2024-07-04T17:12:21.593 INFO:root:teuthology version: 0.0.1.dev347+ga4a89a3 2024-07-04T17:12:21.593 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/mirror/{begin/{0-install 1-ceph 2-logrotate 3-modules} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} conf/{client mds mgr mon osd} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health pg_health} supported-random-distros$/{centos_latest} tasks/mirror} --verbose --owner scheduled_rishabh@teuthology --name rishabh-2024-07-04_16:36:07-fs-wip-rishabh-testing-20240701.141150-debug-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2024-07-04_16:36:07-fs-wip-rishabh-testing-20240701.141150-debug-testing-default-smithi/7787245 -- /home/teuthworker/archive/rishabh-2024-07-04_16:36:07-fs-wip-rishabh-testing-20240701.141150-debug-testing-default-smithi/7787245/orig.config.yaml 2024-07-04T17:12:21.638 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-07-04T17:12:21.726 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-07-04_16:36:07-fs-wip-rishabh-testing-20240701.141150-debug-testing-default-smithi/7787245 branch: wip-rishabh-testing-20240701.141150-debug description: fs/mirror/{begin/{0-install 1-ceph 2-logrotate 3-modules} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} conf/{client mds mgr mon osd} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health pg_health} supported-random-distros$/{centos_latest} tasks/mirror} email: null first_in_suite: false job_id: '7787245' kernel: flavor: default kdb: 1 sha1: 0d32a7bba447dd2470f4de43560317375ca9e9c8 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi meta: - desc: run one cephfs-mirror daemon on primary cluster - desc: configure the permissions for client.mirror - desc: 1 ceph cluster with 1 mon, 1 mgr, 3 osds, 5 mdss mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-07-04_16:36:07-fs-wip-rishabh-testing-20240701.141150-debug-testing-default-smithi no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240701.141150-debug ceph: cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: client: debug cephfs_mirror: 20 log to stderr: false client.mirror: admin socket: /var/run/ceph/cephfs-mirror.asok pid file: /var/run/ceph/cephfs-mirror.pid mgr: debug client: 10 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 9dba1e836b4756bf3a32be2a4936f371dff55008 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 9dba1e836b4756bf3a32be2a4936f371dff55008 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240701.141150-debug sha1: 9dba1e836b4756bf3a32be2a4936f371dff55008 owner: scheduled_rishabh@teuthology priority: 74 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.b - mds.c - mds.d - mds.e - osd.0 - osd.1 - osd.2 - client.0 - client.1 - client.mirror - client.mirror_remote seed: 2237 sha1: 9dba1e836b4756bf3a32be2a4936f371dff55008 sleep_before_teardown: 0 subset: 481/512 suite: fs suite_branch: wip-rishabh-testing-20240701.141150-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_9dba1e836b4756bf3a32be2a4936f371dff55008/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 9dba1e836b4756bf3a32be2a4936f371dff55008 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi042.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGryEf5s/0njFgjBZdSbkWhuJA3zmXyra5E05P04fssEHhz6Me5UzimgC6v5ntl4y1A+jx7lND7JnD5T1968stQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - cephfs-mirror: client: client.mirror - exec: client.mirror: - sudo ceph auth caps client.mirror mon 'profile cephfs-mirror' mds 'allow r' osd 'allow rw tag cephfs metadata=*, allow r tag cephfs data=*' mgr 'allow r' client.mirror_remote: - sudo ceph auth caps client.mirror_remote mon 'allow r' mds 'allow rwps' osd 'allow rw tag cephfs *=*' mgr 'allow r' - ceph-fuse: - client.0 - client.1 - cephfs_test_runner: modules: - tasks.cephfs.test_mirroring.TestMirroring teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: a4a89a3d72be587ba6bd8509090508e8a19e74c5 timestamp: 2024-07-04_16:36:07 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698093 2024-07-04T17:12:21.726 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_9dba1e836b4756bf3a32be2a4936f371dff55008/qa; will attempt to use it 2024-07-04T17:12:21.727 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_9dba1e836b4756bf3a32be2a4936f371dff55008/qa/tasks 2024-07-04T17:12:21.727 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-07-04T17:12:21.729 INFO:teuthology.task.internal:Checking packages... 2024-07-04T17:12:21.749 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '9dba1e836b4756bf3a32be2a4936f371dff55008' 2024-07-04T17:12:21.750 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-07-04T17:12:21.750 INFO:teuthology.packaging:ref: None 2024-07-04T17:12:21.750 INFO:teuthology.packaging:tag: None 2024-07-04T17:12:21.750 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240701.141150-debug 2024-07-04T17:12:21.750 INFO:teuthology.packaging:sha1: 9dba1e836b4756bf3a32be2a4936f371dff55008 2024-07-04T17:12:21.750 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240701.141150-debug 2024-07-04T17:12:21.894 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-4667.g9dba1e83 2024-07-04T17:12:21.895 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-07-04T17:12:21.902 INFO:teuthology.task.internal:no buildpackages task found 2024-07-04T17:12:21.902 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-07-04T17:12:21.943 INFO:teuthology.task.internal:Saving configuration 2024-07-04T17:12:21.958 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-07-04T17:12:21.968 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-07-04T17:12:21.993 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi042.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-07-04_16:36:07-fs-wip-rishabh-testing-20240701.141150-debug-testing-default-smithi/7787245', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-07-04 17:07:48.705937', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGryEf5s/0njFgjBZdSbkWhuJA3zmXyra5E05P04fssEHhz6Me5UzimgC6v5ntl4y1A+jx7lND7JnD5T1968stQ='} 2024-07-04T17:12:21.993 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-07-04T17:12:21.999 INFO:teuthology.task.internal:roles: ubuntu@smithi042.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.b', 'mds.c', 'mds.d', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'client.0', 'client.1', 'client.mirror', 'client.mirror_remote'] 2024-07-04T17:12:21.999 INFO:teuthology.run_tasks:Running task console_log... 2024-07-04T17:12:22.049 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2cb3d1bbe0>, signals=[15]) 2024-07-04T17:12:22.049 INFO:teuthology.run_tasks:Running task internal.connect... 2024-07-04T17:12:22.055 INFO:teuthology.task.internal:Opening connections... 2024-07-04T17:12:22.055 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi042.front.sepia.ceph.com 2024-07-04T17:12:22.056 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-04T17:12:22.132 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-07-04T17:12:22.138 DEBUG:teuthology.orchestra.run.smithi042:> uname -m 2024-07-04T17:12:22.212 INFO:teuthology.orchestra.run.smithi042.stdout:x86_64 2024-07-04T17:12:22.212 DEBUG:teuthology.orchestra.run.smithi042:> cat /etc/os-release 2024-07-04T17:12:22.270 INFO:teuthology.orchestra.run.smithi042.stdout:NAME="CentOS Stream" 2024-07-04T17:12:22.271 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION="9" 2024-07-04T17:12:22.271 INFO:teuthology.orchestra.run.smithi042.stdout:ID="centos" 2024-07-04T17:12:22.271 INFO:teuthology.orchestra.run.smithi042.stdout:ID_LIKE="rhel fedora" 2024-07-04T17:12:22.271 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION_ID="9" 2024-07-04T17:12:22.271 INFO:teuthology.orchestra.run.smithi042.stdout:PLATFORM_ID="platform:el9" 2024-07-04T17:12:22.271 INFO:teuthology.orchestra.run.smithi042.stdout:PRETTY_NAME="CentOS Stream 9" 2024-07-04T17:12:22.271 INFO:teuthology.orchestra.run.smithi042.stdout:ANSI_COLOR="0;31" 2024-07-04T17:12:22.271 INFO:teuthology.orchestra.run.smithi042.stdout:LOGO="fedora-logo-icon" 2024-07-04T17:12:22.271 INFO:teuthology.orchestra.run.smithi042.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-07-04T17:12:22.271 INFO:teuthology.orchestra.run.smithi042.stdout:HOME_URL="https://centos.org/" 2024-07-04T17:12:22.271 INFO:teuthology.orchestra.run.smithi042.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-07-04T17:12:22.271 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-07-04T17:12:22.271 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-07-04T17:12:22.272 INFO:teuthology.lock.ops:Updating smithi042.front.sepia.ceph.com on lock server 2024-07-04T17:12:22.298 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-07-04T17:12:22.307 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-07-04T17:12:22.334 INFO:teuthology.task.internal:Checking for old test directory... 2024-07-04T17:12:22.335 DEBUG:teuthology.orchestra.run.smithi042:> test '!' -e /home/ubuntu/cephtest 2024-07-04T17:12:22.352 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-07-04T17:12:22.400 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-07-04T17:12:22.400 DEBUG:teuthology.orchestra.run.smithi042:> test -z $(ls -A /var/lib/ceph) 2024-07-04T17:12:22.427 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-07-04T17:12:22.457 INFO:teuthology.run_tasks:Running task kernel... 2024-07-04T17:12:22.472 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': 1, 'sha1': '0d32a7bba447dd2470f4de43560317375ca9e9c8'} 2024-07-04T17:12:22.472 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': 1, 'sha1': '0d32a7bba447dd2470f4de43560317375ca9e9c8'}}, timeout 300 2024-07-04T17:12:22.473 DEBUG:teuthology.orchestra.run.smithi042:> test -f /run/.containerenv -o -f /.dockerenv 2024-07-04T17:12:22.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-04T17:12:22.490 DEBUG:teuthology.task.kernel:sha1 for mon.a is 0d32a7bba447dd2470f4de43560317375ca9e9c8 2024-07-04T17:12:22.490 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "0d32a7bba447dd2470f4de43560317375ca9e9c8"... 2024-07-04T17:12:22.490 DEBUG:teuthology.orchestra.run.smithi042:> uname -r 2024-07-04T17:12:22.546 INFO:teuthology.orchestra.run.smithi042.stdout:5.14.0-437.el9.x86_64 2024-07-04T17:12:22.546 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 0d32a7bba447dd2470f4de43560317375ca9e9c8 2024-07-04T17:12:22.547 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-07-04T17:12:22.547 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=0d32a7bba447dd2470f4de43560317375ca9e9c8 2024-07-04T17:12:22.762 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum upgrade -y linux-firmware 2024-07-04T17:12:25.045 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-07-04T17:12:43.311 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 20 MB 00:17 2024-07-04T17:12:55.836 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - CRB 758 kB/s | 6.4 MB 00:08 2024-07-04T17:12:57.779 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 17 kB 00:00 2024-07-04T17:12:58.782 INFO:teuthology.orchestra.run.smithi042.stdout:Extra Packages for Enterprise Linux 26 MB/s | 22 MB 00:00 2024-07-04T17:13:04.255 INFO:teuthology.orchestra.run.smithi042.stdout:lab-extras 48 kB/s | 1.7 kB 00:00 2024-07-04T17:13:06.553 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-07-04T17:13:06.562 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-07-04T17:13:06.562 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-07-04T17:13:06.562 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-07-04T17:13:06.562 INFO:teuthology.orchestra.run.smithi042.stdout:Upgrading: 2024-07-04T17:13:06.562 INFO:teuthology.orchestra.run.smithi042.stdout: linux-firmware noarch 20240603-144.el9 baseos 380 M 2024-07-04T17:13:06.562 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:13:06.562 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-07-04T17:13:06.562 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-07-04T17:13:06.563 INFO:teuthology.orchestra.run.smithi042.stdout:Upgrade 1 Package 2024-07-04T17:13:06.563 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:13:06.563 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 380 M 2024-07-04T17:13:06.563 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-07-04T17:13:11.790 INFO:teuthology.orchestra.run.smithi042.stdout:linux-firmware-20240603-144.el9.noarch.rpm 75 MB/s | 380 MB 00:05 2024-07-04T17:13:11.790 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-07-04T17:13:11.790 INFO:teuthology.orchestra.run.smithi042.stdout:Total 73 MB/s | 380 MB 00:05 2024-07-04T17:13:14.998 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-07-04T17:13:15.090 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-07-04T17:13:15.090 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-07-04T17:13:18.467 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-07-04T17:13:18.467 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-07-04T17:13:27.562 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-07-04T17:13:28.333 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : linux-firmware-20240603-144.el9.noarch 1/2 2024-07-04T17:13:28.423 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : linux-firmware-20240219-143.el9.noarch 2/2 2024-07-04T17:13:29.285 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: linux-firmware-20240219-143.el9.noarch 2/2 2024-07-04T17:13:29.285 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : linux-firmware-20240603-144.el9.noarch 1/2 2024-07-04T17:13:29.993 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 2/2 2024-07-04T17:13:29.993 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:13:29.993 INFO:teuthology.orchestra.run.smithi042.stdout:Upgraded: 2024-07-04T17:13:29.993 INFO:teuthology.orchestra.run.smithi042.stdout: linux-firmware-20240603-144.el9.noarch 2024-07-04T17:13:29.993 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:13:29.994 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-07-04T17:13:30.318 INFO:teuthology.task.kernel:Downloading kernel 0d32a7bba447dd2470f4de43560317375ca9e9c8 on mon.a... 2024-07-04T17:13:30.319 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=0d32a7bba447dd2470f4de43560317375ca9e9c8 2024-07-04T17:13:30.539 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/0d32a7bba447dd2470f4de43560317375ca9e9c8/centos/9/flavors/default/x86_64/ 2024-07-04T17:13:30.540 DEBUG:teuthology.orchestra.run.smithi042:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_g0d32a7bba447-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/0d32a7bba447dd2470f4de43560317375ca9e9c8/centos/9/flavors/default/x86_64/ --input-file=- 2024-07-04T17:13:30.542 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi042.front.sepia.ceph.com to complete... 2024-07-04T17:13:37.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04 17:13:37 URL:https://1.chacra.ceph.com/r/kernel/testing/0d32a7bba447dd2470f4de43560317375ca9e9c8/centos/9/flavors/default/x86_64/kernel-6.9.0_g0d32a7bba447-1.x86_64.rpm [356888676/356888676] -> "/tmp/kernel.x86_64.rpm" [1] 2024-07-04T17:13:37.040 INFO:teuthology.orchestra.run.smithi042.stderr:FINISHED --2024-07-04 17:13:37-- 2024-07-04T17:13:37.040 INFO:teuthology.orchestra.run.smithi042.stderr:Total wall clock time: 6.5s 2024-07-04T17:13:37.040 INFO:teuthology.orchestra.run.smithi042.stderr:Downloaded: 1 files, 340M in 6.1s (55.7 MB/s) 2024-07-04T17:13:37.043 INFO:teuthology.task.kernel:Installing kernel 0d32a7bba447dd2470f4de43560317375ca9e9c8 on mon.a... 2024-07-04T17:13:37.043 DEBUG:teuthology.orchestra.run.smithi042:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-07-04T17:13:39.181 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying... ######################################## 2024-07-04T17:13:39.226 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing... ######################################## 2024-07-04T17:13:39.280 INFO:teuthology.orchestra.run.smithi042.stdout:Updating / installing... 2024-07-04T17:13:47.550 INFO:teuthology.orchestra.run.smithi042.stdout:kernel-6.9.0_g0d32a7bba447-1 ######################################## 2024-07-04T17:14:28.191 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi042.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-07-04T17:14:28.191 DEBUG:teuthology.orchestra.run.smithi042:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-07-04T17:14:28.280 INFO:teuthology.orchestra.run.smithi042.stdout:/boot/System.map-6.9.0-g0d32a7bba447 2024-07-04T17:14:28.280 INFO:teuthology.orchestra.run.smithi042.stdout:/boot/config-6.9.0-g0d32a7bba447 2024-07-04T17:14:28.280 INFO:teuthology.orchestra.run.smithi042.stdout:/boot/vmlinuz-6.9.0-g0d32a7bba447 2024-07-04T17:14:28.280 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/System.map 2024-07-04T17:14:28.280 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/config 2024-07-04T17:14:28.280 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel 2024-07-04T17:14:28.281 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch 2024-07-04T17:14:28.281 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86 2024-07-04T17:14:28.281 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto 2024-07-04T17:14:28.281 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-07-04T17:14:28.281 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-07-04T17:14:28.281 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-07-04T17:14:28.281 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-07-04T17:14:28.281 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-07-04T17:14:28.281 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-07-04T17:14:28.281 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-07-04T17:14:28.281 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/crc32c-intel.ko 2024-07-04T17:14:28.281 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-07-04T17:14:28.281 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-07-04T17:14:28.282 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-07-04T17:14:28.282 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-07-04T17:14:28.282 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/serpent-avx2.ko 2024-07-04T17:14:28.282 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-07-04T17:14:28.282 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-07-04T17:14:28.282 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-07-04T17:14:28.282 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-07-04T17:14:28.282 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-07-04T17:14:28.282 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-07-04T17:14:28.282 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-07-04T17:14:28.282 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/kernel 2024-07-04T17:14:28.285 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/kernel/cpu 2024-07-04T17:14:28.285 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/kernel/cpu/mce 2024-07-04T17:14:28.285 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-07-04T17:14:28.285 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/kvm 2024-07-04T17:14:28.285 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/kvm/kvm-amd.ko 2024-07-04T17:14:28.285 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/kvm/kvm-intel.ko 2024-07-04T17:14:28.285 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/kvm/kvm.ko 2024-07-04T17:14:28.285 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto 2024-07-04T17:14:28.285 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/ansi_cprng.ko 2024-07-04T17:14:28.286 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/anubis.ko 2024-07-04T17:14:28.286 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/arc4.ko 2024-07-04T17:14:28.286 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/async_tx 2024-07-04T17:14:28.286 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/async_tx/async_memcpy.ko 2024-07-04T17:14:28.286 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/async_tx/async_pq.ko 2024-07-04T17:14:28.286 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/async_tx/async_raid6_recov.ko 2024-07-04T17:14:28.286 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/async_tx/async_tx.ko 2024-07-04T17:14:28.286 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/async_tx/async_xor.ko 2024-07-04T17:14:28.286 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/async_tx/raid6test.ko 2024-07-04T17:14:28.286 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/authenc.ko 2024-07-04T17:14:28.286 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/authencesn.ko 2024-07-04T17:14:28.286 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/blake2b_generic.ko 2024-07-04T17:14:28.287 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/blowfish_common.ko 2024-07-04T17:14:28.287 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/blowfish_generic.ko 2024-07-04T17:14:28.287 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/camellia_generic.ko 2024-07-04T17:14:28.287 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/cast5_generic.ko 2024-07-04T17:14:28.287 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/cast6_generic.ko 2024-07-04T17:14:28.287 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/cast_common.ko 2024-07-04T17:14:28.287 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/ccm.ko 2024-07-04T17:14:28.287 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/cmac.ko 2024-07-04T17:14:28.287 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/crc32_generic.ko 2024-07-04T17:14:28.287 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/crypto_engine.ko 2024-07-04T17:14:28.287 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/crypto_user.ko 2024-07-04T17:14:28.287 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/cts.ko 2024-07-04T17:14:28.287 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/des_generic.ko 2024-07-04T17:14:28.288 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/dh_generic.ko 2024-07-04T17:14:28.288 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/echainiv.ko 2024-07-04T17:14:28.288 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/essiv.ko 2024-07-04T17:14:28.288 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/fcrypt.ko 2024-07-04T17:14:28.288 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/khazad.ko 2024-07-04T17:14:28.288 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/lz4.ko 2024-07-04T17:14:28.288 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/lz4hc.ko 2024-07-04T17:14:28.288 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/md4.ko 2024-07-04T17:14:28.288 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/michael_mic.ko 2024-07-04T17:14:28.288 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/pcbc.ko 2024-07-04T17:14:28.288 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/pcrypt.ko 2024-07-04T17:14:28.288 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/rmd160.ko 2024-07-04T17:14:28.288 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/seed.ko 2024-07-04T17:14:28.288 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/serpent_generic.ko 2024-07-04T17:14:28.289 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/tcrypt.ko 2024-07-04T17:14:28.289 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/tea.ko 2024-07-04T17:14:28.289 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/twofish_common.ko 2024-07-04T17:14:28.289 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/twofish_generic.ko 2024-07-04T17:14:28.289 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/vmac.ko 2024-07-04T17:14:28.289 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/wp512.ko 2024-07-04T17:14:28.289 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/xcbc.ko 2024-07-04T17:14:28.289 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/xor.ko 2024-07-04T17:14:28.289 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/xxhash_generic.ko 2024-07-04T17:14:28.289 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers 2024-07-04T17:14:28.289 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/acpi 2024-07-04T17:14:28.289 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/acpi/acpi_ipmi.ko 2024-07-04T17:14:28.289 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/acpi/acpi_pad.ko 2024-07-04T17:14:28.289 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/acpi/ec_sys.ko 2024-07-04T17:14:28.289 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/acpi/platform_profile.ko 2024-07-04T17:14:28.290 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/acpi/sbs.ko 2024-07-04T17:14:28.290 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/acpi/sbshc.ko 2024-07-04T17:14:28.290 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/acpi/video.ko 2024-07-04T17:14:28.290 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata 2024-07-04T17:14:28.290 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/acard-ahci.ko 2024-07-04T17:14:28.290 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/ahci_platform.ko 2024-07-04T17:14:28.290 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/ata_generic.ko 2024-07-04T17:14:28.290 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/libahci_platform.ko 2024-07-04T17:14:28.290 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_acpi.ko 2024-07-04T17:14:28.290 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_ali.ko 2024-07-04T17:14:28.290 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_amd.ko 2024-07-04T17:14:28.290 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_artop.ko 2024-07-04T17:14:28.290 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_atiixp.ko 2024-07-04T17:14:28.290 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_atp867x.ko 2024-07-04T17:14:28.291 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_cmd640.ko 2024-07-04T17:14:28.291 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_cmd64x.ko 2024-07-04T17:14:28.291 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_cypress.ko 2024-07-04T17:14:28.291 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_efar.ko 2024-07-04T17:14:28.291 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_hpt366.ko 2024-07-04T17:14:28.291 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_hpt37x.ko 2024-07-04T17:14:28.291 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_hpt3x2n.ko 2024-07-04T17:14:28.291 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_hpt3x3.ko 2024-07-04T17:14:28.291 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_it8213.ko 2024-07-04T17:14:28.291 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_it821x.ko 2024-07-04T17:14:28.291 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_jmicron.ko 2024-07-04T17:14:28.291 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_marvell.ko 2024-07-04T17:14:28.291 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_mpiix.ko 2024-07-04T17:14:28.291 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_netcell.ko 2024-07-04T17:14:28.291 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_ninja32.ko 2024-07-04T17:14:28.292 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_ns87410.ko 2024-07-04T17:14:28.292 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_ns87415.ko 2024-07-04T17:14:28.292 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_oldpiix.ko 2024-07-04T17:14:28.292 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_opti.ko 2024-07-04T17:14:28.292 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_optidma.ko 2024-07-04T17:14:28.292 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_pcmcia.ko 2024-07-04T17:14:28.292 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_pdc2027x.ko 2024-07-04T17:14:28.292 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-07-04T17:14:28.292 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_piccolo.ko 2024-07-04T17:14:28.292 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_rdc.ko 2024-07-04T17:14:28.292 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_sch.ko 2024-07-04T17:14:28.292 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_serverworks.ko 2024-07-04T17:14:28.292 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_sil680.ko 2024-07-04T17:14:28.293 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_sis.ko 2024-07-04T17:14:28.293 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_sl82c105.ko 2024-07-04T17:14:28.293 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_triflex.ko 2024-07-04T17:14:28.293 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_via.ko 2024-07-04T17:14:28.293 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pdc_adma.ko 2024-07-04T17:14:28.293 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/sata_inic162x.ko 2024-07-04T17:14:28.293 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/sata_mv.ko 2024-07-04T17:14:28.293 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/sata_nv.ko 2024-07-04T17:14:28.293 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/sata_promise.ko 2024-07-04T17:14:28.293 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/sata_qstor.ko 2024-07-04T17:14:28.293 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/sata_sil.ko 2024-07-04T17:14:28.293 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/sata_sil24.ko 2024-07-04T17:14:28.293 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/sata_sis.ko 2024-07-04T17:14:28.293 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/sata_svw.ko 2024-07-04T17:14:28.293 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/sata_sx4.ko 2024-07-04T17:14:28.294 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/sata_uli.ko 2024-07-04T17:14:28.294 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/sata_via.ko 2024-07-04T17:14:28.294 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/sata_vsc.ko 2024-07-04T17:14:28.294 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/auxdisplay 2024-07-04T17:14:28.294 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/auxdisplay/ks0108.ko 2024-07-04T17:14:28.294 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/base 2024-07-04T17:14:28.294 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/base/regmap 2024-07-04T17:14:28.294 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/base/regmap/regmap-i2c.ko 2024-07-04T17:14:28.294 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/base/regmap/regmap-mmio.ko 2024-07-04T17:14:28.294 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/bcma 2024-07-04T17:14:28.294 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/bcma/bcma.ko 2024-07-04T17:14:28.294 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block 2024-07-04T17:14:28.294 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/aoe 2024-07-04T17:14:28.295 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/aoe/aoe.ko 2024-07-04T17:14:28.295 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/brd.ko 2024-07-04T17:14:28.295 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/drbd 2024-07-04T17:14:28.295 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/drbd/drbd.ko 2024-07-04T17:14:28.295 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/floppy.ko 2024-07-04T17:14:28.295 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/loop.ko 2024-07-04T17:14:28.295 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/mtip32xx 2024-07-04T17:14:28.295 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-07-04T17:14:28.295 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/nbd.ko 2024-07-04T17:14:28.295 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/null_blk 2024-07-04T17:14:28.296 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/null_blk/null_blk.ko 2024-07-04T17:14:28.296 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/pktcdvd.ko 2024-07-04T17:14:28.296 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/rbd.ko 2024-07-04T17:14:28.296 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/virtio_blk.ko 2024-07-04T17:14:28.296 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/xen-blkback 2024-07-04T17:14:28.296 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-07-04T17:14:28.296 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/xen-blkfront.ko 2024-07-04T17:14:28.296 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/zram 2024-07-04T17:14:28.296 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/zram/zram.ko 2024-07-04T17:14:28.296 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char 2024-07-04T17:14:28.296 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/hangcheck-timer.ko 2024-07-04T17:14:28.296 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/hw_random 2024-07-04T17:14:28.296 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/hw_random/amd-rng.ko 2024-07-04T17:14:28.297 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/hw_random/intel-rng.ko 2024-07-04T17:14:28.297 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-07-04T17:14:28.297 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/hw_random/via-rng.ko 2024-07-04T17:14:28.297 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/hw_random/virtio-rng.ko 2024-07-04T17:14:28.297 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/ipmi 2024-07-04T17:14:28.297 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-07-04T17:14:28.297 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-07-04T17:14:28.297 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-07-04T17:14:28.297 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/ipmi/ipmi_si.ko 2024-07-04T17:14:28.297 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-07-04T17:14:28.297 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-07-04T17:14:28.297 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/lp.ko 2024-07-04T17:14:28.297 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/mwave 2024-07-04T17:14:28.298 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/mwave/mwave.ko 2024-07-04T17:14:28.298 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/ppdev.ko 2024-07-04T17:14:28.298 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/tlclk.ko 2024-07-04T17:14:28.298 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/tpm 2024-07-04T17:14:28.298 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/tpm/tpm.ko 2024-07-04T17:14:28.298 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/tpm/tpm_atmel.ko 2024-07-04T17:14:28.298 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/tpm/tpm_infineon.ko 2024-07-04T17:14:28.298 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/tpm/tpm_nsc.ko 2024-07-04T17:14:28.298 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/tpm/tpm_tis.ko 2024-07-04T17:14:28.298 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-07-04T17:14:28.298 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/uv_mmtimer.ko 2024-07-04T17:14:28.298 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/virtio_console.ko 2024-07-04T17:14:28.299 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/cpufreq 2024-07-04T17:14:28.299 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-07-04T17:14:28.299 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-07-04T17:14:28.299 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/cpufreq/p4-clockmod.ko 2024-07-04T17:14:28.299 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-07-04T17:14:28.299 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/cpufreq/powernow-k8.ko 2024-07-04T17:14:28.299 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/cpufreq/speedstep-lib.ko 2024-07-04T17:14:28.299 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/crypto 2024-07-04T17:14:28.299 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/crypto/ccp 2024-07-04T17:14:28.299 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-07-04T17:14:28.299 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/crypto/ccp/ccp.ko 2024-07-04T17:14:28.299 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/crypto/intel 2024-07-04T17:14:28.299 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/crypto/intel/qat 2024-07-04T17:14:28.299 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/crypto/intel/qat/qat_common 2024-07-04T17:14:28.300 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-07-04T17:14:28.300 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-07-04T17:14:28.300 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-07-04T17:14:28.300 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/crypto/padlock-aes.ko 2024-07-04T17:14:28.300 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/crypto/padlock-sha.ko 2024-07-04T17:14:28.300 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/crypto/virtio 2024-07-04T17:14:28.300 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-07-04T17:14:28.300 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/dca 2024-07-04T17:14:28.300 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/dca/dca.ko 2024-07-04T17:14:28.300 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/devfreq 2024-07-04T17:14:28.300 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-07-04T17:14:28.300 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/dma 2024-07-04T17:14:28.300 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/dma/dw 2024-07-04T17:14:28.300 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/dma/dw/dw_dmac.ko 2024-07-04T17:14:28.300 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/dma/ioat 2024-07-04T17:14:28.301 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/dma/ioat/ioatdma.ko 2024-07-04T17:14:28.301 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/edac 2024-07-04T17:14:28.301 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/edac/amd64_edac.ko 2024-07-04T17:14:28.301 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/edac/e752x_edac.ko 2024-07-04T17:14:28.301 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/edac/edac_mce_amd.ko 2024-07-04T17:14:28.301 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/edac/i3000_edac.ko 2024-07-04T17:14:28.301 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/edac/i3200_edac.ko 2024-07-04T17:14:28.301 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/edac/i5100_edac.ko 2024-07-04T17:14:28.301 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/edac/i5400_edac.ko 2024-07-04T17:14:28.301 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/edac/i7300_edac.ko 2024-07-04T17:14:28.301 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/edac/i7core_edac.ko 2024-07-04T17:14:28.301 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/edac/i82975x_edac.ko 2024-07-04T17:14:28.301 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/edac/ie31200_edac.ko 2024-07-04T17:14:28.302 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/edac/sb_edac.ko 2024-07-04T17:14:28.302 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/edac/x38_edac.ko 2024-07-04T17:14:28.302 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/firmware 2024-07-04T17:14:28.302 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/firmware/edd.ko 2024-07-04T17:14:28.302 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/firmware/iscsi_ibft.ko 2024-07-04T17:14:28.302 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/gpio 2024-07-04T17:14:28.302 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/gpio/gpio-ich.ko 2024-07-04T17:14:28.302 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/gpio/gpio-viperboard.ko 2024-07-04T17:14:28.302 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid 2024-07-04T17:14:28.302 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-appleir.ko 2024-07-04T17:14:28.302 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-aureal.ko 2024-07-04T17:14:28.302 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-axff.ko 2024-07-04T17:14:28.302 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-dr.ko 2024-07-04T17:14:28.303 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-elecom.ko 2024-07-04T17:14:28.303 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-elo.ko 2024-07-04T17:14:28.303 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-emsff.ko 2024-07-04T17:14:28.303 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-gaff.ko 2024-07-04T17:14:28.303 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-gt683r.ko 2024-07-04T17:14:28.303 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-gyration.ko 2024-07-04T17:14:28.303 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-holtek-kbd.ko 2024-07-04T17:14:28.303 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-holtek-mouse.ko 2024-07-04T17:14:28.303 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-holtekff.ko 2024-07-04T17:14:28.303 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-hyperv.ko 2024-07-04T17:14:28.303 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-icade.ko 2024-07-04T17:14:28.303 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-keytouch.ko 2024-07-04T17:14:28.303 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-kye.ko 2024-07-04T17:14:28.303 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-lcpower.ko 2024-07-04T17:14:28.303 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-led.ko 2024-07-04T17:14:28.304 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-lenovo.ko 2024-07-04T17:14:28.304 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-logitech-dj.ko 2024-07-04T17:14:28.304 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-07-04T17:14:28.304 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-multitouch.ko 2024-07-04T17:14:28.304 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-ortek.ko 2024-07-04T17:14:28.304 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-penmount.ko 2024-07-04T17:14:28.304 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-petalynx.ko 2024-07-04T17:14:28.304 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-picolcd.ko 2024-07-04T17:14:28.304 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-pl.ko 2024-07-04T17:14:28.304 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-plantronics.ko 2024-07-04T17:14:28.304 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-primax.ko 2024-07-04T17:14:28.304 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-rmi.ko 2024-07-04T17:14:28.304 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-roccat-arvo.ko 2024-07-04T17:14:28.304 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-roccat-common.ko 2024-07-04T17:14:28.304 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-roccat-isku.ko 2024-07-04T17:14:28.305 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-roccat-kone.ko 2024-07-04T17:14:28.305 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-07-04T17:14:28.305 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-roccat-konepure.ko 2024-07-04T17:14:28.305 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-07-04T17:14:28.305 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-roccat-lua.ko 2024-07-04T17:14:28.305 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-roccat-pyra.ko 2024-07-04T17:14:28.305 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-roccat-ryos.ko 2024-07-04T17:14:28.305 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-roccat-savu.ko 2024-07-04T17:14:28.305 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-roccat.ko 2024-07-04T17:14:28.305 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-saitek.ko 2024-07-04T17:14:28.305 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-samsung.ko 2024-07-04T17:14:28.305 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-sensor-hub.ko 2024-07-04T17:14:28.305 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-sjoy.ko 2024-07-04T17:14:28.305 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-sony.ko 2024-07-04T17:14:28.306 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-speedlink.ko 2024-07-04T17:14:28.306 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-steelseries.ko 2024-07-04T17:14:28.306 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-sunplus.ko 2024-07-04T17:14:28.306 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-thrustmaster.ko 2024-07-04T17:14:28.306 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-tivo.ko 2024-07-04T17:14:28.306 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-tmff.ko 2024-07-04T17:14:28.306 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-topseed.ko 2024-07-04T17:14:28.306 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-twinhan.ko 2024-07-04T17:14:28.306 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-uclogic.ko 2024-07-04T17:14:28.306 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-waltop.ko 2024-07-04T17:14:28.306 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-wiimote.ko 2024-07-04T17:14:28.306 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-xinmo.ko 2024-07-04T17:14:28.307 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-zpff.ko 2024-07-04T17:14:28.307 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-zydacron.ko 2024-07-04T17:14:28.307 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/uhid.ko 2024-07-04T17:14:28.307 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/wacom.ko 2024-07-04T17:14:28.307 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hv 2024-07-04T17:14:28.307 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hv/hv_balloon.ko 2024-07-04T17:14:28.307 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hv/hv_utils.ko 2024-07-04T17:14:28.307 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hv/hv_vmbus.ko 2024-07-04T17:14:28.307 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon 2024-07-04T17:14:28.307 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/abituguru.ko 2024-07-04T17:14:28.307 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/abituguru3.ko 2024-07-04T17:14:28.307 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/acpi_power_meter.ko 2024-07-04T17:14:28.307 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/ad7414.ko 2024-07-04T17:14:28.307 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/ad7418.ko 2024-07-04T17:14:28.308 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/adc128d818.ko 2024-07-04T17:14:28.308 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/adm1025.ko 2024-07-04T17:14:28.308 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/adm1026.ko 2024-07-04T17:14:28.308 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/adm1029.ko 2024-07-04T17:14:28.308 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/adm1031.ko 2024-07-04T17:14:28.308 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/adm9240.ko 2024-07-04T17:14:28.308 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/ads7828.ko 2024-07-04T17:14:28.308 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/adt7410.ko 2024-07-04T17:14:28.308 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/adt7411.ko 2024-07-04T17:14:28.308 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/adt7462.ko 2024-07-04T17:14:28.308 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/adt7470.ko 2024-07-04T17:14:28.308 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/adt7475.ko 2024-07-04T17:14:28.308 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/adt7x10.ko 2024-07-04T17:14:28.308 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/amc6821.ko 2024-07-04T17:14:28.309 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/applesmc.ko 2024-07-04T17:14:28.309 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/asb100.ko 2024-07-04T17:14:28.309 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/asc7621.ko 2024-07-04T17:14:28.309 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/asus_atk0110.ko 2024-07-04T17:14:28.309 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/atxp1.ko 2024-07-04T17:14:28.309 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/coretemp.ko 2024-07-04T17:14:28.309 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-07-04T17:14:28.309 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/dme1737.ko 2024-07-04T17:14:28.309 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/ds1621.ko 2024-07-04T17:14:28.309 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/ds620.ko 2024-07-04T17:14:28.309 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/emc1403.ko 2024-07-04T17:14:28.309 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/emc6w201.ko 2024-07-04T17:14:28.309 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/f71805f.ko 2024-07-04T17:14:28.310 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/f71882fg.ko 2024-07-04T17:14:28.310 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/f75375s.ko 2024-07-04T17:14:28.310 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/fam15h_power.ko 2024-07-04T17:14:28.310 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/fschmd.ko 2024-07-04T17:14:28.310 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/g760a.ko 2024-07-04T17:14:28.310 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/g762.ko 2024-07-04T17:14:28.310 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/gl518sm.ko 2024-07-04T17:14:28.310 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/gl520sm.ko 2024-07-04T17:14:28.310 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/hwmon-vid.ko 2024-07-04T17:14:28.310 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/i5500_temp.ko 2024-07-04T17:14:28.310 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/i5k_amb.ko 2024-07-04T17:14:28.310 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/ibmaem.ko 2024-07-04T17:14:28.310 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/ibmpex.ko 2024-07-04T17:14:28.310 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/ina209.ko 2024-07-04T17:14:28.311 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/ina2xx.ko 2024-07-04T17:14:28.311 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/it87.ko 2024-07-04T17:14:28.311 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/k10temp.ko 2024-07-04T17:14:28.311 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/k8temp.ko 2024-07-04T17:14:28.311 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/lineage-pem.ko 2024-07-04T17:14:28.311 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/lm63.ko 2024-07-04T17:14:28.311 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/lm73.ko 2024-07-04T17:14:28.311 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/lm75.ko 2024-07-04T17:14:28.311 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/lm77.ko 2024-07-04T17:14:28.311 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/lm78.ko 2024-07-04T17:14:28.311 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/lm80.ko 2024-07-04T17:14:28.311 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/lm83.ko 2024-07-04T17:14:28.311 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/lm85.ko 2024-07-04T17:14:28.311 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/lm87.ko 2024-07-04T17:14:28.312 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/lm90.ko 2024-07-04T17:14:28.312 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/lm92.ko 2024-07-04T17:14:28.312 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/lm93.ko 2024-07-04T17:14:28.312 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/lm95234.ko 2024-07-04T17:14:28.312 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/lm95241.ko 2024-07-04T17:14:28.312 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/lm95245.ko 2024-07-04T17:14:28.312 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/ltc2945.ko 2024-07-04T17:14:28.312 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/ltc4151.ko 2024-07-04T17:14:28.312 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/ltc4215.ko 2024-07-04T17:14:28.312 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/ltc4222.ko 2024-07-04T17:14:28.312 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/ltc4245.ko 2024-07-04T17:14:28.312 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/ltc4260.ko 2024-07-04T17:14:28.312 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/ltc4261.ko 2024-07-04T17:14:28.313 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/max16065.ko 2024-07-04T17:14:28.313 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/max1619.ko 2024-07-04T17:14:28.313 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/max1668.ko 2024-07-04T17:14:28.313 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/max197.ko 2024-07-04T17:14:28.313 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/max6639.ko 2024-07-04T17:14:28.313 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/max6650.ko 2024-07-04T17:14:28.313 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/max6697.ko 2024-07-04T17:14:28.313 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/mcp3021.ko 2024-07-04T17:14:28.313 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/nct6683.ko 2024-07-04T17:14:28.313 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/nct6775-core.ko 2024-07-04T17:14:28.313 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/nct6775.ko 2024-07-04T17:14:28.313 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/nct7802.ko 2024-07-04T17:14:28.313 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/pc87360.ko 2024-07-04T17:14:28.313 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/pc87427.ko 2024-07-04T17:14:28.314 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/pcf8591.ko 2024-07-04T17:14:28.314 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/pmbus 2024-07-04T17:14:28.314 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-07-04T17:14:28.314 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-07-04T17:14:28.314 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-07-04T17:14:28.314 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/pmbus/max16064.ko 2024-07-04T17:14:28.314 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/pmbus/max34440.ko 2024-07-04T17:14:28.314 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/pmbus/max8688.ko 2024-07-04T17:14:28.314 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-07-04T17:14:28.314 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-07-04T17:14:28.314 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-07-04T17:14:28.314 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-07-04T17:14:28.314 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-07-04T17:14:28.315 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-07-04T17:14:28.315 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/powr1220.ko 2024-07-04T17:14:28.315 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/sch5627.ko 2024-07-04T17:14:28.315 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/sch5636.ko 2024-07-04T17:14:28.315 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/sch56xx-common.ko 2024-07-04T17:14:28.315 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/sht15.ko 2024-07-04T17:14:28.315 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/sht21.ko 2024-07-04T17:14:28.315 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/shtc1.ko 2024-07-04T17:14:28.315 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/sis5595.ko 2024-07-04T17:14:28.315 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/smsc47b397.ko 2024-07-04T17:14:28.315 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/smsc47m1.ko 2024-07-04T17:14:28.315 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/smsc47m192.ko 2024-07-04T17:14:28.316 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/thmc50.ko 2024-07-04T17:14:28.316 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/tmp102.ko 2024-07-04T17:14:28.316 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/tmp103.ko 2024-07-04T17:14:28.316 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/tmp401.ko 2024-07-04T17:14:28.316 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/tmp421.ko 2024-07-04T17:14:28.316 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/via-cputemp.ko 2024-07-04T17:14:28.316 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/via686a.ko 2024-07-04T17:14:28.316 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/vt1211.ko 2024-07-04T17:14:28.316 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/vt8231.ko 2024-07-04T17:14:28.316 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/w83627ehf.ko 2024-07-04T17:14:28.316 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/w83627hf.ko 2024-07-04T17:14:28.316 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/w83781d.ko 2024-07-04T17:14:28.316 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/w83791d.ko 2024-07-04T17:14:28.316 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/w83792d.ko 2024-07-04T17:14:28.317 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/w83793.ko 2024-07-04T17:14:28.317 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/w83795.ko 2024-07-04T17:14:28.317 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/w83l785ts.ko 2024-07-04T17:14:28.317 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/w83l786ng.ko 2024-07-04T17:14:28.317 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c 2024-07-04T17:14:28.317 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/algos 2024-07-04T17:14:28.317 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-07-04T17:14:28.317 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-07-04T17:14:28.317 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses 2024-07-04T17:14:28.317 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-07-04T17:14:28.317 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-07-04T17:14:28.317 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-07-04T17:14:28.317 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-07-04T17:14:28.317 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-07-04T17:14:28.317 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-07-04T17:14:28.318 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-07-04T17:14:28.318 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-07-04T17:14:28.318 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-i801.ko 2024-07-04T17:14:28.318 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-isch.ko 2024-07-04T17:14:28.318 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-07-04T17:14:28.318 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-07-04T17:14:28.318 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-07-04T17:14:28.318 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-parport.ko 2024-07-04T17:14:28.318 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-07-04T17:14:28.318 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-07-04T17:14:28.318 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-07-04T17:14:28.318 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-07-04T17:14:28.318 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-07-04T17:14:28.319 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-07-04T17:14:28.319 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-via.ko 2024-07-04T17:14:28.319 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-07-04T17:14:28.319 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-07-04T17:14:28.319 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/i2c-dev.ko 2024-07-04T17:14:28.319 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/i2c-mux.ko 2024-07-04T17:14:28.319 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-07-04T17:14:28.319 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/i2c-smbus.ko 2024-07-04T17:14:28.319 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/i2c-stub.ko 2024-07-04T17:14:28.319 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband 2024-07-04T17:14:28.319 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/core 2024-07-04T17:14:28.319 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/core/ib_cm.ko 2024-07-04T17:14:28.319 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/core/ib_core.ko 2024-07-04T17:14:28.320 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/core/ib_umad.ko 2024-07-04T17:14:28.320 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-07-04T17:14:28.320 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/core/iw_cm.ko 2024-07-04T17:14:28.320 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/core/rdma_cm.ko 2024-07-04T17:14:28.320 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-07-04T17:14:28.320 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/hw 2024-07-04T17:14:28.320 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/hw/cxgb4 2024-07-04T17:14:28.320 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-07-04T17:14:28.320 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/hw/mlx4 2024-07-04T17:14:28.320 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-07-04T17:14:28.320 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/hw/mlx5 2024-07-04T17:14:28.320 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-07-04T17:14:28.320 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/hw/mthca 2024-07-04T17:14:28.320 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-07-04T17:14:28.321 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/ulp 2024-07-04T17:14:28.321 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/ulp/ipoib 2024-07-04T17:14:28.321 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-07-04T17:14:28.321 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/ulp/iser 2024-07-04T17:14:28.321 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-07-04T17:14:28.321 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/ulp/isert 2024-07-04T17:14:28.321 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-07-04T17:14:28.321 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/ulp/srp 2024-07-04T17:14:28.321 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-07-04T17:14:28.321 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/ulp/srpt 2024-07-04T17:14:28.321 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-07-04T17:14:28.321 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input 2024-07-04T17:14:28.322 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/gameport 2024-07-04T17:14:28.322 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-07-04T17:14:28.322 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/gameport/fm801-gp.ko 2024-07-04T17:14:28.322 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/gameport/gameport.ko 2024-07-04T17:14:28.322 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/joydev.ko 2024-07-04T17:14:28.322 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/keyboard 2024-07-04T17:14:28.322 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/keyboard/gpio_keys.ko 2024-07-04T17:14:28.322 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/mouse 2024-07-04T17:14:28.322 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/mouse/appletouch.ko 2024-07-04T17:14:28.322 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/mouse/bcm5974.ko 2024-07-04T17:14:28.322 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/mouse/cyapatp.ko 2024-07-04T17:14:28.322 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/mouse/elan_i2c.ko 2024-07-04T17:14:28.323 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/mouse/sermouse.ko 2024-07-04T17:14:28.323 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-07-04T17:14:28.323 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/mouse/synaptics_usb.ko 2024-07-04T17:14:28.323 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/mouse/vsxxxaa.ko 2024-07-04T17:14:28.323 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/rmi4 2024-07-04T17:14:28.323 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/rmi4/rmi_core.ko 2024-07-04T17:14:28.323 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/serio 2024-07-04T17:14:28.323 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/serio/altera_ps2.ko 2024-07-04T17:14:28.323 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/serio/arc_ps2.ko 2024-07-04T17:14:28.323 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-07-04T17:14:28.323 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/serio/serio_raw.ko 2024-07-04T17:14:28.323 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/sparse-keymap.ko 2024-07-04T17:14:28.323 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/leds 2024-07-04T17:14:28.324 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/leds/leds-blinkm.ko 2024-07-04T17:14:28.324 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/leds/leds-lm3530.ko 2024-07-04T17:14:28.324 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/leds/leds-lp3944.ko 2024-07-04T17:14:28.324 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/leds/leds-ss4200.ko 2024-07-04T17:14:28.324 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/leds/trigger 2024-07-04T17:14:28.324 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-07-04T17:14:28.324 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-07-04T17:14:28.324 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-07-04T17:14:28.324 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-07-04T17:14:28.324 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-07-04T17:14:28.324 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-07-04T17:14:28.324 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-07-04T17:14:28.325 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-07-04T17:14:28.325 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-07-04T17:14:28.325 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md 2024-07-04T17:14:28.325 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/bcache 2024-07-04T17:14:28.325 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/bcache/bcache.ko 2024-07-04T17:14:28.325 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/dm-bio-prison.ko 2024-07-04T17:14:28.325 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/dm-cache-smq.ko 2024-07-04T17:14:28.325 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/dm-cache.ko 2024-07-04T17:14:28.325 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/dm-crypt.ko 2024-07-04T17:14:28.325 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/dm-delay.ko 2024-07-04T17:14:28.325 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/dm-flakey.ko 2024-07-04T17:14:28.325 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/dm-log-userspace.ko 2024-07-04T17:14:28.325 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/dm-multipath.ko 2024-07-04T17:14:28.326 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/dm-queue-length.ko 2024-07-04T17:14:28.326 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/dm-raid.ko 2024-07-04T17:14:28.326 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/dm-round-robin.ko 2024-07-04T17:14:28.326 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/dm-service-time.ko 2024-07-04T17:14:28.326 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/dm-switch.ko 2024-07-04T17:14:28.326 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/dm-thin-pool.ko 2024-07-04T17:14:28.326 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/dm-verity.ko 2024-07-04T17:14:28.326 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/persistent-data 2024-07-04T17:14:28.326 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-07-04T17:14:28.326 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/raid0.ko 2024-07-04T17:14:28.326 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/raid1.ko 2024-07-04T17:14:28.326 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/raid10.ko 2024-07-04T17:14:28.326 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/raid456.ko 2024-07-04T17:14:28.327 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/message 2024-07-04T17:14:28.327 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/message/fusion 2024-07-04T17:14:28.327 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/message/fusion/mptbase.ko 2024-07-04T17:14:28.327 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/message/fusion/mptctl.ko 2024-07-04T17:14:28.327 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/message/fusion/mptfc.ko 2024-07-04T17:14:28.327 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/message/fusion/mptlan.ko 2024-07-04T17:14:28.327 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/message/fusion/mptsas.ko 2024-07-04T17:14:28.327 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/message/fusion/mptscsih.ko 2024-07-04T17:14:28.327 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/message/fusion/mptspi.ko 2024-07-04T17:14:28.327 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/mfd 2024-07-04T17:14:28.327 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/mfd/lpc_ich.ko 2024-07-04T17:14:28.327 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/mfd/lpc_sch.ko 2024-07-04T17:14:28.327 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/mfd/mfd-core.ko 2024-07-04T17:14:28.328 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/mfd/sm501.ko 2024-07-04T17:14:28.328 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/mfd/viperboard.ko 2024-07-04T17:14:28.328 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/mfd/vx855.ko 2024-07-04T17:14:28.328 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/mfd/wl1273-core.ko 2024-07-04T17:14:28.328 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc 2024-07-04T17:14:28.328 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/altera-stapl 2024-07-04T17:14:28.328 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-07-04T17:14:28.328 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/apds9802als.ko 2024-07-04T17:14:28.328 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/apds990x.ko 2024-07-04T17:14:28.328 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/bh1770glc.ko 2024-07-04T17:14:28.328 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/cb710 2024-07-04T17:14:28.328 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/cb710/cb710.ko 2024-07-04T17:14:28.328 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/eeprom 2024-07-04T17:14:28.328 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/eeprom/at24.ko 2024-07-04T17:14:28.328 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-07-04T17:14:28.329 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/eeprom/max6875.ko 2024-07-04T17:14:28.329 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/enclosure.ko 2024-07-04T17:14:28.329 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/hpilo.ko 2024-07-04T17:14:28.329 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/isl29003.ko 2024-07-04T17:14:28.329 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/isl29020.ko 2024-07-04T17:14:28.329 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/lis3lv02d 2024-07-04T17:14:28.329 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-07-04T17:14:28.329 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-07-04T17:14:28.329 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/mei 2024-07-04T17:14:28.329 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/mei/mei-me.ko 2024-07-04T17:14:28.329 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/mei/mei-txe.ko 2024-07-04T17:14:28.329 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/mei/mei.ko 2024-07-04T17:14:28.329 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/sgi-gru 2024-07-04T17:14:28.329 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/sgi-gru/gru.ko 2024-07-04T17:14:28.329 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/sgi-xp 2024-07-04T17:14:28.330 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/sgi-xp/xp.ko 2024-07-04T17:14:28.330 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/sgi-xp/xpc.ko 2024-07-04T17:14:28.330 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-07-04T17:14:28.330 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/tifm_7xx1.ko 2024-07-04T17:14:28.330 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/tifm_core.ko 2024-07-04T17:14:28.330 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/tsl2550.ko 2024-07-04T17:14:28.330 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/vmw_balloon.ko 2024-07-04T17:14:28.330 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/vmw_vmci 2024-07-04T17:14:28.330 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-07-04T17:14:28.330 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net 2024-07-04T17:14:28.330 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/bonding 2024-07-04T17:14:28.330 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/bonding/bonding.ko 2024-07-04T17:14:28.330 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/dsa 2024-07-04T17:14:28.330 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/dsa/b53 2024-07-04T17:14:28.331 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/dsa/b53/b53_common.ko 2024-07-04T17:14:28.331 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/dsa/bcm-sf2.ko 2024-07-04T17:14:28.331 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/dsa/mv88e6060.ko 2024-07-04T17:14:28.331 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/dsa/mv88e6xxx 2024-07-04T17:14:28.331 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-07-04T17:14:28.331 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/dummy.ko 2024-07-04T17:14:28.331 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/eql.ko 2024-07-04T17:14:28.331 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet 2024-07-04T17:14:28.331 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/3com 2024-07-04T17:14:28.331 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-07-04T17:14:28.331 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-07-04T17:14:28.331 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-07-04T17:14:28.331 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-07-04T17:14:28.331 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/8390 2024-07-04T17:14:28.331 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/8390/8390.ko 2024-07-04T17:14:28.332 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-07-04T17:14:28.332 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-07-04T17:14:28.332 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-07-04T17:14:28.332 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/adaptec 2024-07-04T17:14:28.332 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-07-04T17:14:28.332 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/agere 2024-07-04T17:14:28.332 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/agere/et131x.ko 2024-07-04T17:14:28.332 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/alteon 2024-07-04T17:14:28.332 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-07-04T17:14:28.332 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/altera 2024-07-04T17:14:28.332 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-07-04T17:14:28.332 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/amd 2024-07-04T17:14:28.332 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-07-04T17:14:28.332 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-07-04T17:14:28.333 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-07-04T17:14:28.333 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/atheros 2024-07-04T17:14:28.333 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/atheros/alx 2024-07-04T17:14:28.333 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-07-04T17:14:28.333 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/atheros/atl1c 2024-07-04T17:14:28.333 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-07-04T17:14:28.333 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/atheros/atl1e 2024-07-04T17:14:28.333 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-07-04T17:14:28.333 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/atheros/atlx 2024-07-04T17:14:28.333 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-07-04T17:14:28.333 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-07-04T17:14:28.333 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/broadcom 2024-07-04T17:14:28.333 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-07-04T17:14:28.333 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-07-04T17:14:28.334 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-07-04T17:14:28.334 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-07-04T17:14:28.334 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-07-04T17:14:28.334 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/broadcom/genet 2024-07-04T17:14:28.334 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-07-04T17:14:28.334 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-07-04T17:14:28.334 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/brocade 2024-07-04T17:14:28.334 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/brocade/bna 2024-07-04T17:14:28.334 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-07-04T17:14:28.334 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/chelsio 2024-07-04T17:14:28.334 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/chelsio/cxgb 2024-07-04T17:14:28.334 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-07-04T17:14:28.334 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-07-04T17:14:28.335 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-07-04T17:14:28.335 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-07-04T17:14:28.335 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-07-04T17:14:28.335 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-07-04T17:14:28.335 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-07-04T17:14:28.335 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-07-04T17:14:28.335 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-07-04T17:14:28.335 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/cisco 2024-07-04T17:14:28.335 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/cisco/enic 2024-07-04T17:14:28.335 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-07-04T17:14:28.335 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/dec 2024-07-04T17:14:28.335 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/dec/tulip 2024-07-04T17:14:28.335 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-07-04T17:14:28.335 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-07-04T17:14:28.335 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-07-04T17:14:28.336 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-07-04T17:14:28.336 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-07-04T17:14:28.336 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-07-04T17:14:28.336 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/dlink 2024-07-04T17:14:28.336 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-07-04T17:14:28.336 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-07-04T17:14:28.336 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/dnet.ko 2024-07-04T17:14:28.336 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/emulex 2024-07-04T17:14:28.336 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/emulex/benet 2024-07-04T17:14:28.336 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-07-04T17:14:28.336 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/ethoc.ko 2024-07-04T17:14:28.336 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/fealnx.ko 2024-07-04T17:14:28.336 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel 2024-07-04T17:14:28.337 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/e100.ko 2024-07-04T17:14:28.337 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/e1000 2024-07-04T17:14:28.337 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-07-04T17:14:28.337 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/e1000e 2024-07-04T17:14:28.337 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-07-04T17:14:28.337 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/fm10k 2024-07-04T17:14:28.337 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-07-04T17:14:28.337 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/i40e 2024-07-04T17:14:28.337 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-07-04T17:14:28.337 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/iavf 2024-07-04T17:14:28.337 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-07-04T17:14:28.337 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/igb 2024-07-04T17:14:28.337 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-07-04T17:14:28.338 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/igbvf 2024-07-04T17:14:28.338 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-07-04T17:14:28.338 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/ixgbe 2024-07-04T17:14:28.338 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-07-04T17:14:28.338 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/ixgbevf 2024-07-04T17:14:28.338 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-07-04T17:14:28.338 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/jme.ko 2024-07-04T17:14:28.338 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/marvell 2024-07-04T17:14:28.338 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-07-04T17:14:28.338 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/marvell/skge.ko 2024-07-04T17:14:28.338 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-07-04T17:14:28.338 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/mellanox 2024-07-04T17:14:28.339 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/mellanox/mlx4 2024-07-04T17:14:28.339 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-07-04T17:14:28.339 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-07-04T17:14:28.339 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/mellanox/mlx5 2024-07-04T17:14:28.339 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-07-04T17:14:28.339 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-07-04T17:14:28.339 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/micrel 2024-07-04T17:14:28.339 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-07-04T17:14:28.339 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/myricom 2024-07-04T17:14:28.339 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/myricom/myri10ge 2024-07-04T17:14:28.339 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-07-04T17:14:28.339 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/natsemi 2024-07-04T17:14:28.339 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-07-04T17:14:28.340 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-07-04T17:14:28.340 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/neterion 2024-07-04T17:14:28.340 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-07-04T17:14:28.340 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/nvidia 2024-07-04T17:14:28.340 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-07-04T17:14:28.340 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/packetengines 2024-07-04T17:14:28.340 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-07-04T17:14:28.340 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-07-04T17:14:28.340 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/qlogic 2024-07-04T17:14:28.340 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/qlogic/netxen 2024-07-04T17:14:28.340 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-07-04T17:14:28.340 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-07-04T17:14:28.340 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-07-04T17:14:28.341 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-07-04T17:14:28.341 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/rdc 2024-07-04T17:14:28.341 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-07-04T17:14:28.341 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/realtek 2024-07-04T17:14:28.341 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-07-04T17:14:28.341 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-07-04T17:14:28.341 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/realtek/atp.ko 2024-07-04T17:14:28.341 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-07-04T17:14:28.341 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/rocker 2024-07-04T17:14:28.341 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-07-04T17:14:28.341 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/sfc 2024-07-04T17:14:28.341 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-07-04T17:14:28.342 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/silan 2024-07-04T17:14:28.342 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-07-04T17:14:28.342 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/sis 2024-07-04T17:14:28.342 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/sis/sis190.ko 2024-07-04T17:14:28.342 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/sis/sis900.ko 2024-07-04T17:14:28.342 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/smsc 2024-07-04T17:14:28.342 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-07-04T17:14:28.342 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-07-04T17:14:28.342 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-07-04T17:14:28.342 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-07-04T17:14:28.342 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/stmicro 2024-07-04T17:14:28.342 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/stmicro/stmmac 2024-07-04T17:14:28.342 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-07-04T17:14:28.343 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-07-04T17:14:28.343 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/sun 2024-07-04T17:14:28.343 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/sun/cassini.ko 2024-07-04T17:14:28.343 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/sun/niu.ko 2024-07-04T17:14:28.343 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/sun/sungem.ko 2024-07-04T17:14:28.343 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-07-04T17:14:28.343 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/tehuti 2024-07-04T17:14:28.343 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-07-04T17:14:28.343 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/ti 2024-07-04T17:14:28.343 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/ti/tlan.ko 2024-07-04T17:14:28.343 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/via 2024-07-04T17:14:28.343 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-07-04T17:14:28.343 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-07-04T17:14:28.344 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/wiznet 2024-07-04T17:14:28.344 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-07-04T17:14:28.344 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-07-04T17:14:28.344 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/xircom 2024-07-04T17:14:28.344 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-07-04T17:14:28.344 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/geneve.ko 2024-07-04T17:14:28.344 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/hyperv 2024-07-04T17:14:28.344 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-07-04T17:14:28.344 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ieee802154 2024-07-04T17:14:28.344 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ieee802154/fakelb.ko 2024-07-04T17:14:28.344 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ifb.ko 2024-07-04T17:14:28.344 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ipvlan 2024-07-04T17:14:28.344 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ipvlan/ipvlan.ko 2024-07-04T17:14:28.345 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/macvlan.ko 2024-07-04T17:14:28.345 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/macvtap.ko 2024-07-04T17:14:28.345 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/mdio 2024-07-04T17:14:28.345 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/mdio.ko 2024-07-04T17:14:28.345 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-07-04T17:14:28.345 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-07-04T17:14:28.345 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/mdio/mdio-regmap.ko 2024-07-04T17:14:28.345 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/mii.ko 2024-07-04T17:14:28.345 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/net_failover.ko 2024-07-04T17:14:28.345 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/netconsole.ko 2024-07-04T17:14:28.345 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/nlmon.ko 2024-07-04T17:14:28.345 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/pcs 2024-07-04T17:14:28.346 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/pcs/pcs-lynx.ko 2024-07-04T17:14:28.346 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-07-04T17:14:28.346 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy 2024-07-04T17:14:28.346 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/amd.ko 2024-07-04T17:14:28.346 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-07-04T17:14:28.346 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-07-04T17:14:28.346 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/bcm7xxx.ko 2024-07-04T17:14:28.346 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/bcm87xx.ko 2024-07-04T17:14:28.346 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/broadcom.ko 2024-07-04T17:14:28.346 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/cicada.ko 2024-07-04T17:14:28.346 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/davicom.ko 2024-07-04T17:14:28.347 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/dp83640.ko 2024-07-04T17:14:28.347 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/et1011c.ko 2024-07-04T17:14:28.347 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/icplus.ko 2024-07-04T17:14:28.347 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/lxt.ko 2024-07-04T17:14:28.347 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/marvell.ko 2024-07-04T17:14:28.347 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/micrel.ko 2024-07-04T17:14:28.347 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/national.ko 2024-07-04T17:14:28.347 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/phylink.ko 2024-07-04T17:14:28.347 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/qsemi.ko 2024-07-04T17:14:28.347 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/realtek.ko 2024-07-04T17:14:28.347 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/smsc.ko 2024-07-04T17:14:28.347 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/ste10Xp.ko 2024-07-04T17:14:28.347 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/vitesse.ko 2024-07-04T17:14:28.348 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ppp 2024-07-04T17:14:28.348 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ppp/bsd_comp.ko 2024-07-04T17:14:28.348 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ppp/ppp_async.ko 2024-07-04T17:14:28.348 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ppp/ppp_deflate.ko 2024-07-04T17:14:28.348 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ppp/ppp_generic.ko 2024-07-04T17:14:28.348 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ppp/ppp_mppe.ko 2024-07-04T17:14:28.348 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ppp/ppp_synctty.ko 2024-07-04T17:14:28.348 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ppp/pppoe.ko 2024-07-04T17:14:28.348 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ppp/pppox.ko 2024-07-04T17:14:28.348 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ppp/pptp.ko 2024-07-04T17:14:28.348 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/slip 2024-07-04T17:14:28.348 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/slip/slhc.ko 2024-07-04T17:14:28.348 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/slip/slip.ko 2024-07-04T17:14:28.349 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/sungem_phy.ko 2024-07-04T17:14:28.349 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/tap.ko 2024-07-04T17:14:28.349 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/team 2024-07-04T17:14:28.349 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/team/team.ko 2024-07-04T17:14:28.349 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/team/team_mode_activebackup.ko 2024-07-04T17:14:28.349 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/team/team_mode_broadcast.ko 2024-07-04T17:14:28.349 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-07-04T17:14:28.349 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/team/team_mode_random.ko 2024-07-04T17:14:28.349 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-07-04T17:14:28.349 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/tun.ko 2024-07-04T17:14:28.349 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/veth.ko 2024-07-04T17:14:28.349 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/virtio_net.ko 2024-07-04T17:14:28.349 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/vmxnet3 2024-07-04T17:14:28.349 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-07-04T17:14:28.349 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/vxlan 2024-07-04T17:14:28.350 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/vxlan/vxlan.ko 2024-07-04T17:14:28.350 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/xen-netback 2024-07-04T17:14:28.350 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/xen-netback/xen-netback.ko 2024-07-04T17:14:28.350 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/xen-netfront.ko 2024-07-04T17:14:28.350 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ntb 2024-07-04T17:14:28.350 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ntb/ntb.ko 2024-07-04T17:14:28.350 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/nvme 2024-07-04T17:14:28.350 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/nvme/host 2024-07-04T17:14:28.350 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/nvme/host/nvme-core.ko 2024-07-04T17:14:28.350 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/nvme/host/nvme.ko 2024-07-04T17:14:28.350 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/parport 2024-07-04T17:14:28.350 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/parport/parport.ko 2024-07-04T17:14:28.350 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/parport/parport_cs.ko 2024-07-04T17:14:28.350 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/parport/parport_pc.ko 2024-07-04T17:14:28.351 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/parport/parport_serial.ko 2024-07-04T17:14:28.351 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pci 2024-07-04T17:14:28.351 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pci/hotplug 2024-07-04T17:14:28.351 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-07-04T17:14:28.351 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pci/pcie 2024-07-04T17:14:28.351 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pci/pcie/aer_inject.ko 2024-07-04T17:14:28.351 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pci/xen-pcifront.ko 2024-07-04T17:14:28.351 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pcmcia 2024-07-04T17:14:28.351 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pcmcia/i82092.ko 2024-07-04T17:14:28.351 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pcmcia/pd6729.ko 2024-07-04T17:14:28.351 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pcmcia/yenta_socket.ko 2024-07-04T17:14:28.351 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pinctrl 2024-07-04T17:14:28.351 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pinctrl/intel 2024-07-04T17:14:28.351 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-07-04T17:14:28.351 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform 2024-07-04T17:14:28.352 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86 2024-07-04T17:14:28.352 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/acer-wmi.ko 2024-07-04T17:14:28.352 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/acerhdf.ko 2024-07-04T17:14:28.352 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/apple-gmux.ko 2024-07-04T17:14:28.352 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/asus-laptop.ko 2024-07-04T17:14:28.352 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-07-04T17:14:28.352 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/asus-wmi.ko 2024-07-04T17:14:28.352 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/classmate-laptop.ko 2024-07-04T17:14:28.352 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-07-04T17:14:28.352 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-07-04T17:14:28.352 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-07-04T17:14:28.352 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-07-04T17:14:28.352 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/hdaps.ko 2024-07-04T17:14:28.353 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/intel 2024-07-04T17:14:28.353 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-07-04T17:14:28.353 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/intel_ips.ko 2024-07-04T17:14:28.353 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/msi-wmi.ko 2024-07-04T17:14:28.353 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/mxm-wmi.ko 2024-07-04T17:14:28.353 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-07-04T17:14:28.353 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/samsung-laptop.ko 2024-07-04T17:14:28.353 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/samsung-q10.ko 2024-07-04T17:14:28.353 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/topstar-laptop.ko 2024-07-04T17:14:28.353 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-07-04T17:14:28.353 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/toshiba_haps.ko 2024-07-04T17:14:28.353 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/wmi-bmof.ko 2024-07-04T17:14:28.353 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/wmi.ko 2024-07-04T17:14:28.353 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/powercap 2024-07-04T17:14:28.354 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/powercap/intel_rapl_common.ko 2024-07-04T17:14:28.354 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/powercap/intel_rapl_msr.ko 2024-07-04T17:14:28.354 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pps 2024-07-04T17:14:28.354 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pps/clients 2024-07-04T17:14:28.354 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pps/clients/pps-gpio.ko 2024-07-04T17:14:28.354 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pps/clients/pps-ldisc.ko 2024-07-04T17:14:28.354 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pps/clients/pps_parport.ko 2024-07-04T17:14:28.354 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pps/pps_core.ko 2024-07-04T17:14:28.354 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ptp 2024-07-04T17:14:28.354 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ptp/ptp.ko 2024-07-04T17:14:28.354 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ptp/ptp_kvm.ko 2024-07-04T17:14:28.354 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ras 2024-07-04T17:14:28.355 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ras/amd 2024-07-04T17:14:28.355 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ras/amd/atl 2024-07-04T17:14:28.355 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-07-04T17:14:28.355 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ras/amd/fmpm.ko 2024-07-04T17:14:28.355 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc 2024-07-04T17:14:28.355 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-bq32k.ko 2024-07-04T17:14:28.355 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-ds1286.ko 2024-07-04T17:14:28.355 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-ds1307.ko 2024-07-04T17:14:28.355 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-ds1374.ko 2024-07-04T17:14:28.355 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-ds1511.ko 2024-07-04T17:14:28.355 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-ds1553.ko 2024-07-04T17:14:28.355 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-ds1672.ko 2024-07-04T17:14:28.355 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-ds1742.ko 2024-07-04T17:14:28.356 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-ds2404.ko 2024-07-04T17:14:28.356 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-ds3232.ko 2024-07-04T17:14:28.356 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-em3027.ko 2024-07-04T17:14:28.356 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-fm3130.ko 2024-07-04T17:14:28.356 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-isl12022.ko 2024-07-04T17:14:28.356 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-isl1208.ko 2024-07-04T17:14:28.356 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-m41t80.ko 2024-07-04T17:14:28.356 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-m48t35.ko 2024-07-04T17:14:28.356 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-m48t59.ko 2024-07-04T17:14:28.356 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-max6900.ko 2024-07-04T17:14:28.356 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-msm6242.ko 2024-07-04T17:14:28.356 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-pcf2127.ko 2024-07-04T17:14:28.356 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-pcf85063.ko 2024-07-04T17:14:28.357 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-pcf8523.ko 2024-07-04T17:14:28.357 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-pcf8563.ko 2024-07-04T17:14:28.357 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-pcf8583.ko 2024-07-04T17:14:28.357 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-rp5c01.ko 2024-07-04T17:14:28.357 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-rs5c372.ko 2024-07-04T17:14:28.357 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-07-04T17:14:28.357 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-rx8025.ko 2024-07-04T17:14:28.357 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-rx8581.ko 2024-07-04T17:14:28.357 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-07-04T17:14:28.357 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-x1205.ko 2024-07-04T17:14:28.357 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi 2024-07-04T17:14:28.357 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/3w-9xxx.ko 2024-07-04T17:14:28.357 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/3w-sas.ko 2024-07-04T17:14:28.357 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/3w-xxxx.ko 2024-07-04T17:14:28.357 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/BusLogic.ko 2024-07-04T17:14:28.358 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/a100u2w.ko 2024-07-04T17:14:28.358 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/aacraid 2024-07-04T17:14:28.358 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/aacraid/aacraid.ko 2024-07-04T17:14:28.358 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/advansys.ko 2024-07-04T17:14:28.358 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/aic7xxx 2024-07-04T17:14:28.358 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-07-04T17:14:28.358 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-07-04T17:14:28.358 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/am53c974.ko 2024-07-04T17:14:28.358 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/arcmsr 2024-07-04T17:14:28.358 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-07-04T17:14:28.358 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/atp870u.ko 2024-07-04T17:14:28.358 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/be2iscsi 2024-07-04T17:14:28.358 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-07-04T17:14:28.359 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/bfa 2024-07-04T17:14:28.359 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/bfa/bfa.ko 2024-07-04T17:14:28.359 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/bnx2fc 2024-07-04T17:14:28.359 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-07-04T17:14:28.359 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/bnx2i 2024-07-04T17:14:28.359 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-07-04T17:14:28.359 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/ch.ko 2024-07-04T17:14:28.359 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/csiostor 2024-07-04T17:14:28.359 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/csiostor/csiostor.ko 2024-07-04T17:14:28.359 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/cxgbi 2024-07-04T17:14:28.359 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/cxgbi/cxgb3i 2024-07-04T17:14:28.359 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-07-04T17:14:28.359 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/cxgbi/cxgb4i 2024-07-04T17:14:28.359 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-07-04T17:14:28.359 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-07-04T17:14:28.360 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/dc395x.ko 2024-07-04T17:14:28.360 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/device_handler 2024-07-04T17:14:28.360 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-07-04T17:14:28.360 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-07-04T17:14:28.360 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-07-04T17:14:28.360 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-07-04T17:14:28.360 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/dmx3191d.ko 2024-07-04T17:14:28.360 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/esas2r 2024-07-04T17:14:28.360 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/esas2r/esas2r.ko 2024-07-04T17:14:28.360 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/esp_scsi.ko 2024-07-04T17:14:28.360 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/fcoe 2024-07-04T17:14:28.360 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/fcoe/fcoe.ko 2024-07-04T17:14:28.360 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-07-04T17:14:28.360 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/fnic 2024-07-04T17:14:28.360 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/fnic/fnic.ko 2024-07-04T17:14:28.361 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/hpsa.ko 2024-07-04T17:14:28.361 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/hptiop.ko 2024-07-04T17:14:28.361 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/hv_storvsc.ko 2024-07-04T17:14:28.361 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/initio.ko 2024-07-04T17:14:28.361 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/ipr.ko 2024-07-04T17:14:28.361 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/ips.ko 2024-07-04T17:14:28.361 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/isci 2024-07-04T17:14:28.361 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/isci/isci.ko 2024-07-04T17:14:28.361 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-07-04T17:14:28.361 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/iscsi_tcp.ko 2024-07-04T17:14:28.361 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/libfc 2024-07-04T17:14:28.361 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/libfc/libfc.ko 2024-07-04T17:14:28.361 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/libiscsi.ko 2024-07-04T17:14:28.361 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/libiscsi_tcp.ko 2024-07-04T17:14:28.361 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/libsas 2024-07-04T17:14:28.361 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/libsas/libsas.ko 2024-07-04T17:14:28.362 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/lpfc 2024-07-04T17:14:28.362 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/lpfc/lpfc.ko 2024-07-04T17:14:28.362 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/megaraid 2024-07-04T17:14:28.362 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/megaraid.ko 2024-07-04T17:14:28.362 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-07-04T17:14:28.362 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-07-04T17:14:28.362 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-07-04T17:14:28.362 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/mpt3sas 2024-07-04T17:14:28.362 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-07-04T17:14:28.362 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/mvsas 2024-07-04T17:14:28.362 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/mvsas/mvsas.ko 2024-07-04T17:14:28.362 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/mvumi.ko 2024-07-04T17:14:28.362 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/pm8001 2024-07-04T17:14:28.362 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-07-04T17:14:28.363 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/pmcraid.ko 2024-07-04T17:14:28.363 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/qla1280.ko 2024-07-04T17:14:28.363 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/qla2xxx 2024-07-04T17:14:28.363 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-07-04T17:14:28.363 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-07-04T17:14:28.363 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/qla4xxx 2024-07-04T17:14:28.363 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-07-04T17:14:28.363 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/raid_class.ko 2024-07-04T17:14:28.363 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/scsi_debug.ko 2024-07-04T17:14:28.363 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/scsi_transport_fc.ko 2024-07-04T17:14:28.363 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-07-04T17:14:28.363 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/scsi_transport_sas.ko 2024-07-04T17:14:28.363 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/scsi_transport_spi.ko 2024-07-04T17:14:28.363 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/scsi_transport_srp.ko 2024-07-04T17:14:28.363 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/ses.ko 2024-07-04T17:14:28.363 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/st.ko 2024-07-04T17:14:28.364 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/stex.ko 2024-07-04T17:14:28.364 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/sym53c8xx_2 2024-07-04T17:14:28.364 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-07-04T17:14:28.364 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/virtio_scsi.ko 2024-07-04T17:14:28.364 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/vmw_pvscsi.ko 2024-07-04T17:14:28.364 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/wd719x.ko 2024-07-04T17:14:28.364 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ssb 2024-07-04T17:14:28.364 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ssb/ssb.ko 2024-07-04T17:14:28.364 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/target 2024-07-04T17:14:28.364 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/target/iscsi 2024-07-04T17:14:28.364 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-07-04T17:14:28.364 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/target/loopback 2024-07-04T17:14:28.364 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/target/loopback/tcm_loop.ko 2024-07-04T17:14:28.364 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/target/target_core_file.ko 2024-07-04T17:14:28.364 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/target/target_core_iblock.ko 2024-07-04T17:14:28.365 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/target/target_core_mod.ko 2024-07-04T17:14:28.365 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/target/target_core_pscsi.ko 2024-07-04T17:14:28.365 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/target/target_core_user.ko 2024-07-04T17:14:28.365 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/target/tcm_fc 2024-07-04T17:14:28.365 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-07-04T17:14:28.365 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal 2024-07-04T17:14:28.365 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel 2024-07-04T17:14:28.365 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/int340x_thermal 2024-07-04T17:14:28.365 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-07-04T17:14:28.365 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-07-04T17:14:28.365 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-07-04T17:14:28.365 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-07-04T17:14:28.365 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-07-04T17:14:28.365 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-07-04T17:14:28.365 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-07-04T17:14:28.366 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-07-04T17:14:28.366 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-07-04T17:14:28.366 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-07-04T17:14:28.366 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-07-04T17:14:28.366 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-07-04T17:14:28.366 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-07-04T17:14:28.366 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-07-04T17:14:28.366 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-07-04T17:14:28.366 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-07-04T17:14:28.366 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-07-04T17:14:28.366 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-07-04T17:14:28.366 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/tty 2024-07-04T17:14:28.366 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/tty/ipwireless 2024-07-04T17:14:28.367 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-07-04T17:14:28.367 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/tty/n_gsm.ko 2024-07-04T17:14:28.367 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/tty/n_hdlc.ko 2024-07-04T17:14:28.367 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/tty/nozomi.ko 2024-07-04T17:14:28.367 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/tty/serial 2024-07-04T17:14:28.367 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/tty/serial/8250 2024-07-04T17:14:28.367 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-07-04T17:14:28.367 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/tty/serial/arc_uart.ko 2024-07-04T17:14:28.367 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/tty/serial/jsm 2024-07-04T17:14:28.367 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/tty/serial/jsm/jsm.ko 2024-07-04T17:14:28.367 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ufs 2024-07-04T17:14:28.367 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ufs/core 2024-07-04T17:14:28.367 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ufs/core/ufshcd-core.ko 2024-07-04T17:14:28.367 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ufs/host 2024-07-04T17:14:28.368 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-07-04T17:14:28.368 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/uio 2024-07-04T17:14:28.368 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/uio/uio.ko 2024-07-04T17:14:28.368 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/uio/uio_aec.ko 2024-07-04T17:14:28.368 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/uio/uio_cif.ko 2024-07-04T17:14:28.368 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/uio/uio_pci_generic.ko 2024-07-04T17:14:28.368 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/uio/uio_sercos3.ko 2024-07-04T17:14:28.368 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb 2024-07-04T17:14:28.368 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/class 2024-07-04T17:14:28.368 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/class/cdc-acm.ko 2024-07-04T17:14:28.368 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/class/cdc-wdm.ko 2024-07-04T17:14:28.368 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/class/usblp.ko 2024-07-04T17:14:28.368 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/class/usbtmc.ko 2024-07-04T17:14:28.368 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/host 2024-07-04T17:14:28.368 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/host/sl811-hcd.ko 2024-07-04T17:14:28.369 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/image 2024-07-04T17:14:28.369 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/image/mdc800.ko 2024-07-04T17:14:28.369 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/image/microtek.ko 2024-07-04T17:14:28.369 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc 2024-07-04T17:14:28.369 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/adutux.ko 2024-07-04T17:14:28.369 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/appledisplay.ko 2024-07-04T17:14:28.369 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/emi26.ko 2024-07-04T17:14:28.369 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/emi62.ko 2024-07-04T17:14:28.369 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/ezusb.ko 2024-07-04T17:14:28.369 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/idmouse.ko 2024-07-04T17:14:28.369 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/iowarrior.ko 2024-07-04T17:14:28.369 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/isight_firmware.ko 2024-07-04T17:14:28.369 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/ldusb.ko 2024-07-04T17:14:28.369 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/legousbtower.ko 2024-07-04T17:14:28.369 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/sisusbvga 2024-07-04T17:14:28.369 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-07-04T17:14:28.370 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/trancevibrator.ko 2024-07-04T17:14:28.370 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/usb3503.ko 2024-07-04T17:14:28.370 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/usblcd.ko 2024-07-04T17:14:28.370 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/usbsevseg.ko 2024-07-04T17:14:28.370 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/uss720.ko 2024-07-04T17:14:28.370 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/yurex.ko 2024-07-04T17:14:28.370 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/phy 2024-07-04T17:14:28.370 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/phy/phy-generic.ko 2024-07-04T17:14:28.370 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial 2024-07-04T17:14:28.370 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/aircable.ko 2024-07-04T17:14:28.370 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/ark3116.ko 2024-07-04T17:14:28.370 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/belkin_sa.ko 2024-07-04T17:14:28.370 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/ch341.ko 2024-07-04T17:14:28.370 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/cp210x.ko 2024-07-04T17:14:28.370 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/cyberjack.ko 2024-07-04T17:14:28.371 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/cypress_m8.ko 2024-07-04T17:14:28.371 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/digi_acceleport.ko 2024-07-04T17:14:28.371 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/empeg.ko 2024-07-04T17:14:28.371 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/ftdi_sio.ko 2024-07-04T17:14:28.371 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/garmin_gps.ko 2024-07-04T17:14:28.371 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/io_edgeport.ko 2024-07-04T17:14:28.371 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/io_ti.ko 2024-07-04T17:14:28.371 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/ipaq.ko 2024-07-04T17:14:28.371 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/ipw.ko 2024-07-04T17:14:28.371 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/ir-usb.ko 2024-07-04T17:14:28.371 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-07-04T17:14:28.371 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/keyspan.ko 2024-07-04T17:14:28.371 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/keyspan_pda.ko 2024-07-04T17:14:28.371 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/kl5kusb105.ko 2024-07-04T17:14:28.371 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/kobil_sct.ko 2024-07-04T17:14:28.372 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/mct_u232.ko 2024-07-04T17:14:28.372 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/mos7720.ko 2024-07-04T17:14:28.372 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/mos7840.ko 2024-07-04T17:14:28.372 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/navman.ko 2024-07-04T17:14:28.372 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/omninet.ko 2024-07-04T17:14:28.372 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/opticon.ko 2024-07-04T17:14:28.372 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/option.ko 2024-07-04T17:14:28.372 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/oti6858.ko 2024-07-04T17:14:28.372 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/pl2303.ko 2024-07-04T17:14:28.372 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/qcaux.ko 2024-07-04T17:14:28.372 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/qcserial.ko 2024-07-04T17:14:28.372 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/quatech2.ko 2024-07-04T17:14:28.372 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/safe_serial.ko 2024-07-04T17:14:28.372 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/sierra.ko 2024-07-04T17:14:28.373 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/spcp8x5.ko 2024-07-04T17:14:28.373 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/ssu100.ko 2024-07-04T17:14:28.373 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/symbolserial.ko 2024-07-04T17:14:28.373 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-07-04T17:14:28.373 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-07-04T17:14:28.373 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/usb_debug.ko 2024-07-04T17:14:28.373 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/usb_wwan.ko 2024-07-04T17:14:28.373 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/visor.ko 2024-07-04T17:14:28.373 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/whiteheat.ko 2024-07-04T17:14:28.373 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/xsens_mt.ko 2024-07-04T17:14:28.373 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/storage 2024-07-04T17:14:28.373 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/storage/uas.ko 2024-07-04T17:14:28.373 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/storage/ums-alauda.ko 2024-07-04T17:14:28.373 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/storage/ums-cypress.ko 2024-07-04T17:14:28.374 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/storage/ums-datafab.ko 2024-07-04T17:14:28.374 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-07-04T17:14:28.374 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/storage/ums-freecom.ko 2024-07-04T17:14:28.374 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/storage/ums-isd200.ko 2024-07-04T17:14:28.374 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-07-04T17:14:28.374 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/storage/ums-karma.ko 2024-07-04T17:14:28.374 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/storage/ums-onetouch.ko 2024-07-04T17:14:28.374 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/storage/ums-realtek.ko 2024-07-04T17:14:28.374 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/storage/ums-sddr09.ko 2024-07-04T17:14:28.374 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/storage/ums-sddr55.ko 2024-07-04T17:14:28.374 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/storage/ums-usbat.ko 2024-07-04T17:14:28.374 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/storage/usb-storage.ko 2024-07-04T17:14:28.374 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/usbip 2024-07-04T17:14:28.374 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/usbip/usbip-core.ko 2024-07-04T17:14:28.375 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/usbip/usbip-host.ko 2024-07-04T17:14:28.375 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-07-04T17:14:28.375 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/vfio 2024-07-04T17:14:28.375 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/vfio/pci 2024-07-04T17:14:28.375 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-07-04T17:14:28.375 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/vfio/pci/vfio-pci.ko 2024-07-04T17:14:28.375 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/vfio/vfio.ko 2024-07-04T17:14:28.375 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-07-04T17:14:28.375 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/vhost 2024-07-04T17:14:28.375 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/vhost/vhost.ko 2024-07-04T17:14:28.375 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/vhost/vhost_iotlb.ko 2024-07-04T17:14:28.375 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/vhost/vhost_net.ko 2024-07-04T17:14:28.375 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/vhost/vhost_scsi.ko 2024-07-04T17:14:28.376 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/video 2024-07-04T17:14:28.376 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/video/backlight 2024-07-04T17:14:28.376 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/video/backlight/apple_bl.ko 2024-07-04T17:14:28.376 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/video/backlight/lcd.ko 2024-07-04T17:14:28.376 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/video/backlight/platform_lcd.ko 2024-07-04T17:14:28.376 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/virtio 2024-07-04T17:14:28.376 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/virtio/virtio.ko 2024-07-04T17:14:28.376 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/virtio/virtio_balloon.ko 2024-07-04T17:14:28.376 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/virtio/virtio_mmio.ko 2024-07-04T17:14:28.376 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/virtio/virtio_pci.ko 2024-07-04T17:14:28.376 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-07-04T17:14:28.376 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-07-04T17:14:28.376 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/virtio/virtio_ring.ko 2024-07-04T17:14:28.376 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1 2024-07-04T17:14:28.377 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1/masters 2024-07-04T17:14:28.377 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1/masters/ds2482.ko 2024-07-04T17:14:28.377 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1/masters/ds2490.ko 2024-07-04T17:14:28.377 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1/slaves 2024-07-04T17:14:28.377 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-07-04T17:14:28.377 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-07-04T17:14:28.377 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-07-04T17:14:28.377 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-07-04T17:14:28.377 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-07-04T17:14:28.377 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-07-04T17:14:28.377 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-07-04T17:14:28.377 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-07-04T17:14:28.377 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-07-04T17:14:28.378 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1/slaves/w1_smem.ko 2024-07-04T17:14:28.378 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1/slaves/w1_therm.ko 2024-07-04T17:14:28.378 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1/wire.ko 2024-07-04T17:14:28.378 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog 2024-07-04T17:14:28.378 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/alim1535_wdt.ko 2024-07-04T17:14:28.378 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/alim7101_wdt.ko 2024-07-04T17:14:28.378 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/f71808e_wdt.ko 2024-07-04T17:14:28.378 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/hpwdt.ko 2024-07-04T17:14:28.378 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/i6300esb.ko 2024-07-04T17:14:28.378 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-07-04T17:14:28.378 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/iTCO_wdt.ko 2024-07-04T17:14:28.378 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/ib700wdt.ko 2024-07-04T17:14:28.378 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/ibmasr.ko 2024-07-04T17:14:28.379 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-07-04T17:14:28.379 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/it8712f_wdt.ko 2024-07-04T17:14:28.379 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/it87_wdt.ko 2024-07-04T17:14:28.379 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/machzwd.ko 2024-07-04T17:14:28.379 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/nv_tco.ko 2024-07-04T17:14:28.379 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/pcwd_pci.ko 2024-07-04T17:14:28.379 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/pcwd_usb.ko 2024-07-04T17:14:28.379 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-07-04T17:14:28.379 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/sch311x_wdt.ko 2024-07-04T17:14:28.379 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/softdog.ko 2024-07-04T17:14:28.379 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/sp5100_tco.ko 2024-07-04T17:14:28.379 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/via_wdt.ko 2024-07-04T17:14:28.379 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-07-04T17:14:28.379 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/w83877f_wdt.ko 2024-07-04T17:14:28.380 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/w83977f_wdt.ko 2024-07-04T17:14:28.380 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/wdt_pci.ko 2024-07-04T17:14:28.380 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/xen_wdt.ko 2024-07-04T17:14:28.380 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/xen 2024-07-04T17:14:28.380 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/xen/xen-acpi-processor.ko 2024-07-04T17:14:28.380 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/xen/xen-evtchn.ko 2024-07-04T17:14:28.380 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/xen/xen-gntalloc.ko 2024-07-04T17:14:28.380 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/xen/xen-gntdev.ko 2024-07-04T17:14:28.380 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/xen/xen-pciback 2024-07-04T17:14:28.380 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-07-04T17:14:28.380 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/xen/xen-privcmd.ko 2024-07-04T17:14:28.380 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/xen/xenfs 2024-07-04T17:14:28.380 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/xen/xenfs/xenfs.ko 2024-07-04T17:14:28.380 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs 2024-07-04T17:14:28.381 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/9p 2024-07-04T17:14:28.381 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/9p/9p.ko 2024-07-04T17:14:28.381 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/affs 2024-07-04T17:14:28.381 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/affs/affs.ko 2024-07-04T17:14:28.381 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/befs 2024-07-04T17:14:28.381 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/befs/befs.ko 2024-07-04T17:14:28.381 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/binfmt_misc.ko 2024-07-04T17:14:28.381 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/btrfs 2024-07-04T17:14:28.381 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/btrfs/btrfs.ko 2024-07-04T17:14:28.381 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/cachefiles 2024-07-04T17:14:28.381 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/cachefiles/cachefiles.ko 2024-07-04T17:14:28.381 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ceph 2024-07-04T17:14:28.381 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ceph/ceph.ko 2024-07-04T17:14:28.382 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/coda 2024-07-04T17:14:28.382 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/coda/coda.ko 2024-07-04T17:14:28.382 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/cramfs 2024-07-04T17:14:28.382 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/cramfs/cramfs.ko 2024-07-04T17:14:28.382 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/dlm 2024-07-04T17:14:28.382 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/dlm/dlm.ko 2024-07-04T17:14:28.382 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ecryptfs 2024-07-04T17:14:28.382 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ecryptfs/ecryptfs.ko 2024-07-04T17:14:28.382 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/f2fs 2024-07-04T17:14:28.382 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/f2fs/f2fs.ko 2024-07-04T17:14:28.382 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/fat 2024-07-04T17:14:28.382 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/fat/fat.ko 2024-07-04T17:14:28.382 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/fat/msdos.ko 2024-07-04T17:14:28.382 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/fat/vfat.ko 2024-07-04T17:14:28.383 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/fuse 2024-07-04T17:14:28.383 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/fuse/cuse.ko 2024-07-04T17:14:28.383 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/fuse/fuse.ko 2024-07-04T17:14:28.383 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/gfs2 2024-07-04T17:14:28.383 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/gfs2/gfs2.ko 2024-07-04T17:14:28.383 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/hfs 2024-07-04T17:14:28.383 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/hfs/hfs.ko 2024-07-04T17:14:28.383 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/hfsplus 2024-07-04T17:14:28.383 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/hfsplus/hfsplus.ko 2024-07-04T17:14:28.383 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/isofs 2024-07-04T17:14:28.383 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/isofs/isofs.ko 2024-07-04T17:14:28.383 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/jfs 2024-07-04T17:14:28.383 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/jfs/jfs.ko 2024-07-04T17:14:28.384 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/lockd 2024-07-04T17:14:28.384 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/lockd/lockd.ko 2024-07-04T17:14:28.384 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/minix 2024-07-04T17:14:28.384 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/minix/minix.ko 2024-07-04T17:14:28.384 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/netfs 2024-07-04T17:14:28.384 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/netfs/netfs.ko 2024-07-04T17:14:28.384 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nfs 2024-07-04T17:14:28.384 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nfs/blocklayout 2024-07-04T17:14:28.384 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-07-04T17:14:28.384 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nfs/filelayout 2024-07-04T17:14:28.384 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-07-04T17:14:28.384 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nfs/flexfilelayout 2024-07-04T17:14:28.384 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-07-04T17:14:28.384 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nfs/nfs.ko 2024-07-04T17:14:28.385 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nfs/nfsv3.ko 2024-07-04T17:14:28.385 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nfs/nfsv4.ko 2024-07-04T17:14:28.385 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nfs_common 2024-07-04T17:14:28.385 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nfs_common/grace.ko 2024-07-04T17:14:28.385 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nfs_common/nfs_acl.ko 2024-07-04T17:14:28.385 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nfsd 2024-07-04T17:14:28.385 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nfsd/nfsd.ko 2024-07-04T17:14:28.385 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nilfs2 2024-07-04T17:14:28.385 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nilfs2/nilfs2.ko 2024-07-04T17:14:28.385 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls 2024-07-04T17:14:28.385 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/mac-celtic.ko 2024-07-04T17:14:28.385 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/mac-centeuro.ko 2024-07-04T17:14:28.385 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/mac-croatian.ko 2024-07-04T17:14:28.385 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/mac-cyrillic.ko 2024-07-04T17:14:28.385 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/mac-gaelic.ko 2024-07-04T17:14:28.386 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/mac-greek.ko 2024-07-04T17:14:28.386 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/mac-iceland.ko 2024-07-04T17:14:28.386 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/mac-inuit.ko 2024-07-04T17:14:28.386 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/mac-roman.ko 2024-07-04T17:14:28.386 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/mac-romanian.ko 2024-07-04T17:14:28.386 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/mac-turkish.ko 2024-07-04T17:14:28.386 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp1250.ko 2024-07-04T17:14:28.386 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp1251.ko 2024-07-04T17:14:28.386 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp1255.ko 2024-07-04T17:14:28.386 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp737.ko 2024-07-04T17:14:28.386 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp775.ko 2024-07-04T17:14:28.386 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp850.ko 2024-07-04T17:14:28.386 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp852.ko 2024-07-04T17:14:28.387 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp855.ko 2024-07-04T17:14:28.387 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp857.ko 2024-07-04T17:14:28.387 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp860.ko 2024-07-04T17:14:28.387 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp861.ko 2024-07-04T17:14:28.387 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp862.ko 2024-07-04T17:14:28.387 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp863.ko 2024-07-04T17:14:28.387 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp864.ko 2024-07-04T17:14:28.387 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp865.ko 2024-07-04T17:14:28.387 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp866.ko 2024-07-04T17:14:28.387 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp869.ko 2024-07-04T17:14:28.387 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp874.ko 2024-07-04T17:14:28.387 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp932.ko 2024-07-04T17:14:28.387 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp936.ko 2024-07-04T17:14:28.387 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp949.ko 2024-07-04T17:14:28.388 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp950.ko 2024-07-04T17:14:28.388 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_euc-jp.ko 2024-07-04T17:14:28.388 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_iso8859-1.ko 2024-07-04T17:14:28.388 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_iso8859-13.ko 2024-07-04T17:14:28.388 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_iso8859-14.ko 2024-07-04T17:14:28.388 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_iso8859-15.ko 2024-07-04T17:14:28.388 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_iso8859-2.ko 2024-07-04T17:14:28.388 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_iso8859-3.ko 2024-07-04T17:14:28.388 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_iso8859-4.ko 2024-07-04T17:14:28.388 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_iso8859-5.ko 2024-07-04T17:14:28.388 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_iso8859-6.ko 2024-07-04T17:14:28.388 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_iso8859-7.ko 2024-07-04T17:14:28.388 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_iso8859-9.ko 2024-07-04T17:14:28.388 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_koi8-r.ko 2024-07-04T17:14:28.388 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_koi8-ru.ko 2024-07-04T17:14:28.388 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_koi8-u.ko 2024-07-04T17:14:28.389 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_ucs2_utils.ko 2024-07-04T17:14:28.389 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_utf8.ko 2024-07-04T17:14:28.389 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ocfs2 2024-07-04T17:14:28.389 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ocfs2/cluster 2024-07-04T17:14:28.389 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-07-04T17:14:28.389 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ocfs2/dlm 2024-07-04T17:14:28.389 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-07-04T17:14:28.389 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ocfs2/dlmfs 2024-07-04T17:14:28.389 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-07-04T17:14:28.389 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ocfs2/ocfs2.ko 2024-07-04T17:14:28.389 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-07-04T17:14:28.389 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-07-04T17:14:28.389 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-07-04T17:14:28.390 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/overlayfs 2024-07-04T17:14:28.390 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/overlayfs/overlay.ko 2024-07-04T17:14:28.390 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/pstore 2024-07-04T17:14:28.390 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/pstore/ramoops.ko 2024-07-04T17:14:28.390 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/reiserfs 2024-07-04T17:14:28.390 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/reiserfs/reiserfs.ko 2024-07-04T17:14:28.390 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/romfs 2024-07-04T17:14:28.390 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/romfs/romfs.ko 2024-07-04T17:14:28.390 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/smb 2024-07-04T17:14:28.390 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/smb/client 2024-07-04T17:14:28.390 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/smb/client/cifs.ko 2024-07-04T17:14:28.390 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/smb/common 2024-07-04T17:14:28.390 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/smb/common/cifs_arc4.ko 2024-07-04T17:14:28.390 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/smb/common/cifs_md4.ko 2024-07-04T17:14:28.391 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/squashfs 2024-07-04T17:14:28.391 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/squashfs/squashfs.ko 2024-07-04T17:14:28.391 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/sysv 2024-07-04T17:14:28.391 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/sysv/sysv.ko 2024-07-04T17:14:28.391 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/udf 2024-07-04T17:14:28.391 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/udf/udf.ko 2024-07-04T17:14:28.391 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ufs 2024-07-04T17:14:28.391 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ufs/ufs.ko 2024-07-04T17:14:28.391 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/xfs 2024-07-04T17:14:28.391 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/xfs/xfs.ko 2024-07-04T17:14:28.391 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/kernel 2024-07-04T17:14:28.391 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/kernel/rcu 2024-07-04T17:14:28.391 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/kernel/rcu/rcutorture.ko 2024-07-04T17:14:28.391 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/kernel/torture.ko 2024-07-04T17:14:28.391 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/kernel/trace 2024-07-04T17:14:28.392 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-07-04T17:14:28.392 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib 2024-07-04T17:14:28.392 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/asn1_encoder.ko 2024-07-04T17:14:28.392 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/crc-itu-t.ko 2024-07-04T17:14:28.392 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/crc8.ko 2024-07-04T17:14:28.392 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/crypto 2024-07-04T17:14:28.392 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/crypto/libarc4.ko 2024-07-04T17:14:28.392 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/crypto/libdes.ko 2024-07-04T17:14:28.392 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/libcrc32c.ko 2024-07-04T17:14:28.392 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/lru_cache.ko 2024-07-04T17:14:28.392 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/lz4 2024-07-04T17:14:28.392 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/lz4/lz4_compress.ko 2024-07-04T17:14:28.392 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/lz4/lz4hc_compress.ko 2024-07-04T17:14:28.392 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/math 2024-07-04T17:14:28.392 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/math/cordic.ko 2024-07-04T17:14:28.393 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/raid6 2024-07-04T17:14:28.393 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/raid6/raid6_pq.ko 2024-07-04T17:14:28.393 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/reed_solomon 2024-07-04T17:14:28.393 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/reed_solomon/reed_solomon.ko 2024-07-04T17:14:28.393 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/ts_bm.ko 2024-07-04T17:14:28.393 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/ts_fsm.ko 2024-07-04T17:14:28.393 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/ts_kmp.ko 2024-07-04T17:14:28.393 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/zstd 2024-07-04T17:14:28.393 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/zstd/zstd_compress.ko 2024-07-04T17:14:28.393 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/mm 2024-07-04T17:14:28.393 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/mm/hwpoison-inject.ko 2024-07-04T17:14:28.393 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net 2024-07-04T17:14:28.393 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/6lowpan 2024-07-04T17:14:28.393 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/6lowpan/6lowpan.ko 2024-07-04T17:14:28.393 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/6lowpan/nhc_dest.ko 2024-07-04T17:14:28.393 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/6lowpan/nhc_fragment.ko 2024-07-04T17:14:28.394 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/6lowpan/nhc_hop.ko 2024-07-04T17:14:28.394 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/6lowpan/nhc_ipv6.ko 2024-07-04T17:14:28.394 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/6lowpan/nhc_mobility.ko 2024-07-04T17:14:28.394 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/6lowpan/nhc_routing.ko 2024-07-04T17:14:28.394 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/6lowpan/nhc_udp.ko 2024-07-04T17:14:28.394 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/802 2024-07-04T17:14:28.394 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/802/garp.ko 2024-07-04T17:14:28.394 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/802/mrp.ko 2024-07-04T17:14:28.394 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/802/p8022.ko 2024-07-04T17:14:28.394 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/802/psnap.ko 2024-07-04T17:14:28.394 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/802/stp.ko 2024-07-04T17:14:28.394 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/8021q 2024-07-04T17:14:28.394 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/8021q/8021q.ko 2024-07-04T17:14:28.394 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/9p 2024-07-04T17:14:28.394 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/9p/9pnet.ko 2024-07-04T17:14:28.395 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/9p/9pnet_fd.ko 2024-07-04T17:14:28.395 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/9p/9pnet_rdma.ko 2024-07-04T17:14:28.395 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/9p/9pnet_virtio.ko 2024-07-04T17:14:28.395 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/batman-adv 2024-07-04T17:14:28.395 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/batman-adv/batman-adv.ko 2024-07-04T17:14:28.395 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge 2024-07-04T17:14:28.395 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/br_netfilter.ko 2024-07-04T17:14:28.395 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/bridge.ko 2024-07-04T17:14:28.395 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter 2024-07-04T17:14:28.395 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-07-04T17:14:28.395 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_among.ko 2024-07-04T17:14:28.395 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_arp.ko 2024-07-04T17:14:28.395 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-07-04T17:14:28.395 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-07-04T17:14:28.395 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_ip.ko 2024-07-04T17:14:28.396 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-07-04T17:14:28.396 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_limit.ko 2024-07-04T17:14:28.396 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_log.ko 2024-07-04T17:14:28.396 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_mark.ko 2024-07-04T17:14:28.396 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-07-04T17:14:28.396 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-07-04T17:14:28.396 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-07-04T17:14:28.396 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-07-04T17:14:28.396 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_snat.ko 2024-07-04T17:14:28.396 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_stp.ko 2024-07-04T17:14:28.396 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-07-04T17:14:28.396 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-07-04T17:14:28.396 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-07-04T17:14:28.396 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-07-04T17:14:28.396 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebtables.ko 2024-07-04T17:14:28.396 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-07-04T17:14:28.397 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-07-04T17:14:28.397 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-07-04T17:14:28.397 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ceph 2024-07-04T17:14:28.397 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ceph/libceph.ko 2024-07-04T17:14:28.397 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/core 2024-07-04T17:14:28.397 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/core/failover.ko 2024-07-04T17:14:28.397 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/core/pktgen.ko 2024-07-04T17:14:28.397 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/dccp 2024-07-04T17:14:28.397 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/dccp/dccp.ko 2024-07-04T17:14:28.397 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/dccp/dccp_diag.ko 2024-07-04T17:14:28.397 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/dccp/dccp_ipv4.ko 2024-07-04T17:14:28.397 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/dccp/dccp_ipv6.ko 2024-07-04T17:14:28.397 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/dns_resolver 2024-07-04T17:14:28.397 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/dns_resolver/dns_resolver.ko 2024-07-04T17:14:28.397 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/dsa 2024-07-04T17:14:28.398 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/dsa/dsa_core.ko 2024-07-04T17:14:28.398 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/dsa/tag_brcm.ko 2024-07-04T17:14:28.398 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/dsa/tag_dsa.ko 2024-07-04T17:14:28.398 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/dsa/tag_none.ko 2024-07-04T17:14:28.398 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/dsa/tag_trailer.ko 2024-07-04T17:14:28.398 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ieee802154 2024-07-04T17:14:28.398 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ieee802154/6lowpan 2024-07-04T17:14:28.398 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-07-04T17:14:28.398 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ieee802154/ieee802154.ko 2024-07-04T17:14:28.398 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ieee802154/ieee802154_socket.ko 2024-07-04T17:14:28.398 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4 2024-07-04T17:14:28.398 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/ah4.ko 2024-07-04T17:14:28.398 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/esp4.ko 2024-07-04T17:14:28.399 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/fou.ko 2024-07-04T17:14:28.399 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/gre.ko 2024-07-04T17:14:28.399 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/inet_diag.ko 2024-07-04T17:14:28.399 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/ip_gre.ko 2024-07-04T17:14:28.399 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/ip_tunnel.ko 2024-07-04T17:14:28.399 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/ip_vti.ko 2024-07-04T17:14:28.399 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/ipcomp.ko 2024-07-04T17:14:28.399 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/ipip.ko 2024-07-04T17:14:28.399 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter 2024-07-04T17:14:28.399 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/arp_tables.ko 2024-07-04T17:14:28.399 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-07-04T17:14:28.399 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-07-04T17:14:28.399 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-07-04T17:14:28.399 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-07-04T17:14:28.400 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-07-04T17:14:28.400 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-07-04T17:14:28.400 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-07-04T17:14:28.400 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-07-04T17:14:28.400 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-07-04T17:14:28.400 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/iptable_security.ko 2024-07-04T17:14:28.400 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-07-04T17:14:28.400 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-07-04T17:14:28.400 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-07-04T17:14:28.400 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-07-04T17:14:28.400 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-07-04T17:14:28.400 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-07-04T17:14:28.401 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-07-04T17:14:28.401 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-07-04T17:14:28.401 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-07-04T17:14:28.401 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-07-04T17:14:28.401 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/tcp_bic.ko 2024-07-04T17:14:28.401 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/tcp_dctcp.ko 2024-07-04T17:14:28.401 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/tcp_diag.ko 2024-07-04T17:14:28.401 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/tcp_highspeed.ko 2024-07-04T17:14:28.401 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/tcp_htcp.ko 2024-07-04T17:14:28.401 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/tcp_hybla.ko 2024-07-04T17:14:28.401 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/tcp_illinois.ko 2024-07-04T17:14:28.401 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/tcp_lp.ko 2024-07-04T17:14:28.401 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/tcp_scalable.ko 2024-07-04T17:14:28.402 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/tcp_vegas.ko 2024-07-04T17:14:28.402 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/tcp_veno.ko 2024-07-04T17:14:28.402 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/tcp_westwood.ko 2024-07-04T17:14:28.402 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/tcp_yeah.ko 2024-07-04T17:14:28.402 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/tunnel4.ko 2024-07-04T17:14:28.402 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/udp_diag.ko 2024-07-04T17:14:28.402 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/udp_tunnel.ko 2024-07-04T17:14:28.402 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/xfrm4_tunnel.ko 2024-07-04T17:14:28.402 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6 2024-07-04T17:14:28.402 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/ah6.ko 2024-07-04T17:14:28.402 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/esp6.ko 2024-07-04T17:14:28.402 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/fou6.ko 2024-07-04T17:14:28.403 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/ip6_tunnel.ko 2024-07-04T17:14:28.403 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-07-04T17:14:28.403 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/ip6_vti.ko 2024-07-04T17:14:28.403 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/ipcomp6.ko 2024-07-04T17:14:28.403 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter 2024-07-04T17:14:28.403 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-07-04T17:14:28.403 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-07-04T17:14:28.403 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-07-04T17:14:28.403 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-07-04T17:14:28.403 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-07-04T17:14:28.403 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-07-04T17:14:28.403 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-07-04T17:14:28.403 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-07-04T17:14:28.404 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-07-04T17:14:28.404 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-07-04T17:14:28.404 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-07-04T17:14:28.404 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-07-04T17:14:28.404 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-07-04T17:14:28.404 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-07-04T17:14:28.404 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-07-04T17:14:28.404 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-07-04T17:14:28.404 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-07-04T17:14:28.404 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-07-04T17:14:28.404 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-07-04T17:14:28.404 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-07-04T17:14:28.404 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-07-04T17:14:28.404 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-07-04T17:14:28.404 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-07-04T17:14:28.404 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-07-04T17:14:28.405 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/sit.ko 2024-07-04T17:14:28.405 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/tunnel6.ko 2024-07-04T17:14:28.405 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/xfrm6_tunnel.ko 2024-07-04T17:14:28.405 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/key 2024-07-04T17:14:28.405 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/key/af_key.ko 2024-07-04T17:14:28.405 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/l2tp 2024-07-04T17:14:28.405 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/l2tp/l2tp_core.ko 2024-07-04T17:14:28.405 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/l2tp/l2tp_debugfs.ko 2024-07-04T17:14:28.405 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/l2tp/l2tp_eth.ko 2024-07-04T17:14:28.405 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/l2tp/l2tp_ip.ko 2024-07-04T17:14:28.405 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/l2tp/l2tp_ip6.ko 2024-07-04T17:14:28.405 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/l2tp/l2tp_netlink.ko 2024-07-04T17:14:28.405 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/l2tp/l2tp_ppp.ko 2024-07-04T17:14:28.405 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/llc 2024-07-04T17:14:28.405 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/llc/llc.ko 2024-07-04T17:14:28.406 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/mac802154 2024-07-04T17:14:28.406 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/mac802154/mac802154.ko 2024-07-04T17:14:28.406 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/mpls 2024-07-04T17:14:28.406 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/mpls/mpls_gso.ko 2024-07-04T17:14:28.406 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter 2024-07-04T17:14:28.406 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset 2024-07-04T17:14:28.406 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset/ip_set.ko 2024-07-04T17:14:28.406 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-07-04T17:14:28.406 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-07-04T17:14:28.406 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-07-04T17:14:28.406 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-07-04T17:14:28.406 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-07-04T17:14:28.406 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-07-04T17:14:28.406 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-07-04T17:14:28.407 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-07-04T17:14:28.407 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-07-04T17:14:28.407 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-07-04T17:14:28.407 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-07-04T17:14:28.407 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-07-04T17:14:28.407 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-07-04T17:14:28.407 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-07-04T17:14:28.407 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-07-04T17:14:28.407 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipvs 2024-07-04T17:14:28.407 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipvs/ip_vs.ko 2024-07-04T17:14:28.407 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-07-04T17:14:28.407 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-07-04T17:14:28.408 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-07-04T17:14:28.408 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-07-04T17:14:28.408 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-07-04T17:14:28.408 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-07-04T17:14:28.408 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-07-04T17:14:28.408 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-07-04T17:14:28.408 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-07-04T17:14:28.408 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-07-04T17:14:28.408 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-07-04T17:14:28.408 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-07-04T17:14:28.408 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-07-04T17:14:28.408 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_conncount.ko 2024-07-04T17:14:28.408 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_conntrack.ko 2024-07-04T17:14:28.408 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-07-04T17:14:28.409 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-07-04T17:14:28.409 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-07-04T17:14:28.409 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_conntrack_h323.ko 2024-07-04T17:14:28.409 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_conntrack_irc.ko 2024-07-04T17:14:28.409 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-07-04T17:14:28.409 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-07-04T17:14:28.409 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-07-04T17:14:28.409 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_conntrack_sane.ko 2024-07-04T17:14:28.409 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_conntrack_sip.ko 2024-07-04T17:14:28.409 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-07-04T17:14:28.409 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-07-04T17:14:28.409 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_dup_netdev.ko 2024-07-04T17:14:28.409 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_flow_table.ko 2024-07-04T17:14:28.410 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_flow_table_inet.ko 2024-07-04T17:14:28.410 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_log_syslog.ko 2024-07-04T17:14:28.410 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_nat.ko 2024-07-04T17:14:28.410 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_nat_amanda.ko 2024-07-04T17:14:28.410 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_nat_ftp.ko 2024-07-04T17:14:28.410 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_nat_irc.ko 2024-07-04T17:14:28.410 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_nat_sip.ko 2024-07-04T17:14:28.410 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_nat_tftp.ko 2024-07-04T17:14:28.410 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_synproxy_core.ko 2024-07-04T17:14:28.410 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_tables.ko 2024-07-04T17:14:28.410 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nfnetlink.ko 2024-07-04T17:14:28.410 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nfnetlink_acct.ko 2024-07-04T17:14:28.410 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nfnetlink_log.ko 2024-07-04T17:14:28.411 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nfnetlink_osf.ko 2024-07-04T17:14:28.411 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nfnetlink_queue.ko 2024-07-04T17:14:28.411 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_chain_nat.ko 2024-07-04T17:14:28.411 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_compat.ko 2024-07-04T17:14:28.411 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_ct.ko 2024-07-04T17:14:28.411 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_dup_netdev.ko 2024-07-04T17:14:28.411 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_fib.ko 2024-07-04T17:14:28.411 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_fwd_netdev.ko 2024-07-04T17:14:28.411 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_hash.ko 2024-07-04T17:14:28.411 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_limit.ko 2024-07-04T17:14:28.411 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_log.ko 2024-07-04T17:14:28.411 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_masq.ko 2024-07-04T17:14:28.411 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_nat.ko 2024-07-04T17:14:28.411 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_numgen.ko 2024-07-04T17:14:28.412 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_queue.ko 2024-07-04T17:14:28.412 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_quota.ko 2024-07-04T17:14:28.412 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_redir.ko 2024-07-04T17:14:28.412 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_reject.ko 2024-07-04T17:14:28.412 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_reject_inet.ko 2024-07-04T17:14:28.412 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_socket.ko 2024-07-04T17:14:28.412 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_tunnel.ko 2024-07-04T17:14:28.412 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_AUDIT.ko 2024-07-04T17:14:28.412 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_CHECKSUM.ko 2024-07-04T17:14:28.412 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_CLASSIFY.ko 2024-07-04T17:14:28.412 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-07-04T17:14:28.412 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_CT.ko 2024-07-04T17:14:28.413 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_DSCP.ko 2024-07-04T17:14:28.413 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_HL.ko 2024-07-04T17:14:28.413 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_HMARK.ko 2024-07-04T17:14:28.413 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_IDLETIMER.ko 2024-07-04T17:14:28.413 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_LED.ko 2024-07-04T17:14:28.413 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_LOG.ko 2024-07-04T17:14:28.413 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_MASQUERADE.ko 2024-07-04T17:14:28.413 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_NETMAP.ko 2024-07-04T17:14:28.413 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_NFLOG.ko 2024-07-04T17:14:28.413 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_NFQUEUE.ko 2024-07-04T17:14:28.413 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_RATEEST.ko 2024-07-04T17:14:28.414 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_REDIRECT.ko 2024-07-04T17:14:28.414 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_SECMARK.ko 2024-07-04T17:14:28.414 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_TCPMSS.ko 2024-07-04T17:14:28.414 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-07-04T17:14:28.414 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_TEE.ko 2024-07-04T17:14:28.414 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_TPROXY.ko 2024-07-04T17:14:28.414 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_TRACE.ko 2024-07-04T17:14:28.414 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_addrtype.ko 2024-07-04T17:14:28.414 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_bpf.ko 2024-07-04T17:14:28.414 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_cgroup.ko 2024-07-04T17:14:28.414 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_cluster.ko 2024-07-04T17:14:28.414 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_comment.ko 2024-07-04T17:14:28.414 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_connbytes.ko 2024-07-04T17:14:28.415 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_connlabel.ko 2024-07-04T17:14:28.415 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_connlimit.ko 2024-07-04T17:14:28.415 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_connmark.ko 2024-07-04T17:14:28.415 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_conntrack.ko 2024-07-04T17:14:28.415 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_cpu.ko 2024-07-04T17:14:28.415 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_dccp.ko 2024-07-04T17:14:28.415 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_devgroup.ko 2024-07-04T17:14:28.415 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_dscp.ko 2024-07-04T17:14:28.415 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_ecn.ko 2024-07-04T17:14:28.415 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_esp.ko 2024-07-04T17:14:28.415 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_hashlimit.ko 2024-07-04T17:14:28.415 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_helper.ko 2024-07-04T17:14:28.415 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_hl.ko 2024-07-04T17:14:28.415 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_ipcomp.ko 2024-07-04T17:14:28.415 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_iprange.ko 2024-07-04T17:14:28.415 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_ipvs.ko 2024-07-04T17:14:28.416 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_l2tp.ko 2024-07-04T17:14:28.416 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_length.ko 2024-07-04T17:14:28.416 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_limit.ko 2024-07-04T17:14:28.416 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_mac.ko 2024-07-04T17:14:28.416 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_mark.ko 2024-07-04T17:14:28.416 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_multiport.ko 2024-07-04T17:14:28.416 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_nat.ko 2024-07-04T17:14:28.416 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_nfacct.ko 2024-07-04T17:14:28.416 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_osf.ko 2024-07-04T17:14:28.416 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_owner.ko 2024-07-04T17:14:28.416 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_physdev.ko 2024-07-04T17:14:28.416 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_pkttype.ko 2024-07-04T17:14:28.417 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_policy.ko 2024-07-04T17:14:28.417 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_quota.ko 2024-07-04T17:14:28.417 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_rateest.ko 2024-07-04T17:14:28.417 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_realm.ko 2024-07-04T17:14:28.417 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_recent.ko 2024-07-04T17:14:28.417 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_sctp.ko 2024-07-04T17:14:28.417 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_set.ko 2024-07-04T17:14:28.417 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_socket.ko 2024-07-04T17:14:28.417 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_state.ko 2024-07-04T17:14:28.417 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_statistic.ko 2024-07-04T17:14:28.417 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_string.ko 2024-07-04T17:14:28.418 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_tcpmss.ko 2024-07-04T17:14:28.418 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_time.ko 2024-07-04T17:14:28.418 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_u32.ko 2024-07-04T17:14:28.418 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netlink 2024-07-04T17:14:28.418 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netlink/netlink_diag.ko 2024-07-04T17:14:28.418 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/nsh 2024-07-04T17:14:28.418 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/nsh/nsh.ko 2024-07-04T17:14:28.418 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/openvswitch 2024-07-04T17:14:28.418 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/openvswitch/openvswitch.ko 2024-07-04T17:14:28.418 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/openvswitch/vport-geneve.ko 2024-07-04T17:14:28.418 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/openvswitch/vport-gre.ko 2024-07-04T17:14:28.418 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/openvswitch/vport-vxlan.ko 2024-07-04T17:14:28.418 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/packet 2024-07-04T17:14:28.419 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/packet/af_packet_diag.ko 2024-07-04T17:14:28.419 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/rds 2024-07-04T17:14:28.419 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/rds/rds.ko 2024-07-04T17:14:28.419 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/rds/rds_rdma.ko 2024-07-04T17:14:28.419 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/rds/rds_tcp.ko 2024-07-04T17:14:28.419 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched 2024-07-04T17:14:28.419 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/act_csum.ko 2024-07-04T17:14:28.419 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/act_gact.ko 2024-07-04T17:14:28.419 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/act_mirred.ko 2024-07-04T17:14:28.419 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/act_nat.ko 2024-07-04T17:14:28.419 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/act_pedit.ko 2024-07-04T17:14:28.419 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/act_police.ko 2024-07-04T17:14:28.419 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/act_simple.ko 2024-07-04T17:14:28.420 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/act_skbedit.ko 2024-07-04T17:14:28.420 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/act_vlan.ko 2024-07-04T17:14:28.420 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/cls_basic.ko 2024-07-04T17:14:28.420 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/cls_bpf.ko 2024-07-04T17:14:28.420 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/cls_flow.ko 2024-07-04T17:14:28.420 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/cls_fw.ko 2024-07-04T17:14:28.420 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/cls_route.ko 2024-07-04T17:14:28.420 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/cls_u32.ko 2024-07-04T17:14:28.420 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/em_cmp.ko 2024-07-04T17:14:28.420 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/em_ipset.ko 2024-07-04T17:14:28.420 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/em_meta.ko 2024-07-04T17:14:28.420 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/em_nbyte.ko 2024-07-04T17:14:28.421 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/em_text.ko 2024-07-04T17:14:28.421 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/em_u32.ko 2024-07-04T17:14:28.421 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_choke.ko 2024-07-04T17:14:28.421 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_codel.ko 2024-07-04T17:14:28.421 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_drr.ko 2024-07-04T17:14:28.421 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_fq.ko 2024-07-04T17:14:28.421 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_gred.ko 2024-07-04T17:14:28.421 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_hfsc.ko 2024-07-04T17:14:28.421 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_hhf.ko 2024-07-04T17:14:28.421 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_htb.ko 2024-07-04T17:14:28.421 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_ingress.ko 2024-07-04T17:14:28.422 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_mqprio.ko 2024-07-04T17:14:28.422 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_mqprio_lib.ko 2024-07-04T17:14:28.422 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_multiq.ko 2024-07-04T17:14:28.422 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_netem.ko 2024-07-04T17:14:28.422 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_pie.ko 2024-07-04T17:14:28.422 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_plug.ko 2024-07-04T17:14:28.422 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_prio.ko 2024-07-04T17:14:28.422 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_qfq.ko 2024-07-04T17:14:28.422 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_red.ko 2024-07-04T17:14:28.422 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_sfb.ko 2024-07-04T17:14:28.422 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_sfq.ko 2024-07-04T17:14:28.422 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_tbf.ko 2024-07-04T17:14:28.422 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_teql.ko 2024-07-04T17:14:28.423 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sctp 2024-07-04T17:14:28.423 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sctp/sctp.ko 2024-07-04T17:14:28.423 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sctp/sctp_diag.ko 2024-07-04T17:14:28.423 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sunrpc 2024-07-04T17:14:28.423 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sunrpc/auth_gss 2024-07-04T17:14:28.423 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-07-04T17:14:28.423 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-07-04T17:14:28.423 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sunrpc/sunrpc.ko 2024-07-04T17:14:28.423 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sunrpc/xprtrdma 2024-07-04T17:14:28.423 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-07-04T17:14:28.423 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/tipc 2024-07-04T17:14:28.423 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/tipc/tipc.ko 2024-07-04T17:14:28.424 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/tipc/tipc_diag.ko 2024-07-04T17:14:28.424 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/unix 2024-07-04T17:14:28.424 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/unix/unix_diag.ko 2024-07-04T17:14:28.424 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/vmw_vsock 2024-07-04T17:14:28.424 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-07-04T17:14:28.424 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-07-04T17:14:28.424 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/vmw_vsock/vsock.ko 2024-07-04T17:14:28.424 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/vmw_vsock/vsock_diag.ko 2024-07-04T17:14:28.424 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/vmw_vsock/vsock_loopback.ko 2024-07-04T17:14:28.424 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/xfrm 2024-07-04T17:14:28.424 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/xfrm/xfrm_ipcomp.ko 2024-07-04T17:14:28.425 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/security 2024-07-04T17:14:28.425 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/security/keys 2024-07-04T17:14:28.425 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/security/keys/encrypted-keys 2024-07-04T17:14:28.425 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-07-04T17:14:28.425 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/security/keys/trusted-keys 2024-07-04T17:14:28.425 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/security/keys/trusted-keys/trusted.ko 2024-07-04T17:14:28.425 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/modules.alias 2024-07-04T17:14:28.425 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/modules.alias.bin 2024-07-04T17:14:28.425 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/modules.builtin 2024-07-04T17:14:28.425 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/modules.builtin.alias.bin 2024-07-04T17:14:28.425 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/modules.builtin.bin 2024-07-04T17:14:28.425 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/modules.builtin.modinfo 2024-07-04T17:14:28.426 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/modules.dep 2024-07-04T17:14:28.426 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/modules.dep.bin 2024-07-04T17:14:28.426 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/modules.devname 2024-07-04T17:14:28.426 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/modules.order 2024-07-04T17:14:28.426 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/modules.softdep 2024-07-04T17:14:28.426 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/modules.symbols 2024-07-04T17:14:28.426 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/modules.symbols.bin 2024-07-04T17:14:28.426 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-g0d32a7bba447/vmlinuz 2024-07-04T17:14:28.430 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-g0d32a7bba447 2024-07-04T17:14:28.431 DEBUG:teuthology.orchestra.run.smithi042:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-07-04T17:14:28.454 INFO:teuthology.orchestra.run.smithi042.stdout:postinstall scriptlet (using /bin/sh): 2024-07-04T17:14:28.455 INFO:teuthology.orchestra.run.smithi042.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-07-04T17:14:28.455 INFO:teuthology.orchestra.run.smithi042.stdout: /usr/bin/kernel-install add 6.9.0-g0d32a7bba447 /lib/modules/6.9.0-g0d32a7bba447/vmlinuz 2024-07-04T17:14:28.455 INFO:teuthology.orchestra.run.smithi042.stdout:fi 2024-07-04T17:14:28.455 INFO:teuthology.orchestra.run.smithi042.stdout:for file in vmlinuz System.map config; do 2024-07-04T17:14:28.455 INFO:teuthology.orchestra.run.smithi042.stdout: if ! cmp --silent "/lib/modules/6.9.0-g0d32a7bba447/${file}" "/boot/${file}-6.9.0-g0d32a7bba447"; then 2024-07-04T17:14:28.455 INFO:teuthology.orchestra.run.smithi042.stdout: cp "/lib/modules/6.9.0-g0d32a7bba447/${file}" "/boot/${file}-6.9.0-g0d32a7bba447" 2024-07-04T17:14:28.455 INFO:teuthology.orchestra.run.smithi042.stdout: fi 2024-07-04T17:14:28.455 INFO:teuthology.orchestra.run.smithi042.stdout:done 2024-07-04T17:14:28.455 INFO:teuthology.orchestra.run.smithi042.stdout:if [ -d "/lib/modules/6.9.0-g0d32a7bba447/dtb" ] && \ 2024-07-04T17:14:28.455 INFO:teuthology.orchestra.run.smithi042.stdout: ! diff -rq "/lib/modules/6.9.0-g0d32a7bba447/dtb" "/boot/dtb-6.9.0-g0d32a7bba447" >/dev/null 2>&1; then 2024-07-04T17:14:28.455 INFO:teuthology.orchestra.run.smithi042.stdout: rm -rf "/boot/dtb-6.9.0-g0d32a7bba447" 2024-07-04T17:14:28.456 INFO:teuthology.orchestra.run.smithi042.stdout: cp -r "/lib/modules/6.9.0-g0d32a7bba447/dtb" "/boot/dtb-6.9.0-g0d32a7bba447" 2024-07-04T17:14:28.456 INFO:teuthology.orchestra.run.smithi042.stdout:fi 2024-07-04T17:14:28.456 INFO:teuthology.orchestra.run.smithi042.stdout:if [ ! -e "/lib/modules/6.9.0-g0d32a7bba447/modules.dep" ]; then 2024-07-04T17:14:28.456 INFO:teuthology.orchestra.run.smithi042.stdout: /usr/sbin/depmod 6.9.0-g0d32a7bba447 2024-07-04T17:14:28.456 INFO:teuthology.orchestra.run.smithi042.stdout:fi 2024-07-04T17:14:28.456 INFO:teuthology.orchestra.run.smithi042.stdout:preuninstall scriptlet (using /bin/sh): 2024-07-04T17:14:28.456 INFO:teuthology.orchestra.run.smithi042.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-07-04T17:14:28.456 INFO:teuthology.orchestra.run.smithi042.stdout:kernel-install remove 6.9.0-g0d32a7bba447 2024-07-04T17:14:28.456 INFO:teuthology.orchestra.run.smithi042.stdout:fi 2024-07-04T17:14:28.457 DEBUG:teuthology.orchestra.run.smithi042:> sudo rpm -qi grub2-tools 2024-07-04T17:14:28.541 INFO:teuthology.orchestra.run.smithi042.stdout:Name : grub2-tools 2024-07-04T17:14:28.541 INFO:teuthology.orchestra.run.smithi042.stdout:Epoch : 1 2024-07-04T17:14:28.541 INFO:teuthology.orchestra.run.smithi042.stdout:Version : 2.06 2024-07-04T17:14:28.541 INFO:teuthology.orchestra.run.smithi042.stdout:Release : 68.el9 2024-07-04T17:14:28.542 INFO:teuthology.orchestra.run.smithi042.stdout:Architecture: x86_64 2024-07-04T17:14:28.542 INFO:teuthology.orchestra.run.smithi042.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-07-04T17:14:28.542 INFO:teuthology.orchestra.run.smithi042.stdout:Group : Unspecified 2024-07-04T17:14:28.542 INFO:teuthology.orchestra.run.smithi042.stdout:Size : 8264254 2024-07-04T17:14:28.542 INFO:teuthology.orchestra.run.smithi042.stdout:License : GPLv3+ 2024-07-04T17:14:28.542 INFO:teuthology.orchestra.run.smithi042.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-07-04T17:14:28.542 INFO:teuthology.orchestra.run.smithi042.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-07-04T17:14:28.542 INFO:teuthology.orchestra.run.smithi042.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-07-04T17:14:28.542 INFO:teuthology.orchestra.run.smithi042.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-07-04T17:14:28.542 INFO:teuthology.orchestra.run.smithi042.stdout:Packager : builder@centos.org 2024-07-04T17:14:28.542 INFO:teuthology.orchestra.run.smithi042.stdout:Vendor : CentOS 2024-07-04T17:14:28.542 INFO:teuthology.orchestra.run.smithi042.stdout:URL : http://www.gnu.org/software/grub/ 2024-07-04T17:14:28.543 INFO:teuthology.orchestra.run.smithi042.stdout:Summary : Support tools for GRUB. 2024-07-04T17:14:28.543 INFO:teuthology.orchestra.run.smithi042.stdout:Description : 2024-07-04T17:14:28.543 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:14:28.543 INFO:teuthology.orchestra.run.smithi042.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-07-04T17:14:28.543 INFO:teuthology.orchestra.run.smithi042.stdout:customizable bootloader with modular architecture. It supports a rich 2024-07-04T17:14:28.543 INFO:teuthology.orchestra.run.smithi042.stdout:variety of kernel formats, file systems, computer architectures and 2024-07-04T17:14:28.543 INFO:teuthology.orchestra.run.smithi042.stdout:hardware devices. 2024-07-04T17:14:28.543 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:14:28.543 INFO:teuthology.orchestra.run.smithi042.stdout:This subpackage provides tools for support of all platforms. 2024-07-04T17:14:28.544 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-07-04T17:14:28.544 INFO:teuthology.task.kernel:Updating grub on smithi042 to boot 6.9.0-g0d32a7bba447 2024-07-04T17:14:28.544 DEBUG:teuthology.orchestra.run.smithi042:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-07-04T17:14:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:Generating grub configuration file ... 2024-07-04T17:14:30.596 INFO:teuthology.orchestra.run.smithi042.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-07-04T17:14:30.630 INFO:teuthology.orchestra.run.smithi042.stderr:done 2024-07-04T17:14:30.633 DEBUG:teuthology.orchestra.run.smithi042:> mktemp 2024-07-04T17:14:30.651 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/tmp.02Een0jvuK 2024-07-04T17:14:30.651 DEBUG:teuthology.orchestra.run.smithi042:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.02Een0jvuK 2024-07-04T17:14:30.726 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0666 /tmp/tmp.02Een0jvuK 2024-07-04T17:14:30.878 DEBUG:teuthology.orchestra.remote:smithi042:/tmp/tmp.02Een0jvuK is 6KB 2024-07-04T17:14:30.899 DEBUG:teuthology.orchestra.run.smithi042:> rm -fr /tmp/tmp.02Een0jvuK 2024-07-04T17:14:30.914 DEBUG:teuthology.orchestra.run.smithi042:> sudo /bin/ls /boot/loader/entries || true 2024-07-04T17:14:30.976 INFO:teuthology.orchestra.run.smithi042.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-07-04T17:14:30.976 INFO:teuthology.orchestra.run.smithi042.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-07-04T17:14:30.976 INFO:teuthology.orchestra.run.smithi042.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-07-04T17:14:30.976 INFO:teuthology.orchestra.run.smithi042.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-g0d32a7bba447.conf 2024-07-04T17:14:30.977 DEBUG:teuthology.orchestra.run.smithi042:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-g0d32a7bba447 2024-07-04T17:14:31.128 DEBUG:teuthology.orchestra.run.smithi042:> sudo shutdown -r now 2024-07-04T17:15:01.162 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-07-04T17:15:01.163 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-07-04T17:15:01.164 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-04T17:15:19.711 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.42 2024-07-04T17:15:28.720 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-07-04T17:15:28.721 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-04T17:16:28.782 DEBUG:teuthology.orchestra.remote:timed out 2024-07-04T17:16:40.792 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-07-04T17:16:40.793 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-04T17:16:41.192 DEBUG:teuthology.orchestra.run.smithi042:> true 2024-07-04T17:16:41.858 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-07-04T17:16:41.858 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-g0d32a7bba447"... 2024-07-04T17:16:41.858 DEBUG:teuthology.orchestra.run.smithi042:> uname -r 2024-07-04T17:16:41.913 INFO:teuthology.orchestra.run.smithi042.stdout:6.9.0-g0d32a7bba447 2024-07-04T17:16:41.914 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-g0d32a7bba447 vs 6.9.0-g0d32a7bba447 2024-07-04T17:16:41.914 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-07-04T17:16:41.914 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-07-04T17:16:42.917 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-07-04T17:16:42.918 DEBUG:teuthology.orchestra.run.smithi042:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-07-04T17:16:43.050 INFO:teuthology.orchestra.run.smithi042.stdout:ttyS1 2024-07-04T17:16:43.109 DEBUG:teuthology.parallel:result is None 2024-07-04T17:16:43.109 INFO:teuthology.run_tasks:Running task internal.base... 2024-07-04T17:16:43.121 INFO:teuthology.task.internal:Creating test directory... 2024-07-04T17:16:43.121 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-07-04T17:16:43.181 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-07-04T17:16:43.187 INFO:teuthology.run_tasks:Running task internal.archive... 2024-07-04T17:16:43.216 INFO:teuthology.task.internal:Creating archive directory... 2024-07-04T17:16:43.217 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-07-04T17:16:43.288 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-07-04T17:16:43.295 INFO:teuthology.task.internal:Enabling coredump saving... 2024-07-04T17:16:43.295 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-07-04T17:16:43.389 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-04T17:16:43.419 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-04T17:16:43.423 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-07-04T17:16:43.429 INFO:teuthology.task.internal:Configuring sudo... 2024-07-04T17:16:43.430 DEBUG:teuthology.orchestra.run.smithi042:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-07-04T17:16:43.517 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-07-04T17:16:43.526 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-07-04T17:16:43.527 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-07-04T17:16:43.581 DEBUG:teuthology.orchestra.run.smithi042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-04T17:16:43.665 DEBUG:teuthology.orchestra.run.smithi042:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-04T17:16:43.760 DEBUG:teuthology.orchestra.run.smithi042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-04T17:16:43.843 DEBUG:teuthology.orchestra.run.smithi042:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-04T17:16:43.935 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:16:43.935 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-07-04T17:16:44.023 DEBUG:teuthology.orchestra.run.smithi042:> sudo service rsyslog restart 2024-07-04T17:16:44.182 INFO:teuthology.orchestra.run.smithi042.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-04T17:16:44.377 INFO:teuthology.run_tasks:Running task internal.timer... 2024-07-04T17:16:44.384 INFO:teuthology.task.internal:Starting timer... 2024-07-04T17:16:44.384 INFO:teuthology.run_tasks:Running task pcp... 2024-07-04T17:16:44.422 INFO:teuthology.run_tasks:Running task selinux... 2024-07-04T17:16:44.452 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-07-04T17:16:44.453 DEBUG:teuthology.orchestra.run.smithi042:> sudo service auditd rotate 2024-07-04T17:16:44.572 INFO:teuthology.orchestra.run.smithi042.stdout:Rotating logs: 2024-07-04T17:16:44.576 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-07-04T17:16:44.576 DEBUG:teuthology.orchestra.run.smithi042:> /usr/sbin/getenforce 2024-07-04T17:16:44.631 INFO:teuthology.orchestra.run.smithi042.stdout:Permissive 2024-07-04T17:16:44.631 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi042.front.sepia.ceph.com': 'permissive'} 2024-07-04T17:16:44.631 DEBUG:teuthology.orchestra.run.smithi042:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-04T17:16:44.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-04T17:16:44.707 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-07-04T17:16:44.707 DEBUG:teuthology.orchestra.run.smithi042:> sudo /usr/sbin/setenforce permissive 2024-07-04T17:16:44.787 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-07-04T17:16:44.796 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-07-04T17:16:44.808 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-07-04T17:16:44.809 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi042.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-07-04T17:23:08.731 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi042.front.sepia.ceph.com')] 2024-07-04T17:23:08.732 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-07-04T17:23:08.733 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-04T17:23:08.854 DEBUG:teuthology.orchestra.run.smithi042:> true 2024-07-04T17:23:09.012 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-07-04T17:23:09.012 INFO:teuthology.run_tasks:Running task clock... 2024-07-04T17:23:09.023 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-07-04T17:23:09.023 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-07-04T17:23:09.024 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-04T17:23:09.119 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-07-04T17:23:09.168 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-07-04T17:23:09.244 INFO:teuthology.orchestra.run.smithi042.stderr:sudo: ntpd: command not found 2024-07-04T17:23:09.278 INFO:teuthology.orchestra.run.smithi042.stdout:506 Cannot talk to daemon 2024-07-04T17:23:09.318 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-07-04T17:23:09.361 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-07-04T17:23:09.501 INFO:teuthology.orchestra.run.smithi042.stderr:bash: line 1: ntpq: command not found 2024-07-04T17:23:09.511 INFO:teuthology.orchestra.run.smithi042.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-04T17:23:09.511 INFO:teuthology.orchestra.run.smithi042.stdout:=============================================================================== 2024-07-04T17:23:09.511 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-04T17:23:09.511 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-04T17:23:09.511 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-04T17:23:09.511 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-04T17:23:09.512 INFO:teuthology.run_tasks:Running task install... 2024-07-04T17:23:09.523 DEBUG:teuthology.task.install:project ceph 2024-07-04T17:23:09.523 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '9dba1e836b4756bf3a32be2a4936f371dff55008'}} 2024-07-04T17:23:09.523 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '9dba1e836b4756bf3a32be2a4936f371dff55008'} 2024-07-04T17:23:09.523 INFO:teuthology.task.install:Using flavor: default 2024-07-04T17:23:09.528 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-07-04T17:23:09.528 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-07-04T17:23:09.529 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9dba1e836b4756bf3a32be2a4936f371dff55008', 'tag': None, 'wait_for_package': False} 2024-07-04T17:23:09.529 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9dba1e836b4756bf3a32be2a4936f371dff55008 2024-07-04T17:23:09.748 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240701.141150-debug/9dba1e836b4756bf3a32be2a4936f371dff55008/centos/9/flavors/default/ 2024-07-04T17:23:09.749 INFO:teuthology.task.install.rpm:Package version is 19.0.0-4667.g9dba1e83 2024-07-04T17:23:09.937 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240701.141150-debug/9dba1e836b4756bf3a32be2a4936f371dff55008/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240701.141150-debug/9dba1e836b4756bf3a32be2a4936f371dff55008/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240701.141150-debug/9dba1e836b4756bf3a32be2a4936f371dff55008/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-07-04T17:23:09.938 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:23:09.938 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-07-04T17:23:10.032 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-07-04T17:23:10.033 DEBUG:teuthology.orchestra.run.smithi042:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9dba1e836b4756bf3a32be2a4936f371dff55008/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-07-04T17:23:10.164 DEBUG:teuthology.orchestra.run.smithi042:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-07-04T17:23:10.281 DEBUG:teuthology.orchestra.run.smithi042:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-07-04T17:23:10.362 INFO:teuthology.orchestra.run.smithi042.stdout:check_obsoletes = 1 2024-07-04T17:23:10.366 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum clean all 2024-07-04T17:23:10.698 INFO:teuthology.orchestra.run.smithi042.stdout:45 files removed 2024-07-04T17:23:10.738 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-07-04T17:23:11.593 INFO:teuthology.orchestra.run.smithi042.stdout:ceph packages for x86_64 208 kB/s | 83 kB 00:00 2024-07-04T17:23:12.119 INFO:teuthology.orchestra.run.smithi042.stdout:ceph noarch packages 56 kB/s | 24 kB 00:00 2024-07-04T17:23:12.578 INFO:teuthology.orchestra.run.smithi042.stdout:ceph source packages 5.2 kB/s | 1.9 kB 00:00 2024-07-04T17:23:15.301 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - BaseOS 3.1 MB/s | 8.2 MB 00:02 2024-07-04T17:23:21.444 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - AppStream 4.1 MB/s | 20 MB 00:04 2024-07-04T17:23:27.400 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.4 MB 00:01 2024-07-04T17:23:29.807 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - Extras packages 17 kB/s | 17 kB 00:01 2024-07-04T17:23:30.929 INFO:teuthology.orchestra.run.smithi042.stdout:Extra Packages for Enterprise Linux 24 MB/s | 22 MB 00:00 2024-07-04T17:23:37.356 INFO:teuthology.orchestra.run.smithi042.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-07-04T17:23:40.123 INFO:teuthology.orchestra.run.smithi042.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-04T17:23:40.124 INFO:teuthology.orchestra.run.smithi042.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-04T17:23:40.143 INFO:teuthology.orchestra.run.smithi042.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-07-04T17:23:40.144 INFO:teuthology.orchestra.run.smithi042.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-07-04T17:23:40.145 INFO:teuthology.orchestra.run.smithi042.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-07-04T17:23:40.146 INFO:teuthology.orchestra.run.smithi042.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-07-04T17:23:40.148 INFO:teuthology.orchestra.run.smithi042.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-07-04T17:23:40.151 INFO:teuthology.orchestra.run.smithi042.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-07-04T17:23:40.152 INFO:teuthology.orchestra.run.smithi042.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-07-04T17:23:40.228 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-07-04T17:23:40.245 INFO:teuthology.orchestra.run.smithi042.stdout:======================================================================================= 2024-07-04T17:23:40.245 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-07-04T17:23:40.245 INFO:teuthology.orchestra.run.smithi042.stdout:======================================================================================= 2024-07-04T17:23:40.245 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-07-04T17:23:40.245 INFO:teuthology.orchestra.run.smithi042.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-07-04T17:23:40.245 INFO:teuthology.orchestra.run.smithi042.stdout: ceph x86_64 2:19.0.0-4667.g9dba1e83.el9 ceph 6.6 k 2024-07-04T17:23:40.245 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-base x86_64 2:19.0.0-4667.g9dba1e83.el9 ceph 5.4 M 2024-07-04T17:23:40.245 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-fuse x86_64 2:19.0.0-4667.g9dba1e83.el9 ceph 871 k 2024-07-04T17:23:40.245 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-4667.g9dba1e83.el9 ceph 148 k 2024-07-04T17:23:40.246 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr x86_64 2:19.0.0-4667.g9dba1e83.el9 ceph 1.5 M 2024-07-04T17:23:40.246 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-cephadm noarch 2:19.0.0-4667.g9dba1e83.el9 ceph-noarch 143 k 2024-07-04T17:23:40.246 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-dashboard noarch 2:19.0.0-4667.g9dba1e83.el9 ceph-noarch 14 M 2024-07-04T17:23:40.246 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-4667.g9dba1e83.el9 ceph-noarch 7.4 M 2024-07-04T17:23:40.246 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-rook noarch 2:19.0.0-4667.g9dba1e83.el9 ceph-noarch 49 k 2024-07-04T17:23:40.246 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-radosgw x86_64 2:19.0.0-4667.g9dba1e83.el9 ceph 10 M 2024-07-04T17:23:40.246 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-test x86_64 2:19.0.0-4667.g9dba1e83.el9 ceph 44 M 2024-07-04T17:23:40.246 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-volume noarch 2:19.0.0-4667.g9dba1e83.el9 ceph-noarch 274 k 2024-07-04T17:23:40.246 INFO:teuthology.orchestra.run.smithi042.stdout: cephadm noarch 2:19.0.0-4667.g9dba1e83.el9 ceph-noarch 764 k 2024-07-04T17:23:40.246 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-mirror x86_64 2:19.0.0-4667.g9dba1e83.el9 ceph 227 k 2024-07-04T17:23:40.247 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-shell x86_64 2:19.0.0-4667.g9dba1e83.el9 ceph 27 k 2024-07-04T17:23:40.247 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-top noarch 2:19.0.0-4667.g9dba1e83.el9 ceph-noarch 18 k 2024-07-04T17:23:40.247 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-07-04T17:23:40.247 INFO:teuthology.orchestra.run.smithi042.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-07-04T17:23:40.247 INFO:teuthology.orchestra.run.smithi042.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-07-04T17:23:40.247 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs-devel x86_64 2:19.0.0-4667.g9dba1e83.el9 ceph 32 k 2024-07-04T17:23:40.247 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs2 x86_64 2:19.0.0-4667.g9dba1e83.el9 ceph 766 k 2024-07-04T17:23:40.247 INFO:teuthology.orchestra.run.smithi042.stdout: librados-devel x86_64 2:19.0.0-4667.g9dba1e83.el9 ceph 127 k 2024-07-04T17:23:40.247 INFO:teuthology.orchestra.run.smithi042.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-07-04T17:23:40.248 INFO:teuthology.orchestra.run.smithi042.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-07-04T17:23:40.248 INFO:teuthology.orchestra.run.smithi042.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-07-04T17:23:40.248 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-07-04T17:23:40.248 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-07-04T17:23:40.248 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-07-04T17:23:40.248 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cephfs x86_64 2:19.0.0-4667.g9dba1e83.el9 ceph 162 k 2024-07-04T17:23:40.248 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rados x86_64 2:19.0.0-4667.g9dba1e83.el9 ceph 320 k 2024-07-04T17:23:40.248 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rbd x86_64 2:19.0.0-4667.g9dba1e83.el9 ceph 304 k 2024-07-04T17:23:40.248 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rgw x86_64 2:19.0.0-4667.g9dba1e83.el9 ceph 100 k 2024-07-04T17:23:40.248 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-fuse x86_64 2:19.0.0-4667.g9dba1e83.el9 ceph 86 k 2024-07-04T17:23:40.248 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-mirror x86_64 2:19.0.0-4667.g9dba1e83.el9 ceph 3.1 M 2024-07-04T17:23:40.249 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-nbd x86_64 2:19.0.0-4667.g9dba1e83.el9 ceph 170 k 2024-07-04T17:23:40.249 INFO:teuthology.orchestra.run.smithi042.stdout: xfsprogs-devel x86_64 6.4.0-2.el9 appstream 95 k 2024-07-04T17:23:40.249 INFO:teuthology.orchestra.run.smithi042.stdout:Upgrading: 2024-07-04T17:23:40.249 INFO:teuthology.orchestra.run.smithi042.stdout: NetworkManager x86_64 1:1.48.2-2.el9 baseos 2.3 M 2024-07-04T17:23:40.249 INFO:teuthology.orchestra.run.smithi042.stdout: NetworkManager-libnm x86_64 1:1.48.2-2.el9 baseos 1.8 M 2024-07-04T17:23:40.249 INFO:teuthology.orchestra.run.smithi042.stdout: NetworkManager-team x86_64 1:1.48.2-2.el9 baseos 41 k 2024-07-04T17:23:40.249 INFO:teuthology.orchestra.run.smithi042.stdout: NetworkManager-tui x86_64 1:1.48.2-2.el9 baseos 247 k 2024-07-04T17:23:40.249 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-07-04T17:23:40.249 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-07-04T17:23:40.249 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-07-04T17:23:40.249 INFO:teuthology.orchestra.run.smithi042.stdout: librados2 x86_64 2:19.0.0-4667.g9dba1e83.el9 ceph 3.3 M 2024-07-04T17:23:40.249 INFO:teuthology.orchestra.run.smithi042.stdout: librbd1 x86_64 2:19.0.0-4667.g9dba1e83.el9 ceph 3.1 M 2024-07-04T17:23:40.250 INFO:teuthology.orchestra.run.smithi042.stdout: openssl x86_64 1:3.2.2-2.el9 baseos 1.4 M 2024-07-04T17:23:40.250 INFO:teuthology.orchestra.run.smithi042.stdout: openssl-libs x86_64 1:3.2.2-2.el9 baseos 2.4 M 2024-07-04T17:23:40.250 INFO:teuthology.orchestra.run.smithi042.stdout: xfsprogs x86_64 6.4.0-2.el9 baseos 1.1 M 2024-07-04T17:23:40.250 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-07-04T17:23:40.250 INFO:teuthology.orchestra.run.smithi042.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-07-04T17:23:40.250 INFO:teuthology.orchestra.run.smithi042.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-07-04T17:23:40.250 INFO:teuthology.orchestra.run.smithi042.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-07-04T17:23:40.250 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-common x86_64 2:19.0.0-4667.g9dba1e83.el9 ceph 20 M 2024-07-04T17:23:40.250 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-grafana-dashboards noarch 2:19.0.0-4667.g9dba1e83.el9 ceph-noarch 34 k 2024-07-04T17:23:40.250 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mds x86_64 2:19.0.0-4667.g9dba1e83.el9 ceph 2.2 M 2024-07-04T17:23:40.251 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-modules-core noarch 2:19.0.0-4667.g9dba1e83.el9 ceph-noarch 277 k 2024-07-04T17:23:40.251 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mon x86_64 2:19.0.0-4667.g9dba1e83.el9 ceph 4.9 M 2024-07-04T17:23:40.251 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-osd x86_64 2:19.0.0-4667.g9dba1e83.el9 ceph 18 M 2024-07-04T17:23:40.251 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-prometheus-alerts noarch 2:19.0.0-4667.g9dba1e83.el9 ceph-noarch 16 k 2024-07-04T17:23:40.251 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-selinux x86_64 2:19.0.0-4667.g9dba1e83.el9 ceph 25 k 2024-07-04T17:23:40.251 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-07-04T17:23:40.251 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-07-04T17:23:40.251 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-07-04T17:23:40.251 INFO:teuthology.orchestra.run.smithi042.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-07-04T17:23:40.251 INFO:teuthology.orchestra.run.smithi042.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-07-04T17:23:40.251 INFO:teuthology.orchestra.run.smithi042.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-07-04T17:23:40.252 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-07-04T17:23:40.252 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-07-04T17:23:40.252 INFO:teuthology.orchestra.run.smithi042.stdout: libcephsqlite x86_64 2:19.0.0-4667.g9dba1e83.el9 ceph 158 k 2024-07-04T17:23:40.252 INFO:teuthology.orchestra.run.smithi042.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-07-04T17:23:40.252 INFO:teuthology.orchestra.run.smithi042.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-07-04T17:23:40.252 INFO:teuthology.orchestra.run.smithi042.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-07-04T17:23:40.252 INFO:teuthology.orchestra.run.smithi042.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-07-04T17:23:40.252 INFO:teuthology.orchestra.run.smithi042.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-07-04T17:23:40.252 INFO:teuthology.orchestra.run.smithi042.stdout: libradosstriper1 x86_64 2:19.0.0-4667.g9dba1e83.el9 ceph 505 k 2024-07-04T17:23:40.252 INFO:teuthology.orchestra.run.smithi042.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-07-04T17:23:40.252 INFO:teuthology.orchestra.run.smithi042.stdout: librgw2 x86_64 2:19.0.0-4667.g9dba1e83.el9 ceph 5.2 M 2024-07-04T17:23:40.253 INFO:teuthology.orchestra.run.smithi042.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-07-04T17:23:40.253 INFO:teuthology.orchestra.run.smithi042.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-07-04T17:23:40.253 INFO:teuthology.orchestra.run.smithi042.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-07-04T17:23:40.253 INFO:teuthology.orchestra.run.smithi042.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-07-04T17:23:40.253 INFO:teuthology.orchestra.run.smithi042.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-07-04T17:23:40.253 INFO:teuthology.orchestra.run.smithi042.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-07-04T17:23:40.253 INFO:teuthology.orchestra.run.smithi042.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-07-04T17:23:40.253 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-07-04T17:23:40.253 INFO:teuthology.orchestra.run.smithi042.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-07-04T17:23:40.253 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-07-04T17:23:40.253 INFO:teuthology.orchestra.run.smithi042.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-07-04T17:23:40.254 INFO:teuthology.orchestra.run.smithi042.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-07-04T17:23:40.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-07-04T17:23:40.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-07-04T17:23:40.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-07-04T17:23:40.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-07-04T17:23:40.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-07-04T17:23:40.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-argparse x86_64 2:19.0.0-4667.g9dba1e83.el9 ceph 45 k 2024-07-04T17:23:40.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-common x86_64 2:19.0.0-4667.g9dba1e83.el9 ceph 136 k 2024-07-04T17:23:40.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-07-04T17:23:40.254 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-07-04T17:23:40.255 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-07-04T17:23:40.255 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-07-04T17:23:40.255 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-07-04T17:23:40.255 INFO:teuthology.orchestra.run.smithi042.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-07-04T17:23:40.255 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-07-04T17:23:40.255 INFO:teuthology.orchestra.run.smithi042.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-07-04T17:23:40.255 INFO:teuthology.orchestra.run.smithi042.stdout: python3-google-auth noarch 1:2.30.0-1.el9 epel 213 k 2024-07-04T17:23:40.255 INFO:teuthology.orchestra.run.smithi042.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-07-04T17:23:40.255 INFO:teuthology.orchestra.run.smithi042.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-07-04T17:23:40.255 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-07-04T17:23:40.255 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-07-04T17:23:40.255 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-07-04T17:23:40.256 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-07-04T17:23:40.256 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-07-04T17:23:40.256 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-07-04T17:23:40.256 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-07-04T17:23:40.256 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-07-04T17:23:40.256 INFO:teuthology.orchestra.run.smithi042.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-07-04T17:23:40.256 INFO:teuthology.orchestra.run.smithi042.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-07-04T17:23:40.256 INFO:teuthology.orchestra.run.smithi042.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-07-04T17:23:40.256 INFO:teuthology.orchestra.run.smithi042.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-07-04T17:23:40.256 INFO:teuthology.orchestra.run.smithi042.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-07-04T17:23:40.256 INFO:teuthology.orchestra.run.smithi042.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-07-04T17:23:40.257 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-07-04T17:23:40.257 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-07-04T17:23:40.257 INFO:teuthology.orchestra.run.smithi042.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-07-04T17:23:40.257 INFO:teuthology.orchestra.run.smithi042.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-07-04T17:23:40.257 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-07-04T17:23:40.257 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-07-04T17:23:40.257 INFO:teuthology.orchestra.run.smithi042.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-07-04T17:23:40.257 INFO:teuthology.orchestra.run.smithi042.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-07-04T17:23:40.257 INFO:teuthology.orchestra.run.smithi042.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-07-04T17:23:40.257 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-07-04T17:23:40.257 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-07-04T17:23:40.257 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-07-04T17:23:40.258 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-07-04T17:23:40.258 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-07-04T17:23:40.258 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-07-04T17:23:40.258 INFO:teuthology.orchestra.run.smithi042.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-07-04T17:23:40.258 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-07-04T17:23:40.258 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-07-04T17:23:40.258 INFO:teuthology.orchestra.run.smithi042.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-07-04T17:23:40.258 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-07-04T17:23:40.258 INFO:teuthology.orchestra.run.smithi042.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-07-04T17:23:40.258 INFO:teuthology.orchestra.run.smithi042.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-07-04T17:23:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-07-04T17:23:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-07-04T17:23:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-07-04T17:23:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-07-04T17:23:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-07-04T17:23:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-07-04T17:23:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-07-04T17:23:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-07-04T17:23:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-07-04T17:23:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-07-04T17:23:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-07-04T17:23:40.259 INFO:teuthology.orchestra.run.smithi042.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-07-04T17:23:40.260 INFO:teuthology.orchestra.run.smithi042.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-07-04T17:23:40.260 INFO:teuthology.orchestra.run.smithi042.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-07-04T17:23:40.260 INFO:teuthology.orchestra.run.smithi042.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-07-04T17:23:40.260 INFO:teuthology.orchestra.run.smithi042.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-07-04T17:23:40.260 INFO:teuthology.orchestra.run.smithi042.stdout:Installing weak dependencies: 2024-07-04T17:23:40.260 INFO:teuthology.orchestra.run.smithi042.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-07-04T17:23:40.260 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:23:40.260 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-07-04T17:23:40.260 INFO:teuthology.orchestra.run.smithi042.stdout:======================================================================================= 2024-07-04T17:23:40.260 INFO:teuthology.orchestra.run.smithi042.stdout:Install 148 Packages 2024-07-04T17:23:40.260 INFO:teuthology.orchestra.run.smithi042.stdout:Upgrade 12 Packages 2024-07-04T17:23:40.261 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:23:40.261 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 231 M 2024-07-04T17:23:40.261 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-07-04T17:23:41.807 INFO:teuthology.orchestra.run.smithi042.stdout:(1/160): ceph-19.0.0-4667.g9dba1e83.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-07-04T17:23:42.282 INFO:teuthology.orchestra.run.smithi042.stdout:(2/160): ceph-fuse-19.0.0-4667.g9dba1e83.el9.x8 1.8 MB/s | 871 kB 00:00 2024-07-04T17:23:42.483 INFO:teuthology.orchestra.run.smithi042.stdout:(3/160): ceph-immutable-object-cache-19.0.0-466 743 kB/s | 148 kB 00:00 2024-07-04T17:23:42.633 INFO:teuthology.orchestra.run.smithi042.stdout:(4/160): ceph-base-19.0.0-4667.g9dba1e83.el9.x8 4.5 MB/s | 5.4 MB 00:01 2024-07-04T17:23:42.875 INFO:teuthology.orchestra.run.smithi042.stdout:(5/160): ceph-mgr-19.0.0-4667.g9dba1e83.el9.x86 6.3 MB/s | 1.5 MB 00:00 2024-07-04T17:23:43.292 INFO:teuthology.orchestra.run.smithi042.stdout:(6/160): ceph-mds-19.0.0-4667.g9dba1e83.el9.x86 2.8 MB/s | 2.2 MB 00:00 2024-07-04T17:23:44.209 INFO:teuthology.orchestra.run.smithi042.stdout:(7/160): ceph-mon-19.0.0-4667.g9dba1e83.el9.x86 3.7 MB/s | 4.9 MB 00:01 2024-07-04T17:23:45.901 INFO:teuthology.orchestra.run.smithi042.stdout:(8/160): ceph-radosgw-19.0.0-4667.g9dba1e83.el9 6.0 MB/s | 10 MB 00:01 2024-07-04T17:23:45.967 INFO:teuthology.orchestra.run.smithi042.stdout:(9/160): ceph-selinux-19.0.0-4667.g9dba1e83.el9 387 kB/s | 25 kB 00:00 2024-07-04T17:23:46.918 INFO:teuthology.orchestra.run.smithi042.stdout:(10/160): ceph-osd-19.0.0-4667.g9dba1e83.el9.x8 4.9 MB/s | 18 MB 00:03 2024-07-04T17:23:47.001 INFO:teuthology.orchestra.run.smithi042.stdout:(11/160): cephfs-mirror-19.0.0-4667.g9dba1e83.e 2.7 MB/s | 227 kB 00:00 2024-07-04T17:23:47.068 INFO:teuthology.orchestra.run.smithi042.stdout:(12/160): cephfs-shell-19.0.0-4667.g9dba1e83.el 411 kB/s | 27 kB 00:00 2024-07-04T17:23:47.144 INFO:teuthology.orchestra.run.smithi042.stdout:(13/160): libcephfs-devel-19.0.0-4667.g9dba1e83 429 kB/s | 32 kB 00:00 2024-07-04T17:23:47.311 INFO:teuthology.orchestra.run.smithi042.stdout:(14/160): libcephfs2-19.0.0-4667.g9dba1e83.el9. 4.5 MB/s | 766 kB 00:00 2024-07-04T17:23:47.578 INFO:teuthology.orchestra.run.smithi042.stdout:(15/160): ceph-common-19.0.0-4667.g9dba1e83.el9 3.2 MB/s | 20 MB 00:06 2024-07-04T17:23:47.611 INFO:teuthology.orchestra.run.smithi042.stdout:(16/160): libcephsqlite-19.0.0-4667.g9dba1e83.e 528 kB/s | 158 kB 00:00 2024-07-04T17:23:47.670 INFO:teuthology.orchestra.run.smithi042.stdout:(17/160): librados-devel-19.0.0-4667.g9dba1e83. 1.4 MB/s | 127 kB 00:00 2024-07-04T17:23:47.720 INFO:teuthology.orchestra.run.smithi042.stdout:(18/160): libradosstriper1-19.0.0-4667.g9dba1e8 4.6 MB/s | 505 kB 00:00 2024-07-04T17:23:47.805 INFO:teuthology.orchestra.run.smithi042.stdout:(19/160): python3-ceph-argparse-19.0.0-4667.g9d 543 kB/s | 45 kB 00:00 2024-07-04T17:23:47.888 INFO:teuthology.orchestra.run.smithi042.stdout:(20/160): python3-ceph-common-19.0.0-4667.g9dba 1.6 MB/s | 136 kB 00:00 2024-07-04T17:23:47.980 INFO:teuthology.orchestra.run.smithi042.stdout:(21/160): python3-cephfs-19.0.0-4667.g9dba1e83. 1.7 MB/s | 162 kB 00:00 2024-07-04T17:23:48.063 INFO:teuthology.orchestra.run.smithi042.stdout:(22/160): python3-rados-19.0.0-4667.g9dba1e83.e 3.8 MB/s | 320 kB 00:00 2024-07-04T17:23:48.130 INFO:teuthology.orchestra.run.smithi042.stdout:(23/160): python3-rbd-19.0.0-4667.g9dba1e83.el9 4.5 MB/s | 304 kB 00:00 2024-07-04T17:23:48.189 INFO:teuthology.orchestra.run.smithi042.stdout:(24/160): python3-rgw-19.0.0-4667.g9dba1e83.el9 1.7 MB/s | 100 kB 00:00 2024-07-04T17:23:48.256 INFO:teuthology.orchestra.run.smithi042.stdout:(25/160): rbd-fuse-19.0.0-4667.g9dba1e83.el9.x8 1.3 MB/s | 86 kB 00:00 2024-07-04T17:23:48.423 INFO:teuthology.orchestra.run.smithi042.stdout:(26/160): librgw2-19.0.0-4667.g9dba1e83.el9.x86 6.9 MB/s | 5.2 MB 00:00 2024-07-04T17:23:48.565 INFO:teuthology.orchestra.run.smithi042.stdout:(27/160): rbd-mirror-19.0.0-4667.g9dba1e83.el9. 10 MB/s | 3.1 MB 00:00 2024-07-04T17:23:48.599 INFO:teuthology.orchestra.run.smithi042.stdout:(28/160): rbd-nbd-19.0.0-4667.g9dba1e83.el9.x86 976 kB/s | 170 kB 00:00 2024-07-04T17:23:48.782 INFO:teuthology.orchestra.run.smithi042.stdout:(29/160): ceph-grafana-dashboards-19.0.0-4667.g 158 kB/s | 34 kB 00:00 2024-07-04T17:23:48.816 INFO:teuthology.orchestra.run.smithi042.stdout:(30/160): ceph-mgr-cephadm-19.0.0-4667.g9dba1e8 662 kB/s | 143 kB 00:00 2024-07-04T17:23:50.025 INFO:teuthology.orchestra.run.smithi042.stdout:(31/160): ceph-mgr-dashboard-19.0.0-4667.g9dba1 11 MB/s | 14 MB 00:01 2024-07-04T17:23:50.133 INFO:teuthology.orchestra.run.smithi042.stdout:(32/160): ceph-mgr-modules-core-19.0.0-4667.g9d 2.5 MB/s | 277 kB 00:00 2024-07-04T17:23:50.317 INFO:teuthology.orchestra.run.smithi042.stdout:(33/160): ceph-mgr-diskprediction-local-19.0.0- 4.9 MB/s | 7.4 MB 00:01 2024-07-04T17:23:50.342 INFO:teuthology.orchestra.run.smithi042.stdout:(34/160): ceph-mgr-rook-19.0.0-4667.g9dba1e83.e 237 kB/s | 49 kB 00:00 2024-07-04T17:23:50.467 INFO:teuthology.orchestra.run.smithi042.stdout:(35/160): ceph-prometheus-alerts-19.0.0-4667.g9 110 kB/s | 16 kB 00:00 2024-07-04T17:23:50.501 INFO:teuthology.orchestra.run.smithi042.stdout:(36/160): ceph-volume-19.0.0-4667.g9dba1e83.el9 1.7 MB/s | 274 kB 00:00 2024-07-04T17:23:50.601 INFO:teuthology.orchestra.run.smithi042.stdout:(37/160): cephfs-top-19.0.0-4667.g9dba1e83.el9. 184 kB/s | 18 kB 00:00 2024-07-04T17:23:50.668 INFO:teuthology.orchestra.run.smithi042.stdout:(38/160): cephadm-19.0.0-4667.g9dba1e83.el9.noa 3.7 MB/s | 764 kB 00:00 2024-07-04T17:23:50.710 INFO:teuthology.orchestra.run.smithi042.stdout:(39/160): fuse-2.9.9-15.el9.x86_64.rpm 746 kB/s | 80 kB 00:00 2024-07-04T17:23:50.785 INFO:teuthology.orchestra.run.smithi042.stdout:(40/160): libquadmath-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 195 kB 00:00 2024-07-04T17:23:50.853 INFO:teuthology.orchestra.run.smithi042.stdout:(41/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.8 MB/s | 253 kB 00:00 2024-07-04T17:23:50.894 INFO:teuthology.orchestra.run.smithi042.stdout:(42/160): libgfortran-11.4.1-3.el9.x86_64.rpm 3.5 MB/s | 803 kB 00:00 2024-07-04T17:23:50.953 INFO:teuthology.orchestra.run.smithi042.stdout:(43/160): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-07-04T17:23:51.004 INFO:teuthology.orchestra.run.smithi042.stdout:(44/160): python3-pycparser-2.20-6.el9.noarch.r 2.7 MB/s | 135 kB 00:00 2024-07-04T17:23:51.046 INFO:teuthology.orchestra.run.smithi042.stdout:(45/160): python3-requests-2.25.1-8.el9.noarch. 3.0 MB/s | 125 kB 00:00 2024-07-04T17:23:51.096 INFO:teuthology.orchestra.run.smithi042.stdout:(46/160): python3-urllib3-1.26.5-6.el9.noarch.r 4.3 MB/s | 215 kB 00:00 2024-07-04T17:23:51.146 INFO:teuthology.orchestra.run.smithi042.stdout:(47/160): python3-cryptography-36.0.1-4.el9.x86 4.2 MB/s | 1.2 MB 00:00 2024-07-04T17:23:51.422 INFO:teuthology.orchestra.run.smithi042.stdout:(48/160): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 689 kB 00:00 2024-07-04T17:23:51.480 INFO:teuthology.orchestra.run.smithi042.stdout:(49/160): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-07-04T17:23:51.531 INFO:teuthology.orchestra.run.smithi042.stdout:(50/160): bison-3.7.4-5.el9.x86_64.rpm 2.4 MB/s | 943 kB 00:00 2024-07-04T17:23:51.556 INFO:teuthology.orchestra.run.smithi042.stdout:(51/160): elfutils-libelf-devel-0.191-4.el9.x86 292 kB/s | 22 kB 00:00 2024-07-04T17:23:51.598 INFO:teuthology.orchestra.run.smithi042.stdout:(52/160): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-07-04T17:23:51.623 INFO:teuthology.orchestra.run.smithi042.stdout:(53/160): flexiblas-3.0.4-8.el9.x86_64.rpm 493 kB/s | 33 kB 00:00 2024-07-04T17:23:51.682 INFO:teuthology.orchestra.run.smithi042.stdout:(54/160): flexiblas-openblas-openmp-3.0.4-8.el9 316 kB/s | 18 kB 00:00 2024-07-04T17:23:51.741 INFO:teuthology.orchestra.run.smithi042.stdout:(55/160): libaio-devel-0.3.111-13.el9.x86_64.rp 194 kB/s | 11 kB 00:00 2024-07-04T17:23:51.799 INFO:teuthology.orchestra.run.smithi042.stdout:(56/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-07-04T17:23:51.858 INFO:teuthology.orchestra.run.smithi042.stdout:(57/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 786 kB/s | 45 kB 00:00 2024-07-04T17:23:52.017 INFO:teuthology.orchestra.run.smithi042.stdout:(58/160): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 662 kB 00:00 2024-07-04T17:23:52.100 INFO:teuthology.orchestra.run.smithi042.stdout:(59/160): libtool-2.4.6-46.el9.x86_64.rpm 6.9 MB/s | 581 kB 00:00 2024-07-04T17:23:52.159 INFO:teuthology.orchestra.run.smithi042.stdout:(60/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 443 kB/s | 25 kB 00:00 2024-07-04T17:23:52.226 INFO:teuthology.orchestra.run.smithi042.stdout:(61/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 721 kB/s | 47 kB 00:00 2024-07-04T17:23:52.410 INFO:teuthology.orchestra.run.smithi042.stdout:(62/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.6 MB/s | 292 kB 00:00 2024-07-04T17:23:52.527 INFO:teuthology.orchestra.run.smithi042.stdout:(63/160): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-07-04T17:23:52.585 INFO:teuthology.orchestra.run.smithi042.stdout:(64/160): openblas-0.3.21-2.el9.x86_64.rpm 612 kB/s | 35 kB 00:00 2024-07-04T17:23:53.253 INFO:teuthology.orchestra.run.smithi042.stdout:(65/160): ceph-test-19.0.0-4667.g9dba1e83.el9.x 6.0 MB/s | 44 MB 00:07 2024-07-04T17:23:53.336 INFO:teuthology.orchestra.run.smithi042.stdout:(66/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 1.7 MB/s | 3.0 MB 00:01 2024-07-04T17:23:53.953 INFO:teuthology.orchestra.run.smithi042.stdout:(67/160): postgresql-13.14-1.el9.x86_64.rpm 2.6 MB/s | 1.6 MB 00:00 2024-07-04T17:23:54.478 INFO:teuthology.orchestra.run.smithi042.stdout:(68/160): postgresql-contrib-13.14-1.el9.x86_64 1.7 MB/s | 911 kB 00:00 2024-07-04T17:23:54.595 INFO:teuthology.orchestra.run.smithi042.stdout:(69/160): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-07-04T17:23:54.871 INFO:teuthology.orchestra.run.smithi042.stdout:(70/160): openblas-openmp-0.3.21-2.el9.x86_64.r 2.1 MB/s | 4.7 MB 00:02 2024-07-04T17:23:55.280 INFO:teuthology.orchestra.run.smithi042.stdout:(71/160): openssl-devel-3.2.2-2.el9.x86_64.rpm 2.2 MB/s | 4.4 MB 00:02 2024-07-04T17:23:55.372 INFO:teuthology.orchestra.run.smithi042.stdout:(72/160): python3-attrs-20.3.0-7.el9.noarch.rpm 964 kB/s | 87 kB 00:00 2024-07-04T17:23:55.514 INFO:teuthology.orchestra.run.smithi042.stdout:(73/160): protobuf-3.14.0-13.el9.x86_64.rpm 1.6 MB/s | 1.0 MB 00:00 2024-07-04T17:23:55.564 INFO:teuthology.orchestra.run.smithi042.stdout:(74/160): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-07-04T17:23:55.722 INFO:teuthology.orchestra.run.smithi042.stdout:(75/160): python3-devel-3.9.18-3.el9.x86_64.rpm 1.5 MB/s | 244 kB 00:00 2024-07-04T17:23:55.948 INFO:teuthology.orchestra.run.smithi042.stdout:(76/160): python3-jinja2-2.11.3-6.el9.noarch.rp 1.1 MB/s | 248 kB 00:00 2024-07-04T17:23:56.015 INFO:teuthology.orchestra.run.smithi042.stdout:(77/160): python3-jsonpatch-1.21-16.el9.noarch. 389 kB/s | 26 kB 00:00 2024-07-04T17:23:56.090 INFO:teuthology.orchestra.run.smithi042.stdout:(78/160): python3-jsonpointer-2.0-4.el9.noarch. 258 kB/s | 19 kB 00:00 2024-07-04T17:23:56.215 INFO:teuthology.orchestra.run.smithi042.stdout:(79/160): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-07-04T17:23:56.274 INFO:teuthology.orchestra.run.smithi042.stdout:(80/160): python3-markupsafe-1.1.1-12.el9.x86_6 603 kB/s | 35 kB 00:00 2024-07-04T17:23:56.950 INFO:teuthology.orchestra.run.smithi042.stdout:(81/160): postgresql-server-13.14-1.el9.x86_64. 2.5 MB/s | 5.8 MB 00:02 2024-07-04T17:23:57.108 INFO:teuthology.orchestra.run.smithi042.stdout:(82/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 250 kB 00:00 2024-07-04T17:23:57.267 INFO:teuthology.orchestra.run.smithi042.stdout:(83/160): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 222 kB 00:00 2024-07-04T17:23:57.359 INFO:teuthology.orchestra.run.smithi042.stdout:(84/160): python3-packaging-20.9-5.el9.noarch.r 850 kB/s | 77 kB 00:00 2024-07-04T17:23:57.426 INFO:teuthology.orchestra.run.smithi042.stdout:(85/160): python3-prettytable-0.7.2-27.el9.noar 640 kB/s | 42 kB 00:00 2024-07-04T17:23:57.626 INFO:teuthology.orchestra.run.smithi042.stdout:(86/160): python3-protobuf-3.14.0-13.el9.noarch 1.3 MB/s | 269 kB 00:00 2024-07-04T17:23:57.768 INFO:teuthology.orchestra.run.smithi042.stdout:(87/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-07-04T17:23:57.952 INFO:teuthology.orchestra.run.smithi042.stdout:(88/160): python3-pyasn1-modules-0.4.8-6.el9.no 1.5 MB/s | 279 kB 00:00 2024-07-04T17:23:58.010 INFO:teuthology.orchestra.run.smithi042.stdout:(89/160): python3-pytz-2021.1-5.el9.noarch.rpm 886 kB/s | 51 kB 00:00 2024-07-04T17:23:58.069 INFO:teuthology.orchestra.run.smithi042.stdout:(90/160): python3-requests-oauthlib-1.3.0-12.el 934 kB/s | 54 kB 00:00 2024-07-04T17:23:58.195 INFO:teuthology.orchestra.run.smithi042.stdout:(91/160): python3-babel-2.9.1-2.el9.noarch.rpm 2.1 MB/s | 6.0 MB 00:02 2024-07-04T17:23:58.253 INFO:teuthology.orchestra.run.smithi042.stdout:(92/160): python3-toml-0.10.2-6.el9.noarch.rpm 726 kB/s | 42 kB 00:00 2024-07-04T17:23:58.337 INFO:teuthology.orchestra.run.smithi042.stdout:(93/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 533 kB/s | 44 kB 00:00 2024-07-04T17:23:58.496 INFO:teuthology.orchestra.run.smithi042.stdout:(94/160): qatlib-24.02.0-1.el9.x86_64.rpm 1.4 MB/s | 221 kB 00:00 2024-07-04T17:23:58.554 INFO:teuthology.orchestra.run.smithi042.stdout:(95/160): qatlib-service-24.02.0-1.el9.x86_64.r 622 kB/s | 36 kB 00:00 2024-07-04T17:23:58.613 INFO:teuthology.orchestra.run.smithi042.stdout:(96/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 819 kB/s | 47 kB 00:00 2024-07-04T17:23:58.805 INFO:teuthology.orchestra.run.smithi042.stdout:(97/160): socat-1.7.4.1-6.el9.x86_64.rpm 1.6 MB/s | 304 kB 00:00 2024-07-04T17:23:58.887 INFO:teuthology.orchestra.run.smithi042.stdout:(98/160): uuid-1.6.2-55.el9.x86_64.rpm 715 kB/s | 58 kB 00:00 2024-07-04T17:23:58.981 INFO:teuthology.orchestra.run.smithi042.stdout:(99/160): xfsprogs-devel-6.4.0-2.el9.x86_64.rpm 1.0 MB/s | 95 kB 00:00 2024-07-04T17:23:59.081 INFO:teuthology.orchestra.run.smithi042.stdout:(100/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 642 kB/s | 64 kB 00:00 2024-07-04T17:23:59.140 INFO:teuthology.orchestra.run.smithi042.stdout:(101/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 779 kB/s | 45 kB 00:00 2024-07-04T17:23:59.315 INFO:teuthology.orchestra.run.smithi042.stdout:(102/160): python3-numpy-1.20.1-5.el9.x86_64.rp 1.7 MB/s | 5.1 MB 00:03 2024-07-04T17:23:59.340 INFO:teuthology.orchestra.run.smithi042.stdout:(103/160): lua-devel-5.4.4-4.el9.x86_64.rpm 112 kB/s | 22 kB 00:00 2024-07-04T17:23:59.491 INFO:teuthology.orchestra.run.smithi042.stdout:(104/160): abseil-cpp-20211102.0-3.el9.x86_64.r 3.6 MB/s | 548 kB 00:00 2024-07-04T17:23:59.533 INFO:teuthology.orchestra.run.smithi042.stdout:(105/160): gperftools-libs-2.9.1-2.el9.x86_64.r 7.4 MB/s | 309 kB 00:00 2024-07-04T17:23:59.558 INFO:teuthology.orchestra.run.smithi042.stdout:(106/160): grpc-data-1.46.7-10.el9.noarch.rpm 807 kB/s | 19 kB 00:00 2024-07-04T17:23:59.608 INFO:teuthology.orchestra.run.smithi042.stdout:(107/160): protobuf-compiler-3.14.0-13.el9.x86_ 2.9 MB/s | 863 kB 00:00 2024-07-04T17:23:59.717 INFO:teuthology.orchestra.run.smithi042.stdout:(108/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 242 kB/s | 26 kB 00:00 2024-07-04T17:23:59.734 INFO:teuthology.orchestra.run.smithi042.stdout:(109/160): liboath-2.6.7-2.el9.x86_64.rpm 3.0 MB/s | 49 kB 00:00 2024-07-04T17:23:59.767 INFO:teuthology.orchestra.run.smithi042.stdout:(110/160): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-07-04T17:23:59.801 INFO:teuthology.orchestra.run.smithi042.stdout:(111/160): luarocks-3.9.2-5.el9.noarch.rpm 4.6 MB/s | 151 kB 00:00 2024-07-04T17:23:59.885 INFO:teuthology.orchestra.run.smithi042.stdout:(112/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.9 MB/s | 839 kB 00:00 2024-07-04T17:23:59.943 INFO:teuthology.orchestra.run.smithi042.stdout:(113/160): python3-asyncssh-2.13.2-5.el9.noarch 9.3 MB/s | 548 kB 00:00 2024-07-04T17:23:59.969 INFO:teuthology.orchestra.run.smithi042.stdout:(114/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.8 MB/s | 43 kB 00:00 2024-07-04T17:23:59.994 INFO:teuthology.orchestra.run.smithi042.stdout:(115/160): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-07-04T17:24:00.020 INFO:teuthology.orchestra.run.smithi042.stdout:(116/160): python3-certifi-2021.10.8-2.el9.noar 614 kB/s | 15 kB 00:00 2024-07-04T17:24:00.062 INFO:teuthology.orchestra.run.smithi042.stdout:(117/160): python3-cheroot-8.6.0-4.el9.noarch.r 4.1 MB/s | 172 kB 00:00 2024-07-04T17:24:00.120 INFO:teuthology.orchestra.run.smithi042.stdout:(118/160): python3-cherrypy-18.6.1-2.el9.noarch 6.1 MB/s | 358 kB 00:00 2024-07-04T17:24:00.171 INFO:teuthology.orchestra.run.smithi042.stdout:(119/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 6.0 MB/s | 303 kB 00:00 2024-07-04T17:24:00.213 INFO:teuthology.orchestra.run.smithi042.stdout:(120/160): python3-google-auth-2.30.0-1.el9.noa 5.1 MB/s | 213 kB 00:00 2024-07-04T17:24:00.355 INFO:teuthology.orchestra.run.smithi042.stdout:(121/160): libarrow-9.0.0-11.el9.x86_64.rpm 5.5 MB/s | 4.4 MB 00:00 2024-07-04T17:24:00.388 INFO:teuthology.orchestra.run.smithi042.stdout:(122/160): python3-grpcio-tools-1.46.7-10.el9.x 4.4 MB/s | 144 kB 00:00 2024-07-04T17:24:00.414 INFO:teuthology.orchestra.run.smithi042.stdout:(123/160): python3-jaraco-8.2.1-3.el9.noarch.rp 442 kB/s | 11 kB 00:00 2024-07-04T17:24:00.439 INFO:teuthology.orchestra.run.smithi042.stdout:(124/160): python3-jaraco-classes-3.2.1-5.el9.n 736 kB/s | 18 kB 00:00 2024-07-04T17:24:00.464 INFO:teuthology.orchestra.run.smithi042.stdout:(125/160): python3-jaraco-collections-3.0.0-8.e 954 kB/s | 23 kB 00:00 2024-07-04T17:24:00.489 INFO:teuthology.orchestra.run.smithi042.stdout:(126/160): python3-jaraco-functools-3.5.0-2.el9 807 kB/s | 19 kB 00:00 2024-07-04T17:24:00.515 INFO:teuthology.orchestra.run.smithi042.stdout:(127/160): python3-jaraco-text-3.2.0-6.el9.noar 812 kB/s | 20 kB 00:00 2024-07-04T17:24:00.623 INFO:teuthology.orchestra.run.smithi042.stdout:(128/160): python3-grpcio-1.46.7-10.el9.x86_64. 5.0 MB/s | 2.0 MB 00:00 2024-07-04T17:24:00.649 INFO:teuthology.orchestra.run.smithi042.stdout:(129/160): python3-logutils-0.3.5-21.el9.noarch 1.9 MB/s | 46 kB 00:00 2024-07-04T17:24:00.682 INFO:teuthology.orchestra.run.smithi042.stdout:(130/160): python3-more-itertools-8.12.0-2.el9. 2.4 MB/s | 79 kB 00:00 2024-07-04T17:24:00.724 INFO:teuthology.orchestra.run.smithi042.stdout:(131/160): python3-kubernetes-26.1.0-2.el9.noar 4.9 MB/s | 1.0 MB 00:00 2024-07-04T17:24:00.758 INFO:teuthology.orchestra.run.smithi042.stdout:(132/160): python3-natsort-7.1.1-5.el9.noarch.r 775 kB/s | 58 kB 00:00 2024-07-04T17:24:00.783 INFO:teuthology.orchestra.run.smithi042.stdout:(133/160): python3-portend-3.1.0-2.el9.noarch.r 680 kB/s | 16 kB 00:00 2024-07-04T17:24:00.817 INFO:teuthology.orchestra.run.smithi042.stdout:(134/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.7 MB/s | 90 kB 00:00 2024-07-04T17:24:00.842 INFO:teuthology.orchestra.run.smithi042.stdout:(135/160): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-07-04T17:24:00.876 INFO:teuthology.orchestra.run.smithi042.stdout:(136/160): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-07-04T17:24:00.901 INFO:teuthology.orchestra.run.smithi042.stdout:(137/160): python3-repoze-lru-0.7-16.el9.noarch 532 kB/s | 31 kB 00:00 2024-07-04T17:24:00.935 INFO:teuthology.orchestra.run.smithi042.stdout:(138/160): python3-rsa-4.9-2.el9.noarch.rpm 1.8 MB/s | 59 kB 00:00 2024-07-04T17:24:00.960 INFO:teuthology.orchestra.run.smithi042.stdout:(139/160): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-07-04T17:24:00.994 INFO:teuthology.orchestra.run.smithi042.stdout:(140/160): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-07-04T17:24:01.019 INFO:teuthology.orchestra.run.smithi042.stdout:(141/160): python3-typing-extensions-4.4.0-2.el 880 kB/s | 51 kB 00:00 2024-07-04T17:24:01.053 INFO:teuthology.orchestra.run.smithi042.stdout:(142/160): python3-websocket-client-1.2.3-2.el9 2.7 MB/s | 90 kB 00:00 2024-07-04T17:24:01.087 INFO:teuthology.orchestra.run.smithi042.stdout:(143/160): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-07-04T17:24:01.112 INFO:teuthology.orchestra.run.smithi042.stdout:(144/160): python3-zc-lockfile-2.0-10.el9.noarc 828 kB/s | 20 kB 00:00 2024-07-04T17:24:01.154 INFO:teuthology.orchestra.run.smithi042.stdout:(145/160): re2-20211101-3.el9.x86_64.rpm 4.6 MB/s | 192 kB 00:00 2024-07-04T17:24:01.188 INFO:teuthology.orchestra.run.smithi042.stdout:(146/160): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-07-04T17:24:01.282 INFO:teuthology.orchestra.run.smithi042.stdout:(147/160): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-07-04T17:24:01.472 INFO:teuthology.orchestra.run.smithi042.stdout:(148/160): librados2-19.0.0-4667.g9dba1e83.el9. 12 MB/s | 3.3 MB 00:00 2024-07-04T17:24:01.747 INFO:teuthology.orchestra.run.smithi042.stdout:(149/160): librbd1-19.0.0-4667.g9dba1e83.el9.x8 6.7 MB/s | 3.1 MB 00:00 2024-07-04T17:24:02.072 INFO:teuthology.orchestra.run.smithi042.stdout:(150/160): NetworkManager-1.48.2-2.el9.x86_64.r 3.9 MB/s | 2.3 MB 00:00 2024-07-04T17:24:02.139 INFO:teuthology.orchestra.run.smithi042.stdout:(151/160): NetworkManager-libnm-1.48.2-2.el9.x8 4.6 MB/s | 1.8 MB 00:00 2024-07-04T17:24:02.165 INFO:teuthology.orchestra.run.smithi042.stdout:(152/160): NetworkManager-team-1.48.2-2.el9.x86 448 kB/s | 41 kB 00:00 2024-07-04T17:24:02.198 INFO:teuthology.orchestra.run.smithi042.stdout:(153/160): NetworkManager-tui-1.48.2-2.el9.x86_ 4.2 MB/s | 247 kB 00:00 2024-07-04T17:24:02.224 INFO:teuthology.orchestra.run.smithi042.stdout:(154/160): elfutils-debuginfod-client-0.191-4.e 635 kB/s | 37 kB 00:00 2024-07-04T17:24:02.274 INFO:teuthology.orchestra.run.smithi042.stdout:(155/160): elfutils-libs-0.191-4.el9.x86_64.rpm 5.2 MB/s | 260 kB 00:00 2024-07-04T17:24:02.358 INFO:teuthology.orchestra.run.smithi042.stdout:(156/160): elfutils-libelf-0.191-4.el9.x86_64.r 1.3 MB/s | 207 kB 00:00 2024-07-04T17:24:02.758 INFO:teuthology.orchestra.run.smithi042.stdout:(157/160): openssl-3.2.2-2.el9.x86_64.rpm 2.8 MB/s | 1.4 MB 00:00 2024-07-04T17:24:03.200 INFO:teuthology.orchestra.run.smithi042.stdout:(158/160): openssl-libs-3.2.2-2.el9.x86_64.rpm 2.8 MB/s | 2.4 MB 00:00 2024-07-04T17:24:03.242 INFO:teuthology.orchestra.run.smithi042.stdout:(159/160): xfsprogs-6.4.0-2.el9.x86_64.rpm 2.2 MB/s | 1.1 MB 00:00 2024-07-04T17:24:07.850 INFO:teuthology.orchestra.run.smithi042.stdout:(160/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 1.7 MB/s | 16 MB 00:09 2024-07-04T17:24:07.861 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-07-04T17:24:07.861 INFO:teuthology.orchestra.run.smithi042.stdout:Total 8.4 MB/s | 231 MB 00:27 2024-07-04T17:24:11.276 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-07-04T17:24:11.585 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-07-04T17:24:11.586 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-07-04T17:24:14.294 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-07-04T17:24:14.295 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-07-04T17:24:21.123 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-07-04T17:24:21.223 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : openssl-libs-1:3.2.2-2.el9.x86_64 1/172 2024-07-04T17:24:21.311 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-07-04T17:24:21.438 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-07-04T17:24:21.498 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-07-04T17:24:21.534 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-07-04T17:24:21.644 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-07-04T17:24:22.082 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-07-04T17:24:22.198 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : librados2-2:19.0.0-4667.g9dba1e83.el9.x86_64 8/172 2024-07-04T17:24:22.392 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librados2-2:19.0.0-4667.g9dba1e83.el9.x86_64 8/172 2024-07-04T17:24:22.424 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephfs2-2:19.0.0-4667.g9dba1e83.el9.x86_64 9/172 2024-07-04T17:24:22.543 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libcephfs2-2:19.0.0-4667.g9dba1e83.el9.x86_64 9/172 2024-07-04T17:24:22.578 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rados-2:19.0.0-4667.g9dba1e83.el9.x86_64 10/172 2024-07-04T17:24:22.644 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-07-04T17:24:22.948 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-07-04T17:24:23.013 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-07-04T17:24:23.038 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephsqlite-2:19.0.0-4667.g9dba1e83.el9.x86_64 14/172 2024-07-04T17:24:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libcephsqlite-2:19.0.0-4667.g9dba1e83.el9.x86_64 14/172 2024-07-04T17:24:23.239 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libradosstriper1-2:19.0.0-4667.g9dba1e83.el9.x86 15/172 2024-07-04T17:24:23.317 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libradosstriper1-2:19.0.0-4667.g9dba1e83.el9.x86 15/172 2024-07-04T17:24:23.348 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-04T17:24:23.419 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-04T17:24:23.729 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-04T17:24:23.922 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-04T17:24:24.263 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : xfsprogs-6.4.0-2.el9.x86_64 18/172 2024-07-04T17:24:24.452 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-04T17:24:24.529 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-04T17:24:24.738 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-04T17:24:24.810 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-04T17:24:25.134 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-04T17:24:25.312 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-07-04T17:24:25.358 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-07-04T17:24:25.412 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/172 2024-07-04T17:24:25.525 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/172 2024-07-04T17:24:25.615 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/172 2024-07-04T17:24:25.666 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-packaging-20.9-5.el9.noarch 26/172 2024-07-04T17:24:26.082 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/172 2024-07-04T17:24:26.145 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 28/172 2024-07-04T17:24:26.186 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/172 2024-07-04T17:24:26.327 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/172 2024-07-04T17:24:26.399 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 31/172 2024-07-04T17:24:26.434 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/172 2024-07-04T17:24:26.490 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 33/172 2024-07-04T17:24:26.603 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 34/172 2024-07-04T17:24:26.646 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ceph-common-2:19.0.0-4667.g9dba1e83.el9. 35/172 2024-07-04T17:24:26.701 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ceph-argparse-2:19.0.0-4667.g9dba1e83.el 36/172 2024-07-04T17:24:26.736 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cephfs-2:19.0.0-4667.g9dba1e83.el9.x86_6 37/172 2024-07-04T17:24:26.860 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 38/172 2024-07-04T17:24:26.934 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : luarocks-3.9.2-5.el9.noarch 39/172 2024-07-04T17:24:27.049 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 40/172 2024-07-04T17:24:27.234 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 41/172 2024-07-04T17:24:27.302 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 42/172 2024-07-04T17:24:28.036 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rsa-4.9-2.el9.noarch 43/172 2024-07-04T17:24:28.151 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 44/172 2024-07-04T17:24:28.192 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 45/172 2024-07-04T17:24:28.238 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 46/172 2024-07-04T17:24:28.307 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 47/172 2024-07-04T17:24:28.362 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librados-devel-2:19.0.0-4667.g9dba1e83.el9.x86_6 48/172 2024-07-04T17:24:28.433 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 49/172 2024-07-04T17:24:28.671 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 50/172 2024-07-04T17:24:28.755 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : openssl-1:3.2.2-2.el9.x86_64 51/172 2024-07-04T17:24:28.898 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 52/172 2024-07-04T17:24:28.981 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 53/172 2024-07-04T17:24:29.086 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/172 2024-07-04T17:24:29.130 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/172 2024-07-04T17:24:29.169 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/172 2024-07-04T17:24:29.238 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/172 2024-07-04T17:24:29.309 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/172 2024-07-04T17:24:29.377 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/172 2024-07-04T17:24:29.443 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/172 2024-07-04T17:24:29.664 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/172 2024-07-04T17:24:29.712 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/172 2024-07-04T17:24:29.770 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 63/172 2024-07-04T17:24:29.917 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 64/172 2024-07-04T17:24:29.953 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-google-auth-1:2.30.0-1.el9.noarch 65/172 2024-07-04T17:24:30.002 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 66/172 2024-07-04T17:24:30.035 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 67/172 2024-07-04T17:24:30.068 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 68/172 2024-07-04T17:24:30.269 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 69/172 2024-07-04T17:24:30.466 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 70/172 2024-07-04T17:24:30.527 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 71/172 2024-07-04T17:24:30.568 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 72/172 2024-07-04T17:24:30.629 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 73/172 2024-07-04T17:24:30.703 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : uuid-1.6.2-55.el9.x86_64 74/172 2024-07-04T17:24:30.760 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 75/172 2024-07-04T17:24:30.807 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/172 2024-07-04T17:24:30.863 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/172 2024-07-04T17:24:30.914 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 78/172 2024-07-04T17:24:30.990 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 79/172 2024-07-04T17:24:31.030 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 80/172 2024-07-04T17:24:31.179 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 81/172 2024-07-04T17:24:31.241 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 82/172 2024-07-04T17:24:31.283 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 83/172 2024-07-04T17:24:31.443 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/172 2024-07-04T17:24:31.512 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-04T17:24:32.535 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-07-04T17:24:32.594 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-07-04T17:24:32.627 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-07-04T17:24:32.762 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-07-04T17:24:32.828 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-07-04T17:24:32.890 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-07-04T17:24:33.032 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-07-04T17:24:33.071 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-07-04T17:24:33.313 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-07-04T17:24:33.350 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-07-04T17:24:33.461 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-07-04T17:24:34.386 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-07-04T17:24:34.538 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-07-04T17:24:36.265 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-07-04T17:24:36.365 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-07-04T17:24:36.402 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-07-04T17:24:36.436 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-07-04T17:24:36.854 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-07-04T17:24:36.894 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : librbd1-2:19.0.0-4667.g9dba1e83.el9.x86_64 104/172 2024-07-04T17:24:37.017 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librbd1-2:19.0.0-4667.g9dba1e83.el9.x86_64 104/172 2024-07-04T17:24:37.060 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rbd-2:19.0.0-4667.g9dba1e83.el9.x86_64 105/172 2024-07-04T17:24:37.117 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-07-04T17:24:37.737 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-07-04T17:24:37.797 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librgw2-2:19.0.0-4667.g9dba1e83.el9.x86_64 108/172 2024-07-04T17:24:37.909 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librgw2-2:19.0.0-4667.g9dba1e83.el9.x86_64 108/172 2024-07-04T17:24:37.949 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rgw-2:19.0.0-4667.g9dba1e83.el9.x86_64 109/172 2024-07-04T17:24:40.705 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-common-2:19.0.0-4667.g9dba1e83.el9.x86_64 110/172 2024-07-04T17:24:40.746 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-common-2:19.0.0-4667.g9dba1e83.el9.x86_64 110/172 2024-07-04T17:24:41.442 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-common-2:19.0.0-4667.g9dba1e83.el9.x86_64 110/172 2024-07-04T17:24:41.491 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-base-2:19.0.0-4667.g9dba1e83.el9.x86_64 111/172 2024-07-04T17:24:41.650 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-base-2:19.0.0-4667.g9dba1e83.el9.x86_64 111/172 2024-07-04T17:24:41.651 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-07-04T17:24:41.651 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-07-04T17:24:41.651 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:24:41.684 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-selinux-2:19.0.0-4667.g9dba1e83.el9.x86_64 112/172 2024-07-04T17:24:55.127 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-selinux-2:19.0.0-4667.g9dba1e83.el9.x86_64 112/172 2024-07-04T17:24:55.127 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /sys 2024-07-04T17:24:55.127 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /proc 2024-07-04T17:24:55.128 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /mnt 2024-07-04T17:24:55.128 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /var/tmp 2024-07-04T17:24:55.128 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /home 2024-07-04T17:24:55.128 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /root 2024-07-04T17:24:55.128 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /tmp 2024-07-04T17:24:55.128 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:24:55.365 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-volume-2:19.0.0-4667.g9dba1e83.el9.noarch 113/172 2024-07-04T17:24:55.421 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-volume-2:19.0.0-4667.g9dba1e83.el9.noarch 113/172 2024-07-04T17:24:55.421 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T17:24:55.421 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-07-04T17:24:55.421 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:24:57.217 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-osd-2:19.0.0-4667.g9dba1e83.el9.x86_64 114/172 2024-07-04T17:24:57.309 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-osd-2:19.0.0-4667.g9dba1e83.el9.x86_64 114/172 2024-07-04T17:24:57.309 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T17:24:57.309 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-07-04T17:24:57.310 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-04T17:24:57.310 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-04T17:24:57.310 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:24:57.559 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mds-2:19.0.0-4667.g9dba1e83.el9.x86_64 115/172 2024-07-04T17:24:57.636 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mds-2:19.0.0-4667.g9dba1e83.el9.x86_64 115/172 2024-07-04T17:24:57.636 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T17:24:57.636 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-07-04T17:24:57.636 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-04T17:24:57.636 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-04T17:24:57.637 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:24:58.168 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mon-2:19.0.0-4667.g9dba1e83.el9.x86_64 116/172 2024-07-04T17:24:58.254 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mon-2:19.0.0-4667.g9dba1e83.el9.x86_64 116/172 2024-07-04T17:24:58.254 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T17:24:58.254 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-07-04T17:24:58.254 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-04T17:24:58.254 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-04T17:24:58.254 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:24:58.403 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-07-04T17:24:58.469 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-07-04T17:24:58.549 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-07-04T17:24:58.812 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-07-04T17:24:58.875 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-07-04T17:24:58.976 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-07-04T17:24:59.096 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-07-04T17:24:59.287 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-07-04T17:24:59.341 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-07-04T17:24:59.390 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-07-04T17:24:59.421 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : fuse-2.9.9-15.el9.x86_64 127/172 2024-07-04T17:25:00.070 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: cephadm-2:19.0.0-4667.g9dba1e83.el9.noarch 128/172 2024-07-04T17:25:00.108 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephadm-2:19.0.0-4667.g9dba1e83.el9.noarch 128/172 2024-07-04T17:25:00.165 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-4667.g9dba1e83.e 129/172 2024-07-04T17:25:00.275 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-4667.g9dba1e83. 130/172 2024-07-04T17:25:00.309 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-4667.g9dba1e83.el9.noa 131/172 2024-07-04T17:25:04.209 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-4667.g9dba1e83.el9.noa 131/172 2024-07-04T17:25:04.283 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-4667.g9dba1e83.el9.n 132/172 2024-07-04T17:25:05.459 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-4667.g9dba1e83.el9.n 132/172 2024-07-04T17:25:05.491 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-4667.g9db 133/172 2024-07-04T17:25:05.659 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-4667.g9db 133/172 2024-07-04T17:25:05.846 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-4667.g9dba1e83.el 134/172 2024-07-04T17:25:05.894 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-2:19.0.0-4667.g9dba1e83.el9.x86_64 135/172 2024-07-04T17:25:05.988 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-2:19.0.0-4667.g9dba1e83.el9.x86_64 135/172 2024-07-04T17:25:05.988 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T17:25:05.988 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-07-04T17:25:05.989 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-04T17:25:05.989 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-04T17:25:05.989 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:06.053 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-rook-2:19.0.0-4667.g9dba1e83.el9.noarch 136/172 2024-07-04T17:25:06.095 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-4667.g9dba1e83.el9.noarch 136/172 2024-07-04T17:25:06.210 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-2:19.0.0-4667.g9dba1e83.el9.x86_64 137/172 2024-07-04T17:25:06.290 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-fuse-2:19.0.0-4667.g9dba1e83.el9.x86_64 138/172 2024-07-04T17:25:07.385 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libtool-2.4.6-46.el9.x86_64 139/172 2024-07-04T17:25:07.422 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-radosgw-2:19.0.0-4667.g9dba1e83.el9.x86_64 140/172 2024-07-04T17:25:07.494 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-4667.g9dba1e83.el9.x86_64 140/172 2024-07-04T17:25:07.494 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T17:25:07.494 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-07-04T17:25:07.494 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-04T17:25:07.495 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-04T17:25:07.495 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:07.540 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-4667.g9dba1 141/172 2024-07-04T17:25:07.627 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-4667.g9dba1 141/172 2024-07-04T17:25:07.627 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T17:25:07.627 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-07-04T17:25:07.627 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:07.699 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephfs-mirror-2:19.0.0-4667.g9dba1e83.el9.x86_64 142/172 2024-07-04T17:25:07.802 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-4667.g9dba1e83.el9.x86_64 142/172 2024-07-04T17:25:07.802 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T17:25:07.802 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-07-04T17:25:07.803 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-04T17:25:07.803 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-04T17:25:07.803 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:08.216 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-mirror-2:19.0.0-4667.g9dba1e83.el9.x86_64 143/172 2024-07-04T17:25:08.301 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: rbd-mirror-2:19.0.0-4667.g9dba1e83.el9.x86_64 143/172 2024-07-04T17:25:08.301 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T17:25:08.301 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-07-04T17:25:08.301 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-04T17:25:08.301 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-04T17:25:08.301 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:13.310 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-test-2:19.0.0-4667.g9dba1e83.el9.x86_64 144/172 2024-07-04T17:25:13.366 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-fuse-2:19.0.0-4667.g9dba1e83.el9.x86_64 145/172 2024-07-04T17:25:13.429 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-nbd-2:19.0.0-4667.g9dba1e83.el9.x86_64 146/172 2024-07-04T17:25:13.468 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : xfsprogs-devel-6.4.0-2.el9.x86_64 147/172 2024-07-04T17:25:13.505 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-07-04T17:25:13.807 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephfs-shell-2:19.0.0-4667.g9dba1e83.el9.x86_64 149/172 2024-07-04T17:25:13.863 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-07-04T17:25:13.896 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephfs-devel-2:19.0.0-4667.g9dba1e83.el9.x86_ 151/172 2024-07-04T17:25:13.961 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-07-04T17:25:14.007 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : NetworkManager-team-1:1.48.2-2.el9.x86_64 153/172 2024-07-04T17:25:14.062 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : NetworkManager-tui-1:1.48.2-2.el9.x86_64 154/172 2024-07-04T17:25:15.042 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-04T17:25:15.084 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-04T17:25:15.151 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-04T17:25:16.660 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephfs-top-2:19.0.0-4667.g9dba1e83.el9.noarch 156/172 2024-07-04T17:25:16.760 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 157/172 2024-07-04T17:25:16.820 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-07-04T17:25:16.973 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-07-04T17:25:17.011 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-07-04T17:25:17.044 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-07-04T17:25:17.078 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-04T17:25:17.120 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-07-04T17:25:17.121 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-04T17:25:17.262 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-04T17:25:17.315 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-07-04T17:25:17.316 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-04T17:25:17.486 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-04T17:25:17.525 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-07-04T17:25:17.561 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-04T17:25:17.561 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-04T17:25:17.897 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-04T17:25:17.897 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-04T17:25:18.042 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-04T17:25:18.086 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-04T17:25:18.129 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-07-04T17:25:18.269 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-04T17:25:18.547 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-04T17:25:18.547 INFO:teuthology.orchestra.run.smithi042.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-07-04T17:25:18.547 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:24.760 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-2:19.0.0-4667.g9dba1e83.el9.x86_64 1/172 2024-07-04T17:25:24.760 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-base-2:19.0.0-4667.g9dba1e83.el9.x86_64 2/172 2024-07-04T17:25:24.760 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-common-2:19.0.0-4667.g9dba1e83.el9.x86_64 3/172 2024-07-04T17:25:24.760 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-fuse-2:19.0.0-4667.g9dba1e83.el9.x86_64 4/172 2024-07-04T17:25:24.760 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-4667.g9dba1 5/172 2024-07-04T17:25:24.760 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mds-2:19.0.0-4667.g9dba1e83.el9.x86_64 6/172 2024-07-04T17:25:24.761 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-2:19.0.0-4667.g9dba1e83.el9.x86_64 7/172 2024-07-04T17:25:24.761 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mon-2:19.0.0-4667.g9dba1e83.el9.x86_64 8/172 2024-07-04T17:25:24.761 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-osd-2:19.0.0-4667.g9dba1e83.el9.x86_64 9/172 2024-07-04T17:25:24.761 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-radosgw-2:19.0.0-4667.g9dba1e83.el9.x86_64 10/172 2024-07-04T17:25:24.761 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-selinux-2:19.0.0-4667.g9dba1e83.el9.x86_64 11/172 2024-07-04T17:25:24.761 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-test-2:19.0.0-4667.g9dba1e83.el9.x86_64 12/172 2024-07-04T17:25:24.761 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephfs-mirror-2:19.0.0-4667.g9dba1e83.el9.x86_64 13/172 2024-07-04T17:25:24.761 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephfs-shell-2:19.0.0-4667.g9dba1e83.el9.x86_64 14/172 2024-07-04T17:25:24.761 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephfs-devel-2:19.0.0-4667.g9dba1e83.el9.x86_ 15/172 2024-07-04T17:25:24.761 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephfs2-2:19.0.0-4667.g9dba1e83.el9.x86_64 16/172 2024-07-04T17:25:24.761 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephsqlite-2:19.0.0-4667.g9dba1e83.el9.x86_64 17/172 2024-07-04T17:25:24.761 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados-devel-2:19.0.0-4667.g9dba1e83.el9.x86_6 18/172 2024-07-04T17:25:24.761 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libradosstriper1-2:19.0.0-4667.g9dba1e83.el9.x86 19/172 2024-07-04T17:25:24.762 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librgw2-2:19.0.0-4667.g9dba1e83.el9.x86_64 20/172 2024-07-04T17:25:24.762 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ceph-argparse-2:19.0.0-4667.g9dba1e83.el 21/172 2024-07-04T17:25:24.762 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ceph-common-2:19.0.0-4667.g9dba1e83.el9. 22/172 2024-07-04T17:25:24.762 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cephfs-2:19.0.0-4667.g9dba1e83.el9.x86_6 23/172 2024-07-04T17:25:24.762 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rados-2:19.0.0-4667.g9dba1e83.el9.x86_64 24/172 2024-07-04T17:25:24.762 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rbd-2:19.0.0-4667.g9dba1e83.el9.x86_64 25/172 2024-07-04T17:25:24.762 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rgw-2:19.0.0-4667.g9dba1e83.el9.x86_64 26/172 2024-07-04T17:25:24.762 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-fuse-2:19.0.0-4667.g9dba1e83.el9.x86_64 27/172 2024-07-04T17:25:24.762 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-mirror-2:19.0.0-4667.g9dba1e83.el9.x86_64 28/172 2024-07-04T17:25:24.762 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-nbd-2:19.0.0-4667.g9dba1e83.el9.x86_64 29/172 2024-07-04T17:25:24.762 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-4667.g9dba1e83. 30/172 2024-07-04T17:25:24.776 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-4667.g9dba1e83.el9.noa 31/172 2024-07-04T17:25:24.776 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-4667.g9dba1e83.el9.n 32/172 2024-07-04T17:25:24.776 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-4667.g9db 33/172 2024-07-04T17:25:24.776 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-4667.g9dba1e83.el 34/172 2024-07-04T17:25:24.776 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-rook-2:19.0.0-4667.g9dba1e83.el9.noarch 35/172 2024-07-04T17:25:24.776 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-4667.g9dba1e83.e 36/172 2024-07-04T17:25:24.776 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-volume-2:19.0.0-4667.g9dba1e83.el9.noarch 37/172 2024-07-04T17:25:24.776 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephadm-2:19.0.0-4667.g9dba1e83.el9.noarch 38/172 2024-07-04T17:25:24.776 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephfs-top-2:19.0.0-4667.g9dba1e83.el9.noarch 39/172 2024-07-04T17:25:24.776 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/172 2024-07-04T17:25:24.776 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-07-04T17:25:24.776 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-07-04T17:25:24.777 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-07-04T17:25:24.777 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-07-04T17:25:24.777 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-07-04T17:25:24.777 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-07-04T17:25:24.777 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-07-04T17:25:24.777 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/172 2024-07-04T17:25:24.777 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-07-04T17:25:24.777 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-07-04T17:25:24.777 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-07-04T17:25:24.777 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-07-04T17:25:24.777 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-07-04T17:25:24.777 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-07-04T17:25:24.778 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-07-04T17:25:24.778 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-07-04T17:25:24.778 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-07-04T17:25:24.778 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-07-04T17:25:24.778 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-07-04T17:25:24.778 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-07-04T17:25:24.778 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/172 2024-07-04T17:25:24.778 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-07-04T17:25:24.778 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-07-04T17:25:24.778 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-07-04T17:25:24.778 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-07-04T17:25:24.778 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-07-04T17:25:24.778 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-07-04T17:25:24.779 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/172 2024-07-04T17:25:24.779 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-07-04T17:25:24.779 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-07-04T17:25:24.779 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-07-04T17:25:24.779 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-07-04T17:25:24.779 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-07-04T17:25:24.779 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-07-04T17:25:24.779 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-07-04T17:25:24.779 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-07-04T17:25:24.779 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-07-04T17:25:24.779 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-07-04T17:25:24.779 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-07-04T17:25:24.779 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-07-04T17:25:24.780 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-07-04T17:25:24.780 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-07-04T17:25:24.780 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-07-04T17:25:24.780 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-07-04T17:25:24.780 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-04T17:25:24.780 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-07-04T17:25:24.780 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-07-04T17:25:24.780 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-07-04T17:25:24.780 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-07-04T17:25:24.780 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-07-04T17:25:24.780 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-07-04T17:25:24.781 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-07-04T17:25:24.781 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-07-04T17:25:24.781 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-07-04T17:25:24.781 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-07-04T17:25:24.781 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-07-04T17:25:24.781 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-07-04T17:25:24.781 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-07-04T17:25:24.781 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-07-04T17:25:24.781 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-07-04T17:25:24.781 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : xfsprogs-devel-6.4.0-2.el9.x86_64 101/172 2024-07-04T17:25:24.781 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-07-04T17:25:24.781 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-07-04T17:25:24.782 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-07-04T17:25:24.782 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-07-04T17:25:24.782 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-07-04T17:25:24.782 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-07-04T17:25:24.782 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-07-04T17:25:24.782 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-07-04T17:25:24.782 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-07-04T17:25:24.782 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-07-04T17:25:24.782 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-07-04T17:25:24.782 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-07-04T17:25:24.782 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-07-04T17:25:24.783 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-07-04T17:25:24.783 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-07-04T17:25:24.783 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-07-04T17:25:24.783 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/172 2024-07-04T17:25:24.783 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-07-04T17:25:24.783 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-07-04T17:25:24.783 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-07-04T17:25:24.783 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-google-auth-1:2.30.0-1.el9.noarch 122/172 2024-07-04T17:25:24.783 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-07-04T17:25:24.783 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-07-04T17:25:24.783 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-07-04T17:25:24.783 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-07-04T17:25:24.784 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-07-04T17:25:24.784 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-07-04T17:25:24.784 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-07-04T17:25:24.784 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-07-04T17:25:24.784 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-07-04T17:25:24.784 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-07-04T17:25:24.784 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-07-04T17:25:24.784 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-07-04T17:25:24.784 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-07-04T17:25:24.784 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-07-04T17:25:24.784 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-07-04T17:25:24.784 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-07-04T17:25:24.784 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-07-04T17:25:24.784 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-07-04T17:25:24.785 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-07-04T17:25:24.785 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/172 2024-07-04T17:25:24.785 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-07-04T17:25:24.785 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-07-04T17:25:24.785 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-07-04T17:25:24.785 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-07-04T17:25:24.785 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-07-04T17:25:24.785 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-07-04T17:25:24.785 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados2-2:19.0.0-4667.g9dba1e83.el9.x86_64 149/172 2024-07-04T17:25:24.785 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-07-04T17:25:24.785 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librbd1-2:19.0.0-4667.g9dba1e83.el9.x86_64 151/172 2024-07-04T17:25:24.785 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-07-04T17:25:24.786 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : NetworkManager-1:1.48.2-2.el9.x86_64 153/172 2024-07-04T17:25:24.786 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-07-04T17:25:24.786 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 155/172 2024-07-04T17:25:24.786 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-07-04T17:25:24.786 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : NetworkManager-team-1:1.48.2-2.el9.x86_64 157/172 2024-07-04T17:25:24.786 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-07-04T17:25:24.786 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : NetworkManager-tui-1:1.48.2-2.el9.x86_64 159/172 2024-07-04T17:25:24.786 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-07-04T17:25:24.786 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-07-04T17:25:24.786 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-04T17:25:24.786 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-07-04T17:25:24.787 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-07-04T17:25:24.787 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-07-04T17:25:24.787 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-07-04T17:25:24.787 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openssl-1:3.2.2-2.el9.x86_64 167/172 2024-07-04T17:25:24.787 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-07-04T17:25:24.787 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openssl-libs-1:3.2.2-2.el9.x86_64 169/172 2024-07-04T17:25:24.787 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-04T17:25:24.787 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : xfsprogs-6.4.0-2.el9.x86_64 171/172 2024-07-04T17:25:27.205 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-04T17:25:27.206 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:27.206 INFO:teuthology.orchestra.run.smithi042.stdout:Upgraded: 2024-07-04T17:25:27.206 INFO:teuthology.orchestra.run.smithi042.stdout: NetworkManager-1:1.48.2-2.el9.x86_64 2024-07-04T17:25:27.206 INFO:teuthology.orchestra.run.smithi042.stdout: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 2024-07-04T17:25:27.206 INFO:teuthology.orchestra.run.smithi042.stdout: NetworkManager-team-1:1.48.2-2.el9.x86_64 2024-07-04T17:25:27.206 INFO:teuthology.orchestra.run.smithi042.stdout: NetworkManager-tui-1:1.48.2-2.el9.x86_64 2024-07-04T17:25:27.206 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-07-04T17:25:27.206 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-07-04T17:25:27.206 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-07-04T17:25:27.206 INFO:teuthology.orchestra.run.smithi042.stdout: librados2-2:19.0.0-4667.g9dba1e83.el9.x86_64 2024-07-04T17:25:27.207 INFO:teuthology.orchestra.run.smithi042.stdout: librbd1-2:19.0.0-4667.g9dba1e83.el9.x86_64 2024-07-04T17:25:27.207 INFO:teuthology.orchestra.run.smithi042.stdout: openssl-1:3.2.2-2.el9.x86_64 2024-07-04T17:25:27.207 INFO:teuthology.orchestra.run.smithi042.stdout: openssl-libs-1:3.2.2-2.el9.x86_64 2024-07-04T17:25:27.207 INFO:teuthology.orchestra.run.smithi042.stdout: xfsprogs-6.4.0-2.el9.x86_64 2024-07-04T17:25:27.207 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-07-04T17:25:27.207 INFO:teuthology.orchestra.run.smithi042.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-07-04T17:25:27.207 INFO:teuthology.orchestra.run.smithi042.stdout: automake-1.16.2-8.el9.noarch 2024-07-04T17:25:27.207 INFO:teuthology.orchestra.run.smithi042.stdout: bison-3.7.4-5.el9.x86_64 2024-07-04T17:25:27.207 INFO:teuthology.orchestra.run.smithi042.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-07-04T17:25:27.207 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-2:19.0.0-4667.g9dba1e83.el9.x86_64 2024-07-04T17:25:27.207 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-base-2:19.0.0-4667.g9dba1e83.el9.x86_64 2024-07-04T17:25:27.207 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-common-2:19.0.0-4667.g9dba1e83.el9.x86_64 2024-07-04T17:25:27.207 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-fuse-2:19.0.0-4667.g9dba1e83.el9.x86_64 2024-07-04T17:25:27.207 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-grafana-dashboards-2:19.0.0-4667.g9dba1e83.el9.noarch 2024-07-04T17:25:27.208 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-immutable-object-cache-2:19.0.0-4667.g9dba1e83.el9.x86_64 2024-07-04T17:25:27.208 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mds-2:19.0.0-4667.g9dba1e83.el9.x86_64 2024-07-04T17:25:27.208 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-2:19.0.0-4667.g9dba1e83.el9.x86_64 2024-07-04T17:25:27.208 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-cephadm-2:19.0.0-4667.g9dba1e83.el9.noarch 2024-07-04T17:25:27.208 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-dashboard-2:19.0.0-4667.g9dba1e83.el9.noarch 2024-07-04T17:25:27.208 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-diskprediction-local-2:19.0.0-4667.g9dba1e83.el9.noarch 2024-07-04T17:25:27.208 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-modules-core-2:19.0.0-4667.g9dba1e83.el9.noarch 2024-07-04T17:25:27.208 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-rook-2:19.0.0-4667.g9dba1e83.el9.noarch 2024-07-04T17:25:27.208 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mon-2:19.0.0-4667.g9dba1e83.el9.x86_64 2024-07-04T17:25:27.208 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-osd-2:19.0.0-4667.g9dba1e83.el9.x86_64 2024-07-04T17:25:27.208 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-prometheus-alerts-2:19.0.0-4667.g9dba1e83.el9.noarch 2024-07-04T17:25:27.208 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-radosgw-2:19.0.0-4667.g9dba1e83.el9.x86_64 2024-07-04T17:25:27.208 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-selinux-2:19.0.0-4667.g9dba1e83.el9.x86_64 2024-07-04T17:25:27.208 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-test-2:19.0.0-4667.g9dba1e83.el9.x86_64 2024-07-04T17:25:27.208 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-volume-2:19.0.0-4667.g9dba1e83.el9.noarch 2024-07-04T17:25:27.209 INFO:teuthology.orchestra.run.smithi042.stdout: cephadm-2:19.0.0-4667.g9dba1e83.el9.noarch 2024-07-04T17:25:27.209 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-mirror-2:19.0.0-4667.g9dba1e83.el9.x86_64 2024-07-04T17:25:27.209 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-shell-2:19.0.0-4667.g9dba1e83.el9.x86_64 2024-07-04T17:25:27.209 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-top-2:19.0.0-4667.g9dba1e83.el9.noarch 2024-07-04T17:25:27.209 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-07-04T17:25:27.209 INFO:teuthology.orchestra.run.smithi042.stdout: flex-2.6.4-9.el9.x86_64 2024-07-04T17:25:27.209 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-07-04T17:25:27.209 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-07-04T17:25:27.209 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-07-04T17:25:27.209 INFO:teuthology.orchestra.run.smithi042.stdout: fuse-2.9.9-15.el9.x86_64 2024-07-04T17:25:27.209 INFO:teuthology.orchestra.run.smithi042.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-07-04T17:25:27.209 INFO:teuthology.orchestra.run.smithi042.stdout: grpc-data-1.46.7-10.el9.noarch 2024-07-04T17:25:27.209 INFO:teuthology.orchestra.run.smithi042.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-07-04T17:25:27.209 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow-9.0.0-11.el9.x86_64 2024-07-04T17:25:27.209 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-07-04T17:25:27.210 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs-devel-2:19.0.0-4667.g9dba1e83.el9.x86_64 2024-07-04T17:25:27.210 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs2-2:19.0.0-4667.g9dba1e83.el9.x86_64 2024-07-04T17:25:27.210 INFO:teuthology.orchestra.run.smithi042.stdout: libcephsqlite-2:19.0.0-4667.g9dba1e83.el9.x86_64 2024-07-04T17:25:27.210 INFO:teuthology.orchestra.run.smithi042.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-07-04T17:25:27.210 INFO:teuthology.orchestra.run.smithi042.stdout: liboath-2.6.7-2.el9.x86_64 2024-07-04T17:25:27.210 INFO:teuthology.orchestra.run.smithi042.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-07-04T17:25:27.210 INFO:teuthology.orchestra.run.smithi042.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-07-04T17:25:27.210 INFO:teuthology.orchestra.run.smithi042.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-07-04T17:25:27.210 INFO:teuthology.orchestra.run.smithi042.stdout: librados-devel-2:19.0.0-4667.g9dba1e83.el9.x86_64 2024-07-04T17:25:27.210 INFO:teuthology.orchestra.run.smithi042.stdout: libradosstriper1-2:19.0.0-4667.g9dba1e83.el9.x86_64 2024-07-04T17:25:27.210 INFO:teuthology.orchestra.run.smithi042.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-07-04T17:25:27.210 INFO:teuthology.orchestra.run.smithi042.stdout: librgw2-2:19.0.0-4667.g9dba1e83.el9.x86_64 2024-07-04T17:25:27.210 INFO:teuthology.orchestra.run.smithi042.stdout: libtool-2.4.6-46.el9.x86_64 2024-07-04T17:25:27.211 INFO:teuthology.orchestra.run.smithi042.stdout: libunwind-1.6.2-1.el9.x86_64 2024-07-04T17:25:27.211 INFO:teuthology.orchestra.run.smithi042.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-07-04T17:25:27.211 INFO:teuthology.orchestra.run.smithi042.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-07-04T17:25:27.211 INFO:teuthology.orchestra.run.smithi042.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-07-04T17:25:27.211 INFO:teuthology.orchestra.run.smithi042.stdout: lua-5.4.4-4.el9.x86_64 2024-07-04T17:25:27.211 INFO:teuthology.orchestra.run.smithi042.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-07-04T17:25:27.211 INFO:teuthology.orchestra.run.smithi042.stdout: luarocks-3.9.2-5.el9.noarch 2024-07-04T17:25:27.211 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-0.3.21-2.el9.x86_64 2024-07-04T17:25:27.211 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-07-04T17:25:27.211 INFO:teuthology.orchestra.run.smithi042.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-07-04T17:25:27.211 INFO:teuthology.orchestra.run.smithi042.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-07-04T17:25:27.211 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-13.14-1.el9.x86_64 2024-07-04T17:25:27.211 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-07-04T17:25:27.212 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-07-04T17:25:27.212 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-server-13.14-1.el9.x86_64 2024-07-04T17:25:27.212 INFO:teuthology.orchestra.run.smithi042.stdout: protobuf-3.14.0-13.el9.x86_64 2024-07-04T17:25:27.212 INFO:teuthology.orchestra.run.smithi042.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-07-04T17:25:27.212 INFO:teuthology.orchestra.run.smithi042.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-07-04T17:25:27.212 INFO:teuthology.orchestra.run.smithi042.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-07-04T17:25:27.212 INFO:teuthology.orchestra.run.smithi042.stdout: python3-babel-2.9.1-2.el9.noarch 2024-07-04T17:25:27.212 INFO:teuthology.orchestra.run.smithi042.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-07-04T17:25:27.212 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-07-04T17:25:27.212 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-argparse-2:19.0.0-4667.g9dba1e83.el9.x86_64 2024-07-04T17:25:27.212 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-common-2:19.0.0-4667.g9dba1e83.el9.x86_64 2024-07-04T17:25:27.212 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cephfs-2:19.0.0-4667.g9dba1e83.el9.x86_64 2024-07-04T17:25:27.212 INFO:teuthology.orchestra.run.smithi042.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-07-04T17:25:27.212 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-07-04T17:25:27.212 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-07-04T17:25:27.213 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-07-04T17:25:27.213 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-07-04T17:25:27.213 INFO:teuthology.orchestra.run.smithi042.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-07-04T17:25:27.213 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-07-04T17:25:27.213 INFO:teuthology.orchestra.run.smithi042.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-07-04T17:25:27.213 INFO:teuthology.orchestra.run.smithi042.stdout: python3-google-auth-1:2.30.0-1.el9.noarch 2024-07-04T17:25:27.213 INFO:teuthology.orchestra.run.smithi042.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-07-04T17:25:27.213 INFO:teuthology.orchestra.run.smithi042.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-07-04T17:25:27.213 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-07-04T17:25:27.213 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-07-04T17:25:27.213 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-07-04T17:25:27.213 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-07-04T17:25:27.213 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-07-04T17:25:27.213 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-07-04T17:25:27.214 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-07-04T17:25:27.214 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-07-04T17:25:27.214 INFO:teuthology.orchestra.run.smithi042.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-07-04T17:25:27.214 INFO:teuthology.orchestra.run.smithi042.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-07-04T17:25:27.214 INFO:teuthology.orchestra.run.smithi042.stdout: python3-mako-1.1.4-6.el9.noarch 2024-07-04T17:25:27.214 INFO:teuthology.orchestra.run.smithi042.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-07-04T17:25:27.214 INFO:teuthology.orchestra.run.smithi042.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-07-04T17:25:27.214 INFO:teuthology.orchestra.run.smithi042.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-07-04T17:25:27.214 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-07-04T17:25:27.214 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-07-04T17:25:27.214 INFO:teuthology.orchestra.run.smithi042.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-07-04T17:25:27.214 INFO:teuthology.orchestra.run.smithi042.stdout: python3-packaging-20.9-5.el9.noarch 2024-07-04T17:25:27.214 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-07-04T17:25:27.215 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ply-3.11-14.el9.noarch 2024-07-04T17:25:27.215 INFO:teuthology.orchestra.run.smithi042.stdout: python3-portend-3.1.0-2.el9.noarch 2024-07-04T17:25:27.215 INFO:teuthology.orchestra.run.smithi042.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-07-04T17:25:27.215 INFO:teuthology.orchestra.run.smithi042.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-07-04T17:25:27.215 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-07-04T17:25:27.215 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-07-04T17:25:27.215 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-07-04T17:25:27.215 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pycparser-2.20-6.el9.noarch 2024-07-04T17:25:27.215 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-07-04T17:25:27.215 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pytz-2021.1-5.el9.noarch 2024-07-04T17:25:27.215 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rados-2:19.0.0-4667.g9dba1e83.el9.x86_64 2024-07-04T17:25:27.215 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rbd-2:19.0.0-4667.g9dba1e83.el9.x86_64 2024-07-04T17:25:27.215 INFO:teuthology.orchestra.run.smithi042.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-07-04T17:25:27.215 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-2.25.1-8.el9.noarch 2024-07-04T17:25:27.215 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-07-04T17:25:27.216 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rgw-2:19.0.0-4667.g9dba1e83.el9.x86_64 2024-07-04T17:25:27.216 INFO:teuthology.orchestra.run.smithi042.stdout: python3-routes-2.5.1-5.el9.noarch 2024-07-04T17:25:27.216 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rsa-4.9-2.el9.noarch 2024-07-04T17:25:27.216 INFO:teuthology.orchestra.run.smithi042.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-07-04T17:25:27.216 INFO:teuthology.orchestra.run.smithi042.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-07-04T17:25:27.216 INFO:teuthology.orchestra.run.smithi042.stdout: python3-toml-0.10.2-6.el9.noarch 2024-07-04T17:25:27.216 INFO:teuthology.orchestra.run.smithi042.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-07-04T17:25:27.216 INFO:teuthology.orchestra.run.smithi042.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-07-04T17:25:27.216 INFO:teuthology.orchestra.run.smithi042.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-07-04T17:25:27.216 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webob-1.8.7-6.el9.noarch 2024-07-04T17:25:27.216 INFO:teuthology.orchestra.run.smithi042.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-07-04T17:25:27.216 INFO:teuthology.orchestra.run.smithi042.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-07-04T17:25:27.216 INFO:teuthology.orchestra.run.smithi042.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-07-04T17:25:27.216 INFO:teuthology.orchestra.run.smithi042.stdout: qatlib-24.02.0-1.el9.x86_64 2024-07-04T17:25:27.216 INFO:teuthology.orchestra.run.smithi042.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-07-04T17:25:27.217 INFO:teuthology.orchestra.run.smithi042.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-07-04T17:25:27.217 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-fuse-2:19.0.0-4667.g9dba1e83.el9.x86_64 2024-07-04T17:25:27.217 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-mirror-2:19.0.0-4667.g9dba1e83.el9.x86_64 2024-07-04T17:25:27.217 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-nbd-2:19.0.0-4667.g9dba1e83.el9.x86_64 2024-07-04T17:25:27.217 INFO:teuthology.orchestra.run.smithi042.stdout: re2-1:20211101-3.el9.x86_64 2024-07-04T17:25:27.217 INFO:teuthology.orchestra.run.smithi042.stdout: socat-1.7.4.1-6.el9.x86_64 2024-07-04T17:25:27.217 INFO:teuthology.orchestra.run.smithi042.stdout: thrift-0.15.0-2.el9.x86_64 2024-07-04T17:25:27.217 INFO:teuthology.orchestra.run.smithi042.stdout: uuid-1.6.2-55.el9.x86_64 2024-07-04T17:25:27.217 INFO:teuthology.orchestra.run.smithi042.stdout: xfsprogs-devel-6.4.0-2.el9.x86_64 2024-07-04T17:25:27.217 INFO:teuthology.orchestra.run.smithi042.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-07-04T17:25:27.218 INFO:teuthology.orchestra.run.smithi042.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-07-04T17:25:27.218 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:27.218 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-07-04T17:25:27.862 DEBUG:teuthology.parallel:result is None 2024-07-04T17:25:27.863 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9dba1e836b4756bf3a32be2a4936f371dff55008 2024-07-04T17:25:28.012 DEBUG:teuthology.orchestra.run.smithi042:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-07-04T17:25:28.074 INFO:teuthology.orchestra.run.smithi042.stdout:19.0.0-4667.g9dba1e83.el9 2024-07-04T17:25:28.075 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-4667.g9dba1e83.el9 2024-07-04T17:25:28.075 INFO:teuthology.task.install:The correct ceph version 19.0.0-4667.g9dba1e83 is installed. 2024-07-04T17:25:28.077 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-07-04T17:25:28.077 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:25:28.077 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-07-04T17:25:28.146 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-07-04T17:25:28.146 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:25:28.146 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/daemon-helper 2024-07-04T17:25:28.258 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-07-04T17:25:28.361 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-07-04T17:25:28.362 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:25:28.362 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/adjust-ulimits 2024-07-04T17:25:28.429 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-07-04T17:25:28.531 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-07-04T17:25:28.531 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:25:28.531 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/stdin-killer 2024-07-04T17:25:28.593 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-07-04T17:25:28.695 INFO:teuthology.run_tasks:Running task ceph... 2024-07-04T17:25:28.775 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-07-04T17:25:28.775 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 777 /var/log/ceph 2024-07-04T17:25:28.841 INFO:tasks.ceph:Disabling ceph logrotate... 2024-07-04T17:25:28.841 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /etc/logrotate.d/ceph 2024-07-04T17:25:28.908 INFO:tasks.ceph:Creating extra log directories... 2024-07-04T17:25:28.908 DEBUG:teuthology.orchestra.run.smithi042:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-07-04T17:25:28.975 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-07-04T17:25:28.975 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-07-04T17:25:28.975 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-07-04T17:25:28.975 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:25:28.975 DEBUG:teuthology.orchestra.run.smithi042:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-07-04T17:25:29.012 DEBUG:teuthology.orchestra.run.smithi042:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-07-04T17:25:29.201 DEBUG:teuthology.orchestra.run.smithi042:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-07-04T17:25:29.271 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-07-04T17:25:29.271 INFO:tasks.ceph:config {'conf': {'client': {'debug cephfs_mirror': 20, 'log to stderr': False}, 'client.mirror': {'admin socket': '/var/run/ceph/cephfs-mirror.asok', 'pid file': '/var/run/ceph/cephfs-mirror.pid'}, 'mgr': {'debug client': 10, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-07-04T17:25:29.272 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-07-04_16:36:07-fs-wip-rishabh-testing-20240701.141150-debug-testing-default-smithi/7787245', 'branch': 'wip-rishabh-testing-20240701.141150-debug', 'description': 'fs/mirror/{begin/{0-install 1-ceph 2-logrotate 3-modules} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} conf/{client mds mgr mon osd} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health pg_health} supported-random-distros$/{centos_latest} tasks/mirror}', 'email': None, 'first_in_suite': False, 'job_id': '7787245', 'kernel': {'flavor': 'default', 'kdb': 1, 'sha1': '0d32a7bba447dd2470f4de43560317375ca9e9c8'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'meta': [{'desc': 'run one cephfs-mirror daemon on primary cluster'}, {'desc': 'configure the permissions for client.mirror'}, {'desc': '1 ceph cluster with 1 mon, 1 mgr, 3 osds, 5 mdss'}], 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-07-04_16:36:07-fs-wip-rishabh-testing-20240701.141150-debug-testing-default-smithi', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240701.141150-debug'}, 'ceph': {'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'debug cephfs_mirror': 20, 'log to stderr': False}, 'client.mirror': {'admin socket': '/var/run/ceph/cephfs-mirror.asok', 'pid file': '/var/run/ceph/cephfs-mirror.pid'}, 'mgr': {'debug client': 10, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '9dba1e836b4756bf3a32be2a4936f371dff55008'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '9dba1e836b4756bf3a32be2a4936f371dff55008'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240701.141150-debug', 'sha1': '9dba1e836b4756bf3a32be2a4936f371dff55008'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 74, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.x', 'mds.a', 'mds.b', 'mds.c', 'mds.d', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'client.0', 'client.1', 'client.mirror', 'client.mirror_remote']], 'seed': 2237, 'sha1': '9dba1e836b4756bf3a32be2a4936f371dff55008', 'sleep_before_teardown': 0, 'subset': '481/512', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240701.141150-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_9dba1e836b4756bf3a32be2a4936f371dff55008/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '9dba1e836b4756bf3a32be2a4936f371dff55008', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi042.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGryEf5s/0njFgjBZdSbkWhuJA3zmXyra5E05P04fssEHhz6Me5UzimgC6v5ntl4y1A+jx7lND7JnD5T1968stQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': 1, 'sha1': '0d32a7bba447dd2470f4de43560317375ca9e9c8'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '9dba1e836b4756bf3a32be2a4936f371dff55008'}}, {'ceph': {'create_rbd_pool': False, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'debug cephfs_mirror': 20, 'log to stderr': False}, 'client.mirror': {'admin socket': '/var/run/ceph/cephfs-mirror.asok', 'pid file': '/var/run/ceph/cephfs-mirror.pid'}, 'mgr': {'debug client': 10, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '9dba1e836b4756bf3a32be2a4936f371dff55008', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'cephfs-mirror': {'client': 'client.mirror'}}, {'exec': {'client.mirror': ["sudo ceph auth caps client.mirror mon 'profile cephfs-mirror' mds 'allow r' osd 'allow rw tag cephfs metadata=*, allow r tag cephfs data=*' mgr 'allow r'"], 'client.mirror_remote': ["sudo ceph auth caps client.mirror_remote mon 'allow r' mds 'allow rwps' osd 'allow rw tag cephfs *=*' mgr 'allow r'"]}}, {'ceph-fuse': ['client.0', 'client.1']}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_mirroring.TestMirroring']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'a4a89a3d72be587ba6bd8509090508e8a19e74c5', 'timestamp': '2024-07-04_16:36:07', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2698093'} 2024-07-04T17:25:29.272 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-07-04T17:25:29.311 DEBUG:teuthology.orchestra.run.smithi042:> sudo install -d -m0777 -- /var/run/ceph 2024-07-04T17:25:29.412 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:25:29.412 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/scratch_devs of=/dev/stdout 2024-07-04T17:25:29.448 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-07-04T17:25:29.449 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_1 2024-07-04T17:25:29.525 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-07-04T17:25:29.525 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-04T17:25:29.525 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-07-04T17:25:29.526 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-04T17:25:29.526 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-07-04T17:25:29.526 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-07-04 17:25:18.981349869 +0000 2024-07-04T17:25:29.526 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-07-04 17:22:16.719629605 +0000 2024-07-04T17:25:29.526 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-07-04 17:22:16.719629605 +0000 2024-07-04T17:25:29.526 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-07-04 17:22:16.719629605 +0000 2024-07-04T17:25:29.526 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-07-04T17:25:29.624 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-07-04T17:25:29.624 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-07-04T17:25:29.624 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000265026 s, 1.9 MB/s 2024-07-04T17:25:29.627 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-07-04T17:25:29.670 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_2 2024-07-04T17:25:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-07-04T17:25:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-04T17:25:29.749 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-07-04T17:25:29.749 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-04T17:25:29.749 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-07-04T17:25:29.749 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-07-04 17:25:18.982349861 +0000 2024-07-04T17:25:29.749 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-07-04 17:22:17.353624892 +0000 2024-07-04T17:25:29.749 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-07-04 17:22:17.353624892 +0000 2024-07-04T17:25:29.749 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-07-04 17:22:17.353624892 +0000 2024-07-04T17:25:29.749 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-07-04T17:25:29.846 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-07-04T17:25:29.846 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-07-04T17:25:29.846 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000194921 s, 2.6 MB/s 2024-07-04T17:25:29.848 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-07-04T17:25:29.886 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_3 2024-07-04T17:25:29.964 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-07-04T17:25:29.964 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-04T17:25:29.964 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-07-04T17:25:29.964 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-04T17:25:29.964 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-07-04T17:25:29.964 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-07-04 17:25:18.982349861 +0000 2024-07-04T17:25:29.964 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-07-04 17:22:17.927620624 +0000 2024-07-04T17:25:29.964 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-07-04 17:22:17.927620624 +0000 2024-07-04T17:25:29.964 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-07-04 17:22:17.927620624 +0000 2024-07-04T17:25:29.964 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-07-04T17:25:30.077 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-07-04T17:25:30.077 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-07-04T17:25:30.077 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000250409 s, 2.0 MB/s 2024-07-04T17:25:30.080 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-07-04T17:25:30.126 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_4 2024-07-04T17:25:30.205 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-07-04T17:25:30.205 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-04T17:25:30.206 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-07-04T17:25:30.206 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-04T17:25:30.206 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-07-04T17:25:30.206 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-07-04 17:25:18.983349854 +0000 2024-07-04T17:25:30.206 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-07-04 17:22:18.515616253 +0000 2024-07-04T17:25:30.206 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-07-04 17:22:18.516616245 +0000 2024-07-04T17:25:30.206 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-07-04 17:22:18.515616253 +0000 2024-07-04T17:25:30.206 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-07-04T17:25:30.307 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-07-04T17:25:30.307 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-07-04T17:25:30.307 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000230631 s, 2.2 MB/s 2024-07-04T17:25:30.310 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-07-04T17:25:30.356 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-07-04T17:25:30.356 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi042.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-07-04T17:25:30.356 INFO:tasks.ceph:Generating config... 2024-07-04T17:25:30.358 INFO:tasks.ceph:[client] debug cephfs_mirror = 20 2024-07-04T17:25:30.358 INFO:tasks.ceph:[client] log to stderr = False 2024-07-04T17:25:30.358 INFO:tasks.ceph:[client.mirror] admin socket = /var/run/ceph/cephfs-mirror.asok 2024-07-04T17:25:30.358 INFO:tasks.ceph:[client.mirror] pid file = /var/run/ceph/cephfs-mirror.pid 2024-07-04T17:25:30.358 INFO:tasks.ceph:[mgr] debug client = 10 2024-07-04T17:25:30.358 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-07-04T17:25:30.358 INFO:tasks.ceph:[mgr] debug ms = 1 2024-07-04T17:25:30.358 INFO:tasks.ceph:[mon] debug mon = 20 2024-07-04T17:25:30.358 INFO:tasks.ceph:[mon] debug ms = 1 2024-07-04T17:25:30.358 INFO:tasks.ceph:[mon] debug paxos = 20 2024-07-04T17:25:30.358 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-07-04T17:25:30.358 INFO:tasks.ceph:[osd] bdev async discard = True 2024-07-04T17:25:30.359 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-07-04T17:25:30.359 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-07-04T17:25:30.359 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-07-04T17:25:30.359 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-07-04T17:25:30.359 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-07-04T17:25:30.359 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-07-04T17:25:30.359 INFO:tasks.ceph:[osd] debug ms = 1 2024-07-04T17:25:30.359 INFO:tasks.ceph:[osd] debug osd = 20 2024-07-04T17:25:30.359 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-07-04T17:25:30.359 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-07-04T17:25:30.359 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-07-04T17:25:30.359 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-07-04T17:25:30.359 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-07-04T17:25:30.359 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-07-04T17:25:30.360 INFO:tasks.ceph:Setting up mon.a... 2024-07-04T17:25:30.360 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-07-04T17:25:30.593 INFO:teuthology.orchestra.run.smithi042.stdout:creating /etc/ceph/ceph.keyring 2024-07-04T17:25:30.607 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-07-04T17:25:30.782 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-07-04T17:25:30.843 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.42')] 2024-07-04T17:25:30.843 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.42'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 10}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug cephfs_mirror': 20, 'log to stderr': False}, 'mon.a': {}, 'mds.a': {}, 'mds.b': {}, 'mds.c': {}, 'mds.d': {}, 'mds.e': {}, 'client.mirror': {'admin socket': '/var/run/ceph/cephfs-mirror.asok', 'pid file': '/var/run/ceph/cephfs-mirror.pid'}} 2024-07-04T17:25:30.844 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:25:30.844 DEBUG:teuthology.orchestra.run.smithi042:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-07-04T17:25:30.880 DEBUG:teuthology.orchestra.run.smithi042:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.42 --print /home/ubuntu/cephtest/ceph.monmap 2024-07-04T17:25:31.078 INFO:teuthology.orchestra.run.smithi042.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-07-04T17:25:31.079 INFO:teuthology.orchestra.run.smithi042.stdout:monmaptool: generated fsid be4f300a-93ef-4489-b540-37ece39b0aac 2024-07-04T17:25:31.079 INFO:teuthology.orchestra.run.smithi042.stdout:setting min_mon_release = quincy 2024-07-04T17:25:31.079 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 0 2024-07-04T17:25:31.079 INFO:teuthology.orchestra.run.smithi042.stdout:fsid be4f300a-93ef-4489-b540-37ece39b0aac 2024-07-04T17:25:31.079 INFO:teuthology.orchestra.run.smithi042.stdout:last_changed 2024-07-04T17:25:31.078488+0000 2024-07-04T17:25:31.079 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-07-04T17:25:31.078488+0000 2024-07-04T17:25:31.079 INFO:teuthology.orchestra.run.smithi042.stdout:min_mon_release 17 (quincy) 2024-07-04T17:25:31.079 INFO:teuthology.orchestra.run.smithi042.stdout:election_strategy: 1 2024-07-04T17:25:31.079 INFO:teuthology.orchestra.run.smithi042.stdout:0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-07-04T17:25:31.079 INFO:teuthology.orchestra.run.smithi042.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2024-07-04T17:25:31.089 DEBUG:teuthology.orchestra.run.smithi042:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-07-04T17:25:31.139 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID be4f300a-93ef-4489-b540-37ece39b0aac... 2024-07-04T17:25:31.140 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-07-04T17:25:31.274 INFO:teuthology.orchestra.run.smithi042.stdout:[global] 2024-07-04T17:25:31.275 INFO:teuthology.orchestra.run.smithi042.stdout: chdir = "" 2024-07-04T17:25:31.275 INFO:teuthology.orchestra.run.smithi042.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-07-04T17:25:31.275 INFO:teuthology.orchestra.run.smithi042.stdout: auth supported = cephx 2024-07-04T17:25:31.275 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:31.275 INFO:teuthology.orchestra.run.smithi042.stdout: filestore xattr use omap = true 2024-07-04T17:25:31.275 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:31.275 INFO:teuthology.orchestra.run.smithi042.stdout: mon clock drift allowed = 1.000 2024-07-04T17:25:31.275 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:31.275 INFO:teuthology.orchestra.run.smithi042.stdout: osd crush chooseleaf type = 0 2024-07-04T17:25:31.275 INFO:teuthology.orchestra.run.smithi042.stdout: auth debug = true 2024-07-04T17:25:31.275 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:31.275 INFO:teuthology.orchestra.run.smithi042.stdout: ms die on old message = true 2024-07-04T17:25:31.275 INFO:teuthology.orchestra.run.smithi042.stdout: ms die on bug = true 2024-07-04T17:25:31.275 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:31.276 INFO:teuthology.orchestra.run.smithi042.stdout: mon max pg per osd = 10000 # >= luminous 2024-07-04T17:25:31.276 INFO:teuthology.orchestra.run.smithi042.stdout: mon pg warn max object skew = 0 2024-07-04T17:25:31.276 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:31.276 INFO:teuthology.orchestra.run.smithi042.stdout: # disable pg_autoscaler by default for new pools 2024-07-04T17:25:31.276 INFO:teuthology.orchestra.run.smithi042.stdout: osd_pool_default_pg_autoscale_mode = off 2024-07-04T17:25:31.276 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:31.276 INFO:teuthology.orchestra.run.smithi042.stdout: osd pool default size = 2 2024-07-04T17:25:31.276 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:31.276 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd allow primary affinity = true 2024-07-04T17:25:31.276 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd allow pg remap = true 2024-07-04T17:25:31.276 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on legacy crush tunables = false 2024-07-04T17:25:31.276 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on crush straw calc version zero = false 2024-07-04T17:25:31.276 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on no sortbitwise = false 2024-07-04T17:25:31.276 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on osd down out interval zero = false 2024-07-04T17:25:31.276 INFO:teuthology.orchestra.run.smithi042.stdout: mon warn on too few osds = false 2024-07-04T17:25:31.277 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-07-04T17:25:31.277 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_pool_no_redundancy = false 2024-07-04T17:25:31.277 INFO:teuthology.orchestra.run.smithi042.stdout: mon_allow_pool_size_one = true 2024-07-04T17:25:31.277 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:31.277 INFO:teuthology.orchestra.run.smithi042.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-07-04T17:25:31.277 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:31.277 INFO:teuthology.orchestra.run.smithi042.stdout: osd default data pool replay window = 5 2024-07-04T17:25:31.277 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:31.277 INFO:teuthology.orchestra.run.smithi042.stdout: mon allow pool delete = true 2024-07-04T17:25:31.277 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:31.277 INFO:teuthology.orchestra.run.smithi042.stdout: mon cluster log file level = debug 2024-07-04T17:25:31.277 INFO:teuthology.orchestra.run.smithi042.stdout: debug asserts on shutdown = true 2024-07-04T17:25:31.277 INFO:teuthology.orchestra.run.smithi042.stdout: mon health detail to clog = false 2024-07-04T17:25:31.277 INFO:teuthology.orchestra.run.smithi042.stdout: mon host = 172.21.15.42 2024-07-04T17:25:31.278 INFO:teuthology.orchestra.run.smithi042.stdout: fsid = be4f300a-93ef-4489-b540-37ece39b0aac 2024-07-04T17:25:31.278 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:31.278 INFO:teuthology.orchestra.run.smithi042.stdout:[osd] 2024-07-04T17:25:31.278 INFO:teuthology.orchestra.run.smithi042.stdout: osd journal size = 100 2024-07-04T17:25:31.278 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:31.278 INFO:teuthology.orchestra.run.smithi042.stdout: osd scrub load threshold = 5.0 2024-07-04T17:25:31.278 INFO:teuthology.orchestra.run.smithi042.stdout: osd scrub max interval = 600 2024-07-04T17:25:31.278 INFO:teuthology.orchestra.run.smithi042.stdout: osd mclock profile = high_recovery_ops 2024-07-04T17:25:31.278 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:31.278 INFO:teuthology.orchestra.run.smithi042.stdout: osd recover clone overlap = true 2024-07-04T17:25:31.278 INFO:teuthology.orchestra.run.smithi042.stdout: osd recovery max chunk = 1048576 2024-07-04T17:25:31.278 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:31.278 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug shutdown = true 2024-07-04T17:25:31.278 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug op order = true 2024-07-04T17:25:31.278 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug verify stray on activate = true 2024-07-04T17:25:31.279 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug trim objects = true 2024-07-04T17:25:31.279 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:31.279 INFO:teuthology.orchestra.run.smithi042.stdout: osd open classes on start = true 2024-07-04T17:25:31.279 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug pg log writeout = true 2024-07-04T17:25:31.279 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:31.279 INFO:teuthology.orchestra.run.smithi042.stdout: osd deep scrub update digest min age = 30 2024-07-04T17:25:31.279 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:31.279 INFO:teuthology.orchestra.run.smithi042.stdout: osd map max advance = 10 2024-07-04T17:25:31.279 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:31.279 INFO:teuthology.orchestra.run.smithi042.stdout: journal zero on create = true 2024-07-04T17:25:31.279 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:31.279 INFO:teuthology.orchestra.run.smithi042.stdout: filestore ondisk finisher threads = 3 2024-07-04T17:25:31.279 INFO:teuthology.orchestra.run.smithi042.stdout: filestore apply finisher threads = 3 2024-07-04T17:25:31.279 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:31.279 INFO:teuthology.orchestra.run.smithi042.stdout: bdev debug aio = true 2024-07-04T17:25:31.280 INFO:teuthology.orchestra.run.smithi042.stdout: osd debug misdirected ops = true 2024-07-04T17:25:31.280 INFO:teuthology.orchestra.run.smithi042.stdout: bdev async discard = True 2024-07-04T17:25:31.280 INFO:teuthology.orchestra.run.smithi042.stdout: bdev enable discard = True 2024-07-04T17:25:31.280 INFO:teuthology.orchestra.run.smithi042.stdout: bluestore allocator = bitmap 2024-07-04T17:25:31.280 INFO:teuthology.orchestra.run.smithi042.stdout: bluestore block size = 96636764160 2024-07-04T17:25:31.280 INFO:teuthology.orchestra.run.smithi042.stdout: bluestore fsck on mount = True 2024-07-04T17:25:31.280 INFO:teuthology.orchestra.run.smithi042.stdout: debug bluefs = 1/20 2024-07-04T17:25:31.280 INFO:teuthology.orchestra.run.smithi042.stdout: debug bluestore = 1/20 2024-07-04T17:25:31.280 INFO:teuthology.orchestra.run.smithi042.stdout: debug ms = 1 2024-07-04T17:25:31.280 INFO:teuthology.orchestra.run.smithi042.stdout: debug osd = 20 2024-07-04T17:25:31.280 INFO:teuthology.orchestra.run.smithi042.stdout: debug rocksdb = 4/10 2024-07-04T17:25:31.280 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd backfillfull_ratio = 0.85 2024-07-04T17:25:31.280 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd full ratio = 0.9 2024-07-04T17:25:31.280 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd nearfull ratio = 0.8 2024-07-04T17:25:31.280 INFO:teuthology.orchestra.run.smithi042.stdout: osd failsafe full ratio = 0.95 2024-07-04T17:25:31.281 INFO:teuthology.orchestra.run.smithi042.stdout: osd objectstore = bluestore 2024-07-04T17:25:31.281 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:31.281 INFO:teuthology.orchestra.run.smithi042.stdout:[mgr] 2024-07-04T17:25:31.281 INFO:teuthology.orchestra.run.smithi042.stdout: debug ms = 1 2024-07-04T17:25:31.281 INFO:teuthology.orchestra.run.smithi042.stdout: debug mgr = 20 2024-07-04T17:25:31.281 INFO:teuthology.orchestra.run.smithi042.stdout: debug mon = 20 2024-07-04T17:25:31.281 INFO:teuthology.orchestra.run.smithi042.stdout: debug auth = 20 2024-07-04T17:25:31.281 INFO:teuthology.orchestra.run.smithi042.stdout: mon reweight min pgs per osd = 4 2024-07-04T17:25:31.281 INFO:teuthology.orchestra.run.smithi042.stdout: mon reweight min bytes per osd = 10 2024-07-04T17:25:31.281 INFO:teuthology.orchestra.run.smithi042.stdout: mgr/telemetry/nag = false 2024-07-04T17:25:31.281 INFO:teuthology.orchestra.run.smithi042.stdout: debug client = 10 2024-07-04T17:25:31.281 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:31.281 INFO:teuthology.orchestra.run.smithi042.stdout:[mon] 2024-07-04T17:25:31.281 INFO:teuthology.orchestra.run.smithi042.stdout: debug ms = 1 2024-07-04T17:25:31.282 INFO:teuthology.orchestra.run.smithi042.stdout: debug mon = 20 2024-07-04T17:25:31.282 INFO:teuthology.orchestra.run.smithi042.stdout: debug paxos = 20 2024-07-04T17:25:31.282 INFO:teuthology.orchestra.run.smithi042.stdout: debug auth = 20 2024-07-04T17:25:31.282 INFO:teuthology.orchestra.run.smithi042.stdout: mon data avail warn = 5 2024-07-04T17:25:31.282 INFO:teuthology.orchestra.run.smithi042.stdout: mon mgr mkfs grace = 240 2024-07-04T17:25:31.282 INFO:teuthology.orchestra.run.smithi042.stdout: mon reweight min pgs per osd = 4 2024-07-04T17:25:31.282 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd reporter subtree level = osd 2024-07-04T17:25:31.282 INFO:teuthology.orchestra.run.smithi042.stdout: mon osd prime pg temp = true 2024-07-04T17:25:31.282 INFO:teuthology.orchestra.run.smithi042.stdout: mon reweight min bytes per osd = 10 2024-07-04T17:25:31.282 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:31.282 INFO:teuthology.orchestra.run.smithi042.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-07-04T17:25:31.282 INFO:teuthology.orchestra.run.smithi042.stdout: auth mon ticket ttl = 660 # 11m 2024-07-04T17:25:31.282 INFO:teuthology.orchestra.run.smithi042.stdout: auth service ticket ttl = 240 # 4m 2024-07-04T17:25:31.282 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:31.282 INFO:teuthology.orchestra.run.smithi042.stdout: # don't complain about insecure global_id in the test suite 2024-07-04T17:25:31.283 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-07-04T17:25:31.283 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-07-04T17:25:31.283 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:31.283 INFO:teuthology.orchestra.run.smithi042.stdout: # 1m isn't quite enough 2024-07-04T17:25:31.283 INFO:teuthology.orchestra.run.smithi042.stdout: mon_down_mkfs_grace = 2m 2024-07-04T17:25:31.283 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:31.283 INFO:teuthology.orchestra.run.smithi042.stdout: mon_warn_on_filestore_osds = false 2024-07-04T17:25:31.283 INFO:teuthology.orchestra.run.smithi042.stdout: mon down mkfs grace = 300 2024-07-04T17:25:31.283 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:31.283 INFO:teuthology.orchestra.run.smithi042.stdout:[client] 2024-07-04T17:25:31.283 INFO:teuthology.orchestra.run.smithi042.stdout: rgw cache enabled = true 2024-07-04T17:25:31.283 INFO:teuthology.orchestra.run.smithi042.stdout: rgw enable ops log = true 2024-07-04T17:25:31.283 INFO:teuthology.orchestra.run.smithi042.stdout: rgw enable usage log = true 2024-07-04T17:25:31.283 INFO:teuthology.orchestra.run.smithi042.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-07-04T17:25:31.283 INFO:teuthology.orchestra.run.smithi042.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-07-04T17:25:31.283 INFO:teuthology.orchestra.run.smithi042.stdout: debug cephfs_mirror = 20 2024-07-04T17:25:31.284 INFO:teuthology.orchestra.run.smithi042.stdout: log to stderr = False 2024-07-04T17:25:31.284 INFO:teuthology.orchestra.run.smithi042.stdout:[mon.a] 2024-07-04T17:25:31.284 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.a] 2024-07-04T17:25:31.284 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.b] 2024-07-04T17:25:31.284 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.c] 2024-07-04T17:25:31.284 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.d] 2024-07-04T17:25:31.284 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.e] 2024-07-04T17:25:31.284 INFO:teuthology.orchestra.run.smithi042.stdout:[client.mirror] 2024-07-04T17:25:31.284 INFO:teuthology.orchestra.run.smithi042.stdout: admin socket = /var/run/ceph/cephfs-mirror.asok 2024-07-04T17:25:31.284 INFO:teuthology.orchestra.run.smithi042.stdout: pid file = /var/run/ceph/cephfs-mirror.pid 2024-07-04T17:25:31.312 INFO:tasks.ceph:Creating admin key on mon.a... 2024-07-04T17:25:31.312 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-07-04T17:25:31.532 INFO:tasks.ceph:Copying monmap to all nodes... 2024-07-04T17:25:31.532 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:25:31.532 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-07-04T17:25:31.583 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:25:31.583 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-07-04T17:25:31.635 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi042.front.sepia.ceph.com 2024-07-04T17:25:31.636 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:25:31.636 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/ceph/ceph.keyring 2024-07-04T17:25:31.636 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-07-04T17:25:31.760 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:25:31.760 DEBUG:teuthology.orchestra.run.smithi042:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-07-04T17:25:31.858 INFO:tasks.ceph:Setting up mon nodes... 2024-07-04T17:25:31.858 INFO:tasks.ceph:Setting up mgr nodes... 2024-07-04T17:25:31.858 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-07-04T17:25:32.138 INFO:teuthology.orchestra.run.smithi042.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-07-04T17:25:32.156 INFO:tasks.ceph:Setting up mds nodes... 2024-07-04T17:25:32.156 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-07-04T17:25:32.386 INFO:teuthology.orchestra.run.smithi042.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-07-04T17:25:32.401 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-07-04T17:25:32.489 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-07-04T17:25:32.733 INFO:teuthology.orchestra.run.smithi042.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-07-04T17:25:32.750 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-07-04T17:25:32.840 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-07-04T17:25:33.081 INFO:teuthology.orchestra.run.smithi042.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-07-04T17:25:33.097 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-07-04T17:25:33.172 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-07-04T17:25:33.395 INFO:teuthology.orchestra.run.smithi042.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-07-04T17:25:33.409 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-07-04T17:25:33.495 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2024-07-04T17:25:33.714 INFO:teuthology.orchestra.run.smithi042.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2024-07-04T17:25:33.729 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2024-07-04T17:25:33.816 INFO:tasks.ceph_client:Setting up client nodes... 2024-07-04T17:25:33.817 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-07-04T17:25:34.010 INFO:teuthology.orchestra.run.smithi042.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-07-04T17:25:34.074 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-07-04T17:25:34.269 INFO:teuthology.orchestra.run.smithi042.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-07-04T17:25:34.317 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror /etc/ceph/ceph.client.mirror.keyring && sudo chmod 0644 /etc/ceph/ceph.client.mirror.keyring 2024-07-04T17:25:34.516 INFO:teuthology.orchestra.run.smithi042.stdout:creating /etc/ceph/ceph.client.mirror.keyring 2024-07-04T17:25:34.574 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror_remote /etc/ceph/ceph.client.mirror_remote.keyring && sudo chmod 0644 /etc/ceph/ceph.client.mirror_remote.keyring 2024-07-04T17:25:34.774 INFO:teuthology.orchestra.run.smithi042.stdout:creating /etc/ceph/ceph.client.mirror_remote.keyring 2024-07-04T17:25:34.824 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-07-04T17:25:34.824 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi042.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-07-04T17:25:34.824 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-07-04T17:25:34.901 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-07-04T17:25:34.901 INFO:tasks.ceph:role: osd.0 2024-07-04T17:25:34.901 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi042.front.sepia.ceph.com 2024-07-04T17:25:34.902 DEBUG:teuthology.orchestra.run.smithi042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-07-04T17:25:34.985 INFO:teuthology.orchestra.run.smithi042.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-07-04T17:25:34.985 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 attr=2, projid32bit=1 2024-07-04T17:25:34.985 INFO:teuthology.orchestra.run.smithi042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-07-04T17:25:34.985 INFO:teuthology.orchestra.run.smithi042.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-07-04T17:25:34.986 INFO:teuthology.orchestra.run.smithi042.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-07-04T17:25:34.986 INFO:teuthology.orchestra.run.smithi042.stdout: = sunit=0 swidth=0 blks 2024-07-04T17:25:34.986 INFO:teuthology.orchestra.run.smithi042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-07-04T17:25:34.986 INFO:teuthology.orchestra.run.smithi042.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-07-04T17:25:34.986 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-07-04T17:25:34.986 INFO:teuthology.orchestra.run.smithi042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-07-04T17:25:35.023 INFO:teuthology.orchestra.run.smithi042.stdout:Discarding blocks...Done. 2024-07-04T17:25:35.029 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi042.front.sepia.ceph.com -o noatime 2024-07-04T17:25:35.030 DEBUG:teuthology.orchestra.run.smithi042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-07-04T17:25:35.142 DEBUG:teuthology.orchestra.run.smithi042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-07-04T17:25:35.217 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-07-04T17:25:35.292 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-07-04T17:25:35.292 INFO:tasks.ceph:role: osd.1 2024-07-04T17:25:35.292 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi042.front.sepia.ceph.com 2024-07-04T17:25:35.292 DEBUG:teuthology.orchestra.run.smithi042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-07-04T17:25:35.375 INFO:teuthology.orchestra.run.smithi042.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-07-04T17:25:35.376 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 attr=2, projid32bit=1 2024-07-04T17:25:35.376 INFO:teuthology.orchestra.run.smithi042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-07-04T17:25:35.376 INFO:teuthology.orchestra.run.smithi042.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-07-04T17:25:35.376 INFO:teuthology.orchestra.run.smithi042.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-07-04T17:25:35.376 INFO:teuthology.orchestra.run.smithi042.stdout: = sunit=0 swidth=0 blks 2024-07-04T17:25:35.376 INFO:teuthology.orchestra.run.smithi042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-07-04T17:25:35.376 INFO:teuthology.orchestra.run.smithi042.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-07-04T17:25:35.376 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-07-04T17:25:35.376 INFO:teuthology.orchestra.run.smithi042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-07-04T17:25:35.413 INFO:teuthology.orchestra.run.smithi042.stdout:Discarding blocks...Done. 2024-07-04T17:25:35.419 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi042.front.sepia.ceph.com -o noatime 2024-07-04T17:25:35.419 DEBUG:teuthology.orchestra.run.smithi042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-07-04T17:25:35.522 DEBUG:teuthology.orchestra.run.smithi042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-07-04T17:25:35.591 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-07-04T17:25:35.658 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-07-04T17:25:35.659 INFO:tasks.ceph:role: osd.2 2024-07-04T17:25:35.659 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi042.front.sepia.ceph.com 2024-07-04T17:25:35.659 DEBUG:teuthology.orchestra.run.smithi042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-07-04T17:25:35.729 INFO:teuthology.orchestra.run.smithi042.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-07-04T17:25:35.729 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 attr=2, projid32bit=1 2024-07-04T17:25:35.729 INFO:teuthology.orchestra.run.smithi042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-07-04T17:25:35.729 INFO:teuthology.orchestra.run.smithi042.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-07-04T17:25:35.729 INFO:teuthology.orchestra.run.smithi042.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-07-04T17:25:35.729 INFO:teuthology.orchestra.run.smithi042.stdout: = sunit=0 swidth=0 blks 2024-07-04T17:25:35.729 INFO:teuthology.orchestra.run.smithi042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-07-04T17:25:35.729 INFO:teuthology.orchestra.run.smithi042.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-07-04T17:25:35.730 INFO:teuthology.orchestra.run.smithi042.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-07-04T17:25:35.730 INFO:teuthology.orchestra.run.smithi042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-07-04T17:25:35.766 INFO:teuthology.orchestra.run.smithi042.stdout:Discarding blocks...Done. 2024-07-04T17:25:35.772 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi042.front.sepia.ceph.com -o noatime 2024-07-04T17:25:35.772 DEBUG:teuthology.orchestra.run.smithi042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-07-04T17:25:35.870 DEBUG:teuthology.orchestra.run.smithi042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-07-04T17:25:35.942 DEBUG:teuthology.orchestra.run.smithi042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-07-04T17:25:36.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:36.139+0000 7fcd4cf2c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-07-04T17:25:36.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:36.140+0000 7fcd4cf2c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-07-04T17:25:36.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:36.140+0000 7fcd4cf2c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-07-04T17:25:36.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:36.141+0000 7fcd4cf2c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-07-04T17:25:36.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:36.141+0000 7fcd4cf2c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-07-04T17:25:36.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:36.144+0000 7fcd4cf2c640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-07-04T17:25:39.119 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-07-04T17:25:39.207 DEBUG:teuthology.orchestra.run.smithi042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-07-04T17:25:39.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:39.421+0000 7f6290944640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-07-04T17:25:39.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:39.421+0000 7f6290944640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-07-04T17:25:39.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:39.421+0000 7f6290944640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-07-04T17:25:39.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:39.421+0000 7f6290944640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-07-04T17:25:39.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:39.421+0000 7f6290944640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-07-04T17:25:39.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:39.424+0000 7f6290944640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-07-04T17:25:42.648 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-07-04T17:25:42.743 DEBUG:teuthology.orchestra.run.smithi042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-07-04T17:25:42.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:42.956+0000 7f1453a18640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-07-04T17:25:42.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:42.957+0000 7f1453a18640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-07-04T17:25:42.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:42.957+0000 7f1453a18640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-07-04T17:25:42.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:42.957+0000 7f1453a18640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-07-04T17:25:42.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:42.957+0000 7f1453a18640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-07-04T17:25:42.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:42.959+0000 7f1453a18640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-07-04T17:25:46.203 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-07-04T17:25:46.297 INFO:tasks.ceph:Reading keys from all nodes... 2024-07-04T17:25:46.297 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:25:46.298 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-07-04T17:25:46.388 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:25:46.388 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-07-04T17:25:46.475 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:25:46.475 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-07-04T17:25:46.551 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:25:46.551 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-07-04T17:25:46.629 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:25:46.629 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-07-04T17:25:46.713 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:25:46.713 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2024-07-04T17:25:46.802 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:25:46.802 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-07-04T17:25:46.889 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:25:46.889 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-07-04T17:25:46.978 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:25:46.978 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-07-04T17:25:47.074 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:25:47.075 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-07-04T17:25:47.127 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:25:47.128 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-07-04T17:25:47.181 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:25:47.181 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/etc/ceph/ceph.client.mirror.keyring of=/dev/stdout 2024-07-04T17:25:47.232 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:25:47.232 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/etc/ceph/ceph.client.mirror_remote.keyring of=/dev/stdout 2024-07-04T17:25:47.287 INFO:tasks.ceph:Adding keys to all mons... 2024-07-04T17:25:47.288 DEBUG:teuthology.orchestra.run.smithi042:> sudo tee -a /etc/ceph/ceph.keyring 2024-07-04T17:25:47.372 INFO:teuthology.orchestra.run.smithi042.stdout:[mgr.x] 2024-07-04T17:25:47.373 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAM24Zm47tDCBAAjszudEEB4QHLlWqPXyZc2g== 2024-07-04T17:25:47.373 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.a] 2024-07-04T17:25:47.373 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAM24Zm2VMEFxAAQqdcGux6Ker7jEr1dGAfzA== 2024-07-04T17:25:47.373 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.b] 2024-07-04T17:25:47.373 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAM24ZmS/S7KxAAfjTZrLYR9qLxuTvd2zsCcQ== 2024-07-04T17:25:47.373 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.c] 2024-07-04T17:25:47.373 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAN24Zm2JrRBBAAFMYIM2fLsr9EOfcyh3tZpA== 2024-07-04T17:25:47.373 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.d] 2024-07-04T17:25:47.373 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAN24ZmPCGLFxAAhsh3XaAHCyauCGxh/3DFsw== 2024-07-04T17:25:47.373 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.e] 2024-07-04T17:25:47.373 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAN24ZmVIaSKhAAhoBdDalwQ4eoo0rqb6p0Vw== 2024-07-04T17:25:47.373 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.0] 2024-07-04T17:25:47.374 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAQ24Zm4spcCBAA4kTvvR7IOVa01K5ZCrJ35w== 2024-07-04T17:25:47.374 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.1] 2024-07-04T17:25:47.374 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAT24Zmz1sfGRAAgfH5eIiQrufnPq6F94rHWg== 2024-07-04T17:25:47.374 INFO:teuthology.orchestra.run.smithi042.stdout:[osd.2] 2024-07-04T17:25:47.374 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAW24ZmJdQHORAAEjAM9OBqSIRhX148GC6pcw== 2024-07-04T17:25:47.374 INFO:teuthology.orchestra.run.smithi042.stdout:[client.0] 2024-07-04T17:25:47.374 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAO24ZmUMmYABAAh4pNlkb9JsHQtK8uCJFQCA== 2024-07-04T17:25:47.374 INFO:teuthology.orchestra.run.smithi042.stdout:[client.1] 2024-07-04T17:25:47.374 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAO24Zm0pAJEBAAntDeQ/oWDlJ+3xC7q5EQnQ== 2024-07-04T17:25:47.374 INFO:teuthology.orchestra.run.smithi042.stdout:[client.mirror] 2024-07-04T17:25:47.374 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAO24ZmBJzBHhAAR4Zg36YTtVizxm/g/Cdk0w== 2024-07-04T17:25:47.375 INFO:teuthology.orchestra.run.smithi042.stdout:[client.mirror_remote] 2024-07-04T17:25:47.375 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAO24Zm+BsqLhAA19GVuEjfWQeMe/fSc+dmoA== 2024-07-04T17:25:47.377 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-07-04T17:25:47.641 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-07-04T17:25:47.861 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-07-04T17:25:48.085 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-07-04T17:25:48.308 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-07-04T17:25:48.532 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-07-04T17:25:48.744 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-07-04T17:25:48.963 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-07-04T17:25:49.176 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-07-04T17:25:49.396 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-07-04T17:25:49.622 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-07-04T17:25:49.844 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.mirror --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-07-04T17:25:50.060 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.mirror_remote --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-07-04T17:25:50.279 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-07-04T17:25:50.280 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-07-04T17:25:50.365 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-07-04T17:25:50.600 DEBUG:teuthology.orchestra.run.smithi042:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-07-04T17:25:50.679 DEBUG:teuthology.orchestra.run.smithi042:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-07-04T17:25:50.731 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-07-04T17:25:50.731 INFO:tasks.ceph.mon.a:Restarting daemon 2024-07-04T17:25:50.732 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-07-04T17:25:50.736 INFO:tasks.ceph.mon.a:Started 2024-07-04T17:25:50.736 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-07-04T17:25:50.736 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-07-04T17:25:50.736 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-07-04T17:25:50.740 INFO:tasks.ceph.mgr.x:Started 2024-07-04T17:25:50.741 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-07-04T17:25:50.741 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-07-04T17:25:50.745 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-07-04T17:25:50.745 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph config set client debug_client 20 2024-07-04T17:25:50.790 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-07-04T17:25:50.790 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph config set client debug_ms 1 2024-07-04T17:25:50.836 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-07-04T17:25:50.836 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-07-04T17:25:50.883 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-07-04T17:25:50.884 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-07-04T17:25:50.930 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-07-04T17:25:50.931 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-07-04T17:25:50.980 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-07-04T17:25:50.980 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-07-04T17:25:51.028 INFO:tasks.ceph:setting config [mds] debug_ms = 1 2024-07-04T17:25:51.028 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph config set mds debug_ms 1 2024-07-04T17:25:51.079 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-07-04T17:25:51.079 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-07-04T17:25:51.130 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-07-04T17:25:51.130 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-07-04T17:25:51.182 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-07-04T17:25:51.182 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-07-04T17:25:51.235 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-07-04T17:25:51.235 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-07-04T17:25:51.294 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-07-04T17:25:51.294 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-07-04T17:25:51.345 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-07-04T17:25:51.345 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-07-04T17:25:51.401 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-07-04T17:25:51.401 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-07-04T17:25:51.451 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-07-04T17:25:51.451 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-07-04T17:25:51.509 INFO:tasks.ceph:setting config [mon] mon_warn_on_pool_no_app = False 2024-07-04T17:25:51.509 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph config set mon mon_warn_on_pool_no_app False 2024-07-04T17:25:51.573 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-07-04T17:25:51.573 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-07-04T17:25:51.629 DEBUG:tasks.ceph:set 18 configs 2024-07-04T17:25:51.630 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi042') 2024-07-04T17:25:51.630 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi042') 2024-07-04T17:25:51.650 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi042') 2024-07-04T17:25:51.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.797+0000 7f7a85600640 1 -- 172.21.15.42:0/108368851 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a7802cd00 msgr2=0x7f7a7802e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:51.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.797+0000 7f7a85600640 1 --2- 172.21.15.42:0/108368851 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a7802cd00 0x7f7a7802e0e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7a70009860 tx=0x7f7a7002a5f0 comp rx=0 tx=0).stop 2024-07-04T17:25:51.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.801+0000 7f7a85600640 1 -- 172.21.15.42:0/108368851 shutdown_connections 2024-07-04T17:25:51.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.801+0000 7f7a85600640 1 --2- 172.21.15.42:0/108368851 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a7802cd00 0x7f7a7802e0e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:51.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.801+0000 7f7a85600640 1 -- 172.21.15.42:0/108368851 >> 172.21.15.42:0/108368851 conn(0x7f7a7801b310 msgr2=0x7f7a78005380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:51.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.802+0000 7f7a85600640 1 -- 172.21.15.42:0/108368851 shutdown_connections 2024-07-04T17:25:51.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.802+0000 7f7a85600640 1 -- 172.21.15.42:0/108368851 wait complete. 2024-07-04T17:25:51.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.807+0000 7f7a85600640 1 Processor -- start 2024-07-04T17:25:51.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.807+0000 7f7a85600640 1 -- start start 2024-07-04T17:25:51.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.804+0000 7f2fed800640 1 -- 172.21.15.42:0/2888786281 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f2fbc005310 con 0x7f2fe80ab270 2024-07-04T17:25:51.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.810+0000 7f7a85600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a7802cd00 0x7f7a780c6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:25:51.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.811+0000 7f7a7ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a7802cd00 0x7f7a780c6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:25:51.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.811+0000 7f7a7ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a7802cd00 0x7f7a780c6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60872/0 (socket says 172.21.15.42:60872) 2024-07-04T17:25:51.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.811+0000 7f7a7ea00640 1 -- 172.21.15.42:0/1713120588 learned_addr learned my addr 172.21.15.42:0/1713120588 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:25:51.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.811+0000 7f7a85600640 1 -- 172.21.15.42:0/1713120588 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7a780370e0 con 0x7f7a7802cd00 2024-07-04T17:25:51.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.812+0000 7f2fdfe00640 1 -- 172.21.15.42:0/2888786281 <== mon.0 v2:172.21.15.42:3300/0 10 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v6)=0 v6) ==== 123+0+0 (secure 0 0 0) 0x7f2fe0032030 con 0x7f2fe80ab270 2024-07-04T17:25:51.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.815+0000 7f7a7ea00640 1 -- 172.21.15.42:0/1713120588 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a78030de0 con 0x7f7a7802cd00 2024-07-04T17:25:51.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.817+0000 7f647b400640 1 -- 172.21.15.42:0/4000538017 <== mon.0 v2:172.21.15.42:3300/0 8 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64740107f0 con 0x7f64840702f0 2024-07-04T17:25:51.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.814+0000 7f2fdfe00640 1 -- 172.21.15.42:0/2888786281 <== mon.0 v2:172.21.15.42:3300/0 11 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fe0058020 con 0x7f2fe80ab270 2024-07-04T17:25:51.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.817+0000 7f7a7ea00640 1 --2- 172.21.15.42:0/1713120588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a7802cd00 0x7f7a780c6800 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7a70009980 tx=0x7f7a700055e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:25:51.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.818+0000 7f7346000640 1 -- 172.21.15.42:0/1647425414 <== mon.0 v2:172.21.15.42:3300/0 9 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7340024780 con 0x7f73500a94d0 2024-07-04T17:25:51.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.818+0000 7f7a76a00640 1 -- 172.21.15.42:0/1713120588 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a70004020 con 0x7f7a7802cd00 2024-07-04T17:25:51.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.819+0000 7f34afe00640 1 -- 172.21.15.42:0/1219269425 <== mon.0 v2:172.21.15.42:3300/0 10 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34bc0c3460 con 0x7f34b4030140 2024-07-04T17:25:51.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.819+0000 7f7a76a00640 1 -- 172.21.15.42:0/1713120588 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a70004200 con 0x7f7a7802cd00 2024-07-04T17:25:51.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.821+0000 7f7a76a00640 1 -- 172.21.15.42:0/1713120588 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a700049b0 con 0x7f7a7802cd00 2024-07-04T17:25:51.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.821+0000 7f7a85600640 1 -- 172.21.15.42:0/1713120588 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a7816cc60 con 0x7f7a7802cd00 2024-07-04T17:25:51.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.822+0000 7f7a85600640 1 -- 172.21.15.42:0/1713120588 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a7816c9d0 con 0x7f7a7802cd00 2024-07-04T17:25:51.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.822+0000 7f7a76a00640 1 -- 172.21.15.42:0/1713120588 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7a700035f0 con 0x7f7a7802cd00 2024-07-04T17:25:51.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.824+0000 7f7a76a00640 1 -- 172.21.15.42:0/1713120588 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7a70033070 con 0x7f7a7802cd00 2024-07-04T17:25:51.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.828+0000 7f7a97e00640 1 -- 172.21.15.42:0/3561101475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a900c28b0 msgr2=0x7f7a900c2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:51.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.828+0000 7f7a97e00640 1 --2- 172.21.15.42:0/3561101475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a900c28b0 0x7f7a900c2cc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7a84005fd0 tx=0x7f7a8402a560 comp rx=0 tx=0).stop 2024-07-04T17:25:51.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.829+0000 7f7a97e00640 1 -- 172.21.15.42:0/3561101475 shutdown_connections 2024-07-04T17:25:51.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.829+0000 7f7a97e00640 1 --2- 172.21.15.42:0/3561101475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a900c28b0 0x7f7a900c2cc0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:51.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.829+0000 7f7a97e00640 1 -- 172.21.15.42:0/3561101475 >> 172.21.15.42:0/3561101475 conn(0x7f7a9001b5f0 msgr2=0x7f7a9001bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:51.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.830+0000 7f7a97e00640 1 -- 172.21.15.42:0/3561101475 shutdown_connections 2024-07-04T17:25:51.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.832+0000 7f7a85600640 1 -- 172.21.15.42:0/1713120588 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a7802e0e0 con 0x7f7a7802cd00 2024-07-04T17:25:51.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.834+0000 7f2fed800640 1 -- 172.21.15.42:0/2888786281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fe80ab270 msgr2=0x7f2fe81d6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:51.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.836+0000 7f2fed800640 1 --2- 172.21.15.42:0/2888786281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fe80ab270 0x7f2fe81d6e20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0005e30 tx=0x7f2fe0004a40 comp rx=0 tx=0).stop 2024-07-04T17:25:51.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.830+0000 7f7a97e00640 1 -- 172.21.15.42:0/3561101475 wait complete. 2024-07-04T17:25:51.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.836+0000 7f2fed800640 1 -- 172.21.15.42:0/2888786281 shutdown_connections 2024-07-04T17:25:51.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.836+0000 7f2fed800640 1 --2- 172.21.15.42:0/2888786281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fe80ab270 0x7f2fe81d6e20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:51.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.836+0000 7f2fed800640 1 -- 172.21.15.42:0/2888786281 >> 172.21.15.42:0/2888786281 conn(0x7f2fe806fb80 msgr2=0x7f2fe81411a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:51.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.836+0000 7f2fed800640 1 -- 172.21.15.42:0/2888786281 shutdown_connections 2024-07-04T17:25:51.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.837+0000 7f7a76a00640 1 -- 172.21.15.42:0/1713120588 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73745 (secure 0 0 0) 0x7f7a7003f030 con 0x7f7a7802cd00 2024-07-04T17:25:51.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.837+0000 7f2fed800640 1 -- 172.21.15.42:0/2888786281 wait complete. 2024-07-04T17:25:51.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.843+0000 7f7a97e00640 1 Processor -- start 2024-07-04T17:25:51.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.844+0000 7f7a97e00640 1 -- start start 2024-07-04T17:25:51.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.846+0000 7f7a97e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a9002ccf0 0x7f7a900b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:25:51.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.849+0000 7f7a97e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7a90005520 con 0x7f7a9002ccf0 2024-07-04T17:25:51.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.850+0000 7f7a96a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a9002ccf0 0x7f7a900b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:25:51.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.850+0000 7f7a96a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a9002ccf0 0x7f7a900b7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60892/0 (socket says 172.21.15.42:60892) 2024-07-04T17:25:51.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.850+0000 7f7a96a00640 1 -- 172.21.15.42:0/209048539 learned_addr learned my addr 172.21.15.42:0/209048539 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:25:51.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.852+0000 7f7a96a00640 1 -- 172.21.15.42:0/209048539 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a9015a390 con 0x7f7a9002ccf0 2024-07-04T17:25:51.853 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi042') 2024-07-04T17:25:51.853 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi042') 2024-07-04T17:25:51.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.853+0000 7f7a96a00640 1 --2- 172.21.15.42:0/209048539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a9002ccf0 0x7f7a900b7e30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7a98065c30 tx=0x7f7a98075ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:25:51.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.854+0000 7f7a8ea00640 1 -- 172.21.15.42:0/209048539 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a98090070 con 0x7f7a9002ccf0 2024-07-04T17:25:51.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.854+0000 7f7a8ea00640 1 -- 172.21.15.42:0/209048539 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a9806a550 con 0x7f7a9002ccf0 2024-07-04T17:25:51.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.855+0000 7f7a8ea00640 1 -- 172.21.15.42:0/209048539 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a980744d0 con 0x7f7a9002ccf0 2024-07-04T17:25:51.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.856+0000 7f7a97e00640 1 -- 172.21.15.42:0/209048539 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a9015a080 con 0x7f7a9002ccf0 2024-07-04T17:25:51.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.856+0000 7f7a97e00640 1 -- 172.21.15.42:0/209048539 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a90033b80 con 0x7f7a9002ccf0 2024-07-04T17:25:51.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.860+0000 7f7a83e00640 1 -- 172.21.15.42:0/209048539 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a6c005290 con 0x7f7a9002ccf0 2024-07-04T17:25:51.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.861+0000 7f7a8ea00640 1 -- 172.21.15.42:0/209048539 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7a98074050 con 0x7f7a9002ccf0 2024-07-04T17:25:51.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.864+0000 7f7a8ea00640 1 -- 172.21.15.42:0/209048539 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7a98075720 con 0x7f7a9002ccf0 2024-07-04T17:25:51.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.870+0000 7f7a8ea00640 1 -- 172.21.15.42:0/209048539 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73745 (secure 0 0 0) 0x7f7a9806f070 con 0x7f7a9002ccf0 2024-07-04T17:25:51.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.888+0000 7f34c1c00640 1 -- 172.21.15.42:0/1219269425 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f34900052f0 con 0x7f34b4030140 2024-07-04T17:25:51.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.902+0000 7f34afe00640 1 -- 172.21.15.42:0/1219269425 <== mon.0 v2:172.21.15.42:3300/0 11 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v7)=0 v7) ==== 108+0+0 (secure 0 0 0) 0x7f34b403d4d0 con 0x7f34b4030140 2024-07-04T17:25:51.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.907+0000 7f34c1c00640 1 -- 172.21.15.42:0/1219269425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34b4030140 msgr2=0x7f34b40ca7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:51.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.907+0000 7f34c1c00640 1 --2- 172.21.15.42:0/1219269425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34b4030140 0x7f34b40ca7a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f34bc07a670 tx=0x7f34bc076ad0 comp rx=0 tx=0).stop 2024-07-04T17:25:51.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.908+0000 7f34c1c00640 1 -- 172.21.15.42:0/1219269425 shutdown_connections 2024-07-04T17:25:51.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.908+0000 7f34c1c00640 1 --2- 172.21.15.42:0/1219269425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34b4030140 0x7f34b40ca7a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:51.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.908+0000 7f34c1c00640 1 -- 172.21.15.42:0/1219269425 >> 172.21.15.42:0/1219269425 conn(0x7f34b401f830 msgr2=0x7f34b40c5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:51.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.908+0000 7f34c1c00640 1 -- 172.21.15.42:0/1219269425 shutdown_connections 2024-07-04T17:25:51.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.908+0000 7f34c1c00640 1 -- 172.21.15.42:0/1219269425 wait complete. 2024-07-04T17:25:51.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.914+0000 7fa3b7400640 1 -- 172.21.15.42:0/3580211115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3a8030130 msgr2=0x7fa3a8030f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:51.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.914+0000 7fa3b7400640 1 --2- 172.21.15.42:0/3580211115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3a8030130 0x7fa3a8030f80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4005fd0 tx=0x7fa3a402a4d0 comp rx=0 tx=0).stop 2024-07-04T17:25:51.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.914+0000 7fa3b7400640 1 -- 172.21.15.42:0/3580211115 shutdown_connections 2024-07-04T17:25:51.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.914+0000 7fa3b7400640 1 --2- 172.21.15.42:0/3580211115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3a8030130 0x7fa3a8030f80 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:51.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.914+0000 7fa3b7400640 1 -- 172.21.15.42:0/3580211115 >> 172.21.15.42:0/3580211115 conn(0x7fa3a801f820 msgr2=0x7fa3a8020190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:51.916 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi042') 2024-07-04T17:25:51.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.918+0000 7fa3b7400640 1 -- 172.21.15.42:0/3580211115 shutdown_connections 2024-07-04T17:25:51.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.918+0000 7fa3b7400640 1 -- 172.21.15.42:0/3580211115 wait complete. 2024-07-04T17:25:51.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.944+0000 7fa3b7400640 1 Processor -- start 2024-07-04T17:25:51.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.945+0000 7fa3b7400640 1 -- start start 2024-07-04T17:25:51.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.968+0000 7f7354c00640 1 -- 172.21.15.42:0/1647425414 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f73240052f0 con 0x7f73500a94d0 2024-07-04T17:25:51.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.969+0000 7fa3b7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3a8030130 0x7fa3a815efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:25:51.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.969+0000 7fa3b7400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a80cb220 con 0x7fa3a8030130 2024-07-04T17:25:51.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.970+0000 7fa3b4a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3a8030130 0x7fa3a815efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:25:51.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.970+0000 7fa3b4a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3a8030130 0x7fa3a815efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60908/0 (socket says 172.21.15.42:60908) 2024-07-04T17:25:51.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.972+0000 7fa3b4a00640 1 -- 172.21.15.42:0/2117695909 learned_addr learned my addr 172.21.15.42:0/2117695909 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:25:51.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.975+0000 7f7346000640 1 -- 172.21.15.42:0/1647425414 <== mon.0 v2:172.21.15.42:3300/0 10 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v8)=0 v8) ==== 117+0+0 (secure 0 0 0) 0x7f734000f3e0 con 0x7f73500a94d0 2024-07-04T17:25:51.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.977+0000 7fa3b4a00640 1 -- 172.21.15.42:0/2117695909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a8160060 con 0x7fa3a8030130 2024-07-04T17:25:51.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.977+0000 7fa3b4a00640 1 --2- 172.21.15.42:0/2117695909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3a8030130 0x7fa3a815efc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa3a000b360 tx=0x7fa3a000b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:25:51.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.978+0000 7fa3acc00640 1 -- 172.21.15.42:0/2117695909 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3a0015020 con 0x7fa3a8030130 2024-07-04T17:25:51.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.978+0000 7fa3acc00640 1 -- 172.21.15.42:0/2117695909 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3a0011850 con 0x7fa3a8030130 2024-07-04T17:25:51.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.979+0000 7fa3b7400640 1 -- 172.21.15.42:0/2117695909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a815f630 con 0x7fa3a8030130 2024-07-04T17:25:51.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.979+0000 7fa3b7400640 1 -- 172.21.15.42:0/2117695909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a80090f0 con 0x7fa3a8030130 2024-07-04T17:25:51.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.982+0000 7f7354c00640 1 -- 172.21.15.42:0/1647425414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73500a94d0 msgr2=0x7f735014ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:51.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.982+0000 7f7354c00640 1 --2- 172.21.15.42:0/1647425414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73500a94d0 0x7f735014ee80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f734000cfd0 tx=0x7f7340001570 comp rx=0 tx=0).stop 2024-07-04T17:25:51.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.980+0000 7fa3acc00640 1 -- 172.21.15.42:0/2117695909 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3a0011bd0 con 0x7fa3a8030130 2024-07-04T17:25:51.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.983+0000 7f7354c00640 1 -- 172.21.15.42:0/1647425414 shutdown_connections 2024-07-04T17:25:51.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.983+0000 7f7354c00640 1 --2- 172.21.15.42:0/1647425414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73500a94d0 0x7f735014ee80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:51.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.983+0000 7f7354c00640 1 -- 172.21.15.42:0/1647425414 >> 172.21.15.42:0/1647425414 conn(0x7f735006fb80 msgr2=0x7f7350075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:51.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.984+0000 7f7354c00640 1 -- 172.21.15.42:0/1647425414 shutdown_connections 2024-07-04T17:25:51.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.984+0000 7fa39e000640 1 -- 172.21.15.42:0/2117695909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa38c005290 con 0x7fa3a8030130 2024-07-04T17:25:51.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.985+0000 7fa3acc00640 1 -- 172.21.15.42:0/2117695909 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa3a0018020 con 0x7fa3a8030130 2024-07-04T17:25:51.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.985+0000 7fa3acc00640 1 -- 172.21.15.42:0/2117695909 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa3a0019900 con 0x7fa3a8030130 2024-07-04T17:25:51.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.987+0000 7f7354c00640 1 -- 172.21.15.42:0/1647425414 wait complete. 2024-07-04T17:25:51.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:51.990+0000 7fa3acc00640 1 -- 172.21.15.42:0/2117695909 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73745 (secure 0 0 0) 0x7fa3a001f020 con 0x7fa3a8030130 2024-07-04T17:25:52.007 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi042') 2024-07-04T17:25:52.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.015+0000 7efd54c00640 1 -- 172.21.15.42:0/2946873677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd4802ccd0 msgr2=0x7efd4802e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:52.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.019+0000 7efd54c00640 1 --2- 172.21.15.42:0/2946873677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd4802ccd0 0x7efd4802e0b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efd44005fd0 tx=0x7efd4402a5f0 comp rx=0 tx=0).stop 2024-07-04T17:25:52.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.021+0000 7f7a83e00640 1 -- 172.21.15.42:0/209048539 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f7a6c005580 con 0x7f7a9002ccf0 2024-07-04T17:25:52.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.030+0000 7f7a8ea00640 1 -- 172.21.15.42:0/209048539 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v9)=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f7a98075920 con 0x7f7a9002ccf0 2024-07-04T17:25:52.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.030+0000 7efd54c00640 1 -- 172.21.15.42:0/2946873677 shutdown_connections 2024-07-04T17:25:52.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.030+0000 7efd54c00640 1 --2- 172.21.15.42:0/2946873677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd4802ccd0 0x7efd4802e0b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:52.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.030+0000 7efd54c00640 1 -- 172.21.15.42:0/2946873677 >> 172.21.15.42:0/2946873677 conn(0x7efd4801b2e0 msgr2=0x7efd48005350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:52.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.030+0000 7efd54c00640 1 -- 172.21.15.42:0/2946873677 shutdown_connections 2024-07-04T17:25:52.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.031+0000 7efd54c00640 1 -- 172.21.15.42:0/2946873677 wait complete. 2024-07-04T17:25:52.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.042+0000 7f7a83e00640 1 -- 172.21.15.42:0/209048539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a9002ccf0 msgr2=0x7f7a900b7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:52.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.042+0000 7f7a83e00640 1 --2- 172.21.15.42:0/209048539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a9002ccf0 0x7f7a900b7e30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7a98065c30 tx=0x7f7a98075ff0 comp rx=0 tx=0).stop 2024-07-04T17:25:52.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.042+0000 7f7a83e00640 1 -- 172.21.15.42:0/209048539 shutdown_connections 2024-07-04T17:25:52.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.042+0000 7f7a83e00640 1 --2- 172.21.15.42:0/209048539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a9002ccf0 0x7f7a900b7e30 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:52.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.042+0000 7f7a83e00640 1 -- 172.21.15.42:0/209048539 >> 172.21.15.42:0/209048539 conn(0x7f7a9001b5f0 msgr2=0x7f7a9002c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:52.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.042+0000 7f7a83e00640 1 -- 172.21.15.42:0/209048539 shutdown_connections 2024-07-04T17:25:52.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.042+0000 7f7a83e00640 1 -- 172.21.15.42:0/209048539 wait complete. 2024-07-04T17:25:52.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.045+0000 7efd54c00640 1 Processor -- start 2024-07-04T17:25:52.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.049+0000 7efd54c00640 1 -- start start 2024-07-04T17:25:52.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.051+0000 7efd54c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd4802ccd0 0x7efd4802f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:25:52.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.052+0000 7efd54c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efd480370b0 con 0x7efd4802ccd0 2024-07-04T17:25:52.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.056+0000 7efd4f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd4802ccd0 0x7efd4802f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:25:52.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.056+0000 7efd4f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd4802ccd0 0x7efd4802f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60926/0 (socket says 172.21.15.42:60926) 2024-07-04T17:25:52.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.056+0000 7efd4f400640 1 -- 172.21.15.42:0/3372104898 learned_addr learned my addr 172.21.15.42:0/3372104898 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:25:52.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.062+0000 7efd4f400640 1 -- 172.21.15.42:0/3372104898 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd480c6180 con 0x7efd4802ccd0 2024-07-04T17:25:52.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.063+0000 7efd4f400640 1 --2- 172.21.15.42:0/3372104898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd4802ccd0 0x7efd4802f770 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efd44034040 tx=0x7efd440049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:25:52.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.064+0000 7efd43400640 1 -- 172.21.15.42:0/3372104898 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd4403e030 con 0x7efd4802ccd0 2024-07-04T17:25:52.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.068+0000 7efd43400640 1 -- 172.21.15.42:0/3372104898 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd440035f0 con 0x7efd4802ccd0 2024-07-04T17:25:52.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.068+0000 7efd54c00640 1 -- 172.21.15.42:0/3372104898 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd48033ff0 con 0x7efd4802ccd0 2024-07-04T17:25:52.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.069+0000 7f4c55e00640 1 -- 172.21.15.42:0/2074539775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c500bbc40 msgr2=0x7f4c500b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:52.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.068+0000 7efd43400640 1 -- 172.21.15.42:0/3372104898 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd44035040 con 0x7efd4802ccd0 2024-07-04T17:25:52.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.069+0000 7efd43400640 1 -- 172.21.15.42:0/3372104898 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efd44041030 con 0x7efd4802ccd0 2024-07-04T17:25:52.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.069+0000 7f4c55e00640 1 --2- 172.21.15.42:0/2074539775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c500bbc40 0x7f4c500b1a30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4c38005fd0 tx=0x7f4c3802a610 comp rx=0 tx=0).stop 2024-07-04T17:25:52.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.072+0000 7f4c55e00640 1 -- 172.21.15.42:0/2074539775 shutdown_connections 2024-07-04T17:25:52.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.072+0000 7f4c55e00640 1 --2- 172.21.15.42:0/2074539775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c500bbc40 0x7f4c500b1a30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:52.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.072+0000 7efd54c00640 1 -- 172.21.15.42:0/3372104898 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd4802f1a0 con 0x7efd4802ccd0 2024-07-04T17:25:52.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.072+0000 7f4c55e00640 1 -- 172.21.15.42:0/2074539775 >> 172.21.15.42:0/2074539775 conn(0x7f4c500beea0 msgr2=0x7f4c500bf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:52.076 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '1'], check_status=True, hostname='smithi042') 2024-07-04T17:25:52.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.072+0000 7f4c55e00640 1 -- 172.21.15.42:0/2074539775 shutdown_connections 2024-07-04T17:25:52.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.072+0000 7f4c55e00640 1 -- 172.21.15.42:0/2074539775 wait complete. 2024-07-04T17:25:52.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.076+0000 7f7a85600640 1 -- 172.21.15.42:0/1713120588 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f7a78008160 con 0x7f7a7802cd00 2024-07-04T17:25:52.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.080+0000 7f6478c00640 1 -- 172.21.15.42:0/4000538017 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f6484149000 con 0x7f64840702f0 2024-07-04T17:25:52.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.079+0000 7f4c55e00640 1 Processor -- start 2024-07-04T17:25:52.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.079+0000 7f4c55e00640 1 -- start start 2024-07-04T17:25:52.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.082+0000 7efd54c00640 1 -- 172.21.15.42:0/3372104898 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd4802e0b0 con 0x7efd4802ccd0 2024-07-04T17:25:52.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.082+0000 7efd43400640 1 -- 172.21.15.42:0/3372104898 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efd4403f070 con 0x7efd4802ccd0 2024-07-04T17:25:52.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.090+0000 7f4c55e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c500b0a50 0x7f4c50073130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:25:52.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.090+0000 7f4c55e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4c50146190 con 0x7f4c500b0a50 2024-07-04T17:25:52.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.090+0000 7f4c4ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c500b0a50 0x7f4c50073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:25:52.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.090+0000 7f4c4ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c500b0a50 0x7f4c50073130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60942/0 (socket says 172.21.15.42:60942) 2024-07-04T17:25:52.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.091+0000 7f4c4ea00640 1 -- 172.21.15.42:0/212692055 learned_addr learned my addr 172.21.15.42:0/212692055 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:25:52.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.095+0000 7f4c4ea00640 1 -- 172.21.15.42:0/212692055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c50071a20 con 0x7f4c500b0a50 2024-07-04T17:25:52.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.098+0000 7f4c4ea00640 1 --2- 172.21.15.42:0/212692055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c500b0a50 0x7f4c50073130 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009850 tx=0x7f4c38004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:25:52.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.102+0000 7f7a76a00640 1 -- 172.21.15.42:0/1713120588 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v10)=0 v10) ==== 115+0+0 (secure 0 0 0) 0x7f7a700316d0 con 0x7f7a7802cd00 2024-07-04T17:25:52.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.106+0000 7f7a85600640 1 -- 172.21.15.42:0/1713120588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a7802cd00 msgr2=0x7f7a780c6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:52.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.106+0000 7f7a85600640 1 --2- 172.21.15.42:0/1713120588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a7802cd00 0x7f7a780c6800 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7a70009980 tx=0x7f7a700055e0 comp rx=0 tx=0).stop 2024-07-04T17:25:52.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.106+0000 7f7a85600640 1 -- 172.21.15.42:0/1713120588 shutdown_connections 2024-07-04T17:25:52.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.106+0000 7f7a85600640 1 --2- 172.21.15.42:0/1713120588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a7802cd00 0x7f7a780c6800 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:52.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.106+0000 7f7a85600640 1 -- 172.21.15.42:0/1713120588 >> 172.21.15.42:0/1713120588 conn(0x7f7a7801b310 msgr2=0x7f7a78034d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:52.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.103+0000 7efd43400640 1 -- 172.21.15.42:0/3372104898 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73745 (secure 0 0 0) 0x7efd44043750 con 0x7efd4802ccd0 2024-07-04T17:25:52.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.107+0000 7f7a85600640 1 -- 172.21.15.42:0/1713120588 shutdown_connections 2024-07-04T17:25:52.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.107+0000 7f7a85600640 1 -- 172.21.15.42:0/1713120588 wait complete. 2024-07-04T17:25:52.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.109+0000 7f4c47400640 1 -- 172.21.15.42:0/212692055 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c38041070 con 0x7f4c500b0a50 2024-07-04T17:25:52.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.110+0000 7f4c55e00640 1 -- 172.21.15.42:0/212692055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c50073920 con 0x7f4c500b0a50 2024-07-04T17:25:52.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.110+0000 7f4c55e00640 1 -- 172.21.15.42:0/212692055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c5014dfd0 con 0x7f4c500b0a50 2024-07-04T17:25:52.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.111+0000 7f4c47400640 1 -- 172.21.15.42:0/212692055 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c380043f0 con 0x7f4c500b0a50 2024-07-04T17:25:52.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.112+0000 7f4c55e00640 1 -- 172.21.15.42:0/212692055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c500b1a30 con 0x7f4c500b0a50 2024-07-04T17:25:52.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.117+0000 7f4c47400640 1 -- 172.21.15.42:0/212692055 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c3804a060 con 0x7f4c500b0a50 2024-07-04T17:25:52.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.118+0000 7f647b400640 1 -- 172.21.15.42:0/4000538017 <== mon.0 v2:172.21.15.42:3300/0 9 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v11)=0 v11) ==== 106+0+0 (secure 0 0 0) 0x7f64840b0890 con 0x7f64840702f0 2024-07-04T17:25:52.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.118+0000 7f4c47400640 1 -- 172.21.15.42:0/212692055 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4c38005280 con 0x7f4c500b0a50 2024-07-04T17:25:52.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.120+0000 7f4c47400640 1 -- 172.21.15.42:0/212692055 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4c38031a00 con 0x7f4c500b0a50 2024-07-04T17:25:52.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.122+0000 7f4c47400640 1 -- 172.21.15.42:0/212692055 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73745 (secure 0 0 0) 0x7f4c38005570 con 0x7f4c500b0a50 2024-07-04T17:25:52.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.127+0000 7f6478c00640 1 -- 172.21.15.42:0/4000538017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64840702f0 msgr2=0x7f64840731b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:52.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.127+0000 7f6478c00640 1 --2- 172.21.15.42:0/4000538017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64840702f0 0x7f64840731b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6474001e60 tx=0x7f647400c6f0 comp rx=0 tx=0).stop 2024-07-04T17:25:52.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.127+0000 7f6478c00640 1 -- 172.21.15.42:0/4000538017 shutdown_connections 2024-07-04T17:25:52.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.127+0000 7f6478c00640 1 --2- 172.21.15.42:0/4000538017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64840702f0 0x7f64840731b0 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6474001e60 tx=0x7f647400c6f0 comp rx=0 tx=0).stop 2024-07-04T17:25:52.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.127+0000 7f6478c00640 1 -- 172.21.15.42:0/4000538017 >> 172.21.15.42:0/4000538017 conn(0x7f648406fb80 msgr2=0x7f6484075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:52.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.128+0000 7f6478c00640 1 -- 172.21.15.42:0/4000538017 shutdown_connections 2024-07-04T17:25:52.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.128+0000 7f6478c00640 1 -- 172.21.15.42:0/4000538017 wait complete. 2024-07-04T17:25:52.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.149+0000 7fa39e000640 1 -- 172.21.15.42:0/2117695909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7fa38c005580 con 0x7fa3a8030130 2024-07-04T17:25:52.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.154+0000 7fa3acc00640 1 -- 172.21.15.42:0/2117695909 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v12)=0 v12) ==== 121+0+0 (secure 0 0 0) 0x7fa3a001f020 con 0x7fa3a8030130 2024-07-04T17:25:52.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.156+0000 7fa39e000640 1 -- 172.21.15.42:0/2117695909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3a8030130 msgr2=0x7fa3a815efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:52.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.156+0000 7fa39e000640 1 --2- 172.21.15.42:0/2117695909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3a8030130 0x7fa3a815efc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa3a000b360 tx=0x7fa3a000b820 comp rx=0 tx=0).stop 2024-07-04T17:25:52.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.156+0000 7fa39e000640 1 -- 172.21.15.42:0/2117695909 shutdown_connections 2024-07-04T17:25:52.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.156+0000 7fa39e000640 1 --2- 172.21.15.42:0/2117695909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3a8030130 0x7fa3a815efc0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:52.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.156+0000 7fa39e000640 1 -- 172.21.15.42:0/2117695909 >> 172.21.15.42:0/2117695909 conn(0x7fa3a801f820 msgr2=0x7fa3a803a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:52.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.157+0000 7fa39e000640 1 -- 172.21.15.42:0/2117695909 shutdown_connections 2024-07-04T17:25:52.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.157+0000 7fa39e000640 1 -- 172.21.15.42:0/2117695909 wait complete. 2024-07-04T17:25:52.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.158+0000 7f18b5800640 1 -- 172.21.15.42:0/716677635 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18b00abbc0 msgr2=0x7f18b00b37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:52.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.158+0000 7f18b5800640 1 --2- 172.21.15.42:0/716677635 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18b00abbc0 0x7f18b00b37c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f189c005fd0 tx=0x7f189c02a560 comp rx=0 tx=0).stop 2024-07-04T17:25:52.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.158+0000 7f18b5800640 1 -- 172.21.15.42:0/716677635 shutdown_connections 2024-07-04T17:25:52.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.159+0000 7f18b5800640 1 --2- 172.21.15.42:0/716677635 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18b00abbc0 0x7f18b00b37c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:52.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.159+0000 7f18b5800640 1 -- 172.21.15.42:0/716677635 >> 172.21.15.42:0/716677635 conn(0x7f18b006c440 msgr2=0x7f18b006c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:52.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.159+0000 7f18b5800640 1 -- 172.21.15.42:0/716677635 shutdown_connections 2024-07-04T17:25:52.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.159+0000 7f18b5800640 1 -- 172.21.15.42:0/716677635 wait complete. 2024-07-04T17:25:52.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.163+0000 7f18b5800640 1 Processor -- start 2024-07-04T17:25:52.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.168+0000 7f18b5800640 1 -- start start 2024-07-04T17:25:52.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.171+0000 7f18b5800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18b00abbc0 0x7f18b00ae990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:25:52.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.172+0000 7f18aea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18b00abbc0 0x7f18b00ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:25:52.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.172+0000 7f18aea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18b00abbc0 0x7f18b00ae990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60960/0 (socket says 172.21.15.42:60960) 2024-07-04T17:25:52.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.172+0000 7f18aea00640 1 -- 172.21.15.42:0/3754008007 learned_addr learned my addr 172.21.15.42:0/3754008007 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:25:52.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.172+0000 7f18b5800640 1 -- 172.21.15.42:0/3754008007 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f18b00aa4d0 con 0x7f18b00abbc0 2024-07-04T17:25:52.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.173+0000 7f18aea00640 1 -- 172.21.15.42:0/3754008007 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b01e8be0 con 0x7f18b00abbc0 2024-07-04T17:25:52.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.174+0000 7f18aea00640 1 --2- 172.21.15.42:0/3754008007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18b00abbc0 0x7f18b00ae990 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f189c009850 tx=0x7f189c004d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:25:52.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.175+0000 7f18a6a00640 1 -- 172.21.15.42:0/3754008007 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f189c03e030 con 0x7f18b00abbc0 2024-07-04T17:25:52.176 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi042') 2024-07-04T17:25:52.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.176+0000 7f18b5800640 1 -- 172.21.15.42:0/3754008007 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b0143000 con 0x7f18b00abbc0 2024-07-04T17:25:52.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.176+0000 7f18b5800640 1 -- 172.21.15.42:0/3754008007 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b0142b00 con 0x7f18b00abbc0 2024-07-04T17:25:52.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.176+0000 7f18a6a00640 1 -- 172.21.15.42:0/3754008007 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f189c0035f0 con 0x7f18b00abbc0 2024-07-04T17:25:52.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.176+0000 7f18a6a00640 1 -- 172.21.15.42:0/3754008007 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f189c035040 con 0x7f18b00abbc0 2024-07-04T17:25:52.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.180+0000 7f18a6a00640 1 -- 172.21.15.42:0/3754008007 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f189c041030 con 0x7f18b00abbc0 2024-07-04T17:25:52.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.180+0000 7f18a6a00640 1 -- 172.21.15.42:0/3754008007 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f189c03f070 con 0x7f18b00abbc0 2024-07-04T17:25:52.181 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi042') 2024-07-04T17:25:52.182 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi042') 2024-07-04T17:25:52.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.180+0000 7f189be00640 1 -- 172.21.15.42:0/3754008007 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1884005290 con 0x7f18b00abbc0 2024-07-04T17:25:52.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.186+0000 7f18a6a00640 1 -- 172.21.15.42:0/3754008007 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73745 (secure 0 0 0) 0x7f189c003de0 con 0x7f18b00abbc0 2024-07-04T17:25:52.202 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi042') 2024-07-04T17:25:52.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.203+0000 7f055e800640 1 -- 172.21.15.42:0/4144813604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05580af300 msgr2=0x7f05580a9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:52.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.203+0000 7f055e800640 1 --2- 172.21.15.42:0/4144813604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05580af300 0x7f05580a9d40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f05480097b0 tx=0x7f054802a560 comp rx=0 tx=0).stop 2024-07-04T17:25:52.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.204+0000 7f055e800640 1 -- 172.21.15.42:0/4144813604 shutdown_connections 2024-07-04T17:25:52.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.204+0000 7f055e800640 1 --2- 172.21.15.42:0/4144813604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05580af300 0x7f05580a9d40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:52.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.204+0000 7f055e800640 1 -- 172.21.15.42:0/4144813604 >> 172.21.15.42:0/4144813604 conn(0x7f055806fb80 msgr2=0x7f05580afaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:52.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.204+0000 7f055e800640 1 -- 172.21.15.42:0/4144813604 shutdown_connections 2024-07-04T17:25:52.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.205+0000 7f055e800640 1 -- 172.21.15.42:0/4144813604 wait complete. 2024-07-04T17:25:52.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.208+0000 7f055e800640 1 Processor -- start 2024-07-04T17:25:52.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.208+0000 7f055e800640 1 -- start start 2024-07-04T17:25:52.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.210+0000 7f055e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05580ad1c0 0x7f05581389f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:25:52.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.210+0000 7f055e800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0558141f10 con 0x7f05580ad1c0 2024-07-04T17:25:52.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.211+0000 7f0557400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05580ad1c0 0x7f05581389f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:25:52.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.211+0000 7f0557400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05580ad1c0 0x7f05581389f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60970/0 (socket says 172.21.15.42:60970) 2024-07-04T17:25:52.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.211+0000 7f0557400640 1 -- 172.21.15.42:0/761706079 learned_addr learned my addr 172.21.15.42:0/761706079 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:25:52.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.212+0000 7f0557400640 1 -- 172.21.15.42:0/761706079 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055813f590 con 0x7f05580ad1c0 2024-07-04T17:25:52.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.213+0000 7f0557400640 1 --2- 172.21.15.42:0/761706079 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05580ad1c0 0x7f05581389f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f05480097b0 tx=0x7f0548033e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:25:52.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.214+0000 7f054f400640 1 -- 172.21.15.42:0/761706079 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0548031530 con 0x7f05580ad1c0 2024-07-04T17:25:52.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.215+0000 7f054f400640 1 -- 172.21.15.42:0/761706079 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0548031b80 con 0x7f05580ad1c0 2024-07-04T17:25:52.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.215+0000 7f054f400640 1 -- 172.21.15.42:0/761706079 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0548031530 con 0x7f05580ad1c0 2024-07-04T17:25:52.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.217+0000 7f055e800640 1 -- 172.21.15.42:0/761706079 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f055813f830 con 0x7f05580ad1c0 2024-07-04T17:25:52.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.218+0000 7f055e800640 1 -- 172.21.15.42:0/761706079 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0558138570 con 0x7f05580ad1c0 2024-07-04T17:25:52.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.218+0000 7f054f400640 1 -- 172.21.15.42:0/761706079 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0548033070 con 0x7f05580ad1c0 2024-07-04T17:25:52.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.220+0000 7f054f400640 1 -- 172.21.15.42:0/761706079 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0548031d30 con 0x7f05580ad1c0 2024-07-04T17:25:52.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.220+0000 7f054cc00640 1 -- 172.21.15.42:0/761706079 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f052c005290 con 0x7f05580ad1c0 2024-07-04T17:25:52.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.220+0000 7f1b53e00640 1 -- 172.21.15.42:0/1000915247 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b4c030140 msgr2=0x7f1b4c030f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:52.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.220+0000 7f1b53e00640 1 --2- 172.21.15.42:0/1000915247 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b4c030140 0x7f1b4c030f90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1b44013eb0 tx=0x7f1b44032a30 comp rx=0 tx=0).stop 2024-07-04T17:25:52.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.221+0000 7f1b53e00640 1 -- 172.21.15.42:0/1000915247 shutdown_connections 2024-07-04T17:25:52.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.221+0000 7f1b53e00640 1 --2- 172.21.15.42:0/1000915247 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b4c030140 0x7f1b4c030f90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:52.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.221+0000 7f1b53e00640 1 -- 172.21.15.42:0/1000915247 >> 172.21.15.42:0/1000915247 conn(0x7f1b4c01f830 msgr2=0x7f1b4c0201a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:52.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.221+0000 7f1b53e00640 1 -- 172.21.15.42:0/1000915247 shutdown_connections 2024-07-04T17:25:52.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.221+0000 7f1b53e00640 1 -- 172.21.15.42:0/1000915247 wait complete. 2024-07-04T17:25:52.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.224+0000 7f054f400640 1 -- 172.21.15.42:0/761706079 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73745 (secure 0 0 0) 0x7f0548051080 con 0x7f05580ad1c0 2024-07-04T17:25:52.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.224+0000 7f1b53e00640 1 Processor -- start 2024-07-04T17:25:52.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.225+0000 7f1b53e00640 1 -- start start 2024-07-04T17:25:52.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.227+0000 7f1b53e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b4c030140 0x7f1b4c00aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:25:52.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.227+0000 7f1b53e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c0cb230 con 0x7f1b4c030140 2024-07-04T17:25:52.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.227+0000 7f1b52a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b4c030140 0x7f1b4c00aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:25:52.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.228+0000 7f1b52a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b4c030140 0x7f1b4c00aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60992/0 (socket says 172.21.15.42:60992) 2024-07-04T17:25:52.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.228+0000 7f1b52a00640 1 -- 172.21.15.42:0/2551523305 learned_addr learned my addr 172.21.15.42:0/2551523305 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:25:52.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.228+0000 7f1b52a00640 1 -- 172.21.15.42:0/2551523305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b4c00cd60 con 0x7f1b4c030140 2024-07-04T17:25:52.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.229+0000 7f1b52a00640 1 --2- 172.21.15.42:0/2551523305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b4c030140 0x7f1b4c00aaf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1b400046b0 tx=0x7f1b4000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:25:52.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.230+0000 7f1b4aa00640 1 -- 172.21.15.42:0/2551523305 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b400116c0 con 0x7f1b4c030140 2024-07-04T17:25:52.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.230+0000 7f1b4aa00640 1 -- 172.21.15.42:0/2551523305 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b40017070 con 0x7f1b4c030140 2024-07-04T17:25:52.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.230+0000 7f1b4aa00640 1 -- 172.21.15.42:0/2551523305 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b40011b00 con 0x7f1b4c030140 2024-07-04T17:25:52.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.230+0000 7f1b53e00640 1 -- 172.21.15.42:0/2551523305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b4c1653a0 con 0x7f1b4c030140 2024-07-04T17:25:52.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.232+0000 7f1b4aa00640 1 -- 172.21.15.42:0/2551523305 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1b40011ca0 con 0x7f1b4c030140 2024-07-04T17:25:52.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.232+0000 7f1b53e00640 1 -- 172.21.15.42:0/2551523305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b4c00c270 con 0x7f1b4c030140 2024-07-04T17:25:52.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.233+0000 7f1b4aa00640 1 -- 172.21.15.42:0/2551523305 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1b400127c0 con 0x7f1b4c030140 2024-07-04T17:25:52.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.234+0000 7f1b53e00640 1 -- 172.21.15.42:0/2551523305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b4c0cb8d0 con 0x7f1b4c030140 2024-07-04T17:25:52.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.238+0000 7f1b4aa00640 1 -- 172.21.15.42:0/2551523305 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73745 (secure 0 0 0) 0x7f1b40022020 con 0x7f1b4c030140 2024-07-04T17:25:52.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.266+0000 7f9001600640 1 -- 172.21.15.42:0/1611107191 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fe0003aa0 msgr2=0x7f8fe0003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:52.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.266+0000 7f9001600640 1 --2- 172.21.15.42:0/1611107191 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fe0003aa0 0x7f8fe0003ef0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800c260 tx=0x7f8fe0005830 comp rx=0 tx=0).stop 2024-07-04T17:25:52.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.267+0000 7f9001600640 1 -- 172.21.15.42:0/1611107191 shutdown_connections 2024-07-04T17:25:52.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.267+0000 7f9001600640 1 --2- 172.21.15.42:0/1611107191 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fe0003aa0 0x7f8fe0003ef0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:52.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.267+0000 7f9001600640 1 --2- 172.21.15.42:0/1611107191 >> v2:172.21.15.42:3300/0 conn(0x7f8ffc071670 0x7f8ffc070440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:52.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.268+0000 7f9001600640 1 -- 172.21.15.42:0/1611107191 >> 172.21.15.42:0/1611107191 conn(0x7f8ffc06fc40 msgr2=0x7f8ffc06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:52.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.268+0000 7f9001600640 1 -- 172.21.15.42:0/1611107191 shutdown_connections 2024-07-04T17:25:52.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.268+0000 7f9001600640 1 -- 172.21.15.42:0/1611107191 wait complete. 2024-07-04T17:25:52.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.272+0000 7f9001600640 1 Processor -- start 2024-07-04T17:25:52.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.273+0000 7f9001600640 1 -- start start 2024-07-04T17:25:52.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.276+0000 7f9001600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fe0003aa0 0x7f8ffc141160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:25:52.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.276+0000 7f9001600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc13cfc0 con 0x7f8fe0003aa0 2024-07-04T17:25:52.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.276+0000 7f8ffaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fe0003aa0 0x7f8ffc141160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:25:52.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.277+0000 7f8ffaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fe0003aa0 0x7f8ffc141160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32782/0 (socket says 172.21.15.42:32782) 2024-07-04T17:25:52.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.277+0000 7f8ffaa00640 1 -- 172.21.15.42:0/854646910 learned_addr learned my addr 172.21.15.42:0/854646910 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:25:52.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.278+0000 7f8ffaa00640 1 -- 172.21.15.42:0/854646910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ffc1416a0 con 0x7f8fe0003aa0 2024-07-04T17:25:52.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.279+0000 7f8ffaa00640 1 --2- 172.21.15.42:0/854646910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fe0003aa0 0x7f8ffc141160 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8000c00 tx=0x7f8fe8006290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:25:52.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.279+0000 7f8ff2a00640 1 -- 172.21.15.42:0/854646910 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fe802b600 con 0x7f8fe0003aa0 2024-07-04T17:25:52.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.279+0000 7f8ff2a00640 1 -- 172.21.15.42:0/854646910 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fe802b7a0 con 0x7f8fe0003aa0 2024-07-04T17:25:52.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.279+0000 7f8ff2a00640 1 -- 172.21.15.42:0/854646910 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fe802bb00 con 0x7f8fe0003aa0 2024-07-04T17:25:52.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.279+0000 7f9001600640 1 -- 172.21.15.42:0/854646910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ffc140c50 con 0x7f8fe0003aa0 2024-07-04T17:25:52.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.281+0000 7f9001600640 1 -- 172.21.15.42:0/854646910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ffc13b760 con 0x7f8fe0003aa0 2024-07-04T17:25:52.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.283+0000 7f8ff2a00640 1 -- 172.21.15.42:0/854646910 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8fe8003980 con 0x7f8fe0003aa0 2024-07-04T17:25:52.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.283+0000 7f8ff2a00640 1 -- 172.21.15.42:0/854646910 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8fe803a070 con 0x7f8fe0003aa0 2024-07-04T17:25:52.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.284+0000 7f9001600640 1 -- 172.21.15.42:0/854646910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ffc070440 con 0x7f8fe0003aa0 2024-07-04T17:25:52.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.288+0000 7f8ff2a00640 1 -- 172.21.15.42:0/854646910 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73745 (secure 0 0 0) 0x7f8fe804c080 con 0x7f8fe0003aa0 2024-07-04T17:25:52.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.305+0000 7f01c5200640 1 -- 172.21.15.42:0/3894448474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01b80cbd40 msgr2=0x7f01b8030f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:52.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.305+0000 7f01c5200640 1 --2- 172.21.15.42:0/3894448474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01b80cbd40 0x7f01b8030f80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0097a0 tx=0x7f01ac02a4b0 comp rx=0 tx=0).stop 2024-07-04T17:25:52.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.306+0000 7f01c5200640 1 -- 172.21.15.42:0/3894448474 shutdown_connections 2024-07-04T17:25:52.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.306+0000 7f01c5200640 1 --2- 172.21.15.42:0/3894448474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01b80cbd40 0x7f01b8030f80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:52.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.306+0000 7f01c5200640 1 -- 172.21.15.42:0/3894448474 >> 172.21.15.42:0/3894448474 conn(0x7f01b801f820 msgr2=0x7f01b8020190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:52.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.306+0000 7f01c5200640 1 -- 172.21.15.42:0/3894448474 shutdown_connections 2024-07-04T17:25:52.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.306+0000 7f01c5200640 1 -- 172.21.15.42:0/3894448474 wait complete. 2024-07-04T17:25:52.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.309+0000 7f4c55e00640 1 -- 172.21.15.42:0/212692055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f4c50069b70 con 0x7f4c500b0a50 2024-07-04T17:25:52.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.310+0000 7f01c5200640 1 Processor -- start 2024-07-04T17:25:52.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.310+0000 7f01c5200640 1 -- start start 2024-07-04T17:25:52.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.312+0000 7f01c5200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01b8030130 0x7f01b8034700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:25:52.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.312+0000 7f01c5200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f01b80cb220 con 0x7f01b8030130 2024-07-04T17:25:52.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.312+0000 7f01bde00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01b8030130 0x7f01b8034700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:25:52.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.312+0000 7f01bde00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01b8030130 0x7f01b8034700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32806/0 (socket says 172.21.15.42:32806) 2024-07-04T17:25:52.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.312+0000 7f01bde00640 1 -- 172.21.15.42:0/4179656439 learned_addr learned my addr 172.21.15.42:0/4179656439 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:25:52.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.313+0000 7f01bde00640 1 -- 172.21.15.42:0/4179656439 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b8036400 con 0x7f01b8030130 2024-07-04T17:25:52.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.313+0000 7f4c47400640 1 -- 172.21.15.42:0/212692055 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v13)=0 v13) ==== 119+0+0 (secure 0 0 0) 0x7f4c38031d30 con 0x7f4c500b0a50 2024-07-04T17:25:52.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.315+0000 7f4c55e00640 1 -- 172.21.15.42:0/212692055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c500b0a50 msgr2=0x7f4c50073130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:52.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.315+0000 7f4c55e00640 1 --2- 172.21.15.42:0/212692055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c500b0a50 0x7f4c50073130 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009850 tx=0x7f4c38004260 comp rx=0 tx=0).stop 2024-07-04T17:25:52.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.315+0000 7f4c55e00640 1 -- 172.21.15.42:0/212692055 shutdown_connections 2024-07-04T17:25:52.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.315+0000 7f4c55e00640 1 --2- 172.21.15.42:0/212692055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c500b0a50 0x7f4c50073130 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:52.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.315+0000 7f4c55e00640 1 -- 172.21.15.42:0/212692055 >> 172.21.15.42:0/212692055 conn(0x7f4c500beea0 msgr2=0x7f4c5014f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:52.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.315+0000 7f01bde00640 1 --2- 172.21.15.42:0/4179656439 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01b8030130 0x7f01b8034700 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0098c0 tx=0x7f01ac033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:25:52.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.315+0000 7f4c55e00640 1 -- 172.21.15.42:0/212692055 shutdown_connections 2024-07-04T17:25:52.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.315+0000 7f4c55e00640 1 -- 172.21.15.42:0/212692055 wait complete. 2024-07-04T17:25:52.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.316+0000 7f01b6000640 1 -- 172.21.15.42:0/4179656439 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01ac031bc0 con 0x7f01b8030130 2024-07-04T17:25:52.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.316+0000 7f01b6000640 1 -- 172.21.15.42:0/4179656439 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01ac031e50 con 0x7f01b8030130 2024-07-04T17:25:52.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.316+0000 7f01b6000640 1 -- 172.21.15.42:0/4179656439 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01ac005190 con 0x7f01b8030130 2024-07-04T17:25:52.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.316+0000 7f01c5200640 1 -- 172.21.15.42:0/4179656439 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01b8034250 con 0x7f01b8030130 2024-07-04T17:25:52.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.317+0000 7f01c5200640 1 -- 172.21.15.42:0/4179656439 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01b816e8a0 con 0x7f01b8030130 2024-07-04T17:25:52.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.318+0000 7f01b6000640 1 -- 172.21.15.42:0/4179656439 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f01ac033070 con 0x7f01b8030130 2024-07-04T17:25:52.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.319+0000 7f01b6000640 1 -- 172.21.15.42:0/4179656439 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f01ac004b00 con 0x7f01b8030130 2024-07-04T17:25:52.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.320+0000 7f01c5200640 1 -- 172.21.15.42:0/4179656439 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0198005290 con 0x7f01b8030130 2024-07-04T17:25:52.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.320+0000 7efd54c00640 1 -- 172.21.15.42:0/3372104898 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7efd48036510 con 0x7efd4802ccd0 2024-07-04T17:25:52.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.322+0000 7f01b6000640 1 -- 172.21.15.42:0/4179656439 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73745 (secure 0 0 0) 0x7f01ac03f030 con 0x7f01b8030130 2024-07-04T17:25:52.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.325+0000 7efd43400640 1 -- 172.21.15.42:0/3372104898 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v14)=0 v14) ==== 121+0+0 (secure 0 0 0) 0x7efd44043930 con 0x7efd4802ccd0 2024-07-04T17:25:52.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.327+0000 7efd54c00640 1 -- 172.21.15.42:0/3372104898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd4802ccd0 msgr2=0x7efd4802f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:52.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.327+0000 7efd54c00640 1 --2- 172.21.15.42:0/3372104898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd4802ccd0 0x7efd4802f770 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efd44034040 tx=0x7efd440049d0 comp rx=0 tx=0).stop 2024-07-04T17:25:52.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.327+0000 7efd54c00640 1 -- 172.21.15.42:0/3372104898 shutdown_connections 2024-07-04T17:25:52.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.327+0000 7efd54c00640 1 --2- 172.21.15.42:0/3372104898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd4802ccd0 0x7efd4802f770 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:52.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.327+0000 7efd54c00640 1 -- 172.21.15.42:0/3372104898 >> 172.21.15.42:0/3372104898 conn(0x7efd4801b2e0 msgr2=0x7efd48006330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:52.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.328+0000 7efd54c00640 1 -- 172.21.15.42:0/3372104898 shutdown_connections 2024-07-04T17:25:52.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.328+0000 7efd54c00640 1 -- 172.21.15.42:0/3372104898 wait complete. 2024-07-04T17:25:52.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.353+0000 7f189be00640 1 -- 172.21.15.42:0/3754008007 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f1884005580 con 0x7f18b00abbc0 2024-07-04T17:25:52.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.358+0000 7f18a6a00640 1 -- 172.21.15.42:0/3754008007 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v15)=0 v15) ==== 120+0+0 (secure 0 0 0) 0x7f189c043410 con 0x7f18b00abbc0 2024-07-04T17:25:52.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.359+0000 7f189be00640 1 -- 172.21.15.42:0/3754008007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18b00abbc0 msgr2=0x7f18b00ae990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:52.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.359+0000 7f189be00640 1 --2- 172.21.15.42:0/3754008007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18b00abbc0 0x7f18b00ae990 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f189c009850 tx=0x7f189c004d20 comp rx=0 tx=0).stop 2024-07-04T17:25:52.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.362+0000 7f189be00640 1 -- 172.21.15.42:0/3754008007 shutdown_connections 2024-07-04T17:25:52.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.362+0000 7f189be00640 1 --2- 172.21.15.42:0/3754008007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18b00abbc0 0x7f18b00ae990 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:52.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.362+0000 7f189be00640 1 -- 172.21.15.42:0/3754008007 >> 172.21.15.42:0/3754008007 conn(0x7f18b006c440 msgr2=0x7f18b00750b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:52.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.362+0000 7f189be00640 1 -- 172.21.15.42:0/3754008007 shutdown_connections 2024-07-04T17:25:52.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.362+0000 7f189be00640 1 -- 172.21.15.42:0/3754008007 wait complete. 2024-07-04T17:25:52.363 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi042') 2024-07-04T17:25:52.364 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi042') 2024-07-04T17:25:52.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.372+0000 7f054cc00640 1 -- 172.21.15.42:0/761706079 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f052c005850 con 0x7f05580ad1c0 2024-07-04T17:25:52.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.374+0000 7f054f400640 1 -- 172.21.15.42:0/761706079 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v16)=0 v16) ==== 121+0+0 (secure 0 0 0) 0x7f054803f030 con 0x7f05580ad1c0 2024-07-04T17:25:52.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.376+0000 7f054cc00640 1 -- 172.21.15.42:0/761706079 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05580ad1c0 msgr2=0x7f05581389f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:52.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.376+0000 7f054cc00640 1 --2- 172.21.15.42:0/761706079 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05580ad1c0 0x7f05581389f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f05480097b0 tx=0x7f0548033e80 comp rx=0 tx=0).stop 2024-07-04T17:25:52.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.377+0000 7f054cc00640 1 -- 172.21.15.42:0/761706079 shutdown_connections 2024-07-04T17:25:52.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.377+0000 7f054cc00640 1 --2- 172.21.15.42:0/761706079 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05580ad1c0 0x7f05581389f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:52.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.377+0000 7f054cc00640 1 -- 172.21.15.42:0/761706079 >> 172.21.15.42:0/761706079 conn(0x7f055806fb80 msgr2=0x7f05580b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:52.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.377+0000 7f054cc00640 1 -- 172.21.15.42:0/761706079 shutdown_connections 2024-07-04T17:25:52.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.377+0000 7f054cc00640 1 -- 172.21.15.42:0/761706079 wait complete. 2024-07-04T17:25:52.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.382+0000 7f1b53e00640 1 -- 172.21.15.42:0/2551523305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f1b4c0c9f40 con 0x7f1b4c030140 2024-07-04T17:25:52.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.387+0000 7f1b4aa00640 1 -- 172.21.15.42:0/2551523305 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v17)=0 v17) ==== 120+0+0 (secure 0 0 0) 0x7f1b400129c0 con 0x7f1b4c030140 2024-07-04T17:25:52.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.389+0000 7f1b53e00640 1 -- 172.21.15.42:0/2551523305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b4c030140 msgr2=0x7f1b4c00aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:52.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.389+0000 7f1b53e00640 1 --2- 172.21.15.42:0/2551523305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b4c030140 0x7f1b4c00aaf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1b400046b0 tx=0x7f1b4000c810 comp rx=0 tx=0).stop 2024-07-04T17:25:52.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.389+0000 7f1b53e00640 1 -- 172.21.15.42:0/2551523305 shutdown_connections 2024-07-04T17:25:52.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.389+0000 7f1b53e00640 1 --2- 172.21.15.42:0/2551523305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b4c030140 0x7f1b4c00aaf0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:52.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.389+0000 7f1b53e00640 1 -- 172.21.15.42:0/2551523305 >> 172.21.15.42:0/2551523305 conn(0x7f1b4c01f830 msgr2=0x7f1b4c004790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:52.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.390+0000 7f1b53e00640 1 -- 172.21.15.42:0/2551523305 shutdown_connections 2024-07-04T17:25:52.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.390+0000 7f1b53e00640 1 -- 172.21.15.42:0/2551523305 wait complete. 2024-07-04T17:25:52.396 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi042') 2024-07-04T17:25:52.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.402+0000 7f9001600640 1 -- 172.21.15.42:0/854646910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f8ffc13d700 con 0x7f8fe0003aa0 2024-07-04T17:25:52.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.405+0000 7f8ff2a00640 1 -- 172.21.15.42:0/854646910 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v18)=0 v18) ==== 125+0+0 (secure 0 0 0) 0x7f8ffc071a80 con 0x7f8fe0003aa0 2024-07-04T17:25:52.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.406+0000 7f9001600640 1 -- 172.21.15.42:0/854646910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fe0003aa0 msgr2=0x7f8ffc141160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:52.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.406+0000 7f9001600640 1 --2- 172.21.15.42:0/854646910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fe0003aa0 0x7f8ffc141160 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8000c00 tx=0x7f8fe8006290 comp rx=0 tx=0).stop 2024-07-04T17:25:52.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.406+0000 7f9001600640 1 -- 172.21.15.42:0/854646910 shutdown_connections 2024-07-04T17:25:52.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.406+0000 7f9001600640 1 --2- 172.21.15.42:0/854646910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fe0003aa0 0x7f8ffc141160 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:52.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.406+0000 7f9001600640 1 -- 172.21.15.42:0/854646910 >> 172.21.15.42:0/854646910 conn(0x7f8ffc06fc40 msgr2=0x7f8ffc06cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:52.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.406+0000 7f9001600640 1 -- 172.21.15.42:0/854646910 shutdown_connections 2024-07-04T17:25:52.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.406+0000 7f9001600640 1 -- 172.21.15.42:0/854646910 wait complete. 2024-07-04T17:25:52.431 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi042') 2024-07-04T17:25:52.432 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi042') 2024-07-04T17:25:52.432 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi042') 2024-07-04T17:25:52.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.434+0000 7f01c5200640 1 -- 172.21.15.42:0/4179656439 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f0198005850 con 0x7f01b8030130 2024-07-04T17:25:52.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.436+0000 7f01b6000640 1 -- 172.21.15.42:0/4179656439 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v19)=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7f01ac0093d0 con 0x7f01b8030130 2024-07-04T17:25:52.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.437+0000 7f01c5200640 1 -- 172.21.15.42:0/4179656439 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01b8030130 msgr2=0x7f01b8034700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:52.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.437+0000 7f01c5200640 1 --2- 172.21.15.42:0/4179656439 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01b8030130 0x7f01b8034700 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0098c0 tx=0x7f01ac033c10 comp rx=0 tx=0).stop 2024-07-04T17:25:52.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.437+0000 7f01c5200640 1 -- 172.21.15.42:0/4179656439 shutdown_connections 2024-07-04T17:25:52.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.437+0000 7f01c5200640 1 --2- 172.21.15.42:0/4179656439 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01b8030130 0x7f01b8034700 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:52.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.437+0000 7f01c5200640 1 -- 172.21.15.42:0/4179656439 >> 172.21.15.42:0/4179656439 conn(0x7f01b801f820 msgr2=0x7f01b8039d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:52.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.437+0000 7f01c5200640 1 -- 172.21.15.42:0/4179656439 shutdown_connections 2024-07-04T17:25:52.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.437+0000 7f01c5200640 1 -- 172.21.15.42:0/4179656439 wait complete. 2024-07-04T17:25:52.462 INFO:tasks.ceph:Setting crush tunables to default 2024-07-04T17:25:52.462 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph osd crush tunables default 2024-07-04T17:25:52.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.762+0000 7fb881e00640 1 -- 172.21.15.42:0/1857600691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb87c1418d0 msgr2=0x7fb87c141ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:52.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.762+0000 7fb881e00640 1 --2- 172.21.15.42:0/1857600691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb87c1418d0 0x7fb87c141ce0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb868005fd0 tx=0x7fb86802a5f0 comp rx=0 tx=0).stop 2024-07-04T17:25:52.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.762+0000 7fb881e00640 1 -- 172.21.15.42:0/1857600691 shutdown_connections 2024-07-04T17:25:52.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.762+0000 7fb881e00640 1 --2- 172.21.15.42:0/1857600691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb87c1418d0 0x7fb87c141ce0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:52.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.762+0000 7fb881e00640 1 -- 172.21.15.42:0/1857600691 >> 172.21.15.42:0/1857600691 conn(0x7fb87c06fe40 msgr2=0x7fb87c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:52.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.763+0000 7fb881e00640 1 -- 172.21.15.42:0/1857600691 shutdown_connections 2024-07-04T17:25:52.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.763+0000 7fb881e00640 1 -- 172.21.15.42:0/1857600691 wait complete. 2024-07-04T17:25:52.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.765+0000 7fb881e00640 1 Processor -- start 2024-07-04T17:25:52.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.765+0000 7fb881e00640 1 -- start start 2024-07-04T17:25:52.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.767+0000 7fb881e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb87c0b1d20 0x7fb87c1385b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:25:52.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.767+0000 7fb881e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c073a90 con 0x7fb87c0b1d20 2024-07-04T17:25:52.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.767+0000 7fb87aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb87c0b1d20 0x7fb87c1385b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:25:52.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.767+0000 7fb87aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb87c0b1d20 0x7fb87c1385b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32824/0 (socket says 172.21.15.42:32824) 2024-07-04T17:25:52.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.767+0000 7fb87aa00640 1 -- 172.21.15.42:0/689584531 learned_addr learned my addr 172.21.15.42:0/689584531 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:25:52.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.768+0000 7fb87aa00640 1 -- 172.21.15.42:0/689584531 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb87c0715a0 con 0x7fb87c0b1d20 2024-07-04T17:25:52.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.768+0000 7fb87aa00640 1 --2- 172.21.15.42:0/689584531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb87c0b1d20 0x7fb87c1385b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00c6d0 tx=0x7fb86c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:25:52.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.769+0000 7fb873400640 1 -- 172.21.15.42:0/689584531 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb86c003aa0 con 0x7fb87c0b1d20 2024-07-04T17:25:52.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.769+0000 7fb873400640 1 -- 172.21.15.42:0/689584531 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb86c003c40 con 0x7fb87c0b1d20 2024-07-04T17:25:52.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.769+0000 7fb873400640 1 -- 172.21.15.42:0/689584531 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb86c0050d0 con 0x7fb87c0b1d20 2024-07-04T17:25:52.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.769+0000 7fb881e00640 1 -- 172.21.15.42:0/689584531 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb87c0af2f0 con 0x7fb87c0b1d20 2024-07-04T17:25:52.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.769+0000 7fb881e00640 1 -- 172.21.15.42:0/689584531 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb87c0afab0 con 0x7fb87c0b1d20 2024-07-04T17:25:52.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.769+0000 7fb873400640 1 -- 172.21.15.42:0/689584531 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb86c005270 con 0x7fb87c0b1d20 2024-07-04T17:25:52.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.769+0000 7fb873400640 1 -- 172.21.15.42:0/689584531 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb86c012c50 con 0x7fb87c0b1d20 2024-07-04T17:25:52.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.770+0000 7fb881e00640 1 -- 172.21.15.42:0/689584531 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb87c0740c0 con 0x7fb87c0b1d20 2024-07-04T17:25:52.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.772+0000 7fb873400640 1 -- 172.21.15.42:0/689584531 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73745 (secure 0 0 0) 0x7fb87c0afab0 con 0x7fb87c0b1d20 2024-07-04T17:25:52.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.843+0000 7fb881e00640 1 -- 172.21.15.42:0/689584531 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fb87c1e34c0 con 0x7fb87c0b1d20 2024-07-04T17:25:52.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.845+0000 7fb873400640 1 -- 172.21.15.42:0/689584531 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fb86c020020 con 0x7fb87c0b1d20 2024-07-04T17:25:52.846 INFO:teuthology.orchestra.run.smithi042.stderr:adjusted tunables profile to default 2024-07-04T17:25:52.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.846+0000 7fb881e00640 1 -- 172.21.15.42:0/689584531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb87c0b1d20 msgr2=0x7fb87c1385b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:52.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.846+0000 7fb881e00640 1 --2- 172.21.15.42:0/689584531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb87c0b1d20 0x7fb87c1385b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00c6d0 tx=0x7fb86c00cb90 comp rx=0 tx=0).stop 2024-07-04T17:25:52.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.847+0000 7fb881e00640 1 -- 172.21.15.42:0/689584531 shutdown_connections 2024-07-04T17:25:52.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.847+0000 7fb881e00640 1 --2- 172.21.15.42:0/689584531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb87c0b1d20 0x7fb87c1385b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:52.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.847+0000 7fb881e00640 1 -- 172.21.15.42:0/689584531 >> 172.21.15.42:0/689584531 conn(0x7fb87c06fe40 msgr2=0x7fb87c13fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:52.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.847+0000 7fb881e00640 1 -- 172.21.15.42:0/689584531 shutdown_connections 2024-07-04T17:25:52.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:52.847+0000 7fb881e00640 1 -- 172.21.15.42:0/689584531 wait complete. 2024-07-04T17:25:52.871 INFO:tasks.ceph:check_enable_crimson: False 2024-07-04T17:25:52.871 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-07-04T17:25:52.872 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:25:52.872 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-07-04T17:25:52.948 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:25:52.948 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-07-04T17:25:53.073 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:25:53.073 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-07-04T17:25:53.145 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph osd new 7868bbeb-8129-4123-bc11-d96d23f65a77 0 2024-07-04T17:25:53.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.437+0000 7fe35f000640 1 -- 172.21.15.42:0/2113538328 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3580ab310 msgr2=0x7fe3580a99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:53.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.437+0000 7fe35f000640 1 --2- 172.21.15.42:0/2113538328 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3580ab310 0x7fe3580a99a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe348005fd0 tx=0x7fe34802a560 comp rx=0 tx=0).stop 2024-07-04T17:25:53.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.438+0000 7fe35f000640 1 -- 172.21.15.42:0/2113538328 shutdown_connections 2024-07-04T17:25:53.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.438+0000 7fe35f000640 1 --2- 172.21.15.42:0/2113538328 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3580ab310 0x7fe3580a99a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:53.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.438+0000 7fe35f000640 1 -- 172.21.15.42:0/2113538328 >> 172.21.15.42:0/2113538328 conn(0x7fe3580b33f0 msgr2=0x7fe3580b3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:53.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.438+0000 7fe35f000640 1 -- 172.21.15.42:0/2113538328 shutdown_connections 2024-07-04T17:25:53.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.438+0000 7fe35f000640 1 -- 172.21.15.42:0/2113538328 wait complete. 2024-07-04T17:25:53.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.440+0000 7fe35f000640 1 Processor -- start 2024-07-04T17:25:53.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.440+0000 7fe35f000640 1 -- start start 2024-07-04T17:25:53.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.441+0000 7fe35f000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe358073b40 0x7fe3580afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:25:53.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.441+0000 7fe35f000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe3580aaa80 con 0x7fe358073b40 2024-07-04T17:25:53.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.442+0000 7fe357e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe358073b40 0x7fe3580afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:25:53.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.442+0000 7fe357e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe358073b40 0x7fe3580afc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32848/0 (socket says 172.21.15.42:32848) 2024-07-04T17:25:53.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.442+0000 7fe357e00640 1 -- 172.21.15.42:0/1549446623 learned_addr learned my addr 172.21.15.42:0/1549446623 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:25:53.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.442+0000 7fe357e00640 1 -- 172.21.15.42:0/1549446623 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe358072fa0 con 0x7fe358073b40 2024-07-04T17:25:53.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.442+0000 7fe357e00640 1 --2- 172.21.15.42:0/1549446623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe358073b40 0x7fe3580afc30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe348009850 tx=0x7fe3480049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:25:53.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.442+0000 7fe34fe00640 1 -- 172.21.15.42:0/1549446623 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe34803e030 con 0x7fe358073b40 2024-07-04T17:25:53.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.443+0000 7fe34fe00640 1 -- 172.21.15.42:0/1549446623 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3480035f0 con 0x7fe358073b40 2024-07-04T17:25:53.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.443+0000 7fe34fe00640 1 -- 172.21.15.42:0/1549446623 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe348035040 con 0x7fe358073b40 2024-07-04T17:25:53.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.443+0000 7fe35f000640 1 -- 172.21.15.42:0/1549446623 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3580b0170 con 0x7fe358073b40 2024-07-04T17:25:53.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.443+0000 7fe35f000640 1 -- 172.21.15.42:0/1549446623 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe358140470 con 0x7fe358073b40 2024-07-04T17:25:53.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.443+0000 7fe34fe00640 1 -- 172.21.15.42:0/1549446623 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe348041030 con 0x7fe358073b40 2024-07-04T17:25:53.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.443+0000 7fe34fe00640 1 -- 172.21.15.42:0/1549446623 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fe34803f070 con 0x7fe358073b40 2024-07-04T17:25:53.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.444+0000 7fe35f000640 1 -- 172.21.15.42:0/1549446623 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3580ab310 con 0x7fe358073b40 2024-07-04T17:25:53.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.446+0000 7fe34fe00640 1 -- 172.21.15.42:0/1549446623 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73745 (secure 0 0 0) 0x7fe348043e20 con 0x7fe358073b40 2024-07-04T17:25:53.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.524+0000 7fe35f000640 1 -- 172.21.15.42:0/1549446623 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7868bbeb-8129-4123-bc11-d96d23f65a77", "id": 0} v 0) -- 0x7fe358069b70 con 0x7fe358073b40 2024-07-04T17:25:53.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.526+0000 7fe34fe00640 1 -- 172.21.15.42:0/1549446623 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7868bbeb-8129-4123-bc11-d96d23f65a77", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fe358140470 con 0x7fe358073b40 2024-07-04T17:25:53.527 INFO:teuthology.orchestra.run.smithi042.stdout:0 2024-07-04T17:25:53.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.528+0000 7fe35f000640 1 -- 172.21.15.42:0/1549446623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe358073b40 msgr2=0x7fe3580afc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:53.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.528+0000 7fe35f000640 1 --2- 172.21.15.42:0/1549446623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe358073b40 0x7fe3580afc30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe348009850 tx=0x7fe3480049d0 comp rx=0 tx=0).stop 2024-07-04T17:25:53.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.528+0000 7fe35f000640 1 -- 172.21.15.42:0/1549446623 shutdown_connections 2024-07-04T17:25:53.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.528+0000 7fe35f000640 1 --2- 172.21.15.42:0/1549446623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe358073b40 0x7fe3580afc30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:53.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.528+0000 7fe35f000640 1 -- 172.21.15.42:0/1549446623 >> 172.21.15.42:0/1549446623 conn(0x7fe3580b33f0 msgr2=0x7fe35806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:53.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.528+0000 7fe35f000640 1 -- 172.21.15.42:0/1549446623 shutdown_connections 2024-07-04T17:25:53.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.528+0000 7fe35f000640 1 -- 172.21.15.42:0/1549446623 wait complete. 2024-07-04T17:25:53.551 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph osd new 8d5148e1-e5d7-4bca-abef-00e5b1f9e35a 1 2024-07-04T17:25:53.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.804+0000 7f38e3600640 1 -- 172.21.15.42:0/3702639549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38dc0b1d20 msgr2=0x7f38dc0726f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:53.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.804+0000 7f38e3600640 1 --2- 172.21.15.42:0/3702639549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38dc0b1d20 0x7f38dc0726f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f38cc005fd0 tx=0x7f38cc02a5f0 comp rx=0 tx=0).stop 2024-07-04T17:25:53.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.804+0000 7f38e3600640 1 -- 172.21.15.42:0/3702639549 shutdown_connections 2024-07-04T17:25:53.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.804+0000 7f38e3600640 1 --2- 172.21.15.42:0/3702639549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38dc0b1d20 0x7f38dc0726f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:53.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.804+0000 7f38e3600640 1 -- 172.21.15.42:0/3702639549 >> 172.21.15.42:0/3702639549 conn(0x7f38dc06fe40 msgr2=0x7f38dc06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:53.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.804+0000 7f38e3600640 1 -- 172.21.15.42:0/3702639549 shutdown_connections 2024-07-04T17:25:53.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.804+0000 7f38e3600640 1 -- 172.21.15.42:0/3702639549 wait complete. 2024-07-04T17:25:53.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.806+0000 7f38e3600640 1 Processor -- start 2024-07-04T17:25:53.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.806+0000 7f38e3600640 1 -- start start 2024-07-04T17:25:53.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.808+0000 7f38e3600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38dc0b0740 0x7f38dc1439e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:25:53.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.808+0000 7f38e3600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc0737b0 con 0x7f38dc0b0740 2024-07-04T17:25:53.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.808+0000 7f38e0c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38dc0b0740 0x7f38dc1439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:25:53.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.808+0000 7f38e0c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38dc0b0740 0x7f38dc1439e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32860/0 (socket says 172.21.15.42:32860) 2024-07-04T17:25:53.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.808+0000 7f38e0c00640 1 -- 172.21.15.42:0/559555477 learned_addr learned my addr 172.21.15.42:0/559555477 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:25:53.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.808+0000 7f38e0c00640 1 -- 172.21.15.42:0/559555477 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38dc13e600 con 0x7f38dc0b0740 2024-07-04T17:25:53.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.809+0000 7f38e0c00640 1 --2- 172.21.15.42:0/559555477 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38dc0b0740 0x7f38dc1439e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f38d000c380 tx=0x7f38d000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:25:53.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.809+0000 7f38d8c00640 1 -- 172.21.15.42:0/559555477 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38d0017070 con 0x7f38dc0b0740 2024-07-04T17:25:53.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.809+0000 7f38d8c00640 1 -- 172.21.15.42:0/559555477 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38d00052f0 con 0x7f38dc0b0740 2024-07-04T17:25:53.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.809+0000 7f38d8c00640 1 -- 172.21.15.42:0/559555477 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38d00035f0 con 0x7f38dc0b0740 2024-07-04T17:25:53.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.809+0000 7f38e3600640 1 -- 172.21.15.42:0/559555477 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38dc0be2f0 con 0x7f38dc0b0740 2024-07-04T17:25:53.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.809+0000 7f38e3600640 1 -- 172.21.15.42:0/559555477 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38dc0bd880 con 0x7f38dc0b0740 2024-07-04T17:25:53.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.809+0000 7f38d8c00640 1 -- 172.21.15.42:0/559555477 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f38d001a070 con 0x7f38dc0b0740 2024-07-04T17:25:53.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.809+0000 7f38d8c00640 1 -- 172.21.15.42:0/559555477 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f38d0011c50 con 0x7f38dc0b0740 2024-07-04T17:25:53.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.810+0000 7f38e3600640 1 -- 172.21.15.42:0/559555477 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38dc0008d0 con 0x7f38dc0b0740 2024-07-04T17:25:53.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.813+0000 7f38d8c00640 1 -- 172.21.15.42:0/559555477 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73745 (secure 0 0 0) 0x7f38d0021070 con 0x7f38dc0b0740 2024-07-04T17:25:53.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.891+0000 7f38e3600640 1 -- 172.21.15.42:0/559555477 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8d5148e1-e5d7-4bca-abef-00e5b1f9e35a", "id": 1} v 0) -- 0x7f38dc0725e0 con 0x7f38dc0b0740 2024-07-04T17:25:53.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.893+0000 7f38d8c00640 1 -- 172.21.15.42:0/559555477 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8d5148e1-e5d7-4bca-abef-00e5b1f9e35a", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f38dc0bd880 con 0x7f38dc0b0740 2024-07-04T17:25:53.893 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-07-04T17:25:53.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.894+0000 7f38e3600640 1 -- 172.21.15.42:0/559555477 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38dc0b0740 msgr2=0x7f38dc1439e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:53.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.894+0000 7f38e3600640 1 --2- 172.21.15.42:0/559555477 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38dc0b0740 0x7f38dc1439e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f38d000c380 tx=0x7f38d000c840 comp rx=0 tx=0).stop 2024-07-04T17:25:53.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.894+0000 7f38e3600640 1 -- 172.21.15.42:0/559555477 shutdown_connections 2024-07-04T17:25:53.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.894+0000 7f38e3600640 1 --2- 172.21.15.42:0/559555477 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38dc0b0740 0x7f38dc1439e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:53.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.894+0000 7f38e3600640 1 -- 172.21.15.42:0/559555477 >> 172.21.15.42:0/559555477 conn(0x7f38dc06fe40 msgr2=0x7f38dc075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:53.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.895+0000 7f38e3600640 1 -- 172.21.15.42:0/559555477 shutdown_connections 2024-07-04T17:25:53.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:53.895+0000 7f38e3600640 1 -- 172.21.15.42:0/559555477 wait complete. 2024-07-04T17:25:53.919 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph osd new c24efcdf-9c7d-46d6-a323-76fb5250d5fa 2 2024-07-04T17:25:54.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.172+0000 7f0403600640 1 -- 172.21.15.42:0/3871605297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03fc06c9d0 msgr2=0x7f03fc0b56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:54.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.172+0000 7f0403600640 1 --2- 172.21.15.42:0/3871605297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03fc06c9d0 0x7f03fc0b56b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f03f00097b0 tx=0x7f03f002a5d0 comp rx=0 tx=0).stop 2024-07-04T17:25:54.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.172+0000 7f0403600640 1 -- 172.21.15.42:0/3871605297 shutdown_connections 2024-07-04T17:25:54.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.172+0000 7f0403600640 1 --2- 172.21.15.42:0/3871605297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03fc06c9d0 0x7f03fc0b56b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:54.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.172+0000 7f0403600640 1 -- 172.21.15.42:0/3871605297 >> 172.21.15.42:0/3871605297 conn(0x7f03fc071d80 msgr2=0x7f03fc06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:54.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.172+0000 7f0403600640 1 -- 172.21.15.42:0/3871605297 shutdown_connections 2024-07-04T17:25:54.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.173+0000 7f0403600640 1 -- 172.21.15.42:0/3871605297 wait complete. 2024-07-04T17:25:54.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.175+0000 7f0403600640 1 Processor -- start 2024-07-04T17:25:54.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.175+0000 7f0403600640 1 -- start start 2024-07-04T17:25:54.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.176+0000 7f0403600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03fc0aaf00 0x7f03fc06edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:25:54.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.176+0000 7f0403600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc13f420 con 0x7f03fc0aaf00 2024-07-04T17:25:54.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.176+0000 7f0400c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03fc0aaf00 0x7f03fc06edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:25:54.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.176+0000 7f0400c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03fc0aaf00 0x7f03fc06edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35340/0 (socket says 172.21.15.42:35340) 2024-07-04T17:25:54.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.176+0000 7f0400c00640 1 -- 172.21.15.42:0/731157346 learned_addr learned my addr 172.21.15.42:0/731157346 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:25:54.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.177+0000 7f0400c00640 1 -- 172.21.15.42:0/731157346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03fc1d6bf0 con 0x7f03fc0aaf00 2024-07-04T17:25:54.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.177+0000 7f0400c00640 1 --2- 172.21.15.42:0/731157346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03fc0aaf00 0x7f03fc06edc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f03f002ab40 tx=0x7f03f0032aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:25:54.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.177+0000 7f03f8c00640 1 -- 172.21.15.42:0/731157346 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03f0040070 con 0x7f03fc0aaf00 2024-07-04T17:25:54.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.178+0000 7f03f8c00640 1 -- 172.21.15.42:0/731157346 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03f00036e0 con 0x7f03fc0aaf00 2024-07-04T17:25:54.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.178+0000 7f03f8c00640 1 -- 172.21.15.42:0/731157346 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03f0003a00 con 0x7f03fc0aaf00 2024-07-04T17:25:54.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.178+0000 7f0403600640 1 -- 172.21.15.42:0/731157346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03fc13cac0 con 0x7f03fc0aaf00 2024-07-04T17:25:54.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.178+0000 7f0403600640 1 -- 172.21.15.42:0/731157346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03fc1d6220 con 0x7f03fc0aaf00 2024-07-04T17:25:54.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.178+0000 7f03f8c00640 1 -- 172.21.15.42:0/731157346 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f03f0039040 con 0x7f03fc0aaf00 2024-07-04T17:25:54.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.178+0000 7f03f8c00640 1 -- 172.21.15.42:0/731157346 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f03f0045460 con 0x7f03fc0aaf00 2024-07-04T17:25:54.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.179+0000 7f0403600640 1 -- 172.21.15.42:0/731157346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03fc0b5940 con 0x7f03fc0aaf00 2024-07-04T17:25:54.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.181+0000 7f03f8c00640 1 -- 172.21.15.42:0/731157346 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73745 (secure 0 0 0) 0x7f03f0003ba0 con 0x7f03fc0aaf00 2024-07-04T17:25:54.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.261+0000 7f0403600640 1 -- 172.21.15.42:0/731157346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c24efcdf-9c7d-46d6-a323-76fb5250d5fa", "id": 2} v 0) -- 0x7f03fc069b70 con 0x7f03fc0aaf00 2024-07-04T17:25:54.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.264+0000 7f03f8c00640 1 -- 172.21.15.42:0/731157346 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c24efcdf-9c7d-46d6-a323-76fb5250d5fa", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f03f0003e20 con 0x7f03fc0aaf00 2024-07-04T17:25:54.265 INFO:teuthology.orchestra.run.smithi042.stdout:2 2024-07-04T17:25:54.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.266+0000 7f0403600640 1 -- 172.21.15.42:0/731157346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03fc0aaf00 msgr2=0x7f03fc06edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:54.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.266+0000 7f0403600640 1 --2- 172.21.15.42:0/731157346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03fc0aaf00 0x7f03fc06edc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f03f002ab40 tx=0x7f03f0032aa0 comp rx=0 tx=0).stop 2024-07-04T17:25:54.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.266+0000 7f0403600640 1 -- 172.21.15.42:0/731157346 shutdown_connections 2024-07-04T17:25:54.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.266+0000 7f0403600640 1 --2- 172.21.15.42:0/731157346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03fc0aaf00 0x7f03fc06edc0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:54.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.266+0000 7f0403600640 1 -- 172.21.15.42:0/731157346 >> 172.21.15.42:0/731157346 conn(0x7f03fc071d80 msgr2=0x7f03fc073cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:54.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.266+0000 7f0403600640 1 -- 172.21.15.42:0/731157346 shutdown_connections 2024-07-04T17:25:54.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.266+0000 7f0403600640 1 -- 172.21.15.42:0/731157346 wait complete. 2024-07-04T17:25:54.291 INFO:tasks.ceph.osd.0:Restarting daemon 2024-07-04T17:25:54.291 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-07-04T17:25:54.298 INFO:tasks.ceph.osd.0:Started 2024-07-04T17:25:54.298 INFO:tasks.ceph.osd.1:Restarting daemon 2024-07-04T17:25:54.298 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-07-04T17:25:54.305 INFO:tasks.ceph.osd.1:Started 2024-07-04T17:25:54.305 INFO:tasks.ceph.osd.2:Restarting daemon 2024-07-04T17:25:54.305 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-07-04T17:25:54.311 INFO:tasks.ceph.osd.2:Started 2024-07-04T17:25:54.311 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:25:54.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.593+0000 7fbd6fe00640 1 -- 172.21.15.42:0/2057171744 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd680b38b0 msgr2=0x7fbd680b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:54.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.593+0000 7fbd6fe00640 1 --2- 172.21.15.42:0/2057171744 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd680b38b0 0x7fbd680b2d20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c005fd0 tx=0x7fbd5c02a5f0 comp rx=0 tx=0).stop 2024-07-04T17:25:54.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.593+0000 7fbd6fe00640 1 -- 172.21.15.42:0/2057171744 shutdown_connections 2024-07-04T17:25:54.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.593+0000 7fbd6fe00640 1 --2- 172.21.15.42:0/2057171744 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd680b38b0 0x7fbd680b2d20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:54.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.593+0000 7fbd6fe00640 1 -- 172.21.15.42:0/2057171744 >> 172.21.15.42:0/2057171744 conn(0x7fbd680bf700 msgr2=0x7fbd680af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:54.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.593+0000 7fbd6fe00640 1 -- 172.21.15.42:0/2057171744 shutdown_connections 2024-07-04T17:25:54.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.593+0000 7fbd6fe00640 1 -- 172.21.15.42:0/2057171744 wait complete. 2024-07-04T17:25:54.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.596+0000 7fbd6fe00640 1 Processor -- start 2024-07-04T17:25:54.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.597+0000 7fbd6fe00640 1 -- start start 2024-07-04T17:25:54.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.599+0000 7fbd6fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd680b1e00 0x7fbd680ad7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:25:54.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.599+0000 7fbd6fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbd680b06e0 con 0x7fbd680b1e00 2024-07-04T17:25:54.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.599+0000 7fbd6d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd680b1e00 0x7fbd680ad7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:25:54.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.599+0000 7fbd6d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd680b1e00 0x7fbd680ad7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35392/0 (socket says 172.21.15.42:35392) 2024-07-04T17:25:54.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.599+0000 7fbd6d400640 1 -- 172.21.15.42:0/2413728397 learned_addr learned my addr 172.21.15.42:0/2413728397 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:25:54.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.599+0000 7fbd6d400640 1 -- 172.21.15.42:0/2413728397 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd680add70 con 0x7fbd680b1e00 2024-07-04T17:25:54.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.600+0000 7fbd6d400640 1 --2- 172.21.15.42:0/2413728397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd680b1e00 0x7fbd680ad7e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c009850 tx=0x7fbd5c0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:25:54.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.600+0000 7fbd65600640 1 -- 172.21.15.42:0/2413728397 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd5c03e030 con 0x7fbd680b1e00 2024-07-04T17:25:54.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.600+0000 7fbd65600640 1 -- 172.21.15.42:0/2413728397 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd5c0035f0 con 0x7fbd680b1e00 2024-07-04T17:25:54.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.600+0000 7fbd65600640 1 -- 172.21.15.42:0/2413728397 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd5c035040 con 0x7fbd680b1e00 2024-07-04T17:25:54.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.600+0000 7fbd6fe00640 1 -- 172.21.15.42:0/2413728397 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd680aecd0 con 0x7fbd680b1e00 2024-07-04T17:25:54.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.600+0000 7fbd6fe00640 1 -- 172.21.15.42:0/2413728397 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd6813f2f0 con 0x7fbd680b1e00 2024-07-04T17:25:54.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.601+0000 7fbd65600640 1 -- 172.21.15.42:0/2413728397 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbd5c041030 con 0x7fbd680b1e00 2024-07-04T17:25:54.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.601+0000 7fbd65600640 1 -- 172.21.15.42:0/2413728397 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fbd5c03f070 con 0x7fbd680b1e00 2024-07-04T17:25:54.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.602+0000 7fbd6fe00640 1 -- 172.21.15.42:0/2413728397 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd6806c590 con 0x7fbd680b1e00 2024-07-04T17:25:54.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.604+0000 7fbd65600640 1 -- 172.21.15.42:0/2413728397 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73745 (secure 0 0 0) 0x7fbd5c003ca0 con 0x7fbd680b1e00 2024-07-04T17:25:54.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.675+0000 7fbd6fe00640 1 -- 172.21.15.42:0/2413728397 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbd68069b70 con 0x7fbd680b1e00 2024-07-04T17:25:54.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.676+0000 7fbd65600640 1 -- 172.21.15.42:0/2413728397 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v5) ==== 74+0+3427 (secure 0 0 0) 0x7fbd5c043410 con 0x7fbd680b1e00 2024-07-04T17:25:54.677 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:25:54.677 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":5,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:25:54.262511+0000","last_up_change":"0.000000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:25:54.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.677+0000 7fbd6fe00640 1 -- 172.21.15.42:0/2413728397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd680b1e00 msgr2=0x7fbd680ad7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:54.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.677+0000 7fbd6fe00640 1 --2- 172.21.15.42:0/2413728397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd680b1e00 0x7fbd680ad7e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c009850 tx=0x7fbd5c0049d0 comp rx=0 tx=0).stop 2024-07-04T17:25:54.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.678+0000 7fbd6fe00640 1 -- 172.21.15.42:0/2413728397 shutdown_connections 2024-07-04T17:25:54.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.678+0000 7fbd6fe00640 1 --2- 172.21.15.42:0/2413728397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd680b1e00 0x7fbd680ad7e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:54.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.678+0000 7fbd6fe00640 1 -- 172.21.15.42:0/2413728397 >> 172.21.15.42:0/2413728397 conn(0x7fbd680bf700 msgr2=0x7fbd680bd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:54.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.678+0000 7fbd6fe00640 1 -- 172.21.15.42:0/2413728397 shutdown_connections 2024-07-04T17:25:54.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:25:54.678+0000 7fbd6fe00640 1 -- 172.21.15.42:0/2413728397 wait complete. 2024-07-04T17:25:54.701 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-07-04T17:25:54.701 INFO:tasks.ceph:Waiting for OSDs to come up 2024-07-04T17:25:55.003 DEBUG:teuthology.orchestra.run.smithi042:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-07-04T17:25:55.055 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-07-04T17:25:55.054+0000 7f80e600d640 -1 Falling back to public interface 2024-07-04T17:25:55.066 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-07-04T17:25:55.066+0000 7f261800e640 -1 Falling back to public interface 2024-07-04T17:25:55.133 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-07-04T17:25:55.133+0000 7fbb18ba3640 -1 Falling back to public interface 2024-07-04T17:25:55.252 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.251+0000 7f4971200640 1 -- 172.21.15.42:0/1329252080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f496c071a50 msgr2=0x7f496c0bbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:55.252 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.251+0000 7f4971200640 1 --2- 172.21.15.42:0/1329252080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f496c071a50 0x7f496c0bbcd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f495c005fd0 tx=0x7f495c02a5f0 comp rx=0 tx=0).stop 2024-07-04T17:25:55.252 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.251+0000 7f4971200640 1 -- 172.21.15.42:0/1329252080 shutdown_connections 2024-07-04T17:25:55.252 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.251+0000 7f4971200640 1 --2- 172.21.15.42:0/1329252080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f496c071a50 0x7f496c0bbcd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:55.252 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.251+0000 7f4971200640 1 -- 172.21.15.42:0/1329252080 >> 172.21.15.42:0/1329252080 conn(0x7f496c06c200 msgr2=0x7f496c06c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:55.252 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.251+0000 7f4971200640 1 -- 172.21.15.42:0/1329252080 shutdown_connections 2024-07-04T17:25:55.252 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.251+0000 7f4971200640 1 -- 172.21.15.42:0/1329252080 wait complete. 2024-07-04T17:25:55.253 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.253+0000 7f4971200640 1 Processor -- start 2024-07-04T17:25:55.253 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.253+0000 7f4971200640 1 -- start start 2024-07-04T17:25:55.255 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.254+0000 7f4971200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f496c071a50 0x7f496c0b66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:25:55.255 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.254+0000 7f4971200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f496c14b6b0 con 0x7f496c071a50 2024-07-04T17:25:55.255 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.254+0000 7f496b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f496c071a50 0x7f496c0b66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:25:55.255 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.254+0000 7f496b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f496c071a50 0x7f496c0b66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35410/0 (socket says 172.21.15.42:35410) 2024-07-04T17:25:55.255 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.254+0000 7f496b400640 1 -- 172.21.15.42:0/58494097 learned_addr learned my addr 172.21.15.42:0/58494097 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:25:55.256 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.255+0000 7f496b400640 1 -- 172.21.15.42:0/58494097 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f496c13c8d0 con 0x7f496c071a50 2024-07-04T17:25:55.257 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.255+0000 7f496b400640 1 --2- 172.21.15.42:0/58494097 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f496c071a50 0x7f496c0b66d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f495c009850 tx=0x7f495c0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:25:55.257 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.255+0000 7f4963400640 1 -- 172.21.15.42:0/58494097 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f495c03e030 con 0x7f496c071a50 2024-07-04T17:25:55.257 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.255+0000 7f4963400640 1 -- 172.21.15.42:0/58494097 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f495c0035f0 con 0x7f496c071a50 2024-07-04T17:25:55.257 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.255+0000 7f4963400640 1 -- 172.21.15.42:0/58494097 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f495c035040 con 0x7f496c071a50 2024-07-04T17:25:55.257 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.255+0000 7f4971200640 1 -- 172.21.15.42:0/58494097 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f496c0b6cc0 con 0x7f496c071a50 2024-07-04T17:25:55.257 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.256+0000 7f4971200640 1 -- 172.21.15.42:0/58494097 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f496c0b7340 con 0x7f496c071a50 2024-07-04T17:25:55.257 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.256+0000 7f4963400640 1 -- 172.21.15.42:0/58494097 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f495c041030 con 0x7f496c071a50 2024-07-04T17:25:55.257 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.256+0000 7f4963400640 1 -- 172.21.15.42:0/58494097 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f495c03f070 con 0x7f496c071a50 2024-07-04T17:25:55.258 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.257+0000 7f4971200640 1 -- 172.21.15.42:0/58494097 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f496c0bc720 con 0x7f496c071a50 2024-07-04T17:25:55.260 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.259+0000 7f4963400640 1 -- 172.21.15.42:0/58494097 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73745 (secure 0 0 0) 0x7f496c0bc720 con 0x7f496c071a50 2024-07-04T17:25:55.327 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.327+0000 7f4971200640 1 -- 172.21.15.42:0/58494097 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f496c14bd90 con 0x7f496c071a50 2024-07-04T17:25:55.328 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.327+0000 7f4963400640 1 -- 172.21.15.42:0/58494097 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v5) ==== 74+0+3427 (secure 0 0 0) 0x7f495c003ca0 con 0x7f496c071a50 2024-07-04T17:25:55.328 INFO:teuthology.misc.health.smithi042.stdout: 2024-07-04T17:25:55.328 INFO:teuthology.misc.health.smithi042.stdout:{"epoch":5,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:25:54.262511+0000","last_up_change":"0.000000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:25:55.330 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.329+0000 7f4971200640 1 -- 172.21.15.42:0/58494097 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f496c071a50 msgr2=0x7f496c0b66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:25:55.330 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.329+0000 7f4971200640 1 --2- 172.21.15.42:0/58494097 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f496c071a50 0x7f496c0b66d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f495c009850 tx=0x7f495c0049d0 comp rx=0 tx=0).stop 2024-07-04T17:25:55.330 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.329+0000 7f4971200640 1 -- 172.21.15.42:0/58494097 shutdown_connections 2024-07-04T17:25:55.330 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.329+0000 7f4971200640 1 --2- 172.21.15.42:0/58494097 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f496c071a50 0x7f496c0b66d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:25:55.330 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.329+0000 7f4971200640 1 -- 172.21.15.42:0/58494097 >> 172.21.15.42:0/58494097 conn(0x7f496c06c200 msgr2=0x7f496c0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:25:55.330 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.329+0000 7f4971200640 1 -- 172.21.15.42:0/58494097 shutdown_connections 2024-07-04T17:25:55.330 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:25:55.329+0000 7f4971200640 1 -- 172.21.15.42:0/58494097 wait complete. 2024-07-04T17:25:55.360 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-07-04T17:25:58.264 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:25:58.263+0000 7f0a95e00640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-07-04T17:25:58.852 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-07-04T17:25:58.851+0000 7f80e600d640 -1 osd.0 0 log_to_monitors true 2024-07-04T17:25:58.919 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-07-04T17:25:58.919+0000 7f261800e640 -1 osd.2 0 log_to_monitors true 2024-07-04T17:25:58.989 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-07-04T17:25:58.988+0000 7fbb18ba3640 -1 osd.1 0 log_to_monitors true 2024-07-04T17:25:59.273 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-04T17:26:01.663 DEBUG:teuthology.orchestra.run.smithi042:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-07-04T17:26:01.930 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-07-04T17:26:01.929+0000 7fbb13a00640 -1 osd.1 0 waiting for initial osdmap 2024-07-04T17:26:01.937 INFO:tasks.ceph.osd.1.smithi042.stderr:2024-07-04T17:26:01.937+0000 7fbb09a00640 -1 osd.1 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-04T17:26:01.947 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-07-04T17:26:01.946+0000 7f2612800640 -1 osd.2 0 waiting for initial osdmap 2024-07-04T17:26:01.958 INFO:tasks.ceph.osd.2.smithi042.stderr:2024-07-04T17:26:01.958+0000 7f2608800640 -1 osd.2 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-04T17:26:01.977 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-07-04T17:26:01.976+0000 7f80e0a00640 -1 osd.0 0 waiting for initial osdmap 2024-07-04T17:26:01.996 INFO:tasks.ceph.osd.0.smithi042.stderr:2024-07-04T17:26:01.996+0000 7f80d6a00640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-04T17:26:02.003 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.002+0000 7fef7dc00640 1 -- 172.21.15.42:0/3105097421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef7806cf50 msgr2=0x7fef780b02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:02.003 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.002+0000 7fef7dc00640 1 --2- 172.21.15.42:0/3105097421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef7806cf50 0x7fef780b02b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fef68005fd0 tx=0x7fef6802ae80 comp rx=0 tx=0).stop 2024-07-04T17:26:02.004 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.003+0000 7fef7dc00640 1 -- 172.21.15.42:0/3105097421 shutdown_connections 2024-07-04T17:26:02.004 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.003+0000 7fef7dc00640 1 --2- 172.21.15.42:0/3105097421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef7806cf50 0x7fef780b02b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:02.004 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.003+0000 7fef7dc00640 1 -- 172.21.15.42:0/3105097421 >> 172.21.15.42:0/3105097421 conn(0x7fef780bf700 msgr2=0x7fef780af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:02.004 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.003+0000 7fef7dc00640 1 -- 172.21.15.42:0/3105097421 shutdown_connections 2024-07-04T17:26:02.004 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.003+0000 7fef7dc00640 1 -- 172.21.15.42:0/3105097421 wait complete. 2024-07-04T17:26:02.005 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.005+0000 7fef7dc00640 1 Processor -- start 2024-07-04T17:26:02.005 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.005+0000 7fef7dc00640 1 -- start start 2024-07-04T17:26:02.006 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.006+0000 7fef7dc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef7806cf50 0x7fef781d5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:02.006 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.006+0000 7fef7dc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fef7814d130 con 0x7fef7806cf50 2024-07-04T17:26:02.007 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.006+0000 7fef76a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef7806cf50 0x7fef781d5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:02.007 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.006+0000 7fef76a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef7806cf50 0x7fef781d5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35544/0 (socket says 172.21.15.42:35544) 2024-07-04T17:26:02.007 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.006+0000 7fef76a00640 1 -- 172.21.15.42:0/1949038777 learned_addr learned my addr 172.21.15.42:0/1949038777 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:02.007 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.007+0000 7fef76a00640 1 -- 172.21.15.42:0/1949038777 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef78071a00 con 0x7fef7806cf50 2024-07-04T17:26:02.008 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.007+0000 7fef76a00640 1 --2- 172.21.15.42:0/1949038777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef7806cf50 0x7fef781d5660 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fef6800b3c0 tx=0x7fef6802cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:02.008 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.007+0000 7fef6ea00640 1 -- 172.21.15.42:0/1949038777 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef68041020 con 0x7fef7806cf50 2024-07-04T17:26:02.008 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.007+0000 7fef6ea00640 1 -- 172.21.15.42:0/1949038777 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef680095a0 con 0x7fef7806cf50 2024-07-04T17:26:02.008 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.007+0000 7fef6ea00640 1 -- 172.21.15.42:0/1949038777 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef68009920 con 0x7fef7806cf50 2024-07-04T17:26:02.008 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.007+0000 7fef7dc00640 1 -- 172.21.15.42:0/1949038777 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef781469a0 con 0x7fef7806cf50 2024-07-04T17:26:02.009 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.007+0000 7fef7dc00640 1 -- 172.21.15.42:0/1949038777 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef78144f00 con 0x7fef7806cf50 2024-07-04T17:26:02.009 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.009+0000 7fef6ea00640 1 -- 172.21.15.42:0/1949038777 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 4) ==== 49355+0+0 (secure 0 0 0) 0x7fef68044020 con 0x7fef7806cf50 2024-07-04T17:26:02.009 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.009+0000 7fef7dc00640 1 -- 172.21.15.42:0/1949038777 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef78146b70 con 0x7fef7806cf50 2024-07-04T17:26:02.009 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.009+0000 7fef6ea00640 1 --2- 172.21.15.42:0/1949038777 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fef5803d220 0x7fef5803f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:02.010 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.009+0000 7fef6ea00640 1 -- 172.21.15.42:0/1949038777 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1361+0+0 (secure 0 0 0) 0x7fef68039040 con 0x7fef7806cf50 2024-07-04T17:26:02.010 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.009+0000 7fef76000640 1 --2- 172.21.15.42:0/1949038777 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fef5803d220 0x7fef5803f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:02.010 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.010+0000 7fef76000640 1 --2- 172.21.15.42:0/1949038777 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fef5803d220 0x7fef5803f730 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fef64001560 tx=0x7fef640079f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:02.015 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.015+0000 7fef6ea00640 1 -- 172.21.15.42:0/1949038777 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fef6804b030 con 0x7fef7806cf50 2024-07-04T17:26:02.218 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.217+0000 7fef7dc00640 1 -- 172.21.15.42:0/1949038777 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef780b02b0 con 0x7fef7806cf50 2024-07-04T17:26:02.219 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.219+0000 7fef6ea00640 1 -- 172.21.15.42:0/1949038777 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v7) ==== 74+0+3427 (secure 0 0 0) 0x7fef68030e80 con 0x7fef7806cf50 2024-07-04T17:26:02.219 INFO:teuthology.misc.health.smithi042.stdout: 2024-07-04T17:26:02.219 INFO:teuthology.misc.health.smithi042.stdout:{"epoch":7,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:26:00.297997+0000","last_up_change":"0.000000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:26:02.222 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.221+0000 7fef7dc00640 1 -- 172.21.15.42:0/1949038777 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fef5803d220 msgr2=0x7fef5803f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:02.222 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.221+0000 7fef7dc00640 1 --2- 172.21.15.42:0/1949038777 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fef5803d220 0x7fef5803f730 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fef64001560 tx=0x7fef640079f0 comp rx=0 tx=0).stop 2024-07-04T17:26:02.222 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.221+0000 7fef7dc00640 1 -- 172.21.15.42:0/1949038777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef7806cf50 msgr2=0x7fef781d5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:02.222 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.221+0000 7fef7dc00640 1 --2- 172.21.15.42:0/1949038777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef7806cf50 0x7fef781d5660 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fef6800b3c0 tx=0x7fef6802cd70 comp rx=0 tx=0).stop 2024-07-04T17:26:02.222 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.221+0000 7fef7dc00640 1 -- 172.21.15.42:0/1949038777 shutdown_connections 2024-07-04T17:26:02.222 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.221+0000 7fef7dc00640 1 --2- 172.21.15.42:0/1949038777 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fef5803d220 0x7fef5803f730 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:02.223 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.221+0000 7fef7dc00640 1 --2- 172.21.15.42:0/1949038777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef7806cf50 0x7fef781d5660 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:02.223 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.221+0000 7fef7dc00640 1 -- 172.21.15.42:0/1949038777 >> 172.21.15.42:0/1949038777 conn(0x7fef780bf700 msgr2=0x7fef780be7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:02.223 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.221+0000 7fef7dc00640 1 -- 172.21.15.42:0/1949038777 shutdown_connections 2024-07-04T17:26:02.223 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:02.221+0000 7fef7dc00640 1 -- 172.21.15.42:0/1949038777 wait complete. 2024-07-04T17:26:02.243 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-07-04T17:26:08.545 DEBUG:teuthology.orchestra.run.smithi042:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-07-04T17:26:08.818 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:08.816+0000 7f2a70200640 1 -- 172.21.15.42:0/4030855032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a680b0cd0 msgr2=0x7f2a680b2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:08.819 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:08.816+0000 7f2a70200640 1 --2- 172.21.15.42:0/4030855032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a680b0cd0 0x7f2a680b2c00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c00a8c0 tx=0x7f2a5c02b860 comp rx=0 tx=0).stop 2024-07-04T17:26:08.819 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:08.816+0000 7f2a70200640 1 -- 172.21.15.42:0/4030855032 shutdown_connections 2024-07-04T17:26:08.819 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:08.816+0000 7f2a70200640 1 --2- 172.21.15.42:0/4030855032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a680b0cd0 0x7f2a680b2c00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:08.819 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:08.816+0000 7f2a70200640 1 -- 172.21.15.42:0/4030855032 >> 172.21.15.42:0/4030855032 conn(0x7f2a680bf700 msgr2=0x7f2a680af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:08.819 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:08.817+0000 7f2a70200640 1 -- 172.21.15.42:0/4030855032 shutdown_connections 2024-07-04T17:26:08.819 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:08.817+0000 7f2a70200640 1 -- 172.21.15.42:0/4030855032 wait complete. 2024-07-04T17:26:08.821 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:08.819+0000 7f2a70200640 1 Processor -- start 2024-07-04T17:26:08.821 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:08.820+0000 7f2a70200640 1 -- start start 2024-07-04T17:26:08.823 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:08.821+0000 7f2a70200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a680b0cd0 0x7f2a681d4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:08.823 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:08.821+0000 7f2a70200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2a680b0600 con 0x7f2a680b0cd0 2024-07-04T17:26:08.823 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:08.822+0000 7f2a6d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a680b0cd0 0x7f2a681d4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:08.823 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:08.822+0000 7f2a6d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a680b0cd0 0x7f2a681d4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38942/0 (socket says 172.21.15.42:38942) 2024-07-04T17:26:08.823 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:08.822+0000 7f2a6d800640 1 -- 172.21.15.42:0/1855796162 learned_addr learned my addr 172.21.15.42:0/1855796162 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:08.824 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:08.822+0000 7f2a6d800640 1 -- 172.21.15.42:0/1855796162 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a681d6620 con 0x7f2a680b0cd0 2024-07-04T17:26:08.824 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:08.822+0000 7f2a6d800640 1 --2- 172.21.15.42:0/1855796162 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a680b0cd0 0x7f2a681d4890 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c034040 tx=0x7f2a5c031c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:08.825 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:08.823+0000 7f2a65600640 1 -- 172.21.15.42:0/1855796162 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a5c003720 con 0x7f2a680b0cd0 2024-07-04T17:26:08.825 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:08.823+0000 7f2a65600640 1 -- 172.21.15.42:0/1855796162 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a5c0038c0 con 0x7f2a680b0cd0 2024-07-04T17:26:08.825 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:08.823+0000 7f2a65600640 1 -- 172.21.15.42:0/1855796162 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a5c003c20 con 0x7f2a680b0cd0 2024-07-04T17:26:08.825 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:08.823+0000 7f2a70200640 1 -- 172.21.15.42:0/1855796162 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a681d6b40 con 0x7f2a680b0cd0 2024-07-04T17:26:08.825 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:08.823+0000 7f2a70200640 1 -- 172.21.15.42:0/1855796162 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a681484f0 con 0x7f2a680b0cd0 2024-07-04T17:26:08.825 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:08.824+0000 7f2a70200640 1 -- 172.21.15.42:0/1855796162 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a680b2c00 con 0x7f2a680b0cd0 2024-07-04T17:26:08.825 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:08.824+0000 7f2a65600640 1 -- 172.21.15.42:0/1855796162 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f2a5c04e440 con 0x7f2a680b0cd0 2024-07-04T17:26:08.826 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:08.825+0000 7f2a65600640 1 --2- 172.21.15.42:0/1855796162 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f2a4803d300 0x7f2a4803f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:08.826 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:08.825+0000 7f2a65600640 1 -- 172.21.15.42:0/1855796162 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 2587+0+0 (secure 0 0 0) 0x7f2a5c072820 con 0x7f2a680b0cd0 2024-07-04T17:26:08.827 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:08.825+0000 7f2a6ce00640 1 --2- 172.21.15.42:0/1855796162 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f2a4803d300 0x7f2a4803f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:08.827 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:08.826+0000 7f2a6ce00640 1 --2- 172.21.15.42:0/1855796162 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f2a4803d300 0x7f2a4803f810 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2a540015c0 tx=0x7f2a54007a90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:08.832 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:08.831+0000 7f2a65600640 1 -- 172.21.15.42:0/1855796162 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f2a5c03ae20 con 0x7f2a680b0cd0 2024-07-04T17:26:09.042 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:09.041+0000 7f2a70200640 1 -- 172.21.15.42:0/1855796162 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a6814ab60 con 0x7f2a680b0cd0 2024-07-04T17:26:09.044 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:09.042+0000 7f2a65600640 1 -- 172.21.15.42:0/1855796162 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) ==== 74+0+6766 (secure 0 0 0) 0x7f2a5c003dc0 con 0x7f2a680b0cd0 2024-07-04T17:26:09.044 INFO:teuthology.misc.health.smithi042.stdout: 2024-07-04T17:26:09.044 INFO:teuthology.misc.health.smithi042.stdout:{"epoch":11,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:26:05.946381+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:26:09.047 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:09.045+0000 7f2a70200640 1 -- 172.21.15.42:0/1855796162 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f2a4803d300 msgr2=0x7f2a4803f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:09.047 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:09.045+0000 7f2a70200640 1 --2- 172.21.15.42:0/1855796162 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f2a4803d300 0x7f2a4803f810 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2a540015c0 tx=0x7f2a54007a90 comp rx=0 tx=0).stop 2024-07-04T17:26:09.047 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:09.045+0000 7f2a70200640 1 -- 172.21.15.42:0/1855796162 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a680b0cd0 msgr2=0x7f2a681d4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:09.047 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:09.045+0000 7f2a70200640 1 --2- 172.21.15.42:0/1855796162 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a680b0cd0 0x7f2a681d4890 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c034040 tx=0x7f2a5c031c90 comp rx=0 tx=0).stop 2024-07-04T17:26:09.047 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:09.045+0000 7f2a70200640 1 -- 172.21.15.42:0/1855796162 shutdown_connections 2024-07-04T17:26:09.047 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:09.045+0000 7f2a70200640 1 --2- 172.21.15.42:0/1855796162 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f2a4803d300 0x7f2a4803f810 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:09.047 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:09.045+0000 7f2a70200640 1 --2- 172.21.15.42:0/1855796162 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a680b0cd0 0x7f2a681d4890 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:09.047 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:09.045+0000 7f2a70200640 1 -- 172.21.15.42:0/1855796162 >> 172.21.15.42:0/1855796162 conn(0x7f2a680bf700 msgr2=0x7f2a680a9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:09.047 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:09.045+0000 7f2a70200640 1 -- 172.21.15.42:0/1855796162 shutdown_connections 2024-07-04T17:26:09.047 INFO:teuthology.misc.health.smithi042.stderr:2024-07-04T17:26:09.045+0000 7f2a70200640 1 -- 172.21.15.42:0/1855796162 wait complete. 2024-07-04T17:26:09.068 DEBUG:teuthology.misc:3 of 3 OSDs are up 2024-07-04T17:26:09.068 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-07-04T17:26:09.068 INFO:tasks.ceph.mds.a:Restarting daemon 2024-07-04T17:26:09.069 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-07-04T17:26:09.075 INFO:tasks.ceph.mds.a:Started 2024-07-04T17:26:09.075 INFO:tasks.ceph.mds.b:Restarting daemon 2024-07-04T17:26:09.075 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-07-04T17:26:09.082 INFO:tasks.ceph.mds.b:Started 2024-07-04T17:26:09.082 INFO:tasks.ceph.mds.c:Restarting daemon 2024-07-04T17:26:09.082 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-07-04T17:26:09.088 INFO:tasks.ceph.mds.c:Started 2024-07-04T17:26:09.088 INFO:tasks.ceph.mds.d:Restarting daemon 2024-07-04T17:26:09.088 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-07-04T17:26:09.093 INFO:tasks.ceph.mds.d:Started 2024-07-04T17:26:09.093 INFO:tasks.ceph.mds.e:Restarting daemon 2024-07-04T17:26:09.093 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-07-04T17:26:09.099 INFO:tasks.ceph.mds.e:Started 2024-07-04T17:26:09.099 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-07-04T17:26:09.100 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:26:09.353 INFO:tasks.ceph.mds.e.smithi042.stdout:starting mds.e at 2024-07-04T17:26:09.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.392+0000 7fc99c600640 1 -- 172.21.15.42:0/2381933712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc99406cea0 msgr2=0x7fc9940b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:09.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.392+0000 7fc99c600640 1 --2- 172.21.15.42:0/2381933712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc99406cea0 0x7fc9940b3030 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc9880071c0 tx=0x7fc98802b170 comp rx=0 tx=0).stop 2024-07-04T17:26:09.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.392+0000 7fc99c600640 1 -- 172.21.15.42:0/2381933712 shutdown_connections 2024-07-04T17:26:09.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.393+0000 7fc99c600640 1 --2- 172.21.15.42:0/2381933712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc99406cea0 0x7fc9940b3030 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:09.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.393+0000 7fc99c600640 1 -- 172.21.15.42:0/2381933712 >> 172.21.15.42:0/2381933712 conn(0x7fc9940bf700 msgr2=0x7fc9940af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:09.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.393+0000 7fc99c600640 1 -- 172.21.15.42:0/2381933712 shutdown_connections 2024-07-04T17:26:09.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.393+0000 7fc99c600640 1 -- 172.21.15.42:0/2381933712 wait complete. 2024-07-04T17:26:09.396 INFO:tasks.ceph.mds.b.smithi042.stdout:starting mds.b at 2024-07-04T17:26:09.396 INFO:tasks.ceph.mds.d.smithi042.stdout:starting mds.d at 2024-07-04T17:26:09.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.395+0000 7fc99c600640 1 Processor -- start 2024-07-04T17:26:09.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.396+0000 7fc99c600640 1 -- start start 2024-07-04T17:26:09.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.397+0000 7fc99c600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc99406cea0 0x7fc994146f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:09.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.397+0000 7fc99c600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc99406ef50 con 0x7fc99406cea0 2024-07-04T17:26:09.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.398+0000 7fc999c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc99406cea0 0x7fc994146f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:09.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.398+0000 7fc999c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc99406cea0 0x7fc994146f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38994/0 (socket says 172.21.15.42:38994) 2024-07-04T17:26:09.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.398+0000 7fc999c00640 1 -- 172.21.15.42:0/1221685522 learned_addr learned my addr 172.21.15.42:0/1221685522 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:09.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.398+0000 7fc999c00640 1 -- 172.21.15.42:0/1221685522 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc994144ed0 con 0x7fc99406cea0 2024-07-04T17:26:09.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.399+0000 7fc999c00640 1 --2- 172.21.15.42:0/1221685522 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc99406cea0 0x7fc994146f30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc988035040 tx=0x7fc988009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:09.401 INFO:tasks.ceph.mds.b.smithi042.stderr:2024-07-04T17:26:09.399+0000 7fe3c6e00640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6830/0: (98) Address already in use 2024-07-04T17:26:09.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.400+0000 7fc991600640 1 -- 172.21.15.42:0/1221685522 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc988003800 con 0x7fc99406cea0 2024-07-04T17:26:09.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.400+0000 7fc991600640 1 -- 172.21.15.42:0/1221685522 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc988003a90 con 0x7fc99406cea0 2024-07-04T17:26:09.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.400+0000 7fc991600640 1 -- 172.21.15.42:0/1221685522 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc988003e30 con 0x7fc99406cea0 2024-07-04T17:26:09.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.400+0000 7fc99c600640 1 -- 172.21.15.42:0/1221685522 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9941f0c20 con 0x7fc99406cea0 2024-07-04T17:26:09.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.400+0000 7fc99c600640 1 -- 172.21.15.42:0/1221685522 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9941429e0 con 0x7fc99406cea0 2024-07-04T17:26:09.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.403+0000 7fc99c600640 1 -- 172.21.15.42:0/1221685522 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9940b39f0 con 0x7fc99406cea0 2024-07-04T17:26:09.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.403+0000 7fc991600640 1 -- 172.21.15.42:0/1221685522 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fc988044060 con 0x7fc99406cea0 2024-07-04T17:26:09.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.403+0000 7fc991600640 1 --2- 172.21.15.42:0/1221685522 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fc97403d350 0x7fc97403f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:09.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.403+0000 7fc991600640 1 -- 172.21.15.42:0/1221685522 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 2587+0+0 (secure 0 0 0) 0x7fc988071710 con 0x7fc99406cea0 2024-07-04T17:26:09.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.404+0000 7fc999200640 1 --2- 172.21.15.42:0/1221685522 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fc97403d350 0x7fc97403f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:09.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.404+0000 7fc999200640 1 --2- 172.21.15.42:0/1221685522 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fc97403d350 0x7fc97403f860 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc980001560 tx=0x7fc980007a90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:09.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.409+0000 7fc991600640 1 -- 172.21.15.42:0/1221685522 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fc9940b39f0 con 0x7fc99406cea0 2024-07-04T17:26:09.413 INFO:tasks.ceph.mds.a.smithi042.stdout:starting mds.a at 2024-07-04T17:26:09.413 INFO:tasks.ceph.mds.c.smithi042.stdout:starting mds.c at 2024-07-04T17:26:09.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.658+0000 7fc99c600640 1 -- 172.21.15.42:0/1221685522 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9940b3030 con 0x7fc99406cea0 2024-07-04T17:26:09.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.659+0000 7fc991600640 1 -- 172.21.15.42:0/1221685522 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1434 (secure 0 0 0) 0x7fc988037df0 con 0x7fc99406cea0 2024-07-04T17:26:09.661 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:26:09.661 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":2,"btime":"2024-07-04T17:26:09:361752+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4192,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/3484212194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":3484212194},{"type":"v1","addr":"172.21.15.42:6827","nonce":3484212194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-07-04T17:26:09.661 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 2 2024-07-04T17:26:09.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.662+0000 7fc99c600640 1 -- 172.21.15.42:0/1221685522 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fc97403d350 msgr2=0x7fc97403f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:09.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.662+0000 7fc99c600640 1 --2- 172.21.15.42:0/1221685522 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fc97403d350 0x7fc97403f860 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc980001560 tx=0x7fc980007a90 comp rx=0 tx=0).stop 2024-07-04T17:26:09.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.662+0000 7fc99c600640 1 -- 172.21.15.42:0/1221685522 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc99406cea0 msgr2=0x7fc994146f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:09.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.662+0000 7fc99c600640 1 --2- 172.21.15.42:0/1221685522 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc99406cea0 0x7fc994146f30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc988035040 tx=0x7fc988009670 comp rx=0 tx=0).stop 2024-07-04T17:26:09.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.662+0000 7fc99c600640 1 -- 172.21.15.42:0/1221685522 shutdown_connections 2024-07-04T17:26:09.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.662+0000 7fc99c600640 1 --2- 172.21.15.42:0/1221685522 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fc97403d350 0x7fc97403f860 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:09.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.662+0000 7fc99c600640 1 --2- 172.21.15.42:0/1221685522 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc99406cea0 0x7fc994146f30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:09.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.662+0000 7fc99c600640 1 -- 172.21.15.42:0/1221685522 >> 172.21.15.42:0/1221685522 conn(0x7fc9940bf700 msgr2=0x7fc994140310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:09.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.662+0000 7fc99c600640 1 -- 172.21.15.42:0/1221685522 shutdown_connections 2024-07-04T17:26:09.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:09.662+0000 7fc99c600640 1 -- 172.21.15.42:0/1221685522 wait complete. 2024-07-04T17:26:11.688 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:26:11.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:11.980+0000 7ffbe6600640 1 -- 172.21.15.42:0/4213885816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbe00bad70 msgr2=0x7ffbe006c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:11.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:11.980+0000 7ffbe6600640 1 --2- 172.21.15.42:0/4213885816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbe00bad70 0x7ffbe006c2c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800a8c0 tx=0x7ffbc802b7d0 comp rx=0 tx=0).stop 2024-07-04T17:26:11.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:11.980+0000 7ffbe6600640 1 -- 172.21.15.42:0/4213885816 shutdown_connections 2024-07-04T17:26:11.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:11.980+0000 7ffbe6600640 1 --2- 172.21.15.42:0/4213885816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbe00bad70 0x7ffbe006c2c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:11.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:11.980+0000 7ffbe6600640 1 -- 172.21.15.42:0/4213885816 >> 172.21.15.42:0/4213885816 conn(0x7ffbe00bebc0 msgr2=0x7ffbe00befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:11.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:11.980+0000 7ffbe6600640 1 -- 172.21.15.42:0/4213885816 shutdown_connections 2024-07-04T17:26:11.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:11.980+0000 7ffbe6600640 1 -- 172.21.15.42:0/4213885816 wait complete. 2024-07-04T17:26:11.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:11.982+0000 7ffbe6600640 1 Processor -- start 2024-07-04T17:26:11.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:11.982+0000 7ffbe6600640 1 -- start start 2024-07-04T17:26:11.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:11.983+0000 7ffbe6600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbe006d580 0x7ffbe01d8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:11.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:11.983+0000 7ffbe6600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe00b2810 con 0x7ffbe006d580 2024-07-04T17:26:11.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:11.984+0000 7ffbe5200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbe006d580 0x7ffbe01d8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:11.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:11.984+0000 7ffbe5200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbe006d580 0x7ffbe01d8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39078/0 (socket says 172.21.15.42:39078) 2024-07-04T17:26:11.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:11.984+0000 7ffbe5200640 1 -- 172.21.15.42:0/243279823 learned_addr learned my addr 172.21.15.42:0/243279823 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:11.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:11.984+0000 7ffbe5200640 1 -- 172.21.15.42:0/243279823 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe01d32a0 con 0x7ffbe006d580 2024-07-04T17:26:11.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:11.984+0000 7ffbe5200640 1 --2- 172.21.15.42:0/243279823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbe006d580 0x7ffbe01d8140 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000c3b0 tx=0x7ffbd000c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:11.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:11.984+0000 7ffbdd600640 1 -- 172.21.15.42:0/243279823 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffbd0011780 con 0x7ffbe006d580 2024-07-04T17:26:11.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:11.985+0000 7ffbdd600640 1 -- 172.21.15.42:0/243279823 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffbd0017070 con 0x7ffbe006d580 2024-07-04T17:26:11.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:11.985+0000 7ffbdd600640 1 -- 172.21.15.42:0/243279823 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffbd0011c00 con 0x7ffbe006d580 2024-07-04T17:26:11.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:11.985+0000 7ffbe6600640 1 -- 172.21.15.42:0/243279823 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbe0071dc0 con 0x7ffbe006d580 2024-07-04T17:26:11.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:11.985+0000 7ffbe6600640 1 -- 172.21.15.42:0/243279823 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbe013b6d0 con 0x7ffbe006d580 2024-07-04T17:26:11.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:11.986+0000 7ffbe6600640 1 -- 172.21.15.42:0/243279823 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbe006c2c0 con 0x7ffbe006d580 2024-07-04T17:26:11.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:11.986+0000 7ffbdd600640 1 -- 172.21.15.42:0/243279823 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7ffbd001c460 con 0x7ffbe006d580 2024-07-04T17:26:11.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:11.986+0000 7ffbdd600640 1 --2- 172.21.15.42:0/243279823 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ffbbc03d2b0 0x7ffbbc03f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:11.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:11.986+0000 7ffbdd600640 1 -- 172.21.15.42:0/243279823 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 2587+0+0 (secure 0 0 0) 0x7ffbd0050fb0 con 0x7ffbe006d580 2024-07-04T17:26:11.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:11.987+0000 7ffbd7e00640 1 --2- 172.21.15.42:0/243279823 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ffbbc03d2b0 0x7ffbbc03f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:11.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:11.987+0000 7ffbd7e00640 1 --2- 172.21.15.42:0/243279823 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ffbbc03d2b0 0x7ffbbc03f7c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8036000 tx=0x7ffbc8004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:11.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:11.992+0000 7ffbdd600640 1 -- 172.21.15.42:0/243279823 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7ffbd0050200 con 0x7ffbe006d580 2024-07-04T17:26:12.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.230+0000 7ffbe6600640 1 -- 172.21.15.42:0/243279823 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffbe013b900 con 0x7ffbe006d580 2024-07-04T17:26:12.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.231+0000 7ffbdd600640 1 -- 172.21.15.42:0/243279823 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+4724 (secure 0 0 0) 0x7ffbd001c800 con 0x7ffbe006d580 2024-07-04T17:26:12.233 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:26:12.233 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":3,"btime":"2024-07-04T17:26:10:362834+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4192,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/3484212194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":3484212194},{"type":"v1","addr":"172.21.15.42:6827","nonce":3484212194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4202,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6830/815222782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6829","nonce":815222782},{"type":"v1","addr":"172.21.15.42:6830","nonce":815222782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/999331079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":999331079},{"type":"v1","addr":"172.21.15.42:6831","nonce":999331079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4204,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6833/2286778970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2286778970},{"type":"v1","addr":"172.21.15.42:6833","nonce":2286778970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4205,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/3941225191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3941225191},{"type":"v1","addr":"172.21.15.42:6835","nonce":3941225191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-07-04T17:26:12.233 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 3 2024-07-04T17:26:12.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.234+0000 7ffbe6600640 1 -- 172.21.15.42:0/243279823 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ffbbc03d2b0 msgr2=0x7ffbbc03f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:12.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.234+0000 7ffbe6600640 1 --2- 172.21.15.42:0/243279823 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ffbbc03d2b0 0x7ffbbc03f7c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8036000 tx=0x7ffbc8004730 comp rx=0 tx=0).stop 2024-07-04T17:26:12.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.234+0000 7ffbe6600640 1 -- 172.21.15.42:0/243279823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbe006d580 msgr2=0x7ffbe01d8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:12.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.234+0000 7ffbe6600640 1 --2- 172.21.15.42:0/243279823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbe006d580 0x7ffbe01d8140 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000c3b0 tx=0x7ffbd000c870 comp rx=0 tx=0).stop 2024-07-04T17:26:12.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.234+0000 7ffbe6600640 1 -- 172.21.15.42:0/243279823 shutdown_connections 2024-07-04T17:26:12.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.234+0000 7ffbe6600640 1 --2- 172.21.15.42:0/243279823 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ffbbc03d2b0 0x7ffbbc03f7c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:12.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.234+0000 7ffbe6600640 1 --2- 172.21.15.42:0/243279823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbe006d580 0x7ffbe01d8140 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:12.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.234+0000 7ffbe6600640 1 -- 172.21.15.42:0/243279823 >> 172.21.15.42:0/243279823 conn(0x7ffbe00bebc0 msgr2=0x7ffbe0140a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:12.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.234+0000 7ffbe6600640 1 -- 172.21.15.42:0/243279823 shutdown_connections 2024-07-04T17:26:12.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.235+0000 7ffbe6600640 1 -- 172.21.15.42:0/243279823 wait complete. 2024-07-04T17:26:12.261 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-04T17:26:12.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.546+0000 7f65f2a00640 1 -- 172.21.15.42:0/3521664363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f65ec0b3030 msgr2=0x7f65ec0b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:12.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.546+0000 7f65f2a00640 1 --2- 172.21.15.42:0/3521664363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f65ec0b3030 0x7f65ec0b01f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f65d400a8c0 tx=0x7f65d402b860 comp rx=0 tx=0).stop 2024-07-04T17:26:12.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.546+0000 7f65f2a00640 1 -- 172.21.15.42:0/3521664363 shutdown_connections 2024-07-04T17:26:12.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.546+0000 7f65f2a00640 1 --2- 172.21.15.42:0/3521664363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f65ec0b3030 0x7f65ec0b01f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:12.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.546+0000 7f65f2a00640 1 -- 172.21.15.42:0/3521664363 >> 172.21.15.42:0/3521664363 conn(0x7f65ec0bf700 msgr2=0x7f65ec0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:12.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.546+0000 7f65f2a00640 1 -- 172.21.15.42:0/3521664363 shutdown_connections 2024-07-04T17:26:12.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.546+0000 7f65f2a00640 1 -- 172.21.15.42:0/3521664363 wait complete. 2024-07-04T17:26:12.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.548+0000 7f65f2a00640 1 Processor -- start 2024-07-04T17:26:12.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.548+0000 7f65f2a00640 1 -- start start 2024-07-04T17:26:12.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.550+0000 7f65f2a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f65ec06cea0 0x7f65ec144470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:12.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.550+0000 7f65f2a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec06ef50 con 0x7f65ec06cea0 2024-07-04T17:26:12.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.550+0000 7f65ebe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f65ec06cea0 0x7f65ec144470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:12.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.550+0000 7f65ebe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f65ec06cea0 0x7f65ec144470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39094/0 (socket says 172.21.15.42:39094) 2024-07-04T17:26:12.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.550+0000 7f65ebe00640 1 -- 172.21.15.42:0/3436226356 learned_addr learned my addr 172.21.15.42:0/3436226356 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:12.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.550+0000 7f65ebe00640 1 -- 172.21.15.42:0/3436226356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ec149590 con 0x7f65ec06cea0 2024-07-04T17:26:12.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.551+0000 7f65ebe00640 1 --2- 172.21.15.42:0/3436226356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f65ec06cea0 0x7f65ec144470 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00cb40 tx=0x7f65dc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:12.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.551+0000 7f65e3e00640 1 -- 172.21.15.42:0/3436226356 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65dc016020 con 0x7f65ec06cea0 2024-07-04T17:26:12.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.551+0000 7f65e3e00640 1 -- 172.21.15.42:0/3436226356 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65dc00ac30 con 0x7f65ec06cea0 2024-07-04T17:26:12.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.551+0000 7f65e3e00640 1 -- 172.21.15.42:0/3436226356 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65dc012420 con 0x7f65ec06cea0 2024-07-04T17:26:12.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.551+0000 7f65f2a00640 1 -- 172.21.15.42:0/3436226356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65ec146b60 con 0x7f65ec06cea0 2024-07-04T17:26:12.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.551+0000 7f65f2a00640 1 -- 172.21.15.42:0/3436226356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65ec1479a0 con 0x7f65ec06cea0 2024-07-04T17:26:12.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.552+0000 7f65f2a00640 1 -- 172.21.15.42:0/3436226356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65ec0b01f0 con 0x7f65ec06cea0 2024-07-04T17:26:12.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.552+0000 7f65e3e00640 1 -- 172.21.15.42:0/3436226356 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f65dc019020 con 0x7f65ec06cea0 2024-07-04T17:26:12.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.553+0000 7f65e3e00640 1 --2- 172.21.15.42:0/3436226356 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f65c803d2b0 0x7f65c803f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:12.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.553+0000 7f65e3e00640 1 -- 172.21.15.42:0/3436226356 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 2587+0+0 (secure 0 0 0) 0x7f65dc051550 con 0x7f65ec06cea0 2024-07-04T17:26:12.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.553+0000 7f65eb400640 1 --2- 172.21.15.42:0/3436226356 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f65c803d2b0 0x7f65c803f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:12.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.554+0000 7f65eb400640 1 --2- 172.21.15.42:0/3436226356 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f65c803d2b0 0x7f65c803f7c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f65d4036000 tx=0x7f65d4004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:12.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.558+0000 7f65e3e00640 1 -- 172.21.15.42:0/3436226356 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f65dc020030 con 0x7f65ec06cea0 2024-07-04T17:26:12.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.769+0000 7f65f2a00640 1 -- 172.21.15.42:0/3436226356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f65ec069b70 con 0x7f65ec06cea0 2024-07-04T17:26:12.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.770+0000 7f65e3e00640 1 -- 172.21.15.42:0/3436226356 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v11) ==== 84+0+65 (secure 0 0 0) 0x7f65dc00d150 con 0x7f65ec06cea0 2024-07-04T17:26:12.772 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:26:12.772 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-07-04T17:26:12.772 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:26:12.772 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 1, 2024-07-04T17:26:12.773 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": ".mgr" 2024-07-04T17:26:12.773 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:26:12.773 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-07-04T17:26:12.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.773+0000 7f65f2a00640 1 -- 172.21.15.42:0/3436226356 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f65c803d2b0 msgr2=0x7f65c803f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:12.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.773+0000 7f65f2a00640 1 --2- 172.21.15.42:0/3436226356 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f65c803d2b0 0x7f65c803f7c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f65d4036000 tx=0x7f65d4004730 comp rx=0 tx=0).stop 2024-07-04T17:26:12.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.773+0000 7f65f2a00640 1 -- 172.21.15.42:0/3436226356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f65ec06cea0 msgr2=0x7f65ec144470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:12.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.773+0000 7f65f2a00640 1 --2- 172.21.15.42:0/3436226356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f65ec06cea0 0x7f65ec144470 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00cb40 tx=0x7f65dc00a060 comp rx=0 tx=0).stop 2024-07-04T17:26:12.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.773+0000 7f65f2a00640 1 -- 172.21.15.42:0/3436226356 shutdown_connections 2024-07-04T17:26:12.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.773+0000 7f65f2a00640 1 --2- 172.21.15.42:0/3436226356 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f65c803d2b0 0x7f65c803f7c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:12.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.773+0000 7f65f2a00640 1 --2- 172.21.15.42:0/3436226356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f65ec06cea0 0x7f65ec144470 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:12.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.773+0000 7f65f2a00640 1 -- 172.21.15.42:0/3436226356 >> 172.21.15.42:0/3436226356 conn(0x7f65ec0bf700 msgr2=0x7f65ec0aa3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:12.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.774+0000 7f65f2a00640 1 -- 172.21.15.42:0/3436226356 shutdown_connections 2024-07-04T17:26:12.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:12.774+0000 7f65f2a00640 1 -- 172.21.15.42:0/3436226356 wait complete. 2024-07-04T17:26:12.800 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-04T17:26:12.801 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-04T17:26:13.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.088+0000 7f52e7e00640 1 -- 172.21.15.42:0/3116969106 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52e80701d0 msgr2=0x7f52e80abe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:13.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.088+0000 7f52e7e00640 1 --2- 172.21.15.42:0/3116969106 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52e80701d0 0x7f52e80abe40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f52d80097b0 tx=0x7f52d802a540 comp rx=0 tx=0).stop 2024-07-04T17:26:13.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.088+0000 7f52e7e00640 1 -- 172.21.15.42:0/3116969106 shutdown_connections 2024-07-04T17:26:13.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.088+0000 7f52e7e00640 1 --2- 172.21.15.42:0/3116969106 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52e80701d0 0x7f52e80abe40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:13.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.088+0000 7f52e7e00640 1 -- 172.21.15.42:0/3116969106 >> 172.21.15.42:0/3116969106 conn(0x7f52e80afaa0 msgr2=0x7f52e80afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:13.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.089+0000 7f52e7e00640 1 -- 172.21.15.42:0/3116969106 shutdown_connections 2024-07-04T17:26:13.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.089+0000 7f52e7e00640 1 -- 172.21.15.42:0/3116969106 wait complete. 2024-07-04T17:26:13.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.091+0000 7f52e7e00640 1 Processor -- start 2024-07-04T17:26:13.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.091+0000 7f52e7e00640 1 -- start start 2024-07-04T17:26:13.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.092+0000 7f52e7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52e80701d0 0x7f52e814c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:13.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.092+0000 7f52e7e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f52e80ab190 con 0x7f52e80701d0 2024-07-04T17:26:13.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.093+0000 7f52e6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52e80701d0 0x7f52e814c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:13.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.093+0000 7f52e6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52e80701d0 0x7f52e814c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39122/0 (socket says 172.21.15.42:39122) 2024-07-04T17:26:13.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.093+0000 7f52e6a00640 1 -- 172.21.15.42:0/643034203 learned_addr learned my addr 172.21.15.42:0/643034203 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:13.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.093+0000 7f52e6a00640 1 -- 172.21.15.42:0/643034203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e814ce60 con 0x7f52e80701d0 2024-07-04T17:26:13.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.093+0000 7f52e6a00640 1 --2- 172.21.15.42:0/643034203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52e80701d0 0x7f52e814c870 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f52d80098d0 tx=0x7f52d8033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:13.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.094+0000 7f52dea00640 1 -- 172.21.15.42:0/643034203 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52d8031bc0 con 0x7f52e80701d0 2024-07-04T17:26:13.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.094+0000 7f52dea00640 1 -- 172.21.15.42:0/643034203 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52d8031e50 con 0x7f52e80701d0 2024-07-04T17:26:13.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.094+0000 7f52dea00640 1 -- 172.21.15.42:0/643034203 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52d8033200 con 0x7f52e80701d0 2024-07-04T17:26:13.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.094+0000 7f52e7e00640 1 -- 172.21.15.42:0/643034203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e8144a20 con 0x7f52e80701d0 2024-07-04T17:26:13.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.094+0000 7f52e7e00640 1 -- 172.21.15.42:0/643034203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e814f3c0 con 0x7f52e80701d0 2024-07-04T17:26:13.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.095+0000 7f52e7e00640 1 -- 172.21.15.42:0/643034203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52e80abe40 con 0x7f52e80701d0 2024-07-04T17:26:13.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.095+0000 7f52dea00640 1 -- 172.21.15.42:0/643034203 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f52d80333a0 con 0x7f52e80701d0 2024-07-04T17:26:13.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.096+0000 7f52dea00640 1 --2- 172.21.15.42:0/643034203 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f52c803d2b0 0x7f52c803f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:13.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.096+0000 7f52dea00640 1 -- 172.21.15.42:0/643034203 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 2587+0+0 (secure 0 0 0) 0x7f52d8072700 con 0x7f52e80701d0 2024-07-04T17:26:13.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.096+0000 7f52e6000640 1 --2- 172.21.15.42:0/643034203 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f52c803d2b0 0x7f52c803f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:13.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.097+0000 7f52e6000640 1 --2- 172.21.15.42:0/643034203 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f52c803d2b0 0x7f52c803f7c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f52d40015c0 tx=0x7f52d4007a90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:13.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.101+0000 7f52dea00640 1 -- 172.21.15.42:0/643034203 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f52d803f030 con 0x7f52e80701d0 2024-07-04T17:26:13.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.314+0000 7f52e7e00640 1 -- 172.21.15.42:0/643034203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f52e80aa6e0 con 0x7f52e80701d0 2024-07-04T17:26:13.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.978+0000 7f52dea00640 1 -- 172.21.15.42:0/643034203 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v12) ==== 138+0+0 (secure 0 0 0) 0x7f52d8033690 con 0x7f52e80701d0 2024-07-04T17:26:13.980 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' created 2024-07-04T17:26:13.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.982+0000 7f52e7e00640 1 -- 172.21.15.42:0/643034203 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f52c803d2b0 msgr2=0x7f52c803f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:13.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.982+0000 7f52e7e00640 1 --2- 172.21.15.42:0/643034203 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f52c803d2b0 0x7f52c803f7c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f52d40015c0 tx=0x7f52d4007a90 comp rx=0 tx=0).stop 2024-07-04T17:26:13.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.982+0000 7f52e7e00640 1 -- 172.21.15.42:0/643034203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52e80701d0 msgr2=0x7f52e814c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:13.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.982+0000 7f52e7e00640 1 --2- 172.21.15.42:0/643034203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52e80701d0 0x7f52e814c870 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f52d80098d0 tx=0x7f52d8033e40 comp rx=0 tx=0).stop 2024-07-04T17:26:13.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.982+0000 7f52e7e00640 1 -- 172.21.15.42:0/643034203 shutdown_connections 2024-07-04T17:26:13.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.983+0000 7f52e7e00640 1 --2- 172.21.15.42:0/643034203 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f52c803d2b0 0x7f52c803f7c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:13.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.983+0000 7f52e7e00640 1 --2- 172.21.15.42:0/643034203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52e80701d0 0x7f52e814c870 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:13.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.983+0000 7f52e7e00640 1 -- 172.21.15.42:0/643034203 >> 172.21.15.42:0/643034203 conn(0x7f52e80afaa0 msgr2=0x7f52e80b36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:13.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.983+0000 7f52e7e00640 1 -- 172.21.15.42:0/643034203 shutdown_connections 2024-07-04T17:26:13.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:13.983+0000 7f52e7e00640 1 -- 172.21.15.42:0/643034203 wait complete. 2024-07-04T17:26:14.023 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-04T17:26:14.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.289+0000 7f5bb7000640 1 -- 172.21.15.42:0/2172852254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bb0073b00 msgr2=0x7f5bb006d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:14.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.289+0000 7f5bb7000640 1 --2- 172.21.15.42:0/2172852254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bb0073b00 0x7f5bb006d160 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5ba00097b0 tx=0x7f5ba002a5d0 comp rx=0 tx=0).stop 2024-07-04T17:26:14.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.289+0000 7f5bb7000640 1 -- 172.21.15.42:0/2172852254 shutdown_connections 2024-07-04T17:26:14.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.289+0000 7f5bb7000640 1 --2- 172.21.15.42:0/2172852254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bb0073b00 0x7f5bb006d160 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:14.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.289+0000 7f5bb7000640 1 -- 172.21.15.42:0/2172852254 >> 172.21.15.42:0/2172852254 conn(0x7f5bb006c020 msgr2=0x7f5bb006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:14.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.289+0000 7f5bb7000640 1 -- 172.21.15.42:0/2172852254 shutdown_connections 2024-07-04T17:26:14.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.289+0000 7f5bb7000640 1 -- 172.21.15.42:0/2172852254 wait complete. 2024-07-04T17:26:14.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.291+0000 7f5bb7000640 1 Processor -- start 2024-07-04T17:26:14.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.291+0000 7f5bb7000640 1 -- start start 2024-07-04T17:26:14.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.292+0000 7f5bb7000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bb0073b00 0x7f5bb0144460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:14.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.292+0000 7f5bb7000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb013e3a0 con 0x7f5bb0073b00 2024-07-04T17:26:14.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.292+0000 7f5bafe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bb0073b00 0x7f5bb0144460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:14.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.293+0000 7f5bafe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bb0073b00 0x7f5bb0144460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34716/0 (socket says 172.21.15.42:34716) 2024-07-04T17:26:14.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.293+0000 7f5bafe00640 1 -- 172.21.15.42:0/2574957910 learned_addr learned my addr 172.21.15.42:0/2574957910 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:14.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.293+0000 7f5bafe00640 1 -- 172.21.15.42:0/2574957910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb01d4730 con 0x7f5bb0073b00 2024-07-04T17:26:14.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.293+0000 7f5bafe00640 1 --2- 172.21.15.42:0/2574957910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bb0073b00 0x7f5bb0144460 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5ba00098d0 tx=0x7f5ba0033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:14.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.293+0000 7f5ba7e00640 1 -- 172.21.15.42:0/2574957910 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ba00319b0 con 0x7f5bb0073b00 2024-07-04T17:26:14.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.293+0000 7f5ba7e00640 1 -- 172.21.15.42:0/2574957910 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ba0031c40 con 0x7f5bb0073b00 2024-07-04T17:26:14.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.293+0000 7f5ba7e00640 1 -- 172.21.15.42:0/2574957910 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ba00330d0 con 0x7f5bb0073b00 2024-07-04T17:26:14.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.294+0000 7f5bb7000640 1 -- 172.21.15.42:0/2574957910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb01d5640 con 0x7f5bb0073b00 2024-07-04T17:26:14.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.294+0000 7f5bb7000640 1 -- 172.21.15.42:0/2574957910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb01d6360 con 0x7f5bb0073b00 2024-07-04T17:26:14.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.295+0000 7f5ba7e00640 1 -- 172.21.15.42:0/2574957910 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f5ba0033270 con 0x7f5bb0073b00 2024-07-04T17:26:14.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.295+0000 7f5bb7000640 1 -- 172.21.15.42:0/2574957910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bb013e9e0 con 0x7f5bb0073b00 2024-07-04T17:26:14.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.295+0000 7f5ba7e00640 1 --2- 172.21.15.42:0/2574957910 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f5b9003d2b0 0x7f5b9003f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:14.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.295+0000 7f5ba7e00640 1 -- 172.21.15.42:0/2574957910 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2957+0+0 (secure 0 0 0) 0x7f5ba0072610 con 0x7f5bb0073b00 2024-07-04T17:26:14.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.296+0000 7f5baf400640 1 --2- 172.21.15.42:0/2574957910 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f5b9003d2b0 0x7f5b9003f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:14.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.296+0000 7f5baf400640 1 --2- 172.21.15.42:0/2574957910 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f5b9003d2b0 0x7f5b9003f7c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c0015c0 tx=0x7f5b9c007a90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:14.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.301+0000 7f5ba7e00640 1 -- 172.21.15.42:0/2574957910 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f5bb01d4730 con 0x7f5bb0073b00 2024-07-04T17:26:14.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.504+0000 7f5bb7000640 1 -- 172.21.15.42:0/2574957910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f5bb0069b70 con 0x7f5bb0073b00 2024-07-04T17:26:14.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.992+0000 7f5ba7e00640 1 -- 172.21.15.42:0/2574957910 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v13) ==== 170+0+0 (secure 0 0 0) 0x7f5ba003f030 con 0x7f5bb0073b00 2024-07-04T17:26:14.994 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' created 2024-07-04T17:26:14.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.997+0000 7f5bb7000640 1 -- 172.21.15.42:0/2574957910 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f5b9003d2b0 msgr2=0x7f5b9003f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:14.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.997+0000 7f5bb7000640 1 --2- 172.21.15.42:0/2574957910 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f5b9003d2b0 0x7f5b9003f7c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c0015c0 tx=0x7f5b9c007a90 comp rx=0 tx=0).stop 2024-07-04T17:26:14.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.997+0000 7f5bb7000640 1 -- 172.21.15.42:0/2574957910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bb0073b00 msgr2=0x7f5bb0144460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:14.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.997+0000 7f5bb7000640 1 --2- 172.21.15.42:0/2574957910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bb0073b00 0x7f5bb0144460 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5ba00098d0 tx=0x7f5ba0033ad0 comp rx=0 tx=0).stop 2024-07-04T17:26:15.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.997+0000 7f5bb7000640 1 -- 172.21.15.42:0/2574957910 shutdown_connections 2024-07-04T17:26:15.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.998+0000 7f5bb7000640 1 --2- 172.21.15.42:0/2574957910 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f5b9003d2b0 0x7f5b9003f7c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:15.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.998+0000 7f5bb7000640 1 --2- 172.21.15.42:0/2574957910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bb0073b00 0x7f5bb0144460 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:15.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.998+0000 7f5bb7000640 1 -- 172.21.15.42:0/2574957910 >> 172.21.15.42:0/2574957910 conn(0x7f5bb006c020 msgr2=0x7f5bb00b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:15.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.998+0000 7f5bb7000640 1 -- 172.21.15.42:0/2574957910 shutdown_connections 2024-07-04T17:26:15.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:14.998+0000 7f5bb7000640 1 -- 172.21.15.42:0/2574957910 wait complete. 2024-07-04T17:26:15.049 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-04T17:26:15.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:15.307+0000 7f6ea3200640 1 -- 172.21.15.42:0/3665526083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ea40701d0 msgr2=0x7f6ea40abe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:15.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:15.307+0000 7f6ea3200640 1 --2- 172.21.15.42:0/3665526083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ea40701d0 0x7f6ea40abe40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6e940097b0 tx=0x7f6e9402a540 comp rx=0 tx=0).stop 2024-07-04T17:26:15.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:15.307+0000 7f6ea3200640 1 -- 172.21.15.42:0/3665526083 shutdown_connections 2024-07-04T17:26:15.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:15.307+0000 7f6ea3200640 1 --2- 172.21.15.42:0/3665526083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ea40701d0 0x7f6ea40abe40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:15.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:15.307+0000 7f6ea3200640 1 -- 172.21.15.42:0/3665526083 >> 172.21.15.42:0/3665526083 conn(0x7f6ea40afaa0 msgr2=0x7f6ea40afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:15.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:15.307+0000 7f6ea3200640 1 -- 172.21.15.42:0/3665526083 shutdown_connections 2024-07-04T17:26:15.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:15.307+0000 7f6ea3200640 1 -- 172.21.15.42:0/3665526083 wait complete. 2024-07-04T17:26:15.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:15.310+0000 7f6ea3200640 1 Processor -- start 2024-07-04T17:26:15.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:15.310+0000 7f6ea3200640 1 -- start start 2024-07-04T17:26:15.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:15.311+0000 7f6ea3200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ea40701d0 0x7f6ea414ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:15.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:15.311+0000 7f6ea3200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea40ab190 con 0x7f6ea40701d0 2024-07-04T17:26:15.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:15.312+0000 7f6ea1e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ea40701d0 0x7f6ea414ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:15.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:15.312+0000 7f6ea1e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ea40701d0 0x7f6ea414ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34722/0 (socket says 172.21.15.42:34722) 2024-07-04T17:26:15.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:15.312+0000 7f6ea1e00640 1 -- 172.21.15.42:0/1487555793 learned_addr learned my addr 172.21.15.42:0/1487555793 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:15.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:15.312+0000 7f6ea1e00640 1 -- 172.21.15.42:0/1487555793 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea41d67b0 con 0x7f6ea40701d0 2024-07-04T17:26:15.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:15.313+0000 7f6ea1e00640 1 --2- 172.21.15.42:0/1487555793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ea40701d0 0x7f6ea414ea20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6e940098d0 tx=0x7f6e94033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:15.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:15.313+0000 7f6e9a000640 1 -- 172.21.15.42:0/1487555793 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e94031bc0 con 0x7f6ea40701d0 2024-07-04T17:26:15.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:15.313+0000 7f6e9a000640 1 -- 172.21.15.42:0/1487555793 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e94031e50 con 0x7f6ea40701d0 2024-07-04T17:26:15.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:15.313+0000 7f6e9a000640 1 -- 172.21.15.42:0/1487555793 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e94033200 con 0x7f6ea40701d0 2024-07-04T17:26:15.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:15.313+0000 7f6ea3200640 1 -- 172.21.15.42:0/1487555793 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea414ef60 con 0x7f6ea40701d0 2024-07-04T17:26:15.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:15.313+0000 7f6ea3200640 1 -- 172.21.15.42:0/1487555793 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea4144400 con 0x7f6ea40701d0 2024-07-04T17:26:15.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:15.314+0000 7f6ea3200640 1 -- 172.21.15.42:0/1487555793 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ea414e660 con 0x7f6ea40701d0 2024-07-04T17:26:15.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:15.315+0000 7f6e9a000640 1 -- 172.21.15.42:0/1487555793 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f6e940333a0 con 0x7f6ea40701d0 2024-07-04T17:26:15.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:15.315+0000 7f6e9a000640 1 --2- 172.21.15.42:0/1487555793 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f6e8403d2b0 0x7f6e8403f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:15.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:15.315+0000 7f6e9a000640 1 -- 172.21.15.42:0/1487555793 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 3339+0+0 (secure 0 0 0) 0x7f6e940729f0 con 0x7f6ea40701d0 2024-07-04T17:26:15.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:15.316+0000 7f6ea1400640 1 --2- 172.21.15.42:0/1487555793 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f6e8403d2b0 0x7f6e8403f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:15.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:15.316+0000 7f6ea1400640 1 --2- 172.21.15.42:0/1487555793 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f6e8403d2b0 0x7f6e8403f7c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6e900015c0 tx=0x7f6e90007a90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:15.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:15.321+0000 7f6e9a000640 1 -- 172.21.15.42:0/1487555793 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f6ea414e660 con 0x7f6ea40701d0 2024-07-04T17:26:15.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:15.571+0000 7f6ea3200640 1 -- 172.21.15.42:0/1487555793 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f6ea40abe40 con 0x7f6ea40701d0 2024-07-04T17:26:16.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.006+0000 7f6e9a000640 1 -- 172.21.15.42:0/1487555793 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f6e9403f030 con 0x7f6ea40701d0 2024-07-04T17:26:16.009 INFO:teuthology.orchestra.run.smithi042.stderr:new fs with metadata pool 2 and data pool 3 2024-07-04T17:26:16.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.013+0000 7f6ea3200640 1 -- 172.21.15.42:0/1487555793 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f6e8403d2b0 msgr2=0x7f6e8403f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:16.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.013+0000 7f6ea3200640 1 --2- 172.21.15.42:0/1487555793 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f6e8403d2b0 0x7f6e8403f7c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6e900015c0 tx=0x7f6e90007a90 comp rx=0 tx=0).stop 2024-07-04T17:26:16.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.013+0000 7f6ea3200640 1 -- 172.21.15.42:0/1487555793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ea40701d0 msgr2=0x7f6ea414ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:16.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.013+0000 7f6ea3200640 1 --2- 172.21.15.42:0/1487555793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ea40701d0 0x7f6ea414ea20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6e940098d0 tx=0x7f6e94033e40 comp rx=0 tx=0).stop 2024-07-04T17:26:16.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.014+0000 7f6ea3200640 1 -- 172.21.15.42:0/1487555793 shutdown_connections 2024-07-04T17:26:16.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.014+0000 7f6ea3200640 1 --2- 172.21.15.42:0/1487555793 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f6e8403d2b0 0x7f6e8403f7c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:16.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.014+0000 7f6ea3200640 1 --2- 172.21.15.42:0/1487555793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ea40701d0 0x7f6ea414ea20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:16.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.014+0000 7f6ea3200640 1 -- 172.21.15.42:0/1487555793 >> 172.21.15.42:0/1487555793 conn(0x7f6ea40afaa0 msgr2=0x7f6ea40b37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:16.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.014+0000 7f6ea3200640 1 -- 172.21.15.42:0/1487555793 shutdown_connections 2024-07-04T17:26:16.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.014+0000 7f6ea3200640 1 -- 172.21.15.42:0/1487555793 wait complete. 2024-07-04T17:26:16.064 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:26:16.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.321+0000 7efee0e00640 1 -- 172.21.15.42:0/1680284891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efedc06edd0 msgr2=0x7efedc0b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:16.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.321+0000 7efee0e00640 1 --2- 172.21.15.42:0/1680284891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efedc06edd0 0x7efedc0b3030 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efecc005fd0 tx=0x7efecc02a5f0 comp rx=0 tx=0).stop 2024-07-04T17:26:16.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.321+0000 7efee0e00640 1 -- 172.21.15.42:0/1680284891 shutdown_connections 2024-07-04T17:26:16.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.321+0000 7efee0e00640 1 --2- 172.21.15.42:0/1680284891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efedc06edd0 0x7efedc0b3030 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:16.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.321+0000 7efee0e00640 1 -- 172.21.15.42:0/1680284891 >> 172.21.15.42:0/1680284891 conn(0x7efedc0bf700 msgr2=0x7efedc0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:16.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.321+0000 7efee0e00640 1 -- 172.21.15.42:0/1680284891 shutdown_connections 2024-07-04T17:26:16.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.321+0000 7efee0e00640 1 -- 172.21.15.42:0/1680284891 wait complete. 2024-07-04T17:26:16.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.323+0000 7efee0e00640 1 Processor -- start 2024-07-04T17:26:16.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.323+0000 7efee0e00640 1 -- start start 2024-07-04T17:26:16.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.325+0000 7efee0e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efedc06cea0 0x7efedc148390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:16.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.325+0000 7efee0e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efedc0b0580 con 0x7efedc06cea0 2024-07-04T17:26:16.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.325+0000 7efed9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efedc06cea0 0x7efedc148390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:16.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.325+0000 7efed9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efedc06cea0 0x7efedc148390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34748/0 (socket says 172.21.15.42:34748) 2024-07-04T17:26:16.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.325+0000 7efed9e00640 1 -- 172.21.15.42:0/331279499 learned_addr learned my addr 172.21.15.42:0/331279499 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:16.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.325+0000 7efed9e00640 1 -- 172.21.15.42:0/331279499 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efedc149dd0 con 0x7efedc06cea0 2024-07-04T17:26:16.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.326+0000 7efed9e00640 1 --2- 172.21.15.42:0/331279499 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efedc06cea0 0x7efedc148390 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efecc009850 tx=0x7efecc0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:16.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.326+0000 7efed2000640 1 -- 172.21.15.42:0/331279499 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efecc03e030 con 0x7efedc06cea0 2024-07-04T17:26:16.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.326+0000 7efed2000640 1 -- 172.21.15.42:0/331279499 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efecc0035f0 con 0x7efedc06cea0 2024-07-04T17:26:16.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.326+0000 7efee0e00640 1 -- 172.21.15.42:0/331279499 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efedc1e4260 con 0x7efedc06cea0 2024-07-04T17:26:16.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.326+0000 7efed2000640 1 -- 172.21.15.42:0/331279499 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efecc035040 con 0x7efedc06cea0 2024-07-04T17:26:16.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.326+0000 7efee0e00640 1 -- 172.21.15.42:0/331279499 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efedc1454c0 con 0x7efedc06cea0 2024-07-04T17:26:16.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.327+0000 7efee0e00640 1 -- 172.21.15.42:0/331279499 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efedc0b1d40 con 0x7efedc06cea0 2024-07-04T17:26:16.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.327+0000 7efed2000640 1 -- 172.21.15.42:0/331279499 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7efecc041030 con 0x7efedc06cea0 2024-07-04T17:26:16.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.328+0000 7efed2000640 1 --2- 172.21.15.42:0/331279499 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7efeb803d2b0 0x7efeb803f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:16.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.328+0000 7efed2000640 1 -- 172.21.15.42:0/331279499 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 3439+0+0 (secure 0 0 0) 0x7efecc03f070 con 0x7efedc06cea0 2024-07-04T17:26:16.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.328+0000 7efed3e00640 1 --2- 172.21.15.42:0/331279499 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7efeb803d2b0 0x7efeb803f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:16.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.329+0000 7efed3e00640 1 --2- 172.21.15.42:0/331279499 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7efeb803d2b0 0x7efeb803f7c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7efec4001560 tx=0x7efec40079f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:16.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.333+0000 7efed2000640 1 -- 172.21.15.42:0/331279499 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7efedc0b1d40 con 0x7efedc06cea0 2024-07-04T17:26:16.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.541+0000 7efee0e00640 1 -- 172.21.15.42:0/331279499 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efedc0b3030 con 0x7efedc06cea0 2024-07-04T17:26:16.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.543+0000 7efed2000640 1 -- 172.21.15.42:0/331279499 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+10616 (secure 0 0 0) 0x7efecc04bd60 con 0x7efedc06cea0 2024-07-04T17:26:16.545 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:26:16.545 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":14,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:26:15.573690+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:26:13.335715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-04T17:26:14.519899+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:26:16.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.545+0000 7efee0e00640 1 -- 172.21.15.42:0/331279499 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7efeb803d2b0 msgr2=0x7efeb803f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:16.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.545+0000 7efee0e00640 1 --2- 172.21.15.42:0/331279499 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7efeb803d2b0 0x7efeb803f7c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7efec4001560 tx=0x7efec40079f0 comp rx=0 tx=0).stop 2024-07-04T17:26:16.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.545+0000 7efee0e00640 1 -- 172.21.15.42:0/331279499 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efedc06cea0 msgr2=0x7efedc148390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:16.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.545+0000 7efee0e00640 1 --2- 172.21.15.42:0/331279499 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efedc06cea0 0x7efedc148390 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efecc009850 tx=0x7efecc0049d0 comp rx=0 tx=0).stop 2024-07-04T17:26:16.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.546+0000 7efee0e00640 1 -- 172.21.15.42:0/331279499 shutdown_connections 2024-07-04T17:26:16.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.546+0000 7efee0e00640 1 --2- 172.21.15.42:0/331279499 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7efeb803d2b0 0x7efeb803f7c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:16.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.546+0000 7efee0e00640 1 --2- 172.21.15.42:0/331279499 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efedc06cea0 0x7efedc148390 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:16.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.546+0000 7efee0e00640 1 -- 172.21.15.42:0/331279499 >> 172.21.15.42:0/331279499 conn(0x7efedc0bf700 msgr2=0x7efedc0af0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:16.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.546+0000 7efee0e00640 1 -- 172.21.15.42:0/331279499 shutdown_connections 2024-07-04T17:26:16.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.546+0000 7efee0e00640 1 -- 172.21.15.42:0/331279499 wait complete. 2024-07-04T17:26:16.570 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:26:16.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.839+0000 7f9efca00640 1 -- 172.21.15.42:0/1249420428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ef8073e80 msgr2=0x7f9ef806d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:16.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.839+0000 7f9efca00640 1 --2- 172.21.15.42:0/1249420428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ef8073e80 0x7f9ef806d8d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8074460 tx=0x7f9ee402a520 comp rx=0 tx=0).stop 2024-07-04T17:26:16.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.840+0000 7f9efca00640 1 -- 172.21.15.42:0/1249420428 shutdown_connections 2024-07-04T17:26:16.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.840+0000 7f9efca00640 1 --2- 172.21.15.42:0/1249420428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ef8073e80 0x7f9ef806d8d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:16.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.840+0000 7f9efca00640 1 -- 172.21.15.42:0/1249420428 >> 172.21.15.42:0/1249420428 conn(0x7f9ef8070830 msgr2=0x7f9ef80bfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:16.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.840+0000 7f9efca00640 1 -- 172.21.15.42:0/1249420428 shutdown_connections 2024-07-04T17:26:16.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.840+0000 7f9efca00640 1 -- 172.21.15.42:0/1249420428 wait complete. 2024-07-04T17:26:16.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.842+0000 7f9efca00640 1 Processor -- start 2024-07-04T17:26:16.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.842+0000 7f9efca00640 1 -- start start 2024-07-04T17:26:16.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.843+0000 7f9efca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ef80bca50 0x7f9ef8148e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:16.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.843+0000 7f9efca00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef81485b0 con 0x7f9ef80bca50 2024-07-04T17:26:16.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.843+0000 7f9ef7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ef80bca50 0x7f9ef8148e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:16.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.843+0000 7f9ef7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ef80bca50 0x7f9ef8148e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34778/0 (socket says 172.21.15.42:34778) 2024-07-04T17:26:16.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.843+0000 7f9ef7400640 1 -- 172.21.15.42:0/3317401967 learned_addr learned my addr 172.21.15.42:0/3317401967 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:16.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.844+0000 7f9ef7400640 1 -- 172.21.15.42:0/3317401967 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef81e4bb0 con 0x7f9ef80bca50 2024-07-04T17:26:16.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.844+0000 7f9ef7400640 1 --2- 172.21.15.42:0/3317401967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ef80bca50 0x7f9ef8148e60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9ee800c380 tx=0x7f9ee800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:16.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.844+0000 7f9eef400640 1 -- 172.21.15.42:0/3317401967 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ee80116c0 con 0x7f9ef80bca50 2024-07-04T17:26:16.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.844+0000 7f9eef400640 1 -- 172.21.15.42:0/3317401967 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ee8017070 con 0x7f9ef80bca50 2024-07-04T17:26:16.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.844+0000 7f9eef400640 1 -- 172.21.15.42:0/3317401967 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ee8011b20 con 0x7f9ef80bca50 2024-07-04T17:26:16.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.844+0000 7f9efca00640 1 -- 172.21.15.42:0/3317401967 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef81d4c80 con 0x7f9ef80bca50 2024-07-04T17:26:16.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.844+0000 7f9efca00640 1 -- 172.21.15.42:0/3317401967 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef8144420 con 0x7f9ef80bca50 2024-07-04T17:26:16.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.846+0000 7f9efca00640 1 -- 172.21.15.42:0/3317401967 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ef8149dc0 con 0x7f9ef80bca50 2024-07-04T17:26:16.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.846+0000 7f9eef400640 1 -- 172.21.15.42:0/3317401967 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f9ee8011cc0 con 0x7f9ef80bca50 2024-07-04T17:26:16.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.846+0000 7f9eef400640 1 --2- 172.21.15.42:0/3317401967 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f9ed803d2b0 0x7f9ed803f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:16.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.846+0000 7f9eef400640 1 -- 172.21.15.42:0/3317401967 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 3439+0+0 (secure 0 0 0) 0x7f9ee80512d0 con 0x7f9ef80bca50 2024-07-04T17:26:16.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.846+0000 7f9ef6a00640 1 --2- 172.21.15.42:0/3317401967 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f9ed803d2b0 0x7f9ed803f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:16.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.847+0000 7f9ef6a00640 1 --2- 172.21.15.42:0/3317401967 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f9ed803d2b0 0x7f9ed803f7c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4034000 tx=0x7f9ee4004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:16.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:16.851+0000 7f9eef400640 1 -- 172.21.15.42:0/3317401967 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f9ef80bce60 con 0x7f9ef80bca50 2024-07-04T17:26:17.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.058+0000 7f9efca00640 1 -- 172.21.15.42:0/3317401967 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ef8148c50 con 0x7f9ef80bca50 2024-07-04T17:26:17.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.059+0000 7f9eef400640 1 -- 172.21.15.42:0/3317401967 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) ==== 74+0+10603 (secure 0 0 0) 0x7f9ee8010cf0 con 0x7f9ef80bca50 2024-07-04T17:26:17.061 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:26:17.061 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":15,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:26:17.008553+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:26:13.335715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-04T17:26:14.519899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:26:17.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.062+0000 7f9efca00640 1 -- 172.21.15.42:0/3317401967 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f9ed803d2b0 msgr2=0x7f9ed803f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:17.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.062+0000 7f9efca00640 1 --2- 172.21.15.42:0/3317401967 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f9ed803d2b0 0x7f9ed803f7c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4034000 tx=0x7f9ee4004730 comp rx=0 tx=0).stop 2024-07-04T17:26:17.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.062+0000 7f9efca00640 1 -- 172.21.15.42:0/3317401967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ef80bca50 msgr2=0x7f9ef8148e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:17.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.062+0000 7f9efca00640 1 --2- 172.21.15.42:0/3317401967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ef80bca50 0x7f9ef8148e60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9ee800c380 tx=0x7f9ee800c840 comp rx=0 tx=0).stop 2024-07-04T17:26:17.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.062+0000 7f9efca00640 1 -- 172.21.15.42:0/3317401967 shutdown_connections 2024-07-04T17:26:17.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.062+0000 7f9efca00640 1 --2- 172.21.15.42:0/3317401967 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f9ed803d2b0 0x7f9ed803f7c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:17.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.062+0000 7f9efca00640 1 --2- 172.21.15.42:0/3317401967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ef80bca50 0x7f9ef8148e60 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:17.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.062+0000 7f9efca00640 1 -- 172.21.15.42:0/3317401967 >> 172.21.15.42:0/3317401967 conn(0x7f9ef8070830 msgr2=0x7f9ef80b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:17.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.062+0000 7f9efca00640 1 -- 172.21.15.42:0/3317401967 shutdown_connections 2024-07-04T17:26:17.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.062+0000 7f9efca00640 1 -- 172.21.15.42:0/3317401967 wait complete. 2024-07-04T17:26:17.087 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-07-04T17:26:17.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.370+0000 7fa2b9a00640 1 -- 172.21.15.42:0/1498485981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2b40ac9d0 msgr2=0x7fa2b40aef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:17.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.370+0000 7fa2b9a00640 1 --2- 172.21.15.42:0/1498485981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2b40ac9d0 0x7fa2b40aef80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00097b0 tx=0x7fa2a002a4a0 comp rx=0 tx=0).stop 2024-07-04T17:26:17.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.370+0000 7fa2b9a00640 1 -- 172.21.15.42:0/1498485981 shutdown_connections 2024-07-04T17:26:17.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.370+0000 7fa2b9a00640 1 --2- 172.21.15.42:0/1498485981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2b40ac9d0 0x7fa2b40aef80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:17.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.370+0000 7fa2b9a00640 1 -- 172.21.15.42:0/1498485981 >> 172.21.15.42:0/1498485981 conn(0x7fa2b40afaa0 msgr2=0x7fa2b40afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:17.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.370+0000 7fa2b9a00640 1 -- 172.21.15.42:0/1498485981 shutdown_connections 2024-07-04T17:26:17.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.370+0000 7fa2b9a00640 1 -- 172.21.15.42:0/1498485981 wait complete. 2024-07-04T17:26:17.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.373+0000 7fa2b9a00640 1 Processor -- start 2024-07-04T17:26:17.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.373+0000 7fa2b9a00640 1 -- start start 2024-07-04T17:26:17.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.375+0000 7fa2b9a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2b40ac9d0 0x7fa2b4141e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:17.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.375+0000 7fa2b9a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b41415a0 con 0x7fa2b40ac9d0 2024-07-04T17:26:17.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.375+0000 7fa2b3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2b40ac9d0 0x7fa2b4141e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:17.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.375+0000 7fa2b3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2b40ac9d0 0x7fa2b4141e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34796/0 (socket says 172.21.15.42:34796) 2024-07-04T17:26:17.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.375+0000 7fa2b3e00640 1 -- 172.21.15.42:0/3024701923 learned_addr learned my addr 172.21.15.42:0/3024701923 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:17.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.375+0000 7fa2b3e00640 1 -- 172.21.15.42:0/3024701923 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b4142590 con 0x7fa2b40ac9d0 2024-07-04T17:26:17.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.376+0000 7fa2b3e00640 1 --2- 172.21.15.42:0/3024701923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2b40ac9d0 0x7fa2b4141e60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400cb40 tx=0x7fa2a40021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:17.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.376+0000 7fa2abe00640 1 -- 172.21.15.42:0/3024701923 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2a4016020 con 0x7fa2b40ac9d0 2024-07-04T17:26:17.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.376+0000 7fa2abe00640 1 -- 172.21.15.42:0/3024701923 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2a400ac20 con 0x7fa2b40ac9d0 2024-07-04T17:26:17.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.376+0000 7fa2abe00640 1 -- 172.21.15.42:0/3024701923 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2a4012420 con 0x7fa2b40ac9d0 2024-07-04T17:26:17.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.376+0000 7fa2b9a00640 1 -- 172.21.15.42:0/3024701923 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2b41461f0 con 0x7fa2b40ac9d0 2024-07-04T17:26:17.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.376+0000 7fa2b9a00640 1 -- 172.21.15.42:0/3024701923 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2b40aab20 con 0x7fa2b40ac9d0 2024-07-04T17:26:17.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.377+0000 7fa2b9a00640 1 -- 172.21.15.42:0/3024701923 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2b4141c50 con 0x7fa2b40ac9d0 2024-07-04T17:26:17.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.378+0000 7fa2abe00640 1 -- 172.21.15.42:0/3024701923 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fa2a4019020 con 0x7fa2b40ac9d0 2024-07-04T17:26:17.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.378+0000 7fa2abe00640 1 --2- 172.21.15.42:0/3024701923 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa29403d2b0 0x7fa29403f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:17.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.378+0000 7fa2abe00640 1 -- 172.21.15.42:0/3024701923 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7fa2a4051800 con 0x7fa2b40ac9d0 2024-07-04T17:26:17.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.379+0000 7fa2b3400640 1 --2- 172.21.15.42:0/3024701923 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa29403d2b0 0x7fa29403f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:17.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.379+0000 7fa2b3400640 1 --2- 172.21.15.42:0/3024701923 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa29403d2b0 0x7fa29403f7c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0034000 tx=0x7fa2a0004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:17.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.384+0000 7fa2abe00640 1 -- 172.21.15.42:0/3024701923 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fa2a4020030 con 0x7fa2b40ac9d0 2024-07-04T17:26:17.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:17.635+0000 7fa2b9a00640 1 -- 172.21.15.42:0/3024701923 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa2b4068e30 con 0x7fa2b40ac9d0 2024-07-04T17:26:18.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.025+0000 7fa2abe00640 1 -- 172.21.15.42:0/3024701923 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7fa2b40b1280 con 0x7fa2b40ac9d0 2024-07-04T17:26:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.028+0000 7fa2b9a00640 1 -- 172.21.15.42:0/3024701923 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa29403d2b0 msgr2=0x7fa29403f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.028+0000 7fa2b9a00640 1 --2- 172.21.15.42:0/3024701923 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa29403d2b0 0x7fa29403f7c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0034000 tx=0x7fa2a0004730 comp rx=0 tx=0).stop 2024-07-04T17:26:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.028+0000 7fa2b9a00640 1 -- 172.21.15.42:0/3024701923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2b40ac9d0 msgr2=0x7fa2b4141e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:18.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.028+0000 7fa2b9a00640 1 --2- 172.21.15.42:0/3024701923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2b40ac9d0 0x7fa2b4141e60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400cb40 tx=0x7fa2a40021d0 comp rx=0 tx=0).stop 2024-07-04T17:26:18.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.029+0000 7fa2b9a00640 1 -- 172.21.15.42:0/3024701923 shutdown_connections 2024-07-04T17:26:18.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.029+0000 7fa2b9a00640 1 --2- 172.21.15.42:0/3024701923 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa29403d2b0 0x7fa29403f7c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:18.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.029+0000 7fa2b9a00640 1 --2- 172.21.15.42:0/3024701923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2b40ac9d0 0x7fa2b4141e60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:18.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.029+0000 7fa2b9a00640 1 -- 172.21.15.42:0/3024701923 >> 172.21.15.42:0/3024701923 conn(0x7fa2b40afaa0 msgr2=0x7fa2b413f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:18.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.029+0000 7fa2b9a00640 1 -- 172.21.15.42:0/3024701923 shutdown_connections 2024-07-04T17:26:18.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.029+0000 7fa2b9a00640 1 -- 172.21.15.42:0/3024701923 wait complete. 2024-07-04T17:26:18.059 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-04T17:26:18.060 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-07-04T17:26:18.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.355+0000 7f507d400640 1 -- 172.21.15.42:0/3047029359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50780ab2f0 msgr2=0x7f50780a9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:18.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.355+0000 7f507d400640 1 --2- 172.21.15.42:0/3047029359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50780ab2f0 0x7f50780a9990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f50680097b0 tx=0x7f506802a540 comp rx=0 tx=0).stop 2024-07-04T17:26:18.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.355+0000 7f507d400640 1 -- 172.21.15.42:0/3047029359 shutdown_connections 2024-07-04T17:26:18.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.355+0000 7f507d400640 1 --2- 172.21.15.42:0/3047029359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50780ab2f0 0x7f50780a9990 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:18.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.355+0000 7f507d400640 1 -- 172.21.15.42:0/3047029359 >> 172.21.15.42:0/3047029359 conn(0x7f507806c020 msgr2=0x7f507806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:18.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.355+0000 7f507d400640 1 -- 172.21.15.42:0/3047029359 shutdown_connections 2024-07-04T17:26:18.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.355+0000 7f507d400640 1 -- 172.21.15.42:0/3047029359 wait complete. 2024-07-04T17:26:18.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.358+0000 7f507d400640 1 Processor -- start 2024-07-04T17:26:18.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.358+0000 7f507d400640 1 -- start start 2024-07-04T17:26:18.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.359+0000 7f507d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f507806d070 0x7f50780bb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:18.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.359+0000 7f507d400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f507813f430 con 0x7f507806d070 2024-07-04T17:26:18.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.359+0000 7f5076800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f507806d070 0x7f50780bb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:18.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.359+0000 7f5076800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f507806d070 0x7f50780bb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34824/0 (socket says 172.21.15.42:34824) 2024-07-04T17:26:18.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.359+0000 7f5076800640 1 -- 172.21.15.42:0/500926381 learned_addr learned my addr 172.21.15.42:0/500926381 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:18.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.360+0000 7f5076800640 1 -- 172.21.15.42:0/500926381 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50780af300 con 0x7f507806d070 2024-07-04T17:26:18.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.360+0000 7f5076800640 1 --2- 172.21.15.42:0/500926381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f507806d070 0x7f50780bb610 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f50680098d0 tx=0x7f5068033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:18.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.360+0000 7f506ea00640 1 -- 172.21.15.42:0/500926381 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5068031bc0 con 0x7f507806d070 2024-07-04T17:26:18.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.360+0000 7f506ea00640 1 -- 172.21.15.42:0/500926381 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5068031e50 con 0x7f507806d070 2024-07-04T17:26:18.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.360+0000 7f506ea00640 1 -- 172.21.15.42:0/500926381 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5068033200 con 0x7f507806d070 2024-07-04T17:26:18.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.360+0000 7f507d400640 1 -- 172.21.15.42:0/500926381 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50780bce10 con 0x7f507806d070 2024-07-04T17:26:18.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.361+0000 7f507d400640 1 -- 172.21.15.42:0/500926381 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50780bcc20 con 0x7f507806d070 2024-07-04T17:26:18.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.362+0000 7f506ea00640 1 -- 172.21.15.42:0/500926381 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f50680333a0 con 0x7f507806d070 2024-07-04T17:26:18.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.362+0000 7f507d400640 1 -- 172.21.15.42:0/500926381 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f507813fb00 con 0x7f507806d070 2024-07-04T17:26:18.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.362+0000 7f506ea00640 1 --2- 172.21.15.42:0/500926381 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f505403d2b0 0x7f505403f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:18.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.362+0000 7f506ea00640 1 -- 172.21.15.42:0/500926381 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f5068072a70 con 0x7f507806d070 2024-07-04T17:26:18.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.363+0000 7f5075e00640 1 --2- 172.21.15.42:0/500926381 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f505403d2b0 0x7f505403f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:18.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.363+0000 7f5075e00640 1 --2- 172.21.15.42:0/500926381 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f505403d2b0 0x7f505403f7c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f50600015c0 tx=0x7f5060007a90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:18.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.367+0000 7f506ea00640 1 -- 172.21.15.42:0/500926381 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f506803f030 con 0x7f507806d070 2024-07-04T17:26:18.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:18.623+0000 7f507d400640 1 -- 172.21.15.42:0/500926381 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f50780b4260 con 0x7f507806d070 2024-07-04T17:26:19.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.026+0000 7f506ea00640 1 -- 172.21.15.42:0/500926381 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f5068033690 con 0x7f507806d070 2024-07-04T17:26:19.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.030+0000 7f507d400640 1 -- 172.21.15.42:0/500926381 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f505403d2b0 msgr2=0x7f505403f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:19.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.030+0000 7f507d400640 1 --2- 172.21.15.42:0/500926381 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f505403d2b0 0x7f505403f7c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f50600015c0 tx=0x7f5060007a90 comp rx=0 tx=0).stop 2024-07-04T17:26:19.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.030+0000 7f507d400640 1 -- 172.21.15.42:0/500926381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f507806d070 msgr2=0x7f50780bb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:19.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.030+0000 7f507d400640 1 --2- 172.21.15.42:0/500926381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f507806d070 0x7f50780bb610 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f50680098d0 tx=0x7f5068033e40 comp rx=0 tx=0).stop 2024-07-04T17:26:19.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.030+0000 7f507d400640 1 -- 172.21.15.42:0/500926381 shutdown_connections 2024-07-04T17:26:19.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.030+0000 7f507d400640 1 --2- 172.21.15.42:0/500926381 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f505403d2b0 0x7f505403f7c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:19.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.030+0000 7f507d400640 1 --2- 172.21.15.42:0/500926381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f507806d070 0x7f50780bb610 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:19.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.030+0000 7f507d400640 1 -- 172.21.15.42:0/500926381 >> 172.21.15.42:0/500926381 conn(0x7f507806c020 msgr2=0x7f50780713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:19.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.030+0000 7f507d400640 1 -- 172.21.15.42:0/500926381 shutdown_connections 2024-07-04T17:26:19.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.030+0000 7f507d400640 1 -- 172.21.15.42:0/500926381 wait complete. 2024-07-04T17:26:19.063 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-07-04T17:26:19.064 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:26:19.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.357+0000 7f0c7de00640 1 -- 172.21.15.42:0/1984804259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c780b1760 msgr2=0x7f0c780b8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:19.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.357+0000 7f0c7de00640 1 --2- 172.21.15.42:0/1984804259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c780b1760 0x7f0c780b8da0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0c680097b0 tx=0x7f0c6802a540 comp rx=0 tx=0).stop 2024-07-04T17:26:19.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.357+0000 7f0c7de00640 1 -- 172.21.15.42:0/1984804259 shutdown_connections 2024-07-04T17:26:19.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.357+0000 7f0c7de00640 1 --2- 172.21.15.42:0/1984804259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c780b1760 0x7f0c780b8da0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:19.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.357+0000 7f0c7de00640 1 -- 172.21.15.42:0/1984804259 >> 172.21.15.42:0/1984804259 conn(0x7f0c780bebc0 msgr2=0x7f0c780befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:19.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.357+0000 7f0c7de00640 1 -- 172.21.15.42:0/1984804259 shutdown_connections 2024-07-04T17:26:19.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.357+0000 7f0c7de00640 1 -- 172.21.15.42:0/1984804259 wait complete. 2024-07-04T17:26:19.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.360+0000 7f0c7de00640 1 Processor -- start 2024-07-04T17:26:19.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.360+0000 7f0c7de00640 1 -- start start 2024-07-04T17:26:19.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.361+0000 7f0c7de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c7806c020 0x7f0c781d5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:19.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.361+0000 7f0c7de00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7814eed0 con 0x7f0c7806c020 2024-07-04T17:26:19.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.361+0000 7f0c7ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c7806c020 0x7f0c781d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:19.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.361+0000 7f0c7ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c7806c020 0x7f0c781d5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34832/0 (socket says 172.21.15.42:34832) 2024-07-04T17:26:19.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.361+0000 7f0c7ca00640 1 -- 172.21.15.42:0/23403274 learned_addr learned my addr 172.21.15.42:0/23403274 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:19.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.362+0000 7f0c7ca00640 1 -- 172.21.15.42:0/23403274 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c781d7050 con 0x7f0c7806c020 2024-07-04T17:26:19.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.362+0000 7f0c7ca00640 1 --2- 172.21.15.42:0/23403274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c7806c020 0x7f0c781d5ca0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0c680098d0 tx=0x7f0c68033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:19.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.362+0000 7f0c75600640 1 -- 172.21.15.42:0/23403274 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c68031bc0 con 0x7f0c7806c020 2024-07-04T17:26:19.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.362+0000 7f0c75600640 1 -- 172.21.15.42:0/23403274 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c68031e50 con 0x7f0c7806c020 2024-07-04T17:26:19.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.363+0000 7f0c75600640 1 -- 172.21.15.42:0/23403274 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c68033200 con 0x7f0c7806c020 2024-07-04T17:26:19.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.363+0000 7f0c7de00640 1 -- 172.21.15.42:0/23403274 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c781d6310 con 0x7f0c7806c020 2024-07-04T17:26:19.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.363+0000 7f0c7de00640 1 -- 172.21.15.42:0/23403274 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c781daa80 con 0x7f0c7806c020 2024-07-04T17:26:19.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.364+0000 7f0c7de00640 1 -- 172.21.15.42:0/23403274 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c781dacb0 con 0x7f0c7806c020 2024-07-04T17:26:19.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.364+0000 7f0c75600640 1 -- 172.21.15.42:0/23403274 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f0c680333a0 con 0x7f0c7806c020 2024-07-04T17:26:19.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.364+0000 7f0c75600640 1 --2- 172.21.15.42:0/23403274 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f0c5403d300 0x7f0c5403f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:19.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.364+0000 7f0c75600640 1 -- 172.21.15.42:0/23403274 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f0c68072aa0 con 0x7f0c7806c020 2024-07-04T17:26:19.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.365+0000 7f0c6fe00640 1 --2- 172.21.15.42:0/23403274 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f0c5403d300 0x7f0c5403f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:19.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.366+0000 7f0c6fe00640 1 --2- 172.21.15.42:0/23403274 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f0c5403d300 0x7f0c5403f810 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0c780b8760 tx=0x7f0c60007a90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:19.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.369+0000 7f0c75600640 1 -- 172.21.15.42:0/23403274 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f0c68074e20 con 0x7f0c7806c020 2024-07-04T17:26:19.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.622+0000 7f0c7de00640 1 -- 172.21.15.42:0/23403274 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c780b8da0 con 0x7f0c7806c020 2024-07-04T17:26:19.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.623+0000 7f0c75600640 1 -- 172.21.15.42:0/23403274 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6035 (secure 0 0 0) 0x7f0c6803f030 con 0x7f0c7806c020 2024-07-04T17:26:19.625 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:26:19.625 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":8,"btime":"2024-07-04T17:26:19:025883+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4192,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/3484212194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":3484212194},{"type":"v1","addr":"172.21.15.42:6827","nonce":3484212194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4202,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6830/815222782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6829","nonce":815222782},{"type":"v1","addr":"172.21.15.42:6830","nonce":815222782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/999331079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":999331079},{"type":"v1","addr":"172.21.15.42:6831","nonce":999331079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4204,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6833/2286778970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2286778970},{"type":"v1","addr":"172.21.15.42:6833","nonce":2286778970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:26:15.573602+0000","modified":"2024-07-04T17:26:18.625664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4205":{"gid":4205,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6835/3941225191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3941225191},{"type":"v1","addr":"172.21.15.42:6835","nonce":3941225191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4205,"qdb_cluster":[4205]},"id":1}]} 2024-07-04T17:26:19.625 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 8 2024-07-04T17:26:19.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.626+0000 7f0c7de00640 1 -- 172.21.15.42:0/23403274 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f0c5403d300 msgr2=0x7f0c5403f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:19.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.626+0000 7f0c7de00640 1 --2- 172.21.15.42:0/23403274 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f0c5403d300 0x7f0c5403f810 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0c780b8760 tx=0x7f0c60007a90 comp rx=0 tx=0).stop 2024-07-04T17:26:19.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.626+0000 7f0c7de00640 1 -- 172.21.15.42:0/23403274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c7806c020 msgr2=0x7f0c781d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:19.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.626+0000 7f0c7de00640 1 --2- 172.21.15.42:0/23403274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c7806c020 0x7f0c781d5ca0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0c680098d0 tx=0x7f0c68033e40 comp rx=0 tx=0).stop 2024-07-04T17:26:19.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.626+0000 7f0c7de00640 1 -- 172.21.15.42:0/23403274 shutdown_connections 2024-07-04T17:26:19.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.626+0000 7f0c7de00640 1 --2- 172.21.15.42:0/23403274 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f0c5403d300 0x7f0c5403f810 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:19.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.626+0000 7f0c7de00640 1 --2- 172.21.15.42:0/23403274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c7806c020 0x7f0c781d5ca0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:19.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.626+0000 7f0c7de00640 1 -- 172.21.15.42:0/23403274 >> 172.21.15.42:0/23403274 conn(0x7f0c780bebc0 msgr2=0x7f0c780bbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:19.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.626+0000 7f0c7de00640 1 -- 172.21.15.42:0/23403274 shutdown_connections 2024-07-04T17:26:19.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.626+0000 7f0c7de00640 1 -- 172.21.15.42:0/23403274 wait complete. 2024-07-04T17:26:19.655 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:26:19.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.948+0000 7f2ddfe00640 1 -- 172.21.15.42:0/3908245104 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2de00ba8b0 msgr2=0x7f2de00b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:19.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.948+0000 7f2ddfe00640 1 --2- 172.21.15.42:0/3908245104 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2de00ba8b0 0x7f2de00b3550 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2dc80097b0 tx=0x7f2dc802a540 comp rx=0 tx=0).stop 2024-07-04T17:26:19.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.948+0000 7f2ddfe00640 1 -- 172.21.15.42:0/3908245104 shutdown_connections 2024-07-04T17:26:19.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.948+0000 7f2ddfe00640 1 --2- 172.21.15.42:0/3908245104 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2de00ba8b0 0x7f2de00b3550 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:19.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.948+0000 7f2ddfe00640 1 -- 172.21.15.42:0/3908245104 >> 172.21.15.42:0/3908245104 conn(0x7f2de00bebc0 msgr2=0x7f2de00befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:19.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.948+0000 7f2ddfe00640 1 -- 172.21.15.42:0/3908245104 shutdown_connections 2024-07-04T17:26:19.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.948+0000 7f2ddfe00640 1 -- 172.21.15.42:0/3908245104 wait complete. 2024-07-04T17:26:19.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.950+0000 7f2ddfe00640 1 Processor -- start 2024-07-04T17:26:19.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.950+0000 7f2ddfe00640 1 -- start start 2024-07-04T17:26:19.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.951+0000 7f2ddfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2de00b1d20 0x7f2de00ae790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:19.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.951+0000 7f2ddfe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2de00b8b40 con 0x7f2de00b1d20 2024-07-04T17:26:19.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.952+0000 7f2ddea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2de00b1d20 0x7f2de00ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:19.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.952+0000 7f2ddea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2de00b1d20 0x7f2de00ae790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34848/0 (socket says 172.21.15.42:34848) 2024-07-04T17:26:19.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.952+0000 7f2ddea00640 1 -- 172.21.15.42:0/2008747599 learned_addr learned my addr 172.21.15.42:0/2008747599 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:19.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.952+0000 7f2ddea00640 1 -- 172.21.15.42:0/2008747599 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de01d6380 con 0x7f2de00b1d20 2024-07-04T17:26:19.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.952+0000 7f2ddea00640 1 --2- 172.21.15.42:0/2008747599 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2de00b1d20 0x7f2de00ae790 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2dc80098d0 tx=0x7f2dc8033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:19.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.953+0000 7f2dd7400640 1 -- 172.21.15.42:0/2008747599 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2dc8031bc0 con 0x7f2de00b1d20 2024-07-04T17:26:19.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.953+0000 7f2dd7400640 1 -- 172.21.15.42:0/2008747599 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2dc8031e50 con 0x7f2de00b1d20 2024-07-04T17:26:19.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.953+0000 7f2dd7400640 1 -- 172.21.15.42:0/2008747599 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2dc8033200 con 0x7f2de00b1d20 2024-07-04T17:26:19.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.953+0000 7f2ddfe00640 1 -- 172.21.15.42:0/2008747599 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2de01d5520 con 0x7f2de00b1d20 2024-07-04T17:26:19.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.953+0000 7f2ddfe00640 1 -- 172.21.15.42:0/2008747599 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2de01dcb20 con 0x7f2de00b1d20 2024-07-04T17:26:19.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.954+0000 7f2ddfe00640 1 -- 172.21.15.42:0/2008747599 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2de00b55d0 con 0x7f2de00b1d20 2024-07-04T17:26:19.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.954+0000 7f2dd7400640 1 -- 172.21.15.42:0/2008747599 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f2dc80333a0 con 0x7f2de00b1d20 2024-07-04T17:26:19.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.955+0000 7f2dd7400640 1 --2- 172.21.15.42:0/2008747599 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f2dbc03d2b0 0x7f2dbc03f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:19.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.955+0000 7f2dd7400640 1 -- 172.21.15.42:0/2008747599 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f2dc8072a50 con 0x7f2de00b1d20 2024-07-04T17:26:19.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.955+0000 7f2dde000640 1 --2- 172.21.15.42:0/2008747599 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f2dbc03d2b0 0x7f2dbc03f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:19.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.956+0000 7f2dde000640 1 --2- 172.21.15.42:0/2008747599 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f2dbc03d2b0 0x7f2dbc03f7c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2de01442c0 tx=0x7f2dd0007a90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:19.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:19.960+0000 7f2dd7400640 1 -- 172.21.15.42:0/2008747599 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f2de00b55d0 con 0x7f2de00b1d20 2024-07-04T17:26:20.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.168+0000 7f2ddfe00640 1 -- 172.21.15.42:0/2008747599 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2de00b9240 con 0x7f2de00b1d20 2024-07-04T17:26:20.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.171+0000 7f2dd7400640 1 -- 172.21.15.42:0/2008747599 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) ==== 74+0+10603 (secure 0 0 0) 0x7f2dc803f030 con 0x7f2de00b1d20 2024-07-04T17:26:20.173 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:26:20.173 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":15,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:26:17.008553+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:26:13.335715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-04T17:26:14.519899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:26:20.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.173+0000 7f2ddfe00640 1 -- 172.21.15.42:0/2008747599 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f2dbc03d2b0 msgr2=0x7f2dbc03f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:20.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.173+0000 7f2ddfe00640 1 --2- 172.21.15.42:0/2008747599 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f2dbc03d2b0 0x7f2dbc03f7c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2de01442c0 tx=0x7f2dd0007a90 comp rx=0 tx=0).stop 2024-07-04T17:26:20.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.173+0000 7f2ddfe00640 1 -- 172.21.15.42:0/2008747599 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2de00b1d20 msgr2=0x7f2de00ae790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:20.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.173+0000 7f2ddfe00640 1 --2- 172.21.15.42:0/2008747599 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2de00b1d20 0x7f2de00ae790 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2dc80098d0 tx=0x7f2dc8033e40 comp rx=0 tx=0).stop 2024-07-04T17:26:20.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.174+0000 7f2ddfe00640 1 -- 172.21.15.42:0/2008747599 shutdown_connections 2024-07-04T17:26:20.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.174+0000 7f2ddfe00640 1 --2- 172.21.15.42:0/2008747599 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f2dbc03d2b0 0x7f2dbc03f7c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:20.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.174+0000 7f2ddfe00640 1 --2- 172.21.15.42:0/2008747599 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2de00b1d20 0x7f2de00ae790 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:20.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.174+0000 7f2ddfe00640 1 -- 172.21.15.42:0/2008747599 >> 172.21.15.42:0/2008747599 conn(0x7f2de00bebc0 msgr2=0x7f2de013edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:20.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.174+0000 7f2ddfe00640 1 -- 172.21.15.42:0/2008747599 shutdown_connections 2024-07-04T17:26:20.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.174+0000 7f2ddfe00640 1 -- 172.21.15.42:0/2008747599 wait complete. 2024-07-04T17:26:20.199 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-04T17:26:20.200 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:26:20.378 INFO:tasks.ceph.mds.c.smithi042.stderr:2024-07-04T17:26:20.376+0000 7f6272a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-04T17:26:20.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.482+0000 7f1f9d600640 1 -- 172.21.15.42:0/1391308205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f980b0cd0 msgr2=0x7f1f980b34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:20.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.482+0000 7f1f9d600640 1 --2- 172.21.15.42:0/1391308205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f980b0cd0 0x7f1f980b34b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000a8c0 tx=0x7f1f8002b860 comp rx=0 tx=0).stop 2024-07-04T17:26:20.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.483+0000 7f1f9d600640 1 -- 172.21.15.42:0/1391308205 shutdown_connections 2024-07-04T17:26:20.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.483+0000 7f1f9d600640 1 --2- 172.21.15.42:0/1391308205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f980b0cd0 0x7f1f980b34b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:20.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.483+0000 7f1f9d600640 1 -- 172.21.15.42:0/1391308205 >> 172.21.15.42:0/1391308205 conn(0x7f1f980bf700 msgr2=0x7f1f980af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:20.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.483+0000 7f1f9d600640 1 -- 172.21.15.42:0/1391308205 shutdown_connections 2024-07-04T17:26:20.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.483+0000 7f1f9d600640 1 -- 172.21.15.42:0/1391308205 wait complete. 2024-07-04T17:26:20.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.485+0000 7f1f9d600640 1 Processor -- start 2024-07-04T17:26:20.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.485+0000 7f1f9d600640 1 -- start start 2024-07-04T17:26:20.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.486+0000 7f1f9d600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f980abdd0 0x7f1f98072100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:20.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.486+0000 7f1f9d600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1f980b03a0 con 0x7f1f980abdd0 2024-07-04T17:26:20.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.487+0000 7f1f96a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f980abdd0 0x7f1f98072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:20.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.487+0000 7f1f96a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f980abdd0 0x7f1f98072100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34860/0 (socket says 172.21.15.42:34860) 2024-07-04T17:26:20.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.487+0000 7f1f96a00640 1 -- 172.21.15.42:0/381261048 learned_addr learned my addr 172.21.15.42:0/381261048 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:20.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.487+0000 7f1f96a00640 1 -- 172.21.15.42:0/381261048 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f980b8900 con 0x7f1f980abdd0 2024-07-04T17:26:20.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.487+0000 7f1f96a00640 1 --2- 172.21.15.42:0/381261048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f980abdd0 0x7f1f98072100 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1f8800c380 tx=0x7f1f8800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:20.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.488+0000 7f1f8ea00640 1 -- 172.21.15.42:0/381261048 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f880116c0 con 0x7f1f980abdd0 2024-07-04T17:26:20.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.488+0000 7f1f8ea00640 1 -- 172.21.15.42:0/381261048 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f88017070 con 0x7f1f980abdd0 2024-07-04T17:26:20.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.488+0000 7f1f9d600640 1 -- 172.21.15.42:0/381261048 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f980726d0 con 0x7f1f980abdd0 2024-07-04T17:26:20.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.488+0000 7f1f8ea00640 1 -- 172.21.15.42:0/381261048 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f88011b20 con 0x7f1f980abdd0 2024-07-04T17:26:20.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.488+0000 7f1f9d600640 1 -- 172.21.15.42:0/381261048 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f981dad70 con 0x7f1f980abdd0 2024-07-04T17:26:20.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.489+0000 7f1f9d600640 1 -- 172.21.15.42:0/381261048 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f980b2da0 con 0x7f1f980abdd0 2024-07-04T17:26:20.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.489+0000 7f1f8ea00640 1 -- 172.21.15.42:0/381261048 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f1f88011cc0 con 0x7f1f980abdd0 2024-07-04T17:26:20.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.490+0000 7f1f8ea00640 1 --2- 172.21.15.42:0/381261048 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f1f7403d2b0 0x7f1f7403f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:20.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.490+0000 7f1f8ea00640 1 -- 172.21.15.42:0/381261048 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f1f88051280 con 0x7f1f980abdd0 2024-07-04T17:26:20.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.490+0000 7f1f8fe00640 1 --2- 172.21.15.42:0/381261048 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f1f7403d2b0 0x7f1f7403f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:20.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.491+0000 7f1f8fe00640 1 --2- 172.21.15.42:0/381261048 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f1f7403d2b0 0x7f1f7403f7c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1f80036000 tx=0x7f1f80004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:20.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.495+0000 7f1f8ea00640 1 -- 172.21.15.42:0/381261048 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f1f980b10e0 con 0x7f1f980abdd0 2024-07-04T17:26:20.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.699+0000 7f1f9d600640 1 -- 172.21.15.42:0/381261048 --> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1f980b83b0 con 0x7f1f7403d2b0 2024-07-04T17:26:20.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.707+0000 7f1f8ea00640 1 -- 172.21.15.42:0/381261048 <== mgr.4103 v2:172.21.15.42:6824/3523793950 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245956 (secure 0 0 0) 0x7f1f980b83b0 con 0x7f1f7403d2b0 2024-07-04T17:26:20.712 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:26:20.714 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:26:20.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.712+0000 7f1f9d600640 1 -- 172.21.15.42:0/381261048 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f1f7403d2b0 msgr2=0x7f1f7403f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:20.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.712+0000 7f1f9d600640 1 --2- 172.21.15.42:0/381261048 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f1f7403d2b0 0x7f1f7403f7c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1f80036000 tx=0x7f1f80004730 comp rx=0 tx=0).stop 2024-07-04T17:26:20.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.712+0000 7f1f9d600640 1 -- 172.21.15.42:0/381261048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f980abdd0 msgr2=0x7f1f98072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:20.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.712+0000 7f1f9d600640 1 --2- 172.21.15.42:0/381261048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f980abdd0 0x7f1f98072100 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1f8800c380 tx=0x7f1f8800c840 comp rx=0 tx=0).stop 2024-07-04T17:26:20.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.712+0000 7f1f9d600640 1 -- 172.21.15.42:0/381261048 shutdown_connections 2024-07-04T17:26:20.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.712+0000 7f1f9d600640 1 --2- 172.21.15.42:0/381261048 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f1f7403d2b0 0x7f1f7403f7c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:20.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.712+0000 7f1f9d600640 1 --2- 172.21.15.42:0/381261048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f980abdd0 0x7f1f98072100 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:20.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.712+0000 7f1f9d600640 1 -- 172.21.15.42:0/381261048 >> 172.21.15.42:0/381261048 conn(0x7f1f980bf700 msgr2=0x7f1f980aaba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:20.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.713+0000 7f1f9d600640 1 -- 172.21.15.42:0/381261048 shutdown_connections 2024-07-04T17:26:20.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:20.713+0000 7f1f9d600640 1 -- 172.21.15.42:0/381261048 wait complete. 2024-07-04T17:26:20.739 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-07-04T17:26:19.262920+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":124,"num_write_kb":1120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91,"ondisk_log_size":91,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":120,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":95164,"kb_used_data":1608,"kb_used_omap":4,"kb_used_meta":93499,"kb_avail":283020356,"statfs":{"total":289910292480,"available":289812844544,"internally_reserved":0,"allocated":1646592,"data_stored":1306663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":95743311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.281772"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.023948+0000","last_change":"2024-07-04T17:26:16.062462+0000","last_active":"2024-07-04T17:26:17.023948+0000","last_peered":"2024-07-04T17:26:17.023948+0000","last_clean":"2024-07-04T17:26:17.023948+0000","last_became_active":"2024-07-04T17:26:16.058531+0000","last_became_peered":"2024-07-04T17:26:16.058531+0000","last_unstale":"2024-07-04T17:26:17.023948+0000","last_undegraded":"2024-07-04T17:26:17.023948+0000","last_fullsized":"2024-07-04T17:26:17.023948+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:31:33.921497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.028401+0000","last_change":"2024-07-04T17:26:16.064253+0000","last_active":"2024-07-04T17:26:17.028401+0000","last_peered":"2024-07-04T17:26:17.028401+0000","last_clean":"2024-07-04T17:26:17.028401+0000","last_became_active":"2024-07-04T17:26:16.063971+0000","last_became_peered":"2024-07-04T17:26:16.063971+0000","last_unstale":"2024-07-04T17:26:17.028401+0000","last_undegraded":"2024-07-04T17:26:17.028401+0000","last_fullsized":"2024-07-04T17:26:17.028401+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:22:14.191409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022213+0000","last_change":"2024-07-04T17:26:16.059637+0000","last_active":"2024-07-04T17:26:17.022213+0000","last_peered":"2024-07-04T17:26:17.022213+0000","last_clean":"2024-07-04T17:26:17.022213+0000","last_became_active":"2024-07-04T17:26:16.058613+0000","last_became_peered":"2024-07-04T17:26:16.058613+0000","last_unstale":"2024-07-04T17:26:17.022213+0000","last_undegraded":"2024-07-04T17:26:17.022213+0000","last_fullsized":"2024-07-04T17:26:17.022213+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:02:40.668757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702530+0000","last_change":"2024-07-04T17:26:16.072904+0000","last_active":"2024-07-04T17:26:17.702530+0000","last_peered":"2024-07-04T17:26:17.702530+0000","last_clean":"2024-07-04T17:26:17.702530+0000","last_became_active":"2024-07-04T17:26:16.072640+0000","last_became_peered":"2024-07-04T17:26:16.072640+0000","last_unstale":"2024-07-04T17:26:17.702530+0000","last_undegraded":"2024-07-04T17:26:17.702530+0000","last_fullsized":"2024-07-04T17:26:17.702530+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:03:07.909265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021289+0000","last_change":"2024-07-04T17:26:16.044297+0000","last_active":"2024-07-04T17:26:17.021289+0000","last_peered":"2024-07-04T17:26:17.021289+0000","last_clean":"2024-07-04T17:26:17.021289+0000","last_became_active":"2024-07-04T17:26:16.043526+0000","last_became_peered":"2024-07-04T17:26:16.043526+0000","last_unstale":"2024-07-04T17:26:17.021289+0000","last_undegraded":"2024-07-04T17:26:17.021289+0000","last_fullsized":"2024-07-04T17:26:17.021289+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:11:11.392035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702192+0000","last_change":"2024-07-04T17:26:16.072472+0000","last_active":"2024-07-04T17:26:17.702192+0000","last_peered":"2024-07-04T17:26:17.702192+0000","last_clean":"2024-07-04T17:26:17.702192+0000","last_became_active":"2024-07-04T17:26:16.072066+0000","last_became_peered":"2024-07-04T17:26:16.072066+0000","last_unstale":"2024-07-04T17:26:17.702192+0000","last_undegraded":"2024-07-04T17:26:17.702192+0000","last_fullsized":"2024-07-04T17:26:17.702192+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:11:17.509522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021458+0000","last_change":"2024-07-04T17:26:16.070453+0000","last_active":"2024-07-04T17:26:17.021458+0000","last_peered":"2024-07-04T17:26:17.021458+0000","last_clean":"2024-07-04T17:26:17.021458+0000","last_became_active":"2024-07-04T17:26:16.065958+0000","last_became_peered":"2024-07-04T17:26:16.065958+0000","last_unstale":"2024-07-04T17:26:17.021458+0000","last_undegraded":"2024-07-04T17:26:17.021458+0000","last_fullsized":"2024-07-04T17:26:17.021458+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:51:59.570861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.700790+0000","last_change":"2024-07-04T17:26:16.072492+0000","last_active":"2024-07-04T17:26:17.700790+0000","last_peered":"2024-07-04T17:26:17.700790+0000","last_clean":"2024-07-04T17:26:17.700790+0000","last_became_active":"2024-07-04T17:26:16.067345+0000","last_became_peered":"2024-07-04T17:26:16.067345+0000","last_unstale":"2024-07-04T17:26:17.700790+0000","last_undegraded":"2024-07-04T17:26:17.700790+0000","last_fullsized":"2024-07-04T17:26:17.700790+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:02:12.965714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.028868+0000","last_change":"2024-07-04T17:26:16.055031+0000","last_active":"2024-07-04T17:26:17.028868+0000","last_peered":"2024-07-04T17:26:17.028868+0000","last_clean":"2024-07-04T17:26:17.028868+0000","last_became_active":"2024-07-04T17:26:16.051647+0000","last_became_peered":"2024-07-04T17:26:16.051647+0000","last_unstale":"2024-07-04T17:26:17.028868+0000","last_undegraded":"2024-07-04T17:26:17.028868+0000","last_fullsized":"2024-07-04T17:26:17.028868+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:41:20.224285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022523+0000","last_change":"2024-07-04T17:26:16.045710+0000","last_active":"2024-07-04T17:26:17.022523+0000","last_peered":"2024-07-04T17:26:17.022523+0000","last_clean":"2024-07-04T17:26:17.022523+0000","last_became_active":"2024-07-04T17:26:16.045371+0000","last_became_peered":"2024-07-04T17:26:16.045371+0000","last_unstale":"2024-07-04T17:26:17.022523+0000","last_undegraded":"2024-07-04T17:26:17.022523+0000","last_fullsized":"2024-07-04T17:26:17.022523+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:53:44.096508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022006+0000","last_change":"2024-07-04T17:26:16.059936+0000","last_active":"2024-07-04T17:26:17.022006+0000","last_peered":"2024-07-04T17:26:17.022006+0000","last_clean":"2024-07-04T17:26:17.022006+0000","last_became_active":"2024-07-04T17:26:16.058990+0000","last_became_peered":"2024-07-04T17:26:16.058990+0000","last_unstale":"2024-07-04T17:26:17.022006+0000","last_undegraded":"2024-07-04T17:26:17.022006+0000","last_fullsized":"2024-07-04T17:26:17.022006+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:47:09.376866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022753+0000","last_change":"2024-07-04T17:26:16.053390+0000","last_active":"2024-07-04T17:26:17.022753+0000","last_peered":"2024-07-04T17:26:17.022753+0000","last_clean":"2024-07-04T17:26:17.022753+0000","last_became_active":"2024-07-04T17:26:16.052716+0000","last_became_peered":"2024-07-04T17:26:16.052716+0000","last_unstale":"2024-07-04T17:26:17.022753+0000","last_undegraded":"2024-07-04T17:26:17.022753+0000","last_fullsized":"2024-07-04T17:26:17.022753+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:15:52.371580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021947+0000","last_change":"2024-07-04T17:26:16.037676+0000","last_active":"2024-07-04T17:26:17.021947+0000","last_peered":"2024-07-04T17:26:17.021947+0000","last_clean":"2024-07-04T17:26:17.021947+0000","last_became_active":"2024-07-04T17:26:16.037311+0000","last_became_peered":"2024-07-04T17:26:16.037311+0000","last_unstale":"2024-07-04T17:26:17.021947+0000","last_undegraded":"2024-07-04T17:26:17.021947+0000","last_fullsized":"2024-07-04T17:26:17.021947+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:31:49.427076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020967+0000","last_change":"2024-07-04T17:26:16.064558+0000","last_active":"2024-07-04T17:26:17.020967+0000","last_peered":"2024-07-04T17:26:17.020967+0000","last_clean":"2024-07-04T17:26:17.020967+0000","last_became_active":"2024-07-04T17:26:16.054068+0000","last_became_peered":"2024-07-04T17:26:16.054068+0000","last_unstale":"2024-07-04T17:26:17.020967+0000","last_undegraded":"2024-07-04T17:26:17.020967+0000","last_fullsized":"2024-07-04T17:26:17.020967+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:30:12.289936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026054+0000","last_change":"2024-07-04T17:26:16.056551+0000","last_active":"2024-07-04T17:26:17.026054+0000","last_peered":"2024-07-04T17:26:17.026054+0000","last_clean":"2024-07-04T17:26:17.026054+0000","last_became_active":"2024-07-04T17:26:16.056173+0000","last_became_peered":"2024-07-04T17:26:16.056173+0000","last_unstale":"2024-07-04T17:26:17.026054+0000","last_undegraded":"2024-07-04T17:26:17.026054+0000","last_fullsized":"2024-07-04T17:26:17.026054+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:20:14.576761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.700980+0000","last_change":"2024-07-04T17:26:16.073053+0000","last_active":"2024-07-04T17:26:17.700980+0000","last_peered":"2024-07-04T17:26:17.700980+0000","last_clean":"2024-07-04T17:26:17.700980+0000","last_became_active":"2024-07-04T17:26:16.072132+0000","last_became_peered":"2024-07-04T17:26:16.072132+0000","last_unstale":"2024-07-04T17:26:17.700980+0000","last_undegraded":"2024-07-04T17:26:17.700980+0000","last_fullsized":"2024-07-04T17:26:17.700980+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:11:05.657045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.029516+0000","last_change":"2024-07-04T17:26:16.055789+0000","last_active":"2024-07-04T17:26:17.029516+0000","last_peered":"2024-07-04T17:26:17.029516+0000","last_clean":"2024-07-04T17:26:17.029516+0000","last_became_active":"2024-07-04T17:26:16.055206+0000","last_became_peered":"2024-07-04T17:26:16.055206+0000","last_unstale":"2024-07-04T17:26:17.029516+0000","last_undegraded":"2024-07-04T17:26:17.029516+0000","last_fullsized":"2024-07-04T17:26:17.029516+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:31:59.564668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.703184+0000","last_change":"2024-07-04T17:26:16.070972+0000","last_active":"2024-07-04T17:26:17.703184+0000","last_peered":"2024-07-04T17:26:17.703184+0000","last_clean":"2024-07-04T17:26:17.703184+0000","last_became_active":"2024-07-04T17:26:16.067224+0000","last_became_peered":"2024-07-04T17:26:16.067224+0000","last_unstale":"2024-07-04T17:26:17.703184+0000","last_undegraded":"2024-07-04T17:26:17.703184+0000","last_fullsized":"2024-07-04T17:26:17.703184+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:22:14.191409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022486+0000","last_change":"2024-07-04T17:26:16.059368+0000","last_active":"2024-07-04T17:26:17.022486+0000","last_peered":"2024-07-04T17:26:17.022486+0000","last_clean":"2024-07-04T17:26:17.022486+0000","last_became_active":"2024-07-04T17:26:16.058568+0000","last_became_peered":"2024-07-04T17:26:16.058568+0000","last_unstale":"2024-07-04T17:26:17.022486+0000","last_undegraded":"2024-07-04T17:26:17.022486+0000","last_fullsized":"2024-07-04T17:26:17.022486+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:18:06.133593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022008+0000","last_change":"2024-07-04T17:26:16.083687+0000","last_active":"2024-07-04T17:26:17.022008+0000","last_peered":"2024-07-04T17:26:17.022008+0000","last_clean":"2024-07-04T17:26:17.022008+0000","last_became_active":"2024-07-04T17:26:16.083145+0000","last_became_peered":"2024-07-04T17:26:16.083145+0000","last_unstale":"2024-07-04T17:26:17.022008+0000","last_undegraded":"2024-07-04T17:26:17.022008+0000","last_fullsized":"2024-07-04T17:26:17.022008+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:36:50.247043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020646+0000","last_change":"2024-07-04T17:26:16.047023+0000","last_active":"2024-07-04T17:26:17.020646+0000","last_peered":"2024-07-04T17:26:17.020646+0000","last_clean":"2024-07-04T17:26:17.020646+0000","last_became_active":"2024-07-04T17:26:16.046631+0000","last_became_peered":"2024-07-04T17:26:16.046631+0000","last_unstale":"2024-07-04T17:26:17.020646+0000","last_undegraded":"2024-07-04T17:26:17.020646+0000","last_fullsized":"2024-07-04T17:26:17.020646+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:13:00.963495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026306+0000","last_change":"2024-07-04T17:26:16.075119+0000","last_active":"2024-07-04T17:26:17.026306+0000","last_peered":"2024-07-04T17:26:17.026306+0000","last_clean":"2024-07-04T17:26:17.026306+0000","last_became_active":"2024-07-04T17:26:16.074600+0000","last_became_peered":"2024-07-04T17:26:16.074600+0000","last_unstale":"2024-07-04T17:26:17.026306+0000","last_undegraded":"2024-07-04T17:26:17.026306+0000","last_fullsized":"2024-07-04T17:26:17.026306+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:17:18.408614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021060+0000","last_change":"2024-07-04T17:26:16.052698+0000","last_active":"2024-07-04T17:26:17.021060+0000","last_peered":"2024-07-04T17:26:17.021060+0000","last_clean":"2024-07-04T17:26:17.021060+0000","last_became_active":"2024-07-04T17:26:16.052278+0000","last_became_peered":"2024-07-04T17:26:16.052278+0000","last_unstale":"2024-07-04T17:26:17.021060+0000","last_undegraded":"2024-07-04T17:26:17.021060+0000","last_fullsized":"2024-07-04T17:26:17.021060+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:27:20.732374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020541+0000","last_change":"2024-07-04T17:26:16.052254+0000","last_active":"2024-07-04T17:26:17.020541+0000","last_peered":"2024-07-04T17:26:17.020541+0000","last_clean":"2024-07-04T17:26:17.020541+0000","last_became_active":"2024-07-04T17:26:16.046560+0000","last_became_peered":"2024-07-04T17:26:16.046560+0000","last_unstale":"2024-07-04T17:26:17.020541+0000","last_undegraded":"2024-07-04T17:26:17.020541+0000","last_fullsized":"2024-07-04T17:26:17.020541+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:50:25.908300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702155+0000","last_change":"2024-07-04T17:26:16.061658+0000","last_active":"2024-07-04T17:26:17.702155+0000","last_peered":"2024-07-04T17:26:17.702155+0000","last_clean":"2024-07-04T17:26:17.702155+0000","last_became_active":"2024-07-04T17:26:16.061257+0000","last_became_peered":"2024-07-04T17:26:16.061257+0000","last_unstale":"2024-07-04T17:26:17.702155+0000","last_undegraded":"2024-07-04T17:26:17.702155+0000","last_fullsized":"2024-07-04T17:26:17.702155+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:17:18.408614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.703496+0000","last_change":"2024-07-04T17:26:16.073828+0000","last_active":"2024-07-04T17:26:17.703496+0000","last_peered":"2024-07-04T17:26:17.703496+0000","last_clean":"2024-07-04T17:26:17.703496+0000","last_became_active":"2024-07-04T17:26:16.073494+0000","last_became_peered":"2024-07-04T17:26:16.073494+0000","last_unstale":"2024-07-04T17:26:17.703496+0000","last_undegraded":"2024-07-04T17:26:17.703496+0000","last_fullsized":"2024-07-04T17:26:17.703496+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:41:20.224285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.028237+0000","last_change":"2024-07-04T17:26:16.046050+0000","last_active":"2024-07-04T17:26:17.028237+0000","last_peered":"2024-07-04T17:26:17.028237+0000","last_clean":"2024-07-04T17:26:17.028237+0000","last_became_active":"2024-07-04T17:26:16.043726+0000","last_became_peered":"2024-07-04T17:26:16.043726+0000","last_unstale":"2024-07-04T17:26:17.028237+0000","last_undegraded":"2024-07-04T17:26:17.028237+0000","last_fullsized":"2024-07-04T17:26:17.028237+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:43:29.877017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022171+0000","last_change":"2024-07-04T17:26:16.047247+0000","last_active":"2024-07-04T17:26:17.022171+0000","last_peered":"2024-07-04T17:26:17.022171+0000","last_clean":"2024-07-04T17:26:17.022171+0000","last_became_active":"2024-07-04T17:26:16.046845+0000","last_became_peered":"2024-07-04T17:26:16.046845+0000","last_unstale":"2024-07-04T17:26:17.022171+0000","last_undegraded":"2024-07-04T17:26:17.022171+0000","last_fullsized":"2024-07-04T17:26:17.022171+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:22:33.108870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020889+0000","last_change":"2024-07-04T17:26:16.047621+0000","last_active":"2024-07-04T17:26:17.020889+0000","last_peered":"2024-07-04T17:26:17.020889+0000","last_clean":"2024-07-04T17:26:17.020889+0000","last_became_active":"2024-07-04T17:26:16.047328+0000","last_became_peered":"2024-07-04T17:26:16.047328+0000","last_unstale":"2024-07-04T17:26:17.020889+0000","last_undegraded":"2024-07-04T17:26:17.020889+0000","last_fullsized":"2024-07-04T17:26:17.020889+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:03:58.364101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020557+0000","last_change":"2024-07-04T17:26:16.046513+0000","last_active":"2024-07-04T17:26:17.020557+0000","last_peered":"2024-07-04T17:26:17.020557+0000","last_clean":"2024-07-04T17:26:17.020557+0000","last_became_active":"2024-07-04T17:26:16.045399+0000","last_became_peered":"2024-07-04T17:26:16.045399+0000","last_unstale":"2024-07-04T17:26:17.020557+0000","last_undegraded":"2024-07-04T17:26:17.020557+0000","last_fullsized":"2024-07-04T17:26:17.020557+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:31:59.564668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.701258+0000","last_change":"2024-07-04T17:26:16.057205+0000","last_active":"2024-07-04T17:26:17.701258+0000","last_peered":"2024-07-04T17:26:17.701258+0000","last_clean":"2024-07-04T17:26:17.701258+0000","last_became_active":"2024-07-04T17:26:16.056891+0000","last_became_peered":"2024-07-04T17:26:16.056891+0000","last_unstale":"2024-07-04T17:26:17.701258+0000","last_undegraded":"2024-07-04T17:26:17.701258+0000","last_fullsized":"2024-07-04T17:26:17.701258+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:16:58.662657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.701357+0000","last_change":"2024-07-04T17:26:16.072774+0000","last_active":"2024-07-04T17:26:17.701357+0000","last_peered":"2024-07-04T17:26:17.701357+0000","last_clean":"2024-07-04T17:26:17.701357+0000","last_became_active":"2024-07-04T17:26:16.071711+0000","last_became_peered":"2024-07-04T17:26:16.071711+0000","last_unstale":"2024-07-04T17:26:17.701357+0000","last_undegraded":"2024-07-04T17:26:17.701357+0000","last_fullsized":"2024-07-04T17:26:17.701357+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:20:14.576761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026507+0000","last_change":"2024-07-04T17:26:16.047214+0000","last_active":"2024-07-04T17:26:17.026507+0000","last_peered":"2024-07-04T17:26:17.026507+0000","last_clean":"2024-07-04T17:26:17.026507+0000","last_became_active":"2024-07-04T17:26:16.046836+0000","last_became_peered":"2024-07-04T17:26:16.046836+0000","last_unstale":"2024-07-04T17:26:17.026507+0000","last_undegraded":"2024-07-04T17:26:17.026507+0000","last_fullsized":"2024-07-04T17:26:17.026507+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:04:04.805872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"14'1","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.027730+0000","last_change":"2024-07-04T17:26:15.033936+0000","last_active":"2024-07-04T17:26:17.027730+0000","last_peered":"2024-07-04T17:26:17.027730+0000","last_clean":"2024-07-04T17:26:17.027730+0000","last_became_active":"2024-07-04T17:26:15.033601+0000","last_became_peered":"2024-07-04T17:26:15.033601+0000","last_unstale":"2024-07-04T17:26:17.027730+0000","last_undegraded":"2024-07-04T17:26:17.027730+0000","last_fullsized":"2024-07-04T17:26:17.027730+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:17:16.904464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021045+0000","last_change":"2024-07-04T17:26:16.053083+0000","last_active":"2024-07-04T17:26:17.021045+0000","last_peered":"2024-07-04T17:26:17.021045+0000","last_clean":"2024-07-04T17:26:17.021045+0000","last_became_active":"2024-07-04T17:26:16.052739+0000","last_became_peered":"2024-07-04T17:26:16.052739+0000","last_unstale":"2024-07-04T17:26:17.021045+0000","last_undegraded":"2024-07-04T17:26:17.021045+0000","last_fullsized":"2024-07-04T17:26:17.021045+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:00:52.128797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"14'1","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.701569+0000","last_change":"2024-07-04T17:26:15.039637+0000","last_active":"2024-07-04T17:26:17.701569+0000","last_peered":"2024-07-04T17:26:17.701569+0000","last_clean":"2024-07-04T17:26:17.701569+0000","last_became_active":"2024-07-04T17:26:15.039305+0000","last_became_peered":"2024-07-04T17:26:15.039305+0000","last_unstale":"2024-07-04T17:26:17.701569+0000","last_undegraded":"2024-07-04T17:26:17.701569+0000","last_fullsized":"2024-07-04T17:26:17.701569+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:04:46.667644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021302+0000","last_change":"2024-07-04T17:26:15.027075+0000","last_active":"2024-07-04T17:26:17.021302+0000","last_peered":"2024-07-04T17:26:17.021302+0000","last_clean":"2024-07-04T17:26:17.021302+0000","last_became_active":"2024-07-04T17:26:15.026375+0000","last_became_peered":"2024-07-04T17:26:15.026375+0000","last_unstale":"2024-07-04T17:26:17.021302+0000","last_undegraded":"2024-07-04T17:26:17.021302+0000","last_fullsized":"2024-07-04T17:26:17.021302+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:24:04.318931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702805+0000","last_change":"2024-07-04T17:26:16.074834+0000","last_active":"2024-07-04T17:26:17.702805+0000","last_peered":"2024-07-04T17:26:17.702805+0000","last_clean":"2024-07-04T17:26:17.702805+0000","last_became_active":"2024-07-04T17:26:16.074498+0000","last_became_peered":"2024-07-04T17:26:16.074498+0000","last_unstale":"2024-07-04T17:26:17.702805+0000","last_undegraded":"2024-07-04T17:26:17.702805+0000","last_fullsized":"2024-07-04T17:26:17.702805+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:08:53.606368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020171+0000","last_change":"2024-07-04T17:26:15.046968+0000","last_active":"2024-07-04T17:26:17.020171+0000","last_peered":"2024-07-04T17:26:17.020171+0000","last_clean":"2024-07-04T17:26:17.020171+0000","last_became_active":"2024-07-04T17:26:15.046617+0000","last_became_peered":"2024-07-04T17:26:15.046617+0000","last_unstale":"2024-07-04T17:26:17.020171+0000","last_undegraded":"2024-07-04T17:26:17.020171+0000","last_fullsized":"2024-07-04T17:26:17.020171+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:13:00.368941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026043+0000","last_change":"2024-07-04T17:26:16.041627+0000","last_active":"2024-07-04T17:26:17.026043+0000","last_peered":"2024-07-04T17:26:17.026043+0000","last_clean":"2024-07-04T17:26:17.026043+0000","last_became_active":"2024-07-04T17:26:16.041274+0000","last_became_peered":"2024-07-04T17:26:16.041274+0000","last_unstale":"2024-07-04T17:26:17.026043+0000","last_undegraded":"2024-07-04T17:26:17.026043+0000","last_fullsized":"2024-07-04T17:26:17.026043+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:41:46.272734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.029021+0000","last_change":"2024-07-04T17:26:15.037603+0000","last_active":"2024-07-04T17:26:17.029021+0000","last_peered":"2024-07-04T17:26:17.029021+0000","last_clean":"2024-07-04T17:26:17.029021+0000","last_became_active":"2024-07-04T17:26:15.030406+0000","last_became_peered":"2024-07-04T17:26:15.030406+0000","last_unstale":"2024-07-04T17:26:17.029021+0000","last_undegraded":"2024-07-04T17:26:17.029021+0000","last_fullsized":"2024-07-04T17:26:17.029021+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:13:00.368941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.703750+0000","last_change":"2024-07-04T17:26:16.060475+0000","last_active":"2024-07-04T17:26:17.703750+0000","last_peered":"2024-07-04T17:26:17.703750+0000","last_clean":"2024-07-04T17:26:17.703750+0000","last_became_active":"2024-07-04T17:26:16.059861+0000","last_became_peered":"2024-07-04T17:26:16.059861+0000","last_unstale":"2024-07-04T17:26:17.703750+0000","last_undegraded":"2024-07-04T17:26:17.703750+0000","last_fullsized":"2024-07-04T17:26:17.703750+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:21:38.693638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.023859+0000","last_change":"2024-07-04T17:26:15.046597+0000","last_active":"2024-07-04T17:26:17.023859+0000","last_peered":"2024-07-04T17:26:17.023859+0000","last_clean":"2024-07-04T17:26:17.023859+0000","last_became_active":"2024-07-04T17:26:15.046243+0000","last_became_peered":"2024-07-04T17:26:15.046243+0000","last_unstale":"2024-07-04T17:26:17.023859+0000","last_undegraded":"2024-07-04T17:26:17.023859+0000","last_fullsized":"2024-07-04T17:26:17.023859+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:18:04.371426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.024536+0000","last_change":"2024-07-04T17:26:16.059218+0000","last_active":"2024-07-04T17:26:17.024536+0000","last_peered":"2024-07-04T17:26:17.024536+0000","last_clean":"2024-07-04T17:26:17.024536+0000","last_became_active":"2024-07-04T17:26:16.058877+0000","last_became_peered":"2024-07-04T17:26:16.058877+0000","last_unstale":"2024-07-04T17:26:17.024536+0000","last_undegraded":"2024-07-04T17:26:17.024536+0000","last_fullsized":"2024-07-04T17:26:17.024536+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:48:48.817671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"14'3","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.029077+0000","last_change":"2024-07-04T17:26:15.031110+0000","last_active":"2024-07-04T17:26:17.029077+0000","last_peered":"2024-07-04T17:26:17.029077+0000","last_clean":"2024-07-04T17:26:17.029077+0000","last_became_active":"2024-07-04T17:26:15.030786+0000","last_became_peered":"2024-07-04T17:26:15.030786+0000","last_unstale":"2024-07-04T17:26:17.029077+0000","last_undegraded":"2024-07-04T17:26:17.029077+0000","last_fullsized":"2024-07-04T17:26:17.029077+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:34:46.876523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.028971+0000","last_change":"2024-07-04T17:26:16.046513+0000","last_active":"2024-07-04T17:26:17.028971+0000","last_peered":"2024-07-04T17:26:17.028971+0000","last_clean":"2024-07-04T17:26:17.028971+0000","last_became_active":"2024-07-04T17:26:16.044971+0000","last_became_peered":"2024-07-04T17:26:16.044971+0000","last_unstale":"2024-07-04T17:26:17.028971+0000","last_undegraded":"2024-07-04T17:26:17.028971+0000","last_fullsized":"2024-07-04T17:26:17.028971+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:21:38.693638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"14'6","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702536+0000","last_change":"2024-07-04T17:26:15.042022+0000","last_active":"2024-07-04T17:26:17.702536+0000","last_peered":"2024-07-04T17:26:17.702536+0000","last_clean":"2024-07-04T17:26:17.702536+0000","last_became_active":"2024-07-04T17:26:15.041669+0000","last_became_peered":"2024-07-04T17:26:15.041669+0000","last_unstale":"2024-07-04T17:26:17.702536+0000","last_undegraded":"2024-07-04T17:26:17.702536+0000","last_fullsized":"2024-07-04T17:26:17.702536+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:35:22.891399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.024207+0000","last_change":"2024-07-04T17:26:16.059656+0000","last_active":"2024-07-04T17:26:17.024207+0000","last_peered":"2024-07-04T17:26:17.024207+0000","last_clean":"2024-07-04T17:26:17.024207+0000","last_became_active":"2024-07-04T17:26:16.058951+0000","last_became_peered":"2024-07-04T17:26:16.058951+0000","last_unstale":"2024-07-04T17:26:17.024207+0000","last_undegraded":"2024-07-04T17:26:17.024207+0000","last_fullsized":"2024-07-04T17:26:17.024207+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:52:45.300002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"14'2","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021264+0000","last_change":"2024-07-04T17:26:15.035545+0000","last_active":"2024-07-04T17:26:17.021264+0000","last_peered":"2024-07-04T17:26:17.021264+0000","last_clean":"2024-07-04T17:26:17.021264+0000","last_became_active":"2024-07-04T17:26:15.034822+0000","last_became_peered":"2024-07-04T17:26:15.034822+0000","last_unstale":"2024-07-04T17:26:17.021264+0000","last_undegraded":"2024-07-04T17:26:17.021264+0000","last_fullsized":"2024-07-04T17:26:17.021264+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:17:16.904464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020669+0000","last_change":"2024-07-04T17:26:16.064292+0000","last_active":"2024-07-04T17:26:17.020669+0000","last_peered":"2024-07-04T17:26:17.020669+0000","last_clean":"2024-07-04T17:26:17.020669+0000","last_became_active":"2024-07-04T17:26:16.053620+0000","last_became_peered":"2024-07-04T17:26:16.053620+0000","last_unstale":"2024-07-04T17:26:17.020669+0000","last_undegraded":"2024-07-04T17:26:17.020669+0000","last_fullsized":"2024-07-04T17:26:17.020669+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:40:49.177003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.701897+0000","last_change":"2024-07-04T17:26:15.041695+0000","last_active":"2024-07-04T17:26:17.701897+0000","last_peered":"2024-07-04T17:26:17.701897+0000","last_clean":"2024-07-04T17:26:17.701897+0000","last_became_active":"2024-07-04T17:26:15.041353+0000","last_became_peered":"2024-07-04T17:26:15.041353+0000","last_unstale":"2024-07-04T17:26:17.701897+0000","last_undegraded":"2024-07-04T17:26:17.701897+0000","last_fullsized":"2024-07-04T17:26:17.701897+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:05:39.042848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026437+0000","last_change":"2024-07-04T17:26:16.063355+0000","last_active":"2024-07-04T17:26:17.026437+0000","last_peered":"2024-07-04T17:26:17.026437+0000","last_clean":"2024-07-04T17:26:17.026437+0000","last_became_active":"2024-07-04T17:26:16.056707+0000","last_became_peered":"2024-07-04T17:26:16.056707+0000","last_unstale":"2024-07-04T17:26:17.026437+0000","last_undegraded":"2024-07-04T17:26:17.026437+0000","last_fullsized":"2024-07-04T17:26:17.026437+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:11:17.509522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021433+0000","last_change":"2024-07-04T17:26:15.050680+0000","last_active":"2024-07-04T17:26:17.021433+0000","last_peered":"2024-07-04T17:26:17.021433+0000","last_clean":"2024-07-04T17:26:17.021433+0000","last_became_active":"2024-07-04T17:26:15.049449+0000","last_became_peered":"2024-07-04T17:26:15.049449+0000","last_unstale":"2024-07-04T17:26:17.021433+0000","last_undegraded":"2024-07-04T17:26:17.021433+0000","last_fullsized":"2024-07-04T17:26:17.021433+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:25:38.473912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026199+0000","last_change":"2024-07-04T17:26:16.063777+0000","last_active":"2024-07-04T17:26:17.026199+0000","last_peered":"2024-07-04T17:26:17.026199+0000","last_clean":"2024-07-04T17:26:17.026199+0000","last_became_active":"2024-07-04T17:26:16.063580+0000","last_became_peered":"2024-07-04T17:26:16.063580+0000","last_unstale":"2024-07-04T17:26:17.026199+0000","last_undegraded":"2024-07-04T17:26:17.026199+0000","last_fullsized":"2024-07-04T17:26:17.026199+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:21:48.005606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"14'8","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702841+0000","last_change":"2024-07-04T17:26:15.043346+0000","last_active":"2024-07-04T17:26:17.702841+0000","last_peered":"2024-07-04T17:26:17.702841+0000","last_clean":"2024-07-04T17:26:17.702841+0000","last_became_active":"2024-07-04T17:26:15.041219+0000","last_became_peered":"2024-07-04T17:26:15.041219+0000","last_unstale":"2024-07-04T17:26:17.702841+0000","last_undegraded":"2024-07-04T17:26:17.702841+0000","last_fullsized":"2024-07-04T17:26:17.702841+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:13:00.368941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.701310+0000","last_change":"2024-07-04T17:26:16.057277+0000","last_active":"2024-07-04T17:26:17.701310+0000","last_peered":"2024-07-04T17:26:17.701310+0000","last_clean":"2024-07-04T17:26:17.701310+0000","last_became_active":"2024-07-04T17:26:16.056946+0000","last_became_peered":"2024-07-04T17:26:16.056946+0000","last_unstale":"2024-07-04T17:26:17.701310+0000","last_undegraded":"2024-07-04T17:26:17.701310+0000","last_fullsized":"2024-07-04T17:26:17.701310+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:41:46.272734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"14'1","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.027552+0000","last_change":"2024-07-04T17:26:15.049872+0000","last_active":"2024-07-04T17:26:17.027552+0000","last_peered":"2024-07-04T17:26:17.027552+0000","last_clean":"2024-07-04T17:26:17.027552+0000","last_became_active":"2024-07-04T17:26:15.048975+0000","last_became_peered":"2024-07-04T17:26:15.048975+0000","last_unstale":"2024-07-04T17:26:17.027552+0000","last_undegraded":"2024-07-04T17:26:17.027552+0000","last_fullsized":"2024-07-04T17:26:17.027552+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:18:52.190216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.704159+0000","last_change":"2024-07-04T17:26:16.071254+0000","last_active":"2024-07-04T17:26:17.704159+0000","last_peered":"2024-07-04T17:26:17.704159+0000","last_clean":"2024-07-04T17:26:17.704159+0000","last_became_active":"2024-07-04T17:26:16.070624+0000","last_became_peered":"2024-07-04T17:26:16.070624+0000","last_unstale":"2024-07-04T17:26:17.704159+0000","last_undegraded":"2024-07-04T17:26:17.704159+0000","last_fullsized":"2024-07-04T17:26:17.704159+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:50:25.908300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"14'2","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.704347+0000","last_change":"2024-07-04T17:26:15.048120+0000","last_active":"2024-07-04T17:26:17.704347+0000","last_peered":"2024-07-04T17:26:17.704347+0000","last_clean":"2024-07-04T17:26:17.704347+0000","last_became_active":"2024-07-04T17:26:15.047745+0000","last_became_peered":"2024-07-04T17:26:15.047745+0000","last_unstale":"2024-07-04T17:26:17.704347+0000","last_undegraded":"2024-07-04T17:26:17.704347+0000","last_fullsized":"2024-07-04T17:26:17.704347+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:17:16.904464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.024662+0000","last_change":"2024-07-04T17:26:16.047526+0000","last_active":"2024-07-04T17:26:17.024662+0000","last_peered":"2024-07-04T17:26:17.024662+0000","last_clean":"2024-07-04T17:26:17.024662+0000","last_became_active":"2024-07-04T17:26:16.046143+0000","last_became_peered":"2024-07-04T17:26:16.046143+0000","last_unstale":"2024-07-04T17:26:17.024662+0000","last_undegraded":"2024-07-04T17:26:17.024662+0000","last_fullsized":"2024-07-04T17:26:17.024662+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:48:44.510528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"14'1","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.029637+0000","last_change":"2024-07-04T17:26:15.021551+0000","last_active":"2024-07-04T17:26:17.029637+0000","last_peered":"2024-07-04T17:26:17.029637+0000","last_clean":"2024-07-04T17:26:17.029637+0000","last_became_active":"2024-07-04T17:26:15.021324+0000","last_became_peered":"2024-07-04T17:26:15.021324+0000","last_unstale":"2024-07-04T17:26:17.029637+0000","last_undegraded":"2024-07-04T17:26:17.029637+0000","last_fullsized":"2024-07-04T17:26:17.029637+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:18:04.371426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022385+0000","last_change":"2024-07-04T17:26:16.043753+0000","last_active":"2024-07-04T17:26:17.022385+0000","last_peered":"2024-07-04T17:26:17.022385+0000","last_clean":"2024-07-04T17:26:17.022385+0000","last_became_active":"2024-07-04T17:26:16.037154+0000","last_became_peered":"2024-07-04T17:26:16.037154+0000","last_unstale":"2024-07-04T17:26:17.022385+0000","last_undegraded":"2024-07-04T17:26:17.022385+0000","last_fullsized":"2024-07-04T17:26:17.022385+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:30:49.842921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702886+0000","last_change":"2024-07-04T17:26:15.039837+0000","last_active":"2024-07-04T17:26:17.702886+0000","last_peered":"2024-07-04T17:26:17.702886+0000","last_clean":"2024-07-04T17:26:17.702886+0000","last_became_active":"2024-07-04T17:26:15.039334+0000","last_became_peered":"2024-07-04T17:26:15.039334+0000","last_unstale":"2024-07-04T17:26:17.702886+0000","last_undegraded":"2024-07-04T17:26:17.702886+0000","last_fullsized":"2024-07-04T17:26:17.702886+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:46.107542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.030226+0000","last_change":"2024-07-04T17:26:16.061791+0000","last_active":"2024-07-04T17:26:17.030226+0000","last_peered":"2024-07-04T17:26:17.030226+0000","last_clean":"2024-07-04T17:26:17.030226+0000","last_became_active":"2024-07-04T17:26:16.061453+0000","last_became_peered":"2024-07-04T17:26:16.061453+0000","last_unstale":"2024-07-04T17:26:17.030226+0000","last_undegraded":"2024-07-04T17:26:17.030226+0000","last_fullsized":"2024-07-04T17:26:17.030226+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:13:00.963495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'52","reported_seq":87,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026937+0000","last_change":"2024-07-04T17:26:04.953559+0000","last_active":"2024-07-04T17:26:17.026937+0000","last_peered":"2024-07-04T17:26:17.026937+0000","last_clean":"2024-07-04T17:26:17.026937+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:26:17.026937+0000","last_undegraded":"2024-07-04T17:26:17.026937+0000","last_fullsized":"2024-07-04T17:26:17.026937+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_clean_scrub_stamp":"2024-07-04T17:26:03.933432+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.064181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.027937+0000","last_change":"2024-07-04T17:26:15.026582+0000","last_active":"2024-07-04T17:26:17.027937+0000","last_peered":"2024-07-04T17:26:17.027937+0000","last_clean":"2024-07-04T17:26:17.027937+0000","last_became_active":"2024-07-04T17:26:15.026170+0000","last_became_peered":"2024-07-04T17:26:15.026170+0000","last_unstale":"2024-07-04T17:26:17.027937+0000","last_undegraded":"2024-07-04T17:26:17.027937+0000","last_fullsized":"2024-07-04T17:26:17.027937+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:45:20.253806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026496+0000","last_change":"2024-07-04T17:26:16.044579+0000","last_active":"2024-07-04T17:26:17.026496+0000","last_peered":"2024-07-04T17:26:17.026496+0000","last_clean":"2024-07-04T17:26:17.026496+0000","last_became_active":"2024-07-04T17:26:16.044255+0000","last_became_peered":"2024-07-04T17:26:16.044255+0000","last_unstale":"2024-07-04T17:26:17.026496+0000","last_undegraded":"2024-07-04T17:26:17.026496+0000","last_fullsized":"2024-07-04T17:26:17.026496+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:30:20.910427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.703153+0000","last_change":"2024-07-04T17:26:15.043650+0000","last_active":"2024-07-04T17:26:17.703153+0000","last_peered":"2024-07-04T17:26:17.703153+0000","last_clean":"2024-07-04T17:26:17.703153+0000","last_became_active":"2024-07-04T17:26:15.042392+0000","last_became_peered":"2024-07-04T17:26:15.042392+0000","last_unstale":"2024-07-04T17:26:17.703153+0000","last_undegraded":"2024-07-04T17:26:17.703153+0000","last_fullsized":"2024-07-04T17:26:17.703153+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:18:04.371426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021141+0000","last_change":"2024-07-04T17:26:16.051770+0000","last_active":"2024-07-04T17:26:17.021141+0000","last_peered":"2024-07-04T17:26:17.021141+0000","last_clean":"2024-07-04T17:26:17.021141+0000","last_became_active":"2024-07-04T17:26:16.046616+0000","last_became_peered":"2024-07-04T17:26:16.046616+0000","last_unstale":"2024-07-04T17:26:17.021141+0000","last_undegraded":"2024-07-04T17:26:17.021141+0000","last_fullsized":"2024-07-04T17:26:17.021141+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:03:12.070345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702913+0000","last_change":"2024-07-04T17:26:15.042707+0000","last_active":"2024-07-04T17:26:17.702913+0000","last_peered":"2024-07-04T17:26:17.702913+0000","last_clean":"2024-07-04T17:26:17.702913+0000","last_became_active":"2024-07-04T17:26:15.041346+0000","last_became_peered":"2024-07-04T17:26:15.041346+0000","last_unstale":"2024-07-04T17:26:17.702913+0000","last_undegraded":"2024-07-04T17:26:17.702913+0000","last_fullsized":"2024-07-04T17:26:17.702913+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:31:03.314632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026776+0000","last_change":"2024-07-04T17:26:16.039792+0000","last_active":"2024-07-04T17:26:17.026776+0000","last_peered":"2024-07-04T17:26:17.026776+0000","last_clean":"2024-07-04T17:26:17.026776+0000","last_became_active":"2024-07-04T17:26:16.039470+0000","last_became_peered":"2024-07-04T17:26:16.039470+0000","last_unstale":"2024-07-04T17:26:17.026776+0000","last_undegraded":"2024-07-04T17:26:17.026776+0000","last_fullsized":"2024-07-04T17:26:17.026776+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:08:53.606368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022753+0000","last_change":"2024-07-04T17:26:15.046123+0000","last_active":"2024-07-04T17:26:17.022753+0000","last_peered":"2024-07-04T17:26:17.022753+0000","last_clean":"2024-07-04T17:26:17.022753+0000","last_became_active":"2024-07-04T17:26:15.045824+0000","last_became_peered":"2024-07-04T17:26:15.045824+0000","last_unstale":"2024-07-04T17:26:17.022753+0000","last_undegraded":"2024-07-04T17:26:17.022753+0000","last_fullsized":"2024-07-04T17:26:17.022753+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:46.107542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021092+0000","last_change":"2024-07-04T17:26:16.044581+0000","last_active":"2024-07-04T17:26:17.021092+0000","last_peered":"2024-07-04T17:26:17.021092+0000","last_clean":"2024-07-04T17:26:17.021092+0000","last_became_active":"2024-07-04T17:26:16.043791+0000","last_became_peered":"2024-07-04T17:26:16.043791+0000","last_unstale":"2024-07-04T17:26:17.021092+0000","last_undegraded":"2024-07-04T17:26:17.021092+0000","last_fullsized":"2024-07-04T17:26:17.021092+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:38:17.647736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.029309+0000","last_change":"2024-07-04T17:26:15.032798+0000","last_active":"2024-07-04T17:26:17.029309+0000","last_peered":"2024-07-04T17:26:17.029309+0000","last_clean":"2024-07-04T17:26:17.029309+0000","last_became_active":"2024-07-04T17:26:15.029686+0000","last_became_peered":"2024-07-04T17:26:15.029686+0000","last_unstale":"2024-07-04T17:26:17.029309+0000","last_undegraded":"2024-07-04T17:26:17.029309+0000","last_fullsized":"2024-07-04T17:26:17.029309+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:25:38.473912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702222+0000","last_change":"2024-07-04T17:26:16.062731+0000","last_active":"2024-07-04T17:26:17.702222+0000","last_peered":"2024-07-04T17:26:17.702222+0000","last_clean":"2024-07-04T17:26:17.702222+0000","last_became_active":"2024-07-04T17:26:16.062519+0000","last_became_peered":"2024-07-04T17:26:16.062519+0000","last_unstale":"2024-07-04T17:26:17.702222+0000","last_undegraded":"2024-07-04T17:26:17.702222+0000","last_fullsized":"2024-07-04T17:26:17.702222+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:30:20.910427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.029919+0000","last_change":"2024-07-04T17:26:15.036880+0000","last_active":"2024-07-04T17:26:17.029919+0000","last_peered":"2024-07-04T17:26:17.029919+0000","last_clean":"2024-07-04T17:26:17.029919+0000","last_became_active":"2024-07-04T17:26:15.036529+0000","last_became_peered":"2024-07-04T17:26:15.036529+0000","last_unstale":"2024-07-04T17:26:17.029919+0000","last_undegraded":"2024-07-04T17:26:17.029919+0000","last_fullsized":"2024-07-04T17:26:17.029919+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:04:46.667644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.029152+0000","last_change":"2024-07-04T17:26:16.046186+0000","last_active":"2024-07-04T17:26:17.029152+0000","last_peered":"2024-07-04T17:26:17.029152+0000","last_clean":"2024-07-04T17:26:17.029152+0000","last_became_active":"2024-07-04T17:26:16.044588+0000","last_became_peered":"2024-07-04T17:26:16.044588+0000","last_unstale":"2024-07-04T17:26:17.029152+0000","last_undegraded":"2024-07-04T17:26:17.029152+0000","last_fullsized":"2024-07-04T17:26:17.029152+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:50:25.908300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"15'1","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.030544+0000","last_change":"2024-07-04T17:26:15.022516+0000","last_active":"2024-07-04T17:26:17.030544+0000","last_peered":"2024-07-04T17:26:17.030544+0000","last_clean":"2024-07-04T17:26:17.030544+0000","last_became_active":"2024-07-04T17:26:15.022185+0000","last_became_peered":"2024-07-04T17:26:15.022185+0000","last_unstale":"2024-07-04T17:26:17.030544+0000","last_undegraded":"2024-07-04T17:26:17.030544+0000","last_fullsized":"2024-07-04T17:26:17.030544+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:24:04.318931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702984+0000","last_change":"2024-07-04T17:26:16.059869+0000","last_active":"2024-07-04T17:26:17.702984+0000","last_peered":"2024-07-04T17:26:17.702984+0000","last_clean":"2024-07-04T17:26:17.702984+0000","last_became_active":"2024-07-04T17:26:16.059535+0000","last_became_peered":"2024-07-04T17:26:16.059535+0000","last_unstale":"2024-07-04T17:26:17.702984+0000","last_undegraded":"2024-07-04T17:26:17.702984+0000","last_fullsized":"2024-07-04T17:26:17.702984+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:43:29.877017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"14'2","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.029845+0000","last_change":"2024-07-04T17:26:15.040384+0000","last_active":"2024-07-04T17:26:17.029845+0000","last_peered":"2024-07-04T17:26:17.029845+0000","last_clean":"2024-07-04T17:26:17.029845+0000","last_became_active":"2024-07-04T17:26:15.038959+0000","last_became_peered":"2024-07-04T17:26:15.038959+0000","last_unstale":"2024-07-04T17:26:17.029845+0000","last_undegraded":"2024-07-04T17:26:17.029845+0000","last_fullsized":"2024-07-04T17:26:17.029845+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:18:52.190216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.023117+0000","last_change":"2024-07-04T17:26:16.053639+0000","last_active":"2024-07-04T17:26:17.023117+0000","last_peered":"2024-07-04T17:26:17.023117+0000","last_clean":"2024-07-04T17:26:17.023117+0000","last_became_active":"2024-07-04T17:26:16.052766+0000","last_became_peered":"2024-07-04T17:26:16.052766+0000","last_unstale":"2024-07-04T17:26:17.023117+0000","last_undegraded":"2024-07-04T17:26:17.023117+0000","last_fullsized":"2024-07-04T17:26:17.023117+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:22:12.063177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020302+0000","last_change":"2024-07-04T17:26:15.027718+0000","last_active":"2024-07-04T17:26:17.020302+0000","last_peered":"2024-07-04T17:26:17.020302+0000","last_clean":"2024-07-04T17:26:17.020302+0000","last_became_active":"2024-07-04T17:26:15.026662+0000","last_became_peered":"2024-07-04T17:26:15.026662+0000","last_unstale":"2024-07-04T17:26:17.020302+0000","last_undegraded":"2024-07-04T17:26:17.020302+0000","last_fullsized":"2024-07-04T17:26:17.020302+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:45:20.253806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020487+0000","last_change":"2024-07-04T17:26:16.066686+0000","last_active":"2024-07-04T17:26:17.020487+0000","last_peered":"2024-07-04T17:26:17.020487+0000","last_clean":"2024-07-04T17:26:17.020487+0000","last_became_active":"2024-07-04T17:26:16.066397+0000","last_became_peered":"2024-07-04T17:26:16.066397+0000","last_unstale":"2024-07-04T17:26:17.020487+0000","last_undegraded":"2024-07-04T17:26:17.020487+0000","last_fullsized":"2024-07-04T17:26:17.020487+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:21:38.693638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026169+0000","last_change":"2024-07-04T17:26:15.034670+0000","last_active":"2024-07-04T17:26:17.026169+0000","last_peered":"2024-07-04T17:26:17.026169+0000","last_clean":"2024-07-04T17:26:17.026169+0000","last_became_active":"2024-07-04T17:26:15.034267+0000","last_became_peered":"2024-07-04T17:26:15.034267+0000","last_unstale":"2024-07-04T17:26:17.026169+0000","last_undegraded":"2024-07-04T17:26:17.026169+0000","last_fullsized":"2024-07-04T17:26:17.026169+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:46.107542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702508+0000","last_change":"2024-07-04T17:26:16.054400+0000","last_active":"2024-07-04T17:26:17.702508+0000","last_peered":"2024-07-04T17:26:17.702508+0000","last_clean":"2024-07-04T17:26:17.702508+0000","last_became_active":"2024-07-04T17:26:16.054057+0000","last_became_peered":"2024-07-04T17:26:16.054057+0000","last_unstale":"2024-07-04T17:26:17.702508+0000","last_undegraded":"2024-07-04T17:26:17.702508+0000","last_fullsized":"2024-07-04T17:26:17.702508+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:04:04.805872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026719+0000","last_change":"2024-07-04T17:26:15.032894+0000","last_active":"2024-07-04T17:26:17.026719+0000","last_peered":"2024-07-04T17:26:17.026719+0000","last_clean":"2024-07-04T17:26:17.026719+0000","last_became_active":"2024-07-04T17:26:15.032634+0000","last_became_peered":"2024-07-04T17:26:15.032634+0000","last_unstale":"2024-07-04T17:26:17.026719+0000","last_undegraded":"2024-07-04T17:26:17.026719+0000","last_fullsized":"2024-07-04T17:26:17.026719+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:05:39.042848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026622+0000","last_change":"2024-07-04T17:26:16.065536+0000","last_active":"2024-07-04T17:26:17.026622+0000","last_peered":"2024-07-04T17:26:17.026622+0000","last_clean":"2024-07-04T17:26:17.026622+0000","last_became_active":"2024-07-04T17:26:16.056278+0000","last_became_peered":"2024-07-04T17:26:16.056278+0000","last_unstale":"2024-07-04T17:26:17.026622+0000","last_undegraded":"2024-07-04T17:26:17.026622+0000","last_fullsized":"2024-07-04T17:26:17.026622+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:03:07.909265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"14'2","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021818+0000","last_change":"2024-07-04T17:26:15.033912+0000","last_active":"2024-07-04T17:26:17.021818+0000","last_peered":"2024-07-04T17:26:17.021818+0000","last_clean":"2024-07-04T17:26:17.021818+0000","last_became_active":"2024-07-04T17:26:15.027824+0000","last_became_peered":"2024-07-04T17:26:15.027824+0000","last_unstale":"2024-07-04T17:26:17.021818+0000","last_undegraded":"2024-07-04T17:26:17.021818+0000","last_fullsized":"2024-07-04T17:26:17.021818+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:34:46.876523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020817+0000","last_change":"2024-07-04T17:26:16.065218+0000","last_active":"2024-07-04T17:26:17.020817+0000","last_peered":"2024-07-04T17:26:17.020817+0000","last_clean":"2024-07-04T17:26:17.020817+0000","last_became_active":"2024-07-04T17:26:16.064904+0000","last_became_peered":"2024-07-04T17:26:16.064904+0000","last_unstale":"2024-07-04T17:26:17.020817+0000","last_undegraded":"2024-07-04T17:26:17.020817+0000","last_fullsized":"2024-07-04T17:26:17.020817+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:33:23.722640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702115+0000","last_change":"2024-07-04T17:26:16.073564+0000","last_active":"2024-07-04T17:26:17.702115+0000","last_peered":"2024-07-04T17:26:17.702115+0000","last_clean":"2024-07-04T17:26:17.702115+0000","last_became_active":"2024-07-04T17:26:16.073189+0000","last_became_peered":"2024-07-04T17:26:16.073189+0000","last_unstale":"2024-07-04T17:26:17.702115+0000","last_undegraded":"2024-07-04T17:26:17.702115+0000","last_fullsized":"2024-07-04T17:26:17.702115+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:21:48.005606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"14'2","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022583+0000","last_change":"2024-07-04T17:26:15.049001+0000","last_active":"2024-07-04T17:26:17.022583+0000","last_peered":"2024-07-04T17:26:17.022583+0000","last_clean":"2024-07-04T17:26:17.022583+0000","last_became_active":"2024-07-04T17:26:15.048437+0000","last_became_peered":"2024-07-04T17:26:15.048437+0000","last_unstale":"2024-07-04T17:26:17.022583+0000","last_undegraded":"2024-07-04T17:26:17.022583+0000","last_fullsized":"2024-07-04T17:26:17.022583+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:15:08.055102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.028692+0000","last_change":"2024-07-04T17:26:16.064782+0000","last_active":"2024-07-04T17:26:17.028692+0000","last_peered":"2024-07-04T17:26:17.028692+0000","last_clean":"2024-07-04T17:26:17.028692+0000","last_became_active":"2024-07-04T17:26:16.064518+0000","last_became_peered":"2024-07-04T17:26:16.064518+0000","last_unstale":"2024-07-04T17:26:17.028692+0000","last_undegraded":"2024-07-04T17:26:17.028692+0000","last_fullsized":"2024-07-04T17:26:17.028692+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:01:31.824881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"14'2","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022678+0000","last_change":"2024-07-04T17:26:15.047259+0000","last_active":"2024-07-04T17:26:17.022678+0000","last_peered":"2024-07-04T17:26:17.022678+0000","last_clean":"2024-07-04T17:26:17.022678+0000","last_became_active":"2024-07-04T17:26:15.046951+0000","last_became_peered":"2024-07-04T17:26:15.046951+0000","last_unstale":"2024-07-04T17:26:17.022678+0000","last_undegraded":"2024-07-04T17:26:17.022678+0000","last_fullsized":"2024-07-04T17:26:17.022678+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:04:46.667644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.024493+0000","last_change":"2024-07-04T17:26:16.058196+0000","last_active":"2024-07-04T17:26:17.024493+0000","last_peered":"2024-07-04T17:26:17.024493+0000","last_clean":"2024-07-04T17:26:17.024493+0000","last_became_active":"2024-07-04T17:26:16.051038+0000","last_became_peered":"2024-07-04T17:26:16.051038+0000","last_unstale":"2024-07-04T17:26:17.024493+0000","last_undegraded":"2024-07-04T17:26:17.024493+0000","last_fullsized":"2024-07-04T17:26:17.024493+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:04:50.688662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"14'1","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.704046+0000","last_change":"2024-07-04T17:26:15.040464+0000","last_active":"2024-07-04T17:26:17.704046+0000","last_peered":"2024-07-04T17:26:17.704046+0000","last_clean":"2024-07-04T17:26:17.704046+0000","last_became_active":"2024-07-04T17:26:15.040008+0000","last_became_peered":"2024-07-04T17:26:15.040008+0000","last_unstale":"2024-07-04T17:26:17.704046+0000","last_undegraded":"2024-07-04T17:26:17.704046+0000","last_fullsized":"2024-07-04T17:26:17.704046+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:34:46.876523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.703450+0000","last_change":"2024-07-04T17:26:16.060159+0000","last_active":"2024-07-04T17:26:17.703450+0000","last_peered":"2024-07-04T17:26:17.703450+0000","last_clean":"2024-07-04T17:26:17.703450+0000","last_became_active":"2024-07-04T17:26:16.059458+0000","last_became_peered":"2024-07-04T17:26:16.059458+0000","last_unstale":"2024-07-04T17:26:17.703450+0000","last_undegraded":"2024-07-04T17:26:17.703450+0000","last_fullsized":"2024-07-04T17:26:17.703450+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:01:31.824881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"14'4","reported_seq":18,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.027492+0000","last_change":"2024-07-04T17:26:15.049639+0000","last_active":"2024-07-04T17:26:17.027492+0000","last_peered":"2024-07-04T17:26:17.027492+0000","last_clean":"2024-07-04T17:26:17.027492+0000","last_became_active":"2024-07-04T17:26:15.048590+0000","last_became_peered":"2024-07-04T17:26:15.048590+0000","last_unstale":"2024-07-04T17:26:17.027492+0000","last_undegraded":"2024-07-04T17:26:17.027492+0000","last_fullsized":"2024-07-04T17:26:17.027492+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:05:39.042848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738373,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31332,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340508,"statfs":{"total":96636764160,"available":96604680192,"internally_reserved":0,"allocated":151552,"data_stored":42085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738374,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31920,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339920,"statfs":{"total":96636764160,"available":96604078080,"internally_reserved":0,"allocated":753664,"data_stored":633113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738372,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31912,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339928,"statfs":{"total":96636764160,"available":96604086272,"internally_reserved":0,"allocated":741376,"data_stored":631465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:26:20.743 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:26:21.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.065+0000 7f8980e00640 1 -- 172.21.15.42:0/2845724355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f897c0b1d20 msgr2=0x7f897c0b6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:21.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.065+0000 7f8980e00640 1 --2- 172.21.15.42:0/2845724355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f897c0b1d20 0x7f897c0b6230 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8968005fd0 tx=0x7f896802a8b0 comp rx=0 tx=0).stop 2024-07-04T17:26:21.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.065+0000 7f8980e00640 1 -- 172.21.15.42:0/2845724355 shutdown_connections 2024-07-04T17:26:21.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.065+0000 7f8980e00640 1 --2- 172.21.15.42:0/2845724355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f897c0b1d20 0x7f897c0b6230 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:21.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.065+0000 7f8980e00640 1 -- 172.21.15.42:0/2845724355 >> 172.21.15.42:0/2845724355 conn(0x7f897c0bebc0 msgr2=0x7f897c0befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:21.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.066+0000 7f8980e00640 1 -- 172.21.15.42:0/2845724355 shutdown_connections 2024-07-04T17:26:21.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.066+0000 7f8980e00640 1 -- 172.21.15.42:0/2845724355 wait complete. 2024-07-04T17:26:21.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.068+0000 7f8980e00640 1 Processor -- start 2024-07-04T17:26:21.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.068+0000 7f8980e00640 1 -- start start 2024-07-04T17:26:21.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.069+0000 7f8980e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f897c0b1d20 0x7f897c1428f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:21.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.069+0000 7f8980e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f897c0b8b40 con 0x7f897c0b1d20 2024-07-04T17:26:21.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.069+0000 7f897b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f897c0b1d20 0x7f897c1428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:21.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.069+0000 7f897b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f897c0b1d20 0x7f897c1428f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34878/0 (socket says 172.21.15.42:34878) 2024-07-04T17:26:21.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.069+0000 7f897b400640 1 -- 172.21.15.42:0/151965067 learned_addr learned my addr 172.21.15.42:0/151965067 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:21.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.070+0000 7f897b400640 1 -- 172.21.15.42:0/151965067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f897c1d3990 con 0x7f897c0b1d20 2024-07-04T17:26:21.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.070+0000 7f897b400640 1 --2- 172.21.15.42:0/151965067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f897c0b1d20 0x7f897c1428f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f896c00c6b0 tx=0x7f896c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:21.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.070+0000 7f8973400640 1 -- 172.21.15.42:0/151965067 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f896c016020 con 0x7f897c0b1d20 2024-07-04T17:26:21.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.070+0000 7f8973400640 1 -- 172.21.15.42:0/151965067 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f896c00a5c0 con 0x7f897c0b1d20 2024-07-04T17:26:21.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.070+0000 7f8973400640 1 -- 172.21.15.42:0/151965067 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f896c00a960 con 0x7f897c0b1d20 2024-07-04T17:26:21.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.070+0000 7f8980e00640 1 -- 172.21.15.42:0/151965067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f897c1d5090 con 0x7f897c0b1d20 2024-07-04T17:26:21.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.070+0000 7f8980e00640 1 -- 172.21.15.42:0/151965067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f897c1d8c10 con 0x7f897c0b1d20 2024-07-04T17:26:21.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.071+0000 7f8980e00640 1 -- 172.21.15.42:0/151965067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f897c0b91d0 con 0x7f897c0b1d20 2024-07-04T17:26:21.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.071+0000 7f8973400640 1 -- 172.21.15.42:0/151965067 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f896c019020 con 0x7f897c0b1d20 2024-07-04T17:26:21.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.072+0000 7f8973400640 1 --2- 172.21.15.42:0/151965067 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f895c03d2b0 0x7f895c03f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:21.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.072+0000 7f8973400640 1 -- 172.21.15.42:0/151965067 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f896c0513b0 con 0x7f897c0b1d20 2024-07-04T17:26:21.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.072+0000 7f897aa00640 1 --2- 172.21.15.42:0/151965067 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f895c03d2b0 0x7f895c03f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:21.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.073+0000 7f897aa00640 1 --2- 172.21.15.42:0/151965067 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f895c03d2b0 0x7f895c03f7c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8968034000 tx=0x7f8968004ab0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:21.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.077+0000 7f8973400640 1 -- 172.21.15.42:0/151965067 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f896c020020 con 0x7f897c0b1d20 2024-07-04T17:26:21.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.286+0000 7f8980e00640 1 -- 172.21.15.42:0/151965067 --> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f897c068e30 con 0x7f895c03d2b0 2024-07-04T17:26:21.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.294+0000 7f8973400640 1 -- 172.21.15.42:0/151965067 <== mgr.4103 v2:172.21.15.42:6824/3523793950 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245956 (secure 0 0 0) 0x7f897c068e30 con 0x7f895c03d2b0 2024-07-04T17:26:21.299 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:26:21.301 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:26:21.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.300+0000 7f8980e00640 1 -- 172.21.15.42:0/151965067 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f895c03d2b0 msgr2=0x7f895c03f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:21.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.300+0000 7f8980e00640 1 --2- 172.21.15.42:0/151965067 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f895c03d2b0 0x7f895c03f7c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8968034000 tx=0x7f8968004ab0 comp rx=0 tx=0).stop 2024-07-04T17:26:21.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.300+0000 7f8980e00640 1 -- 172.21.15.42:0/151965067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f897c0b1d20 msgr2=0x7f897c1428f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:21.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.300+0000 7f8980e00640 1 --2- 172.21.15.42:0/151965067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f897c0b1d20 0x7f897c1428f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f896c00c6b0 tx=0x7f896c00cb70 comp rx=0 tx=0).stop 2024-07-04T17:26:21.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.300+0000 7f8980e00640 1 -- 172.21.15.42:0/151965067 shutdown_connections 2024-07-04T17:26:21.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.300+0000 7f8980e00640 1 --2- 172.21.15.42:0/151965067 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f895c03d2b0 0x7f895c03f7c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:21.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.300+0000 7f8980e00640 1 --2- 172.21.15.42:0/151965067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f897c0b1d20 0x7f897c1428f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:21.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.300+0000 7f8980e00640 1 -- 172.21.15.42:0/151965067 >> 172.21.15.42:0/151965067 conn(0x7f897c0bebc0 msgr2=0x7f897c0b6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:21.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.300+0000 7f8980e00640 1 -- 172.21.15.42:0/151965067 shutdown_connections 2024-07-04T17:26:21.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.300+0000 7f8980e00640 1 -- 172.21.15.42:0/151965067 wait complete. 2024-07-04T17:26:21.329 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-07-04T17:26:21.263824+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":124,"num_write_kb":1120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91,"ondisk_log_size":91,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":120,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":95164,"kb_used_data":1608,"kb_used_omap":4,"kb_used_meta":93499,"kb_avail":283020356,"statfs":{"total":289910292480,"available":289812844544,"internally_reserved":0,"allocated":1646592,"data_stored":1306663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":95743311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.269201"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.023948+0000","last_change":"2024-07-04T17:26:16.062462+0000","last_active":"2024-07-04T17:26:17.023948+0000","last_peered":"2024-07-04T17:26:17.023948+0000","last_clean":"2024-07-04T17:26:17.023948+0000","last_became_active":"2024-07-04T17:26:16.058531+0000","last_became_peered":"2024-07-04T17:26:16.058531+0000","last_unstale":"2024-07-04T17:26:17.023948+0000","last_undegraded":"2024-07-04T17:26:17.023948+0000","last_fullsized":"2024-07-04T17:26:17.023948+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:31:33.921497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.028401+0000","last_change":"2024-07-04T17:26:16.064253+0000","last_active":"2024-07-04T17:26:17.028401+0000","last_peered":"2024-07-04T17:26:17.028401+0000","last_clean":"2024-07-04T17:26:17.028401+0000","last_became_active":"2024-07-04T17:26:16.063971+0000","last_became_peered":"2024-07-04T17:26:16.063971+0000","last_unstale":"2024-07-04T17:26:17.028401+0000","last_undegraded":"2024-07-04T17:26:17.028401+0000","last_fullsized":"2024-07-04T17:26:17.028401+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:22:14.191409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022213+0000","last_change":"2024-07-04T17:26:16.059637+0000","last_active":"2024-07-04T17:26:17.022213+0000","last_peered":"2024-07-04T17:26:17.022213+0000","last_clean":"2024-07-04T17:26:17.022213+0000","last_became_active":"2024-07-04T17:26:16.058613+0000","last_became_peered":"2024-07-04T17:26:16.058613+0000","last_unstale":"2024-07-04T17:26:17.022213+0000","last_undegraded":"2024-07-04T17:26:17.022213+0000","last_fullsized":"2024-07-04T17:26:17.022213+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:02:40.668757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702530+0000","last_change":"2024-07-04T17:26:16.072904+0000","last_active":"2024-07-04T17:26:17.702530+0000","last_peered":"2024-07-04T17:26:17.702530+0000","last_clean":"2024-07-04T17:26:17.702530+0000","last_became_active":"2024-07-04T17:26:16.072640+0000","last_became_peered":"2024-07-04T17:26:16.072640+0000","last_unstale":"2024-07-04T17:26:17.702530+0000","last_undegraded":"2024-07-04T17:26:17.702530+0000","last_fullsized":"2024-07-04T17:26:17.702530+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:03:07.909265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021289+0000","last_change":"2024-07-04T17:26:16.044297+0000","last_active":"2024-07-04T17:26:17.021289+0000","last_peered":"2024-07-04T17:26:17.021289+0000","last_clean":"2024-07-04T17:26:17.021289+0000","last_became_active":"2024-07-04T17:26:16.043526+0000","last_became_peered":"2024-07-04T17:26:16.043526+0000","last_unstale":"2024-07-04T17:26:17.021289+0000","last_undegraded":"2024-07-04T17:26:17.021289+0000","last_fullsized":"2024-07-04T17:26:17.021289+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:11:11.392035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702192+0000","last_change":"2024-07-04T17:26:16.072472+0000","last_active":"2024-07-04T17:26:17.702192+0000","last_peered":"2024-07-04T17:26:17.702192+0000","last_clean":"2024-07-04T17:26:17.702192+0000","last_became_active":"2024-07-04T17:26:16.072066+0000","last_became_peered":"2024-07-04T17:26:16.072066+0000","last_unstale":"2024-07-04T17:26:17.702192+0000","last_undegraded":"2024-07-04T17:26:17.702192+0000","last_fullsized":"2024-07-04T17:26:17.702192+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:11:17.509522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021458+0000","last_change":"2024-07-04T17:26:16.070453+0000","last_active":"2024-07-04T17:26:17.021458+0000","last_peered":"2024-07-04T17:26:17.021458+0000","last_clean":"2024-07-04T17:26:17.021458+0000","last_became_active":"2024-07-04T17:26:16.065958+0000","last_became_peered":"2024-07-04T17:26:16.065958+0000","last_unstale":"2024-07-04T17:26:17.021458+0000","last_undegraded":"2024-07-04T17:26:17.021458+0000","last_fullsized":"2024-07-04T17:26:17.021458+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:51:59.570861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.700790+0000","last_change":"2024-07-04T17:26:16.072492+0000","last_active":"2024-07-04T17:26:17.700790+0000","last_peered":"2024-07-04T17:26:17.700790+0000","last_clean":"2024-07-04T17:26:17.700790+0000","last_became_active":"2024-07-04T17:26:16.067345+0000","last_became_peered":"2024-07-04T17:26:16.067345+0000","last_unstale":"2024-07-04T17:26:17.700790+0000","last_undegraded":"2024-07-04T17:26:17.700790+0000","last_fullsized":"2024-07-04T17:26:17.700790+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:02:12.965714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.028868+0000","last_change":"2024-07-04T17:26:16.055031+0000","last_active":"2024-07-04T17:26:17.028868+0000","last_peered":"2024-07-04T17:26:17.028868+0000","last_clean":"2024-07-04T17:26:17.028868+0000","last_became_active":"2024-07-04T17:26:16.051647+0000","last_became_peered":"2024-07-04T17:26:16.051647+0000","last_unstale":"2024-07-04T17:26:17.028868+0000","last_undegraded":"2024-07-04T17:26:17.028868+0000","last_fullsized":"2024-07-04T17:26:17.028868+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:41:20.224285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022523+0000","last_change":"2024-07-04T17:26:16.045710+0000","last_active":"2024-07-04T17:26:17.022523+0000","last_peered":"2024-07-04T17:26:17.022523+0000","last_clean":"2024-07-04T17:26:17.022523+0000","last_became_active":"2024-07-04T17:26:16.045371+0000","last_became_peered":"2024-07-04T17:26:16.045371+0000","last_unstale":"2024-07-04T17:26:17.022523+0000","last_undegraded":"2024-07-04T17:26:17.022523+0000","last_fullsized":"2024-07-04T17:26:17.022523+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:53:44.096508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022006+0000","last_change":"2024-07-04T17:26:16.059936+0000","last_active":"2024-07-04T17:26:17.022006+0000","last_peered":"2024-07-04T17:26:17.022006+0000","last_clean":"2024-07-04T17:26:17.022006+0000","last_became_active":"2024-07-04T17:26:16.058990+0000","last_became_peered":"2024-07-04T17:26:16.058990+0000","last_unstale":"2024-07-04T17:26:17.022006+0000","last_undegraded":"2024-07-04T17:26:17.022006+0000","last_fullsized":"2024-07-04T17:26:17.022006+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:47:09.376866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022753+0000","last_change":"2024-07-04T17:26:16.053390+0000","last_active":"2024-07-04T17:26:17.022753+0000","last_peered":"2024-07-04T17:26:17.022753+0000","last_clean":"2024-07-04T17:26:17.022753+0000","last_became_active":"2024-07-04T17:26:16.052716+0000","last_became_peered":"2024-07-04T17:26:16.052716+0000","last_unstale":"2024-07-04T17:26:17.022753+0000","last_undegraded":"2024-07-04T17:26:17.022753+0000","last_fullsized":"2024-07-04T17:26:17.022753+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:15:52.371580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021947+0000","last_change":"2024-07-04T17:26:16.037676+0000","last_active":"2024-07-04T17:26:17.021947+0000","last_peered":"2024-07-04T17:26:17.021947+0000","last_clean":"2024-07-04T17:26:17.021947+0000","last_became_active":"2024-07-04T17:26:16.037311+0000","last_became_peered":"2024-07-04T17:26:16.037311+0000","last_unstale":"2024-07-04T17:26:17.021947+0000","last_undegraded":"2024-07-04T17:26:17.021947+0000","last_fullsized":"2024-07-04T17:26:17.021947+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:31:49.427076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020967+0000","last_change":"2024-07-04T17:26:16.064558+0000","last_active":"2024-07-04T17:26:17.020967+0000","last_peered":"2024-07-04T17:26:17.020967+0000","last_clean":"2024-07-04T17:26:17.020967+0000","last_became_active":"2024-07-04T17:26:16.054068+0000","last_became_peered":"2024-07-04T17:26:16.054068+0000","last_unstale":"2024-07-04T17:26:17.020967+0000","last_undegraded":"2024-07-04T17:26:17.020967+0000","last_fullsized":"2024-07-04T17:26:17.020967+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:30:12.289936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026054+0000","last_change":"2024-07-04T17:26:16.056551+0000","last_active":"2024-07-04T17:26:17.026054+0000","last_peered":"2024-07-04T17:26:17.026054+0000","last_clean":"2024-07-04T17:26:17.026054+0000","last_became_active":"2024-07-04T17:26:16.056173+0000","last_became_peered":"2024-07-04T17:26:16.056173+0000","last_unstale":"2024-07-04T17:26:17.026054+0000","last_undegraded":"2024-07-04T17:26:17.026054+0000","last_fullsized":"2024-07-04T17:26:17.026054+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:20:14.576761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.700980+0000","last_change":"2024-07-04T17:26:16.073053+0000","last_active":"2024-07-04T17:26:17.700980+0000","last_peered":"2024-07-04T17:26:17.700980+0000","last_clean":"2024-07-04T17:26:17.700980+0000","last_became_active":"2024-07-04T17:26:16.072132+0000","last_became_peered":"2024-07-04T17:26:16.072132+0000","last_unstale":"2024-07-04T17:26:17.700980+0000","last_undegraded":"2024-07-04T17:26:17.700980+0000","last_fullsized":"2024-07-04T17:26:17.700980+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:11:05.657045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.029516+0000","last_change":"2024-07-04T17:26:16.055789+0000","last_active":"2024-07-04T17:26:17.029516+0000","last_peered":"2024-07-04T17:26:17.029516+0000","last_clean":"2024-07-04T17:26:17.029516+0000","last_became_active":"2024-07-04T17:26:16.055206+0000","last_became_peered":"2024-07-04T17:26:16.055206+0000","last_unstale":"2024-07-04T17:26:17.029516+0000","last_undegraded":"2024-07-04T17:26:17.029516+0000","last_fullsized":"2024-07-04T17:26:17.029516+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:31:59.564668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.703184+0000","last_change":"2024-07-04T17:26:16.070972+0000","last_active":"2024-07-04T17:26:17.703184+0000","last_peered":"2024-07-04T17:26:17.703184+0000","last_clean":"2024-07-04T17:26:17.703184+0000","last_became_active":"2024-07-04T17:26:16.067224+0000","last_became_peered":"2024-07-04T17:26:16.067224+0000","last_unstale":"2024-07-04T17:26:17.703184+0000","last_undegraded":"2024-07-04T17:26:17.703184+0000","last_fullsized":"2024-07-04T17:26:17.703184+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:22:14.191409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022486+0000","last_change":"2024-07-04T17:26:16.059368+0000","last_active":"2024-07-04T17:26:17.022486+0000","last_peered":"2024-07-04T17:26:17.022486+0000","last_clean":"2024-07-04T17:26:17.022486+0000","last_became_active":"2024-07-04T17:26:16.058568+0000","last_became_peered":"2024-07-04T17:26:16.058568+0000","last_unstale":"2024-07-04T17:26:17.022486+0000","last_undegraded":"2024-07-04T17:26:17.022486+0000","last_fullsized":"2024-07-04T17:26:17.022486+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:18:06.133593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022008+0000","last_change":"2024-07-04T17:26:16.083687+0000","last_active":"2024-07-04T17:26:17.022008+0000","last_peered":"2024-07-04T17:26:17.022008+0000","last_clean":"2024-07-04T17:26:17.022008+0000","last_became_active":"2024-07-04T17:26:16.083145+0000","last_became_peered":"2024-07-04T17:26:16.083145+0000","last_unstale":"2024-07-04T17:26:17.022008+0000","last_undegraded":"2024-07-04T17:26:17.022008+0000","last_fullsized":"2024-07-04T17:26:17.022008+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:36:50.247043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020646+0000","last_change":"2024-07-04T17:26:16.047023+0000","last_active":"2024-07-04T17:26:17.020646+0000","last_peered":"2024-07-04T17:26:17.020646+0000","last_clean":"2024-07-04T17:26:17.020646+0000","last_became_active":"2024-07-04T17:26:16.046631+0000","last_became_peered":"2024-07-04T17:26:16.046631+0000","last_unstale":"2024-07-04T17:26:17.020646+0000","last_undegraded":"2024-07-04T17:26:17.020646+0000","last_fullsized":"2024-07-04T17:26:17.020646+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:13:00.963495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026306+0000","last_change":"2024-07-04T17:26:16.075119+0000","last_active":"2024-07-04T17:26:17.026306+0000","last_peered":"2024-07-04T17:26:17.026306+0000","last_clean":"2024-07-04T17:26:17.026306+0000","last_became_active":"2024-07-04T17:26:16.074600+0000","last_became_peered":"2024-07-04T17:26:16.074600+0000","last_unstale":"2024-07-04T17:26:17.026306+0000","last_undegraded":"2024-07-04T17:26:17.026306+0000","last_fullsized":"2024-07-04T17:26:17.026306+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:17:18.408614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021060+0000","last_change":"2024-07-04T17:26:16.052698+0000","last_active":"2024-07-04T17:26:17.021060+0000","last_peered":"2024-07-04T17:26:17.021060+0000","last_clean":"2024-07-04T17:26:17.021060+0000","last_became_active":"2024-07-04T17:26:16.052278+0000","last_became_peered":"2024-07-04T17:26:16.052278+0000","last_unstale":"2024-07-04T17:26:17.021060+0000","last_undegraded":"2024-07-04T17:26:17.021060+0000","last_fullsized":"2024-07-04T17:26:17.021060+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:27:20.732374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020541+0000","last_change":"2024-07-04T17:26:16.052254+0000","last_active":"2024-07-04T17:26:17.020541+0000","last_peered":"2024-07-04T17:26:17.020541+0000","last_clean":"2024-07-04T17:26:17.020541+0000","last_became_active":"2024-07-04T17:26:16.046560+0000","last_became_peered":"2024-07-04T17:26:16.046560+0000","last_unstale":"2024-07-04T17:26:17.020541+0000","last_undegraded":"2024-07-04T17:26:17.020541+0000","last_fullsized":"2024-07-04T17:26:17.020541+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:50:25.908300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702155+0000","last_change":"2024-07-04T17:26:16.061658+0000","last_active":"2024-07-04T17:26:17.702155+0000","last_peered":"2024-07-04T17:26:17.702155+0000","last_clean":"2024-07-04T17:26:17.702155+0000","last_became_active":"2024-07-04T17:26:16.061257+0000","last_became_peered":"2024-07-04T17:26:16.061257+0000","last_unstale":"2024-07-04T17:26:17.702155+0000","last_undegraded":"2024-07-04T17:26:17.702155+0000","last_fullsized":"2024-07-04T17:26:17.702155+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:17:18.408614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.703496+0000","last_change":"2024-07-04T17:26:16.073828+0000","last_active":"2024-07-04T17:26:17.703496+0000","last_peered":"2024-07-04T17:26:17.703496+0000","last_clean":"2024-07-04T17:26:17.703496+0000","last_became_active":"2024-07-04T17:26:16.073494+0000","last_became_peered":"2024-07-04T17:26:16.073494+0000","last_unstale":"2024-07-04T17:26:17.703496+0000","last_undegraded":"2024-07-04T17:26:17.703496+0000","last_fullsized":"2024-07-04T17:26:17.703496+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:41:20.224285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.028237+0000","last_change":"2024-07-04T17:26:16.046050+0000","last_active":"2024-07-04T17:26:17.028237+0000","last_peered":"2024-07-04T17:26:17.028237+0000","last_clean":"2024-07-04T17:26:17.028237+0000","last_became_active":"2024-07-04T17:26:16.043726+0000","last_became_peered":"2024-07-04T17:26:16.043726+0000","last_unstale":"2024-07-04T17:26:17.028237+0000","last_undegraded":"2024-07-04T17:26:17.028237+0000","last_fullsized":"2024-07-04T17:26:17.028237+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:43:29.877017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022171+0000","last_change":"2024-07-04T17:26:16.047247+0000","last_active":"2024-07-04T17:26:17.022171+0000","last_peered":"2024-07-04T17:26:17.022171+0000","last_clean":"2024-07-04T17:26:17.022171+0000","last_became_active":"2024-07-04T17:26:16.046845+0000","last_became_peered":"2024-07-04T17:26:16.046845+0000","last_unstale":"2024-07-04T17:26:17.022171+0000","last_undegraded":"2024-07-04T17:26:17.022171+0000","last_fullsized":"2024-07-04T17:26:17.022171+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:22:33.108870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020889+0000","last_change":"2024-07-04T17:26:16.047621+0000","last_active":"2024-07-04T17:26:17.020889+0000","last_peered":"2024-07-04T17:26:17.020889+0000","last_clean":"2024-07-04T17:26:17.020889+0000","last_became_active":"2024-07-04T17:26:16.047328+0000","last_became_peered":"2024-07-04T17:26:16.047328+0000","last_unstale":"2024-07-04T17:26:17.020889+0000","last_undegraded":"2024-07-04T17:26:17.020889+0000","last_fullsized":"2024-07-04T17:26:17.020889+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:03:58.364101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020557+0000","last_change":"2024-07-04T17:26:16.046513+0000","last_active":"2024-07-04T17:26:17.020557+0000","last_peered":"2024-07-04T17:26:17.020557+0000","last_clean":"2024-07-04T17:26:17.020557+0000","last_became_active":"2024-07-04T17:26:16.045399+0000","last_became_peered":"2024-07-04T17:26:16.045399+0000","last_unstale":"2024-07-04T17:26:17.020557+0000","last_undegraded":"2024-07-04T17:26:17.020557+0000","last_fullsized":"2024-07-04T17:26:17.020557+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:31:59.564668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.701258+0000","last_change":"2024-07-04T17:26:16.057205+0000","last_active":"2024-07-04T17:26:17.701258+0000","last_peered":"2024-07-04T17:26:17.701258+0000","last_clean":"2024-07-04T17:26:17.701258+0000","last_became_active":"2024-07-04T17:26:16.056891+0000","last_became_peered":"2024-07-04T17:26:16.056891+0000","last_unstale":"2024-07-04T17:26:17.701258+0000","last_undegraded":"2024-07-04T17:26:17.701258+0000","last_fullsized":"2024-07-04T17:26:17.701258+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:16:58.662657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.701357+0000","last_change":"2024-07-04T17:26:16.072774+0000","last_active":"2024-07-04T17:26:17.701357+0000","last_peered":"2024-07-04T17:26:17.701357+0000","last_clean":"2024-07-04T17:26:17.701357+0000","last_became_active":"2024-07-04T17:26:16.071711+0000","last_became_peered":"2024-07-04T17:26:16.071711+0000","last_unstale":"2024-07-04T17:26:17.701357+0000","last_undegraded":"2024-07-04T17:26:17.701357+0000","last_fullsized":"2024-07-04T17:26:17.701357+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:20:14.576761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026507+0000","last_change":"2024-07-04T17:26:16.047214+0000","last_active":"2024-07-04T17:26:17.026507+0000","last_peered":"2024-07-04T17:26:17.026507+0000","last_clean":"2024-07-04T17:26:17.026507+0000","last_became_active":"2024-07-04T17:26:16.046836+0000","last_became_peered":"2024-07-04T17:26:16.046836+0000","last_unstale":"2024-07-04T17:26:17.026507+0000","last_undegraded":"2024-07-04T17:26:17.026507+0000","last_fullsized":"2024-07-04T17:26:17.026507+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:04:04.805872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"14'1","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.027730+0000","last_change":"2024-07-04T17:26:15.033936+0000","last_active":"2024-07-04T17:26:17.027730+0000","last_peered":"2024-07-04T17:26:17.027730+0000","last_clean":"2024-07-04T17:26:17.027730+0000","last_became_active":"2024-07-04T17:26:15.033601+0000","last_became_peered":"2024-07-04T17:26:15.033601+0000","last_unstale":"2024-07-04T17:26:17.027730+0000","last_undegraded":"2024-07-04T17:26:17.027730+0000","last_fullsized":"2024-07-04T17:26:17.027730+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:17:16.904464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021045+0000","last_change":"2024-07-04T17:26:16.053083+0000","last_active":"2024-07-04T17:26:17.021045+0000","last_peered":"2024-07-04T17:26:17.021045+0000","last_clean":"2024-07-04T17:26:17.021045+0000","last_became_active":"2024-07-04T17:26:16.052739+0000","last_became_peered":"2024-07-04T17:26:16.052739+0000","last_unstale":"2024-07-04T17:26:17.021045+0000","last_undegraded":"2024-07-04T17:26:17.021045+0000","last_fullsized":"2024-07-04T17:26:17.021045+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:00:52.128797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"14'1","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.701569+0000","last_change":"2024-07-04T17:26:15.039637+0000","last_active":"2024-07-04T17:26:17.701569+0000","last_peered":"2024-07-04T17:26:17.701569+0000","last_clean":"2024-07-04T17:26:17.701569+0000","last_became_active":"2024-07-04T17:26:15.039305+0000","last_became_peered":"2024-07-04T17:26:15.039305+0000","last_unstale":"2024-07-04T17:26:17.701569+0000","last_undegraded":"2024-07-04T17:26:17.701569+0000","last_fullsized":"2024-07-04T17:26:17.701569+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:04:46.667644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021302+0000","last_change":"2024-07-04T17:26:15.027075+0000","last_active":"2024-07-04T17:26:17.021302+0000","last_peered":"2024-07-04T17:26:17.021302+0000","last_clean":"2024-07-04T17:26:17.021302+0000","last_became_active":"2024-07-04T17:26:15.026375+0000","last_became_peered":"2024-07-04T17:26:15.026375+0000","last_unstale":"2024-07-04T17:26:17.021302+0000","last_undegraded":"2024-07-04T17:26:17.021302+0000","last_fullsized":"2024-07-04T17:26:17.021302+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:24:04.318931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702805+0000","last_change":"2024-07-04T17:26:16.074834+0000","last_active":"2024-07-04T17:26:17.702805+0000","last_peered":"2024-07-04T17:26:17.702805+0000","last_clean":"2024-07-04T17:26:17.702805+0000","last_became_active":"2024-07-04T17:26:16.074498+0000","last_became_peered":"2024-07-04T17:26:16.074498+0000","last_unstale":"2024-07-04T17:26:17.702805+0000","last_undegraded":"2024-07-04T17:26:17.702805+0000","last_fullsized":"2024-07-04T17:26:17.702805+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:08:53.606368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020171+0000","last_change":"2024-07-04T17:26:15.046968+0000","last_active":"2024-07-04T17:26:17.020171+0000","last_peered":"2024-07-04T17:26:17.020171+0000","last_clean":"2024-07-04T17:26:17.020171+0000","last_became_active":"2024-07-04T17:26:15.046617+0000","last_became_peered":"2024-07-04T17:26:15.046617+0000","last_unstale":"2024-07-04T17:26:17.020171+0000","last_undegraded":"2024-07-04T17:26:17.020171+0000","last_fullsized":"2024-07-04T17:26:17.020171+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:13:00.368941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026043+0000","last_change":"2024-07-04T17:26:16.041627+0000","last_active":"2024-07-04T17:26:17.026043+0000","last_peered":"2024-07-04T17:26:17.026043+0000","last_clean":"2024-07-04T17:26:17.026043+0000","last_became_active":"2024-07-04T17:26:16.041274+0000","last_became_peered":"2024-07-04T17:26:16.041274+0000","last_unstale":"2024-07-04T17:26:17.026043+0000","last_undegraded":"2024-07-04T17:26:17.026043+0000","last_fullsized":"2024-07-04T17:26:17.026043+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:41:46.272734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.029021+0000","last_change":"2024-07-04T17:26:15.037603+0000","last_active":"2024-07-04T17:26:17.029021+0000","last_peered":"2024-07-04T17:26:17.029021+0000","last_clean":"2024-07-04T17:26:17.029021+0000","last_became_active":"2024-07-04T17:26:15.030406+0000","last_became_peered":"2024-07-04T17:26:15.030406+0000","last_unstale":"2024-07-04T17:26:17.029021+0000","last_undegraded":"2024-07-04T17:26:17.029021+0000","last_fullsized":"2024-07-04T17:26:17.029021+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:13:00.368941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.703750+0000","last_change":"2024-07-04T17:26:16.060475+0000","last_active":"2024-07-04T17:26:17.703750+0000","last_peered":"2024-07-04T17:26:17.703750+0000","last_clean":"2024-07-04T17:26:17.703750+0000","last_became_active":"2024-07-04T17:26:16.059861+0000","last_became_peered":"2024-07-04T17:26:16.059861+0000","last_unstale":"2024-07-04T17:26:17.703750+0000","last_undegraded":"2024-07-04T17:26:17.703750+0000","last_fullsized":"2024-07-04T17:26:17.703750+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:21:38.693638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.023859+0000","last_change":"2024-07-04T17:26:15.046597+0000","last_active":"2024-07-04T17:26:17.023859+0000","last_peered":"2024-07-04T17:26:17.023859+0000","last_clean":"2024-07-04T17:26:17.023859+0000","last_became_active":"2024-07-04T17:26:15.046243+0000","last_became_peered":"2024-07-04T17:26:15.046243+0000","last_unstale":"2024-07-04T17:26:17.023859+0000","last_undegraded":"2024-07-04T17:26:17.023859+0000","last_fullsized":"2024-07-04T17:26:17.023859+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:18:04.371426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.024536+0000","last_change":"2024-07-04T17:26:16.059218+0000","last_active":"2024-07-04T17:26:17.024536+0000","last_peered":"2024-07-04T17:26:17.024536+0000","last_clean":"2024-07-04T17:26:17.024536+0000","last_became_active":"2024-07-04T17:26:16.058877+0000","last_became_peered":"2024-07-04T17:26:16.058877+0000","last_unstale":"2024-07-04T17:26:17.024536+0000","last_undegraded":"2024-07-04T17:26:17.024536+0000","last_fullsized":"2024-07-04T17:26:17.024536+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:48:48.817671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"14'3","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.029077+0000","last_change":"2024-07-04T17:26:15.031110+0000","last_active":"2024-07-04T17:26:17.029077+0000","last_peered":"2024-07-04T17:26:17.029077+0000","last_clean":"2024-07-04T17:26:17.029077+0000","last_became_active":"2024-07-04T17:26:15.030786+0000","last_became_peered":"2024-07-04T17:26:15.030786+0000","last_unstale":"2024-07-04T17:26:17.029077+0000","last_undegraded":"2024-07-04T17:26:17.029077+0000","last_fullsized":"2024-07-04T17:26:17.029077+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:34:46.876523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.028971+0000","last_change":"2024-07-04T17:26:16.046513+0000","last_active":"2024-07-04T17:26:17.028971+0000","last_peered":"2024-07-04T17:26:17.028971+0000","last_clean":"2024-07-04T17:26:17.028971+0000","last_became_active":"2024-07-04T17:26:16.044971+0000","last_became_peered":"2024-07-04T17:26:16.044971+0000","last_unstale":"2024-07-04T17:26:17.028971+0000","last_undegraded":"2024-07-04T17:26:17.028971+0000","last_fullsized":"2024-07-04T17:26:17.028971+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:21:38.693638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"14'6","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702536+0000","last_change":"2024-07-04T17:26:15.042022+0000","last_active":"2024-07-04T17:26:17.702536+0000","last_peered":"2024-07-04T17:26:17.702536+0000","last_clean":"2024-07-04T17:26:17.702536+0000","last_became_active":"2024-07-04T17:26:15.041669+0000","last_became_peered":"2024-07-04T17:26:15.041669+0000","last_unstale":"2024-07-04T17:26:17.702536+0000","last_undegraded":"2024-07-04T17:26:17.702536+0000","last_fullsized":"2024-07-04T17:26:17.702536+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:35:22.891399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.024207+0000","last_change":"2024-07-04T17:26:16.059656+0000","last_active":"2024-07-04T17:26:17.024207+0000","last_peered":"2024-07-04T17:26:17.024207+0000","last_clean":"2024-07-04T17:26:17.024207+0000","last_became_active":"2024-07-04T17:26:16.058951+0000","last_became_peered":"2024-07-04T17:26:16.058951+0000","last_unstale":"2024-07-04T17:26:17.024207+0000","last_undegraded":"2024-07-04T17:26:17.024207+0000","last_fullsized":"2024-07-04T17:26:17.024207+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:52:45.300002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"14'2","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021264+0000","last_change":"2024-07-04T17:26:15.035545+0000","last_active":"2024-07-04T17:26:17.021264+0000","last_peered":"2024-07-04T17:26:17.021264+0000","last_clean":"2024-07-04T17:26:17.021264+0000","last_became_active":"2024-07-04T17:26:15.034822+0000","last_became_peered":"2024-07-04T17:26:15.034822+0000","last_unstale":"2024-07-04T17:26:17.021264+0000","last_undegraded":"2024-07-04T17:26:17.021264+0000","last_fullsized":"2024-07-04T17:26:17.021264+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:17:16.904464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020669+0000","last_change":"2024-07-04T17:26:16.064292+0000","last_active":"2024-07-04T17:26:17.020669+0000","last_peered":"2024-07-04T17:26:17.020669+0000","last_clean":"2024-07-04T17:26:17.020669+0000","last_became_active":"2024-07-04T17:26:16.053620+0000","last_became_peered":"2024-07-04T17:26:16.053620+0000","last_unstale":"2024-07-04T17:26:17.020669+0000","last_undegraded":"2024-07-04T17:26:17.020669+0000","last_fullsized":"2024-07-04T17:26:17.020669+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:40:49.177003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.701897+0000","last_change":"2024-07-04T17:26:15.041695+0000","last_active":"2024-07-04T17:26:17.701897+0000","last_peered":"2024-07-04T17:26:17.701897+0000","last_clean":"2024-07-04T17:26:17.701897+0000","last_became_active":"2024-07-04T17:26:15.041353+0000","last_became_peered":"2024-07-04T17:26:15.041353+0000","last_unstale":"2024-07-04T17:26:17.701897+0000","last_undegraded":"2024-07-04T17:26:17.701897+0000","last_fullsized":"2024-07-04T17:26:17.701897+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:05:39.042848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026437+0000","last_change":"2024-07-04T17:26:16.063355+0000","last_active":"2024-07-04T17:26:17.026437+0000","last_peered":"2024-07-04T17:26:17.026437+0000","last_clean":"2024-07-04T17:26:17.026437+0000","last_became_active":"2024-07-04T17:26:16.056707+0000","last_became_peered":"2024-07-04T17:26:16.056707+0000","last_unstale":"2024-07-04T17:26:17.026437+0000","last_undegraded":"2024-07-04T17:26:17.026437+0000","last_fullsized":"2024-07-04T17:26:17.026437+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:11:17.509522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021433+0000","last_change":"2024-07-04T17:26:15.050680+0000","last_active":"2024-07-04T17:26:17.021433+0000","last_peered":"2024-07-04T17:26:17.021433+0000","last_clean":"2024-07-04T17:26:17.021433+0000","last_became_active":"2024-07-04T17:26:15.049449+0000","last_became_peered":"2024-07-04T17:26:15.049449+0000","last_unstale":"2024-07-04T17:26:17.021433+0000","last_undegraded":"2024-07-04T17:26:17.021433+0000","last_fullsized":"2024-07-04T17:26:17.021433+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:25:38.473912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026199+0000","last_change":"2024-07-04T17:26:16.063777+0000","last_active":"2024-07-04T17:26:17.026199+0000","last_peered":"2024-07-04T17:26:17.026199+0000","last_clean":"2024-07-04T17:26:17.026199+0000","last_became_active":"2024-07-04T17:26:16.063580+0000","last_became_peered":"2024-07-04T17:26:16.063580+0000","last_unstale":"2024-07-04T17:26:17.026199+0000","last_undegraded":"2024-07-04T17:26:17.026199+0000","last_fullsized":"2024-07-04T17:26:17.026199+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:21:48.005606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"14'8","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702841+0000","last_change":"2024-07-04T17:26:15.043346+0000","last_active":"2024-07-04T17:26:17.702841+0000","last_peered":"2024-07-04T17:26:17.702841+0000","last_clean":"2024-07-04T17:26:17.702841+0000","last_became_active":"2024-07-04T17:26:15.041219+0000","last_became_peered":"2024-07-04T17:26:15.041219+0000","last_unstale":"2024-07-04T17:26:17.702841+0000","last_undegraded":"2024-07-04T17:26:17.702841+0000","last_fullsized":"2024-07-04T17:26:17.702841+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:13:00.368941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.701310+0000","last_change":"2024-07-04T17:26:16.057277+0000","last_active":"2024-07-04T17:26:17.701310+0000","last_peered":"2024-07-04T17:26:17.701310+0000","last_clean":"2024-07-04T17:26:17.701310+0000","last_became_active":"2024-07-04T17:26:16.056946+0000","last_became_peered":"2024-07-04T17:26:16.056946+0000","last_unstale":"2024-07-04T17:26:17.701310+0000","last_undegraded":"2024-07-04T17:26:17.701310+0000","last_fullsized":"2024-07-04T17:26:17.701310+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:41:46.272734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"14'1","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.027552+0000","last_change":"2024-07-04T17:26:15.049872+0000","last_active":"2024-07-04T17:26:17.027552+0000","last_peered":"2024-07-04T17:26:17.027552+0000","last_clean":"2024-07-04T17:26:17.027552+0000","last_became_active":"2024-07-04T17:26:15.048975+0000","last_became_peered":"2024-07-04T17:26:15.048975+0000","last_unstale":"2024-07-04T17:26:17.027552+0000","last_undegraded":"2024-07-04T17:26:17.027552+0000","last_fullsized":"2024-07-04T17:26:17.027552+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:18:52.190216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.704159+0000","last_change":"2024-07-04T17:26:16.071254+0000","last_active":"2024-07-04T17:26:17.704159+0000","last_peered":"2024-07-04T17:26:17.704159+0000","last_clean":"2024-07-04T17:26:17.704159+0000","last_became_active":"2024-07-04T17:26:16.070624+0000","last_became_peered":"2024-07-04T17:26:16.070624+0000","last_unstale":"2024-07-04T17:26:17.704159+0000","last_undegraded":"2024-07-04T17:26:17.704159+0000","last_fullsized":"2024-07-04T17:26:17.704159+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:50:25.908300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"14'2","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.704347+0000","last_change":"2024-07-04T17:26:15.048120+0000","last_active":"2024-07-04T17:26:17.704347+0000","last_peered":"2024-07-04T17:26:17.704347+0000","last_clean":"2024-07-04T17:26:17.704347+0000","last_became_active":"2024-07-04T17:26:15.047745+0000","last_became_peered":"2024-07-04T17:26:15.047745+0000","last_unstale":"2024-07-04T17:26:17.704347+0000","last_undegraded":"2024-07-04T17:26:17.704347+0000","last_fullsized":"2024-07-04T17:26:17.704347+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:17:16.904464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.024662+0000","last_change":"2024-07-04T17:26:16.047526+0000","last_active":"2024-07-04T17:26:17.024662+0000","last_peered":"2024-07-04T17:26:17.024662+0000","last_clean":"2024-07-04T17:26:17.024662+0000","last_became_active":"2024-07-04T17:26:16.046143+0000","last_became_peered":"2024-07-04T17:26:16.046143+0000","last_unstale":"2024-07-04T17:26:17.024662+0000","last_undegraded":"2024-07-04T17:26:17.024662+0000","last_fullsized":"2024-07-04T17:26:17.024662+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:48:44.510528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"14'1","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.029637+0000","last_change":"2024-07-04T17:26:15.021551+0000","last_active":"2024-07-04T17:26:17.029637+0000","last_peered":"2024-07-04T17:26:17.029637+0000","last_clean":"2024-07-04T17:26:17.029637+0000","last_became_active":"2024-07-04T17:26:15.021324+0000","last_became_peered":"2024-07-04T17:26:15.021324+0000","last_unstale":"2024-07-04T17:26:17.029637+0000","last_undegraded":"2024-07-04T17:26:17.029637+0000","last_fullsized":"2024-07-04T17:26:17.029637+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:18:04.371426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022385+0000","last_change":"2024-07-04T17:26:16.043753+0000","last_active":"2024-07-04T17:26:17.022385+0000","last_peered":"2024-07-04T17:26:17.022385+0000","last_clean":"2024-07-04T17:26:17.022385+0000","last_became_active":"2024-07-04T17:26:16.037154+0000","last_became_peered":"2024-07-04T17:26:16.037154+0000","last_unstale":"2024-07-04T17:26:17.022385+0000","last_undegraded":"2024-07-04T17:26:17.022385+0000","last_fullsized":"2024-07-04T17:26:17.022385+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:30:49.842921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702886+0000","last_change":"2024-07-04T17:26:15.039837+0000","last_active":"2024-07-04T17:26:17.702886+0000","last_peered":"2024-07-04T17:26:17.702886+0000","last_clean":"2024-07-04T17:26:17.702886+0000","last_became_active":"2024-07-04T17:26:15.039334+0000","last_became_peered":"2024-07-04T17:26:15.039334+0000","last_unstale":"2024-07-04T17:26:17.702886+0000","last_undegraded":"2024-07-04T17:26:17.702886+0000","last_fullsized":"2024-07-04T17:26:17.702886+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:46.107542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.030226+0000","last_change":"2024-07-04T17:26:16.061791+0000","last_active":"2024-07-04T17:26:17.030226+0000","last_peered":"2024-07-04T17:26:17.030226+0000","last_clean":"2024-07-04T17:26:17.030226+0000","last_became_active":"2024-07-04T17:26:16.061453+0000","last_became_peered":"2024-07-04T17:26:16.061453+0000","last_unstale":"2024-07-04T17:26:17.030226+0000","last_undegraded":"2024-07-04T17:26:17.030226+0000","last_fullsized":"2024-07-04T17:26:17.030226+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:13:00.963495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'52","reported_seq":87,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026937+0000","last_change":"2024-07-04T17:26:04.953559+0000","last_active":"2024-07-04T17:26:17.026937+0000","last_peered":"2024-07-04T17:26:17.026937+0000","last_clean":"2024-07-04T17:26:17.026937+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:26:17.026937+0000","last_undegraded":"2024-07-04T17:26:17.026937+0000","last_fullsized":"2024-07-04T17:26:17.026937+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_clean_scrub_stamp":"2024-07-04T17:26:03.933432+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.064181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.027937+0000","last_change":"2024-07-04T17:26:15.026582+0000","last_active":"2024-07-04T17:26:17.027937+0000","last_peered":"2024-07-04T17:26:17.027937+0000","last_clean":"2024-07-04T17:26:17.027937+0000","last_became_active":"2024-07-04T17:26:15.026170+0000","last_became_peered":"2024-07-04T17:26:15.026170+0000","last_unstale":"2024-07-04T17:26:17.027937+0000","last_undegraded":"2024-07-04T17:26:17.027937+0000","last_fullsized":"2024-07-04T17:26:17.027937+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:45:20.253806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026496+0000","last_change":"2024-07-04T17:26:16.044579+0000","last_active":"2024-07-04T17:26:17.026496+0000","last_peered":"2024-07-04T17:26:17.026496+0000","last_clean":"2024-07-04T17:26:17.026496+0000","last_became_active":"2024-07-04T17:26:16.044255+0000","last_became_peered":"2024-07-04T17:26:16.044255+0000","last_unstale":"2024-07-04T17:26:17.026496+0000","last_undegraded":"2024-07-04T17:26:17.026496+0000","last_fullsized":"2024-07-04T17:26:17.026496+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:30:20.910427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.703153+0000","last_change":"2024-07-04T17:26:15.043650+0000","last_active":"2024-07-04T17:26:17.703153+0000","last_peered":"2024-07-04T17:26:17.703153+0000","last_clean":"2024-07-04T17:26:17.703153+0000","last_became_active":"2024-07-04T17:26:15.042392+0000","last_became_peered":"2024-07-04T17:26:15.042392+0000","last_unstale":"2024-07-04T17:26:17.703153+0000","last_undegraded":"2024-07-04T17:26:17.703153+0000","last_fullsized":"2024-07-04T17:26:17.703153+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:18:04.371426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021141+0000","last_change":"2024-07-04T17:26:16.051770+0000","last_active":"2024-07-04T17:26:17.021141+0000","last_peered":"2024-07-04T17:26:17.021141+0000","last_clean":"2024-07-04T17:26:17.021141+0000","last_became_active":"2024-07-04T17:26:16.046616+0000","last_became_peered":"2024-07-04T17:26:16.046616+0000","last_unstale":"2024-07-04T17:26:17.021141+0000","last_undegraded":"2024-07-04T17:26:17.021141+0000","last_fullsized":"2024-07-04T17:26:17.021141+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:03:12.070345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702913+0000","last_change":"2024-07-04T17:26:15.042707+0000","last_active":"2024-07-04T17:26:17.702913+0000","last_peered":"2024-07-04T17:26:17.702913+0000","last_clean":"2024-07-04T17:26:17.702913+0000","last_became_active":"2024-07-04T17:26:15.041346+0000","last_became_peered":"2024-07-04T17:26:15.041346+0000","last_unstale":"2024-07-04T17:26:17.702913+0000","last_undegraded":"2024-07-04T17:26:17.702913+0000","last_fullsized":"2024-07-04T17:26:17.702913+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:31:03.314632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026776+0000","last_change":"2024-07-04T17:26:16.039792+0000","last_active":"2024-07-04T17:26:17.026776+0000","last_peered":"2024-07-04T17:26:17.026776+0000","last_clean":"2024-07-04T17:26:17.026776+0000","last_became_active":"2024-07-04T17:26:16.039470+0000","last_became_peered":"2024-07-04T17:26:16.039470+0000","last_unstale":"2024-07-04T17:26:17.026776+0000","last_undegraded":"2024-07-04T17:26:17.026776+0000","last_fullsized":"2024-07-04T17:26:17.026776+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:08:53.606368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022753+0000","last_change":"2024-07-04T17:26:15.046123+0000","last_active":"2024-07-04T17:26:17.022753+0000","last_peered":"2024-07-04T17:26:17.022753+0000","last_clean":"2024-07-04T17:26:17.022753+0000","last_became_active":"2024-07-04T17:26:15.045824+0000","last_became_peered":"2024-07-04T17:26:15.045824+0000","last_unstale":"2024-07-04T17:26:17.022753+0000","last_undegraded":"2024-07-04T17:26:17.022753+0000","last_fullsized":"2024-07-04T17:26:17.022753+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:46.107542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021092+0000","last_change":"2024-07-04T17:26:16.044581+0000","last_active":"2024-07-04T17:26:17.021092+0000","last_peered":"2024-07-04T17:26:17.021092+0000","last_clean":"2024-07-04T17:26:17.021092+0000","last_became_active":"2024-07-04T17:26:16.043791+0000","last_became_peered":"2024-07-04T17:26:16.043791+0000","last_unstale":"2024-07-04T17:26:17.021092+0000","last_undegraded":"2024-07-04T17:26:17.021092+0000","last_fullsized":"2024-07-04T17:26:17.021092+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:38:17.647736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.029309+0000","last_change":"2024-07-04T17:26:15.032798+0000","last_active":"2024-07-04T17:26:17.029309+0000","last_peered":"2024-07-04T17:26:17.029309+0000","last_clean":"2024-07-04T17:26:17.029309+0000","last_became_active":"2024-07-04T17:26:15.029686+0000","last_became_peered":"2024-07-04T17:26:15.029686+0000","last_unstale":"2024-07-04T17:26:17.029309+0000","last_undegraded":"2024-07-04T17:26:17.029309+0000","last_fullsized":"2024-07-04T17:26:17.029309+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:25:38.473912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702222+0000","last_change":"2024-07-04T17:26:16.062731+0000","last_active":"2024-07-04T17:26:17.702222+0000","last_peered":"2024-07-04T17:26:17.702222+0000","last_clean":"2024-07-04T17:26:17.702222+0000","last_became_active":"2024-07-04T17:26:16.062519+0000","last_became_peered":"2024-07-04T17:26:16.062519+0000","last_unstale":"2024-07-04T17:26:17.702222+0000","last_undegraded":"2024-07-04T17:26:17.702222+0000","last_fullsized":"2024-07-04T17:26:17.702222+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:30:20.910427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.029919+0000","last_change":"2024-07-04T17:26:15.036880+0000","last_active":"2024-07-04T17:26:17.029919+0000","last_peered":"2024-07-04T17:26:17.029919+0000","last_clean":"2024-07-04T17:26:17.029919+0000","last_became_active":"2024-07-04T17:26:15.036529+0000","last_became_peered":"2024-07-04T17:26:15.036529+0000","last_unstale":"2024-07-04T17:26:17.029919+0000","last_undegraded":"2024-07-04T17:26:17.029919+0000","last_fullsized":"2024-07-04T17:26:17.029919+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:04:46.667644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.029152+0000","last_change":"2024-07-04T17:26:16.046186+0000","last_active":"2024-07-04T17:26:17.029152+0000","last_peered":"2024-07-04T17:26:17.029152+0000","last_clean":"2024-07-04T17:26:17.029152+0000","last_became_active":"2024-07-04T17:26:16.044588+0000","last_became_peered":"2024-07-04T17:26:16.044588+0000","last_unstale":"2024-07-04T17:26:17.029152+0000","last_undegraded":"2024-07-04T17:26:17.029152+0000","last_fullsized":"2024-07-04T17:26:17.029152+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:50:25.908300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"15'1","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.030544+0000","last_change":"2024-07-04T17:26:15.022516+0000","last_active":"2024-07-04T17:26:17.030544+0000","last_peered":"2024-07-04T17:26:17.030544+0000","last_clean":"2024-07-04T17:26:17.030544+0000","last_became_active":"2024-07-04T17:26:15.022185+0000","last_became_peered":"2024-07-04T17:26:15.022185+0000","last_unstale":"2024-07-04T17:26:17.030544+0000","last_undegraded":"2024-07-04T17:26:17.030544+0000","last_fullsized":"2024-07-04T17:26:17.030544+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:24:04.318931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702984+0000","last_change":"2024-07-04T17:26:16.059869+0000","last_active":"2024-07-04T17:26:17.702984+0000","last_peered":"2024-07-04T17:26:17.702984+0000","last_clean":"2024-07-04T17:26:17.702984+0000","last_became_active":"2024-07-04T17:26:16.059535+0000","last_became_peered":"2024-07-04T17:26:16.059535+0000","last_unstale":"2024-07-04T17:26:17.702984+0000","last_undegraded":"2024-07-04T17:26:17.702984+0000","last_fullsized":"2024-07-04T17:26:17.702984+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:43:29.877017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"14'2","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.029845+0000","last_change":"2024-07-04T17:26:15.040384+0000","last_active":"2024-07-04T17:26:17.029845+0000","last_peered":"2024-07-04T17:26:17.029845+0000","last_clean":"2024-07-04T17:26:17.029845+0000","last_became_active":"2024-07-04T17:26:15.038959+0000","last_became_peered":"2024-07-04T17:26:15.038959+0000","last_unstale":"2024-07-04T17:26:17.029845+0000","last_undegraded":"2024-07-04T17:26:17.029845+0000","last_fullsized":"2024-07-04T17:26:17.029845+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:18:52.190216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.023117+0000","last_change":"2024-07-04T17:26:16.053639+0000","last_active":"2024-07-04T17:26:17.023117+0000","last_peered":"2024-07-04T17:26:17.023117+0000","last_clean":"2024-07-04T17:26:17.023117+0000","last_became_active":"2024-07-04T17:26:16.052766+0000","last_became_peered":"2024-07-04T17:26:16.052766+0000","last_unstale":"2024-07-04T17:26:17.023117+0000","last_undegraded":"2024-07-04T17:26:17.023117+0000","last_fullsized":"2024-07-04T17:26:17.023117+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:22:12.063177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020302+0000","last_change":"2024-07-04T17:26:15.027718+0000","last_active":"2024-07-04T17:26:17.020302+0000","last_peered":"2024-07-04T17:26:17.020302+0000","last_clean":"2024-07-04T17:26:17.020302+0000","last_became_active":"2024-07-04T17:26:15.026662+0000","last_became_peered":"2024-07-04T17:26:15.026662+0000","last_unstale":"2024-07-04T17:26:17.020302+0000","last_undegraded":"2024-07-04T17:26:17.020302+0000","last_fullsized":"2024-07-04T17:26:17.020302+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:45:20.253806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020487+0000","last_change":"2024-07-04T17:26:16.066686+0000","last_active":"2024-07-04T17:26:17.020487+0000","last_peered":"2024-07-04T17:26:17.020487+0000","last_clean":"2024-07-04T17:26:17.020487+0000","last_became_active":"2024-07-04T17:26:16.066397+0000","last_became_peered":"2024-07-04T17:26:16.066397+0000","last_unstale":"2024-07-04T17:26:17.020487+0000","last_undegraded":"2024-07-04T17:26:17.020487+0000","last_fullsized":"2024-07-04T17:26:17.020487+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:21:38.693638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026169+0000","last_change":"2024-07-04T17:26:15.034670+0000","last_active":"2024-07-04T17:26:17.026169+0000","last_peered":"2024-07-04T17:26:17.026169+0000","last_clean":"2024-07-04T17:26:17.026169+0000","last_became_active":"2024-07-04T17:26:15.034267+0000","last_became_peered":"2024-07-04T17:26:15.034267+0000","last_unstale":"2024-07-04T17:26:17.026169+0000","last_undegraded":"2024-07-04T17:26:17.026169+0000","last_fullsized":"2024-07-04T17:26:17.026169+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:46.107542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702508+0000","last_change":"2024-07-04T17:26:16.054400+0000","last_active":"2024-07-04T17:26:17.702508+0000","last_peered":"2024-07-04T17:26:17.702508+0000","last_clean":"2024-07-04T17:26:17.702508+0000","last_became_active":"2024-07-04T17:26:16.054057+0000","last_became_peered":"2024-07-04T17:26:16.054057+0000","last_unstale":"2024-07-04T17:26:17.702508+0000","last_undegraded":"2024-07-04T17:26:17.702508+0000","last_fullsized":"2024-07-04T17:26:17.702508+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:04:04.805872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026719+0000","last_change":"2024-07-04T17:26:15.032894+0000","last_active":"2024-07-04T17:26:17.026719+0000","last_peered":"2024-07-04T17:26:17.026719+0000","last_clean":"2024-07-04T17:26:17.026719+0000","last_became_active":"2024-07-04T17:26:15.032634+0000","last_became_peered":"2024-07-04T17:26:15.032634+0000","last_unstale":"2024-07-04T17:26:17.026719+0000","last_undegraded":"2024-07-04T17:26:17.026719+0000","last_fullsized":"2024-07-04T17:26:17.026719+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:05:39.042848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026622+0000","last_change":"2024-07-04T17:26:16.065536+0000","last_active":"2024-07-04T17:26:17.026622+0000","last_peered":"2024-07-04T17:26:17.026622+0000","last_clean":"2024-07-04T17:26:17.026622+0000","last_became_active":"2024-07-04T17:26:16.056278+0000","last_became_peered":"2024-07-04T17:26:16.056278+0000","last_unstale":"2024-07-04T17:26:17.026622+0000","last_undegraded":"2024-07-04T17:26:17.026622+0000","last_fullsized":"2024-07-04T17:26:17.026622+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:03:07.909265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"14'2","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021818+0000","last_change":"2024-07-04T17:26:15.033912+0000","last_active":"2024-07-04T17:26:17.021818+0000","last_peered":"2024-07-04T17:26:17.021818+0000","last_clean":"2024-07-04T17:26:17.021818+0000","last_became_active":"2024-07-04T17:26:15.027824+0000","last_became_peered":"2024-07-04T17:26:15.027824+0000","last_unstale":"2024-07-04T17:26:17.021818+0000","last_undegraded":"2024-07-04T17:26:17.021818+0000","last_fullsized":"2024-07-04T17:26:17.021818+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:34:46.876523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020817+0000","last_change":"2024-07-04T17:26:16.065218+0000","last_active":"2024-07-04T17:26:17.020817+0000","last_peered":"2024-07-04T17:26:17.020817+0000","last_clean":"2024-07-04T17:26:17.020817+0000","last_became_active":"2024-07-04T17:26:16.064904+0000","last_became_peered":"2024-07-04T17:26:16.064904+0000","last_unstale":"2024-07-04T17:26:17.020817+0000","last_undegraded":"2024-07-04T17:26:17.020817+0000","last_fullsized":"2024-07-04T17:26:17.020817+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:33:23.722640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702115+0000","last_change":"2024-07-04T17:26:16.073564+0000","last_active":"2024-07-04T17:26:17.702115+0000","last_peered":"2024-07-04T17:26:17.702115+0000","last_clean":"2024-07-04T17:26:17.702115+0000","last_became_active":"2024-07-04T17:26:16.073189+0000","last_became_peered":"2024-07-04T17:26:16.073189+0000","last_unstale":"2024-07-04T17:26:17.702115+0000","last_undegraded":"2024-07-04T17:26:17.702115+0000","last_fullsized":"2024-07-04T17:26:17.702115+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:21:48.005606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"14'2","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022583+0000","last_change":"2024-07-04T17:26:15.049001+0000","last_active":"2024-07-04T17:26:17.022583+0000","last_peered":"2024-07-04T17:26:17.022583+0000","last_clean":"2024-07-04T17:26:17.022583+0000","last_became_active":"2024-07-04T17:26:15.048437+0000","last_became_peered":"2024-07-04T17:26:15.048437+0000","last_unstale":"2024-07-04T17:26:17.022583+0000","last_undegraded":"2024-07-04T17:26:17.022583+0000","last_fullsized":"2024-07-04T17:26:17.022583+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:15:08.055102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.028692+0000","last_change":"2024-07-04T17:26:16.064782+0000","last_active":"2024-07-04T17:26:17.028692+0000","last_peered":"2024-07-04T17:26:17.028692+0000","last_clean":"2024-07-04T17:26:17.028692+0000","last_became_active":"2024-07-04T17:26:16.064518+0000","last_became_peered":"2024-07-04T17:26:16.064518+0000","last_unstale":"2024-07-04T17:26:17.028692+0000","last_undegraded":"2024-07-04T17:26:17.028692+0000","last_fullsized":"2024-07-04T17:26:17.028692+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:01:31.824881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"14'2","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022678+0000","last_change":"2024-07-04T17:26:15.047259+0000","last_active":"2024-07-04T17:26:17.022678+0000","last_peered":"2024-07-04T17:26:17.022678+0000","last_clean":"2024-07-04T17:26:17.022678+0000","last_became_active":"2024-07-04T17:26:15.046951+0000","last_became_peered":"2024-07-04T17:26:15.046951+0000","last_unstale":"2024-07-04T17:26:17.022678+0000","last_undegraded":"2024-07-04T17:26:17.022678+0000","last_fullsized":"2024-07-04T17:26:17.022678+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:04:46.667644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.024493+0000","last_change":"2024-07-04T17:26:16.058196+0000","last_active":"2024-07-04T17:26:17.024493+0000","last_peered":"2024-07-04T17:26:17.024493+0000","last_clean":"2024-07-04T17:26:17.024493+0000","last_became_active":"2024-07-04T17:26:16.051038+0000","last_became_peered":"2024-07-04T17:26:16.051038+0000","last_unstale":"2024-07-04T17:26:17.024493+0000","last_undegraded":"2024-07-04T17:26:17.024493+0000","last_fullsized":"2024-07-04T17:26:17.024493+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:04:50.688662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"14'1","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.704046+0000","last_change":"2024-07-04T17:26:15.040464+0000","last_active":"2024-07-04T17:26:17.704046+0000","last_peered":"2024-07-04T17:26:17.704046+0000","last_clean":"2024-07-04T17:26:17.704046+0000","last_became_active":"2024-07-04T17:26:15.040008+0000","last_became_peered":"2024-07-04T17:26:15.040008+0000","last_unstale":"2024-07-04T17:26:17.704046+0000","last_undegraded":"2024-07-04T17:26:17.704046+0000","last_fullsized":"2024-07-04T17:26:17.704046+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:34:46.876523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.703450+0000","last_change":"2024-07-04T17:26:16.060159+0000","last_active":"2024-07-04T17:26:17.703450+0000","last_peered":"2024-07-04T17:26:17.703450+0000","last_clean":"2024-07-04T17:26:17.703450+0000","last_became_active":"2024-07-04T17:26:16.059458+0000","last_became_peered":"2024-07-04T17:26:16.059458+0000","last_unstale":"2024-07-04T17:26:17.703450+0000","last_undegraded":"2024-07-04T17:26:17.703450+0000","last_fullsized":"2024-07-04T17:26:17.703450+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:01:31.824881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"14'4","reported_seq":18,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.027492+0000","last_change":"2024-07-04T17:26:15.049639+0000","last_active":"2024-07-04T17:26:17.027492+0000","last_peered":"2024-07-04T17:26:17.027492+0000","last_clean":"2024-07-04T17:26:17.027492+0000","last_became_active":"2024-07-04T17:26:15.048590+0000","last_became_peered":"2024-07-04T17:26:15.048590+0000","last_unstale":"2024-07-04T17:26:17.027492+0000","last_undegraded":"2024-07-04T17:26:17.027492+0000","last_fullsized":"2024-07-04T17:26:17.027492+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:05:39.042848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738373,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31332,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340508,"statfs":{"total":96636764160,"available":96604680192,"internally_reserved":0,"allocated":151552,"data_stored":42085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738374,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31920,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339920,"statfs":{"total":96636764160,"available":96604078080,"internally_reserved":0,"allocated":753664,"data_stored":633113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738372,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31912,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339928,"statfs":{"total":96636764160,"available":96604086272,"internally_reserved":0,"allocated":741376,"data_stored":631465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:26:21.332 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-04T17:26:21.333 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-07-04T17:26:21.333 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-07-04T17:26:21.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.667+0000 7f176b600640 1 -- 172.21.15.42:0/3218405622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1764141590 msgr2=0x7f17641419a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:21.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.667+0000 7f176b600640 1 --2- 172.21.15.42:0/3218405622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1764141590 0x7f17641419a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f174c005fd0 tx=0x7f174c02a5f0 comp rx=0 tx=0).stop 2024-07-04T17:26:21.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.667+0000 7f176b600640 1 -- 172.21.15.42:0/3218405622 shutdown_connections 2024-07-04T17:26:21.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.667+0000 7f176b600640 1 --2- 172.21.15.42:0/3218405622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1764141590 0x7f17641419a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:21.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.667+0000 7f176b600640 1 -- 172.21.15.42:0/3218405622 >> 172.21.15.42:0/3218405622 conn(0x7f176406c940 msgr2=0x7f176406ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:21.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.667+0000 7f176b600640 1 -- 172.21.15.42:0/3218405622 shutdown_connections 2024-07-04T17:26:21.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.667+0000 7f176b600640 1 -- 172.21.15.42:0/3218405622 wait complete. 2024-07-04T17:26:21.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.669+0000 7f176b600640 1 Processor -- start 2024-07-04T17:26:21.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.669+0000 7f176b600640 1 -- start start 2024-07-04T17:26:21.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.671+0000 7f176b600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17640bc040 0x7f17640c25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:21.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.671+0000 7f176b600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f17640b7530 con 0x7f17640bc040 2024-07-04T17:26:21.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.671+0000 7f1768c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17640bc040 0x7f17640c25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:21.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.671+0000 7f1768c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17640bc040 0x7f17640c25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34896/0 (socket says 172.21.15.42:34896) 2024-07-04T17:26:21.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.671+0000 7f1768c00640 1 -- 172.21.15.42:0/1404363642 learned_addr learned my addr 172.21.15.42:0/1404363642 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:21.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.671+0000 7f1768c00640 1 -- 172.21.15.42:0/1404363642 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1764140700 con 0x7f17640bc040 2024-07-04T17:26:21.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.671+0000 7f1768c00640 1 --2- 172.21.15.42:0/1404363642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17640bc040 0x7f17640c25e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f175400c6b0 tx=0x7f175400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:21.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.672+0000 7f1760c00640 1 -- 172.21.15.42:0/1404363642 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1754003aa0 con 0x7f17640bc040 2024-07-04T17:26:21.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.672+0000 7f1760c00640 1 -- 172.21.15.42:0/1404363642 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1754003c60 con 0x7f17640bc040 2024-07-04T17:26:21.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.672+0000 7f1760c00640 1 -- 172.21.15.42:0/1404363642 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17540050d0 con 0x7f17640bc040 2024-07-04T17:26:21.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.672+0000 7f176b600640 1 -- 172.21.15.42:0/1404363642 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1764140fe0 con 0x7f17640bc040 2024-07-04T17:26:21.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.672+0000 7f176b600640 1 -- 172.21.15.42:0/1404363642 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17641e7460 con 0x7f17640bc040 2024-07-04T17:26:21.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.673+0000 7f176b600640 1 -- 172.21.15.42:0/1404363642 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17640b7bb0 con 0x7f17640bc040 2024-07-04T17:26:21.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.673+0000 7f1760c00640 1 -- 172.21.15.42:0/1404363642 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f1754005290 con 0x7f17640bc040 2024-07-04T17:26:21.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.674+0000 7f1760c00640 1 --2- 172.21.15.42:0/1404363642 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f174003d2b0 0x7f174003f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:21.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.674+0000 7f1760c00640 1 -- 172.21.15.42:0/1404363642 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f1754051340 con 0x7f17640bc040 2024-07-04T17:26:21.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.674+0000 7f1763e00640 1 --2- 172.21.15.42:0/1404363642 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f174003d2b0 0x7f174003f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:21.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.675+0000 7f1763e00640 1 --2- 172.21.15.42:0/1404363642 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f174003d2b0 0x7f174003f7c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f174c033000 tx=0x7f174c004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:21.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.679+0000 7f1760c00640 1 -- 172.21.15.42:0/1404363642 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f175400d150 con 0x7f17640bc040 2024-07-04T17:26:21.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.889+0000 7f176b600640 1 -- 172.21.15.42:0/1404363642 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f1764069b70 con 0x7f17640bc040 2024-07-04T17:26:21.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.890+0000 7f1760c00640 1 -- 172.21.15.42:0/1404363642 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v22) ==== 86+0+160 (secure 0 0 0) 0x7f175400a610 con 0x7f17640bc040 2024-07-04T17:26:21.892 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:26:21.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.893+0000 7f176b600640 1 -- 172.21.15.42:0/1404363642 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f174003d2b0 msgr2=0x7f174003f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:21.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.893+0000 7f176b600640 1 --2- 172.21.15.42:0/1404363642 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f174003d2b0 0x7f174003f7c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f174c033000 tx=0x7f174c004730 comp rx=0 tx=0).stop 2024-07-04T17:26:21.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.893+0000 7f176b600640 1 -- 172.21.15.42:0/1404363642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17640bc040 msgr2=0x7f17640c25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:21.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.893+0000 7f176b600640 1 --2- 172.21.15.42:0/1404363642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17640bc040 0x7f17640c25e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f175400c6b0 tx=0x7f175400cb70 comp rx=0 tx=0).stop 2024-07-04T17:26:21.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.893+0000 7f176b600640 1 -- 172.21.15.42:0/1404363642 shutdown_connections 2024-07-04T17:26:21.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.893+0000 7f176b600640 1 --2- 172.21.15.42:0/1404363642 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f174003d2b0 0x7f174003f7c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:21.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.893+0000 7f176b600640 1 --2- 172.21.15.42:0/1404363642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17640bc040 0x7f17640c25e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:21.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.893+0000 7f176b600640 1 -- 172.21.15.42:0/1404363642 >> 172.21.15.42:0/1404363642 conn(0x7f176406c940 msgr2=0x7f17640740b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:21.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.893+0000 7f176b600640 1 -- 172.21.15.42:0/1404363642 shutdown_connections 2024-07-04T17:26:21.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:21.893+0000 7f176b600640 1 -- 172.21.15.42:0/1404363642 wait complete. 2024-07-04T17:26:21.920 INFO:teuthology.orchestra.run.smithi042.stdout:[{"version":22,"timestamp":"2024-07-04T17:26:01.977405+0000","name":"","changes":[{"name":"osd.0/osd_mclock_max_capacity_iops_ssd","new_value":"3541.919687"}]}] 2024-07-04T17:26:21.920 INFO:tasks.ceph_manager:config epoch is 22 2024-07-04T17:26:21.921 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-07-04T17:26:21.921 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-07-04T17:26:21.921 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-07-04T17:26:22.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.208+0000 7f4594200640 1 -- 172.21.15.42:0/3444800617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f458c0bc040 msgr2=0x7f458c0bd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:22.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.208+0000 7f4594200640 1 --2- 172.21.15.42:0/3444800617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f458c0bc040 0x7f458c0bd2d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f45800097b0 tx=0x7f458002a530 comp rx=0 tx=0).stop 2024-07-04T17:26:22.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.209+0000 7f4594200640 1 -- 172.21.15.42:0/3444800617 shutdown_connections 2024-07-04T17:26:22.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.209+0000 7f4594200640 1 --2- 172.21.15.42:0/3444800617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f458c0bc040 0x7f458c0bd2d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:22.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.209+0000 7f4594200640 1 -- 172.21.15.42:0/3444800617 >> 172.21.15.42:0/3444800617 conn(0x7f458c06c940 msgr2=0x7f458c06ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:22.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.209+0000 7f4594200640 1 -- 172.21.15.42:0/3444800617 shutdown_connections 2024-07-04T17:26:22.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.209+0000 7f4594200640 1 -- 172.21.15.42:0/3444800617 wait complete. 2024-07-04T17:26:22.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.211+0000 7f4594200640 1 Processor -- start 2024-07-04T17:26:22.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.211+0000 7f4594200640 1 -- start start 2024-07-04T17:26:22.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.212+0000 7f4594200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f458c0bc040 0x7f458c0bfae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:22.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.212+0000 7f4594200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f458c0b7530 con 0x7f458c0bc040 2024-07-04T17:26:22.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.212+0000 7f4591800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f458c0bc040 0x7f458c0bfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:22.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.212+0000 7f4591800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f458c0bc040 0x7f458c0bfae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34906/0 (socket says 172.21.15.42:34906) 2024-07-04T17:26:22.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.212+0000 7f4591800640 1 -- 172.21.15.42:0/2326826385 learned_addr learned my addr 172.21.15.42:0/2326826385 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:22.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.213+0000 7f4591800640 1 -- 172.21.15.42:0/2326826385 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458c0c25e0 con 0x7f458c0bc040 2024-07-04T17:26:22.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.213+0000 7f4591800640 1 --2- 172.21.15.42:0/2326826385 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f458c0bc040 0x7f458c0bfae0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f45800098d0 tx=0x7f4580033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:22.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.213+0000 7f4589600640 1 -- 172.21.15.42:0/2326826385 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4580031880 con 0x7f458c0bc040 2024-07-04T17:26:22.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.214+0000 7f4589600640 1 -- 172.21.15.42:0/2326826385 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4580031b10 con 0x7f458c0bc040 2024-07-04T17:26:22.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.214+0000 7f4589600640 1 -- 172.21.15.42:0/2326826385 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4580031df0 con 0x7f458c0bc040 2024-07-04T17:26:22.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.214+0000 7f4594200640 1 -- 172.21.15.42:0/2326826385 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f458c13fba0 con 0x7f458c0bc040 2024-07-04T17:26:22.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.214+0000 7f4594200640 1 -- 172.21.15.42:0/2326826385 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f458c0c1470 con 0x7f458c0bc040 2024-07-04T17:26:22.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.215+0000 7f4589600640 1 -- 172.21.15.42:0/2326826385 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f4580033070 con 0x7f458c0bc040 2024-07-04T17:26:22.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.215+0000 7f4594200640 1 -- 172.21.15.42:0/2326826385 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f458c0c1760 con 0x7f458c0bc040 2024-07-04T17:26:22.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.215+0000 7f4589600640 1 --2- 172.21.15.42:0/2326826385 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f456c03d2b0 0x7f456c03f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:22.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.215+0000 7f4589600640 1 -- 172.21.15.42:0/2326826385 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f45800728c0 con 0x7f458c0bc040 2024-07-04T17:26:22.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.216+0000 7f4590e00640 1 --2- 172.21.15.42:0/2326826385 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f456c03d2b0 0x7f456c03f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:22.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.216+0000 7f4590e00640 1 --2- 172.21.15.42:0/2326826385 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f456c03d2b0 0x7f456c03f7c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4578007fd0 tx=0x7f4578007920 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:22.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.221+0000 7f4589600640 1 -- 172.21.15.42:0/2326826385 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f458c0c1760 con 0x7f458c0bc040 2024-07-04T17:26:22.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.482+0000 7f4594200640 1 -- 172.21.15.42:0/2326826385 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f458c0b7c00 con 0x7f458c0bc040 2024-07-04T17:26:22.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.485+0000 7f4589600640 1 -- 172.21.15.42:0/2326826385 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+95488 (secure 0 0 0) 0x7f458003f030 con 0x7f458c0bc040 2024-07-04T17:26:22.488 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:26:22.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.488+0000 7f4594200640 1 -- 172.21.15.42:0/2326826385 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f456c03d2b0 msgr2=0x7f456c03f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:22.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.488+0000 7f4594200640 1 --2- 172.21.15.42:0/2326826385 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f456c03d2b0 0x7f456c03f7c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4578007fd0 tx=0x7f4578007920 comp rx=0 tx=0).stop 2024-07-04T17:26:22.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.488+0000 7f4594200640 1 -- 172.21.15.42:0/2326826385 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f458c0bc040 msgr2=0x7f458c0bfae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:22.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.488+0000 7f4594200640 1 --2- 172.21.15.42:0/2326826385 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f458c0bc040 0x7f458c0bfae0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f45800098d0 tx=0x7f4580033ad0 comp rx=0 tx=0).stop 2024-07-04T17:26:22.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.488+0000 7f4594200640 1 -- 172.21.15.42:0/2326826385 shutdown_connections 2024-07-04T17:26:22.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.488+0000 7f4594200640 1 --2- 172.21.15.42:0/2326826385 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f456c03d2b0 0x7f456c03f7c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:22.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.488+0000 7f4594200640 1 --2- 172.21.15.42:0/2326826385 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f458c0bc040 0x7f458c0bfae0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:22.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.488+0000 7f4594200640 1 -- 172.21.15.42:0/2326826385 >> 172.21.15.42:0/2326826385 conn(0x7f458c06c940 msgr2=0x7f458c140eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:22.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.488+0000 7f4594200640 1 -- 172.21.15.42:0/2326826385 shutdown_connections 2024-07-04T17:26:22.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.489+0000 7f4594200640 1 -- 172.21.15.42:0/2326826385 wait complete. 2024-07-04T17:26:22.514 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":5,"flags":0,"active_gid":4103,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":3523793950},{"type":"v1","addr":"172.21.15.42:6825","nonce":3523793950}]},"active_addr":"172.21.15.42:6825/3523793950","active_change":"2024-07-04T17:25:57.240346+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":880857241}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":3526015921}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":2974639683}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":83074206}]}]} 2024-07-04T17:26:22.516 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-07-04T17:26:22.517 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-07-04T17:26:22.517 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:26:22.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.805+0000 7f9694e00640 1 -- 172.21.15.42:0/2253680896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96900b1d20 msgr2=0x7f96900b6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:22.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.805+0000 7f9694e00640 1 --2- 172.21.15.42:0/2253680896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96900b1d20 0x7f96900b6230 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9678005fd0 tx=0x7f967802a5f0 comp rx=0 tx=0).stop 2024-07-04T17:26:22.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.805+0000 7f9694e00640 1 -- 172.21.15.42:0/2253680896 shutdown_connections 2024-07-04T17:26:22.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.805+0000 7f9694e00640 1 --2- 172.21.15.42:0/2253680896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96900b1d20 0x7f96900b6230 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:22.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.805+0000 7f9694e00640 1 -- 172.21.15.42:0/2253680896 >> 172.21.15.42:0/2253680896 conn(0x7f96900bebc0 msgr2=0x7f96900befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:22.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.805+0000 7f9694e00640 1 -- 172.21.15.42:0/2253680896 shutdown_connections 2024-07-04T17:26:22.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.805+0000 7f9694e00640 1 -- 172.21.15.42:0/2253680896 wait complete. 2024-07-04T17:26:22.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.808+0000 7f9694e00640 1 Processor -- start 2024-07-04T17:26:22.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.808+0000 7f9694e00640 1 -- start start 2024-07-04T17:26:22.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.810+0000 7f9694e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96900b1d20 0x7f96900ab950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:22.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.810+0000 7f9694e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f96900b8b40 con 0x7f96900b1d20 2024-07-04T17:26:22.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.810+0000 7f968f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96900b1d20 0x7f96900ab950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:22.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.811+0000 7f968f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96900b1d20 0x7f96900ab950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34916/0 (socket says 172.21.15.42:34916) 2024-07-04T17:26:22.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.811+0000 7f968f400640 1 -- 172.21.15.42:0/553264624 learned_addr learned my addr 172.21.15.42:0/553264624 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:22.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.811+0000 7f968f400640 1 -- 172.21.15.42:0/553264624 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9690142ff0 con 0x7f96900b1d20 2024-07-04T17:26:22.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.811+0000 7f968f400640 1 --2- 172.21.15.42:0/553264624 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96900b1d20 0x7f96900ab950 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f968000c320 tx=0x7f968000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:22.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.812+0000 7f9687400640 1 -- 172.21.15.42:0/553264624 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9680017070 con 0x7f96900b1d20 2024-07-04T17:26:22.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.812+0000 7f9687400640 1 -- 172.21.15.42:0/553264624 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9680005310 con 0x7f96900b1d20 2024-07-04T17:26:22.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.812+0000 7f9687400640 1 -- 172.21.15.42:0/553264624 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96800035f0 con 0x7f96900b1d20 2024-07-04T17:26:22.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.812+0000 7f9694e00640 1 -- 172.21.15.42:0/553264624 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96900ad840 con 0x7f96900b1d20 2024-07-04T17:26:22.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.812+0000 7f9694e00640 1 -- 172.21.15.42:0/553264624 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96900ad2e0 con 0x7f96900b1d20 2024-07-04T17:26:22.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.813+0000 7f9687400640 1 -- 172.21.15.42:0/553264624 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f968001a070 con 0x7f96900b1d20 2024-07-04T17:26:22.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.813+0000 7f9694e00640 1 -- 172.21.15.42:0/553264624 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96901e2360 con 0x7f96900b1d20 2024-07-04T17:26:22.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.814+0000 7f9687400640 1 --2- 172.21.15.42:0/553264624 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f966c03d2b0 0x7f966c03f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:22.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.814+0000 7f9687400640 1 -- 172.21.15.42:0/553264624 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f9680050800 con 0x7f96900b1d20 2024-07-04T17:26:22.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.815+0000 7f9687e00640 1 --2- 172.21.15.42:0/553264624 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f966c03d2b0 0x7f966c03f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:22.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.815+0000 7f9687e00640 1 --2- 172.21.15.42:0/553264624 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f966c03d2b0 0x7f966c03f7c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9678033000 tx=0x7f9678004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:22.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:22.819+0000 7f9687400640 1 -- 172.21.15.42:0/553264624 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f96901e2360 con 0x7f96900b1d20 2024-07-04T17:26:23.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.031+0000 7f9694e00640 1 -- 172.21.15.42:0/553264624 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f96900b91d0 con 0x7f96900b1d20 2024-07-04T17:26:23.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.034+0000 7f9687400640 1 -- 172.21.15.42:0/553264624 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) ==== 74+0+10603 (secure 0 0 0) 0x7f969013ec10 con 0x7f96900b1d20 2024-07-04T17:26:23.036 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:26:23.036 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":15,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:26:17.008553+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:26:13.335715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-04T17:26:14.519899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:26:23.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.036+0000 7f9694e00640 1 -- 172.21.15.42:0/553264624 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f966c03d2b0 msgr2=0x7f966c03f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:23.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.036+0000 7f9694e00640 1 --2- 172.21.15.42:0/553264624 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f966c03d2b0 0x7f966c03f7c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9678033000 tx=0x7f9678004730 comp rx=0 tx=0).stop 2024-07-04T17:26:23.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.036+0000 7f9694e00640 1 -- 172.21.15.42:0/553264624 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96900b1d20 msgr2=0x7f96900ab950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:23.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.036+0000 7f9694e00640 1 --2- 172.21.15.42:0/553264624 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96900b1d20 0x7f96900ab950 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f968000c320 tx=0x7f968000c7e0 comp rx=0 tx=0).stop 2024-07-04T17:26:23.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.037+0000 7f9694e00640 1 -- 172.21.15.42:0/553264624 shutdown_connections 2024-07-04T17:26:23.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.037+0000 7f9694e00640 1 --2- 172.21.15.42:0/553264624 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f966c03d2b0 0x7f966c03f7c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:23.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.037+0000 7f9694e00640 1 --2- 172.21.15.42:0/553264624 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96900b1d20 0x7f96900ab950 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:23.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.037+0000 7f9694e00640 1 -- 172.21.15.42:0/553264624 >> 172.21.15.42:0/553264624 conn(0x7f96900bebc0 msgr2=0x7f969006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:23.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.037+0000 7f9694e00640 1 -- 172.21.15.42:0/553264624 shutdown_connections 2024-07-04T17:26:23.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.037+0000 7f9694e00640 1 -- 172.21.15.42:0/553264624 wait complete. 2024-07-04T17:26:23.065 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-07-04T17:26:23.065 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:26:23.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.346+0000 7ff87e800640 1 -- 172.21.15.42:0/4221151465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8780b38b0 msgr2=0x7ff8780b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:23.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.346+0000 7ff87e800640 1 --2- 172.21.15.42:0/4221151465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8780b38b0 0x7ff8780b2d20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff8680097b0 tx=0x7ff86802a540 comp rx=0 tx=0).stop 2024-07-04T17:26:23.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.346+0000 7ff87e800640 1 -- 172.21.15.42:0/4221151465 shutdown_connections 2024-07-04T17:26:23.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.346+0000 7ff87e800640 1 --2- 172.21.15.42:0/4221151465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8780b38b0 0x7ff8780b2d20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:23.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.346+0000 7ff87e800640 1 -- 172.21.15.42:0/4221151465 >> 172.21.15.42:0/4221151465 conn(0x7ff8780bf700 msgr2=0x7ff8780af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:23.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.346+0000 7ff87e800640 1 -- 172.21.15.42:0/4221151465 shutdown_connections 2024-07-04T17:26:23.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.347+0000 7ff87e800640 1 -- 172.21.15.42:0/4221151465 wait complete. 2024-07-04T17:26:23.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.349+0000 7ff87e800640 1 Processor -- start 2024-07-04T17:26:23.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.349+0000 7ff87e800640 1 -- start start 2024-07-04T17:26:23.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.350+0000 7ff87e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8780b1e00 0x7ff8781d4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:23.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.350+0000 7ff87e800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff8780b06e0 con 0x7ff8780b1e00 2024-07-04T17:26:23.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.351+0000 7ff877400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8780b1e00 0x7ff8781d4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:23.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.351+0000 7ff877400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8780b1e00 0x7ff8781d4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34942/0 (socket says 172.21.15.42:34942) 2024-07-04T17:26:23.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.351+0000 7ff877400640 1 -- 172.21.15.42:0/672536185 learned_addr learned my addr 172.21.15.42:0/672536185 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:23.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.351+0000 7ff877400640 1 -- 172.21.15.42:0/672536185 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8781d6830 con 0x7ff8780b1e00 2024-07-04T17:26:23.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.351+0000 7ff877400640 1 --2- 172.21.15.42:0/672536185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8780b1e00 0x7ff8781d4ae0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff8680098d0 tx=0x7ff868033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:23.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.352+0000 7ff86f400640 1 -- 172.21.15.42:0/672536185 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff868031bc0 con 0x7ff8780b1e00 2024-07-04T17:26:23.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.352+0000 7ff86f400640 1 -- 172.21.15.42:0/672536185 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff868031e50 con 0x7ff8780b1e00 2024-07-04T17:26:23.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.352+0000 7ff86f400640 1 -- 172.21.15.42:0/672536185 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff868033200 con 0x7ff8780b1e00 2024-07-04T17:26:23.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.352+0000 7ff87e800640 1 -- 172.21.15.42:0/672536185 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8781d6a80 con 0x7ff8780b1e00 2024-07-04T17:26:23.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.352+0000 7ff87e800640 1 -- 172.21.15.42:0/672536185 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8780aa930 con 0x7ff8780b1e00 2024-07-04T17:26:23.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.353+0000 7ff87e800640 1 -- 172.21.15.42:0/672536185 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff87806c590 con 0x7ff8780b1e00 2024-07-04T17:26:23.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.353+0000 7ff86f400640 1 -- 172.21.15.42:0/672536185 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7ff8680333a0 con 0x7ff8780b1e00 2024-07-04T17:26:23.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.354+0000 7ff86f400640 1 --2- 172.21.15.42:0/672536185 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff85803d2b0 0x7ff85803f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:23.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.354+0000 7ff86f400640 1 -- 172.21.15.42:0/672536185 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7ff868072ae0 con 0x7ff8780b1e00 2024-07-04T17:26:23.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.354+0000 7ff876a00640 1 --2- 172.21.15.42:0/672536185 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff85803d2b0 0x7ff85803f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:23.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.355+0000 7ff876a00640 1 --2- 172.21.15.42:0/672536185 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff85803d2b0 0x7ff85803f7c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff87806ed80 tx=0x7ff864007a90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:23.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.359+0000 7ff86f400640 1 -- 172.21.15.42:0/672536185 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7ff86803f030 con 0x7ff8780b1e00 2024-07-04T17:26:23.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.572+0000 7ff87e800640 1 -- 172.21.15.42:0/672536185 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff878140dd0 con 0x7ff8780b1e00 2024-07-04T17:26:23.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.574+0000 7ff86f400640 1 -- 172.21.15.42:0/672536185 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) ==== 74+0+10603 (secure 0 0 0) 0x7ff868003b10 con 0x7ff8780b1e00 2024-07-04T17:26:23.575 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:26:23.576 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":15,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:26:17.008553+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:26:13.335715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-04T17:26:14.519899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:26:23.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.577+0000 7ff87e800640 1 -- 172.21.15.42:0/672536185 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff85803d2b0 msgr2=0x7ff85803f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:23.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.577+0000 7ff87e800640 1 --2- 172.21.15.42:0/672536185 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff85803d2b0 0x7ff85803f7c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff87806ed80 tx=0x7ff864007a90 comp rx=0 tx=0).stop 2024-07-04T17:26:23.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.577+0000 7ff87e800640 1 -- 172.21.15.42:0/672536185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8780b1e00 msgr2=0x7ff8781d4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:23.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.577+0000 7ff87e800640 1 --2- 172.21.15.42:0/672536185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8780b1e00 0x7ff8781d4ae0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff8680098d0 tx=0x7ff868033e40 comp rx=0 tx=0).stop 2024-07-04T17:26:23.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.577+0000 7ff87e800640 1 -- 172.21.15.42:0/672536185 shutdown_connections 2024-07-04T17:26:23.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.577+0000 7ff87e800640 1 --2- 172.21.15.42:0/672536185 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff85803d2b0 0x7ff85803f7c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:23.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.577+0000 7ff87e800640 1 --2- 172.21.15.42:0/672536185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8780b1e00 0x7ff8781d4ae0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:23.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.577+0000 7ff87e800640 1 -- 172.21.15.42:0/672536185 >> 172.21.15.42:0/672536185 conn(0x7ff8780bf700 msgr2=0x7ff8781412f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:23.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.577+0000 7ff87e800640 1 -- 172.21.15.42:0/672536185 shutdown_connections 2024-07-04T17:26:23.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.577+0000 7ff87e800640 1 -- 172.21.15.42:0/672536185 wait complete. 2024-07-04T17:26:23.606 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-07-04T17:26:23.606 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-07-04T17:26:23.607 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-07-04T17:26:23.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.856+0000 7f96c1200640 1 -- 172.21.15.42:0/2046986028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96bc0b2ea0 msgr2=0x7f96bc0bdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:23.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.856+0000 7f96c1200640 1 --2- 172.21.15.42:0/2046986028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96bc0b2ea0 0x7f96bc0bdc60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f96ac0097b0 tx=0x7f96ac02a530 comp rx=0 tx=0).stop 2024-07-04T17:26:23.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.856+0000 7f96c1200640 1 -- 172.21.15.42:0/2046986028 shutdown_connections 2024-07-04T17:26:23.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.856+0000 7f96c1200640 1 --2- 172.21.15.42:0/2046986028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96bc0b2ea0 0x7f96bc0bdc60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:23.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.856+0000 7f96c1200640 1 -- 172.21.15.42:0/2046986028 >> 172.21.15.42:0/2046986028 conn(0x7f96bc06c020 msgr2=0x7f96bc06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:23.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.856+0000 7f96c1200640 1 -- 172.21.15.42:0/2046986028 shutdown_connections 2024-07-04T17:26:23.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.856+0000 7f96c1200640 1 -- 172.21.15.42:0/2046986028 wait complete. 2024-07-04T17:26:23.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.859+0000 7f96c1200640 1 Processor -- start 2024-07-04T17:26:23.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.859+0000 7f96c1200640 1 -- start start 2024-07-04T17:26:23.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.860+0000 7fa998200640 1 -- 172.21.15.42:0/3001712921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9900bdf70 msgr2=0x7fa9900bbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:23.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.860+0000 7fa998200640 1 --2- 172.21.15.42:0/3001712921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9900bdf70 0x7fa9900bbed0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa9840097b0 tx=0x7fa98402a530 comp rx=0 tx=0).stop 2024-07-04T17:26:23.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.860+0000 7f96c1200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96bc072150 0x7f96bc14da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:23.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.860+0000 7f96c1200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc14c040 con 0x7f96bc072150 2024-07-04T17:26:23.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.860+0000 7fa998200640 1 -- 172.21.15.42:0/3001712921 shutdown_connections 2024-07-04T17:26:23.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.860+0000 7fa998200640 1 --2- 172.21.15.42:0/3001712921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9900bdf70 0x7fa9900bbed0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:23.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.860+0000 7fa998200640 1 -- 172.21.15.42:0/3001712921 >> 172.21.15.42:0/3001712921 conn(0x7fa99006c9a0 msgr2=0x7fa99006de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:23.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.860+0000 7fa998200640 1 -- 172.21.15.42:0/3001712921 shutdown_connections 2024-07-04T17:26:23.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.861+0000 7f96b9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96bc072150 0x7f96bc14da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:23.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.861+0000 7fa998200640 1 -- 172.21.15.42:0/3001712921 wait complete. 2024-07-04T17:26:23.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.861+0000 7f96b9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96bc072150 0x7f96bc14da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43906/0 (socket says 172.21.15.42:43906) 2024-07-04T17:26:23.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.861+0000 7f96b9e00640 1 -- 172.21.15.42:0/1989501990 learned_addr learned my addr 172.21.15.42:0/1989501990 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:23.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.861+0000 7fd411800640 1 -- 172.21.15.42:0/1654938961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd40c0b2930 msgr2=0x7fd40c0b0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:23.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.861+0000 7fd411800640 1 --2- 172.21.15.42:0/1654938961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd40c0b2930 0x7fd40c0b0160 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc0097b0 tx=0x7fd3fc02a4a0 comp rx=0 tx=0).stop 2024-07-04T17:26:23.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.861+0000 7fd411800640 1 -- 172.21.15.42:0/1654938961 shutdown_connections 2024-07-04T17:26:23.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.861+0000 7fd411800640 1 --2- 172.21.15.42:0/1654938961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd40c0b2930 0x7fd40c0b0160 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:23.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.861+0000 7fd411800640 1 -- 172.21.15.42:0/1654938961 >> 172.21.15.42:0/1654938961 conn(0x7fd40c0bf4d0 msgr2=0x7fd40c0af870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:23.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.861+0000 7fd411800640 1 -- 172.21.15.42:0/1654938961 shutdown_connections 2024-07-04T17:26:23.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.861+0000 7fd411800640 1 -- 172.21.15.42:0/1654938961 wait complete. 2024-07-04T17:26:23.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.861+0000 7f96b9e00640 1 -- 172.21.15.42:0/1989501990 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96bc1d7200 con 0x7f96bc072150 2024-07-04T17:26:23.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.861+0000 7f96b9e00640 1 --2- 172.21.15.42:0/1989501990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96bc072150 0x7f96bc14da80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f96ac0098d0 tx=0x7f96ac033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:23.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.862+0000 7f96b2000640 1 -- 172.21.15.42:0/1989501990 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96ac031a90 con 0x7f96bc072150 2024-07-04T17:26:23.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.862+0000 7f96b2000640 1 -- 172.21.15.42:0/1989501990 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96ac031d20 con 0x7f96bc072150 2024-07-04T17:26:23.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.862+0000 7f96c1200640 1 -- 172.21.15.42:0/1989501990 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96bc14d510 con 0x7f96bc072150 2024-07-04T17:26:23.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.862+0000 7f96b2000640 1 -- 172.21.15.42:0/1989501990 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96ac033200 con 0x7f96bc072150 2024-07-04T17:26:23.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.862+0000 7f96c1200640 1 -- 172.21.15.42:0/1989501990 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96bc143a60 con 0x7f96bc072150 2024-07-04T17:26:23.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.863+0000 7fa998200640 1 Processor -- start 2024-07-04T17:26:23.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.863+0000 7fd411800640 1 Processor -- start 2024-07-04T17:26:23.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.864+0000 7fd411800640 1 -- start start 2024-07-04T17:26:23.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.863+0000 7f96c1200640 1 -- 172.21.15.42:0/1989501990 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f96bc0bdc60 con 0x7f96bc072150 2024-07-04T17:26:23.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.863+0000 7f96b2000640 1 -- 172.21.15.42:0/1989501990 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f96ac0333a0 con 0x7f96bc072150 2024-07-04T17:26:23.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.864+0000 7f96b2000640 1 --2- 172.21.15.42:0/1989501990 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f969c03d2b0 0x7f969c03f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:23.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.864+0000 7f96b2000640 1 -- 172.21.15.42:0/1989501990 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f96ac072930 con 0x7f96bc072150 2024-07-04T17:26:23.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.864+0000 7f96b2000640 1 --2- 172.21.15.42:0/1989501990 >> [v2:172.21.15.42:6800/584646941,v1:172.21.15.42:6801/584646941] conn(0x7f969c044220 0x7f969c0466b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:23.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.864+0000 7f96b2000640 1 -- 172.21.15.42:0/1989501990 --> [v2:172.21.15.42:6800/584646941,v1:172.21.15.42:6801/584646941] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f969c046d80 con 0x7f969c044220 2024-07-04T17:26:23.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.864+0000 7f96b2000640 1 -- 172.21.15.42:0/1989501990 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7f96ac0772a0 con 0x7f96bc072150 2024-07-04T17:26:23.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.864+0000 7f96ba800640 1 --2- 172.21.15.42:0/1989501990 >> [v2:172.21.15.42:6800/584646941,v1:172.21.15.42:6801/584646941] conn(0x7f969c044220 0x7f969c0466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:23.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.864+0000 7f96b9400640 1 --2- 172.21.15.42:0/1989501990 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f969c03d2b0 0x7f969c03f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:23.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.865+0000 7fa998200640 1 -- start start 2024-07-04T17:26:23.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.865+0000 7fd411800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd40c072250 0x7fd40c0aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:23.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.865+0000 7fd411800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c14bcd0 con 0x7fd40c072250 2024-07-04T17:26:23.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.865+0000 7fd40aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd40c072250 0x7fd40c0aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:23.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.865+0000 7fd40aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd40c072250 0x7fd40c0aeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43912/0 (socket says 172.21.15.42:43912) 2024-07-04T17:26:23.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.865+0000 7fd40aa00640 1 -- 172.21.15.42:0/705035826 learned_addr learned my addr 172.21.15.42:0/705035826 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:23.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.866+0000 7fd40aa00640 1 -- 172.21.15.42:0/705035826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd40c0b5280 con 0x7fd40c072250 2024-07-04T17:26:23.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.866+0000 7f96b9400640 1 --2- 172.21.15.42:0/1989501990 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f969c03d2b0 0x7f969c03f7c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f96a8007fd0 tx=0x7f96a8007920 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:23.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.866+0000 7fd40aa00640 1 --2- 172.21.15.42:0/705035826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd40c072250 0x7fd40c0aeb10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc0098d0 tx=0x7fd3fc033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:23.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.866+0000 7fd402a00640 1 -- 172.21.15.42:0/705035826 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3fc031bc0 con 0x7fd40c072250 2024-07-04T17:26:23.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.866+0000 7fd402a00640 1 -- 172.21.15.42:0/705035826 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3fc031e50 con 0x7fd40c072250 2024-07-04T17:26:23.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.866+0000 7fd402a00640 1 -- 172.21.15.42:0/705035826 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3fc033200 con 0x7fd40c072250 2024-07-04T17:26:23.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.867+0000 7fa998200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9900bc530 0x7fa9901e6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:23.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.867+0000 7fa998200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa990141760 con 0x7fa9900bc530 2024-07-04T17:26:23.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.867+0000 7f96ba800640 1 --2- 172.21.15.42:0/1989501990 >> [v2:172.21.15.42:6800/584646941,v1:172.21.15.42:6801/584646941] conn(0x7f969c044220 0x7f969c0466b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:23.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.867+0000 7fa995800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9900bc530 0x7fa9901e6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:23.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.867+0000 7fa995800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9900bc530 0x7fa9901e6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43916/0 (socket says 172.21.15.42:43916) 2024-07-04T17:26:23.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.867+0000 7fa995800640 1 -- 172.21.15.42:0/1641443607 learned_addr learned my addr 172.21.15.42:0/1641443607 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:23.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.867+0000 7fd411800640 1 -- 172.21.15.42:0/705035826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd40c0b6910 con 0x7fd40c072250 2024-07-04T17:26:23.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.867+0000 7fd411800640 1 -- 172.21.15.42:0/705035826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd40c0bb020 con 0x7fd40c072250 2024-07-04T17:26:23.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.868+0000 7fa995800640 1 -- 172.21.15.42:0/1641443607 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9901e71e0 con 0x7fa9900bc530 2024-07-04T17:26:23.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.868+0000 7f96b2000640 1 -- 172.21.15.42:0/1989501990 <== osd.0 v2:172.21.15.42:6800/584646941 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f969c046d80 con 0x7f969c044220 2024-07-04T17:26:23.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.868+0000 7fa995800640 1 --2- 172.21.15.42:0/1641443607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9900bc530 0x7fa9901e6be0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa9840098d0 tx=0x7fa984033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:23.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.869+0000 7fd411800640 1 -- 172.21.15.42:0/705035826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd40c0bb310 con 0x7fd40c072250 2024-07-04T17:26:23.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.869+0000 7fa98d600640 1 -- 172.21.15.42:0/1641443607 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa984031880 con 0x7fa9900bc530 2024-07-04T17:26:23.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.869+0000 7fa98d600640 1 -- 172.21.15.42:0/1641443607 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa984031b10 con 0x7fa9900bc530 2024-07-04T17:26:23.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.869+0000 7fa98d600640 1 -- 172.21.15.42:0/1641443607 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa984031df0 con 0x7fa9900bc530 2024-07-04T17:26:23.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.869+0000 7fa998200640 1 -- 172.21.15.42:0/1641443607 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9901376e0 con 0x7fa9900bc530 2024-07-04T17:26:23.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.869+0000 7fd402a00640 1 -- 172.21.15.42:0/705035826 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fd3fc0333a0 con 0x7fd40c072250 2024-07-04T17:26:23.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.869+0000 7fa998200640 1 -- 172.21.15.42:0/1641443607 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9900c3db0 con 0x7fa9900bc530 2024-07-04T17:26:23.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.870+0000 7fd402a00640 1 --2- 172.21.15.42:0/705035826 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fd3ec03d2b0 0x7fd3ec03f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:23.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.870+0000 7fd402a00640 1 -- 172.21.15.42:0/705035826 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7fd3fc072990 con 0x7fd40c072250 2024-07-04T17:26:23.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.870+0000 7fd402a00640 1 --2- 172.21.15.42:0/705035826 >> [v2:172.21.15.42:6808/2100781882,v1:172.21.15.42:6809/2100781882] conn(0x7fd3ec044140 0x7fd3ec0465d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:23.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.870+0000 7fd402a00640 1 -- 172.21.15.42:0/705035826 --> [v2:172.21.15.42:6808/2100781882,v1:172.21.15.42:6809/2100781882] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd3ec046ca0 con 0x7fd3ec044140 2024-07-04T17:26:23.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.870+0000 7fa998200640 1 -- 172.21.15.42:0/1641443607 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa9900c39d0 con 0x7fa9900bc530 2024-07-04T17:26:23.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.870+0000 7fd40a000640 1 --2- 172.21.15.42:0/705035826 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fd3ec03d2b0 0x7fd3ec03f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:23.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.870+0000 7fd40b400640 1 --2- 172.21.15.42:0/705035826 >> [v2:172.21.15.42:6808/2100781882,v1:172.21.15.42:6809/2100781882] conn(0x7fd3ec044140 0x7fd3ec0465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:23.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.871+0000 7fd402a00640 1 -- 172.21.15.42:0/705035826 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7fd3fc033690 con 0x7fd40c072250 2024-07-04T17:26:23.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.871+0000 7fd40b400640 1 --2- 172.21.15.42:0/705035826 >> [v2:172.21.15.42:6808/2100781882,v1:172.21.15.42:6809/2100781882] conn(0x7fd3ec044140 0x7fd3ec0465d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:23.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.871+0000 7fd40a000640 1 --2- 172.21.15.42:0/705035826 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fd3ec03d2b0 0x7fd3ec03f7c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd3f80015c0 tx=0x7fd3f8007a90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:23.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.872+0000 7fa98d600640 1 -- 172.21.15.42:0/1641443607 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fa984033070 con 0x7fa9900bc530 2024-07-04T17:26:23.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.872+0000 7fa98d600640 1 --2- 172.21.15.42:0/1641443607 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa97003d300 0x7fa97003f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:23.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.872+0000 7fa98d600640 1 -- 172.21.15.42:0/1641443607 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7fa984072e90 con 0x7fa9900bc530 2024-07-04T17:26:23.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.872+0000 7fd402a00640 1 -- 172.21.15.42:0/705035826 <== osd.2 v2:172.21.15.42:6808/2100781882 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd3ec046ca0 con 0x7fd3ec044140 2024-07-04T17:26:23.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.872+0000 7fa994e00640 1 --2- 172.21.15.42:0/1641443607 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa97003d300 0x7fa97003f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:23.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.872+0000 7fa98d600640 1 --2- 172.21.15.42:0/1641443607 >> [v2:172.21.15.42:6816/3110347191,v1:172.21.15.42:6817/3110347191] conn(0x7fa9700442f0 0x7fa970046780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:23.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.872+0000 7fa98d600640 1 -- 172.21.15.42:0/1641443607 --> [v2:172.21.15.42:6816/3110347191,v1:172.21.15.42:6817/3110347191] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa984050e50 con 0x7fa9700442f0 2024-07-04T17:26:23.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.872+0000 7fa98d600640 1 -- 172.21.15.42:0/1641443607 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7fa9840731c0 con 0x7fa9900bc530 2024-07-04T17:26:23.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.873+0000 7fa996200640 1 --2- 172.21.15.42:0/1641443607 >> [v2:172.21.15.42:6816/3110347191,v1:172.21.15.42:6817/3110347191] conn(0x7fa9700442f0 0x7fa970046780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:23.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.873+0000 7fa994e00640 1 --2- 172.21.15.42:0/1641443607 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa97003d300 0x7fa97003f810 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa97c007fd0 tx=0x7fa97c007920 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:23.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.873+0000 7fa996200640 1 --2- 172.21.15.42:0/1641443607 >> [v2:172.21.15.42:6816/3110347191,v1:172.21.15.42:6817/3110347191] conn(0x7fa9700442f0 0x7fa970046780 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:23.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.874+0000 7fa98d600640 1 -- 172.21.15.42:0/1641443607 <== osd.1 v2:172.21.15.42:6816/3110347191 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa984050e50 con 0x7fa9700442f0 2024-07-04T17:26:23.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.885+0000 7f96c1200640 1 -- 172.21.15.42:0/1989501990 --> [v2:172.21.15.42:6800/584646941,v1:172.21.15.42:6801/584646941] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f96bc0b30f0 con 0x7f969c044220 2024-07-04T17:26:23.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.886+0000 7f96b2000640 1 -- 172.21.15.42:0/1989501990 <== osd.0 v2:172.21.15.42:6800/584646941 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f96b400ad10 con 0x7f969c044220 2024-07-04T17:26:23.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.886+0000 7f96c1200640 1 -- 172.21.15.42:0/1989501990 >> [v2:172.21.15.42:6800/584646941,v1:172.21.15.42:6801/584646941] conn(0x7f969c044220 msgr2=0x7f969c0466b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:23.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.886+0000 7f96c1200640 1 --2- 172.21.15.42:0/1989501990 >> [v2:172.21.15.42:6800/584646941,v1:172.21.15.42:6801/584646941] conn(0x7f969c044220 0x7f969c0466b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:23.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.886+0000 7f96c1200640 1 -- 172.21.15.42:0/1989501990 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f969c03d2b0 msgr2=0x7f969c03f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:23.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.886+0000 7f96c1200640 1 --2- 172.21.15.42:0/1989501990 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f969c03d2b0 0x7f969c03f7c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f96a8007fd0 tx=0x7f96a8007920 comp rx=0 tx=0).stop 2024-07-04T17:26:23.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.886+0000 7f96c1200640 1 -- 172.21.15.42:0/1989501990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96bc072150 msgr2=0x7f96bc14da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:23.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.886+0000 7f96c1200640 1 --2- 172.21.15.42:0/1989501990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96bc072150 0x7f96bc14da80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f96ac0098d0 tx=0x7f96ac033c10 comp rx=0 tx=0).stop 2024-07-04T17:26:23.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.886+0000 7f96c1200640 1 -- 172.21.15.42:0/1989501990 shutdown_connections 2024-07-04T17:26:23.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.886+0000 7f96c1200640 1 --2- 172.21.15.42:0/1989501990 >> [v2:172.21.15.42:6800/584646941,v1:172.21.15.42:6801/584646941] conn(0x7f969c044220 0x7f969c0466b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:23.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.886+0000 7f96c1200640 1 --2- 172.21.15.42:0/1989501990 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f969c03d2b0 0x7f969c03f7c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:23.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.886+0000 7f96c1200640 1 --2- 172.21.15.42:0/1989501990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96bc072150 0x7f96bc14da80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:23.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.886+0000 7f96c1200640 1 -- 172.21.15.42:0/1989501990 >> 172.21.15.42:0/1989501990 conn(0x7f96bc06c020 msgr2=0x7f96bc0b14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:23.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.887+0000 7f96c1200640 1 -- 172.21.15.42:0/1989501990 shutdown_connections 2024-07-04T17:26:23.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.887+0000 7f96c1200640 1 -- 172.21.15.42:0/1989501990 wait complete. 2024-07-04T17:26:23.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.905+0000 7fd411800640 1 -- 172.21.15.42:0/705035826 --> [v2:172.21.15.42:6808/2100781882,v1:172.21.15.42:6809/2100781882] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd40c14c3d0 con 0x7fd3ec044140 2024-07-04T17:26:23.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.906+0000 7fd402a00640 1 -- 172.21.15.42:0/705035826 <== osd.2 v2:172.21.15.42:6808/2100781882 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd40c14c3d0 con 0x7fd3ec044140 2024-07-04T17:26:23.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.906+0000 7fd411800640 1 -- 172.21.15.42:0/705035826 >> [v2:172.21.15.42:6808/2100781882,v1:172.21.15.42:6809/2100781882] conn(0x7fd3ec044140 msgr2=0x7fd3ec0465d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:23.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.906+0000 7fd411800640 1 --2- 172.21.15.42:0/705035826 >> [v2:172.21.15.42:6808/2100781882,v1:172.21.15.42:6809/2100781882] conn(0x7fd3ec044140 0x7fd3ec0465d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:23.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.907+0000 7fd411800640 1 -- 172.21.15.42:0/705035826 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fd3ec03d2b0 msgr2=0x7fd3ec03f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:23.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.907+0000 7fd411800640 1 --2- 172.21.15.42:0/705035826 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fd3ec03d2b0 0x7fd3ec03f7c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd3f80015c0 tx=0x7fd3f8007a90 comp rx=0 tx=0).stop 2024-07-04T17:26:23.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.907+0000 7fd411800640 1 -- 172.21.15.42:0/705035826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd40c072250 msgr2=0x7fd40c0aeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:23.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.907+0000 7fd411800640 1 --2- 172.21.15.42:0/705035826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd40c072250 0x7fd40c0aeb10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc0098d0 tx=0x7fd3fc033e40 comp rx=0 tx=0).stop 2024-07-04T17:26:23.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.907+0000 7fd411800640 1 -- 172.21.15.42:0/705035826 shutdown_connections 2024-07-04T17:26:23.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.907+0000 7fd411800640 1 --2- 172.21.15.42:0/705035826 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fd3ec03d2b0 0x7fd3ec03f7c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:23.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.907+0000 7fd411800640 1 --2- 172.21.15.42:0/705035826 >> [v2:172.21.15.42:6808/2100781882,v1:172.21.15.42:6809/2100781882] conn(0x7fd3ec044140 0x7fd3ec0465d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:23.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.907+0000 7fd411800640 1 --2- 172.21.15.42:0/705035826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd40c072250 0x7fd40c0aeb10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:23.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.907+0000 7fd411800640 1 -- 172.21.15.42:0/705035826 >> 172.21.15.42:0/705035826 conn(0x7fd40c0bf4d0 msgr2=0x7fd40c0732a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:23.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.907+0000 7fd411800640 1 -- 172.21.15.42:0/705035826 shutdown_connections 2024-07-04T17:26:23.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.907+0000 7fd411800640 1 -- 172.21.15.42:0/705035826 wait complete. 2024-07-04T17:26:23.910 INFO:teuthology.orchestra.run.smithi042.stdout:34359738374 2024-07-04T17:26:23.910 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-07-04T17:26:23.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.908+0000 7fa998200640 1 -- 172.21.15.42:0/1641443607 --> [v2:172.21.15.42:6816/3110347191,v1:172.21.15.42:6817/3110347191] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa9900be1c0 con 0x7fa9700442f0 2024-07-04T17:26:23.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.908+0000 7fa98d600640 1 -- 172.21.15.42:0/1641443607 <== osd.1 v2:172.21.15.42:6816/3110347191 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa98800ad10 con 0x7fa9700442f0 2024-07-04T17:26:23.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.909+0000 7fa998200640 1 -- 172.21.15.42:0/1641443607 >> [v2:172.21.15.42:6816/3110347191,v1:172.21.15.42:6817/3110347191] conn(0x7fa9700442f0 msgr2=0x7fa970046780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:23.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.909+0000 7fa998200640 1 --2- 172.21.15.42:0/1641443607 >> [v2:172.21.15.42:6816/3110347191,v1:172.21.15.42:6817/3110347191] conn(0x7fa9700442f0 0x7fa970046780 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:23.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.909+0000 7fa998200640 1 -- 172.21.15.42:0/1641443607 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa97003d300 msgr2=0x7fa97003f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:23.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.909+0000 7fa998200640 1 --2- 172.21.15.42:0/1641443607 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa97003d300 0x7fa97003f810 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa97c007fd0 tx=0x7fa97c007920 comp rx=0 tx=0).stop 2024-07-04T17:26:23.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.909+0000 7fa998200640 1 -- 172.21.15.42:0/1641443607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9900bc530 msgr2=0x7fa9901e6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:23.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.909+0000 7fa998200640 1 --2- 172.21.15.42:0/1641443607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9900bc530 0x7fa9901e6be0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa9840098d0 tx=0x7fa984033ad0 comp rx=0 tx=0).stop 2024-07-04T17:26:23.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.909+0000 7fa998200640 1 -- 172.21.15.42:0/1641443607 shutdown_connections 2024-07-04T17:26:23.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.909+0000 7fa998200640 1 --2- 172.21.15.42:0/1641443607 >> [v2:172.21.15.42:6816/3110347191,v1:172.21.15.42:6817/3110347191] conn(0x7fa9700442f0 0x7fa970046780 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:23.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.909+0000 7fa998200640 1 --2- 172.21.15.42:0/1641443607 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa97003d300 0x7fa97003f810 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:23.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.909+0000 7fa998200640 1 --2- 172.21.15.42:0/1641443607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9900bc530 0x7fa9901e6be0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:23.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.909+0000 7fa998200640 1 -- 172.21.15.42:0/1641443607 >> 172.21.15.42:0/1641443607 conn(0x7fa99006c9a0 msgr2=0x7fa9900b6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:23.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.909+0000 7fa998200640 1 -- 172.21.15.42:0/1641443607 shutdown_connections 2024-07-04T17:26:23.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:23.909+0000 7fa998200640 1 -- 172.21.15.42:0/1641443607 wait complete. 2024-07-04T17:26:23.932 INFO:teuthology.orchestra.run.smithi042.stdout:34359738375 2024-07-04T17:26:23.932 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-07-04T17:26:23.933 INFO:teuthology.orchestra.run.smithi042.stdout:34359738376 2024-07-04T17:26:23.933 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-07-04T17:26:24.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.172+0000 7fb36a000640 1 -- 172.21.15.42:0/1029324073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3640b38d0 msgr2=0x7fb3640b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:24.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.172+0000 7fb36a000640 1 --2- 172.21.15.42:0/1029324073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3640b38d0 0x7fb3640b2d20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb3540097b0 tx=0x7fb35402a530 comp rx=0 tx=0).stop 2024-07-04T17:26:24.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.172+0000 7fb36a000640 1 -- 172.21.15.42:0/1029324073 shutdown_connections 2024-07-04T17:26:24.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.172+0000 7fb36a000640 1 --2- 172.21.15.42:0/1029324073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3640b38d0 0x7fb3640b2d20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:24.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.172+0000 7fb36a000640 1 -- 172.21.15.42:0/1029324073 >> 172.21.15.42:0/1029324073 conn(0x7fb3640bf700 msgr2=0x7fb3640af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:24.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.172+0000 7fb36a000640 1 -- 172.21.15.42:0/1029324073 shutdown_connections 2024-07-04T17:26:24.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.172+0000 7fb36a000640 1 -- 172.21.15.42:0/1029324073 wait complete. 2024-07-04T17:26:24.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.174+0000 7fb36a000640 1 Processor -- start 2024-07-04T17:26:24.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.174+0000 7fb36a000640 1 -- start start 2024-07-04T17:26:24.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.175+0000 7fb36a000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3640b01f0 0x7fb3641d7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:24.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.175+0000 7fb36a000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb36406cf50 con 0x7fb3640b01f0 2024-07-04T17:26:24.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.176+0000 7fb363400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3640b01f0 0x7fb3641d7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:24.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.176+0000 7fb363400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3640b01f0 0x7fb3641d7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43934/0 (socket says 172.21.15.42:43934) 2024-07-04T17:26:24.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.176+0000 7fb363400640 1 -- 172.21.15.42:0/571693833 learned_addr learned my addr 172.21.15.42:0/571693833 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:24.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.176+0000 7fb363400640 1 -- 172.21.15.42:0/571693833 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3640722a0 con 0x7fb3640b01f0 2024-07-04T17:26:24.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.176+0000 7fb363400640 1 --2- 172.21.15.42:0/571693833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3640b01f0 0x7fb3641d7320 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb3540098d0 tx=0x7fb354033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:24.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.177+0000 7fb35b400640 1 -- 172.21.15.42:0/571693833 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb354031880 con 0x7fb3640b01f0 2024-07-04T17:26:24.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.177+0000 7fb35b400640 1 -- 172.21.15.42:0/571693833 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb354031b10 con 0x7fb3640b01f0 2024-07-04T17:26:24.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.177+0000 7fb35b400640 1 -- 172.21.15.42:0/571693833 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb354031df0 con 0x7fb3640b01f0 2024-07-04T17:26:24.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.177+0000 7fb36a000640 1 -- 172.21.15.42:0/571693833 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3641e56b0 con 0x7fb3640b01f0 2024-07-04T17:26:24.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.177+0000 7fb36a000640 1 -- 172.21.15.42:0/571693833 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3640af4d0 con 0x7fb3640b01f0 2024-07-04T17:26:24.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.178+0000 7fb35b400640 1 -- 172.21.15.42:0/571693833 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fb354033070 con 0x7fb3640b01f0 2024-07-04T17:26:24.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.178+0000 7fb36a000640 1 -- 172.21.15.42:0/571693833 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3640008d0 con 0x7fb3640b01f0 2024-07-04T17:26:24.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.179+0000 7fb35b400640 1 --2- 172.21.15.42:0/571693833 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb34403d2b0 0x7fb34403f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:24.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.179+0000 7fb35b400640 1 -- 172.21.15.42:0/571693833 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7fb3540728a0 con 0x7fb3640b01f0 2024-07-04T17:26:24.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.179+0000 7fb362a00640 1 --2- 172.21.15.42:0/571693833 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb34403d2b0 0x7fb34403f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:24.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.180+0000 7fb362a00640 1 --2- 172.21.15.42:0/571693833 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb34403d2b0 0x7fb34403f7c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb350007fd0 tx=0x7fb350007920 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:24.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.188+0000 7fb35b400640 1 -- 172.21.15.42:0/571693833 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fb35403f030 con 0x7fb3640b01f0 2024-07-04T17:26:24.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.219+0000 7f8424200640 1 -- 172.21.15.42:0/2427968159 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f841c0aabc0 msgr2=0x7f841c0a9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:24.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.219+0000 7f8424200640 1 --2- 172.21.15.42:0/2427968159 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f841c0aabc0 0x7f841c0a9fa0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8410005fd0 tx=0x7f841002a5f0 comp rx=0 tx=0).stop 2024-07-04T17:26:24.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.219+0000 7f8424200640 1 -- 172.21.15.42:0/2427968159 shutdown_connections 2024-07-04T17:26:24.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.219+0000 7f8424200640 1 --2- 172.21.15.42:0/2427968159 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f841c0aabc0 0x7f841c0a9fa0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:24.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.219+0000 7f8424200640 1 -- 172.21.15.42:0/2427968159 >> 172.21.15.42:0/2427968159 conn(0x7f841c0bf700 msgr2=0x7f841c0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:24.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.219+0000 7f8424200640 1 -- 172.21.15.42:0/2427968159 shutdown_connections 2024-07-04T17:26:24.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.219+0000 7f8424200640 1 -- 172.21.15.42:0/2427968159 wait complete. 2024-07-04T17:26:24.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.222+0000 7f8424200640 1 Processor -- start 2024-07-04T17:26:24.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.222+0000 7f8424200640 1 -- start start 2024-07-04T17:26:24.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.222+0000 7f8caf600640 1 -- 172.21.15.42:0/2083258652 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ca80b37d0 msgr2=0x7f8ca80b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:24.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.222+0000 7f8caf600640 1 --2- 172.21.15.42:0/2083258652 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ca80b37d0 0x7f8ca80b01f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8c94005fd0 tx=0x7f8c9402a5f0 comp rx=0 tx=0).stop 2024-07-04T17:26:24.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.222+0000 7f8caf600640 1 -- 172.21.15.42:0/2083258652 shutdown_connections 2024-07-04T17:26:24.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.222+0000 7f8caf600640 1 --2- 172.21.15.42:0/2083258652 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ca80b37d0 0x7f8ca80b01f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:24.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.222+0000 7f8caf600640 1 -- 172.21.15.42:0/2083258652 >> 172.21.15.42:0/2083258652 conn(0x7f8ca80bf700 msgr2=0x7f8ca80af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:24.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.222+0000 7f8caf600640 1 -- 172.21.15.42:0/2083258652 shutdown_connections 2024-07-04T17:26:24.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.222+0000 7f8caf600640 1 -- 172.21.15.42:0/2083258652 wait complete. 2024-07-04T17:26:24.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.223+0000 7f8424200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f841c0b38f0 0x7f841c0aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:24.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.223+0000 7f8424200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f841c0ab180 con 0x7f841c0b38f0 2024-07-04T17:26:24.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.223+0000 7f8421800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f841c0b38f0 0x7f841c0aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:24.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.223+0000 7f8421800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f841c0b38f0 0x7f841c0aefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43952/0 (socket says 172.21.15.42:43952) 2024-07-04T17:26:24.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.223+0000 7f8421800640 1 -- 172.21.15.42:0/1704373379 learned_addr learned my addr 172.21.15.42:0/1704373379 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:24.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.224+0000 7f8421800640 1 -- 172.21.15.42:0/1704373379 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841c1dc800 con 0x7f841c0b38f0 2024-07-04T17:26:24.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.224+0000 7f8421800640 1 --2- 172.21.15.42:0/1704373379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f841c0b38f0 0x7f841c0aefc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8410034040 tx=0x7f84100049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:24.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.224+0000 7f8caf600640 1 Processor -- start 2024-07-04T17:26:24.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.225+0000 7f8419600640 1 -- 172.21.15.42:0/1704373379 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f841003e030 con 0x7f841c0b38f0 2024-07-04T17:26:24.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.225+0000 7f8caf600640 1 -- start start 2024-07-04T17:26:24.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.225+0000 7f8419600640 1 -- 172.21.15.42:0/1704373379 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84100035f0 con 0x7f841c0b38f0 2024-07-04T17:26:24.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.225+0000 7f8419600640 1 -- 172.21.15.42:0/1704373379 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8410035040 con 0x7f841c0b38f0 2024-07-04T17:26:24.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.225+0000 7f8424200640 1 -- 172.21.15.42:0/1704373379 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f841c1db1f0 con 0x7f841c0b38f0 2024-07-04T17:26:24.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.225+0000 7f8424200640 1 -- 172.21.15.42:0/1704373379 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f841c073ca0 con 0x7f841c0b38f0 2024-07-04T17:26:24.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.226+0000 7f8419600640 1 -- 172.21.15.42:0/1704373379 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f8410041030 con 0x7f841c0b38f0 2024-07-04T17:26:24.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.226+0000 7f8419600640 1 --2- 172.21.15.42:0/1704373379 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f83fc03d2b0 0x7f83fc03f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:24.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.227+0000 7f8caf600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ca80b2d20 0x7f8ca806ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:24.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.227+0000 7f8caf600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca806ce30 con 0x7f8ca80b2d20 2024-07-04T17:26:24.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.227+0000 7f8419600640 1 -- 172.21.15.42:0/1704373379 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f841003f070 con 0x7f841c0b38f0 2024-07-04T17:26:24.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.227+0000 7f8cacc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ca80b2d20 0x7f8ca806ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:24.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.227+0000 7f8cacc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ca80b2d20 0x7f8ca806ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43966/0 (socket says 172.21.15.42:43966) 2024-07-04T17:26:24.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.227+0000 7f8cacc00640 1 -- 172.21.15.42:0/3015061429 learned_addr learned my addr 172.21.15.42:0/3015061429 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:24.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.227+0000 7f8420e00640 1 --2- 172.21.15.42:0/1704373379 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f83fc03d2b0 0x7f83fc03f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:24.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.227+0000 7f8424200640 1 -- 172.21.15.42:0/1704373379 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f841c0aabc0 con 0x7f841c0b38f0 2024-07-04T17:26:24.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.227+0000 7f8cacc00640 1 -- 172.21.15.42:0/3015061429 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca80aa250 con 0x7f8ca80b2d20 2024-07-04T17:26:24.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.228+0000 7f8cacc00640 1 --2- 172.21.15.42:0/3015061429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ca80b2d20 0x7f8ca806ff70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c001eb0 tx=0x7f8c9c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:24.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.228+0000 7f8420e00640 1 --2- 172.21.15.42:0/1704373379 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f83fc03d2b0 0x7f83fc03f7c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8408001500 tx=0x7f8408007a30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:24.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.232+0000 7f8ca4c00640 1 -- 172.21.15.42:0/3015061429 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c9c017070 con 0x7f8ca80b2d20 2024-07-04T17:26:24.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.232+0000 7f8ca4c00640 1 -- 172.21.15.42:0/3015061429 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c9c0052f0 con 0x7f8ca80b2d20 2024-07-04T17:26:24.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.232+0000 7f8ca4c00640 1 -- 172.21.15.42:0/3015061429 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c9c0035f0 con 0x7f8ca80b2d20 2024-07-04T17:26:24.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.232+0000 7f8caf600640 1 -- 172.21.15.42:0/3015061429 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca80baa30 con 0x7f8ca80b2d20 2024-07-04T17:26:24.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.233+0000 7f8caf600640 1 -- 172.21.15.42:0/3015061429 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca80a9b20 con 0x7f8ca80b2d20 2024-07-04T17:26:24.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.233+0000 7f8419600640 1 -- 172.21.15.42:0/1704373379 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f8410003df0 con 0x7f841c0b38f0 2024-07-04T17:26:24.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.234+0000 7f8caf600640 1 -- 172.21.15.42:0/3015061429 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ca80704b0 con 0x7f8ca80b2d20 2024-07-04T17:26:24.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.234+0000 7f8ca4c00640 1 -- 172.21.15.42:0/3015061429 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f8c9c01a070 con 0x7f8ca80b2d20 2024-07-04T17:26:24.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.234+0000 7f8ca4c00640 1 --2- 172.21.15.42:0/3015061429 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f8c8803d300 0x7f8c8803f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:24.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.234+0000 7f8ca4c00640 1 -- 172.21.15.42:0/3015061429 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f8c9c050880 con 0x7f8ca80b2d20 2024-07-04T17:26:24.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.236+0000 7f8ca7e00640 1 --2- 172.21.15.42:0/3015061429 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f8c8803d300 0x7f8c8803f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:24.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.236+0000 7f8ca7e00640 1 --2- 172.21.15.42:0/3015061429 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f8c8803d300 0x7f8c8803f810 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8c94033000 tx=0x7f8c94004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:24.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.240+0000 7f8ca4c00640 1 -- 172.21.15.42:0/3015061429 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f8c9c003a20 con 0x7f8ca80b2d20 2024-07-04T17:26:24.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.394+0000 7fb36a000640 1 -- 172.21.15.42:0/571693833 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fb3640b2d20 con 0x7fb3640b01f0 2024-07-04T17:26:24.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.394+0000 7fb35b400640 1 -- 172.21.15.42:0/571693833 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb35403a3a0 con 0x7fb3640b01f0 2024-07-04T17:26:24.396 INFO:teuthology.orchestra.run.smithi042.stdout:34359738372 2024-07-04T17:26:24.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.397+0000 7fb36a000640 1 -- 172.21.15.42:0/571693833 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb34403d2b0 msgr2=0x7fb34403f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:24.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.397+0000 7fb36a000640 1 --2- 172.21.15.42:0/571693833 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb34403d2b0 0x7fb34403f7c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb350007fd0 tx=0x7fb350007920 comp rx=0 tx=0).stop 2024-07-04T17:26:24.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.397+0000 7fb36a000640 1 -- 172.21.15.42:0/571693833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3640b01f0 msgr2=0x7fb3641d7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:24.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.397+0000 7fb36a000640 1 --2- 172.21.15.42:0/571693833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3640b01f0 0x7fb3641d7320 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb3540098d0 tx=0x7fb354033ad0 comp rx=0 tx=0).stop 2024-07-04T17:26:24.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.397+0000 7fb36a000640 1 -- 172.21.15.42:0/571693833 shutdown_connections 2024-07-04T17:26:24.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.397+0000 7fb36a000640 1 --2- 172.21.15.42:0/571693833 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb34403d2b0 0x7fb34403f7c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:24.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.397+0000 7fb36a000640 1 --2- 172.21.15.42:0/571693833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3640b01f0 0x7fb3641d7320 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:24.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.397+0000 7fb36a000640 1 -- 172.21.15.42:0/571693833 >> 172.21.15.42:0/571693833 conn(0x7fb3640bf700 msgr2=0x7fb364141420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:24.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.397+0000 7fb36a000640 1 -- 172.21.15.42:0/571693833 shutdown_connections 2024-07-04T17:26:24.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.397+0000 7fb36a000640 1 -- 172.21.15.42:0/571693833 wait complete. 2024-07-04T17:26:24.421 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738374 got 34359738372 for osd.0 2024-07-04T17:26:24.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.438+0000 7f8424200640 1 -- 172.21.15.42:0/1704373379 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f841c0af510 con 0x7f841c0b38f0 2024-07-04T17:26:24.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.438+0000 7f8419600640 1 -- 172.21.15.42:0/1704373379 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f841004bd60 con 0x7f841c0b38f0 2024-07-04T17:26:24.440 INFO:teuthology.orchestra.run.smithi042.stdout:34359738374 2024-07-04T17:26:24.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.442+0000 7f8424200640 1 -- 172.21.15.42:0/1704373379 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f83fc03d2b0 msgr2=0x7f83fc03f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:24.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.442+0000 7f8424200640 1 --2- 172.21.15.42:0/1704373379 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f83fc03d2b0 0x7f83fc03f7c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8408001500 tx=0x7f8408007a30 comp rx=0 tx=0).stop 2024-07-04T17:26:24.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.442+0000 7f8424200640 1 -- 172.21.15.42:0/1704373379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f841c0b38f0 msgr2=0x7f841c0aefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:24.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.442+0000 7f8424200640 1 --2- 172.21.15.42:0/1704373379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f841c0b38f0 0x7f841c0aefc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8410034040 tx=0x7f84100049d0 comp rx=0 tx=0).stop 2024-07-04T17:26:24.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.442+0000 7f8424200640 1 -- 172.21.15.42:0/1704373379 shutdown_connections 2024-07-04T17:26:24.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.442+0000 7f8424200640 1 --2- 172.21.15.42:0/1704373379 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f83fc03d2b0 0x7f83fc03f7c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:24.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.442+0000 7f8424200640 1 --2- 172.21.15.42:0/1704373379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f841c0b38f0 0x7f841c0aefc0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:24.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.442+0000 7f8424200640 1 -- 172.21.15.42:0/1704373379 >> 172.21.15.42:0/1704373379 conn(0x7f841c0bf700 msgr2=0x7f841c0be7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:24.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.442+0000 7f8424200640 1 -- 172.21.15.42:0/1704373379 shutdown_connections 2024-07-04T17:26:24.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.442+0000 7f8424200640 1 -- 172.21.15.42:0/1704373379 wait complete. 2024-07-04T17:26:24.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.447+0000 7f8caf600640 1 -- 172.21.15.42:0/3015061429 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f8ca806d490 con 0x7f8ca80b2d20 2024-07-04T17:26:24.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.448+0000 7f8ca4c00640 1 -- 172.21.15.42:0/3015061429 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8c9c003ca0 con 0x7f8ca80b2d20 2024-07-04T17:26:24.449 INFO:teuthology.orchestra.run.smithi042.stdout:34359738373 2024-07-04T17:26:24.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.450+0000 7f8caf600640 1 -- 172.21.15.42:0/3015061429 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f8c8803d300 msgr2=0x7f8c8803f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:24.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.450+0000 7f8caf600640 1 --2- 172.21.15.42:0/3015061429 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f8c8803d300 0x7f8c8803f810 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8c94033000 tx=0x7f8c94004730 comp rx=0 tx=0).stop 2024-07-04T17:26:24.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.450+0000 7f8caf600640 1 -- 172.21.15.42:0/3015061429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ca80b2d20 msgr2=0x7f8ca806ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:24.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.450+0000 7f8caf600640 1 --2- 172.21.15.42:0/3015061429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ca80b2d20 0x7f8ca806ff70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c001eb0 tx=0x7f8c9c00c810 comp rx=0 tx=0).stop 2024-07-04T17:26:24.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.450+0000 7f8caf600640 1 -- 172.21.15.42:0/3015061429 shutdown_connections 2024-07-04T17:26:24.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.450+0000 7f8caf600640 1 --2- 172.21.15.42:0/3015061429 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f8c8803d300 0x7f8c8803f810 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:24.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.450+0000 7f8caf600640 1 --2- 172.21.15.42:0/3015061429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ca80b2d20 0x7f8ca806ff70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:24.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.450+0000 7f8caf600640 1 -- 172.21.15.42:0/3015061429 >> 172.21.15.42:0/3015061429 conn(0x7f8ca80bf700 msgr2=0x7f8ca8141760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:24.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.450+0000 7f8caf600640 1 -- 172.21.15.42:0/3015061429 shutdown_connections 2024-07-04T17:26:24.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:24.451+0000 7f8caf600640 1 -- 172.21.15.42:0/3015061429 wait complete. 2024-07-04T17:26:24.470 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738376 got 34359738374 for osd.1 2024-07-04T17:26:24.476 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738375 got 34359738373 for osd.2 2024-07-04T17:26:25.422 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-07-04T17:26:25.471 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-07-04T17:26:25.476 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-07-04T17:26:25.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.699+0000 7f9fd7e00640 1 -- 172.21.15.42:0/2724074944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fd014d550 msgr2=0x7f9fd00bd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:25.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.699+0000 7f9fd7e00640 1 --2- 172.21.15.42:0/2724074944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fd014d550 0x7f9fd00bd100 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9fc40097b0 tx=0x7f9fc402a540 comp rx=0 tx=0).stop 2024-07-04T17:26:25.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.699+0000 7f9fd7e00640 1 -- 172.21.15.42:0/2724074944 shutdown_connections 2024-07-04T17:26:25.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.699+0000 7f9fd7e00640 1 --2- 172.21.15.42:0/2724074944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fd014d550 0x7f9fd00bd100 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:25.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.699+0000 7f9fd7e00640 1 -- 172.21.15.42:0/2724074944 >> 172.21.15.42:0/2724074944 conn(0x7f9fd00c03e0 msgr2=0x7f9fd00c07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:25.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.699+0000 7f9fd7e00640 1 -- 172.21.15.42:0/2724074944 shutdown_connections 2024-07-04T17:26:25.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.699+0000 7f9fd7e00640 1 -- 172.21.15.42:0/2724074944 wait complete. 2024-07-04T17:26:25.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.702+0000 7f9fd7e00640 1 Processor -- start 2024-07-04T17:26:25.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.703+0000 7f9fd7e00640 1 -- start start 2024-07-04T17:26:25.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.705+0000 7f9fd7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fd00ad8e0 0x7f9fd01e6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:25.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.705+0000 7f9fd7e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd014df30 con 0x7f9fd00ad8e0 2024-07-04T17:26:25.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.705+0000 7f9fd5400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fd00ad8e0 0x7f9fd01e6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:25.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.705+0000 7f9fd5400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fd00ad8e0 0x7f9fd01e6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43996/0 (socket says 172.21.15.42:43996) 2024-07-04T17:26:25.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.705+0000 7f9fd5400640 1 -- 172.21.15.42:0/2552249689 learned_addr learned my addr 172.21.15.42:0/2552249689 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:25.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.705+0000 7f9fd5400640 1 -- 172.21.15.42:0/2552249689 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd01458b0 con 0x7f9fd00ad8e0 2024-07-04T17:26:25.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.706+0000 7f9fd5400640 1 --2- 172.21.15.42:0/2552249689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fd00ad8e0 0x7f9fd01e6860 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9fc40098d0 tx=0x7f9fc4033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:25.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.706+0000 7f9fcd600640 1 -- 172.21.15.42:0/2552249689 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fc4031880 con 0x7f9fd00ad8e0 2024-07-04T17:26:25.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.706+0000 7f9fcd600640 1 -- 172.21.15.42:0/2552249689 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fc4031b10 con 0x7f9fd00ad8e0 2024-07-04T17:26:25.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.706+0000 7f9fcd600640 1 -- 172.21.15.42:0/2552249689 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fc4031df0 con 0x7f9fd00ad8e0 2024-07-04T17:26:25.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.706+0000 7f9fd7e00640 1 -- 172.21.15.42:0/2552249689 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fd0142270 con 0x7f9fd00ad8e0 2024-07-04T17:26:25.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.706+0000 7f9fd7e00640 1 -- 172.21.15.42:0/2552249689 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fd0151240 con 0x7f9fd00ad8e0 2024-07-04T17:26:25.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.708+0000 7f9fcd600640 1 -- 172.21.15.42:0/2552249689 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f9fc4033070 con 0x7f9fd00ad8e0 2024-07-04T17:26:25.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.708+0000 7f9fd7e00640 1 -- 172.21.15.42:0/2552249689 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fd00bd100 con 0x7f9fd00ad8e0 2024-07-04T17:26:25.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.708+0000 7f9fcd600640 1 --2- 172.21.15.42:0/2552249689 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f9fb003d260 0x7f9fb003f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:25.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.709+0000 7f9fcd600640 1 -- 172.21.15.42:0/2552249689 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f9fc4072900 con 0x7f9fd00ad8e0 2024-07-04T17:26:25.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.709+0000 7f9fd4a00640 1 --2- 172.21.15.42:0/2552249689 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f9fb003d260 0x7f9fb003f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:25.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.710+0000 7f9fd4a00640 1 --2- 172.21.15.42:0/2552249689 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f9fb003d260 0x7f9fb003f770 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc007fd0 tx=0x7f9fbc007920 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:25.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.714+0000 7f9fcd600640 1 -- 172.21.15.42:0/2552249689 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f9fc403f030 con 0x7f9fd00ad8e0 2024-07-04T17:26:25.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.736+0000 7fa10a200640 1 -- 172.21.15.42:0/533880425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1040aabc0 msgr2=0x7fa1040a9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:25.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.736+0000 7fa10a200640 1 --2- 172.21.15.42:0/533880425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1040aabc0 0x7fa1040a9fa0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4005fd0 tx=0x7fa0f402a5f0 comp rx=0 tx=0).stop 2024-07-04T17:26:25.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.737+0000 7fa10a200640 1 -- 172.21.15.42:0/533880425 shutdown_connections 2024-07-04T17:26:25.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.737+0000 7fa10a200640 1 --2- 172.21.15.42:0/533880425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1040aabc0 0x7fa1040a9fa0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:25.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.737+0000 7fa10a200640 1 -- 172.21.15.42:0/533880425 >> 172.21.15.42:0/533880425 conn(0x7fa1040bf700 msgr2=0x7fa1040af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:25.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.737+0000 7fa10a200640 1 -- 172.21.15.42:0/533880425 shutdown_connections 2024-07-04T17:26:25.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.737+0000 7fa10a200640 1 -- 172.21.15.42:0/533880425 wait complete. 2024-07-04T17:26:25.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.739+0000 7fa10a200640 1 Processor -- start 2024-07-04T17:26:25.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.739+0000 7fa10a200640 1 -- start start 2024-07-04T17:26:25.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.740+0000 7fa10a200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1040b38f0 0x7fa1040aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:25.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.740+0000 7fa10a200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa1040ab180 con 0x7fa1040b38f0 2024-07-04T17:26:25.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.740+0000 7fa103400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1040b38f0 0x7fa1040aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:25.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.741+0000 7fa103400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1040b38f0 0x7fa1040aefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44016/0 (socket says 172.21.15.42:44016) 2024-07-04T17:26:25.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.741+0000 7fa103400640 1 -- 172.21.15.42:0/4272833303 learned_addr learned my addr 172.21.15.42:0/4272833303 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:25.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.741+0000 7fa103400640 1 -- 172.21.15.42:0/4272833303 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1041dc800 con 0x7fa1040b38f0 2024-07-04T17:26:25.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.741+0000 7fa103400640 1 --2- 172.21.15.42:0/4272833303 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1040b38f0 0x7fa1040aefc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4009850 tx=0x7fa0f40049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:25.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.742+0000 7fa0fb400640 1 -- 172.21.15.42:0/4272833303 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0f403e030 con 0x7fa1040b38f0 2024-07-04T17:26:25.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.742+0000 7fa0fb400640 1 -- 172.21.15.42:0/4272833303 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0f40035f0 con 0x7fa1040b38f0 2024-07-04T17:26:25.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.742+0000 7fa0fb400640 1 -- 172.21.15.42:0/4272833303 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0f4035040 con 0x7fa1040b38f0 2024-07-04T17:26:25.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.742+0000 7fa10a200640 1 -- 172.21.15.42:0/4272833303 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1041db1f0 con 0x7fa1040b38f0 2024-07-04T17:26:25.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.742+0000 7fa10a200640 1 -- 172.21.15.42:0/4272833303 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa104073ca0 con 0x7fa1040b38f0 2024-07-04T17:26:25.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.743+0000 7fa10a200640 1 -- 172.21.15.42:0/4272833303 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1040aabc0 con 0x7fa1040b38f0 2024-07-04T17:26:25.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.743+0000 7fa0fb400640 1 -- 172.21.15.42:0/4272833303 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fa0f4041030 con 0x7fa1040b38f0 2024-07-04T17:26:25.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.744+0000 7fa0fb400640 1 --2- 172.21.15.42:0/4272833303 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa0e403d2b0 0x7fa0e403f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:25.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.744+0000 7fa0fb400640 1 -- 172.21.15.42:0/4272833303 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7fa0f403f070 con 0x7fa1040b38f0 2024-07-04T17:26:25.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.744+0000 7fa102a00640 1 --2- 172.21.15.42:0/4272833303 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa0e403d2b0 0x7fa0e403f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:25.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.745+0000 7fa102a00640 1 --2- 172.21.15.42:0/4272833303 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa0e403d2b0 0x7fa0e403f7c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0001560 tx=0x7fa0f00079f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:25.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.749+0000 7fa0fb400640 1 -- 172.21.15.42:0/4272833303 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fa0f404bd20 con 0x7fa1040b38f0 2024-07-04T17:26:25.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.769+0000 7f1477400640 1 -- 172.21.15.42:0/2303952725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f147006cea0 msgr2=0x7f14700b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:25.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.769+0000 7f1477400640 1 --2- 172.21.15.42:0/2303952725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f147006cea0 0x7f14700b3030 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f145c00a8c0 tx=0x7f145c02b860 comp rx=0 tx=0).stop 2024-07-04T17:26:25.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.769+0000 7f1477400640 1 -- 172.21.15.42:0/2303952725 shutdown_connections 2024-07-04T17:26:25.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.769+0000 7f1477400640 1 --2- 172.21.15.42:0/2303952725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f147006cea0 0x7f14700b3030 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:25.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.769+0000 7f1477400640 1 -- 172.21.15.42:0/2303952725 >> 172.21.15.42:0/2303952725 conn(0x7f14700bf700 msgr2=0x7f14700af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:25.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.769+0000 7f1477400640 1 -- 172.21.15.42:0/2303952725 shutdown_connections 2024-07-04T17:26:25.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.769+0000 7f1477400640 1 -- 172.21.15.42:0/2303952725 wait complete. 2024-07-04T17:26:25.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.773+0000 7f1477400640 1 Processor -- start 2024-07-04T17:26:25.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.773+0000 7f1477400640 1 -- start start 2024-07-04T17:26:25.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.775+0000 7f1477400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f147006cea0 0x7f14700aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:25.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.775+0000 7f1477400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f14700b0370 con 0x7f147006cea0 2024-07-04T17:26:25.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.775+0000 7f1474a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f147006cea0 0x7f14700aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:25.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.775+0000 7f1474a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f147006cea0 0x7f14700aee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44034/0 (socket says 172.21.15.42:44034) 2024-07-04T17:26:25.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.775+0000 7f1474a00640 1 -- 172.21.15.42:0/1158989545 learned_addr learned my addr 172.21.15.42:0/1158989545 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:25.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.776+0000 7f1474a00640 1 -- 172.21.15.42:0/1158989545 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1470071d10 con 0x7f147006cea0 2024-07-04T17:26:25.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.776+0000 7f1474a00640 1 --2- 172.21.15.42:0/1158989545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f147006cea0 0x7f14700aee50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f146400c380 tx=0x7f146400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:25.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.776+0000 7f146cc00640 1 -- 172.21.15.42:0/1158989545 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14640116c0 con 0x7f147006cea0 2024-07-04T17:26:25.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.777+0000 7f146cc00640 1 -- 172.21.15.42:0/1158989545 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1464017070 con 0x7f147006cea0 2024-07-04T17:26:25.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.777+0000 7f146cc00640 1 -- 172.21.15.42:0/1158989545 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1464011b20 con 0x7f147006cea0 2024-07-04T17:26:25.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.777+0000 7f1477400640 1 -- 172.21.15.42:0/1158989545 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14701d50f0 con 0x7f147006cea0 2024-07-04T17:26:25.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.777+0000 7f1477400640 1 -- 172.21.15.42:0/1158989545 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14701da8b0 con 0x7f147006cea0 2024-07-04T17:26:25.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.778+0000 7f1477400640 1 -- 172.21.15.42:0/1158989545 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1470070bb0 con 0x7f147006cea0 2024-07-04T17:26:25.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.778+0000 7f146cc00640 1 -- 172.21.15.42:0/1158989545 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f1464011cc0 con 0x7f147006cea0 2024-07-04T17:26:25.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.779+0000 7f146cc00640 1 --2- 172.21.15.42:0/1158989545 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f145003d2b0 0x7f145003f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:25.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.779+0000 7f146cc00640 1 -- 172.21.15.42:0/1158989545 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f1464051370 con 0x7f147006cea0 2024-07-04T17:26:25.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.780+0000 7f146fe00640 1 --2- 172.21.15.42:0/1158989545 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f145003d2b0 0x7f145003f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:25.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.781+0000 7f146fe00640 1 --2- 172.21.15.42:0/1158989545 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f145003d2b0 0x7f145003f7c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f145c036000 tx=0x7f145c004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:25.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.785+0000 7f146cc00640 1 -- 172.21.15.42:0/1158989545 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f146400a140 con 0x7f147006cea0 2024-07-04T17:26:25.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.927+0000 7f9fd7e00640 1 -- 172.21.15.42:0/2552249689 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f9fd0145ad0 con 0x7f9fd00ad8e0 2024-07-04T17:26:25.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.928+0000 7f9fcd600640 1 -- 172.21.15.42:0/2552249689 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9fc403a3a0 con 0x7f9fd00ad8e0 2024-07-04T17:26:25.930 INFO:teuthology.orchestra.run.smithi042.stdout:34359738372 2024-07-04T17:26:25.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.930+0000 7f9fd7e00640 1 -- 172.21.15.42:0/2552249689 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f9fb003d260 msgr2=0x7f9fb003f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:25.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.930+0000 7f9fd7e00640 1 --2- 172.21.15.42:0/2552249689 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f9fb003d260 0x7f9fb003f770 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc007fd0 tx=0x7f9fbc007920 comp rx=0 tx=0).stop 2024-07-04T17:26:25.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.931+0000 7f9fd7e00640 1 -- 172.21.15.42:0/2552249689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fd00ad8e0 msgr2=0x7f9fd01e6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:25.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.931+0000 7f9fd7e00640 1 --2- 172.21.15.42:0/2552249689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fd00ad8e0 0x7f9fd01e6860 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9fc40098d0 tx=0x7f9fc4033ad0 comp rx=0 tx=0).stop 2024-07-04T17:26:25.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.931+0000 7f9fd7e00640 1 -- 172.21.15.42:0/2552249689 shutdown_connections 2024-07-04T17:26:25.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.931+0000 7f9fd7e00640 1 --2- 172.21.15.42:0/2552249689 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f9fb003d260 0x7f9fb003f770 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:25.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.931+0000 7f9fd7e00640 1 --2- 172.21.15.42:0/2552249689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fd00ad8e0 0x7f9fd01e6860 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:25.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.931+0000 7f9fd7e00640 1 -- 172.21.15.42:0/2552249689 >> 172.21.15.42:0/2552249689 conn(0x7f9fd00c03e0 msgr2=0x7f9fd00b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:25.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.931+0000 7f9fd7e00640 1 -- 172.21.15.42:0/2552249689 shutdown_connections 2024-07-04T17:26:25.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.931+0000 7f9fd7e00640 1 -- 172.21.15.42:0/2552249689 wait complete. 2024-07-04T17:26:25.955 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738374 got 34359738372 for osd.0 2024-07-04T17:26:25.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.963+0000 7fa10a200640 1 -- 172.21.15.42:0/4272833303 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fa1040af510 con 0x7fa1040b38f0 2024-07-04T17:26:25.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.963+0000 7fa0fb400640 1 -- 172.21.15.42:0/4272833303 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa0f4030c90 con 0x7fa1040b38f0 2024-07-04T17:26:25.965 INFO:teuthology.orchestra.run.smithi042.stdout:34359738374 2024-07-04T17:26:25.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.966+0000 7fa10a200640 1 -- 172.21.15.42:0/4272833303 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa0e403d2b0 msgr2=0x7fa0e403f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:25.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.966+0000 7fa10a200640 1 --2- 172.21.15.42:0/4272833303 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa0e403d2b0 0x7fa0e403f7c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0001560 tx=0x7fa0f00079f0 comp rx=0 tx=0).stop 2024-07-04T17:26:25.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.966+0000 7fa10a200640 1 -- 172.21.15.42:0/4272833303 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1040b38f0 msgr2=0x7fa1040aefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:25.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.966+0000 7fa10a200640 1 --2- 172.21.15.42:0/4272833303 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1040b38f0 0x7fa1040aefc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4009850 tx=0x7fa0f40049d0 comp rx=0 tx=0).stop 2024-07-04T17:26:25.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.966+0000 7fa10a200640 1 -- 172.21.15.42:0/4272833303 shutdown_connections 2024-07-04T17:26:25.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.966+0000 7fa10a200640 1 --2- 172.21.15.42:0/4272833303 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa0e403d2b0 0x7fa0e403f7c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:25.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.966+0000 7fa10a200640 1 --2- 172.21.15.42:0/4272833303 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1040b38f0 0x7fa1040aefc0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:25.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.966+0000 7fa10a200640 1 -- 172.21.15.42:0/4272833303 >> 172.21.15.42:0/4272833303 conn(0x7fa1040bf700 msgr2=0x7fa1040be7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:25.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.966+0000 7fa10a200640 1 -- 172.21.15.42:0/4272833303 shutdown_connections 2024-07-04T17:26:25.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.966+0000 7fa10a200640 1 -- 172.21.15.42:0/4272833303 wait complete. 2024-07-04T17:26:25.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.989+0000 7f1477400640 1 -- 172.21.15.42:0/1158989545 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f14700b3030 con 0x7f147006cea0 2024-07-04T17:26:25.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.989+0000 7f146cc00640 1 -- 172.21.15.42:0/1158989545 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1464010cf0 con 0x7f147006cea0 2024-07-04T17:26:25.991 INFO:teuthology.orchestra.run.smithi042.stdout:34359738373 2024-07-04T17:26:25.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.992+0000 7f1477400640 1 -- 172.21.15.42:0/1158989545 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f145003d2b0 msgr2=0x7f145003f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:25.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.992+0000 7f1477400640 1 --2- 172.21.15.42:0/1158989545 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f145003d2b0 0x7f145003f7c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f145c036000 tx=0x7f145c004730 comp rx=0 tx=0).stop 2024-07-04T17:26:25.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.992+0000 7f1477400640 1 -- 172.21.15.42:0/1158989545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f147006cea0 msgr2=0x7f14700aee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:25.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.992+0000 7f1477400640 1 --2- 172.21.15.42:0/1158989545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f147006cea0 0x7f14700aee50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f146400c380 tx=0x7f146400c840 comp rx=0 tx=0).stop 2024-07-04T17:26:25.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.992+0000 7f1477400640 1 -- 172.21.15.42:0/1158989545 shutdown_connections 2024-07-04T17:26:25.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.992+0000 7f1477400640 1 --2- 172.21.15.42:0/1158989545 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f145003d2b0 0x7f145003f7c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:25.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.992+0000 7f1477400640 1 --2- 172.21.15.42:0/1158989545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f147006cea0 0x7f14700aee50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:25.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.992+0000 7f1477400640 1 -- 172.21.15.42:0/1158989545 >> 172.21.15.42:0/1158989545 conn(0x7f14700bf700 msgr2=0x7f14701492f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:25.995 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738376 got 34359738374 for osd.1 2024-07-04T17:26:25.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.992+0000 7f1477400640 1 -- 172.21.15.42:0/1158989545 shutdown_connections 2024-07-04T17:26:25.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:25.992+0000 7f1477400640 1 -- 172.21.15.42:0/1158989545 wait complete. 2024-07-04T17:26:26.016 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738375 got 34359738373 for osd.2 2024-07-04T17:26:26.955 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-07-04T17:26:26.995 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-07-04T17:26:27.016 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-07-04T17:26:27.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.238+0000 7fedad000640 1 -- 172.21.15.42:0/3503011330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feda80b38d0 msgr2=0x7feda80b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:27.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.238+0000 7fedad000640 1 --2- 172.21.15.42:0/3503011330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feda80b38d0 0x7feda80b2d20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fed980097b0 tx=0x7fed9802a540 comp rx=0 tx=0).stop 2024-07-04T17:26:27.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.238+0000 7fedad000640 1 -- 172.21.15.42:0/3503011330 shutdown_connections 2024-07-04T17:26:27.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.238+0000 7fedad000640 1 --2- 172.21.15.42:0/3503011330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feda80b38d0 0x7feda80b2d20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:27.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.238+0000 7fedad000640 1 -- 172.21.15.42:0/3503011330 >> 172.21.15.42:0/3503011330 conn(0x7feda80bf700 msgr2=0x7feda80af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:27.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.238+0000 7fedad000640 1 -- 172.21.15.42:0/3503011330 shutdown_connections 2024-07-04T17:26:27.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.239+0000 7fedad000640 1 -- 172.21.15.42:0/3503011330 wait complete. 2024-07-04T17:26:27.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.242+0000 7fedad000640 1 Processor -- start 2024-07-04T17:26:27.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.242+0000 7fedad000640 1 -- start start 2024-07-04T17:26:27.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.244+0000 7fedad000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feda80b01f0 0x7feda80aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:27.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.244+0000 7fedad000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7feda806cf50 con 0x7feda80b01f0 2024-07-04T17:26:27.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.244+0000 7feda5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feda80b01f0 0x7feda80aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:27.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.244+0000 7feda5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feda80b01f0 0x7feda80aeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44060/0 (socket says 172.21.15.42:44060) 2024-07-04T17:26:27.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.244+0000 7feda5e00640 1 -- 172.21.15.42:0/3415010107 learned_addr learned my addr 172.21.15.42:0/3415010107 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:27.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.245+0000 7feda5e00640 1 -- 172.21.15.42:0/3415010107 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda80af040 con 0x7feda80b01f0 2024-07-04T17:26:27.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.245+0000 7feda5e00640 1 --2- 172.21.15.42:0/3415010107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feda80b01f0 0x7feda80aeb00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fed980098d0 tx=0x7fed98033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:27.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.245+0000 7fed9e000640 1 -- 172.21.15.42:0/3415010107 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed98031bc0 con 0x7feda80b01f0 2024-07-04T17:26:27.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.246+0000 7fed9e000640 1 -- 172.21.15.42:0/3415010107 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed98031e50 con 0x7feda80b01f0 2024-07-04T17:26:27.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.246+0000 7fed9e000640 1 -- 172.21.15.42:0/3415010107 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed98033200 con 0x7feda80b01f0 2024-07-04T17:26:27.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.246+0000 7fedad000640 1 -- 172.21.15.42:0/3415010107 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda80aa060 con 0x7feda80b01f0 2024-07-04T17:26:27.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.246+0000 7fedad000640 1 -- 172.21.15.42:0/3415010107 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda8137170 con 0x7feda80b01f0 2024-07-04T17:26:27.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.247+0000 7fedad000640 1 -- 172.21.15.42:0/3415010107 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feda80b2d20 con 0x7feda80b01f0 2024-07-04T17:26:27.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.247+0000 7fed9e000640 1 -- 172.21.15.42:0/3415010107 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fed980333a0 con 0x7feda80b01f0 2024-07-04T17:26:27.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.248+0000 7fed9e000640 1 --2- 172.21.15.42:0/3415010107 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fed8403d2b0 0x7fed8403f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:27.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.248+0000 7fed9e000640 1 -- 172.21.15.42:0/3415010107 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7fed980729c0 con 0x7feda80b01f0 2024-07-04T17:26:27.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.248+0000 7fed9fe00640 1 --2- 172.21.15.42:0/3415010107 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fed8403d2b0 0x7fed8403f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:27.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.249+0000 7fed9fe00640 1 --2- 172.21.15.42:0/3415010107 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fed8403d2b0 0x7fed8403f7c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fed900015c0 tx=0x7fed90007a90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:27.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.253+0000 7fed9e000640 1 -- 172.21.15.42:0/3415010107 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fed9803f030 con 0x7feda80b01f0 2024-07-04T17:26:27.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.269+0000 7f4dd1200640 1 -- 172.21.15.42:0/257988902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4dcc06cea0 msgr2=0x7f4dcc0b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:27.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.269+0000 7f4dd1200640 1 --2- 172.21.15.42:0/257988902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4dcc06cea0 0x7f4dcc0b3030 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4db8005fd0 tx=0x7f4db802b930 comp rx=0 tx=0).stop 2024-07-04T17:26:27.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.269+0000 7f4dd1200640 1 -- 172.21.15.42:0/257988902 shutdown_connections 2024-07-04T17:26:27.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.269+0000 7f4dd1200640 1 --2- 172.21.15.42:0/257988902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4dcc06cea0 0x7f4dcc0b3030 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:27.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.269+0000 7f4dd1200640 1 -- 172.21.15.42:0/257988902 >> 172.21.15.42:0/257988902 conn(0x7f4dcc0bf700 msgr2=0x7f4dcc0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:27.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.269+0000 7f4dd1200640 1 -- 172.21.15.42:0/257988902 shutdown_connections 2024-07-04T17:26:27.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.269+0000 7f4dd1200640 1 -- 172.21.15.42:0/257988902 wait complete. 2024-07-04T17:26:27.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.271+0000 7f4dd1200640 1 Processor -- start 2024-07-04T17:26:27.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.271+0000 7f4dd1200640 1 -- start start 2024-07-04T17:26:27.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.272+0000 7f4dd1200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4dcc06cea0 0x7f4dcc0aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:27.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.272+0000 7f4dd1200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc0b0370 con 0x7f4dcc06cea0 2024-07-04T17:26:27.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.273+0000 7f4dc9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4dcc06cea0 0x7f4dcc0aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:27.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.273+0000 7f4dc9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4dcc06cea0 0x7f4dcc0aeba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44078/0 (socket says 172.21.15.42:44078) 2024-07-04T17:26:27.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.273+0000 7f4dc9e00640 1 -- 172.21.15.42:0/1327155543 learned_addr learned my addr 172.21.15.42:0/1327155543 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:27.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.273+0000 7f4dc9e00640 1 -- 172.21.15.42:0/1327155543 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dcc0ac410 con 0x7f4dcc06cea0 2024-07-04T17:26:27.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.274+0000 7f4dc9e00640 1 --2- 172.21.15.42:0/1327155543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4dcc06cea0 0x7f4dcc0aeba0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc00c380 tx=0x7f4dbc00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:27.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.274+0000 7f4dc2000640 1 -- 172.21.15.42:0/1327155543 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dbc017070 con 0x7f4dcc06cea0 2024-07-04T17:26:27.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.274+0000 7f4dc2000640 1 -- 172.21.15.42:0/1327155543 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4dbc0052f0 con 0x7f4dcc06cea0 2024-07-04T17:26:27.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.274+0000 7f4dc2000640 1 -- 172.21.15.42:0/1327155543 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dbc0035f0 con 0x7f4dcc06cea0 2024-07-04T17:26:27.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.274+0000 7f4dd1200640 1 -- 172.21.15.42:0/1327155543 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dcc1d4350 con 0x7f4dcc06cea0 2024-07-04T17:26:27.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.274+0000 7f4dd1200640 1 -- 172.21.15.42:0/1327155543 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dcc0babd0 con 0x7f4dcc06cea0 2024-07-04T17:26:27.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.275+0000 7f4dd1200640 1 -- 172.21.15.42:0/1327155543 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dcc0b3030 con 0x7f4dcc06cea0 2024-07-04T17:26:27.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.276+0000 7f4dc2000640 1 -- 172.21.15.42:0/1327155543 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f4dbc01a070 con 0x7f4dcc06cea0 2024-07-04T17:26:27.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.276+0000 7f4dc2000640 1 --2- 172.21.15.42:0/1327155543 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f4dac03d2b0 0x7f4dac03f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:27.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.276+0000 7f4dc2000640 1 -- 172.21.15.42:0/1327155543 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f4dbc0508f0 con 0x7f4dcc06cea0 2024-07-04T17:26:27.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.277+0000 7f4dc9400640 1 --2- 172.21.15.42:0/1327155543 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f4dac03d2b0 0x7f4dac03f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:27.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.277+0000 7f4dc9400640 1 --2- 172.21.15.42:0/1327155543 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f4dac03d2b0 0x7f4dac03f7c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4db8035000 tx=0x7f4db8004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:27.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.282+0000 7f4dc2000640 1 -- 172.21.15.42:0/1327155543 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f4dbc011630 con 0x7f4dcc06cea0 2024-07-04T17:26:27.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.300+0000 7f4ff9000640 1 -- 172.21.15.42:0/1635177954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ff40adaa0 msgr2=0x7f4ff406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:27.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.300+0000 7f4ff9000640 1 --2- 172.21.15.42:0/1635177954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ff40adaa0 0x7f4ff406d110 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4fe40097b0 tx=0x7f4fe402a540 comp rx=0 tx=0).stop 2024-07-04T17:26:27.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.300+0000 7f4ff9000640 1 -- 172.21.15.42:0/1635177954 shutdown_connections 2024-07-04T17:26:27.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.300+0000 7f4ff9000640 1 --2- 172.21.15.42:0/1635177954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ff40adaa0 0x7f4ff406d110 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:27.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.300+0000 7f4ff9000640 1 -- 172.21.15.42:0/1635177954 >> 172.21.15.42:0/1635177954 conn(0x7f4ff406c020 msgr2=0x7f4ff406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:27.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.300+0000 7f4ff9000640 1 -- 172.21.15.42:0/1635177954 shutdown_connections 2024-07-04T17:26:27.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.300+0000 7f4ff9000640 1 -- 172.21.15.42:0/1635177954 wait complete. 2024-07-04T17:26:27.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.302+0000 7f4ff9000640 1 Processor -- start 2024-07-04T17:26:27.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.302+0000 7f4ff9000640 1 -- start start 2024-07-04T17:26:27.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.303+0000 7f4ff9000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ff40adaa0 0x7f4ff413ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:27.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.303+0000 7f4ff9000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff41413d0 con 0x7f4ff40adaa0 2024-07-04T17:26:27.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.304+0000 7f4ff3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ff40adaa0 0x7f4ff413ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:27.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.304+0000 7f4ff3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ff40adaa0 0x7f4ff413ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44104/0 (socket says 172.21.15.42:44104) 2024-07-04T17:26:27.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.304+0000 7f4ff3400640 1 -- 172.21.15.42:0/3903851445 learned_addr learned my addr 172.21.15.42:0/3903851445 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:27.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.304+0000 7f4ff3400640 1 -- 172.21.15.42:0/3903851445 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff4073cf0 con 0x7f4ff40adaa0 2024-07-04T17:26:27.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.305+0000 7f4ff3400640 1 --2- 172.21.15.42:0/3903851445 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ff40adaa0 0x7f4ff413ded0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4fe402ab30 tx=0x7f4fe4033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:27.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.305+0000 7f4feb400640 1 -- 172.21.15.42:0/3903851445 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fe4031bc0 con 0x7f4ff40adaa0 2024-07-04T17:26:27.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.305+0000 7f4feb400640 1 -- 172.21.15.42:0/3903851445 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4fe4031e50 con 0x7f4ff40adaa0 2024-07-04T17:26:27.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.305+0000 7f4ff9000640 1 -- 172.21.15.42:0/3903851445 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ff413e6c0 con 0x7f4ff40adaa0 2024-07-04T17:26:27.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.305+0000 7f4feb400640 1 -- 172.21.15.42:0/3903851445 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fe4033200 con 0x7f4ff40adaa0 2024-07-04T17:26:27.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.305+0000 7f4ff9000640 1 -- 172.21.15.42:0/3903851445 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ff4152eb0 con 0x7f4ff40adaa0 2024-07-04T17:26:27.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.306+0000 7f4ff9000640 1 -- 172.21.15.42:0/3903851445 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ff4142c60 con 0x7f4ff40adaa0 2024-07-04T17:26:27.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.307+0000 7f4feb400640 1 -- 172.21.15.42:0/3903851445 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f4fe40333a0 con 0x7f4ff40adaa0 2024-07-04T17:26:27.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.307+0000 7f4feb400640 1 --2- 172.21.15.42:0/3903851445 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f4fd403d2b0 0x7f4fd403f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:27.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.307+0000 7f4feb400640 1 -- 172.21.15.42:0/3903851445 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f4fe4072aa0 con 0x7f4ff40adaa0 2024-07-04T17:26:27.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.308+0000 7f4ff2a00640 1 --2- 172.21.15.42:0/3903851445 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f4fd403d2b0 0x7f4fd403f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:27.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.308+0000 7f4ff2a00640 1 --2- 172.21.15.42:0/3903851445 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f4fd403d2b0 0x7f4fd403f7c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4147530 tx=0x7f4fe0007a90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:27.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.317+0000 7f4feb400640 1 -- 172.21.15.42:0/3903851445 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f4fe403f030 con 0x7f4ff40adaa0 2024-07-04T17:26:27.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.459+0000 7fedad000640 1 -- 172.21.15.42:0/3415010107 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7feda80ac8d0 con 0x7feda80b01f0 2024-07-04T17:26:27.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.460+0000 7fed9e000640 1 -- 172.21.15.42:0/3415010107 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fed98033690 con 0x7feda80b01f0 2024-07-04T17:26:27.462 INFO:teuthology.orchestra.run.smithi042.stdout:34359738374 2024-07-04T17:26:27.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.462+0000 7fedad000640 1 -- 172.21.15.42:0/3415010107 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fed8403d2b0 msgr2=0x7fed8403f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:27.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.462+0000 7fedad000640 1 --2- 172.21.15.42:0/3415010107 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fed8403d2b0 0x7fed8403f7c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fed900015c0 tx=0x7fed90007a90 comp rx=0 tx=0).stop 2024-07-04T17:26:27.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.462+0000 7fedad000640 1 -- 172.21.15.42:0/3415010107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feda80b01f0 msgr2=0x7feda80aeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:27.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.462+0000 7fedad000640 1 --2- 172.21.15.42:0/3415010107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feda80b01f0 0x7feda80aeb00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fed980098d0 tx=0x7fed98033e40 comp rx=0 tx=0).stop 2024-07-04T17:26:27.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.463+0000 7fedad000640 1 -- 172.21.15.42:0/3415010107 shutdown_connections 2024-07-04T17:26:27.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.463+0000 7fedad000640 1 --2- 172.21.15.42:0/3415010107 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fed8403d2b0 0x7fed8403f7c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:27.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.463+0000 7fedad000640 1 --2- 172.21.15.42:0/3415010107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feda80b01f0 0x7feda80aeb00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:27.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.463+0000 7fedad000640 1 -- 172.21.15.42:0/3415010107 >> 172.21.15.42:0/3415010107 conn(0x7feda80bf700 msgr2=0x7feda813f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:27.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.463+0000 7fedad000640 1 -- 172.21.15.42:0/3415010107 shutdown_connections 2024-07-04T17:26:27.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.463+0000 7fedad000640 1 -- 172.21.15.42:0/3415010107 wait complete. 2024-07-04T17:26:27.488 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738374 got 34359738374 for osd.0 2024-07-04T17:26:27.489 DEBUG:teuthology.parallel:result is None 2024-07-04T17:26:27.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.494+0000 7f4dd1200640 1 -- 172.21.15.42:0/1327155543 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f4dcc0bae00 con 0x7f4dcc06cea0 2024-07-04T17:26:27.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.495+0000 7f4dc2000640 1 -- 172.21.15.42:0/1327155543 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4dbc050be0 con 0x7f4dcc06cea0 2024-07-04T17:26:27.497 INFO:teuthology.orchestra.run.smithi042.stdout:34359738376 2024-07-04T17:26:27.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.498+0000 7f4dd1200640 1 -- 172.21.15.42:0/1327155543 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f4dac03d2b0 msgr2=0x7f4dac03f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:27.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.498+0000 7f4dd1200640 1 --2- 172.21.15.42:0/1327155543 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f4dac03d2b0 0x7f4dac03f7c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4db8035000 tx=0x7f4db8004730 comp rx=0 tx=0).stop 2024-07-04T17:26:27.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.498+0000 7f4dd1200640 1 -- 172.21.15.42:0/1327155543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4dcc06cea0 msgr2=0x7f4dcc0aeba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:27.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.498+0000 7f4dd1200640 1 --2- 172.21.15.42:0/1327155543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4dcc06cea0 0x7f4dcc0aeba0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc00c380 tx=0x7f4dbc00c840 comp rx=0 tx=0).stop 2024-07-04T17:26:27.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.498+0000 7f4dd1200640 1 -- 172.21.15.42:0/1327155543 shutdown_connections 2024-07-04T17:26:27.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.498+0000 7f4dd1200640 1 --2- 172.21.15.42:0/1327155543 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f4dac03d2b0 0x7f4dac03f7c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:27.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.498+0000 7f4dd1200640 1 --2- 172.21.15.42:0/1327155543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4dcc06cea0 0x7f4dcc0aeba0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:27.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.498+0000 7f4dd1200640 1 -- 172.21.15.42:0/1327155543 >> 172.21.15.42:0/1327155543 conn(0x7f4dcc0bf700 msgr2=0x7f4dcc14b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:27.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.498+0000 7f4dd1200640 1 -- 172.21.15.42:0/1327155543 shutdown_connections 2024-07-04T17:26:27.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.498+0000 7f4dd1200640 1 -- 172.21.15.42:0/1327155543 wait complete. 2024-07-04T17:26:27.522 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738376 got 34359738376 for osd.1 2024-07-04T17:26:27.523 DEBUG:teuthology.parallel:result is None 2024-07-04T17:26:27.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.525+0000 7f4ff9000640 1 -- 172.21.15.42:0/3903851445 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f4ff40687d0 con 0x7f4ff40adaa0 2024-07-04T17:26:27.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.526+0000 7f4feb400640 1 -- 172.21.15.42:0/3903851445 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4fe4003b10 con 0x7f4ff40adaa0 2024-07-04T17:26:27.527 INFO:teuthology.orchestra.run.smithi042.stdout:34359738375 2024-07-04T17:26:27.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.528+0000 7f4ff9000640 1 -- 172.21.15.42:0/3903851445 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f4fd403d2b0 msgr2=0x7f4fd403f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:27.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.528+0000 7f4ff9000640 1 --2- 172.21.15.42:0/3903851445 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f4fd403d2b0 0x7f4fd403f7c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4147530 tx=0x7f4fe0007a90 comp rx=0 tx=0).stop 2024-07-04T17:26:27.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.528+0000 7f4ff9000640 1 -- 172.21.15.42:0/3903851445 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ff40adaa0 msgr2=0x7f4ff413ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:27.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.528+0000 7f4ff9000640 1 --2- 172.21.15.42:0/3903851445 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ff40adaa0 0x7f4ff413ded0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4fe402ab30 tx=0x7f4fe4033e40 comp rx=0 tx=0).stop 2024-07-04T17:26:27.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.528+0000 7f4ff9000640 1 -- 172.21.15.42:0/3903851445 shutdown_connections 2024-07-04T17:26:27.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.528+0000 7f4ff9000640 1 --2- 172.21.15.42:0/3903851445 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f4fd403d2b0 0x7f4fd403f7c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:27.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.528+0000 7f4ff9000640 1 --2- 172.21.15.42:0/3903851445 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ff40adaa0 0x7f4ff413ded0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:27.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.528+0000 7f4ff9000640 1 -- 172.21.15.42:0/3903851445 >> 172.21.15.42:0/3903851445 conn(0x7f4ff406c020 msgr2=0x7f4ff40b3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:27.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.529+0000 7f4ff9000640 1 -- 172.21.15.42:0/3903851445 shutdown_connections 2024-07-04T17:26:27.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.529+0000 7f4ff9000640 1 -- 172.21.15.42:0/3903851445 wait complete. 2024-07-04T17:26:27.553 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738375 got 34359738375 for osd.2 2024-07-04T17:26:27.554 DEBUG:teuthology.parallel:result is None 2024-07-04T17:26:27.554 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-04T17:26:27.554 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:26:27.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.887+0000 7fe529200640 1 -- 172.21.15.42:0/564143450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5240b2d20 msgr2=0x7fe5240bb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:27.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.887+0000 7fe529200640 1 --2- 172.21.15.42:0/564143450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5240b2d20 0x7fe5240bb1f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0097b0 tx=0x7fe50c02a4a0 comp rx=0 tx=0).stop 2024-07-04T17:26:27.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.887+0000 7fe529200640 1 -- 172.21.15.42:0/564143450 shutdown_connections 2024-07-04T17:26:27.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.887+0000 7fe529200640 1 --2- 172.21.15.42:0/564143450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5240b2d20 0x7fe5240bb1f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:27.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.887+0000 7fe529200640 1 -- 172.21.15.42:0/564143450 >> 172.21.15.42:0/564143450 conn(0x7fe5240bf700 msgr2=0x7fe5240af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:27.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.887+0000 7fe529200640 1 -- 172.21.15.42:0/564143450 shutdown_connections 2024-07-04T17:26:27.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.887+0000 7fe529200640 1 -- 172.21.15.42:0/564143450 wait complete. 2024-07-04T17:26:27.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.889+0000 7fe529200640 1 Processor -- start 2024-07-04T17:26:27.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.889+0000 7fe529200640 1 -- start start 2024-07-04T17:26:27.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.891+0000 7fe529200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5240b2d20 0x7fe5241d6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:27.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.891+0000 7fe529200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe52406c020 con 0x7fe5240b2d20 2024-07-04T17:26:27.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.891+0000 7fe521e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5240b2d20 0x7fe5241d6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:27.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.891+0000 7fe521e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5240b2d20 0x7fe5241d6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44134/0 (socket says 172.21.15.42:44134) 2024-07-04T17:26:27.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.891+0000 7fe521e00640 1 -- 172.21.15.42:0/3844547590 learned_addr learned my addr 172.21.15.42:0/3844547590 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:27.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.891+0000 7fe521e00640 1 -- 172.21.15.42:0/3844547590 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5241e1370 con 0x7fe5240b2d20 2024-07-04T17:26:27.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.892+0000 7fe521e00640 1 --2- 172.21.15.42:0/3844547590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5240b2d20 0x7fe5241d6cc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe51400c3b0 tx=0x7fe51400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:27.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.892+0000 7fe51a000640 1 -- 172.21.15.42:0/3844547590 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe514011780 con 0x7fe5240b2d20 2024-07-04T17:26:27.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.892+0000 7fe51a000640 1 -- 172.21.15.42:0/3844547590 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe514017070 con 0x7fe5240b2d20 2024-07-04T17:26:27.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.892+0000 7fe51a000640 1 -- 172.21.15.42:0/3844547590 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe514011c00 con 0x7fe5240b2d20 2024-07-04T17:26:27.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.892+0000 7fe529200640 1 -- 172.21.15.42:0/3844547590 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5241e1600 con 0x7fe5240b2d20 2024-07-04T17:26:27.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.892+0000 7fe529200640 1 -- 172.21.15.42:0/3844547590 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5240acb90 con 0x7fe5240b2d20 2024-07-04T17:26:27.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.893+0000 7fe51a000640 1 -- 172.21.15.42:0/3844547590 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fe51401c460 con 0x7fe5240b2d20 2024-07-04T17:26:27.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.893+0000 7fe529200640 1 -- 172.21.15.42:0/3844547590 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe524137e00 con 0x7fe5240b2d20 2024-07-04T17:26:27.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.894+0000 7fe51a000640 1 --2- 172.21.15.42:0/3844547590 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fe50003d2b0 0x7fe50003f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:27.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.894+0000 7fe51a000640 1 -- 172.21.15.42:0/3844547590 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7fe5140511c0 con 0x7fe5240b2d20 2024-07-04T17:26:27.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.894+0000 7fe51be00640 1 --2- 172.21.15.42:0/3844547590 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fe50003d2b0 0x7fe50003f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:27.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.895+0000 7fe51be00640 1 --2- 172.21.15.42:0/3844547590 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fe50003d2b0 0x7fe50003f7c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe50c034000 tx=0x7fe50c004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:27.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:27.899+0000 7fe51a000640 1 -- 172.21.15.42:0/3844547590 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fe514011da0 con 0x7fe5240b2d20 2024-07-04T17:26:28.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.105+0000 7fe529200640 1 -- 172.21.15.42:0/3844547590 --> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe5240ac520 con 0x7fe50003d2b0 2024-07-04T17:26:28.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.118+0000 7fe51a000640 1 -- 172.21.15.42:0/3844547590 <== mgr.4103 v2:172.21.15.42:6824/3523793950 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245906 (secure 0 0 0) 0x7fe5240ac520 con 0x7fe50003d2b0 2024-07-04T17:26:28.123 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:26:28.125 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:26:28.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.125+0000 7fe529200640 1 -- 172.21.15.42:0/3844547590 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fe50003d2b0 msgr2=0x7fe50003f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:28.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.125+0000 7fe529200640 1 --2- 172.21.15.42:0/3844547590 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fe50003d2b0 0x7fe50003f7c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe50c034000 tx=0x7fe50c004730 comp rx=0 tx=0).stop 2024-07-04T17:26:28.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.125+0000 7fe529200640 1 -- 172.21.15.42:0/3844547590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5240b2d20 msgr2=0x7fe5241d6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:28.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.125+0000 7fe529200640 1 --2- 172.21.15.42:0/3844547590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5240b2d20 0x7fe5241d6cc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe51400c3b0 tx=0x7fe51400c870 comp rx=0 tx=0).stop 2024-07-04T17:26:28.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.125+0000 7fe529200640 1 -- 172.21.15.42:0/3844547590 shutdown_connections 2024-07-04T17:26:28.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.125+0000 7fe529200640 1 --2- 172.21.15.42:0/3844547590 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fe50003d2b0 0x7fe50003f7c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:28.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.125+0000 7fe529200640 1 --2- 172.21.15.42:0/3844547590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5240b2d20 0x7fe5241d6cc0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:28.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.125+0000 7fe529200640 1 -- 172.21.15.42:0/3844547590 >> 172.21.15.42:0/3844547590 conn(0x7fe5240bf700 msgr2=0x7fe524140070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:28.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.125+0000 7fe529200640 1 -- 172.21.15.42:0/3844547590 shutdown_connections 2024-07-04T17:26:28.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.126+0000 7fe529200640 1 -- 172.21.15.42:0/3844547590 wait complete. 2024-07-04T17:26:28.154 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-07-04T17:26:27.266101+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":124,"num_write_kb":1120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91,"ondisk_log_size":91,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":95164,"kb_used_data":1608,"kb_used_omap":4,"kb_used_meta":93499,"kb_avail":283020356,"statfs":{"total":289910292480,"available":289812844544,"internally_reserved":0,"allocated":1646592,"data_stored":1306663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":95743311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.248494"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.023948+0000","last_change":"2024-07-04T17:26:16.062462+0000","last_active":"2024-07-04T17:26:17.023948+0000","last_peered":"2024-07-04T17:26:17.023948+0000","last_clean":"2024-07-04T17:26:17.023948+0000","last_became_active":"2024-07-04T17:26:16.058531+0000","last_became_peered":"2024-07-04T17:26:16.058531+0000","last_unstale":"2024-07-04T17:26:17.023948+0000","last_undegraded":"2024-07-04T17:26:17.023948+0000","last_fullsized":"2024-07-04T17:26:17.023948+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:31:33.921497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.028401+0000","last_change":"2024-07-04T17:26:16.064253+0000","last_active":"2024-07-04T17:26:17.028401+0000","last_peered":"2024-07-04T17:26:17.028401+0000","last_clean":"2024-07-04T17:26:17.028401+0000","last_became_active":"2024-07-04T17:26:16.063971+0000","last_became_peered":"2024-07-04T17:26:16.063971+0000","last_unstale":"2024-07-04T17:26:17.028401+0000","last_undegraded":"2024-07-04T17:26:17.028401+0000","last_fullsized":"2024-07-04T17:26:17.028401+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:22:14.191409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022213+0000","last_change":"2024-07-04T17:26:16.059637+0000","last_active":"2024-07-04T17:26:17.022213+0000","last_peered":"2024-07-04T17:26:17.022213+0000","last_clean":"2024-07-04T17:26:17.022213+0000","last_became_active":"2024-07-04T17:26:16.058613+0000","last_became_peered":"2024-07-04T17:26:16.058613+0000","last_unstale":"2024-07-04T17:26:17.022213+0000","last_undegraded":"2024-07-04T17:26:17.022213+0000","last_fullsized":"2024-07-04T17:26:17.022213+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:02:40.668757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702530+0000","last_change":"2024-07-04T17:26:16.072904+0000","last_active":"2024-07-04T17:26:17.702530+0000","last_peered":"2024-07-04T17:26:17.702530+0000","last_clean":"2024-07-04T17:26:17.702530+0000","last_became_active":"2024-07-04T17:26:16.072640+0000","last_became_peered":"2024-07-04T17:26:16.072640+0000","last_unstale":"2024-07-04T17:26:17.702530+0000","last_undegraded":"2024-07-04T17:26:17.702530+0000","last_fullsized":"2024-07-04T17:26:17.702530+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:03:07.909265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021289+0000","last_change":"2024-07-04T17:26:16.044297+0000","last_active":"2024-07-04T17:26:17.021289+0000","last_peered":"2024-07-04T17:26:17.021289+0000","last_clean":"2024-07-04T17:26:17.021289+0000","last_became_active":"2024-07-04T17:26:16.043526+0000","last_became_peered":"2024-07-04T17:26:16.043526+0000","last_unstale":"2024-07-04T17:26:17.021289+0000","last_undegraded":"2024-07-04T17:26:17.021289+0000","last_fullsized":"2024-07-04T17:26:17.021289+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:11:11.392035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702192+0000","last_change":"2024-07-04T17:26:16.072472+0000","last_active":"2024-07-04T17:26:17.702192+0000","last_peered":"2024-07-04T17:26:17.702192+0000","last_clean":"2024-07-04T17:26:17.702192+0000","last_became_active":"2024-07-04T17:26:16.072066+0000","last_became_peered":"2024-07-04T17:26:16.072066+0000","last_unstale":"2024-07-04T17:26:17.702192+0000","last_undegraded":"2024-07-04T17:26:17.702192+0000","last_fullsized":"2024-07-04T17:26:17.702192+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:11:17.509522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021458+0000","last_change":"2024-07-04T17:26:16.070453+0000","last_active":"2024-07-04T17:26:17.021458+0000","last_peered":"2024-07-04T17:26:17.021458+0000","last_clean":"2024-07-04T17:26:17.021458+0000","last_became_active":"2024-07-04T17:26:16.065958+0000","last_became_peered":"2024-07-04T17:26:16.065958+0000","last_unstale":"2024-07-04T17:26:17.021458+0000","last_undegraded":"2024-07-04T17:26:17.021458+0000","last_fullsized":"2024-07-04T17:26:17.021458+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:51:59.570861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.700790+0000","last_change":"2024-07-04T17:26:16.072492+0000","last_active":"2024-07-04T17:26:17.700790+0000","last_peered":"2024-07-04T17:26:17.700790+0000","last_clean":"2024-07-04T17:26:17.700790+0000","last_became_active":"2024-07-04T17:26:16.067345+0000","last_became_peered":"2024-07-04T17:26:16.067345+0000","last_unstale":"2024-07-04T17:26:17.700790+0000","last_undegraded":"2024-07-04T17:26:17.700790+0000","last_fullsized":"2024-07-04T17:26:17.700790+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:02:12.965714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.028868+0000","last_change":"2024-07-04T17:26:16.055031+0000","last_active":"2024-07-04T17:26:17.028868+0000","last_peered":"2024-07-04T17:26:17.028868+0000","last_clean":"2024-07-04T17:26:17.028868+0000","last_became_active":"2024-07-04T17:26:16.051647+0000","last_became_peered":"2024-07-04T17:26:16.051647+0000","last_unstale":"2024-07-04T17:26:17.028868+0000","last_undegraded":"2024-07-04T17:26:17.028868+0000","last_fullsized":"2024-07-04T17:26:17.028868+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:41:20.224285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022523+0000","last_change":"2024-07-04T17:26:16.045710+0000","last_active":"2024-07-04T17:26:17.022523+0000","last_peered":"2024-07-04T17:26:17.022523+0000","last_clean":"2024-07-04T17:26:17.022523+0000","last_became_active":"2024-07-04T17:26:16.045371+0000","last_became_peered":"2024-07-04T17:26:16.045371+0000","last_unstale":"2024-07-04T17:26:17.022523+0000","last_undegraded":"2024-07-04T17:26:17.022523+0000","last_fullsized":"2024-07-04T17:26:17.022523+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:53:44.096508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022006+0000","last_change":"2024-07-04T17:26:16.059936+0000","last_active":"2024-07-04T17:26:17.022006+0000","last_peered":"2024-07-04T17:26:17.022006+0000","last_clean":"2024-07-04T17:26:17.022006+0000","last_became_active":"2024-07-04T17:26:16.058990+0000","last_became_peered":"2024-07-04T17:26:16.058990+0000","last_unstale":"2024-07-04T17:26:17.022006+0000","last_undegraded":"2024-07-04T17:26:17.022006+0000","last_fullsized":"2024-07-04T17:26:17.022006+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:47:09.376866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022753+0000","last_change":"2024-07-04T17:26:16.053390+0000","last_active":"2024-07-04T17:26:17.022753+0000","last_peered":"2024-07-04T17:26:17.022753+0000","last_clean":"2024-07-04T17:26:17.022753+0000","last_became_active":"2024-07-04T17:26:16.052716+0000","last_became_peered":"2024-07-04T17:26:16.052716+0000","last_unstale":"2024-07-04T17:26:17.022753+0000","last_undegraded":"2024-07-04T17:26:17.022753+0000","last_fullsized":"2024-07-04T17:26:17.022753+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:15:52.371580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021947+0000","last_change":"2024-07-04T17:26:16.037676+0000","last_active":"2024-07-04T17:26:17.021947+0000","last_peered":"2024-07-04T17:26:17.021947+0000","last_clean":"2024-07-04T17:26:17.021947+0000","last_became_active":"2024-07-04T17:26:16.037311+0000","last_became_peered":"2024-07-04T17:26:16.037311+0000","last_unstale":"2024-07-04T17:26:17.021947+0000","last_undegraded":"2024-07-04T17:26:17.021947+0000","last_fullsized":"2024-07-04T17:26:17.021947+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:31:49.427076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020967+0000","last_change":"2024-07-04T17:26:16.064558+0000","last_active":"2024-07-04T17:26:17.020967+0000","last_peered":"2024-07-04T17:26:17.020967+0000","last_clean":"2024-07-04T17:26:17.020967+0000","last_became_active":"2024-07-04T17:26:16.054068+0000","last_became_peered":"2024-07-04T17:26:16.054068+0000","last_unstale":"2024-07-04T17:26:17.020967+0000","last_undegraded":"2024-07-04T17:26:17.020967+0000","last_fullsized":"2024-07-04T17:26:17.020967+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:30:12.289936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026054+0000","last_change":"2024-07-04T17:26:16.056551+0000","last_active":"2024-07-04T17:26:17.026054+0000","last_peered":"2024-07-04T17:26:17.026054+0000","last_clean":"2024-07-04T17:26:17.026054+0000","last_became_active":"2024-07-04T17:26:16.056173+0000","last_became_peered":"2024-07-04T17:26:16.056173+0000","last_unstale":"2024-07-04T17:26:17.026054+0000","last_undegraded":"2024-07-04T17:26:17.026054+0000","last_fullsized":"2024-07-04T17:26:17.026054+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:20:14.576761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.700980+0000","last_change":"2024-07-04T17:26:16.073053+0000","last_active":"2024-07-04T17:26:17.700980+0000","last_peered":"2024-07-04T17:26:17.700980+0000","last_clean":"2024-07-04T17:26:17.700980+0000","last_became_active":"2024-07-04T17:26:16.072132+0000","last_became_peered":"2024-07-04T17:26:16.072132+0000","last_unstale":"2024-07-04T17:26:17.700980+0000","last_undegraded":"2024-07-04T17:26:17.700980+0000","last_fullsized":"2024-07-04T17:26:17.700980+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:11:05.657045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.029516+0000","last_change":"2024-07-04T17:26:16.055789+0000","last_active":"2024-07-04T17:26:17.029516+0000","last_peered":"2024-07-04T17:26:17.029516+0000","last_clean":"2024-07-04T17:26:17.029516+0000","last_became_active":"2024-07-04T17:26:16.055206+0000","last_became_peered":"2024-07-04T17:26:16.055206+0000","last_unstale":"2024-07-04T17:26:17.029516+0000","last_undegraded":"2024-07-04T17:26:17.029516+0000","last_fullsized":"2024-07-04T17:26:17.029516+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:31:59.564668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.703184+0000","last_change":"2024-07-04T17:26:16.070972+0000","last_active":"2024-07-04T17:26:17.703184+0000","last_peered":"2024-07-04T17:26:17.703184+0000","last_clean":"2024-07-04T17:26:17.703184+0000","last_became_active":"2024-07-04T17:26:16.067224+0000","last_became_peered":"2024-07-04T17:26:16.067224+0000","last_unstale":"2024-07-04T17:26:17.703184+0000","last_undegraded":"2024-07-04T17:26:17.703184+0000","last_fullsized":"2024-07-04T17:26:17.703184+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:22:14.191409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022486+0000","last_change":"2024-07-04T17:26:16.059368+0000","last_active":"2024-07-04T17:26:17.022486+0000","last_peered":"2024-07-04T17:26:17.022486+0000","last_clean":"2024-07-04T17:26:17.022486+0000","last_became_active":"2024-07-04T17:26:16.058568+0000","last_became_peered":"2024-07-04T17:26:16.058568+0000","last_unstale":"2024-07-04T17:26:17.022486+0000","last_undegraded":"2024-07-04T17:26:17.022486+0000","last_fullsized":"2024-07-04T17:26:17.022486+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:18:06.133593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022008+0000","last_change":"2024-07-04T17:26:16.083687+0000","last_active":"2024-07-04T17:26:17.022008+0000","last_peered":"2024-07-04T17:26:17.022008+0000","last_clean":"2024-07-04T17:26:17.022008+0000","last_became_active":"2024-07-04T17:26:16.083145+0000","last_became_peered":"2024-07-04T17:26:16.083145+0000","last_unstale":"2024-07-04T17:26:17.022008+0000","last_undegraded":"2024-07-04T17:26:17.022008+0000","last_fullsized":"2024-07-04T17:26:17.022008+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:36:50.247043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020646+0000","last_change":"2024-07-04T17:26:16.047023+0000","last_active":"2024-07-04T17:26:17.020646+0000","last_peered":"2024-07-04T17:26:17.020646+0000","last_clean":"2024-07-04T17:26:17.020646+0000","last_became_active":"2024-07-04T17:26:16.046631+0000","last_became_peered":"2024-07-04T17:26:16.046631+0000","last_unstale":"2024-07-04T17:26:17.020646+0000","last_undegraded":"2024-07-04T17:26:17.020646+0000","last_fullsized":"2024-07-04T17:26:17.020646+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:13:00.963495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026306+0000","last_change":"2024-07-04T17:26:16.075119+0000","last_active":"2024-07-04T17:26:17.026306+0000","last_peered":"2024-07-04T17:26:17.026306+0000","last_clean":"2024-07-04T17:26:17.026306+0000","last_became_active":"2024-07-04T17:26:16.074600+0000","last_became_peered":"2024-07-04T17:26:16.074600+0000","last_unstale":"2024-07-04T17:26:17.026306+0000","last_undegraded":"2024-07-04T17:26:17.026306+0000","last_fullsized":"2024-07-04T17:26:17.026306+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:17:18.408614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021060+0000","last_change":"2024-07-04T17:26:16.052698+0000","last_active":"2024-07-04T17:26:17.021060+0000","last_peered":"2024-07-04T17:26:17.021060+0000","last_clean":"2024-07-04T17:26:17.021060+0000","last_became_active":"2024-07-04T17:26:16.052278+0000","last_became_peered":"2024-07-04T17:26:16.052278+0000","last_unstale":"2024-07-04T17:26:17.021060+0000","last_undegraded":"2024-07-04T17:26:17.021060+0000","last_fullsized":"2024-07-04T17:26:17.021060+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:27:20.732374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020541+0000","last_change":"2024-07-04T17:26:16.052254+0000","last_active":"2024-07-04T17:26:17.020541+0000","last_peered":"2024-07-04T17:26:17.020541+0000","last_clean":"2024-07-04T17:26:17.020541+0000","last_became_active":"2024-07-04T17:26:16.046560+0000","last_became_peered":"2024-07-04T17:26:16.046560+0000","last_unstale":"2024-07-04T17:26:17.020541+0000","last_undegraded":"2024-07-04T17:26:17.020541+0000","last_fullsized":"2024-07-04T17:26:17.020541+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:50:25.908300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702155+0000","last_change":"2024-07-04T17:26:16.061658+0000","last_active":"2024-07-04T17:26:17.702155+0000","last_peered":"2024-07-04T17:26:17.702155+0000","last_clean":"2024-07-04T17:26:17.702155+0000","last_became_active":"2024-07-04T17:26:16.061257+0000","last_became_peered":"2024-07-04T17:26:16.061257+0000","last_unstale":"2024-07-04T17:26:17.702155+0000","last_undegraded":"2024-07-04T17:26:17.702155+0000","last_fullsized":"2024-07-04T17:26:17.702155+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:17:18.408614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.703496+0000","last_change":"2024-07-04T17:26:16.073828+0000","last_active":"2024-07-04T17:26:17.703496+0000","last_peered":"2024-07-04T17:26:17.703496+0000","last_clean":"2024-07-04T17:26:17.703496+0000","last_became_active":"2024-07-04T17:26:16.073494+0000","last_became_peered":"2024-07-04T17:26:16.073494+0000","last_unstale":"2024-07-04T17:26:17.703496+0000","last_undegraded":"2024-07-04T17:26:17.703496+0000","last_fullsized":"2024-07-04T17:26:17.703496+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:41:20.224285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.028237+0000","last_change":"2024-07-04T17:26:16.046050+0000","last_active":"2024-07-04T17:26:17.028237+0000","last_peered":"2024-07-04T17:26:17.028237+0000","last_clean":"2024-07-04T17:26:17.028237+0000","last_became_active":"2024-07-04T17:26:16.043726+0000","last_became_peered":"2024-07-04T17:26:16.043726+0000","last_unstale":"2024-07-04T17:26:17.028237+0000","last_undegraded":"2024-07-04T17:26:17.028237+0000","last_fullsized":"2024-07-04T17:26:17.028237+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:43:29.877017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022171+0000","last_change":"2024-07-04T17:26:16.047247+0000","last_active":"2024-07-04T17:26:17.022171+0000","last_peered":"2024-07-04T17:26:17.022171+0000","last_clean":"2024-07-04T17:26:17.022171+0000","last_became_active":"2024-07-04T17:26:16.046845+0000","last_became_peered":"2024-07-04T17:26:16.046845+0000","last_unstale":"2024-07-04T17:26:17.022171+0000","last_undegraded":"2024-07-04T17:26:17.022171+0000","last_fullsized":"2024-07-04T17:26:17.022171+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:22:33.108870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020889+0000","last_change":"2024-07-04T17:26:16.047621+0000","last_active":"2024-07-04T17:26:17.020889+0000","last_peered":"2024-07-04T17:26:17.020889+0000","last_clean":"2024-07-04T17:26:17.020889+0000","last_became_active":"2024-07-04T17:26:16.047328+0000","last_became_peered":"2024-07-04T17:26:16.047328+0000","last_unstale":"2024-07-04T17:26:17.020889+0000","last_undegraded":"2024-07-04T17:26:17.020889+0000","last_fullsized":"2024-07-04T17:26:17.020889+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:03:58.364101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020557+0000","last_change":"2024-07-04T17:26:16.046513+0000","last_active":"2024-07-04T17:26:17.020557+0000","last_peered":"2024-07-04T17:26:17.020557+0000","last_clean":"2024-07-04T17:26:17.020557+0000","last_became_active":"2024-07-04T17:26:16.045399+0000","last_became_peered":"2024-07-04T17:26:16.045399+0000","last_unstale":"2024-07-04T17:26:17.020557+0000","last_undegraded":"2024-07-04T17:26:17.020557+0000","last_fullsized":"2024-07-04T17:26:17.020557+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:31:59.564668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.701258+0000","last_change":"2024-07-04T17:26:16.057205+0000","last_active":"2024-07-04T17:26:17.701258+0000","last_peered":"2024-07-04T17:26:17.701258+0000","last_clean":"2024-07-04T17:26:17.701258+0000","last_became_active":"2024-07-04T17:26:16.056891+0000","last_became_peered":"2024-07-04T17:26:16.056891+0000","last_unstale":"2024-07-04T17:26:17.701258+0000","last_undegraded":"2024-07-04T17:26:17.701258+0000","last_fullsized":"2024-07-04T17:26:17.701258+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:16:58.662657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.701357+0000","last_change":"2024-07-04T17:26:16.072774+0000","last_active":"2024-07-04T17:26:17.701357+0000","last_peered":"2024-07-04T17:26:17.701357+0000","last_clean":"2024-07-04T17:26:17.701357+0000","last_became_active":"2024-07-04T17:26:16.071711+0000","last_became_peered":"2024-07-04T17:26:16.071711+0000","last_unstale":"2024-07-04T17:26:17.701357+0000","last_undegraded":"2024-07-04T17:26:17.701357+0000","last_fullsized":"2024-07-04T17:26:17.701357+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:20:14.576761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026507+0000","last_change":"2024-07-04T17:26:16.047214+0000","last_active":"2024-07-04T17:26:17.026507+0000","last_peered":"2024-07-04T17:26:17.026507+0000","last_clean":"2024-07-04T17:26:17.026507+0000","last_became_active":"2024-07-04T17:26:16.046836+0000","last_became_peered":"2024-07-04T17:26:16.046836+0000","last_unstale":"2024-07-04T17:26:17.026507+0000","last_undegraded":"2024-07-04T17:26:17.026507+0000","last_fullsized":"2024-07-04T17:26:17.026507+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:04:04.805872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"14'1","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.027730+0000","last_change":"2024-07-04T17:26:15.033936+0000","last_active":"2024-07-04T17:26:17.027730+0000","last_peered":"2024-07-04T17:26:17.027730+0000","last_clean":"2024-07-04T17:26:17.027730+0000","last_became_active":"2024-07-04T17:26:15.033601+0000","last_became_peered":"2024-07-04T17:26:15.033601+0000","last_unstale":"2024-07-04T17:26:17.027730+0000","last_undegraded":"2024-07-04T17:26:17.027730+0000","last_fullsized":"2024-07-04T17:26:17.027730+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:17:16.904464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021045+0000","last_change":"2024-07-04T17:26:16.053083+0000","last_active":"2024-07-04T17:26:17.021045+0000","last_peered":"2024-07-04T17:26:17.021045+0000","last_clean":"2024-07-04T17:26:17.021045+0000","last_became_active":"2024-07-04T17:26:16.052739+0000","last_became_peered":"2024-07-04T17:26:16.052739+0000","last_unstale":"2024-07-04T17:26:17.021045+0000","last_undegraded":"2024-07-04T17:26:17.021045+0000","last_fullsized":"2024-07-04T17:26:17.021045+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:00:52.128797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"14'1","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.701569+0000","last_change":"2024-07-04T17:26:15.039637+0000","last_active":"2024-07-04T17:26:17.701569+0000","last_peered":"2024-07-04T17:26:17.701569+0000","last_clean":"2024-07-04T17:26:17.701569+0000","last_became_active":"2024-07-04T17:26:15.039305+0000","last_became_peered":"2024-07-04T17:26:15.039305+0000","last_unstale":"2024-07-04T17:26:17.701569+0000","last_undegraded":"2024-07-04T17:26:17.701569+0000","last_fullsized":"2024-07-04T17:26:17.701569+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:04:46.667644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021302+0000","last_change":"2024-07-04T17:26:15.027075+0000","last_active":"2024-07-04T17:26:17.021302+0000","last_peered":"2024-07-04T17:26:17.021302+0000","last_clean":"2024-07-04T17:26:17.021302+0000","last_became_active":"2024-07-04T17:26:15.026375+0000","last_became_peered":"2024-07-04T17:26:15.026375+0000","last_unstale":"2024-07-04T17:26:17.021302+0000","last_undegraded":"2024-07-04T17:26:17.021302+0000","last_fullsized":"2024-07-04T17:26:17.021302+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:24:04.318931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702805+0000","last_change":"2024-07-04T17:26:16.074834+0000","last_active":"2024-07-04T17:26:17.702805+0000","last_peered":"2024-07-04T17:26:17.702805+0000","last_clean":"2024-07-04T17:26:17.702805+0000","last_became_active":"2024-07-04T17:26:16.074498+0000","last_became_peered":"2024-07-04T17:26:16.074498+0000","last_unstale":"2024-07-04T17:26:17.702805+0000","last_undegraded":"2024-07-04T17:26:17.702805+0000","last_fullsized":"2024-07-04T17:26:17.702805+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:08:53.606368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020171+0000","last_change":"2024-07-04T17:26:15.046968+0000","last_active":"2024-07-04T17:26:17.020171+0000","last_peered":"2024-07-04T17:26:17.020171+0000","last_clean":"2024-07-04T17:26:17.020171+0000","last_became_active":"2024-07-04T17:26:15.046617+0000","last_became_peered":"2024-07-04T17:26:15.046617+0000","last_unstale":"2024-07-04T17:26:17.020171+0000","last_undegraded":"2024-07-04T17:26:17.020171+0000","last_fullsized":"2024-07-04T17:26:17.020171+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:13:00.368941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026043+0000","last_change":"2024-07-04T17:26:16.041627+0000","last_active":"2024-07-04T17:26:17.026043+0000","last_peered":"2024-07-04T17:26:17.026043+0000","last_clean":"2024-07-04T17:26:17.026043+0000","last_became_active":"2024-07-04T17:26:16.041274+0000","last_became_peered":"2024-07-04T17:26:16.041274+0000","last_unstale":"2024-07-04T17:26:17.026043+0000","last_undegraded":"2024-07-04T17:26:17.026043+0000","last_fullsized":"2024-07-04T17:26:17.026043+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:41:46.272734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.029021+0000","last_change":"2024-07-04T17:26:15.037603+0000","last_active":"2024-07-04T17:26:17.029021+0000","last_peered":"2024-07-04T17:26:17.029021+0000","last_clean":"2024-07-04T17:26:17.029021+0000","last_became_active":"2024-07-04T17:26:15.030406+0000","last_became_peered":"2024-07-04T17:26:15.030406+0000","last_unstale":"2024-07-04T17:26:17.029021+0000","last_undegraded":"2024-07-04T17:26:17.029021+0000","last_fullsized":"2024-07-04T17:26:17.029021+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:13:00.368941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.703750+0000","last_change":"2024-07-04T17:26:16.060475+0000","last_active":"2024-07-04T17:26:17.703750+0000","last_peered":"2024-07-04T17:26:17.703750+0000","last_clean":"2024-07-04T17:26:17.703750+0000","last_became_active":"2024-07-04T17:26:16.059861+0000","last_became_peered":"2024-07-04T17:26:16.059861+0000","last_unstale":"2024-07-04T17:26:17.703750+0000","last_undegraded":"2024-07-04T17:26:17.703750+0000","last_fullsized":"2024-07-04T17:26:17.703750+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:21:38.693638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.023859+0000","last_change":"2024-07-04T17:26:15.046597+0000","last_active":"2024-07-04T17:26:17.023859+0000","last_peered":"2024-07-04T17:26:17.023859+0000","last_clean":"2024-07-04T17:26:17.023859+0000","last_became_active":"2024-07-04T17:26:15.046243+0000","last_became_peered":"2024-07-04T17:26:15.046243+0000","last_unstale":"2024-07-04T17:26:17.023859+0000","last_undegraded":"2024-07-04T17:26:17.023859+0000","last_fullsized":"2024-07-04T17:26:17.023859+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:18:04.371426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.024536+0000","last_change":"2024-07-04T17:26:16.059218+0000","last_active":"2024-07-04T17:26:17.024536+0000","last_peered":"2024-07-04T17:26:17.024536+0000","last_clean":"2024-07-04T17:26:17.024536+0000","last_became_active":"2024-07-04T17:26:16.058877+0000","last_became_peered":"2024-07-04T17:26:16.058877+0000","last_unstale":"2024-07-04T17:26:17.024536+0000","last_undegraded":"2024-07-04T17:26:17.024536+0000","last_fullsized":"2024-07-04T17:26:17.024536+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:48:48.817671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"14'3","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.029077+0000","last_change":"2024-07-04T17:26:15.031110+0000","last_active":"2024-07-04T17:26:17.029077+0000","last_peered":"2024-07-04T17:26:17.029077+0000","last_clean":"2024-07-04T17:26:17.029077+0000","last_became_active":"2024-07-04T17:26:15.030786+0000","last_became_peered":"2024-07-04T17:26:15.030786+0000","last_unstale":"2024-07-04T17:26:17.029077+0000","last_undegraded":"2024-07-04T17:26:17.029077+0000","last_fullsized":"2024-07-04T17:26:17.029077+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:34:46.876523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.028971+0000","last_change":"2024-07-04T17:26:16.046513+0000","last_active":"2024-07-04T17:26:17.028971+0000","last_peered":"2024-07-04T17:26:17.028971+0000","last_clean":"2024-07-04T17:26:17.028971+0000","last_became_active":"2024-07-04T17:26:16.044971+0000","last_became_peered":"2024-07-04T17:26:16.044971+0000","last_unstale":"2024-07-04T17:26:17.028971+0000","last_undegraded":"2024-07-04T17:26:17.028971+0000","last_fullsized":"2024-07-04T17:26:17.028971+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:21:38.693638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"14'6","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702536+0000","last_change":"2024-07-04T17:26:15.042022+0000","last_active":"2024-07-04T17:26:17.702536+0000","last_peered":"2024-07-04T17:26:17.702536+0000","last_clean":"2024-07-04T17:26:17.702536+0000","last_became_active":"2024-07-04T17:26:15.041669+0000","last_became_peered":"2024-07-04T17:26:15.041669+0000","last_unstale":"2024-07-04T17:26:17.702536+0000","last_undegraded":"2024-07-04T17:26:17.702536+0000","last_fullsized":"2024-07-04T17:26:17.702536+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:35:22.891399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.024207+0000","last_change":"2024-07-04T17:26:16.059656+0000","last_active":"2024-07-04T17:26:17.024207+0000","last_peered":"2024-07-04T17:26:17.024207+0000","last_clean":"2024-07-04T17:26:17.024207+0000","last_became_active":"2024-07-04T17:26:16.058951+0000","last_became_peered":"2024-07-04T17:26:16.058951+0000","last_unstale":"2024-07-04T17:26:17.024207+0000","last_undegraded":"2024-07-04T17:26:17.024207+0000","last_fullsized":"2024-07-04T17:26:17.024207+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:52:45.300002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"14'2","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021264+0000","last_change":"2024-07-04T17:26:15.035545+0000","last_active":"2024-07-04T17:26:17.021264+0000","last_peered":"2024-07-04T17:26:17.021264+0000","last_clean":"2024-07-04T17:26:17.021264+0000","last_became_active":"2024-07-04T17:26:15.034822+0000","last_became_peered":"2024-07-04T17:26:15.034822+0000","last_unstale":"2024-07-04T17:26:17.021264+0000","last_undegraded":"2024-07-04T17:26:17.021264+0000","last_fullsized":"2024-07-04T17:26:17.021264+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:17:16.904464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020669+0000","last_change":"2024-07-04T17:26:16.064292+0000","last_active":"2024-07-04T17:26:17.020669+0000","last_peered":"2024-07-04T17:26:17.020669+0000","last_clean":"2024-07-04T17:26:17.020669+0000","last_became_active":"2024-07-04T17:26:16.053620+0000","last_became_peered":"2024-07-04T17:26:16.053620+0000","last_unstale":"2024-07-04T17:26:17.020669+0000","last_undegraded":"2024-07-04T17:26:17.020669+0000","last_fullsized":"2024-07-04T17:26:17.020669+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:40:49.177003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.701897+0000","last_change":"2024-07-04T17:26:15.041695+0000","last_active":"2024-07-04T17:26:17.701897+0000","last_peered":"2024-07-04T17:26:17.701897+0000","last_clean":"2024-07-04T17:26:17.701897+0000","last_became_active":"2024-07-04T17:26:15.041353+0000","last_became_peered":"2024-07-04T17:26:15.041353+0000","last_unstale":"2024-07-04T17:26:17.701897+0000","last_undegraded":"2024-07-04T17:26:17.701897+0000","last_fullsized":"2024-07-04T17:26:17.701897+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:05:39.042848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026437+0000","last_change":"2024-07-04T17:26:16.063355+0000","last_active":"2024-07-04T17:26:17.026437+0000","last_peered":"2024-07-04T17:26:17.026437+0000","last_clean":"2024-07-04T17:26:17.026437+0000","last_became_active":"2024-07-04T17:26:16.056707+0000","last_became_peered":"2024-07-04T17:26:16.056707+0000","last_unstale":"2024-07-04T17:26:17.026437+0000","last_undegraded":"2024-07-04T17:26:17.026437+0000","last_fullsized":"2024-07-04T17:26:17.026437+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:11:17.509522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021433+0000","last_change":"2024-07-04T17:26:15.050680+0000","last_active":"2024-07-04T17:26:17.021433+0000","last_peered":"2024-07-04T17:26:17.021433+0000","last_clean":"2024-07-04T17:26:17.021433+0000","last_became_active":"2024-07-04T17:26:15.049449+0000","last_became_peered":"2024-07-04T17:26:15.049449+0000","last_unstale":"2024-07-04T17:26:17.021433+0000","last_undegraded":"2024-07-04T17:26:17.021433+0000","last_fullsized":"2024-07-04T17:26:17.021433+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:25:38.473912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026199+0000","last_change":"2024-07-04T17:26:16.063777+0000","last_active":"2024-07-04T17:26:17.026199+0000","last_peered":"2024-07-04T17:26:17.026199+0000","last_clean":"2024-07-04T17:26:17.026199+0000","last_became_active":"2024-07-04T17:26:16.063580+0000","last_became_peered":"2024-07-04T17:26:16.063580+0000","last_unstale":"2024-07-04T17:26:17.026199+0000","last_undegraded":"2024-07-04T17:26:17.026199+0000","last_fullsized":"2024-07-04T17:26:17.026199+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:21:48.005606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"14'8","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702841+0000","last_change":"2024-07-04T17:26:15.043346+0000","last_active":"2024-07-04T17:26:17.702841+0000","last_peered":"2024-07-04T17:26:17.702841+0000","last_clean":"2024-07-04T17:26:17.702841+0000","last_became_active":"2024-07-04T17:26:15.041219+0000","last_became_peered":"2024-07-04T17:26:15.041219+0000","last_unstale":"2024-07-04T17:26:17.702841+0000","last_undegraded":"2024-07-04T17:26:17.702841+0000","last_fullsized":"2024-07-04T17:26:17.702841+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:13:00.368941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.701310+0000","last_change":"2024-07-04T17:26:16.057277+0000","last_active":"2024-07-04T17:26:17.701310+0000","last_peered":"2024-07-04T17:26:17.701310+0000","last_clean":"2024-07-04T17:26:17.701310+0000","last_became_active":"2024-07-04T17:26:16.056946+0000","last_became_peered":"2024-07-04T17:26:16.056946+0000","last_unstale":"2024-07-04T17:26:17.701310+0000","last_undegraded":"2024-07-04T17:26:17.701310+0000","last_fullsized":"2024-07-04T17:26:17.701310+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:41:46.272734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"14'1","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.027552+0000","last_change":"2024-07-04T17:26:15.049872+0000","last_active":"2024-07-04T17:26:17.027552+0000","last_peered":"2024-07-04T17:26:17.027552+0000","last_clean":"2024-07-04T17:26:17.027552+0000","last_became_active":"2024-07-04T17:26:15.048975+0000","last_became_peered":"2024-07-04T17:26:15.048975+0000","last_unstale":"2024-07-04T17:26:17.027552+0000","last_undegraded":"2024-07-04T17:26:17.027552+0000","last_fullsized":"2024-07-04T17:26:17.027552+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:18:52.190216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.704159+0000","last_change":"2024-07-04T17:26:16.071254+0000","last_active":"2024-07-04T17:26:17.704159+0000","last_peered":"2024-07-04T17:26:17.704159+0000","last_clean":"2024-07-04T17:26:17.704159+0000","last_became_active":"2024-07-04T17:26:16.070624+0000","last_became_peered":"2024-07-04T17:26:16.070624+0000","last_unstale":"2024-07-04T17:26:17.704159+0000","last_undegraded":"2024-07-04T17:26:17.704159+0000","last_fullsized":"2024-07-04T17:26:17.704159+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:50:25.908300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"14'2","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.704347+0000","last_change":"2024-07-04T17:26:15.048120+0000","last_active":"2024-07-04T17:26:17.704347+0000","last_peered":"2024-07-04T17:26:17.704347+0000","last_clean":"2024-07-04T17:26:17.704347+0000","last_became_active":"2024-07-04T17:26:15.047745+0000","last_became_peered":"2024-07-04T17:26:15.047745+0000","last_unstale":"2024-07-04T17:26:17.704347+0000","last_undegraded":"2024-07-04T17:26:17.704347+0000","last_fullsized":"2024-07-04T17:26:17.704347+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:17:16.904464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.024662+0000","last_change":"2024-07-04T17:26:16.047526+0000","last_active":"2024-07-04T17:26:17.024662+0000","last_peered":"2024-07-04T17:26:17.024662+0000","last_clean":"2024-07-04T17:26:17.024662+0000","last_became_active":"2024-07-04T17:26:16.046143+0000","last_became_peered":"2024-07-04T17:26:16.046143+0000","last_unstale":"2024-07-04T17:26:17.024662+0000","last_undegraded":"2024-07-04T17:26:17.024662+0000","last_fullsized":"2024-07-04T17:26:17.024662+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:48:44.510528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"14'1","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.029637+0000","last_change":"2024-07-04T17:26:15.021551+0000","last_active":"2024-07-04T17:26:17.029637+0000","last_peered":"2024-07-04T17:26:17.029637+0000","last_clean":"2024-07-04T17:26:17.029637+0000","last_became_active":"2024-07-04T17:26:15.021324+0000","last_became_peered":"2024-07-04T17:26:15.021324+0000","last_unstale":"2024-07-04T17:26:17.029637+0000","last_undegraded":"2024-07-04T17:26:17.029637+0000","last_fullsized":"2024-07-04T17:26:17.029637+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:18:04.371426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022385+0000","last_change":"2024-07-04T17:26:16.043753+0000","last_active":"2024-07-04T17:26:17.022385+0000","last_peered":"2024-07-04T17:26:17.022385+0000","last_clean":"2024-07-04T17:26:17.022385+0000","last_became_active":"2024-07-04T17:26:16.037154+0000","last_became_peered":"2024-07-04T17:26:16.037154+0000","last_unstale":"2024-07-04T17:26:17.022385+0000","last_undegraded":"2024-07-04T17:26:17.022385+0000","last_fullsized":"2024-07-04T17:26:17.022385+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:30:49.842921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702886+0000","last_change":"2024-07-04T17:26:15.039837+0000","last_active":"2024-07-04T17:26:17.702886+0000","last_peered":"2024-07-04T17:26:17.702886+0000","last_clean":"2024-07-04T17:26:17.702886+0000","last_became_active":"2024-07-04T17:26:15.039334+0000","last_became_peered":"2024-07-04T17:26:15.039334+0000","last_unstale":"2024-07-04T17:26:17.702886+0000","last_undegraded":"2024-07-04T17:26:17.702886+0000","last_fullsized":"2024-07-04T17:26:17.702886+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:46.107542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.030226+0000","last_change":"2024-07-04T17:26:16.061791+0000","last_active":"2024-07-04T17:26:17.030226+0000","last_peered":"2024-07-04T17:26:17.030226+0000","last_clean":"2024-07-04T17:26:17.030226+0000","last_became_active":"2024-07-04T17:26:16.061453+0000","last_became_peered":"2024-07-04T17:26:16.061453+0000","last_unstale":"2024-07-04T17:26:17.030226+0000","last_undegraded":"2024-07-04T17:26:17.030226+0000","last_fullsized":"2024-07-04T17:26:17.030226+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:13:00.963495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'52","reported_seq":87,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026937+0000","last_change":"2024-07-04T17:26:04.953559+0000","last_active":"2024-07-04T17:26:17.026937+0000","last_peered":"2024-07-04T17:26:17.026937+0000","last_clean":"2024-07-04T17:26:17.026937+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:26:17.026937+0000","last_undegraded":"2024-07-04T17:26:17.026937+0000","last_fullsized":"2024-07-04T17:26:17.026937+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_clean_scrub_stamp":"2024-07-04T17:26:03.933432+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.064181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.027937+0000","last_change":"2024-07-04T17:26:15.026582+0000","last_active":"2024-07-04T17:26:17.027937+0000","last_peered":"2024-07-04T17:26:17.027937+0000","last_clean":"2024-07-04T17:26:17.027937+0000","last_became_active":"2024-07-04T17:26:15.026170+0000","last_became_peered":"2024-07-04T17:26:15.026170+0000","last_unstale":"2024-07-04T17:26:17.027937+0000","last_undegraded":"2024-07-04T17:26:17.027937+0000","last_fullsized":"2024-07-04T17:26:17.027937+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:45:20.253806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026496+0000","last_change":"2024-07-04T17:26:16.044579+0000","last_active":"2024-07-04T17:26:17.026496+0000","last_peered":"2024-07-04T17:26:17.026496+0000","last_clean":"2024-07-04T17:26:17.026496+0000","last_became_active":"2024-07-04T17:26:16.044255+0000","last_became_peered":"2024-07-04T17:26:16.044255+0000","last_unstale":"2024-07-04T17:26:17.026496+0000","last_undegraded":"2024-07-04T17:26:17.026496+0000","last_fullsized":"2024-07-04T17:26:17.026496+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:30:20.910427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.703153+0000","last_change":"2024-07-04T17:26:15.043650+0000","last_active":"2024-07-04T17:26:17.703153+0000","last_peered":"2024-07-04T17:26:17.703153+0000","last_clean":"2024-07-04T17:26:17.703153+0000","last_became_active":"2024-07-04T17:26:15.042392+0000","last_became_peered":"2024-07-04T17:26:15.042392+0000","last_unstale":"2024-07-04T17:26:17.703153+0000","last_undegraded":"2024-07-04T17:26:17.703153+0000","last_fullsized":"2024-07-04T17:26:17.703153+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:18:04.371426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021141+0000","last_change":"2024-07-04T17:26:16.051770+0000","last_active":"2024-07-04T17:26:17.021141+0000","last_peered":"2024-07-04T17:26:17.021141+0000","last_clean":"2024-07-04T17:26:17.021141+0000","last_became_active":"2024-07-04T17:26:16.046616+0000","last_became_peered":"2024-07-04T17:26:16.046616+0000","last_unstale":"2024-07-04T17:26:17.021141+0000","last_undegraded":"2024-07-04T17:26:17.021141+0000","last_fullsized":"2024-07-04T17:26:17.021141+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:03:12.070345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702913+0000","last_change":"2024-07-04T17:26:15.042707+0000","last_active":"2024-07-04T17:26:17.702913+0000","last_peered":"2024-07-04T17:26:17.702913+0000","last_clean":"2024-07-04T17:26:17.702913+0000","last_became_active":"2024-07-04T17:26:15.041346+0000","last_became_peered":"2024-07-04T17:26:15.041346+0000","last_unstale":"2024-07-04T17:26:17.702913+0000","last_undegraded":"2024-07-04T17:26:17.702913+0000","last_fullsized":"2024-07-04T17:26:17.702913+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:31:03.314632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026776+0000","last_change":"2024-07-04T17:26:16.039792+0000","last_active":"2024-07-04T17:26:17.026776+0000","last_peered":"2024-07-04T17:26:17.026776+0000","last_clean":"2024-07-04T17:26:17.026776+0000","last_became_active":"2024-07-04T17:26:16.039470+0000","last_became_peered":"2024-07-04T17:26:16.039470+0000","last_unstale":"2024-07-04T17:26:17.026776+0000","last_undegraded":"2024-07-04T17:26:17.026776+0000","last_fullsized":"2024-07-04T17:26:17.026776+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:08:53.606368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022753+0000","last_change":"2024-07-04T17:26:15.046123+0000","last_active":"2024-07-04T17:26:17.022753+0000","last_peered":"2024-07-04T17:26:17.022753+0000","last_clean":"2024-07-04T17:26:17.022753+0000","last_became_active":"2024-07-04T17:26:15.045824+0000","last_became_peered":"2024-07-04T17:26:15.045824+0000","last_unstale":"2024-07-04T17:26:17.022753+0000","last_undegraded":"2024-07-04T17:26:17.022753+0000","last_fullsized":"2024-07-04T17:26:17.022753+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:46.107542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021092+0000","last_change":"2024-07-04T17:26:16.044581+0000","last_active":"2024-07-04T17:26:17.021092+0000","last_peered":"2024-07-04T17:26:17.021092+0000","last_clean":"2024-07-04T17:26:17.021092+0000","last_became_active":"2024-07-04T17:26:16.043791+0000","last_became_peered":"2024-07-04T17:26:16.043791+0000","last_unstale":"2024-07-04T17:26:17.021092+0000","last_undegraded":"2024-07-04T17:26:17.021092+0000","last_fullsized":"2024-07-04T17:26:17.021092+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:38:17.647736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.029309+0000","last_change":"2024-07-04T17:26:15.032798+0000","last_active":"2024-07-04T17:26:17.029309+0000","last_peered":"2024-07-04T17:26:17.029309+0000","last_clean":"2024-07-04T17:26:17.029309+0000","last_became_active":"2024-07-04T17:26:15.029686+0000","last_became_peered":"2024-07-04T17:26:15.029686+0000","last_unstale":"2024-07-04T17:26:17.029309+0000","last_undegraded":"2024-07-04T17:26:17.029309+0000","last_fullsized":"2024-07-04T17:26:17.029309+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:25:38.473912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702222+0000","last_change":"2024-07-04T17:26:16.062731+0000","last_active":"2024-07-04T17:26:17.702222+0000","last_peered":"2024-07-04T17:26:17.702222+0000","last_clean":"2024-07-04T17:26:17.702222+0000","last_became_active":"2024-07-04T17:26:16.062519+0000","last_became_peered":"2024-07-04T17:26:16.062519+0000","last_unstale":"2024-07-04T17:26:17.702222+0000","last_undegraded":"2024-07-04T17:26:17.702222+0000","last_fullsized":"2024-07-04T17:26:17.702222+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:30:20.910427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.029919+0000","last_change":"2024-07-04T17:26:15.036880+0000","last_active":"2024-07-04T17:26:17.029919+0000","last_peered":"2024-07-04T17:26:17.029919+0000","last_clean":"2024-07-04T17:26:17.029919+0000","last_became_active":"2024-07-04T17:26:15.036529+0000","last_became_peered":"2024-07-04T17:26:15.036529+0000","last_unstale":"2024-07-04T17:26:17.029919+0000","last_undegraded":"2024-07-04T17:26:17.029919+0000","last_fullsized":"2024-07-04T17:26:17.029919+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:04:46.667644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.029152+0000","last_change":"2024-07-04T17:26:16.046186+0000","last_active":"2024-07-04T17:26:17.029152+0000","last_peered":"2024-07-04T17:26:17.029152+0000","last_clean":"2024-07-04T17:26:17.029152+0000","last_became_active":"2024-07-04T17:26:16.044588+0000","last_became_peered":"2024-07-04T17:26:16.044588+0000","last_unstale":"2024-07-04T17:26:17.029152+0000","last_undegraded":"2024-07-04T17:26:17.029152+0000","last_fullsized":"2024-07-04T17:26:17.029152+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:50:25.908300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"15'1","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.030544+0000","last_change":"2024-07-04T17:26:15.022516+0000","last_active":"2024-07-04T17:26:17.030544+0000","last_peered":"2024-07-04T17:26:17.030544+0000","last_clean":"2024-07-04T17:26:17.030544+0000","last_became_active":"2024-07-04T17:26:15.022185+0000","last_became_peered":"2024-07-04T17:26:15.022185+0000","last_unstale":"2024-07-04T17:26:17.030544+0000","last_undegraded":"2024-07-04T17:26:17.030544+0000","last_fullsized":"2024-07-04T17:26:17.030544+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:24:04.318931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702984+0000","last_change":"2024-07-04T17:26:16.059869+0000","last_active":"2024-07-04T17:26:17.702984+0000","last_peered":"2024-07-04T17:26:17.702984+0000","last_clean":"2024-07-04T17:26:17.702984+0000","last_became_active":"2024-07-04T17:26:16.059535+0000","last_became_peered":"2024-07-04T17:26:16.059535+0000","last_unstale":"2024-07-04T17:26:17.702984+0000","last_undegraded":"2024-07-04T17:26:17.702984+0000","last_fullsized":"2024-07-04T17:26:17.702984+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:43:29.877017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"14'2","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.029845+0000","last_change":"2024-07-04T17:26:15.040384+0000","last_active":"2024-07-04T17:26:17.029845+0000","last_peered":"2024-07-04T17:26:17.029845+0000","last_clean":"2024-07-04T17:26:17.029845+0000","last_became_active":"2024-07-04T17:26:15.038959+0000","last_became_peered":"2024-07-04T17:26:15.038959+0000","last_unstale":"2024-07-04T17:26:17.029845+0000","last_undegraded":"2024-07-04T17:26:17.029845+0000","last_fullsized":"2024-07-04T17:26:17.029845+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:18:52.190216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.023117+0000","last_change":"2024-07-04T17:26:16.053639+0000","last_active":"2024-07-04T17:26:17.023117+0000","last_peered":"2024-07-04T17:26:17.023117+0000","last_clean":"2024-07-04T17:26:17.023117+0000","last_became_active":"2024-07-04T17:26:16.052766+0000","last_became_peered":"2024-07-04T17:26:16.052766+0000","last_unstale":"2024-07-04T17:26:17.023117+0000","last_undegraded":"2024-07-04T17:26:17.023117+0000","last_fullsized":"2024-07-04T17:26:17.023117+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:22:12.063177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020302+0000","last_change":"2024-07-04T17:26:15.027718+0000","last_active":"2024-07-04T17:26:17.020302+0000","last_peered":"2024-07-04T17:26:17.020302+0000","last_clean":"2024-07-04T17:26:17.020302+0000","last_became_active":"2024-07-04T17:26:15.026662+0000","last_became_peered":"2024-07-04T17:26:15.026662+0000","last_unstale":"2024-07-04T17:26:17.020302+0000","last_undegraded":"2024-07-04T17:26:17.020302+0000","last_fullsized":"2024-07-04T17:26:17.020302+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:45:20.253806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020487+0000","last_change":"2024-07-04T17:26:16.066686+0000","last_active":"2024-07-04T17:26:17.020487+0000","last_peered":"2024-07-04T17:26:17.020487+0000","last_clean":"2024-07-04T17:26:17.020487+0000","last_became_active":"2024-07-04T17:26:16.066397+0000","last_became_peered":"2024-07-04T17:26:16.066397+0000","last_unstale":"2024-07-04T17:26:17.020487+0000","last_undegraded":"2024-07-04T17:26:17.020487+0000","last_fullsized":"2024-07-04T17:26:17.020487+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:21:38.693638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026169+0000","last_change":"2024-07-04T17:26:15.034670+0000","last_active":"2024-07-04T17:26:17.026169+0000","last_peered":"2024-07-04T17:26:17.026169+0000","last_clean":"2024-07-04T17:26:17.026169+0000","last_became_active":"2024-07-04T17:26:15.034267+0000","last_became_peered":"2024-07-04T17:26:15.034267+0000","last_unstale":"2024-07-04T17:26:17.026169+0000","last_undegraded":"2024-07-04T17:26:17.026169+0000","last_fullsized":"2024-07-04T17:26:17.026169+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:46.107542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702508+0000","last_change":"2024-07-04T17:26:16.054400+0000","last_active":"2024-07-04T17:26:17.702508+0000","last_peered":"2024-07-04T17:26:17.702508+0000","last_clean":"2024-07-04T17:26:17.702508+0000","last_became_active":"2024-07-04T17:26:16.054057+0000","last_became_peered":"2024-07-04T17:26:16.054057+0000","last_unstale":"2024-07-04T17:26:17.702508+0000","last_undegraded":"2024-07-04T17:26:17.702508+0000","last_fullsized":"2024-07-04T17:26:17.702508+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:04:04.805872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026719+0000","last_change":"2024-07-04T17:26:15.032894+0000","last_active":"2024-07-04T17:26:17.026719+0000","last_peered":"2024-07-04T17:26:17.026719+0000","last_clean":"2024-07-04T17:26:17.026719+0000","last_became_active":"2024-07-04T17:26:15.032634+0000","last_became_peered":"2024-07-04T17:26:15.032634+0000","last_unstale":"2024-07-04T17:26:17.026719+0000","last_undegraded":"2024-07-04T17:26:17.026719+0000","last_fullsized":"2024-07-04T17:26:17.026719+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:05:39.042848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026622+0000","last_change":"2024-07-04T17:26:16.065536+0000","last_active":"2024-07-04T17:26:17.026622+0000","last_peered":"2024-07-04T17:26:17.026622+0000","last_clean":"2024-07-04T17:26:17.026622+0000","last_became_active":"2024-07-04T17:26:16.056278+0000","last_became_peered":"2024-07-04T17:26:16.056278+0000","last_unstale":"2024-07-04T17:26:17.026622+0000","last_undegraded":"2024-07-04T17:26:17.026622+0000","last_fullsized":"2024-07-04T17:26:17.026622+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:03:07.909265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"14'2","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021818+0000","last_change":"2024-07-04T17:26:15.033912+0000","last_active":"2024-07-04T17:26:17.021818+0000","last_peered":"2024-07-04T17:26:17.021818+0000","last_clean":"2024-07-04T17:26:17.021818+0000","last_became_active":"2024-07-04T17:26:15.027824+0000","last_became_peered":"2024-07-04T17:26:15.027824+0000","last_unstale":"2024-07-04T17:26:17.021818+0000","last_undegraded":"2024-07-04T17:26:17.021818+0000","last_fullsized":"2024-07-04T17:26:17.021818+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:34:46.876523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020817+0000","last_change":"2024-07-04T17:26:16.065218+0000","last_active":"2024-07-04T17:26:17.020817+0000","last_peered":"2024-07-04T17:26:17.020817+0000","last_clean":"2024-07-04T17:26:17.020817+0000","last_became_active":"2024-07-04T17:26:16.064904+0000","last_became_peered":"2024-07-04T17:26:16.064904+0000","last_unstale":"2024-07-04T17:26:17.020817+0000","last_undegraded":"2024-07-04T17:26:17.020817+0000","last_fullsized":"2024-07-04T17:26:17.020817+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:33:23.722640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702115+0000","last_change":"2024-07-04T17:26:16.073564+0000","last_active":"2024-07-04T17:26:17.702115+0000","last_peered":"2024-07-04T17:26:17.702115+0000","last_clean":"2024-07-04T17:26:17.702115+0000","last_became_active":"2024-07-04T17:26:16.073189+0000","last_became_peered":"2024-07-04T17:26:16.073189+0000","last_unstale":"2024-07-04T17:26:17.702115+0000","last_undegraded":"2024-07-04T17:26:17.702115+0000","last_fullsized":"2024-07-04T17:26:17.702115+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:21:48.005606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"14'2","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022583+0000","last_change":"2024-07-04T17:26:15.049001+0000","last_active":"2024-07-04T17:26:17.022583+0000","last_peered":"2024-07-04T17:26:17.022583+0000","last_clean":"2024-07-04T17:26:17.022583+0000","last_became_active":"2024-07-04T17:26:15.048437+0000","last_became_peered":"2024-07-04T17:26:15.048437+0000","last_unstale":"2024-07-04T17:26:17.022583+0000","last_undegraded":"2024-07-04T17:26:17.022583+0000","last_fullsized":"2024-07-04T17:26:17.022583+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:15:08.055102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.028692+0000","last_change":"2024-07-04T17:26:16.064782+0000","last_active":"2024-07-04T17:26:17.028692+0000","last_peered":"2024-07-04T17:26:17.028692+0000","last_clean":"2024-07-04T17:26:17.028692+0000","last_became_active":"2024-07-04T17:26:16.064518+0000","last_became_peered":"2024-07-04T17:26:16.064518+0000","last_unstale":"2024-07-04T17:26:17.028692+0000","last_undegraded":"2024-07-04T17:26:17.028692+0000","last_fullsized":"2024-07-04T17:26:17.028692+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:01:31.824881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"14'2","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022678+0000","last_change":"2024-07-04T17:26:15.047259+0000","last_active":"2024-07-04T17:26:17.022678+0000","last_peered":"2024-07-04T17:26:17.022678+0000","last_clean":"2024-07-04T17:26:17.022678+0000","last_became_active":"2024-07-04T17:26:15.046951+0000","last_became_peered":"2024-07-04T17:26:15.046951+0000","last_unstale":"2024-07-04T17:26:17.022678+0000","last_undegraded":"2024-07-04T17:26:17.022678+0000","last_fullsized":"2024-07-04T17:26:17.022678+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:04:46.667644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.024493+0000","last_change":"2024-07-04T17:26:16.058196+0000","last_active":"2024-07-04T17:26:17.024493+0000","last_peered":"2024-07-04T17:26:17.024493+0000","last_clean":"2024-07-04T17:26:17.024493+0000","last_became_active":"2024-07-04T17:26:16.051038+0000","last_became_peered":"2024-07-04T17:26:16.051038+0000","last_unstale":"2024-07-04T17:26:17.024493+0000","last_undegraded":"2024-07-04T17:26:17.024493+0000","last_fullsized":"2024-07-04T17:26:17.024493+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:04:50.688662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"14'1","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.704046+0000","last_change":"2024-07-04T17:26:15.040464+0000","last_active":"2024-07-04T17:26:17.704046+0000","last_peered":"2024-07-04T17:26:17.704046+0000","last_clean":"2024-07-04T17:26:17.704046+0000","last_became_active":"2024-07-04T17:26:15.040008+0000","last_became_peered":"2024-07-04T17:26:15.040008+0000","last_unstale":"2024-07-04T17:26:17.704046+0000","last_undegraded":"2024-07-04T17:26:17.704046+0000","last_fullsized":"2024-07-04T17:26:17.704046+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:34:46.876523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.703450+0000","last_change":"2024-07-04T17:26:16.060159+0000","last_active":"2024-07-04T17:26:17.703450+0000","last_peered":"2024-07-04T17:26:17.703450+0000","last_clean":"2024-07-04T17:26:17.703450+0000","last_became_active":"2024-07-04T17:26:16.059458+0000","last_became_peered":"2024-07-04T17:26:16.059458+0000","last_unstale":"2024-07-04T17:26:17.703450+0000","last_undegraded":"2024-07-04T17:26:17.703450+0000","last_fullsized":"2024-07-04T17:26:17.703450+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:01:31.824881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"14'4","reported_seq":18,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.027492+0000","last_change":"2024-07-04T17:26:15.049639+0000","last_active":"2024-07-04T17:26:17.027492+0000","last_peered":"2024-07-04T17:26:17.027492+0000","last_clean":"2024-07-04T17:26:17.027492+0000","last_became_active":"2024-07-04T17:26:15.048590+0000","last_became_peered":"2024-07-04T17:26:15.048590+0000","last_unstale":"2024-07-04T17:26:17.027492+0000","last_undegraded":"2024-07-04T17:26:17.027492+0000","last_fullsized":"2024-07-04T17:26:17.027492+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:05:39.042848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738375,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31332,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340508,"statfs":{"total":96636764160,"available":96604680192,"internally_reserved":0,"allocated":151552,"data_stored":42085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738376,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31920,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339920,"statfs":{"total":96636764160,"available":96604078080,"internally_reserved":0,"allocated":753664,"data_stored":633113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738374,"num_pgs":74,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31912,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339928,"statfs":{"total":96636764160,"available":96604086272,"internally_reserved":0,"allocated":741376,"data_stored":631465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:26:28.158 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:26:28.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.483+0000 7f31bd600640 1 -- 172.21.15.42:0/4010930362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31b80b17d0 msgr2=0x7f31b80b1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:28.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.483+0000 7f31bd600640 1 --2- 172.21.15.42:0/4010930362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31b80b17d0 0x7f31b80b1be0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f31a00097b0 tx=0x7f31a002a4b0 comp rx=0 tx=0).stop 2024-07-04T17:26:28.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.483+0000 7f31bd600640 1 -- 172.21.15.42:0/4010930362 shutdown_connections 2024-07-04T17:26:28.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.483+0000 7f31bd600640 1 --2- 172.21.15.42:0/4010930362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31b80b17d0 0x7f31b80b1be0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:28.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.483+0000 7f31bd600640 1 -- 172.21.15.42:0/4010930362 >> 172.21.15.42:0/4010930362 conn(0x7f31b80bebc0 msgr2=0x7f31b80befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:28.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.483+0000 7f31bd600640 1 -- 172.21.15.42:0/4010930362 shutdown_connections 2024-07-04T17:26:28.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.484+0000 7f31bd600640 1 -- 172.21.15.42:0/4010930362 wait complete. 2024-07-04T17:26:28.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.485+0000 7f31bd600640 1 Processor -- start 2024-07-04T17:26:28.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.486+0000 7f31bd600640 1 -- start start 2024-07-04T17:26:28.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.487+0000 7f31bd600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31b806c200 0x7f31b81d4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:28.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.487+0000 7f31bd600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f31b814ae80 con 0x7f31b806c200 2024-07-04T17:26:28.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.487+0000 7f31b7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31b806c200 0x7f31b81d4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:28.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.487+0000 7f31b7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31b806c200 0x7f31b81d4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44158/0 (socket says 172.21.15.42:44158) 2024-07-04T17:26:28.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.487+0000 7f31b7e00640 1 -- 172.21.15.42:0/2607120939 learned_addr learned my addr 172.21.15.42:0/2607120939 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:28.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.487+0000 7f31b7e00640 1 -- 172.21.15.42:0/2607120939 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b81d6680 con 0x7f31b806c200 2024-07-04T17:26:28.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.488+0000 7f31b7e00640 1 --2- 172.21.15.42:0/2607120939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31b806c200 0x7f31b81d4bf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f31a800c380 tx=0x7f31a800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:28.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.488+0000 7f31b4c00640 1 -- 172.21.15.42:0/2607120939 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31a80116c0 con 0x7f31b806c200 2024-07-04T17:26:28.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.488+0000 7f31b4c00640 1 -- 172.21.15.42:0/2607120939 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31a8017070 con 0x7f31b806c200 2024-07-04T17:26:28.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.488+0000 7f31b4c00640 1 -- 172.21.15.42:0/2607120939 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31a8011b20 con 0x7f31b806c200 2024-07-04T17:26:28.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.488+0000 7f31bd600640 1 -- 172.21.15.42:0/2607120939 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31b8142540 con 0x7f31b806c200 2024-07-04T17:26:28.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.488+0000 7f31bd600640 1 -- 172.21.15.42:0/2607120939 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31b81d5e70 con 0x7f31b806c200 2024-07-04T17:26:28.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.489+0000 7f31bd600640 1 -- 172.21.15.42:0/2607120939 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31b814b4c0 con 0x7f31b806c200 2024-07-04T17:26:28.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.489+0000 7f31b4c00640 1 -- 172.21.15.42:0/2607120939 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f31a8011cc0 con 0x7f31b806c200 2024-07-04T17:26:28.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.490+0000 7f31b4c00640 1 --2- 172.21.15.42:0/2607120939 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f319403d2b0 0x7f319403f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:28.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.490+0000 7f31b4c00640 1 -- 172.21.15.42:0/2607120939 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f31a8051340 con 0x7f31b806c200 2024-07-04T17:26:28.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.490+0000 7f31afe00640 1 --2- 172.21.15.42:0/2607120939 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f319403d2b0 0x7f319403f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:28.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.491+0000 7f31afe00640 1 --2- 172.21.15.42:0/2607120939 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f319403d2b0 0x7f319403f7c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f31a0034000 tx=0x7f31a0004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:28.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.495+0000 7f31b4c00640 1 -- 172.21.15.42:0/2607120939 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f31a8010cf0 con 0x7f31b806c200 2024-07-04T17:26:28.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.700+0000 7f31bd600640 1 -- 172.21.15.42:0/2607120939 --> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f31b8142770 con 0x7f319403d2b0 2024-07-04T17:26:28.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.714+0000 7f31b4c00640 1 -- 172.21.15.42:0/2607120939 <== mgr.4103 v2:172.21.15.42:6824/3523793950 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245906 (secure 0 0 0) 0x7f31b8142770 con 0x7f319403d2b0 2024-07-04T17:26:28.719 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:26:28.720 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:26:28.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.720+0000 7f31bd600640 1 -- 172.21.15.42:0/2607120939 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f319403d2b0 msgr2=0x7f319403f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:28.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.720+0000 7f31bd600640 1 --2- 172.21.15.42:0/2607120939 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f319403d2b0 0x7f319403f7c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f31a0034000 tx=0x7f31a0004730 comp rx=0 tx=0).stop 2024-07-04T17:26:28.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.720+0000 7f31bd600640 1 -- 172.21.15.42:0/2607120939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31b806c200 msgr2=0x7f31b81d4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:28.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.720+0000 7f31bd600640 1 --2- 172.21.15.42:0/2607120939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31b806c200 0x7f31b81d4bf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f31a800c380 tx=0x7f31a800c840 comp rx=0 tx=0).stop 2024-07-04T17:26:28.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.720+0000 7f31bd600640 1 -- 172.21.15.42:0/2607120939 shutdown_connections 2024-07-04T17:26:28.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.720+0000 7f31bd600640 1 --2- 172.21.15.42:0/2607120939 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f319403d2b0 0x7f319403f7c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:28.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.720+0000 7f31bd600640 1 --2- 172.21.15.42:0/2607120939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31b806c200 0x7f31b81d4bf0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:28.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.720+0000 7f31bd600640 1 -- 172.21.15.42:0/2607120939 >> 172.21.15.42:0/2607120939 conn(0x7f31b80bebc0 msgr2=0x7f31b81d72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:28.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.720+0000 7f31bd600640 1 -- 172.21.15.42:0/2607120939 shutdown_connections 2024-07-04T17:26:28.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:28.720+0000 7f31bd600640 1 -- 172.21.15.42:0/2607120939 wait complete. 2024-07-04T17:26:28.750 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-07-04T17:26:27.266101+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":124,"num_write_kb":1120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91,"ondisk_log_size":91,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":95164,"kb_used_data":1608,"kb_used_omap":4,"kb_used_meta":93499,"kb_avail":283020356,"statfs":{"total":289910292480,"available":289812844544,"internally_reserved":0,"allocated":1646592,"data_stored":1306663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":95743311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.248494"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.023948+0000","last_change":"2024-07-04T17:26:16.062462+0000","last_active":"2024-07-04T17:26:17.023948+0000","last_peered":"2024-07-04T17:26:17.023948+0000","last_clean":"2024-07-04T17:26:17.023948+0000","last_became_active":"2024-07-04T17:26:16.058531+0000","last_became_peered":"2024-07-04T17:26:16.058531+0000","last_unstale":"2024-07-04T17:26:17.023948+0000","last_undegraded":"2024-07-04T17:26:17.023948+0000","last_fullsized":"2024-07-04T17:26:17.023948+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:31:33.921497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.028401+0000","last_change":"2024-07-04T17:26:16.064253+0000","last_active":"2024-07-04T17:26:17.028401+0000","last_peered":"2024-07-04T17:26:17.028401+0000","last_clean":"2024-07-04T17:26:17.028401+0000","last_became_active":"2024-07-04T17:26:16.063971+0000","last_became_peered":"2024-07-04T17:26:16.063971+0000","last_unstale":"2024-07-04T17:26:17.028401+0000","last_undegraded":"2024-07-04T17:26:17.028401+0000","last_fullsized":"2024-07-04T17:26:17.028401+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:22:14.191409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022213+0000","last_change":"2024-07-04T17:26:16.059637+0000","last_active":"2024-07-04T17:26:17.022213+0000","last_peered":"2024-07-04T17:26:17.022213+0000","last_clean":"2024-07-04T17:26:17.022213+0000","last_became_active":"2024-07-04T17:26:16.058613+0000","last_became_peered":"2024-07-04T17:26:16.058613+0000","last_unstale":"2024-07-04T17:26:17.022213+0000","last_undegraded":"2024-07-04T17:26:17.022213+0000","last_fullsized":"2024-07-04T17:26:17.022213+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:02:40.668757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702530+0000","last_change":"2024-07-04T17:26:16.072904+0000","last_active":"2024-07-04T17:26:17.702530+0000","last_peered":"2024-07-04T17:26:17.702530+0000","last_clean":"2024-07-04T17:26:17.702530+0000","last_became_active":"2024-07-04T17:26:16.072640+0000","last_became_peered":"2024-07-04T17:26:16.072640+0000","last_unstale":"2024-07-04T17:26:17.702530+0000","last_undegraded":"2024-07-04T17:26:17.702530+0000","last_fullsized":"2024-07-04T17:26:17.702530+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:03:07.909265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021289+0000","last_change":"2024-07-04T17:26:16.044297+0000","last_active":"2024-07-04T17:26:17.021289+0000","last_peered":"2024-07-04T17:26:17.021289+0000","last_clean":"2024-07-04T17:26:17.021289+0000","last_became_active":"2024-07-04T17:26:16.043526+0000","last_became_peered":"2024-07-04T17:26:16.043526+0000","last_unstale":"2024-07-04T17:26:17.021289+0000","last_undegraded":"2024-07-04T17:26:17.021289+0000","last_fullsized":"2024-07-04T17:26:17.021289+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:11:11.392035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702192+0000","last_change":"2024-07-04T17:26:16.072472+0000","last_active":"2024-07-04T17:26:17.702192+0000","last_peered":"2024-07-04T17:26:17.702192+0000","last_clean":"2024-07-04T17:26:17.702192+0000","last_became_active":"2024-07-04T17:26:16.072066+0000","last_became_peered":"2024-07-04T17:26:16.072066+0000","last_unstale":"2024-07-04T17:26:17.702192+0000","last_undegraded":"2024-07-04T17:26:17.702192+0000","last_fullsized":"2024-07-04T17:26:17.702192+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:11:17.509522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021458+0000","last_change":"2024-07-04T17:26:16.070453+0000","last_active":"2024-07-04T17:26:17.021458+0000","last_peered":"2024-07-04T17:26:17.021458+0000","last_clean":"2024-07-04T17:26:17.021458+0000","last_became_active":"2024-07-04T17:26:16.065958+0000","last_became_peered":"2024-07-04T17:26:16.065958+0000","last_unstale":"2024-07-04T17:26:17.021458+0000","last_undegraded":"2024-07-04T17:26:17.021458+0000","last_fullsized":"2024-07-04T17:26:17.021458+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:51:59.570861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.700790+0000","last_change":"2024-07-04T17:26:16.072492+0000","last_active":"2024-07-04T17:26:17.700790+0000","last_peered":"2024-07-04T17:26:17.700790+0000","last_clean":"2024-07-04T17:26:17.700790+0000","last_became_active":"2024-07-04T17:26:16.067345+0000","last_became_peered":"2024-07-04T17:26:16.067345+0000","last_unstale":"2024-07-04T17:26:17.700790+0000","last_undegraded":"2024-07-04T17:26:17.700790+0000","last_fullsized":"2024-07-04T17:26:17.700790+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:02:12.965714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.028868+0000","last_change":"2024-07-04T17:26:16.055031+0000","last_active":"2024-07-04T17:26:17.028868+0000","last_peered":"2024-07-04T17:26:17.028868+0000","last_clean":"2024-07-04T17:26:17.028868+0000","last_became_active":"2024-07-04T17:26:16.051647+0000","last_became_peered":"2024-07-04T17:26:16.051647+0000","last_unstale":"2024-07-04T17:26:17.028868+0000","last_undegraded":"2024-07-04T17:26:17.028868+0000","last_fullsized":"2024-07-04T17:26:17.028868+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:41:20.224285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022523+0000","last_change":"2024-07-04T17:26:16.045710+0000","last_active":"2024-07-04T17:26:17.022523+0000","last_peered":"2024-07-04T17:26:17.022523+0000","last_clean":"2024-07-04T17:26:17.022523+0000","last_became_active":"2024-07-04T17:26:16.045371+0000","last_became_peered":"2024-07-04T17:26:16.045371+0000","last_unstale":"2024-07-04T17:26:17.022523+0000","last_undegraded":"2024-07-04T17:26:17.022523+0000","last_fullsized":"2024-07-04T17:26:17.022523+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:53:44.096508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022006+0000","last_change":"2024-07-04T17:26:16.059936+0000","last_active":"2024-07-04T17:26:17.022006+0000","last_peered":"2024-07-04T17:26:17.022006+0000","last_clean":"2024-07-04T17:26:17.022006+0000","last_became_active":"2024-07-04T17:26:16.058990+0000","last_became_peered":"2024-07-04T17:26:16.058990+0000","last_unstale":"2024-07-04T17:26:17.022006+0000","last_undegraded":"2024-07-04T17:26:17.022006+0000","last_fullsized":"2024-07-04T17:26:17.022006+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:47:09.376866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022753+0000","last_change":"2024-07-04T17:26:16.053390+0000","last_active":"2024-07-04T17:26:17.022753+0000","last_peered":"2024-07-04T17:26:17.022753+0000","last_clean":"2024-07-04T17:26:17.022753+0000","last_became_active":"2024-07-04T17:26:16.052716+0000","last_became_peered":"2024-07-04T17:26:16.052716+0000","last_unstale":"2024-07-04T17:26:17.022753+0000","last_undegraded":"2024-07-04T17:26:17.022753+0000","last_fullsized":"2024-07-04T17:26:17.022753+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:15:52.371580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021947+0000","last_change":"2024-07-04T17:26:16.037676+0000","last_active":"2024-07-04T17:26:17.021947+0000","last_peered":"2024-07-04T17:26:17.021947+0000","last_clean":"2024-07-04T17:26:17.021947+0000","last_became_active":"2024-07-04T17:26:16.037311+0000","last_became_peered":"2024-07-04T17:26:16.037311+0000","last_unstale":"2024-07-04T17:26:17.021947+0000","last_undegraded":"2024-07-04T17:26:17.021947+0000","last_fullsized":"2024-07-04T17:26:17.021947+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:31:49.427076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020967+0000","last_change":"2024-07-04T17:26:16.064558+0000","last_active":"2024-07-04T17:26:17.020967+0000","last_peered":"2024-07-04T17:26:17.020967+0000","last_clean":"2024-07-04T17:26:17.020967+0000","last_became_active":"2024-07-04T17:26:16.054068+0000","last_became_peered":"2024-07-04T17:26:16.054068+0000","last_unstale":"2024-07-04T17:26:17.020967+0000","last_undegraded":"2024-07-04T17:26:17.020967+0000","last_fullsized":"2024-07-04T17:26:17.020967+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:30:12.289936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026054+0000","last_change":"2024-07-04T17:26:16.056551+0000","last_active":"2024-07-04T17:26:17.026054+0000","last_peered":"2024-07-04T17:26:17.026054+0000","last_clean":"2024-07-04T17:26:17.026054+0000","last_became_active":"2024-07-04T17:26:16.056173+0000","last_became_peered":"2024-07-04T17:26:16.056173+0000","last_unstale":"2024-07-04T17:26:17.026054+0000","last_undegraded":"2024-07-04T17:26:17.026054+0000","last_fullsized":"2024-07-04T17:26:17.026054+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:20:14.576761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.700980+0000","last_change":"2024-07-04T17:26:16.073053+0000","last_active":"2024-07-04T17:26:17.700980+0000","last_peered":"2024-07-04T17:26:17.700980+0000","last_clean":"2024-07-04T17:26:17.700980+0000","last_became_active":"2024-07-04T17:26:16.072132+0000","last_became_peered":"2024-07-04T17:26:16.072132+0000","last_unstale":"2024-07-04T17:26:17.700980+0000","last_undegraded":"2024-07-04T17:26:17.700980+0000","last_fullsized":"2024-07-04T17:26:17.700980+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:11:05.657045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.029516+0000","last_change":"2024-07-04T17:26:16.055789+0000","last_active":"2024-07-04T17:26:17.029516+0000","last_peered":"2024-07-04T17:26:17.029516+0000","last_clean":"2024-07-04T17:26:17.029516+0000","last_became_active":"2024-07-04T17:26:16.055206+0000","last_became_peered":"2024-07-04T17:26:16.055206+0000","last_unstale":"2024-07-04T17:26:17.029516+0000","last_undegraded":"2024-07-04T17:26:17.029516+0000","last_fullsized":"2024-07-04T17:26:17.029516+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:31:59.564668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.703184+0000","last_change":"2024-07-04T17:26:16.070972+0000","last_active":"2024-07-04T17:26:17.703184+0000","last_peered":"2024-07-04T17:26:17.703184+0000","last_clean":"2024-07-04T17:26:17.703184+0000","last_became_active":"2024-07-04T17:26:16.067224+0000","last_became_peered":"2024-07-04T17:26:16.067224+0000","last_unstale":"2024-07-04T17:26:17.703184+0000","last_undegraded":"2024-07-04T17:26:17.703184+0000","last_fullsized":"2024-07-04T17:26:17.703184+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:22:14.191409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022486+0000","last_change":"2024-07-04T17:26:16.059368+0000","last_active":"2024-07-04T17:26:17.022486+0000","last_peered":"2024-07-04T17:26:17.022486+0000","last_clean":"2024-07-04T17:26:17.022486+0000","last_became_active":"2024-07-04T17:26:16.058568+0000","last_became_peered":"2024-07-04T17:26:16.058568+0000","last_unstale":"2024-07-04T17:26:17.022486+0000","last_undegraded":"2024-07-04T17:26:17.022486+0000","last_fullsized":"2024-07-04T17:26:17.022486+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:18:06.133593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022008+0000","last_change":"2024-07-04T17:26:16.083687+0000","last_active":"2024-07-04T17:26:17.022008+0000","last_peered":"2024-07-04T17:26:17.022008+0000","last_clean":"2024-07-04T17:26:17.022008+0000","last_became_active":"2024-07-04T17:26:16.083145+0000","last_became_peered":"2024-07-04T17:26:16.083145+0000","last_unstale":"2024-07-04T17:26:17.022008+0000","last_undegraded":"2024-07-04T17:26:17.022008+0000","last_fullsized":"2024-07-04T17:26:17.022008+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:36:50.247043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020646+0000","last_change":"2024-07-04T17:26:16.047023+0000","last_active":"2024-07-04T17:26:17.020646+0000","last_peered":"2024-07-04T17:26:17.020646+0000","last_clean":"2024-07-04T17:26:17.020646+0000","last_became_active":"2024-07-04T17:26:16.046631+0000","last_became_peered":"2024-07-04T17:26:16.046631+0000","last_unstale":"2024-07-04T17:26:17.020646+0000","last_undegraded":"2024-07-04T17:26:17.020646+0000","last_fullsized":"2024-07-04T17:26:17.020646+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:13:00.963495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026306+0000","last_change":"2024-07-04T17:26:16.075119+0000","last_active":"2024-07-04T17:26:17.026306+0000","last_peered":"2024-07-04T17:26:17.026306+0000","last_clean":"2024-07-04T17:26:17.026306+0000","last_became_active":"2024-07-04T17:26:16.074600+0000","last_became_peered":"2024-07-04T17:26:16.074600+0000","last_unstale":"2024-07-04T17:26:17.026306+0000","last_undegraded":"2024-07-04T17:26:17.026306+0000","last_fullsized":"2024-07-04T17:26:17.026306+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:17:18.408614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021060+0000","last_change":"2024-07-04T17:26:16.052698+0000","last_active":"2024-07-04T17:26:17.021060+0000","last_peered":"2024-07-04T17:26:17.021060+0000","last_clean":"2024-07-04T17:26:17.021060+0000","last_became_active":"2024-07-04T17:26:16.052278+0000","last_became_peered":"2024-07-04T17:26:16.052278+0000","last_unstale":"2024-07-04T17:26:17.021060+0000","last_undegraded":"2024-07-04T17:26:17.021060+0000","last_fullsized":"2024-07-04T17:26:17.021060+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:27:20.732374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020541+0000","last_change":"2024-07-04T17:26:16.052254+0000","last_active":"2024-07-04T17:26:17.020541+0000","last_peered":"2024-07-04T17:26:17.020541+0000","last_clean":"2024-07-04T17:26:17.020541+0000","last_became_active":"2024-07-04T17:26:16.046560+0000","last_became_peered":"2024-07-04T17:26:16.046560+0000","last_unstale":"2024-07-04T17:26:17.020541+0000","last_undegraded":"2024-07-04T17:26:17.020541+0000","last_fullsized":"2024-07-04T17:26:17.020541+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:50:25.908300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702155+0000","last_change":"2024-07-04T17:26:16.061658+0000","last_active":"2024-07-04T17:26:17.702155+0000","last_peered":"2024-07-04T17:26:17.702155+0000","last_clean":"2024-07-04T17:26:17.702155+0000","last_became_active":"2024-07-04T17:26:16.061257+0000","last_became_peered":"2024-07-04T17:26:16.061257+0000","last_unstale":"2024-07-04T17:26:17.702155+0000","last_undegraded":"2024-07-04T17:26:17.702155+0000","last_fullsized":"2024-07-04T17:26:17.702155+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:17:18.408614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.703496+0000","last_change":"2024-07-04T17:26:16.073828+0000","last_active":"2024-07-04T17:26:17.703496+0000","last_peered":"2024-07-04T17:26:17.703496+0000","last_clean":"2024-07-04T17:26:17.703496+0000","last_became_active":"2024-07-04T17:26:16.073494+0000","last_became_peered":"2024-07-04T17:26:16.073494+0000","last_unstale":"2024-07-04T17:26:17.703496+0000","last_undegraded":"2024-07-04T17:26:17.703496+0000","last_fullsized":"2024-07-04T17:26:17.703496+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:41:20.224285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.028237+0000","last_change":"2024-07-04T17:26:16.046050+0000","last_active":"2024-07-04T17:26:17.028237+0000","last_peered":"2024-07-04T17:26:17.028237+0000","last_clean":"2024-07-04T17:26:17.028237+0000","last_became_active":"2024-07-04T17:26:16.043726+0000","last_became_peered":"2024-07-04T17:26:16.043726+0000","last_unstale":"2024-07-04T17:26:17.028237+0000","last_undegraded":"2024-07-04T17:26:17.028237+0000","last_fullsized":"2024-07-04T17:26:17.028237+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:43:29.877017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022171+0000","last_change":"2024-07-04T17:26:16.047247+0000","last_active":"2024-07-04T17:26:17.022171+0000","last_peered":"2024-07-04T17:26:17.022171+0000","last_clean":"2024-07-04T17:26:17.022171+0000","last_became_active":"2024-07-04T17:26:16.046845+0000","last_became_peered":"2024-07-04T17:26:16.046845+0000","last_unstale":"2024-07-04T17:26:17.022171+0000","last_undegraded":"2024-07-04T17:26:17.022171+0000","last_fullsized":"2024-07-04T17:26:17.022171+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:22:33.108870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020889+0000","last_change":"2024-07-04T17:26:16.047621+0000","last_active":"2024-07-04T17:26:17.020889+0000","last_peered":"2024-07-04T17:26:17.020889+0000","last_clean":"2024-07-04T17:26:17.020889+0000","last_became_active":"2024-07-04T17:26:16.047328+0000","last_became_peered":"2024-07-04T17:26:16.047328+0000","last_unstale":"2024-07-04T17:26:17.020889+0000","last_undegraded":"2024-07-04T17:26:17.020889+0000","last_fullsized":"2024-07-04T17:26:17.020889+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:03:58.364101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020557+0000","last_change":"2024-07-04T17:26:16.046513+0000","last_active":"2024-07-04T17:26:17.020557+0000","last_peered":"2024-07-04T17:26:17.020557+0000","last_clean":"2024-07-04T17:26:17.020557+0000","last_became_active":"2024-07-04T17:26:16.045399+0000","last_became_peered":"2024-07-04T17:26:16.045399+0000","last_unstale":"2024-07-04T17:26:17.020557+0000","last_undegraded":"2024-07-04T17:26:17.020557+0000","last_fullsized":"2024-07-04T17:26:17.020557+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:31:59.564668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.701258+0000","last_change":"2024-07-04T17:26:16.057205+0000","last_active":"2024-07-04T17:26:17.701258+0000","last_peered":"2024-07-04T17:26:17.701258+0000","last_clean":"2024-07-04T17:26:17.701258+0000","last_became_active":"2024-07-04T17:26:16.056891+0000","last_became_peered":"2024-07-04T17:26:16.056891+0000","last_unstale":"2024-07-04T17:26:17.701258+0000","last_undegraded":"2024-07-04T17:26:17.701258+0000","last_fullsized":"2024-07-04T17:26:17.701258+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:16:58.662657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.701357+0000","last_change":"2024-07-04T17:26:16.072774+0000","last_active":"2024-07-04T17:26:17.701357+0000","last_peered":"2024-07-04T17:26:17.701357+0000","last_clean":"2024-07-04T17:26:17.701357+0000","last_became_active":"2024-07-04T17:26:16.071711+0000","last_became_peered":"2024-07-04T17:26:16.071711+0000","last_unstale":"2024-07-04T17:26:17.701357+0000","last_undegraded":"2024-07-04T17:26:17.701357+0000","last_fullsized":"2024-07-04T17:26:17.701357+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:20:14.576761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026507+0000","last_change":"2024-07-04T17:26:16.047214+0000","last_active":"2024-07-04T17:26:17.026507+0000","last_peered":"2024-07-04T17:26:17.026507+0000","last_clean":"2024-07-04T17:26:17.026507+0000","last_became_active":"2024-07-04T17:26:16.046836+0000","last_became_peered":"2024-07-04T17:26:16.046836+0000","last_unstale":"2024-07-04T17:26:17.026507+0000","last_undegraded":"2024-07-04T17:26:17.026507+0000","last_fullsized":"2024-07-04T17:26:17.026507+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:04:04.805872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"14'1","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.027730+0000","last_change":"2024-07-04T17:26:15.033936+0000","last_active":"2024-07-04T17:26:17.027730+0000","last_peered":"2024-07-04T17:26:17.027730+0000","last_clean":"2024-07-04T17:26:17.027730+0000","last_became_active":"2024-07-04T17:26:15.033601+0000","last_became_peered":"2024-07-04T17:26:15.033601+0000","last_unstale":"2024-07-04T17:26:17.027730+0000","last_undegraded":"2024-07-04T17:26:17.027730+0000","last_fullsized":"2024-07-04T17:26:17.027730+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:17:16.904464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021045+0000","last_change":"2024-07-04T17:26:16.053083+0000","last_active":"2024-07-04T17:26:17.021045+0000","last_peered":"2024-07-04T17:26:17.021045+0000","last_clean":"2024-07-04T17:26:17.021045+0000","last_became_active":"2024-07-04T17:26:16.052739+0000","last_became_peered":"2024-07-04T17:26:16.052739+0000","last_unstale":"2024-07-04T17:26:17.021045+0000","last_undegraded":"2024-07-04T17:26:17.021045+0000","last_fullsized":"2024-07-04T17:26:17.021045+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:00:52.128797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"14'1","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.701569+0000","last_change":"2024-07-04T17:26:15.039637+0000","last_active":"2024-07-04T17:26:17.701569+0000","last_peered":"2024-07-04T17:26:17.701569+0000","last_clean":"2024-07-04T17:26:17.701569+0000","last_became_active":"2024-07-04T17:26:15.039305+0000","last_became_peered":"2024-07-04T17:26:15.039305+0000","last_unstale":"2024-07-04T17:26:17.701569+0000","last_undegraded":"2024-07-04T17:26:17.701569+0000","last_fullsized":"2024-07-04T17:26:17.701569+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:04:46.667644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021302+0000","last_change":"2024-07-04T17:26:15.027075+0000","last_active":"2024-07-04T17:26:17.021302+0000","last_peered":"2024-07-04T17:26:17.021302+0000","last_clean":"2024-07-04T17:26:17.021302+0000","last_became_active":"2024-07-04T17:26:15.026375+0000","last_became_peered":"2024-07-04T17:26:15.026375+0000","last_unstale":"2024-07-04T17:26:17.021302+0000","last_undegraded":"2024-07-04T17:26:17.021302+0000","last_fullsized":"2024-07-04T17:26:17.021302+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:24:04.318931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702805+0000","last_change":"2024-07-04T17:26:16.074834+0000","last_active":"2024-07-04T17:26:17.702805+0000","last_peered":"2024-07-04T17:26:17.702805+0000","last_clean":"2024-07-04T17:26:17.702805+0000","last_became_active":"2024-07-04T17:26:16.074498+0000","last_became_peered":"2024-07-04T17:26:16.074498+0000","last_unstale":"2024-07-04T17:26:17.702805+0000","last_undegraded":"2024-07-04T17:26:17.702805+0000","last_fullsized":"2024-07-04T17:26:17.702805+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:08:53.606368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020171+0000","last_change":"2024-07-04T17:26:15.046968+0000","last_active":"2024-07-04T17:26:17.020171+0000","last_peered":"2024-07-04T17:26:17.020171+0000","last_clean":"2024-07-04T17:26:17.020171+0000","last_became_active":"2024-07-04T17:26:15.046617+0000","last_became_peered":"2024-07-04T17:26:15.046617+0000","last_unstale":"2024-07-04T17:26:17.020171+0000","last_undegraded":"2024-07-04T17:26:17.020171+0000","last_fullsized":"2024-07-04T17:26:17.020171+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:13:00.368941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026043+0000","last_change":"2024-07-04T17:26:16.041627+0000","last_active":"2024-07-04T17:26:17.026043+0000","last_peered":"2024-07-04T17:26:17.026043+0000","last_clean":"2024-07-04T17:26:17.026043+0000","last_became_active":"2024-07-04T17:26:16.041274+0000","last_became_peered":"2024-07-04T17:26:16.041274+0000","last_unstale":"2024-07-04T17:26:17.026043+0000","last_undegraded":"2024-07-04T17:26:17.026043+0000","last_fullsized":"2024-07-04T17:26:17.026043+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:41:46.272734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.029021+0000","last_change":"2024-07-04T17:26:15.037603+0000","last_active":"2024-07-04T17:26:17.029021+0000","last_peered":"2024-07-04T17:26:17.029021+0000","last_clean":"2024-07-04T17:26:17.029021+0000","last_became_active":"2024-07-04T17:26:15.030406+0000","last_became_peered":"2024-07-04T17:26:15.030406+0000","last_unstale":"2024-07-04T17:26:17.029021+0000","last_undegraded":"2024-07-04T17:26:17.029021+0000","last_fullsized":"2024-07-04T17:26:17.029021+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:13:00.368941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.703750+0000","last_change":"2024-07-04T17:26:16.060475+0000","last_active":"2024-07-04T17:26:17.703750+0000","last_peered":"2024-07-04T17:26:17.703750+0000","last_clean":"2024-07-04T17:26:17.703750+0000","last_became_active":"2024-07-04T17:26:16.059861+0000","last_became_peered":"2024-07-04T17:26:16.059861+0000","last_unstale":"2024-07-04T17:26:17.703750+0000","last_undegraded":"2024-07-04T17:26:17.703750+0000","last_fullsized":"2024-07-04T17:26:17.703750+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:21:38.693638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.023859+0000","last_change":"2024-07-04T17:26:15.046597+0000","last_active":"2024-07-04T17:26:17.023859+0000","last_peered":"2024-07-04T17:26:17.023859+0000","last_clean":"2024-07-04T17:26:17.023859+0000","last_became_active":"2024-07-04T17:26:15.046243+0000","last_became_peered":"2024-07-04T17:26:15.046243+0000","last_unstale":"2024-07-04T17:26:17.023859+0000","last_undegraded":"2024-07-04T17:26:17.023859+0000","last_fullsized":"2024-07-04T17:26:17.023859+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:18:04.371426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.024536+0000","last_change":"2024-07-04T17:26:16.059218+0000","last_active":"2024-07-04T17:26:17.024536+0000","last_peered":"2024-07-04T17:26:17.024536+0000","last_clean":"2024-07-04T17:26:17.024536+0000","last_became_active":"2024-07-04T17:26:16.058877+0000","last_became_peered":"2024-07-04T17:26:16.058877+0000","last_unstale":"2024-07-04T17:26:17.024536+0000","last_undegraded":"2024-07-04T17:26:17.024536+0000","last_fullsized":"2024-07-04T17:26:17.024536+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:48:48.817671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"14'3","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.029077+0000","last_change":"2024-07-04T17:26:15.031110+0000","last_active":"2024-07-04T17:26:17.029077+0000","last_peered":"2024-07-04T17:26:17.029077+0000","last_clean":"2024-07-04T17:26:17.029077+0000","last_became_active":"2024-07-04T17:26:15.030786+0000","last_became_peered":"2024-07-04T17:26:15.030786+0000","last_unstale":"2024-07-04T17:26:17.029077+0000","last_undegraded":"2024-07-04T17:26:17.029077+0000","last_fullsized":"2024-07-04T17:26:17.029077+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:34:46.876523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.028971+0000","last_change":"2024-07-04T17:26:16.046513+0000","last_active":"2024-07-04T17:26:17.028971+0000","last_peered":"2024-07-04T17:26:17.028971+0000","last_clean":"2024-07-04T17:26:17.028971+0000","last_became_active":"2024-07-04T17:26:16.044971+0000","last_became_peered":"2024-07-04T17:26:16.044971+0000","last_unstale":"2024-07-04T17:26:17.028971+0000","last_undegraded":"2024-07-04T17:26:17.028971+0000","last_fullsized":"2024-07-04T17:26:17.028971+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:21:38.693638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"14'6","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702536+0000","last_change":"2024-07-04T17:26:15.042022+0000","last_active":"2024-07-04T17:26:17.702536+0000","last_peered":"2024-07-04T17:26:17.702536+0000","last_clean":"2024-07-04T17:26:17.702536+0000","last_became_active":"2024-07-04T17:26:15.041669+0000","last_became_peered":"2024-07-04T17:26:15.041669+0000","last_unstale":"2024-07-04T17:26:17.702536+0000","last_undegraded":"2024-07-04T17:26:17.702536+0000","last_fullsized":"2024-07-04T17:26:17.702536+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:35:22.891399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.024207+0000","last_change":"2024-07-04T17:26:16.059656+0000","last_active":"2024-07-04T17:26:17.024207+0000","last_peered":"2024-07-04T17:26:17.024207+0000","last_clean":"2024-07-04T17:26:17.024207+0000","last_became_active":"2024-07-04T17:26:16.058951+0000","last_became_peered":"2024-07-04T17:26:16.058951+0000","last_unstale":"2024-07-04T17:26:17.024207+0000","last_undegraded":"2024-07-04T17:26:17.024207+0000","last_fullsized":"2024-07-04T17:26:17.024207+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:52:45.300002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"14'2","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021264+0000","last_change":"2024-07-04T17:26:15.035545+0000","last_active":"2024-07-04T17:26:17.021264+0000","last_peered":"2024-07-04T17:26:17.021264+0000","last_clean":"2024-07-04T17:26:17.021264+0000","last_became_active":"2024-07-04T17:26:15.034822+0000","last_became_peered":"2024-07-04T17:26:15.034822+0000","last_unstale":"2024-07-04T17:26:17.021264+0000","last_undegraded":"2024-07-04T17:26:17.021264+0000","last_fullsized":"2024-07-04T17:26:17.021264+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:17:16.904464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020669+0000","last_change":"2024-07-04T17:26:16.064292+0000","last_active":"2024-07-04T17:26:17.020669+0000","last_peered":"2024-07-04T17:26:17.020669+0000","last_clean":"2024-07-04T17:26:17.020669+0000","last_became_active":"2024-07-04T17:26:16.053620+0000","last_became_peered":"2024-07-04T17:26:16.053620+0000","last_unstale":"2024-07-04T17:26:17.020669+0000","last_undegraded":"2024-07-04T17:26:17.020669+0000","last_fullsized":"2024-07-04T17:26:17.020669+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:40:49.177003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.701897+0000","last_change":"2024-07-04T17:26:15.041695+0000","last_active":"2024-07-04T17:26:17.701897+0000","last_peered":"2024-07-04T17:26:17.701897+0000","last_clean":"2024-07-04T17:26:17.701897+0000","last_became_active":"2024-07-04T17:26:15.041353+0000","last_became_peered":"2024-07-04T17:26:15.041353+0000","last_unstale":"2024-07-04T17:26:17.701897+0000","last_undegraded":"2024-07-04T17:26:17.701897+0000","last_fullsized":"2024-07-04T17:26:17.701897+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:05:39.042848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026437+0000","last_change":"2024-07-04T17:26:16.063355+0000","last_active":"2024-07-04T17:26:17.026437+0000","last_peered":"2024-07-04T17:26:17.026437+0000","last_clean":"2024-07-04T17:26:17.026437+0000","last_became_active":"2024-07-04T17:26:16.056707+0000","last_became_peered":"2024-07-04T17:26:16.056707+0000","last_unstale":"2024-07-04T17:26:17.026437+0000","last_undegraded":"2024-07-04T17:26:17.026437+0000","last_fullsized":"2024-07-04T17:26:17.026437+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:11:17.509522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021433+0000","last_change":"2024-07-04T17:26:15.050680+0000","last_active":"2024-07-04T17:26:17.021433+0000","last_peered":"2024-07-04T17:26:17.021433+0000","last_clean":"2024-07-04T17:26:17.021433+0000","last_became_active":"2024-07-04T17:26:15.049449+0000","last_became_peered":"2024-07-04T17:26:15.049449+0000","last_unstale":"2024-07-04T17:26:17.021433+0000","last_undegraded":"2024-07-04T17:26:17.021433+0000","last_fullsized":"2024-07-04T17:26:17.021433+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:25:38.473912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026199+0000","last_change":"2024-07-04T17:26:16.063777+0000","last_active":"2024-07-04T17:26:17.026199+0000","last_peered":"2024-07-04T17:26:17.026199+0000","last_clean":"2024-07-04T17:26:17.026199+0000","last_became_active":"2024-07-04T17:26:16.063580+0000","last_became_peered":"2024-07-04T17:26:16.063580+0000","last_unstale":"2024-07-04T17:26:17.026199+0000","last_undegraded":"2024-07-04T17:26:17.026199+0000","last_fullsized":"2024-07-04T17:26:17.026199+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:21:48.005606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"14'8","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702841+0000","last_change":"2024-07-04T17:26:15.043346+0000","last_active":"2024-07-04T17:26:17.702841+0000","last_peered":"2024-07-04T17:26:17.702841+0000","last_clean":"2024-07-04T17:26:17.702841+0000","last_became_active":"2024-07-04T17:26:15.041219+0000","last_became_peered":"2024-07-04T17:26:15.041219+0000","last_unstale":"2024-07-04T17:26:17.702841+0000","last_undegraded":"2024-07-04T17:26:17.702841+0000","last_fullsized":"2024-07-04T17:26:17.702841+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:13:00.368941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.701310+0000","last_change":"2024-07-04T17:26:16.057277+0000","last_active":"2024-07-04T17:26:17.701310+0000","last_peered":"2024-07-04T17:26:17.701310+0000","last_clean":"2024-07-04T17:26:17.701310+0000","last_became_active":"2024-07-04T17:26:16.056946+0000","last_became_peered":"2024-07-04T17:26:16.056946+0000","last_unstale":"2024-07-04T17:26:17.701310+0000","last_undegraded":"2024-07-04T17:26:17.701310+0000","last_fullsized":"2024-07-04T17:26:17.701310+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:41:46.272734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"14'1","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.027552+0000","last_change":"2024-07-04T17:26:15.049872+0000","last_active":"2024-07-04T17:26:17.027552+0000","last_peered":"2024-07-04T17:26:17.027552+0000","last_clean":"2024-07-04T17:26:17.027552+0000","last_became_active":"2024-07-04T17:26:15.048975+0000","last_became_peered":"2024-07-04T17:26:15.048975+0000","last_unstale":"2024-07-04T17:26:17.027552+0000","last_undegraded":"2024-07-04T17:26:17.027552+0000","last_fullsized":"2024-07-04T17:26:17.027552+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:18:52.190216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.704159+0000","last_change":"2024-07-04T17:26:16.071254+0000","last_active":"2024-07-04T17:26:17.704159+0000","last_peered":"2024-07-04T17:26:17.704159+0000","last_clean":"2024-07-04T17:26:17.704159+0000","last_became_active":"2024-07-04T17:26:16.070624+0000","last_became_peered":"2024-07-04T17:26:16.070624+0000","last_unstale":"2024-07-04T17:26:17.704159+0000","last_undegraded":"2024-07-04T17:26:17.704159+0000","last_fullsized":"2024-07-04T17:26:17.704159+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:50:25.908300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"14'2","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.704347+0000","last_change":"2024-07-04T17:26:15.048120+0000","last_active":"2024-07-04T17:26:17.704347+0000","last_peered":"2024-07-04T17:26:17.704347+0000","last_clean":"2024-07-04T17:26:17.704347+0000","last_became_active":"2024-07-04T17:26:15.047745+0000","last_became_peered":"2024-07-04T17:26:15.047745+0000","last_unstale":"2024-07-04T17:26:17.704347+0000","last_undegraded":"2024-07-04T17:26:17.704347+0000","last_fullsized":"2024-07-04T17:26:17.704347+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:17:16.904464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.024662+0000","last_change":"2024-07-04T17:26:16.047526+0000","last_active":"2024-07-04T17:26:17.024662+0000","last_peered":"2024-07-04T17:26:17.024662+0000","last_clean":"2024-07-04T17:26:17.024662+0000","last_became_active":"2024-07-04T17:26:16.046143+0000","last_became_peered":"2024-07-04T17:26:16.046143+0000","last_unstale":"2024-07-04T17:26:17.024662+0000","last_undegraded":"2024-07-04T17:26:17.024662+0000","last_fullsized":"2024-07-04T17:26:17.024662+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:48:44.510528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"14'1","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.029637+0000","last_change":"2024-07-04T17:26:15.021551+0000","last_active":"2024-07-04T17:26:17.029637+0000","last_peered":"2024-07-04T17:26:17.029637+0000","last_clean":"2024-07-04T17:26:17.029637+0000","last_became_active":"2024-07-04T17:26:15.021324+0000","last_became_peered":"2024-07-04T17:26:15.021324+0000","last_unstale":"2024-07-04T17:26:17.029637+0000","last_undegraded":"2024-07-04T17:26:17.029637+0000","last_fullsized":"2024-07-04T17:26:17.029637+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:18:04.371426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022385+0000","last_change":"2024-07-04T17:26:16.043753+0000","last_active":"2024-07-04T17:26:17.022385+0000","last_peered":"2024-07-04T17:26:17.022385+0000","last_clean":"2024-07-04T17:26:17.022385+0000","last_became_active":"2024-07-04T17:26:16.037154+0000","last_became_peered":"2024-07-04T17:26:16.037154+0000","last_unstale":"2024-07-04T17:26:17.022385+0000","last_undegraded":"2024-07-04T17:26:17.022385+0000","last_fullsized":"2024-07-04T17:26:17.022385+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:30:49.842921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702886+0000","last_change":"2024-07-04T17:26:15.039837+0000","last_active":"2024-07-04T17:26:17.702886+0000","last_peered":"2024-07-04T17:26:17.702886+0000","last_clean":"2024-07-04T17:26:17.702886+0000","last_became_active":"2024-07-04T17:26:15.039334+0000","last_became_peered":"2024-07-04T17:26:15.039334+0000","last_unstale":"2024-07-04T17:26:17.702886+0000","last_undegraded":"2024-07-04T17:26:17.702886+0000","last_fullsized":"2024-07-04T17:26:17.702886+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:46.107542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.030226+0000","last_change":"2024-07-04T17:26:16.061791+0000","last_active":"2024-07-04T17:26:17.030226+0000","last_peered":"2024-07-04T17:26:17.030226+0000","last_clean":"2024-07-04T17:26:17.030226+0000","last_became_active":"2024-07-04T17:26:16.061453+0000","last_became_peered":"2024-07-04T17:26:16.061453+0000","last_unstale":"2024-07-04T17:26:17.030226+0000","last_undegraded":"2024-07-04T17:26:17.030226+0000","last_fullsized":"2024-07-04T17:26:17.030226+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:13:00.963495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'52","reported_seq":87,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026937+0000","last_change":"2024-07-04T17:26:04.953559+0000","last_active":"2024-07-04T17:26:17.026937+0000","last_peered":"2024-07-04T17:26:17.026937+0000","last_clean":"2024-07-04T17:26:17.026937+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:26:17.026937+0000","last_undegraded":"2024-07-04T17:26:17.026937+0000","last_fullsized":"2024-07-04T17:26:17.026937+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_clean_scrub_stamp":"2024-07-04T17:26:03.933432+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.064181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.027937+0000","last_change":"2024-07-04T17:26:15.026582+0000","last_active":"2024-07-04T17:26:17.027937+0000","last_peered":"2024-07-04T17:26:17.027937+0000","last_clean":"2024-07-04T17:26:17.027937+0000","last_became_active":"2024-07-04T17:26:15.026170+0000","last_became_peered":"2024-07-04T17:26:15.026170+0000","last_unstale":"2024-07-04T17:26:17.027937+0000","last_undegraded":"2024-07-04T17:26:17.027937+0000","last_fullsized":"2024-07-04T17:26:17.027937+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:45:20.253806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026496+0000","last_change":"2024-07-04T17:26:16.044579+0000","last_active":"2024-07-04T17:26:17.026496+0000","last_peered":"2024-07-04T17:26:17.026496+0000","last_clean":"2024-07-04T17:26:17.026496+0000","last_became_active":"2024-07-04T17:26:16.044255+0000","last_became_peered":"2024-07-04T17:26:16.044255+0000","last_unstale":"2024-07-04T17:26:17.026496+0000","last_undegraded":"2024-07-04T17:26:17.026496+0000","last_fullsized":"2024-07-04T17:26:17.026496+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:30:20.910427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.703153+0000","last_change":"2024-07-04T17:26:15.043650+0000","last_active":"2024-07-04T17:26:17.703153+0000","last_peered":"2024-07-04T17:26:17.703153+0000","last_clean":"2024-07-04T17:26:17.703153+0000","last_became_active":"2024-07-04T17:26:15.042392+0000","last_became_peered":"2024-07-04T17:26:15.042392+0000","last_unstale":"2024-07-04T17:26:17.703153+0000","last_undegraded":"2024-07-04T17:26:17.703153+0000","last_fullsized":"2024-07-04T17:26:17.703153+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:18:04.371426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021141+0000","last_change":"2024-07-04T17:26:16.051770+0000","last_active":"2024-07-04T17:26:17.021141+0000","last_peered":"2024-07-04T17:26:17.021141+0000","last_clean":"2024-07-04T17:26:17.021141+0000","last_became_active":"2024-07-04T17:26:16.046616+0000","last_became_peered":"2024-07-04T17:26:16.046616+0000","last_unstale":"2024-07-04T17:26:17.021141+0000","last_undegraded":"2024-07-04T17:26:17.021141+0000","last_fullsized":"2024-07-04T17:26:17.021141+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:03:12.070345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702913+0000","last_change":"2024-07-04T17:26:15.042707+0000","last_active":"2024-07-04T17:26:17.702913+0000","last_peered":"2024-07-04T17:26:17.702913+0000","last_clean":"2024-07-04T17:26:17.702913+0000","last_became_active":"2024-07-04T17:26:15.041346+0000","last_became_peered":"2024-07-04T17:26:15.041346+0000","last_unstale":"2024-07-04T17:26:17.702913+0000","last_undegraded":"2024-07-04T17:26:17.702913+0000","last_fullsized":"2024-07-04T17:26:17.702913+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:31:03.314632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026776+0000","last_change":"2024-07-04T17:26:16.039792+0000","last_active":"2024-07-04T17:26:17.026776+0000","last_peered":"2024-07-04T17:26:17.026776+0000","last_clean":"2024-07-04T17:26:17.026776+0000","last_became_active":"2024-07-04T17:26:16.039470+0000","last_became_peered":"2024-07-04T17:26:16.039470+0000","last_unstale":"2024-07-04T17:26:17.026776+0000","last_undegraded":"2024-07-04T17:26:17.026776+0000","last_fullsized":"2024-07-04T17:26:17.026776+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:08:53.606368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022753+0000","last_change":"2024-07-04T17:26:15.046123+0000","last_active":"2024-07-04T17:26:17.022753+0000","last_peered":"2024-07-04T17:26:17.022753+0000","last_clean":"2024-07-04T17:26:17.022753+0000","last_became_active":"2024-07-04T17:26:15.045824+0000","last_became_peered":"2024-07-04T17:26:15.045824+0000","last_unstale":"2024-07-04T17:26:17.022753+0000","last_undegraded":"2024-07-04T17:26:17.022753+0000","last_fullsized":"2024-07-04T17:26:17.022753+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:46.107542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021092+0000","last_change":"2024-07-04T17:26:16.044581+0000","last_active":"2024-07-04T17:26:17.021092+0000","last_peered":"2024-07-04T17:26:17.021092+0000","last_clean":"2024-07-04T17:26:17.021092+0000","last_became_active":"2024-07-04T17:26:16.043791+0000","last_became_peered":"2024-07-04T17:26:16.043791+0000","last_unstale":"2024-07-04T17:26:17.021092+0000","last_undegraded":"2024-07-04T17:26:17.021092+0000","last_fullsized":"2024-07-04T17:26:17.021092+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:38:17.647736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.029309+0000","last_change":"2024-07-04T17:26:15.032798+0000","last_active":"2024-07-04T17:26:17.029309+0000","last_peered":"2024-07-04T17:26:17.029309+0000","last_clean":"2024-07-04T17:26:17.029309+0000","last_became_active":"2024-07-04T17:26:15.029686+0000","last_became_peered":"2024-07-04T17:26:15.029686+0000","last_unstale":"2024-07-04T17:26:17.029309+0000","last_undegraded":"2024-07-04T17:26:17.029309+0000","last_fullsized":"2024-07-04T17:26:17.029309+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:25:38.473912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702222+0000","last_change":"2024-07-04T17:26:16.062731+0000","last_active":"2024-07-04T17:26:17.702222+0000","last_peered":"2024-07-04T17:26:17.702222+0000","last_clean":"2024-07-04T17:26:17.702222+0000","last_became_active":"2024-07-04T17:26:16.062519+0000","last_became_peered":"2024-07-04T17:26:16.062519+0000","last_unstale":"2024-07-04T17:26:17.702222+0000","last_undegraded":"2024-07-04T17:26:17.702222+0000","last_fullsized":"2024-07-04T17:26:17.702222+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:30:20.910427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.029919+0000","last_change":"2024-07-04T17:26:15.036880+0000","last_active":"2024-07-04T17:26:17.029919+0000","last_peered":"2024-07-04T17:26:17.029919+0000","last_clean":"2024-07-04T17:26:17.029919+0000","last_became_active":"2024-07-04T17:26:15.036529+0000","last_became_peered":"2024-07-04T17:26:15.036529+0000","last_unstale":"2024-07-04T17:26:17.029919+0000","last_undegraded":"2024-07-04T17:26:17.029919+0000","last_fullsized":"2024-07-04T17:26:17.029919+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:04:46.667644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.029152+0000","last_change":"2024-07-04T17:26:16.046186+0000","last_active":"2024-07-04T17:26:17.029152+0000","last_peered":"2024-07-04T17:26:17.029152+0000","last_clean":"2024-07-04T17:26:17.029152+0000","last_became_active":"2024-07-04T17:26:16.044588+0000","last_became_peered":"2024-07-04T17:26:16.044588+0000","last_unstale":"2024-07-04T17:26:17.029152+0000","last_undegraded":"2024-07-04T17:26:17.029152+0000","last_fullsized":"2024-07-04T17:26:17.029152+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:50:25.908300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"15'1","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.030544+0000","last_change":"2024-07-04T17:26:15.022516+0000","last_active":"2024-07-04T17:26:17.030544+0000","last_peered":"2024-07-04T17:26:17.030544+0000","last_clean":"2024-07-04T17:26:17.030544+0000","last_became_active":"2024-07-04T17:26:15.022185+0000","last_became_peered":"2024-07-04T17:26:15.022185+0000","last_unstale":"2024-07-04T17:26:17.030544+0000","last_undegraded":"2024-07-04T17:26:17.030544+0000","last_fullsized":"2024-07-04T17:26:17.030544+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:24:04.318931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702984+0000","last_change":"2024-07-04T17:26:16.059869+0000","last_active":"2024-07-04T17:26:17.702984+0000","last_peered":"2024-07-04T17:26:17.702984+0000","last_clean":"2024-07-04T17:26:17.702984+0000","last_became_active":"2024-07-04T17:26:16.059535+0000","last_became_peered":"2024-07-04T17:26:16.059535+0000","last_unstale":"2024-07-04T17:26:17.702984+0000","last_undegraded":"2024-07-04T17:26:17.702984+0000","last_fullsized":"2024-07-04T17:26:17.702984+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:43:29.877017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"14'2","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.029845+0000","last_change":"2024-07-04T17:26:15.040384+0000","last_active":"2024-07-04T17:26:17.029845+0000","last_peered":"2024-07-04T17:26:17.029845+0000","last_clean":"2024-07-04T17:26:17.029845+0000","last_became_active":"2024-07-04T17:26:15.038959+0000","last_became_peered":"2024-07-04T17:26:15.038959+0000","last_unstale":"2024-07-04T17:26:17.029845+0000","last_undegraded":"2024-07-04T17:26:17.029845+0000","last_fullsized":"2024-07-04T17:26:17.029845+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:18:52.190216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.023117+0000","last_change":"2024-07-04T17:26:16.053639+0000","last_active":"2024-07-04T17:26:17.023117+0000","last_peered":"2024-07-04T17:26:17.023117+0000","last_clean":"2024-07-04T17:26:17.023117+0000","last_became_active":"2024-07-04T17:26:16.052766+0000","last_became_peered":"2024-07-04T17:26:16.052766+0000","last_unstale":"2024-07-04T17:26:17.023117+0000","last_undegraded":"2024-07-04T17:26:17.023117+0000","last_fullsized":"2024-07-04T17:26:17.023117+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:22:12.063177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020302+0000","last_change":"2024-07-04T17:26:15.027718+0000","last_active":"2024-07-04T17:26:17.020302+0000","last_peered":"2024-07-04T17:26:17.020302+0000","last_clean":"2024-07-04T17:26:17.020302+0000","last_became_active":"2024-07-04T17:26:15.026662+0000","last_became_peered":"2024-07-04T17:26:15.026662+0000","last_unstale":"2024-07-04T17:26:17.020302+0000","last_undegraded":"2024-07-04T17:26:17.020302+0000","last_fullsized":"2024-07-04T17:26:17.020302+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:45:20.253806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020487+0000","last_change":"2024-07-04T17:26:16.066686+0000","last_active":"2024-07-04T17:26:17.020487+0000","last_peered":"2024-07-04T17:26:17.020487+0000","last_clean":"2024-07-04T17:26:17.020487+0000","last_became_active":"2024-07-04T17:26:16.066397+0000","last_became_peered":"2024-07-04T17:26:16.066397+0000","last_unstale":"2024-07-04T17:26:17.020487+0000","last_undegraded":"2024-07-04T17:26:17.020487+0000","last_fullsized":"2024-07-04T17:26:17.020487+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:21:38.693638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026169+0000","last_change":"2024-07-04T17:26:15.034670+0000","last_active":"2024-07-04T17:26:17.026169+0000","last_peered":"2024-07-04T17:26:17.026169+0000","last_clean":"2024-07-04T17:26:17.026169+0000","last_became_active":"2024-07-04T17:26:15.034267+0000","last_became_peered":"2024-07-04T17:26:15.034267+0000","last_unstale":"2024-07-04T17:26:17.026169+0000","last_undegraded":"2024-07-04T17:26:17.026169+0000","last_fullsized":"2024-07-04T17:26:17.026169+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:46.107542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702508+0000","last_change":"2024-07-04T17:26:16.054400+0000","last_active":"2024-07-04T17:26:17.702508+0000","last_peered":"2024-07-04T17:26:17.702508+0000","last_clean":"2024-07-04T17:26:17.702508+0000","last_became_active":"2024-07-04T17:26:16.054057+0000","last_became_peered":"2024-07-04T17:26:16.054057+0000","last_unstale":"2024-07-04T17:26:17.702508+0000","last_undegraded":"2024-07-04T17:26:17.702508+0000","last_fullsized":"2024-07-04T17:26:17.702508+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:04:04.805872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026719+0000","last_change":"2024-07-04T17:26:15.032894+0000","last_active":"2024-07-04T17:26:17.026719+0000","last_peered":"2024-07-04T17:26:17.026719+0000","last_clean":"2024-07-04T17:26:17.026719+0000","last_became_active":"2024-07-04T17:26:15.032634+0000","last_became_peered":"2024-07-04T17:26:15.032634+0000","last_unstale":"2024-07-04T17:26:17.026719+0000","last_undegraded":"2024-07-04T17:26:17.026719+0000","last_fullsized":"2024-07-04T17:26:17.026719+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:05:39.042848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.026622+0000","last_change":"2024-07-04T17:26:16.065536+0000","last_active":"2024-07-04T17:26:17.026622+0000","last_peered":"2024-07-04T17:26:17.026622+0000","last_clean":"2024-07-04T17:26:17.026622+0000","last_became_active":"2024-07-04T17:26:16.056278+0000","last_became_peered":"2024-07-04T17:26:16.056278+0000","last_unstale":"2024-07-04T17:26:17.026622+0000","last_undegraded":"2024-07-04T17:26:17.026622+0000","last_fullsized":"2024-07-04T17:26:17.026622+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:03:07.909265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"14'2","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.021818+0000","last_change":"2024-07-04T17:26:15.033912+0000","last_active":"2024-07-04T17:26:17.021818+0000","last_peered":"2024-07-04T17:26:17.021818+0000","last_clean":"2024-07-04T17:26:17.021818+0000","last_became_active":"2024-07-04T17:26:15.027824+0000","last_became_peered":"2024-07-04T17:26:15.027824+0000","last_unstale":"2024-07-04T17:26:17.021818+0000","last_undegraded":"2024-07-04T17:26:17.021818+0000","last_fullsized":"2024-07-04T17:26:17.021818+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:34:46.876523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.020817+0000","last_change":"2024-07-04T17:26:16.065218+0000","last_active":"2024-07-04T17:26:17.020817+0000","last_peered":"2024-07-04T17:26:17.020817+0000","last_clean":"2024-07-04T17:26:17.020817+0000","last_became_active":"2024-07-04T17:26:16.064904+0000","last_became_peered":"2024-07-04T17:26:16.064904+0000","last_unstale":"2024-07-04T17:26:17.020817+0000","last_undegraded":"2024-07-04T17:26:17.020817+0000","last_fullsized":"2024-07-04T17:26:17.020817+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:33:23.722640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.702115+0000","last_change":"2024-07-04T17:26:16.073564+0000","last_active":"2024-07-04T17:26:17.702115+0000","last_peered":"2024-07-04T17:26:17.702115+0000","last_clean":"2024-07-04T17:26:17.702115+0000","last_became_active":"2024-07-04T17:26:16.073189+0000","last_became_peered":"2024-07-04T17:26:16.073189+0000","last_unstale":"2024-07-04T17:26:17.702115+0000","last_undegraded":"2024-07-04T17:26:17.702115+0000","last_fullsized":"2024-07-04T17:26:17.702115+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:21:48.005606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"14'2","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022583+0000","last_change":"2024-07-04T17:26:15.049001+0000","last_active":"2024-07-04T17:26:17.022583+0000","last_peered":"2024-07-04T17:26:17.022583+0000","last_clean":"2024-07-04T17:26:17.022583+0000","last_became_active":"2024-07-04T17:26:15.048437+0000","last_became_peered":"2024-07-04T17:26:15.048437+0000","last_unstale":"2024-07-04T17:26:17.022583+0000","last_undegraded":"2024-07-04T17:26:17.022583+0000","last_fullsized":"2024-07-04T17:26:17.022583+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:15:08.055102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.028692+0000","last_change":"2024-07-04T17:26:16.064782+0000","last_active":"2024-07-04T17:26:17.028692+0000","last_peered":"2024-07-04T17:26:17.028692+0000","last_clean":"2024-07-04T17:26:17.028692+0000","last_became_active":"2024-07-04T17:26:16.064518+0000","last_became_peered":"2024-07-04T17:26:16.064518+0000","last_unstale":"2024-07-04T17:26:17.028692+0000","last_undegraded":"2024-07-04T17:26:17.028692+0000","last_fullsized":"2024-07-04T17:26:17.028692+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:01:31.824881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"14'2","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.022678+0000","last_change":"2024-07-04T17:26:15.047259+0000","last_active":"2024-07-04T17:26:17.022678+0000","last_peered":"2024-07-04T17:26:17.022678+0000","last_clean":"2024-07-04T17:26:17.022678+0000","last_became_active":"2024-07-04T17:26:15.046951+0000","last_became_peered":"2024-07-04T17:26:15.046951+0000","last_unstale":"2024-07-04T17:26:17.022678+0000","last_undegraded":"2024-07-04T17:26:17.022678+0000","last_fullsized":"2024-07-04T17:26:17.022678+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:04:46.667644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.024493+0000","last_change":"2024-07-04T17:26:16.058196+0000","last_active":"2024-07-04T17:26:17.024493+0000","last_peered":"2024-07-04T17:26:17.024493+0000","last_clean":"2024-07-04T17:26:17.024493+0000","last_became_active":"2024-07-04T17:26:16.051038+0000","last_became_peered":"2024-07-04T17:26:16.051038+0000","last_unstale":"2024-07-04T17:26:17.024493+0000","last_undegraded":"2024-07-04T17:26:17.024493+0000","last_fullsized":"2024-07-04T17:26:17.024493+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:04:50.688662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"14'1","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.704046+0000","last_change":"2024-07-04T17:26:15.040464+0000","last_active":"2024-07-04T17:26:17.704046+0000","last_peered":"2024-07-04T17:26:17.704046+0000","last_clean":"2024-07-04T17:26:17.704046+0000","last_became_active":"2024-07-04T17:26:15.040008+0000","last_became_peered":"2024-07-04T17:26:15.040008+0000","last_unstale":"2024-07-04T17:26:17.704046+0000","last_undegraded":"2024-07-04T17:26:17.704046+0000","last_fullsized":"2024-07-04T17:26:17.704046+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:34:46.876523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.703450+0000","last_change":"2024-07-04T17:26:16.060159+0000","last_active":"2024-07-04T17:26:17.703450+0000","last_peered":"2024-07-04T17:26:17.703450+0000","last_clean":"2024-07-04T17:26:17.703450+0000","last_became_active":"2024-07-04T17:26:16.059458+0000","last_became_peered":"2024-07-04T17:26:16.059458+0000","last_unstale":"2024-07-04T17:26:17.703450+0000","last_undegraded":"2024-07-04T17:26:17.703450+0000","last_fullsized":"2024-07-04T17:26:17.703450+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:14.980863+0000","last_clean_scrub_stamp":"2024-07-04T17:26:14.980863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:01:31.824881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"14'4","reported_seq":18,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-07-04T17:26:17.027492+0000","last_change":"2024-07-04T17:26:15.049639+0000","last_active":"2024-07-04T17:26:17.027492+0000","last_peered":"2024-07-04T17:26:17.027492+0000","last_clean":"2024-07-04T17:26:17.027492+0000","last_became_active":"2024-07-04T17:26:15.048590+0000","last_became_peered":"2024-07-04T17:26:15.048590+0000","last_unstale":"2024-07-04T17:26:17.027492+0000","last_undegraded":"2024-07-04T17:26:17.027492+0000","last_fullsized":"2024-07-04T17:26:17.027492+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:13.971683+0000","last_clean_scrub_stamp":"2024-07-04T17:26:13.971683+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:05:39.042848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738375,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31332,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340508,"statfs":{"total":96636764160,"available":96604680192,"internally_reserved":0,"allocated":151552,"data_stored":42085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738376,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31920,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339920,"statfs":{"total":96636764160,"available":96604078080,"internally_reserved":0,"allocated":753664,"data_stored":633113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738374,"num_pgs":74,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31912,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339928,"statfs":{"total":96636764160,"available":96604086272,"internally_reserved":0,"allocated":741376,"data_stored":631465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:26:28.754 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-04T17:26:28.754 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-07-04T17:26:28.754 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-07-04T17:26:28.754 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-07-04T17:26:29.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.085+0000 7f6361800640 1 -- 172.21.15.42:0/1265539801 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f635c0adac0 msgr2=0x7f635c06e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:29.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.085+0000 7f6361800640 1 --2- 172.21.15.42:0/1265539801 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f635c0adac0 0x7f635c06e120 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f63440097b0 tx=0x7f634402a540 comp rx=0 tx=0).stop 2024-07-04T17:26:29.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.085+0000 7f6361800640 1 -- 172.21.15.42:0/1265539801 shutdown_connections 2024-07-04T17:26:29.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.085+0000 7f6361800640 1 --2- 172.21.15.42:0/1265539801 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f635c0adac0 0x7f635c06e120 secure :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f63440097b0 tx=0x7f634402a540 comp rx=0 tx=0).stop 2024-07-04T17:26:29.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.085+0000 7f6361800640 1 -- 172.21.15.42:0/1265539801 >> 172.21.15.42:0/1265539801 conn(0x7f635c071960 msgr2=0x7f635c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:29.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.086+0000 7f6361800640 1 -- 172.21.15.42:0/1265539801 shutdown_connections 2024-07-04T17:26:29.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.086+0000 7f6361800640 1 -- 172.21.15.42:0/1265539801 wait complete. 2024-07-04T17:26:29.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.088+0000 7f6361800640 1 Processor -- start 2024-07-04T17:26:29.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.088+0000 7f6361800640 1 -- start start 2024-07-04T17:26:29.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.090+0000 7f6361800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f635c0be540 0x7f635c1d80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:29.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.090+0000 7f6361800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f635c06e660 con 0x7f635c0be540 2024-07-04T17:26:29.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.090+0000 7f635be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f635c0be540 0x7f635c1d80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:29.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.091+0000 7f635be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f635c0be540 0x7f635c1d80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44164/0 (socket says 172.21.15.42:44164) 2024-07-04T17:26:29.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.091+0000 7f635be00640 1 -- 172.21.15.42:0/1177641202 learned_addr learned my addr 172.21.15.42:0/1177641202 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:29.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.091+0000 7f635be00640 1 -- 172.21.15.42:0/1177641202 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f635c0bd090 con 0x7f635c0be540 2024-07-04T17:26:29.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.091+0000 7f635be00640 1 --2- 172.21.15.42:0/1177641202 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f635c0be540 0x7f635c1d80e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f634c001eb0 tx=0x7f634c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:29.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.092+0000 7f6358c00640 1 -- 172.21.15.42:0/1177641202 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f634c0116c0 con 0x7f635c0be540 2024-07-04T17:26:29.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.092+0000 7f6358c00640 1 -- 172.21.15.42:0/1177641202 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f634c017070 con 0x7f635c0be540 2024-07-04T17:26:29.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.092+0000 7f6358c00640 1 -- 172.21.15.42:0/1177641202 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f634c011ae0 con 0x7f635c0be540 2024-07-04T17:26:29.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.092+0000 7f6361800640 1 -- 172.21.15.42:0/1177641202 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f635c0b0950 con 0x7f635c0be540 2024-07-04T17:26:29.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.092+0000 7f6361800640 1 -- 172.21.15.42:0/1177641202 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f635c0b0760 con 0x7f635c0be540 2024-07-04T17:26:29.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.093+0000 7f6358c00640 1 -- 172.21.15.42:0/1177641202 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f634c011c80 con 0x7f635c0be540 2024-07-04T17:26:29.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.093+0000 7f6361800640 1 -- 172.21.15.42:0/1177641202 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f635c0b44b0 con 0x7f635c0be540 2024-07-04T17:26:29.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.094+0000 7f6358c00640 1 --2- 172.21.15.42:0/1177641202 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f633803d2b0 0x7f633803f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:29.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.094+0000 7f6358c00640 1 -- 172.21.15.42:0/1177641202 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f634c051320 con 0x7f635c0be540 2024-07-04T17:26:29.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.094+0000 7f6353400640 1 --2- 172.21.15.42:0/1177641202 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f633803d2b0 0x7f633803f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:29.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.095+0000 7f6353400640 1 --2- 172.21.15.42:0/1177641202 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f633803d2b0 0x7f633803f7c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6344034000 tx=0x7f6344004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:29.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.101+0000 7f6358c00640 1 -- 172.21.15.42:0/1177641202 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f635c0b44b0 con 0x7f635c0be540 2024-07-04T17:26:29.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.364+0000 7f6361800640 1 -- 172.21.15.42:0/1177641202 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f635c06ed20 con 0x7f635c0be540 2024-07-04T17:26:29.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.365+0000 7f6358c00640 1 -- 172.21.15.42:0/1177641202 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f634c00a140 con 0x7f635c0be540 2024-07-04T17:26:29.367 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:26:29.367 INFO:teuthology.orchestra.run.smithi042.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-07-04T17:26:29.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.368+0000 7f6361800640 1 -- 172.21.15.42:0/1177641202 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f633803d2b0 msgr2=0x7f633803f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:29.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.368+0000 7f6361800640 1 --2- 172.21.15.42:0/1177641202 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f633803d2b0 0x7f633803f7c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6344034000 tx=0x7f6344004730 comp rx=0 tx=0).stop 2024-07-04T17:26:29.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.368+0000 7f6361800640 1 -- 172.21.15.42:0/1177641202 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f635c0be540 msgr2=0x7f635c1d80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:29.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.368+0000 7f6361800640 1 --2- 172.21.15.42:0/1177641202 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f635c0be540 0x7f635c1d80e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f634c001eb0 tx=0x7f634c00c810 comp rx=0 tx=0).stop 2024-07-04T17:26:29.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.368+0000 7f6361800640 1 -- 172.21.15.42:0/1177641202 shutdown_connections 2024-07-04T17:26:29.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.368+0000 7f6361800640 1 --2- 172.21.15.42:0/1177641202 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f633803d2b0 0x7f633803f7c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:29.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.368+0000 7f6361800640 1 --2- 172.21.15.42:0/1177641202 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f635c0be540 0x7f635c1d80e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:29.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.368+0000 7f6361800640 1 -- 172.21.15.42:0/1177641202 >> 172.21.15.42:0/1177641202 conn(0x7f635c071960 msgr2=0x7f635c1510c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:29.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.369+0000 7f6361800640 1 -- 172.21.15.42:0/1177641202 shutdown_connections 2024-07-04T17:26:29.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.369+0000 7f6361800640 1 -- 172.21.15.42:0/1177641202 wait complete. 2024-07-04T17:26:29.396 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-07-04T17:26:29.396 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:26:29.418 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-04T17:26:29.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.685+0000 7f5aa6600640 1 -- 172.21.15.42:0/1554568146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5aa0144fe0 msgr2=0x7f5aa006c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:29.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.685+0000 7f5aa6600640 1 --2- 172.21.15.42:0/1554568146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5aa0144fe0 0x7f5aa006c020 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5a88005fd0 tx=0x7f5a8802a8b0 comp rx=0 tx=0).stop 2024-07-04T17:26:29.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.685+0000 7f5aa6600640 1 -- 172.21.15.42:0/1554568146 shutdown_connections 2024-07-04T17:26:29.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.685+0000 7f5aa6600640 1 --2- 172.21.15.42:0/1554568146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5aa0144fe0 0x7f5aa006c020 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:29.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.685+0000 7f5aa6600640 1 -- 172.21.15.42:0/1554568146 >> 172.21.15.42:0/1554568146 conn(0x7f5aa00bebc0 msgr2=0x7f5aa00befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:29.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.686+0000 7f5aa6600640 1 -- 172.21.15.42:0/1554568146 shutdown_connections 2024-07-04T17:26:29.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.686+0000 7f5aa6600640 1 -- 172.21.15.42:0/1554568146 wait complete. 2024-07-04T17:26:29.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.688+0000 7f5aa6600640 1 Processor -- start 2024-07-04T17:26:29.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.688+0000 7f5aa6600640 1 -- start start 2024-07-04T17:26:29.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.690+0000 7f5aa6600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5aa00ba830 0x7f5aa01d6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:29.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.690+0000 7f5aa6600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa00b1e30 con 0x7f5aa00ba830 2024-07-04T17:26:29.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.690+0000 7f5aa5200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5aa00ba830 0x7f5aa01d6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:29.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.690+0000 7f5aa5200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5aa00ba830 0x7f5aa01d6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44194/0 (socket says 172.21.15.42:44194) 2024-07-04T17:26:29.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.690+0000 7f5aa5200640 1 -- 172.21.15.42:0/3109219757 learned_addr learned my addr 172.21.15.42:0/3109219757 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:29.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.690+0000 7f5aa5200640 1 -- 172.21.15.42:0/3109219757 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa0071420 con 0x7f5aa00ba830 2024-07-04T17:26:29.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.691+0000 7f5aa5200640 1 --2- 172.21.15.42:0/3109219757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5aa00ba830 0x7f5aa01d6ae0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000c380 tx=0x7f5a9000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:29.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.691+0000 7f5a9cc00640 1 -- 172.21.15.42:0/3109219757 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a900116c0 con 0x7f5aa00ba830 2024-07-04T17:26:29.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.691+0000 7f5a9cc00640 1 -- 172.21.15.42:0/3109219757 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a90017070 con 0x7f5aa00ba830 2024-07-04T17:26:29.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.691+0000 7f5aa6600640 1 -- 172.21.15.42:0/3109219757 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aa01d5df0 con 0x7f5aa00ba830 2024-07-04T17:26:29.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.691+0000 7f5a9cc00640 1 -- 172.21.15.42:0/3109219757 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a90011b20 con 0x7f5aa00ba830 2024-07-04T17:26:29.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.691+0000 7f5aa6600640 1 -- 172.21.15.42:0/3109219757 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aa01dc8e0 con 0x7f5aa00ba830 2024-07-04T17:26:29.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.692+0000 7f5aa6600640 1 -- 172.21.15.42:0/3109219757 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5aa01dcb70 con 0x7f5aa00ba830 2024-07-04T17:26:29.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.693+0000 7f5a9cc00640 1 -- 172.21.15.42:0/3109219757 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f5a90011cc0 con 0x7f5aa00ba830 2024-07-04T17:26:29.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.693+0000 7f5a9cc00640 1 --2- 172.21.15.42:0/3109219757 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f5a7c03d2b0 0x7f5a7c03f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:29.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.693+0000 7f5a9cc00640 1 -- 172.21.15.42:0/3109219757 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f5a90051370 con 0x7f5aa00ba830 2024-07-04T17:26:29.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.693+0000 7f5a9fe00640 1 --2- 172.21.15.42:0/3109219757 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f5a7c03d2b0 0x7f5a7c03f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:29.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.694+0000 7f5a9fe00640 1 --2- 172.21.15.42:0/3109219757 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f5a7c03d2b0 0x7f5a7c03f7c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5a88034000 tx=0x7f5a88004ab0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:29.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.698+0000 7f5a9cc00640 1 -- 172.21.15.42:0/3109219757 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f5a9000a140 con 0x7f5aa00ba830 2024-07-04T17:26:29.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.950+0000 7f5aa6600640 1 -- 172.21.15.42:0/3109219757 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5aa006c020 con 0x7f5aa00ba830 2024-07-04T17:26:29.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.951+0000 7f5a9cc00640 1 -- 172.21.15.42:0/3109219757 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6035 (secure 0 0 0) 0x7f5a90010cf0 con 0x7f5aa00ba830 2024-07-04T17:26:29.953 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:26:29.953 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":8,"btime":"2024-07-04T17:26:19:025883+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4192,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/3484212194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":3484212194},{"type":"v1","addr":"172.21.15.42:6827","nonce":3484212194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4202,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6830/815222782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6829","nonce":815222782},{"type":"v1","addr":"172.21.15.42:6830","nonce":815222782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/999331079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":999331079},{"type":"v1","addr":"172.21.15.42:6831","nonce":999331079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4204,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6833/2286778970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2286778970},{"type":"v1","addr":"172.21.15.42:6833","nonce":2286778970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:26:15.573602+0000","modified":"2024-07-04T17:26:18.625664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4205":{"gid":4205,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6835/3941225191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3941225191},{"type":"v1","addr":"172.21.15.42:6835","nonce":3941225191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4205,"qdb_cluster":[4205]},"id":1}]} 2024-07-04T17:26:29.954 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 8 2024-07-04T17:26:29.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.954+0000 7f5aa6600640 1 -- 172.21.15.42:0/3109219757 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f5a7c03d2b0 msgr2=0x7f5a7c03f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:29.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.954+0000 7f5aa6600640 1 --2- 172.21.15.42:0/3109219757 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f5a7c03d2b0 0x7f5a7c03f7c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5a88034000 tx=0x7f5a88004ab0 comp rx=0 tx=0).stop 2024-07-04T17:26:29.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.954+0000 7f5aa6600640 1 -- 172.21.15.42:0/3109219757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5aa00ba830 msgr2=0x7f5aa01d6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:29.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.954+0000 7f5aa6600640 1 --2- 172.21.15.42:0/3109219757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5aa00ba830 0x7f5aa01d6ae0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000c380 tx=0x7f5a9000c840 comp rx=0 tx=0).stop 2024-07-04T17:26:29.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.954+0000 7f5aa6600640 1 -- 172.21.15.42:0/3109219757 shutdown_connections 2024-07-04T17:26:29.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.954+0000 7f5aa6600640 1 --2- 172.21.15.42:0/3109219757 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f5a7c03d2b0 0x7f5a7c03f7c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:29.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.954+0000 7f5aa6600640 1 --2- 172.21.15.42:0/3109219757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5aa00ba830 0x7f5aa01d6ae0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:29.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.954+0000 7f5aa6600640 1 -- 172.21.15.42:0/3109219757 >> 172.21.15.42:0/3109219757 conn(0x7f5aa00bebc0 msgr2=0x7f5aa00b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:29.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.954+0000 7f5aa6600640 1 -- 172.21.15.42:0/3109219757 shutdown_connections 2024-07-04T17:26:29.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:29.955+0000 7f5aa6600640 1 -- 172.21.15.42:0/3109219757 wait complete. 2024-07-04T17:26:29.981 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:26:30.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.272+0000 7f16c1a00640 1 -- 172.21.15.42:0/1533112709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16bc0b1760 msgr2=0x7f16bc0b8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:30.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.272+0000 7f16c1a00640 1 --2- 172.21.15.42:0/1533112709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16bc0b1760 0x7f16bc0b8da0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f16ac005fd0 tx=0x7f16ac02a8b0 comp rx=0 tx=0).stop 2024-07-04T17:26:30.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.272+0000 7f16c1a00640 1 -- 172.21.15.42:0/1533112709 shutdown_connections 2024-07-04T17:26:30.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.272+0000 7f16c1a00640 1 --2- 172.21.15.42:0/1533112709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16bc0b1760 0x7f16bc0b8da0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:30.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.272+0000 7f16c1a00640 1 -- 172.21.15.42:0/1533112709 >> 172.21.15.42:0/1533112709 conn(0x7f16bc0bebc0 msgr2=0x7f16bc0befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:30.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.272+0000 7f16c1a00640 1 -- 172.21.15.42:0/1533112709 shutdown_connections 2024-07-04T17:26:30.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.272+0000 7f16c1a00640 1 -- 172.21.15.42:0/1533112709 wait complete. 2024-07-04T17:26:30.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.274+0000 7f16c1a00640 1 Processor -- start 2024-07-04T17:26:30.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.275+0000 7f16c1a00640 1 -- start start 2024-07-04T17:26:30.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.276+0000 7f16c1a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16bc06c020 0x7f16bc146130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:30.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.276+0000 7f16c1a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc14eed0 con 0x7f16bc06c020 2024-07-04T17:26:30.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.276+0000 7f16bbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16bc06c020 0x7f16bc146130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:30.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.276+0000 7f16bbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16bc06c020 0x7f16bc146130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44208/0 (socket says 172.21.15.42:44208) 2024-07-04T17:26:30.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.276+0000 7f16bbe00640 1 -- 172.21.15.42:0/3580324701 learned_addr learned my addr 172.21.15.42:0/3580324701 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:30.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.277+0000 7f16bbe00640 1 -- 172.21.15.42:0/3580324701 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16bc1452b0 con 0x7f16bc06c020 2024-07-04T17:26:30.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.277+0000 7f16bbe00640 1 --2- 172.21.15.42:0/3580324701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16bc06c020 0x7f16bc146130 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f16ac006010 tx=0x7f16ac033b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:30.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.277+0000 7f16b3e00640 1 -- 172.21.15.42:0/3580324701 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16ac0319f0 con 0x7f16bc06c020 2024-07-04T17:26:30.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.277+0000 7f16b3e00640 1 -- 172.21.15.42:0/3580324701 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16ac031b90 con 0x7f16bc06c020 2024-07-04T17:26:30.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.277+0000 7f16c1a00640 1 -- 172.21.15.42:0/3580324701 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16bc147e30 con 0x7f16bc06c020 2024-07-04T17:26:30.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.277+0000 7f16b3e00640 1 -- 172.21.15.42:0/3580324701 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16ac033070 con 0x7f16bc06c020 2024-07-04T17:26:30.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.278+0000 7f16c1a00640 1 -- 172.21.15.42:0/3580324701 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16bc1d7620 con 0x7f16bc06c020 2024-07-04T17:26:30.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.279+0000 7f16c1a00640 1 -- 172.21.15.42:0/3580324701 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16bc145480 con 0x7f16bc06c020 2024-07-04T17:26:30.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.279+0000 7f16b3e00640 1 -- 172.21.15.42:0/3580324701 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f16ac033210 con 0x7f16bc06c020 2024-07-04T17:26:30.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.280+0000 7f16b3e00640 1 --2- 172.21.15.42:0/3580324701 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f169c03d2b0 0x7f169c03f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:30.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.280+0000 7f16b3e00640 1 -- 172.21.15.42:0/3580324701 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f16ac072b80 con 0x7f16bc06c020 2024-07-04T17:26:30.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.280+0000 7f16bb400640 1 --2- 172.21.15.42:0/3580324701 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f169c03d2b0 0x7f169c03f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:30.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.281+0000 7f16bb400640 1 --2- 172.21.15.42:0/3580324701 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f169c03d2b0 0x7f169c03f7c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f16bc0b8760 tx=0x7f16a8007a90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:30.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.285+0000 7f16b3e00640 1 -- 172.21.15.42:0/3580324701 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f16ac03f030 con 0x7f16bc06c020 2024-07-04T17:26:30.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.537+0000 7f16c1a00640 1 -- 172.21.15.42:0/3580324701 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16bc0b8da0 con 0x7f16bc06c020 2024-07-04T17:26:30.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.538+0000 7f16b3e00640 1 -- 172.21.15.42:0/3580324701 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6035 (secure 0 0 0) 0x7f16ac033500 con 0x7f16bc06c020 2024-07-04T17:26:30.540 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:26:30.540 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":8,"btime":"2024-07-04T17:26:19:025883+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4192,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/3484212194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":3484212194},{"type":"v1","addr":"172.21.15.42:6827","nonce":3484212194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4202,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6830/815222782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6829","nonce":815222782},{"type":"v1","addr":"172.21.15.42:6830","nonce":815222782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/999331079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":999331079},{"type":"v1","addr":"172.21.15.42:6831","nonce":999331079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4204,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6833/2286778970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2286778970},{"type":"v1","addr":"172.21.15.42:6833","nonce":2286778970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:26:15.573602+0000","modified":"2024-07-04T17:26:18.625664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4205":{"gid":4205,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6835/3941225191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3941225191},{"type":"v1","addr":"172.21.15.42:6835","nonce":3941225191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4205,"qdb_cluster":[4205]},"id":1}]} 2024-07-04T17:26:30.540 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 8 2024-07-04T17:26:30.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.541+0000 7f16c1a00640 1 -- 172.21.15.42:0/3580324701 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f169c03d2b0 msgr2=0x7f169c03f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:30.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.541+0000 7f16c1a00640 1 --2- 172.21.15.42:0/3580324701 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f169c03d2b0 0x7f169c03f7c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f16bc0b8760 tx=0x7f16a8007a90 comp rx=0 tx=0).stop 2024-07-04T17:26:30.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.541+0000 7f16c1a00640 1 -- 172.21.15.42:0/3580324701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16bc06c020 msgr2=0x7f16bc146130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:30.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.541+0000 7f16c1a00640 1 --2- 172.21.15.42:0/3580324701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16bc06c020 0x7f16bc146130 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f16ac006010 tx=0x7f16ac033b10 comp rx=0 tx=0).stop 2024-07-04T17:26:30.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.541+0000 7f16c1a00640 1 -- 172.21.15.42:0/3580324701 shutdown_connections 2024-07-04T17:26:30.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.541+0000 7f16c1a00640 1 --2- 172.21.15.42:0/3580324701 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f169c03d2b0 0x7f169c03f7c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:30.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.541+0000 7f16c1a00640 1 --2- 172.21.15.42:0/3580324701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16bc06c020 0x7f16bc146130 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:30.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.541+0000 7f16c1a00640 1 -- 172.21.15.42:0/3580324701 >> 172.21.15.42:0/3580324701 conn(0x7f16bc0bebc0 msgr2=0x7f16bc0bbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:30.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.541+0000 7f16c1a00640 1 -- 172.21.15.42:0/3580324701 shutdown_connections 2024-07-04T17:26:30.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.541+0000 7f16c1a00640 1 -- 172.21.15.42:0/3580324701 wait complete. 2024-07-04T17:26:30.568 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:26:30.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.843+0000 7ff969000640 1 -- 172.21.15.42:0/3914370030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9640b38d0 msgr2=0x7ff9640b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:30.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.843+0000 7ff969000640 1 --2- 172.21.15.42:0/3914370030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9640b38d0 0x7ff9640b2d20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff9540097b0 tx=0x7ff95402a530 comp rx=0 tx=0).stop 2024-07-04T17:26:30.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.843+0000 7ff969000640 1 -- 172.21.15.42:0/3914370030 shutdown_connections 2024-07-04T17:26:30.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.843+0000 7ff969000640 1 --2- 172.21.15.42:0/3914370030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9640b38d0 0x7ff9640b2d20 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:30.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.843+0000 7ff969000640 1 -- 172.21.15.42:0/3914370030 >> 172.21.15.42:0/3914370030 conn(0x7ff9640bf700 msgr2=0x7ff9640af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:30.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.843+0000 7ff969000640 1 -- 172.21.15.42:0/3914370030 shutdown_connections 2024-07-04T17:26:30.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.843+0000 7ff969000640 1 -- 172.21.15.42:0/3914370030 wait complete. 2024-07-04T17:26:30.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.845+0000 7ff969000640 1 Processor -- start 2024-07-04T17:26:30.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.846+0000 7ff969000640 1 -- start start 2024-07-04T17:26:30.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.847+0000 7ff969000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9640b01f0 0x7ff9640bdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:30.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.847+0000 7ff969000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff96406cf50 con 0x7ff9640b01f0 2024-07-04T17:26:30.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.847+0000 7ff961e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9640b01f0 0x7ff9640bdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:30.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.847+0000 7ff961e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9640b01f0 0x7ff9640bdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44216/0 (socket says 172.21.15.42:44216) 2024-07-04T17:26:30.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.847+0000 7ff961e00640 1 -- 172.21.15.42:0/3096490858 learned_addr learned my addr 172.21.15.42:0/3096490858 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:30.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.848+0000 7ff961e00640 1 -- 172.21.15.42:0/3096490858 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9640be210 con 0x7ff9640b01f0 2024-07-04T17:26:30.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.848+0000 7ff961e00640 1 --2- 172.21.15.42:0/3096490858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9640b01f0 0x7ff9640bdc20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff9540098d0 tx=0x7ff954033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:30.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.848+0000 7ff95aa00640 1 -- 172.21.15.42:0/3096490858 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff954031880 con 0x7ff9640b01f0 2024-07-04T17:26:30.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.848+0000 7ff95aa00640 1 -- 172.21.15.42:0/3096490858 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff954031b10 con 0x7ff9640b01f0 2024-07-04T17:26:30.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.848+0000 7ff95aa00640 1 -- 172.21.15.42:0/3096490858 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff954031df0 con 0x7ff9640b01f0 2024-07-04T17:26:30.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.848+0000 7ff969000640 1 -- 172.21.15.42:0/3096490858 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9641e39a0 con 0x7ff9640b01f0 2024-07-04T17:26:30.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.848+0000 7ff969000640 1 -- 172.21.15.42:0/3096490858 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9641444a0 con 0x7ff9640b01f0 2024-07-04T17:26:30.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.850+0000 7ff969000640 1 -- 172.21.15.42:0/3096490858 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9640b2d20 con 0x7ff9640b01f0 2024-07-04T17:26:30.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.850+0000 7ff95aa00640 1 -- 172.21.15.42:0/3096490858 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7ff954033070 con 0x7ff9640b01f0 2024-07-04T17:26:30.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.850+0000 7ff95aa00640 1 --2- 172.21.15.42:0/3096490858 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff94003d2b0 0x7ff94003f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:30.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.850+0000 7ff95aa00640 1 -- 172.21.15.42:0/3096490858 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7ff954072890 con 0x7ff9640b01f0 2024-07-04T17:26:30.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.851+0000 7ff959400640 1 --2- 172.21.15.42:0/3096490858 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff94003d2b0 0x7ff94003f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:30.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.852+0000 7ff959400640 1 --2- 172.21.15.42:0/3096490858 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff94003d2b0 0x7ff94003f7c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff94c007fd0 tx=0x7ff94c007920 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:30.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:30.856+0000 7ff95aa00640 1 -- 172.21.15.42:0/3096490858 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7ff95403f030 con 0x7ff9640b01f0 2024-07-04T17:26:31.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.072+0000 7ff969000640 1 -- 172.21.15.42:0/3096490858 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9641421c0 con 0x7ff9640b01f0 2024-07-04T17:26:31.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.074+0000 7ff95aa00640 1 -- 172.21.15.42:0/3096490858 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) ==== 74+0+10603 (secure 0 0 0) 0x7ff95403a3a0 con 0x7ff9640b01f0 2024-07-04T17:26:31.076 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:26:31.076 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":15,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:26:17.008553+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:26:13.335715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-04T17:26:14.519899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:26:31.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.077+0000 7ff969000640 1 -- 172.21.15.42:0/3096490858 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff94003d2b0 msgr2=0x7ff94003f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:31.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.077+0000 7ff969000640 1 --2- 172.21.15.42:0/3096490858 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff94003d2b0 0x7ff94003f7c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff94c007fd0 tx=0x7ff94c007920 comp rx=0 tx=0).stop 2024-07-04T17:26:31.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.077+0000 7ff969000640 1 -- 172.21.15.42:0/3096490858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9640b01f0 msgr2=0x7ff9640bdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:31.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.077+0000 7ff969000640 1 --2- 172.21.15.42:0/3096490858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9640b01f0 0x7ff9640bdc20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff9540098d0 tx=0x7ff954033ad0 comp rx=0 tx=0).stop 2024-07-04T17:26:31.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.077+0000 7ff969000640 1 -- 172.21.15.42:0/3096490858 shutdown_connections 2024-07-04T17:26:31.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.077+0000 7ff969000640 1 --2- 172.21.15.42:0/3096490858 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff94003d2b0 0x7ff94003f7c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:31.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.077+0000 7ff969000640 1 --2- 172.21.15.42:0/3096490858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9640b01f0 0x7ff9640bdc20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:31.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.077+0000 7ff969000640 1 -- 172.21.15.42:0/3096490858 >> 172.21.15.42:0/3096490858 conn(0x7ff9640bf700 msgr2=0x7ff96413a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:31.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.077+0000 7ff969000640 1 -- 172.21.15.42:0/3096490858 shutdown_connections 2024-07-04T17:26:31.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.077+0000 7ff969000640 1 -- 172.21.15.42:0/3096490858 wait complete. 2024-07-04T17:26:31.104 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:26:31.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.384+0000 7ff400a00640 1 -- 172.21.15.42:0/1770538319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3fc06cea0 msgr2=0x7ff3fc0b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:31.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.384+0000 7ff400a00640 1 --2- 172.21.15.42:0/1770538319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3fc06cea0 0x7ff3fc0b3030 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec00b2b0 tx=0x7ff3ec02ad10 comp rx=0 tx=0).stop 2024-07-04T17:26:31.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.384+0000 7ff400a00640 1 -- 172.21.15.42:0/1770538319 shutdown_connections 2024-07-04T17:26:31.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.384+0000 7ff400a00640 1 --2- 172.21.15.42:0/1770538319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3fc06cea0 0x7ff3fc0b3030 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:31.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.384+0000 7ff400a00640 1 -- 172.21.15.42:0/1770538319 >> 172.21.15.42:0/1770538319 conn(0x7ff3fc0bf700 msgr2=0x7ff3fc0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:31.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.384+0000 7ff400a00640 1 -- 172.21.15.42:0/1770538319 shutdown_connections 2024-07-04T17:26:31.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.384+0000 7ff400a00640 1 -- 172.21.15.42:0/1770538319 wait complete. 2024-07-04T17:26:31.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.387+0000 7ff400a00640 1 Processor -- start 2024-07-04T17:26:31.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.387+0000 7ff400a00640 1 -- start start 2024-07-04T17:26:31.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.388+0000 7ff400a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3fc06cea0 0x7ff3fc070050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:31.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.388+0000 7ff400a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc06edd0 con 0x7ff3fc06cea0 2024-07-04T17:26:31.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.388+0000 7ff3f9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3fc06cea0 0x7ff3fc070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:31.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.389+0000 7ff3f9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3fc06cea0 0x7ff3fc070050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44230/0 (socket says 172.21.15.42:44230) 2024-07-04T17:26:31.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.389+0000 7ff3f9e00640 1 -- 172.21.15.42:0/3756834909 learned_addr learned my addr 172.21.15.42:0/3756834909 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:31.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.389+0000 7ff3f9e00640 1 -- 172.21.15.42:0/3756834909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3fc0730f0 con 0x7ff3fc06cea0 2024-07-04T17:26:31.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.389+0000 7ff3f9e00640 1 --2- 172.21.15.42:0/3756834909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3fc06cea0 0x7ff3fc070050 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec000c00 tx=0x7ff3ec0097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:31.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.390+0000 7ff3f2000640 1 -- 172.21.15.42:0/3756834909 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3ec038040 con 0x7ff3fc06cea0 2024-07-04T17:26:31.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.390+0000 7ff3f2000640 1 -- 172.21.15.42:0/3756834909 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3ec041070 con 0x7ff3fc06cea0 2024-07-04T17:26:31.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.390+0000 7ff3f2000640 1 -- 172.21.15.42:0/3756834909 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3ec004420 con 0x7ff3fc06cea0 2024-07-04T17:26:31.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.390+0000 7ff400a00640 1 -- 172.21.15.42:0/3756834909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3fc073ee0 con 0x7ff3fc06cea0 2024-07-04T17:26:31.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.390+0000 7ff400a00640 1 -- 172.21.15.42:0/3756834909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3fc072d70 con 0x7ff3fc06cea0 2024-07-04T17:26:31.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.391+0000 7ff400a00640 1 -- 172.21.15.42:0/3756834909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3fc0b3030 con 0x7ff3fc06cea0 2024-07-04T17:26:31.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.391+0000 7ff3f2000640 1 -- 172.21.15.42:0/3756834909 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7ff3ec0035f0 con 0x7ff3fc06cea0 2024-07-04T17:26:31.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.392+0000 7ff3f2000640 1 --2- 172.21.15.42:0/3756834909 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff3dc03d2b0 0x7ff3dc03f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:31.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.392+0000 7ff3f2000640 1 -- 172.21.15.42:0/3756834909 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7ff3ec071d40 con 0x7ff3fc06cea0 2024-07-04T17:26:31.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.392+0000 7ff3f9400640 1 --2- 172.21.15.42:0/3756834909 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff3dc03d2b0 0x7ff3dc03f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:31.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.393+0000 7ff3f9400640 1 --2- 172.21.15.42:0/3756834909 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff3dc03d2b0 0x7ff3dc03f7c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff3e80015c0 tx=0x7ff3e8007a90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:31.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.397+0000 7ff3f2000640 1 -- 172.21.15.42:0/3756834909 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7ff3ec077020 con 0x7ff3fc06cea0 2024-07-04T17:26:31.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.649+0000 7ff400a00640 1 -- 172.21.15.42:0/3756834909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3fc0732c0 con 0x7ff3fc06cea0 2024-07-04T17:26:31.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.650+0000 7ff3f2000640 1 -- 172.21.15.42:0/3756834909 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6035 (secure 0 0 0) 0x7ff3ec0453c0 con 0x7ff3fc06cea0 2024-07-04T17:26:31.652 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:26:31.652 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":8,"btime":"2024-07-04T17:26:19:025883+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4192,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/3484212194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":3484212194},{"type":"v1","addr":"172.21.15.42:6827","nonce":3484212194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4202,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6830/815222782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6829","nonce":815222782},{"type":"v1","addr":"172.21.15.42:6830","nonce":815222782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/999331079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":999331079},{"type":"v1","addr":"172.21.15.42:6831","nonce":999331079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4204,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6833/2286778970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2286778970},{"type":"v1","addr":"172.21.15.42:6833","nonce":2286778970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:26:15.573602+0000","modified":"2024-07-04T17:26:18.625664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4205":{"gid":4205,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6835/3941225191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3941225191},{"type":"v1","addr":"172.21.15.42:6835","nonce":3941225191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4205,"qdb_cluster":[4205]},"id":1}]} 2024-07-04T17:26:31.653 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 8 2024-07-04T17:26:31.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.653+0000 7ff400a00640 1 -- 172.21.15.42:0/3756834909 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff3dc03d2b0 msgr2=0x7ff3dc03f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:31.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.653+0000 7ff400a00640 1 --2- 172.21.15.42:0/3756834909 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff3dc03d2b0 0x7ff3dc03f7c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff3e80015c0 tx=0x7ff3e8007a90 comp rx=0 tx=0).stop 2024-07-04T17:26:31.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.653+0000 7ff400a00640 1 -- 172.21.15.42:0/3756834909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3fc06cea0 msgr2=0x7ff3fc070050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:31.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.653+0000 7ff400a00640 1 --2- 172.21.15.42:0/3756834909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3fc06cea0 0x7ff3fc070050 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec000c00 tx=0x7ff3ec0097d0 comp rx=0 tx=0).stop 2024-07-04T17:26:31.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.654+0000 7ff400a00640 1 -- 172.21.15.42:0/3756834909 shutdown_connections 2024-07-04T17:26:31.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.654+0000 7ff400a00640 1 --2- 172.21.15.42:0/3756834909 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff3dc03d2b0 0x7ff3dc03f7c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:31.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.654+0000 7ff400a00640 1 --2- 172.21.15.42:0/3756834909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3fc06cea0 0x7ff3fc070050 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:31.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.654+0000 7ff400a00640 1 -- 172.21.15.42:0/3756834909 >> 172.21.15.42:0/3756834909 conn(0x7ff3fc0bf700 msgr2=0x7ff3fc0a94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:31.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.654+0000 7ff400a00640 1 -- 172.21.15.42:0/3756834909 shutdown_connections 2024-07-04T17:26:31.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:31.654+0000 7ff400a00640 1 -- 172.21.15.42:0/3756834909 wait complete. 2024-07-04T17:26:32.183 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-04T17:26:15.573602+0000', 'modified': '2024-07-04T17:26:18.625664+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4205}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4205': {'gid': 4205, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.42:6835/3941225191', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6834', 'nonce': 3941225191}, {'type': 'v1', 'addr': '172.21.15.42:6835', 'nonce': 3941225191}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4205, 'qdb_cluster': [4205]} 2024-07-04T17:26:32.183 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-04T17:26:32.183 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-07-04T17:26:32.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.481+0000 7f9d1d200640 1 -- 172.21.15.42:0/1716439169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d1806cea0 msgr2=0x7f9d180b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:32.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.481+0000 7f9d1d200640 1 --2- 172.21.15.42:0/1716439169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d1806cea0 0x7f9d180b3030 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9d0800a8c0 tx=0x7f9d0802b860 comp rx=0 tx=0).stop 2024-07-04T17:26:32.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.481+0000 7f9d1d200640 1 -- 172.21.15.42:0/1716439169 shutdown_connections 2024-07-04T17:26:32.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.481+0000 7f9d1d200640 1 --2- 172.21.15.42:0/1716439169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d1806cea0 0x7f9d180b3030 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:32.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.481+0000 7f9d1d200640 1 -- 172.21.15.42:0/1716439169 >> 172.21.15.42:0/1716439169 conn(0x7f9d180bf700 msgr2=0x7f9d180af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:32.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.482+0000 7f9d1d200640 1 -- 172.21.15.42:0/1716439169 shutdown_connections 2024-07-04T17:26:32.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.482+0000 7f9d1d200640 1 -- 172.21.15.42:0/1716439169 wait complete. 2024-07-04T17:26:32.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.484+0000 7f9d1d200640 1 Processor -- start 2024-07-04T17:26:32.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.484+0000 7f9d1d200640 1 -- start start 2024-07-04T17:26:32.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.485+0000 7f9d1d200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d1806cea0 0x7f9d180ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:32.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.485+0000 7f9d1d200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9d180b0370 con 0x7f9d1806cea0 2024-07-04T17:26:32.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.486+0000 7f9d15e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d1806cea0 0x7f9d180ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:32.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.486+0000 7f9d15e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d1806cea0 0x7f9d180ae3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44242/0 (socket says 172.21.15.42:44242) 2024-07-04T17:26:32.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.486+0000 7f9d15e00640 1 -- 172.21.15.42:0/3837098356 learned_addr learned my addr 172.21.15.42:0/3837098356 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:32.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.486+0000 7f9d15e00640 1 -- 172.21.15.42:0/3837098356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d180705e0 con 0x7f9d1806cea0 2024-07-04T17:26:32.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.486+0000 7f9d15e00640 1 --2- 172.21.15.42:0/3837098356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d1806cea0 0x7f9d180ae3f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9d0000c380 tx=0x7f9d0000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:32.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.487+0000 7f9d0e000640 1 -- 172.21.15.42:0/3837098356 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d000116c0 con 0x7f9d1806cea0 2024-07-04T17:26:32.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.487+0000 7f9d0e000640 1 -- 172.21.15.42:0/3837098356 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d00017070 con 0x7f9d1806cea0 2024-07-04T17:26:32.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.487+0000 7f9d0e000640 1 -- 172.21.15.42:0/3837098356 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d00011b20 con 0x7f9d1806cea0 2024-07-04T17:26:32.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.487+0000 7f9d1d200640 1 -- 172.21.15.42:0/3837098356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d180ad590 con 0x7f9d1806cea0 2024-07-04T17:26:32.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.487+0000 7f9d1d200640 1 -- 172.21.15.42:0/3837098356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d18145ac0 con 0x7f9d1806cea0 2024-07-04T17:26:32.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.488+0000 7f9d0e000640 1 -- 172.21.15.42:0/3837098356 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f9d00011cc0 con 0x7f9d1806cea0 2024-07-04T17:26:32.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.489+0000 7f9d0e000640 1 --2- 172.21.15.42:0/3837098356 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f9cf403d2b0 0x7f9cf403f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:32.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.489+0000 7f9d0e000640 1 -- 172.21.15.42:0/3837098356 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f9d00051340 con 0x7f9d1806cea0 2024-07-04T17:26:32.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.489+0000 7f9d15400640 1 --2- 172.21.15.42:0/3837098356 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f9cf403d2b0 0x7f9cf403f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:32.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.490+0000 7f9d15400640 1 --2- 172.21.15.42:0/3837098356 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f9cf403d2b0 0x7f9cf403f7c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9d08036000 tx=0x7f9d08004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:32.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.491+0000 7f9d241bc740 1 Processor -- start 2024-07-04T17:26:32.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.492+0000 7f9d241bc740 1 -- start start 2024-07-04T17:26:32.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.493+0000 7f9d241bc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558118c9d270 0x558119043b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:32.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.493+0000 7f9d241bc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x558118fb45b0 con 0x558118c9d270 2024-07-04T17:26:32.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.493+0000 7f9d15e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558118c9d270 0x558119043b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:32.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.493+0000 7f9d15e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558118c9d270 0x558119043b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44254/0 (socket says 172.21.15.42:44254) 2024-07-04T17:26:32.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.493+0000 7f9d15e00640 1 -- 172.21.15.42:0/923831542 learned_addr learned my addr 172.21.15.42:0/923831542 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:32.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.493+0000 7f9d15e00640 1 -- 172.21.15.42:0/923831542 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558118d08850 con 0x558118c9d270 2024-07-04T17:26:32.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.494+0000 7f9d15e00640 1 --2- 172.21.15.42:0/923831542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558118c9d270 0x558119043b70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9d00010390 tx=0x7f9d0000a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=390bd23a09ff3094 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:32.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.494+0000 7f9d06a00640 1 -- 172.21.15.42:0/923831542 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d0004d070 con 0x558118c9d270 2024-07-04T17:26:32.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.494+0000 7f9d06a00640 1 -- 172.21.15.42:0/923831542 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d000421f0 con 0x558118c9d270 2024-07-04T17:26:32.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.494+0000 7f9d06a00640 1 -- 172.21.15.42:0/923831542 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d00019310 con 0x558118c9d270 2024-07-04T17:26:32.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.495+0000 7f9d241bc740 1 -- 172.21.15.42:0/923831542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558118c9d270 msgr2=0x558119043b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:32.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.495+0000 7f9d241bc740 1 --2- 172.21.15.42:0/923831542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558118c9d270 0x558119043b70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9d00010390 tx=0x7f9d0000a510 comp rx=0 tx=0).stop 2024-07-04T17:26:32.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.495+0000 7f9d241bc740 1 -- 172.21.15.42:0/923831542 shutdown_connections 2024-07-04T17:26:32.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.495+0000 7f9d241bc740 1 --2- 172.21.15.42:0/923831542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558118c9d270 0x558119043b70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:32.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.495+0000 7f9d241bc740 1 -- 172.21.15.42:0/923831542 >> 172.21.15.42:0/923831542 conn(0x558118c913c0 msgr2=0x558119042310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:32.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.495+0000 7f9d241bc740 1 -- 172.21.15.42:0/923831542 shutdown_connections 2024-07-04T17:26:32.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.495+0000 7f9d241bc740 1 -- 172.21.15.42:0/923831542 wait complete. 2024-07-04T17:26:32.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.496+0000 7f9d241bc740 1 -- start start 2024-07-04T17:26:32.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.496+0000 7f9d241bc740 1 Processor -- start 2024-07-04T17:26:32.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.497+0000 7f9d241bc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558118c9d270 0x558119066870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:32.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.497+0000 7f9d241bc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x558118fa7b90 con 0x558118c9d270 2024-07-04T17:26:32.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.498+0000 7f9d241bc740 10 client.0 _ll_register_callbacks cb 0x558118f4c8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T17:26:32.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.498+0000 7f9d15e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558118c9d270 0x558119066870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:32.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.498+0000 7f9d15e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558118c9d270 0x558119066870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44270/0 (socket says 172.21.15.42:44270) 2024-07-04T17:26:32.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.498+0000 7f9d15e00640 1 -- 172.21.15.42:0/887366967 learned_addr learned my addr 172.21.15.42:0/887366967 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:32.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.498+0000 7f9d15e00640 1 -- 172.21.15.42:0/887366967 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558118d7f6b0 con 0x558118c9d270 2024-07-04T17:26:32.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.498+0000 7f9d15e00640 1 --2- 172.21.15.42:0/887366967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558118c9d270 0x558119066870 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9d00010a00 tx=0x7f9d00019540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d022201da04fc702 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:32.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.498+0000 7f9cfb400640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-07-04T17:26:32.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.499+0000 7f9cfb400640 1 -- 172.21.15.42:0/887366967 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d00029070 con 0x558118c9d270 2024-07-04T17:26:32.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.499+0000 7f9cfb400640 1 -- 172.21.15.42:0/887366967 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f9d00015b10 con 0x558118c9d270 2024-07-04T17:26:32.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.499+0000 7f9d241bc740 1 -- 172.21.15.42:0/887366967 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558118f446c0 con 0x558118c9d270 2024-07-04T17:26:32.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.499+0000 7f9cfb400640 1 -- 172.21.15.42:0/887366967 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f9d00033050 con 0x558118c9d270 2024-07-04T17:26:32.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.499+0000 7f9d241bc740 10 client.4317 fetch_fsmap learned FSMap version 8 2024-07-04T17:26:32.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.499+0000 7f9d241bc740 1 -- 172.21.15.42:0/887366967 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55811905e1e0 con 0x558118c9d270 2024-07-04T17:26:32.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.499+0000 7f9cfb400640 1 -- 172.21.15.42:0/887366967 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x558118f446c0 con 0x558118c9d270 2024-07-04T17:26:32.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.499+0000 7f9cfb400640 1 -- 172.21.15.42:0/887366967 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 8) ==== 4107+0+0 (secure 0 0 0) 0x7f9d0000e280 con 0x558118c9d270 2024-07-04T17:26:32.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.499+0000 7f9d241bc740 10 client.4317 fetch_fsmap finished waiting for FSMap version 8 2024-07-04T17:26:32.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.499+0000 7f9d241bc740 10 client.4317 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-04T17:26:32.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.499+0000 7f9d241bc740 20 client.4317 populate_metadata read hostname 'smithi042' 2024-07-04T17:26:32.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.499+0000 7f9d241bc740 1 --2- 172.21.15.42:0/887366967 >> [v2:172.21.15.42:6834/3941225191,v1:172.21.15.42:6835/3941225191] conn(0x558118c743b0 0x55811905e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:32.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.499+0000 7f9d241bc740 4 client.4317 mds_command: new command op to 4205 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T17:26:32.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.499+0000 7f9d241bc740 1 -- 172.21.15.42:0/887366967 --> [v2:172.21.15.42:6834/3941225191,v1:172.21.15.42:6835/3941225191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558118fb6500 con 0x558118c743b0 2024-07-04T17:26:32.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.500+0000 7f9d15400640 1 --2- 172.21.15.42:0/887366967 >> [v2:172.21.15.42:6834/3941225191,v1:172.21.15.42:6835/3941225191] conn(0x558118c743b0 0x55811905e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:32.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.500+0000 7f9cfb400640 10 client.4317 ms_handle_connect on v2:172.21.15.42:6834/3941225191 2024-07-04T17:26:32.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.500+0000 7f9d15400640 1 --2- 172.21.15.42:0/887366967 >> [v2:172.21.15.42:6834/3941225191,v1:172.21.15.42:6835/3941225191] conn(0x558118c743b0 0x55811905e4d0 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=193e749841b81921 server_cookie=c4103762790ac393 in_seq=0 out_seq=0 2024-07-04T17:26:32.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.501+0000 7f9cfb400640 1 -- 172.21.15.42:0/887366967 <== mds.0 v2:172.21.15.42:6834/3941225191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558118fb6500 con 0x558118c743b0 2024-07-04T17:26:32.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.501+0000 7f9cfb400640 10 client.4317 handle_command_reply: tid=0 2024-07-04T17:26:32.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.501+0000 7f9d241bc740 1 client.4317 shutdown 2024-07-04T17:26:32.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.502+0000 7f9d241bc740 1 -- 172.21.15.42:0/887366967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558118c9d270 msgr2=0x558119066870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:32.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.502+0000 7f9d241bc740 1 --2- 172.21.15.42:0/887366967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558118c9d270 0x558119066870 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9d00010a00 tx=0x7f9d00019540 comp rx=0 tx=0).stop 2024-07-04T17:26:32.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.502+0000 7f9d241bc740 1 -- 172.21.15.42:0/887366967 shutdown_connections 2024-07-04T17:26:32.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.502+0000 7f9d241bc740 1 --2- 172.21.15.42:0/887366967 >> [v2:172.21.15.42:6834/3941225191,v1:172.21.15.42:6835/3941225191] conn(0x558118c743b0 0x55811905e4d0 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:32.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.502+0000 7f9d241bc740 1 --2- 172.21.15.42:0/887366967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558118c9d270 0x558119066870 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:32.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.502+0000 7f9d241bc740 1 -- 172.21.15.42:0/887366967 >> 172.21.15.42:0/887366967 conn(0x558118c913c0 msgr2=0x55811903d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:32.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.502+0000 7f9cfb400640 0 client.4317 ms_handle_reset on v2:172.21.15.42:6834/3941225191 2024-07-04T17:26:32.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.502+0000 7f9d241bc740 1 -- 172.21.15.42:0/887366967 shutdown_connections 2024-07-04T17:26:32.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.502+0000 7f9d241bc740 1 -- 172.21.15.42:0/887366967 wait complete. 2024-07-04T17:26:32.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.502+0000 7f9d241bc740 20 client.4317 trim_cache size 0 max 16384 2024-07-04T17:26:32.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.523+0000 7f9d241bc740 1 Processor -- start 2024-07-04T17:26:32.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.523+0000 7f9d241bc740 1 -- start start 2024-07-04T17:26:32.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.524+0000 7f9d241bc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558118c9d270 0x55811903f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:32.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.524+0000 7f9d241bc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9d181453f0 con 0x558118c9d270 2024-07-04T17:26:32.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.525+0000 7f9d15e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558118c9d270 0x55811903f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:32.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.525+0000 7f9d15e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558118c9d270 0x55811903f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44280/0 (socket says 172.21.15.42:44280) 2024-07-04T17:26:32.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.525+0000 7f9d15e00640 1 -- 172.21.15.42:0/3064007995 learned_addr learned my addr 172.21.15.42:0/3064007995 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:32.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.525+0000 7f9d15e00640 1 -- 172.21.15.42:0/3064007995 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581190670e0 con 0x558118c9d270 2024-07-04T17:26:32.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.525+0000 7f9d15e00640 1 --2- 172.21.15.42:0/3064007995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558118c9d270 0x55811903f5e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9d00040010 tx=0x7f9d00007c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6818518e8a6524 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:32.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.525+0000 7f9d06000640 1 -- 172.21.15.42:0/3064007995 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d00029040 con 0x558118c9d270 2024-07-04T17:26:32.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.525+0000 7f9d06000640 1 -- 172.21.15.42:0/3064007995 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d00050300 con 0x558118c9d270 2024-07-04T17:26:32.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.525+0000 7f9d06000640 1 -- 172.21.15.42:0/3064007995 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d0004d040 con 0x558118c9d270 2024-07-04T17:26:32.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.526+0000 7f9d241bc740 1 -- 172.21.15.42:0/3064007995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558118c9d270 msgr2=0x55811903f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:32.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.526+0000 7f9d241bc740 1 --2- 172.21.15.42:0/3064007995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558118c9d270 0x55811903f5e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9d00040010 tx=0x7f9d00007c70 comp rx=0 tx=0).stop 2024-07-04T17:26:32.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.526+0000 7f9d241bc740 1 -- 172.21.15.42:0/3064007995 shutdown_connections 2024-07-04T17:26:32.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.526+0000 7f9d241bc740 1 --2- 172.21.15.42:0/3064007995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558118c9d270 0x55811903f5e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:32.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.526+0000 7f9d241bc740 1 -- 172.21.15.42:0/3064007995 >> 172.21.15.42:0/3064007995 conn(0x558118c913c0 msgr2=0x5581190397a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:32.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.527+0000 7f9d241bc740 1 -- 172.21.15.42:0/3064007995 shutdown_connections 2024-07-04T17:26:32.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.527+0000 7f9d241bc740 1 -- 172.21.15.42:0/3064007995 wait complete. 2024-07-04T17:26:32.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.527+0000 7f9d241bc740 1 -- start start 2024-07-04T17:26:32.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.528+0000 7f9d241bc740 1 Processor -- start 2024-07-04T17:26:32.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.529+0000 7f9d241bc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558118c9d270 0x558119062250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:32.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.529+0000 7f9d241bc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9d180acf20 con 0x558118c9d270 2024-07-04T17:26:32.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.529+0000 7f9d241bc740 10 client.0 _ll_register_callbacks cb 0x7f9d000102e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T17:26:32.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.529+0000 7f9d15e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558118c9d270 0x558119062250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:32.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.529+0000 7f9d15e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558118c9d270 0x558119062250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44290/0 (socket says 172.21.15.42:44290) 2024-07-04T17:26:32.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.529+0000 7f9d15e00640 1 -- 172.21.15.42:0/7122760 learned_addr learned my addr 172.21.15.42:0/7122760 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:32.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.530+0000 7f9d15e00640 1 -- 172.21.15.42:0/7122760 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581190627b0 con 0x558118c9d270 2024-07-04T17:26:32.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.530+0000 7f9d15e00640 1 --2- 172.21.15.42:0/7122760 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558118c9d270 0x558119062250 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9d00010a00 tx=0x7f9d0000e500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cae5d2ec416be21 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:32.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.530+0000 7f9cfb400640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-07-04T17:26:32.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.530+0000 7f9cfb400640 1 -- 172.21.15.42:0/7122760 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d0004d040 con 0x558118c9d270 2024-07-04T17:26:32.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.530+0000 7f9cfb400640 1 -- 172.21.15.42:0/7122760 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f9d0003c1c0 con 0x558118c9d270 2024-07-04T17:26:32.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.530+0000 7f9d241bc740 1 -- 172.21.15.42:0/7122760 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d00050300 con 0x558118c9d270 2024-07-04T17:26:32.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.531+0000 7f9cfb400640 1 -- 172.21.15.42:0/7122760 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f9d00029040 con 0x558118c9d270 2024-07-04T17:26:32.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.531+0000 7f9d241bc740 10 client.4319 fetch_fsmap learned FSMap version 8 2024-07-04T17:26:32.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.531+0000 7f9d241bc740 1 -- 172.21.15.42:0/7122760 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x5581190627b0 con 0x558118c9d270 2024-07-04T17:26:32.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.531+0000 7f9cfb400640 1 -- 172.21.15.42:0/7122760 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d00050300 con 0x558118c9d270 2024-07-04T17:26:32.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.531+0000 7f9cfb400640 1 -- 172.21.15.42:0/7122760 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 8) ==== 4107+0+0 (secure 0 0 0) 0x7f9d0003c3c0 con 0x558118c9d270 2024-07-04T17:26:32.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.531+0000 7f9d241bc740 10 client.4319 fetch_fsmap finished waiting for FSMap version 8 2024-07-04T17:26:32.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.531+0000 7f9d241bc740 10 client.4319 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-04T17:26:32.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.531+0000 7f9d241bc740 20 client.4319 populate_metadata read hostname 'smithi042' 2024-07-04T17:26:32.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.531+0000 7f9d241bc740 1 --2- 172.21.15.42:0/7122760 >> [v2:172.21.15.42:6834/3941225191,v1:172.21.15.42:6835/3941225191] conn(0x558118c5be40 0x5581190f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:32.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.531+0000 7f9d241bc740 4 client.4319 mds_command: new command op to 4205 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-04T17:26:32.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.531+0000 7f9d241bc740 1 -- 172.21.15.42:0/7122760 --> [v2:172.21.15.42:6834/3941225191,v1:172.21.15.42:6835/3941225191] -- command(tid 0: {"prefix": "status"}) -- 0x558118fb1fa0 con 0x558118c5be40 2024-07-04T17:26:32.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.531+0000 7f9d15400640 1 --2- 172.21.15.42:0/7122760 >> [v2:172.21.15.42:6834/3941225191,v1:172.21.15.42:6835/3941225191] conn(0x558118c5be40 0x5581190f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:32.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.532+0000 7f9cfb400640 10 client.4319 ms_handle_connect on v2:172.21.15.42:6834/3941225191 2024-07-04T17:26:32.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.532+0000 7f9d15400640 1 --2- 172.21.15.42:0/7122760 >> [v2:172.21.15.42:6834/3941225191,v1:172.21.15.42:6835/3941225191] conn(0x558118c5be40 0x5581190f41e0 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f8c4526c5f15110 server_cookie=961a8236f44e2e69 in_seq=0 out_seq=0 2024-07-04T17:26:32.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.532+0000 7f9cfb400640 1 -- 172.21.15.42:0/7122760 <== mds.0 v2:172.21.15.42:6834/3941225191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x558118fb1fa0 con 0x558118c5be40 2024-07-04T17:26:32.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.532+0000 7f9cfb400640 10 client.4319 handle_command_reply: tid=0 2024-07-04T17:26:32.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.532+0000 7f9d241bc740 1 client.4319 shutdown 2024-07-04T17:26:32.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.532+0000 7f9d241bc740 1 -- 172.21.15.42:0/7122760 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558118c9d270 msgr2=0x558119062250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:32.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.532+0000 7f9d241bc740 1 --2- 172.21.15.42:0/7122760 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558118c9d270 0x558119062250 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9d00010a00 tx=0x7f9d0000e500 comp rx=0 tx=0).stop 2024-07-04T17:26:32.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.533+0000 7f9d241bc740 1 -- 172.21.15.42:0/7122760 shutdown_connections 2024-07-04T17:26:32.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.533+0000 7f9d241bc740 1 --2- 172.21.15.42:0/7122760 >> [v2:172.21.15.42:6834/3941225191,v1:172.21.15.42:6835/3941225191] conn(0x558118c5be40 0x5581190f41e0 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:32.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.533+0000 7f9d241bc740 1 --2- 172.21.15.42:0/7122760 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558118c9d270 0x558119062250 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:32.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.533+0000 7f9d241bc740 1 -- 172.21.15.42:0/7122760 >> 172.21.15.42:0/7122760 conn(0x558118c913c0 msgr2=0x55811903f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:32.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.533+0000 7f9cfb400640 0 client.4319 ms_handle_reset on v2:172.21.15.42:6834/3941225191 2024-07-04T17:26:32.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.533+0000 7f9d241bc740 1 -- 172.21.15.42:0/7122760 shutdown_connections 2024-07-04T17:26:32.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.533+0000 7f9d241bc740 1 -- 172.21.15.42:0/7122760 wait complete. 2024-07-04T17:26:32.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.533+0000 7f9d241bc740 20 client.4319 trim_cache size 0 max 16384 2024-07-04T17:26:32.536 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:26:32.536 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "be4f300a-93ef-4489-b540-37ece39b0aac", 2024-07-04T17:26:32.536 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-07-04T17:26:32.537 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 4205, 2024-07-04T17:26:32.537 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-07-04T17:26:32.537 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-07-04T17:26:32.537 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-07-04T17:26:32.537 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 16.524225660999999, 2024-07-04T17:26:32.537 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 8, 2024-07-04T17:26:32.537 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 15, 2024-07-04T17:26:32.537 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 15, 2024-07-04T17:26:32.537 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 23.113734463 2024-07-04T17:26:32.537 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:26:32.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.534+0000 7f9d1d200640 1 -- 172.21.15.42:0/3837098356 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f9cf403d2b0 msgr2=0x7f9cf403f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:32.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.534+0000 7f9d1d200640 1 --2- 172.21.15.42:0/3837098356 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f9cf403d2b0 0x7f9cf403f7c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9d08036000 tx=0x7f9d08004730 comp rx=0 tx=0).stop 2024-07-04T17:26:32.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.534+0000 7f9d1d200640 1 -- 172.21.15.42:0/3837098356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d1806cea0 msgr2=0x7f9d180ae3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:32.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.534+0000 7f9d1d200640 1 --2- 172.21.15.42:0/3837098356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d1806cea0 0x7f9d180ae3f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9d0000c380 tx=0x7f9d0000c840 comp rx=0 tx=0).stop 2024-07-04T17:26:32.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.534+0000 7f9d1d200640 1 -- 172.21.15.42:0/3837098356 shutdown_connections 2024-07-04T17:26:32.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.534+0000 7f9d1d200640 1 --2- 172.21.15.42:0/3837098356 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f9cf403d2b0 0x7f9cf403f7c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:32.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.534+0000 7f9d1d200640 1 --2- 172.21.15.42:0/3837098356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d1806cea0 0x7f9d180ae3f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:32.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.534+0000 7f9d1d200640 1 -- 172.21.15.42:0/3837098356 >> 172.21.15.42:0/3837098356 conn(0x7f9d180bf700 msgr2=0x7f9d1814bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:32.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.534+0000 7f9d1d200640 1 -- 172.21.15.42:0/3837098356 shutdown_connections 2024-07-04T17:26:32.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.534+0000 7f9d1d200640 1 -- 172.21.15.42:0/3837098356 wait complete. 2024-07-04T17:26:32.559 INFO:teuthology.run_tasks:Running task sequential... 2024-07-04T17:26:32.568 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-07-04T17:26:32.568 INFO:teuthology.task.sequential:In sequential, running task print... 2024-07-04T17:26:32.569 INFO:teuthology.task.print:Enabling mgr modules 2024-07-04T17:26:32.569 INFO:teuthology.run_tasks:Running task cephfs-mirror... 2024-07-04T17:26:32.619 INFO:tasks.cephfs_mirror.client.mirror:Restarting daemon 2024-07-04T17:26:32.620 DEBUG:teuthology.orchestra.run.smithi042:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term cephfs-mirror --cluster ceph --id mirror 2024-07-04T17:26:32.627 INFO:tasks.cephfs_mirror.client.mirror:Started 2024-07-04T17:26:32.628 INFO:teuthology.run_tasks:Running task exec... 2024-07-04T17:26:32.643 INFO:teuthology.task.exec:Executing custom commands... 2024-07-04T17:26:32.643 INFO:teuthology.task.exec:Running commands on role client.mirror host ubuntu@smithi042.front.sepia.ceph.com 2024-07-04T17:26:32.644 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph auth caps client.mirror mon '"'"'profile cephfs-mirror'"'"' mds '"'"'allow r'"'"' osd '"'"'allow rw tag cephfs metadata=*, allow r tag cephfs data=*'"'"' mgr '"'"'allow r'"'"'' 2024-07-04T17:26:32.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.936+0000 7f9483200640 1 -- 172.21.15.42:0/1365873445 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f947c072f50 msgr2=0x7f947c06d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:32.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.936+0000 7f9483200640 1 --2- 172.21.15.42:0/1365873445 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f947c072f50 0x7f947c06d430 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f946c005fd0 tx=0x7f946c02a5f0 comp rx=0 tx=0).stop 2024-07-04T17:26:32.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.936+0000 7f9483200640 1 -- 172.21.15.42:0/1365873445 shutdown_connections 2024-07-04T17:26:32.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.936+0000 7f9483200640 1 --2- 172.21.15.42:0/1365873445 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f947c072f50 0x7f947c06d430 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:32.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.936+0000 7f9483200640 1 -- 172.21.15.42:0/1365873445 >> 172.21.15.42:0/1365873445 conn(0x7f947c06c2b0 msgr2=0x7f947c06c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:32.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.937+0000 7f9483200640 1 -- 172.21.15.42:0/1365873445 shutdown_connections 2024-07-04T17:26:32.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.937+0000 7f9483200640 1 -- 172.21.15.42:0/1365873445 wait complete. 2024-07-04T17:26:32.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.939+0000 7f9483200640 1 Processor -- start 2024-07-04T17:26:32.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.939+0000 7f9483200640 1 -- start start 2024-07-04T17:26:32.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.940+0000 7f9483200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f947c072f50 0x7f947c0af910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:32.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.940+0000 7f9483200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f947c1416e0 con 0x7f947c072f50 2024-07-04T17:26:32.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.940+0000 7f947be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f947c072f50 0x7f947c0af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:32.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.940+0000 7f947be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f947c072f50 0x7f947c0af910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44340/0 (socket says 172.21.15.42:44340) 2024-07-04T17:26:32.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.940+0000 7f947be00640 1 -- 172.21.15.42:0/472551706 learned_addr learned my addr 172.21.15.42:0/472551706 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:32.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.941+0000 7f947be00640 1 -- 172.21.15.42:0/472551706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f947c0addc0 con 0x7f947c072f50 2024-07-04T17:26:32.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.941+0000 7f947be00640 1 --2- 172.21.15.42:0/472551706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f947c072f50 0x7f947c0af910 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f946c009850 tx=0x7f946c0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:32.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.941+0000 7f9473e00640 1 -- 172.21.15.42:0/472551706 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f946c03e030 con 0x7f947c072f50 2024-07-04T17:26:32.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.941+0000 7f9473e00640 1 -- 172.21.15.42:0/472551706 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f946c0035f0 con 0x7f947c072f50 2024-07-04T17:26:32.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.941+0000 7f9473e00640 1 -- 172.21.15.42:0/472551706 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f946c035040 con 0x7f947c072f50 2024-07-04T17:26:32.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.941+0000 7f9483200640 1 -- 172.21.15.42:0/472551706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f947c0b0f70 con 0x7f947c072f50 2024-07-04T17:26:32.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.942+0000 7f9483200640 1 -- 172.21.15.42:0/472551706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f947c0b1430 con 0x7f947c072f50 2024-07-04T17:26:32.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.943+0000 7f9483200640 1 -- 172.21.15.42:0/472551706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f947c151be0 con 0x7f947c072f50 2024-07-04T17:26:32.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.943+0000 7f9473e00640 1 -- 172.21.15.42:0/472551706 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f946c041030 con 0x7f947c072f50 2024-07-04T17:26:32.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.943+0000 7f9473e00640 1 --2- 172.21.15.42:0/472551706 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f945803d2b0 0x7f945803f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:32.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.943+0000 7f9473e00640 1 -- 172.21.15.42:0/472551706 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f946c03f070 con 0x7f947c072f50 2024-07-04T17:26:32.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.944+0000 7f947b400640 1 --2- 172.21.15.42:0/472551706 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f945803d2b0 0x7f945803f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:32.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.944+0000 7f947b400640 1 --2- 172.21.15.42:0/472551706 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f945803d2b0 0x7f945803f7c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f947c146bc0 tx=0x7f94640079f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:32.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:32.948+0000 7f9473e00640 1 -- 172.21.15.42:0/472551706 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f946c077020 con 0x7f947c072f50 2024-07-04T17:26:33.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.238+0000 7f9483200640 1 -- 172.21.15.42:0/472551706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.mirror", "caps": ["mon", "profile cephfs-mirror", "mds", "allow r", "osd", "allow rw tag cephfs metadata=*, allow r tag cephfs data=*", "mgr", "allow r"]} v 0) -- 0x7f947c1e3960 con 0x7f947c072f50 2024-07-04T17:26:33.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.243+0000 7f9473e00640 1 -- 172.21.15.42:0/472551706 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.mirror", "caps": ["mon", "profile cephfs-mirror", "mds", "allow r", "osd", "allow rw tag cephfs metadata=*, allow r tag cephfs data=*", "mgr", "allow r"]}]=0 updated caps for client.mirror v3) ==== 260+0+216 (secure 0 0 0) 0x7f946c04bd60 con 0x7f947c072f50 2024-07-04T17:26:33.246 INFO:teuthology.orchestra.run.smithi042.stdout:[client.mirror] 2024-07-04T17:26:33.246 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAO24ZmBJzBHhAAR4Zg36YTtVizxm/g/Cdk0w== 2024-07-04T17:26:33.246 INFO:teuthology.orchestra.run.smithi042.stdout: caps mds = "allow r" 2024-07-04T17:26:33.246 INFO:teuthology.orchestra.run.smithi042.stdout: caps mgr = "allow r" 2024-07-04T17:26:33.246 INFO:teuthology.orchestra.run.smithi042.stdout: caps mon = "profile cephfs-mirror" 2024-07-04T17:26:33.246 INFO:teuthology.orchestra.run.smithi042.stdout: caps osd = "allow rw tag cephfs metadata=*, allow r tag cephfs data=*" 2024-07-04T17:26:33.246 INFO:teuthology.orchestra.run.smithi042.stderr:updated caps for client.mirror 2024-07-04T17:26:33.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.246+0000 7f9483200640 1 -- 172.21.15.42:0/472551706 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f945803d2b0 msgr2=0x7f945803f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:33.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.246+0000 7f9483200640 1 --2- 172.21.15.42:0/472551706 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f945803d2b0 0x7f945803f7c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f947c146bc0 tx=0x7f94640079f0 comp rx=0 tx=0).stop 2024-07-04T17:26:33.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.246+0000 7f9483200640 1 -- 172.21.15.42:0/472551706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f947c072f50 msgr2=0x7f947c0af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:33.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.246+0000 7f9483200640 1 --2- 172.21.15.42:0/472551706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f947c072f50 0x7f947c0af910 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f946c009850 tx=0x7f946c0049d0 comp rx=0 tx=0).stop 2024-07-04T17:26:33.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.246+0000 7f9483200640 1 -- 172.21.15.42:0/472551706 shutdown_connections 2024-07-04T17:26:33.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.246+0000 7f9483200640 1 --2- 172.21.15.42:0/472551706 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f945803d2b0 0x7f945803f7c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:33.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.246+0000 7f9483200640 1 --2- 172.21.15.42:0/472551706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f947c072f50 0x7f947c0af910 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:33.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.246+0000 7f9483200640 1 -- 172.21.15.42:0/472551706 >> 172.21.15.42:0/472551706 conn(0x7f947c06c2b0 msgr2=0x7f947c0b5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:33.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.247+0000 7f9483200640 1 -- 172.21.15.42:0/472551706 shutdown_connections 2024-07-04T17:26:33.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.247+0000 7f9483200640 1 -- 172.21.15.42:0/472551706 wait complete. 2024-07-04T17:26:33.275 INFO:teuthology.task.exec:Running commands on role client.mirror_remote host ubuntu@smithi042.front.sepia.ceph.com 2024-07-04T17:26:33.276 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph auth caps client.mirror_remote mon '"'"'allow r'"'"' mds '"'"'allow rwps'"'"' osd '"'"'allow rw tag cephfs *=*'"'"' mgr '"'"'allow r'"'"'' 2024-07-04T17:26:33.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.639+0000 7ff97e400640 1 -- 172.21.15.42:0/1599452777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff97806d430 msgr2=0x7ff9780710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:33.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.639+0000 7ff97e400640 1 --2- 172.21.15.42:0/1599452777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff97806d430 0x7ff9780710e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff964005fd0 tx=0x7ff96402a8b0 comp rx=0 tx=0).stop 2024-07-04T17:26:33.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.640+0000 7ff97e400640 1 -- 172.21.15.42:0/1599452777 shutdown_connections 2024-07-04T17:26:33.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.640+0000 7ff97e400640 1 --2- 172.21.15.42:0/1599452777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff97806d430 0x7ff9780710e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:33.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.640+0000 7ff97e400640 1 -- 172.21.15.42:0/1599452777 >> 172.21.15.42:0/1599452777 conn(0x7ff97806c2b0 msgr2=0x7ff97806c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:33.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.640+0000 7ff97e400640 1 -- 172.21.15.42:0/1599452777 shutdown_connections 2024-07-04T17:26:33.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.640+0000 7ff97e400640 1 -- 172.21.15.42:0/1599452777 wait complete. 2024-07-04T17:26:33.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.643+0000 7ff97e400640 1 Processor -- start 2024-07-04T17:26:33.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.643+0000 7ff97e400640 1 -- start start 2024-07-04T17:26:33.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.644+0000 7ff97e400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff97806d430 0x7ff9780b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:33.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.644+0000 7ff97e400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff9781416e0 con 0x7ff97806d430 2024-07-04T17:26:33.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.645+0000 7ff977400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff97806d430 0x7ff9780b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:33.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.645+0000 7ff977400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff97806d430 0x7ff9780b0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44358/0 (socket says 172.21.15.42:44358) 2024-07-04T17:26:33.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.645+0000 7ff977400640 1 -- 172.21.15.42:0/836691826 learned_addr learned my addr 172.21.15.42:0/836691826 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:33.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.645+0000 7ff977400640 1 -- 172.21.15.42:0/836691826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9780b1c60 con 0x7ff97806d430 2024-07-04T17:26:33.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.646+0000 7ff977400640 1 --2- 172.21.15.42:0/836691826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff97806d430 0x7ff9780b0de0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff96800c380 tx=0x7ff96800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:33.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.646+0000 7ff96f400640 1 -- 172.21.15.42:0/836691826 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9680116c0 con 0x7ff97806d430 2024-07-04T17:26:33.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.646+0000 7ff96f400640 1 -- 172.21.15.42:0/836691826 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff968017070 con 0x7ff97806d430 2024-07-04T17:26:33.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.646+0000 7ff96f400640 1 -- 172.21.15.42:0/836691826 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff968011b20 con 0x7ff97806d430 2024-07-04T17:26:33.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.646+0000 7ff97e400640 1 -- 172.21.15.42:0/836691826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9780af160 con 0x7ff97806d430 2024-07-04T17:26:33.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.646+0000 7ff97e400640 1 -- 172.21.15.42:0/836691826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9781e6f00 con 0x7ff97806d430 2024-07-04T17:26:33.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.648+0000 7ff97e400640 1 -- 172.21.15.42:0/836691826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff97806ca20 con 0x7ff97806d430 2024-07-04T17:26:33.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.648+0000 7ff96f400640 1 -- 172.21.15.42:0/836691826 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7ff968011cc0 con 0x7ff97806d430 2024-07-04T17:26:33.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.649+0000 7ff96f400640 1 --2- 172.21.15.42:0/836691826 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff95803d2b0 0x7ff95803f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:33.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.649+0000 7ff96f400640 1 -- 172.21.15.42:0/836691826 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7ff968051300 con 0x7ff97806d430 2024-07-04T17:26:33.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.649+0000 7ff976a00640 1 --2- 172.21.15.42:0/836691826 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff95803d2b0 0x7ff95803f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:33.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.650+0000 7ff976a00640 1 --2- 172.21.15.42:0/836691826 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff95803d2b0 0x7ff95803f7c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff964034000 tx=0x7ff964004ab0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:33.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.654+0000 7ff96f400640 1 -- 172.21.15.42:0/836691826 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7ff97806ca20 con 0x7ff97806d430 2024-07-04T17:26:33.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.940+0000 7ff97e400640 1 -- 172.21.15.42:0/836691826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.mirror_remote", "caps": ["mon", "allow r", "mds", "allow rwps", "osd", "allow rw tag cephfs *=*", "mgr", "allow r"]} v 0) -- 0x7ff978141db0 con 0x7ff97806d430 2024-07-04T17:26:33.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.945+0000 7ff96f400640 1 -- 172.21.15.42:0/836691826 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.mirror_remote", "caps": ["mon", "allow r", "mds", "allow rwps", "osd", "allow rw tag cephfs *=*", "mgr", "allow r"]}]=0 updated caps for client.mirror_remote v4) ==== 229+0+178 (secure 0 0 0) 0x7ff97806d840 con 0x7ff97806d430 2024-07-04T17:26:33.947 INFO:teuthology.orchestra.run.smithi042.stdout:[client.mirror_remote] 2024-07-04T17:26:33.947 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAO24Zm+BsqLhAA19GVuEjfWQeMe/fSc+dmoA== 2024-07-04T17:26:33.947 INFO:teuthology.orchestra.run.smithi042.stdout: caps mds = "allow rwps" 2024-07-04T17:26:33.947 INFO:teuthology.orchestra.run.smithi042.stdout: caps mgr = "allow r" 2024-07-04T17:26:33.947 INFO:teuthology.orchestra.run.smithi042.stdout: caps mon = "allow r" 2024-07-04T17:26:33.947 INFO:teuthology.orchestra.run.smithi042.stdout: caps osd = "allow rw tag cephfs *=*" 2024-07-04T17:26:33.947 INFO:teuthology.orchestra.run.smithi042.stderr:updated caps for client.mirror_remote 2024-07-04T17:26:33.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.948+0000 7ff97e400640 1 -- 172.21.15.42:0/836691826 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff95803d2b0 msgr2=0x7ff95803f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:33.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.948+0000 7ff97e400640 1 --2- 172.21.15.42:0/836691826 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff95803d2b0 0x7ff95803f7c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff964034000 tx=0x7ff964004ab0 comp rx=0 tx=0).stop 2024-07-04T17:26:33.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.948+0000 7ff97e400640 1 -- 172.21.15.42:0/836691826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff97806d430 msgr2=0x7ff9780b0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:33.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.948+0000 7ff97e400640 1 --2- 172.21.15.42:0/836691826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff97806d430 0x7ff9780b0de0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff96800c380 tx=0x7ff96800c840 comp rx=0 tx=0).stop 2024-07-04T17:26:33.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.948+0000 7ff97e400640 1 -- 172.21.15.42:0/836691826 shutdown_connections 2024-07-04T17:26:33.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.948+0000 7ff97e400640 1 --2- 172.21.15.42:0/836691826 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff95803d2b0 0x7ff95803f7c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:33.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.948+0000 7ff97e400640 1 --2- 172.21.15.42:0/836691826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff97806d430 0x7ff9780b0de0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:33.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.948+0000 7ff97e400640 1 -- 172.21.15.42:0/836691826 >> 172.21.15.42:0/836691826 conn(0x7ff97806c2b0 msgr2=0x7ff9780b88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:33.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.948+0000 7ff97e400640 1 -- 172.21.15.42:0/836691826 shutdown_connections 2024-07-04T17:26:33.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:33.948+0000 7ff97e400640 1 -- 172.21.15.42:0/836691826 wait complete. 2024-07-04T17:26:33.979 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-07-04T17:26:34.003 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-07-04T17:26:34.003 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-07-04T17:26:34.004 INFO:tasks.ceph_fuse:client.0 config is {} 2024-07-04T17:26:34.004 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-07-04T17:26:34.004 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-04T17:26:34.004 INFO:tasks.ceph_fuse:client.1 config is {} 2024-07-04T17:26:34.004 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-07-04T17:26:34.004 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-07-04T17:26:34.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:26:34.004 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-07-04T17:26:34.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:26:34.062 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link delete ceph-brx 2024-07-04T17:26:34.156 INFO:teuthology.orchestra.run.smithi042.stderr:Cannot find device "ceph-brx" 2024-07-04T17:26:34.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-04T17:26:34.161 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-07-04T17:26:34.161 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-07-04T17:26:34.161 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-04T17:26:34.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.450+0000 7f8fb3c00640 1 -- 172.21.15.42:0/786147298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fac0b02b0 msgr2=0x7f8fac0b3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:34.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.450+0000 7f8fb3c00640 1 --2- 172.21.15.42:0/786147298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fac0b02b0 0x7f8fac0b3650 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8fa00097b0 tx=0x7f8fa002a4a0 comp rx=0 tx=0).stop 2024-07-04T17:26:34.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.450+0000 7f8fb3c00640 1 -- 172.21.15.42:0/786147298 shutdown_connections 2024-07-04T17:26:34.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.450+0000 7f8fb3c00640 1 --2- 172.21.15.42:0/786147298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fac0b02b0 0x7f8fac0b3650 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:34.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.450+0000 7f8fb3c00640 1 -- 172.21.15.42:0/786147298 >> 172.21.15.42:0/786147298 conn(0x7f8fac0bf700 msgr2=0x7f8fac0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:34.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.450+0000 7f8fb3c00640 1 -- 172.21.15.42:0/786147298 shutdown_connections 2024-07-04T17:26:34.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.450+0000 7f8fb3c00640 1 -- 172.21.15.42:0/786147298 wait complete. 2024-07-04T17:26:34.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.452+0000 7f8fb3c00640 1 Processor -- start 2024-07-04T17:26:34.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.452+0000 7f8fb3c00640 1 -- start start 2024-07-04T17:26:34.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.454+0000 7f8fb3c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fac0b02b0 0x7f8fac0b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:34.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.454+0000 7f8fb3c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac06c710 con 0x7f8fac0b02b0 2024-07-04T17:26:34.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.454+0000 7f8fb1200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fac0b02b0 0x7f8fac0b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:34.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.454+0000 7f8fb1200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fac0b02b0 0x7f8fac0b7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39396/0 (socket says 172.21.15.42:39396) 2024-07-04T17:26:34.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.454+0000 7f8fb1200640 1 -- 172.21.15.42:0/4003796159 learned_addr learned my addr 172.21.15.42:0/4003796159 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:34.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.454+0000 7f8fb1200640 1 -- 172.21.15.42:0/4003796159 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fac0b9420 con 0x7f8fac0b02b0 2024-07-04T17:26:34.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.455+0000 7f8fb1200640 1 --2- 172.21.15.42:0/4003796159 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fac0b02b0 0x7f8fac0b7140 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8fa00098d0 tx=0x7f8fa0033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:34.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.455+0000 7f8fa8c00640 1 -- 172.21.15.42:0/4003796159 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fa0031880 con 0x7f8fac0b02b0 2024-07-04T17:26:34.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.455+0000 7f8fa8c00640 1 -- 172.21.15.42:0/4003796159 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fa0031b10 con 0x7f8fac0b02b0 2024-07-04T17:26:34.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.455+0000 7f8fa8c00640 1 -- 172.21.15.42:0/4003796159 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fa0031df0 con 0x7f8fac0b02b0 2024-07-04T17:26:34.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.455+0000 7f8fb3c00640 1 -- 172.21.15.42:0/4003796159 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fac1e4570 con 0x7f8fac0b02b0 2024-07-04T17:26:34.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.455+0000 7f8fb3c00640 1 -- 172.21.15.42:0/4003796159 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fac0ba900 con 0x7f8fac0b02b0 2024-07-04T17:26:34.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.456+0000 7f8fa8c00640 1 -- 172.21.15.42:0/4003796159 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f8fa00330d0 con 0x7f8fac0b02b0 2024-07-04T17:26:34.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.456+0000 7f8fb3c00640 1 -- 172.21.15.42:0/4003796159 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fac06cd50 con 0x7f8fac0b02b0 2024-07-04T17:26:34.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.457+0000 7f8fa8c00640 1 --2- 172.21.15.42:0/4003796159 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f8f8c03d2b0 0x7f8f8c03f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:34.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.457+0000 7f8fa8c00640 1 -- 172.21.15.42:0/4003796159 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f8fa0072900 con 0x7f8fac0b02b0 2024-07-04T17:26:34.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.457+0000 7f8fabe00640 1 --2- 172.21.15.42:0/4003796159 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f8f8c03d2b0 0x7f8f8c03f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:34.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.458+0000 7f8fabe00640 1 --2- 172.21.15.42:0/4003796159 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f8f8c03d2b0 0x7f8f8c03f7c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8f980015e0 tx=0x7f8f98001290 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:34.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.463+0000 7f8fa8c00640 1 -- 172.21.15.42:0/4003796159 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f8fa003f030 con 0x7f8fac0b02b0 2024-07-04T17:26:34.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.714+0000 7f8fb3c00640 1 -- 172.21.15.42:0/4003796159 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8fac0be150 con 0x7f8fac0b02b0 2024-07-04T17:26:34.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.714+0000 7f8fa8c00640 1 -- 172.21.15.42:0/4003796159 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f8fa003a3a0 con 0x7f8fac0b02b0 2024-07-04T17:26:34.716 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-04T17:26:34.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.717+0000 7f8fb3c00640 1 -- 172.21.15.42:0/4003796159 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f8f8c03d2b0 msgr2=0x7f8f8c03f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:34.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.717+0000 7f8fb3c00640 1 --2- 172.21.15.42:0/4003796159 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f8f8c03d2b0 0x7f8f8c03f7c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8f980015e0 tx=0x7f8f98001290 comp rx=0 tx=0).stop 2024-07-04T17:26:34.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.717+0000 7f8fb3c00640 1 -- 172.21.15.42:0/4003796159 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fac0b02b0 msgr2=0x7f8fac0b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:34.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.717+0000 7f8fb3c00640 1 --2- 172.21.15.42:0/4003796159 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fac0b02b0 0x7f8fac0b7140 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8fa00098d0 tx=0x7f8fa0033ad0 comp rx=0 tx=0).stop 2024-07-04T17:26:34.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.717+0000 7f8fb3c00640 1 -- 172.21.15.42:0/4003796159 shutdown_connections 2024-07-04T17:26:34.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.717+0000 7f8fb3c00640 1 --2- 172.21.15.42:0/4003796159 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f8f8c03d2b0 0x7f8f8c03f7c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:34.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.717+0000 7f8fb3c00640 1 --2- 172.21.15.42:0/4003796159 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8fac0b02b0 0x7f8fac0b7140 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:34.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.717+0000 7f8fb3c00640 1 -- 172.21.15.42:0/4003796159 >> 172.21.15.42:0/4003796159 conn(0x7f8fac0bf700 msgr2=0x7f8fac145320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:34.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.717+0000 7f8fb3c00640 1 -- 172.21.15.42:0/4003796159 shutdown_connections 2024-07-04T17:26:34.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34.717+0000 7f8fb3c00640 1 -- 172.21.15.42:0/4003796159 wait complete. 2024-07-04T17:26:34.748 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-07-04T17:26:34.748 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-04T17:26:34.748 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2024-07-04T17:26:34.748 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-04T17:26:34.748 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-04T17:26:34.748 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-07-04T17:26:34.748 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-07-04T17:26:34.748 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-04T17:26:34.748 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-04T17:26:34.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:26:34.749 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2024-07-04T17:26:34.802 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-04T17:26:34.802 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-04T17:26:34.802 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-07-04T17:26:34.802 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:26:34.802 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-07-04T17:26:34.802 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:26:34.802 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:26:34.802 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2024-07-04T17:26:34.802 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f0 2024-07-04T17:26:34.802 INFO:teuthology.orchestra.run.smithi042.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:26:34.803 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2024-07-04T17:26:34.803 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f1 2024-07-04T17:26:34.803 INFO:teuthology.orchestra.run.smithi042.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-04T17:26:34.803 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:26:34.803 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f0 2024-07-04T17:26:34.803 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-07-04T17:26:34.803 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 42575sec preferred_lft 42575sec 2024-07-04T17:26:34.803 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::ec4:7aff:febd:1440/64 scope link noprefixroute 2024-07-04T17:26:34.803 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:26:34.803 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:26:34.803 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:26:34.803 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f1 2024-07-04T17:26:34.804 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-07-04T17:26:34.805 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-04T17:26:34.805 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-07-04T17:26:34.805 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link add name ceph-brx type bridge 2024-07-04T17:26:34.805 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip addr flush dev ceph-brx 2024-07-04T17:26:34.805 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set ceph-brx up 2024-07-04T17:26:34.805 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-07-04T17:26:34.805 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-07-04T17:26:34.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-04T17:26:35.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:26:35.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:26:35.307 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-04T17:26:35.379 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-07-04T17:26:35.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:26:35.383 DEBUG:teuthology.orchestra.run.smithi042:> route 2024-07-04T17:26:35.439 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2024-07-04T17:26:35.439 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-04T17:26:35.439 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-04T17:26:35.439 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-04T17:26:35.439 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-04T17:26:35.440 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-04T17:26:35.440 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-07-04T17:26:35.440 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-04T17:26:35.440 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-04T17:26:35.440 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-04T17:26:35.440 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-07-04T17:26:35.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-04T17:26:36.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:26:36.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:26:36.168 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-07-04T17:26:36.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:26:36.212 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list-id 2024-07-04T17:26:36.301 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-04T17:26:36.301 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-07-04T17:26:36.301 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:26:36.301 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-07-04T17:26:36.301 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-07-04T17:26:36.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-04T17:26:36.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:26:36.527 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-07-04T17:26:36.528 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-04T17:26:36.528 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-07-04T17:26:36.528 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-07-04T17:26:36.528 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-07-04T17:26:36.528 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-07-04T17:26:36.528 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-07-04T17:26:36.528 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-07-04T17:26:36.528 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-07-04T17:26:36.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-04T17:26:36.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:26:36.813 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-04T17:26:36.814 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-07-04T17:26:36.814 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.0 up 2024-07-04T17:26:36.814 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set dev brx.0 master ceph-brx 2024-07-04T17:26:36.814 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-07-04T17:26:36.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-04T17:26:37.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:26:37.008 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-07-04T17:26:37.008 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-04T17:26:37.009 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-04T17:26:37.049 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-04T17:26:37.049 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-04T17:26:37.050 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-04T17:26:37.140 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:26:37.207 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:26:37.247 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:26:37.247 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:26:37.247 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:26:37.247 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:26:37.247 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:26:37.247 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:26:37.247 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:26:37.247 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:26:37.248 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:26:37.248 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:26:37.248 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:26:37.248 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:26:37.248 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:26:37.248 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:26:37.248 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:26:37.248 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:26:37.248 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:26:37.248 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:26:37.248 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:26:37.249 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:26:37.249 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:26:37.249 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:26:37.249 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:26:37.249 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:26:37.249 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:26:37.249 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:26:37.249 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:26:37.249 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:26:37.250 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:26:37.250 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:26:37.250 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:26:37.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:26:37.251 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:26:37.345 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-07-04T17:26:37.345 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-07-04T17:26:37.352 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:26:37.422 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:26:37.466 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:26:37.467 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:26:37.467 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:26:37.467 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:26:37.467 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:26:37.467 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:26:37.467 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:26:37.467 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:26:37.467 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:26:37.467 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:26:37.467 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:26:37.468 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:26:37.468 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:26:37.468 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:26:37.468 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:26:37.468 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:26:37.468 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:26:37.468 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:26:37.468 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:26:37.468 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:26:37.468 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:26:37.468 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:26:37.468 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:26:37.468 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:26:37.468 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:26:37.468 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:26:37.469 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:26:37.469 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:26:37.469 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:26:37.469 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:26:37.469 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:26:37.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:26:37.470 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:26:37.623 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:2024-07-04T17:26:37.622+0000 7f5ae191b480 -1 init, newargv = 0x559b94204220 newargc=15 2024-07-04T17:26:37.624 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[42663]: starting ceph client 2024-07-04T17:26:37.633 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[42663]: starting fuse 2024-07-04T17:26:38.658 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:26:38.772 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:26:38.834 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:26:38.835 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:26:38.835 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:26:38.835 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:26:38.835 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:26:38.836 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:26:38.836 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:26:38.836 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:26:38.836 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:26:38.836 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:26:38.837 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:26:38.837 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:26:38.837 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:26:38.837 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:26:38.837 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:26:38.838 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:26:38.838 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:26:38.838 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:26:38.838 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:26:38.838 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:26:38.839 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:26:38.839 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:26:38.839 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:26:38.839 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:26:38.839 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:26:38.840 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:26:38.840 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:26:38.840 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:26:38.840 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:26:38.840 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:26:38.840 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:26:38.840 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-07-04T17:26:38.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:26:38.841 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:26:38.898 INFO:teuthology.orchestra.run.smithi042.stdout:42 2024-07-04T17:26:38.898 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-07-04T17:26:38.899 DEBUG:teuthology.orchestra.run.smithi042:> sudo stdin-killer -- python3 -c ' 2024-07-04T17:26:38.899 DEBUG:teuthology.orchestra.run.smithi042:> import glob 2024-07-04T17:26:38.899 DEBUG:teuthology.orchestra.run.smithi042:> import re 2024-07-04T17:26:38.899 DEBUG:teuthology.orchestra.run.smithi042:> import os 2024-07-04T17:26:38.899 DEBUG:teuthology.orchestra.run.smithi042:> import subprocess 2024-07-04T17:26:38.899 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:26:38.899 DEBUG:teuthology.orchestra.run.smithi042:> def _find_admin_socket(client_name): 2024-07-04T17:26:38.899 DEBUG:teuthology.orchestra.run.smithi042:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-07-04T17:26:38.899 DEBUG:teuthology.orchestra.run.smithi042:> files = glob.glob(asok_path) 2024-07-04T17:26:38.899 DEBUG:teuthology.orchestra.run.smithi042:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-07-04T17:26:38.899 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:26:38.899 DEBUG:teuthology.orchestra.run.smithi042:> # Given a non-glob path, it better be there 2024-07-04T17:26:38.899 DEBUG:teuthology.orchestra.run.smithi042:> if "*" not in asok_path: 2024-07-04T17:26:38.899 DEBUG:teuthology.orchestra.run.smithi042:> assert(len(files) == 1) 2024-07-04T17:26:38.900 DEBUG:teuthology.orchestra.run.smithi042:> return files[0] 2024-07-04T17:26:38.900 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:26:38.900 DEBUG:teuthology.orchestra.run.smithi042:> for f in files: 2024-07-04T17:26:38.900 DEBUG:teuthology.orchestra.run.smithi042:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-07-04T17:26:38.900 DEBUG:teuthology.orchestra.run.smithi042:> if os.path.exists("/proc/{0}".format(pid)): 2024-07-04T17:26:38.900 DEBUG:teuthology.orchestra.run.smithi042:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-07-04T17:26:38.900 DEBUG:teuthology.orchestra.run.smithi042:> contents = proc_f.read() 2024-07-04T17:26:38.900 DEBUG:teuthology.orchestra.run.smithi042:> if mountpoint in contents: 2024-07-04T17:26:38.900 DEBUG:teuthology.orchestra.run.smithi042:> return f 2024-07-04T17:26:38.900 DEBUG:teuthology.orchestra.run.smithi042:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-07-04T17:26:38.900 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:26:38.900 DEBUG:teuthology.orchestra.run.smithi042:> print(_find_admin_socket("client.0")) 2024-07-04T17:26:38.900 DEBUG:teuthology.orchestra.run.smithi042:> ' 2024-07-04T17:26:39.078 INFO:teuthology.orchestra.run.smithi042.stdout:/var/run/ceph/ceph-client.0.42663.asok 2024-07-04T17:26:39.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:26:39.096 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.42663.asok 2024-07-04T17:26:39.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:26:39.096 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.42663.asok status 2024-07-04T17:26:39.401 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:26:39.402 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": { 2024-07-04T17:26:39.402 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_sha1": "9dba1e836b4756bf3a32be2a4936f371dff55008", 2024-07-04T17:26:39.402 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.0.0-4667-g9dba1e83 (9dba1e836b4756bf3a32be2a4936f371dff55008) squid (dev)", 2024-07-04T17:26:39.402 INFO:teuthology.orchestra.run.smithi042.stdout: "entity_id": "0", 2024-07-04T17:26:39.402 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": "smithi042", 2024-07-04T17:26:39.402 INFO:teuthology.orchestra.run.smithi042.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-07-04T17:26:39.402 INFO:teuthology.orchestra.run.smithi042.stdout: "pid": "42663", 2024-07-04T17:26:39.402 INFO:teuthology.orchestra.run.smithi042.stdout: "root": "/" 2024-07-04T17:26:39.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:26:39.402 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_count": 0, 2024-07-04T17:26:39.403 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_pinned_count": 0, 2024-07-04T17:26:39.403 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 4336, 2024-07-04T17:26:39.403 INFO:teuthology.orchestra.run.smithi042.stdout: "inst": { 2024-07-04T17:26:39.403 INFO:teuthology.orchestra.run.smithi042.stdout: "name": { 2024-07-04T17:26:39.403 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "client", 2024-07-04T17:26:39.403 INFO:teuthology.orchestra.run.smithi042.stdout: "num": 4336 2024-07-04T17:26:39.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:26:39.403 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-07-04T17:26:39.403 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-07-04T17:26:39.403 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-07-04T17:26:39.403 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3932377136 2024-07-04T17:26:39.403 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:26:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:26:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-07-04T17:26:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-07-04T17:26:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-07-04T17:26:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3932377136 2024-07-04T17:26:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:26:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: "inst_str": "client.4336 192.168.0.1:0/3932377136", 2024-07-04T17:26:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: "addr_str": "192.168.0.1:0/3932377136", 2024-07-04T17:26:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: "inode_count": 1, 2024-07-04T17:26:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_epoch": 8, 2024-07-04T17:26:39.405 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch": 15, 2024-07-04T17:26:39.405 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch_barrier": 0, 2024-07-04T17:26:39.405 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklisted": false, 2024-07-04T17:26:39.405 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs" 2024-07-04T17:26:39.405 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:26:39.422 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-07-04T17:26:39.423 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-04T17:26:39.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.713+0000 7fcf97e00640 1 -- 172.21.15.42:0/233747838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf980bcf90 msgr2=0x7fcf980bd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:39.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.713+0000 7fcf97e00640 1 --2- 172.21.15.42:0/233747838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf980bcf90 0x7fcf980bd3a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcf800097b0 tx=0x7fcf8002a540 comp rx=0 tx=0).stop 2024-07-04T17:26:39.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.713+0000 7fcf97e00640 1 -- 172.21.15.42:0/233747838 shutdown_connections 2024-07-04T17:26:39.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.713+0000 7fcf97e00640 1 --2- 172.21.15.42:0/233747838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf980bcf90 0x7fcf980bd3a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:39.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.713+0000 7fcf97e00640 1 -- 172.21.15.42:0/233747838 >> 172.21.15.42:0/233747838 conn(0x7fcf980688a0 msgr2=0x7fcf9806db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:39.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.714+0000 7fcf97e00640 1 -- 172.21.15.42:0/233747838 shutdown_connections 2024-07-04T17:26:39.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.714+0000 7fcf97e00640 1 -- 172.21.15.42:0/233747838 wait complete. 2024-07-04T17:26:39.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.716+0000 7fcf97e00640 1 Processor -- start 2024-07-04T17:26:39.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.716+0000 7fcf97e00640 1 -- start start 2024-07-04T17:26:39.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.717+0000 7fcf97e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf980bbef0 0x7fcf980c2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:39.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.717+0000 7fcf97e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcf980b79a0 con 0x7fcf980bbef0 2024-07-04T17:26:39.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.718+0000 7fcf96a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf980bbef0 0x7fcf980c2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:39.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.718+0000 7fcf96a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf980bbef0 0x7fcf980c2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39412/0 (socket says 172.21.15.42:39412) 2024-07-04T17:26:39.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.718+0000 7fcf96a00640 1 -- 172.21.15.42:0/2535642716 learned_addr learned my addr 172.21.15.42:0/2535642716 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:39.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.718+0000 7fcf96a00640 1 -- 172.21.15.42:0/2535642716 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf980c1fa0 con 0x7fcf980bbef0 2024-07-04T17:26:39.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.718+0000 7fcf96a00640 1 --2- 172.21.15.42:0/2535642716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf980bbef0 0x7fcf980c2480 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcf8800c320 tx=0x7fcf8800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:39.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.719+0000 7fcf8f400640 1 -- 172.21.15.42:0/2535642716 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf880116c0 con 0x7fcf980bbef0 2024-07-04T17:26:39.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.719+0000 7fcf8f400640 1 -- 172.21.15.42:0/2535642716 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf88017070 con 0x7fcf980bbef0 2024-07-04T17:26:39.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.719+0000 7fcf8f400640 1 -- 172.21.15.42:0/2535642716 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf88011b00 con 0x7fcf980bbef0 2024-07-04T17:26:39.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.719+0000 7fcf97e00640 1 -- 172.21.15.42:0/2535642716 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf98136b40 con 0x7fcf980bbef0 2024-07-04T17:26:39.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.719+0000 7fcf97e00640 1 -- 172.21.15.42:0/2535642716 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf980c1c70 con 0x7fcf980bbef0 2024-07-04T17:26:39.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.720+0000 7fcf8f400640 1 -- 172.21.15.42:0/2535642716 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fcf88011ca0 con 0x7fcf980bbef0 2024-07-04T17:26:39.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.720+0000 7fcf97e00640 1 -- 172.21.15.42:0/2535642716 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf980b8020 con 0x7fcf980bbef0 2024-07-04T17:26:39.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.721+0000 7fcf8f400640 1 --2- 172.21.15.42:0/2535642716 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fcf7403d300 0x7fcf7403f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:39.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.721+0000 7fcf8f400640 1 -- 172.21.15.42:0/2535642716 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7fcf88051270 con 0x7fcf980bbef0 2024-07-04T17:26:39.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.721+0000 7fcf96000640 1 --2- 172.21.15.42:0/2535642716 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fcf7403d300 0x7fcf7403f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:39.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.722+0000 7fcf96000640 1 --2- 172.21.15.42:0/2535642716 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fcf7403d300 0x7fcf7403f810 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcf80034000 tx=0x7fcf80004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:39.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.726+0000 7fcf8f400640 1 -- 172.21.15.42:0/2535642716 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fcf980bc300 con 0x7fcf980bbef0 2024-07-04T17:26:39.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.976+0000 7fcf97e00640 1 -- 172.21.15.42:0/2535642716 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcf98068e30 con 0x7fcf980bbef0 2024-07-04T17:26:39.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.978+0000 7fcf8f400640 1 -- 172.21.15.42:0/2535642716 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fcf8800a140 con 0x7fcf980bbef0 2024-07-04T17:26:39.979 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-04T17:26:39.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.980+0000 7fcf97e00640 1 -- 172.21.15.42:0/2535642716 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fcf7403d300 msgr2=0x7fcf7403f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:39.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.980+0000 7fcf97e00640 1 --2- 172.21.15.42:0/2535642716 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fcf7403d300 0x7fcf7403f810 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcf80034000 tx=0x7fcf80004730 comp rx=0 tx=0).stop 2024-07-04T17:26:39.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.980+0000 7fcf97e00640 1 -- 172.21.15.42:0/2535642716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf980bbef0 msgr2=0x7fcf980c2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:39.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.980+0000 7fcf97e00640 1 --2- 172.21.15.42:0/2535642716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf980bbef0 0x7fcf980c2480 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcf8800c320 tx=0x7fcf8800c7e0 comp rx=0 tx=0).stop 2024-07-04T17:26:39.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.981+0000 7fcf97e00640 1 -- 172.21.15.42:0/2535642716 shutdown_connections 2024-07-04T17:26:39.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.981+0000 7fcf97e00640 1 --2- 172.21.15.42:0/2535642716 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fcf7403d300 0x7fcf7403f810 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:39.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.981+0000 7fcf97e00640 1 --2- 172.21.15.42:0/2535642716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf980bbef0 0x7fcf980c2480 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:39.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.981+0000 7fcf97e00640 1 -- 172.21.15.42:0/2535642716 >> 172.21.15.42:0/2535642716 conn(0x7fcf980688a0 msgr2=0x7fcf98075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:39.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.981+0000 7fcf97e00640 1 -- 172.21.15.42:0/2535642716 shutdown_connections 2024-07-04T17:26:39.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:39.981+0000 7fcf97e00640 1 -- 172.21.15.42:0/2535642716 wait complete. 2024-07-04T17:26:40.011 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-07-04T17:26:40.011 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-04T17:26:40.011 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2024-07-04T17:26:40.011 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-07-04T17:26:40.011 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-07-04T17:26:40.011 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-07-04T17:26:40.011 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-07-04T17:26:40.011 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-04T17:26:40.012 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-07-04T17:26:40.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:26:40.012 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2024-07-04T17:26:40.067 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-04T17:26:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-04T17:26:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-07-04T17:26:40.067 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:26:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-07-04T17:26:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:26:40.068 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:26:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2024-07-04T17:26:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f0 2024-07-04T17:26:40.068 INFO:teuthology.orchestra.run.smithi042.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:26:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2024-07-04T17:26:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f1 2024-07-04T17:26:40.068 INFO:teuthology.orchestra.run.smithi042.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-04T17:26:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:26:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f0 2024-07-04T17:26:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-07-04T17:26:40.068 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 42570sec preferred_lft 42570sec 2024-07-04T17:26:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::ec4:7aff:febd:1440/64 scope link noprefixroute 2024-07-04T17:26:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:26:40.069 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:26:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:26:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f1 2024-07-04T17:26:40.069 INFO:teuthology.orchestra.run.smithi042.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-04T17:26:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether e6:ab:57:fb:ca:9e brd ff:ff:ff:ff:ff:ff 2024-07-04T17:26:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-04T17:26:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:26:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::8839:a9ff:fe74:17f5/64 scope link 2024-07-04T17:26:40.069 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:26:40.069 INFO:teuthology.orchestra.run.smithi042.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-04T17:26:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether e6:ab:57:fb:ca:9e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:26:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::e4ab:57ff:fefb:ca9e/64 scope link 2024-07-04T17:26:40.070 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:26:40.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:26:40.071 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-04T17:26:40.142 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-07-04T17:26:40.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:26:40.146 DEBUG:teuthology.orchestra.run.smithi042:> route 2024-07-04T17:26:40.198 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2024-07-04T17:26:40.198 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-04T17:26:40.199 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-04T17:26:40.199 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-04T17:26:40.199 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-04T17:26:40.200 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-04T17:26:40.200 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-07-04T17:26:40.200 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-04T17:26:40.200 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-04T17:26:40.200 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-04T17:26:40.200 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-07-04T17:26:40.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-04T17:26:40.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:26:40.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:26:40.416 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-07-04T17:26:40.467 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-04T17:26:40.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:26:40.467 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list-id 2024-07-04T17:26:40.523 INFO:teuthology.orchestra.run.smithi042.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-07-04T17:26:40.524 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-04T17:26:40.524 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-07-04T17:26:40.524 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:26:40.524 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 1 2024-07-04T17:26:40.524 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-07-04T17:26:40.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-04T17:26:40.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:26:40.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:26:40.715 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-07-04T17:26:40.792 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-04T17:26:40.792 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-04T17:26:40.792 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-07-04T17:26:40.792 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:26:40.792 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-07-04T17:26:40.792 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:26:40.793 INFO:teuthology.orchestra.run.smithi042.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-04T17:26:40.793 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 82:f0:7f:d9:75:41 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-07-04T17:26:40.793 INFO:teuthology.orchestra.run.smithi042.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-07-04T17:26:40.793 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:26:40.793 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::80f0:7fff:fed9:7541/64 scope link 2024-07-04T17:26:40.793 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:26:40.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:26:40.796 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-07-04T17:26:40.877 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-04T17:26:40.877 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-04T17:26:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-07-04T17:26:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:26:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-07-04T17:26:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:26:40.878 INFO:teuthology.orchestra.run.smithi042.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-04T17:26:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 82:f0:7f:d9:75:41 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-07-04T17:26:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-07-04T17:26:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:26:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::80f0:7fff:fed9:7541/64 scope link 2024-07-04T17:26:40.878 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:26:40.881 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.2/16 2024-07-04T17:26:40.881 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-04T17:26:40.882 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-07-04T17:26:40.882 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.1 2024-07-04T17:26:40.882 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-07-04T17:26:40.882 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-07-04T17:26:40.882 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-07-04T17:26:40.882 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-07-04T17:26:40.882 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-07-04T17:26:40.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-04T17:26:41.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:26:41.168 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-04T17:26:41.168 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-07-04T17:26:41.168 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.1 up 2024-07-04T17:26:41.168 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set dev brx.1 master ceph-brx 2024-07-04T17:26:41.168 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-07-04T17:26:41.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-04T17:26:41.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:26:41.380 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-07-04T17:26:41.380 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-04T17:26:41.381 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-07-04T17:26:41.428 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-07-04T17:26:41.428 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-04T17:26:41.428 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-07-04T17:26:41.478 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:26:41.562 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:26:41.620 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:26:41.620 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:26:41.620 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:26:41.620 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:26:41.620 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:26:41.620 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:26:41.620 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:26:41.621 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:26:41.621 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:26:41.621 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:26:41.621 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:26:41.621 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:26:41.621 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:26:41.621 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:26:41.621 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:26:41.621 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:26:41.621 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:26:41.621 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:26:41.621 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:26:41.621 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:26:41.621 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:26:41.622 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:26:41.622 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:26:41.622 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:26:41.622 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:26:41.622 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:26:41.622 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:26:41.622 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:26:41.622 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:26:41.622 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:26:41.622 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:26:41.622 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-07-04T17:26:41.622 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:26:41.623 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:26:41.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:26:41.623 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:26:41.676 INFO:teuthology.orchestra.run.smithi042.stdout:42 2024-07-04T17:26:41.677 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-07-04T17:26:41.677 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-07-04T17:26:41.682 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:26:41.751 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:26:41.794 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:26:41.794 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:26:41.794 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:26:41.794 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:26:41.794 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:26:41.794 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:26:41.794 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:26:41.795 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:26:41.795 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:26:41.795 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:26:41.795 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:26:41.795 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:26:41.795 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:26:41.795 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:26:41.795 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:26:41.795 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:26:41.795 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:26:41.795 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:26:41.795 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:26:41.795 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:26:41.796 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:26:41.796 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:26:41.796 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:26:41.796 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:26:41.796 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:26:41.796 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:26:41.796 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:26:41.796 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:26:41.796 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:26:41.796 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:26:41.796 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:26:41.796 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-07-04T17:26:41.796 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:26:41.796 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:26:41.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:26:41.797 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:26:41.881 INFO:teuthology.orchestra.run.smithi042.stdout:42 2024-07-04T17:26:41.930 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi042.stderr:2024-07-04T17:26:41.929+0000 7f9516a3c480 -1 init, newargv = 0x7f94f8001080 newargc=15 2024-07-04T17:26:41.930 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi042.stderr:ceph-fuse[43229]: starting ceph client 2024-07-04T17:26:41.938 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi042.stderr:ceph-fuse[43229]: starting fuse 2024-07-04T17:26:42.983 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:26:43.078 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:26:43.135 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:26:43.135 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:26:43.135 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:26:43.135 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:26:43.135 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:26:43.135 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:26:43.136 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:26:43.136 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:26:43.136 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:26:43.136 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:26:43.136 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:26:43.136 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:26:43.136 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:26:43.136 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:26:43.136 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:26:43.136 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:26:43.136 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:26:43.136 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:26:43.137 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:26:43.137 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:26:43.137 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:26:43.137 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:26:43.137 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:26:43.137 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:26:43.137 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:26:43.137 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:26:43.137 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:26:43.137 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:26:43.137 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:26:43.138 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:26:43.138 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:26:43.138 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-07-04T17:26:43.138 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:26:43.138 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:26:43.138 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.1 2024-07-04T17:26:43.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:26:43.139 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:26:43.195 INFO:teuthology.orchestra.run.smithi042.stdout:42 2024-07-04T17:26:43.195 INFO:teuthology.orchestra.run.smithi042.stdout:43 2024-07-04T17:26:43.195 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 43] 2024-07-04T17:26:43.195 DEBUG:teuthology.orchestra.run.smithi042:> sudo stdin-killer -- python3 -c ' 2024-07-04T17:26:43.195 DEBUG:teuthology.orchestra.run.smithi042:> import glob 2024-07-04T17:26:43.196 DEBUG:teuthology.orchestra.run.smithi042:> import re 2024-07-04T17:26:43.196 DEBUG:teuthology.orchestra.run.smithi042:> import os 2024-07-04T17:26:43.196 DEBUG:teuthology.orchestra.run.smithi042:> import subprocess 2024-07-04T17:26:43.196 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:26:43.196 DEBUG:teuthology.orchestra.run.smithi042:> def _find_admin_socket(client_name): 2024-07-04T17:26:43.196 DEBUG:teuthology.orchestra.run.smithi042:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-07-04T17:26:43.196 DEBUG:teuthology.orchestra.run.smithi042:> files = glob.glob(asok_path) 2024-07-04T17:26:43.196 DEBUG:teuthology.orchestra.run.smithi042:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-07-04T17:26:43.196 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:26:43.196 DEBUG:teuthology.orchestra.run.smithi042:> # Given a non-glob path, it better be there 2024-07-04T17:26:43.196 DEBUG:teuthology.orchestra.run.smithi042:> if "*" not in asok_path: 2024-07-04T17:26:43.196 DEBUG:teuthology.orchestra.run.smithi042:> assert(len(files) == 1) 2024-07-04T17:26:43.197 DEBUG:teuthology.orchestra.run.smithi042:> return files[0] 2024-07-04T17:26:43.197 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:26:43.197 DEBUG:teuthology.orchestra.run.smithi042:> for f in files: 2024-07-04T17:26:43.197 DEBUG:teuthology.orchestra.run.smithi042:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-07-04T17:26:43.197 DEBUG:teuthology.orchestra.run.smithi042:> if os.path.exists("/proc/{0}".format(pid)): 2024-07-04T17:26:43.197 DEBUG:teuthology.orchestra.run.smithi042:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-07-04T17:26:43.197 DEBUG:teuthology.orchestra.run.smithi042:> contents = proc_f.read() 2024-07-04T17:26:43.197 DEBUG:teuthology.orchestra.run.smithi042:> if mountpoint in contents: 2024-07-04T17:26:43.197 DEBUG:teuthology.orchestra.run.smithi042:> return f 2024-07-04T17:26:43.197 DEBUG:teuthology.orchestra.run.smithi042:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-07-04T17:26:43.197 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:26:43.197 DEBUG:teuthology.orchestra.run.smithi042:> print(_find_admin_socket("client.1")) 2024-07-04T17:26:43.198 DEBUG:teuthology.orchestra.run.smithi042:> ' 2024-07-04T17:26:43.377 INFO:teuthology.orchestra.run.smithi042.stdout:/var/run/ceph/ceph-client.1.43229.asok 2024-07-04T17:26:43.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:26:43.395 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.43229.asok 2024-07-04T17:26:43.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:26:43.395 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.43229.asok status 2024-07-04T17:26:43.706 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:26:43.707 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": { 2024-07-04T17:26:43.707 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_sha1": "9dba1e836b4756bf3a32be2a4936f371dff55008", 2024-07-04T17:26:43.707 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.0.0-4667-g9dba1e83 (9dba1e836b4756bf3a32be2a4936f371dff55008) squid (dev)", 2024-07-04T17:26:43.707 INFO:teuthology.orchestra.run.smithi042.stdout: "entity_id": "1", 2024-07-04T17:26:43.707 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": "smithi042", 2024-07-04T17:26:43.707 INFO:teuthology.orchestra.run.smithi042.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-07-04T17:26:43.707 INFO:teuthology.orchestra.run.smithi042.stdout: "pid": "43229", 2024-07-04T17:26:43.707 INFO:teuthology.orchestra.run.smithi042.stdout: "root": "/" 2024-07-04T17:26:43.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:26:43.707 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_count": 0, 2024-07-04T17:26:43.708 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_pinned_count": 0, 2024-07-04T17:26:43.708 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 4342, 2024-07-04T17:26:43.708 INFO:teuthology.orchestra.run.smithi042.stdout: "inst": { 2024-07-04T17:26:43.708 INFO:teuthology.orchestra.run.smithi042.stdout: "name": { 2024-07-04T17:26:43.708 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "client", 2024-07-04T17:26:43.708 INFO:teuthology.orchestra.run.smithi042.stdout: "num": 4342 2024-07-04T17:26:43.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:26:43.708 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-07-04T17:26:43.708 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-07-04T17:26:43.708 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.2:0", 2024-07-04T17:26:43.708 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 638197095 2024-07-04T17:26:43.708 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:26:43.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:26:43.708 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-07-04T17:26:43.708 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-07-04T17:26:43.709 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.2:0", 2024-07-04T17:26:43.709 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 638197095 2024-07-04T17:26:43.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:26:43.709 INFO:teuthology.orchestra.run.smithi042.stdout: "inst_str": "client.4342 192.168.0.2:0/638197095", 2024-07-04T17:26:43.709 INFO:teuthology.orchestra.run.smithi042.stdout: "addr_str": "192.168.0.2:0/638197095", 2024-07-04T17:26:43.709 INFO:teuthology.orchestra.run.smithi042.stdout: "inode_count": 1, 2024-07-04T17:26:43.709 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_epoch": 8, 2024-07-04T17:26:43.709 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch": 15, 2024-07-04T17:26:43.709 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch_barrier": 0, 2024-07-04T17:26:43.709 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklisted": false, 2024-07-04T17:26:43.709 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs" 2024-07-04T17:26:43.709 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:26:43.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:26:43.728 DEBUG:teuthology.orchestra.run.smithi042:> stat --file-system '--printf=%T 2024-07-04T17:26:43.728 DEBUG:teuthology.orchestra.run.smithi042:> ' -- /home/ubuntu/cephtest/mnt.0 2024-07-04T17:26:43.784 INFO:teuthology.orchestra.run.smithi042.stdout:fuseblk 2024-07-04T17:26:43.785 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-07-04T17:26:43.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:26:43.785 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-04T17:26:43.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:26:43.878 DEBUG:teuthology.orchestra.run.smithi042:> stat --file-system '--printf=%T 2024-07-04T17:26:43.878 DEBUG:teuthology.orchestra.run.smithi042:> ' -- /home/ubuntu/cephtest/mnt.1 2024-07-04T17:26:43.936 INFO:teuthology.orchestra.run.smithi042.stdout:fuseblk 2024-07-04T17:26:43.936 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-07-04T17:26:43.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:26:43.936 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-07-04T17:26:44.033 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-07-04T17:26:44.198 INFO:tasks.cephfs_test_runner:Starting test: test_add_ancestor_and_child_directory (tasks.cephfs.test_mirroring.TestMirroring) 2024-07-04T17:26:44.199 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:26:44.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.499+0000 7f887a200640 1 -- 172.21.15.42:0/548516986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f887406c2c0 msgr2=0x7f88740b5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:44.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.499+0000 7f887a200640 1 --2- 172.21.15.42:0/548516986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f887406c2c0 0x7f88740b5650 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8860005fd0 tx=0x7f886002ae00 comp rx=0 tx=0).stop 2024-07-04T17:26:44.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.499+0000 7f887a200640 1 -- 172.21.15.42:0/548516986 shutdown_connections 2024-07-04T17:26:44.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.499+0000 7f887a200640 1 --2- 172.21.15.42:0/548516986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f887406c2c0 0x7f88740b5650 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:44.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.499+0000 7f887a200640 1 -- 172.21.15.42:0/548516986 >> 172.21.15.42:0/548516986 conn(0x7f88740bebc0 msgr2=0x7f88740befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:44.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.500+0000 7f887a200640 1 -- 172.21.15.42:0/548516986 shutdown_connections 2024-07-04T17:26:44.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.500+0000 7f887a200640 1 -- 172.21.15.42:0/548516986 wait complete. 2024-07-04T17:26:44.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.502+0000 7f887a200640 1 Processor -- start 2024-07-04T17:26:44.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.502+0000 7f887a200640 1 -- start start 2024-07-04T17:26:44.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.503+0000 7f887a200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f887406c2c0 0x7f8874070440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:44.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.503+0000 7f887a200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f88740b2810 con 0x7f887406c2c0 2024-07-04T17:26:44.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.503+0000 7f8878e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f887406c2c0 0x7f8874070440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:44.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.504+0000 7f8878e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f887406c2c0 0x7f8874070440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36738/0 (socket says 172.21.15.42:36738) 2024-07-04T17:26:44.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.504+0000 7f8878e00640 1 -- 172.21.15.42:0/339199177 learned_addr learned my addr 172.21.15.42:0/339199177 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:44.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.504+0000 7f8878e00640 1 -- 172.21.15.42:0/339199177 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f887413f660 con 0x7f887406c2c0 2024-07-04T17:26:44.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.504+0000 7f8878e00640 1 --2- 172.21.15.42:0/339199177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f887406c2c0 0x7f8874070440 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f886800c380 tx=0x7f886800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:44.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.505+0000 7f8870c00640 1 -- 172.21.15.42:0/339199177 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8868017070 con 0x7f887406c2c0 2024-07-04T17:26:44.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.505+0000 7f8870c00640 1 -- 172.21.15.42:0/339199177 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88680052f0 con 0x7f887406c2c0 2024-07-04T17:26:44.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.505+0000 7f8870c00640 1 -- 172.21.15.42:0/339199177 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88680035f0 con 0x7f887406c2c0 2024-07-04T17:26:44.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.505+0000 7f887a200640 1 -- 172.21.15.42:0/339199177 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8874070b50 con 0x7f887406c2c0 2024-07-04T17:26:44.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.505+0000 7f887a200640 1 -- 172.21.15.42:0/339199177 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f887413edf0 con 0x7f887406c2c0 2024-07-04T17:26:44.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.506+0000 7f887a200640 1 -- 172.21.15.42:0/339199177 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8874070980 con 0x7f887406c2c0 2024-07-04T17:26:44.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.506+0000 7f8870c00640 1 -- 172.21.15.42:0/339199177 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f886801a070 con 0x7f887406c2c0 2024-07-04T17:26:44.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.507+0000 7f8870c00640 1 --2- 172.21.15.42:0/339199177 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f885403d2b0 0x7f885403f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:44.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.507+0000 7f8870c00640 1 -- 172.21.15.42:0/339199177 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f8868050860 con 0x7f887406c2c0 2024-07-04T17:26:44.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.507+0000 7f8873e00640 1 --2- 172.21.15.42:0/339199177 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f885403d2b0 0x7f885403f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:44.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.508+0000 7f8873e00640 1 --2- 172.21.15.42:0/339199177 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f885403d2b0 0x7f885403f7c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8860034000 tx=0x7f8860004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:44.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.512+0000 7f8870c00640 1 -- 172.21.15.42:0/339199177 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f887406c6d0 con 0x7f887406c2c0 2024-07-04T17:26:44.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.724+0000 7f887a200640 1 -- 172.21.15.42:0/339199177 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f887413f080 con 0x7f887406c2c0 2024-07-04T17:26:44.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.726+0000 7f8870c00640 1 -- 172.21.15.42:0/339199177 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) ==== 74+0+10603 (secure 0 0 0) 0x7f887413f080 con 0x7f887406c2c0 2024-07-04T17:26:44.728 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:26:44.728 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":15,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:26:17.008553+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:26:13.335715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-04T17:26:14.519899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:26:44.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.729+0000 7f887a200640 1 -- 172.21.15.42:0/339199177 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f885403d2b0 msgr2=0x7f885403f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:44.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.729+0000 7f887a200640 1 --2- 172.21.15.42:0/339199177 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f885403d2b0 0x7f885403f7c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8860034000 tx=0x7f8860004730 comp rx=0 tx=0).stop 2024-07-04T17:26:44.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.729+0000 7f887a200640 1 -- 172.21.15.42:0/339199177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f887406c2c0 msgr2=0x7f8874070440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:44.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.729+0000 7f887a200640 1 --2- 172.21.15.42:0/339199177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f887406c2c0 0x7f8874070440 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f886800c380 tx=0x7f886800c840 comp rx=0 tx=0).stop 2024-07-04T17:26:44.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.729+0000 7f887a200640 1 -- 172.21.15.42:0/339199177 shutdown_connections 2024-07-04T17:26:44.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.729+0000 7f887a200640 1 --2- 172.21.15.42:0/339199177 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f885403d2b0 0x7f885403f7c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:44.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.729+0000 7f887a200640 1 --2- 172.21.15.42:0/339199177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f887406c2c0 0x7f8874070440 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:44.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.729+0000 7f887a200640 1 -- 172.21.15.42:0/339199177 >> 172.21.15.42:0/339199177 conn(0x7f88740bebc0 msgr2=0x7f887413fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:44.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.729+0000 7f887a200640 1 -- 172.21.15.42:0/339199177 shutdown_connections 2024-07-04T17:26:44.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:44.729+0000 7f887a200640 1 -- 172.21.15.42:0/339199177 wait complete. 2024-07-04T17:26:44.756 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-04T17:26:03.297630+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-07-04T17:26:13.335715+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-07-04T17:26:14.519899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3600000143051147, 'score_stable': 1.3600000143051147, 'optimal_score': 1, 'raw_score_acting': 1.3600000143051147, 'raw_score_stable': 1.3600000143051147, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-04T17:26:44.756 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-04T17:26:45.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.042+0000 7fb56fa00640 1 -- 172.21.15.42:0/2779397889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5680aff80 msgr2=0x7fb56806ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:45.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.042+0000 7fb56fa00640 1 --2- 172.21.15.42:0/2779397889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5680aff80 0x7fb56806ff60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb558009fd0 tx=0x7fb55802b140 comp rx=0 tx=0).stop 2024-07-04T17:26:45.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.042+0000 7fb56fa00640 1 -- 172.21.15.42:0/2779397889 shutdown_connections 2024-07-04T17:26:45.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.043+0000 7fb56fa00640 1 --2- 172.21.15.42:0/2779397889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5680aff80 0x7fb56806ff60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:45.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.043+0000 7fb56fa00640 1 -- 172.21.15.42:0/2779397889 >> 172.21.15.42:0/2779397889 conn(0x7fb5680bf4d0 msgr2=0x7fb5680af870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:45.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.043+0000 7fb56fa00640 1 -- 172.21.15.42:0/2779397889 shutdown_connections 2024-07-04T17:26:45.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.043+0000 7fb56fa00640 1 -- 172.21.15.42:0/2779397889 wait complete. 2024-07-04T17:26:45.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.045+0000 7fb56fa00640 1 Processor -- start 2024-07-04T17:26:45.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.045+0000 7fb56fa00640 1 -- start start 2024-07-04T17:26:45.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.046+0000 7fb56fa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5680aff80 0x7fb568070d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:45.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.046+0000 7fb56fa00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb5680b2e60 con 0x7fb5680aff80 2024-07-04T17:26:45.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.047+0000 7fb56d000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5680aff80 0x7fb568070d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:45.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.047+0000 7fb56d000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5680aff80 0x7fb568070d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36764/0 (socket says 172.21.15.42:36764) 2024-07-04T17:26:45.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.047+0000 7fb56d000640 1 -- 172.21.15.42:0/1820133455 learned_addr learned my addr 172.21.15.42:0/1820133455 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:45.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.047+0000 7fb56d000640 1 -- 172.21.15.42:0/1820133455 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5681d9a70 con 0x7fb5680aff80 2024-07-04T17:26:45.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.047+0000 7fb56d000640 1 --2- 172.21.15.42:0/1820133455 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5680aff80 0x7fb568070d20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb55800a010 tx=0x7fb55802bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:45.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.048+0000 7fb564c00640 1 -- 172.21.15.42:0/1820133455 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb558003aa0 con 0x7fb5680aff80 2024-07-04T17:26:45.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.048+0000 7fb564c00640 1 -- 172.21.15.42:0/1820133455 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb558003d30 con 0x7fb5680aff80 2024-07-04T17:26:45.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.048+0000 7fb564c00640 1 -- 172.21.15.42:0/1820133455 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5580338c0 con 0x7fb5680aff80 2024-07-04T17:26:45.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.048+0000 7fb56fa00640 1 -- 172.21.15.42:0/1820133455 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb568145030 con 0x7fb5680aff80 2024-07-04T17:26:45.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.048+0000 7fb56fa00640 1 -- 172.21.15.42:0/1820133455 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5681d9360 con 0x7fb5680aff80 2024-07-04T17:26:45.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.049+0000 7fb56fa00640 1 -- 172.21.15.42:0/1820133455 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5680b3730 con 0x7fb5680aff80 2024-07-04T17:26:45.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.049+0000 7fb564c00640 1 -- 172.21.15.42:0/1820133455 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fb558042020 con 0x7fb5680aff80 2024-07-04T17:26:45.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.050+0000 7fb564c00640 1 --2- 172.21.15.42:0/1820133455 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb54803d2b0 0x7fb54803f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:45.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.050+0000 7fb564c00640 1 -- 172.21.15.42:0/1820133455 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7fb558040020 con 0x7fb5680aff80 2024-07-04T17:26:45.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.050+0000 7fb567e00640 1 --2- 172.21.15.42:0/1820133455 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb54803d2b0 0x7fb54803f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:45.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.051+0000 7fb567e00640 1 --2- 172.21.15.42:0/1820133455 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb54803d2b0 0x7fb54803f7c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb554001560 tx=0x7fb554007a90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:45.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.055+0000 7fb564c00640 1 -- 172.21.15.42:0/1820133455 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fb5680b3730 con 0x7fb5680aff80 2024-07-04T17:26:45.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.264+0000 7fb56fa00640 1 -- 172.21.15.42:0/1820133455 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb56806ff60 con 0x7fb5680aff80 2024-07-04T17:26:45.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.265+0000 7fb564c00640 1 -- 172.21.15.42:0/1820133455 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v15) ==== 93+0+10 (secure 0 0 0) 0x7fb558033cf0 con 0x7fb5680aff80 2024-07-04T17:26:45.267 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-07-04T17:26:45.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.268+0000 7fb56fa00640 1 -- 172.21.15.42:0/1820133455 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb54803d2b0 msgr2=0x7fb54803f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:45.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.268+0000 7fb56fa00640 1 --2- 172.21.15.42:0/1820133455 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb54803d2b0 0x7fb54803f7c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb554001560 tx=0x7fb554007a90 comp rx=0 tx=0).stop 2024-07-04T17:26:45.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.268+0000 7fb56fa00640 1 -- 172.21.15.42:0/1820133455 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5680aff80 msgr2=0x7fb568070d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:45.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.268+0000 7fb56fa00640 1 --2- 172.21.15.42:0/1820133455 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5680aff80 0x7fb568070d20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb55800a010 tx=0x7fb55802bf20 comp rx=0 tx=0).stop 2024-07-04T17:26:45.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.268+0000 7fb56fa00640 1 -- 172.21.15.42:0/1820133455 shutdown_connections 2024-07-04T17:26:45.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.268+0000 7fb56fa00640 1 --2- 172.21.15.42:0/1820133455 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb54803d2b0 0x7fb54803f7c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:45.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.268+0000 7fb56fa00640 1 --2- 172.21.15.42:0/1820133455 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5680aff80 0x7fb568070d20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:45.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.268+0000 7fb56fa00640 1 -- 172.21.15.42:0/1820133455 >> 172.21.15.42:0/1820133455 conn(0x7fb5680bf4d0 msgr2=0x7fb5680bde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:45.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.268+0000 7fb56fa00640 1 -- 172.21.15.42:0/1820133455 shutdown_connections 2024-07-04T17:26:45.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.269+0000 7fb56fa00640 1 -- 172.21.15.42:0/1820133455 wait complete. 2024-07-04T17:26:45.297 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-07-04T17:26:45.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.588+0000 7fc639a00640 1 -- 172.21.15.42:0/771542831 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6340b0fe0 msgr2=0x7fc6340adef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:45.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.588+0000 7fc639a00640 1 --2- 172.21.15.42:0/771542831 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6340b0fe0 0x7fc6340adef0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc620005fd0 tx=0x7fc62002a5f0 comp rx=0 tx=0).stop 2024-07-04T17:26:45.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.588+0000 7fc639a00640 1 -- 172.21.15.42:0/771542831 shutdown_connections 2024-07-04T17:26:45.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.588+0000 7fc639a00640 1 --2- 172.21.15.42:0/771542831 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6340b0fe0 0x7fc6340adef0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:45.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.588+0000 7fc639a00640 1 -- 172.21.15.42:0/771542831 >> 172.21.15.42:0/771542831 conn(0x7fc6340afaa0 msgr2=0x7fc6340afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:45.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.588+0000 7fc639a00640 1 -- 172.21.15.42:0/771542831 shutdown_connections 2024-07-04T17:26:45.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.588+0000 7fc639a00640 1 -- 172.21.15.42:0/771542831 wait complete. 2024-07-04T17:26:45.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.590+0000 7fc639a00640 1 Processor -- start 2024-07-04T17:26:45.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.590+0000 7fc639a00640 1 -- start start 2024-07-04T17:26:45.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.592+0000 7fc639a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6340b0bd0 0x7fc6341d7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:45.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.592+0000 7fc639a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc6340acca0 con 0x7fc6340b0bd0 2024-07-04T17:26:45.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.592+0000 7fc633e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6340b0bd0 0x7fc6341d7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:45.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.592+0000 7fc633e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6340b0bd0 0x7fc6341d7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36794/0 (socket says 172.21.15.42:36794) 2024-07-04T17:26:45.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.592+0000 7fc633e00640 1 -- 172.21.15.42:0/4018781876 learned_addr learned my addr 172.21.15.42:0/4018781876 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:45.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.593+0000 7fc633e00640 1 -- 172.21.15.42:0/4018781876 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6341e43c0 con 0x7fc6340b0bd0 2024-07-04T17:26:45.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.593+0000 7fc633e00640 1 --2- 172.21.15.42:0/4018781876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6340b0bd0 0x7fc6341d7f50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc624001eb0 tx=0x7fc62400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:45.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.593+0000 7fc62be00640 1 -- 172.21.15.42:0/4018781876 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc624017070 con 0x7fc6340b0bd0 2024-07-04T17:26:45.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.593+0000 7fc62be00640 1 -- 172.21.15.42:0/4018781876 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6240052f0 con 0x7fc6340b0bd0 2024-07-04T17:26:45.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.593+0000 7fc62be00640 1 -- 172.21.15.42:0/4018781876 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6240035f0 con 0x7fc6340b0bd0 2024-07-04T17:26:45.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.593+0000 7fc639a00640 1 -- 172.21.15.42:0/4018781876 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6341d3990 con 0x7fc6340b0bd0 2024-07-04T17:26:45.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.593+0000 7fc639a00640 1 -- 172.21.15.42:0/4018781876 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6341453c0 con 0x7fc6340b0bd0 2024-07-04T17:26:45.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.595+0000 7fc639a00640 1 -- 172.21.15.42:0/4018781876 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6341d3bc0 con 0x7fc6340b0bd0 2024-07-04T17:26:45.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.595+0000 7fc62be00640 1 -- 172.21.15.42:0/4018781876 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fc62401a070 con 0x7fc6340b0bd0 2024-07-04T17:26:45.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.595+0000 7fc62be00640 1 --2- 172.21.15.42:0/4018781876 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fc61403d2b0 0x7fc61403f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:45.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.595+0000 7fc62be00640 1 -- 172.21.15.42:0/4018781876 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7fc6240508d0 con 0x7fc6340b0bd0 2024-07-04T17:26:45.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.596+0000 7fc633400640 1 --2- 172.21.15.42:0/4018781876 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fc61403d2b0 0x7fc61403f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:45.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.596+0000 7fc633400640 1 --2- 172.21.15.42:0/4018781876 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fc61403d2b0 0x7fc61403f7c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc620033000 tx=0x7fc620004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:45.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.601+0000 7fc62be00640 1 -- 172.21.15.42:0/4018781876 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fc6240102e0 con 0x7fc6340b0bd0 2024-07-04T17:26:45.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.814+0000 7fc639a00640 1 -- 172.21.15.42:0/4018781876 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc6340ad340 con 0x7fc6340b0bd0 2024-07-04T17:26:45.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.815+0000 7fc62be00640 1 -- 172.21.15.42:0/4018781876 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v15) ==== 104+0+11 (secure 0 0 0) 0x7fc624050bc0 con 0x7fc6340b0bd0 2024-07-04T17:26:45.817 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-07-04T17:26:45.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.818+0000 7fc639a00640 1 -- 172.21.15.42:0/4018781876 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fc61403d2b0 msgr2=0x7fc61403f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:45.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.818+0000 7fc639a00640 1 --2- 172.21.15.42:0/4018781876 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fc61403d2b0 0x7fc61403f7c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc620033000 tx=0x7fc620004730 comp rx=0 tx=0).stop 2024-07-04T17:26:45.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.818+0000 7fc639a00640 1 -- 172.21.15.42:0/4018781876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6340b0bd0 msgr2=0x7fc6341d7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:45.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.818+0000 7fc639a00640 1 --2- 172.21.15.42:0/4018781876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6340b0bd0 0x7fc6341d7f50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc624001eb0 tx=0x7fc62400c810 comp rx=0 tx=0).stop 2024-07-04T17:26:45.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.818+0000 7fc639a00640 1 -- 172.21.15.42:0/4018781876 shutdown_connections 2024-07-04T17:26:45.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.818+0000 7fc639a00640 1 --2- 172.21.15.42:0/4018781876 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fc61403d2b0 0x7fc61403f7c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:45.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.818+0000 7fc639a00640 1 --2- 172.21.15.42:0/4018781876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6340b0bd0 0x7fc6341d7f50 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:45.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.818+0000 7fc639a00640 1 -- 172.21.15.42:0/4018781876 >> 172.21.15.42:0/4018781876 conn(0x7fc6340afaa0 msgr2=0x7fc634075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:45.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.818+0000 7fc639a00640 1 -- 172.21.15.42:0/4018781876 shutdown_connections 2024-07-04T17:26:45.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:45.818+0000 7fc639a00640 1 -- 172.21.15.42:0/4018781876 wait complete. 2024-07-04T17:26:45.848 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-07-04T17:26:46.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.136+0000 7fd4bd800640 1 -- 172.21.15.42:0/1993381629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4b8073470 msgr2=0x7fd4b80a9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:46.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.136+0000 7fd4bd800640 1 --2- 172.21.15.42:0/1993381629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4b8073470 0x7fd4b80a9990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd4a00097b0 tx=0x7fd4a002a540 comp rx=0 tx=0).stop 2024-07-04T17:26:46.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.136+0000 7fd4bd800640 1 -- 172.21.15.42:0/1993381629 shutdown_connections 2024-07-04T17:26:46.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.136+0000 7fd4bd800640 1 --2- 172.21.15.42:0/1993381629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4b8073470 0x7fd4b80a9990 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:46.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.136+0000 7fd4bd800640 1 -- 172.21.15.42:0/1993381629 >> 172.21.15.42:0/1993381629 conn(0x7fd4b806c020 msgr2=0x7fd4b806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:46.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.136+0000 7fd4bd800640 1 -- 172.21.15.42:0/1993381629 shutdown_connections 2024-07-04T17:26:46.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.136+0000 7fd4bd800640 1 -- 172.21.15.42:0/1993381629 wait complete. 2024-07-04T17:26:46.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.138+0000 7fd4bd800640 1 Processor -- start 2024-07-04T17:26:46.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.139+0000 7fd4bd800640 1 -- start start 2024-07-04T17:26:46.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.140+0000 7fd4bd800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4b8073470 0x7fd4b81d6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:46.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.140+0000 7fd4bd800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b813f430 con 0x7fd4b8073470 2024-07-04T17:26:46.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.140+0000 7fd4b6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4b8073470 0x7fd4b81d6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:46.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.140+0000 7fd4b6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4b8073470 0x7fd4b81d6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36822/0 (socket says 172.21.15.42:36822) 2024-07-04T17:26:46.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.140+0000 7fd4b6a00640 1 -- 172.21.15.42:0/3231046472 learned_addr learned my addr 172.21.15.42:0/3231046472 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:46.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.141+0000 7fd4b6a00640 1 -- 172.21.15.42:0/3231046472 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b80ac320 con 0x7fd4b8073470 2024-07-04T17:26:46.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.141+0000 7fd4b6a00640 1 --2- 172.21.15.42:0/3231046472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4b8073470 0x7fd4b81d6630 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800cb40 tx=0x7fd4a800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:46.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.141+0000 7fd4af400640 1 -- 172.21.15.42:0/3231046472 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4a8016020 con 0x7fd4b8073470 2024-07-04T17:26:46.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.141+0000 7fd4af400640 1 -- 172.21.15.42:0/3231046472 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4a800ac30 con 0x7fd4b8073470 2024-07-04T17:26:46.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.141+0000 7fd4af400640 1 -- 172.21.15.42:0/3231046472 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4a8012420 con 0x7fd4b8073470 2024-07-04T17:26:46.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.141+0000 7fd4bd800640 1 -- 172.21.15.42:0/3231046472 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4b81d5a30 con 0x7fd4b8073470 2024-07-04T17:26:46.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.142+0000 7fd4bd800640 1 -- 172.21.15.42:0/3231046472 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4b81d4b50 con 0x7fd4b8073470 2024-07-04T17:26:46.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.143+0000 7fd4bd800640 1 -- 172.21.15.42:0/3231046472 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4b813fb00 con 0x7fd4b8073470 2024-07-04T17:26:46.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.143+0000 7fd4af400640 1 -- 172.21.15.42:0/3231046472 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fd4a8019020 con 0x7fd4b8073470 2024-07-04T17:26:46.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.144+0000 7fd4af400640 1 --2- 172.21.15.42:0/3231046472 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fd49403d300 0x7fd49403f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:46.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.144+0000 7fd4af400640 1 -- 172.21.15.42:0/3231046472 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7fd4a80518a0 con 0x7fd4b8073470 2024-07-04T17:26:46.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.144+0000 7fd4ae000640 1 --2- 172.21.15.42:0/3231046472 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fd49403d300 0x7fd49403f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:46.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.145+0000 7fd4ae000640 1 --2- 172.21.15.42:0/3231046472 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fd49403d300 0x7fd49403f810 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0034000 tx=0x7fd4a0004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:46.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.149+0000 7fd4af400640 1 -- 172.21.15.42:0/3231046472 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fd4a8020030 con 0x7fd4b8073470 2024-07-04T17:26:46.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.356+0000 7fd4bd800640 1 -- 172.21.15.42:0/3231046472 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd4b80ac040 con 0x7fd4b8073470 2024-07-04T17:26:46.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.358+0000 7fd4af400640 1 -- 172.21.15.42:0/3231046472 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v15) ==== 100+0+11 (secure 0 0 0) 0x7fd4a800d150 con 0x7fd4b8073470 2024-07-04T17:26:46.360 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-07-04T17:26:46.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.361+0000 7fd4bd800640 1 -- 172.21.15.42:0/3231046472 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fd49403d300 msgr2=0x7fd49403f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:46.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.361+0000 7fd4bd800640 1 --2- 172.21.15.42:0/3231046472 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fd49403d300 0x7fd49403f810 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0034000 tx=0x7fd4a0004730 comp rx=0 tx=0).stop 2024-07-04T17:26:46.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.361+0000 7fd4bd800640 1 -- 172.21.15.42:0/3231046472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4b8073470 msgr2=0x7fd4b81d6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:46.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.361+0000 7fd4bd800640 1 --2- 172.21.15.42:0/3231046472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4b8073470 0x7fd4b81d6630 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800cb40 tx=0x7fd4a800a060 comp rx=0 tx=0).stop 2024-07-04T17:26:46.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.362+0000 7fd4bd800640 1 -- 172.21.15.42:0/3231046472 shutdown_connections 2024-07-04T17:26:46.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.362+0000 7fd4bd800640 1 --2- 172.21.15.42:0/3231046472 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fd49403d300 0x7fd49403f810 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:46.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.362+0000 7fd4bd800640 1 --2- 172.21.15.42:0/3231046472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4b8073470 0x7fd4b81d6630 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:46.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.362+0000 7fd4bd800640 1 -- 172.21.15.42:0/3231046472 >> 172.21.15.42:0/3231046472 conn(0x7fd4b806c020 msgr2=0x7fd4b8070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:46.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.362+0000 7fd4bd800640 1 -- 172.21.15.42:0/3231046472 shutdown_connections 2024-07-04T17:26:46.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.362+0000 7fd4bd800640 1 -- 172.21.15.42:0/3231046472 wait complete. 2024-07-04T17:26:46.391 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_mirroring.TestMirroring.test_add_ancestor_and_child_directory' 2024-07-04T17:26:46.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.679+0000 7f03fc800640 1 -- 172.21.15.42:0/32485062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03f40b2d20 msgr2=0x7f03f40b37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:46.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.679+0000 7f03fc800640 1 --2- 172.21.15.42:0/32485062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03f40b2d20 0x7f03f40b37d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f03e8005fd0 tx=0x7f03e802a5f0 comp rx=0 tx=0).stop 2024-07-04T17:26:46.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.679+0000 7f03fc800640 1 -- 172.21.15.42:0/32485062 shutdown_connections 2024-07-04T17:26:46.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.679+0000 7f03fc800640 1 --2- 172.21.15.42:0/32485062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03f40b2d20 0x7f03f40b37d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:46.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.679+0000 7f03fc800640 1 -- 172.21.15.42:0/32485062 >> 172.21.15.42:0/32485062 conn(0x7f03f40bf700 msgr2=0x7f03f40af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:46.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.679+0000 7f03fc800640 1 -- 172.21.15.42:0/32485062 shutdown_connections 2024-07-04T17:26:46.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.679+0000 7f03fc800640 1 -- 172.21.15.42:0/32485062 wait complete. 2024-07-04T17:26:46.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.681+0000 7f03fc800640 1 Processor -- start 2024-07-04T17:26:46.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.681+0000 7f03fc800640 1 -- start start 2024-07-04T17:26:46.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.682+0000 7f03fc800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03f40b01f0 0x7f03f41438d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:46.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.682+0000 7f03fc800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f03f406cc20 con 0x7f03f40b01f0 2024-07-04T17:26:46.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.683+0000 7f03f9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03f40b01f0 0x7f03f41438d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:46.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.683+0000 7f03f9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03f40b01f0 0x7f03f41438d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36836/0 (socket says 172.21.15.42:36836) 2024-07-04T17:26:46.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.683+0000 7f03f9e00640 1 -- 172.21.15.42:0/3312063212 learned_addr learned my addr 172.21.15.42:0/3312063212 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:46.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.683+0000 7f03f9e00640 1 -- 172.21.15.42:0/3312063212 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f4141b10 con 0x7f03f40b01f0 2024-07-04T17:26:46.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.683+0000 7f03f9e00640 1 --2- 172.21.15.42:0/3312063212 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03f40b01f0 0x7f03f41438d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009850 tx=0x7f03e80049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:46.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.684+0000 7f03ee000640 1 -- 172.21.15.42:0/3312063212 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03e803e030 con 0x7f03f40b01f0 2024-07-04T17:26:46.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.684+0000 7f03ee000640 1 -- 172.21.15.42:0/3312063212 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03e80035f0 con 0x7f03f40b01f0 2024-07-04T17:26:46.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.684+0000 7f03ee000640 1 -- 172.21.15.42:0/3312063212 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03e8035040 con 0x7f03f40b01f0 2024-07-04T17:26:46.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.684+0000 7f03fc800640 1 -- 172.21.15.42:0/3312063212 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f41d62f0 con 0x7f03f40b01f0 2024-07-04T17:26:46.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.684+0000 7f03fc800640 1 -- 172.21.15.42:0/3312063212 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f4141fd0 con 0x7f03f40b01f0 2024-07-04T17:26:46.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.685+0000 7f03ee000640 1 -- 172.21.15.42:0/3312063212 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f03e8041030 con 0x7f03f40b01f0 2024-07-04T17:26:46.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.685+0000 7f03fc800640 1 -- 172.21.15.42:0/3312063212 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03f406d2c0 con 0x7f03f40b01f0 2024-07-04T17:26:46.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.686+0000 7f03ee000640 1 --2- 172.21.15.42:0/3312063212 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f03d803d2b0 0x7f03d803f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:46.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.686+0000 7f03ee000640 1 -- 172.21.15.42:0/3312063212 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f03e803f070 con 0x7f03f40b01f0 2024-07-04T17:26:46.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.686+0000 7f03f9400640 1 --2- 172.21.15.42:0/3312063212 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f03d803d2b0 0x7f03d803f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:46.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.687+0000 7f03f9400640 1 --2- 172.21.15.42:0/3312063212 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f03d803d2b0 0x7f03d803f7c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f03e4001560 tx=0x7f03e40079f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:46.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.691+0000 7f03ee000640 1 -- 172.21.15.42:0/3312063212 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f03e804bd60 con 0x7f03f40b01f0 2024-07-04T17:26:46.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:46.973+0000 7f03fc800640 1 -- 172.21.15.42:0/3312063212 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_mirroring.TestMirroring.test_add_ancestor_and_child_directory"]} v 0) -- 0x7f03f40af6e0 con 0x7f03f40b01f0 2024-07-04T17:26:47.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.360+0000 7f03ee000640 1 -- 172.21.15.42:0/3312063212 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_mirroring.TestMirroring.test_add_ancestor_and_child_directory"]}]=0 v50) ==== 161+0+0 (secure 0 0 0) 0x7f03e8031400 con 0x7f03f40b01f0 2024-07-04T17:26:47.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.365+0000 7f03fc800640 1 -- 172.21.15.42:0/3312063212 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f03d803d2b0 msgr2=0x7f03d803f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:47.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.365+0000 7f03fc800640 1 --2- 172.21.15.42:0/3312063212 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f03d803d2b0 0x7f03d803f7c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f03e4001560 tx=0x7f03e40079f0 comp rx=0 tx=0).stop 2024-07-04T17:26:47.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.365+0000 7f03fc800640 1 -- 172.21.15.42:0/3312063212 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03f40b01f0 msgr2=0x7f03f41438d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:47.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.365+0000 7f03fc800640 1 --2- 172.21.15.42:0/3312063212 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03f40b01f0 0x7f03f41438d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009850 tx=0x7f03e80049d0 comp rx=0 tx=0).stop 2024-07-04T17:26:47.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.365+0000 7f03fc800640 1 -- 172.21.15.42:0/3312063212 shutdown_connections 2024-07-04T17:26:47.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.365+0000 7f03fc800640 1 --2- 172.21.15.42:0/3312063212 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f03d803d2b0 0x7f03d803f7c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:47.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.365+0000 7f03fc800640 1 --2- 172.21.15.42:0/3312063212 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03f40b01f0 0x7f03f41438d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:47.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.365+0000 7f03fc800640 1 -- 172.21.15.42:0/3312063212 >> 172.21.15.42:0/3312063212 conn(0x7f03f40bf700 msgr2=0x7f03f40bc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:47.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.365+0000 7f03fc800640 1 -- 172.21.15.42:0/3312063212 shutdown_connections 2024-07-04T17:26:47.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.366+0000 7f03fc800640 1 -- 172.21.15.42:0/3312063212 wait complete. 2024-07-04T17:26:47.404 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-07-04T17:26:47.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.698+0000 7fc1b3000640 1 -- 172.21.15.42:0/2193633435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1ac0b2d20 msgr2=0x7fc1ac0bb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:47.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.698+0000 7fc1b3000640 1 --2- 172.21.15.42:0/2193633435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1ac0b2d20 0x7fc1ac0bb1f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc1940097b0 tx=0x7fc19402a4a0 comp rx=0 tx=0).stop 2024-07-04T17:26:47.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.698+0000 7fc1b3000640 1 -- 172.21.15.42:0/2193633435 shutdown_connections 2024-07-04T17:26:47.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.698+0000 7fc1b3000640 1 --2- 172.21.15.42:0/2193633435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1ac0b2d20 0x7fc1ac0bb1f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:47.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.698+0000 7fc1b3000640 1 -- 172.21.15.42:0/2193633435 >> 172.21.15.42:0/2193633435 conn(0x7fc1ac0bf700 msgr2=0x7fc1ac0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:47.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.699+0000 7fc1b3000640 1 -- 172.21.15.42:0/2193633435 shutdown_connections 2024-07-04T17:26:47.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.699+0000 7fc1b3000640 1 -- 172.21.15.42:0/2193633435 wait complete. 2024-07-04T17:26:47.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.701+0000 7fc1b3000640 1 Processor -- start 2024-07-04T17:26:47.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.701+0000 7fc1b3000640 1 -- start start 2024-07-04T17:26:47.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.702+0000 7fc1b3000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1ac0b2d20 0x7fc1ac143b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:47.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.702+0000 7fc1b3000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac06c020 con 0x7fc1ac0b2d20 2024-07-04T17:26:47.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.702+0000 7fc1abe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1ac0b2d20 0x7fc1ac143b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:47.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.702+0000 7fc1abe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1ac0b2d20 0x7fc1ac143b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36858/0 (socket says 172.21.15.42:36858) 2024-07-04T17:26:47.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.702+0000 7fc1abe00640 1 -- 172.21.15.42:0/304374074 learned_addr learned my addr 172.21.15.42:0/304374074 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:47.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.703+0000 7fc1abe00640 1 -- 172.21.15.42:0/304374074 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1ac143890 con 0x7fc1ac0b2d20 2024-07-04T17:26:47.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.703+0000 7fc1abe00640 1 --2- 172.21.15.42:0/304374074 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1ac0b2d20 0x7fc1ac143b10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00c3b0 tx=0x7fc19c00c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:47.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.703+0000 7fc1a8c00640 1 -- 172.21.15.42:0/304374074 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc19c011780 con 0x7fc1ac0b2d20 2024-07-04T17:26:47.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.703+0000 7fc1a8c00640 1 -- 172.21.15.42:0/304374074 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc19c017070 con 0x7fc1ac0b2d20 2024-07-04T17:26:47.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.703+0000 7fc1a8c00640 1 -- 172.21.15.42:0/304374074 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc19c011c00 con 0x7fc1ac0b2d20 2024-07-04T17:26:47.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.704+0000 7fc1b3000640 1 -- 172.21.15.42:0/304374074 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1ac1d8980 con 0x7fc1ac0b2d20 2024-07-04T17:26:47.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.704+0000 7fc1b3000640 1 -- 172.21.15.42:0/304374074 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1ac13e540 con 0x7fc1ac0b2d20 2024-07-04T17:26:47.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.705+0000 7fc1b3000640 1 -- 172.21.15.42:0/304374074 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1ac06c710 con 0x7fc1ac0b2d20 2024-07-04T17:26:47.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.705+0000 7fc1a8c00640 1 -- 172.21.15.42:0/304374074 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fc19c01c460 con 0x7fc1ac0b2d20 2024-07-04T17:26:47.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.706+0000 7fc1a8c00640 1 --2- 172.21.15.42:0/304374074 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fc18803d2b0 0x7fc18803f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:47.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.706+0000 7fc1a8c00640 1 -- 172.21.15.42:0/304374074 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7fc19c051270 con 0x7fc1ac0b2d20 2024-07-04T17:26:47.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.706+0000 7fc1a3e00640 1 --2- 172.21.15.42:0/304374074 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fc18803d2b0 0x7fc18803f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:47.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.707+0000 7fc1a3e00640 1 --2- 172.21.15.42:0/304374074 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fc18803d2b0 0x7fc18803f7c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc194034000 tx=0x7fc194004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:47.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.711+0000 7fc1a8c00640 1 -- 172.21.15.42:0/304374074 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fc19c010c60 con 0x7fc1ac0b2d20 2024-07-04T17:26:47.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.919+0000 7fc1b3000640 1 -- 172.21.15.42:0/304374074 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fc1ac1445a0 con 0x7fc1ac0b2d20 2024-07-04T17:26:47.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.920+0000 7fc1a8c00640 1 -- 172.21.15.42:0/304374074 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v22) ==== 59+0+1628 (secure 0 0 0) 0x7fc19c01c800 con 0x7fc1ac0b2d20 2024-07-04T17:26:47.922 INFO:teuthology.orchestra.run.smithi042.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-07-04T17:26:47.922 INFO:teuthology.orchestra.run.smithi042.stdout:mon advanced mon_op_complaint_time 120 2024-07-04T17:26:47.922 INFO:teuthology.orchestra.run.smithi042.stdout:mon dev mon_warn_on_pool_no_app false 2024-07-04T17:26:47.922 INFO:teuthology.orchestra.run.smithi042.stdout:osd advanced osd_op_complaint_time 180.000000 2024-07-04T17:26:47.922 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3541.919687 2024-07-04T17:26:47.922 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3558.529833 2024-07-04T17:26:47.922 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3688.441103 2024-07-04T17:26:47.922 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced debug_mds 20/20 2024-07-04T17:26:47.923 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced debug_mds_balancer 20/20 2024-07-04T17:26:47.923 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced debug_ms 1/1 2024-07-04T17:26:47.923 INFO:teuthology.orchestra.run.smithi042.stdout:mds dev mds_debug_frag true 2024-07-04T17:26:47.923 INFO:teuthology.orchestra.run.smithi042.stdout:mds dev mds_debug_scatterstat true 2024-07-04T17:26:47.923 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced mds_op_complaint_time 180.000000 2024-07-04T17:26:47.923 INFO:teuthology.orchestra.run.smithi042.stdout:mds dev mds_verify_scatter true 2024-07-04T17:26:47.923 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced osd_op_complaint_time 180.000000 2024-07-04T17:26:47.923 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced rados_mon_op_timeout 900 2024-07-04T17:26:47.923 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced rados_osd_op_timeout 900 2024-07-04T17:26:47.923 INFO:teuthology.orchestra.run.smithi042.stdout:client advanced client_mount_timeout 600 2024-07-04T17:26:47.923 INFO:teuthology.orchestra.run.smithi042.stdout:client advanced debug_client 20/20 2024-07-04T17:26:47.923 INFO:teuthology.orchestra.run.smithi042.stdout:client advanced debug_ms 1/1 2024-07-04T17:26:47.923 INFO:teuthology.orchestra.run.smithi042.stdout:client advanced rados_mon_op_timeout 900 2024-07-04T17:26:47.924 INFO:teuthology.orchestra.run.smithi042.stdout:client advanced rados_osd_op_timeout 900 2024-07-04T17:26:47.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.923+0000 7fc1b3000640 1 -- 172.21.15.42:0/304374074 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fc18803d2b0 msgr2=0x7fc18803f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:47.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.923+0000 7fc1b3000640 1 --2- 172.21.15.42:0/304374074 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fc18803d2b0 0x7fc18803f7c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc194034000 tx=0x7fc194004730 comp rx=0 tx=0).stop 2024-07-04T17:26:47.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.923+0000 7fc1b3000640 1 -- 172.21.15.42:0/304374074 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1ac0b2d20 msgr2=0x7fc1ac143b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:47.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.923+0000 7fc1b3000640 1 --2- 172.21.15.42:0/304374074 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1ac0b2d20 0x7fc1ac143b10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00c3b0 tx=0x7fc19c00c870 comp rx=0 tx=0).stop 2024-07-04T17:26:47.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.923+0000 7fc1b3000640 1 -- 172.21.15.42:0/304374074 shutdown_connections 2024-07-04T17:26:47.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.923+0000 7fc1b3000640 1 --2- 172.21.15.42:0/304374074 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fc18803d2b0 0x7fc18803f7c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:47.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.923+0000 7fc1b3000640 1 --2- 172.21.15.42:0/304374074 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc1ac0b2d20 0x7fc1ac143b10 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:47.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.923+0000 7fc1b3000640 1 -- 172.21.15.42:0/304374074 >> 172.21.15.42:0/304374074 conn(0x7fc1ac0bf700 msgr2=0x7fc1ac075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:47.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.924+0000 7fc1b3000640 1 -- 172.21.15.42:0/304374074 shutdown_connections 2024-07-04T17:26:47.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:47.924+0000 7fc1b3000640 1 -- 172.21.15.42:0/304374074 wait complete. 2024-07-04T17:26:47.952 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-07-04T17:26:48.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.249+0000 7f5e0a800640 1 -- 172.21.15.42:0/2504440165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e040701d0 msgr2=0x7f5e040abe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:48.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.249+0000 7f5e0a800640 1 --2- 172.21.15.42:0/2504440165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e040701d0 0x7f5e040abe40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5df8005fd0 tx=0x7f5df802a8b0 comp rx=0 tx=0).stop 2024-07-04T17:26:48.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.249+0000 7f5e0a800640 1 -- 172.21.15.42:0/2504440165 shutdown_connections 2024-07-04T17:26:48.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.249+0000 7f5e0a800640 1 --2- 172.21.15.42:0/2504440165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e040701d0 0x7f5e040abe40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:48.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.249+0000 7f5e0a800640 1 -- 172.21.15.42:0/2504440165 >> 172.21.15.42:0/2504440165 conn(0x7f5e040afaa0 msgr2=0x7f5e040afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:48.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.250+0000 7f5e0a800640 1 -- 172.21.15.42:0/2504440165 shutdown_connections 2024-07-04T17:26:48.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.250+0000 7f5e0a800640 1 -- 172.21.15.42:0/2504440165 wait complete. 2024-07-04T17:26:48.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.252+0000 7f5e0a800640 1 Processor -- start 2024-07-04T17:26:48.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.252+0000 7f5e0a800640 1 -- start start 2024-07-04T17:26:48.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.253+0000 7f5e0a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e040701d0 0x7f5e040ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:48.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.253+0000 7f5e0a800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5e040ab190 con 0x7f5e040701d0 2024-07-04T17:26:48.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.254+0000 7f5e09400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e040701d0 0x7f5e040ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:48.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.254+0000 7f5e09400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e040701d0 0x7f5e040ad7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36864/0 (socket says 172.21.15.42:36864) 2024-07-04T17:26:48.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.254+0000 7f5e09400640 1 -- 172.21.15.42:0/508854755 learned_addr learned my addr 172.21.15.42:0/508854755 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:48.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.254+0000 7f5e09400640 1 -- 172.21.15.42:0/508854755 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0414b6f0 con 0x7f5e040701d0 2024-07-04T17:26:48.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.255+0000 7f5e09400640 1 --2- 172.21.15.42:0/508854755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e040701d0 0x7f5e040ad7c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009c50 tx=0x7f5df8031670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:48.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.255+0000 7f5e01600640 1 -- 172.21.15.42:0/508854755 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5df80318c0 con 0x7f5e040701d0 2024-07-04T17:26:48.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.255+0000 7f5e01600640 1 -- 172.21.15.42:0/508854755 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5df8031b50 con 0x7f5e040701d0 2024-07-04T17:26:48.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.255+0000 7f5e0a800640 1 -- 172.21.15.42:0/508854755 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e0414ac70 con 0x7f5e040701d0 2024-07-04T17:26:48.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.255+0000 7f5e01600640 1 -- 172.21.15.42:0/508854755 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5df8033070 con 0x7f5e040701d0 2024-07-04T17:26:48.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.255+0000 7f5e0a800640 1 -- 172.21.15.42:0/508854755 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e0414bc10 con 0x7f5e040701d0 2024-07-04T17:26:48.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.256+0000 7f5e0a800640 1 -- 172.21.15.42:0/508854755 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e040abe40 con 0x7f5e040701d0 2024-07-04T17:26:48.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.257+0000 7f5e01600640 1 -- 172.21.15.42:0/508854755 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f5df8033210 con 0x7f5e040701d0 2024-07-04T17:26:48.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.257+0000 7f5e01600640 1 --2- 172.21.15.42:0/508854755 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f5de403d2b0 0x7f5de403f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:48.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.257+0000 7f5e01600640 1 -- 172.21.15.42:0/508854755 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f5df80729b0 con 0x7f5e040701d0 2024-07-04T17:26:48.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.258+0000 7f5e08a00640 1 --2- 172.21.15.42:0/508854755 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f5de403d2b0 0x7f5de403f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:48.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.259+0000 7f5e08a00640 1 --2- 172.21.15.42:0/508854755 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f5de403d2b0 0x7f5de403f7c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5df0007fd0 tx=0x7f5df0007920 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:48.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.263+0000 7f5e01600640 1 -- 172.21.15.42:0/508854755 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f5df803f030 con 0x7f5e040701d0 2024-07-04T17:26:48.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.475+0000 7f5e0a800640 1 -- 172.21.15.42:0/508854755 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f5e040aa6e0 con 0x7f5e040701d0 2024-07-04T17:26:48.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.481+0000 7f5e01600640 1 -- 172.21.15.42:0/508854755 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v23)=0 v23) ==== 122+0+0 (secure 0 0 0) 0x7f5df8004040 con 0x7f5e040701d0 2024-07-04T17:26:48.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.484+0000 7f5e0a800640 1 -- 172.21.15.42:0/508854755 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f5de403d2b0 msgr2=0x7f5de403f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:48.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.484+0000 7f5e0a800640 1 --2- 172.21.15.42:0/508854755 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f5de403d2b0 0x7f5de403f7c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5df0007fd0 tx=0x7f5df0007920 comp rx=0 tx=0).stop 2024-07-04T17:26:48.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.484+0000 7f5e0a800640 1 -- 172.21.15.42:0/508854755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e040701d0 msgr2=0x7f5e040ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:48.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.484+0000 7f5e0a800640 1 --2- 172.21.15.42:0/508854755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e040701d0 0x7f5e040ad7c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009c50 tx=0x7f5df8031670 comp rx=0 tx=0).stop 2024-07-04T17:26:48.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.484+0000 7f5e0a800640 1 -- 172.21.15.42:0/508854755 shutdown_connections 2024-07-04T17:26:48.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.484+0000 7f5e0a800640 1 --2- 172.21.15.42:0/508854755 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f5de403d2b0 0x7f5de403f7c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:48.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.484+0000 7f5e0a800640 1 --2- 172.21.15.42:0/508854755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e040701d0 0x7f5e040ad7c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:48.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.484+0000 7f5e0a800640 1 -- 172.21.15.42:0/508854755 >> 172.21.15.42:0/508854755 conn(0x7f5e040afaa0 msgr2=0x7f5e040b36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:48.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.484+0000 7f5e0a800640 1 -- 172.21.15.42:0/508854755 shutdown_connections 2024-07-04T17:26:48.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:48.484+0000 7f5e0a800640 1 -- 172.21.15.42:0/508854755 wait complete. 2024-07-04T17:26:48.511 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-04T17:26:48.511 DEBUG:teuthology.orchestra.run.smithi042:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-04T17:26:48.619 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-04T17:26:48.619 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:26:48.619 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:26:48.675 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:26:48.675 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:26:48.732 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-04T17:26:49.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.015+0000 7f09aac00640 1 -- 172.21.15.42:0/70863663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09a40b8f80 msgr2=0x7f09a406c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:49.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.015+0000 7f09aac00640 1 --2- 172.21.15.42:0/70863663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09a40b8f80 0x7f09a406c020 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0998005fd0 tx=0x7f099802aac0 comp rx=0 tx=0).stop 2024-07-04T17:26:49.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.015+0000 7f09aac00640 1 -- 172.21.15.42:0/70863663 shutdown_connections 2024-07-04T17:26:49.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.015+0000 7f09aac00640 1 --2- 172.21.15.42:0/70863663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09a40b8f80 0x7f09a406c020 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:49.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.015+0000 7f09aac00640 1 -- 172.21.15.42:0/70863663 >> 172.21.15.42:0/70863663 conn(0x7f09a40bebc0 msgr2=0x7f09a40befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:49.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.016+0000 7f09aac00640 1 -- 172.21.15.42:0/70863663 shutdown_connections 2024-07-04T17:26:49.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.016+0000 7f09aac00640 1 -- 172.21.15.42:0/70863663 wait complete. 2024-07-04T17:26:49.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.018+0000 7f09aac00640 1 Processor -- start 2024-07-04T17:26:49.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.018+0000 7f09aac00640 1 -- start start 2024-07-04T17:26:49.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.019+0000 7f09aac00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09a40b2110 0x7f09a40aa410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:49.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.019+0000 7f09aac00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f09a40b28b0 con 0x7f09a40b2110 2024-07-04T17:26:49.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.020+0000 7f09a9800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09a40b2110 0x7f09a40aa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:49.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.020+0000 7f09a9800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09a40b2110 0x7f09a40aa410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36886/0 (socket says 172.21.15.42:36886) 2024-07-04T17:26:49.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.020+0000 7f09a9800640 1 -- 172.21.15.42:0/2485561863 learned_addr learned my addr 172.21.15.42:0/2485561863 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:49.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.020+0000 7f09a9800640 1 -- 172.21.15.42:0/2485561863 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a40b8750 con 0x7f09a40b2110 2024-07-04T17:26:49.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.020+0000 7f09a9800640 1 --2- 172.21.15.42:0/2485561863 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09a40b2110 0x7f09a40aa410 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0998006010 tx=0x7f0998003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:49.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.021+0000 7f09a1600640 1 -- 172.21.15.42:0/2485561863 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0998044070 con 0x7f09a40b2110 2024-07-04T17:26:49.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.021+0000 7f09a1600640 1 -- 172.21.15.42:0/2485561863 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0998004380 con 0x7f09a40b2110 2024-07-04T17:26:49.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.021+0000 7f09aac00640 1 -- 172.21.15.42:0/2485561863 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a40b8240 con 0x7f09a40b2110 2024-07-04T17:26:49.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.021+0000 7f09a1600640 1 -- 172.21.15.42:0/2485561863 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09980046c0 con 0x7f09a40b2110 2024-07-04T17:26:49.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.021+0000 7f09aac00640 1 -- 172.21.15.42:0/2485561863 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a40b0da0 con 0x7f09a40b2110 2024-07-04T17:26:49.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.022+0000 7f09aac00640 1 -- 172.21.15.42:0/2485561863 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09a40ba920 con 0x7f09a40b2110 2024-07-04T17:26:49.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.022+0000 7f09a1600640 1 -- 172.21.15.42:0/2485561863 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f0998047070 con 0x7f09a40b2110 2024-07-04T17:26:49.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.023+0000 7f09a1600640 1 --2- 172.21.15.42:0/2485561863 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f098403d2b0 0x7f098403f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:49.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.023+0000 7f09a1600640 1 -- 172.21.15.42:0/2485561863 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f099803c070 con 0x7f09a40b2110 2024-07-04T17:26:49.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.023+0000 7f09a8e00640 1 --2- 172.21.15.42:0/2485561863 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f098403d2b0 0x7f098403f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:49.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.024+0000 7f09a8e00640 1 --2- 172.21.15.42:0/2485561863 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f098403d2b0 0x7f098403f7c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f09a4147550 tx=0x7f09900079f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:49.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.028+0000 7f09a1600640 1 -- 172.21.15.42:0/2485561863 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f09a40b0da0 con 0x7f09a40b2110 2024-07-04T17:26:49.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.254+0000 7f09aac00640 1 -- 172.21.15.42:0/2485561863 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f09a4068e30 con 0x7f09a40b2110 2024-07-04T17:26:49.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.255+0000 7f09a1600640 1 -- 172.21.15.42:0/2485561863 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v15) ==== 80+0+0 (secure 0 0 0) 0x7f0998032030 con 0x7f09a40b2110 2024-07-04T17:26:49.257 INFO:teuthology.orchestra.run.smithi042.stderr:listed 0 entries 2024-07-04T17:26:49.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.259+0000 7f09aac00640 1 -- 172.21.15.42:0/2485561863 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f098403d2b0 msgr2=0x7f098403f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:49.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.259+0000 7f09aac00640 1 --2- 172.21.15.42:0/2485561863 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f098403d2b0 0x7f098403f7c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f09a4147550 tx=0x7f09900079f0 comp rx=0 tx=0).stop 2024-07-04T17:26:49.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.259+0000 7f09aac00640 1 -- 172.21.15.42:0/2485561863 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09a40b2110 msgr2=0x7f09a40aa410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:49.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.259+0000 7f09aac00640 1 --2- 172.21.15.42:0/2485561863 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09a40b2110 0x7f09a40aa410 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0998006010 tx=0x7f0998003f10 comp rx=0 tx=0).stop 2024-07-04T17:26:49.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.259+0000 7f09aac00640 1 -- 172.21.15.42:0/2485561863 shutdown_connections 2024-07-04T17:26:49.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.259+0000 7f09aac00640 1 --2- 172.21.15.42:0/2485561863 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f098403d2b0 0x7f098403f7c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:49.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.259+0000 7f09aac00640 1 --2- 172.21.15.42:0/2485561863 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09a40b2110 0x7f09a40aa410 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:49.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.259+0000 7f09aac00640 1 -- 172.21.15.42:0/2485561863 >> 172.21.15.42:0/2485561863 conn(0x7f09a40bebc0 msgr2=0x7f09a40b9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:49.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.259+0000 7f09aac00640 1 -- 172.21.15.42:0/2485561863 shutdown_connections 2024-07-04T17:26:49.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.259+0000 7f09aac00640 1 -- 172.21.15.42:0/2485561863 wait complete. 2024-07-04T17:26:49.286 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:26:49.286 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:26:49.339 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-04T17:26:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.620+0000 7f2240c00640 1 -- 172.21.15.42:0/2804571800 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f223c06d580 msgr2=0x7f223c06c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.620+0000 7f2240c00640 1 --2- 172.21.15.42:0/2804571800 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f223c06d580 0x7f223c06c2c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f222c005fd0 tx=0x7f222c02bb50 comp rx=0 tx=0).stop 2024-07-04T17:26:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.621+0000 7f2240c00640 1 -- 172.21.15.42:0/2804571800 shutdown_connections 2024-07-04T17:26:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.621+0000 7f2240c00640 1 --2- 172.21.15.42:0/2804571800 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f223c06d580 0x7f223c06c2c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.621+0000 7f2240c00640 1 -- 172.21.15.42:0/2804571800 >> 172.21.15.42:0/2804571800 conn(0x7f223c0bebc0 msgr2=0x7f223c0befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.621+0000 7f2240c00640 1 -- 172.21.15.42:0/2804571800 shutdown_connections 2024-07-04T17:26:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.621+0000 7f2240c00640 1 -- 172.21.15.42:0/2804571800 wait complete. 2024-07-04T17:26:49.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.623+0000 7f2240c00640 1 Processor -- start 2024-07-04T17:26:49.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.623+0000 7f2240c00640 1 -- start start 2024-07-04T17:26:49.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.625+0000 7f2240c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f223c06d580 0x7f223c1d4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:49.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.625+0000 7f2240c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f223c0b2810 con 0x7f223c06d580 2024-07-04T17:26:49.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.625+0000 7f223b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f223c06d580 0x7f223c1d4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:49.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.625+0000 7f223b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f223c06d580 0x7f223c1d4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36892/0 (socket says 172.21.15.42:36892) 2024-07-04T17:26:49.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.625+0000 7f223b400640 1 -- 172.21.15.42:0/3190729004 learned_addr learned my addr 172.21.15.42:0/3190729004 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:49.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.625+0000 7f223b400640 1 -- 172.21.15.42:0/3190729004 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f223c070890 con 0x7f223c06d580 2024-07-04T17:26:49.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.625+0000 7f223b400640 1 --2- 172.21.15.42:0/3190729004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f223c06d580 0x7f223c1d4e30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f222c00ac40 tx=0x7f222c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:49.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.626+0000 7f2233400640 1 -- 172.21.15.42:0/3190729004 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f222c004380 con 0x7f223c06d580 2024-07-04T17:26:49.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.626+0000 7f2233400640 1 -- 172.21.15.42:0/3190729004 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f222c004520 con 0x7f223c06d580 2024-07-04T17:26:49.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.626+0000 7f2233400640 1 -- 172.21.15.42:0/3190729004 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f222c004860 con 0x7f223c06d580 2024-07-04T17:26:49.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.626+0000 7f2240c00640 1 -- 172.21.15.42:0/3190729004 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f223c072330 con 0x7f223c06d580 2024-07-04T17:26:49.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.626+0000 7f2240c00640 1 -- 172.21.15.42:0/3190729004 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f223c06fc90 con 0x7f223c06d580 2024-07-04T17:26:49.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.627+0000 7f2240c00640 1 -- 172.21.15.42:0/3190729004 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f223c075a10 con 0x7f223c06d580 2024-07-04T17:26:49.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.627+0000 7f2233400640 1 -- 172.21.15.42:0/3190729004 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f222c02e480 con 0x7f223c06d580 2024-07-04T17:26:49.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.628+0000 7f2233400640 1 --2- 172.21.15.42:0/3190729004 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f221c03d2b0 0x7f221c03f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:49.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.628+0000 7f2233400640 1 -- 172.21.15.42:0/3190729004 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f222c072f00 con 0x7f223c06d580 2024-07-04T17:26:49.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.628+0000 7f223aa00640 1 --2- 172.21.15.42:0/3190729004 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f221c03d2b0 0x7f221c03f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:49.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.629+0000 7f223aa00640 1 --2- 172.21.15.42:0/3190729004 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f221c03d2b0 0x7f221c03f7c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2228007fd0 tx=0x7f2228007920 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:49.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.633+0000 7f2233400640 1 -- 172.21.15.42:0/3190729004 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f223c075a10 con 0x7f223c06d580 2024-07-04T17:26:49.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.855+0000 7f2240c00640 1 -- 172.21.15.42:0/3190729004 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f223c06c2c0 con 0x7f223c06d580 2024-07-04T17:26:49.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.856+0000 7f2233400640 1 -- 172.21.15.42:0/3190729004 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v15) ==== 80+0+0 (secure 0 0 0) 0x7f222c033070 con 0x7f223c06d580 2024-07-04T17:26:49.858 INFO:teuthology.orchestra.run.smithi042.stderr:listed 0 entries 2024-07-04T17:26:49.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.859+0000 7f2240c00640 1 -- 172.21.15.42:0/3190729004 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f221c03d2b0 msgr2=0x7f221c03f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:49.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.859+0000 7f2240c00640 1 --2- 172.21.15.42:0/3190729004 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f221c03d2b0 0x7f221c03f7c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2228007fd0 tx=0x7f2228007920 comp rx=0 tx=0).stop 2024-07-04T17:26:49.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.859+0000 7f2240c00640 1 -- 172.21.15.42:0/3190729004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f223c06d580 msgr2=0x7f223c1d4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:49.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.859+0000 7f2240c00640 1 --2- 172.21.15.42:0/3190729004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f223c06d580 0x7f223c1d4e30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f222c00ac40 tx=0x7f222c003f10 comp rx=0 tx=0).stop 2024-07-04T17:26:49.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.859+0000 7f2240c00640 1 -- 172.21.15.42:0/3190729004 shutdown_connections 2024-07-04T17:26:49.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.859+0000 7f2240c00640 1 --2- 172.21.15.42:0/3190729004 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f221c03d2b0 0x7f221c03f7c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:49.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.859+0000 7f2240c00640 1 --2- 172.21.15.42:0/3190729004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f223c06d580 0x7f223c1d4e30 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:49.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.859+0000 7f2240c00640 1 -- 172.21.15.42:0/3190729004 >> 172.21.15.42:0/3190729004 conn(0x7f223c0bebc0 msgr2=0x7f223c141560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:49.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.860+0000 7f2240c00640 1 -- 172.21.15.42:0/3190729004 shutdown_connections 2024-07-04T17:26:49.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:49.860+0000 7f2240c00640 1 -- 172.21.15.42:0/3190729004 wait complete. 2024-07-04T17:26:49.890 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi042.front.sepia.ceph.com... 2024-07-04T17:26:49.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:26:49.890 DEBUG:teuthology.orchestra.run.smithi042:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-07-04T17:26:49.979 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:26:49.979 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-07-04T17:26:49.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:26:49.980 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-04T17:26:50.050 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:26:50.050 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:26:50.104 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:26:50.104 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:26:50.158 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-04T17:26:50.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.430+0000 7f066de00640 1 -- 172.21.15.42:0/1032366849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06680bad70 msgr2=0x7f066806c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:50.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.430+0000 7f066de00640 1 --2- 172.21.15.42:0/1032366849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06680bad70 0x7f066806c2c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0654005fd0 tx=0x7f065402bb60 comp rx=0 tx=0).stop 2024-07-04T17:26:50.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.430+0000 7f066de00640 1 -- 172.21.15.42:0/1032366849 shutdown_connections 2024-07-04T17:26:50.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.430+0000 7f066de00640 1 --2- 172.21.15.42:0/1032366849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06680bad70 0x7f066806c2c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:50.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.430+0000 7f066de00640 1 -- 172.21.15.42:0/1032366849 >> 172.21.15.42:0/1032366849 conn(0x7f06680bebc0 msgr2=0x7f06680befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:50.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.431+0000 7f066de00640 1 -- 172.21.15.42:0/1032366849 shutdown_connections 2024-07-04T17:26:50.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.431+0000 7f066de00640 1 -- 172.21.15.42:0/1032366849 wait complete. 2024-07-04T17:26:50.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.433+0000 7f066de00640 1 Processor -- start 2024-07-04T17:26:50.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.433+0000 7f066de00640 1 -- start start 2024-07-04T17:26:50.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.434+0000 7f066de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f066806d580 0x7f0668070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:50.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.434+0000 7f066de00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f06680b2810 con 0x7f066806d580 2024-07-04T17:26:50.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.435+0000 7f066ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f066806d580 0x7f0668070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:50.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.435+0000 7f066ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f066806d580 0x7f0668070cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36922/0 (socket says 172.21.15.42:36922) 2024-07-04T17:26:50.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.435+0000 7f066ca00640 1 -- 172.21.15.42:0/2256802541 learned_addr learned my addr 172.21.15.42:0/2256802541 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:50.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.435+0000 7f066ca00640 1 -- 172.21.15.42:0/2256802541 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0668070800 con 0x7f066806d580 2024-07-04T17:26:50.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.435+0000 7f066ca00640 1 --2- 172.21.15.42:0/2256802541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f066806d580 0x7f0668070cc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f065800c3b0 tx=0x7f065800c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:50.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.436+0000 7f0664c00640 1 -- 172.21.15.42:0/2256802541 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0658011780 con 0x7f066806d580 2024-07-04T17:26:50.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.436+0000 7f0664c00640 1 -- 172.21.15.42:0/2256802541 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0658017070 con 0x7f066806d580 2024-07-04T17:26:50.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.436+0000 7f0664c00640 1 -- 172.21.15.42:0/2256802541 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0658011c00 con 0x7f066806d580 2024-07-04T17:26:50.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.436+0000 7f066de00640 1 -- 172.21.15.42:0/2256802541 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0668072280 con 0x7f066806d580 2024-07-04T17:26:50.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.436+0000 7f066de00640 1 -- 172.21.15.42:0/2256802541 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06681d67e0 con 0x7f066806d580 2024-07-04T17:26:50.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.437+0000 7f066de00640 1 -- 172.21.15.42:0/2256802541 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f066806c2c0 con 0x7f066806d580 2024-07-04T17:26:50.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.437+0000 7f0664c00640 1 -- 172.21.15.42:0/2256802541 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f065801c460 con 0x7f066806d580 2024-07-04T17:26:50.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.438+0000 7f0664c00640 1 --2- 172.21.15.42:0/2256802541 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f064803d2b0 0x7f064803f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:50.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.438+0000 7f0664c00640 1 -- 172.21.15.42:0/2256802541 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f0658051320 con 0x7f066806d580 2024-07-04T17:26:50.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.438+0000 7f0667e00640 1 --2- 172.21.15.42:0/2256802541 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f064803d2b0 0x7f064803f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:50.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.439+0000 7f0667e00640 1 --2- 172.21.15.42:0/2256802541 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f064803d2b0 0x7f064803f7c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0654035000 tx=0x7f0654004ab0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:50.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.443+0000 7f0664c00640 1 -- 172.21.15.42:0/2256802541 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f065800a140 con 0x7f066806d580 2024-07-04T17:26:50.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.667+0000 7f066de00640 1 -- 172.21.15.42:0/2256802541 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f06680709d0 con 0x7f066806d580 2024-07-04T17:26:50.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.668+0000 7f0664c00640 1 -- 172.21.15.42:0/2256802541 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v15) ==== 80+0+0 (secure 0 0 0) 0x7f065801c800 con 0x7f066806d580 2024-07-04T17:26:50.670 INFO:teuthology.orchestra.run.smithi042.stderr:listed 0 entries 2024-07-04T17:26:50.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.672+0000 7f066de00640 1 -- 172.21.15.42:0/2256802541 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f064803d2b0 msgr2=0x7f064803f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:50.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.672+0000 7f066de00640 1 --2- 172.21.15.42:0/2256802541 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f064803d2b0 0x7f064803f7c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0654035000 tx=0x7f0654004ab0 comp rx=0 tx=0).stop 2024-07-04T17:26:50.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.672+0000 7f066de00640 1 -- 172.21.15.42:0/2256802541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f066806d580 msgr2=0x7f0668070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:50.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.672+0000 7f066de00640 1 --2- 172.21.15.42:0/2256802541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f066806d580 0x7f0668070cc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f065800c3b0 tx=0x7f065800c870 comp rx=0 tx=0).stop 2024-07-04T17:26:50.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.672+0000 7f066de00640 1 -- 172.21.15.42:0/2256802541 shutdown_connections 2024-07-04T17:26:50.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.672+0000 7f066de00640 1 --2- 172.21.15.42:0/2256802541 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f064803d2b0 0x7f064803f7c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:50.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.672+0000 7f066de00640 1 --2- 172.21.15.42:0/2256802541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f066806d580 0x7f0668070cc0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:50.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.672+0000 7f066de00640 1 -- 172.21.15.42:0/2256802541 >> 172.21.15.42:0/2256802541 conn(0x7f06680bebc0 msgr2=0x7f06680bcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:50.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.672+0000 7f066de00640 1 -- 172.21.15.42:0/2256802541 shutdown_connections 2024-07-04T17:26:50.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:50.672+0000 7f066de00640 1 -- 172.21.15.42:0/2256802541 wait complete. 2024-07-04T17:26:50.698 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:26:50.698 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:26:50.750 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-04T17:26:51.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.088+0000 7f8ea2600640 1 -- 172.21.15.42:0/325835560 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e9c06cea0 msgr2=0x7f8e9c0b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:51.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.088+0000 7f8ea2600640 1 --2- 172.21.15.42:0/325835560 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e9c06cea0 0x7f8e9c0b3030 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8e84005fd0 tx=0x7f8e8402a5f0 comp rx=0 tx=0).stop 2024-07-04T17:26:51.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.088+0000 7f8ea2600640 1 -- 172.21.15.42:0/325835560 shutdown_connections 2024-07-04T17:26:51.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.088+0000 7f8ea2600640 1 --2- 172.21.15.42:0/325835560 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e9c06cea0 0x7f8e9c0b3030 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:51.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.088+0000 7f8ea2600640 1 -- 172.21.15.42:0/325835560 >> 172.21.15.42:0/325835560 conn(0x7f8e9c0bf700 msgr2=0x7f8e9c0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:51.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.088+0000 7f8ea2600640 1 -- 172.21.15.42:0/325835560 shutdown_connections 2024-07-04T17:26:51.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.088+0000 7f8ea2600640 1 -- 172.21.15.42:0/325835560 wait complete. 2024-07-04T17:26:51.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.090+0000 7f8ea2600640 1 Processor -- start 2024-07-04T17:26:51.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.090+0000 7f8ea2600640 1 -- start start 2024-07-04T17:26:51.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.092+0000 7f8ea2600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e9c06cea0 0x7f8e9c149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:51.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.092+0000 7f8ea2600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c0b0580 con 0x7f8e9c06cea0 2024-07-04T17:26:51.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.092+0000 7f8e9b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e9c06cea0 0x7f8e9c149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:51.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.092+0000 7f8e9b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e9c06cea0 0x7f8e9c149920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36948/0 (socket says 172.21.15.42:36948) 2024-07-04T17:26:51.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.092+0000 7f8e9b400640 1 -- 172.21.15.42:0/1541731288 learned_addr learned my addr 172.21.15.42:0/1541731288 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:51.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.093+0000 7f8e9b400640 1 -- 172.21.15.42:0/1541731288 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e9c1d5f50 con 0x7f8e9c06cea0 2024-07-04T17:26:51.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.093+0000 7f8e9b400640 1 --2- 172.21.15.42:0/1541731288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e9c06cea0 0x7f8e9c149920 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c00c380 tx=0x7f8e8c00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:51.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.093+0000 7f8e93400640 1 -- 172.21.15.42:0/1541731288 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e8c017070 con 0x7f8e9c06cea0 2024-07-04T17:26:51.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.093+0000 7f8e93400640 1 -- 172.21.15.42:0/1541731288 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e8c0052f0 con 0x7f8e9c06cea0 2024-07-04T17:26:51.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.093+0000 7f8ea2600640 1 -- 172.21.15.42:0/1541731288 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e9c149f10 con 0x7f8e9c06cea0 2024-07-04T17:26:51.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.093+0000 7f8e93400640 1 -- 172.21.15.42:0/1541731288 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e8c0035f0 con 0x7f8e9c06cea0 2024-07-04T17:26:51.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.093+0000 7f8ea2600640 1 -- 172.21.15.42:0/1541731288 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e9c1d7190 con 0x7f8e9c06cea0 2024-07-04T17:26:51.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.094+0000 7f8ea2600640 1 -- 172.21.15.42:0/1541731288 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e9c063f70 con 0x7f8e9c06cea0 2024-07-04T17:26:51.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.095+0000 7f8e93400640 1 -- 172.21.15.42:0/1541731288 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f8e8c01a070 con 0x7f8e9c06cea0 2024-07-04T17:26:51.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.095+0000 7f8e93400640 1 --2- 172.21.15.42:0/1541731288 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f8e7803d2b0 0x7f8e7803f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:51.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.095+0000 7f8e93400640 1 -- 172.21.15.42:0/1541731288 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f8e8c0507c0 con 0x7f8e9c06cea0 2024-07-04T17:26:51.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.096+0000 7f8e93e00640 1 --2- 172.21.15.42:0/1541731288 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f8e7803d2b0 0x7f8e7803f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:51.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.096+0000 7f8e93e00640 1 --2- 172.21.15.42:0/1541731288 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f8e7803d2b0 0x7f8e7803f7c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8e84033000 tx=0x7f8e84004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:51.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.101+0000 7f8e93400640 1 -- 172.21.15.42:0/1541731288 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f8e9c063f70 con 0x7f8e9c06cea0 2024-07-04T17:26:51.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.325+0000 7f8ea2600640 1 -- 172.21.15.42:0/1541731288 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8e9c0ae850 con 0x7f8e9c06cea0 2024-07-04T17:26:51.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.326+0000 7f8e93400640 1 -- 172.21.15.42:0/1541731288 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v15) ==== 80+0+0 (secure 0 0 0) 0x7f8e9c0ae850 con 0x7f8e9c06cea0 2024-07-04T17:26:51.327 INFO:teuthology.orchestra.run.smithi042.stderr:listed 0 entries 2024-07-04T17:26:51.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.329+0000 7f8ea2600640 1 -- 172.21.15.42:0/1541731288 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f8e7803d2b0 msgr2=0x7f8e7803f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:51.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.329+0000 7f8ea2600640 1 --2- 172.21.15.42:0/1541731288 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f8e7803d2b0 0x7f8e7803f7c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8e84033000 tx=0x7f8e84004730 comp rx=0 tx=0).stop 2024-07-04T17:26:51.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.329+0000 7f8ea2600640 1 -- 172.21.15.42:0/1541731288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e9c06cea0 msgr2=0x7f8e9c149920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:51.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.329+0000 7f8ea2600640 1 --2- 172.21.15.42:0/1541731288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e9c06cea0 0x7f8e9c149920 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c00c380 tx=0x7f8e8c00c840 comp rx=0 tx=0).stop 2024-07-04T17:26:51.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.329+0000 7f8ea2600640 1 -- 172.21.15.42:0/1541731288 shutdown_connections 2024-07-04T17:26:51.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.329+0000 7f8ea2600640 1 --2- 172.21.15.42:0/1541731288 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f8e7803d2b0 0x7f8e7803f7c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:51.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.329+0000 7f8ea2600640 1 --2- 172.21.15.42:0/1541731288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e9c06cea0 0x7f8e9c149920 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:51.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.329+0000 7f8ea2600640 1 -- 172.21.15.42:0/1541731288 >> 172.21.15.42:0/1541731288 conn(0x7f8e9c0bf700 msgr2=0x7f8e9c075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:51.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.329+0000 7f8ea2600640 1 -- 172.21.15.42:0/1541731288 shutdown_connections 2024-07-04T17:26:51.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.329+0000 7f8ea2600640 1 -- 172.21.15.42:0/1541731288 wait complete. 2024-07-04T17:26:51.357 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi042.front.sepia.ceph.com... 2024-07-04T17:26:51.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:26:51.357 DEBUG:teuthology.orchestra.run.smithi042:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-07-04T17:26:51.446 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:26:51.446 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-07-04T17:26:51.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:26:51.446 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-07-04T17:26:51.503 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-07-04T17:26:51.503 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:26:51.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.785+0000 7f4780600640 1 -- 172.21.15.42:0/3899301171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47780b0cd0 msgr2=0x7f47780b34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:51.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.785+0000 7f4780600640 1 --2- 172.21.15.42:0/3899301171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47780b0cd0 0x7f47780b34b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f476400a8c0 tx=0x7f476402b860 comp rx=0 tx=0).stop 2024-07-04T17:26:51.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.786+0000 7f4780600640 1 -- 172.21.15.42:0/3899301171 shutdown_connections 2024-07-04T17:26:51.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.786+0000 7f4780600640 1 --2- 172.21.15.42:0/3899301171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47780b0cd0 0x7f47780b34b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:51.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.786+0000 7f4780600640 1 -- 172.21.15.42:0/3899301171 >> 172.21.15.42:0/3899301171 conn(0x7f47780bf700 msgr2=0x7f47780af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:51.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.786+0000 7f4780600640 1 -- 172.21.15.42:0/3899301171 shutdown_connections 2024-07-04T17:26:51.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.786+0000 7f4780600640 1 -- 172.21.15.42:0/3899301171 wait complete. 2024-07-04T17:26:51.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.788+0000 7f4780600640 1 Processor -- start 2024-07-04T17:26:51.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.788+0000 7f4780600640 1 -- start start 2024-07-04T17:26:51.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.789+0000 7f4780600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47780abdd0 0x7f4778072100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:51.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.789+0000 7f4780600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f47780b03a0 con 0x7f47780abdd0 2024-07-04T17:26:51.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.790+0000 7f477dc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47780abdd0 0x7f4778072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:51.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.790+0000 7f477dc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47780abdd0 0x7f4778072100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36970/0 (socket says 172.21.15.42:36970) 2024-07-04T17:26:51.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.790+0000 7f477dc00640 1 -- 172.21.15.42:0/2831456345 learned_addr learned my addr 172.21.15.42:0/2831456345 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:51.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.790+0000 7f477dc00640 1 -- 172.21.15.42:0/2831456345 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47780b8900 con 0x7f47780abdd0 2024-07-04T17:26:51.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.791+0000 7f477dc00640 1 --2- 172.21.15.42:0/2831456345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47780abdd0 0x7f4778072100 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f476c00c6b0 tx=0x7f476c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:51.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.791+0000 7f4775600640 1 -- 172.21.15.42:0/2831456345 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f476c016020 con 0x7f47780abdd0 2024-07-04T17:26:51.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.791+0000 7f4775600640 1 -- 172.21.15.42:0/2831456345 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f476c00a5c0 con 0x7f47780abdd0 2024-07-04T17:26:51.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.791+0000 7f4775600640 1 -- 172.21.15.42:0/2831456345 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f476c00a960 con 0x7f47780abdd0 2024-07-04T17:26:51.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.791+0000 7f4780600640 1 -- 172.21.15.42:0/2831456345 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47780726d0 con 0x7f47780abdd0 2024-07-04T17:26:51.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.791+0000 7f4780600640 1 -- 172.21.15.42:0/2831456345 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47781dad70 con 0x7f47780abdd0 2024-07-04T17:26:51.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.792+0000 7f4780600640 1 -- 172.21.15.42:0/2831456345 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47780b2da0 con 0x7f47780abdd0 2024-07-04T17:26:51.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.792+0000 7f4775600640 1 -- 172.21.15.42:0/2831456345 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f476c019020 con 0x7f47780abdd0 2024-07-04T17:26:51.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.793+0000 7f4775600640 1 --2- 172.21.15.42:0/2831456345 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f475803d2b0 0x7f475803f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:51.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.793+0000 7f4775600640 1 -- 172.21.15.42:0/2831456345 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f476c051390 con 0x7f47780abdd0 2024-07-04T17:26:51.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.793+0000 7f477d200640 1 --2- 172.21.15.42:0/2831456345 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f475803d2b0 0x7f475803f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:51.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.794+0000 7f477d200640 1 --2- 172.21.15.42:0/2831456345 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f475803d2b0 0x7f475803f7c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4764036000 tx=0x7f4764004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:51.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:51.798+0000 7f4775600640 1 -- 172.21.15.42:0/2831456345 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f476c020020 con 0x7f47780abdd0 2024-07-04T17:26:52.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.044+0000 7f4780600640 1 -- 172.21.15.42:0/2831456345 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47780bdce0 con 0x7f47780abdd0 2024-07-04T17:26:52.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.045+0000 7f4775600640 1 -- 172.21.15.42:0/2831456345 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6035 (secure 0 0 0) 0x7f47780b10e0 con 0x7f47780abdd0 2024-07-04T17:26:52.047 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:26:52.047 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":8,"btime":"2024-07-04T17:26:19:025883+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4192,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/3484212194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":3484212194},{"type":"v1","addr":"172.21.15.42:6827","nonce":3484212194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4202,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6830/815222782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6829","nonce":815222782},{"type":"v1","addr":"172.21.15.42:6830","nonce":815222782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/999331079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":999331079},{"type":"v1","addr":"172.21.15.42:6831","nonce":999331079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4204,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6833/2286778970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2286778970},{"type":"v1","addr":"172.21.15.42:6833","nonce":2286778970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:26:15.573602+0000","modified":"2024-07-04T17:26:18.625664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4205":{"gid":4205,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6835/3941225191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3941225191},{"type":"v1","addr":"172.21.15.42:6835","nonce":3941225191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4205,"qdb_cluster":[4205]},"id":1}]} 2024-07-04T17:26:52.047 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 8 2024-07-04T17:26:52.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.047+0000 7f4780600640 1 -- 172.21.15.42:0/2831456345 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f475803d2b0 msgr2=0x7f475803f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:52.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.047+0000 7f4780600640 1 --2- 172.21.15.42:0/2831456345 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f475803d2b0 0x7f475803f7c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4764036000 tx=0x7f4764004730 comp rx=0 tx=0).stop 2024-07-04T17:26:52.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.047+0000 7f4780600640 1 -- 172.21.15.42:0/2831456345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47780abdd0 msgr2=0x7f4778072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:52.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.048+0000 7f4780600640 1 --2- 172.21.15.42:0/2831456345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47780abdd0 0x7f4778072100 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f476c00c6b0 tx=0x7f476c00cb70 comp rx=0 tx=0).stop 2024-07-04T17:26:52.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.048+0000 7f4780600640 1 -- 172.21.15.42:0/2831456345 shutdown_connections 2024-07-04T17:26:52.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.048+0000 7f4780600640 1 --2- 172.21.15.42:0/2831456345 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f475803d2b0 0x7f475803f7c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:52.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.048+0000 7f4780600640 1 --2- 172.21.15.42:0/2831456345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47780abdd0 0x7f4778072100 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:52.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.048+0000 7f4780600640 1 -- 172.21.15.42:0/2831456345 >> 172.21.15.42:0/2831456345 conn(0x7f47780bf700 msgr2=0x7f47780aaba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:52.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.048+0000 7f4780600640 1 -- 172.21.15.42:0/2831456345 shutdown_connections 2024-07-04T17:26:52.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.048+0000 7f4780600640 1 -- 172.21.15.42:0/2831456345 wait complete. 2024-07-04T17:26:52.075 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:26:52.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.354+0000 7fe991800640 1 -- 172.21.15.42:0/3601760846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe98c0b17d0 msgr2=0x7fe98c0b1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:52.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.354+0000 7fe991800640 1 --2- 172.21.15.42:0/3601760846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe98c0b17d0 0x7fe98c0b1be0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe9780097b0 tx=0x7fe97802a4a0 comp rx=0 tx=0).stop 2024-07-04T17:26:52.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.354+0000 7fe991800640 1 -- 172.21.15.42:0/3601760846 shutdown_connections 2024-07-04T17:26:52.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.354+0000 7fe991800640 1 --2- 172.21.15.42:0/3601760846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe98c0b17d0 0x7fe98c0b1be0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:52.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.354+0000 7fe991800640 1 -- 172.21.15.42:0/3601760846 >> 172.21.15.42:0/3601760846 conn(0x7fe98c0bebc0 msgr2=0x7fe98c0befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:52.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.354+0000 7fe991800640 1 -- 172.21.15.42:0/3601760846 shutdown_connections 2024-07-04T17:26:52.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.354+0000 7fe991800640 1 -- 172.21.15.42:0/3601760846 wait complete. 2024-07-04T17:26:52.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.356+0000 7fe991800640 1 Processor -- start 2024-07-04T17:26:52.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.356+0000 7fe991800640 1 -- start start 2024-07-04T17:26:52.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.358+0000 7fe991800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe98c06c200 0x7fe98c0b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:52.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.358+0000 7fe991800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c14ae80 con 0x7fe98c06c200 2024-07-04T17:26:52.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.358+0000 7fe98be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe98c06c200 0x7fe98c0b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:52.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.358+0000 7fe98be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe98c06c200 0x7fe98c0b75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36982/0 (socket says 172.21.15.42:36982) 2024-07-04T17:26:52.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.358+0000 7fe98be00640 1 -- 172.21.15.42:0/2672477875 learned_addr learned my addr 172.21.15.42:0/2672477875 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:52.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.358+0000 7fe98be00640 1 -- 172.21.15.42:0/2672477875 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98c1e39c0 con 0x7fe98c06c200 2024-07-04T17:26:52.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.359+0000 7fe98be00640 1 --2- 172.21.15.42:0/2672477875 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe98c06c200 0x7fe98c0b75c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe97c00cb40 tx=0x7fe97c0021e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:52.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.359+0000 7fe983e00640 1 -- 172.21.15.42:0/2672477875 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe97c016020 con 0x7fe98c06c200 2024-07-04T17:26:52.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.359+0000 7fe983e00640 1 -- 172.21.15.42:0/2672477875 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe97c00abd0 con 0x7fe98c06c200 2024-07-04T17:26:52.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.359+0000 7fe991800640 1 -- 172.21.15.42:0/2672477875 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe98c147430 con 0x7fe98c06c200 2024-07-04T17:26:52.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.359+0000 7fe983e00640 1 -- 172.21.15.42:0/2672477875 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe97c012420 con 0x7fe98c06c200 2024-07-04T17:26:52.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.359+0000 7fe991800640 1 -- 172.21.15.42:0/2672477875 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe98c0b0250 con 0x7fe98c06c200 2024-07-04T17:26:52.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.360+0000 7fe991800640 1 -- 172.21.15.42:0/2672477875 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe98c0b44b0 con 0x7fe98c06c200 2024-07-04T17:26:52.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.360+0000 7fe983e00640 1 -- 172.21.15.42:0/2672477875 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fe97c019020 con 0x7fe98c06c200 2024-07-04T17:26:52.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.361+0000 7fe983e00640 1 --2- 172.21.15.42:0/2672477875 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fe96c03d2b0 0x7fe96c03f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:52.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.361+0000 7fe983e00640 1 -- 172.21.15.42:0/2672477875 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7fe97c051b30 con 0x7fe98c06c200 2024-07-04T17:26:52.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.361+0000 7fe98b400640 1 --2- 172.21.15.42:0/2672477875 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fe96c03d2b0 0x7fe96c03f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:52.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.362+0000 7fe98b400640 1 --2- 172.21.15.42:0/2672477875 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fe96c03d2b0 0x7fe96c03f7c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe978034000 tx=0x7fe978004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:52.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.366+0000 7fe983e00640 1 -- 172.21.15.42:0/2672477875 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fe98c0b44b0 con 0x7fe98c06c200 2024-07-04T17:26:52.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.619+0000 7fe991800640 1 -- 172.21.15.42:0/2672477875 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe98c14b4c0 con 0x7fe98c06c200 2024-07-04T17:26:52.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.621+0000 7fe983e00640 1 -- 172.21.15.42:0/2672477875 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6035 (secure 0 0 0) 0x7fe97c020020 con 0x7fe98c06c200 2024-07-04T17:26:52.623 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:26:52.623 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":8,"btime":"2024-07-04T17:26:19:025883+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4192,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/3484212194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":3484212194},{"type":"v1","addr":"172.21.15.42:6827","nonce":3484212194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4202,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6830/815222782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6829","nonce":815222782},{"type":"v1","addr":"172.21.15.42:6830","nonce":815222782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/999331079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":999331079},{"type":"v1","addr":"172.21.15.42:6831","nonce":999331079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4204,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6833/2286778970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2286778970},{"type":"v1","addr":"172.21.15.42:6833","nonce":2286778970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:26:15.573602+0000","modified":"2024-07-04T17:26:18.625664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4205":{"gid":4205,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6835/3941225191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3941225191},{"type":"v1","addr":"172.21.15.42:6835","nonce":3941225191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4205,"qdb_cluster":[4205]},"id":1}]} 2024-07-04T17:26:52.623 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 8 2024-07-04T17:26:52.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.623+0000 7fe991800640 1 -- 172.21.15.42:0/2672477875 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fe96c03d2b0 msgr2=0x7fe96c03f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:52.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.623+0000 7fe991800640 1 --2- 172.21.15.42:0/2672477875 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fe96c03d2b0 0x7fe96c03f7c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe978034000 tx=0x7fe978004730 comp rx=0 tx=0).stop 2024-07-04T17:26:52.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.623+0000 7fe991800640 1 -- 172.21.15.42:0/2672477875 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe98c06c200 msgr2=0x7fe98c0b75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:52.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.623+0000 7fe991800640 1 --2- 172.21.15.42:0/2672477875 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe98c06c200 0x7fe98c0b75c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe97c00cb40 tx=0x7fe97c0021e0 comp rx=0 tx=0).stop 2024-07-04T17:26:52.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.624+0000 7fe991800640 1 -- 172.21.15.42:0/2672477875 shutdown_connections 2024-07-04T17:26:52.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.624+0000 7fe991800640 1 --2- 172.21.15.42:0/2672477875 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fe96c03d2b0 0x7fe96c03f7c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:52.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.624+0000 7fe991800640 1 --2- 172.21.15.42:0/2672477875 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe98c06c200 0x7fe98c0b75c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:52.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.624+0000 7fe991800640 1 -- 172.21.15.42:0/2672477875 >> 172.21.15.42:0/2672477875 conn(0x7fe98c0bebc0 msgr2=0x7fe98c0b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:52.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.624+0000 7fe991800640 1 -- 172.21.15.42:0/2672477875 shutdown_connections 2024-07-04T17:26:52.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.624+0000 7fe991800640 1 -- 172.21.15.42:0/2672477875 wait complete. 2024-07-04T17:26:52.649 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:26:52.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.928+0000 7fa431200640 1 -- 172.21.15.42:0/3628914072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa42c0b17d0 msgr2=0x7fa42c0b1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:52.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.928+0000 7fa431200640 1 --2- 172.21.15.42:0/3628914072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa42c0b17d0 0x7fa42c0b1be0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa4140097b0 tx=0x7fa41402a4a0 comp rx=0 tx=0).stop 2024-07-04T17:26:52.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.928+0000 7fa431200640 1 -- 172.21.15.42:0/3628914072 shutdown_connections 2024-07-04T17:26:52.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.928+0000 7fa431200640 1 --2- 172.21.15.42:0/3628914072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa42c0b17d0 0x7fa42c0b1be0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:52.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.928+0000 7fa431200640 1 -- 172.21.15.42:0/3628914072 >> 172.21.15.42:0/3628914072 conn(0x7fa42c0bebc0 msgr2=0x7fa42c0befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:52.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.928+0000 7fa431200640 1 -- 172.21.15.42:0/3628914072 shutdown_connections 2024-07-04T17:26:52.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.928+0000 7fa431200640 1 -- 172.21.15.42:0/3628914072 wait complete. 2024-07-04T17:26:52.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.930+0000 7fa431200640 1 Processor -- start 2024-07-04T17:26:52.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.930+0000 7fa431200640 1 -- start start 2024-07-04T17:26:52.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.931+0000 7fa431200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa42c06c200 0x7fa42c0b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:52.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.931+0000 7fa431200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa42c14ae80 con 0x7fa42c06c200 2024-07-04T17:26:52.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.932+0000 7fa42b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa42c06c200 0x7fa42c0b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:52.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.932+0000 7fa42b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa42c06c200 0x7fa42c0b75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37002/0 (socket says 172.21.15.42:37002) 2024-07-04T17:26:52.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.932+0000 7fa42b400640 1 -- 172.21.15.42:0/1174260019 learned_addr learned my addr 172.21.15.42:0/1174260019 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:52.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.932+0000 7fa42b400640 1 -- 172.21.15.42:0/1174260019 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42c1e39c0 con 0x7fa42c06c200 2024-07-04T17:26:52.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.933+0000 7fa42b400640 1 --2- 172.21.15.42:0/1174260019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa42c06c200 0x7fa42c0b75c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa41c00c3b0 tx=0x7fa41c00c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:52.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.933+0000 7fa423400640 1 -- 172.21.15.42:0/1174260019 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa41c011780 con 0x7fa42c06c200 2024-07-04T17:26:52.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.933+0000 7fa423400640 1 -- 172.21.15.42:0/1174260019 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa41c017070 con 0x7fa42c06c200 2024-07-04T17:26:52.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.933+0000 7fa431200640 1 -- 172.21.15.42:0/1174260019 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa42c147430 con 0x7fa42c06c200 2024-07-04T17:26:52.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.933+0000 7fa423400640 1 -- 172.21.15.42:0/1174260019 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa41c011c00 con 0x7fa42c06c200 2024-07-04T17:26:52.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.933+0000 7fa431200640 1 -- 172.21.15.42:0/1174260019 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa42c0b0250 con 0x7fa42c06c200 2024-07-04T17:26:52.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.934+0000 7fa431200640 1 -- 172.21.15.42:0/1174260019 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa42c0b44b0 con 0x7fa42c06c200 2024-07-04T17:26:52.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.934+0000 7fa423400640 1 -- 172.21.15.42:0/1174260019 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fa41c01c460 con 0x7fa42c06c200 2024-07-04T17:26:52.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.935+0000 7fa423400640 1 --2- 172.21.15.42:0/1174260019 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa40803d260 0x7fa40803f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:52.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.935+0000 7fa423400640 1 -- 172.21.15.42:0/1174260019 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7fa41c051270 con 0x7fa42c06c200 2024-07-04T17:26:52.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.935+0000 7fa423e00640 1 --2- 172.21.15.42:0/1174260019 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa40803d260 0x7fa40803f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:52.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.936+0000 7fa423e00640 1 --2- 172.21.15.42:0/1174260019 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa40803d260 0x7fa40803f770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa414034000 tx=0x7fa414004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:52.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:52.940+0000 7fa423400640 1 -- 172.21.15.42:0/1174260019 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fa42c0b44b0 con 0x7fa42c06c200 2024-07-04T17:26:53.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.141+0000 7fa431200640 1 -- 172.21.15.42:0/1174260019 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa42c14b4c0 con 0x7fa42c06c200 2024-07-04T17:26:53.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.144+0000 7fa423400640 1 -- 172.21.15.42:0/1174260019 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) ==== 74+0+10603 (secure 0 0 0) 0x7fa41c011e20 con 0x7fa42c06c200 2024-07-04T17:26:53.145 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:26:53.146 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":15,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:26:17.008553+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:26:13.335715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-04T17:26:14.519899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:26:53.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.146+0000 7fa431200640 1 -- 172.21.15.42:0/1174260019 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa40803d260 msgr2=0x7fa40803f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:53.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.146+0000 7fa431200640 1 --2- 172.21.15.42:0/1174260019 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa40803d260 0x7fa40803f770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa414034000 tx=0x7fa414004730 comp rx=0 tx=0).stop 2024-07-04T17:26:53.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.146+0000 7fa431200640 1 -- 172.21.15.42:0/1174260019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa42c06c200 msgr2=0x7fa42c0b75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:53.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.146+0000 7fa431200640 1 --2- 172.21.15.42:0/1174260019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa42c06c200 0x7fa42c0b75c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa41c00c3b0 tx=0x7fa41c00c870 comp rx=0 tx=0).stop 2024-07-04T17:26:53.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.146+0000 7fa431200640 1 -- 172.21.15.42:0/1174260019 shutdown_connections 2024-07-04T17:26:53.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.146+0000 7fa431200640 1 --2- 172.21.15.42:0/1174260019 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa40803d260 0x7fa40803f770 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:53.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.146+0000 7fa431200640 1 --2- 172.21.15.42:0/1174260019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa42c06c200 0x7fa42c0b75c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:53.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.146+0000 7fa431200640 1 -- 172.21.15.42:0/1174260019 >> 172.21.15.42:0/1174260019 conn(0x7fa42c0bebc0 msgr2=0x7fa42c0b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:53.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.146+0000 7fa431200640 1 -- 172.21.15.42:0/1174260019 shutdown_connections 2024-07-04T17:26:53.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.146+0000 7fa431200640 1 -- 172.21.15.42:0/1174260019 wait complete. 2024-07-04T17:26:53.174 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-04T17:26:53.175 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:26:53.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.453+0000 7f6068c00640 1 -- 172.21.15.42:0/3724691705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60640b37b0 msgr2=0x7f60640b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:53.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.453+0000 7f6068c00640 1 --2- 172.21.15.42:0/3724691705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60640b37b0 0x7f60640b01f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f60500097b0 tx=0x7f605002a530 comp rx=0 tx=0).stop 2024-07-04T17:26:53.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.453+0000 7f6068c00640 1 -- 172.21.15.42:0/3724691705 shutdown_connections 2024-07-04T17:26:53.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.453+0000 7f6068c00640 1 --2- 172.21.15.42:0/3724691705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60640b37b0 0x7f60640b01f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:53.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.453+0000 7f6068c00640 1 -- 172.21.15.42:0/3724691705 >> 172.21.15.42:0/3724691705 conn(0x7f60640bf700 msgr2=0x7f60640af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:53.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.453+0000 7f6068c00640 1 -- 172.21.15.42:0/3724691705 shutdown_connections 2024-07-04T17:26:53.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.453+0000 7f6068c00640 1 -- 172.21.15.42:0/3724691705 wait complete. 2024-07-04T17:26:53.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.455+0000 7f6068c00640 1 Processor -- start 2024-07-04T17:26:53.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.455+0000 7f6068c00640 1 -- start start 2024-07-04T17:26:53.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.456+0000 7f6068c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60640b2e00 0x7f60640bdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:53.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.456+0000 7f6068c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f606406ce30 con 0x7f60640b2e00 2024-07-04T17:26:53.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.456+0000 7f6061e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60640b2e00 0x7f60640bdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:53.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.456+0000 7f6061e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60640b2e00 0x7f60640bdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37012/0 (socket says 172.21.15.42:37012) 2024-07-04T17:26:53.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.456+0000 7f6061e00640 1 -- 172.21.15.42:0/1618780510 learned_addr learned my addr 172.21.15.42:0/1618780510 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:53.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.457+0000 7f6061e00640 1 -- 172.21.15.42:0/1618780510 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60640ac4b0 con 0x7f60640b2e00 2024-07-04T17:26:53.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.457+0000 7f6061e00640 1 --2- 172.21.15.42:0/1618780510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60640b2e00 0x7f60640bdc20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f605400c340 tx=0x7f605400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:53.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.457+0000 7f605a000640 1 -- 172.21.15.42:0/1618780510 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6054017070 con 0x7f60640b2e00 2024-07-04T17:26:53.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.457+0000 7f605a000640 1 -- 172.21.15.42:0/1618780510 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60540035f0 con 0x7f60640b2e00 2024-07-04T17:26:53.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.457+0000 7f6068c00640 1 -- 172.21.15.42:0/1618780510 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60640af350 con 0x7f60640b2e00 2024-07-04T17:26:53.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.457+0000 7f605a000640 1 -- 172.21.15.42:0/1618780510 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6054003970 con 0x7f60640b2e00 2024-07-04T17:26:53.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.457+0000 7f6068c00640 1 -- 172.21.15.42:0/1618780510 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f606413e140 con 0x7f60640b2e00 2024-07-04T17:26:53.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.459+0000 7f6068c00640 1 -- 172.21.15.42:0/1618780510 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60640af170 con 0x7f60640b2e00 2024-07-04T17:26:53.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.459+0000 7f605a000640 1 -- 172.21.15.42:0/1618780510 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f605401a070 con 0x7f60640b2e00 2024-07-04T17:26:53.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.459+0000 7f605a000640 1 --2- 172.21.15.42:0/1618780510 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f604403d260 0x7f604403f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:53.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.459+0000 7f605a000640 1 -- 172.21.15.42:0/1618780510 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f60540507d0 con 0x7f60640b2e00 2024-07-04T17:26:53.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.460+0000 7f6061400640 1 --2- 172.21.15.42:0/1618780510 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f604403d260 0x7f604403f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:53.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.460+0000 7f6061400640 1 --2- 172.21.15.42:0/1618780510 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f604403d260 0x7f604403f770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6050034000 tx=0x7f6050004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:53.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.464+0000 7f605a000640 1 -- 172.21.15.42:0/1618780510 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f60640af170 con 0x7f60640b2e00 2024-07-04T17:26:53.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.712+0000 7f6068c00640 1 -- 172.21.15.42:0/1618780510 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f606406d480 con 0x7f60640b2e00 2024-07-04T17:26:53.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.713+0000 7f605a000640 1 -- 172.21.15.42:0/1618780510 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6035 (secure 0 0 0) 0x7f6054003da0 con 0x7f60640b2e00 2024-07-04T17:26:53.715 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:26:53.715 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":8,"btime":"2024-07-04T17:26:19:025883+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4192,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/3484212194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":3484212194},{"type":"v1","addr":"172.21.15.42:6827","nonce":3484212194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4202,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6830/815222782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6829","nonce":815222782},{"type":"v1","addr":"172.21.15.42:6830","nonce":815222782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/999331079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":999331079},{"type":"v1","addr":"172.21.15.42:6831","nonce":999331079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4204,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6833/2286778970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2286778970},{"type":"v1","addr":"172.21.15.42:6833","nonce":2286778970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:26:15.573602+0000","modified":"2024-07-04T17:26:18.625664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4205":{"gid":4205,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6835/3941225191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3941225191},{"type":"v1","addr":"172.21.15.42:6835","nonce":3941225191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4205,"qdb_cluster":[4205]},"id":1}]} 2024-07-04T17:26:53.716 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 8 2024-07-04T17:26:53.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.716+0000 7f6068c00640 1 -- 172.21.15.42:0/1618780510 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f604403d260 msgr2=0x7f604403f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:53.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.716+0000 7f6068c00640 1 --2- 172.21.15.42:0/1618780510 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f604403d260 0x7f604403f770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6050034000 tx=0x7f6050004730 comp rx=0 tx=0).stop 2024-07-04T17:26:53.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.716+0000 7f6068c00640 1 -- 172.21.15.42:0/1618780510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60640b2e00 msgr2=0x7f60640bdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:53.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.716+0000 7f6068c00640 1 --2- 172.21.15.42:0/1618780510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60640b2e00 0x7f60640bdc20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f605400c340 tx=0x7f605400c800 comp rx=0 tx=0).stop 2024-07-04T17:26:53.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.716+0000 7f6068c00640 1 -- 172.21.15.42:0/1618780510 shutdown_connections 2024-07-04T17:26:53.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.716+0000 7f6068c00640 1 --2- 172.21.15.42:0/1618780510 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f604403d260 0x7f604403f770 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:53.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.716+0000 7f6068c00640 1 --2- 172.21.15.42:0/1618780510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60640b2e00 0x7f60640bdc20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:53.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.716+0000 7f6068c00640 1 -- 172.21.15.42:0/1618780510 >> 172.21.15.42:0/1618780510 conn(0x7f60640bf700 msgr2=0x7f60640aba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:53.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.717+0000 7f6068c00640 1 -- 172.21.15.42:0/1618780510 shutdown_connections 2024-07-04T17:26:53.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:53.717+0000 7f6068c00640 1 -- 172.21.15.42:0/1618780510 wait complete. 2024-07-04T17:26:53.745 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:26:54.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.031+0000 7ffbeaa00640 1 -- 172.21.15.42:0/1487989739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbe40b9280 msgr2=0x7ffbe40c38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:54.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.031+0000 7ffbeaa00640 1 --2- 172.21.15.42:0/1487989739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbe40b9280 0x7ffbe40c38b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0005fd0 tx=0x7ffbd002a8b0 comp rx=0 tx=0).stop 2024-07-04T17:26:54.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.031+0000 7ffbeaa00640 1 -- 172.21.15.42:0/1487989739 shutdown_connections 2024-07-04T17:26:54.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.031+0000 7ffbeaa00640 1 --2- 172.21.15.42:0/1487989739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbe40b9280 0x7ffbe40c38b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:54.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.031+0000 7ffbeaa00640 1 -- 172.21.15.42:0/1487989739 >> 172.21.15.42:0/1487989739 conn(0x7ffbe406dd10 msgr2=0x7ffbe406e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:54.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.031+0000 7ffbeaa00640 1 -- 172.21.15.42:0/1487989739 shutdown_connections 2024-07-04T17:26:54.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.031+0000 7ffbeaa00640 1 -- 172.21.15.42:0/1487989739 wait complete. 2024-07-04T17:26:54.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.033+0000 7ffbeaa00640 1 Processor -- start 2024-07-04T17:26:54.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.034+0000 7ffbeaa00640 1 -- start start 2024-07-04T17:26:54.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.035+0000 7ffbeaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbe40b5ae0 0x7ffbe40be580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:54.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.035+0000 7ffbeaa00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe40b76d0 con 0x7ffbe40b5ae0 2024-07-04T17:26:54.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.035+0000 7ffbe9600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbe40b5ae0 0x7ffbe40be580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:54.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.035+0000 7ffbe9600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbe40b5ae0 0x7ffbe40be580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33634/0 (socket says 172.21.15.42:33634) 2024-07-04T17:26:54.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.035+0000 7ffbe9600640 1 -- 172.21.15.42:0/772128827 learned_addr learned my addr 172.21.15.42:0/772128827 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:54.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.036+0000 7ffbe9600640 1 -- 172.21.15.42:0/772128827 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe413dfb0 con 0x7ffbe40b5ae0 2024-07-04T17:26:54.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.036+0000 7ffbe9600640 1 --2- 172.21.15.42:0/772128827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbe40b5ae0 0x7ffbe40be580 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4001eb0 tx=0x7ffbd400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:54.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.036+0000 7ffbe1600640 1 -- 172.21.15.42:0/772128827 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffbd40116c0 con 0x7ffbe40b5ae0 2024-07-04T17:26:54.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.036+0000 7ffbe1600640 1 -- 172.21.15.42:0/772128827 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffbd4017070 con 0x7ffbe40b5ae0 2024-07-04T17:26:54.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.036+0000 7ffbe1600640 1 -- 172.21.15.42:0/772128827 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffbd4011ae0 con 0x7ffbe40b5ae0 2024-07-04T17:26:54.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.036+0000 7ffbeaa00640 1 -- 172.21.15.42:0/772128827 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbe41e4230 con 0x7ffbe40b5ae0 2024-07-04T17:26:54.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.036+0000 7ffbeaa00640 1 -- 172.21.15.42:0/772128827 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbe413ba40 con 0x7ffbe40b5ae0 2024-07-04T17:26:54.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.037+0000 7ffbeaa00640 1 -- 172.21.15.42:0/772128827 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbe40b7d30 con 0x7ffbe40b5ae0 2024-07-04T17:26:54.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.038+0000 7ffbe1600640 1 -- 172.21.15.42:0/772128827 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7ffbd4011c80 con 0x7ffbe40b5ae0 2024-07-04T17:26:54.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.038+0000 7ffbe1600640 1 --2- 172.21.15.42:0/772128827 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ffbc403d2b0 0x7ffbc403f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:54.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.038+0000 7ffbe1600640 1 -- 172.21.15.42:0/772128827 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7ffbd4051320 con 0x7ffbe40b5ae0 2024-07-04T17:26:54.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.039+0000 7ffbe8c00640 1 --2- 172.21.15.42:0/772128827 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ffbc403d2b0 0x7ffbc403f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:54.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.039+0000 7ffbe8c00640 1 --2- 172.21.15.42:0/772128827 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ffbc403d2b0 0x7ffbc403f7c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0034000 tx=0x7ffbd0004ab0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:54.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.043+0000 7ffbe1600640 1 -- 172.21.15.42:0/772128827 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7ffbd400a140 con 0x7ffbe40b5ae0 2024-07-04T17:26:54.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.287+0000 7ffbeaa00640 1 -- 172.21.15.42:0/772128827 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffbe41e3250 con 0x7ffbe40b5ae0 2024-07-04T17:26:54.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.288+0000 7ffbe1600640 1 -- 172.21.15.42:0/772128827 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6035 (secure 0 0 0) 0x7ffbd4010cf0 con 0x7ffbe40b5ae0 2024-07-04T17:26:54.290 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:26:54.290 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":8,"btime":"2024-07-04T17:26:19:025883+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4192,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/3484212194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":3484212194},{"type":"v1","addr":"172.21.15.42:6827","nonce":3484212194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4202,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6830/815222782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6829","nonce":815222782},{"type":"v1","addr":"172.21.15.42:6830","nonce":815222782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/999331079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":999331079},{"type":"v1","addr":"172.21.15.42:6831","nonce":999331079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4204,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6833/2286778970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2286778970},{"type":"v1","addr":"172.21.15.42:6833","nonce":2286778970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:26:15.573602+0000","modified":"2024-07-04T17:26:18.625664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4205":{"gid":4205,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6835/3941225191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":3941225191},{"type":"v1","addr":"172.21.15.42:6835","nonce":3941225191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4205,"qdb_cluster":[4205]},"id":1}]} 2024-07-04T17:26:54.290 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 8 2024-07-04T17:26:54.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.291+0000 7ffbeaa00640 1 -- 172.21.15.42:0/772128827 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ffbc403d2b0 msgr2=0x7ffbc403f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:54.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.291+0000 7ffbeaa00640 1 --2- 172.21.15.42:0/772128827 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ffbc403d2b0 0x7ffbc403f7c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0034000 tx=0x7ffbd0004ab0 comp rx=0 tx=0).stop 2024-07-04T17:26:54.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.291+0000 7ffbeaa00640 1 -- 172.21.15.42:0/772128827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbe40b5ae0 msgr2=0x7ffbe40be580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:54.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.291+0000 7ffbeaa00640 1 --2- 172.21.15.42:0/772128827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbe40b5ae0 0x7ffbe40be580 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4001eb0 tx=0x7ffbd400c810 comp rx=0 tx=0).stop 2024-07-04T17:26:54.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.291+0000 7ffbeaa00640 1 -- 172.21.15.42:0/772128827 shutdown_connections 2024-07-04T17:26:54.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.291+0000 7ffbeaa00640 1 --2- 172.21.15.42:0/772128827 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ffbc403d2b0 0x7ffbc403f7c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:54.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.291+0000 7ffbeaa00640 1 --2- 172.21.15.42:0/772128827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbe40b5ae0 0x7ffbe40be580 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:54.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.291+0000 7ffbeaa00640 1 -- 172.21.15.42:0/772128827 >> 172.21.15.42:0/772128827 conn(0x7ffbe406dd10 msgr2=0x7ffbe4075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:54.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.291+0000 7ffbeaa00640 1 -- 172.21.15.42:0/772128827 shutdown_connections 2024-07-04T17:26:54.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.291+0000 7ffbeaa00640 1 -- 172.21.15.42:0/772128827 wait complete. 2024-07-04T17:26:54.320 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:26:54.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.604+0000 7fa76f200640 1 -- 172.21.15.42:0/2815423480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7680b37d0 msgr2=0x7fa7680b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:54.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.604+0000 7fa76f200640 1 --2- 172.21.15.42:0/2815423480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7680b37d0 0x7fa7680b01f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa754005fd0 tx=0x7fa75402a5f0 comp rx=0 tx=0).stop 2024-07-04T17:26:54.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.604+0000 7fa76f200640 1 -- 172.21.15.42:0/2815423480 shutdown_connections 2024-07-04T17:26:54.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.604+0000 7fa76f200640 1 --2- 172.21.15.42:0/2815423480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7680b37d0 0x7fa7680b01f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:54.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.604+0000 7fa76f200640 1 -- 172.21.15.42:0/2815423480 >> 172.21.15.42:0/2815423480 conn(0x7fa7680bf700 msgr2=0x7fa7680af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:54.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.604+0000 7fa76f200640 1 -- 172.21.15.42:0/2815423480 shutdown_connections 2024-07-04T17:26:54.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.604+0000 7fa76f200640 1 -- 172.21.15.42:0/2815423480 wait complete. 2024-07-04T17:26:54.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.606+0000 7fa76f200640 1 Processor -- start 2024-07-04T17:26:54.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.607+0000 7fa76f200640 1 -- start start 2024-07-04T17:26:54.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.608+0000 7fa76f200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7680b2d20 0x7fa768140ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:54.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.608+0000 7fa76f200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa76806ce30 con 0x7fa7680b2d20 2024-07-04T17:26:54.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.608+0000 7fa767e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7680b2d20 0x7fa768140ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:54.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.608+0000 7fa767e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7680b2d20 0x7fa768140ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33640/0 (socket says 172.21.15.42:33640) 2024-07-04T17:26:54.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.608+0000 7fa767e00640 1 -- 172.21.15.42:0/878757063 learned_addr learned my addr 172.21.15.42:0/878757063 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:54.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.609+0000 7fa767e00640 1 -- 172.21.15.42:0/878757063 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7681d7d50 con 0x7fa7680b2d20 2024-07-04T17:26:54.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.609+0000 7fa767e00640 1 --2- 172.21.15.42:0/878757063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7680b2d20 0x7fa768140ee0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa758001eb0 tx=0x7fa75800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:54.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.609+0000 7fa75fe00640 1 -- 172.21.15.42:0/878757063 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa758017070 con 0x7fa7680b2d20 2024-07-04T17:26:54.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.609+0000 7fa75fe00640 1 -- 172.21.15.42:0/878757063 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7580052f0 con 0x7fa7680b2d20 2024-07-04T17:26:54.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.609+0000 7fa76f200640 1 -- 172.21.15.42:0/878757063 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7681e7000 con 0x7fa7680b2d20 2024-07-04T17:26:54.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.609+0000 7fa75fe00640 1 -- 172.21.15.42:0/878757063 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7580035f0 con 0x7fa7680b2d20 2024-07-04T17:26:54.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.609+0000 7fa76f200640 1 -- 172.21.15.42:0/878757063 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa768141420 con 0x7fa7680b2d20 2024-07-04T17:26:54.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.610+0000 7fa76f200640 1 -- 172.21.15.42:0/878757063 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa76806d490 con 0x7fa7680b2d20 2024-07-04T17:26:54.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.611+0000 7fa75fe00640 1 -- 172.21.15.42:0/878757063 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fa75801a070 con 0x7fa7680b2d20 2024-07-04T17:26:54.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.611+0000 7fa75fe00640 1 --2- 172.21.15.42:0/878757063 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa74803d2b0 0x7fa74803f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:54.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.611+0000 7fa75fe00640 1 -- 172.21.15.42:0/878757063 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7fa7580508d0 con 0x7fa7680b2d20 2024-07-04T17:26:54.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.612+0000 7fa767400640 1 --2- 172.21.15.42:0/878757063 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa74803d2b0 0x7fa74803f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:54.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.612+0000 7fa767400640 1 --2- 172.21.15.42:0/878757063 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa74803d2b0 0x7fa74803f7c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa754033000 tx=0x7fa754004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:54.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.617+0000 7fa75fe00640 1 -- 172.21.15.42:0/878757063 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fa7580102e0 con 0x7fa7680b2d20 2024-07-04T17:26:54.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.826+0000 7fa76f200640 1 -- 172.21.15.42:0/878757063 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7680af6e0 con 0x7fa7680b2d20 2024-07-04T17:26:54.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.829+0000 7fa75fe00640 1 -- 172.21.15.42:0/878757063 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) ==== 74+0+10603 (secure 0 0 0) 0x7fa758050bc0 con 0x7fa7680b2d20 2024-07-04T17:26:54.831 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:26:54.831 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":15,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:26:17.008553+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:26:13.335715+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-04T17:26:14.519899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:26:54.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.832+0000 7fa76f200640 1 -- 172.21.15.42:0/878757063 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa74803d2b0 msgr2=0x7fa74803f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:54.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.832+0000 7fa76f200640 1 --2- 172.21.15.42:0/878757063 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa74803d2b0 0x7fa74803f7c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa754033000 tx=0x7fa754004730 comp rx=0 tx=0).stop 2024-07-04T17:26:54.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.832+0000 7fa76f200640 1 -- 172.21.15.42:0/878757063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7680b2d20 msgr2=0x7fa768140ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:54.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.832+0000 7fa76f200640 1 --2- 172.21.15.42:0/878757063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7680b2d20 0x7fa768140ee0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa758001eb0 tx=0x7fa75800c810 comp rx=0 tx=0).stop 2024-07-04T17:26:54.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.832+0000 7fa76f200640 1 -- 172.21.15.42:0/878757063 shutdown_connections 2024-07-04T17:26:54.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.832+0000 7fa76f200640 1 --2- 172.21.15.42:0/878757063 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa74803d2b0 0x7fa74803f7c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:54.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.832+0000 7fa76f200640 1 --2- 172.21.15.42:0/878757063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7680b2d20 0x7fa768140ee0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:54.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.832+0000 7fa76f200640 1 -- 172.21.15.42:0/878757063 >> 172.21.15.42:0/878757063 conn(0x7fa7680bf700 msgr2=0x7fa7680bc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:54.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.832+0000 7fa76f200640 1 -- 172.21.15.42:0/878757063 shutdown_connections 2024-07-04T17:26:54.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:54.832+0000 7fa76f200640 1 -- 172.21.15.42:0/878757063 wait complete. 2024-07-04T17:26:54.863 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-07-04T17:26:55.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.156+0000 7f560fe00640 1 -- 172.21.15.42:0/3979874993 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5610140440 msgr2=0x7f561006c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:55.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.156+0000 7f560fe00640 1 --2- 172.21.15.42:0/3979874993 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5610140440 0x7f561006c8b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00a8c0 tx=0x7f55fc02b8f0 comp rx=0 tx=0).stop 2024-07-04T17:26:55.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.156+0000 7f560fe00640 1 -- 172.21.15.42:0/3979874993 shutdown_connections 2024-07-04T17:26:55.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.156+0000 7f560fe00640 1 --2- 172.21.15.42:0/3979874993 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5610140440 0x7f561006c8b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:55.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.156+0000 7f560fe00640 1 -- 172.21.15.42:0/3979874993 >> 172.21.15.42:0/3979874993 conn(0x7f56100bebc0 msgr2=0x7f56100befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:55.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.156+0000 7f560fe00640 1 -- 172.21.15.42:0/3979874993 shutdown_connections 2024-07-04T17:26:55.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.156+0000 7f560fe00640 1 -- 172.21.15.42:0/3979874993 wait complete. 2024-07-04T17:26:55.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.158+0000 7f560fe00640 1 Processor -- start 2024-07-04T17:26:55.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.158+0000 7f560fe00640 1 -- start start 2024-07-04T17:26:55.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.160+0000 7f560fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56100b20c0 0x7f56101d4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:55.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.160+0000 7f560fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f56100b5480 con 0x7f56100b20c0 2024-07-04T17:26:55.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.160+0000 7f560ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56100b20c0 0x7f56101d4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:55.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.160+0000 7f560ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56100b20c0 0x7f56101d4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33668/0 (socket says 172.21.15.42:33668) 2024-07-04T17:26:55.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.160+0000 7f560ea00640 1 -- 172.21.15.42:0/2919073065 learned_addr learned my addr 172.21.15.42:0/2919073065 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:55.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.160+0000 7f560ea00640 1 -- 172.21.15.42:0/2919073065 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f561013d2c0 con 0x7f56100b20c0 2024-07-04T17:26:55.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.161+0000 7f560ea00640 1 --2- 172.21.15.42:0/2919073065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56100b20c0 0x7f56101d4ac0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f560000c2f0 tx=0x7f560000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:55.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.161+0000 7f5606a00640 1 -- 172.21.15.42:0/2919073065 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5600017070 con 0x7f56100b20c0 2024-07-04T17:26:55.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.161+0000 7f5606a00640 1 -- 172.21.15.42:0/2919073065 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5600005410 con 0x7f56100b20c0 2024-07-04T17:26:55.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.161+0000 7f560fe00640 1 -- 172.21.15.42:0/2919073065 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56101e21a0 con 0x7f56100b20c0 2024-07-04T17:26:55.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.161+0000 7f5606a00640 1 -- 172.21.15.42:0/2919073065 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5600003780 con 0x7f56100b20c0 2024-07-04T17:26:55.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.161+0000 7f560fe00640 1 -- 172.21.15.42:0/2919073065 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56101e2670 con 0x7f56100b20c0 2024-07-04T17:26:55.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.162+0000 7f560fe00640 1 -- 172.21.15.42:0/2919073065 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56101d7c60 con 0x7f56100b20c0 2024-07-04T17:26:55.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.162+0000 7f5606a00640 1 -- 172.21.15.42:0/2919073065 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f560001a070 con 0x7f56100b20c0 2024-07-04T17:26:55.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.163+0000 7f5606a00640 1 --2- 172.21.15.42:0/2919073065 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f55f003d2b0 0x7f55f003f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:55.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.163+0000 7f5606a00640 1 -- 172.21.15.42:0/2919073065 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3439+0+0 (secure 0 0 0) 0x7f5600050840 con 0x7f56100b20c0 2024-07-04T17:26:55.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.163+0000 7f560e000640 1 --2- 172.21.15.42:0/2919073065 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f55f003d2b0 0x7f55f003f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:55.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.164+0000 7f560e000640 1 --2- 172.21.15.42:0/2919073065 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f55f003d2b0 0x7f55f003f7c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f55fc036000 tx=0x7f55fc004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:55.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.168+0000 7f5606a00640 1 -- 172.21.15.42:0/2919073065 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f56101d7c60 con 0x7f56100b20c0 2024-07-04T17:26:55.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.424+0000 7f560fe00640 1 -- 172.21.15.42:0/2919073065 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f56100ba440 con 0x7f56100b20c0 2024-07-04T17:26:55.427 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-07-04T17:26:55.426+0000 7f8616e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-04T17:26:55.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.427+0000 7f5606a00640 1 -- 172.21.15.42:0/2919073065 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f560001ad80 con 0x7f56100b20c0 2024-07-04T17:26:55.429 INFO:teuthology.orchestra.run.smithi042.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-04T17:26:55.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.431+0000 7f560fe00640 1 -- 172.21.15.42:0/2919073065 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f55f003d2b0 msgr2=0x7f55f003f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:55.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.431+0000 7f560fe00640 1 --2- 172.21.15.42:0/2919073065 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f55f003d2b0 0x7f55f003f7c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f55fc036000 tx=0x7f55fc004730 comp rx=0 tx=0).stop 2024-07-04T17:26:55.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.431+0000 7f560fe00640 1 -- 172.21.15.42:0/2919073065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56100b20c0 msgr2=0x7f56101d4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:55.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.431+0000 7f560fe00640 1 --2- 172.21.15.42:0/2919073065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56100b20c0 0x7f56101d4ac0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f560000c2f0 tx=0x7f560000c7b0 comp rx=0 tx=0).stop 2024-07-04T17:26:55.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.431+0000 7f560fe00640 1 -- 172.21.15.42:0/2919073065 shutdown_connections 2024-07-04T17:26:55.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.431+0000 7f560fe00640 1 --2- 172.21.15.42:0/2919073065 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f55f003d2b0 0x7f55f003f7c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:55.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.431+0000 7f560fe00640 1 --2- 172.21.15.42:0/2919073065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56100b20c0 0x7f56101d4ac0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:55.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.431+0000 7f560fe00640 1 -- 172.21.15.42:0/2919073065 >> 172.21.15.42:0/2919073065 conn(0x7f56100bebc0 msgr2=0x7f56100b95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:55.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.433+0000 7f560fe00640 1 -- 172.21.15.42:0/2919073065 shutdown_connections 2024-07-04T17:26:55.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.433+0000 7f560fe00640 1 -- 172.21.15.42:0/2919073065 wait complete. 2024-07-04T17:26:55.441 INFO:tasks.ceph.mds.c.smithi042.stderr: -1834> 2024-07-04T17:26:20.376+0000 7f6272a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-04T17:26:55.468 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-07-04T17:26:55.588 INFO:tasks.ceph.mds.c.smithi042.stdout:starting mds.c at 2024-07-04T17:26:55.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.740+0000 7f7110a00640 1 -- 172.21.15.42:0/1577468083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f710c0b25c0 msgr2=0x7f710c0afed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:55.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.740+0000 7f7110a00640 1 --2- 172.21.15.42:0/1577468083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f710c0b25c0 0x7f710c0afed0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f70f40097b0 tx=0x7f70f402a530 comp rx=0 tx=0).stop 2024-07-04T17:26:55.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.740+0000 7f7110a00640 1 -- 172.21.15.42:0/1577468083 shutdown_connections 2024-07-04T17:26:55.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.740+0000 7f7110a00640 1 --2- 172.21.15.42:0/1577468083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f710c0b25c0 0x7f710c0afed0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:55.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.740+0000 7f7110a00640 1 -- 172.21.15.42:0/1577468083 >> 172.21.15.42:0/1577468083 conn(0x7f710c06fe40 msgr2=0x7f710c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:55.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.740+0000 7f7110a00640 1 -- 172.21.15.42:0/1577468083 shutdown_connections 2024-07-04T17:26:55.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.740+0000 7f7110a00640 1 -- 172.21.15.42:0/1577468083 wait complete. 2024-07-04T17:26:55.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.742+0000 7f7110a00640 1 Processor -- start 2024-07-04T17:26:55.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.742+0000 7f7110a00640 1 -- start start 2024-07-04T17:26:55.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.744+0000 7f7110a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f710c0af8a0 0x7f710c073300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:55.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.744+0000 7f7110a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f710c1417c0 con 0x7f710c0af8a0 2024-07-04T17:26:55.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.744+0000 7f7109e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f710c0af8a0 0x7f710c073300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:55.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.744+0000 7f7109e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f710c0af8a0 0x7f710c073300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33710/0 (socket says 172.21.15.42:33710) 2024-07-04T17:26:55.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.744+0000 7f7109e00640 1 -- 172.21.15.42:0/707745779 learned_addr learned my addr 172.21.15.42:0/707745779 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:55.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.744+0000 7f7109e00640 1 -- 172.21.15.42:0/707745779 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f710c073b60 con 0x7f710c0af8a0 2024-07-04T17:26:55.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.745+0000 7f7109e00640 1 --2- 172.21.15.42:0/707745779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f710c0af8a0 0x7f710c073300 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00ca70 tx=0x7f70fc00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:55.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.745+0000 7f7102a00640 1 -- 172.21.15.42:0/707745779 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70fc016020 con 0x7f710c0af8a0 2024-07-04T17:26:55.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.745+0000 7f7102a00640 1 -- 172.21.15.42:0/707745779 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70fc00ab70 con 0x7f710c0af8a0 2024-07-04T17:26:55.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.745+0000 7f7102a00640 1 -- 172.21.15.42:0/707745779 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70fc012420 con 0x7f710c0af8a0 2024-07-04T17:26:55.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.745+0000 7f7110a00640 1 -- 172.21.15.42:0/707745779 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f710c1e3fd0 con 0x7f710c0af8a0 2024-07-04T17:26:55.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.745+0000 7f7110a00640 1 -- 172.21.15.42:0/707745779 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f710c074020 con 0x7f710c0af8a0 2024-07-04T17:26:55.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.746+0000 7f7110a00640 1 -- 172.21.15.42:0/707745779 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f710c141e70 con 0x7f710c0af8a0 2024-07-04T17:26:55.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.746+0000 7f7102a00640 1 -- 172.21.15.42:0/707745779 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f70fc019020 con 0x7f710c0af8a0 2024-07-04T17:26:55.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.747+0000 7f7102a00640 1 --2- 172.21.15.42:0/707745779 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f70e803d2b0 0x7f70e803f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:55.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.747+0000 7f7102a00640 1 -- 172.21.15.42:0/707745779 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 3525+0+0 (secure 0 0 0) 0x7f70fc0518e0 con 0x7f710c0af8a0 2024-07-04T17:26:55.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.747+0000 7f7101400640 1 --2- 172.21.15.42:0/707745779 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f70e803d2b0 0x7f70e803f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:55.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.748+0000 7f7101400640 1 --2- 172.21.15.42:0/707745779 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f70e803d2b0 0x7f70e803f7c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f70f4034000 tx=0x7f70f4004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:55.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:55.752+0000 7f7102a00640 1 -- 172.21.15.42:0/707745779 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f70fc020030 con 0x7f710c0af8a0 2024-07-04T17:26:56.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.005+0000 7f7110a00640 1 -- 172.21.15.42:0/707745779 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f710c1453d0 con 0x7f710c0af8a0 2024-07-04T17:26:56.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.504+0000 7f7102a00640 1 -- 172.21.15.42:0/707745779 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f710c0b29d0 con 0x7f710c0af8a0 2024-07-04T17:26:56.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.508+0000 7f7110a00640 1 -- 172.21.15.42:0/707745779 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f70e803d2b0 msgr2=0x7f70e803f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:56.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.508+0000 7f7110a00640 1 --2- 172.21.15.42:0/707745779 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f70e803d2b0 0x7f70e803f7c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f70f4034000 tx=0x7f70f4004730 comp rx=0 tx=0).stop 2024-07-04T17:26:56.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.508+0000 7f7110a00640 1 -- 172.21.15.42:0/707745779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f710c0af8a0 msgr2=0x7f710c073300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:56.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.508+0000 7f7110a00640 1 --2- 172.21.15.42:0/707745779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f710c0af8a0 0x7f710c073300 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00ca70 tx=0x7f70fc00cf30 comp rx=0 tx=0).stop 2024-07-04T17:26:56.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.508+0000 7f7110a00640 1 -- 172.21.15.42:0/707745779 shutdown_connections 2024-07-04T17:26:56.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.508+0000 7f7110a00640 1 --2- 172.21.15.42:0/707745779 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f70e803d2b0 0x7f70e803f7c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:56.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.508+0000 7f7110a00640 1 --2- 172.21.15.42:0/707745779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f710c0af8a0 0x7f710c073300 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:56.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.508+0000 7f7110a00640 1 -- 172.21.15.42:0/707745779 >> 172.21.15.42:0/707745779 conn(0x7f710c06fe40 msgr2=0x7f710c145bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:56.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.508+0000 7f7110a00640 1 -- 172.21.15.42:0/707745779 shutdown_connections 2024-07-04T17:26:56.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.509+0000 7f7110a00640 1 -- 172.21.15.42:0/707745779 wait complete. 2024-07-04T17:26:56.538 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-04T17:26:56.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.807+0000 7f87f1e00640 1 -- 172.21.15.42:0/2810744959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87ec148b70 msgr2=0x7f87ec0701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:56.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.807+0000 7f87f1e00640 1 --2- 172.21.15.42:0/2810744959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87ec148b70 0x7f87ec0701d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f87e00097b0 tx=0x7f87e002a530 comp rx=0 tx=0).stop 2024-07-04T17:26:56.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.807+0000 7f87f1e00640 1 -- 172.21.15.42:0/2810744959 shutdown_connections 2024-07-04T17:26:56.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.807+0000 7f87f1e00640 1 --2- 172.21.15.42:0/2810744959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87ec148b70 0x7f87ec0701d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:56.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.807+0000 7f87f1e00640 1 -- 172.21.15.42:0/2810744959 >> 172.21.15.42:0/2810744959 conn(0x7f87ec0afaa0 msgr2=0x7f87ec0afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:56.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.808+0000 7f87f1e00640 1 -- 172.21.15.42:0/2810744959 shutdown_connections 2024-07-04T17:26:56.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.808+0000 7f87f1e00640 1 -- 172.21.15.42:0/2810744959 wait complete. 2024-07-04T17:26:56.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.810+0000 7f87f1e00640 1 Processor -- start 2024-07-04T17:26:56.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.810+0000 7f87f1e00640 1 -- start start 2024-07-04T17:26:56.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.812+0000 7f87f1e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87ec073d50 0x7f87ec141ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:56.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.812+0000 7f87f1e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec0aac90 con 0x7f87ec073d50 2024-07-04T17:26:56.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.812+0000 7f87f0a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87ec073d50 0x7f87ec141ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:56.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.812+0000 7f87f0a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87ec073d50 0x7f87ec141ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33732/0 (socket says 172.21.15.42:33732) 2024-07-04T17:26:56.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.812+0000 7f87f0a00640 1 -- 172.21.15.42:0/3859473161 learned_addr learned my addr 172.21.15.42:0/3859473161 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:56.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.812+0000 7f87f0a00640 1 -- 172.21.15.42:0/3859473161 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87ec072940 con 0x7f87ec073d50 2024-07-04T17:26:56.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.813+0000 7f87f0a00640 1 --2- 172.21.15.42:0/3859473161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87ec073d50 0x7f87ec141ef0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f87e00098d0 tx=0x7f87e0033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:56.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.813+0000 7f87e8c00640 1 -- 172.21.15.42:0/3859473161 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87e0031880 con 0x7f87ec073d50 2024-07-04T17:26:56.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.813+0000 7f87e8c00640 1 -- 172.21.15.42:0/3859473161 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87e0031b10 con 0x7f87ec073d50 2024-07-04T17:26:56.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.813+0000 7f87e8c00640 1 -- 172.21.15.42:0/3859473161 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87e0031df0 con 0x7f87ec073d50 2024-07-04T17:26:56.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.813+0000 7f87f1e00640 1 -- 172.21.15.42:0/3859473161 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87ec071850 con 0x7f87ec073d50 2024-07-04T17:26:56.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.813+0000 7f87f1e00640 1 -- 172.21.15.42:0/3859473161 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87ec142a60 con 0x7f87ec073d50 2024-07-04T17:26:56.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.814+0000 7f87f1e00640 1 -- 172.21.15.42:0/3859473161 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87ec071a20 con 0x7f87ec073d50 2024-07-04T17:26:56.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.815+0000 7f87e8c00640 1 -- 172.21.15.42:0/3859473161 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f87e00330d0 con 0x7f87ec073d50 2024-07-04T17:26:56.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.815+0000 7f87e8c00640 1 --2- 172.21.15.42:0/3859473161 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f87d003d2b0 0x7f87d003f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:56.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.815+0000 7f87e8c00640 1 -- 172.21.15.42:0/3859473161 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 3525+0+0 (secure 0 0 0) 0x7f87e00728d0 con 0x7f87ec073d50 2024-07-04T17:26:56.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.815+0000 7f87ebe00640 1 --2- 172.21.15.42:0/3859473161 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f87d003d2b0 0x7f87d003f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:56.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.816+0000 7f87ebe00640 1 --2- 172.21.15.42:0/3859473161 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f87d003d2b0 0x7f87d003f7c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f87dc007fd0 tx=0x7f87dc007920 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:56.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:56.820+0000 7f87e8c00640 1 -- 172.21.15.42:0/3859473161 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f87e003f030 con 0x7f87ec073d50 2024-07-04T17:26:57.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.022+0000 7f87f1e00640 1 -- 172.21.15.42:0/3859473161 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f87ec0701d0 con 0x7f87ec073d50 2024-07-04T17:26:57.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.519+0000 7f87e8c00640 1 -- 172.21.15.42:0/3859473161 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v17) ==== 181+0+0 (secure 0 0 0) 0x7f87e003a3a0 con 0x7f87ec073d50 2024-07-04T17:26:57.521 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' removed 2024-07-04T17:26:57.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.523+0000 7f87f1e00640 1 -- 172.21.15.42:0/3859473161 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f87d003d2b0 msgr2=0x7f87d003f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:57.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.523+0000 7f87f1e00640 1 --2- 172.21.15.42:0/3859473161 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f87d003d2b0 0x7f87d003f7c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f87dc007fd0 tx=0x7f87dc007920 comp rx=0 tx=0).stop 2024-07-04T17:26:57.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.523+0000 7f87f1e00640 1 -- 172.21.15.42:0/3859473161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87ec073d50 msgr2=0x7f87ec141ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:57.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.523+0000 7f87f1e00640 1 --2- 172.21.15.42:0/3859473161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87ec073d50 0x7f87ec141ef0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f87e00098d0 tx=0x7f87e0033ad0 comp rx=0 tx=0).stop 2024-07-04T17:26:57.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.523+0000 7f87f1e00640 1 -- 172.21.15.42:0/3859473161 shutdown_connections 2024-07-04T17:26:57.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.523+0000 7f87f1e00640 1 --2- 172.21.15.42:0/3859473161 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f87d003d2b0 0x7f87d003f7c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:57.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.523+0000 7f87f1e00640 1 --2- 172.21.15.42:0/3859473161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87ec073d50 0x7f87ec141ef0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:57.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.523+0000 7f87f1e00640 1 -- 172.21.15.42:0/3859473161 >> 172.21.15.42:0/3859473161 conn(0x7f87ec0afaa0 msgr2=0x7f87ec144540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:57.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.523+0000 7f87f1e00640 1 -- 172.21.15.42:0/3859473161 shutdown_connections 2024-07-04T17:26:57.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.523+0000 7f87f1e00640 1 -- 172.21.15.42:0/3859473161 wait complete. 2024-07-04T17:26:57.570 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-04T17:26:57.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.819+0000 7ff05f200640 1 -- 172.21.15.42:0/51413310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0600701d0 msgr2=0x7ff0600abe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:57.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.819+0000 7ff05f200640 1 --2- 172.21.15.42:0/51413310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0600701d0 0x7ff0600abe40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff050005fd0 tx=0x7ff05002a5f0 comp rx=0 tx=0).stop 2024-07-04T17:26:57.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.819+0000 7ff05f200640 1 -- 172.21.15.42:0/51413310 shutdown_connections 2024-07-04T17:26:57.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.819+0000 7ff05f200640 1 --2- 172.21.15.42:0/51413310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0600701d0 0x7ff0600abe40 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:57.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.819+0000 7ff05f200640 1 -- 172.21.15.42:0/51413310 >> 172.21.15.42:0/51413310 conn(0x7ff0600afaa0 msgr2=0x7ff0600afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:57.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.819+0000 7ff05f200640 1 -- 172.21.15.42:0/51413310 shutdown_connections 2024-07-04T17:26:57.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.819+0000 7ff05f200640 1 -- 172.21.15.42:0/51413310 wait complete. 2024-07-04T17:26:57.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.821+0000 7ff05f200640 1 Processor -- start 2024-07-04T17:26:57.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.821+0000 7ff05f200640 1 -- start start 2024-07-04T17:26:57.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.823+0000 7ff05f200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0600701d0 0x7ff06014c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:57.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.823+0000 7ff05f200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff0600ab190 con 0x7ff0600701d0 2024-07-04T17:26:57.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.823+0000 7ff05de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0600701d0 0x7ff06014c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:57.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.823+0000 7ff05de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0600701d0 0x7ff06014c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33750/0 (socket says 172.21.15.42:33750) 2024-07-04T17:26:57.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.823+0000 7ff05de00640 1 -- 172.21.15.42:0/1910479866 learned_addr learned my addr 172.21.15.42:0/1910479866 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:57.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.823+0000 7ff05de00640 1 -- 172.21.15.42:0/1910479866 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06014ce60 con 0x7ff0600701d0 2024-07-04T17:26:57.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.824+0000 7ff05de00640 1 --2- 172.21.15.42:0/1910479866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0600701d0 0x7ff06014c870 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff050009850 tx=0x7ff0500049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:57.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.824+0000 7ff056000640 1 -- 172.21.15.42:0/1910479866 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff05003e030 con 0x7ff0600701d0 2024-07-04T17:26:57.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.824+0000 7ff056000640 1 -- 172.21.15.42:0/1910479866 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0500035f0 con 0x7ff0600701d0 2024-07-04T17:26:57.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.824+0000 7ff056000640 1 -- 172.21.15.42:0/1910479866 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff050035040 con 0x7ff0600701d0 2024-07-04T17:26:57.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.824+0000 7ff05f200640 1 -- 172.21.15.42:0/1910479866 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff060144a20 con 0x7ff0600701d0 2024-07-04T17:26:57.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.824+0000 7ff05f200640 1 -- 172.21.15.42:0/1910479866 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff06014f3c0 con 0x7ff0600701d0 2024-07-04T17:26:57.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.825+0000 7ff056000640 1 -- 172.21.15.42:0/1910479866 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7ff050041030 con 0x7ff0600701d0 2024-07-04T17:26:57.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.825+0000 7ff05f200640 1 -- 172.21.15.42:0/1910479866 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0600abe40 con 0x7ff0600701d0 2024-07-04T17:26:57.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.826+0000 7ff056000640 1 --2- 172.21.15.42:0/1910479866 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff03c03d300 0x7ff03c03f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:57.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.826+0000 7ff056000640 1 -- 172.21.15.42:0/1910479866 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 3145+0+0 (secure 0 0 0) 0x7ff05003f070 con 0x7ff0600701d0 2024-07-04T17:26:57.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.826+0000 7ff057e00640 1 --2- 172.21.15.42:0/1910479866 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff03c03d300 0x7ff03c03f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:57.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.827+0000 7ff057e00640 1 --2- 172.21.15.42:0/1910479866 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff03c03d300 0x7ff03c03f810 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff048001560 tx=0x7ff0480079f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:57.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:57.830+0000 7ff056000640 1 -- 172.21.15.42:0/1910479866 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7ff050070dc0 con 0x7ff0600701d0 2024-07-04T17:26:58.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:58.039+0000 7ff05f200640 1 -- 172.21.15.42:0/1910479866 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff060147810 con 0x7ff0600701d0 2024-07-04T17:26:58.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:58.528+0000 7ff056000640 1 -- 172.21.15.42:0/1910479866 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v18) ==== 169+0+0 (secure 0 0 0) 0x7ff05004bd60 con 0x7ff0600701d0 2024-07-04T17:26:58.530 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' removed 2024-07-04T17:26:58.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:58.532+0000 7ff05f200640 1 -- 172.21.15.42:0/1910479866 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff03c03d300 msgr2=0x7ff03c03f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:58.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:58.532+0000 7ff05f200640 1 --2- 172.21.15.42:0/1910479866 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff03c03d300 0x7ff03c03f810 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff048001560 tx=0x7ff0480079f0 comp rx=0 tx=0).stop 2024-07-04T17:26:58.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:58.532+0000 7ff05f200640 1 -- 172.21.15.42:0/1910479866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0600701d0 msgr2=0x7ff06014c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:58.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:58.532+0000 7ff05f200640 1 --2- 172.21.15.42:0/1910479866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0600701d0 0x7ff06014c870 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff050009850 tx=0x7ff0500049d0 comp rx=0 tx=0).stop 2024-07-04T17:26:58.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:58.532+0000 7ff05f200640 1 -- 172.21.15.42:0/1910479866 shutdown_connections 2024-07-04T17:26:58.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:58.532+0000 7ff05f200640 1 --2- 172.21.15.42:0/1910479866 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff03c03d300 0x7ff03c03f810 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:58.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:58.532+0000 7ff05f200640 1 --2- 172.21.15.42:0/1910479866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0600701d0 0x7ff06014c870 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:58.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:58.532+0000 7ff05f200640 1 -- 172.21.15.42:0/1910479866 >> 172.21.15.42:0/1910479866 conn(0x7ff0600afaa0 msgr2=0x7ff0600b36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:58.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:58.533+0000 7ff05f200640 1 -- 172.21.15.42:0/1910479866 shutdown_connections 2024-07-04T17:26:58.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:58.533+0000 7ff05f200640 1 -- 172.21.15.42:0/1910479866 wait complete. 2024-07-04T17:26:58.576 INFO:tasks.ceph.mds.a:Restarting daemon 2024-07-04T17:26:58.576 INFO:tasks.ceph.mds.a:Stopping old one... 2024-07-04T17:26:58.577 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-07-04T17:26:58.577 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:26:58.577 INFO:tasks.ceph.mds.b:Restarting daemon 2024-07-04T17:26:58.577 INFO:tasks.ceph.mds.b:Stopping old one... 2024-07-04T17:26:58.577 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-07-04T17:26:58.577 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:26:58.578 INFO:tasks.ceph.mds.c:Restarting daemon 2024-07-04T17:26:58.578 INFO:tasks.ceph.mds.c:Stopping old one... 2024-07-04T17:26:58.578 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-07-04T17:26:58.578 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:26:58.578 INFO:tasks.ceph.mds.d:Restarting daemon 2024-07-04T17:26:58.578 INFO:tasks.ceph.mds.d:Stopping old one... 2024-07-04T17:26:58.579 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-07-04T17:26:58.579 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:26:58.579 INFO:tasks.ceph.mds.e:Restarting daemon 2024-07-04T17:26:58.579 INFO:tasks.ceph.mds.e:Stopping old one... 2024-07-04T17:26:58.579 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-07-04T17:26:58.579 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:26:58.603 INFO:tasks.ceph.mds.b:Stopped 2024-07-04T17:26:58.603 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-07-04T17:26:58.615 INFO:tasks.ceph.mds.a:Stopped 2024-07-04T17:26:58.616 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-07-04T17:26:58.617 INFO:tasks.ceph.mds.e:Stopped 2024-07-04T17:26:58.617 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-07-04T17:26:58.617 INFO:tasks.ceph.mds.d:Stopped 2024-07-04T17:26:58.618 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-07-04T17:26:58.620 INFO:tasks.ceph.mds.b:Started 2024-07-04T17:26:58.621 DEBUG:teuthology.parallel:result is None 2024-07-04T17:26:58.623 INFO:tasks.ceph.mds.a:Started 2024-07-04T17:26:58.624 DEBUG:teuthology.parallel:result is None 2024-07-04T17:26:58.624 INFO:tasks.ceph.mds.c:Stopped 2024-07-04T17:26:58.624 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-07-04T17:26:58.628 INFO:tasks.ceph.mds.e:Started 2024-07-04T17:26:58.629 INFO:tasks.ceph.mds.d:Started 2024-07-04T17:26:58.629 DEBUG:teuthology.parallel:result is None 2024-07-04T17:26:58.629 DEBUG:teuthology.parallel:result is None 2024-07-04T17:26:58.633 INFO:tasks.ceph.mds.c:Started 2024-07-04T17:26:58.633 DEBUG:teuthology.parallel:result is None 2024-07-04T17:26:58.634 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-07-04T17:26:58.895 INFO:tasks.ceph.mds.e.smithi042.stdout:starting mds.e at 2024-07-04T17:26:58.903 INFO:tasks.ceph.mds.d.smithi042.stdout:starting mds.d at 2024-07-04T17:26:58.938 INFO:tasks.ceph.mds.c.smithi042.stdout:starting mds.c at 2024-07-04T17:26:58.947 INFO:tasks.ceph.mds.a.smithi042.stdout:starting mds.a at 2024-07-04T17:26:58.949 INFO:tasks.ceph.mds.b.smithi042.stdout:starting mds.b at 2024-07-04T17:26:58.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:58.997+0000 7f08bba00640 1 -- 172.21.15.42:0/2594473341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08b406cea0 msgr2=0x7f08b40b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:58.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:58.997+0000 7f08bba00640 1 --2- 172.21.15.42:0/2594473341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08b406cea0 0x7f08b40b3030 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f089c00a8c0 tx=0x7f089c02b860 comp rx=0 tx=0).stop 2024-07-04T17:26:58.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:58.997+0000 7f08bba00640 1 -- 172.21.15.42:0/2594473341 shutdown_connections 2024-07-04T17:26:58.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:58.997+0000 7f08bba00640 1 --2- 172.21.15.42:0/2594473341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08b406cea0 0x7f08b40b3030 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:58.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:58.997+0000 7f08bba00640 1 -- 172.21.15.42:0/2594473341 >> 172.21.15.42:0/2594473341 conn(0x7f08b40bf700 msgr2=0x7f08b40af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:58.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:58.997+0000 7f08bba00640 1 -- 172.21.15.42:0/2594473341 shutdown_connections 2024-07-04T17:26:59.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:58.997+0000 7f08bba00640 1 -- 172.21.15.42:0/2594473341 wait complete. 2024-07-04T17:26:59.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:58.999+0000 7f08bba00640 1 Processor -- start 2024-07-04T17:26:59.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.000+0000 7f08bba00640 1 -- start start 2024-07-04T17:26:59.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.001+0000 7f08bba00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08b406cea0 0x7f08b40b5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:59.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.001+0000 7f08bba00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f08b40b0370 con 0x7f08b406cea0 2024-07-04T17:26:59.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.001+0000 7f08b9000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08b406cea0 0x7f08b40b5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:59.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.001+0000 7f08b9000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08b406cea0 0x7f08b40b5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33878/0 (socket says 172.21.15.42:33878) 2024-07-04T17:26:59.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.001+0000 7f08b9000640 1 -- 172.21.15.42:0/3046526568 learned_addr learned my addr 172.21.15.42:0/3046526568 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:59.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.001+0000 7f08b9000640 1 -- 172.21.15.42:0/3046526568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b40b5d20 con 0x7f08b406cea0 2024-07-04T17:26:59.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.002+0000 7f08b9000640 1 --2- 172.21.15.42:0/3046526568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08b406cea0 0x7f08b40b5630 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f08a400c380 tx=0x7f08a400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:59.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.002+0000 7f08b1600640 1 -- 172.21.15.42:0/3046526568 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08a40116c0 con 0x7f08b406cea0 2024-07-04T17:26:59.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.002+0000 7f08b1600640 1 -- 172.21.15.42:0/3046526568 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08a4017070 con 0x7f08b406cea0 2024-07-04T17:26:59.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.002+0000 7f08b1600640 1 -- 172.21.15.42:0/3046526568 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08a4011b20 con 0x7f08b406cea0 2024-07-04T17:26:59.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.002+0000 7f08bba00640 1 -- 172.21.15.42:0/3046526568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b4073410 con 0x7f08b406cea0 2024-07-04T17:26:59.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.002+0000 7f08bba00640 1 -- 172.21.15.42:0/3046526568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b41dcb40 con 0x7f08b406cea0 2024-07-04T17:26:59.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.003+0000 7f08bba00640 1 -- 172.21.15.42:0/3046526568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08b40b3030 con 0x7f08b406cea0 2024-07-04T17:26:59.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.003+0000 7f08b1600640 1 -- 172.21.15.42:0/3046526568 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f08a4011cc0 con 0x7f08b406cea0 2024-07-04T17:26:59.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.004+0000 7f08b1600640 1 --2- 172.21.15.42:0/3046526568 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f089003d2b0 0x7f089003f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:59.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.004+0000 7f08b1600640 1 -- 172.21.15.42:0/3046526568 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2673+0+0 (secure 0 0 0) 0x7f08a4051080 con 0x7f08b406cea0 2024-07-04T17:26:59.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.004+0000 7f08abe00640 1 --2- 172.21.15.42:0/3046526568 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f089003d2b0 0x7f089003f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:59.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.005+0000 7f08abe00640 1 --2- 172.21.15.42:0/3046526568 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f089003d2b0 0x7f089003f7c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f089c036000 tx=0x7f089c004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:59.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.009+0000 7f08b1600640 1 -- 172.21.15.42:0/3046526568 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f08a400a140 con 0x7f08b406cea0 2024-07-04T17:26:59.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.229+0000 7f08bba00640 1 -- 172.21.15.42:0/3046526568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f08b4069b70 con 0x7f08b406cea0 2024-07-04T17:26:59.544 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-04T17:26:59.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.546+0000 7f08b1600640 1 -- 172.21.15.42:0/3046526568 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v20) ==== 97+0+0 (secure 0 0 0) 0x7f08a4010cf0 con 0x7f08b406cea0 2024-07-04T17:26:59.553 INFO:teuthology.orchestra.run.smithi042.stderr: removed all blocklist entries 2024-07-04T17:26:59.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.550+0000 7f08bba00640 1 -- 172.21.15.42:0/3046526568 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f089003d2b0 msgr2=0x7f089003f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:59.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.550+0000 7f08bba00640 1 --2- 172.21.15.42:0/3046526568 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f089003d2b0 0x7f089003f7c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f089c036000 tx=0x7f089c004730 comp rx=0 tx=0).stop 2024-07-04T17:26:59.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.550+0000 7f08bba00640 1 -- 172.21.15.42:0/3046526568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08b406cea0 msgr2=0x7f08b40b5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:59.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.550+0000 7f08bba00640 1 --2- 172.21.15.42:0/3046526568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08b406cea0 0x7f08b40b5630 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f08a400c380 tx=0x7f08a400c840 comp rx=0 tx=0).stop 2024-07-04T17:26:59.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.551+0000 7f08bba00640 1 -- 172.21.15.42:0/3046526568 shutdown_connections 2024-07-04T17:26:59.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.551+0000 7f08bba00640 1 --2- 172.21.15.42:0/3046526568 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f089003d2b0 0x7f089003f7c0 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f089c036000 tx=0x7f089c004730 comp rx=0 tx=0).stop 2024-07-04T17:26:59.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.551+0000 7f08bba00640 1 --2- 172.21.15.42:0/3046526568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08b406cea0 0x7f08b40b5630 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:59.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.551+0000 7f08bba00640 1 -- 172.21.15.42:0/3046526568 >> 172.21.15.42:0/3046526568 conn(0x7f08b40bf700 msgr2=0x7f08b414d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:59.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.551+0000 7f08bba00640 1 -- 172.21.15.42:0/3046526568 shutdown_connections 2024-07-04T17:26:59.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.551+0000 7f08bba00640 1 -- 172.21.15.42:0/3046526568 wait complete. 2024-07-04T17:26:59.579 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-07-04T17:26:59.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.885+0000 7f4967e00640 1 -- 172.21.15.42:0/3726730999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f496806c200 msgr2=0x7f496814ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:26:59.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.885+0000 7f4967e00640 1 --2- 172.21.15.42:0/3726730999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f496806c200 0x7f496814ef10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f49580097b0 tx=0x7f495802a4a0 comp rx=0 tx=0).stop 2024-07-04T17:26:59.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.885+0000 7f4967e00640 1 -- 172.21.15.42:0/3726730999 shutdown_connections 2024-07-04T17:26:59.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.885+0000 7f4967e00640 1 --2- 172.21.15.42:0/3726730999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f496806c200 0x7f496814ef10 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:26:59.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.885+0000 7f4967e00640 1 -- 172.21.15.42:0/3726730999 >> 172.21.15.42:0/3726730999 conn(0x7f49680bebc0 msgr2=0x7f49680befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:26:59.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.885+0000 7f4967e00640 1 -- 172.21.15.42:0/3726730999 shutdown_connections 2024-07-04T17:26:59.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.885+0000 7f4967e00640 1 -- 172.21.15.42:0/3726730999 wait complete. 2024-07-04T17:26:59.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.888+0000 7f4967e00640 1 Processor -- start 2024-07-04T17:26:59.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.888+0000 7f4967e00640 1 -- start start 2024-07-04T17:26:59.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.890+0000 7f4967e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f496806c200 0x7f49680bde70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:59.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.890+0000 7f4967e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f496814ae50 con 0x7f496806c200 2024-07-04T17:26:59.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.890+0000 7f4966a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f496806c200 0x7f49680bde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:59.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.890+0000 7f4966a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f496806c200 0x7f49680bde70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33904/0 (socket says 172.21.15.42:33904) 2024-07-04T17:26:59.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.890+0000 7f4966a00640 1 -- 172.21.15.42:0/3602532312 learned_addr learned my addr 172.21.15.42:0/3602532312 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:26:59.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.891+0000 7f4966a00640 1 -- 172.21.15.42:0/3602532312 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4968071c20 con 0x7f496806c200 2024-07-04T17:26:59.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.891+0000 7f4966a00640 1 --2- 172.21.15.42:0/3602532312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f496806c200 0x7f49680bde70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f49580098d0 tx=0x7f4958033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:59.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.891+0000 7f495f400640 1 -- 172.21.15.42:0/3602532312 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4958031880 con 0x7f496806c200 2024-07-04T17:26:59.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.891+0000 7f495f400640 1 -- 172.21.15.42:0/3602532312 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4958031b10 con 0x7f496806c200 2024-07-04T17:26:59.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.891+0000 7f495f400640 1 -- 172.21.15.42:0/3602532312 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4958031df0 con 0x7f496806c200 2024-07-04T17:26:59.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.891+0000 7f4967e00640 1 -- 172.21.15.42:0/3602532312 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4968073210 con 0x7f496806c200 2024-07-04T17:26:59.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.891+0000 7f4967e00640 1 -- 172.21.15.42:0/3602532312 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f496806ffb0 con 0x7f496806c200 2024-07-04T17:26:59.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.893+0000 7f4967e00640 1 -- 172.21.15.42:0/3602532312 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f496814b490 con 0x7f496806c200 2024-07-04T17:26:59.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.893+0000 7f495f400640 1 -- 172.21.15.42:0/3602532312 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f49580330d0 con 0x7f496806c200 2024-07-04T17:26:59.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.894+0000 7f495f400640 1 --2- 172.21.15.42:0/3602532312 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f494403d2b0 0x7f494403f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:26:59.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.894+0000 7f495f400640 1 -- 172.21.15.42:0/3602532312 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 2587+0+0 (secure 0 0 0) 0x7f4958072560 con 0x7f496806c200 2024-07-04T17:26:59.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.894+0000 7f495fe00640 1 --2- 172.21.15.42:0/3602532312 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f494403d2b0 0x7f494403f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:26:59.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.895+0000 7f495fe00640 1 --2- 172.21.15.42:0/3602532312 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f494403d2b0 0x7f494403f7c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4950007fd0 tx=0x7f4950007920 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:26:59.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:26:59.899+0000 7f495f400640 1 -- 172.21.15.42:0/3602532312 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f495803f030 con 0x7f496806c200 2024-07-04T17:27:00.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.176+0000 7f4967e00640 1 -- 172.21.15.42:0/3602532312 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f49681487a0 con 0x7f496806c200 2024-07-04T17:27:00.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.177+0000 7f495f400640 1 -- 172.21.15.42:0/3602532312 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+5405 (secure 0 0 0) 0x7f495803a3a0 con 0x7f496806c200 2024-07-04T17:27:00.178 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:27:00.179 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:27:00.179 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_dump": [ 2024-07-04T17:27:00.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:00.179 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.a", 2024-07-04T17:27:00.179 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAM24Zm2VMEFxAAQqdcGux6Ker7jEr1dGAfzA==", 2024-07-04T17:27:00.179 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:27:00.179 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:27:00.179 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:27:00.179 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:27:00.179 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:27:00.179 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:00.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:00.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:00.180 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.b", 2024-07-04T17:27:00.180 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAM24ZmS/S7KxAAfjTZrLYR9qLxuTvd2zsCcQ==", 2024-07-04T17:27:00.180 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:27:00.180 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:27:00.180 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:27:00.180 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:27:00.180 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:27:00.180 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:00.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:00.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:00.180 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.c", 2024-07-04T17:27:00.180 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAN24Zm2JrRBBAAFMYIM2fLsr9EOfcyh3tZpA==", 2024-07-04T17:27:00.180 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:27:00.181 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:27:00.181 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:27:00.181 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:27:00.181 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:27:00.181 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:00.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:00.181 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:00.181 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.d", 2024-07-04T17:27:00.181 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAN24ZmPCGLFxAAhsh3XaAHCyauCGxh/3DFsw==", 2024-07-04T17:27:00.181 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:27:00.181 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:27:00.181 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:27:00.181 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:27:00.181 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:27:00.182 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:00.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:00.182 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:00.182 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.e", 2024-07-04T17:27:00.182 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAN24ZmVIaSKhAAhoBdDalwQ4eoo0rqb6p0Vw==", 2024-07-04T17:27:00.182 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:27:00.182 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:27:00.182 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:27:00.182 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:27:00.182 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:27:00.182 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:00.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:00.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:00.183 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.0", 2024-07-04T17:27:00.183 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAQ24Zm4spcCBAA4kTvvR7IOVa01K5ZCrJ35w==", 2024-07-04T17:27:00.183 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:27:00.183 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-07-04T17:27:00.183 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-07-04T17:27:00.183 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:27:00.183 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:00.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:00.183 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:00.183 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.1", 2024-07-04T17:27:00.183 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAT24Zmz1sfGRAAgfH5eIiQrufnPq6F94rHWg==", 2024-07-04T17:27:00.183 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:27:00.183 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-07-04T17:27:00.184 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-07-04T17:27:00.184 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:27:00.184 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:00.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:00.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:00.184 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.2", 2024-07-04T17:27:00.184 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAW24ZmJdQHORAAEjAM9OBqSIRhX148GC6pcw==", 2024-07-04T17:27:00.184 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:27:00.184 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-07-04T17:27:00.184 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-07-04T17:27:00.184 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:27:00.184 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:00.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:00.184 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:00.185 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.0", 2024-07-04T17:27:00.185 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAO24ZmUMmYABAAh4pNlkb9JsHQtK8uCJFQCA==", 2024-07-04T17:27:00.185 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:27:00.185 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:27:00.185 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-07-04T17:27:00.185 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow rw", 2024-07-04T17:27:00.185 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rwx" 2024-07-04T17:27:00.185 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:00.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:00.185 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:00.185 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.1", 2024-07-04T17:27:00.185 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAO24Zm0pAJEBAAntDeQ/oWDlJ+3xC7q5EQnQ==", 2024-07-04T17:27:00.185 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:27:00.185 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:27:00.186 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-07-04T17:27:00.186 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow rw", 2024-07-04T17:27:00.186 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rwx" 2024-07-04T17:27:00.186 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:00.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:00.186 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:00.186 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.admin", 2024-07-04T17:27:00.186 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAL24ZmMe3iHhAAFciH9cbUAIHLSAJvXy30EQ==", 2024-07-04T17:27:00.186 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:27:00.186 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-07-04T17:27:00.186 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:27:00.186 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:27:00.186 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:27:00.186 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:00.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:00.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:00.187 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.bootstrap-mds", 2024-07-04T17:27:00.187 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAf24ZmrNlsAxAADHFyZXhB4uvq3ERJ2t4dNA==", 2024-07-04T17:27:00.187 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:27:00.187 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile bootstrap-mds" 2024-07-04T17:27:00.187 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:00.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:00.187 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:00.187 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.bootstrap-mgr", 2024-07-04T17:27:00.187 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAf24Zm4+1sAxAALp9gMJeyBRaze+a+9LDIFg==", 2024-07-04T17:27:00.187 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:27:00.187 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile bootstrap-mgr" 2024-07-04T17:27:00.188 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:00.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:00.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:00.188 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.bootstrap-osd", 2024-07-04T17:27:00.188 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAf24ZmcQBtAxAAKjiDGZuFbXgrx+lJgTnrkg==", 2024-07-04T17:27:00.188 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:27:00.188 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile bootstrap-osd" 2024-07-04T17:27:00.188 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:00.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:00.188 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:00.188 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.bootstrap-rbd", 2024-07-04T17:27:00.188 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAf24ZmeRJtAxAApCmVpCfP5mBFC+mD/X9wxg==", 2024-07-04T17:27:00.188 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:27:00.188 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile bootstrap-rbd" 2024-07-04T17:27:00.188 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:00.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:00.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:00.189 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-07-04T17:27:00.189 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAf24ZmISRtAxAAX2Lqo5XZ9xW/PuPRNku6bg==", 2024-07-04T17:27:00.189 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:27:00.189 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-07-04T17:27:00.189 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:00.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:00.189 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:00.189 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.bootstrap-rgw", 2024-07-04T17:27:00.189 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAf24ZmAThtAxAAjvEpdRERqYtVZldGnmnZfw==", 2024-07-04T17:27:00.189 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:27:00.189 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile bootstrap-rgw" 2024-07-04T17:27:00.189 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:00.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:00.190 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:00.190 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.mirror", 2024-07-04T17:27:00.190 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAO24ZmBJzBHhAAR4Zg36YTtVizxm/g/Cdk0w==", 2024-07-04T17:27:00.190 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:27:00.190 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow r", 2024-07-04T17:27:00.190 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-07-04T17:27:00.190 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "profile cephfs-mirror", 2024-07-04T17:27:00.190 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rw tag cephfs metadata=*, allow r tag cephfs data=*" 2024-07-04T17:27:00.190 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:00.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:00.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:00.191 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.mirror_remote", 2024-07-04T17:27:00.191 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAO24Zm+BsqLhAA19GVuEjfWQeMe/fSc+dmoA==", 2024-07-04T17:27:00.191 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:27:00.191 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow rwps", 2024-07-04T17:27:00.191 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-07-04T17:27:00.191 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow r", 2024-07-04T17:27:00.191 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rw tag cephfs *=*" 2024-07-04T17:27:00.191 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:00.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:00.191 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:00.191 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mgr.x", 2024-07-04T17:27:00.191 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAM24Zm47tDCBAAjszudEEB4QHLlWqPXyZc2g==", 2024-07-04T17:27:00.192 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:27:00.192 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-07-04T17:27:00.192 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile mgr", 2024-07-04T17:27:00.192 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:27:00.192 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:00.192 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:00.192 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-07-04T17:27:00.192 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:27:00.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.179+0000 7f4967e00640 1 -- 172.21.15.42:0/3602532312 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f494403d2b0 msgr2=0x7f494403f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:00.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.179+0000 7f4967e00640 1 --2- 172.21.15.42:0/3602532312 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f494403d2b0 0x7f494403f7c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4950007fd0 tx=0x7f4950007920 comp rx=0 tx=0).stop 2024-07-04T17:27:00.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.179+0000 7f4967e00640 1 -- 172.21.15.42:0/3602532312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f496806c200 msgr2=0x7f49680bde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:00.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.179+0000 7f4967e00640 1 --2- 172.21.15.42:0/3602532312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f496806c200 0x7f49680bde70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f49580098d0 tx=0x7f4958033ad0 comp rx=0 tx=0).stop 2024-07-04T17:27:00.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.179+0000 7f4967e00640 1 -- 172.21.15.42:0/3602532312 shutdown_connections 2024-07-04T17:27:00.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.179+0000 7f4967e00640 1 --2- 172.21.15.42:0/3602532312 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f494403d2b0 0x7f494403f7c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:00.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.179+0000 7f4967e00640 1 --2- 172.21.15.42:0/3602532312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f496806c200 0x7f49680bde70 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:00.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.179+0000 7f4967e00640 1 -- 172.21.15.42:0/3602532312 >> 172.21.15.42:0/3602532312 conn(0x7f49680bebc0 msgr2=0x7f49680bbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:00.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.180+0000 7f4967e00640 1 -- 172.21.15.42:0/3602532312 shutdown_connections 2024-07-04T17:27:00.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.180+0000 7f4967e00640 1 -- 172.21.15.42:0/3602532312 wait complete. 2024-07-04T17:27:00.204 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-07-04T17:27:00.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.488+0000 7feebfe00640 1 -- 172.21.15.42:0/1146787255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feec00bad70 msgr2=0x7feec006c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:00.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.488+0000 7feebfe00640 1 --2- 172.21.15.42:0/1146787255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feec00bad70 0x7feec006c2c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7feeb0009fd0 tx=0x7feeb002a820 comp rx=0 tx=0).stop 2024-07-04T17:27:00.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.488+0000 7feebfe00640 1 -- 172.21.15.42:0/1146787255 shutdown_connections 2024-07-04T17:27:00.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.488+0000 7feebfe00640 1 --2- 172.21.15.42:0/1146787255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feec00bad70 0x7feec006c2c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:00.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.488+0000 7feebfe00640 1 -- 172.21.15.42:0/1146787255 >> 172.21.15.42:0/1146787255 conn(0x7feec00bebc0 msgr2=0x7feec00befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:00.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.488+0000 7feebfe00640 1 -- 172.21.15.42:0/1146787255 shutdown_connections 2024-07-04T17:27:00.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.488+0000 7feebfe00640 1 -- 172.21.15.42:0/1146787255 wait complete. 2024-07-04T17:27:00.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.491+0000 7feebfe00640 1 Processor -- start 2024-07-04T17:27:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.491+0000 7feebfe00640 1 -- start start 2024-07-04T17:27:00.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.493+0000 7feebfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feec00b5650 0x7feec01d7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:00.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.493+0000 7feebfe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7feec00b27c0 con 0x7feec00b5650 2024-07-04T17:27:00.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.493+0000 7feebea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feec00b5650 0x7feec01d7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:00.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.493+0000 7feebea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feec00b5650 0x7feec01d7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33926/0 (socket says 172.21.15.42:33926) 2024-07-04T17:27:00.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.493+0000 7feebea00640 1 -- 172.21.15.42:0/3179486002 learned_addr learned my addr 172.21.15.42:0/3179486002 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:00.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.494+0000 7feebea00640 1 -- 172.21.15.42:0/3179486002 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec00be620 con 0x7feec00b5650 2024-07-04T17:27:00.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.494+0000 7feebea00640 1 --2- 172.21.15.42:0/3179486002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feec00b5650 0x7feec01d7130 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7feeb000b310 tx=0x7feeb0031600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:00.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.494+0000 7feeb6a00640 1 -- 172.21.15.42:0/3179486002 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feeb0042070 con 0x7feec00b5650 2024-07-04T17:27:00.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.494+0000 7feeb6a00640 1 -- 172.21.15.42:0/3179486002 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feeb0031c40 con 0x7feec00b5650 2024-07-04T17:27:00.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.495+0000 7feebfe00640 1 -- 172.21.15.42:0/3179486002 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec0071eb0 con 0x7feec00b5650 2024-07-04T17:27:00.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.495+0000 7feeb6a00640 1 -- 172.21.15.42:0/3179486002 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feeb003c430 con 0x7feec00b5650 2024-07-04T17:27:00.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.495+0000 7feebfe00640 1 -- 172.21.15.42:0/3179486002 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec01dca40 con 0x7feec00b5650 2024-07-04T17:27:00.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.496+0000 7feebfe00640 1 -- 172.21.15.42:0/3179486002 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feec01d7670 con 0x7feec00b5650 2024-07-04T17:27:00.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.496+0000 7feeb6a00640 1 -- 172.21.15.42:0/3179486002 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7feeb0045070 con 0x7feec00b5650 2024-07-04T17:27:00.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.497+0000 7feeb6a00640 1 --2- 172.21.15.42:0/3179486002 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fee9c03d2b0 0x7fee9c03f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:00.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.497+0000 7feeb6a00640 1 -- 172.21.15.42:0/3179486002 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 2587+0+0 (secure 0 0 0) 0x7feeb00727b0 con 0x7feec00b5650 2024-07-04T17:27:00.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.497+0000 7feebe000640 1 --2- 172.21.15.42:0/3179486002 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fee9c03d2b0 0x7fee9c03f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:00.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.498+0000 7feebe000640 1 --2- 172.21.15.42:0/3179486002 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fee9c03d2b0 0x7fee9c03f7c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7feea8007fd0 tx=0x7feea8007920 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:00.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.502+0000 7feeb6a00640 1 -- 172.21.15.42:0/3179486002 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7feeb004f350 con 0x7feec00b5650 2024-07-04T17:27:00.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.795+0000 7feebfe00640 1 -- 172.21.15.42:0/3179486002 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7feec006c2c0 con 0x7feec00b5650 2024-07-04T17:27:00.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.798+0000 7feeb6a00640 1 -- 172.21.15.42:0/3179486002 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7feeb0009340 con 0x7feec00b5650 2024-07-04T17:27:00.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.801+0000 7feebfe00640 1 -- 172.21.15.42:0/3179486002 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fee9c03d2b0 msgr2=0x7fee9c03f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:00.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.801+0000 7feebfe00640 1 --2- 172.21.15.42:0/3179486002 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fee9c03d2b0 0x7fee9c03f7c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7feea8007fd0 tx=0x7feea8007920 comp rx=0 tx=0).stop 2024-07-04T17:27:00.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.801+0000 7feebfe00640 1 -- 172.21.15.42:0/3179486002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feec00b5650 msgr2=0x7feec01d7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:00.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.801+0000 7feebfe00640 1 --2- 172.21.15.42:0/3179486002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feec00b5650 0x7feec01d7130 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7feeb000b310 tx=0x7feeb0031600 comp rx=0 tx=0).stop 2024-07-04T17:27:00.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.801+0000 7feebfe00640 1 -- 172.21.15.42:0/3179486002 shutdown_connections 2024-07-04T17:27:00.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.801+0000 7feebfe00640 1 --2- 172.21.15.42:0/3179486002 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fee9c03d2b0 0x7fee9c03f7c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:00.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.801+0000 7feebfe00640 1 --2- 172.21.15.42:0/3179486002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feec00b5650 0x7feec01d7130 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:00.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.801+0000 7feebfe00640 1 -- 172.21.15.42:0/3179486002 >> 172.21.15.42:0/3179486002 conn(0x7feec00bebc0 msgr2=0x7feec01417d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:00.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.801+0000 7feebfe00640 1 -- 172.21.15.42:0/3179486002 shutdown_connections 2024-07-04T17:27:00.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:00.801+0000 7feebfe00640 1 -- 172.21.15.42:0/3179486002 wait complete. 2024-07-04T17:27:00.829 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-07-04T17:27:01.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.116+0000 7fb537200640 1 -- 172.21.15.42:0/552185468 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5380b1d20 msgr2=0x7fb5380b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:01.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.116+0000 7fb537200640 1 --2- 172.21.15.42:0/552185468 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5380b1d20 0x7fb5380b6250 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb5200097b0 tx=0x7fb52002a540 comp rx=0 tx=0).stop 2024-07-04T17:27:01.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.116+0000 7fb537200640 1 -- 172.21.15.42:0/552185468 shutdown_connections 2024-07-04T17:27:01.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.116+0000 7fb537200640 1 --2- 172.21.15.42:0/552185468 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5380b1d20 0x7fb5380b6250 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:01.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.116+0000 7fb537200640 1 -- 172.21.15.42:0/552185468 >> 172.21.15.42:0/552185468 conn(0x7fb5380bebc0 msgr2=0x7fb5380befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:01.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.116+0000 7fb537200640 1 -- 172.21.15.42:0/552185468 shutdown_connections 2024-07-04T17:27:01.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.116+0000 7fb537200640 1 -- 172.21.15.42:0/552185468 wait complete. 2024-07-04T17:27:01.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.119+0000 7fb537200640 1 Processor -- start 2024-07-04T17:27:01.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.119+0000 7fb537200640 1 -- start start 2024-07-04T17:27:01.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.121+0000 7fb537200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5380b1d20 0x7fb5381d9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:01.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.121+0000 7fb537200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb5380b8b40 con 0x7fb5380b1d20 2024-07-04T17:27:01.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.121+0000 7fb535e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5380b1d20 0x7fb5381d9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:01.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.121+0000 7fb535e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5380b1d20 0x7fb5381d9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33946/0 (socket says 172.21.15.42:33946) 2024-07-04T17:27:01.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.121+0000 7fb535e00640 1 -- 172.21.15.42:0/1415422184 learned_addr learned my addr 172.21.15.42:0/1415422184 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:01.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.122+0000 7fb535e00640 1 -- 172.21.15.42:0/1415422184 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5380ae940 con 0x7fb5380b1d20 2024-07-04T17:27:01.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.122+0000 7fb535e00640 1 --2- 172.21.15.42:0/1415422184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5380b1d20 0x7fb5381d9050 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb52800c320 tx=0x7fb52800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:01.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.122+0000 7fb52e000640 1 -- 172.21.15.42:0/1415422184 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5280116c0 con 0x7fb5380b1d20 2024-07-04T17:27:01.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.122+0000 7fb52e000640 1 -- 172.21.15.42:0/1415422184 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb528017070 con 0x7fb5380b1d20 2024-07-04T17:27:01.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.122+0000 7fb52e000640 1 -- 172.21.15.42:0/1415422184 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb528011b40 con 0x7fb5380b1d20 2024-07-04T17:27:01.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.122+0000 7fb537200640 1 -- 172.21.15.42:0/1415422184 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5381d71f0 con 0x7fb5380b1d20 2024-07-04T17:27:01.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.123+0000 7fb537200640 1 -- 172.21.15.42:0/1415422184 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5381d6ca0 con 0x7fb5380b1d20 2024-07-04T17:27:01.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.124+0000 7fb537200640 1 -- 172.21.15.42:0/1415422184 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5380b9170 con 0x7fb5380b1d20 2024-07-04T17:27:01.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.124+0000 7fb52e000640 1 -- 172.21.15.42:0/1415422184 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fb528011ce0 con 0x7fb5380b1d20 2024-07-04T17:27:01.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.125+0000 7fb52e000640 1 --2- 172.21.15.42:0/1415422184 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb51403d2b0 0x7fb51403f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:01.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.125+0000 7fb52e000640 1 -- 172.21.15.42:0/1415422184 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 2587+0+0 (secure 0 0 0) 0x7fb528050fb0 con 0x7fb5380b1d20 2024-07-04T17:27:01.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.125+0000 7fb52fe00640 1 --2- 172.21.15.42:0/1415422184 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb51403d2b0 0x7fb51403f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:01.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.126+0000 7fb52fe00640 1 --2- 172.21.15.42:0/1415422184 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb51403d2b0 0x7fb51403f7c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb520034000 tx=0x7fb520004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:01.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.130+0000 7fb52e000640 1 -- 172.21.15.42:0/1415422184 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fb52800a140 con 0x7fb5380b1d20 2024-07-04T17:27:01.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.414+0000 7fb537200640 1 -- 172.21.15.42:0/1415422184 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fb538068e30 con 0x7fb5380b1d20 2024-07-04T17:27:01.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.416+0000 7fb52e000640 1 -- 172.21.15.42:0/1415422184 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7fb528050200 con 0x7fb5380b1d20 2024-07-04T17:27:01.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.420+0000 7fb537200640 1 -- 172.21.15.42:0/1415422184 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb51403d2b0 msgr2=0x7fb51403f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:01.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.420+0000 7fb537200640 1 --2- 172.21.15.42:0/1415422184 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb51403d2b0 0x7fb51403f7c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb520034000 tx=0x7fb520004730 comp rx=0 tx=0).stop 2024-07-04T17:27:01.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.420+0000 7fb537200640 1 -- 172.21.15.42:0/1415422184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5380b1d20 msgr2=0x7fb5381d9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:01.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.420+0000 7fb537200640 1 --2- 172.21.15.42:0/1415422184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5380b1d20 0x7fb5381d9050 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb52800c320 tx=0x7fb52800c7e0 comp rx=0 tx=0).stop 2024-07-04T17:27:01.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.420+0000 7fb537200640 1 -- 172.21.15.42:0/1415422184 shutdown_connections 2024-07-04T17:27:01.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.420+0000 7fb537200640 1 --2- 172.21.15.42:0/1415422184 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb51403d2b0 0x7fb51403f7c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:01.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.420+0000 7fb537200640 1 --2- 172.21.15.42:0/1415422184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5380b1d20 0x7fb5381d9050 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:01.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.420+0000 7fb537200640 1 -- 172.21.15.42:0/1415422184 >> 172.21.15.42:0/1415422184 conn(0x7fb5380bebc0 msgr2=0x7fb53813edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:01.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.421+0000 7fb537200640 1 -- 172.21.15.42:0/1415422184 shutdown_connections 2024-07-04T17:27:01.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.421+0000 7fb537200640 1 -- 172.21.15.42:0/1415422184 wait complete. 2024-07-04T17:27:01.444 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-07-04T17:27:01.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.729+0000 7f303a400640 1 -- 172.21.15.42:0/925665331 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30340b2d20 msgr2=0x7f30340bb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:01.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.729+0000 7f303a400640 1 --2- 172.21.15.42:0/925665331 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30340b2d20 0x7f30340bb1f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f301c0097b0 tx=0x7f301c02a4a0 comp rx=0 tx=0).stop 2024-07-04T17:27:01.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.729+0000 7f303a400640 1 -- 172.21.15.42:0/925665331 shutdown_connections 2024-07-04T17:27:01.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.729+0000 7f303a400640 1 --2- 172.21.15.42:0/925665331 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30340b2d20 0x7f30340bb1f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:01.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.729+0000 7f303a400640 1 -- 172.21.15.42:0/925665331 >> 172.21.15.42:0/925665331 conn(0x7f30340bf700 msgr2=0x7f30340af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:01.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.729+0000 7f303a400640 1 -- 172.21.15.42:0/925665331 shutdown_connections 2024-07-04T17:27:01.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.729+0000 7f303a400640 1 -- 172.21.15.42:0/925665331 wait complete. 2024-07-04T17:27:01.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.731+0000 7f303a400640 1 Processor -- start 2024-07-04T17:27:01.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.731+0000 7f303a400640 1 -- start start 2024-07-04T17:27:01.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.732+0000 7f303a400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30340b2d20 0x7f30341e6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:01.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.732+0000 7f303a400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f303406c020 con 0x7f30340b2d20 2024-07-04T17:27:01.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.733+0000 7f3033400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30340b2d20 0x7f30341e6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:01.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.733+0000 7f3033400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30340b2d20 0x7f30341e6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33954/0 (socket says 172.21.15.42:33954) 2024-07-04T17:27:01.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.733+0000 7f3033400640 1 -- 172.21.15.42:0/876626694 learned_addr learned my addr 172.21.15.42:0/876626694 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:01.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.733+0000 7f3033400640 1 -- 172.21.15.42:0/876626694 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30341d76a0 con 0x7f30340b2d20 2024-07-04T17:27:01.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.733+0000 7f3033400640 1 --2- 172.21.15.42:0/876626694 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30340b2d20 0x7f30341e6d00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f302400c2f0 tx=0x7f302400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:01.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.734+0000 7f302b400640 1 -- 172.21.15.42:0/876626694 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3024017070 con 0x7f30340b2d20 2024-07-04T17:27:01.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.734+0000 7f302b400640 1 -- 172.21.15.42:0/876626694 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3024005410 con 0x7f30340b2d20 2024-07-04T17:27:01.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.734+0000 7f302b400640 1 -- 172.21.15.42:0/876626694 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3024003780 con 0x7f30340b2d20 2024-07-04T17:27:01.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.734+0000 7f303a400640 1 -- 172.21.15.42:0/876626694 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30341e7340 con 0x7f30340b2d20 2024-07-04T17:27:01.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.734+0000 7f303a400640 1 -- 172.21.15.42:0/876626694 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30341d7320 con 0x7f30340b2d20 2024-07-04T17:27:01.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.735+0000 7f302b400640 1 -- 172.21.15.42:0/876626694 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f302401a070 con 0x7f30340b2d20 2024-07-04T17:27:01.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.735+0000 7f303a400640 1 -- 172.21.15.42:0/876626694 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f303406c750 con 0x7f30340b2d20 2024-07-04T17:27:01.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.736+0000 7f302b400640 1 --2- 172.21.15.42:0/876626694 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f301003d2b0 0x7f301003f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:01.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.736+0000 7f302b400640 1 -- 172.21.15.42:0/876626694 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 2587+0+0 (secure 0 0 0) 0x7f3024050440 con 0x7f30340b2d20 2024-07-04T17:27:01.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.736+0000 7f302be00640 1 --2- 172.21.15.42:0/876626694 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f301003d2b0 0x7f301003f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:01.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.737+0000 7f302be00640 1 --2- 172.21.15.42:0/876626694 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f301003d2b0 0x7f301003f7c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f301c034000 tx=0x7f301c004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:01.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:01.741+0000 7f302b400640 1 -- 172.21.15.42:0/876626694 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f3024010300 con 0x7f30340b2d20 2024-07-04T17:27:02.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.030+0000 7f303a400640 1 -- 172.21.15.42:0/876626694 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f3034144470 con 0x7f30340b2d20 2024-07-04T17:27:02.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.034+0000 7f302b400640 1 -- 172.21.15.42:0/876626694 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f3024010e20 con 0x7f30340b2d20 2024-07-04T17:27:02.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.036+0000 7f303a400640 1 -- 172.21.15.42:0/876626694 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f301003d2b0 msgr2=0x7f301003f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:02.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.037+0000 7f303a400640 1 --2- 172.21.15.42:0/876626694 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f301003d2b0 0x7f301003f7c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f301c034000 tx=0x7f301c004730 comp rx=0 tx=0).stop 2024-07-04T17:27:02.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.037+0000 7f303a400640 1 -- 172.21.15.42:0/876626694 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30340b2d20 msgr2=0x7f30341e6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:02.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.037+0000 7f303a400640 1 --2- 172.21.15.42:0/876626694 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30340b2d20 0x7f30341e6d00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f302400c2f0 tx=0x7f302400c7b0 comp rx=0 tx=0).stop 2024-07-04T17:27:02.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.037+0000 7f303a400640 1 -- 172.21.15.42:0/876626694 shutdown_connections 2024-07-04T17:27:02.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.037+0000 7f303a400640 1 --2- 172.21.15.42:0/876626694 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f301003d2b0 0x7f301003f7c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:02.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.037+0000 7f303a400640 1 --2- 172.21.15.42:0/876626694 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30340b2d20 0x7f30341e6d00 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:02.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.037+0000 7f303a400640 1 -- 172.21.15.42:0/876626694 >> 172.21.15.42:0/876626694 conn(0x7f30340bf700 msgr2=0x7f30340a94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:02.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.037+0000 7f303a400640 1 -- 172.21.15.42:0/876626694 shutdown_connections 2024-07-04T17:27:02.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.037+0000 7f303a400640 1 -- 172.21.15.42:0/876626694 wait complete. 2024-07-04T17:27:02.062 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-07-04T17:27:02.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.336+0000 7ff35ca00640 1 -- 172.21.15.42:0/170761622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3580adac0 msgr2=0x7ff35806e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:02.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.336+0000 7ff35ca00640 1 --2- 172.21.15.42:0/170761622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3580adac0 0x7ff35806e120 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff3400097b0 tx=0x7ff34002a540 comp rx=0 tx=0).stop 2024-07-04T17:27:02.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.336+0000 7ff35ca00640 1 -- 172.21.15.42:0/170761622 shutdown_connections 2024-07-04T17:27:02.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.336+0000 7ff35ca00640 1 --2- 172.21.15.42:0/170761622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3580adac0 0x7ff35806e120 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:02.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.336+0000 7ff35ca00640 1 -- 172.21.15.42:0/170761622 >> 172.21.15.42:0/170761622 conn(0x7ff358071960 msgr2=0x7ff35806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:02.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.337+0000 7ff35ca00640 1 -- 172.21.15.42:0/170761622 shutdown_connections 2024-07-04T17:27:02.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.337+0000 7ff35ca00640 1 -- 172.21.15.42:0/170761622 wait complete. 2024-07-04T17:27:02.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.339+0000 7ff35ca00640 1 Processor -- start 2024-07-04T17:27:02.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.339+0000 7ff35ca00640 1 -- start start 2024-07-04T17:27:02.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.340+0000 7ff35ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3580adac0 0x7ff3580b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:02.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.340+0000 7ff35ca00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff35806e660 con 0x7ff3580adac0 2024-07-04T17:27:02.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.340+0000 7ff357400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3580adac0 0x7ff3580b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:02.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.340+0000 7ff357400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3580adac0 0x7ff3580b04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33986/0 (socket says 172.21.15.42:33986) 2024-07-04T17:27:02.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.340+0000 7ff357400640 1 -- 172.21.15.42:0/3408029260 learned_addr learned my addr 172.21.15.42:0/3408029260 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:02.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.341+0000 7ff357400640 1 -- 172.21.15.42:0/3408029260 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3580b5cd0 con 0x7ff3580adac0 2024-07-04T17:27:02.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.341+0000 7ff357400640 1 --2- 172.21.15.42:0/3408029260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3580adac0 0x7ff3580b04a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff34800ca90 tx=0x7ff34800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:02.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.341+0000 7ff34f400640 1 -- 172.21.15.42:0/3408029260 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff348016020 con 0x7ff3580adac0 2024-07-04T17:27:02.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.341+0000 7ff34f400640 1 -- 172.21.15.42:0/3408029260 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff34800ac40 con 0x7ff3580adac0 2024-07-04T17:27:02.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.341+0000 7ff34f400640 1 -- 172.21.15.42:0/3408029260 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff348012420 con 0x7ff3580adac0 2024-07-04T17:27:02.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.341+0000 7ff35ca00640 1 -- 172.21.15.42:0/3408029260 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3581e7390 con 0x7ff3580adac0 2024-07-04T17:27:02.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.341+0000 7ff35ca00640 1 -- 172.21.15.42:0/3408029260 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3580b5800 con 0x7ff3580adac0 2024-07-04T17:27:02.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.342+0000 7ff35ca00640 1 -- 172.21.15.42:0/3408029260 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff35806ed20 con 0x7ff3580adac0 2024-07-04T17:27:02.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.343+0000 7ff34f400640 1 -- 172.21.15.42:0/3408029260 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7ff348019020 con 0x7ff3580adac0 2024-07-04T17:27:02.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.343+0000 7ff34f400640 1 --2- 172.21.15.42:0/3408029260 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff33403d2b0 0x7ff33403f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:02.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.343+0000 7ff34f400640 1 -- 172.21.15.42:0/3408029260 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 2587+0+0 (secure 0 0 0) 0x7ff348051850 con 0x7ff3580adac0 2024-07-04T17:27:02.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.344+0000 7ff356a00640 1 --2- 172.21.15.42:0/3408029260 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff33403d2b0 0x7ff33403f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:02.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.344+0000 7ff356a00640 1 --2- 172.21.15.42:0/3408029260 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff33403d2b0 0x7ff33403f7c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff340034000 tx=0x7ff340004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:02.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.348+0000 7ff34f400640 1 -- 172.21.15.42:0/3408029260 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7ff348020020 con 0x7ff3580adac0 2024-07-04T17:27:02.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.631+0000 7ff35ca00640 1 -- 172.21.15.42:0/3408029260 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7ff3580b50f0 con 0x7ff3580adac0 2024-07-04T17:27:02.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.635+0000 7ff34f400640 1 -- 172.21.15.42:0/3408029260 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7ff348010cd0 con 0x7ff3580adac0 2024-07-04T17:27:02.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.637+0000 7ff35ca00640 1 -- 172.21.15.42:0/3408029260 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff33403d2b0 msgr2=0x7ff33403f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:02.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.637+0000 7ff35ca00640 1 --2- 172.21.15.42:0/3408029260 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff33403d2b0 0x7ff33403f7c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff340034000 tx=0x7ff340004730 comp rx=0 tx=0).stop 2024-07-04T17:27:02.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.637+0000 7ff35ca00640 1 -- 172.21.15.42:0/3408029260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3580adac0 msgr2=0x7ff3580b04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:02.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.637+0000 7ff35ca00640 1 --2- 172.21.15.42:0/3408029260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3580adac0 0x7ff3580b04a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff34800ca90 tx=0x7ff34800cf50 comp rx=0 tx=0).stop 2024-07-04T17:27:02.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.637+0000 7ff35ca00640 1 -- 172.21.15.42:0/3408029260 shutdown_connections 2024-07-04T17:27:02.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.637+0000 7ff35ca00640 1 --2- 172.21.15.42:0/3408029260 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff33403d2b0 0x7ff33403f7c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:02.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.637+0000 7ff35ca00640 1 --2- 172.21.15.42:0/3408029260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3580adac0 0x7ff3580b04a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:02.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.637+0000 7ff35ca00640 1 -- 172.21.15.42:0/3408029260 >> 172.21.15.42:0/3408029260 conn(0x7ff358071960 msgr2=0x7ff3581d7d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:02.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.638+0000 7ff35ca00640 1 -- 172.21.15.42:0/3408029260 shutdown_connections 2024-07-04T17:27:02.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.638+0000 7ff35ca00640 1 -- 172.21.15.42:0/3408029260 wait complete. 2024-07-04T17:27:02.664 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-07-04T17:27:02.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.937+0000 7fb33dc00640 1 -- 172.21.15.42:0/790282412 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb33806d580 msgr2=0x7fb33806c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:02.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.937+0000 7fb33dc00640 1 --2- 172.21.15.42:0/790282412 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb33806d580 0x7fb33806c2c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb32800a8c0 tx=0x7fb32802b7d0 comp rx=0 tx=0).stop 2024-07-04T17:27:02.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.937+0000 7fb33dc00640 1 -- 172.21.15.42:0/790282412 shutdown_connections 2024-07-04T17:27:02.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.937+0000 7fb33dc00640 1 --2- 172.21.15.42:0/790282412 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb33806d580 0x7fb33806c2c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:02.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.937+0000 7fb33dc00640 1 -- 172.21.15.42:0/790282412 >> 172.21.15.42:0/790282412 conn(0x7fb3380bebc0 msgr2=0x7fb3380befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:02.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.937+0000 7fb33dc00640 1 -- 172.21.15.42:0/790282412 shutdown_connections 2024-07-04T17:27:02.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.937+0000 7fb33dc00640 1 -- 172.21.15.42:0/790282412 wait complete. 2024-07-04T17:27:02.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.939+0000 7fb33dc00640 1 Processor -- start 2024-07-04T17:27:02.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.939+0000 7fb33dc00640 1 -- start start 2024-07-04T17:27:02.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.940+0000 7fb33dc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb33806d580 0x7fb3380afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:02.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.940+0000 7fb33dc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb3380b2810 con 0x7fb33806d580 2024-07-04T17:27:02.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.940+0000 7fb337e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb33806d580 0x7fb3380afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:02.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.940+0000 7fb337e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb33806d580 0x7fb3380afcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34008/0 (socket says 172.21.15.42:34008) 2024-07-04T17:27:02.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.940+0000 7fb337e00640 1 -- 172.21.15.42:0/1645619701 learned_addr learned my addr 172.21.15.42:0/1645619701 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:02.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.941+0000 7fb337e00640 1 -- 172.21.15.42:0/1645619701 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3380ae930 con 0x7fb33806d580 2024-07-04T17:27:02.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.941+0000 7fb337e00640 1 --2- 172.21.15.42:0/1645619701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb33806d580 0x7fb3380afcb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb328000c00 tx=0x7fb328003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:02.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.941+0000 7fb334c00640 1 -- 172.21.15.42:0/1645619701 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb328039040 con 0x7fb33806d580 2024-07-04T17:27:02.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.941+0000 7fb334c00640 1 -- 172.21.15.42:0/1645619701 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb328044070 con 0x7fb33806d580 2024-07-04T17:27:02.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.941+0000 7fb334c00640 1 -- 172.21.15.42:0/1645619701 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3280085d0 con 0x7fb33806d580 2024-07-04T17:27:02.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.941+0000 7fb33dc00640 1 -- 172.21.15.42:0/1645619701 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3380af560 con 0x7fb33806d580 2024-07-04T17:27:02.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.942+0000 7fb33dc00640 1 -- 172.21.15.42:0/1645619701 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb33813ef50 con 0x7fb33806d580 2024-07-04T17:27:02.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.943+0000 7fb33dc00640 1 -- 172.21.15.42:0/1645619701 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3380aeb00 con 0x7fb33806d580 2024-07-04T17:27:02.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.943+0000 7fb334c00640 1 -- 172.21.15.42:0/1645619701 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fb328032070 con 0x7fb33806d580 2024-07-04T17:27:02.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.944+0000 7fb334c00640 1 --2- 172.21.15.42:0/1645619701 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb31403d2b0 0x7fb31403f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:02.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.944+0000 7fb334c00640 1 -- 172.21.15.42:0/1645619701 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 2587+0+0 (secure 0 0 0) 0x7fb328072070 con 0x7fb33806d580 2024-07-04T17:27:02.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.944+0000 7fb32fe00640 1 --2- 172.21.15.42:0/1645619701 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb31403d2b0 0x7fb31403f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:02.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.945+0000 7fb32fe00640 1 --2- 172.21.15.42:0/1645619701 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb31403d2b0 0x7fb31403f7c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb3200015c0 tx=0x7fb320007a90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:02.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:02.949+0000 7fb334c00640 1 -- 172.21.15.42:0/1645619701 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fb328008950 con 0x7fb33806d580 2024-07-04T17:27:03.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.220+0000 7fb33dc00640 1 -- 172.21.15.42:0/1645619701 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fb33806c2c0 con 0x7fb33806d580 2024-07-04T17:27:03.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.225+0000 7fb334c00640 1 -- 172.21.15.42:0/1645619701 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7fb328008cd0 con 0x7fb33806d580 2024-07-04T17:27:03.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.227+0000 7fb33dc00640 1 -- 172.21.15.42:0/1645619701 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb31403d2b0 msgr2=0x7fb31403f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:03.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.227+0000 7fb33dc00640 1 --2- 172.21.15.42:0/1645619701 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb31403d2b0 0x7fb31403f7c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb3200015c0 tx=0x7fb320007a90 comp rx=0 tx=0).stop 2024-07-04T17:27:03.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.227+0000 7fb33dc00640 1 -- 172.21.15.42:0/1645619701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb33806d580 msgr2=0x7fb3380afcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:03.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.227+0000 7fb33dc00640 1 --2- 172.21.15.42:0/1645619701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb33806d580 0x7fb3380afcb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb328000c00 tx=0x7fb328003b90 comp rx=0 tx=0).stop 2024-07-04T17:27:03.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.228+0000 7fb33dc00640 1 -- 172.21.15.42:0/1645619701 shutdown_connections 2024-07-04T17:27:03.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.228+0000 7fb33dc00640 1 --2- 172.21.15.42:0/1645619701 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb31403d2b0 0x7fb31403f7c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:03.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.228+0000 7fb33dc00640 1 --2- 172.21.15.42:0/1645619701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb33806d580 0x7fb3380afcb0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:03.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.228+0000 7fb33dc00640 1 -- 172.21.15.42:0/1645619701 >> 172.21.15.42:0/1645619701 conn(0x7fb3380bebc0 msgr2=0x7fb3380bd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:03.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.228+0000 7fb33dc00640 1 -- 172.21.15.42:0/1645619701 shutdown_connections 2024-07-04T17:27:03.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.228+0000 7fb33dc00640 1 -- 172.21.15.42:0/1645619701 wait complete. 2024-07-04T17:27:03.256 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-07-04T17:27:03.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.538+0000 7fb4e3200640 1 -- 172.21.15.42:0/4182236964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4e40bad70 msgr2=0x7fb4e406c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:03.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.538+0000 7fb4e3200640 1 --2- 172.21.15.42:0/4182236964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4e40bad70 0x7fb4e406c2c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4009230 tx=0x7fb4d402a3f0 comp rx=0 tx=0).stop 2024-07-04T17:27:03.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.538+0000 7fb4e3200640 1 -- 172.21.15.42:0/4182236964 shutdown_connections 2024-07-04T17:27:03.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.538+0000 7fb4e3200640 1 --2- 172.21.15.42:0/4182236964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4e40bad70 0x7fb4e406c2c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:03.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.538+0000 7fb4e3200640 1 -- 172.21.15.42:0/4182236964 >> 172.21.15.42:0/4182236964 conn(0x7fb4e40bebc0 msgr2=0x7fb4e40befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:03.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.538+0000 7fb4e3200640 1 -- 172.21.15.42:0/4182236964 shutdown_connections 2024-07-04T17:27:03.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.538+0000 7fb4e3200640 1 -- 172.21.15.42:0/4182236964 wait complete. 2024-07-04T17:27:03.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.540+0000 7fb4e3200640 1 Processor -- start 2024-07-04T17:27:03.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.540+0000 7fb4e3200640 1 -- start start 2024-07-04T17:27:03.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.542+0000 7fb4e3200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4e40b5650 0x7fb4e4071530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:03.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.542+0000 7fb4e3200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e40b27c0 con 0x7fb4e40b5650 2024-07-04T17:27:03.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.542+0000 7fb4e1e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4e40b5650 0x7fb4e4071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:03.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.542+0000 7fb4e1e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4e40b5650 0x7fb4e4071530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34022/0 (socket says 172.21.15.42:34022) 2024-07-04T17:27:03.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.542+0000 7fb4e1e00640 1 -- 172.21.15.42:0/539447727 learned_addr learned my addr 172.21.15.42:0/539447727 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:03.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.542+0000 7fb4e1e00640 1 -- 172.21.15.42:0/539447727 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e40731d0 con 0x7fb4e40b5650 2024-07-04T17:27:03.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.542+0000 7fb4e1e00640 1 --2- 172.21.15.42:0/539447727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4e40b5650 0x7fb4e4071530 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4009290 tx=0x7fb4d4033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:03.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.543+0000 7fb4da000640 1 -- 172.21.15.42:0/539447727 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4d40319b0 con 0x7fb4e40b5650 2024-07-04T17:27:03.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.543+0000 7fb4da000640 1 -- 172.21.15.42:0/539447727 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb4d4031c40 con 0x7fb4e40b5650 2024-07-04T17:27:03.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.543+0000 7fb4e3200640 1 -- 172.21.15.42:0/539447727 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4e41e3d60 con 0x7fb4e40b5650 2024-07-04T17:27:03.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.543+0000 7fb4da000640 1 -- 172.21.15.42:0/539447727 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4d4033070 con 0x7fb4e40b5650 2024-07-04T17:27:03.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.543+0000 7fb4e3200640 1 -- 172.21.15.42:0/539447727 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4e413ef50 con 0x7fb4e40b5650 2024-07-04T17:27:03.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.544+0000 7fb4e3200640 1 -- 172.21.15.42:0/539447727 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4e406c2c0 con 0x7fb4e40b5650 2024-07-04T17:27:03.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.544+0000 7fb4da000640 1 -- 172.21.15.42:0/539447727 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fb4d4033210 con 0x7fb4e40b5650 2024-07-04T17:27:03.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.545+0000 7fb4da000640 1 --2- 172.21.15.42:0/539447727 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb4c003d2b0 0x7fb4c003f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:03.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.545+0000 7fb4da000640 1 -- 172.21.15.42:0/539447727 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 2587+0+0 (secure 0 0 0) 0x7fb4d40725f0 con 0x7fb4e40b5650 2024-07-04T17:27:03.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.545+0000 7fb4dbe00640 1 --2- 172.21.15.42:0/539447727 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb4c003d2b0 0x7fb4c003f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:03.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.546+0000 7fb4dbe00640 1 --2- 172.21.15.42:0/539447727 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb4c003d2b0 0x7fb4c003f7c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc007fd0 tx=0x7fb4cc007920 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:03.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.549+0000 7fb4da000640 1 -- 172.21.15.42:0/539447727 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fb4d403f030 con 0x7fb4e40b5650 2024-07-04T17:27:03.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.836+0000 7fb4e3200640 1 -- 172.21.15.42:0/539447727 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fb4e4068e30 con 0x7fb4e40b5650 2024-07-04T17:27:03.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.839+0000 7fb4da000640 1 -- 172.21.15.42:0/539447727 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7fb4d4003b10 con 0x7fb4e40b5650 2024-07-04T17:27:03.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.842+0000 7fb4e3200640 1 -- 172.21.15.42:0/539447727 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb4c003d2b0 msgr2=0x7fb4c003f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:03.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.842+0000 7fb4e3200640 1 --2- 172.21.15.42:0/539447727 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb4c003d2b0 0x7fb4c003f7c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc007fd0 tx=0x7fb4cc007920 comp rx=0 tx=0).stop 2024-07-04T17:27:03.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.842+0000 7fb4e3200640 1 -- 172.21.15.42:0/539447727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4e40b5650 msgr2=0x7fb4e4071530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:03.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.842+0000 7fb4e3200640 1 --2- 172.21.15.42:0/539447727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4e40b5650 0x7fb4e4071530 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4009290 tx=0x7fb4d4033ad0 comp rx=0 tx=0).stop 2024-07-04T17:27:03.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.842+0000 7fb4e3200640 1 -- 172.21.15.42:0/539447727 shutdown_connections 2024-07-04T17:27:03.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.842+0000 7fb4e3200640 1 --2- 172.21.15.42:0/539447727 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb4c003d2b0 0x7fb4c003f7c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:03.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.842+0000 7fb4e3200640 1 --2- 172.21.15.42:0/539447727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4e40b5650 0x7fb4e4071530 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:03.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.842+0000 7fb4e3200640 1 -- 172.21.15.42:0/539447727 >> 172.21.15.42:0/539447727 conn(0x7fb4e40bebc0 msgr2=0x7fb4e40b0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:03.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.842+0000 7fb4e3200640 1 -- 172.21.15.42:0/539447727 shutdown_connections 2024-07-04T17:27:03.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:03.842+0000 7fb4e3200640 1 -- 172.21.15.42:0/539447727 wait complete. 2024-07-04T17:27:03.868 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-04T17:27:04.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.155+0000 7fcdeca00640 1 -- 172.21.15.42:0/451965984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcde8073e80 msgr2=0x7fcde806d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:04.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.155+0000 7fcdeca00640 1 --2- 172.21.15.42:0/451965984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcde8073e80 0x7fcde806d8d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcde8074460 tx=0x7fcdd002a520 comp rx=0 tx=0).stop 2024-07-04T17:27:04.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.155+0000 7fcdeca00640 1 -- 172.21.15.42:0/451965984 shutdown_connections 2024-07-04T17:27:04.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.155+0000 7fcdeca00640 1 --2- 172.21.15.42:0/451965984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcde8073e80 0x7fcde806d8d0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:04.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.155+0000 7fcdeca00640 1 -- 172.21.15.42:0/451965984 >> 172.21.15.42:0/451965984 conn(0x7fcde8070830 msgr2=0x7fcde80bfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:04.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.155+0000 7fcdeca00640 1 -- 172.21.15.42:0/451965984 shutdown_connections 2024-07-04T17:27:04.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.155+0000 7fcdeca00640 1 -- 172.21.15.42:0/451965984 wait complete. 2024-07-04T17:27:04.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.158+0000 7fcdeca00640 1 Processor -- start 2024-07-04T17:27:04.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.158+0000 7fcdeca00640 1 -- start start 2024-07-04T17:27:04.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.160+0000 7fcdeca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcde8073e80 0x7fcde8148e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:04.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.160+0000 7fcdeca00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcde81485b0 con 0x7fcde8073e80 2024-07-04T17:27:04.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.160+0000 7fcde7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcde8073e80 0x7fcde8148e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:04.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.160+0000 7fcde7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcde8073e80 0x7fcde8148e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38300/0 (socket says 172.21.15.42:38300) 2024-07-04T17:27:04.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.160+0000 7fcde7400640 1 -- 172.21.15.42:0/442188170 learned_addr learned my addr 172.21.15.42:0/442188170 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:04.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.160+0000 7fcde7400640 1 -- 172.21.15.42:0/442188170 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde81e4bb0 con 0x7fcde8073e80 2024-07-04T17:27:04.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.161+0000 7fcde7400640 1 --2- 172.21.15.42:0/442188170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcde8073e80 0x7fcde8148e60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcdd800cb40 tx=0x7fcdd800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:04.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.161+0000 7fcddf400640 1 -- 172.21.15.42:0/442188170 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcdd8016020 con 0x7fcde8073e80 2024-07-04T17:27:04.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.161+0000 7fcddf400640 1 -- 172.21.15.42:0/442188170 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcdd800abf0 con 0x7fcde8073e80 2024-07-04T17:27:04.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.161+0000 7fcdeca00640 1 -- 172.21.15.42:0/442188170 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcde81d4c80 con 0x7fcde8073e80 2024-07-04T17:27:04.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.161+0000 7fcddf400640 1 -- 172.21.15.42:0/442188170 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcdd8012420 con 0x7fcde8073e80 2024-07-04T17:27:04.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.161+0000 7fcdeca00640 1 -- 172.21.15.42:0/442188170 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcde8144420 con 0x7fcde8073e80 2024-07-04T17:27:04.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.162+0000 7fcdeca00640 1 -- 172.21.15.42:0/442188170 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcde8149dc0 con 0x7fcde8073e80 2024-07-04T17:27:04.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.163+0000 7fcddf400640 1 -- 172.21.15.42:0/442188170 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fcdd8019020 con 0x7fcde8073e80 2024-07-04T17:27:04.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.163+0000 7fcddf400640 1 --2- 172.21.15.42:0/442188170 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fcdc403d2b0 0x7fcdc403f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:04.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.163+0000 7fcddf400640 1 -- 172.21.15.42:0/442188170 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 2587+0+0 (secure 0 0 0) 0x7fcdd8051500 con 0x7fcde8073e80 2024-07-04T17:27:04.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.164+0000 7fcde6a00640 1 --2- 172.21.15.42:0/442188170 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fcdc403d2b0 0x7fcdc403f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:04.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.164+0000 7fcde6a00640 1 --2- 172.21.15.42:0/442188170 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fcdc403d2b0 0x7fcdc403f7c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0034000 tx=0x7fcdd0004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:04.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.168+0000 7fcddf400640 1 -- 172.21.15.42:0/442188170 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fcdd8020030 con 0x7fcde8073e80 2024-07-04T17:27:04.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.370+0000 7fcdeca00640 1 -- 172.21.15.42:0/442188170 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fcde8148c50 con 0x7fcde8073e80 2024-07-04T17:27:04.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.371+0000 7fcddf400640 1 -- 172.21.15.42:0/442188170 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v23) ==== 84+0+65 (secure 0 0 0) 0x7fcde80bce60 con 0x7fcde8073e80 2024-07-04T17:27:04.373 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:27:04.373 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-07-04T17:27:04.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:04.373 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 1, 2024-07-04T17:27:04.373 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": ".mgr" 2024-07-04T17:27:04.374 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:04.374 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-07-04T17:27:04.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.374+0000 7fcdeca00640 1 -- 172.21.15.42:0/442188170 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fcdc403d2b0 msgr2=0x7fcdc403f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:04.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.374+0000 7fcdeca00640 1 --2- 172.21.15.42:0/442188170 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fcdc403d2b0 0x7fcdc403f7c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0034000 tx=0x7fcdd0004730 comp rx=0 tx=0).stop 2024-07-04T17:27:04.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.374+0000 7fcdeca00640 1 -- 172.21.15.42:0/442188170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcde8073e80 msgr2=0x7fcde8148e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:04.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.374+0000 7fcdeca00640 1 --2- 172.21.15.42:0/442188170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcde8073e80 0x7fcde8148e60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcdd800cb40 tx=0x7fcdd800cf40 comp rx=0 tx=0).stop 2024-07-04T17:27:04.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.374+0000 7fcdeca00640 1 -- 172.21.15.42:0/442188170 shutdown_connections 2024-07-04T17:27:04.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.374+0000 7fcdeca00640 1 --2- 172.21.15.42:0/442188170 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fcdc403d2b0 0x7fcdc403f7c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:04.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.374+0000 7fcdeca00640 1 --2- 172.21.15.42:0/442188170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcde8073e80 0x7fcde8148e60 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:04.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.374+0000 7fcdeca00640 1 -- 172.21.15.42:0/442188170 >> 172.21.15.42:0/442188170 conn(0x7fcde8070830 msgr2=0x7fcde80b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:04.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.374+0000 7fcdeca00640 1 -- 172.21.15.42:0/442188170 shutdown_connections 2024-07-04T17:27:04.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.374+0000 7fcdeca00640 1 -- 172.21.15.42:0/442188170 wait complete. 2024-07-04T17:27:04.403 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-04T17:27:04.403 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-04T17:27:04.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.694+0000 7fae37e00640 1 -- 172.21.15.42:0/2647289199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae380ac5b0 msgr2=0x7fae380b0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:04.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.694+0000 7fae37e00640 1 --2- 172.21.15.42:0/2647289199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae380ac5b0 0x7fae380b0db0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fae200097b0 tx=0x7fae2002a4b0 comp rx=0 tx=0).stop 2024-07-04T17:27:04.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.694+0000 7fae37e00640 1 -- 172.21.15.42:0/2647289199 shutdown_connections 2024-07-04T17:27:04.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.694+0000 7fae37e00640 1 --2- 172.21.15.42:0/2647289199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae380ac5b0 0x7fae380b0db0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:04.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.694+0000 7fae37e00640 1 -- 172.21.15.42:0/2647289199 >> 172.21.15.42:0/2647289199 conn(0x7fae380afaa0 msgr2=0x7fae380afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:04.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.694+0000 7fae37e00640 1 -- 172.21.15.42:0/2647289199 shutdown_connections 2024-07-04T17:27:04.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.694+0000 7fae37e00640 1 -- 172.21.15.42:0/2647289199 wait complete. 2024-07-04T17:27:04.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.696+0000 7fae37e00640 1 Processor -- start 2024-07-04T17:27:04.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.697+0000 7fae37e00640 1 -- start start 2024-07-04T17:27:04.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.698+0000 7fae37e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae3806da60 0x7fae3813f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:04.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.698+0000 7fae37e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fae380ae590 con 0x7fae3806da60 2024-07-04T17:27:04.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.698+0000 7fae36a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae3806da60 0x7fae3813f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:04.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.698+0000 7fae36a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae3806da60 0x7fae3813f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38318/0 (socket says 172.21.15.42:38318) 2024-07-04T17:27:04.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.698+0000 7fae36a00640 1 -- 172.21.15.42:0/4052324136 learned_addr learned my addr 172.21.15.42:0/4052324136 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:04.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.698+0000 7fae36a00640 1 -- 172.21.15.42:0/4052324136 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae381d8d10 con 0x7fae3806da60 2024-07-04T17:27:04.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.699+0000 7fae36a00640 1 --2- 172.21.15.42:0/4052324136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae3806da60 0x7fae3813f3a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fae2800c320 tx=0x7fae2800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:04.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.699+0000 7fae2ea00640 1 -- 172.21.15.42:0/4052324136 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae280116c0 con 0x7fae3806da60 2024-07-04T17:27:04.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.699+0000 7fae2ea00640 1 -- 172.21.15.42:0/4052324136 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae28017070 con 0x7fae3806da60 2024-07-04T17:27:04.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.699+0000 7fae2ea00640 1 -- 172.21.15.42:0/4052324136 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae28011b40 con 0x7fae3806da60 2024-07-04T17:27:04.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.699+0000 7fae37e00640 1 -- 172.21.15.42:0/4052324136 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae381e3640 con 0x7fae3806da60 2024-07-04T17:27:04.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.699+0000 7fae37e00640 1 -- 172.21.15.42:0/4052324136 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae381d8160 con 0x7fae3806da60 2024-07-04T17:27:04.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.700+0000 7fae37e00640 1 -- 172.21.15.42:0/4052324136 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae380b0db0 con 0x7fae3806da60 2024-07-04T17:27:04.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.701+0000 7fae2ea00640 1 -- 172.21.15.42:0/4052324136 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fae28011ce0 con 0x7fae3806da60 2024-07-04T17:27:04.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.701+0000 7fae2ea00640 1 --2- 172.21.15.42:0/4052324136 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fae1403d2b0 0x7fae1403f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:04.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.701+0000 7fae2ea00640 1 -- 172.21.15.42:0/4052324136 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 2587+0+0 (secure 0 0 0) 0x7fae28050ef0 con 0x7fae3806da60 2024-07-04T17:27:04.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.702+0000 7fae36000640 1 --2- 172.21.15.42:0/4052324136 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fae1403d2b0 0x7fae1403f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:04.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.702+0000 7fae36000640 1 --2- 172.21.15.42:0/4052324136 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fae1403d2b0 0x7fae1403f7c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fae20034000 tx=0x7fae20004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:04.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.706+0000 7fae2ea00640 1 -- 172.21.15.42:0/4052324136 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fae2800a140 con 0x7fae3806da60 2024-07-04T17:27:04.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:04.915+0000 7fae37e00640 1 -- 172.21.15.42:0/4052324136 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fae38068e30 con 0x7fae3806da60 2024-07-04T17:27:05.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:05.815+0000 7fae2ea00640 1 -- 172.21.15.42:0/4052324136 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v24) ==== 138+0+0 (secure 0 0 0) 0x7fae2801ac80 con 0x7fae3806da60 2024-07-04T17:27:05.817 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' created 2024-07-04T17:27:05.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:05.819+0000 7fae37e00640 1 -- 172.21.15.42:0/4052324136 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fae1403d2b0 msgr2=0x7fae1403f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:05.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:05.819+0000 7fae37e00640 1 --2- 172.21.15.42:0/4052324136 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fae1403d2b0 0x7fae1403f7c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fae20034000 tx=0x7fae20004730 comp rx=0 tx=0).stop 2024-07-04T17:27:05.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:05.819+0000 7fae37e00640 1 -- 172.21.15.42:0/4052324136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae3806da60 msgr2=0x7fae3813f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:05.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:05.819+0000 7fae37e00640 1 --2- 172.21.15.42:0/4052324136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae3806da60 0x7fae3813f3a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fae2800c320 tx=0x7fae2800c7e0 comp rx=0 tx=0).stop 2024-07-04T17:27:05.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:05.819+0000 7fae37e00640 1 -- 172.21.15.42:0/4052324136 shutdown_connections 2024-07-04T17:27:05.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:05.819+0000 7fae37e00640 1 --2- 172.21.15.42:0/4052324136 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fae1403d2b0 0x7fae1403f7c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:05.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:05.819+0000 7fae37e00640 1 --2- 172.21.15.42:0/4052324136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae3806da60 0x7fae3813f3a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:05.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:05.819+0000 7fae37e00640 1 -- 172.21.15.42:0/4052324136 >> 172.21.15.42:0/4052324136 conn(0x7fae380afaa0 msgr2=0x7fae380aebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:05.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:05.820+0000 7fae37e00640 1 -- 172.21.15.42:0/4052324136 shutdown_connections 2024-07-04T17:27:05.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:05.820+0000 7fae37e00640 1 -- 172.21.15.42:0/4052324136 wait complete. 2024-07-04T17:27:05.860 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-04T17:27:06.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.142+0000 7fc92fe00640 1 -- 172.21.15.42:0/789640228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc930070470 msgr2=0x7fc930073bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:06.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.142+0000 7fc92fe00640 1 --2- 172.21.15.42:0/789640228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc930070470 0x7fc930073bd0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc918005fd0 tx=0x7fc91802a8b0 comp rx=0 tx=0).stop 2024-07-04T17:27:06.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.142+0000 7fc92fe00640 1 -- 172.21.15.42:0/789640228 shutdown_connections 2024-07-04T17:27:06.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.142+0000 7fc92fe00640 1 --2- 172.21.15.42:0/789640228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc930070470 0x7fc930073bd0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:06.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.142+0000 7fc92fe00640 1 -- 172.21.15.42:0/789640228 >> 172.21.15.42:0/789640228 conn(0x7fc93006c020 msgr2=0x7fc93006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:06.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.142+0000 7fc92fe00640 1 -- 172.21.15.42:0/789640228 shutdown_connections 2024-07-04T17:27:06.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.142+0000 7fc92fe00640 1 -- 172.21.15.42:0/789640228 wait complete. 2024-07-04T17:27:06.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.144+0000 7fc92fe00640 1 Processor -- start 2024-07-04T17:27:06.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.145+0000 7fc92fe00640 1 -- start start 2024-07-04T17:27:06.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.146+0000 7fc92fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc930070470 0x7fc9301d6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:06.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.146+0000 7fc92fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc9300aac40 con 0x7fc930070470 2024-07-04T17:27:06.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.146+0000 7fc92ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc930070470 0x7fc9301d6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:06.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.146+0000 7fc92ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc930070470 0x7fc9301d6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38330/0 (socket says 172.21.15.42:38330) 2024-07-04T17:27:06.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.146+0000 7fc92ea00640 1 -- 172.21.15.42:0/3769995515 learned_addr learned my addr 172.21.15.42:0/3769995515 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:06.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.147+0000 7fc92ea00640 1 -- 172.21.15.42:0/3769995515 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc930072cf0 con 0x7fc930070470 2024-07-04T17:27:06.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.147+0000 7fc92ea00640 1 --2- 172.21.15.42:0/3769995515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc930070470 0x7fc9301d6b90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc92000c6b0 tx=0x7fc92000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:06.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.147+0000 7fc926a00640 1 -- 172.21.15.42:0/3769995515 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc920016020 con 0x7fc930070470 2024-07-04T17:27:06.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.147+0000 7fc926a00640 1 -- 172.21.15.42:0/3769995515 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc92000a5c0 con 0x7fc930070470 2024-07-04T17:27:06.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.147+0000 7fc92fe00640 1 -- 172.21.15.42:0/3769995515 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc930072f80 con 0x7fc930070470 2024-07-04T17:27:06.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.147+0000 7fc926a00640 1 -- 172.21.15.42:0/3769995515 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc92000a960 con 0x7fc930070470 2024-07-04T17:27:06.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.147+0000 7fc92fe00640 1 -- 172.21.15.42:0/3769995515 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9300ad670 con 0x7fc930070470 2024-07-04T17:27:06.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.149+0000 7fc92fe00640 1 -- 172.21.15.42:0/3769995515 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc930141d20 con 0x7fc930070470 2024-07-04T17:27:06.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.149+0000 7fc926a00640 1 -- 172.21.15.42:0/3769995515 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fc920019020 con 0x7fc930070470 2024-07-04T17:27:06.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.149+0000 7fc926a00640 1 --2- 172.21.15.42:0/3769995515 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fc90c03d260 0x7fc90c03f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:06.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.149+0000 7fc926a00640 1 -- 172.21.15.42:0/3769995515 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 2957+0+0 (secure 0 0 0) 0x7fc9200511d0 con 0x7fc930070470 2024-07-04T17:27:06.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.150+0000 7fc92e000640 1 --2- 172.21.15.42:0/3769995515 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fc90c03d260 0x7fc90c03f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:06.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.151+0000 7fc92e000640 1 --2- 172.21.15.42:0/3769995515 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fc90c03d260 0x7fc90c03f770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc918005fd0 tx=0x7fc918004ab0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:06.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.155+0000 7fc926a00640 1 -- 172.21.15.42:0/3769995515 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fc920020020 con 0x7fc930070470 2024-07-04T17:27:06.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.372+0000 7fc92fe00640 1 -- 172.21.15.42:0/3769995515 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc9300731b0 con 0x7fc930070470 2024-07-04T17:27:06.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.819+0000 7fc926a00640 1 -- 172.21.15.42:0/3769995515 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v25) ==== 170+0+0 (secure 0 0 0) 0x7fc930070880 con 0x7fc930070470 2024-07-04T17:27:06.821 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' created 2024-07-04T17:27:06.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.823+0000 7fc92fe00640 1 -- 172.21.15.42:0/3769995515 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fc90c03d260 msgr2=0x7fc90c03f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:06.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.823+0000 7fc92fe00640 1 --2- 172.21.15.42:0/3769995515 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fc90c03d260 0x7fc90c03f770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc918005fd0 tx=0x7fc918004ab0 comp rx=0 tx=0).stop 2024-07-04T17:27:06.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.823+0000 7fc92fe00640 1 -- 172.21.15.42:0/3769995515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc930070470 msgr2=0x7fc9301d6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:06.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.823+0000 7fc92fe00640 1 --2- 172.21.15.42:0/3769995515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc930070470 0x7fc9301d6b90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc92000c6b0 tx=0x7fc92000cb70 comp rx=0 tx=0).stop 2024-07-04T17:27:06.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.823+0000 7fc92fe00640 1 -- 172.21.15.42:0/3769995515 shutdown_connections 2024-07-04T17:27:06.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.823+0000 7fc92fe00640 1 --2- 172.21.15.42:0/3769995515 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fc90c03d260 0x7fc90c03f770 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:06.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.823+0000 7fc92fe00640 1 --2- 172.21.15.42:0/3769995515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc930070470 0x7fc9301d6b90 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:06.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.823+0000 7fc92fe00640 1 -- 172.21.15.42:0/3769995515 >> 172.21.15.42:0/3769995515 conn(0x7fc93006c020 msgr2=0x7fc9300b0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:06.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.823+0000 7fc92fe00640 1 -- 172.21.15.42:0/3769995515 shutdown_connections 2024-07-04T17:27:06.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:06.824+0000 7fc92fe00640 1 -- 172.21.15.42:0/3769995515 wait complete. 2024-07-04T17:27:06.868 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-04T17:27:07.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.134+0000 7fceab200640 1 -- 172.21.15.42:0/1909223508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fceac0701d0 msgr2=0x7fceac0abe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:07.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.134+0000 7fceab200640 1 --2- 172.21.15.42:0/1909223508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fceac0701d0 0x7fceac0abe40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fce9c0097b0 tx=0x7fce9c02a530 comp rx=0 tx=0).stop 2024-07-04T17:27:07.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.134+0000 7fceab200640 1 -- 172.21.15.42:0/1909223508 shutdown_connections 2024-07-04T17:27:07.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.134+0000 7fceab200640 1 --2- 172.21.15.42:0/1909223508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fceac0701d0 0x7fceac0abe40 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:07.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.134+0000 7fceab200640 1 -- 172.21.15.42:0/1909223508 >> 172.21.15.42:0/1909223508 conn(0x7fceac0afaa0 msgr2=0x7fceac0afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:07.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.134+0000 7fceab200640 1 -- 172.21.15.42:0/1909223508 shutdown_connections 2024-07-04T17:27:07.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.135+0000 7fceab200640 1 -- 172.21.15.42:0/1909223508 wait complete. 2024-07-04T17:27:07.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.137+0000 7fceab200640 1 Processor -- start 2024-07-04T17:27:07.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.137+0000 7fceab200640 1 -- start start 2024-07-04T17:27:07.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.138+0000 7fceab200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fceac0701d0 0x7fceac0ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:07.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.138+0000 7fceab200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fceac0ab190 con 0x7fceac0701d0 2024-07-04T17:27:07.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.139+0000 7fcea9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fceac0701d0 0x7fceac0ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:07.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.139+0000 7fcea9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fceac0701d0 0x7fceac0ad7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38360/0 (socket says 172.21.15.42:38360) 2024-07-04T17:27:07.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.139+0000 7fcea9e00640 1 -- 172.21.15.42:0/3469118837 learned_addr learned my addr 172.21.15.42:0/3469118837 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:07.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.139+0000 7fcea9e00640 1 -- 172.21.15.42:0/3469118837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceac14b6f0 con 0x7fceac0701d0 2024-07-04T17:27:07.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.139+0000 7fcea9e00640 1 --2- 172.21.15.42:0/3469118837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fceac0701d0 0x7fceac0ad7c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fce9c0098d0 tx=0x7fce9c033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:07.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.140+0000 7fcea2000640 1 -- 172.21.15.42:0/3469118837 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce9c031a90 con 0x7fceac0701d0 2024-07-04T17:27:07.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.140+0000 7fcea2000640 1 -- 172.21.15.42:0/3469118837 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce9c031d20 con 0x7fceac0701d0 2024-07-04T17:27:07.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.140+0000 7fcea2000640 1 -- 172.21.15.42:0/3469118837 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce9c033200 con 0x7fceac0701d0 2024-07-04T17:27:07.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.140+0000 7fceab200640 1 -- 172.21.15.42:0/3469118837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceac14ac70 con 0x7fceac0701d0 2024-07-04T17:27:07.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.140+0000 7fceab200640 1 -- 172.21.15.42:0/3469118837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceac14bc10 con 0x7fceac0701d0 2024-07-04T17:27:07.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.141+0000 7fceab200640 1 -- 172.21.15.42:0/3469118837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceac0abe40 con 0x7fceac0701d0 2024-07-04T17:27:07.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.142+0000 7fcea2000640 1 -- 172.21.15.42:0/3469118837 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fce9c0333a0 con 0x7fceac0701d0 2024-07-04T17:27:07.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.142+0000 7fcea2000640 1 --2- 172.21.15.42:0/3469118837 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fce8c03d2b0 0x7fce8c03f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:07.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.142+0000 7fcea2000640 1 -- 172.21.15.42:0/3469118837 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 3339+0+0 (secure 0 0 0) 0x7fce9c072860 con 0x7fceac0701d0 2024-07-04T17:27:07.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.142+0000 7fcea9400640 1 --2- 172.21.15.42:0/3469118837 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fce8c03d2b0 0x7fce8c03f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:07.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.143+0000 7fcea9400640 1 --2- 172.21.15.42:0/3469118837 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fce8c03d2b0 0x7fce8c03f7c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fce98007fd0 tx=0x7fce98007920 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:07.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.148+0000 7fcea2000640 1 -- 172.21.15.42:0/3469118837 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fce9c03f030 con 0x7fceac0701d0 2024-07-04T17:27:07.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.399+0000 7fceab200640 1 -- 172.21.15.42:0/3469118837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fceac0aa6e0 con 0x7fceac0701d0 2024-07-04T17:27:07.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.844+0000 7fcea2000640 1 -- 172.21.15.42:0/3469118837 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7fce9c04b3a0 con 0x7fceac0701d0 2024-07-04T17:27:07.850 INFO:teuthology.orchestra.run.smithi042.stderr:new fs with metadata pool 4 and data pool 5 2024-07-04T17:27:07.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.852+0000 7fceab200640 1 -- 172.21.15.42:0/3469118837 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fce8c03d2b0 msgr2=0x7fce8c03f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:07.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.852+0000 7fceab200640 1 --2- 172.21.15.42:0/3469118837 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fce8c03d2b0 0x7fce8c03f7c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fce98007fd0 tx=0x7fce98007920 comp rx=0 tx=0).stop 2024-07-04T17:27:07.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.852+0000 7fceab200640 1 -- 172.21.15.42:0/3469118837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fceac0701d0 msgr2=0x7fceac0ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:07.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.852+0000 7fceab200640 1 --2- 172.21.15.42:0/3469118837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fceac0701d0 0x7fceac0ad7c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fce9c0098d0 tx=0x7fce9c033c10 comp rx=0 tx=0).stop 2024-07-04T17:27:07.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.856+0000 7fceab200640 1 -- 172.21.15.42:0/3469118837 shutdown_connections 2024-07-04T17:27:07.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.856+0000 7fceab200640 1 --2- 172.21.15.42:0/3469118837 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fce8c03d2b0 0x7fce8c03f7c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:07.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.856+0000 7fceab200640 1 --2- 172.21.15.42:0/3469118837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fceac0701d0 0x7fceac0ad7c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:07.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.856+0000 7fceab200640 1 -- 172.21.15.42:0/3469118837 >> 172.21.15.42:0/3469118837 conn(0x7fceac0afaa0 msgr2=0x7fceac0b36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:07.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.858+0000 7fceab200640 1 -- 172.21.15.42:0/3469118837 shutdown_connections 2024-07-04T17:27:07.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:07.860+0000 7fceab200640 1 -- 172.21.15.42:0/3469118837 wait complete. 2024-07-04T17:27:07.919 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:27:08.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.168+0000 7f0f2f200640 1 -- 172.21.15.42:0/645694567 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f2806c200 msgr2=0x7f0f2814ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:08.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.168+0000 7f0f2f200640 1 --2- 172.21.15.42:0/645694567 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f2806c200 0x7f0f2814ef10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0f180097b0 tx=0x7f0f1802a4b0 comp rx=0 tx=0).stop 2024-07-04T17:27:08.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.169+0000 7f0f2f200640 1 -- 172.21.15.42:0/645694567 shutdown_connections 2024-07-04T17:27:08.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.169+0000 7f0f2f200640 1 --2- 172.21.15.42:0/645694567 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f2806c200 0x7f0f2814ef10 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:08.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.169+0000 7f0f2f200640 1 -- 172.21.15.42:0/645694567 >> 172.21.15.42:0/645694567 conn(0x7f0f280bebc0 msgr2=0x7f0f280befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:08.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.169+0000 7f0f2f200640 1 -- 172.21.15.42:0/645694567 shutdown_connections 2024-07-04T17:27:08.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.169+0000 7f0f2f200640 1 -- 172.21.15.42:0/645694567 wait complete. 2024-07-04T17:27:08.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.171+0000 7f0f2f200640 1 Processor -- start 2024-07-04T17:27:08.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.171+0000 7f0f2f200640 1 -- start start 2024-07-04T17:27:08.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.173+0000 7f0f2f200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f2806c200 0x7f0f28143fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:08.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.173+0000 7f0f2f200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2814ae50 con 0x7f0f2806c200 2024-07-04T17:27:08.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.173+0000 7f0f2de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f2806c200 0x7f0f28143fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:08.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.173+0000 7f0f2de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f2806c200 0x7f0f28143fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38376/0 (socket says 172.21.15.42:38376) 2024-07-04T17:27:08.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.173+0000 7f0f2de00640 1 -- 172.21.15.42:0/2155080677 learned_addr learned my addr 172.21.15.42:0/2155080677 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:08.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.174+0000 7f0f2de00640 1 -- 172.21.15.42:0/2155080677 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f28142500 con 0x7f0f2806c200 2024-07-04T17:27:08.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.174+0000 7f0f2de00640 1 --2- 172.21.15.42:0/2155080677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f2806c200 0x7f0f28143fc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0f180098d0 tx=0x7f0f18033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:08.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.175+0000 7f0f22000640 1 -- 172.21.15.42:0/2155080677 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f18031bc0 con 0x7f0f2806c200 2024-07-04T17:27:08.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.175+0000 7f0f22000640 1 -- 172.21.15.42:0/2155080677 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f18031e50 con 0x7f0f2806c200 2024-07-04T17:27:08.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.175+0000 7f0f22000640 1 -- 172.21.15.42:0/2155080677 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f18033260 con 0x7f0f2806c200 2024-07-04T17:27:08.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.175+0000 7f0f2f200640 1 -- 172.21.15.42:0/2155080677 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f281d6200 con 0x7f0f2806c200 2024-07-04T17:27:08.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.175+0000 7f0f2f200640 1 -- 172.21.15.42:0/2155080677 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f281d50c0 con 0x7f0f2806c200 2024-07-04T17:27:08.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.176+0000 7f0f2f200640 1 -- 172.21.15.42:0/2155080677 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f2814b490 con 0x7f0f2806c200 2024-07-04T17:27:08.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.176+0000 7f0f22000640 1 -- 172.21.15.42:0/2155080677 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f0f18033400 con 0x7f0f2806c200 2024-07-04T17:27:08.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.177+0000 7f0f22000640 1 --2- 172.21.15.42:0/2155080677 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f0f0c03d2b0 0x7f0f0c03f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:08.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.177+0000 7f0f22000640 1 -- 172.21.15.42:0/2155080677 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 3439+0+0 (secure 0 0 0) 0x7f0f18072a30 con 0x7f0f2806c200 2024-07-04T17:27:08.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.177+0000 7f0f2d400640 1 --2- 172.21.15.42:0/2155080677 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f0f0c03d2b0 0x7f0f0c03f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:08.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.178+0000 7f0f2d400640 1 --2- 172.21.15.42:0/2155080677 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f0f0c03d2b0 0x7f0f0c03f7c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0f280bac80 tx=0x7f0f1c007a90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:08.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.182+0000 7f0f22000640 1 -- 172.21.15.42:0/2155080677 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f0f1803f030 con 0x7f0f2806c200 2024-07-04T17:27:08.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.386+0000 7f0f2f200640 1 -- 172.21.15.42:0/2155080677 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f28068e30 con 0x7f0f2806c200 2024-07-04T17:27:08.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.387+0000 7f0f22000640 1 -- 172.21.15.42:0/2155080677 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+10616 (secure 0 0 0) 0x7f0f1803a3a0 con 0x7f0f2806c200 2024-07-04T17:27:08.389 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:27:08.389 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":26,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:27:07.402089+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:27:04.932839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-07-04T17:27:06.393713+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.0800000429153442,"score_stable":1.0800000429153442,"optimal_score":1,"raw_score_acting":1.0800000429153442,"raw_score_stable":1.0800000429153442,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:27:08.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.389+0000 7f0f2f200640 1 -- 172.21.15.42:0/2155080677 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f0f0c03d2b0 msgr2=0x7f0f0c03f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:08.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.389+0000 7f0f2f200640 1 --2- 172.21.15.42:0/2155080677 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f0f0c03d2b0 0x7f0f0c03f7c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0f280bac80 tx=0x7f0f1c007a90 comp rx=0 tx=0).stop 2024-07-04T17:27:08.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.389+0000 7f0f2f200640 1 -- 172.21.15.42:0/2155080677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f2806c200 msgr2=0x7f0f28143fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:08.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.389+0000 7f0f2f200640 1 --2- 172.21.15.42:0/2155080677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f2806c200 0x7f0f28143fc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0f180098d0 tx=0x7f0f18033e40 comp rx=0 tx=0).stop 2024-07-04T17:27:08.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.390+0000 7f0f2f200640 1 -- 172.21.15.42:0/2155080677 shutdown_connections 2024-07-04T17:27:08.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.390+0000 7f0f2f200640 1 --2- 172.21.15.42:0/2155080677 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f0f0c03d2b0 0x7f0f0c03f7c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:08.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.390+0000 7f0f2f200640 1 --2- 172.21.15.42:0/2155080677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f2806c200 0x7f0f28143fc0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:08.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.390+0000 7f0f2f200640 1 -- 172.21.15.42:0/2155080677 >> 172.21.15.42:0/2155080677 conn(0x7f0f280bebc0 msgr2=0x7f0f280bbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:08.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.390+0000 7f0f2f200640 1 -- 172.21.15.42:0/2155080677 shutdown_connections 2024-07-04T17:27:08.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.390+0000 7f0f2f200640 1 -- 172.21.15.42:0/2155080677 wait complete. 2024-07-04T17:27:08.415 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:27:08.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.692+0000 7f20df800640 1 -- 172.21.15.42:0/1852557544 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20c0003a50 msgr2=0x7f20c0003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:08.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.692+0000 7f20df800640 1 --2- 172.21.15.42:0/1852557544 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20c0003a50 0x7f20c0003ea0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f20cc006f70 tx=0x7f20c0005830 comp rx=0 tx=0).stop 2024-07-04T17:27:08.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.693+0000 7f20df800640 1 -- 172.21.15.42:0/1852557544 shutdown_connections 2024-07-04T17:27:08.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.693+0000 7f20df800640 1 --2- 172.21.15.42:0/1852557544 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20c0003a50 0x7f20c0003ea0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:08.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.693+0000 7f20df800640 1 --2- 172.21.15.42:0/1852557544 >> v2:172.21.15.42:3300/0 conn(0x7f20d80b3030 0x7f20d80b01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:08.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.693+0000 7f20df800640 1 -- 172.21.15.42:0/1852557544 >> 172.21.15.42:0/1852557544 conn(0x7f20d80bf700 msgr2=0x7f20d80af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:08.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.693+0000 7f20df800640 1 -- 172.21.15.42:0/1852557544 shutdown_connections 2024-07-04T17:27:08.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.693+0000 7f20df800640 1 -- 172.21.15.42:0/1852557544 wait complete. 2024-07-04T17:27:08.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.695+0000 7f20df800640 1 Processor -- start 2024-07-04T17:27:08.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.695+0000 7f20df800640 1 -- start start 2024-07-04T17:27:08.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.696+0000 7f20df800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20c0003a50 0x7f20d80b77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:08.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.696+0000 7f20df800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f20d806ef50 con 0x7f20c0003a50 2024-07-04T17:27:08.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.697+0000 7f20dce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20c0003a50 0x7f20d80b77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:08.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.697+0000 7f20dce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20c0003a50 0x7f20d80b77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38394/0 (socket says 172.21.15.42:38394) 2024-07-04T17:27:08.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.697+0000 7f20dce00640 1 -- 172.21.15.42:0/827484543 learned_addr learned my addr 172.21.15.42:0/827484543 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:08.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.697+0000 7f20dce00640 1 -- 172.21.15.42:0/827484543 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d80b7dd0 con 0x7f20c0003a50 2024-07-04T17:27:08.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.697+0000 7f20dce00640 1 --2- 172.21.15.42:0/827484543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20c0003a50 0x7f20d80b77d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f20cc000c00 tx=0x7f20cc0016e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:08.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.698+0000 7f20d4c00640 1 -- 172.21.15.42:0/827484543 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20cc042070 con 0x7f20c0003a50 2024-07-04T17:27:08.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.698+0000 7f20d4c00640 1 -- 172.21.15.42:0/827484543 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20cc03a070 con 0x7f20c0003a50 2024-07-04T17:27:08.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.698+0000 7f20d4c00640 1 -- 172.21.15.42:0/827484543 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20cc003990 con 0x7f20c0003a50 2024-07-04T17:27:08.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.698+0000 7f20df800640 1 -- 172.21.15.42:0/827484543 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d80b5640 con 0x7f20c0003a50 2024-07-04T17:27:08.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.698+0000 7f20df800640 1 -- 172.21.15.42:0/827484543 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d80b5f00 con 0x7f20c0003a50 2024-07-04T17:27:08.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.699+0000 7f20df800640 1 -- 172.21.15.42:0/827484543 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20d80b01f0 con 0x7f20c0003a50 2024-07-04T17:27:08.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.699+0000 7f20d4c00640 1 -- 172.21.15.42:0/827484543 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f20cc045070 con 0x7f20c0003a50 2024-07-04T17:27:08.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.699+0000 7f20d4c00640 1 --2- 172.21.15.42:0/827484543 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f20b803d2b0 0x7f20b803f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:08.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.700+0000 7f20d4c00640 1 -- 172.21.15.42:0/827484543 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 3439+0+0 (secure 0 0 0) 0x7f20cc071120 con 0x7f20c0003a50 2024-07-04T17:27:08.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.700+0000 7f20d7e00640 1 --2- 172.21.15.42:0/827484543 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f20b803d2b0 0x7f20b803f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:08.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.700+0000 7f20d7e00640 1 --2- 172.21.15.42:0/827484543 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f20b803d2b0 0x7f20b803f7c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f20c4005eb0 tx=0x7f20c4005f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:08.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.704+0000 7f20d4c00640 1 -- 172.21.15.42:0/827484543 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f20cc003dc0 con 0x7f20c0003a50 2024-07-04T17:27:08.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.912+0000 7f20df800640 1 -- 172.21.15.42:0/827484543 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f20d80b1f10 con 0x7f20c0003a50 2024-07-04T17:27:08.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.913+0000 7f20d4c00640 1 -- 172.21.15.42:0/827484543 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+10603 (secure 0 0 0) 0x7f20cc02fe00 con 0x7f20c0003a50 2024-07-04T17:27:08.915 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:27:08.915 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:27:08.845516+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:27:04.932839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-07-04T17:27:06.393713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.0800000429153442,"score_stable":1.0800000429153442,"optimal_score":1,"raw_score_acting":1.0800000429153442,"raw_score_stable":1.0800000429153442,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:27:08.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.916+0000 7f20df800640 1 -- 172.21.15.42:0/827484543 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f20b803d2b0 msgr2=0x7f20b803f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:08.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.916+0000 7f20df800640 1 --2- 172.21.15.42:0/827484543 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f20b803d2b0 0x7f20b803f7c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f20c4005eb0 tx=0x7f20c4005f90 comp rx=0 tx=0).stop 2024-07-04T17:27:08.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.916+0000 7f20df800640 1 -- 172.21.15.42:0/827484543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20c0003a50 msgr2=0x7f20d80b77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:08.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.916+0000 7f20df800640 1 --2- 172.21.15.42:0/827484543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20c0003a50 0x7f20d80b77d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f20cc000c00 tx=0x7f20cc0016e0 comp rx=0 tx=0).stop 2024-07-04T17:27:08.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.916+0000 7f20df800640 1 -- 172.21.15.42:0/827484543 shutdown_connections 2024-07-04T17:27:08.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.916+0000 7f20df800640 1 --2- 172.21.15.42:0/827484543 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f20b803d2b0 0x7f20b803f7c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:08.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.916+0000 7f20df800640 1 --2- 172.21.15.42:0/827484543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20c0003a50 0x7f20d80b77d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:08.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.916+0000 7f20df800640 1 -- 172.21.15.42:0/827484543 >> 172.21.15.42:0/827484543 conn(0x7f20d80bf700 msgr2=0x7f20d814be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:08.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.916+0000 7f20df800640 1 -- 172.21.15.42:0/827484543 shutdown_connections 2024-07-04T17:27:08.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:08.916+0000 7f20df800640 1 -- 172.21.15.42:0/827484543 wait complete. 2024-07-04T17:27:08.940 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-07-04T17:27:09.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.220+0000 7fe821600640 1 -- 172.21.15.42:0/3260015946 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe81c0701d0 msgr2=0x7fe81c0abe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:09.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.220+0000 7fe821600640 1 --2- 172.21.15.42:0/3260015946 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe81c0701d0 0x7fe81c0abe40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe80c0097b0 tx=0x7fe80c02a540 comp rx=0 tx=0).stop 2024-07-04T17:27:09.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.220+0000 7fe821600640 1 -- 172.21.15.42:0/3260015946 shutdown_connections 2024-07-04T17:27:09.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.220+0000 7fe821600640 1 --2- 172.21.15.42:0/3260015946 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe81c0701d0 0x7fe81c0abe40 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:09.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.220+0000 7fe821600640 1 -- 172.21.15.42:0/3260015946 >> 172.21.15.42:0/3260015946 conn(0x7fe81c0afaa0 msgr2=0x7fe81c0afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:09.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.220+0000 7fe821600640 1 -- 172.21.15.42:0/3260015946 shutdown_connections 2024-07-04T17:27:09.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.220+0000 7fe821600640 1 -- 172.21.15.42:0/3260015946 wait complete. 2024-07-04T17:27:09.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.222+0000 7fe821600640 1 Processor -- start 2024-07-04T17:27:09.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.222+0000 7fe821600640 1 -- start start 2024-07-04T17:27:09.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.223+0000 7fe821600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe81c0701d0 0x7fe81c0bf210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:09.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.223+0000 7fe821600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe81c0ab190 con 0x7fe81c0701d0 2024-07-04T17:27:09.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.224+0000 7fe81be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe81c0701d0 0x7fe81c0bf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:09.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.224+0000 7fe81be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe81c0701d0 0x7fe81c0bf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38406/0 (socket says 172.21.15.42:38406) 2024-07-04T17:27:09.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.224+0000 7fe81be00640 1 -- 172.21.15.42:0/2038045287 learned_addr learned my addr 172.21.15.42:0/2038045287 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:09.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.224+0000 7fe81be00640 1 -- 172.21.15.42:0/2038045287 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe81c071c30 con 0x7fe81c0701d0 2024-07-04T17:27:09.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.225+0000 7fe81be00640 1 --2- 172.21.15.42:0/2038045287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe81c0701d0 0x7fe81c0bf210 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe80c0098d0 tx=0x7fe80c033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:09.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.225+0000 7fe813e00640 1 -- 172.21.15.42:0/2038045287 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe80c031bc0 con 0x7fe81c0701d0 2024-07-04T17:27:09.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.225+0000 7fe813e00640 1 -- 172.21.15.42:0/2038045287 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe80c031e50 con 0x7fe81c0701d0 2024-07-04T17:27:09.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.225+0000 7fe821600640 1 -- 172.21.15.42:0/2038045287 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe81c144f90 con 0x7fe81c0701d0 2024-07-04T17:27:09.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.225+0000 7fe813e00640 1 -- 172.21.15.42:0/2038045287 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe80c033200 con 0x7fe81c0701d0 2024-07-04T17:27:09.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.225+0000 7fe821600640 1 -- 172.21.15.42:0/2038045287 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe81c145460 con 0x7fe81c0701d0 2024-07-04T17:27:09.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.226+0000 7fe813e00640 1 -- 172.21.15.42:0/2038045287 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fe80c0333a0 con 0x7fe81c0701d0 2024-07-04T17:27:09.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.226+0000 7fe821600640 1 -- 172.21.15.42:0/2038045287 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe81c14d9b0 con 0x7fe81c0701d0 2024-07-04T17:27:09.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.227+0000 7fe813e00640 1 --2- 172.21.15.42:0/2038045287 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fe7fc03d2b0 0x7fe7fc03f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:09.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.227+0000 7fe813e00640 1 -- 172.21.15.42:0/2038045287 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3439+0+0 (secure 0 0 0) 0x7fe80c072990 con 0x7fe81c0701d0 2024-07-04T17:27:09.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.227+0000 7fe81b400640 1 --2- 172.21.15.42:0/2038045287 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fe7fc03d2b0 0x7fe7fc03f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:09.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.228+0000 7fe81b400640 1 --2- 172.21.15.42:0/2038045287 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fe7fc03d2b0 0x7fe7fc03f7c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe8080015c0 tx=0x7fe808007a90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:09.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.231+0000 7fe813e00640 1 -- 172.21.15.42:0/2038045287 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fe81c145460 con 0x7fe81c0701d0 2024-07-04T17:27:09.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.491+0000 7fe821600640 1 -- 172.21.15.42:0/2038045287 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe81c0abe40 con 0x7fe81c0701d0 2024-07-04T17:27:09.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.859+0000 7fe813e00640 1 -- 172.21.15.42:0/2038045287 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7fe80c03f030 con 0x7fe81c0701d0 2024-07-04T17:27:09.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.862+0000 7fe821600640 1 -- 172.21.15.42:0/2038045287 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fe7fc03d2b0 msgr2=0x7fe7fc03f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:09.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.862+0000 7fe821600640 1 --2- 172.21.15.42:0/2038045287 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fe7fc03d2b0 0x7fe7fc03f7c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe8080015c0 tx=0x7fe808007a90 comp rx=0 tx=0).stop 2024-07-04T17:27:09.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.863+0000 7fe821600640 1 -- 172.21.15.42:0/2038045287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe81c0701d0 msgr2=0x7fe81c0bf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:09.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.863+0000 7fe821600640 1 --2- 172.21.15.42:0/2038045287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe81c0701d0 0x7fe81c0bf210 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe80c0098d0 tx=0x7fe80c033e40 comp rx=0 tx=0).stop 2024-07-04T17:27:09.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.863+0000 7fe821600640 1 -- 172.21.15.42:0/2038045287 shutdown_connections 2024-07-04T17:27:09.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.863+0000 7fe821600640 1 --2- 172.21.15.42:0/2038045287 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fe7fc03d2b0 0x7fe7fc03f7c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:09.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.863+0000 7fe821600640 1 --2- 172.21.15.42:0/2038045287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe81c0701d0 0x7fe81c0bf210 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:09.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.863+0000 7fe821600640 1 -- 172.21.15.42:0/2038045287 >> 172.21.15.42:0/2038045287 conn(0x7fe81c0afaa0 msgr2=0x7fe81c1d7640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:09.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.863+0000 7fe821600640 1 -- 172.21.15.42:0/2038045287 shutdown_connections 2024-07-04T17:27:09.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:09.863+0000 7fe821600640 1 -- 172.21.15.42:0/2038045287 wait complete. 2024-07-04T17:27:09.894 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-04T17:27:09.894 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-07-04T17:27:10.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.188+0000 7fd0cce00640 1 -- 172.21.15.42:0/3947172596 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0c80a99b0 msgr2=0x7fd0c806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:10.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.188+0000 7fd0cce00640 1 --2- 172.21.15.42:0/3947172596 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0c80a99b0 0x7fd0c806d110 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd0b40097b0 tx=0x7fd0b402a540 comp rx=0 tx=0).stop 2024-07-04T17:27:10.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.189+0000 7fd0cce00640 1 -- 172.21.15.42:0/3947172596 shutdown_connections 2024-07-04T17:27:10.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.189+0000 7fd0cce00640 1 --2- 172.21.15.42:0/3947172596 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0c80a99b0 0x7fd0c806d110 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:10.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.189+0000 7fd0cce00640 1 -- 172.21.15.42:0/3947172596 >> 172.21.15.42:0/3947172596 conn(0x7fd0c806c020 msgr2=0x7fd0c806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:10.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.189+0000 7fd0cce00640 1 -- 172.21.15.42:0/3947172596 shutdown_connections 2024-07-04T17:27:10.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.189+0000 7fd0cce00640 1 -- 172.21.15.42:0/3947172596 wait complete. 2024-07-04T17:27:10.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.191+0000 7fd0cce00640 1 Processor -- start 2024-07-04T17:27:10.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.191+0000 7fd0cce00640 1 -- start start 2024-07-04T17:27:10.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.192+0000 7fd0cce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0c80a99b0 0x7fd0c81e39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:10.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.192+0000 7fd0cce00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c813f430 con 0x7fd0c80a99b0 2024-07-04T17:27:10.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.193+0000 7fd0c5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0c80a99b0 0x7fd0c81e39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:10.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.193+0000 7fd0c5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0c80a99b0 0x7fd0c81e39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38420/0 (socket says 172.21.15.42:38420) 2024-07-04T17:27:10.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.193+0000 7fd0c5e00640 1 -- 172.21.15.42:0/1869918106 learned_addr learned my addr 172.21.15.42:0/1869918106 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:10.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.193+0000 7fd0c5e00640 1 -- 172.21.15.42:0/1869918106 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0c80b7320 con 0x7fd0c80a99b0 2024-07-04T17:27:10.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.194+0000 7fd0c5e00640 1 --2- 172.21.15.42:0/1869918106 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0c80a99b0 0x7fd0c81e39e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800c320 tx=0x7fd0b800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:10.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.194+0000 7fd0be000640 1 -- 172.21.15.42:0/1869918106 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0b80116c0 con 0x7fd0c80a99b0 2024-07-04T17:27:10.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.194+0000 7fd0be000640 1 -- 172.21.15.42:0/1869918106 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0b8017070 con 0x7fd0c80a99b0 2024-07-04T17:27:10.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.194+0000 7fd0be000640 1 -- 172.21.15.42:0/1869918106 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0b8011b40 con 0x7fd0c80a99b0 2024-07-04T17:27:10.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.194+0000 7fd0cce00640 1 -- 172.21.15.42:0/1869918106 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0c80b6570 con 0x7fd0c80a99b0 2024-07-04T17:27:10.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.194+0000 7fd0cce00640 1 -- 172.21.15.42:0/1869918106 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0c80b6f10 con 0x7fd0c80a99b0 2024-07-04T17:27:10.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.195+0000 7fd0cce00640 1 -- 172.21.15.42:0/1869918106 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0c813fa70 con 0x7fd0c80a99b0 2024-07-04T17:27:10.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.196+0000 7fd0be000640 1 -- 172.21.15.42:0/1869918106 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fd0b8011ce0 con 0x7fd0c80a99b0 2024-07-04T17:27:10.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.196+0000 7fd0be000640 1 --2- 172.21.15.42:0/1869918106 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fd0a803d2b0 0x7fd0a803f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:10.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.196+0000 7fd0be000640 1 -- 172.21.15.42:0/1869918106 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3439+0+0 (secure 0 0 0) 0x7fd0b8051280 con 0x7fd0c80a99b0 2024-07-04T17:27:10.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.197+0000 7fd0c5400640 1 --2- 172.21.15.42:0/1869918106 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fd0a803d2b0 0x7fd0a803f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:10.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.197+0000 7fd0c5400640 1 --2- 172.21.15.42:0/1869918106 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fd0a803d2b0 0x7fd0a803f7c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4034000 tx=0x7fd0b4004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:10.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.201+0000 7fd0be000640 1 -- 172.21.15.42:0/1869918106 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fd0b801ac80 con 0x7fd0c80a99b0 2024-07-04T17:27:10.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.460+0000 7fd0cce00640 1 -- 172.21.15.42:0/1869918106 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd0c81449c0 con 0x7fd0c80a99b0 2024-07-04T17:27:10.558 INFO:tasks.ceph.mds.b.smithi042.stderr:2024-07-04T17:27:10.557+0000 7f77ef200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-04T17:27:10.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.868+0000 7fd0be000640 1 -- 172.21.15.42:0/1869918106 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7fd0b80102e0 con 0x7fd0c80a99b0 2024-07-04T17:27:10.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.873+0000 7fd0cce00640 1 -- 172.21.15.42:0/1869918106 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fd0a803d2b0 msgr2=0x7fd0a803f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:10.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.873+0000 7fd0cce00640 1 --2- 172.21.15.42:0/1869918106 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fd0a803d2b0 0x7fd0a803f7c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4034000 tx=0x7fd0b4004730 comp rx=0 tx=0).stop 2024-07-04T17:27:10.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.873+0000 7fd0cce00640 1 -- 172.21.15.42:0/1869918106 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0c80a99b0 msgr2=0x7fd0c81e39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:10.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.873+0000 7fd0cce00640 1 --2- 172.21.15.42:0/1869918106 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0c80a99b0 0x7fd0c81e39e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800c320 tx=0x7fd0b800c7e0 comp rx=0 tx=0).stop 2024-07-04T17:27:10.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.873+0000 7fd0cce00640 1 -- 172.21.15.42:0/1869918106 shutdown_connections 2024-07-04T17:27:10.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.873+0000 7fd0cce00640 1 --2- 172.21.15.42:0/1869918106 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fd0a803d2b0 0x7fd0a803f7c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:10.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.873+0000 7fd0cce00640 1 --2- 172.21.15.42:0/1869918106 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0c80a99b0 0x7fd0c81e39e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:10.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.873+0000 7fd0cce00640 1 -- 172.21.15.42:0/1869918106 >> 172.21.15.42:0/1869918106 conn(0x7fd0c806c020 msgr2=0x7fd0c806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:10.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.874+0000 7fd0cce00640 1 -- 172.21.15.42:0/1869918106 shutdown_connections 2024-07-04T17:27:10.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:10.874+0000 7fd0cce00640 1 -- 172.21.15.42:0/1869918106 wait complete. 2024-07-04T17:27:10.906 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-07-04T17:27:10.906 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:27:11.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.192+0000 7f240bc00640 1 -- 172.21.15.42:0/23697043 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24040b37b0 msgr2=0x7f24040b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:11.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.192+0000 7f240bc00640 1 --2- 172.21.15.42:0/23697043 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24040b37b0 0x7f24040b01f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f23f00097b0 tx=0x7f23f002a540 comp rx=0 tx=0).stop 2024-07-04T17:27:11.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.192+0000 7f240bc00640 1 -- 172.21.15.42:0/23697043 shutdown_connections 2024-07-04T17:27:11.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.192+0000 7f240bc00640 1 --2- 172.21.15.42:0/23697043 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24040b37b0 0x7f24040b01f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:11.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.192+0000 7f240bc00640 1 -- 172.21.15.42:0/23697043 >> 172.21.15.42:0/23697043 conn(0x7f24040bf700 msgr2=0x7f24040af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:11.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.192+0000 7f240bc00640 1 -- 172.21.15.42:0/23697043 shutdown_connections 2024-07-04T17:27:11.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.192+0000 7f240bc00640 1 -- 172.21.15.42:0/23697043 wait complete. 2024-07-04T17:27:11.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.194+0000 7f240bc00640 1 Processor -- start 2024-07-04T17:27:11.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.195+0000 7f240bc00640 1 -- start start 2024-07-04T17:27:11.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.196+0000 7f240bc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24040b2e00 0x7f24040ae970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:11.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.196+0000 7f240bc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f240406ce30 con 0x7f24040b2e00 2024-07-04T17:27:11.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.196+0000 7f2409200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24040b2e00 0x7f24040ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:11.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.196+0000 7f2409200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24040b2e00 0x7f24040ae970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38428/0 (socket says 172.21.15.42:38428) 2024-07-04T17:27:11.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.196+0000 7f2409200640 1 -- 172.21.15.42:0/2743308623 learned_addr learned my addr 172.21.15.42:0/2743308623 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:11.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.197+0000 7f2409200640 1 -- 172.21.15.42:0/2743308623 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2404072280 con 0x7f24040b2e00 2024-07-04T17:27:11.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.197+0000 7f2409200640 1 --2- 172.21.15.42:0/2743308623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24040b2e00 0x7f24040ae970 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f23f800c320 tx=0x7f23f800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:11.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.197+0000 7f2400c00640 1 -- 172.21.15.42:0/2743308623 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23f80116c0 con 0x7f24040b2e00 2024-07-04T17:27:11.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.197+0000 7f2400c00640 1 -- 172.21.15.42:0/2743308623 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23f8017070 con 0x7f24040b2e00 2024-07-04T17:27:11.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.197+0000 7f2400c00640 1 -- 172.21.15.42:0/2743308623 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23f8011b00 con 0x7f24040b2e00 2024-07-04T17:27:11.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.197+0000 7f240bc00640 1 -- 172.21.15.42:0/2743308623 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24040737c0 con 0x7f24040b2e00 2024-07-04T17:27:11.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.197+0000 7f240bc00640 1 -- 172.21.15.42:0/2743308623 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2404072060 con 0x7f24040b2e00 2024-07-04T17:27:11.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.199+0000 7f2400c00640 1 -- 172.21.15.42:0/2743308623 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f23f8011ca0 con 0x7f24040b2e00 2024-07-04T17:27:11.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.199+0000 7f240bc00640 1 -- 172.21.15.42:0/2743308623 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f240406d440 con 0x7f24040b2e00 2024-07-04T17:27:11.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.199+0000 7f2400c00640 1 --2- 172.21.15.42:0/2743308623 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f23e403d2b0 0x7f23e403f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:11.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.199+0000 7f2400c00640 1 -- 172.21.15.42:0/2743308623 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3439+0+0 (secure 0 0 0) 0x7f23f80511e0 con 0x7f24040b2e00 2024-07-04T17:27:11.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.200+0000 7f2403e00640 1 --2- 172.21.15.42:0/2743308623 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f23e403d2b0 0x7f23e403f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:11.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.200+0000 7f2403e00640 1 --2- 172.21.15.42:0/2743308623 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f23e403d2b0 0x7f23e403f7c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f23f0034000 tx=0x7f23f0004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:11.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.204+0000 7f2400c00640 1 -- 172.21.15.42:0/2743308623 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f23f800a140 con 0x7f24040b2e00 2024-07-04T17:27:11.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.453+0000 7f240bc00640 1 -- 172.21.15.42:0/2743308623 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24040739f0 con 0x7f24040b2e00 2024-07-04T17:27:11.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.454+0000 7f2400c00640 1 -- 172.21.15.42:0/2743308623 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6042 (secure 0 0 0) 0x7f23f8010cf0 con 0x7f24040b2e00 2024-07-04T17:27:11.456 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:27:11.456 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":17,"btime":"2024-07-04T17:27:10:861234+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/399707138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":399707138},{"type":"v1","addr":"172.21.15.42:6827","nonce":399707138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4414,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1510714231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1510714231},{"type":"v1","addr":"172.21.15.42:6829","nonce":1510714231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/3069680926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":3069680926},{"type":"v1","addr":"172.21.15.42:6831","nonce":3069680926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4422,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6833/1348446958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1348446958},{"type":"v1","addr":"172.21.15.42:6833","nonce":1348446958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:27:07.402007+0000","modified":"2024-07-04T17:27:10.462342+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4423":{"gid":4423,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.42:6835/807364821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":807364821},{"type":"v1","addr":"172.21.15.42:6835","nonce":807364821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4423,"qdb_cluster":[4423]},"id":2}]} 2024-07-04T17:27:11.456 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 17 2024-07-04T17:27:11.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.457+0000 7f240bc00640 1 -- 172.21.15.42:0/2743308623 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f23e403d2b0 msgr2=0x7f23e403f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:11.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.457+0000 7f240bc00640 1 --2- 172.21.15.42:0/2743308623 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f23e403d2b0 0x7f23e403f7c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f23f0034000 tx=0x7f23f0004730 comp rx=0 tx=0).stop 2024-07-04T17:27:11.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.457+0000 7f240bc00640 1 -- 172.21.15.42:0/2743308623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24040b2e00 msgr2=0x7f24040ae970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:11.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.457+0000 7f240bc00640 1 --2- 172.21.15.42:0/2743308623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24040b2e00 0x7f24040ae970 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f23f800c320 tx=0x7f23f800c7e0 comp rx=0 tx=0).stop 2024-07-04T17:27:11.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.457+0000 7f240bc00640 1 -- 172.21.15.42:0/2743308623 shutdown_connections 2024-07-04T17:27:11.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.457+0000 7f240bc00640 1 --2- 172.21.15.42:0/2743308623 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f23e403d2b0 0x7f23e403f7c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:11.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.457+0000 7f240bc00640 1 --2- 172.21.15.42:0/2743308623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f24040b2e00 0x7f24040ae970 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:11.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.457+0000 7f240bc00640 1 -- 172.21.15.42:0/2743308623 >> 172.21.15.42:0/2743308623 conn(0x7f24040bf700 msgr2=0x7f24040b1180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:11.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.457+0000 7f240bc00640 1 -- 172.21.15.42:0/2743308623 shutdown_connections 2024-07-04T17:27:11.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.457+0000 7f240bc00640 1 -- 172.21.15.42:0/2743308623 wait complete. 2024-07-04T17:27:11.484 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:27:11.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.760+0000 7fb307e00640 1 -- 172.21.15.42:0/2849416357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb30806c020 msgr2=0x7fb30814a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:11.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.760+0000 7fb307e00640 1 --2- 172.21.15.42:0/2849416357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb30806c020 0x7fb30814a9e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4005fd0 tx=0x7fb2f402a560 comp rx=0 tx=0).stop 2024-07-04T17:27:11.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.760+0000 7fb307e00640 1 -- 172.21.15.42:0/2849416357 shutdown_connections 2024-07-04T17:27:11.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.760+0000 7fb307e00640 1 --2- 172.21.15.42:0/2849416357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb30806c020 0x7fb30814a9e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:11.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.760+0000 7fb307e00640 1 -- 172.21.15.42:0/2849416357 >> 172.21.15.42:0/2849416357 conn(0x7fb3080bebc0 msgr2=0x7fb3080befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:11.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.761+0000 7fb307e00640 1 -- 172.21.15.42:0/2849416357 shutdown_connections 2024-07-04T17:27:11.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.761+0000 7fb307e00640 1 -- 172.21.15.42:0/2849416357 wait complete. 2024-07-04T17:27:11.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.763+0000 7fb307e00640 1 Processor -- start 2024-07-04T17:27:11.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.763+0000 7fb307e00640 1 -- start start 2024-07-04T17:27:11.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.764+0000 7fb307e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb30806c020 0x7fb308149590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:11.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.764+0000 7fb307e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb30814b500 con 0x7fb30806c020 2024-07-04T17:27:11.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.764+0000 7fb306a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb30806c020 0x7fb308149590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:11.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.764+0000 7fb306a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb30806c020 0x7fb308149590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38446/0 (socket says 172.21.15.42:38446) 2024-07-04T17:27:11.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.764+0000 7fb306a00640 1 -- 172.21.15.42:0/4123830706 learned_addr learned my addr 172.21.15.42:0/4123830706 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:11.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.765+0000 7fb306a00640 1 -- 172.21.15.42:0/4123830706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3080ae290 con 0x7fb30806c020 2024-07-04T17:27:11.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.765+0000 7fb306a00640 1 --2- 172.21.15.42:0/4123830706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb30806c020 0x7fb308149590 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8001eb0 tx=0x7fb2f800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:11.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.765+0000 7fb2fea00640 1 -- 172.21.15.42:0/4123830706 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2f8017070 con 0x7fb30806c020 2024-07-04T17:27:11.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.766+0000 7fb2fea00640 1 -- 172.21.15.42:0/4123830706 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2f80052f0 con 0x7fb30806c020 2024-07-04T17:27:11.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.766+0000 7fb307e00640 1 -- 172.21.15.42:0/4123830706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3081e7790 con 0x7fb30806c020 2024-07-04T17:27:11.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.766+0000 7fb2fea00640 1 -- 172.21.15.42:0/4123830706 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2f80035f0 con 0x7fb30806c020 2024-07-04T17:27:11.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.766+0000 7fb307e00640 1 -- 172.21.15.42:0/4123830706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3080adda0 con 0x7fb30806c020 2024-07-04T17:27:11.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.767+0000 7fb307e00640 1 -- 172.21.15.42:0/4123830706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb30814bb40 con 0x7fb30806c020 2024-07-04T17:27:11.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.767+0000 7fb2fea00640 1 -- 172.21.15.42:0/4123830706 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fb2f801a070 con 0x7fb30806c020 2024-07-04T17:27:11.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.767+0000 7fb2fea00640 1 --2- 172.21.15.42:0/4123830706 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb2e803d2b0 0x7fb2e803f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:11.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.767+0000 7fb2fea00640 1 -- 172.21.15.42:0/4123830706 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3439+0+0 (secure 0 0 0) 0x7fb2f8050810 con 0x7fb30806c020 2024-07-04T17:27:11.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.768+0000 7fb306000640 1 --2- 172.21.15.42:0/4123830706 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb2e803d2b0 0x7fb2e803f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:11.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.769+0000 7fb306000640 1 --2- 172.21.15.42:0/4123830706 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb2e803d2b0 0x7fb2e803f7c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4005fd0 tx=0x7fb2f4004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:11.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.773+0000 7fb2fea00640 1 -- 172.21.15.42:0/4123830706 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fb2f8011630 con 0x7fb30806c020 2024-07-04T17:27:11.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.978+0000 7fb307e00640 1 -- 172.21.15.42:0/4123830706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb30814a9c0 con 0x7fb30806c020 2024-07-04T17:27:11.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.979+0000 7fb2fea00640 1 -- 172.21.15.42:0/4123830706 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+10603 (secure 0 0 0) 0x7fb2f8050bc0 con 0x7fb30806c020 2024-07-04T17:27:11.980 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:27:11.981 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:27:08.845516+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:27:04.932839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-07-04T17:27:06.393713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.0800000429153442,"score_stable":1.0800000429153442,"optimal_score":1,"raw_score_acting":1.0800000429153442,"raw_score_stable":1.0800000429153442,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:27:11.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.982+0000 7fb307e00640 1 -- 172.21.15.42:0/4123830706 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb2e803d2b0 msgr2=0x7fb2e803f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:11.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.982+0000 7fb307e00640 1 --2- 172.21.15.42:0/4123830706 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb2e803d2b0 0x7fb2e803f7c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4005fd0 tx=0x7fb2f4004730 comp rx=0 tx=0).stop 2024-07-04T17:27:11.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.982+0000 7fb307e00640 1 -- 172.21.15.42:0/4123830706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb30806c020 msgr2=0x7fb308149590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:11.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.982+0000 7fb307e00640 1 --2- 172.21.15.42:0/4123830706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb30806c020 0x7fb308149590 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8001eb0 tx=0x7fb2f800c810 comp rx=0 tx=0).stop 2024-07-04T17:27:11.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.982+0000 7fb307e00640 1 -- 172.21.15.42:0/4123830706 shutdown_connections 2024-07-04T17:27:11.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.982+0000 7fb307e00640 1 --2- 172.21.15.42:0/4123830706 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb2e803d2b0 0x7fb2e803f7c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:11.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.982+0000 7fb307e00640 1 --2- 172.21.15.42:0/4123830706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb30806c020 0x7fb308149590 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:11.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.982+0000 7fb307e00640 1 -- 172.21.15.42:0/4123830706 >> 172.21.15.42:0/4123830706 conn(0x7fb3080bebc0 msgr2=0x7fb3080befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:11.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.982+0000 7fb307e00640 1 -- 172.21.15.42:0/4123830706 shutdown_connections 2024-07-04T17:27:11.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:11.982+0000 7fb307e00640 1 -- 172.21.15.42:0/4123830706 wait complete. 2024-07-04T17:27:12.007 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-04T17:27:12.007 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:27:12.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.284+0000 7f2cc7200640 1 -- 172.21.15.42:0/2110607359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cc8144fe0 msgr2=0x7f2cc806c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:12.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.284+0000 7f2cc7200640 1 --2- 172.21.15.42:0/2110607359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cc8144fe0 0x7f2cc806c020 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2cb00097b0 tx=0x7f2cb002a540 comp rx=0 tx=0).stop 2024-07-04T17:27:12.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.284+0000 7f2cc7200640 1 -- 172.21.15.42:0/2110607359 shutdown_connections 2024-07-04T17:27:12.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.284+0000 7f2cc7200640 1 --2- 172.21.15.42:0/2110607359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cc8144fe0 0x7f2cc806c020 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:12.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.284+0000 7f2cc7200640 1 -- 172.21.15.42:0/2110607359 >> 172.21.15.42:0/2110607359 conn(0x7f2cc80bebc0 msgr2=0x7f2cc80befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:12.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.284+0000 7f2cc7200640 1 -- 172.21.15.42:0/2110607359 shutdown_connections 2024-07-04T17:27:12.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.284+0000 7f2cc7200640 1 -- 172.21.15.42:0/2110607359 wait complete. 2024-07-04T17:27:12.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.286+0000 7f2cc7200640 1 Processor -- start 2024-07-04T17:27:12.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.286+0000 7f2cc7200640 1 -- start start 2024-07-04T17:27:12.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.287+0000 7f2cc7200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cc80ba830 0x7f2cc8072db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:12.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.287+0000 7f2cc7200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc80b1e30 con 0x7f2cc80ba830 2024-07-04T17:27:12.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.287+0000 7f2cc5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cc80ba830 0x7f2cc8072db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:12.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.287+0000 7f2cc5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cc80ba830 0x7f2cc8072db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38464/0 (socket says 172.21.15.42:38464) 2024-07-04T17:27:12.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.287+0000 7f2cc5e00640 1 -- 172.21.15.42:0/408716251 learned_addr learned my addr 172.21.15.42:0/408716251 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:12.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.288+0000 7f2cc5e00640 1 -- 172.21.15.42:0/408716251 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc813e220 con 0x7f2cc80ba830 2024-07-04T17:27:12.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.288+0000 7f2cc5e00640 1 --2- 172.21.15.42:0/408716251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cc80ba830 0x7f2cc8072db0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2cb800cb40 tx=0x7f2cb800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:12.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.288+0000 7f2cbea00640 1 -- 172.21.15.42:0/408716251 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2cb8016020 con 0x7f2cc80ba830 2024-07-04T17:27:12.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.288+0000 7f2cbea00640 1 -- 172.21.15.42:0/408716251 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2cb800abf0 con 0x7f2cc80ba830 2024-07-04T17:27:12.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.288+0000 7f2cbea00640 1 -- 172.21.15.42:0/408716251 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2cb8012420 con 0x7f2cc80ba830 2024-07-04T17:27:12.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.289+0000 7f2cc7200640 1 -- 172.21.15.42:0/408716251 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cc813e900 con 0x7f2cc80ba830 2024-07-04T17:27:12.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.289+0000 7f2cc7200640 1 -- 172.21.15.42:0/408716251 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cc81d7510 con 0x7f2cc80ba830 2024-07-04T17:27:12.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.290+0000 7f2cc7200640 1 -- 172.21.15.42:0/408716251 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cc806c020 con 0x7f2cc80ba830 2024-07-04T17:27:12.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.290+0000 7f2cbea00640 1 -- 172.21.15.42:0/408716251 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f2cb8019020 con 0x7f2cc80ba830 2024-07-04T17:27:12.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.290+0000 7f2cbea00640 1 --2- 172.21.15.42:0/408716251 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f2ca403d2b0 0x7f2ca403f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:12.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.290+0000 7f2cbea00640 1 -- 172.21.15.42:0/408716251 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3439+0+0 (secure 0 0 0) 0x7f2cb80518c0 con 0x7f2cc80ba830 2024-07-04T17:27:12.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.291+0000 7f2cbd400640 1 --2- 172.21.15.42:0/408716251 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f2ca403d2b0 0x7f2ca403f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:12.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.292+0000 7f2cbd400640 1 --2- 172.21.15.42:0/408716251 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f2ca403d2b0 0x7f2ca403f7c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0034000 tx=0x7f2cb0004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:12.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.296+0000 7f2cbea00640 1 -- 172.21.15.42:0/408716251 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f2cb8020030 con 0x7f2cc80ba830 2024-07-04T17:27:12.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.495+0000 7f2cc7200640 1 -- 172.21.15.42:0/408716251 --> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2cc8072be0 con 0x7f2ca403d2b0 2024-07-04T17:27:12.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.501+0000 7f2cbea00640 1 -- 172.21.15.42:0/408716251 <== mgr.4103 v2:172.21.15.42:6824/3523793950 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245890 (secure 0 0 0) 0x7f2cc8072be0 con 0x7f2ca403d2b0 2024-07-04T17:27:12.506 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:27:12.508 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:27:12.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.506+0000 7f2cc7200640 1 -- 172.21.15.42:0/408716251 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f2ca403d2b0 msgr2=0x7f2ca403f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:12.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.506+0000 7f2cc7200640 1 --2- 172.21.15.42:0/408716251 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f2ca403d2b0 0x7f2ca403f7c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0034000 tx=0x7f2cb0004730 comp rx=0 tx=0).stop 2024-07-04T17:27:12.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.506+0000 7f2cc7200640 1 -- 172.21.15.42:0/408716251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cc80ba830 msgr2=0x7f2cc8072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:12.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.506+0000 7f2cc7200640 1 --2- 172.21.15.42:0/408716251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cc80ba830 0x7f2cc8072db0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2cb800cb40 tx=0x7f2cb800cf40 comp rx=0 tx=0).stop 2024-07-04T17:27:12.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.506+0000 7f2cc7200640 1 -- 172.21.15.42:0/408716251 shutdown_connections 2024-07-04T17:27:12.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.506+0000 7f2cc7200640 1 --2- 172.21.15.42:0/408716251 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f2ca403d2b0 0x7f2ca403f7c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:12.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.506+0000 7f2cc7200640 1 --2- 172.21.15.42:0/408716251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2cc80ba830 0x7f2cc8072db0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:12.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.506+0000 7f2cc7200640 1 -- 172.21.15.42:0/408716251 >> 172.21.15.42:0/408716251 conn(0x7f2cc80bebc0 msgr2=0x7f2cc80b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:12.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.506+0000 7f2cc7200640 1 -- 172.21.15.42:0/408716251 shutdown_connections 2024-07-04T17:27:12.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.506+0000 7f2cc7200640 1 -- 172.21.15.42:0/408716251 wait complete. 2024-07-04T17:27:12.531 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":59,"stamp":"2024-07-04T17:27:11.280013+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":123,"num_write_kb":1120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90,"ondisk_log_size":90,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":95412,"kb_used_data":1868,"kb_used_omap":4,"kb_used_meta":93499,"kb_avail":283020108,"statfs":{"total":289910292480,"available":289812590592,"internally_reserved":0,"allocated":1912832,"data_stored":1411109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":95743311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.462503"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.918042+0000","last_change":"2024-07-04T17:27:07.916245+0000","last_active":"2024-07-04T17:27:07.918042+0000","last_peered":"2024-07-04T17:27:07.918042+0000","last_clean":"2024-07-04T17:27:07.918042+0000","last_became_active":"2024-07-04T17:27:07.912310+0000","last_became_peered":"2024-07-04T17:27:07.912310+0000","last_unstale":"2024-07-04T17:27:07.918042+0000","last_undegraded":"2024-07-04T17:27:07.918042+0000","last_fullsized":"2024-07-04T17:27:07.918042+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:57:29.976142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.907428+0000","last_change":"2024-07-04T17:27:07.884739+0000","last_active":"2024-07-04T17:27:07.907428+0000","last_peered":"2024-07-04T17:27:07.907428+0000","last_clean":"2024-07-04T17:27:07.907428+0000","last_became_active":"2024-07-04T17:27:07.884425+0000","last_became_peered":"2024-07-04T17:27:07.884425+0000","last_unstale":"2024-07-04T17:27:07.907428+0000","last_undegraded":"2024-07-04T17:27:07.907428+0000","last_fullsized":"2024-07-04T17:27:07.907428+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:56:08.436905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.900778+0000","last_change":"2024-07-04T17:27:07.891626+0000","last_active":"2024-07-04T17:27:07.900778+0000","last_peered":"2024-07-04T17:27:07.900778+0000","last_clean":"2024-07-04T17:27:07.900778+0000","last_became_active":"2024-07-04T17:27:07.891070+0000","last_became_peered":"2024-07-04T17:27:07.891070+0000","last_unstale":"2024-07-04T17:27:07.900778+0000","last_undegraded":"2024-07-04T17:27:07.900778+0000","last_fullsized":"2024-07-04T17:27:07.900778+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:32:46.895301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.893681+0000","last_change":"2024-07-04T17:27:07.889861+0000","last_active":"2024-07-04T17:27:07.893681+0000","last_peered":"2024-07-04T17:27:07.893681+0000","last_clean":"2024-07-04T17:27:07.893681+0000","last_became_active":"2024-07-04T17:27:07.889003+0000","last_became_peered":"2024-07-04T17:27:07.889003+0000","last_unstale":"2024-07-04T17:27:07.893681+0000","last_undegraded":"2024-07-04T17:27:07.893681+0000","last_fullsized":"2024-07-04T17:27:07.893681+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.727000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.917350+0000","last_change":"2024-07-04T17:27:07.905295+0000","last_active":"2024-07-04T17:27:07.917350+0000","last_peered":"2024-07-04T17:27:07.917350+0000","last_clean":"2024-07-04T17:27:07.917350+0000","last_became_active":"2024-07-04T17:27:07.904234+0000","last_became_peered":"2024-07-04T17:27:07.904234+0000","last_unstale":"2024-07-04T17:27:07.917350+0000","last_undegraded":"2024-07-04T17:27:07.917350+0000","last_fullsized":"2024-07-04T17:27:07.917350+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:01:29.057248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.885941+0000","last_change":"2024-07-04T17:27:07.883415+0000","last_active":"2024-07-04T17:27:07.885941+0000","last_peered":"2024-07-04T17:27:07.885941+0000","last_clean":"2024-07-04T17:27:07.885941+0000","last_became_active":"2024-07-04T17:27:07.878729+0000","last_became_peered":"2024-07-04T17:27:07.878729+0000","last_unstale":"2024-07-04T17:27:07.885941+0000","last_undegraded":"2024-07-04T17:27:07.885941+0000","last_fullsized":"2024-07-04T17:27:07.885941+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:16:39.185466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.898812+0000","last_change":"2024-07-04T17:27:07.894901+0000","last_active":"2024-07-04T17:27:07.898812+0000","last_peered":"2024-07-04T17:27:07.898812+0000","last_clean":"2024-07-04T17:27:07.898812+0000","last_became_active":"2024-07-04T17:27:07.894469+0000","last_became_peered":"2024-07-04T17:27:07.894469+0000","last_unstale":"2024-07-04T17:27:07.898812+0000","last_undegraded":"2024-07-04T17:27:07.898812+0000","last_fullsized":"2024-07-04T17:27:07.898812+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:19:44.491304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.901643+0000","last_change":"2024-07-04T17:27:07.899412+0000","last_active":"2024-07-04T17:27:07.901643+0000","last_peered":"2024-07-04T17:27:07.901643+0000","last_clean":"2024-07-04T17:27:07.901643+0000","last_became_active":"2024-07-04T17:27:07.898386+0000","last_became_peered":"2024-07-04T17:27:07.898386+0000","last_unstale":"2024-07-04T17:27:07.901643+0000","last_undegraded":"2024-07-04T17:27:07.901643+0000","last_fullsized":"2024-07-04T17:27:07.901643+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:10:35.175672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.900628+0000","last_change":"2024-07-04T17:27:07.897001+0000","last_active":"2024-07-04T17:27:07.900628+0000","last_peered":"2024-07-04T17:27:07.900628+0000","last_clean":"2024-07-04T17:27:07.900628+0000","last_became_active":"2024-07-04T17:27:07.896453+0000","last_became_peered":"2024-07-04T17:27:07.896453+0000","last_unstale":"2024-07-04T17:27:07.900628+0000","last_undegraded":"2024-07-04T17:27:07.900628+0000","last_fullsized":"2024-07-04T17:27:07.900628+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:48:13.563809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.878674+0000","last_change":"2024-07-04T17:27:07.871832+0000","last_active":"2024-07-04T17:27:07.878674+0000","last_peered":"2024-07-04T17:27:07.878674+0000","last_clean":"2024-07-04T17:27:07.878674+0000","last_became_active":"2024-07-04T17:27:07.870928+0000","last_became_peered":"2024-07-04T17:27:07.870928+0000","last_unstale":"2024-07-04T17:27:07.878674+0000","last_undegraded":"2024-07-04T17:27:07.878674+0000","last_fullsized":"2024-07-04T17:27:07.878674+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:55:31.586864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.909301+0000","last_change":"2024-07-04T17:27:07.893125+0000","last_active":"2024-07-04T17:27:07.909301+0000","last_peered":"2024-07-04T17:27:07.909301+0000","last_clean":"2024-07-04T17:27:07.909301+0000","last_became_active":"2024-07-04T17:27:07.892921+0000","last_became_peered":"2024-07-04T17:27:07.892921+0000","last_unstale":"2024-07-04T17:27:07.909301+0000","last_undegraded":"2024-07-04T17:27:07.909301+0000","last_fullsized":"2024-07-04T17:27:07.909301+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:56:46.200219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.889611+0000","last_change":"2024-07-04T17:27:07.886093+0000","last_active":"2024-07-04T17:27:07.889611+0000","last_peered":"2024-07-04T17:27:07.889611+0000","last_clean":"2024-07-04T17:27:07.889611+0000","last_became_active":"2024-07-04T17:27:07.885701+0000","last_became_peered":"2024-07-04T17:27:07.885701+0000","last_unstale":"2024-07-04T17:27:07.889611+0000","last_undegraded":"2024-07-04T17:27:07.889611+0000","last_fullsized":"2024-07-04T17:27:07.889611+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:20:20.786366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.899347+0000","last_change":"2024-07-04T17:27:07.896119+0000","last_active":"2024-07-04T17:27:07.899347+0000","last_peered":"2024-07-04T17:27:07.899347+0000","last_clean":"2024-07-04T17:27:07.899347+0000","last_became_active":"2024-07-04T17:27:07.895583+0000","last_became_peered":"2024-07-04T17:27:07.895583+0000","last_unstale":"2024-07-04T17:27:07.899347+0000","last_undegraded":"2024-07-04T17:27:07.899347+0000","last_fullsized":"2024-07-04T17:27:07.899347+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:17:39.221662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.897612+0000","last_change":"2024-07-04T17:27:07.879943+0000","last_active":"2024-07-04T17:27:07.897612+0000","last_peered":"2024-07-04T17:27:07.897612+0000","last_clean":"2024-07-04T17:27:07.897612+0000","last_became_active":"2024-07-04T17:27:07.875927+0000","last_became_peered":"2024-07-04T17:27:07.875927+0000","last_unstale":"2024-07-04T17:27:07.897612+0000","last_undegraded":"2024-07-04T17:27:07.897612+0000","last_fullsized":"2024-07-04T17:27:07.897612+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:27:04.910602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.890336+0000","last_change":"2024-07-04T17:27:07.873017+0000","last_active":"2024-07-04T17:27:07.890336+0000","last_peered":"2024-07-04T17:27:07.890336+0000","last_clean":"2024-07-04T17:27:07.890336+0000","last_became_active":"2024-07-04T17:27:07.872715+0000","last_became_peered":"2024-07-04T17:27:07.872715+0000","last_unstale":"2024-07-04T17:27:07.890336+0000","last_undegraded":"2024-07-04T17:27:07.890336+0000","last_fullsized":"2024-07-04T17:27:07.890336+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:38:38.667113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.907454+0000","last_change":"2024-07-04T17:27:07.884482+0000","last_active":"2024-07-04T17:27:07.907454+0000","last_peered":"2024-07-04T17:27:07.907454+0000","last_clean":"2024-07-04T17:27:07.907454+0000","last_became_active":"2024-07-04T17:27:07.882978+0000","last_became_peered":"2024-07-04T17:27:07.882978+0000","last_unstale":"2024-07-04T17:27:07.907454+0000","last_undegraded":"2024-07-04T17:27:07.907454+0000","last_fullsized":"2024-07-04T17:27:07.907454+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:27:08.653236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.898899+0000","last_change":"2024-07-04T17:27:07.877401+0000","last_active":"2024-07-04T17:27:07.898899+0000","last_peered":"2024-07-04T17:27:07.898899+0000","last_clean":"2024-07-04T17:27:07.898899+0000","last_became_active":"2024-07-04T17:27:07.877014+0000","last_became_peered":"2024-07-04T17:27:07.877014+0000","last_unstale":"2024-07-04T17:27:07.898899+0000","last_undegraded":"2024-07-04T17:27:07.898899+0000","last_fullsized":"2024-07-04T17:27:07.898899+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:24:58.267453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.907361+0000","last_change":"2024-07-04T17:27:07.883614+0000","last_active":"2024-07-04T17:27:07.907361+0000","last_peered":"2024-07-04T17:27:07.907361+0000","last_clean":"2024-07-04T17:27:07.907361+0000","last_became_active":"2024-07-04T17:27:07.883289+0000","last_became_peered":"2024-07-04T17:27:07.883289+0000","last_unstale":"2024-07-04T17:27:07.907361+0000","last_undegraded":"2024-07-04T17:27:07.907361+0000","last_fullsized":"2024-07-04T17:27:07.907361+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:06:36.067329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.909268+0000","last_change":"2024-07-04T17:27:07.893065+0000","last_active":"2024-07-04T17:27:07.909268+0000","last_peered":"2024-07-04T17:27:07.909268+0000","last_clean":"2024-07-04T17:27:07.909268+0000","last_became_active":"2024-07-04T17:27:07.892736+0000","last_became_peered":"2024-07-04T17:27:07.892736+0000","last_unstale":"2024-07-04T17:27:07.909268+0000","last_undegraded":"2024-07-04T17:27:07.909268+0000","last_fullsized":"2024-07-04T17:27:07.909268+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:40:58.276630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.917950+0000","last_change":"2024-07-04T17:27:07.916151+0000","last_active":"2024-07-04T17:27:07.917950+0000","last_peered":"2024-07-04T17:27:07.917950+0000","last_clean":"2024-07-04T17:27:07.917950+0000","last_became_active":"2024-07-04T17:27:07.909999+0000","last_became_peered":"2024-07-04T17:27:07.909999+0000","last_unstale":"2024-07-04T17:27:07.917950+0000","last_undegraded":"2024-07-04T17:27:07.917950+0000","last_fullsized":"2024-07-04T17:27:07.917950+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:32:46.895301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.908720+0000","last_change":"2024-07-04T17:27:07.887157+0000","last_active":"2024-07-04T17:27:07.908720+0000","last_peered":"2024-07-04T17:27:07.908720+0000","last_clean":"2024-07-04T17:27:07.908720+0000","last_became_active":"2024-07-04T17:27:07.886783+0000","last_became_peered":"2024-07-04T17:27:07.886783+0000","last_unstale":"2024-07-04T17:27:07.908720+0000","last_undegraded":"2024-07-04T17:27:07.908720+0000","last_fullsized":"2024-07-04T17:27:07.908720+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:34:35.587534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.917662+0000","last_change":"2024-07-04T17:27:07.904537+0000","last_active":"2024-07-04T17:27:07.917662+0000","last_peered":"2024-07-04T17:27:07.917662+0000","last_clean":"2024-07-04T17:27:07.917662+0000","last_became_active":"2024-07-04T17:27:07.903476+0000","last_became_peered":"2024-07-04T17:27:07.903476+0000","last_unstale":"2024-07-04T17:27:07.917662+0000","last_undegraded":"2024-07-04T17:27:07.917662+0000","last_fullsized":"2024-07-04T17:27:07.917662+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:55:31.586864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.907929+0000","last_change":"2024-07-04T17:27:07.883846+0000","last_active":"2024-07-04T17:27:07.907929+0000","last_peered":"2024-07-04T17:27:07.907929+0000","last_clean":"2024-07-04T17:27:07.907929+0000","last_became_active":"2024-07-04T17:27:07.881609+0000","last_became_peered":"2024-07-04T17:27:07.881609+0000","last_unstale":"2024-07-04T17:27:07.907929+0000","last_undegraded":"2024-07-04T17:27:07.907929+0000","last_fullsized":"2024-07-04T17:27:07.907929+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:38:45.217394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.901619+0000","last_change":"2024-07-04T17:27:07.898354+0000","last_active":"2024-07-04T17:27:07.901619+0000","last_peered":"2024-07-04T17:27:07.901619+0000","last_clean":"2024-07-04T17:27:07.901619+0000","last_became_active":"2024-07-04T17:27:07.897988+0000","last_became_peered":"2024-07-04T17:27:07.897988+0000","last_unstale":"2024-07-04T17:27:07.901619+0000","last_undegraded":"2024-07-04T17:27:07.901619+0000","last_fullsized":"2024-07-04T17:27:07.901619+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:23:41.800377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.900613+0000","last_change":"2024-07-04T17:27:07.896753+0000","last_active":"2024-07-04T17:27:07.900613+0000","last_peered":"2024-07-04T17:27:07.900613+0000","last_clean":"2024-07-04T17:27:07.900613+0000","last_became_active":"2024-07-04T17:27:07.896053+0000","last_became_peered":"2024-07-04T17:27:07.896053+0000","last_unstale":"2024-07-04T17:27:07.900613+0000","last_undegraded":"2024-07-04T17:27:07.900613+0000","last_fullsized":"2024-07-04T17:27:07.900613+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:57:29.976142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.885849+0000","last_change":"2024-07-04T17:27:07.877946+0000","last_active":"2024-07-04T17:27:07.885849+0000","last_peered":"2024-07-04T17:27:07.885849+0000","last_clean":"2024-07-04T17:27:07.885849+0000","last_became_active":"2024-07-04T17:27:07.877581+0000","last_became_peered":"2024-07-04T17:27:07.877581+0000","last_unstale":"2024-07-04T17:27:07.885849+0000","last_undegraded":"2024-07-04T17:27:07.885849+0000","last_fullsized":"2024-07-04T17:27:07.885849+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:31:06.033299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.909621+0000","last_change":"2024-07-04T17:27:07.901813+0000","last_active":"2024-07-04T17:27:07.909621+0000","last_peered":"2024-07-04T17:27:07.909621+0000","last_clean":"2024-07-04T17:27:07.909621+0000","last_became_active":"2024-07-04T17:27:07.901333+0000","last_became_peered":"2024-07-04T17:27:07.901333+0000","last_unstale":"2024-07-04T17:27:07.909621+0000","last_undegraded":"2024-07-04T17:27:07.909621+0000","last_fullsized":"2024-07-04T17:27:07.909621+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:25:52.643999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.917928+0000","last_change":"2024-07-04T17:27:07.911625+0000","last_active":"2024-07-04T17:27:07.917928+0000","last_peered":"2024-07-04T17:27:07.917928+0000","last_clean":"2024-07-04T17:27:07.917928+0000","last_became_active":"2024-07-04T17:27:07.910841+0000","last_became_peered":"2024-07-04T17:27:07.910841+0000","last_unstale":"2024-07-04T17:27:07.917928+0000","last_undegraded":"2024-07-04T17:27:07.917928+0000","last_fullsized":"2024-07-04T17:27:07.917928+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:20:20.786366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.907327+0000","last_change":"2024-07-04T17:27:07.883042+0000","last_active":"2024-07-04T17:27:07.907327+0000","last_peered":"2024-07-04T17:27:07.907327+0000","last_clean":"2024-07-04T17:27:07.907327+0000","last_became_active":"2024-07-04T17:27:07.882772+0000","last_became_peered":"2024-07-04T17:27:07.882772+0000","last_unstale":"2024-07-04T17:27:07.907327+0000","last_undegraded":"2024-07-04T17:27:07.907327+0000","last_fullsized":"2024-07-04T17:27:07.907327+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:34:46.000005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.897584+0000","last_change":"2024-07-04T17:27:07.878292+0000","last_active":"2024-07-04T17:27:07.897584+0000","last_peered":"2024-07-04T17:27:07.897584+0000","last_clean":"2024-07-04T17:27:07.897584+0000","last_became_active":"2024-07-04T17:27:07.874906+0000","last_became_peered":"2024-07-04T17:27:07.874906+0000","last_unstale":"2024-07-04T17:27:07.897584+0000","last_undegraded":"2024-07-04T17:27:07.897584+0000","last_fullsized":"2024-07-04T17:27:07.897584+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:47:16.318473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.917482+0000","last_change":"2024-07-04T17:27:07.906719+0000","last_active":"2024-07-04T17:27:07.917482+0000","last_peered":"2024-07-04T17:27:07.917482+0000","last_clean":"2024-07-04T17:27:07.917482+0000","last_became_active":"2024-07-04T17:27:07.903370+0000","last_became_peered":"2024-07-04T17:27:07.903370+0000","last_unstale":"2024-07-04T17:27:07.917482+0000","last_undegraded":"2024-07-04T17:27:07.917482+0000","last_fullsized":"2024-07-04T17:27:07.917482+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:12:04.323408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.917745+0000","last_change":"2024-07-04T17:27:07.910603+0000","last_active":"2024-07-04T17:27:07.917745+0000","last_peered":"2024-07-04T17:27:07.917745+0000","last_clean":"2024-07-04T17:27:07.917745+0000","last_became_active":"2024-07-04T17:27:07.908845+0000","last_became_peered":"2024-07-04T17:27:07.908845+0000","last_unstale":"2024-07-04T17:27:07.917745+0000","last_undegraded":"2024-07-04T17:27:07.917745+0000","last_fullsized":"2024-07-04T17:27:07.917745+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:16:39.185466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.907663+0000","last_change":"2024-07-04T17:27:07.888430+0000","last_active":"2024-07-04T17:27:07.907663+0000","last_peered":"2024-07-04T17:27:07.907663+0000","last_clean":"2024-07-04T17:27:07.907663+0000","last_became_active":"2024-07-04T17:27:07.886722+0000","last_became_peered":"2024-07-04T17:27:07.886722+0000","last_unstale":"2024-07-04T17:27:07.907663+0000","last_undegraded":"2024-07-04T17:27:07.907663+0000","last_fullsized":"2024-07-04T17:27:07.907663+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:20:22.623733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"26'4","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.913855+0000","last_change":"2024-07-04T17:27:06.866478+0000","last_active":"2024-07-04T17:27:07.913855+0000","last_peered":"2024-07-04T17:27:07.913855+0000","last_clean":"2024-07-04T17:27:07.913855+0000","last_became_active":"2024-07-04T17:27:06.865591+0000","last_became_peered":"2024-07-04T17:27:06.865591+0000","last_unstale":"2024-07-04T17:27:07.913855+0000","last_undegraded":"2024-07-04T17:27:07.913855+0000","last_fullsized":"2024-07-04T17:27:07.913855+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:09:15.298068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.900211+0000","last_change":"2024-07-04T17:27:07.896653+0000","last_active":"2024-07-04T17:27:07.900211+0000","last_peered":"2024-07-04T17:27:07.900211+0000","last_clean":"2024-07-04T17:27:07.900211+0000","last_became_active":"2024-07-04T17:27:07.895759+0000","last_became_peered":"2024-07-04T17:27:07.895759+0000","last_unstale":"2024-07-04T17:27:07.900211+0000","last_undegraded":"2024-07-04T17:27:07.900211+0000","last_fullsized":"2024-07-04T17:27:07.900211+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:09:29.424181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"26'1","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.892150+0000","last_change":"2024-07-04T17:27:06.863457+0000","last_active":"2024-07-04T17:27:07.892150+0000","last_peered":"2024-07-04T17:27:07.892150+0000","last_clean":"2024-07-04T17:27:07.892150+0000","last_became_active":"2024-07-04T17:27:06.862649+0000","last_became_peered":"2024-07-04T17:27:06.862649+0000","last_unstale":"2024-07-04T17:27:07.892150+0000","last_undegraded":"2024-07-04T17:27:07.892150+0000","last_fullsized":"2024-07-04T17:27:07.892150+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:49:24.997305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.917781+0000","last_change":"2024-07-04T17:27:07.911112+0000","last_active":"2024-07-04T17:27:07.917781+0000","last_peered":"2024-07-04T17:27:07.917781+0000","last_clean":"2024-07-04T17:27:07.917781+0000","last_became_active":"2024-07-04T17:27:07.910473+0000","last_became_peered":"2024-07-04T17:27:07.910473+0000","last_unstale":"2024-07-04T17:27:07.917781+0000","last_undegraded":"2024-07-04T17:27:07.917781+0000","last_fullsized":"2024-07-04T17:27:07.917781+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:22:57.186029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"26'2","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.897532+0000","last_change":"2024-07-04T17:27:06.860195+0000","last_active":"2024-07-04T17:27:07.897532+0000","last_peered":"2024-07-04T17:27:07.897532+0000","last_clean":"2024-07-04T17:27:07.897532+0000","last_became_active":"2024-07-04T17:27:06.859440+0000","last_became_peered":"2024-07-04T17:27:06.859440+0000","last_unstale":"2024-07-04T17:27:07.897532+0000","last_undegraded":"2024-07-04T17:27:07.897532+0000","last_fullsized":"2024-07-04T17:27:07.897532+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:11:21.518052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.907560+0000","last_change":"2024-07-04T17:27:07.901586+0000","last_active":"2024-07-04T17:27:07.907560+0000","last_peered":"2024-07-04T17:27:07.907560+0000","last_clean":"2024-07-04T17:27:07.907560+0000","last_became_active":"2024-07-04T17:27:07.900901+0000","last_became_peered":"2024-07-04T17:27:07.900901+0000","last_unstale":"2024-07-04T17:27:07.907560+0000","last_undegraded":"2024-07-04T17:27:07.907560+0000","last_fullsized":"2024-07-04T17:27:07.907560+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:11:22.527978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"26'2","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.898863+0000","last_change":"2024-07-04T17:27:06.885584+0000","last_active":"2024-07-04T17:27:07.898863+0000","last_peered":"2024-07-04T17:27:07.898863+0000","last_clean":"2024-07-04T17:27:07.898863+0000","last_became_active":"2024-07-04T17:27:06.885226+0000","last_became_peered":"2024-07-04T17:27:06.885226+0000","last_unstale":"2024-07-04T17:27:07.898863+0000","last_undegraded":"2024-07-04T17:27:07.898863+0000","last_fullsized":"2024-07-04T17:27:07.898863+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:10:29.469978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.878698+0000","last_change":"2024-07-04T17:27:07.872169+0000","last_active":"2024-07-04T17:27:07.878698+0000","last_peered":"2024-07-04T17:27:07.878698+0000","last_clean":"2024-07-04T17:27:07.878698+0000","last_became_active":"2024-07-04T17:27:07.870986+0000","last_became_peered":"2024-07-04T17:27:07.870986+0000","last_unstale":"2024-07-04T17:27:07.878698+0000","last_undegraded":"2024-07-04T17:27:07.878698+0000","last_fullsized":"2024-07-04T17:27:07.878698+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:16:25.982847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"26'1","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.899296+0000","last_change":"2024-07-04T17:27:06.883716+0000","last_active":"2024-07-04T17:27:07.899296+0000","last_peered":"2024-07-04T17:27:07.899296+0000","last_clean":"2024-07-04T17:27:07.899296+0000","last_became_active":"2024-07-04T17:27:06.882604+0000","last_became_peered":"2024-07-04T17:27:06.882604+0000","last_unstale":"2024-07-04T17:27:07.899296+0000","last_undegraded":"2024-07-04T17:27:07.899296+0000","last_fullsized":"2024-07-04T17:27:07.899296+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:04:44.886174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.915617+0000","last_change":"2024-07-04T17:27:07.904214+0000","last_active":"2024-07-04T17:27:07.915617+0000","last_peered":"2024-07-04T17:27:07.915617+0000","last_clean":"2024-07-04T17:27:07.915617+0000","last_became_active":"2024-07-04T17:27:07.903782+0000","last_became_peered":"2024-07-04T17:27:07.903782+0000","last_unstale":"2024-07-04T17:27:07.915617+0000","last_undegraded":"2024-07-04T17:27:07.915617+0000","last_fullsized":"2024-07-04T17:27:07.915617+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:35:44.467250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"26'1","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.905336+0000","last_change":"2024-07-04T17:27:06.881468+0000","last_active":"2024-07-04T17:27:07.905336+0000","last_peered":"2024-07-04T17:27:07.905336+0000","last_clean":"2024-07-04T17:27:07.905336+0000","last_became_active":"2024-07-04T17:27:06.881150+0000","last_became_peered":"2024-07-04T17:27:06.881150+0000","last_unstale":"2024-07-04T17:27:07.905336+0000","last_undegraded":"2024-07-04T17:27:07.905336+0000","last_fullsized":"2024-07-04T17:27:07.905336+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:49:39.642772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.886191+0000","last_change":"2024-07-04T17:27:07.866107+0000","last_active":"2024-07-04T17:27:07.886191+0000","last_peered":"2024-07-04T17:27:07.886191+0000","last_clean":"2024-07-04T17:27:07.886191+0000","last_became_active":"2024-07-04T17:27:07.865761+0000","last_became_peered":"2024-07-04T17:27:07.865761+0000","last_unstale":"2024-07-04T17:27:07.886191+0000","last_undegraded":"2024-07-04T17:27:07.886191+0000","last_fullsized":"2024-07-04T17:27:07.886191+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:29:13.459059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.858077+0000","last_change":"2024-07-04T17:27:06.872335+0000","last_active":"2024-07-04T17:27:07.858077+0000","last_peered":"2024-07-04T17:27:07.858077+0000","last_clean":"2024-07-04T17:27:07.858077+0000","last_became_active":"2024-07-04T17:27:06.871960+0000","last_became_peered":"2024-07-04T17:27:06.871960+0000","last_unstale":"2024-07-04T17:27:07.858077+0000","last_undegraded":"2024-07-04T17:27:07.858077+0000","last_fullsized":"2024-07-04T17:27:07.858077+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:57:28.966215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.889305+0000","last_change":"2024-07-04T17:27:07.885943+0000","last_active":"2024-07-04T17:27:07.889305+0000","last_peered":"2024-07-04T17:27:07.889305+0000","last_clean":"2024-07-04T17:27:07.889305+0000","last_became_active":"2024-07-04T17:27:07.876919+0000","last_became_peered":"2024-07-04T17:27:07.876919+0000","last_unstale":"2024-07-04T17:27:07.889305+0000","last_undegraded":"2024-07-04T17:27:07.889305+0000","last_fullsized":"2024-07-04T17:27:07.889305+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:22:57.186029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.870646+0000","last_change":"2024-07-04T17:27:06.883828+0000","last_active":"2024-07-04T17:27:07.870646+0000","last_peered":"2024-07-04T17:27:07.870646+0000","last_clean":"2024-07-04T17:27:07.870646+0000","last_became_active":"2024-07-04T17:27:06.883508+0000","last_became_peered":"2024-07-04T17:27:06.883508+0000","last_unstale":"2024-07-04T17:27:07.870646+0000","last_undegraded":"2024-07-04T17:27:07.870646+0000","last_fullsized":"2024-07-04T17:27:07.870646+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:44:59.048384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.885152+0000","last_change":"2024-07-04T17:27:07.877680+0000","last_active":"2024-07-04T17:27:07.885152+0000","last_peered":"2024-07-04T17:27:07.885152+0000","last_clean":"2024-07-04T17:27:07.885152+0000","last_became_active":"2024-07-04T17:27:07.877486+0000","last_became_peered":"2024-07-04T17:27:07.877486+0000","last_unstale":"2024-07-04T17:27:07.885152+0000","last_undegraded":"2024-07-04T17:27:07.885152+0000","last_fullsized":"2024-07-04T17:27:07.885152+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:12:04.323408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.862548+0000","last_change":"2024-07-04T17:27:06.861151+0000","last_active":"2024-07-04T17:27:07.862548+0000","last_peered":"2024-07-04T17:27:07.862548+0000","last_clean":"2024-07-04T17:27:07.862548+0000","last_became_active":"2024-07-04T17:27:06.860975+0000","last_became_peered":"2024-07-04T17:27:06.860975+0000","last_unstale":"2024-07-04T17:27:07.862548+0000","last_undegraded":"2024-07-04T17:27:07.862548+0000","last_fullsized":"2024-07-04T17:27:07.862548+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:17:47.882494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.902977+0000","last_change":"2024-07-04T17:27:07.898440+0000","last_active":"2024-07-04T17:27:07.902977+0000","last_peered":"2024-07-04T17:27:07.902977+0000","last_clean":"2024-07-04T17:27:07.902977+0000","last_became_active":"2024-07-04T17:27:07.897813+0000","last_became_peered":"2024-07-04T17:27:07.897813+0000","last_unstale":"2024-07-04T17:27:07.902977+0000","last_undegraded":"2024-07-04T17:27:07.902977+0000","last_fullsized":"2024-07-04T17:27:07.902977+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:52:16.369596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.862680+0000","last_change":"2024-07-04T17:27:06.873231+0000","last_active":"2024-07-04T17:27:07.862680+0000","last_peered":"2024-07-04T17:27:07.862680+0000","last_clean":"2024-07-04T17:27:07.862680+0000","last_became_active":"2024-07-04T17:27:06.872934+0000","last_became_peered":"2024-07-04T17:27:06.872934+0000","last_unstale":"2024-07-04T17:27:07.862680+0000","last_undegraded":"2024-07-04T17:27:07.862680+0000","last_fullsized":"2024-07-04T17:27:07.862680+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:42:02.672719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.908299+0000","last_change":"2024-07-04T17:27:07.891116+0000","last_active":"2024-07-04T17:27:07.908299+0000","last_peered":"2024-07-04T17:27:07.908299+0000","last_clean":"2024-07-04T17:27:07.908299+0000","last_became_active":"2024-07-04T17:27:07.890572+0000","last_became_peered":"2024-07-04T17:27:07.890572+0000","last_unstale":"2024-07-04T17:27:07.908299+0000","last_undegraded":"2024-07-04T17:27:07.908299+0000","last_fullsized":"2024-07-04T17:27:07.908299+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:18:55.534844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"26'3","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.897648+0000","last_change":"2024-07-04T17:27:06.868145+0000","last_active":"2024-07-04T17:27:07.897648+0000","last_peered":"2024-07-04T17:27:07.897648+0000","last_clean":"2024-07-04T17:27:07.897648+0000","last_became_active":"2024-07-04T17:27:06.867751+0000","last_became_peered":"2024-07-04T17:27:06.867751+0000","last_unstale":"2024-07-04T17:27:07.897648+0000","last_undegraded":"2024-07-04T17:27:07.897648+0000","last_fullsized":"2024-07-04T17:27:07.897648+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:13:08.657441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.907645+0000","last_change":"2024-07-04T17:27:07.889067+0000","last_active":"2024-07-04T17:27:07.907645+0000","last_peered":"2024-07-04T17:27:07.907645+0000","last_clean":"2024-07-04T17:27:07.907645+0000","last_became_active":"2024-07-04T17:27:07.888715+0000","last_became_peered":"2024-07-04T17:27:07.888715+0000","last_unstale":"2024-07-04T17:27:07.907645+0000","last_undegraded":"2024-07-04T17:27:07.907645+0000","last_fullsized":"2024-07-04T17:27:07.907645+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:35:18.197218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"26'6","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.915686+0000","last_change":"2024-07-04T17:27:06.872191+0000","last_active":"2024-07-04T17:27:07.915686+0000","last_peered":"2024-07-04T17:27:07.915686+0000","last_clean":"2024-07-04T17:27:07.915686+0000","last_became_active":"2024-07-04T17:27:06.871594+0000","last_became_peered":"2024-07-04T17:27:06.871594+0000","last_unstale":"2024-07-04T17:27:07.915686+0000","last_undegraded":"2024-07-04T17:27:07.915686+0000","last_fullsized":"2024-07-04T17:27:07.915686+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:05:09.285237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.885791+0000","last_change":"2024-07-04T17:27:07.878309+0000","last_active":"2024-07-04T17:27:07.885791+0000","last_peered":"2024-07-04T17:27:07.885791+0000","last_clean":"2024-07-04T17:27:07.885791+0000","last_became_active":"2024-07-04T17:27:07.877975+0000","last_became_peered":"2024-07-04T17:27:07.877975+0000","last_unstale":"2024-07-04T17:27:07.885791+0000","last_undegraded":"2024-07-04T17:27:07.885791+0000","last_fullsized":"2024-07-04T17:27:07.885791+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:52:51.670741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"26'2","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.906095+0000","last_change":"2024-07-04T17:27:06.883778+0000","last_active":"2024-07-04T17:27:07.906095+0000","last_peered":"2024-07-04T17:27:07.906095+0000","last_clean":"2024-07-04T17:27:07.906095+0000","last_became_active":"2024-07-04T17:27:06.883449+0000","last_became_peered":"2024-07-04T17:27:06.883449+0000","last_unstale":"2024-07-04T17:27:07.906095+0000","last_undegraded":"2024-07-04T17:27:07.906095+0000","last_fullsized":"2024-07-04T17:27:07.906095+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:05:09.285237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.891647+0000","last_change":"2024-07-04T17:27:07.886433+0000","last_active":"2024-07-04T17:27:07.891647+0000","last_peered":"2024-07-04T17:27:07.891647+0000","last_clean":"2024-07-04T17:27:07.891647+0000","last_became_active":"2024-07-04T17:27:07.878497+0000","last_became_peered":"2024-07-04T17:27:07.878497+0000","last_unstale":"2024-07-04T17:27:07.891647+0000","last_undegraded":"2024-07-04T17:27:07.891647+0000","last_fullsized":"2024-07-04T17:27:07.891647+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:18:54.268323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.848972+0000","last_change":"2024-07-04T17:27:06.873153+0000","last_active":"2024-07-04T17:27:07.848972+0000","last_peered":"2024-07-04T17:27:07.848972+0000","last_clean":"2024-07-04T17:27:07.848972+0000","last_became_active":"2024-07-04T17:27:06.872701+0000","last_became_peered":"2024-07-04T17:27:06.872701+0000","last_unstale":"2024-07-04T17:27:07.848972+0000","last_undegraded":"2024-07-04T17:27:07.848972+0000","last_fullsized":"2024-07-04T17:27:07.848972+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:23:40.790450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.917599+0000","last_change":"2024-07-04T17:27:07.909034+0000","last_active":"2024-07-04T17:27:07.917599+0000","last_peered":"2024-07-04T17:27:07.917599+0000","last_clean":"2024-07-04T17:27:07.917599+0000","last_became_active":"2024-07-04T17:27:07.908392+0000","last_became_peered":"2024-07-04T17:27:07.908392+0000","last_unstale":"2024-07-04T17:27:07.917599+0000","last_undegraded":"2024-07-04T17:27:07.917599+0000","last_fullsized":"2024-07-04T17:27:07.917599+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:31:06.033299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.868421+0000","last_change":"2024-07-04T17:27:06.883046+0000","last_active":"2024-07-04T17:27:07.868421+0000","last_peered":"2024-07-04T17:27:07.868421+0000","last_clean":"2024-07-04T17:27:07.868421+0000","last_became_active":"2024-07-04T17:27:06.881811+0000","last_became_peered":"2024-07-04T17:27:06.881811+0000","last_unstale":"2024-07-04T17:27:07.868421+0000","last_undegraded":"2024-07-04T17:27:07.868421+0000","last_fullsized":"2024-07-04T17:27:07.868421+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:40:58.357800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.890930+0000","last_change":"2024-07-04T17:27:07.886267+0000","last_active":"2024-07-04T17:27:07.890930+0000","last_peered":"2024-07-04T17:27:07.890930+0000","last_clean":"2024-07-04T17:27:07.890930+0000","last_became_active":"2024-07-04T17:27:07.885526+0000","last_became_peered":"2024-07-04T17:27:07.885526+0000","last_unstale":"2024-07-04T17:27:07.890930+0000","last_undegraded":"2024-07-04T17:27:07.890930+0000","last_fullsized":"2024-07-04T17:27:07.890930+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:06:16.576182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"26'8","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.914645+0000","last_change":"2024-07-04T17:27:06.865126+0000","last_active":"2024-07-04T17:27:07.914645+0000","last_peered":"2024-07-04T17:27:07.914645+0000","last_clean":"2024-07-04T17:27:07.914645+0000","last_became_active":"2024-07-04T17:27:06.864781+0000","last_became_peered":"2024-07-04T17:27:06.864781+0000","last_unstale":"2024-07-04T17:27:07.914645+0000","last_undegraded":"2024-07-04T17:27:07.914645+0000","last_fullsized":"2024-07-04T17:27:07.914645+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:44:59.048384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.918346+0000","last_change":"2024-07-04T17:27:07.915949+0000","last_active":"2024-07-04T17:27:07.918346+0000","last_peered":"2024-07-04T17:27:07.918346+0000","last_clean":"2024-07-04T17:27:07.918346+0000","last_became_active":"2024-07-04T17:27:07.915603+0000","last_became_peered":"2024-07-04T17:27:07.915603+0000","last_unstale":"2024-07-04T17:27:07.918346+0000","last_undegraded":"2024-07-04T17:27:07.918346+0000","last_fullsized":"2024-07-04T17:27:07.918346+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.727000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"26'1","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.906024+0000","last_change":"2024-07-04T17:27:06.877349+0000","last_active":"2024-07-04T17:27:07.906024+0000","last_peered":"2024-07-04T17:27:07.906024+0000","last_clean":"2024-07-04T17:27:07.906024+0000","last_became_active":"2024-07-04T17:27:06.876814+0000","last_became_peered":"2024-07-04T17:27:06.876814+0000","last_unstale":"2024-07-04T17:27:07.906024+0000","last_undegraded":"2024-07-04T17:27:07.906024+0000","last_fullsized":"2024-07-04T17:27:07.906024+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:52:15.359670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.900639+0000","last_change":"2024-07-04T17:27:07.899041+0000","last_active":"2024-07-04T17:27:07.900639+0000","last_peered":"2024-07-04T17:27:07.900639+0000","last_clean":"2024-07-04T17:27:07.900639+0000","last_became_active":"2024-07-04T17:27:07.895875+0000","last_became_peered":"2024-07-04T17:27:07.895875+0000","last_unstale":"2024-07-04T17:27:07.900639+0000","last_undegraded":"2024-07-04T17:27:07.900639+0000","last_fullsized":"2024-07-04T17:27:07.900639+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:55:16.424967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"26'2","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.895709+0000","last_change":"2024-07-04T17:27:06.885939+0000","last_active":"2024-07-04T17:27:07.895709+0000","last_peered":"2024-07-04T17:27:07.895709+0000","last_clean":"2024-07-04T17:27:07.895709+0000","last_became_active":"2024-07-04T17:27:06.885638+0000","last_became_peered":"2024-07-04T17:27:06.885638+0000","last_unstale":"2024-07-04T17:27:07.895709+0000","last_undegraded":"2024-07-04T17:27:07.895709+0000","last_fullsized":"2024-07-04T17:27:07.895709+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:31:45.030996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.913683+0000","last_change":"2024-07-04T17:27:07.904773+0000","last_active":"2024-07-04T17:27:07.913683+0000","last_peered":"2024-07-04T17:27:07.913683+0000","last_clean":"2024-07-04T17:27:07.913683+0000","last_became_active":"2024-07-04T17:27:07.900622+0000","last_became_peered":"2024-07-04T17:27:07.900622+0000","last_unstale":"2024-07-04T17:27:07.913683+0000","last_undegraded":"2024-07-04T17:27:07.913683+0000","last_fullsized":"2024-07-04T17:27:07.913683+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:13:09.667367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"26'1","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.891393+0000","last_change":"2024-07-04T17:27:06.877855+0000","last_active":"2024-07-04T17:27:07.891393+0000","last_peered":"2024-07-04T17:27:07.891393+0000","last_clean":"2024-07-04T17:27:07.891393+0000","last_became_active":"2024-07-04T17:27:06.876852+0000","last_became_peered":"2024-07-04T17:27:06.876852+0000","last_unstale":"2024-07-04T17:27:07.891393+0000","last_undegraded":"2024-07-04T17:27:07.891393+0000","last_fullsized":"2024-07-04T17:27:07.891393+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:19:43.481377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.918953+0000","last_change":"2024-07-04T17:27:07.912224+0000","last_active":"2024-07-04T17:27:07.918953+0000","last_peered":"2024-07-04T17:27:07.918953+0000","last_clean":"2024-07-04T17:27:07.918953+0000","last_became_active":"2024-07-04T17:27:07.911921+0000","last_became_peered":"2024-07-04T17:27:07.911921+0000","last_unstale":"2024-07-04T17:27:07.918953+0000","last_undegraded":"2024-07-04T17:27:07.918953+0000","last_fullsized":"2024-07-04T17:27:07.918953+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:18:54.268323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.862276+0000","last_change":"2024-07-04T17:27:06.859888+0000","last_active":"2024-07-04T17:27:07.862276+0000","last_peered":"2024-07-04T17:27:07.862276+0000","last_clean":"2024-07-04T17:27:07.862276+0000","last_became_active":"2024-07-04T17:27:06.859565+0000","last_became_peered":"2024-07-04T17:27:06.859565+0000","last_unstale":"2024-07-04T17:27:07.862276+0000","last_undegraded":"2024-07-04T17:27:07.862276+0000","last_fullsized":"2024-07-04T17:27:07.862276+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:25:51.634073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.917397+0000","last_change":"2024-07-04T17:27:07.905531+0000","last_active":"2024-07-04T17:27:07.917397+0000","last_peered":"2024-07-04T17:27:07.917397+0000","last_clean":"2024-07-04T17:27:07.917397+0000","last_became_active":"2024-07-04T17:27:07.903880+0000","last_became_peered":"2024-07-04T17:27:07.903880+0000","last_unstale":"2024-07-04T17:27:07.917397+0000","last_undegraded":"2024-07-04T17:27:07.917397+0000","last_fullsized":"2024-07-04T17:27:07.917397+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:34:47.905292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"26'2","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.893123+0000","last_change":"2024-07-04T17:27:06.859869+0000","last_active":"2024-07-04T17:27:07.893123+0000","last_peered":"2024-07-04T17:27:07.893123+0000","last_clean":"2024-07-04T17:27:07.893123+0000","last_became_active":"2024-07-04T17:27:06.859475+0000","last_became_peered":"2024-07-04T17:27:06.859475+0000","last_unstale":"2024-07-04T17:27:07.893123+0000","last_undegraded":"2024-07-04T17:27:07.893123+0000","last_fullsized":"2024-07-04T17:27:07.893123+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:00:10.490785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.915751+0000","last_change":"2024-07-04T17:27:07.904396+0000","last_active":"2024-07-04T17:27:07.915751+0000","last_peered":"2024-07-04T17:27:07.915751+0000","last_clean":"2024-07-04T17:27:07.915751+0000","last_became_active":"2024-07-04T17:27:07.903961+0000","last_became_peered":"2024-07-04T17:27:07.903961+0000","last_unstale":"2024-07-04T17:27:07.915751+0000","last_undegraded":"2024-07-04T17:27:07.915751+0000","last_fullsized":"2024-07-04T17:27:07.915751+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:29:13.459059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.868334+0000","last_change":"2024-07-04T17:27:06.885598+0000","last_active":"2024-07-04T17:27:07.868334+0000","last_peered":"2024-07-04T17:27:07.868334+0000","last_clean":"2024-07-04T17:27:07.868334+0000","last_became_active":"2024-07-04T17:27:06.885220+0000","last_became_peered":"2024-07-04T17:27:06.885220+0000","last_unstale":"2024-07-04T17:27:07.868334+0000","last_undegraded":"2024-07-04T17:27:07.868334+0000","last_fullsized":"2024-07-04T17:27:07.868334+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:49:35.335628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.917575+0000","last_change":"2024-07-04T17:27:07.905869+0000","last_active":"2024-07-04T17:27:07.917575+0000","last_peered":"2024-07-04T17:27:07.917575+0000","last_clean":"2024-07-04T17:27:07.917575+0000","last_became_active":"2024-07-04T17:27:07.905559+0000","last_became_peered":"2024-07-04T17:27:07.905559+0000","last_unstale":"2024-07-04T17:27:07.917575+0000","last_undegraded":"2024-07-04T17:27:07.917575+0000","last_fullsized":"2024-07-04T17:27:07.917575+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:16:25.982847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.845383+0000","last_change":"2024-07-04T17:27:06.868310+0000","last_active":"2024-07-04T17:27:07.845383+0000","last_peered":"2024-07-04T17:27:07.845383+0000","last_clean":"2024-07-04T17:27:07.845383+0000","last_became_active":"2024-07-04T17:27:06.866928+0000","last_became_peered":"2024-07-04T17:27:06.866928+0000","last_unstale":"2024-07-04T17:27:07.845383+0000","last_undegraded":"2024-07-04T17:27:07.845383+0000","last_fullsized":"2024-07-04T17:27:07.845383+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:31:45.030996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.885796+0000","last_change":"2024-07-04T17:27:07.872007+0000","last_active":"2024-07-04T17:27:07.885796+0000","last_peered":"2024-07-04T17:27:07.885796+0000","last_clean":"2024-07-04T17:27:07.885796+0000","last_became_active":"2024-07-04T17:27:07.871647+0000","last_became_peered":"2024-07-04T17:27:07.871647+0000","last_unstale":"2024-07-04T17:27:07.885796+0000","last_undegraded":"2024-07-04T17:27:07.885796+0000","last_fullsized":"2024-07-04T17:27:07.885796+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:34:47.905292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.848155+0000","last_change":"2024-07-04T17:27:06.871847+0000","last_active":"2024-07-04T17:27:07.848155+0000","last_peered":"2024-07-04T17:27:07.848155+0000","last_clean":"2024-07-04T17:27:07.848155+0000","last_became_active":"2024-07-04T17:27:06.871475+0000","last_became_peered":"2024-07-04T17:27:06.871475+0000","last_unstale":"2024-07-04T17:27:07.848155+0000","last_undegraded":"2024-07-04T17:27:07.848155+0000","last_fullsized":"2024-07-04T17:27:07.848155+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:09:28.414254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.918332+0000","last_change":"2024-07-04T17:27:07.911944+0000","last_active":"2024-07-04T17:27:07.918332+0000","last_peered":"2024-07-04T17:27:07.918332+0000","last_clean":"2024-07-04T17:27:07.918332+0000","last_became_active":"2024-07-04T17:27:07.911639+0000","last_became_peered":"2024-07-04T17:27:07.911639+0000","last_unstale":"2024-07-04T17:27:07.918332+0000","last_undegraded":"2024-07-04T17:27:07.918332+0000","last_fullsized":"2024-07-04T17:27:07.918332+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:06:16.576182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"26'2","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.911057+0000","last_change":"2024-07-04T17:27:06.865834+0000","last_active":"2024-07-04T17:27:07.911057+0000","last_peered":"2024-07-04T17:27:07.911057+0000","last_clean":"2024-07-04T17:27:07.911057+0000","last_became_active":"2024-07-04T17:27:06.865463+0000","last_became_peered":"2024-07-04T17:27:06.865463+0000","last_unstale":"2024-07-04T17:27:07.911057+0000","last_undegraded":"2024-07-04T17:27:07.911057+0000","last_fullsized":"2024-07-04T17:27:07.911057+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:35:43.457323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.909298+0000","last_change":"2024-07-04T17:27:07.882402+0000","last_active":"2024-07-04T17:27:07.909298+0000","last_peered":"2024-07-04T17:27:07.909298+0000","last_clean":"2024-07-04T17:27:07.909298+0000","last_became_active":"2024-07-04T17:27:07.882061+0000","last_became_peered":"2024-07-04T17:27:07.882061+0000","last_unstale":"2024-07-04T17:27:07.909298+0000","last_undegraded":"2024-07-04T17:27:07.909298+0000","last_fullsized":"2024-07-04T17:27:07.909298+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:12:04.631448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.862863+0000","last_change":"2024-07-04T17:27:06.866198+0000","last_active":"2024-07-04T17:27:07.862863+0000","last_peered":"2024-07-04T17:27:07.862863+0000","last_clean":"2024-07-04T17:27:07.862863+0000","last_became_active":"2024-07-04T17:27:06.865891+0000","last_became_peered":"2024-07-04T17:27:06.865891+0000","last_unstale":"2024-07-04T17:27:07.862863+0000","last_undegraded":"2024-07-04T17:27:07.862863+0000","last_fullsized":"2024-07-04T17:27:07.862863+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:59:47.703820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.907207+0000","last_change":"2024-07-04T17:27:07.883459+0000","last_active":"2024-07-04T17:27:07.907207+0000","last_peered":"2024-07-04T17:27:07.907207+0000","last_clean":"2024-07-04T17:27:07.907207+0000","last_became_active":"2024-07-04T17:27:07.883202+0000","last_became_peered":"2024-07-04T17:27:07.883202+0000","last_unstale":"2024-07-04T17:27:07.907207+0000","last_undegraded":"2024-07-04T17:27:07.907207+0000","last_fullsized":"2024-07-04T17:27:07.907207+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:23.798405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.844844+0000","last_change":"2024-07-04T17:27:06.872489+0000","last_active":"2024-07-04T17:27:07.844844+0000","last_peered":"2024-07-04T17:27:07.844844+0000","last_clean":"2024-07-04T17:27:07.844844+0000","last_became_active":"2024-07-04T17:27:06.872125+0000","last_became_peered":"2024-07-04T17:27:06.872125+0000","last_unstale":"2024-07-04T17:27:07.844844+0000","last_undegraded":"2024-07-04T17:27:07.844844+0000","last_fullsized":"2024-07-04T17:27:07.844844+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:10:29.469978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.897536+0000","last_change":"2024-07-04T17:27:07.878014+0000","last_active":"2024-07-04T17:27:07.897536+0000","last_peered":"2024-07-04T17:27:07.897536+0000","last_clean":"2024-07-04T17:27:07.897536+0000","last_became_active":"2024-07-04T17:27:07.874485+0000","last_became_peered":"2024-07-04T17:27:07.874485+0000","last_unstale":"2024-07-04T17:27:07.897536+0000","last_undegraded":"2024-07-04T17:27:07.897536+0000","last_fullsized":"2024-07-04T17:27:07.897536+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:43:39.860474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.861881+0000","last_change":"2024-07-04T17:27:06.870890+0000","last_active":"2024-07-04T17:27:07.861881+0000","last_peered":"2024-07-04T17:27:07.861881+0000","last_clean":"2024-07-04T17:27:07.861881+0000","last_became_active":"2024-07-04T17:27:06.870581+0000","last_became_peered":"2024-07-04T17:27:06.870581+0000","last_unstale":"2024-07-04T17:27:07.861881+0000","last_undegraded":"2024-07-04T17:27:07.861881+0000","last_fullsized":"2024-07-04T17:27:07.861881+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:55:15.415040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.906785+0000","last_change":"2024-07-04T17:27:07.882615+0000","last_active":"2024-07-04T17:27:07.906785+0000","last_peered":"2024-07-04T17:27:07.906785+0000","last_clean":"2024-07-04T17:27:07.906785+0000","last_became_active":"2024-07-04T17:27:07.882132+0000","last_became_peered":"2024-07-04T17:27:07.882132+0000","last_unstale":"2024-07-04T17:27:07.906785+0000","last_undegraded":"2024-07-04T17:27:07.906785+0000","last_fullsized":"2024-07-04T17:27:07.906785+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:21:38.559811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.849512+0000","last_change":"2024-07-04T17:27:06.861673+0000","last_active":"2024-07-04T17:27:07.849512+0000","last_peered":"2024-07-04T17:27:07.849512+0000","last_clean":"2024-07-04T17:27:07.849512+0000","last_became_active":"2024-07-04T17:27:06.861297+0000","last_became_peered":"2024-07-04T17:27:06.861297+0000","last_unstale":"2024-07-04T17:27:07.849512+0000","last_undegraded":"2024-07-04T17:27:07.849512+0000","last_fullsized":"2024-07-04T17:27:07.849512+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:10:34.165745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.917621+0000","last_change":"2024-07-04T17:27:07.909121+0000","last_active":"2024-07-04T17:27:07.917621+0000","last_peered":"2024-07-04T17:27:07.917621+0000","last_clean":"2024-07-04T17:27:07.917621+0000","last_became_active":"2024-07-04T17:27:07.908646+0000","last_became_peered":"2024-07-04T17:27:07.908646+0000","last_unstale":"2024-07-04T17:27:07.917621+0000","last_undegraded":"2024-07-04T17:27:07.917621+0000","last_fullsized":"2024-07-04T17:27:07.917621+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:52:51.670741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.867042+0000","last_change":"2024-07-04T17:27:06.883045+0000","last_active":"2024-07-04T17:27:07.867042+0000","last_peered":"2024-07-04T17:27:07.867042+0000","last_clean":"2024-07-04T17:27:07.867042+0000","last_became_active":"2024-07-04T17:27:06.882363+0000","last_became_peered":"2024-07-04T17:27:06.882363+0000","last_unstale":"2024-07-04T17:27:07.867042+0000","last_undegraded":"2024-07-04T17:27:07.867042+0000","last_fullsized":"2024-07-04T17:27:07.867042+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:05:41.513762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.885769+0000","last_change":"2024-07-04T17:27:07.871918+0000","last_active":"2024-07-04T17:27:07.885769+0000","last_peered":"2024-07-04T17:27:07.885769+0000","last_clean":"2024-07-04T17:27:07.885769+0000","last_became_active":"2024-07-04T17:27:07.871229+0000","last_became_peered":"2024-07-04T17:27:07.871229+0000","last_unstale":"2024-07-04T17:27:07.885769+0000","last_undegraded":"2024-07-04T17:27:07.885769+0000","last_fullsized":"2024-07-04T17:27:07.885769+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:01:29.057248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.858698+0000","last_change":"2024-07-04T17:27:06.859760+0000","last_active":"2024-07-04T17:27:07.858698+0000","last_peered":"2024-07-04T17:27:07.858698+0000","last_clean":"2024-07-04T17:27:07.858698+0000","last_became_active":"2024-07-04T17:27:06.859396+0000","last_became_peered":"2024-07-04T17:27:06.859396+0000","last_unstale":"2024-07-04T17:27:07.858698+0000","last_undegraded":"2024-07-04T17:27:07.858698+0000","last_fullsized":"2024-07-04T17:27:07.858698+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:48:12.553882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.908258+0000","last_change":"2024-07-04T17:27:07.891018+0000","last_active":"2024-07-04T17:27:07.908258+0000","last_peered":"2024-07-04T17:27:07.908258+0000","last_clean":"2024-07-04T17:27:07.908258+0000","last_became_active":"2024-07-04T17:27:07.889885+0000","last_became_peered":"2024-07-04T17:27:07.889885+0000","last_unstale":"2024-07-04T17:27:07.908258+0000","last_undegraded":"2024-07-04T17:27:07.908258+0000","last_fullsized":"2024-07-04T17:27:07.908258+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:26:23.039967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.848028+0000","last_change":"2024-07-04T17:27:06.874596+0000","last_active":"2024-07-04T17:27:07.848028+0000","last_peered":"2024-07-04T17:27:07.848028+0000","last_clean":"2024-07-04T17:27:07.848028+0000","last_became_active":"2024-07-04T17:27:06.874252+0000","last_became_peered":"2024-07-04T17:27:06.874252+0000","last_unstale":"2024-07-04T17:27:07.848028+0000","last_undegraded":"2024-07-04T17:27:07.848028+0000","last_fullsized":"2024-07-04T17:27:07.848028+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:17:38.211735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'52","reported_seq":107,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.841949+0000","last_change":"2024-07-04T17:26:04.953559+0000","last_active":"2024-07-04T17:27:07.841949+0000","last_peered":"2024-07-04T17:27:07.841949+0000","last_clean":"2024-07-04T17:27:07.841949+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:27:07.841949+0000","last_undegraded":"2024-07-04T17:27:07.841949+0000","last_fullsized":"2024-07-04T17:27:07.841949+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_clean_scrub_stamp":"2024-07-04T17:26:03.933432+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.064181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738384,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":241664,"data_stored":77560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738385,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":856064,"data_stored":670212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738383,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":815104,"data_stored":663337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:27:12.535 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:27:12.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.857+0000 7f6387200640 1 -- 172.21.15.42:0/958169981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f638806c2c0 msgr2=0x7f63880b5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:12.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.857+0000 7f6387200640 1 --2- 172.21.15.42:0/958169981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f638806c2c0 0x7f63880b5650 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6370009fd0 tx=0x7f637002b0b0 comp rx=0 tx=0).stop 2024-07-04T17:27:12.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.857+0000 7f6387200640 1 -- 172.21.15.42:0/958169981 shutdown_connections 2024-07-04T17:27:12.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.857+0000 7f6387200640 1 --2- 172.21.15.42:0/958169981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f638806c2c0 0x7f63880b5650 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:12.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.857+0000 7f6387200640 1 -- 172.21.15.42:0/958169981 >> 172.21.15.42:0/958169981 conn(0x7f63880bebc0 msgr2=0x7f63880befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:12.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.857+0000 7f6387200640 1 -- 172.21.15.42:0/958169981 shutdown_connections 2024-07-04T17:27:12.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.857+0000 7f6387200640 1 -- 172.21.15.42:0/958169981 wait complete. 2024-07-04T17:27:12.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.859+0000 7f6387200640 1 Processor -- start 2024-07-04T17:27:12.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.859+0000 7f6387200640 1 -- start start 2024-07-04T17:27:12.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.860+0000 7f6387200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f638806c2c0 0x7f63881366c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:12.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.860+0000 7f6387200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f63880b27c0 con 0x7f638806c2c0 2024-07-04T17:27:12.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.861+0000 7f6385e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f638806c2c0 0x7f63881366c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:12.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.861+0000 7f6385e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f638806c2c0 0x7f63881366c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38476/0 (socket says 172.21.15.42:38476) 2024-07-04T17:27:12.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.861+0000 7f6385e00640 1 -- 172.21.15.42:0/3554215682 learned_addr learned my addr 172.21.15.42:0/3554215682 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:12.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.861+0000 7f6385e00640 1 -- 172.21.15.42:0/3554215682 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6388131760 con 0x7f638806c2c0 2024-07-04T17:27:12.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.861+0000 7f6385e00640 1 --2- 172.21.15.42:0/3554215682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f638806c2c0 0x7f63881366c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f637800c380 tx=0x7f637800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:12.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.862+0000 7f637e000640 1 -- 172.21.15.42:0/3554215682 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63780116c0 con 0x7f638806c2c0 2024-07-04T17:27:12.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.862+0000 7f637e000640 1 -- 172.21.15.42:0/3554215682 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6378017070 con 0x7f638806c2c0 2024-07-04T17:27:12.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.862+0000 7f637e000640 1 -- 172.21.15.42:0/3554215682 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6378011b20 con 0x7f638806c2c0 2024-07-04T17:27:12.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.862+0000 7f6387200640 1 -- 172.21.15.42:0/3554215682 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63881e2180 con 0x7f638806c2c0 2024-07-04T17:27:12.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.862+0000 7f6387200640 1 -- 172.21.15.42:0/3554215682 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6388131e40 con 0x7f638806c2c0 2024-07-04T17:27:12.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.863+0000 7f6387200640 1 -- 172.21.15.42:0/3554215682 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6388141c70 con 0x7f638806c2c0 2024-07-04T17:27:12.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.863+0000 7f637e000640 1 -- 172.21.15.42:0/3554215682 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f6378011cc0 con 0x7f638806c2c0 2024-07-04T17:27:12.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.864+0000 7f637e000640 1 --2- 172.21.15.42:0/3554215682 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f636403d2b0 0x7f636403f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:12.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.864+0000 7f637e000640 1 -- 172.21.15.42:0/3554215682 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3439+0+0 (secure 0 0 0) 0x7f6378051340 con 0x7f638806c2c0 2024-07-04T17:27:12.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.864+0000 7f6385400640 1 --2- 172.21.15.42:0/3554215682 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f636403d2b0 0x7f636403f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:12.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.865+0000 7f6385400640 1 --2- 172.21.15.42:0/3554215682 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f636403d2b0 0x7f636403f7c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6370033000 tx=0x7f6370004ab0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:12.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:12.869+0000 7f637e000640 1 -- 172.21.15.42:0/3554215682 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f6388141c70 con 0x7f638806c2c0 2024-07-04T17:27:13.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.073+0000 7f6387200640 1 -- 172.21.15.42:0/3554215682 --> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f63880b67a0 con 0x7f636403d2b0 2024-07-04T17:27:13.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.085+0000 7f637e000640 1 -- 172.21.15.42:0/3554215682 <== mgr.4103 v2:172.21.15.42:6824/3523793950 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245890 (secure 0 0 0) 0x7f63880b67a0 con 0x7f636403d2b0 2024-07-04T17:27:13.090 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:27:13.091 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:27:13.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.090+0000 7f6387200640 1 -- 172.21.15.42:0/3554215682 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f636403d2b0 msgr2=0x7f636403f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:13.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.090+0000 7f6387200640 1 --2- 172.21.15.42:0/3554215682 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f636403d2b0 0x7f636403f7c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6370033000 tx=0x7f6370004ab0 comp rx=0 tx=0).stop 2024-07-04T17:27:13.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.090+0000 7f6387200640 1 -- 172.21.15.42:0/3554215682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f638806c2c0 msgr2=0x7f63881366c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:13.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.090+0000 7f6387200640 1 --2- 172.21.15.42:0/3554215682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f638806c2c0 0x7f63881366c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f637800c380 tx=0x7f637800c840 comp rx=0 tx=0).stop 2024-07-04T17:27:13.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.090+0000 7f6387200640 1 -- 172.21.15.42:0/3554215682 shutdown_connections 2024-07-04T17:27:13.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.090+0000 7f6387200640 1 --2- 172.21.15.42:0/3554215682 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f636403d2b0 0x7f636403f7c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:13.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.090+0000 7f6387200640 1 --2- 172.21.15.42:0/3554215682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f638806c2c0 0x7f63881366c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:13.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.090+0000 7f6387200640 1 -- 172.21.15.42:0/3554215682 >> 172.21.15.42:0/3554215682 conn(0x7f63880bebc0 msgr2=0x7f638813ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:13.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.091+0000 7f6387200640 1 -- 172.21.15.42:0/3554215682 shutdown_connections 2024-07-04T17:27:13.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.091+0000 7f6387200640 1 -- 172.21.15.42:0/3554215682 wait complete. 2024-07-04T17:27:13.116 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":59,"stamp":"2024-07-04T17:27:11.280013+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":123,"num_write_kb":1120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90,"ondisk_log_size":90,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":95412,"kb_used_data":1868,"kb_used_omap":4,"kb_used_meta":93499,"kb_avail":283020108,"statfs":{"total":289910292480,"available":289812590592,"internally_reserved":0,"allocated":1912832,"data_stored":1411109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":95743311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.462503"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.918042+0000","last_change":"2024-07-04T17:27:07.916245+0000","last_active":"2024-07-04T17:27:07.918042+0000","last_peered":"2024-07-04T17:27:07.918042+0000","last_clean":"2024-07-04T17:27:07.918042+0000","last_became_active":"2024-07-04T17:27:07.912310+0000","last_became_peered":"2024-07-04T17:27:07.912310+0000","last_unstale":"2024-07-04T17:27:07.918042+0000","last_undegraded":"2024-07-04T17:27:07.918042+0000","last_fullsized":"2024-07-04T17:27:07.918042+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:57:29.976142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.907428+0000","last_change":"2024-07-04T17:27:07.884739+0000","last_active":"2024-07-04T17:27:07.907428+0000","last_peered":"2024-07-04T17:27:07.907428+0000","last_clean":"2024-07-04T17:27:07.907428+0000","last_became_active":"2024-07-04T17:27:07.884425+0000","last_became_peered":"2024-07-04T17:27:07.884425+0000","last_unstale":"2024-07-04T17:27:07.907428+0000","last_undegraded":"2024-07-04T17:27:07.907428+0000","last_fullsized":"2024-07-04T17:27:07.907428+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:56:08.436905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.900778+0000","last_change":"2024-07-04T17:27:07.891626+0000","last_active":"2024-07-04T17:27:07.900778+0000","last_peered":"2024-07-04T17:27:07.900778+0000","last_clean":"2024-07-04T17:27:07.900778+0000","last_became_active":"2024-07-04T17:27:07.891070+0000","last_became_peered":"2024-07-04T17:27:07.891070+0000","last_unstale":"2024-07-04T17:27:07.900778+0000","last_undegraded":"2024-07-04T17:27:07.900778+0000","last_fullsized":"2024-07-04T17:27:07.900778+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:32:46.895301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.893681+0000","last_change":"2024-07-04T17:27:07.889861+0000","last_active":"2024-07-04T17:27:07.893681+0000","last_peered":"2024-07-04T17:27:07.893681+0000","last_clean":"2024-07-04T17:27:07.893681+0000","last_became_active":"2024-07-04T17:27:07.889003+0000","last_became_peered":"2024-07-04T17:27:07.889003+0000","last_unstale":"2024-07-04T17:27:07.893681+0000","last_undegraded":"2024-07-04T17:27:07.893681+0000","last_fullsized":"2024-07-04T17:27:07.893681+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.727000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.917350+0000","last_change":"2024-07-04T17:27:07.905295+0000","last_active":"2024-07-04T17:27:07.917350+0000","last_peered":"2024-07-04T17:27:07.917350+0000","last_clean":"2024-07-04T17:27:07.917350+0000","last_became_active":"2024-07-04T17:27:07.904234+0000","last_became_peered":"2024-07-04T17:27:07.904234+0000","last_unstale":"2024-07-04T17:27:07.917350+0000","last_undegraded":"2024-07-04T17:27:07.917350+0000","last_fullsized":"2024-07-04T17:27:07.917350+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:01:29.057248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.885941+0000","last_change":"2024-07-04T17:27:07.883415+0000","last_active":"2024-07-04T17:27:07.885941+0000","last_peered":"2024-07-04T17:27:07.885941+0000","last_clean":"2024-07-04T17:27:07.885941+0000","last_became_active":"2024-07-04T17:27:07.878729+0000","last_became_peered":"2024-07-04T17:27:07.878729+0000","last_unstale":"2024-07-04T17:27:07.885941+0000","last_undegraded":"2024-07-04T17:27:07.885941+0000","last_fullsized":"2024-07-04T17:27:07.885941+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:16:39.185466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.898812+0000","last_change":"2024-07-04T17:27:07.894901+0000","last_active":"2024-07-04T17:27:07.898812+0000","last_peered":"2024-07-04T17:27:07.898812+0000","last_clean":"2024-07-04T17:27:07.898812+0000","last_became_active":"2024-07-04T17:27:07.894469+0000","last_became_peered":"2024-07-04T17:27:07.894469+0000","last_unstale":"2024-07-04T17:27:07.898812+0000","last_undegraded":"2024-07-04T17:27:07.898812+0000","last_fullsized":"2024-07-04T17:27:07.898812+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:19:44.491304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.901643+0000","last_change":"2024-07-04T17:27:07.899412+0000","last_active":"2024-07-04T17:27:07.901643+0000","last_peered":"2024-07-04T17:27:07.901643+0000","last_clean":"2024-07-04T17:27:07.901643+0000","last_became_active":"2024-07-04T17:27:07.898386+0000","last_became_peered":"2024-07-04T17:27:07.898386+0000","last_unstale":"2024-07-04T17:27:07.901643+0000","last_undegraded":"2024-07-04T17:27:07.901643+0000","last_fullsized":"2024-07-04T17:27:07.901643+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:10:35.175672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.900628+0000","last_change":"2024-07-04T17:27:07.897001+0000","last_active":"2024-07-04T17:27:07.900628+0000","last_peered":"2024-07-04T17:27:07.900628+0000","last_clean":"2024-07-04T17:27:07.900628+0000","last_became_active":"2024-07-04T17:27:07.896453+0000","last_became_peered":"2024-07-04T17:27:07.896453+0000","last_unstale":"2024-07-04T17:27:07.900628+0000","last_undegraded":"2024-07-04T17:27:07.900628+0000","last_fullsized":"2024-07-04T17:27:07.900628+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:48:13.563809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.878674+0000","last_change":"2024-07-04T17:27:07.871832+0000","last_active":"2024-07-04T17:27:07.878674+0000","last_peered":"2024-07-04T17:27:07.878674+0000","last_clean":"2024-07-04T17:27:07.878674+0000","last_became_active":"2024-07-04T17:27:07.870928+0000","last_became_peered":"2024-07-04T17:27:07.870928+0000","last_unstale":"2024-07-04T17:27:07.878674+0000","last_undegraded":"2024-07-04T17:27:07.878674+0000","last_fullsized":"2024-07-04T17:27:07.878674+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:55:31.586864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.909301+0000","last_change":"2024-07-04T17:27:07.893125+0000","last_active":"2024-07-04T17:27:07.909301+0000","last_peered":"2024-07-04T17:27:07.909301+0000","last_clean":"2024-07-04T17:27:07.909301+0000","last_became_active":"2024-07-04T17:27:07.892921+0000","last_became_peered":"2024-07-04T17:27:07.892921+0000","last_unstale":"2024-07-04T17:27:07.909301+0000","last_undegraded":"2024-07-04T17:27:07.909301+0000","last_fullsized":"2024-07-04T17:27:07.909301+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:56:46.200219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.889611+0000","last_change":"2024-07-04T17:27:07.886093+0000","last_active":"2024-07-04T17:27:07.889611+0000","last_peered":"2024-07-04T17:27:07.889611+0000","last_clean":"2024-07-04T17:27:07.889611+0000","last_became_active":"2024-07-04T17:27:07.885701+0000","last_became_peered":"2024-07-04T17:27:07.885701+0000","last_unstale":"2024-07-04T17:27:07.889611+0000","last_undegraded":"2024-07-04T17:27:07.889611+0000","last_fullsized":"2024-07-04T17:27:07.889611+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:20:20.786366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.899347+0000","last_change":"2024-07-04T17:27:07.896119+0000","last_active":"2024-07-04T17:27:07.899347+0000","last_peered":"2024-07-04T17:27:07.899347+0000","last_clean":"2024-07-04T17:27:07.899347+0000","last_became_active":"2024-07-04T17:27:07.895583+0000","last_became_peered":"2024-07-04T17:27:07.895583+0000","last_unstale":"2024-07-04T17:27:07.899347+0000","last_undegraded":"2024-07-04T17:27:07.899347+0000","last_fullsized":"2024-07-04T17:27:07.899347+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:17:39.221662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.897612+0000","last_change":"2024-07-04T17:27:07.879943+0000","last_active":"2024-07-04T17:27:07.897612+0000","last_peered":"2024-07-04T17:27:07.897612+0000","last_clean":"2024-07-04T17:27:07.897612+0000","last_became_active":"2024-07-04T17:27:07.875927+0000","last_became_peered":"2024-07-04T17:27:07.875927+0000","last_unstale":"2024-07-04T17:27:07.897612+0000","last_undegraded":"2024-07-04T17:27:07.897612+0000","last_fullsized":"2024-07-04T17:27:07.897612+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:27:04.910602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.890336+0000","last_change":"2024-07-04T17:27:07.873017+0000","last_active":"2024-07-04T17:27:07.890336+0000","last_peered":"2024-07-04T17:27:07.890336+0000","last_clean":"2024-07-04T17:27:07.890336+0000","last_became_active":"2024-07-04T17:27:07.872715+0000","last_became_peered":"2024-07-04T17:27:07.872715+0000","last_unstale":"2024-07-04T17:27:07.890336+0000","last_undegraded":"2024-07-04T17:27:07.890336+0000","last_fullsized":"2024-07-04T17:27:07.890336+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:38:38.667113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.907454+0000","last_change":"2024-07-04T17:27:07.884482+0000","last_active":"2024-07-04T17:27:07.907454+0000","last_peered":"2024-07-04T17:27:07.907454+0000","last_clean":"2024-07-04T17:27:07.907454+0000","last_became_active":"2024-07-04T17:27:07.882978+0000","last_became_peered":"2024-07-04T17:27:07.882978+0000","last_unstale":"2024-07-04T17:27:07.907454+0000","last_undegraded":"2024-07-04T17:27:07.907454+0000","last_fullsized":"2024-07-04T17:27:07.907454+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:27:08.653236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.898899+0000","last_change":"2024-07-04T17:27:07.877401+0000","last_active":"2024-07-04T17:27:07.898899+0000","last_peered":"2024-07-04T17:27:07.898899+0000","last_clean":"2024-07-04T17:27:07.898899+0000","last_became_active":"2024-07-04T17:27:07.877014+0000","last_became_peered":"2024-07-04T17:27:07.877014+0000","last_unstale":"2024-07-04T17:27:07.898899+0000","last_undegraded":"2024-07-04T17:27:07.898899+0000","last_fullsized":"2024-07-04T17:27:07.898899+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:24:58.267453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.907361+0000","last_change":"2024-07-04T17:27:07.883614+0000","last_active":"2024-07-04T17:27:07.907361+0000","last_peered":"2024-07-04T17:27:07.907361+0000","last_clean":"2024-07-04T17:27:07.907361+0000","last_became_active":"2024-07-04T17:27:07.883289+0000","last_became_peered":"2024-07-04T17:27:07.883289+0000","last_unstale":"2024-07-04T17:27:07.907361+0000","last_undegraded":"2024-07-04T17:27:07.907361+0000","last_fullsized":"2024-07-04T17:27:07.907361+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:06:36.067329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.909268+0000","last_change":"2024-07-04T17:27:07.893065+0000","last_active":"2024-07-04T17:27:07.909268+0000","last_peered":"2024-07-04T17:27:07.909268+0000","last_clean":"2024-07-04T17:27:07.909268+0000","last_became_active":"2024-07-04T17:27:07.892736+0000","last_became_peered":"2024-07-04T17:27:07.892736+0000","last_unstale":"2024-07-04T17:27:07.909268+0000","last_undegraded":"2024-07-04T17:27:07.909268+0000","last_fullsized":"2024-07-04T17:27:07.909268+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:40:58.276630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.917950+0000","last_change":"2024-07-04T17:27:07.916151+0000","last_active":"2024-07-04T17:27:07.917950+0000","last_peered":"2024-07-04T17:27:07.917950+0000","last_clean":"2024-07-04T17:27:07.917950+0000","last_became_active":"2024-07-04T17:27:07.909999+0000","last_became_peered":"2024-07-04T17:27:07.909999+0000","last_unstale":"2024-07-04T17:27:07.917950+0000","last_undegraded":"2024-07-04T17:27:07.917950+0000","last_fullsized":"2024-07-04T17:27:07.917950+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:32:46.895301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.908720+0000","last_change":"2024-07-04T17:27:07.887157+0000","last_active":"2024-07-04T17:27:07.908720+0000","last_peered":"2024-07-04T17:27:07.908720+0000","last_clean":"2024-07-04T17:27:07.908720+0000","last_became_active":"2024-07-04T17:27:07.886783+0000","last_became_peered":"2024-07-04T17:27:07.886783+0000","last_unstale":"2024-07-04T17:27:07.908720+0000","last_undegraded":"2024-07-04T17:27:07.908720+0000","last_fullsized":"2024-07-04T17:27:07.908720+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:34:35.587534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.917662+0000","last_change":"2024-07-04T17:27:07.904537+0000","last_active":"2024-07-04T17:27:07.917662+0000","last_peered":"2024-07-04T17:27:07.917662+0000","last_clean":"2024-07-04T17:27:07.917662+0000","last_became_active":"2024-07-04T17:27:07.903476+0000","last_became_peered":"2024-07-04T17:27:07.903476+0000","last_unstale":"2024-07-04T17:27:07.917662+0000","last_undegraded":"2024-07-04T17:27:07.917662+0000","last_fullsized":"2024-07-04T17:27:07.917662+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:55:31.586864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.907929+0000","last_change":"2024-07-04T17:27:07.883846+0000","last_active":"2024-07-04T17:27:07.907929+0000","last_peered":"2024-07-04T17:27:07.907929+0000","last_clean":"2024-07-04T17:27:07.907929+0000","last_became_active":"2024-07-04T17:27:07.881609+0000","last_became_peered":"2024-07-04T17:27:07.881609+0000","last_unstale":"2024-07-04T17:27:07.907929+0000","last_undegraded":"2024-07-04T17:27:07.907929+0000","last_fullsized":"2024-07-04T17:27:07.907929+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:38:45.217394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.901619+0000","last_change":"2024-07-04T17:27:07.898354+0000","last_active":"2024-07-04T17:27:07.901619+0000","last_peered":"2024-07-04T17:27:07.901619+0000","last_clean":"2024-07-04T17:27:07.901619+0000","last_became_active":"2024-07-04T17:27:07.897988+0000","last_became_peered":"2024-07-04T17:27:07.897988+0000","last_unstale":"2024-07-04T17:27:07.901619+0000","last_undegraded":"2024-07-04T17:27:07.901619+0000","last_fullsized":"2024-07-04T17:27:07.901619+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:23:41.800377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.900613+0000","last_change":"2024-07-04T17:27:07.896753+0000","last_active":"2024-07-04T17:27:07.900613+0000","last_peered":"2024-07-04T17:27:07.900613+0000","last_clean":"2024-07-04T17:27:07.900613+0000","last_became_active":"2024-07-04T17:27:07.896053+0000","last_became_peered":"2024-07-04T17:27:07.896053+0000","last_unstale":"2024-07-04T17:27:07.900613+0000","last_undegraded":"2024-07-04T17:27:07.900613+0000","last_fullsized":"2024-07-04T17:27:07.900613+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:57:29.976142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.885849+0000","last_change":"2024-07-04T17:27:07.877946+0000","last_active":"2024-07-04T17:27:07.885849+0000","last_peered":"2024-07-04T17:27:07.885849+0000","last_clean":"2024-07-04T17:27:07.885849+0000","last_became_active":"2024-07-04T17:27:07.877581+0000","last_became_peered":"2024-07-04T17:27:07.877581+0000","last_unstale":"2024-07-04T17:27:07.885849+0000","last_undegraded":"2024-07-04T17:27:07.885849+0000","last_fullsized":"2024-07-04T17:27:07.885849+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:31:06.033299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.909621+0000","last_change":"2024-07-04T17:27:07.901813+0000","last_active":"2024-07-04T17:27:07.909621+0000","last_peered":"2024-07-04T17:27:07.909621+0000","last_clean":"2024-07-04T17:27:07.909621+0000","last_became_active":"2024-07-04T17:27:07.901333+0000","last_became_peered":"2024-07-04T17:27:07.901333+0000","last_unstale":"2024-07-04T17:27:07.909621+0000","last_undegraded":"2024-07-04T17:27:07.909621+0000","last_fullsized":"2024-07-04T17:27:07.909621+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:25:52.643999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.917928+0000","last_change":"2024-07-04T17:27:07.911625+0000","last_active":"2024-07-04T17:27:07.917928+0000","last_peered":"2024-07-04T17:27:07.917928+0000","last_clean":"2024-07-04T17:27:07.917928+0000","last_became_active":"2024-07-04T17:27:07.910841+0000","last_became_peered":"2024-07-04T17:27:07.910841+0000","last_unstale":"2024-07-04T17:27:07.917928+0000","last_undegraded":"2024-07-04T17:27:07.917928+0000","last_fullsized":"2024-07-04T17:27:07.917928+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:20:20.786366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.907327+0000","last_change":"2024-07-04T17:27:07.883042+0000","last_active":"2024-07-04T17:27:07.907327+0000","last_peered":"2024-07-04T17:27:07.907327+0000","last_clean":"2024-07-04T17:27:07.907327+0000","last_became_active":"2024-07-04T17:27:07.882772+0000","last_became_peered":"2024-07-04T17:27:07.882772+0000","last_unstale":"2024-07-04T17:27:07.907327+0000","last_undegraded":"2024-07-04T17:27:07.907327+0000","last_fullsized":"2024-07-04T17:27:07.907327+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:34:46.000005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.897584+0000","last_change":"2024-07-04T17:27:07.878292+0000","last_active":"2024-07-04T17:27:07.897584+0000","last_peered":"2024-07-04T17:27:07.897584+0000","last_clean":"2024-07-04T17:27:07.897584+0000","last_became_active":"2024-07-04T17:27:07.874906+0000","last_became_peered":"2024-07-04T17:27:07.874906+0000","last_unstale":"2024-07-04T17:27:07.897584+0000","last_undegraded":"2024-07-04T17:27:07.897584+0000","last_fullsized":"2024-07-04T17:27:07.897584+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:47:16.318473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.917482+0000","last_change":"2024-07-04T17:27:07.906719+0000","last_active":"2024-07-04T17:27:07.917482+0000","last_peered":"2024-07-04T17:27:07.917482+0000","last_clean":"2024-07-04T17:27:07.917482+0000","last_became_active":"2024-07-04T17:27:07.903370+0000","last_became_peered":"2024-07-04T17:27:07.903370+0000","last_unstale":"2024-07-04T17:27:07.917482+0000","last_undegraded":"2024-07-04T17:27:07.917482+0000","last_fullsized":"2024-07-04T17:27:07.917482+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:12:04.323408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.917745+0000","last_change":"2024-07-04T17:27:07.910603+0000","last_active":"2024-07-04T17:27:07.917745+0000","last_peered":"2024-07-04T17:27:07.917745+0000","last_clean":"2024-07-04T17:27:07.917745+0000","last_became_active":"2024-07-04T17:27:07.908845+0000","last_became_peered":"2024-07-04T17:27:07.908845+0000","last_unstale":"2024-07-04T17:27:07.917745+0000","last_undegraded":"2024-07-04T17:27:07.917745+0000","last_fullsized":"2024-07-04T17:27:07.917745+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:16:39.185466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.907663+0000","last_change":"2024-07-04T17:27:07.888430+0000","last_active":"2024-07-04T17:27:07.907663+0000","last_peered":"2024-07-04T17:27:07.907663+0000","last_clean":"2024-07-04T17:27:07.907663+0000","last_became_active":"2024-07-04T17:27:07.886722+0000","last_became_peered":"2024-07-04T17:27:07.886722+0000","last_unstale":"2024-07-04T17:27:07.907663+0000","last_undegraded":"2024-07-04T17:27:07.907663+0000","last_fullsized":"2024-07-04T17:27:07.907663+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:20:22.623733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"26'4","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.913855+0000","last_change":"2024-07-04T17:27:06.866478+0000","last_active":"2024-07-04T17:27:07.913855+0000","last_peered":"2024-07-04T17:27:07.913855+0000","last_clean":"2024-07-04T17:27:07.913855+0000","last_became_active":"2024-07-04T17:27:06.865591+0000","last_became_peered":"2024-07-04T17:27:06.865591+0000","last_unstale":"2024-07-04T17:27:07.913855+0000","last_undegraded":"2024-07-04T17:27:07.913855+0000","last_fullsized":"2024-07-04T17:27:07.913855+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:09:15.298068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.900211+0000","last_change":"2024-07-04T17:27:07.896653+0000","last_active":"2024-07-04T17:27:07.900211+0000","last_peered":"2024-07-04T17:27:07.900211+0000","last_clean":"2024-07-04T17:27:07.900211+0000","last_became_active":"2024-07-04T17:27:07.895759+0000","last_became_peered":"2024-07-04T17:27:07.895759+0000","last_unstale":"2024-07-04T17:27:07.900211+0000","last_undegraded":"2024-07-04T17:27:07.900211+0000","last_fullsized":"2024-07-04T17:27:07.900211+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:09:29.424181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"26'1","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.892150+0000","last_change":"2024-07-04T17:27:06.863457+0000","last_active":"2024-07-04T17:27:07.892150+0000","last_peered":"2024-07-04T17:27:07.892150+0000","last_clean":"2024-07-04T17:27:07.892150+0000","last_became_active":"2024-07-04T17:27:06.862649+0000","last_became_peered":"2024-07-04T17:27:06.862649+0000","last_unstale":"2024-07-04T17:27:07.892150+0000","last_undegraded":"2024-07-04T17:27:07.892150+0000","last_fullsized":"2024-07-04T17:27:07.892150+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:49:24.997305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.917781+0000","last_change":"2024-07-04T17:27:07.911112+0000","last_active":"2024-07-04T17:27:07.917781+0000","last_peered":"2024-07-04T17:27:07.917781+0000","last_clean":"2024-07-04T17:27:07.917781+0000","last_became_active":"2024-07-04T17:27:07.910473+0000","last_became_peered":"2024-07-04T17:27:07.910473+0000","last_unstale":"2024-07-04T17:27:07.917781+0000","last_undegraded":"2024-07-04T17:27:07.917781+0000","last_fullsized":"2024-07-04T17:27:07.917781+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:22:57.186029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"26'2","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.897532+0000","last_change":"2024-07-04T17:27:06.860195+0000","last_active":"2024-07-04T17:27:07.897532+0000","last_peered":"2024-07-04T17:27:07.897532+0000","last_clean":"2024-07-04T17:27:07.897532+0000","last_became_active":"2024-07-04T17:27:06.859440+0000","last_became_peered":"2024-07-04T17:27:06.859440+0000","last_unstale":"2024-07-04T17:27:07.897532+0000","last_undegraded":"2024-07-04T17:27:07.897532+0000","last_fullsized":"2024-07-04T17:27:07.897532+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:11:21.518052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.907560+0000","last_change":"2024-07-04T17:27:07.901586+0000","last_active":"2024-07-04T17:27:07.907560+0000","last_peered":"2024-07-04T17:27:07.907560+0000","last_clean":"2024-07-04T17:27:07.907560+0000","last_became_active":"2024-07-04T17:27:07.900901+0000","last_became_peered":"2024-07-04T17:27:07.900901+0000","last_unstale":"2024-07-04T17:27:07.907560+0000","last_undegraded":"2024-07-04T17:27:07.907560+0000","last_fullsized":"2024-07-04T17:27:07.907560+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:11:22.527978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"26'2","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.898863+0000","last_change":"2024-07-04T17:27:06.885584+0000","last_active":"2024-07-04T17:27:07.898863+0000","last_peered":"2024-07-04T17:27:07.898863+0000","last_clean":"2024-07-04T17:27:07.898863+0000","last_became_active":"2024-07-04T17:27:06.885226+0000","last_became_peered":"2024-07-04T17:27:06.885226+0000","last_unstale":"2024-07-04T17:27:07.898863+0000","last_undegraded":"2024-07-04T17:27:07.898863+0000","last_fullsized":"2024-07-04T17:27:07.898863+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:10:29.469978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.878698+0000","last_change":"2024-07-04T17:27:07.872169+0000","last_active":"2024-07-04T17:27:07.878698+0000","last_peered":"2024-07-04T17:27:07.878698+0000","last_clean":"2024-07-04T17:27:07.878698+0000","last_became_active":"2024-07-04T17:27:07.870986+0000","last_became_peered":"2024-07-04T17:27:07.870986+0000","last_unstale":"2024-07-04T17:27:07.878698+0000","last_undegraded":"2024-07-04T17:27:07.878698+0000","last_fullsized":"2024-07-04T17:27:07.878698+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:16:25.982847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"26'1","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.899296+0000","last_change":"2024-07-04T17:27:06.883716+0000","last_active":"2024-07-04T17:27:07.899296+0000","last_peered":"2024-07-04T17:27:07.899296+0000","last_clean":"2024-07-04T17:27:07.899296+0000","last_became_active":"2024-07-04T17:27:06.882604+0000","last_became_peered":"2024-07-04T17:27:06.882604+0000","last_unstale":"2024-07-04T17:27:07.899296+0000","last_undegraded":"2024-07-04T17:27:07.899296+0000","last_fullsized":"2024-07-04T17:27:07.899296+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:04:44.886174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.915617+0000","last_change":"2024-07-04T17:27:07.904214+0000","last_active":"2024-07-04T17:27:07.915617+0000","last_peered":"2024-07-04T17:27:07.915617+0000","last_clean":"2024-07-04T17:27:07.915617+0000","last_became_active":"2024-07-04T17:27:07.903782+0000","last_became_peered":"2024-07-04T17:27:07.903782+0000","last_unstale":"2024-07-04T17:27:07.915617+0000","last_undegraded":"2024-07-04T17:27:07.915617+0000","last_fullsized":"2024-07-04T17:27:07.915617+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:35:44.467250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"26'1","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.905336+0000","last_change":"2024-07-04T17:27:06.881468+0000","last_active":"2024-07-04T17:27:07.905336+0000","last_peered":"2024-07-04T17:27:07.905336+0000","last_clean":"2024-07-04T17:27:07.905336+0000","last_became_active":"2024-07-04T17:27:06.881150+0000","last_became_peered":"2024-07-04T17:27:06.881150+0000","last_unstale":"2024-07-04T17:27:07.905336+0000","last_undegraded":"2024-07-04T17:27:07.905336+0000","last_fullsized":"2024-07-04T17:27:07.905336+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:49:39.642772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.886191+0000","last_change":"2024-07-04T17:27:07.866107+0000","last_active":"2024-07-04T17:27:07.886191+0000","last_peered":"2024-07-04T17:27:07.886191+0000","last_clean":"2024-07-04T17:27:07.886191+0000","last_became_active":"2024-07-04T17:27:07.865761+0000","last_became_peered":"2024-07-04T17:27:07.865761+0000","last_unstale":"2024-07-04T17:27:07.886191+0000","last_undegraded":"2024-07-04T17:27:07.886191+0000","last_fullsized":"2024-07-04T17:27:07.886191+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:29:13.459059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.858077+0000","last_change":"2024-07-04T17:27:06.872335+0000","last_active":"2024-07-04T17:27:07.858077+0000","last_peered":"2024-07-04T17:27:07.858077+0000","last_clean":"2024-07-04T17:27:07.858077+0000","last_became_active":"2024-07-04T17:27:06.871960+0000","last_became_peered":"2024-07-04T17:27:06.871960+0000","last_unstale":"2024-07-04T17:27:07.858077+0000","last_undegraded":"2024-07-04T17:27:07.858077+0000","last_fullsized":"2024-07-04T17:27:07.858077+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:57:28.966215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.889305+0000","last_change":"2024-07-04T17:27:07.885943+0000","last_active":"2024-07-04T17:27:07.889305+0000","last_peered":"2024-07-04T17:27:07.889305+0000","last_clean":"2024-07-04T17:27:07.889305+0000","last_became_active":"2024-07-04T17:27:07.876919+0000","last_became_peered":"2024-07-04T17:27:07.876919+0000","last_unstale":"2024-07-04T17:27:07.889305+0000","last_undegraded":"2024-07-04T17:27:07.889305+0000","last_fullsized":"2024-07-04T17:27:07.889305+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:22:57.186029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.870646+0000","last_change":"2024-07-04T17:27:06.883828+0000","last_active":"2024-07-04T17:27:07.870646+0000","last_peered":"2024-07-04T17:27:07.870646+0000","last_clean":"2024-07-04T17:27:07.870646+0000","last_became_active":"2024-07-04T17:27:06.883508+0000","last_became_peered":"2024-07-04T17:27:06.883508+0000","last_unstale":"2024-07-04T17:27:07.870646+0000","last_undegraded":"2024-07-04T17:27:07.870646+0000","last_fullsized":"2024-07-04T17:27:07.870646+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:44:59.048384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.885152+0000","last_change":"2024-07-04T17:27:07.877680+0000","last_active":"2024-07-04T17:27:07.885152+0000","last_peered":"2024-07-04T17:27:07.885152+0000","last_clean":"2024-07-04T17:27:07.885152+0000","last_became_active":"2024-07-04T17:27:07.877486+0000","last_became_peered":"2024-07-04T17:27:07.877486+0000","last_unstale":"2024-07-04T17:27:07.885152+0000","last_undegraded":"2024-07-04T17:27:07.885152+0000","last_fullsized":"2024-07-04T17:27:07.885152+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:12:04.323408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.862548+0000","last_change":"2024-07-04T17:27:06.861151+0000","last_active":"2024-07-04T17:27:07.862548+0000","last_peered":"2024-07-04T17:27:07.862548+0000","last_clean":"2024-07-04T17:27:07.862548+0000","last_became_active":"2024-07-04T17:27:06.860975+0000","last_became_peered":"2024-07-04T17:27:06.860975+0000","last_unstale":"2024-07-04T17:27:07.862548+0000","last_undegraded":"2024-07-04T17:27:07.862548+0000","last_fullsized":"2024-07-04T17:27:07.862548+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:17:47.882494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.902977+0000","last_change":"2024-07-04T17:27:07.898440+0000","last_active":"2024-07-04T17:27:07.902977+0000","last_peered":"2024-07-04T17:27:07.902977+0000","last_clean":"2024-07-04T17:27:07.902977+0000","last_became_active":"2024-07-04T17:27:07.897813+0000","last_became_peered":"2024-07-04T17:27:07.897813+0000","last_unstale":"2024-07-04T17:27:07.902977+0000","last_undegraded":"2024-07-04T17:27:07.902977+0000","last_fullsized":"2024-07-04T17:27:07.902977+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:52:16.369596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.862680+0000","last_change":"2024-07-04T17:27:06.873231+0000","last_active":"2024-07-04T17:27:07.862680+0000","last_peered":"2024-07-04T17:27:07.862680+0000","last_clean":"2024-07-04T17:27:07.862680+0000","last_became_active":"2024-07-04T17:27:06.872934+0000","last_became_peered":"2024-07-04T17:27:06.872934+0000","last_unstale":"2024-07-04T17:27:07.862680+0000","last_undegraded":"2024-07-04T17:27:07.862680+0000","last_fullsized":"2024-07-04T17:27:07.862680+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:42:02.672719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.908299+0000","last_change":"2024-07-04T17:27:07.891116+0000","last_active":"2024-07-04T17:27:07.908299+0000","last_peered":"2024-07-04T17:27:07.908299+0000","last_clean":"2024-07-04T17:27:07.908299+0000","last_became_active":"2024-07-04T17:27:07.890572+0000","last_became_peered":"2024-07-04T17:27:07.890572+0000","last_unstale":"2024-07-04T17:27:07.908299+0000","last_undegraded":"2024-07-04T17:27:07.908299+0000","last_fullsized":"2024-07-04T17:27:07.908299+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:18:55.534844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"26'3","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.897648+0000","last_change":"2024-07-04T17:27:06.868145+0000","last_active":"2024-07-04T17:27:07.897648+0000","last_peered":"2024-07-04T17:27:07.897648+0000","last_clean":"2024-07-04T17:27:07.897648+0000","last_became_active":"2024-07-04T17:27:06.867751+0000","last_became_peered":"2024-07-04T17:27:06.867751+0000","last_unstale":"2024-07-04T17:27:07.897648+0000","last_undegraded":"2024-07-04T17:27:07.897648+0000","last_fullsized":"2024-07-04T17:27:07.897648+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:13:08.657441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.907645+0000","last_change":"2024-07-04T17:27:07.889067+0000","last_active":"2024-07-04T17:27:07.907645+0000","last_peered":"2024-07-04T17:27:07.907645+0000","last_clean":"2024-07-04T17:27:07.907645+0000","last_became_active":"2024-07-04T17:27:07.888715+0000","last_became_peered":"2024-07-04T17:27:07.888715+0000","last_unstale":"2024-07-04T17:27:07.907645+0000","last_undegraded":"2024-07-04T17:27:07.907645+0000","last_fullsized":"2024-07-04T17:27:07.907645+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:35:18.197218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"26'6","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.915686+0000","last_change":"2024-07-04T17:27:06.872191+0000","last_active":"2024-07-04T17:27:07.915686+0000","last_peered":"2024-07-04T17:27:07.915686+0000","last_clean":"2024-07-04T17:27:07.915686+0000","last_became_active":"2024-07-04T17:27:06.871594+0000","last_became_peered":"2024-07-04T17:27:06.871594+0000","last_unstale":"2024-07-04T17:27:07.915686+0000","last_undegraded":"2024-07-04T17:27:07.915686+0000","last_fullsized":"2024-07-04T17:27:07.915686+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:05:09.285237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.885791+0000","last_change":"2024-07-04T17:27:07.878309+0000","last_active":"2024-07-04T17:27:07.885791+0000","last_peered":"2024-07-04T17:27:07.885791+0000","last_clean":"2024-07-04T17:27:07.885791+0000","last_became_active":"2024-07-04T17:27:07.877975+0000","last_became_peered":"2024-07-04T17:27:07.877975+0000","last_unstale":"2024-07-04T17:27:07.885791+0000","last_undegraded":"2024-07-04T17:27:07.885791+0000","last_fullsized":"2024-07-04T17:27:07.885791+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:52:51.670741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"26'2","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.906095+0000","last_change":"2024-07-04T17:27:06.883778+0000","last_active":"2024-07-04T17:27:07.906095+0000","last_peered":"2024-07-04T17:27:07.906095+0000","last_clean":"2024-07-04T17:27:07.906095+0000","last_became_active":"2024-07-04T17:27:06.883449+0000","last_became_peered":"2024-07-04T17:27:06.883449+0000","last_unstale":"2024-07-04T17:27:07.906095+0000","last_undegraded":"2024-07-04T17:27:07.906095+0000","last_fullsized":"2024-07-04T17:27:07.906095+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:05:09.285237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.891647+0000","last_change":"2024-07-04T17:27:07.886433+0000","last_active":"2024-07-04T17:27:07.891647+0000","last_peered":"2024-07-04T17:27:07.891647+0000","last_clean":"2024-07-04T17:27:07.891647+0000","last_became_active":"2024-07-04T17:27:07.878497+0000","last_became_peered":"2024-07-04T17:27:07.878497+0000","last_unstale":"2024-07-04T17:27:07.891647+0000","last_undegraded":"2024-07-04T17:27:07.891647+0000","last_fullsized":"2024-07-04T17:27:07.891647+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:18:54.268323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.848972+0000","last_change":"2024-07-04T17:27:06.873153+0000","last_active":"2024-07-04T17:27:07.848972+0000","last_peered":"2024-07-04T17:27:07.848972+0000","last_clean":"2024-07-04T17:27:07.848972+0000","last_became_active":"2024-07-04T17:27:06.872701+0000","last_became_peered":"2024-07-04T17:27:06.872701+0000","last_unstale":"2024-07-04T17:27:07.848972+0000","last_undegraded":"2024-07-04T17:27:07.848972+0000","last_fullsized":"2024-07-04T17:27:07.848972+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:23:40.790450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.917599+0000","last_change":"2024-07-04T17:27:07.909034+0000","last_active":"2024-07-04T17:27:07.917599+0000","last_peered":"2024-07-04T17:27:07.917599+0000","last_clean":"2024-07-04T17:27:07.917599+0000","last_became_active":"2024-07-04T17:27:07.908392+0000","last_became_peered":"2024-07-04T17:27:07.908392+0000","last_unstale":"2024-07-04T17:27:07.917599+0000","last_undegraded":"2024-07-04T17:27:07.917599+0000","last_fullsized":"2024-07-04T17:27:07.917599+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:31:06.033299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.868421+0000","last_change":"2024-07-04T17:27:06.883046+0000","last_active":"2024-07-04T17:27:07.868421+0000","last_peered":"2024-07-04T17:27:07.868421+0000","last_clean":"2024-07-04T17:27:07.868421+0000","last_became_active":"2024-07-04T17:27:06.881811+0000","last_became_peered":"2024-07-04T17:27:06.881811+0000","last_unstale":"2024-07-04T17:27:07.868421+0000","last_undegraded":"2024-07-04T17:27:07.868421+0000","last_fullsized":"2024-07-04T17:27:07.868421+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:40:58.357800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.890930+0000","last_change":"2024-07-04T17:27:07.886267+0000","last_active":"2024-07-04T17:27:07.890930+0000","last_peered":"2024-07-04T17:27:07.890930+0000","last_clean":"2024-07-04T17:27:07.890930+0000","last_became_active":"2024-07-04T17:27:07.885526+0000","last_became_peered":"2024-07-04T17:27:07.885526+0000","last_unstale":"2024-07-04T17:27:07.890930+0000","last_undegraded":"2024-07-04T17:27:07.890930+0000","last_fullsized":"2024-07-04T17:27:07.890930+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:06:16.576182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"26'8","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.914645+0000","last_change":"2024-07-04T17:27:06.865126+0000","last_active":"2024-07-04T17:27:07.914645+0000","last_peered":"2024-07-04T17:27:07.914645+0000","last_clean":"2024-07-04T17:27:07.914645+0000","last_became_active":"2024-07-04T17:27:06.864781+0000","last_became_peered":"2024-07-04T17:27:06.864781+0000","last_unstale":"2024-07-04T17:27:07.914645+0000","last_undegraded":"2024-07-04T17:27:07.914645+0000","last_fullsized":"2024-07-04T17:27:07.914645+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:44:59.048384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.918346+0000","last_change":"2024-07-04T17:27:07.915949+0000","last_active":"2024-07-04T17:27:07.918346+0000","last_peered":"2024-07-04T17:27:07.918346+0000","last_clean":"2024-07-04T17:27:07.918346+0000","last_became_active":"2024-07-04T17:27:07.915603+0000","last_became_peered":"2024-07-04T17:27:07.915603+0000","last_unstale":"2024-07-04T17:27:07.918346+0000","last_undegraded":"2024-07-04T17:27:07.918346+0000","last_fullsized":"2024-07-04T17:27:07.918346+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.727000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"26'1","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.906024+0000","last_change":"2024-07-04T17:27:06.877349+0000","last_active":"2024-07-04T17:27:07.906024+0000","last_peered":"2024-07-04T17:27:07.906024+0000","last_clean":"2024-07-04T17:27:07.906024+0000","last_became_active":"2024-07-04T17:27:06.876814+0000","last_became_peered":"2024-07-04T17:27:06.876814+0000","last_unstale":"2024-07-04T17:27:07.906024+0000","last_undegraded":"2024-07-04T17:27:07.906024+0000","last_fullsized":"2024-07-04T17:27:07.906024+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:52:15.359670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.900639+0000","last_change":"2024-07-04T17:27:07.899041+0000","last_active":"2024-07-04T17:27:07.900639+0000","last_peered":"2024-07-04T17:27:07.900639+0000","last_clean":"2024-07-04T17:27:07.900639+0000","last_became_active":"2024-07-04T17:27:07.895875+0000","last_became_peered":"2024-07-04T17:27:07.895875+0000","last_unstale":"2024-07-04T17:27:07.900639+0000","last_undegraded":"2024-07-04T17:27:07.900639+0000","last_fullsized":"2024-07-04T17:27:07.900639+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:55:16.424967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"26'2","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.895709+0000","last_change":"2024-07-04T17:27:06.885939+0000","last_active":"2024-07-04T17:27:07.895709+0000","last_peered":"2024-07-04T17:27:07.895709+0000","last_clean":"2024-07-04T17:27:07.895709+0000","last_became_active":"2024-07-04T17:27:06.885638+0000","last_became_peered":"2024-07-04T17:27:06.885638+0000","last_unstale":"2024-07-04T17:27:07.895709+0000","last_undegraded":"2024-07-04T17:27:07.895709+0000","last_fullsized":"2024-07-04T17:27:07.895709+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:31:45.030996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.913683+0000","last_change":"2024-07-04T17:27:07.904773+0000","last_active":"2024-07-04T17:27:07.913683+0000","last_peered":"2024-07-04T17:27:07.913683+0000","last_clean":"2024-07-04T17:27:07.913683+0000","last_became_active":"2024-07-04T17:27:07.900622+0000","last_became_peered":"2024-07-04T17:27:07.900622+0000","last_unstale":"2024-07-04T17:27:07.913683+0000","last_undegraded":"2024-07-04T17:27:07.913683+0000","last_fullsized":"2024-07-04T17:27:07.913683+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:13:09.667367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"26'1","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.891393+0000","last_change":"2024-07-04T17:27:06.877855+0000","last_active":"2024-07-04T17:27:07.891393+0000","last_peered":"2024-07-04T17:27:07.891393+0000","last_clean":"2024-07-04T17:27:07.891393+0000","last_became_active":"2024-07-04T17:27:06.876852+0000","last_became_peered":"2024-07-04T17:27:06.876852+0000","last_unstale":"2024-07-04T17:27:07.891393+0000","last_undegraded":"2024-07-04T17:27:07.891393+0000","last_fullsized":"2024-07-04T17:27:07.891393+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:19:43.481377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.918953+0000","last_change":"2024-07-04T17:27:07.912224+0000","last_active":"2024-07-04T17:27:07.918953+0000","last_peered":"2024-07-04T17:27:07.918953+0000","last_clean":"2024-07-04T17:27:07.918953+0000","last_became_active":"2024-07-04T17:27:07.911921+0000","last_became_peered":"2024-07-04T17:27:07.911921+0000","last_unstale":"2024-07-04T17:27:07.918953+0000","last_undegraded":"2024-07-04T17:27:07.918953+0000","last_fullsized":"2024-07-04T17:27:07.918953+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:18:54.268323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.862276+0000","last_change":"2024-07-04T17:27:06.859888+0000","last_active":"2024-07-04T17:27:07.862276+0000","last_peered":"2024-07-04T17:27:07.862276+0000","last_clean":"2024-07-04T17:27:07.862276+0000","last_became_active":"2024-07-04T17:27:06.859565+0000","last_became_peered":"2024-07-04T17:27:06.859565+0000","last_unstale":"2024-07-04T17:27:07.862276+0000","last_undegraded":"2024-07-04T17:27:07.862276+0000","last_fullsized":"2024-07-04T17:27:07.862276+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:25:51.634073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.917397+0000","last_change":"2024-07-04T17:27:07.905531+0000","last_active":"2024-07-04T17:27:07.917397+0000","last_peered":"2024-07-04T17:27:07.917397+0000","last_clean":"2024-07-04T17:27:07.917397+0000","last_became_active":"2024-07-04T17:27:07.903880+0000","last_became_peered":"2024-07-04T17:27:07.903880+0000","last_unstale":"2024-07-04T17:27:07.917397+0000","last_undegraded":"2024-07-04T17:27:07.917397+0000","last_fullsized":"2024-07-04T17:27:07.917397+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:34:47.905292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"26'2","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.893123+0000","last_change":"2024-07-04T17:27:06.859869+0000","last_active":"2024-07-04T17:27:07.893123+0000","last_peered":"2024-07-04T17:27:07.893123+0000","last_clean":"2024-07-04T17:27:07.893123+0000","last_became_active":"2024-07-04T17:27:06.859475+0000","last_became_peered":"2024-07-04T17:27:06.859475+0000","last_unstale":"2024-07-04T17:27:07.893123+0000","last_undegraded":"2024-07-04T17:27:07.893123+0000","last_fullsized":"2024-07-04T17:27:07.893123+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:00:10.490785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.915751+0000","last_change":"2024-07-04T17:27:07.904396+0000","last_active":"2024-07-04T17:27:07.915751+0000","last_peered":"2024-07-04T17:27:07.915751+0000","last_clean":"2024-07-04T17:27:07.915751+0000","last_became_active":"2024-07-04T17:27:07.903961+0000","last_became_peered":"2024-07-04T17:27:07.903961+0000","last_unstale":"2024-07-04T17:27:07.915751+0000","last_undegraded":"2024-07-04T17:27:07.915751+0000","last_fullsized":"2024-07-04T17:27:07.915751+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:29:13.459059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.868334+0000","last_change":"2024-07-04T17:27:06.885598+0000","last_active":"2024-07-04T17:27:07.868334+0000","last_peered":"2024-07-04T17:27:07.868334+0000","last_clean":"2024-07-04T17:27:07.868334+0000","last_became_active":"2024-07-04T17:27:06.885220+0000","last_became_peered":"2024-07-04T17:27:06.885220+0000","last_unstale":"2024-07-04T17:27:07.868334+0000","last_undegraded":"2024-07-04T17:27:07.868334+0000","last_fullsized":"2024-07-04T17:27:07.868334+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:49:35.335628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.917575+0000","last_change":"2024-07-04T17:27:07.905869+0000","last_active":"2024-07-04T17:27:07.917575+0000","last_peered":"2024-07-04T17:27:07.917575+0000","last_clean":"2024-07-04T17:27:07.917575+0000","last_became_active":"2024-07-04T17:27:07.905559+0000","last_became_peered":"2024-07-04T17:27:07.905559+0000","last_unstale":"2024-07-04T17:27:07.917575+0000","last_undegraded":"2024-07-04T17:27:07.917575+0000","last_fullsized":"2024-07-04T17:27:07.917575+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:16:25.982847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.845383+0000","last_change":"2024-07-04T17:27:06.868310+0000","last_active":"2024-07-04T17:27:07.845383+0000","last_peered":"2024-07-04T17:27:07.845383+0000","last_clean":"2024-07-04T17:27:07.845383+0000","last_became_active":"2024-07-04T17:27:06.866928+0000","last_became_peered":"2024-07-04T17:27:06.866928+0000","last_unstale":"2024-07-04T17:27:07.845383+0000","last_undegraded":"2024-07-04T17:27:07.845383+0000","last_fullsized":"2024-07-04T17:27:07.845383+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:31:45.030996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.885796+0000","last_change":"2024-07-04T17:27:07.872007+0000","last_active":"2024-07-04T17:27:07.885796+0000","last_peered":"2024-07-04T17:27:07.885796+0000","last_clean":"2024-07-04T17:27:07.885796+0000","last_became_active":"2024-07-04T17:27:07.871647+0000","last_became_peered":"2024-07-04T17:27:07.871647+0000","last_unstale":"2024-07-04T17:27:07.885796+0000","last_undegraded":"2024-07-04T17:27:07.885796+0000","last_fullsized":"2024-07-04T17:27:07.885796+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:34:47.905292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.848155+0000","last_change":"2024-07-04T17:27:06.871847+0000","last_active":"2024-07-04T17:27:07.848155+0000","last_peered":"2024-07-04T17:27:07.848155+0000","last_clean":"2024-07-04T17:27:07.848155+0000","last_became_active":"2024-07-04T17:27:06.871475+0000","last_became_peered":"2024-07-04T17:27:06.871475+0000","last_unstale":"2024-07-04T17:27:07.848155+0000","last_undegraded":"2024-07-04T17:27:07.848155+0000","last_fullsized":"2024-07-04T17:27:07.848155+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:09:28.414254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.918332+0000","last_change":"2024-07-04T17:27:07.911944+0000","last_active":"2024-07-04T17:27:07.918332+0000","last_peered":"2024-07-04T17:27:07.918332+0000","last_clean":"2024-07-04T17:27:07.918332+0000","last_became_active":"2024-07-04T17:27:07.911639+0000","last_became_peered":"2024-07-04T17:27:07.911639+0000","last_unstale":"2024-07-04T17:27:07.918332+0000","last_undegraded":"2024-07-04T17:27:07.918332+0000","last_fullsized":"2024-07-04T17:27:07.918332+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:06:16.576182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"26'2","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.911057+0000","last_change":"2024-07-04T17:27:06.865834+0000","last_active":"2024-07-04T17:27:07.911057+0000","last_peered":"2024-07-04T17:27:07.911057+0000","last_clean":"2024-07-04T17:27:07.911057+0000","last_became_active":"2024-07-04T17:27:06.865463+0000","last_became_peered":"2024-07-04T17:27:06.865463+0000","last_unstale":"2024-07-04T17:27:07.911057+0000","last_undegraded":"2024-07-04T17:27:07.911057+0000","last_fullsized":"2024-07-04T17:27:07.911057+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:35:43.457323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.909298+0000","last_change":"2024-07-04T17:27:07.882402+0000","last_active":"2024-07-04T17:27:07.909298+0000","last_peered":"2024-07-04T17:27:07.909298+0000","last_clean":"2024-07-04T17:27:07.909298+0000","last_became_active":"2024-07-04T17:27:07.882061+0000","last_became_peered":"2024-07-04T17:27:07.882061+0000","last_unstale":"2024-07-04T17:27:07.909298+0000","last_undegraded":"2024-07-04T17:27:07.909298+0000","last_fullsized":"2024-07-04T17:27:07.909298+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:12:04.631448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.862863+0000","last_change":"2024-07-04T17:27:06.866198+0000","last_active":"2024-07-04T17:27:07.862863+0000","last_peered":"2024-07-04T17:27:07.862863+0000","last_clean":"2024-07-04T17:27:07.862863+0000","last_became_active":"2024-07-04T17:27:06.865891+0000","last_became_peered":"2024-07-04T17:27:06.865891+0000","last_unstale":"2024-07-04T17:27:07.862863+0000","last_undegraded":"2024-07-04T17:27:07.862863+0000","last_fullsized":"2024-07-04T17:27:07.862863+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:59:47.703820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.907207+0000","last_change":"2024-07-04T17:27:07.883459+0000","last_active":"2024-07-04T17:27:07.907207+0000","last_peered":"2024-07-04T17:27:07.907207+0000","last_clean":"2024-07-04T17:27:07.907207+0000","last_became_active":"2024-07-04T17:27:07.883202+0000","last_became_peered":"2024-07-04T17:27:07.883202+0000","last_unstale":"2024-07-04T17:27:07.907207+0000","last_undegraded":"2024-07-04T17:27:07.907207+0000","last_fullsized":"2024-07-04T17:27:07.907207+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:23.798405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.844844+0000","last_change":"2024-07-04T17:27:06.872489+0000","last_active":"2024-07-04T17:27:07.844844+0000","last_peered":"2024-07-04T17:27:07.844844+0000","last_clean":"2024-07-04T17:27:07.844844+0000","last_became_active":"2024-07-04T17:27:06.872125+0000","last_became_peered":"2024-07-04T17:27:06.872125+0000","last_unstale":"2024-07-04T17:27:07.844844+0000","last_undegraded":"2024-07-04T17:27:07.844844+0000","last_fullsized":"2024-07-04T17:27:07.844844+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:10:29.469978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.897536+0000","last_change":"2024-07-04T17:27:07.878014+0000","last_active":"2024-07-04T17:27:07.897536+0000","last_peered":"2024-07-04T17:27:07.897536+0000","last_clean":"2024-07-04T17:27:07.897536+0000","last_became_active":"2024-07-04T17:27:07.874485+0000","last_became_peered":"2024-07-04T17:27:07.874485+0000","last_unstale":"2024-07-04T17:27:07.897536+0000","last_undegraded":"2024-07-04T17:27:07.897536+0000","last_fullsized":"2024-07-04T17:27:07.897536+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:43:39.860474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.861881+0000","last_change":"2024-07-04T17:27:06.870890+0000","last_active":"2024-07-04T17:27:07.861881+0000","last_peered":"2024-07-04T17:27:07.861881+0000","last_clean":"2024-07-04T17:27:07.861881+0000","last_became_active":"2024-07-04T17:27:06.870581+0000","last_became_peered":"2024-07-04T17:27:06.870581+0000","last_unstale":"2024-07-04T17:27:07.861881+0000","last_undegraded":"2024-07-04T17:27:07.861881+0000","last_fullsized":"2024-07-04T17:27:07.861881+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:55:15.415040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.906785+0000","last_change":"2024-07-04T17:27:07.882615+0000","last_active":"2024-07-04T17:27:07.906785+0000","last_peered":"2024-07-04T17:27:07.906785+0000","last_clean":"2024-07-04T17:27:07.906785+0000","last_became_active":"2024-07-04T17:27:07.882132+0000","last_became_peered":"2024-07-04T17:27:07.882132+0000","last_unstale":"2024-07-04T17:27:07.906785+0000","last_undegraded":"2024-07-04T17:27:07.906785+0000","last_fullsized":"2024-07-04T17:27:07.906785+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:21:38.559811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.849512+0000","last_change":"2024-07-04T17:27:06.861673+0000","last_active":"2024-07-04T17:27:07.849512+0000","last_peered":"2024-07-04T17:27:07.849512+0000","last_clean":"2024-07-04T17:27:07.849512+0000","last_became_active":"2024-07-04T17:27:06.861297+0000","last_became_peered":"2024-07-04T17:27:06.861297+0000","last_unstale":"2024-07-04T17:27:07.849512+0000","last_undegraded":"2024-07-04T17:27:07.849512+0000","last_fullsized":"2024-07-04T17:27:07.849512+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:10:34.165745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.917621+0000","last_change":"2024-07-04T17:27:07.909121+0000","last_active":"2024-07-04T17:27:07.917621+0000","last_peered":"2024-07-04T17:27:07.917621+0000","last_clean":"2024-07-04T17:27:07.917621+0000","last_became_active":"2024-07-04T17:27:07.908646+0000","last_became_peered":"2024-07-04T17:27:07.908646+0000","last_unstale":"2024-07-04T17:27:07.917621+0000","last_undegraded":"2024-07-04T17:27:07.917621+0000","last_fullsized":"2024-07-04T17:27:07.917621+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:52:51.670741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.867042+0000","last_change":"2024-07-04T17:27:06.883045+0000","last_active":"2024-07-04T17:27:07.867042+0000","last_peered":"2024-07-04T17:27:07.867042+0000","last_clean":"2024-07-04T17:27:07.867042+0000","last_became_active":"2024-07-04T17:27:06.882363+0000","last_became_peered":"2024-07-04T17:27:06.882363+0000","last_unstale":"2024-07-04T17:27:07.867042+0000","last_undegraded":"2024-07-04T17:27:07.867042+0000","last_fullsized":"2024-07-04T17:27:07.867042+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:05:41.513762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.885769+0000","last_change":"2024-07-04T17:27:07.871918+0000","last_active":"2024-07-04T17:27:07.885769+0000","last_peered":"2024-07-04T17:27:07.885769+0000","last_clean":"2024-07-04T17:27:07.885769+0000","last_became_active":"2024-07-04T17:27:07.871229+0000","last_became_peered":"2024-07-04T17:27:07.871229+0000","last_unstale":"2024-07-04T17:27:07.885769+0000","last_undegraded":"2024-07-04T17:27:07.885769+0000","last_fullsized":"2024-07-04T17:27:07.885769+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:01:29.057248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.858698+0000","last_change":"2024-07-04T17:27:06.859760+0000","last_active":"2024-07-04T17:27:07.858698+0000","last_peered":"2024-07-04T17:27:07.858698+0000","last_clean":"2024-07-04T17:27:07.858698+0000","last_became_active":"2024-07-04T17:27:06.859396+0000","last_became_peered":"2024-07-04T17:27:06.859396+0000","last_unstale":"2024-07-04T17:27:07.858698+0000","last_undegraded":"2024-07-04T17:27:07.858698+0000","last_fullsized":"2024-07-04T17:27:07.858698+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:48:12.553882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.908258+0000","last_change":"2024-07-04T17:27:07.891018+0000","last_active":"2024-07-04T17:27:07.908258+0000","last_peered":"2024-07-04T17:27:07.908258+0000","last_clean":"2024-07-04T17:27:07.908258+0000","last_became_active":"2024-07-04T17:27:07.889885+0000","last_became_peered":"2024-07-04T17:27:07.889885+0000","last_unstale":"2024-07-04T17:27:07.908258+0000","last_undegraded":"2024-07-04T17:27:07.908258+0000","last_fullsized":"2024-07-04T17:27:07.908258+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:26:23.039967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.848028+0000","last_change":"2024-07-04T17:27:06.874596+0000","last_active":"2024-07-04T17:27:07.848028+0000","last_peered":"2024-07-04T17:27:07.848028+0000","last_clean":"2024-07-04T17:27:07.848028+0000","last_became_active":"2024-07-04T17:27:06.874252+0000","last_became_peered":"2024-07-04T17:27:06.874252+0000","last_unstale":"2024-07-04T17:27:07.848028+0000","last_undegraded":"2024-07-04T17:27:07.848028+0000","last_fullsized":"2024-07-04T17:27:07.848028+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:17:38.211735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'52","reported_seq":107,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-07-04T17:27:07.841949+0000","last_change":"2024-07-04T17:26:04.953559+0000","last_active":"2024-07-04T17:27:07.841949+0000","last_peered":"2024-07-04T17:27:07.841949+0000","last_clean":"2024-07-04T17:27:07.841949+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:27:07.841949+0000","last_undegraded":"2024-07-04T17:27:07.841949+0000","last_fullsized":"2024-07-04T17:27:07.841949+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_clean_scrub_stamp":"2024-07-04T17:26:03.933432+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.064181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738384,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":241664,"data_stored":77560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738385,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":856064,"data_stored":670212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738383,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":815104,"data_stored":663337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:27:13.119 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-04T17:27:13.119 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-07-04T17:27:13.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.457+0000 7fd8d2200640 1 -- 172.21.15.42:0/3014298894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8cc0bc090 msgr2=0x7fd8cc0c37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:13.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.457+0000 7fd8d2200640 1 --2- 172.21.15.42:0/3014298894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8cc0bc090 0x7fd8cc0c37d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd8b80097b0 tx=0x7fd8b802a540 comp rx=0 tx=0).stop 2024-07-04T17:27:13.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.458+0000 7fd8d2200640 1 -- 172.21.15.42:0/3014298894 shutdown_connections 2024-07-04T17:27:13.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.458+0000 7fd8d2200640 1 --2- 172.21.15.42:0/3014298894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8cc0bc090 0x7fd8cc0c37d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:13.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.458+0000 7fd8d2200640 1 -- 172.21.15.42:0/3014298894 >> 172.21.15.42:0/3014298894 conn(0x7fd8cc06e160 msgr2=0x7fd8cc06e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:13.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.458+0000 7fd8d2200640 1 -- 172.21.15.42:0/3014298894 shutdown_connections 2024-07-04T17:27:13.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.458+0000 7fd8d2200640 1 -- 172.21.15.42:0/3014298894 wait complete. 2024-07-04T17:27:13.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.460+0000 7fd8d2200640 1 Processor -- start 2024-07-04T17:27:13.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.460+0000 7fd8d2200640 1 -- start start 2024-07-04T17:27:13.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.461+0000 7fd8d2200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8cc0b3ba0 0x7fd8cc0b5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:13.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.461+0000 7fd8d2200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc0b7360 con 0x7fd8cc0b3ba0 2024-07-04T17:27:13.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.462+0000 7fd8d0e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8cc0b3ba0 0x7fd8cc0b5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:13.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.462+0000 7fd8d0e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8cc0b3ba0 0x7fd8cc0b5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38496/0 (socket says 172.21.15.42:38496) 2024-07-04T17:27:13.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.462+0000 7fd8d0e00640 1 -- 172.21.15.42:0/2084306880 learned_addr learned my addr 172.21.15.42:0/2084306880 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:13.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.462+0000 7fd8d0e00640 1 -- 172.21.15.42:0/2084306880 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8cc0c2dc0 con 0x7fd8cc0b3ba0 2024-07-04T17:27:13.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.462+0000 7fd8d0e00640 1 --2- 172.21.15.42:0/2084306880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8cc0b3ba0 0x7fd8cc0b5f80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00cb40 tx=0x7fd8bc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:13.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.463+0000 7fd8c8c00640 1 -- 172.21.15.42:0/2084306880 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8bc016020 con 0x7fd8cc0b3ba0 2024-07-04T17:27:13.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.463+0000 7fd8c8c00640 1 -- 172.21.15.42:0/2084306880 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8bc00ac30 con 0x7fd8cc0b3ba0 2024-07-04T17:27:13.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.463+0000 7fd8d2200640 1 -- 172.21.15.42:0/2084306880 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8cc0c12f0 con 0x7fd8cc0b3ba0 2024-07-04T17:27:13.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.463+0000 7fd8c8c00640 1 -- 172.21.15.42:0/2084306880 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8bc012420 con 0x7fd8cc0b3ba0 2024-07-04T17:27:13.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.463+0000 7fd8d2200640 1 -- 172.21.15.42:0/2084306880 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8cc0c2fe0 con 0x7fd8cc0b3ba0 2024-07-04T17:27:13.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.464+0000 7fd8d2200640 1 -- 172.21.15.42:0/2084306880 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8cc0b7a10 con 0x7fd8cc0b3ba0 2024-07-04T17:27:13.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.464+0000 7fd8c8c00640 1 -- 172.21.15.42:0/2084306880 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fd8bc019020 con 0x7fd8cc0b3ba0 2024-07-04T17:27:13.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.465+0000 7fd8c8c00640 1 --2- 172.21.15.42:0/2084306880 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fd8ac03d2b0 0x7fd8ac03f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:13.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.465+0000 7fd8c8c00640 1 -- 172.21.15.42:0/2084306880 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3439+0+0 (secure 0 0 0) 0x7fd8bc051820 con 0x7fd8cc0b3ba0 2024-07-04T17:27:13.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.465+0000 7fd8cbe00640 1 --2- 172.21.15.42:0/2084306880 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fd8ac03d2b0 0x7fd8ac03f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:13.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.466+0000 7fd8cbe00640 1 --2- 172.21.15.42:0/2084306880 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fd8ac03d2b0 0x7fd8ac03f7c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8034000 tx=0x7fd8b8004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:13.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.470+0000 7fd8c8c00640 1 -- 172.21.15.42:0/2084306880 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fd8bc020030 con 0x7fd8cc0b3ba0 2024-07-04T17:27:13.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.755+0000 7fd8d2200640 1 -- 172.21.15.42:0/2084306880 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fd8cc068d10 con 0x7fd8cc0b3ba0 2024-07-04T17:27:13.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.759+0000 7fd8c8c00640 1 -- 172.21.15.42:0/2084306880 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7fd8bc011380 con 0x7fd8cc0b3ba0 2024-07-04T17:27:13.761 INFO:teuthology.orchestra.run.smithi042.stdout:[client.0] 2024-07-04T17:27:13.761 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAO24ZmUMmYABAAh4pNlkb9JsHQtK8uCJFQCA== 2024-07-04T17:27:13.761 INFO:teuthology.orchestra.run.smithi042.stdout: caps mds = "allow" 2024-07-04T17:27:13.761 INFO:teuthology.orchestra.run.smithi042.stdout: caps mon = "allow r" 2024-07-04T17:27:13.761 INFO:teuthology.orchestra.run.smithi042.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-07-04T17:27:13.761 INFO:teuthology.orchestra.run.smithi042.stderr:updated caps for client.0 2024-07-04T17:27:13.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.761+0000 7fd8d2200640 1 -- 172.21.15.42:0/2084306880 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fd8ac03d2b0 msgr2=0x7fd8ac03f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:13.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.761+0000 7fd8d2200640 1 --2- 172.21.15.42:0/2084306880 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fd8ac03d2b0 0x7fd8ac03f7c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8034000 tx=0x7fd8b8004730 comp rx=0 tx=0).stop 2024-07-04T17:27:13.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.761+0000 7fd8d2200640 1 -- 172.21.15.42:0/2084306880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8cc0b3ba0 msgr2=0x7fd8cc0b5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:13.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.761+0000 7fd8d2200640 1 --2- 172.21.15.42:0/2084306880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8cc0b3ba0 0x7fd8cc0b5f80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00cb40 tx=0x7fd8bc00a060 comp rx=0 tx=0).stop 2024-07-04T17:27:13.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.762+0000 7fd8d2200640 1 -- 172.21.15.42:0/2084306880 shutdown_connections 2024-07-04T17:27:13.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.762+0000 7fd8d2200640 1 --2- 172.21.15.42:0/2084306880 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fd8ac03d2b0 0x7fd8ac03f7c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:13.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.762+0000 7fd8d2200640 1 --2- 172.21.15.42:0/2084306880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8cc0b3ba0 0x7fd8cc0b5f80 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:13.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.762+0000 7fd8d2200640 1 -- 172.21.15.42:0/2084306880 >> 172.21.15.42:0/2084306880 conn(0x7fd8cc06e160 msgr2=0x7fd8cc075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:13.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.762+0000 7fd8d2200640 1 -- 172.21.15.42:0/2084306880 shutdown_connections 2024-07-04T17:27:13.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:13.762+0000 7fd8d2200640 1 -- 172.21.15.42:0/2084306880 wait complete. 2024-07-04T17:27:13.787 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:27:14.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.071+0000 7f0307200640 1 -- 172.21.15.42:0/1199716993 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03080b17d0 msgr2=0x7f03080b1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:14.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.071+0000 7f0307200640 1 --2- 172.21.15.42:0/1199716993 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03080b17d0 0x7f03080b1be0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f02f00097b0 tx=0x7f02f002a4a0 comp rx=0 tx=0).stop 2024-07-04T17:27:14.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.071+0000 7f0307200640 1 -- 172.21.15.42:0/1199716993 shutdown_connections 2024-07-04T17:27:14.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.071+0000 7f0307200640 1 --2- 172.21.15.42:0/1199716993 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03080b17d0 0x7f03080b1be0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:14.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.071+0000 7f0307200640 1 -- 172.21.15.42:0/1199716993 >> 172.21.15.42:0/1199716993 conn(0x7f03080bebc0 msgr2=0x7f03080befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:14.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.071+0000 7f0307200640 1 -- 172.21.15.42:0/1199716993 shutdown_connections 2024-07-04T17:27:14.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.071+0000 7f0307200640 1 -- 172.21.15.42:0/1199716993 wait complete. 2024-07-04T17:27:14.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.073+0000 7f0307200640 1 Processor -- start 2024-07-04T17:27:14.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.073+0000 7f0307200640 1 -- start start 2024-07-04T17:27:14.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.074+0000 7f0307200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f030806c200 0x7f03081d6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:14.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.074+0000 7f0307200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f030814ae80 con 0x7f030806c200 2024-07-04T17:27:14.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.075+0000 7f0305e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f030806c200 0x7f03081d6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:14.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.075+0000 7f0305e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f030806c200 0x7f03081d6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57950/0 (socket says 172.21.15.42:57950) 2024-07-04T17:27:14.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.075+0000 7f0305e00640 1 -- 172.21.15.42:0/2707106758 learned_addr learned my addr 172.21.15.42:0/2707106758 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:14.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.075+0000 7f0305e00640 1 -- 172.21.15.42:0/2707106758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03081d7140 con 0x7f030806c200 2024-07-04T17:27:14.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.075+0000 7f0305e00640 1 --2- 172.21.15.42:0/2707106758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f030806c200 0x7f03081d6b50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f02f800c3b0 tx=0x7f02f800c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:14.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.076+0000 7f02fe000640 1 -- 172.21.15.42:0/2707106758 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02f8011780 con 0x7f030806c200 2024-07-04T17:27:14.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.076+0000 7f02fe000640 1 -- 172.21.15.42:0/2707106758 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02f8017070 con 0x7f030806c200 2024-07-04T17:27:14.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.076+0000 7f0307200640 1 -- 172.21.15.42:0/2707106758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03081d7620 con 0x7f030806c200 2024-07-04T17:27:14.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.076+0000 7f02fe000640 1 -- 172.21.15.42:0/2707106758 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02f8011c00 con 0x7f030806c200 2024-07-04T17:27:14.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.076+0000 7f0307200640 1 -- 172.21.15.42:0/2707106758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0308148690 con 0x7f030806c200 2024-07-04T17:27:14.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.077+0000 7f0307200640 1 -- 172.21.15.42:0/2707106758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f030814b4c0 con 0x7f030806c200 2024-07-04T17:27:14.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.077+0000 7f02fe000640 1 -- 172.21.15.42:0/2707106758 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f02f801c460 con 0x7f030806c200 2024-07-04T17:27:14.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.078+0000 7f02fe000640 1 --2- 172.21.15.42:0/2707106758 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f02e403d2b0 0x7f02e403f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:14.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.078+0000 7f02fe000640 1 -- 172.21.15.42:0/2707106758 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3439+0+0 (secure 0 0 0) 0x7f02f8051270 con 0x7f030806c200 2024-07-04T17:27:14.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.078+0000 7f02ffe00640 1 --2- 172.21.15.42:0/2707106758 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f02e403d2b0 0x7f02e403f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:14.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.079+0000 7f02ffe00640 1 --2- 172.21.15.42:0/2707106758 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f02e403d2b0 0x7f02e403f7c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f02f0034000 tx=0x7f02f0004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:14.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.083+0000 7f02fe000640 1 -- 172.21.15.42:0/2707106758 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f02f8011da0 con 0x7f030806c200 2024-07-04T17:27:14.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.337+0000 7f0307200640 1 -- 172.21.15.42:0/2707106758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0308068e30 con 0x7f030806c200 2024-07-04T17:27:14.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.338+0000 7f02fe000640 1 -- 172.21.15.42:0/2707106758 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6042 (secure 0 0 0) 0x7f02f801c800 con 0x7f030806c200 2024-07-04T17:27:14.340 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:27:14.340 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":17,"btime":"2024-07-04T17:27:10:861234+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/399707138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":399707138},{"type":"v1","addr":"172.21.15.42:6827","nonce":399707138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4414,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1510714231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1510714231},{"type":"v1","addr":"172.21.15.42:6829","nonce":1510714231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/3069680926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":3069680926},{"type":"v1","addr":"172.21.15.42:6831","nonce":3069680926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4422,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6833/1348446958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1348446958},{"type":"v1","addr":"172.21.15.42:6833","nonce":1348446958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:27:07.402007+0000","modified":"2024-07-04T17:27:10.462342+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4423":{"gid":4423,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.42:6835/807364821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":807364821},{"type":"v1","addr":"172.21.15.42:6835","nonce":807364821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4423,"qdb_cluster":[4423]},"id":2}]} 2024-07-04T17:27:14.340 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 17 2024-07-04T17:27:14.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.341+0000 7f0307200640 1 -- 172.21.15.42:0/2707106758 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f02e403d2b0 msgr2=0x7f02e403f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:14.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.341+0000 7f0307200640 1 --2- 172.21.15.42:0/2707106758 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f02e403d2b0 0x7f02e403f7c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f02f0034000 tx=0x7f02f0004730 comp rx=0 tx=0).stop 2024-07-04T17:27:14.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.341+0000 7f0307200640 1 -- 172.21.15.42:0/2707106758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f030806c200 msgr2=0x7f03081d6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:14.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.341+0000 7f0307200640 1 --2- 172.21.15.42:0/2707106758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f030806c200 0x7f03081d6b50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f02f800c3b0 tx=0x7f02f800c870 comp rx=0 tx=0).stop 2024-07-04T17:27:14.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.341+0000 7f0307200640 1 -- 172.21.15.42:0/2707106758 shutdown_connections 2024-07-04T17:27:14.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.341+0000 7f0307200640 1 --2- 172.21.15.42:0/2707106758 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f02e403d2b0 0x7f02e403f7c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:14.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.341+0000 7f0307200640 1 --2- 172.21.15.42:0/2707106758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f030806c200 0x7f03081d6b50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:14.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.341+0000 7f0307200640 1 -- 172.21.15.42:0/2707106758 >> 172.21.15.42:0/2707106758 conn(0x7f03080bebc0 msgr2=0x7f03080b95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:14.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.341+0000 7f0307200640 1 -- 172.21.15.42:0/2707106758 shutdown_connections 2024-07-04T17:27:14.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:14.341+0000 7f0307200640 1 -- 172.21.15.42:0/2707106758 wait complete. 2024-07-04T17:27:14.873 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-04T17:27:07.402007+0000', 'modified': '2024-07-04T17:27:10.462342+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4423}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4423': {'gid': 4423, 'name': 'b', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.42:6835/807364821', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6834', 'nonce': 807364821}, {'type': 'v1', 'addr': '172.21.15.42:6835', 'nonce': 807364821}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4423, 'qdb_cluster': [4423]} 2024-07-04T17:27:14.873 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-04T17:27:14.874 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-07-04T17:27:15.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.172+0000 7fa7a7200640 1 -- 172.21.15.42:0/2389836507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7a006c020 msgr2=0x7fa7a014a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:15.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.172+0000 7fa7a7200640 1 --2- 172.21.15.42:0/2389836507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7a006c020 0x7fa7a014a9e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa7900097b0 tx=0x7fa79002a540 comp rx=0 tx=0).stop 2024-07-04T17:27:15.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.172+0000 7fa7a7200640 1 -- 172.21.15.42:0/2389836507 shutdown_connections 2024-07-04T17:27:15.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.172+0000 7fa7a7200640 1 --2- 172.21.15.42:0/2389836507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7a006c020 0x7fa7a014a9e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:15.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.172+0000 7fa7a7200640 1 -- 172.21.15.42:0/2389836507 >> 172.21.15.42:0/2389836507 conn(0x7fa7a00bebc0 msgr2=0x7fa7a00befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:15.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.172+0000 7fa7a7200640 1 -- 172.21.15.42:0/2389836507 shutdown_connections 2024-07-04T17:27:15.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.172+0000 7fa7a7200640 1 -- 172.21.15.42:0/2389836507 wait complete. 2024-07-04T17:27:15.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.174+0000 7fa7a7200640 1 Processor -- start 2024-07-04T17:27:15.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.174+0000 7fa7a7200640 1 -- start start 2024-07-04T17:27:15.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.175+0000 7fa7a7200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7a006c020 0x7fa7a0148ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:15.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.175+0000 7fa7a7200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a014b550 con 0x7fa7a006c020 2024-07-04T17:27:15.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.176+0000 7fa7a5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7a006c020 0x7fa7a0148ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:15.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.176+0000 7fa7a5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7a006c020 0x7fa7a0148ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57978/0 (socket says 172.21.15.42:57978) 2024-07-04T17:27:15.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.176+0000 7fa7a5e00640 1 -- 172.21.15.42:0/1284993688 learned_addr learned my addr 172.21.15.42:0/1284993688 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:15.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.176+0000 7fa7a5e00640 1 -- 172.21.15.42:0/1284993688 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a01d4e50 con 0x7fa7a006c020 2024-07-04T17:27:15.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.176+0000 7fa7a5e00640 1 --2- 172.21.15.42:0/1284993688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7a006c020 0x7fa7a0148ff0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa794001eb0 tx=0x7fa79400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:15.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.177+0000 7fa79a000640 1 -- 172.21.15.42:0/1284993688 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7940116c0 con 0x7fa7a006c020 2024-07-04T17:27:15.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.177+0000 7fa79a000640 1 -- 172.21.15.42:0/1284993688 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa794017070 con 0x7fa7a006c020 2024-07-04T17:27:15.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.177+0000 7fa7a7200640 1 -- 172.21.15.42:0/1284993688 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7a01d60b0 con 0x7fa7a006c020 2024-07-04T17:27:15.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.177+0000 7fa79a000640 1 -- 172.21.15.42:0/1284993688 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa794011ae0 con 0x7fa7a006c020 2024-07-04T17:27:15.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.177+0000 7fa7a7200640 1 -- 172.21.15.42:0/1284993688 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7a00b1150 con 0x7fa7a006c020 2024-07-04T17:27:15.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.179+0000 7fa79a000640 1 -- 172.21.15.42:0/1284993688 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fa794011c80 con 0x7fa7a006c020 2024-07-04T17:27:15.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.180+0000 7fa79a000640 1 --2- 172.21.15.42:0/1284993688 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa78403d2b0 0x7fa78403f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:15.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.180+0000 7fa79a000640 1 -- 172.21.15.42:0/1284993688 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3439+0+0 (secure 0 0 0) 0x7fa794051330 con 0x7fa7a006c020 2024-07-04T17:27:15.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.180+0000 7fa7a5400640 1 --2- 172.21.15.42:0/1284993688 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa78403d2b0 0x7fa78403f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:15.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.181+0000 7fa7a5400640 1 --2- 172.21.15.42:0/1284993688 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa78403d2b0 0x7fa78403f7c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa790034000 tx=0x7fa790004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:15.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.182+0000 7fa7af7f6740 1 Processor -- start 2024-07-04T17:27:15.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.182+0000 7fa7af7f6740 1 -- start start 2024-07-04T17:27:15.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.183+0000 7fa7af7f6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f19a5e270 0x555f19e04d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:15.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.183+0000 7fa7af7f6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x555f19d755b0 con 0x555f19a5e270 2024-07-04T17:27:15.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.183+0000 7fa7a5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f19a5e270 0x555f19e04d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:15.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.183+0000 7fa7a5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f19a5e270 0x555f19e04d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57988/0 (socket says 172.21.15.42:57988) 2024-07-04T17:27:15.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.184+0000 7fa7a5e00640 1 -- 172.21.15.42:0/3625449536 learned_addr learned my addr 172.21.15.42:0/3625449536 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:15.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.184+0000 7fa7a5e00640 1 -- 172.21.15.42:0/3625449536 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f19ac9850 con 0x555f19a5e270 2024-07-04T17:27:15.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.184+0000 7fa7a5e00640 1 --2- 172.21.15.42:0/3625449536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f19a5e270 0x555f19e04d30 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa794010420 tx=0x7fa794019fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a59e91a404764d9 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:15.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.184+0000 7fa78e600640 1 -- 172.21.15.42:0/3625449536 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa79404d070 con 0x555f19a5e270 2024-07-04T17:27:15.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.184+0000 7fa78e600640 1 -- 172.21.15.42:0/3625449536 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7940106a0 con 0x555f19a5e270 2024-07-04T17:27:15.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.184+0000 7fa78e600640 1 -- 172.21.15.42:0/3625449536 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa794048f50 con 0x555f19a5e270 2024-07-04T17:27:15.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.185+0000 7fa7af7f6740 1 -- 172.21.15.42:0/3625449536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f19a5e270 msgr2=0x555f19e04d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:15.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.185+0000 7fa7af7f6740 1 --2- 172.21.15.42:0/3625449536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f19a5e270 0x555f19e04d30 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa794010420 tx=0x7fa794019fb0 comp rx=0 tx=0).stop 2024-07-04T17:27:15.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.185+0000 7fa7af7f6740 1 -- 172.21.15.42:0/3625449536 shutdown_connections 2024-07-04T17:27:15.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.185+0000 7fa7af7f6740 1 --2- 172.21.15.42:0/3625449536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f19a5e270 0x555f19e04d30 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:15.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.185+0000 7fa7af7f6740 1 -- 172.21.15.42:0/3625449536 >> 172.21.15.42:0/3625449536 conn(0x555f19a523c0 msgr2=0x555f19e034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:15.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.185+0000 7fa7af7f6740 1 -- 172.21.15.42:0/3625449536 shutdown_connections 2024-07-04T17:27:15.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.185+0000 7fa7af7f6740 1 -- 172.21.15.42:0/3625449536 wait complete. 2024-07-04T17:27:15.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.186+0000 7fa7af7f6740 1 -- start start 2024-07-04T17:27:15.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.187+0000 7fa7af7f6740 1 Processor -- start 2024-07-04T17:27:15.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.188+0000 7fa7af7f6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f19a5e270 0x555f19e478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:15.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.188+0000 7fa7af7f6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x555f19d68b90 con 0x555f19a5e270 2024-07-04T17:27:15.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.188+0000 7fa7af7f6740 10 client.0 _ll_register_callbacks cb 0x555f19d0d8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T17:27:15.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.188+0000 7fa7a5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f19a5e270 0x555f19e478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:15.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.188+0000 7fa7a5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f19a5e270 0x555f19e478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57994/0 (socket says 172.21.15.42:57994) 2024-07-04T17:27:15.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.188+0000 7fa7a5e00640 1 -- 172.21.15.42:0/1564855074 learned_addr learned my addr 172.21.15.42:0/1564855074 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:15.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.189+0000 7fa7a5e00640 1 -- 172.21.15.42:0/1564855074 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f19b406b0 con 0x555f19a5e270 2024-07-04T17:27:15.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.189+0000 7fa7a5e00640 1 --2- 172.21.15.42:0/1564855074 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f19a5e270 0x555f19e478c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa794000c00 tx=0x7fa794010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b4c039e6a0a763 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:15.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.189+0000 7fa782a00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-07-04T17:27:15.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.189+0000 7fa782a00640 1 -- 172.21.15.42:0/1564855074 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa794029070 con 0x555f19a5e270 2024-07-04T17:27:15.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.189+0000 7fa782a00640 1 -- 172.21.15.42:0/1564855074 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 3439+0+0 (secure 0 0 0) 0x7fa794012420 con 0x555f19a5e270 2024-07-04T17:27:15.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.189+0000 7fa7af7f6740 1 -- 172.21.15.42:0/1564855074 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555f19d056c0 con 0x555f19a5e270 2024-07-04T17:27:15.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.189+0000 7fa782a00640 1 -- 172.21.15.42:0/1564855074 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fa794033050 con 0x555f19a5e270 2024-07-04T17:27:15.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.189+0000 7fa7af7f6740 10 client.4494 fetch_fsmap learned FSMap version 17 2024-07-04T17:27:15.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.189+0000 7fa7af7f6740 1 -- 172.21.15.42:0/1564855074 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x555f19e20900 con 0x555f19a5e270 2024-07-04T17:27:15.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.190+0000 7fa782a00640 1 -- 172.21.15.42:0/1564855074 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x555f19d056c0 con 0x555f19a5e270 2024-07-04T17:27:15.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.190+0000 7fa782a00640 1 -- 172.21.15.42:0/1564855074 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 17) ==== 4107+0+0 (secure 0 0 0) 0x7fa794015b10 con 0x555f19a5e270 2024-07-04T17:27:15.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.190+0000 7fa7af7f6740 10 client.4494 fetch_fsmap finished waiting for FSMap version 17 2024-07-04T17:27:15.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.190+0000 7fa7af7f6740 10 client.4494 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-04T17:27:15.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.190+0000 7fa7af7f6740 20 client.4494 populate_metadata read hostname 'smithi042' 2024-07-04T17:27:15.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.190+0000 7fa7af7f6740 1 --2- 172.21.15.42:0/1564855074 >> [v2:172.21.15.42:6834/807364821,v1:172.21.15.42:6835/807364821] conn(0x555f19a353b0 0x555f19e2c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:15.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.190+0000 7fa7af7f6740 4 client.4494 mds_command: new command op to 4423 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T17:27:15.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.190+0000 7fa7af7f6740 1 -- 172.21.15.42:0/1564855074 --> [v2:172.21.15.42:6834/807364821,v1:172.21.15.42:6835/807364821] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f19d77500 con 0x555f19a353b0 2024-07-04T17:27:15.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.190+0000 7fa7a5400640 1 --2- 172.21.15.42:0/1564855074 >> [v2:172.21.15.42:6834/807364821,v1:172.21.15.42:6835/807364821] conn(0x555f19a353b0 0x555f19e2c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:15.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.191+0000 7fa7a5400640 1 --2- 172.21.15.42:0/1564855074 >> [v2:172.21.15.42:6834/807364821,v1:172.21.15.42:6835/807364821] conn(0x555f19a353b0 0x555f19e2c9f0 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e3bf21c0a3a0969 server_cookie=f7702de5edf0155 in_seq=0 out_seq=0 2024-07-04T17:27:15.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.191+0000 7fa782a00640 10 client.4494 ms_handle_connect on v2:172.21.15.42:6834/807364821 2024-07-04T17:27:15.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.192+0000 7fa782a00640 1 -- 172.21.15.42:0/1564855074 <== mds.0 v2:172.21.15.42:6834/807364821 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555f19d77500 con 0x555f19a353b0 2024-07-04T17:27:15.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.192+0000 7fa782a00640 10 client.4494 handle_command_reply: tid=0 2024-07-04T17:27:15.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.192+0000 7fa7af7f6740 1 client.4494 shutdown 2024-07-04T17:27:15.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.192+0000 7fa7af7f6740 1 -- 172.21.15.42:0/1564855074 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f19a5e270 msgr2=0x555f19e478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:15.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.192+0000 7fa7af7f6740 1 --2- 172.21.15.42:0/1564855074 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f19a5e270 0x555f19e478c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa794000c00 tx=0x7fa794010f90 comp rx=0 tx=0).stop 2024-07-04T17:27:15.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.192+0000 7fa7af7f6740 1 -- 172.21.15.42:0/1564855074 shutdown_connections 2024-07-04T17:27:15.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.192+0000 7fa7af7f6740 1 --2- 172.21.15.42:0/1564855074 >> [v2:172.21.15.42:6834/807364821,v1:172.21.15.42:6835/807364821] conn(0x555f19a353b0 0x555f19e2c9f0 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:15.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.192+0000 7fa7af7f6740 1 --2- 172.21.15.42:0/1564855074 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f19a5e270 0x555f19e478c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:15.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.192+0000 7fa7af7f6740 1 -- 172.21.15.42:0/1564855074 >> 172.21.15.42:0/1564855074 conn(0x555f19a523c0 msgr2=0x555f19e30f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:15.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.192+0000 7fa782a00640 0 client.4494 ms_handle_reset on v2:172.21.15.42:6834/807364821 2024-07-04T17:27:15.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.193+0000 7fa7af7f6740 1 -- 172.21.15.42:0/1564855074 shutdown_connections 2024-07-04T17:27:15.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.193+0000 7fa7af7f6740 1 -- 172.21.15.42:0/1564855074 wait complete. 2024-07-04T17:27:15.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.193+0000 7fa7af7f6740 20 client.4494 trim_cache size 0 max 16384 2024-07-04T17:27:15.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.214+0000 7fa7af7f6740 1 Processor -- start 2024-07-04T17:27:15.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.214+0000 7fa7af7f6740 1 -- start start 2024-07-04T17:27:15.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.216+0000 7fa7af7f6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f19a5e270 0x555f19dfe270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:15.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.216+0000 7fa7af7f6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a01d5da0 con 0x555f19a5e270 2024-07-04T17:27:15.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.216+0000 7fa7a5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f19a5e270 0x555f19dfe270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:15.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.216+0000 7fa7a5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f19a5e270 0x555f19dfe270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57996/0 (socket says 172.21.15.42:57996) 2024-07-04T17:27:15.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.216+0000 7fa7a5e00640 1 -- 172.21.15.42:0/132197909 learned_addr learned my addr 172.21.15.42:0/132197909 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:15.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.216+0000 7fa7a5e00640 1 -- 172.21.15.42:0/132197909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f19e050a0 con 0x555f19a5e270 2024-07-04T17:27:15.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.217+0000 7fa7a5e00640 1 --2- 172.21.15.42:0/132197909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f19a5e270 0x555f19dfe270 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa7940128d0 tx=0x7fa79401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc88a9dd2e70e53b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:15.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.217+0000 7fa78dc00640 1 -- 172.21.15.42:0/132197909 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa794029040 con 0x555f19a5e270 2024-07-04T17:27:15.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.217+0000 7fa78dc00640 1 -- 172.21.15.42:0/132197909 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa794015c00 con 0x555f19a5e270 2024-07-04T17:27:15.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.217+0000 7fa78dc00640 1 -- 172.21.15.42:0/132197909 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa79404d040 con 0x555f19a5e270 2024-07-04T17:27:15.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.218+0000 7fa7af7f6740 1 -- 172.21.15.42:0/132197909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f19a5e270 msgr2=0x555f19dfe270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:15.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.218+0000 7fa7af7f6740 1 --2- 172.21.15.42:0/132197909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f19a5e270 0x555f19dfe270 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa7940128d0 tx=0x7fa79401c510 comp rx=0 tx=0).stop 2024-07-04T17:27:15.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.218+0000 7fa7af7f6740 1 -- 172.21.15.42:0/132197909 shutdown_connections 2024-07-04T17:27:15.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.218+0000 7fa7af7f6740 1 --2- 172.21.15.42:0/132197909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f19a5e270 0x555f19dfe270 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:15.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.218+0000 7fa7af7f6740 1 -- 172.21.15.42:0/132197909 >> 172.21.15.42:0/132197909 conn(0x555f19a523c0 msgr2=0x555f19e024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:15.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.218+0000 7fa7af7f6740 1 -- 172.21.15.42:0/132197909 shutdown_connections 2024-07-04T17:27:15.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.218+0000 7fa7af7f6740 1 -- 172.21.15.42:0/132197909 wait complete. 2024-07-04T17:27:15.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.219+0000 7fa7af7f6740 1 -- start start 2024-07-04T17:27:15.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.219+0000 7fa7af7f6740 1 Processor -- start 2024-07-04T17:27:15.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.220+0000 7fa7af7f6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f19a5e270 0x555f19e4d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:15.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.220+0000 7fa7af7f6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x555f19d64b50 con 0x555f19a5e270 2024-07-04T17:27:15.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.220+0000 7fa7af7f6740 10 client.0 _ll_register_callbacks cb 0x7fa794010e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T17:27:15.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.220+0000 7fa7a5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f19a5e270 0x555f19e4d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:15.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.221+0000 7fa7a5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f19a5e270 0x555f19e4d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58002/0 (socket says 172.21.15.42:58002) 2024-07-04T17:27:15.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.221+0000 7fa7a5e00640 1 -- 172.21.15.42:0/2913359473 learned_addr learned my addr 172.21.15.42:0/2913359473 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:15.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.221+0000 7fa7a5e00640 1 -- 172.21.15.42:0/2913359473 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f19e486b0 con 0x555f19a5e270 2024-07-04T17:27:15.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.221+0000 7fa7a5e00640 1 --2- 172.21.15.42:0/2913359473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f19a5e270 0x555f19e4d6f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa79401aeb0 tx=0x7fa794012650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d084e74248db134 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:15.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.221+0000 7fa782a00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-07-04T17:27:15.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.221+0000 7fa782a00640 1 -- 172.21.15.42:0/2913359473 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa79404d040 con 0x555f19a5e270 2024-07-04T17:27:15.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.221+0000 7fa782a00640 1 -- 172.21.15.42:0/2913359473 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 3439+0+0 (secure 0 0 0) 0x7fa79400e490 con 0x555f19a5e270 2024-07-04T17:27:15.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.221+0000 7fa7af7f6740 1 -- 172.21.15.42:0/2913359473 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa794015c00 con 0x555f19a5e270 2024-07-04T17:27:15.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.222+0000 7fa782a00640 1 -- 172.21.15.42:0/2913359473 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fa794029040 con 0x555f19a5e270 2024-07-04T17:27:15.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.222+0000 7fa7af7f6740 10 client.4496 fetch_fsmap learned FSMap version 17 2024-07-04T17:27:15.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.222+0000 7fa7af7f6740 1 -- 172.21.15.42:0/2913359473 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x555f19eb7300 con 0x555f19a5e270 2024-07-04T17:27:15.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.222+0000 7fa782a00640 1 -- 172.21.15.42:0/2913359473 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa794015c00 con 0x555f19a5e270 2024-07-04T17:27:15.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.222+0000 7fa782a00640 1 -- 172.21.15.42:0/2913359473 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 17) ==== 4107+0+0 (secure 0 0 0) 0x7fa79403c170 con 0x555f19a5e270 2024-07-04T17:27:15.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.222+0000 7fa7af7f6740 10 client.4496 fetch_fsmap finished waiting for FSMap version 17 2024-07-04T17:27:15.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.222+0000 7fa7af7f6740 10 client.4496 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-04T17:27:15.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.222+0000 7fa7af7f6740 20 client.4496 populate_metadata read hostname 'smithi042' 2024-07-04T17:27:15.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.222+0000 7fa7af7f6740 1 --2- 172.21.15.42:0/2913359473 >> [v2:172.21.15.42:6834/807364821,v1:172.21.15.42:6835/807364821] conn(0x555f19a1ce40 0x555f19eb66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:15.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.222+0000 7fa7af7f6740 4 client.4496 mds_command: new command op to 4423 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-04T17:27:15.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.222+0000 7fa7af7f6740 1 -- 172.21.15.42:0/2913359473 --> [v2:172.21.15.42:6834/807364821,v1:172.21.15.42:6835/807364821] -- command(tid 0: {"prefix": "status"}) -- 0x555f19e29340 con 0x555f19a1ce40 2024-07-04T17:27:15.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.223+0000 7fa7a5400640 1 --2- 172.21.15.42:0/2913359473 >> [v2:172.21.15.42:6834/807364821,v1:172.21.15.42:6835/807364821] conn(0x555f19a1ce40 0x555f19eb66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:15.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.223+0000 7fa7a5400640 1 --2- 172.21.15.42:0/2913359473 >> [v2:172.21.15.42:6834/807364821,v1:172.21.15.42:6835/807364821] conn(0x555f19a1ce40 0x555f19eb66d0 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c115ea9387629faf server_cookie=8ae7b63239389a7a in_seq=0 out_seq=0 2024-07-04T17:27:15.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.223+0000 7fa782a00640 10 client.4496 ms_handle_connect on v2:172.21.15.42:6834/807364821 2024-07-04T17:27:15.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.224+0000 7fa782a00640 1 -- 172.21.15.42:0/2913359473 <== mds.0 v2:172.21.15.42:6834/807364821 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x555f19e29340 con 0x555f19a1ce40 2024-07-04T17:27:15.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.224+0000 7fa782a00640 10 client.4496 handle_command_reply: tid=0 2024-07-04T17:27:15.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.224+0000 7fa7af7f6740 1 client.4496 shutdown 2024-07-04T17:27:15.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.224+0000 7fa7af7f6740 1 -- 172.21.15.42:0/2913359473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f19a5e270 msgr2=0x555f19e4d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:15.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.224+0000 7fa7af7f6740 1 --2- 172.21.15.42:0/2913359473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f19a5e270 0x555f19e4d6f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa79401aeb0 tx=0x7fa794012650 comp rx=0 tx=0).stop 2024-07-04T17:27:15.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.224+0000 7fa7af7f6740 1 -- 172.21.15.42:0/2913359473 shutdown_connections 2024-07-04T17:27:15.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.224+0000 7fa7af7f6740 1 --2- 172.21.15.42:0/2913359473 >> [v2:172.21.15.42:6834/807364821,v1:172.21.15.42:6835/807364821] conn(0x555f19a1ce40 0x555f19eb66d0 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:15.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.224+0000 7fa7af7f6740 1 --2- 172.21.15.42:0/2913359473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555f19a5e270 0x555f19e4d6f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:15.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.224+0000 7fa7af7f6740 1 -- 172.21.15.42:0/2913359473 >> 172.21.15.42:0/2913359473 conn(0x555f19a523c0 msgr2=0x555f19e024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:15.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.224+0000 7fa782a00640 0 client.4496 ms_handle_reset on v2:172.21.15.42:6834/807364821 2024-07-04T17:27:15.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.224+0000 7fa7af7f6740 1 -- 172.21.15.42:0/2913359473 shutdown_connections 2024-07-04T17:27:15.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.224+0000 7fa7af7f6740 1 -- 172.21.15.42:0/2913359473 wait complete. 2024-07-04T17:27:15.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.224+0000 7fa7af7f6740 20 client.4496 trim_cache size 0 max 16384 2024-07-04T17:27:15.228 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:27:15.228 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "be4f300a-93ef-4489-b540-37ece39b0aac", 2024-07-04T17:27:15.228 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-07-04T17:27:15.228 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 4423, 2024-07-04T17:27:15.228 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-07-04T17:27:15.228 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-07-04T17:27:15.228 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-07-04T17:27:15.228 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 7.376545041, 2024-07-04T17:27:15.228 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 17, 2024-07-04T17:27:15.228 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 27, 2024-07-04T17:27:15.228 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 27, 2024-07-04T17:27:15.228 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 16.271206164999999 2024-07-04T17:27:15.228 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:27:15.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.225+0000 7fa7a7200640 1 -- 172.21.15.42:0/1284993688 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa78403d2b0 msgr2=0x7fa78403f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:15.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.225+0000 7fa7a7200640 1 --2- 172.21.15.42:0/1284993688 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa78403d2b0 0x7fa78403f7c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa790034000 tx=0x7fa790004730 comp rx=0 tx=0).stop 2024-07-04T17:27:15.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.225+0000 7fa7a7200640 1 -- 172.21.15.42:0/1284993688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7a006c020 msgr2=0x7fa7a0148ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:15.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.225+0000 7fa7a7200640 1 --2- 172.21.15.42:0/1284993688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7a006c020 0x7fa7a0148ff0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa794001eb0 tx=0x7fa79400c810 comp rx=0 tx=0).stop 2024-07-04T17:27:15.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.225+0000 7fa7a7200640 1 -- 172.21.15.42:0/1284993688 shutdown_connections 2024-07-04T17:27:15.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.225+0000 7fa7a7200640 1 --2- 172.21.15.42:0/1284993688 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa78403d2b0 0x7fa78403f7c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:15.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.225+0000 7fa7a7200640 1 --2- 172.21.15.42:0/1284993688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7a006c020 0x7fa7a0148ff0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:15.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.225+0000 7fa7a7200640 1 -- 172.21.15.42:0/1284993688 >> 172.21.15.42:0/1284993688 conn(0x7fa7a00bebc0 msgr2=0x7fa7a014de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:15.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.225+0000 7fa7a7200640 1 -- 172.21.15.42:0/1284993688 shutdown_connections 2024-07-04T17:27:15.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.225+0000 7fa7a7200640 1 -- 172.21.15.42:0/1284993688 wait complete. 2024-07-04T17:27:15.254 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-04T17:27:15.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.543+0000 7f200fe00640 1 -- 172.21.15.42:0/2553609841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20100b1760 msgr2=0x7f20100b8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:15.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.543+0000 7f200fe00640 1 --2- 172.21.15.42:0/2553609841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20100b1760 0x7f20100b8da0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2000005fd0 tx=0x7f200002a5f0 comp rx=0 tx=0).stop 2024-07-04T17:27:15.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.543+0000 7f200fe00640 1 -- 172.21.15.42:0/2553609841 shutdown_connections 2024-07-04T17:27:15.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.543+0000 7f200fe00640 1 --2- 172.21.15.42:0/2553609841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20100b1760 0x7f20100b8da0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:15.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.544+0000 7f200fe00640 1 -- 172.21.15.42:0/2553609841 >> 172.21.15.42:0/2553609841 conn(0x7f20100bebc0 msgr2=0x7f20100befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:15.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.544+0000 7f200fe00640 1 -- 172.21.15.42:0/2553609841 shutdown_connections 2024-07-04T17:27:15.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.544+0000 7f200fe00640 1 -- 172.21.15.42:0/2553609841 wait complete. 2024-07-04T17:27:15.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.546+0000 7f200fe00640 1 Processor -- start 2024-07-04T17:27:15.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.546+0000 7f200fe00640 1 -- start start 2024-07-04T17:27:15.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.547+0000 7f200fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f201006c020 0x7f20101d5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:15.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.547+0000 7f200fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f201014eed0 con 0x7f201006c020 2024-07-04T17:27:15.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.548+0000 7f200ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f201006c020 0x7f20101d5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:15.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.548+0000 7f200ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f201006c020 0x7f20101d5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58018/0 (socket says 172.21.15.42:58018) 2024-07-04T17:27:15.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.548+0000 7f200ea00640 1 -- 172.21.15.42:0/2963474428 learned_addr learned my addr 172.21.15.42:0/2963474428 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:15.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.548+0000 7f200ea00640 1 -- 172.21.15.42:0/2963474428 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20101d7490 con 0x7f201006c020 2024-07-04T17:27:15.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.548+0000 7f200ea00640 1 --2- 172.21.15.42:0/2963474428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f201006c020 0x7f20101d5dd0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2000009850 tx=0x7f20000049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:15.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.549+0000 7f2006a00640 1 -- 172.21.15.42:0/2963474428 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f200003e030 con 0x7f201006c020 2024-07-04T17:27:15.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.549+0000 7f2006a00640 1 -- 172.21.15.42:0/2963474428 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20000035f0 con 0x7f201006c020 2024-07-04T17:27:15.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.549+0000 7f2006a00640 1 -- 172.21.15.42:0/2963474428 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2000035040 con 0x7f201006c020 2024-07-04T17:27:15.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.549+0000 7f200fe00640 1 -- 172.21.15.42:0/2963474428 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2010145990 con 0x7f201006c020 2024-07-04T17:27:15.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.549+0000 7f200fe00640 1 -- 172.21.15.42:0/2963474428 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2010148030 con 0x7f201006c020 2024-07-04T17:27:15.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.550+0000 7f200fe00640 1 -- 172.21.15.42:0/2963474428 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20100b8da0 con 0x7f201006c020 2024-07-04T17:27:15.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.550+0000 7f2006a00640 1 -- 172.21.15.42:0/2963474428 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f2000041030 con 0x7f201006c020 2024-07-04T17:27:15.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.551+0000 7f2006a00640 1 --2- 172.21.15.42:0/2963474428 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f1ff003d2b0 0x7f1ff003f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:15.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.551+0000 7f2006a00640 1 -- 172.21.15.42:0/2963474428 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3439+0+0 (secure 0 0 0) 0x7f200003f070 con 0x7f201006c020 2024-07-04T17:27:15.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.551+0000 7f200e000640 1 --2- 172.21.15.42:0/2963474428 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f1ff003d2b0 0x7f1ff003f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:15.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.552+0000 7f200e000640 1 --2- 172.21.15.42:0/2963474428 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f1ff003d2b0 0x7f1ff003f7c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f20100b8760 tx=0x7f1ffc0079f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:15.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.556+0000 7f2006a00640 1 -- 172.21.15.42:0/2963474428 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f2000077020 con 0x7f201006c020 2024-07-04T17:27:15.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.811+0000 7f200fe00640 1 -- 172.21.15.42:0/2963474428 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2010068e30 con 0x7f201006c020 2024-07-04T17:27:15.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.812+0000 7f2006a00640 1 -- 172.21.15.42:0/2963474428 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f200004bd60 con 0x7f201006c020 2024-07-04T17:27:15.814 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-04T17:27:15.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.815+0000 7f200fe00640 1 -- 172.21.15.42:0/2963474428 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f1ff003d2b0 msgr2=0x7f1ff003f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:15.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.815+0000 7f200fe00640 1 --2- 172.21.15.42:0/2963474428 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f1ff003d2b0 0x7f1ff003f7c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f20100b8760 tx=0x7f1ffc0079f0 comp rx=0 tx=0).stop 2024-07-04T17:27:15.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.815+0000 7f200fe00640 1 -- 172.21.15.42:0/2963474428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f201006c020 msgr2=0x7f20101d5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:15.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.815+0000 7f200fe00640 1 --2- 172.21.15.42:0/2963474428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f201006c020 0x7f20101d5dd0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2000009850 tx=0x7f20000049d0 comp rx=0 tx=0).stop 2024-07-04T17:27:15.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.815+0000 7f200fe00640 1 -- 172.21.15.42:0/2963474428 shutdown_connections 2024-07-04T17:27:15.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.815+0000 7f200fe00640 1 --2- 172.21.15.42:0/2963474428 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f1ff003d2b0 0x7f1ff003f7c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:15.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.815+0000 7f200fe00640 1 --2- 172.21.15.42:0/2963474428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f201006c020 0x7f20101d5dd0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:15.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.815+0000 7f200fe00640 1 -- 172.21.15.42:0/2963474428 >> 172.21.15.42:0/2963474428 conn(0x7f20100bebc0 msgr2=0x7f20100bbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:15.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.815+0000 7f200fe00640 1 -- 172.21.15.42:0/2963474428 shutdown_connections 2024-07-04T17:27:15.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:15.815+0000 7f200fe00640 1 -- 172.21.15.42:0/2963474428 wait complete. 2024-07-04T17:27:15.843 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-07-04T17:27:15.843 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-04T17:27:15.843 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2024-07-04T17:27:15.843 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-04T17:27:15.843 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-04T17:27:15.843 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-07-04T17:27:15.843 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-07-04T17:27:15.843 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-04T17:27:15.843 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-04T17:27:15.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:27:15.844 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2024-07-04T17:27:15.900 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-04T17:27:15.900 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-04T17:27:15.900 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-07-04T17:27:15.901 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:27:15.901 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-07-04T17:27:15.901 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:27:15.901 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:27:15.901 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2024-07-04T17:27:15.901 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f0 2024-07-04T17:27:15.901 INFO:teuthology.orchestra.run.smithi042.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:27:15.901 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2024-07-04T17:27:15.901 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f1 2024-07-04T17:27:15.901 INFO:teuthology.orchestra.run.smithi042.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-04T17:27:15.901 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:27:15.901 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f0 2024-07-04T17:27:15.901 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-07-04T17:27:15.901 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 42534sec preferred_lft 42534sec 2024-07-04T17:27:15.902 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::ec4:7aff:febd:1440/64 scope link noprefixroute 2024-07-04T17:27:15.902 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:27:15.902 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:27:15.902 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:27:15.902 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f1 2024-07-04T17:27:15.902 INFO:teuthology.orchestra.run.smithi042.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-04T17:27:15.902 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 32:75:d6:f1:31:e1 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:27:15.902 INFO:teuthology.orchestra.run.smithi042.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-04T17:27:15.902 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:27:15.902 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::8839:a9ff:fe74:17f5/64 scope link 2024-07-04T17:27:15.902 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:27:15.902 INFO:teuthology.orchestra.run.smithi042.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-04T17:27:15.903 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether e6:ab:57:fb:ca:9e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:27:15.903 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::e4ab:57ff:fefb:ca9e/64 scope link 2024-07-04T17:27:15.903 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:27:15.903 INFO:teuthology.orchestra.run.smithi042.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-04T17:27:15.903 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 32:75:d6:f1:31:e1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:27:15.903 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::3075:d6ff:fef1:31e1/64 scope link 2024-07-04T17:27:15.903 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:27:15.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:27:15.904 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-04T17:27:15.978 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-07-04T17:27:15.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:27:15.982 DEBUG:teuthology.orchestra.run.smithi042:> route 2024-07-04T17:27:16.034 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2024-07-04T17:27:16.034 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-04T17:27:16.034 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-04T17:27:16.035 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-04T17:27:16.035 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-04T17:27:16.036 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-04T17:27:16.036 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-07-04T17:27:16.036 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-04T17:27:16.036 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-04T17:27:16.036 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-04T17:27:16.036 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-07-04T17:27:16.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-04T17:27:16.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:27:16.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:27:16.263 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-07-04T17:27:16.313 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-04T17:27:16.314 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2024-07-04T17:27:16.314 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-04T17:27:16.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:27:16.315 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.0 up 2024-07-04T17:27:16.391 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-07-04T17:27:16.392 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-04T17:27:16.392 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-04T17:27:16.447 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-04T17:27:16.448 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-04T17:27:16.448 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-04T17:27:16.498 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:27:16.584 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:27:16.643 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:27:16.643 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:27:16.643 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:27:16.643 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:27:16.644 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:27:16.644 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:27:16.644 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:27:16.644 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:27:16.644 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:27:16.644 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:27:16.644 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:27:16.644 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:27:16.644 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:27:16.644 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:27:16.644 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:27:16.644 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:27:16.644 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:27:16.645 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:27:16.645 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:27:16.645 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:27:16.645 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:27:16.645 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:27:16.645 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:27:16.645 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:27:16.645 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:27:16.645 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:27:16.645 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:27:16.645 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:27:16.645 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:27:16.645 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:27:16.646 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:27:16.646 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:27:16.646 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:27:16.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:27:16.647 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:27:16.698 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-07-04T17:27:16.698 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-07-04T17:27:16.703 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:27:16.772 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:27:16.815 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:27:16.815 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:27:16.815 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:27:16.815 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:27:16.815 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:27:16.815 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:27:16.815 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:27:16.815 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:27:16.815 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:27:16.815 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:27:16.816 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:27:16.816 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:27:16.816 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:27:16.816 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:27:16.816 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:27:16.816 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:27:16.816 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:27:16.816 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:27:16.816 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:27:16.816 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:27:16.816 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:27:16.816 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:27:16.816 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:27:16.816 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:27:16.816 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:27:16.817 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:27:16.817 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:27:16.817 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:27:16.817 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:27:16.817 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:27:16.817 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:27:16.817 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:27:16.817 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:27:16.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:27:16.818 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:27:16.959 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:2024-07-04T17:27:16.957+0000 7f2c1b71f480 -1 init, newargv = 0x7f2bfc004580 newargc=15 2024-07-04T17:27:16.959 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[46040]: starting ceph client 2024-07-04T17:27:16.967 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[46040]: starting fuse 2024-07-04T17:27:18.004 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:27:18.100 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:27:18.161 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:27:18.161 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:27:18.161 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:27:18.161 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:27:18.161 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:27:18.161 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:27:18.161 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:27:18.161 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:27:18.161 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:27:18.161 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:27:18.161 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:27:18.161 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:27:18.162 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:27:18.162 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:27:18.162 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:27:18.162 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:27:18.162 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:27:18.162 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:27:18.162 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:27:18.162 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:27:18.162 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:27:18.162 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:27:18.162 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:27:18.162 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:27:18.162 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:27:18.163 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:27:18.163 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:27:18.163 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:27:18.163 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:27:18.163 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:27:18.163 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:27:18.163 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:27:18.163 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:27:18.163 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-07-04T17:27:18.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:27:18.164 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:27:18.219 INFO:teuthology.orchestra.run.smithi042.stdout:42 2024-07-04T17:27:18.219 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-07-04T17:27:18.219 DEBUG:teuthology.orchestra.run.smithi042:> sudo stdin-killer -- python3 -c ' 2024-07-04T17:27:18.219 DEBUG:teuthology.orchestra.run.smithi042:> import glob 2024-07-04T17:27:18.219 DEBUG:teuthology.orchestra.run.smithi042:> import re 2024-07-04T17:27:18.220 DEBUG:teuthology.orchestra.run.smithi042:> import os 2024-07-04T17:27:18.220 DEBUG:teuthology.orchestra.run.smithi042:> import subprocess 2024-07-04T17:27:18.220 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:27:18.220 DEBUG:teuthology.orchestra.run.smithi042:> def _find_admin_socket(client_name): 2024-07-04T17:27:18.220 DEBUG:teuthology.orchestra.run.smithi042:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-07-04T17:27:18.220 DEBUG:teuthology.orchestra.run.smithi042:> files = glob.glob(asok_path) 2024-07-04T17:27:18.220 DEBUG:teuthology.orchestra.run.smithi042:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-07-04T17:27:18.220 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:27:18.220 DEBUG:teuthology.orchestra.run.smithi042:> # Given a non-glob path, it better be there 2024-07-04T17:27:18.220 DEBUG:teuthology.orchestra.run.smithi042:> if "*" not in asok_path: 2024-07-04T17:27:18.220 DEBUG:teuthology.orchestra.run.smithi042:> assert(len(files) == 1) 2024-07-04T17:27:18.220 DEBUG:teuthology.orchestra.run.smithi042:> return files[0] 2024-07-04T17:27:18.220 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:27:18.220 DEBUG:teuthology.orchestra.run.smithi042:> for f in files: 2024-07-04T17:27:18.221 DEBUG:teuthology.orchestra.run.smithi042:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-07-04T17:27:18.221 DEBUG:teuthology.orchestra.run.smithi042:> if os.path.exists("/proc/{0}".format(pid)): 2024-07-04T17:27:18.221 DEBUG:teuthology.orchestra.run.smithi042:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-07-04T17:27:18.221 DEBUG:teuthology.orchestra.run.smithi042:> contents = proc_f.read() 2024-07-04T17:27:18.221 DEBUG:teuthology.orchestra.run.smithi042:> if mountpoint in contents: 2024-07-04T17:27:18.221 DEBUG:teuthology.orchestra.run.smithi042:> return f 2024-07-04T17:27:18.221 DEBUG:teuthology.orchestra.run.smithi042:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-07-04T17:27:18.221 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:27:18.221 DEBUG:teuthology.orchestra.run.smithi042:> print(_find_admin_socket("client.0")) 2024-07-04T17:27:18.221 DEBUG:teuthology.orchestra.run.smithi042:> ' 2024-07-04T17:27:18.402 INFO:teuthology.orchestra.run.smithi042.stdout:/var/run/ceph/ceph-client.0.46040.asok 2024-07-04T17:27:18.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:27:18.420 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.46040.asok 2024-07-04T17:27:18.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:27:18.420 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.46040.asok status 2024-07-04T17:27:18.730 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:27:18.730 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": { 2024-07-04T17:27:18.731 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_sha1": "9dba1e836b4756bf3a32be2a4936f371dff55008", 2024-07-04T17:27:18.731 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.0.0-4667-g9dba1e83 (9dba1e836b4756bf3a32be2a4936f371dff55008) squid (dev)", 2024-07-04T17:27:18.731 INFO:teuthology.orchestra.run.smithi042.stdout: "entity_id": "0", 2024-07-04T17:27:18.731 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": "smithi042", 2024-07-04T17:27:18.731 INFO:teuthology.orchestra.run.smithi042.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-07-04T17:27:18.731 INFO:teuthology.orchestra.run.smithi042.stdout: "pid": "46040", 2024-07-04T17:27:18.731 INFO:teuthology.orchestra.run.smithi042.stdout: "root": "/" 2024-07-04T17:27:18.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:18.731 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_count": 0, 2024-07-04T17:27:18.731 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_pinned_count": 0, 2024-07-04T17:27:18.731 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 4502, 2024-07-04T17:27:18.731 INFO:teuthology.orchestra.run.smithi042.stdout: "inst": { 2024-07-04T17:27:18.732 INFO:teuthology.orchestra.run.smithi042.stdout: "name": { 2024-07-04T17:27:18.732 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "client", 2024-07-04T17:27:18.732 INFO:teuthology.orchestra.run.smithi042.stdout: "num": 4502 2024-07-04T17:27:18.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:18.732 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-07-04T17:27:18.732 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-07-04T17:27:18.732 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-07-04T17:27:18.732 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2026951793 2024-07-04T17:27:18.732 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:18.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:18.732 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-07-04T17:27:18.732 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-07-04T17:27:18.732 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-07-04T17:27:18.732 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2026951793 2024-07-04T17:27:18.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:18.733 INFO:teuthology.orchestra.run.smithi042.stdout: "inst_str": "client.4502 192.168.0.1:0/2026951793", 2024-07-04T17:27:18.733 INFO:teuthology.orchestra.run.smithi042.stdout: "addr_str": "192.168.0.1:0/2026951793", 2024-07-04T17:27:18.733 INFO:teuthology.orchestra.run.smithi042.stdout: "inode_count": 1, 2024-07-04T17:27:18.733 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_epoch": 17, 2024-07-04T17:27:18.733 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch": 27, 2024-07-04T17:27:18.733 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch_barrier": 0, 2024-07-04T17:27:18.733 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklisted": false, 2024-07-04T17:27:18.733 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs" 2024-07-04T17:27:18.733 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:27:18.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:27:18.751 DEBUG:teuthology.orchestra.run.smithi042:> stat --file-system '--printf=%T 2024-07-04T17:27:18.751 DEBUG:teuthology.orchestra.run.smithi042:> ' -- /home/ubuntu/cephtest/mnt.0 2024-07-04T17:27:18.807 INFO:teuthology.orchestra.run.smithi042.stdout:fuseblk 2024-07-04T17:27:18.807 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-07-04T17:27:18.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:27:18.808 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-04T17:27:18.899 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-04T17:27:19.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.182+0000 7f1f1e000640 1 -- 172.21.15.42:0/1607570304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f1814abc0 msgr2=0x7f1f1806c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:19.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.182+0000 7f1f1e000640 1 --2- 172.21.15.42:0/1607570304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f1814abc0 0x7f1f1806c020 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1f08005fd0 tx=0x7f1f0802a840 comp rx=0 tx=0).stop 2024-07-04T17:27:19.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.182+0000 7f1f1e000640 1 -- 172.21.15.42:0/1607570304 shutdown_connections 2024-07-04T17:27:19.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.182+0000 7f1f1e000640 1 --2- 172.21.15.42:0/1607570304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f1814abc0 0x7f1f1806c020 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:19.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.182+0000 7f1f1e000640 1 -- 172.21.15.42:0/1607570304 >> 172.21.15.42:0/1607570304 conn(0x7f1f180bebc0 msgr2=0x7f1f180befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:19.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.183+0000 7f1f1e000640 1 -- 172.21.15.42:0/1607570304 shutdown_connections 2024-07-04T17:27:19.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.183+0000 7f1f1e000640 1 -- 172.21.15.42:0/1607570304 wait complete. 2024-07-04T17:27:19.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.185+0000 7f1f1e000640 1 Processor -- start 2024-07-04T17:27:19.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.185+0000 7f1f1e000640 1 -- start start 2024-07-04T17:27:19.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.186+0000 7f1f1e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f1814afd0 0x7f1f181e4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:19.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.186+0000 7f1f1e000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1814b8f0 con 0x7f1f1814afd0 2024-07-04T17:27:19.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.186+0000 7f1f1cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f1814afd0 0x7f1f181e4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:19.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.187+0000 7f1f1cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f1814afd0 0x7f1f181e4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58026/0 (socket says 172.21.15.42:58026) 2024-07-04T17:27:19.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.187+0000 7f1f1cc00640 1 -- 172.21.15.42:0/3178809822 learned_addr learned my addr 172.21.15.42:0/3178809822 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:19.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.187+0000 7f1f1cc00640 1 -- 172.21.15.42:0/3178809822 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f181e49c0 con 0x7f1f1814afd0 2024-07-04T17:27:19.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.187+0000 7f1f1cc00640 1 --2- 172.21.15.42:0/3178809822 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f1814afd0 0x7f1f181e4460 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1f08034040 tx=0x7f1f08032c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:19.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.188+0000 7f1f14c00640 1 -- 172.21.15.42:0/3178809822 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f08040070 con 0x7f1f1814afd0 2024-07-04T17:27:19.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.188+0000 7f1f14c00640 1 -- 172.21.15.42:0/3178809822 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f08032e30 con 0x7f1f1814afd0 2024-07-04T17:27:19.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.188+0000 7f1f14c00640 1 -- 172.21.15.42:0/3178809822 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f080043e0 con 0x7f1f1814afd0 2024-07-04T17:27:19.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.188+0000 7f1f1e000640 1 -- 172.21.15.42:0/3178809822 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f180aa130 con 0x7f1f1814afd0 2024-07-04T17:27:19.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.188+0000 7f1f1e000640 1 -- 172.21.15.42:0/3178809822 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f1814a2a0 con 0x7f1f1814afd0 2024-07-04T17:27:19.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.189+0000 7f1f1e000640 1 -- 172.21.15.42:0/3178809822 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f181e4b90 con 0x7f1f1814afd0 2024-07-04T17:27:19.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.189+0000 7f1f14c00640 1 -- 172.21.15.42:0/3178809822 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f1f08043070 con 0x7f1f1814afd0 2024-07-04T17:27:19.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.190+0000 7f1f14c00640 1 --2- 172.21.15.42:0/3178809822 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f1ef803d2b0 0x7f1ef803f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:19.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.190+0000 7f1f14c00640 1 -- 172.21.15.42:0/3178809822 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3439+0+0 (secure 0 0 0) 0x7f1f08072550 con 0x7f1f1814afd0 2024-07-04T17:27:19.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.190+0000 7f1f17e00640 1 --2- 172.21.15.42:0/3178809822 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f1ef803d2b0 0x7f1ef803f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:19.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.191+0000 7f1f17e00640 1 --2- 172.21.15.42:0/3178809822 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f1ef803d2b0 0x7f1ef803f7c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1f04001400 tx=0x7f1f040010e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:19.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.196+0000 7f1f14c00640 1 -- 172.21.15.42:0/3178809822 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f1f08037230 con 0x7f1f1814afd0 2024-07-04T17:27:19.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.451+0000 7f1f1e000640 1 -- 172.21.15.42:0/3178809822 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1f1814bfc0 con 0x7f1f1814afd0 2024-07-04T17:27:19.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.452+0000 7f1f14c00640 1 -- 172.21.15.42:0/3178809822 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f1f08031cc0 con 0x7f1f1814afd0 2024-07-04T17:27:19.453 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-04T17:27:19.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.455+0000 7f1f1e000640 1 -- 172.21.15.42:0/3178809822 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f1ef803d2b0 msgr2=0x7f1ef803f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:19.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.455+0000 7f1f1e000640 1 --2- 172.21.15.42:0/3178809822 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f1ef803d2b0 0x7f1ef803f7c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1f04001400 tx=0x7f1f040010e0 comp rx=0 tx=0).stop 2024-07-04T17:27:19.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.455+0000 7f1f1e000640 1 -- 172.21.15.42:0/3178809822 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f1814afd0 msgr2=0x7f1f181e4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:19.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.455+0000 7f1f1e000640 1 --2- 172.21.15.42:0/3178809822 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f1814afd0 0x7f1f181e4460 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1f08034040 tx=0x7f1f08032c70 comp rx=0 tx=0).stop 2024-07-04T17:27:19.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.455+0000 7f1f1e000640 1 -- 172.21.15.42:0/3178809822 shutdown_connections 2024-07-04T17:27:19.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.455+0000 7f1f1e000640 1 --2- 172.21.15.42:0/3178809822 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f1ef803d2b0 0x7f1ef803f7c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:19.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.455+0000 7f1f1e000640 1 --2- 172.21.15.42:0/3178809822 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f1814afd0 0x7f1f181e4460 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:19.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.455+0000 7f1f1e000640 1 -- 172.21.15.42:0/3178809822 >> 172.21.15.42:0/3178809822 conn(0x7f1f180bebc0 msgr2=0x7f1f180bbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:19.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.455+0000 7f1f1e000640 1 -- 172.21.15.42:0/3178809822 shutdown_connections 2024-07-04T17:27:19.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19.455+0000 7f1f1e000640 1 -- 172.21.15.42:0/3178809822 wait complete. 2024-07-04T17:27:19.481 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-07-04T17:27:19.481 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-04T17:27:19.481 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2024-07-04T17:27:19.481 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-07-04T17:27:19.481 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-07-04T17:27:19.481 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-07-04T17:27:19.481 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-07-04T17:27:19.481 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-04T17:27:19.481 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-07-04T17:27:19.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:27:19.482 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2024-07-04T17:27:19.532 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-04T17:27:19.533 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-04T17:27:19.533 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-07-04T17:27:19.533 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:27:19.533 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-07-04T17:27:19.533 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:27:19.533 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:27:19.533 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2024-07-04T17:27:19.533 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f0 2024-07-04T17:27:19.533 INFO:teuthology.orchestra.run.smithi042.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:27:19.533 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2024-07-04T17:27:19.533 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f1 2024-07-04T17:27:19.534 INFO:teuthology.orchestra.run.smithi042.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-04T17:27:19.534 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:27:19.534 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f0 2024-07-04T17:27:19.534 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-07-04T17:27:19.534 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 42531sec preferred_lft 42531sec 2024-07-04T17:27:19.534 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::ec4:7aff:febd:1440/64 scope link noprefixroute 2024-07-04T17:27:19.534 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:27:19.534 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:27:19.534 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:27:19.534 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f1 2024-07-04T17:27:19.534 INFO:teuthology.orchestra.run.smithi042.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-04T17:27:19.534 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 32:75:d6:f1:31:e1 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:27:19.534 INFO:teuthology.orchestra.run.smithi042.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-04T17:27:19.535 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:27:19.535 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::8839:a9ff:fe74:17f5/64 scope link 2024-07-04T17:27:19.535 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:27:19.535 INFO:teuthology.orchestra.run.smithi042.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-04T17:27:19.535 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether e6:ab:57:fb:ca:9e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:27:19.535 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::e4ab:57ff:fefb:ca9e/64 scope link 2024-07-04T17:27:19.535 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:27:19.535 INFO:teuthology.orchestra.run.smithi042.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-04T17:27:19.535 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 32:75:d6:f1:31:e1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:27:19.535 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::3075:d6ff:fef1:31e1/64 scope link 2024-07-04T17:27:19.535 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:27:19.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:27:19.536 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-04T17:27:19.618 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-07-04T17:27:19.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:27:19.623 DEBUG:teuthology.orchestra.run.smithi042:> route 2024-07-04T17:27:19.681 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2024-07-04T17:27:19.681 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-04T17:27:19.681 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-04T17:27:19.681 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-04T17:27:19.682 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-04T17:27:19.682 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-04T17:27:19.682 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-07-04T17:27:19.682 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-04T17:27:19.682 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-04T17:27:19.682 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-04T17:27:19.682 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-07-04T17:27:19.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-04T17:27:19.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:27:19.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:27:19.907 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-07-04T17:27:19.956 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-04T17:27:19.956 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2024-07-04T17:27:19.957 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-07-04T17:27:19.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:27:19.957 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.1 up 2024-07-04T17:27:20.039 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-07-04T17:27:20.039 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-04T17:27:20.039 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-07-04T17:27:20.092 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-07-04T17:27:20.092 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-04T17:27:20.092 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-07-04T17:27:20.147 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:27:20.236 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:27:20.295 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:27:20.295 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:27:20.295 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:27:20.295 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:27:20.295 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:27:20.296 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:27:20.296 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:27:20.296 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:27:20.296 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:27:20.296 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:27:20.296 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:27:20.296 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:27:20.296 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:27:20.296 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:27:20.296 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:27:20.296 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:27:20.296 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:27:20.296 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:27:20.297 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:27:20.297 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:27:20.297 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:27:20.297 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:27:20.297 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:27:20.297 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:27:20.297 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:27:20.297 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:27:20.297 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:27:20.297 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:27:20.297 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:27:20.297 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:27:20.297 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:27:20.297 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:27:20.298 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:27:20.298 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-07-04T17:27:20.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:27:20.299 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:27:20.357 INFO:teuthology.orchestra.run.smithi042.stdout:42 2024-07-04T17:27:20.357 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-07-04T17:27:20.357 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-07-04T17:27:20.362 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:27:20.435 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:27:20.480 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:27:20.480 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:27:20.480 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:27:20.480 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:27:20.480 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:27:20.480 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:27:20.480 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:27:20.480 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:27:20.480 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:27:20.481 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:27:20.481 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:27:20.481 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:27:20.481 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:27:20.481 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:27:20.481 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:27:20.481 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:27:20.481 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:27:20.481 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:27:20.481 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:27:20.481 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:27:20.481 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:27:20.481 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:27:20.481 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:27:20.482 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:27:20.482 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:27:20.482 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:27:20.482 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:27:20.482 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:27:20.482 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:27:20.482 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:27:20.482 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:27:20.482 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:27:20.482 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:27:20.482 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-07-04T17:27:20.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:27:20.483 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:27:20.573 INFO:teuthology.orchestra.run.smithi042.stdout:42 2024-07-04T17:27:20.612 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi042.stderr:2024-07-04T17:27:20.610+0000 7f396b91e480 -1 init, newargv = 0x7f394c001080 newargc=15 2024-07-04T17:27:20.612 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi042.stderr:ceph-fuse[46509]: starting ceph client 2024-07-04T17:27:20.619 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi042.stderr:ceph-fuse[46509]: starting fuse 2024-07-04T17:27:21.675 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:27:21.770 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:27:21.828 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:27:21.828 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:27:21.828 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:27:21.828 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:27:21.828 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:27:21.828 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:27:21.828 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:27:21.828 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:27:21.828 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:27:21.829 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:27:21.829 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:27:21.829 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:27:21.829 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:27:21.829 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:27:21.829 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:27:21.829 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:27:21.829 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:27:21.829 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:27:21.829 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:27:21.829 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:27:21.829 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:27:21.830 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:27:21.830 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:27:21.830 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:27:21.830 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:27:21.830 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:27:21.830 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:27:21.830 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:27:21.830 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:27:21.830 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:27:21.830 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:27:21.830 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:27:21.830 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:27:21.830 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-07-04T17:27:21.831 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.1 2024-07-04T17:27:21.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:27:21.832 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:27:21.889 INFO:teuthology.orchestra.run.smithi042.stdout:42 2024-07-04T17:27:21.889 INFO:teuthology.orchestra.run.smithi042.stdout:43 2024-07-04T17:27:21.889 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 43] 2024-07-04T17:27:21.890 DEBUG:teuthology.orchestra.run.smithi042:> sudo stdin-killer -- python3 -c ' 2024-07-04T17:27:21.890 DEBUG:teuthology.orchestra.run.smithi042:> import glob 2024-07-04T17:27:21.890 DEBUG:teuthology.orchestra.run.smithi042:> import re 2024-07-04T17:27:21.890 DEBUG:teuthology.orchestra.run.smithi042:> import os 2024-07-04T17:27:21.890 DEBUG:teuthology.orchestra.run.smithi042:> import subprocess 2024-07-04T17:27:21.890 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:27:21.890 DEBUG:teuthology.orchestra.run.smithi042:> def _find_admin_socket(client_name): 2024-07-04T17:27:21.890 DEBUG:teuthology.orchestra.run.smithi042:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-07-04T17:27:21.890 DEBUG:teuthology.orchestra.run.smithi042:> files = glob.glob(asok_path) 2024-07-04T17:27:21.890 DEBUG:teuthology.orchestra.run.smithi042:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-07-04T17:27:21.890 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:27:21.890 DEBUG:teuthology.orchestra.run.smithi042:> # Given a non-glob path, it better be there 2024-07-04T17:27:21.891 DEBUG:teuthology.orchestra.run.smithi042:> if "*" not in asok_path: 2024-07-04T17:27:21.891 DEBUG:teuthology.orchestra.run.smithi042:> assert(len(files) == 1) 2024-07-04T17:27:21.891 DEBUG:teuthology.orchestra.run.smithi042:> return files[0] 2024-07-04T17:27:21.891 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:27:21.891 DEBUG:teuthology.orchestra.run.smithi042:> for f in files: 2024-07-04T17:27:21.891 DEBUG:teuthology.orchestra.run.smithi042:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-07-04T17:27:21.891 DEBUG:teuthology.orchestra.run.smithi042:> if os.path.exists("/proc/{0}".format(pid)): 2024-07-04T17:27:21.891 DEBUG:teuthology.orchestra.run.smithi042:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-07-04T17:27:21.891 DEBUG:teuthology.orchestra.run.smithi042:> contents = proc_f.read() 2024-07-04T17:27:21.891 DEBUG:teuthology.orchestra.run.smithi042:> if mountpoint in contents: 2024-07-04T17:27:21.891 DEBUG:teuthology.orchestra.run.smithi042:> return f 2024-07-04T17:27:21.891 DEBUG:teuthology.orchestra.run.smithi042:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-07-04T17:27:21.891 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:27:21.891 DEBUG:teuthology.orchestra.run.smithi042:> print(_find_admin_socket("client.1")) 2024-07-04T17:27:21.891 DEBUG:teuthology.orchestra.run.smithi042:> ' 2024-07-04T17:27:22.075 INFO:teuthology.orchestra.run.smithi042.stdout:/var/run/ceph/ceph-client.1.46509.asok 2024-07-04T17:27:22.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:27:22.092 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.46509.asok 2024-07-04T17:27:22.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:27:22.092 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.46509.asok status 2024-07-04T17:27:22.403 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:27:22.403 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": { 2024-07-04T17:27:22.403 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_sha1": "9dba1e836b4756bf3a32be2a4936f371dff55008", 2024-07-04T17:27:22.404 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.0.0-4667-g9dba1e83 (9dba1e836b4756bf3a32be2a4936f371dff55008) squid (dev)", 2024-07-04T17:27:22.404 INFO:teuthology.orchestra.run.smithi042.stdout: "entity_id": "1", 2024-07-04T17:27:22.404 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": "smithi042", 2024-07-04T17:27:22.404 INFO:teuthology.orchestra.run.smithi042.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-07-04T17:27:22.404 INFO:teuthology.orchestra.run.smithi042.stdout: "pid": "46509", 2024-07-04T17:27:22.404 INFO:teuthology.orchestra.run.smithi042.stdout: "root": "/" 2024-07-04T17:27:22.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:22.404 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_count": 0, 2024-07-04T17:27:22.404 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_pinned_count": 0, 2024-07-04T17:27:22.404 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 4508, 2024-07-04T17:27:22.404 INFO:teuthology.orchestra.run.smithi042.stdout: "inst": { 2024-07-04T17:27:22.404 INFO:teuthology.orchestra.run.smithi042.stdout: "name": { 2024-07-04T17:27:22.404 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "client", 2024-07-04T17:27:22.404 INFO:teuthology.orchestra.run.smithi042.stdout: "num": 4508 2024-07-04T17:27:22.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:22.405 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-07-04T17:27:22.405 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-07-04T17:27:22.405 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.2:0", 2024-07-04T17:27:22.405 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3720587267 2024-07-04T17:27:22.405 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:22.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:22.405 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-07-04T17:27:22.405 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-07-04T17:27:22.405 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.2:0", 2024-07-04T17:27:22.405 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3720587267 2024-07-04T17:27:22.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:22.405 INFO:teuthology.orchestra.run.smithi042.stdout: "inst_str": "client.4508 192.168.0.2:0/3720587267", 2024-07-04T17:27:22.405 INFO:teuthology.orchestra.run.smithi042.stdout: "addr_str": "192.168.0.2:0/3720587267", 2024-07-04T17:27:22.405 INFO:teuthology.orchestra.run.smithi042.stdout: "inode_count": 1, 2024-07-04T17:27:22.406 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_epoch": 17, 2024-07-04T17:27:22.406 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch": 27, 2024-07-04T17:27:22.406 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch_barrier": 0, 2024-07-04T17:27:22.406 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklisted": false, 2024-07-04T17:27:22.406 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs" 2024-07-04T17:27:22.406 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:27:22.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:27:22.426 DEBUG:teuthology.orchestra.run.smithi042:> stat --file-system '--printf=%T 2024-07-04T17:27:22.426 DEBUG:teuthology.orchestra.run.smithi042:> ' -- /home/ubuntu/cephtest/mnt.1 2024-07-04T17:27:22.480 INFO:teuthology.orchestra.run.smithi042.stdout:fuseblk 2024-07-04T17:27:22.480 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-07-04T17:27:22.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:27:22.480 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-07-04T17:27:22.560 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-07-04T17:27:22.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:22.855+0000 7efc78000640 1 -- 172.21.15.42:0/1666586165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc700ab2f0 msgr2=0x7efc700a9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:22.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:22.855+0000 7efc78000640 1 --2- 172.21.15.42:0/1666586165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc700ab2f0 0x7efc700a9990 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7efc600097b0 tx=0x7efc6002a530 comp rx=0 tx=0).stop 2024-07-04T17:27:22.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:22.855+0000 7efc78000640 1 -- 172.21.15.42:0/1666586165 shutdown_connections 2024-07-04T17:27:22.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:22.855+0000 7efc78000640 1 --2- 172.21.15.42:0/1666586165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc700ab2f0 0x7efc700a9990 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:22.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:22.855+0000 7efc78000640 1 -- 172.21.15.42:0/1666586165 >> 172.21.15.42:0/1666586165 conn(0x7efc7006c020 msgr2=0x7efc7006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:22.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:22.856+0000 7efc78000640 1 -- 172.21.15.42:0/1666586165 shutdown_connections 2024-07-04T17:27:22.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:22.856+0000 7efc78000640 1 -- 172.21.15.42:0/1666586165 wait complete. 2024-07-04T17:27:22.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:22.858+0000 7efc78000640 1 Processor -- start 2024-07-04T17:27:22.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:22.858+0000 7efc78000640 1 -- start start 2024-07-04T17:27:22.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:22.859+0000 7efc78000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc7006d070 0x7efc700bd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:22.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:22.859+0000 7efc78000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efc7013f430 con 0x7efc7006d070 2024-07-04T17:27:22.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:22.860+0000 7efc75600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc7006d070 0x7efc700bd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:22.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:22.860+0000 7efc75600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc7006d070 0x7efc700bd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58040/0 (socket says 172.21.15.42:58040) 2024-07-04T17:27:22.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:22.860+0000 7efc75600640 1 -- 172.21.15.42:0/3515383225 learned_addr learned my addr 172.21.15.42:0/3515383225 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:22.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:22.860+0000 7efc75600640 1 -- 172.21.15.42:0/3515383225 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc700bd490 con 0x7efc7006d070 2024-07-04T17:27:22.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:22.861+0000 7efc75600640 1 --2- 172.21.15.42:0/3515383225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc7006d070 0x7efc700bd990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7efc60000c00 tx=0x7efc60032c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:22.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:22.861+0000 7efc6d600640 1 -- 172.21.15.42:0/3515383225 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc60040070 con 0x7efc7006d070 2024-07-04T17:27:22.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:22.861+0000 7efc6d600640 1 -- 172.21.15.42:0/3515383225 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc60003830 con 0x7efc7006d070 2024-07-04T17:27:22.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:22.861+0000 7efc6d600640 1 -- 172.21.15.42:0/3515383225 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc60003bb0 con 0x7efc7006d070 2024-07-04T17:27:22.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:22.861+0000 7efc78000640 1 -- 172.21.15.42:0/3515383225 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc700ad7e0 con 0x7efc7006d070 2024-07-04T17:27:22.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:22.861+0000 7efc78000640 1 -- 172.21.15.42:0/3515383225 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc701dcbf0 con 0x7efc7006d070 2024-07-04T17:27:22.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:22.863+0000 7efc78000640 1 -- 172.21.15.42:0/3515383225 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc7013fb00 con 0x7efc7006d070 2024-07-04T17:27:22.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:22.863+0000 7efc6d600640 1 -- 172.21.15.42:0/3515383225 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7efc60039040 con 0x7efc7006d070 2024-07-04T17:27:22.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:22.863+0000 7efc6d600640 1 --2- 172.21.15.42:0/3515383225 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7efc5003d2b0 0x7efc5003f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:22.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:22.863+0000 7efc6d600640 1 -- 172.21.15.42:0/3515383225 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3439+0+0 (secure 0 0 0) 0x7efc60072a90 con 0x7efc7006d070 2024-07-04T17:27:22.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:22.864+0000 7efc74c00640 1 --2- 172.21.15.42:0/3515383225 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7efc5003d2b0 0x7efc5003f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:22.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:22.865+0000 7efc74c00640 1 --2- 172.21.15.42:0/3515383225 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7efc5003d2b0 0x7efc5003f7c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efc5c002020 tx=0x7efc5c001610 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:22.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:22.870+0000 7efc6d600640 1 -- 172.21.15.42:0/3515383225 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7efc701dcbf0 con 0x7efc7006d070 2024-07-04T17:27:23.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:23.122+0000 7efc78000640 1 -- 172.21.15.42:0/3515383225 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7efc70069b70 con 0x7efc7006d070 2024-07-04T17:27:23.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:23.912+0000 7efc6d600640 1 -- 172.21.15.42:0/3515383225 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v18) ==== 136+0+0 (secure 0 0 0) 0x7efc60043360 con 0x7efc7006d070 2024-07-04T17:27:23.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:23.916+0000 7efc78000640 1 -- 172.21.15.42:0/3515383225 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7efc5003d2b0 msgr2=0x7efc5003f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:23.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:23.916+0000 7efc78000640 1 --2- 172.21.15.42:0/3515383225 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7efc5003d2b0 0x7efc5003f7c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efc5c002020 tx=0x7efc5c001610 comp rx=0 tx=0).stop 2024-07-04T17:27:23.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:23.916+0000 7efc78000640 1 -- 172.21.15.42:0/3515383225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc7006d070 msgr2=0x7efc700bd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:23.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:23.916+0000 7efc78000640 1 --2- 172.21.15.42:0/3515383225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc7006d070 0x7efc700bd990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7efc60000c00 tx=0x7efc60032c00 comp rx=0 tx=0).stop 2024-07-04T17:27:23.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:23.917+0000 7efc78000640 1 -- 172.21.15.42:0/3515383225 shutdown_connections 2024-07-04T17:27:23.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:23.917+0000 7efc78000640 1 --2- 172.21.15.42:0/3515383225 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7efc5003d2b0 0x7efc5003f7c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:23.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:23.917+0000 7efc78000640 1 --2- 172.21.15.42:0/3515383225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc7006d070 0x7efc700bd990 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:23.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:23.917+0000 7efc78000640 1 -- 172.21.15.42:0/3515383225 >> 172.21.15.42:0/3515383225 conn(0x7efc7006c020 msgr2=0x7efc700b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:23.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:23.917+0000 7efc78000640 1 -- 172.21.15.42:0/3515383225 shutdown_connections 2024-07-04T17:27:23.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:23.917+0000 7efc78000640 1 -- 172.21.15.42:0/3515383225 wait complete. 2024-07-04T17:27:23.955 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-04T17:27:24.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.246+0000 7fa892c00640 1 -- 172.21.15.42:0/1891788265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa88c06c020 msgr2=0x7fa88c0b1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:24.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.246+0000 7fa892c00640 1 --2- 172.21.15.42:0/1891788265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa88c06c020 0x7fa88c0b1b80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa8780097b0 tx=0x7fa87802a530 comp rx=0 tx=0).stop 2024-07-04T17:27:24.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.246+0000 7fa892c00640 1 -- 172.21.15.42:0/1891788265 shutdown_connections 2024-07-04T17:27:24.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.246+0000 7fa892c00640 1 --2- 172.21.15.42:0/1891788265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa88c06c020 0x7fa88c0b1b80 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:24.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.246+0000 7fa892c00640 1 -- 172.21.15.42:0/1891788265 >> 172.21.15.42:0/1891788265 conn(0x7fa88c0bebc0 msgr2=0x7fa88c0befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:24.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.246+0000 7fa892c00640 1 -- 172.21.15.42:0/1891788265 shutdown_connections 2024-07-04T17:27:24.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.246+0000 7fa892c00640 1 -- 172.21.15.42:0/1891788265 wait complete. 2024-07-04T17:27:24.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.249+0000 7fa892c00640 1 Processor -- start 2024-07-04T17:27:24.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.249+0000 7fa892c00640 1 -- start start 2024-07-04T17:27:24.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.250+0000 7fa892c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa88c06c020 0x7fa88c0bd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:24.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.250+0000 7fa892c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa88c0b20c0 con 0x7fa88c06c020 2024-07-04T17:27:24.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.251+0000 7fa891800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa88c06c020 0x7fa88c0bd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:24.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.251+0000 7fa891800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa88c06c020 0x7fa88c0bd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36576/0 (socket says 172.21.15.42:36576) 2024-07-04T17:27:24.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.251+0000 7fa891800640 1 -- 172.21.15.42:0/1979192640 learned_addr learned my addr 172.21.15.42:0/1979192640 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:24.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.251+0000 7fa891800640 1 -- 172.21.15.42:0/1979192640 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88c0a94d0 con 0x7fa88c06c020 2024-07-04T17:27:24.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.252+0000 7fa891800640 1 --2- 172.21.15.42:0/1979192640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa88c06c020 0x7fa88c0bd990 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa88000cb40 tx=0x7fa8800021e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:24.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.252+0000 7fa889600640 1 -- 172.21.15.42:0/1979192640 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa880016020 con 0x7fa88c06c020 2024-07-04T17:27:24.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.252+0000 7fa889600640 1 -- 172.21.15.42:0/1979192640 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa88000abd0 con 0x7fa88c06c020 2024-07-04T17:27:24.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.252+0000 7fa892c00640 1 -- 172.21.15.42:0/1979192640 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa88c1414a0 con 0x7fa88c06c020 2024-07-04T17:27:24.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.252+0000 7fa889600640 1 -- 172.21.15.42:0/1979192640 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa880012420 con 0x7fa88c06c020 2024-07-04T17:27:24.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.252+0000 7fa892c00640 1 -- 172.21.15.42:0/1979192640 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa88c140fa0 con 0x7fa88c06c020 2024-07-04T17:27:24.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.253+0000 7fa892c00640 1 -- 172.21.15.42:0/1979192640 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa88c0b27b0 con 0x7fa88c06c020 2024-07-04T17:27:24.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.254+0000 7fa889600640 1 -- 172.21.15.42:0/1979192640 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fa880019020 con 0x7fa88c06c020 2024-07-04T17:27:24.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.254+0000 7fa889600640 1 --2- 172.21.15.42:0/1979192640 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa86c03d2b0 0x7fa86c03f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:24.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.254+0000 7fa889600640 1 -- 172.21.15.42:0/1979192640 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3439+0+0 (secure 0 0 0) 0x7fa880051a90 con 0x7fa88c06c020 2024-07-04T17:27:24.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.255+0000 7fa890e00640 1 --2- 172.21.15.42:0/1979192640 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa86c03d2b0 0x7fa86c03f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:24.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.255+0000 7fa890e00640 1 --2- 172.21.15.42:0/1979192640 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa86c03d2b0 0x7fa86c03f7c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa878034000 tx=0x7fa878004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:24.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.260+0000 7fa889600640 1 -- 172.21.15.42:0/1979192640 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fa880020020 con 0x7fa88c06c020 2024-07-04T17:27:24.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.475+0000 7fa892c00640 1 -- 172.21.15.42:0/1979192640 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa88c068e30 con 0x7fa88c06c020 2024-07-04T17:27:24.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.476+0000 7fa889600640 1 -- 172.21.15.42:0/1979192640 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v27) ==== 84+0+207 (secure 0 0 0) 0x7fa880011c60 con 0x7fa88c06c020 2024-07-04T17:27:24.477 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:27:24.478 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-07-04T17:27:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 1, 2024-07-04T17:27:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": ".mgr" 2024-07-04T17:27:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 4, 2024-07-04T17:27:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": "cephfs_metadata" 2024-07-04T17:27:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 5, 2024-07-04T17:27:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": "cephfs_data" 2024-07-04T17:27:24.478 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:24.479 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-07-04T17:27:24.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.479+0000 7fa892c00640 1 -- 172.21.15.42:0/1979192640 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa86c03d2b0 msgr2=0x7fa86c03f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:24.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.479+0000 7fa892c00640 1 --2- 172.21.15.42:0/1979192640 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa86c03d2b0 0x7fa86c03f7c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa878034000 tx=0x7fa878004730 comp rx=0 tx=0).stop 2024-07-04T17:27:24.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.479+0000 7fa892c00640 1 -- 172.21.15.42:0/1979192640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa88c06c020 msgr2=0x7fa88c0bd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:24.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.479+0000 7fa892c00640 1 --2- 172.21.15.42:0/1979192640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa88c06c020 0x7fa88c0bd990 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa88000cb40 tx=0x7fa8800021e0 comp rx=0 tx=0).stop 2024-07-04T17:27:24.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.479+0000 7fa892c00640 1 -- 172.21.15.42:0/1979192640 shutdown_connections 2024-07-04T17:27:24.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.479+0000 7fa892c00640 1 --2- 172.21.15.42:0/1979192640 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa86c03d2b0 0x7fa86c03f7c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:24.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.479+0000 7fa892c00640 1 --2- 172.21.15.42:0/1979192640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa88c06c020 0x7fa88c0bd990 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:24.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.479+0000 7fa892c00640 1 -- 172.21.15.42:0/1979192640 >> 172.21.15.42:0/1979192640 conn(0x7fa88c0bebc0 msgr2=0x7fa88c0bbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:24.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.479+0000 7fa892c00640 1 -- 172.21.15.42:0/1979192640 shutdown_connections 2024-07-04T17:27:24.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.479+0000 7fa892c00640 1 -- 172.21.15.42:0/1979192640 wait complete. 2024-07-04T17:27:24.507 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'backup_fs' 2024-07-04T17:27:24.507 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_metadata --pg_num_min 64 2024-07-04T17:27:24.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.799+0000 7f2721e00640 1 -- 172.21.15.42:0/2158062271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f271c0b25d0 msgr2=0x7f271c0b35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:24.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.799+0000 7f2721e00640 1 --2- 172.21.15.42:0/2158062271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f271c0b25d0 0x7f271c0b35f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f27080097b0 tx=0x7f270802a540 comp rx=0 tx=0).stop 2024-07-04T17:27:24.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.799+0000 7f2721e00640 1 -- 172.21.15.42:0/2158062271 shutdown_connections 2024-07-04T17:27:24.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.799+0000 7f2721e00640 1 --2- 172.21.15.42:0/2158062271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f271c0b25d0 0x7f271c0b35f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:24.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.799+0000 7f2721e00640 1 -- 172.21.15.42:0/2158062271 >> 172.21.15.42:0/2158062271 conn(0x7f271c0bf700 msgr2=0x7f271c0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:24.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.799+0000 7f2721e00640 1 -- 172.21.15.42:0/2158062271 shutdown_connections 2024-07-04T17:27:24.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.799+0000 7f2721e00640 1 -- 172.21.15.42:0/2158062271 wait complete. 2024-07-04T17:27:24.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.801+0000 7f2721e00640 1 Processor -- start 2024-07-04T17:27:24.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.801+0000 7f2721e00640 1 -- start start 2024-07-04T17:27:24.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.803+0000 7f2721e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f271c0b01f0 0x7f271c0ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:24.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.803+0000 7f2721e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f271c06cbf0 con 0x7f271c0b01f0 2024-07-04T17:27:24.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.803+0000 7f271aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f271c0b01f0 0x7f271c0ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:24.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.803+0000 7f271aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f271c0b01f0 0x7f271c0ae0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36594/0 (socket says 172.21.15.42:36594) 2024-07-04T17:27:24.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.803+0000 7f271aa00640 1 -- 172.21.15.42:0/181869247 learned_addr learned my addr 172.21.15.42:0/181869247 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:24.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.804+0000 7f271aa00640 1 -- 172.21.15.42:0/181869247 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f271c1e45f0 con 0x7f271c0b01f0 2024-07-04T17:27:24.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.804+0000 7f271aa00640 1 --2- 172.21.15.42:0/181869247 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f271c0b01f0 0x7f271c0ae0f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f270c00c380 tx=0x7f270c00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:24.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.804+0000 7f2713400640 1 -- 172.21.15.42:0/181869247 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f270c0116c0 con 0x7f271c0b01f0 2024-07-04T17:27:24.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.804+0000 7f2713400640 1 -- 172.21.15.42:0/181869247 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f270c017070 con 0x7f271c0b01f0 2024-07-04T17:27:24.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.804+0000 7f2713400640 1 -- 172.21.15.42:0/181869247 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f270c011b20 con 0x7f271c0b01f0 2024-07-04T17:27:24.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.804+0000 7f2721e00640 1 -- 172.21.15.42:0/181869247 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f271c0b5b90 con 0x7f271c0b01f0 2024-07-04T17:27:24.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.804+0000 7f2721e00640 1 -- 172.21.15.42:0/181869247 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f271c141580 con 0x7f271c0b01f0 2024-07-04T17:27:24.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.806+0000 7f2721e00640 1 -- 172.21.15.42:0/181869247 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f271c06d290 con 0x7f271c0b01f0 2024-07-04T17:27:24.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.806+0000 7f2713400640 1 -- 172.21.15.42:0/181869247 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f270c011cc0 con 0x7f271c0b01f0 2024-07-04T17:27:24.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.807+0000 7f2713400640 1 --2- 172.21.15.42:0/181869247 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f26fc03d2b0 0x7f26fc03f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:24.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.807+0000 7f2713400640 1 -- 172.21.15.42:0/181869247 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3439+0+0 (secure 0 0 0) 0x7f270c051340 con 0x7f271c0b01f0 2024-07-04T17:27:24.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.807+0000 7f271a000640 1 --2- 172.21.15.42:0/181869247 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f26fc03d2b0 0x7f26fc03f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:24.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.808+0000 7f271a000640 1 --2- 172.21.15.42:0/181869247 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f26fc03d2b0 0x7f26fc03f7c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2708034000 tx=0x7f2708004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:24.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:24.814+0000 7f2713400640 1 -- 172.21.15.42:0/181869247 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f271c141f90 con 0x7f271c0b01f0 2024-07-04T17:27:25.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:25.024+0000 7f2721e00640 1 -- 172.21.15.42:0/181869247 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64} v 0) -- 0x7f271c0ac5a0 con 0x7f271c0b01f0 2024-07-04T17:27:25.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:25.925+0000 7f2713400640 1 -- 172.21.15.42:0/181869247 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64}]=0 pool 'backup_fs_metadata' created v28) ==== 144+0+0 (secure 0 0 0) 0x7f270c010cf0 con 0x7f271c0b01f0 2024-07-04T17:27:25.928 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'backup_fs_metadata' created 2024-07-04T17:27:25.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:25.930+0000 7f2721e00640 1 -- 172.21.15.42:0/181869247 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f26fc03d2b0 msgr2=0x7f26fc03f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:25.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:25.930+0000 7f2721e00640 1 --2- 172.21.15.42:0/181869247 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f26fc03d2b0 0x7f26fc03f7c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2708034000 tx=0x7f2708004730 comp rx=0 tx=0).stop 2024-07-04T17:27:25.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:25.930+0000 7f2721e00640 1 -- 172.21.15.42:0/181869247 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f271c0b01f0 msgr2=0x7f271c0ae0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:25.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:25.930+0000 7f2721e00640 1 --2- 172.21.15.42:0/181869247 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f271c0b01f0 0x7f271c0ae0f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f270c00c380 tx=0x7f270c00c840 comp rx=0 tx=0).stop 2024-07-04T17:27:25.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:25.931+0000 7f2721e00640 1 -- 172.21.15.42:0/181869247 shutdown_connections 2024-07-04T17:27:25.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:25.931+0000 7f2721e00640 1 --2- 172.21.15.42:0/181869247 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f26fc03d2b0 0x7f26fc03f7c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:25.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:25.931+0000 7f2721e00640 1 --2- 172.21.15.42:0/181869247 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f271c0b01f0 0x7f271c0ae0f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:25.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:25.931+0000 7f2721e00640 1 -- 172.21.15.42:0/181869247 >> 172.21.15.42:0/181869247 conn(0x7f271c0bf700 msgr2=0x7f271c143b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:25.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:25.932+0000 7f2721e00640 1 -- 172.21.15.42:0/181869247 shutdown_connections 2024-07-04T17:27:25.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:25.933+0000 7f2721e00640 1 -- 172.21.15.42:0/181869247 wait complete. 2024-07-04T17:27:25.969 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-04T17:27:26.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.245+0000 7f9365e00640 1 -- 172.21.15.42:0/3162825192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93600a94d0 msgr2=0x7f93600b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:26.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.245+0000 7f9365e00640 1 --2- 172.21.15.42:0/3162825192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93600a94d0 0x7f93600b1090 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f934c005fd0 tx=0x7f934c02a5f0 comp rx=0 tx=0).stop 2024-07-04T17:27:26.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.245+0000 7f9365e00640 1 -- 172.21.15.42:0/3162825192 shutdown_connections 2024-07-04T17:27:26.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.245+0000 7f9365e00640 1 --2- 172.21.15.42:0/3162825192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93600a94d0 0x7f93600b1090 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:26.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.245+0000 7f9365e00640 1 -- 172.21.15.42:0/3162825192 >> 172.21.15.42:0/3162825192 conn(0x7f936006c020 msgr2=0x7f936006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:26.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.245+0000 7f9365e00640 1 -- 172.21.15.42:0/3162825192 shutdown_connections 2024-07-04T17:27:26.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.245+0000 7f9365e00640 1 -- 172.21.15.42:0/3162825192 wait complete. 2024-07-04T17:27:26.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.247+0000 7f9365e00640 1 Processor -- start 2024-07-04T17:27:26.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.247+0000 7f9365e00640 1 -- start start 2024-07-04T17:27:26.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.249+0000 7f9365e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f936006dcf0 0x7f93601436d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:26.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.249+0000 7f9365e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9360072220 con 0x7f936006dcf0 2024-07-04T17:27:26.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.249+0000 7f935ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f936006dcf0 0x7f93601436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:26.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.249+0000 7f935ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f936006dcf0 0x7f93601436d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36604/0 (socket says 172.21.15.42:36604) 2024-07-04T17:27:26.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.249+0000 7f935ea00640 1 -- 172.21.15.42:0/2106492197 learned_addr learned my addr 172.21.15.42:0/2106492197 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:26.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.250+0000 7f935ea00640 1 -- 172.21.15.42:0/2106492197 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9360140ec0 con 0x7f936006dcf0 2024-07-04T17:27:26.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.250+0000 7f935ea00640 1 --2- 172.21.15.42:0/2106492197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f936006dcf0 0x7f93601436d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f935000c340 tx=0x7f935000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:26.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.250+0000 7f9357400640 1 -- 172.21.15.42:0/2106492197 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9350017070 con 0x7f936006dcf0 2024-07-04T17:27:26.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.250+0000 7f9357400640 1 -- 172.21.15.42:0/2106492197 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93500035f0 con 0x7f936006dcf0 2024-07-04T17:27:26.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.250+0000 7f9357400640 1 -- 172.21.15.42:0/2106492197 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9350003970 con 0x7f936006dcf0 2024-07-04T17:27:26.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.250+0000 7f9365e00640 1 -- 172.21.15.42:0/2106492197 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f936013aae0 con 0x7f936006dcf0 2024-07-04T17:27:26.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.251+0000 7f9365e00640 1 -- 172.21.15.42:0/2106492197 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9360140af0 con 0x7f936006dcf0 2024-07-04T17:27:26.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.252+0000 7f9357400640 1 -- 172.21.15.42:0/2106492197 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f935001a070 con 0x7f936006dcf0 2024-07-04T17:27:26.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.252+0000 7f9365e00640 1 -- 172.21.15.42:0/2106492197 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93600b1090 con 0x7f936006dcf0 2024-07-04T17:27:26.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.252+0000 7f9357400640 1 --2- 172.21.15.42:0/2106492197 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f934003d2b0 0x7f934003f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:26.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.252+0000 7f9357400640 1 -- 172.21.15.42:0/2106492197 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3812+0+0 (secure 0 0 0) 0x7f93500508f0 con 0x7f936006dcf0 2024-07-04T17:27:26.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.253+0000 7f935e000640 1 --2- 172.21.15.42:0/2106492197 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f934003d2b0 0x7f934003f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:26.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.253+0000 7f935e000640 1 --2- 172.21.15.42:0/2106492197 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f934003d2b0 0x7f934003f7c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f934c005fd0 tx=0x7f934c004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:26.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.258+0000 7f9357400640 1 -- 172.21.15.42:0/2106492197 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f9350003da0 con 0x7f936006dcf0 2024-07-04T17:27:26.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.468+0000 7f9365e00640 1 -- 172.21.15.42:0/2106492197 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f9360069b70 con 0x7f936006dcf0 2024-07-04T17:27:26.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.930+0000 7f9357400640 1 -- 172.21.15.42:0/2106492197 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'backup_fs_data' created v29) ==== 176+0+0 (secure 0 0 0) 0x7f935001bd70 con 0x7f936006dcf0 2024-07-04T17:27:26.932 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'backup_fs_data' created 2024-07-04T17:27:26.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.933+0000 7f9365e00640 1 -- 172.21.15.42:0/2106492197 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f934003d2b0 msgr2=0x7f934003f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:26.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.933+0000 7f9365e00640 1 --2- 172.21.15.42:0/2106492197 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f934003d2b0 0x7f934003f7c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f934c005fd0 tx=0x7f934c004730 comp rx=0 tx=0).stop 2024-07-04T17:27:26.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.933+0000 7f9365e00640 1 -- 172.21.15.42:0/2106492197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f936006dcf0 msgr2=0x7f93601436d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:26.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.934+0000 7f9365e00640 1 --2- 172.21.15.42:0/2106492197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f936006dcf0 0x7f93601436d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f935000c340 tx=0x7f935000c800 comp rx=0 tx=0).stop 2024-07-04T17:27:26.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.934+0000 7f9365e00640 1 -- 172.21.15.42:0/2106492197 shutdown_connections 2024-07-04T17:27:26.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.934+0000 7f9365e00640 1 --2- 172.21.15.42:0/2106492197 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f934003d2b0 0x7f934003f7c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:26.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.934+0000 7f9365e00640 1 --2- 172.21.15.42:0/2106492197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f936006dcf0 0x7f93601436d0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:26.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.934+0000 7f9365e00640 1 -- 172.21.15.42:0/2106492197 >> 172.21.15.42:0/2106492197 conn(0x7f936006c020 msgr2=0x7f93600b0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:26.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.934+0000 7f9365e00640 1 -- 172.21.15.42:0/2106492197 shutdown_connections 2024-07-04T17:27:26.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:26.934+0000 7f9365e00640 1 -- 172.21.15.42:0/2106492197 wait complete. 2024-07-04T17:27:26.980 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new backup_fs backup_fs_metadata backup_fs_data 2024-07-04T17:27:27.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.247+0000 7f87cde00640 1 -- 172.21.15.42:0/1929114980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87c806ee40 msgr2=0x7f87c8149db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:27.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.247+0000 7f87cde00640 1 --2- 172.21.15.42:0/1929114980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87c806ee40 0x7f87c8149db0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f87b80097b0 tx=0x7f87b802a540 comp rx=0 tx=0).stop 2024-07-04T17:27:27.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.247+0000 7f87cde00640 1 -- 172.21.15.42:0/1929114980 shutdown_connections 2024-07-04T17:27:27.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.247+0000 7f87cde00640 1 --2- 172.21.15.42:0/1929114980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87c806ee40 0x7f87c8149db0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:27.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.248+0000 7f87cde00640 1 -- 172.21.15.42:0/1929114980 >> 172.21.15.42:0/1929114980 conn(0x7f87c80bf4d0 msgr2=0x7f87c80af870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:27.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.248+0000 7f87cde00640 1 -- 172.21.15.42:0/1929114980 shutdown_connections 2024-07-04T17:27:27.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.248+0000 7f87cde00640 1 -- 172.21.15.42:0/1929114980 wait complete. 2024-07-04T17:27:27.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.250+0000 7f87cde00640 1 Processor -- start 2024-07-04T17:27:27.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.250+0000 7f87cde00640 1 -- start start 2024-07-04T17:27:27.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.252+0000 7f87cde00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87c806ee40 0x7f87c8142490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:27.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.252+0000 7f87cde00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f87c80b2750 con 0x7f87c806ee40 2024-07-04T17:27:27.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.252+0000 7f87c6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87c806ee40 0x7f87c8142490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:27.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.252+0000 7f87c6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87c806ee40 0x7f87c8142490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36620/0 (socket says 172.21.15.42:36620) 2024-07-04T17:27:27.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.253+0000 7f87c6a00640 1 -- 172.21.15.42:0/1080086440 learned_addr learned my addr 172.21.15.42:0/1080086440 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:27.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.253+0000 7f87c6a00640 1 -- 172.21.15.42:0/1080086440 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c80732d0 con 0x7f87c806ee40 2024-07-04T17:27:27.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.253+0000 7f87c6a00640 1 --2- 172.21.15.42:0/1080086440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87c806ee40 0x7f87c8142490 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f87b80098d0 tx=0x7f87b8033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:27.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.254+0000 7f87bf400640 1 -- 172.21.15.42:0/1080086440 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87b8031bc0 con 0x7f87c806ee40 2024-07-04T17:27:27.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.254+0000 7f87bf400640 1 -- 172.21.15.42:0/1080086440 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87b8031e50 con 0x7f87c806ee40 2024-07-04T17:27:27.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.254+0000 7f87bf400640 1 -- 172.21.15.42:0/1080086440 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87b8033200 con 0x7f87c806ee40 2024-07-04T17:27:27.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.254+0000 7f87cde00640 1 -- 172.21.15.42:0/1080086440 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87c8073570 con 0x7f87c806ee40 2024-07-04T17:27:27.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.254+0000 7f87cde00640 1 -- 172.21.15.42:0/1080086440 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87c81416a0 con 0x7f87c806ee40 2024-07-04T17:27:27.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.255+0000 7f87cde00640 1 -- 172.21.15.42:0/1080086440 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87c8149db0 con 0x7f87c806ee40 2024-07-04T17:27:27.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.256+0000 7f87bf400640 1 -- 172.21.15.42:0/1080086440 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f87b80333a0 con 0x7f87c806ee40 2024-07-04T17:27:27.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.256+0000 7f87bf400640 1 --2- 172.21.15.42:0/1080086440 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f87a803d2b0 0x7f87a803f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:27.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.256+0000 7f87bf400640 1 -- 172.21.15.42:0/1080086440 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4197+0+0 (secure 0 0 0) 0x7f87b8072d20 con 0x7f87c806ee40 2024-07-04T17:27:27.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.256+0000 7f87c6000640 1 --2- 172.21.15.42:0/1080086440 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f87a803d2b0 0x7f87a803f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:27.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.257+0000 7f87c6000640 1 --2- 172.21.15.42:0/1080086440 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f87a803d2b0 0x7f87a803f7c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f87b40015c0 tx=0x7f87b4007a90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:27.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.262+0000 7f87bf400640 1 -- 172.21.15.42:0/1080086440 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f87b8077070 con 0x7f87c806ee40 2024-07-04T17:27:27.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.511+0000 7f87cde00640 1 -- 172.21.15.42:0/1080086440 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"} v 0) -- 0x7f87c8073d50 con 0x7f87c806ee40 2024-07-04T17:27:27.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.958+0000 7f87bf400640 1 -- 172.21.15.42:0/1080086440 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"}]=0 new fs with metadata pool 6 and data pool 7 v19) ==== 181+0+0 (secure 0 0 0) 0x7f87b803f030 con 0x7f87c806ee40 2024-07-04T17:27:27.961 INFO:teuthology.orchestra.run.smithi042.stderr:new fs with metadata pool 6 and data pool 7 2024-07-04T17:27:27.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.969+0000 7f87cde00640 1 -- 172.21.15.42:0/1080086440 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f87a803d2b0 msgr2=0x7f87a803f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:27.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.969+0000 7f87cde00640 1 --2- 172.21.15.42:0/1080086440 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f87a803d2b0 0x7f87a803f7c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f87b40015c0 tx=0x7f87b4007a90 comp rx=0 tx=0).stop 2024-07-04T17:27:27.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.969+0000 7f87cde00640 1 -- 172.21.15.42:0/1080086440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87c806ee40 msgr2=0x7f87c8142490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:27.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.969+0000 7f87cde00640 1 --2- 172.21.15.42:0/1080086440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87c806ee40 0x7f87c8142490 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f87b80098d0 tx=0x7f87b8033e40 comp rx=0 tx=0).stop 2024-07-04T17:27:27.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.970+0000 7f87cde00640 1 -- 172.21.15.42:0/1080086440 shutdown_connections 2024-07-04T17:27:27.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.970+0000 7f87cde00640 1 --2- 172.21.15.42:0/1080086440 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f87a803d2b0 0x7f87a803f7c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:27.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.970+0000 7f87cde00640 1 --2- 172.21.15.42:0/1080086440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87c806ee40 0x7f87c8142490 secure :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f87b80098d0 tx=0x7f87b8033e40 comp rx=0 tx=0).stop 2024-07-04T17:27:27.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.970+0000 7f87cde00640 1 -- 172.21.15.42:0/1080086440 >> 172.21.15.42:0/1080086440 conn(0x7f87c80bf4d0 msgr2=0x7f87c8143650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:27.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.972+0000 7f87cde00640 1 -- 172.21.15.42:0/1080086440 shutdown_connections 2024-07-04T17:27:27.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:27.972+0000 7f87cde00640 1 -- 172.21.15.42:0/1080086440 wait complete. 2024-07-04T17:27:28.024 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:27:28.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.273+0000 7fa737e00640 1 -- 172.21.15.42:0/1136039430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa73806c200 msgr2=0x7fa73814ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:28.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.273+0000 7fa737e00640 1 --2- 172.21.15.42:0/1136039430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa73806c200 0x7fa73814ef10 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa728005fd0 tx=0x7fa72802a560 comp rx=0 tx=0).stop 2024-07-04T17:27:28.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.273+0000 7fa737e00640 1 -- 172.21.15.42:0/1136039430 shutdown_connections 2024-07-04T17:27:28.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.273+0000 7fa737e00640 1 --2- 172.21.15.42:0/1136039430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa73806c200 0x7fa73814ef10 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:28.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.273+0000 7fa737e00640 1 -- 172.21.15.42:0/1136039430 >> 172.21.15.42:0/1136039430 conn(0x7fa7380bebc0 msgr2=0x7fa7380befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:28.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.273+0000 7fa737e00640 1 -- 172.21.15.42:0/1136039430 shutdown_connections 2024-07-04T17:27:28.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.274+0000 7fa737e00640 1 -- 172.21.15.42:0/1136039430 wait complete. 2024-07-04T17:27:28.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.276+0000 7fa737e00640 1 Processor -- start 2024-07-04T17:27:28.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.276+0000 7fa737e00640 1 -- start start 2024-07-04T17:27:28.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.278+0000 7fa737e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa73806c200 0x7fa738147f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:28.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.278+0000 7fa737e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa73814ae50 con 0x7fa73806c200 2024-07-04T17:27:28.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.278+0000 7fa736a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa73806c200 0x7fa738147f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:28.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.278+0000 7fa736a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa73806c200 0x7fa738147f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36632/0 (socket says 172.21.15.42:36632) 2024-07-04T17:27:28.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.278+0000 7fa736a00640 1 -- 172.21.15.42:0/2715699087 learned_addr learned my addr 172.21.15.42:0/2715699087 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:28.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.278+0000 7fa736a00640 1 -- 172.21.15.42:0/2715699087 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7381d5bf0 con 0x7fa73806c200 2024-07-04T17:27:28.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.279+0000 7fa736a00640 1 --2- 172.21.15.42:0/2715699087 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa73806c200 0x7fa738147f20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa728009850 tx=0x7fa7280049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:28.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.279+0000 7fa72ea00640 1 -- 172.21.15.42:0/2715699087 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa72803e030 con 0x7fa73806c200 2024-07-04T17:27:28.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.279+0000 7fa72ea00640 1 -- 172.21.15.42:0/2715699087 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7280035f0 con 0x7fa73806c200 2024-07-04T17:27:28.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.279+0000 7fa72ea00640 1 -- 172.21.15.42:0/2715699087 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa728035040 con 0x7fa73806c200 2024-07-04T17:27:28.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.279+0000 7fa737e00640 1 -- 172.21.15.42:0/2715699087 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7380b6530 con 0x7fa73806c200 2024-07-04T17:27:28.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.279+0000 7fa737e00640 1 -- 172.21.15.42:0/2715699087 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7380b7130 con 0x7fa73806c200 2024-07-04T17:27:28.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.281+0000 7fa72ea00640 1 -- 172.21.15.42:0/2715699087 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fa728041030 con 0x7fa73806c200 2024-07-04T17:27:28.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.281+0000 7fa737e00640 1 -- 172.21.15.42:0/2715699087 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa73814b490 con 0x7fa73806c200 2024-07-04T17:27:28.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.281+0000 7fa72ea00640 1 --2- 172.21.15.42:0/2715699087 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa71803d2b0 0x7fa71803f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:28.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.281+0000 7fa72ea00640 1 -- 172.21.15.42:0/2715699087 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4303+0+0 (secure 0 0 0) 0x7fa72803f070 con 0x7fa73806c200 2024-07-04T17:27:28.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.281+0000 7fa736000640 1 --2- 172.21.15.42:0/2715699087 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa71803d2b0 0x7fa71803f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:28.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.282+0000 7fa736000640 1 --2- 172.21.15.42:0/2715699087 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa71803d2b0 0x7fa71803f7c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa7380bac80 tx=0x7fa7240079f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:28.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.286+0000 7fa72ea00640 1 -- 172.21.15.42:0/2715699087 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fa72804bd60 con 0x7fa73806c200 2024-07-04T17:27:28.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.494+0000 7fa737e00640 1 -- 172.21.15.42:0/2715699087 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7381d6570 con 0x7fa73806c200 2024-07-04T17:27:28.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.497+0000 7fa72ea00640 1 -- 172.21.15.42:0/2715699087 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) ==== 74+0+14462 (secure 0 0 0) 0x7fa728031400 con 0x7fa73806c200 2024-07-04T17:27:28.499 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:27:28.499 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":30,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:27:27.514060+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:27:04.932839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-07-04T17:27:06.393713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.0800000429153442,"score_stable":1.0800000429153442,"optimal_score":1,"raw_score_acting":1.0800000429153442,"raw_score_stable":1.0800000429153442,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:27:25.040056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:27:26.487236+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:27:28.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.500+0000 7fa737e00640 1 -- 172.21.15.42:0/2715699087 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa71803d2b0 msgr2=0x7fa71803f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:28.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.500+0000 7fa737e00640 1 --2- 172.21.15.42:0/2715699087 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa71803d2b0 0x7fa71803f7c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa7380bac80 tx=0x7fa7240079f0 comp rx=0 tx=0).stop 2024-07-04T17:27:28.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.500+0000 7fa737e00640 1 -- 172.21.15.42:0/2715699087 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa73806c200 msgr2=0x7fa738147f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:28.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.500+0000 7fa737e00640 1 --2- 172.21.15.42:0/2715699087 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa73806c200 0x7fa738147f20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa728009850 tx=0x7fa7280049d0 comp rx=0 tx=0).stop 2024-07-04T17:27:28.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.500+0000 7fa737e00640 1 -- 172.21.15.42:0/2715699087 shutdown_connections 2024-07-04T17:27:28.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.500+0000 7fa737e00640 1 --2- 172.21.15.42:0/2715699087 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fa71803d2b0 0x7fa71803f7c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:28.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.500+0000 7fa737e00640 1 --2- 172.21.15.42:0/2715699087 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa73806c200 0x7fa738147f20 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:28.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.500+0000 7fa737e00640 1 -- 172.21.15.42:0/2715699087 >> 172.21.15.42:0/2715699087 conn(0x7fa7380bebc0 msgr2=0x7fa7380bbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:28.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.500+0000 7fa737e00640 1 -- 172.21.15.42:0/2715699087 shutdown_connections 2024-07-04T17:27:28.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.500+0000 7fa737e00640 1 -- 172.21.15.42:0/2715699087 wait complete. 2024-07-04T17:27:28.525 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:27:28.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.784+0000 7f0e19e00640 1 -- 172.21.15.42:0/411951634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e140a9fa0 msgr2=0x7f0e140bb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:28.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.784+0000 7f0e19e00640 1 --2- 172.21.15.42:0/411951634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e140a9fa0 0x7f0e140bb1f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc005fd0 tx=0x7f0dfc02a8b0 comp rx=0 tx=0).stop 2024-07-04T17:27:28.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.784+0000 7f0e19e00640 1 -- 172.21.15.42:0/411951634 shutdown_connections 2024-07-04T17:27:28.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.784+0000 7f0e19e00640 1 --2- 172.21.15.42:0/411951634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e140a9fa0 0x7f0e140bb1f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:28.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.784+0000 7f0e19e00640 1 -- 172.21.15.42:0/411951634 >> 172.21.15.42:0/411951634 conn(0x7f0e140bf700 msgr2=0x7f0e140af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:28.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.784+0000 7f0e19e00640 1 -- 172.21.15.42:0/411951634 shutdown_connections 2024-07-04T17:27:28.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.784+0000 7f0e19e00640 1 -- 172.21.15.42:0/411951634 wait complete. 2024-07-04T17:27:28.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.786+0000 7f0e19e00640 1 Processor -- start 2024-07-04T17:27:28.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.786+0000 7f0e19e00640 1 -- start start 2024-07-04T17:27:28.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.788+0000 7f0e19e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e140a9fa0 0x7f0e141d4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:28.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.788+0000 7f0e19e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0e140ab180 con 0x7f0e140a9fa0 2024-07-04T17:27:28.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.788+0000 7f0e12a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e140a9fa0 0x7f0e141d4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:28.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.788+0000 7f0e12a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e140a9fa0 0x7f0e141d4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36636/0 (socket says 172.21.15.42:36636) 2024-07-04T17:27:28.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.788+0000 7f0e12a00640 1 -- 172.21.15.42:0/2855665467 learned_addr learned my addr 172.21.15.42:0/2855665467 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:28.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.788+0000 7f0e12a00640 1 -- 172.21.15.42:0/2855665467 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e141d4370 con 0x7f0e140a9fa0 2024-07-04T17:27:28.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.789+0000 7f0e12a00640 1 --2- 172.21.15.42:0/2855665467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e140a9fa0 0x7f0e141d4810 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400c380 tx=0x7f0e0400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:28.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.789+0000 7f0e0be00640 1 -- 172.21.15.42:0/2855665467 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e040116c0 con 0x7f0e140a9fa0 2024-07-04T17:27:28.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.789+0000 7f0e0be00640 1 -- 172.21.15.42:0/2855665467 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e04017070 con 0x7f0e140a9fa0 2024-07-04T17:27:28.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.789+0000 7f0e0be00640 1 -- 172.21.15.42:0/2855665467 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e04011b20 con 0x7f0e140a9fa0 2024-07-04T17:27:28.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.789+0000 7f0e19e00640 1 -- 172.21.15.42:0/2855665467 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e1413b8f0 con 0x7f0e140a9fa0 2024-07-04T17:27:28.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.789+0000 7f0e19e00640 1 -- 172.21.15.42:0/2855665467 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e140ae100 con 0x7f0e140a9fa0 2024-07-04T17:27:28.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.790+0000 7f0e19e00640 1 -- 172.21.15.42:0/2855665467 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e140aabc0 con 0x7f0e140a9fa0 2024-07-04T17:27:28.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.790+0000 7f0e0be00640 1 -- 172.21.15.42:0/2855665467 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f0e04011cc0 con 0x7f0e140a9fa0 2024-07-04T17:27:28.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.791+0000 7f0e0be00640 1 --2- 172.21.15.42:0/2855665467 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f0df003d2b0 0x7f0df003f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:28.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.791+0000 7f0e0be00640 1 -- 172.21.15.42:0/2855665467 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4303+0+0 (secure 0 0 0) 0x7f0e040516a0 con 0x7f0e140a9fa0 2024-07-04T17:27:28.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.791+0000 7f0e0a000640 1 --2- 172.21.15.42:0/2855665467 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f0df003d2b0 0x7f0df003f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:28.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.792+0000 7f0e0a000640 1 --2- 172.21.15.42:0/2855665467 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f0df003d2b0 0x7f0df003f7c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc034000 tx=0x7f0dfc004ab0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:28.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:28.796+0000 7f0e0be00640 1 -- 172.21.15.42:0/2855665467 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f0e04010cf0 con 0x7f0e140a9fa0 2024-07-04T17:27:29.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.013+0000 7f0e19e00640 1 -- 172.21.15.42:0/2855665467 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e141d4540 con 0x7f0e140a9fa0 2024-07-04T17:27:29.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.015+0000 7f0e0be00640 1 -- 172.21.15.42:0/2855665467 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+14449 (secure 0 0 0) 0x7f0e04019310 con 0x7f0e140a9fa0 2024-07-04T17:27:29.016 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:27:29.017 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":31,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:27:28.959929+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:27:04.932839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-07-04T17:27:06.393713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.0800000429153442,"score_stable":1.0800000429153442,"optimal_score":1,"raw_score_acting":1.0800000429153442,"raw_score_stable":1.0800000429153442,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:27:25.040056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:27:26.487236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:27:29.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.017+0000 7f0e19e00640 1 -- 172.21.15.42:0/2855665467 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f0df003d2b0 msgr2=0x7f0df003f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:29.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.017+0000 7f0e19e00640 1 --2- 172.21.15.42:0/2855665467 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f0df003d2b0 0x7f0df003f7c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc034000 tx=0x7f0dfc004ab0 comp rx=0 tx=0).stop 2024-07-04T17:27:29.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.017+0000 7f0e19e00640 1 -- 172.21.15.42:0/2855665467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e140a9fa0 msgr2=0x7f0e141d4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:29.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.017+0000 7f0e19e00640 1 --2- 172.21.15.42:0/2855665467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e140a9fa0 0x7f0e141d4810 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400c380 tx=0x7f0e0400c840 comp rx=0 tx=0).stop 2024-07-04T17:27:29.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.017+0000 7f0e19e00640 1 -- 172.21.15.42:0/2855665467 shutdown_connections 2024-07-04T17:27:29.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.017+0000 7f0e19e00640 1 --2- 172.21.15.42:0/2855665467 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f0df003d2b0 0x7f0df003f7c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:29.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.017+0000 7f0e19e00640 1 --2- 172.21.15.42:0/2855665467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e140a9fa0 0x7f0e141d4810 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:29.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.017+0000 7f0e19e00640 1 -- 172.21.15.42:0/2855665467 >> 172.21.15.42:0/2855665467 conn(0x7f0e140bf700 msgr2=0x7f0e140b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:29.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.017+0000 7f0e19e00640 1 -- 172.21.15.42:0/2855665467 shutdown_connections 2024-07-04T17:27:29.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.018+0000 7f0e19e00640 1 -- 172.21.15.42:0/2855665467 wait complete. 2024-07-04T17:27:29.044 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs standby_count_wanted 0 2024-07-04T17:27:29.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.308+0000 7fbc4fe00640 1 -- 172.21.15.42:0/2998634495 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc480a94d0 msgr2=0x7fbc480b1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:29.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.308+0000 7fbc4fe00640 1 --2- 172.21.15.42:0/2998634495 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc480a94d0 0x7fbc480b1100 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c00b2b0 tx=0x7fbc3c02ad10 comp rx=0 tx=0).stop 2024-07-04T17:27:29.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.308+0000 7fbc4fe00640 1 -- 172.21.15.42:0/2998634495 shutdown_connections 2024-07-04T17:27:29.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.308+0000 7fbc4fe00640 1 --2- 172.21.15.42:0/2998634495 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc480a94d0 0x7fbc480b1100 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:29.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.308+0000 7fbc4fe00640 1 -- 172.21.15.42:0/2998634495 >> 172.21.15.42:0/2998634495 conn(0x7fbc4806c020 msgr2=0x7fbc4806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:29.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.309+0000 7fbc4fe00640 1 -- 172.21.15.42:0/2998634495 shutdown_connections 2024-07-04T17:27:29.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.309+0000 7fbc4fe00640 1 -- 172.21.15.42:0/2998634495 wait complete. 2024-07-04T17:27:29.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.311+0000 7fbc4fe00640 1 Processor -- start 2024-07-04T17:27:29.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.311+0000 7fbc4fe00640 1 -- start start 2024-07-04T17:27:29.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.312+0000 7fbc4fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc48073b50 0x7fbc48152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:29.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.312+0000 7fbc4fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbc48072740 con 0x7fbc48073b50 2024-07-04T17:27:29.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.312+0000 7fbc4d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc48073b50 0x7fbc48152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:29.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.312+0000 7fbc4d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc48073b50 0x7fbc48152550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36644/0 (socket says 172.21.15.42:36644) 2024-07-04T17:27:29.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.312+0000 7fbc4d400640 1 -- 172.21.15.42:0/3114664120 learned_addr learned my addr 172.21.15.42:0/3114664120 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:29.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.313+0000 7fbc4d400640 1 -- 172.21.15.42:0/3114664120 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc481e1720 con 0x7fbc48073b50 2024-07-04T17:27:29.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.313+0000 7fbc4d400640 1 --2- 172.21.15.42:0/3114664120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc48073b50 0x7fbc48152550 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c000c00 tx=0x7fbc3c0097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:29.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.314+0000 7fbc45600640 1 -- 172.21.15.42:0/3114664120 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc3c038040 con 0x7fbc48073b50 2024-07-04T17:27:29.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.314+0000 7fbc45600640 1 -- 172.21.15.42:0/3114664120 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc3c041070 con 0x7fbc48073b50 2024-07-04T17:27:29.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.314+0000 7fbc4fe00640 1 -- 172.21.15.42:0/3114664120 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc48150cc0 con 0x7fbc48073b50 2024-07-04T17:27:29.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.314+0000 7fbc45600640 1 -- 172.21.15.42:0/3114664120 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc3c004420 con 0x7fbc48073b50 2024-07-04T17:27:29.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.314+0000 7fbc4fe00640 1 -- 172.21.15.42:0/3114664120 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc480aa840 con 0x7fbc48073b50 2024-07-04T17:27:29.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.315+0000 7fbc45600640 1 -- 172.21.15.42:0/3114664120 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fbc3c0035f0 con 0x7fbc48073b50 2024-07-04T17:27:29.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.315+0000 7fbc4fe00640 1 -- 172.21.15.42:0/3114664120 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc48072e70 con 0x7fbc48073b50 2024-07-04T17:27:29.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.316+0000 7fbc45600640 1 --2- 172.21.15.42:0/3114664120 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fbc2803d2b0 0x7fbc2803f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:29.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.316+0000 7fbc45600640 1 -- 172.21.15.42:0/3114664120 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4303+0+0 (secure 0 0 0) 0x7fbc3c071e60 con 0x7fbc48073b50 2024-07-04T17:27:29.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.316+0000 7fbc4ca00640 1 --2- 172.21.15.42:0/3114664120 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fbc2803d2b0 0x7fbc2803f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:29.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.317+0000 7fbc4ca00640 1 --2- 172.21.15.42:0/3114664120 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fbc2803d2b0 0x7fbc2803f7c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbc48073660 tx=0x7fbc34007a90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:29.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.321+0000 7fbc45600640 1 -- 172.21.15.42:0/3114664120 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fbc3c04b020 con 0x7fbc48073b50 2024-07-04T17:27:29.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.569+0000 7fbc4fe00640 1 -- 172.21.15.42:0/3114664120 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fbc481e1bf0 con 0x7fbc48073b50 2024-07-04T17:27:29.648 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-04T17:27:29.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.969+0000 7fbc45600640 1 -- 172.21.15.42:0/3114664120 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7fbc3c0453c0 con 0x7fbc48073b50 2024-07-04T17:27:29.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.973+0000 7fbc4fe00640 1 -- 172.21.15.42:0/3114664120 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fbc2803d2b0 msgr2=0x7fbc2803f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:29.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.973+0000 7fbc4fe00640 1 --2- 172.21.15.42:0/3114664120 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fbc2803d2b0 0x7fbc2803f7c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbc48073660 tx=0x7fbc34007a90 comp rx=0 tx=0).stop 2024-07-04T17:27:29.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.973+0000 7fbc4fe00640 1 -- 172.21.15.42:0/3114664120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc48073b50 msgr2=0x7fbc48152550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:29.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.973+0000 7fbc4fe00640 1 --2- 172.21.15.42:0/3114664120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc48073b50 0x7fbc48152550 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c000c00 tx=0x7fbc3c0097d0 comp rx=0 tx=0).stop 2024-07-04T17:27:29.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.973+0000 7fbc4fe00640 1 -- 172.21.15.42:0/3114664120 shutdown_connections 2024-07-04T17:27:29.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.973+0000 7fbc4fe00640 1 --2- 172.21.15.42:0/3114664120 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fbc2803d2b0 0x7fbc2803f7c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:29.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.973+0000 7fbc4fe00640 1 --2- 172.21.15.42:0/3114664120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc48073b50 0x7fbc48152550 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:29.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.973+0000 7fbc4fe00640 1 -- 172.21.15.42:0/3114664120 >> 172.21.15.42:0/3114664120 conn(0x7fbc4806c020 msgr2=0x7fbc48075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:29.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.973+0000 7fbc4fe00640 1 -- 172.21.15.42:0/3114664120 shutdown_connections 2024-07-04T17:27:29.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:29.973+0000 7fbc4fe00640 1 -- 172.21.15.42:0/3114664120 wait complete. 2024-07-04T17:27:30.009 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-04T17:27:30.009 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs allow_standby_replay false 2024-07-04T17:27:30.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.302+0000 7fd79a200640 1 -- 172.21.15.42:0/3141371552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7940b0db0 msgr2=0x7fd7940abc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:30.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.302+0000 7fd79a200640 1 --2- 172.21.15.42:0/3141371552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7940b0db0 0x7fd7940abc80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd7800097b0 tx=0x7fd78002a440 comp rx=0 tx=0).stop 2024-07-04T17:27:30.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.303+0000 7fd79a200640 1 -- 172.21.15.42:0/3141371552 shutdown_connections 2024-07-04T17:27:30.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.303+0000 7fd79a200640 1 --2- 172.21.15.42:0/3141371552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7940b0db0 0x7fd7940abc80 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:30.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.303+0000 7fd79a200640 1 -- 172.21.15.42:0/3141371552 >> 172.21.15.42:0/3141371552 conn(0x7fd7940afaa0 msgr2=0x7fd7940afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:30.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.303+0000 7fd79a200640 1 -- 172.21.15.42:0/3141371552 shutdown_connections 2024-07-04T17:27:30.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.303+0000 7fd79a200640 1 -- 172.21.15.42:0/3141371552 wait complete. 2024-07-04T17:27:30.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.305+0000 7fd79a200640 1 Processor -- start 2024-07-04T17:27:30.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.305+0000 7fd79a200640 1 -- start start 2024-07-04T17:27:30.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.306+0000 7fd79a200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7940ac5b0 0x7fd794138d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:30.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.306+0000 7fd79a200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd7940ae590 con 0x7fd7940ac5b0 2024-07-04T17:27:30.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.306+0000 7fd798e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7940ac5b0 0x7fd794138d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:30.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.307+0000 7fd798e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7940ac5b0 0x7fd794138d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36668/0 (socket says 172.21.15.42:36668) 2024-07-04T17:27:30.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.307+0000 7fd798e00640 1 -- 172.21.15.42:0/798032909 learned_addr learned my addr 172.21.15.42:0/798032909 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:30.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.307+0000 7fd798e00640 1 -- 172.21.15.42:0/798032909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7941d5990 con 0x7fd7940ac5b0 2024-07-04T17:27:30.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.307+0000 7fd798e00640 1 --2- 172.21.15.42:0/798032909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7940ac5b0 0x7fd794138d60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd78800c480 tx=0x7fd78800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:30.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.308+0000 7fd790c00640 1 -- 172.21.15.42:0/798032909 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7880116c0 con 0x7fd7940ac5b0 2024-07-04T17:27:30.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.308+0000 7fd790c00640 1 -- 172.21.15.42:0/798032909 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd788017070 con 0x7fd7940ac5b0 2024-07-04T17:27:30.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.308+0000 7fd79a200640 1 -- 172.21.15.42:0/798032909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd79413f270 con 0x7fd7940ac5b0 2024-07-04T17:27:30.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.308+0000 7fd790c00640 1 -- 172.21.15.42:0/798032909 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd788011b40 con 0x7fd7940ac5b0 2024-07-04T17:27:30.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.308+0000 7fd79a200640 1 -- 172.21.15.42:0/798032909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7940bf870 con 0x7fd7940ac5b0 2024-07-04T17:27:30.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.309+0000 7fd79a200640 1 -- 172.21.15.42:0/798032909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7940abc80 con 0x7fd7940ac5b0 2024-07-04T17:27:30.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.309+0000 7fd790c00640 1 -- 172.21.15.42:0/798032909 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fd788011ce0 con 0x7fd7940ac5b0 2024-07-04T17:27:30.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.310+0000 7fd790c00640 1 --2- 172.21.15.42:0/798032909 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fd77403d2b0 0x7fd77403f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:30.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.310+0000 7fd790c00640 1 -- 172.21.15.42:0/798032909 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4303+0+0 (secure 0 0 0) 0x7fd788051720 con 0x7fd7940ac5b0 2024-07-04T17:27:30.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.310+0000 7fd793e00640 1 --2- 172.21.15.42:0/798032909 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fd77403d2b0 0x7fd77403f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:30.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.311+0000 7fd793e00640 1 --2- 172.21.15.42:0/798032909 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fd77403d2b0 0x7fd77403f7c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd780034000 tx=0x7fd780004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:30.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.315+0000 7fd790c00640 1 -- 172.21.15.42:0/798032909 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fd78801ac80 con 0x7fd7940ac5b0 2024-07-04T17:27:30.561 INFO:tasks.ceph.mds.a.smithi042.stderr:2024-07-04T17:27:30.559+0000 7f63cf400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-04T17:27:30.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.567+0000 7fd79a200640 1 -- 172.21.15.42:0/798032909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd794068e30 con 0x7fd7940ac5b0 2024-07-04T17:27:30.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.985+0000 7fd790c00640 1 -- 172.21.15.42:0/798032909 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"}]=0 v23) ==== 125+0+0 (secure 0 0 0) 0x7fd788051320 con 0x7fd7940ac5b0 2024-07-04T17:27:30.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.989+0000 7fd79a200640 1 -- 172.21.15.42:0/798032909 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fd77403d2b0 msgr2=0x7fd77403f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:30.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.989+0000 7fd79a200640 1 --2- 172.21.15.42:0/798032909 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fd77403d2b0 0x7fd77403f7c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd780034000 tx=0x7fd780004730 comp rx=0 tx=0).stop 2024-07-04T17:27:30.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.989+0000 7fd79a200640 1 -- 172.21.15.42:0/798032909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7940ac5b0 msgr2=0x7fd794138d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:30.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.989+0000 7fd79a200640 1 --2- 172.21.15.42:0/798032909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7940ac5b0 0x7fd794138d60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd78800c480 tx=0x7fd78800c940 comp rx=0 tx=0).stop 2024-07-04T17:27:30.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.989+0000 7fd79a200640 1 -- 172.21.15.42:0/798032909 shutdown_connections 2024-07-04T17:27:30.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.990+0000 7fd79a200640 1 --2- 172.21.15.42:0/798032909 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fd77403d2b0 0x7fd77403f7c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:30.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.990+0000 7fd79a200640 1 --2- 172.21.15.42:0/798032909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7940ac5b0 0x7fd794138d60 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:30.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.990+0000 7fd79a200640 1 -- 172.21.15.42:0/798032909 >> 172.21.15.42:0/798032909 conn(0x7fd7940afaa0 msgr2=0x7fd79413ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:30.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.990+0000 7fd79a200640 1 -- 172.21.15.42:0/798032909 shutdown_connections 2024-07-04T17:27:30.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:30.990+0000 7fd79a200640 1 -- 172.21.15.42:0/798032909 wait complete. 2024-07-04T17:27:31.028 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'backup_fs' 2024-07-04T17:27:31.028 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:27:31.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.320+0000 7f8915400640 1 -- 172.21.15.42:0/2598470730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89100b38d0 msgr2=0x7f89100b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:31.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.320+0000 7f8915400640 1 --2- 172.21.15.42:0/2598470730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89100b38d0 0x7f89100b2d20 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8900005fd0 tx=0x7f890002a5f0 comp rx=0 tx=0).stop 2024-07-04T17:27:31.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.321+0000 7f8915400640 1 -- 172.21.15.42:0/2598470730 shutdown_connections 2024-07-04T17:27:31.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.321+0000 7f8915400640 1 --2- 172.21.15.42:0/2598470730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89100b38d0 0x7f89100b2d20 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:31.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.321+0000 7f8915400640 1 -- 172.21.15.42:0/2598470730 >> 172.21.15.42:0/2598470730 conn(0x7f89100bf700 msgr2=0x7f89100af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:31.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.321+0000 7f8915400640 1 -- 172.21.15.42:0/2598470730 shutdown_connections 2024-07-04T17:27:31.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.321+0000 7f8915400640 1 -- 172.21.15.42:0/2598470730 wait complete. 2024-07-04T17:27:31.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.323+0000 7f8915400640 1 Processor -- start 2024-07-04T17:27:31.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.324+0000 7f8915400640 1 -- start start 2024-07-04T17:27:31.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.325+0000 7f8915400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89100b01f0 0x7f89100ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:31.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.325+0000 7f8915400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f891006cf50 con 0x7f89100b01f0 2024-07-04T17:27:31.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.325+0000 7f890e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89100b01f0 0x7f89100ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:31.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.325+0000 7f890e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89100b01f0 0x7f89100ae7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36684/0 (socket says 172.21.15.42:36684) 2024-07-04T17:27:31.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.326+0000 7f890e800640 1 -- 172.21.15.42:0/4057020155 learned_addr learned my addr 172.21.15.42:0/4057020155 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:31.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.326+0000 7f890e800640 1 -- 172.21.15.42:0/4057020155 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89100b5560 con 0x7f89100b01f0 2024-07-04T17:27:31.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.326+0000 7f890e800640 1 --2- 172.21.15.42:0/4057020155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89100b01f0 0x7f89100ae7c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8900034040 tx=0x7f89000049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:31.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.327+0000 7f8906a00640 1 -- 172.21.15.42:0/4057020155 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f890003e030 con 0x7f89100b01f0 2024-07-04T17:27:31.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.327+0000 7f8906a00640 1 -- 172.21.15.42:0/4057020155 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89000035f0 con 0x7f89100b01f0 2024-07-04T17:27:31.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.327+0000 7f8906a00640 1 -- 172.21.15.42:0/4057020155 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8900035040 con 0x7f89100b01f0 2024-07-04T17:27:31.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.327+0000 7f8915400640 1 -- 172.21.15.42:0/4057020155 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89101e1430 con 0x7f89100b01f0 2024-07-04T17:27:31.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.327+0000 7f8915400640 1 -- 172.21.15.42:0/4057020155 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89101e1240 con 0x7f89100b01f0 2024-07-04T17:27:31.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.328+0000 7f8915400640 1 -- 172.21.15.42:0/4057020155 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89100aed00 con 0x7f89100b01f0 2024-07-04T17:27:31.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.328+0000 7f8906a00640 1 -- 172.21.15.42:0/4057020155 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f8900041030 con 0x7f89100b01f0 2024-07-04T17:27:31.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.329+0000 7f8906a00640 1 --2- 172.21.15.42:0/4057020155 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f88f003d2b0 0x7f88f003f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:31.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.329+0000 7f8906a00640 1 -- 172.21.15.42:0/4057020155 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4303+0+0 (secure 0 0 0) 0x7f890003f070 con 0x7f89100b01f0 2024-07-04T17:27:31.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.330+0000 7f890de00640 1 --2- 172.21.15.42:0/4057020155 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f88f003d2b0 0x7f88f003f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:31.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.330+0000 7f890de00640 1 --2- 172.21.15.42:0/4057020155 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f88f003d2b0 0x7f88f003f7c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f88fc001c60 tx=0x7f88fc007d70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:31.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.335+0000 7f8906a00640 1 -- 172.21.15.42:0/4057020155 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f8900003e10 con 0x7f89100b01f0 2024-07-04T17:27:31.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.588+0000 7f8915400640 1 -- 172.21.15.42:0/4057020155 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89100b2d20 con 0x7f89100b01f0 2024-07-04T17:27:31.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.588+0000 7f8906a00640 1 -- 172.21.15.42:0/4057020155 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+7359 (secure 0 0 0) 0x7f8900072080 con 0x7f89100b01f0 2024-07-04T17:27:31.590 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:27:31.590 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":23,"btime":"2024-07-04T17:27:30:978140+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/399707138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":399707138},{"type":"v1","addr":"172.21.15.42:6827","nonce":399707138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4414,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1510714231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1510714231},{"type":"v1","addr":"172.21.15.42:6829","nonce":1510714231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/3069680926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":3069680926},{"type":"v1","addr":"172.21.15.42:6831","nonce":3069680926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:27:07.402007+0000","modified":"2024-07-04T17:27:10.462342+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4423":{"gid":4423,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.42:6835/807364821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":807364821},{"type":"v1","addr":"172.21.15.42:6835","nonce":807364821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4423,"qdb_cluster":[4423]},"id":2},{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:27:27.513977+0000","modified":"2024-07-04T17:27:30.569774+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4422":{"gid":4422,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.42:6833/1348446958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1348446958},{"type":"v1","addr":"172.21.15.42:6833","nonce":1348446958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4422,"qdb_cluster":[4422]},"id":3}]} 2024-07-04T17:27:31.591 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 23 2024-07-04T17:27:31.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.591+0000 7f8915400640 1 -- 172.21.15.42:0/4057020155 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f88f003d2b0 msgr2=0x7f88f003f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:31.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.591+0000 7f8915400640 1 --2- 172.21.15.42:0/4057020155 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f88f003d2b0 0x7f88f003f7c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f88fc001c60 tx=0x7f88fc007d70 comp rx=0 tx=0).stop 2024-07-04T17:27:31.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.591+0000 7f8915400640 1 -- 172.21.15.42:0/4057020155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89100b01f0 msgr2=0x7f89100ae7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:31.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.591+0000 7f8915400640 1 --2- 172.21.15.42:0/4057020155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89100b01f0 0x7f89100ae7c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8900034040 tx=0x7f89000049d0 comp rx=0 tx=0).stop 2024-07-04T17:27:31.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.591+0000 7f8915400640 1 -- 172.21.15.42:0/4057020155 shutdown_connections 2024-07-04T17:27:31.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.591+0000 7f8915400640 1 --2- 172.21.15.42:0/4057020155 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f88f003d2b0 0x7f88f003f7c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:31.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.591+0000 7f8915400640 1 --2- 172.21.15.42:0/4057020155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89100b01f0 0x7f89100ae7c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:31.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.591+0000 7f8915400640 1 -- 172.21.15.42:0/4057020155 >> 172.21.15.42:0/4057020155 conn(0x7f89100bf700 msgr2=0x7f891013f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:31.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.592+0000 7f8915400640 1 -- 172.21.15.42:0/4057020155 shutdown_connections 2024-07-04T17:27:31.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.592+0000 7f8915400640 1 -- 172.21.15.42:0/4057020155 wait complete. 2024-07-04T17:27:31.620 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:27:31.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.919+0000 7fac88200640 1 -- 172.21.15.42:0/2601418684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac800b3650 msgr2=0x7fac801462f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:31.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.919+0000 7fac88200640 1 --2- 172.21.15.42:0/2601418684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac800b3650 0x7fac801462f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fac6c0097b0 tx=0x7fac6c02a4b0 comp rx=0 tx=0).stop 2024-07-04T17:27:31.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.919+0000 7fac88200640 1 -- 172.21.15.42:0/2601418684 shutdown_connections 2024-07-04T17:27:31.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.919+0000 7fac88200640 1 --2- 172.21.15.42:0/2601418684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac800b3650 0x7fac801462f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:31.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.919+0000 7fac88200640 1 -- 172.21.15.42:0/2601418684 >> 172.21.15.42:0/2601418684 conn(0x7fac800bf700 msgr2=0x7fac800af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:31.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.919+0000 7fac88200640 1 -- 172.21.15.42:0/2601418684 shutdown_connections 2024-07-04T17:27:31.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.919+0000 7fac88200640 1 -- 172.21.15.42:0/2601418684 wait complete. 2024-07-04T17:27:31.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.921+0000 7fac88200640 1 Processor -- start 2024-07-04T17:27:31.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.921+0000 7fac88200640 1 -- start start 2024-07-04T17:27:31.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.922+0000 7fac88200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac800b02b0 0x7fac801e6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:31.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.923+0000 7fac88200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fac8006c590 con 0x7fac800b02b0 2024-07-04T17:27:31.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.923+0000 7fac85800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac800b02b0 0x7fac801e6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:31.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.923+0000 7fac85800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac800b02b0 0x7fac801e6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36694/0 (socket says 172.21.15.42:36694) 2024-07-04T17:27:31.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.923+0000 7fac85800640 1 -- 172.21.15.42:0/924443179 learned_addr learned my addr 172.21.15.42:0/924443179 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:31.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.923+0000 7fac85800640 1 -- 172.21.15.42:0/924443179 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac800b8bd0 con 0x7fac800b02b0 2024-07-04T17:27:31.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.924+0000 7fac85800640 1 --2- 172.21.15.42:0/924443179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac800b02b0 0x7fac801e6b70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fac7000c6b0 tx=0x7fac7000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:31.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.924+0000 7fac7d600640 1 -- 172.21.15.42:0/924443179 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac70007e10 con 0x7fac800b02b0 2024-07-04T17:27:31.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.924+0000 7fac7d600640 1 -- 172.21.15.42:0/924443179 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac70017070 con 0x7fac800b02b0 2024-07-04T17:27:31.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.924+0000 7fac7d600640 1 -- 172.21.15.42:0/924443179 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac70006b50 con 0x7fac800b02b0 2024-07-04T17:27:31.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.924+0000 7fac88200640 1 -- 172.21.15.42:0/924443179 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac801e7210 con 0x7fac800b02b0 2024-07-04T17:27:31.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.924+0000 7fac88200640 1 -- 172.21.15.42:0/924443179 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac800b88a0 con 0x7fac800b02b0 2024-07-04T17:27:31.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.925+0000 7fac88200640 1 -- 172.21.15.42:0/924443179 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac801462f0 con 0x7fac800b02b0 2024-07-04T17:27:31.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.925+0000 7fac7d600640 1 -- 172.21.15.42:0/924443179 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fac70006cf0 con 0x7fac800b02b0 2024-07-04T17:27:31.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.926+0000 7fac7d600640 1 --2- 172.21.15.42:0/924443179 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fac6003d2b0 0x7fac6003f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:31.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.926+0000 7fac7d600640 1 -- 172.21.15.42:0/924443179 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4303+0+0 (secure 0 0 0) 0x7fac700515a0 con 0x7fac800b02b0 2024-07-04T17:27:31.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.926+0000 7fac84e00640 1 --2- 172.21.15.42:0/924443179 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fac6003d2b0 0x7fac6003f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:31.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.927+0000 7fac84e00640 1 --2- 172.21.15.42:0/924443179 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fac6003d2b0 0x7fac6003f7c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fac6c034000 tx=0x7fac6c004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:31.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:31.932+0000 7fac7d600640 1 -- 172.21.15.42:0/924443179 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fac800b06c0 con 0x7fac800b02b0 2024-07-04T17:27:32.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.146+0000 7fac88200640 1 -- 172.21.15.42:0/924443179 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac800beb10 con 0x7fac800b02b0 2024-07-04T17:27:32.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.148+0000 7fac7d600640 1 -- 172.21.15.42:0/924443179 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+14449 (secure 0 0 0) 0x7fac700072e0 con 0x7fac800b02b0 2024-07-04T17:27:32.149 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:27:32.149 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":31,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:27:28.959929+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:27:04.932839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-07-04T17:27:06.393713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.0800000429153442,"score_stable":1.0800000429153442,"optimal_score":1,"raw_score_acting":1.0800000429153442,"raw_score_stable":1.0800000429153442,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:27:25.040056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:27:26.487236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:27:32.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.150+0000 7fac88200640 1 -- 172.21.15.42:0/924443179 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fac6003d2b0 msgr2=0x7fac6003f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:32.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.150+0000 7fac88200640 1 --2- 172.21.15.42:0/924443179 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fac6003d2b0 0x7fac6003f7c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fac6c034000 tx=0x7fac6c004730 comp rx=0 tx=0).stop 2024-07-04T17:27:32.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.150+0000 7fac88200640 1 -- 172.21.15.42:0/924443179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac800b02b0 msgr2=0x7fac801e6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:32.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.150+0000 7fac88200640 1 --2- 172.21.15.42:0/924443179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac800b02b0 0x7fac801e6b70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fac7000c6b0 tx=0x7fac7000cb70 comp rx=0 tx=0).stop 2024-07-04T17:27:32.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.150+0000 7fac88200640 1 -- 172.21.15.42:0/924443179 shutdown_connections 2024-07-04T17:27:32.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.150+0000 7fac88200640 1 --2- 172.21.15.42:0/924443179 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fac6003d2b0 0x7fac6003f7c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:32.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.150+0000 7fac88200640 1 --2- 172.21.15.42:0/924443179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac800b02b0 0x7fac801e6b70 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:32.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.150+0000 7fac88200640 1 -- 172.21.15.42:0/924443179 >> 172.21.15.42:0/924443179 conn(0x7fac800bf700 msgr2=0x7fac80075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:32.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.151+0000 7fac88200640 1 -- 172.21.15.42:0/924443179 shutdown_connections 2024-07-04T17:27:32.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.151+0000 7fac88200640 1 -- 172.21.15.42:0/924443179 wait complete. 2024-07-04T17:27:32.178 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-04T17:27:32.178 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:27:32.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.456+0000 7f02e8000640 1 -- 172.21.15.42:0/3492010542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02e00b02b0 msgr2=0x7f02e00b3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:32.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.456+0000 7f02e8000640 1 --2- 172.21.15.42:0/3492010542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02e00b02b0 0x7f02e00b3650 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f02d4005fd0 tx=0x7f02d402a5f0 comp rx=0 tx=0).stop 2024-07-04T17:27:32.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.456+0000 7f02e8000640 1 -- 172.21.15.42:0/3492010542 shutdown_connections 2024-07-04T17:27:32.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.456+0000 7f02e8000640 1 --2- 172.21.15.42:0/3492010542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02e00b02b0 0x7f02e00b3650 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:32.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.456+0000 7f02e8000640 1 -- 172.21.15.42:0/3492010542 >> 172.21.15.42:0/3492010542 conn(0x7f02e00bf700 msgr2=0x7f02e00af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:32.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.457+0000 7f02e8000640 1 -- 172.21.15.42:0/3492010542 shutdown_connections 2024-07-04T17:27:32.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.457+0000 7f02e8000640 1 -- 172.21.15.42:0/3492010542 wait complete. 2024-07-04T17:27:32.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.459+0000 7f02e8000640 1 Processor -- start 2024-07-04T17:27:32.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.459+0000 7f02e8000640 1 -- start start 2024-07-04T17:27:32.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.460+0000 7f02e8000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02e00b02b0 0x7f02e00af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:32.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.460+0000 7f02e8000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f02e01463b0 con 0x7f02e00b02b0 2024-07-04T17:27:32.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.460+0000 7f02e5600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02e00b02b0 0x7f02e00af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:32.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.460+0000 7f02e5600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02e00b02b0 0x7f02e00af0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36712/0 (socket says 172.21.15.42:36712) 2024-07-04T17:27:32.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.461+0000 7f02e5600640 1 -- 172.21.15.42:0/3021778698 learned_addr learned my addr 172.21.15.42:0/3021778698 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:32.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.461+0000 7f02e5600640 1 -- 172.21.15.42:0/3021778698 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e00b5250 con 0x7f02e00b02b0 2024-07-04T17:27:32.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.461+0000 7f02e5600640 1 --2- 172.21.15.42:0/3021778698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02e00b02b0 0x7f02e00af0b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009850 tx=0x7f02d40049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:32.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.461+0000 7f02dd600640 1 -- 172.21.15.42:0/3021778698 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02d403e030 con 0x7f02e00b02b0 2024-07-04T17:27:32.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.462+0000 7f02dd600640 1 -- 172.21.15.42:0/3021778698 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02d40035f0 con 0x7f02e00b02b0 2024-07-04T17:27:32.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.462+0000 7f02dd600640 1 -- 172.21.15.42:0/3021778698 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02d4035040 con 0x7f02e00b02b0 2024-07-04T17:27:32.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.462+0000 7f02e8000640 1 -- 172.21.15.42:0/3021778698 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02e00b5d60 con 0x7f02e00b02b0 2024-07-04T17:27:32.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.462+0000 7f02e8000640 1 -- 172.21.15.42:0/3021778698 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02e00b5660 con 0x7f02e00b02b0 2024-07-04T17:27:32.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.463+0000 7f02e8000640 1 -- 172.21.15.42:0/3021778698 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02e00b3650 con 0x7f02e00b02b0 2024-07-04T17:27:32.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.463+0000 7f02dd600640 1 -- 172.21.15.42:0/3021778698 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f02d4041030 con 0x7f02e00b02b0 2024-07-04T17:27:32.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.464+0000 7f02dd600640 1 --2- 172.21.15.42:0/3021778698 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f02c003d2b0 0x7f02c003f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:32.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.464+0000 7f02dd600640 1 -- 172.21.15.42:0/3021778698 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4303+0+0 (secure 0 0 0) 0x7f02d403f070 con 0x7f02e00b02b0 2024-07-04T17:27:32.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.464+0000 7f02e4c00640 1 --2- 172.21.15.42:0/3021778698 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f02c003d2b0 0x7f02c003f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:32.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.465+0000 7f02e4c00640 1 --2- 172.21.15.42:0/3021778698 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f02c003d2b0 0x7f02c003f7c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f02cc001560 tx=0x7f02cc0079f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:32.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.469+0000 7f02dd600640 1 -- 172.21.15.42:0/3021778698 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f02e00b5660 con 0x7f02e00b02b0 2024-07-04T17:27:32.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.678+0000 7f02e8000640 1 -- 172.21.15.42:0/3021778698 --> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f02e00b6be0 con 0x7f02c003d2b0 2024-07-04T17:27:32.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.706+0000 7f02dd600640 1 -- 172.21.15.42:0/3021778698 <== mgr.4103 v2:172.21.15.42:6824/3523793950 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+482153 (secure 0 0 0) 0x7f02e00b6be0 con 0x7f02c003d2b0 2024-07-04T17:27:32.711 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:27:32.717 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:27:32.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.717+0000 7f02e8000640 1 -- 172.21.15.42:0/3021778698 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f02c003d2b0 msgr2=0x7f02c003f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:32.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.717+0000 7f02e8000640 1 --2- 172.21.15.42:0/3021778698 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f02c003d2b0 0x7f02c003f7c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f02cc001560 tx=0x7f02cc0079f0 comp rx=0 tx=0).stop 2024-07-04T17:27:32.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.717+0000 7f02e8000640 1 -- 172.21.15.42:0/3021778698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02e00b02b0 msgr2=0x7f02e00af0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:32.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.717+0000 7f02e8000640 1 --2- 172.21.15.42:0/3021778698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02e00b02b0 0x7f02e00af0b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009850 tx=0x7f02d40049d0 comp rx=0 tx=0).stop 2024-07-04T17:27:32.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.717+0000 7f02e8000640 1 -- 172.21.15.42:0/3021778698 shutdown_connections 2024-07-04T17:27:32.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.717+0000 7f02e8000640 1 --2- 172.21.15.42:0/3021778698 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f02c003d2b0 0x7f02c003f7c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:32.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.717+0000 7f02e8000640 1 --2- 172.21.15.42:0/3021778698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02e00b02b0 0x7f02e00af0b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:32.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.717+0000 7f02e8000640 1 -- 172.21.15.42:0/3021778698 >> 172.21.15.42:0/3021778698 conn(0x7f02e00bf700 msgr2=0x7f02e01455f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:32.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.718+0000 7f02e8000640 1 -- 172.21.15.42:0/3021778698 shutdown_connections 2024-07-04T17:27:32.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:32.718+0000 7f02e8000640 1 -- 172.21.15.42:0/3021778698 wait complete. 2024-07-04T17:27:32.751 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-07-04T17:27:31.286881+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596885,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":171,"num_write_kb":1137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":206,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":95572,"kb_used_data":2020,"kb_used_omap":4,"kb_used_meta":93499,"kb_avail":283019948,"statfs":{"total":289910292480,"available":289812426752,"internally_reserved":0,"allocated":2068480,"data_stored":1471921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":95743311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.358643"},"pg_stats":[{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.016871+0000","last_change":"2024-07-04T17:27:28.000192+0000","last_active":"2024-07-04T17:27:28.016871+0000","last_peered":"2024-07-04T17:27:28.016871+0000","last_clean":"2024-07-04T17:27:28.016871+0000","last_became_active":"2024-07-04T17:27:27.999880+0000","last_became_peered":"2024-07-04T17:27:27.999880+0000","last_unstale":"2024-07-04T17:27:28.016871+0000","last_undegraded":"2024-07-04T17:27:28.016871+0000","last_fullsized":"2024-07-04T17:27:28.016871+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:32:32.138712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"10'52","reported_seq":115,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.971855+0000","last_change":"2024-07-04T17:26:04.953559+0000","last_active":"2024-07-04T17:27:27.971855+0000","last_peered":"2024-07-04T17:27:27.971855+0000","last_clean":"2024-07-04T17:27:27.971855+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:27:27.971855+0000","last_undegraded":"2024-07-04T17:27:27.971855+0000","last_fullsized":"2024-07-04T17:27:27.971855+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_clean_scrub_stamp":"2024-07-04T17:26:03.933432+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.064181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.974655+0000","last_change":"2024-07-04T17:27:06.874596+0000","last_active":"2024-07-04T17:27:27.974655+0000","last_peered":"2024-07-04T17:27:27.974655+0000","last_clean":"2024-07-04T17:27:27.974655+0000","last_became_active":"2024-07-04T17:27:06.874252+0000","last_became_peered":"2024-07-04T17:27:06.874252+0000","last_unstale":"2024-07-04T17:27:27.974655+0000","last_undegraded":"2024-07-04T17:27:27.974655+0000","last_fullsized":"2024-07-04T17:27:27.974655+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:17:38.211735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.968322+0000","last_change":"2024-07-04T17:27:07.891018+0000","last_active":"2024-07-04T17:27:27.968322+0000","last_peered":"2024-07-04T17:27:27.968322+0000","last_clean":"2024-07-04T17:27:27.968322+0000","last_became_active":"2024-07-04T17:27:07.889885+0000","last_became_peered":"2024-07-04T17:27:07.889885+0000","last_unstale":"2024-07-04T17:27:27.968322+0000","last_undegraded":"2024-07-04T17:27:27.968322+0000","last_fullsized":"2024-07-04T17:27:27.968322+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:59:08.231146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"30'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.032442+0000","last_change":"2024-07-04T17:27:26.992615+0000","last_active":"2024-07-04T17:27:28.032442+0000","last_peered":"2024-07-04T17:27:28.032442+0000","last_clean":"2024-07-04T17:27:28.032442+0000","last_became_active":"2024-07-04T17:27:26.992123+0000","last_became_peered":"2024-07-04T17:27:26.992123+0000","last_unstale":"2024-07-04T17:27:28.032442+0000","last_undegraded":"2024-07-04T17:27:28.032442+0000","last_fullsized":"2024-07-04T17:27:28.032442+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:11:50.417822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.016336+0000","last_change":"2024-07-04T17:27:27.999411+0000","last_active":"2024-07-04T17:27:28.016336+0000","last_peered":"2024-07-04T17:27:28.016336+0000","last_clean":"2024-07-04T17:27:28.016336+0000","last_became_active":"2024-07-04T17:27:27.994185+0000","last_became_peered":"2024-07-04T17:27:27.994185+0000","last_unstale":"2024-07-04T17:27:28.016336+0000","last_undegraded":"2024-07-04T17:27:28.016336+0000","last_fullsized":"2024-07-04T17:27:28.016336+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:56:44.529403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.968251+0000","last_change":"2024-07-04T17:27:06.859760+0000","last_active":"2024-07-04T17:27:27.968251+0000","last_peered":"2024-07-04T17:27:27.968251+0000","last_clean":"2024-07-04T17:27:27.968251+0000","last_became_active":"2024-07-04T17:27:06.859396+0000","last_became_peered":"2024-07-04T17:27:06.859396+0000","last_unstale":"2024-07-04T17:27:27.968251+0000","last_undegraded":"2024-07-04T17:27:27.968251+0000","last_fullsized":"2024-07-04T17:27:27.968251+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:48:12.553882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.964861+0000","last_change":"2024-07-04T17:27:07.871918+0000","last_active":"2024-07-04T17:27:27.964861+0000","last_peered":"2024-07-04T17:27:27.964861+0000","last_clean":"2024-07-04T17:27:27.964861+0000","last_became_active":"2024-07-04T17:27:07.871229+0000","last_became_peered":"2024-07-04T17:27:07.871229+0000","last_unstale":"2024-07-04T17:27:27.964861+0000","last_undegraded":"2024-07-04T17:27:27.964861+0000","last_fullsized":"2024-07-04T17:27:27.964861+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:00:11.500712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.969486+0000","last_change":"2024-07-04T17:27:26.989371+0000","last_active":"2024-07-04T17:27:27.969486+0000","last_peered":"2024-07-04T17:27:27.969486+0000","last_clean":"2024-07-04T17:27:27.969486+0000","last_became_active":"2024-07-04T17:27:26.988886+0000","last_became_peered":"2024-07-04T17:27:26.988886+0000","last_unstale":"2024-07-04T17:27:27.969486+0000","last_undegraded":"2024-07-04T17:27:27.969486+0000","last_fullsized":"2024-07-04T17:27:27.969486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:59:27.331368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.026664+0000","last_change":"2024-07-04T17:27:28.003755+0000","last_active":"2024-07-04T17:27:28.026664+0000","last_peered":"2024-07-04T17:27:28.026664+0000","last_clean":"2024-07-04T17:27:28.026664+0000","last_became_active":"2024-07-04T17:27:28.002942+0000","last_became_peered":"2024-07-04T17:27:28.002942+0000","last_unstale":"2024-07-04T17:27:28.026664+0000","last_undegraded":"2024-07-04T17:27:28.026664+0000","last_fullsized":"2024-07-04T17:27:28.026664+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:53:22.051231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.971736+0000","last_change":"2024-07-04T17:27:06.883045+0000","last_active":"2024-07-04T17:27:27.971736+0000","last_peered":"2024-07-04T17:27:27.971736+0000","last_clean":"2024-07-04T17:27:27.971736+0000","last_became_active":"2024-07-04T17:27:06.882363+0000","last_became_peered":"2024-07-04T17:27:06.882363+0000","last_unstale":"2024-07-04T17:27:27.971736+0000","last_undegraded":"2024-07-04T17:27:27.971736+0000","last_fullsized":"2024-07-04T17:27:27.971736+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:05:41.513762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.967990+0000","last_change":"2024-07-04T17:27:07.909121+0000","last_active":"2024-07-04T17:27:27.967990+0000","last_peered":"2024-07-04T17:27:27.967990+0000","last_clean":"2024-07-04T17:27:27.967990+0000","last_became_active":"2024-07-04T17:27:07.908646+0000","last_became_peered":"2024-07-04T17:27:07.908646+0000","last_unstale":"2024-07-04T17:27:27.967990+0000","last_undegraded":"2024-07-04T17:27:27.967990+0000","last_fullsized":"2024-07-04T17:27:27.967990+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:34:46.000005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.991395+0000","last_change":"2024-07-04T17:27:26.992253+0000","last_active":"2024-07-04T17:27:27.991395+0000","last_peered":"2024-07-04T17:27:27.991395+0000","last_clean":"2024-07-04T17:27:27.991395+0000","last_became_active":"2024-07-04T17:27:26.991917+0000","last_became_peered":"2024-07-04T17:27:26.991917+0000","last_unstale":"2024-07-04T17:27:27.991395+0000","last_undegraded":"2024-07-04T17:27:27.991395+0000","last_fullsized":"2024-07-04T17:27:27.991395+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:08:43.921506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.017355+0000","last_change":"2024-07-04T17:27:28.002245+0000","last_active":"2024-07-04T17:27:28.017355+0000","last_peered":"2024-07-04T17:27:28.017355+0000","last_clean":"2024-07-04T17:27:28.017355+0000","last_became_active":"2024-07-04T17:27:28.000808+0000","last_became_peered":"2024-07-04T17:27:28.000808+0000","last_unstale":"2024-07-04T17:27:28.017355+0000","last_undegraded":"2024-07-04T17:27:28.017355+0000","last_fullsized":"2024-07-04T17:27:28.017355+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:24:20.430216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.966797+0000","last_change":"2024-07-04T17:27:06.861673+0000","last_active":"2024-07-04T17:27:27.966797+0000","last_peered":"2024-07-04T17:27:27.966797+0000","last_clean":"2024-07-04T17:27:27.966797+0000","last_became_active":"2024-07-04T17:27:06.861297+0000","last_became_peered":"2024-07-04T17:27:06.861297+0000","last_unstale":"2024-07-04T17:27:27.966797+0000","last_undegraded":"2024-07-04T17:27:27.966797+0000","last_fullsized":"2024-07-04T17:27:27.966797+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:10:34.165745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.982964+0000","last_change":"2024-07-04T17:27:07.882615+0000","last_active":"2024-07-04T17:27:27.982964+0000","last_peered":"2024-07-04T17:27:27.982964+0000","last_clean":"2024-07-04T17:27:27.982964+0000","last_became_active":"2024-07-04T17:27:07.882132+0000","last_became_peered":"2024-07-04T17:27:07.882132+0000","last_unstale":"2024-07-04T17:27:27.982964+0000","last_undegraded":"2024-07-04T17:27:27.982964+0000","last_fullsized":"2024-07-04T17:27:27.982964+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:45:52.813695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.958026+0000","last_change":"2024-07-04T17:27:26.986473+0000","last_active":"2024-07-04T17:27:27.958026+0000","last_peered":"2024-07-04T17:27:27.958026+0000","last_clean":"2024-07-04T17:27:27.958026+0000","last_became_active":"2024-07-04T17:27:26.986137+0000","last_became_peered":"2024-07-04T17:27:26.986137+0000","last_unstale":"2024-07-04T17:27:27.958026+0000","last_undegraded":"2024-07-04T17:27:27.958026+0000","last_fullsized":"2024-07-04T17:27:27.958026+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:31:33.657276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.998766+0000","last_change":"2024-07-04T17:27:27.984855+0000","last_active":"2024-07-04T17:27:27.998766+0000","last_peered":"2024-07-04T17:27:27.998766+0000","last_clean":"2024-07-04T17:27:27.998766+0000","last_became_active":"2024-07-04T17:27:27.984390+0000","last_became_peered":"2024-07-04T17:27:27.984390+0000","last_unstale":"2024-07-04T17:27:27.998766+0000","last_undegraded":"2024-07-04T17:27:27.998766+0000","last_fullsized":"2024-07-04T17:27:27.998766+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:53:07.282774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.982594+0000","last_change":"2024-07-04T17:27:06.870890+0000","last_active":"2024-07-04T17:27:27.982594+0000","last_peered":"2024-07-04T17:27:27.982594+0000","last_clean":"2024-07-04T17:27:27.982594+0000","last_became_active":"2024-07-04T17:27:06.870581+0000","last_became_peered":"2024-07-04T17:27:06.870581+0000","last_unstale":"2024-07-04T17:27:27.982594+0000","last_undegraded":"2024-07-04T17:27:27.982594+0000","last_fullsized":"2024-07-04T17:27:27.982594+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:55:15.415040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.957418+0000","last_change":"2024-07-04T17:27:07.878014+0000","last_active":"2024-07-04T17:27:27.957418+0000","last_peered":"2024-07-04T17:27:27.957418+0000","last_clean":"2024-07-04T17:27:27.957418+0000","last_became_active":"2024-07-04T17:27:07.874485+0000","last_became_peered":"2024-07-04T17:27:07.874485+0000","last_unstale":"2024-07-04T17:27:27.957418+0000","last_undegraded":"2024-07-04T17:27:27.957418+0000","last_fullsized":"2024-07-04T17:27:27.957418+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:55:10.737185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.967712+0000","last_change":"2024-07-04T17:27:26.983805+0000","last_active":"2024-07-04T17:27:27.967712+0000","last_peered":"2024-07-04T17:27:27.967712+0000","last_clean":"2024-07-04T17:27:27.967712+0000","last_became_active":"2024-07-04T17:27:26.982536+0000","last_became_peered":"2024-07-04T17:27:26.982536+0000","last_unstale":"2024-07-04T17:27:27.967712+0000","last_undegraded":"2024-07-04T17:27:27.967712+0000","last_fullsized":"2024-07-04T17:27:27.967712+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:44:50.168355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.000829+0000","last_change":"2024-07-04T17:27:27.992927+0000","last_active":"2024-07-04T17:27:28.000829+0000","last_peered":"2024-07-04T17:27:28.000829+0000","last_clean":"2024-07-04T17:27:28.000829+0000","last_became_active":"2024-07-04T17:27:27.992276+0000","last_became_peered":"2024-07-04T17:27:27.992276+0000","last_unstale":"2024-07-04T17:27:28.000829+0000","last_undegraded":"2024-07-04T17:27:28.000829+0000","last_fullsized":"2024-07-04T17:27:28.000829+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:25:29.884373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.969998+0000","last_change":"2024-07-04T17:27:06.872489+0000","last_active":"2024-07-04T17:27:27.969998+0000","last_peered":"2024-07-04T17:27:27.969998+0000","last_clean":"2024-07-04T17:27:27.969998+0000","last_became_active":"2024-07-04T17:27:06.872125+0000","last_became_peered":"2024-07-04T17:27:06.872125+0000","last_unstale":"2024-07-04T17:27:27.969998+0000","last_undegraded":"2024-07-04T17:27:27.969998+0000","last_fullsized":"2024-07-04T17:27:27.969998+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:10:29.469978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.974496+0000","last_change":"2024-07-04T17:27:07.883459+0000","last_active":"2024-07-04T17:27:27.974496+0000","last_peered":"2024-07-04T17:27:27.974496+0000","last_clean":"2024-07-04T17:27:27.974496+0000","last_became_active":"2024-07-04T17:27:07.883202+0000","last_became_peered":"2024-07-04T17:27:07.883202+0000","last_unstale":"2024-07-04T17:27:27.974496+0000","last_undegraded":"2024-07-04T17:27:27.974496+0000","last_fullsized":"2024-07-04T17:27:27.974496+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:29:49.821609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.976708+0000","last_change":"2024-07-04T17:27:26.982200+0000","last_active":"2024-07-04T17:27:27.976708+0000","last_peered":"2024-07-04T17:27:27.976708+0000","last_clean":"2024-07-04T17:27:27.976708+0000","last_became_active":"2024-07-04T17:27:26.981617+0000","last_became_peered":"2024-07-04T17:27:26.981617+0000","last_unstale":"2024-07-04T17:27:27.976708+0000","last_undegraded":"2024-07-04T17:27:27.976708+0000","last_fullsized":"2024-07-04T17:27:27.976708+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:48:22.086567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.031197+0000","last_change":"2024-07-04T17:27:28.026525+0000","last_active":"2024-07-04T17:27:28.031197+0000","last_peered":"2024-07-04T17:27:28.031197+0000","last_clean":"2024-07-04T17:27:28.031197+0000","last_became_active":"2024-07-04T17:27:28.026192+0000","last_became_peered":"2024-07-04T17:27:28.026192+0000","last_unstale":"2024-07-04T17:27:28.031197+0000","last_undegraded":"2024-07-04T17:27:28.031197+0000","last_fullsized":"2024-07-04T17:27:28.031197+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:44:15.882179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"27'1","reported_seq":21,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.959431+0000","last_change":"2024-07-04T17:27:06.866198+0000","last_active":"2024-07-04T17:27:27.959431+0000","last_peered":"2024-07-04T17:27:27.959431+0000","last_clean":"2024-07-04T17:27:27.959431+0000","last_became_active":"2024-07-04T17:27:06.865891+0000","last_became_peered":"2024-07-04T17:27:06.865891+0000","last_unstale":"2024-07-04T17:27:27.959431+0000","last_undegraded":"2024-07-04T17:27:27.959431+0000","last_fullsized":"2024-07-04T17:27:27.959431+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:59:47.703820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.978168+0000","last_change":"2024-07-04T17:27:07.882402+0000","last_active":"2024-07-04T17:27:27.978168+0000","last_peered":"2024-07-04T17:27:27.978168+0000","last_clean":"2024-07-04T17:27:27.978168+0000","last_became_active":"2024-07-04T17:27:07.882061+0000","last_became_peered":"2024-07-04T17:27:07.882061+0000","last_unstale":"2024-07-04T17:27:27.978168+0000","last_undegraded":"2024-07-04T17:27:27.978168+0000","last_fullsized":"2024-07-04T17:27:27.978168+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:49:59.324212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.971233+0000","last_change":"2024-07-04T17:27:26.986939+0000","last_active":"2024-07-04T17:27:27.971233+0000","last_peered":"2024-07-04T17:27:27.971233+0000","last_clean":"2024-07-04T17:27:27.971233+0000","last_became_active":"2024-07-04T17:27:26.986582+0000","last_became_peered":"2024-07-04T17:27:26.986582+0000","last_unstale":"2024-07-04T17:27:27.971233+0000","last_undegraded":"2024-07-04T17:27:27.971233+0000","last_fullsized":"2024-07-04T17:27:27.971233+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:55:06.278743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.015152+0000","last_change":"2024-07-04T17:27:27.998065+0000","last_active":"2024-07-04T17:27:28.015152+0000","last_peered":"2024-07-04T17:27:28.015152+0000","last_clean":"2024-07-04T17:27:28.015152+0000","last_became_active":"2024-07-04T17:27:27.997759+0000","last_became_peered":"2024-07-04T17:27:27.997759+0000","last_unstale":"2024-07-04T17:27:28.015152+0000","last_undegraded":"2024-07-04T17:27:28.015152+0000","last_fullsized":"2024-07-04T17:27:28.015152+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:06:51.782470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"26'2","reported_seq":22,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.976250+0000","last_change":"2024-07-04T17:27:06.865834+0000","last_active":"2024-07-04T17:27:27.976250+0000","last_peered":"2024-07-04T17:27:27.976250+0000","last_clean":"2024-07-04T17:27:27.976250+0000","last_became_active":"2024-07-04T17:27:06.865463+0000","last_became_peered":"2024-07-04T17:27:06.865463+0000","last_unstale":"2024-07-04T17:27:27.976250+0000","last_undegraded":"2024-07-04T17:27:27.976250+0000","last_fullsized":"2024-07-04T17:27:27.976250+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:35:43.457323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.982344+0000","last_change":"2024-07-04T17:27:07.911944+0000","last_active":"2024-07-04T17:27:27.982344+0000","last_peered":"2024-07-04T17:27:27.982344+0000","last_clean":"2024-07-04T17:27:27.982344+0000","last_became_active":"2024-07-04T17:27:07.911639+0000","last_became_peered":"2024-07-04T17:27:07.911639+0000","last_unstale":"2024-07-04T17:27:27.982344+0000","last_undegraded":"2024-07-04T17:27:27.982344+0000","last_fullsized":"2024-07-04T17:27:27.982344+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:56:08.436905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.975139+0000","last_change":"2024-07-04T17:27:26.988445+0000","last_active":"2024-07-04T17:27:27.975139+0000","last_peered":"2024-07-04T17:27:27.975139+0000","last_clean":"2024-07-04T17:27:27.975139+0000","last_became_active":"2024-07-04T17:27:26.986965+0000","last_became_peered":"2024-07-04T17:27:26.986965+0000","last_unstale":"2024-07-04T17:27:27.975139+0000","last_undegraded":"2024-07-04T17:27:27.975139+0000","last_fullsized":"2024-07-04T17:27:27.975139+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:37:50.824328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.999020+0000","last_change":"2024-07-04T17:27:27.983773+0000","last_active":"2024-07-04T17:27:27.999020+0000","last_peered":"2024-07-04T17:27:27.999020+0000","last_clean":"2024-07-04T17:27:27.999020+0000","last_became_active":"2024-07-04T17:27:27.983392+0000","last_became_peered":"2024-07-04T17:27:27.983392+0000","last_unstale":"2024-07-04T17:27:27.999020+0000","last_undegraded":"2024-07-04T17:27:27.999020+0000","last_fullsized":"2024-07-04T17:27:27.999020+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:47:31.123344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.971657+0000","last_change":"2024-07-04T17:27:06.871847+0000","last_active":"2024-07-04T17:27:27.971657+0000","last_peered":"2024-07-04T17:27:27.971657+0000","last_clean":"2024-07-04T17:27:27.971657+0000","last_became_active":"2024-07-04T17:27:06.871475+0000","last_became_peered":"2024-07-04T17:27:06.871475+0000","last_unstale":"2024-07-04T17:27:27.971657+0000","last_undegraded":"2024-07-04T17:27:27.971657+0000","last_fullsized":"2024-07-04T17:27:27.971657+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:09:28.414254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.965007+0000","last_change":"2024-07-04T17:27:07.872007+0000","last_active":"2024-07-04T17:27:27.965007+0000","last_peered":"2024-07-04T17:27:27.965007+0000","last_clean":"2024-07-04T17:27:27.965007+0000","last_became_active":"2024-07-04T17:27:07.871647+0000","last_became_peered":"2024-07-04T17:27:07.871647+0000","last_unstale":"2024-07-04T17:27:27.965007+0000","last_undegraded":"2024-07-04T17:27:27.965007+0000","last_fullsized":"2024-07-04T17:27:27.965007+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:17:48.892420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.982841+0000","last_change":"2024-07-04T17:27:26.985765+0000","last_active":"2024-07-04T17:27:27.982841+0000","last_peered":"2024-07-04T17:27:27.982841+0000","last_clean":"2024-07-04T17:27:27.982841+0000","last_became_active":"2024-07-04T17:27:26.984981+0000","last_became_peered":"2024-07-04T17:27:26.984981+0000","last_unstale":"2024-07-04T17:27:27.982841+0000","last_undegraded":"2024-07-04T17:27:27.982841+0000","last_fullsized":"2024-07-04T17:27:27.982841+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:45:58.822013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.031124+0000","last_change":"2024-07-04T17:27:28.022666+0000","last_active":"2024-07-04T17:27:28.031124+0000","last_peered":"2024-07-04T17:27:28.031124+0000","last_clean":"2024-07-04T17:27:28.031124+0000","last_became_active":"2024-07-04T17:27:28.021315+0000","last_became_peered":"2024-07-04T17:27:28.021315+0000","last_unstale":"2024-07-04T17:27:28.031124+0000","last_undegraded":"2024-07-04T17:27:28.031124+0000","last_fullsized":"2024-07-04T17:27:28.031124+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:37:40.507481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.964714+0000","last_change":"2024-07-04T17:27:06.868310+0000","last_active":"2024-07-04T17:27:27.964714+0000","last_peered":"2024-07-04T17:27:27.964714+0000","last_clean":"2024-07-04T17:27:27.964714+0000","last_became_active":"2024-07-04T17:27:06.866928+0000","last_became_peered":"2024-07-04T17:27:06.866928+0000","last_unstale":"2024-07-04T17:27:27.964714+0000","last_undegraded":"2024-07-04T17:27:27.964714+0000","last_fullsized":"2024-07-04T17:27:27.964714+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:31:45.030996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.968135+0000","last_change":"2024-07-04T17:27:07.905869+0000","last_active":"2024-07-04T17:27:27.968135+0000","last_peered":"2024-07-04T17:27:27.968135+0000","last_clean":"2024-07-04T17:27:27.968135+0000","last_became_active":"2024-07-04T17:27:07.905559+0000","last_became_peered":"2024-07-04T17:27:07.905559+0000","last_unstale":"2024-07-04T17:27:27.968135+0000","last_undegraded":"2024-07-04T17:27:27.968135+0000","last_fullsized":"2024-07-04T17:27:27.968135+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:00:11.500712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"30'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.014114+0000","last_change":"2024-07-04T17:27:26.990435+0000","last_active":"2024-07-04T17:27:28.014114+0000","last_peered":"2024-07-04T17:27:28.014114+0000","last_clean":"2024-07-04T17:27:28.014114+0000","last_became_active":"2024-07-04T17:27:26.989929+0000","last_became_peered":"2024-07-04T17:27:26.989929+0000","last_unstale":"2024-07-04T17:27:28.014114+0000","last_undegraded":"2024-07-04T17:27:28.014114+0000","last_fullsized":"2024-07-04T17:27:28.014114+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:55:04.441377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.020983+0000","last_change":"2024-07-04T17:27:28.013631+0000","last_active":"2024-07-04T17:27:28.020983+0000","last_peered":"2024-07-04T17:27:28.020983+0000","last_clean":"2024-07-04T17:27:28.020983+0000","last_became_active":"2024-07-04T17:27:28.011264+0000","last_became_peered":"2024-07-04T17:27:28.011264+0000","last_unstale":"2024-07-04T17:27:28.020983+0000","last_undegraded":"2024-07-04T17:27:28.020983+0000","last_fullsized":"2024-07-04T17:27:28.020983+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:41:35.479418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.971470+0000","last_change":"2024-07-04T17:27:06.885598+0000","last_active":"2024-07-04T17:27:27.971470+0000","last_peered":"2024-07-04T17:27:27.971470+0000","last_clean":"2024-07-04T17:27:27.971470+0000","last_became_active":"2024-07-04T17:27:06.885220+0000","last_became_peered":"2024-07-04T17:27:06.885220+0000","last_unstale":"2024-07-04T17:27:27.971470+0000","last_undegraded":"2024-07-04T17:27:27.971470+0000","last_fullsized":"2024-07-04T17:27:27.971470+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:49:35.335628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.977849+0000","last_change":"2024-07-04T17:27:07.904396+0000","last_active":"2024-07-04T17:27:27.977849+0000","last_peered":"2024-07-04T17:27:27.977849+0000","last_clean":"2024-07-04T17:27:27.977849+0000","last_became_active":"2024-07-04T17:27:07.903961+0000","last_became_peered":"2024-07-04T17:27:07.903961+0000","last_unstale":"2024-07-04T17:27:27.977849+0000","last_undegraded":"2024-07-04T17:27:27.977849+0000","last_fullsized":"2024-07-04T17:27:27.977849+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:59:48.713747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.969758+0000","last_change":"2024-07-04T17:27:26.980557+0000","last_active":"2024-07-04T17:27:27.969758+0000","last_peered":"2024-07-04T17:27:27.969758+0000","last_clean":"2024-07-04T17:27:27.969758+0000","last_became_active":"2024-07-04T17:27:26.979421+0000","last_became_peered":"2024-07-04T17:27:26.979421+0000","last_unstale":"2024-07-04T17:27:27.969758+0000","last_undegraded":"2024-07-04T17:27:27.969758+0000","last_fullsized":"2024-07-04T17:27:27.969758+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:53:58.600996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.003470+0000","last_change":"2024-07-04T17:27:27.998798+0000","last_active":"2024-07-04T17:27:28.003470+0000","last_peered":"2024-07-04T17:27:28.003470+0000","last_clean":"2024-07-04T17:27:28.003470+0000","last_became_active":"2024-07-04T17:27:27.998444+0000","last_became_peered":"2024-07-04T17:27:27.998444+0000","last_unstale":"2024-07-04T17:27:28.003470+0000","last_undegraded":"2024-07-04T17:27:28.003470+0000","last_fullsized":"2024-07-04T17:27:28.003470+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:20:48.469282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"26'2","reported_seq":22,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.982842+0000","last_change":"2024-07-04T17:27:06.859869+0000","last_active":"2024-07-04T17:27:27.982842+0000","last_peered":"2024-07-04T17:27:27.982842+0000","last_clean":"2024-07-04T17:27:27.982842+0000","last_became_active":"2024-07-04T17:27:06.859475+0000","last_became_peered":"2024-07-04T17:27:06.859475+0000","last_unstale":"2024-07-04T17:27:27.982842+0000","last_undegraded":"2024-07-04T17:27:27.982842+0000","last_fullsized":"2024-07-04T17:27:27.982842+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:00:10.490785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.974216+0000","last_change":"2024-07-04T17:27:07.905531+0000","last_active":"2024-07-04T17:27:27.974216+0000","last_peered":"2024-07-04T17:27:27.974216+0000","last_clean":"2024-07-04T17:27:27.974216+0000","last_became_active":"2024-07-04T17:27:07.903880+0000","last_became_peered":"2024-07-04T17:27:07.903880+0000","last_unstale":"2024-07-04T17:27:27.974216+0000","last_undegraded":"2024-07-04T17:27:27.974216+0000","last_fullsized":"2024-07-04T17:27:27.974216+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:21:38.559811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.968059+0000","last_change":"2024-07-04T17:27:26.993149+0000","last_active":"2024-07-04T17:27:27.968059+0000","last_peered":"2024-07-04T17:27:27.968059+0000","last_clean":"2024-07-04T17:27:27.968059+0000","last_became_active":"2024-07-04T17:27:26.987822+0000","last_became_peered":"2024-07-04T17:27:26.987822+0000","last_unstale":"2024-07-04T17:27:27.968059+0000","last_undegraded":"2024-07-04T17:27:27.968059+0000","last_fullsized":"2024-07-04T17:27:27.968059+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:54:26.165578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.016651+0000","last_change":"2024-07-04T17:27:28.010844+0000","last_active":"2024-07-04T17:27:28.016651+0000","last_peered":"2024-07-04T17:27:28.016651+0000","last_clean":"2024-07-04T17:27:28.016651+0000","last_became_active":"2024-07-04T17:27:28.010550+0000","last_became_peered":"2024-07-04T17:27:28.010550+0000","last_unstale":"2024-07-04T17:27:28.016651+0000","last_undegraded":"2024-07-04T17:27:28.016651+0000","last_fullsized":"2024-07-04T17:27:28.016651+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:15:30.121052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.958711+0000","last_change":"2024-07-04T17:27:06.859888+0000","last_active":"2024-07-04T17:27:27.958711+0000","last_peered":"2024-07-04T17:27:27.958711+0000","last_clean":"2024-07-04T17:27:27.958711+0000","last_became_active":"2024-07-04T17:27:06.859565+0000","last_became_peered":"2024-07-04T17:27:06.859565+0000","last_unstale":"2024-07-04T17:27:27.958711+0000","last_undegraded":"2024-07-04T17:27:27.958711+0000","last_fullsized":"2024-07-04T17:27:27.958711+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:25:51.634073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.980198+0000","last_change":"2024-07-04T17:27:07.912224+0000","last_active":"2024-07-04T17:27:27.980198+0000","last_peered":"2024-07-04T17:27:27.980198+0000","last_clean":"2024-07-04T17:27:27.980198+0000","last_became_active":"2024-07-04T17:27:07.911921+0000","last_became_peered":"2024-07-04T17:27:07.911921+0000","last_unstale":"2024-07-04T17:27:27.980198+0000","last_undegraded":"2024-07-04T17:27:27.980198+0000","last_fullsized":"2024-07-04T17:27:27.980198+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:12:04.631448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"30'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.020350+0000","last_change":"2024-07-04T17:27:26.981903+0000","last_active":"2024-07-04T17:27:28.020350+0000","last_peered":"2024-07-04T17:27:28.020350+0000","last_clean":"2024-07-04T17:27:28.020350+0000","last_became_active":"2024-07-04T17:27:26.981211+0000","last_became_peered":"2024-07-04T17:27:26.981211+0000","last_unstale":"2024-07-04T17:27:28.020350+0000","last_undegraded":"2024-07-04T17:27:28.020350+0000","last_fullsized":"2024-07-04T17:27:28.020350+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:32.422119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.026775+0000","last_change":"2024-07-04T17:27:28.007392+0000","last_active":"2024-07-04T17:27:28.026775+0000","last_peered":"2024-07-04T17:27:28.026775+0000","last_clean":"2024-07-04T17:27:28.026775+0000","last_became_active":"2024-07-04T17:27:28.003205+0000","last_became_peered":"2024-07-04T17:27:28.003205+0000","last_unstale":"2024-07-04T17:27:28.026775+0000","last_undegraded":"2024-07-04T17:27:28.026775+0000","last_fullsized":"2024-07-04T17:27:28.026775+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:18:35.514937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"26'1","reported_seq":21,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.974785+0000","last_change":"2024-07-04T17:27:06.877855+0000","last_active":"2024-07-04T17:27:27.974785+0000","last_peered":"2024-07-04T17:27:27.974785+0000","last_clean":"2024-07-04T17:27:27.974785+0000","last_became_active":"2024-07-04T17:27:06.876852+0000","last_became_peered":"2024-07-04T17:27:06.876852+0000","last_unstale":"2024-07-04T17:27:27.974785+0000","last_undegraded":"2024-07-04T17:27:27.974785+0000","last_fullsized":"2024-07-04T17:27:27.974785+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:19:43.481377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.970474+0000","last_change":"2024-07-04T17:27:07.904773+0000","last_active":"2024-07-04T17:27:27.970474+0000","last_peered":"2024-07-04T17:27:27.970474+0000","last_clean":"2024-07-04T17:27:27.970474+0000","last_became_active":"2024-07-04T17:27:07.900622+0000","last_became_peered":"2024-07-04T17:27:07.900622+0000","last_unstale":"2024-07-04T17:27:27.970474+0000","last_undegraded":"2024-07-04T17:27:27.970474+0000","last_fullsized":"2024-07-04T17:27:27.970474+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:21:38.559811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"30'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.017767+0000","last_change":"2024-07-04T17:27:26.966387+0000","last_active":"2024-07-04T17:27:28.017767+0000","last_peered":"2024-07-04T17:27:28.017767+0000","last_clean":"2024-07-04T17:27:28.017767+0000","last_became_active":"2024-07-04T17:27:26.966045+0000","last_became_peered":"2024-07-04T17:27:26.966045+0000","last_unstale":"2024-07-04T17:27:28.017767+0000","last_undegraded":"2024-07-04T17:27:28.017767+0000","last_fullsized":"2024-07-04T17:27:28.017767+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:51:48.147252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.026321+0000","last_change":"2024-07-04T17:27:28.015465+0000","last_active":"2024-07-04T17:27:28.026321+0000","last_peered":"2024-07-04T17:27:28.026321+0000","last_clean":"2024-07-04T17:27:28.026321+0000","last_became_active":"2024-07-04T17:27:28.013215+0000","last_became_peered":"2024-07-04T17:27:28.013215+0000","last_unstale":"2024-07-04T17:27:28.026321+0000","last_undegraded":"2024-07-04T17:27:28.026321+0000","last_fullsized":"2024-07-04T17:27:28.026321+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:25:12.049628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"26'2","reported_seq":22,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.980551+0000","last_change":"2024-07-04T17:27:06.885939+0000","last_active":"2024-07-04T17:27:27.980551+0000","last_peered":"2024-07-04T17:27:27.980551+0000","last_clean":"2024-07-04T17:27:27.980551+0000","last_became_active":"2024-07-04T17:27:06.885638+0000","last_became_peered":"2024-07-04T17:27:06.885638+0000","last_unstale":"2024-07-04T17:27:27.980551+0000","last_undegraded":"2024-07-04T17:27:27.980551+0000","last_fullsized":"2024-07-04T17:27:27.980551+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:31:45.030996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.977299+0000","last_change":"2024-07-04T17:27:07.899041+0000","last_active":"2024-07-04T17:27:27.977299+0000","last_peered":"2024-07-04T17:27:27.977299+0000","last_clean":"2024-07-04T17:27:27.977299+0000","last_became_active":"2024-07-04T17:27:07.895875+0000","last_became_peered":"2024-07-04T17:27:07.895875+0000","last_unstale":"2024-07-04T17:27:27.977299+0000","last_undegraded":"2024-07-04T17:27:27.977299+0000","last_fullsized":"2024-07-04T17:27:27.977299+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:34:35.587534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.970840+0000","last_change":"2024-07-04T17:27:26.981319+0000","last_active":"2024-07-04T17:27:27.970840+0000","last_peered":"2024-07-04T17:27:27.970840+0000","last_clean":"2024-07-04T17:27:27.970840+0000","last_became_active":"2024-07-04T17:27:26.980593+0000","last_became_peered":"2024-07-04T17:27:26.980593+0000","last_unstale":"2024-07-04T17:27:27.970840+0000","last_undegraded":"2024-07-04T17:27:27.970840+0000","last_fullsized":"2024-07-04T17:27:27.970840+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:11:05.732872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.030472+0000","last_change":"2024-07-04T17:27:28.010746+0000","last_active":"2024-07-04T17:27:28.030472+0000","last_peered":"2024-07-04T17:27:28.030472+0000","last_clean":"2024-07-04T17:27:28.030472+0000","last_became_active":"2024-07-04T17:27:28.009340+0000","last_became_peered":"2024-07-04T17:27:28.009340+0000","last_unstale":"2024-07-04T17:27:28.030472+0000","last_undegraded":"2024-07-04T17:27:28.030472+0000","last_fullsized":"2024-07-04T17:27:28.030472+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:17:51.739014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"26'1","reported_seq":21,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.976403+0000","last_change":"2024-07-04T17:27:06.877349+0000","last_active":"2024-07-04T17:27:27.976403+0000","last_peered":"2024-07-04T17:27:27.976403+0000","last_clean":"2024-07-04T17:27:27.976403+0000","last_became_active":"2024-07-04T17:27:06.876814+0000","last_became_peered":"2024-07-04T17:27:06.876814+0000","last_unstale":"2024-07-04T17:27:27.976403+0000","last_undegraded":"2024-07-04T17:27:27.976403+0000","last_fullsized":"2024-07-04T17:27:27.976403+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:52:15.359670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.982195+0000","last_change":"2024-07-04T17:27:07.915949+0000","last_active":"2024-07-04T17:27:27.982195+0000","last_peered":"2024-07-04T17:27:27.982195+0000","last_clean":"2024-07-04T17:27:27.982195+0000","last_became_active":"2024-07-04T17:27:07.915603+0000","last_became_peered":"2024-07-04T17:27:07.915603+0000","last_unstale":"2024-07-04T17:27:27.982195+0000","last_undegraded":"2024-07-04T17:27:27.982195+0000","last_fullsized":"2024-07-04T17:27:27.982195+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:27:08.653236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"30'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.013749+0000","last_change":"2024-07-04T17:27:26.988800+0000","last_active":"2024-07-04T17:27:28.013749+0000","last_peered":"2024-07-04T17:27:28.013749+0000","last_clean":"2024-07-04T17:27:28.013749+0000","last_became_active":"2024-07-04T17:27:26.988563+0000","last_became_peered":"2024-07-04T17:27:26.988563+0000","last_unstale":"2024-07-04T17:27:28.013749+0000","last_undegraded":"2024-07-04T17:27:28.013749+0000","last_fullsized":"2024-07-04T17:27:28.013749+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:18:34.504516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.034456+0000","last_change":"2024-07-04T17:27:28.032546+0000","last_active":"2024-07-04T17:27:28.034456+0000","last_peered":"2024-07-04T17:27:28.034456+0000","last_clean":"2024-07-04T17:27:28.034456+0000","last_became_active":"2024-07-04T17:27:28.032101+0000","last_became_peered":"2024-07-04T17:27:28.032101+0000","last_unstale":"2024-07-04T17:27:28.034456+0000","last_undegraded":"2024-07-04T17:27:28.034456+0000","last_fullsized":"2024-07-04T17:27:28.034456+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:47:32.101334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"26'8","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.972064+0000","last_change":"2024-07-04T17:27:06.865126+0000","last_active":"2024-07-04T17:27:27.972064+0000","last_peered":"2024-07-04T17:27:27.972064+0000","last_clean":"2024-07-04T17:27:27.972064+0000","last_became_active":"2024-07-04T17:27:06.864781+0000","last_became_peered":"2024-07-04T17:27:06.864781+0000","last_unstale":"2024-07-04T17:27:27.972064+0000","last_undegraded":"2024-07-04T17:27:27.972064+0000","last_fullsized":"2024-07-04T17:27:27.972064+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:44:59.048384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.967458+0000","last_change":"2024-07-04T17:27:07.886267+0000","last_active":"2024-07-04T17:27:27.967458+0000","last_peered":"2024-07-04T17:27:27.967458+0000","last_clean":"2024-07-04T17:27:27.967458+0000","last_became_active":"2024-07-04T17:27:07.885526+0000","last_became_peered":"2024-07-04T17:27:07.885526+0000","last_unstale":"2024-07-04T17:27:27.967458+0000","last_undegraded":"2024-07-04T17:27:27.967458+0000","last_fullsized":"2024-07-04T17:27:27.967458+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:42:03.682646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.958076+0000","last_change":"2024-07-04T17:27:26.992067+0000","last_active":"2024-07-04T17:27:27.958076+0000","last_peered":"2024-07-04T17:27:27.958076+0000","last_clean":"2024-07-04T17:27:27.958076+0000","last_became_active":"2024-07-04T17:27:26.985396+0000","last_became_peered":"2024-07-04T17:27:26.985396+0000","last_unstale":"2024-07-04T17:27:27.958076+0000","last_undegraded":"2024-07-04T17:27:27.958076+0000","last_fullsized":"2024-07-04T17:27:27.958076+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:49:35.175636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.021963+0000","last_change":"2024-07-04T17:27:28.010519+0000","last_active":"2024-07-04T17:27:28.021963+0000","last_peered":"2024-07-04T17:27:28.021963+0000","last_clean":"2024-07-04T17:27:28.021963+0000","last_became_active":"2024-07-04T17:27:28.010200+0000","last_became_peered":"2024-07-04T17:27:28.010200+0000","last_unstale":"2024-07-04T17:27:28.021963+0000","last_undegraded":"2024-07-04T17:27:28.021963+0000","last_fullsized":"2024-07-04T17:27:28.021963+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:37:40.507481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.971449+0000","last_change":"2024-07-04T17:27:06.883046+0000","last_active":"2024-07-04T17:27:27.971449+0000","last_peered":"2024-07-04T17:27:27.971449+0000","last_clean":"2024-07-04T17:27:27.971449+0000","last_became_active":"2024-07-04T17:27:06.881811+0000","last_became_peered":"2024-07-04T17:27:06.881811+0000","last_unstale":"2024-07-04T17:27:27.971449+0000","last_undegraded":"2024-07-04T17:27:27.971449+0000","last_fullsized":"2024-07-04T17:27:27.971449+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:40:58.357800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.968665+0000","last_change":"2024-07-04T17:27:07.909034+0000","last_active":"2024-07-04T17:27:27.968665+0000","last_peered":"2024-07-04T17:27:27.968665+0000","last_clean":"2024-07-04T17:27:27.968665+0000","last_became_active":"2024-07-04T17:27:07.908392+0000","last_became_peered":"2024-07-04T17:27:07.908392+0000","last_unstale":"2024-07-04T17:27:27.968665+0000","last_undegraded":"2024-07-04T17:27:27.968665+0000","last_fullsized":"2024-07-04T17:27:27.968665+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:09:16.307994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"30'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.018119+0000","last_change":"2024-07-04T17:27:26.982698+0000","last_active":"2024-07-04T17:27:28.018119+0000","last_peered":"2024-07-04T17:27:28.018119+0000","last_clean":"2024-07-04T17:27:28.018119+0000","last_became_active":"2024-07-04T17:27:26.982363+0000","last_became_peered":"2024-07-04T17:27:26.982363+0000","last_unstale":"2024-07-04T17:27:28.018119+0000","last_undegraded":"2024-07-04T17:27:28.018119+0000","last_fullsized":"2024-07-04T17:27:28.018119+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:30:08.921831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.030635+0000","last_change":"2024-07-04T17:27:28.020047+0000","last_active":"2024-07-04T17:27:28.030635+0000","last_peered":"2024-07-04T17:27:28.030635+0000","last_clean":"2024-07-04T17:27:28.030635+0000","last_became_active":"2024-07-04T17:27:28.019697+0000","last_became_peered":"2024-07-04T17:27:28.019697+0000","last_unstale":"2024-07-04T17:27:28.030635+0000","last_undegraded":"2024-07-04T17:27:28.030635+0000","last_fullsized":"2024-07-04T17:27:28.030635+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:15:24.893281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.966923+0000","last_change":"2024-07-04T17:27:06.873153+0000","last_active":"2024-07-04T17:27:27.966923+0000","last_peered":"2024-07-04T17:27:27.966923+0000","last_clean":"2024-07-04T17:27:27.966923+0000","last_became_active":"2024-07-04T17:27:06.872701+0000","last_became_peered":"2024-07-04T17:27:06.872701+0000","last_unstale":"2024-07-04T17:27:27.966923+0000","last_undegraded":"2024-07-04T17:27:27.966923+0000","last_fullsized":"2024-07-04T17:27:27.966923+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:23:40.790450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.969976+0000","last_change":"2024-07-04T17:27:07.886433+0000","last_active":"2024-07-04T17:27:27.969976+0000","last_peered":"2024-07-04T17:27:27.969976+0000","last_clean":"2024-07-04T17:27:27.969976+0000","last_became_active":"2024-07-04T17:27:07.878497+0000","last_became_peered":"2024-07-04T17:27:07.878497+0000","last_unstale":"2024-07-04T17:27:27.969976+0000","last_undegraded":"2024-07-04T17:27:27.969976+0000","last_fullsized":"2024-07-04T17:27:27.969976+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:59:48.713747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.10","version":"30'8","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.034024+0000","last_change":"2024-07-04T17:27:26.993836+0000","last_active":"2024-07-04T17:27:28.034024+0000","last_peered":"2024-07-04T17:27:28.034024+0000","last_clean":"2024-07-04T17:27:28.034024+0000","last_became_active":"2024-07-04T17:27:26.993545+0000","last_became_peered":"2024-07-04T17:27:26.993545+0000","last_unstale":"2024-07-04T17:27:28.034024+0000","last_undegraded":"2024-07-04T17:27:28.034024+0000","last_fullsized":"2024-07-04T17:27:28.034024+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:25:11.039208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.998793+0000","last_change":"2024-07-04T17:27:27.985575+0000","last_active":"2024-07-04T17:27:27.998793+0000","last_peered":"2024-07-04T17:27:27.998793+0000","last_clean":"2024-07-04T17:27:27.998793+0000","last_became_active":"2024-07-04T17:27:27.985158+0000","last_became_peered":"2024-07-04T17:27:27.985158+0000","last_unstale":"2024-07-04T17:27:27.998793+0000","last_undegraded":"2024-07-04T17:27:27.998793+0000","last_fullsized":"2024-07-04T17:27:27.998793+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:57:25.963098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"26'2","reported_seq":22,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.978430+0000","last_change":"2024-07-04T17:27:06.883778+0000","last_active":"2024-07-04T17:27:27.978430+0000","last_peered":"2024-07-04T17:27:27.978430+0000","last_clean":"2024-07-04T17:27:27.978430+0000","last_became_active":"2024-07-04T17:27:06.883449+0000","last_became_peered":"2024-07-04T17:27:06.883449+0000","last_unstale":"2024-07-04T17:27:27.978430+0000","last_undegraded":"2024-07-04T17:27:27.978430+0000","last_fullsized":"2024-07-04T17:27:27.978430+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:05:09.285237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.968707+0000","last_change":"2024-07-04T17:27:07.878309+0000","last_active":"2024-07-04T17:27:27.968707+0000","last_peered":"2024-07-04T17:27:27.968707+0000","last_clean":"2024-07-04T17:27:27.968707+0000","last_became_active":"2024-07-04T17:27:07.877975+0000","last_became_peered":"2024-07-04T17:27:07.877975+0000","last_unstale":"2024-07-04T17:27:27.968707+0000","last_undegraded":"2024-07-04T17:27:27.968707+0000","last_fullsized":"2024-07-04T17:27:27.968707+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:23.798405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.973070+0000","last_change":"2024-07-04T17:27:26.979760+0000","last_active":"2024-07-04T17:27:27.973070+0000","last_peered":"2024-07-04T17:27:27.973070+0000","last_clean":"2024-07-04T17:27:27.973070+0000","last_became_active":"2024-07-04T17:27:26.979431+0000","last_became_peered":"2024-07-04T17:27:26.979431+0000","last_unstale":"2024-07-04T17:27:27.973070+0000","last_undegraded":"2024-07-04T17:27:27.973070+0000","last_fullsized":"2024-07-04T17:27:27.973070+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:19:16.598719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.025987+0000","last_change":"2024-07-04T17:27:28.014511+0000","last_active":"2024-07-04T17:27:28.025987+0000","last_peered":"2024-07-04T17:27:28.025987+0000","last_clean":"2024-07-04T17:27:28.025987+0000","last_became_active":"2024-07-04T17:27:28.014044+0000","last_became_peered":"2024-07-04T17:27:28.014044+0000","last_unstale":"2024-07-04T17:27:28.025987+0000","last_undegraded":"2024-07-04T17:27:28.025987+0000","last_fullsized":"2024-07-04T17:27:28.025987+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:44:15.882179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"27'9","reported_seq":29,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.970340+0000","last_change":"2024-07-04T17:27:06.872191+0000","last_active":"2024-07-04T17:27:27.970340+0000","last_peered":"2024-07-04T17:27:27.970340+0000","last_clean":"2024-07-04T17:27:27.970340+0000","last_became_active":"2024-07-04T17:27:06.871594+0000","last_became_peered":"2024-07-04T17:27:06.871594+0000","last_unstale":"2024-07-04T17:27:27.970340+0000","last_undegraded":"2024-07-04T17:27:27.970340+0000","last_fullsized":"2024-07-04T17:27:27.970340+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:05:09.285237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3309,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.974606+0000","last_change":"2024-07-04T17:27:07.889067+0000","last_active":"2024-07-04T17:27:27.974606+0000","last_peered":"2024-07-04T17:27:27.974606+0000","last_clean":"2024-07-04T17:27:27.974606+0000","last_became_active":"2024-07-04T17:27:07.888715+0000","last_became_peered":"2024-07-04T17:27:07.888715+0000","last_unstale":"2024-07-04T17:27:27.974606+0000","last_undegraded":"2024-07-04T17:27:27.974606+0000","last_fullsized":"2024-07-04T17:27:27.974606+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:51:30.952338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.983593+0000","last_change":"2024-07-04T17:27:26.985620+0000","last_active":"2024-07-04T17:27:27.983593+0000","last_peered":"2024-07-04T17:27:27.983593+0000","last_clean":"2024-07-04T17:27:27.983593+0000","last_became_active":"2024-07-04T17:27:26.985096+0000","last_became_peered":"2024-07-04T17:27:26.985096+0000","last_unstale":"2024-07-04T17:27:27.983593+0000","last_undegraded":"2024-07-04T17:27:27.983593+0000","last_fullsized":"2024-07-04T17:27:27.983593+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:05:29.349573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.029366+0000","last_change":"2024-07-04T17:27:28.020943+0000","last_active":"2024-07-04T17:27:28.029366+0000","last_peered":"2024-07-04T17:27:28.029366+0000","last_clean":"2024-07-04T17:27:28.029366+0000","last_became_active":"2024-07-04T17:27:28.020616+0000","last_became_peered":"2024-07-04T17:27:28.020616+0000","last_unstale":"2024-07-04T17:27:28.029366+0000","last_undegraded":"2024-07-04T17:27:28.029366+0000","last_fullsized":"2024-07-04T17:27:28.029366+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:49:36.186056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"26'3","reported_seq":23,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.968300+0000","last_change":"2024-07-04T17:27:06.868145+0000","last_active":"2024-07-04T17:27:27.968300+0000","last_peered":"2024-07-04T17:27:27.968300+0000","last_clean":"2024-07-04T17:27:27.968300+0000","last_became_active":"2024-07-04T17:27:06.867751+0000","last_became_peered":"2024-07-04T17:27:06.867751+0000","last_unstale":"2024-07-04T17:27:27.968300+0000","last_undegraded":"2024-07-04T17:27:27.968300+0000","last_fullsized":"2024-07-04T17:27:27.968300+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:13:08.657441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.958997+0000","last_change":"2024-07-04T17:27:07.891116+0000","last_active":"2024-07-04T17:27:27.958997+0000","last_peered":"2024-07-04T17:27:27.958997+0000","last_clean":"2024-07-04T17:27:27.958997+0000","last_became_active":"2024-07-04T17:27:07.890572+0000","last_became_peered":"2024-07-04T17:27:07.890572+0000","last_unstale":"2024-07-04T17:27:27.958997+0000","last_undegraded":"2024-07-04T17:27:27.958997+0000","last_fullsized":"2024-07-04T17:27:27.958997+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:51:29.047031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.977916+0000","last_change":"2024-07-04T17:27:26.977555+0000","last_active":"2024-07-04T17:27:27.977916+0000","last_peered":"2024-07-04T17:27:27.977916+0000","last_clean":"2024-07-04T17:27:27.977916+0000","last_became_active":"2024-07-04T17:27:26.976808+0000","last_became_peered":"2024-07-04T17:27:26.976808+0000","last_unstale":"2024-07-04T17:27:27.977916+0000","last_undegraded":"2024-07-04T17:27:27.977916+0000","last_fullsized":"2024-07-04T17:27:27.977916+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:40:10.805520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.017334+0000","last_change":"2024-07-04T17:27:28.002590+0000","last_active":"2024-07-04T17:27:28.017334+0000","last_peered":"2024-07-04T17:27:28.017334+0000","last_clean":"2024-07-04T17:27:28.017334+0000","last_became_active":"2024-07-04T17:27:28.000398+0000","last_became_peered":"2024-07-04T17:27:28.000398+0000","last_unstale":"2024-07-04T17:27:28.017334+0000","last_undegraded":"2024-07-04T17:27:28.017334+0000","last_fullsized":"2024-07-04T17:27:28.017334+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:11:06.743293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.959761+0000","last_change":"2024-07-04T17:27:06.873231+0000","last_active":"2024-07-04T17:27:27.959761+0000","last_peered":"2024-07-04T17:27:27.959761+0000","last_clean":"2024-07-04T17:27:27.959761+0000","last_became_active":"2024-07-04T17:27:06.872934+0000","last_became_peered":"2024-07-04T17:27:06.872934+0000","last_unstale":"2024-07-04T17:27:27.959761+0000","last_undegraded":"2024-07-04T17:27:27.959761+0000","last_fullsized":"2024-07-04T17:27:27.959761+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:42:02.672719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.983055+0000","last_change":"2024-07-04T17:27:07.898440+0000","last_active":"2024-07-04T17:27:27.983055+0000","last_peered":"2024-07-04T17:27:27.983055+0000","last_clean":"2024-07-04T17:27:27.983055+0000","last_became_active":"2024-07-04T17:27:07.897813+0000","last_became_peered":"2024-07-04T17:27:07.897813+0000","last_unstale":"2024-07-04T17:27:27.983055+0000","last_undegraded":"2024-07-04T17:27:27.983055+0000","last_fullsized":"2024-07-04T17:27:27.983055+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:38:45.217394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.14","version":"30'6","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.036663+0000","last_change":"2024-07-04T17:27:26.979315+0000","last_active":"2024-07-04T17:27:28.036663+0000","last_peered":"2024-07-04T17:27:28.036663+0000","last_clean":"2024-07-04T17:27:28.036663+0000","last_became_active":"2024-07-04T17:27:26.978520+0000","last_became_peered":"2024-07-04T17:27:26.978520+0000","last_unstale":"2024-07-04T17:27:28.036663+0000","last_undegraded":"2024-07-04T17:27:28.036663+0000","last_fullsized":"2024-07-04T17:27:28.036663+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:35:49.643323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.999104+0000","last_change":"2024-07-04T17:27:27.987156+0000","last_active":"2024-07-04T17:27:27.999104+0000","last_peered":"2024-07-04T17:27:27.999104+0000","last_clean":"2024-07-04T17:27:27.999104+0000","last_became_active":"2024-07-04T17:27:27.986834+0000","last_became_peered":"2024-07-04T17:27:27.986834+0000","last_unstale":"2024-07-04T17:27:27.999104+0000","last_undegraded":"2024-07-04T17:27:27.999104+0000","last_fullsized":"2024-07-04T17:27:27.999104+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:01:26.219741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.977201+0000","last_change":"2024-07-04T17:27:06.861151+0000","last_active":"2024-07-04T17:27:27.977201+0000","last_peered":"2024-07-04T17:27:27.977201+0000","last_clean":"2024-07-04T17:27:27.977201+0000","last_became_active":"2024-07-04T17:27:06.860975+0000","last_became_peered":"2024-07-04T17:27:06.860975+0000","last_unstale":"2024-07-04T17:27:27.977201+0000","last_undegraded":"2024-07-04T17:27:27.977201+0000","last_fullsized":"2024-07-04T17:27:27.977201+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:17:47.882494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.976776+0000","last_change":"2024-07-04T17:27:07.877680+0000","last_active":"2024-07-04T17:27:27.976776+0000","last_peered":"2024-07-04T17:27:27.976776+0000","last_clean":"2024-07-04T17:27:27.976776+0000","last_became_active":"2024-07-04T17:27:07.877486+0000","last_became_peered":"2024-07-04T17:27:07.877486+0000","last_unstale":"2024-07-04T17:27:27.976776+0000","last_undegraded":"2024-07-04T17:27:27.976776+0000","last_fullsized":"2024-07-04T17:27:27.976776+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:27:08.653236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.15","version":"30'3","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.023632+0000","last_change":"2024-07-04T17:27:26.980595+0000","last_active":"2024-07-04T17:27:28.023632+0000","last_peered":"2024-07-04T17:27:28.023632+0000","last_clean":"2024-07-04T17:27:28.023632+0000","last_became_active":"2024-07-04T17:27:26.978486+0000","last_became_peered":"2024-07-04T17:27:26.978486+0000","last_unstale":"2024-07-04T17:27:28.023632+0000","last_undegraded":"2024-07-04T17:27:28.023632+0000","last_fullsized":"2024-07-04T17:27:28.023632+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:51:50.052560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.016362+0000","last_change":"2024-07-04T17:27:28.001911+0000","last_active":"2024-07-04T17:27:28.016362+0000","last_peered":"2024-07-04T17:27:28.016362+0000","last_clean":"2024-07-04T17:27:28.016362+0000","last_became_active":"2024-07-04T17:27:28.001234+0000","last_became_peered":"2024-07-04T17:27:28.001234+0000","last_unstale":"2024-07-04T17:27:28.016362+0000","last_undegraded":"2024-07-04T17:27:28.016362+0000","last_fullsized":"2024-07-04T17:27:28.016362+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:09:53.789876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.967489+0000","last_change":"2024-07-04T17:27:06.883828+0000","last_active":"2024-07-04T17:27:27.967489+0000","last_peered":"2024-07-04T17:27:27.967489+0000","last_clean":"2024-07-04T17:27:27.967489+0000","last_became_active":"2024-07-04T17:27:06.883508+0000","last_became_peered":"2024-07-04T17:27:06.883508+0000","last_unstale":"2024-07-04T17:27:27.967489+0000","last_undegraded":"2024-07-04T17:27:27.967489+0000","last_fullsized":"2024-07-04T17:27:27.967489+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:44:59.048384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.967778+0000","last_change":"2024-07-04T17:27:07.885943+0000","last_active":"2024-07-04T17:27:27.967778+0000","last_peered":"2024-07-04T17:27:27.967778+0000","last_clean":"2024-07-04T17:27:27.967778+0000","last_became_active":"2024-07-04T17:27:07.876919+0000","last_became_peered":"2024-07-04T17:27:07.876919+0000","last_unstale":"2024-07-04T17:27:27.967778+0000","last_undegraded":"2024-07-04T17:27:27.967778+0000","last_fullsized":"2024-07-04T17:27:27.967778+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:09:16.307994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"30'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.029663+0000","last_change":"2024-07-04T17:27:26.987170+0000","last_active":"2024-07-04T17:27:28.029663+0000","last_peered":"2024-07-04T17:27:28.029663+0000","last_clean":"2024-07-04T17:27:28.029663+0000","last_became_active":"2024-07-04T17:27:26.986855+0000","last_became_peered":"2024-07-04T17:27:26.986855+0000","last_unstale":"2024-07-04T17:27:28.029663+0000","last_undegraded":"2024-07-04T17:27:28.029663+0000","last_fullsized":"2024-07-04T17:27:28.029663+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:44:29.281558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.030774+0000","last_change":"2024-07-04T17:27:28.021858+0000","last_active":"2024-07-04T17:27:28.030774+0000","last_peered":"2024-07-04T17:27:28.030774+0000","last_clean":"2024-07-04T17:27:28.030774+0000","last_became_active":"2024-07-04T17:27:28.021553+0000","last_became_peered":"2024-07-04T17:27:28.021553+0000","last_unstale":"2024-07-04T17:27:28.030774+0000","last_undegraded":"2024-07-04T17:27:28.030774+0000","last_fullsized":"2024-07-04T17:27:28.030774+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:01:26.219741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.955437+0000","last_change":"2024-07-04T17:27:06.872335+0000","last_active":"2024-07-04T17:27:27.955437+0000","last_peered":"2024-07-04T17:27:27.955437+0000","last_clean":"2024-07-04T17:27:27.955437+0000","last_became_active":"2024-07-04T17:27:06.871960+0000","last_became_peered":"2024-07-04T17:27:06.871960+0000","last_unstale":"2024-07-04T17:27:27.955437+0000","last_undegraded":"2024-07-04T17:27:27.955437+0000","last_fullsized":"2024-07-04T17:27:27.955437+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:57:28.966215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.972195+0000","last_change":"2024-07-04T17:27:07.866107+0000","last_active":"2024-07-04T17:27:27.972195+0000","last_peered":"2024-07-04T17:27:27.972195+0000","last_clean":"2024-07-04T17:27:27.972195+0000","last_became_active":"2024-07-04T17:27:07.865761+0000","last_became_peered":"2024-07-04T17:27:07.865761+0000","last_unstale":"2024-07-04T17:27:27.972195+0000","last_undegraded":"2024-07-04T17:27:27.972195+0000","last_fullsized":"2024-07-04T17:27:27.972195+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:49:26.007232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"30'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.033379+0000","last_change":"2024-07-04T17:27:26.989568+0000","last_active":"2024-07-04T17:27:28.033379+0000","last_peered":"2024-07-04T17:27:28.033379+0000","last_clean":"2024-07-04T17:27:28.033379+0000","last_became_active":"2024-07-04T17:27:26.989241+0000","last_became_peered":"2024-07-04T17:27:26.989241+0000","last_unstale":"2024-07-04T17:27:28.033379+0000","last_undegraded":"2024-07-04T17:27:28.033379+0000","last_fullsized":"2024-07-04T17:27:28.033379+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:34:39.046205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.998717+0000","last_change":"2024-07-04T17:27:27.983473+0000","last_active":"2024-07-04T17:27:27.998717+0000","last_peered":"2024-07-04T17:27:27.998717+0000","last_clean":"2024-07-04T17:27:27.998717+0000","last_became_active":"2024-07-04T17:27:27.983015+0000","last_became_peered":"2024-07-04T17:27:27.983015+0000","last_unstale":"2024-07-04T17:27:27.998717+0000","last_undegraded":"2024-07-04T17:27:27.998717+0000","last_fullsized":"2024-07-04T17:27:27.998717+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:15:24.893281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"26'1","reported_seq":21,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.973406+0000","last_change":"2024-07-04T17:27:06.881468+0000","last_active":"2024-07-04T17:27:27.973406+0000","last_peered":"2024-07-04T17:27:27.973406+0000","last_clean":"2024-07-04T17:27:27.973406+0000","last_became_active":"2024-07-04T17:27:06.881150+0000","last_became_peered":"2024-07-04T17:27:06.881150+0000","last_unstale":"2024-07-04T17:27:27.973406+0000","last_undegraded":"2024-07-04T17:27:27.973406+0000","last_fullsized":"2024-07-04T17:27:27.973406+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:49:39.642772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.977979+0000","last_change":"2024-07-04T17:27:07.904214+0000","last_active":"2024-07-04T17:27:27.977979+0000","last_peered":"2024-07-04T17:27:27.977979+0000","last_clean":"2024-07-04T17:27:27.977979+0000","last_became_active":"2024-07-04T17:27:07.903782+0000","last_became_peered":"2024-07-04T17:27:07.903782+0000","last_unstale":"2024-07-04T17:27:27.977979+0000","last_undegraded":"2024-07-04T17:27:27.977979+0000","last_fullsized":"2024-07-04T17:27:27.977979+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:42:03.682646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.968185+0000","last_change":"2024-07-04T17:27:26.970514+0000","last_active":"2024-07-04T17:27:27.968185+0000","last_peered":"2024-07-04T17:27:27.968185+0000","last_clean":"2024-07-04T17:27:27.968185+0000","last_became_active":"2024-07-04T17:27:26.970160+0000","last_became_peered":"2024-07-04T17:27:26.970160+0000","last_unstale":"2024-07-04T17:27:27.968185+0000","last_undegraded":"2024-07-04T17:27:27.968185+0000","last_fullsized":"2024-07-04T17:27:27.968185+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:13:57.406777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.016540+0000","last_change":"2024-07-04T17:27:27.999689+0000","last_active":"2024-07-04T17:27:28.016540+0000","last_peered":"2024-07-04T17:27:28.016540+0000","last_clean":"2024-07-04T17:27:28.016540+0000","last_became_active":"2024-07-04T17:27:27.999348+0000","last_became_peered":"2024-07-04T17:27:27.999348+0000","last_unstale":"2024-07-04T17:27:28.016540+0000","last_undegraded":"2024-07-04T17:27:28.016540+0000","last_fullsized":"2024-07-04T17:27:28.016540+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:31:17.246893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"26'1","reported_seq":21,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.977306+0000","last_change":"2024-07-04T17:27:06.883716+0000","last_active":"2024-07-04T17:27:27.977306+0000","last_peered":"2024-07-04T17:27:27.977306+0000","last_clean":"2024-07-04T17:27:27.977306+0000","last_became_active":"2024-07-04T17:27:06.882604+0000","last_became_peered":"2024-07-04T17:27:06.882604+0000","last_unstale":"2024-07-04T17:27:27.977306+0000","last_undegraded":"2024-07-04T17:27:27.977306+0000","last_fullsized":"2024-07-04T17:27:27.977306+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:04:44.886174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.968914+0000","last_change":"2024-07-04T17:27:07.872169+0000","last_active":"2024-07-04T17:27:27.968914+0000","last_peered":"2024-07-04T17:27:27.968914+0000","last_clean":"2024-07-04T17:27:27.968914+0000","last_became_active":"2024-07-04T17:27:07.870986+0000","last_became_peered":"2024-07-04T17:27:07.870986+0000","last_unstale":"2024-07-04T17:27:27.968914+0000","last_undegraded":"2024-07-04T17:27:27.968914+0000","last_fullsized":"2024-07-04T17:27:27.968914+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:34:46.000005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.956997+0000","last_change":"2024-07-04T17:27:26.988054+0000","last_active":"2024-07-04T17:27:27.956997+0000","last_peered":"2024-07-04T17:27:27.956997+0000","last_clean":"2024-07-04T17:27:27.956997+0000","last_became_active":"2024-07-04T17:27:26.985155+0000","last_became_peered":"2024-07-04T17:27:26.985155+0000","last_unstale":"2024-07-04T17:27:27.956997+0000","last_undegraded":"2024-07-04T17:27:27.956997+0000","last_fullsized":"2024-07-04T17:27:27.956997+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:36:05.554366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.021704+0000","last_change":"2024-07-04T17:27:28.018532+0000","last_active":"2024-07-04T17:27:28.021704+0000","last_peered":"2024-07-04T17:27:28.021704+0000","last_clean":"2024-07-04T17:27:28.021704+0000","last_became_active":"2024-07-04T17:27:28.018166+0000","last_became_peered":"2024-07-04T17:27:28.018166+0000","last_unstale":"2024-07-04T17:27:28.021704+0000","last_undegraded":"2024-07-04T17:27:28.021704+0000","last_fullsized":"2024-07-04T17:27:28.021704+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:38:51.351144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"30'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.016248+0000","last_change":"2024-07-04T17:27:26.989355+0000","last_active":"2024-07-04T17:27:28.016248+0000","last_peered":"2024-07-04T17:27:28.016248+0000","last_clean":"2024-07-04T17:27:28.016248+0000","last_became_active":"2024-07-04T17:27:26.988955+0000","last_became_peered":"2024-07-04T17:27:26.988955+0000","last_unstale":"2024-07-04T17:27:28.016248+0000","last_undegraded":"2024-07-04T17:27:28.016248+0000","last_fullsized":"2024-07-04T17:27:28.016248+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:17:50.728594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"26'2","reported_seq":22,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.979708+0000","last_change":"2024-07-04T17:27:06.885584+0000","last_active":"2024-07-04T17:27:27.979708+0000","last_peered":"2024-07-04T17:27:27.979708+0000","last_clean":"2024-07-04T17:27:27.979708+0000","last_became_active":"2024-07-04T17:27:06.885226+0000","last_became_peered":"2024-07-04T17:27:06.885226+0000","last_unstale":"2024-07-04T17:27:27.979708+0000","last_undegraded":"2024-07-04T17:27:27.979708+0000","last_fullsized":"2024-07-04T17:27:27.979708+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:10:29.469978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.975437+0000","last_change":"2024-07-04T17:27:07.901586+0000","last_active":"2024-07-04T17:27:27.975437+0000","last_peered":"2024-07-04T17:27:27.975437+0000","last_clean":"2024-07-04T17:27:27.975437+0000","last_became_active":"2024-07-04T17:27:07.900901+0000","last_became_peered":"2024-07-04T17:27:07.900901+0000","last_unstale":"2024-07-04T17:27:27.975437+0000","last_undegraded":"2024-07-04T17:27:27.975437+0000","last_fullsized":"2024-07-04T17:27:27.975437+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:27:04.910602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.026098+0000","last_change":"2024-07-04T17:27:28.015072+0000","last_active":"2024-07-04T17:27:28.026098+0000","last_peered":"2024-07-04T17:27:28.026098+0000","last_clean":"2024-07-04T17:27:28.026098+0000","last_became_active":"2024-07-04T17:27:28.014401+0000","last_became_peered":"2024-07-04T17:27:28.014401+0000","last_unstale":"2024-07-04T17:27:28.026098+0000","last_undegraded":"2024-07-04T17:27:28.026098+0000","last_fullsized":"2024-07-04T17:27:28.026098+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:28:51.661977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"30'4","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.023736+0000","last_change":"2024-07-04T17:27:26.980348+0000","last_active":"2024-07-04T17:27:28.023736+0000","last_peered":"2024-07-04T17:27:28.023736+0000","last_clean":"2024-07-04T17:27:28.023736+0000","last_became_active":"2024-07-04T17:27:26.979958+0000","last_became_peered":"2024-07-04T17:27:26.979958+0000","last_unstale":"2024-07-04T17:27:28.023736+0000","last_undegraded":"2024-07-04T17:27:28.023736+0000","last_fullsized":"2024-07-04T17:27:28.023736+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:50:18.424434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"26'2","reported_seq":22,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.974359+0000","last_change":"2024-07-04T17:27:06.860195+0000","last_active":"2024-07-04T17:27:27.974359+0000","last_peered":"2024-07-04T17:27:27.974359+0000","last_clean":"2024-07-04T17:27:27.974359+0000","last_became_active":"2024-07-04T17:27:06.859440+0000","last_became_peered":"2024-07-04T17:27:06.859440+0000","last_unstale":"2024-07-04T17:27:27.974359+0000","last_undegraded":"2024-07-04T17:27:27.974359+0000","last_fullsized":"2024-07-04T17:27:27.974359+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:11:21.518052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.980236+0000","last_change":"2024-07-04T17:27:07.911112+0000","last_active":"2024-07-04T17:27:27.980236+0000","last_peered":"2024-07-04T17:27:27.980236+0000","last_clean":"2024-07-04T17:27:27.980236+0000","last_became_active":"2024-07-04T17:27:07.910473+0000","last_became_peered":"2024-07-04T17:27:07.910473+0000","last_unstale":"2024-07-04T17:27:27.980236+0000","last_undegraded":"2024-07-04T17:27:27.980236+0000","last_fullsized":"2024-07-04T17:27:27.980236+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:47:16.318473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.026349+0000","last_change":"2024-07-04T17:27:28.015745+0000","last_active":"2024-07-04T17:27:28.026349+0000","last_peered":"2024-07-04T17:27:28.026349+0000","last_clean":"2024-07-04T17:27:28.026349+0000","last_became_active":"2024-07-04T17:27:28.014012+0000","last_became_peered":"2024-07-04T17:27:28.014012+0000","last_unstale":"2024-07-04T17:27:28.026349+0000","last_undegraded":"2024-07-04T17:27:28.026349+0000","last_fullsized":"2024-07-04T17:27:28.026349+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:36:06.564787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"26'1","reported_seq":21,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.985428+0000","last_change":"2024-07-04T17:27:06.863457+0000","last_active":"2024-07-04T17:27:27.985428+0000","last_peered":"2024-07-04T17:27:27.985428+0000","last_clean":"2024-07-04T17:27:27.985428+0000","last_became_active":"2024-07-04T17:27:06.862649+0000","last_became_peered":"2024-07-04T17:27:06.862649+0000","last_unstale":"2024-07-04T17:27:27.985428+0000","last_undegraded":"2024-07-04T17:27:27.985428+0000","last_fullsized":"2024-07-04T17:27:27.985428+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:49:24.997305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.982983+0000","last_change":"2024-07-04T17:27:07.896653+0000","last_active":"2024-07-04T17:27:27.982983+0000","last_peered":"2024-07-04T17:27:27.982983+0000","last_clean":"2024-07-04T17:27:27.982983+0000","last_became_active":"2024-07-04T17:27:07.895759+0000","last_became_peered":"2024-07-04T17:27:07.895759+0000","last_unstale":"2024-07-04T17:27:27.982983+0000","last_undegraded":"2024-07-04T17:27:27.982983+0000","last_fullsized":"2024-07-04T17:27:27.982983+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:06:36.067329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"30'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.019678+0000","last_change":"2024-07-04T17:27:26.981603+0000","last_active":"2024-07-04T17:27:28.019678+0000","last_peered":"2024-07-04T17:27:28.019678+0000","last_clean":"2024-07-04T17:27:28.019678+0000","last_became_active":"2024-07-04T17:27:26.980997+0000","last_became_peered":"2024-07-04T17:27:26.980997+0000","last_unstale":"2024-07-04T17:27:28.019678+0000","last_undegraded":"2024-07-04T17:27:28.019678+0000","last_fullsized":"2024-07-04T17:27:28.019678+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:24:19.419795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.028462+0000","last_change":"2024-07-04T17:27:28.010451+0000","last_active":"2024-07-04T17:27:28.028462+0000","last_peered":"2024-07-04T17:27:28.028462+0000","last_clean":"2024-07-04T17:27:28.028462+0000","last_became_active":"2024-07-04T17:27:28.003814+0000","last_became_peered":"2024-07-04T17:27:28.003814+0000","last_unstale":"2024-07-04T17:27:28.028462+0000","last_undegraded":"2024-07-04T17:27:28.028462+0000","last_fullsized":"2024-07-04T17:27:28.028462+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:33.432540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"26'4","reported_seq":24,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.977841+0000","last_change":"2024-07-04T17:27:06.866478+0000","last_active":"2024-07-04T17:27:27.977841+0000","last_peered":"2024-07-04T17:27:27.977841+0000","last_clean":"2024-07-04T17:27:27.977841+0000","last_became_active":"2024-07-04T17:27:06.865591+0000","last_became_peered":"2024-07-04T17:27:06.865591+0000","last_unstale":"2024-07-04T17:27:27.977841+0000","last_undegraded":"2024-07-04T17:27:27.977841+0000","last_fullsized":"2024-07-04T17:27:27.977841+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:09:15.298068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.965337+0000","last_change":"2024-07-04T17:27:07.888430+0000","last_active":"2024-07-04T17:27:27.965337+0000","last_peered":"2024-07-04T17:27:27.965337+0000","last_clean":"2024-07-04T17:27:27.965337+0000","last_became_active":"2024-07-04T17:27:07.886722+0000","last_became_peered":"2024-07-04T17:27:07.886722+0000","last_unstale":"2024-07-04T17:27:27.965337+0000","last_undegraded":"2024-07-04T17:27:27.965337+0000","last_fullsized":"2024-07-04T17:27:27.965337+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:44:31.068133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"30'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.015268+0000","last_change":"2024-07-04T17:27:26.986270+0000","last_active":"2024-07-04T17:27:28.015268+0000","last_peered":"2024-07-04T17:27:28.015268+0000","last_clean":"2024-07-04T17:27:28.015268+0000","last_became_active":"2024-07-04T17:27:26.985927+0000","last_became_peered":"2024-07-04T17:27:26.985927+0000","last_unstale":"2024-07-04T17:27:28.015268+0000","last_undegraded":"2024-07-04T17:27:28.015268+0000","last_fullsized":"2024-07-04T17:27:28.015268+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:53:21.040811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.028435+0000","last_change":"2024-07-04T17:27:28.014596+0000","last_active":"2024-07-04T17:27:28.028435+0000","last_peered":"2024-07-04T17:27:28.028435+0000","last_clean":"2024-07-04T17:27:28.028435+0000","last_became_active":"2024-07-04T17:27:28.014261+0000","last_became_peered":"2024-07-04T17:27:28.014261+0000","last_unstale":"2024-07-04T17:27:28.028435+0000","last_undegraded":"2024-07-04T17:27:28.028435+0000","last_fullsized":"2024-07-04T17:27:28.028435+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:40:11.815941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.969165+0000","last_change":"2024-07-04T17:27:07.910603+0000","last_active":"2024-07-04T17:27:27.969165+0000","last_peered":"2024-07-04T17:27:27.969165+0000","last_clean":"2024-07-04T17:27:27.969165+0000","last_became_active":"2024-07-04T17:27:07.908845+0000","last_became_peered":"2024-07-04T17:27:07.908845+0000","last_unstale":"2024-07-04T17:27:27.969165+0000","last_undegraded":"2024-07-04T17:27:27.969165+0000","last_fullsized":"2024-07-04T17:27:27.969165+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:17:48.892420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.029698+0000","last_change":"2024-07-04T17:27:28.027868+0000","last_active":"2024-07-04T17:27:28.029698+0000","last_peered":"2024-07-04T17:27:28.029698+0000","last_clean":"2024-07-04T17:27:28.029698+0000","last_became_active":"2024-07-04T17:27:28.026823+0000","last_became_peered":"2024-07-04T17:27:28.026823+0000","last_unstale":"2024-07-04T17:27:28.029698+0000","last_undegraded":"2024-07-04T17:27:28.029698+0000","last_fullsized":"2024-07-04T17:27:28.029698+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:21:02.597642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.971962+0000","last_change":"2024-07-04T17:27:07.906719+0000","last_active":"2024-07-04T17:27:27.971962+0000","last_peered":"2024-07-04T17:27:27.971962+0000","last_clean":"2024-07-04T17:27:27.971962+0000","last_became_active":"2024-07-04T17:27:07.903370+0000","last_became_peered":"2024-07-04T17:27:07.903370+0000","last_unstale":"2024-07-04T17:27:27.971962+0000","last_undegraded":"2024-07-04T17:27:27.971962+0000","last_fullsized":"2024-07-04T17:27:27.971962+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:55:16.424967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.006519+0000","last_change":"2024-07-04T17:27:27.999222+0000","last_active":"2024-07-04T17:27:28.006519+0000","last_peered":"2024-07-04T17:27:28.006519+0000","last_clean":"2024-07-04T17:27:28.006519+0000","last_became_active":"2024-07-04T17:27:27.998897+0000","last_became_peered":"2024-07-04T17:27:27.998897+0000","last_unstale":"2024-07-04T17:27:28.006519+0000","last_undegraded":"2024-07-04T17:27:28.006519+0000","last_fullsized":"2024-07-04T17:27:28.006519+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:59:28.341788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.958272+0000","last_change":"2024-07-04T17:27:07.878292+0000","last_active":"2024-07-04T17:27:27.958272+0000","last_peered":"2024-07-04T17:27:27.958272+0000","last_clean":"2024-07-04T17:27:27.958272+0000","last_became_active":"2024-07-04T17:27:07.874906+0000","last_became_peered":"2024-07-04T17:27:07.874906+0000","last_unstale":"2024-07-04T17:27:27.958272+0000","last_undegraded":"2024-07-04T17:27:27.958272+0000","last_fullsized":"2024-07-04T17:27:27.958272+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:20:57.602937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.025556+0000","last_change":"2024-07-04T17:27:28.013502+0000","last_active":"2024-07-04T17:27:28.025556+0000","last_peered":"2024-07-04T17:27:28.025556+0000","last_clean":"2024-07-04T17:27:28.025556+0000","last_became_active":"2024-07-04T17:27:28.013162+0000","last_became_peered":"2024-07-04T17:27:28.013162+0000","last_unstale":"2024-07-04T17:27:28.025556+0000","last_undegraded":"2024-07-04T17:27:28.025556+0000","last_fullsized":"2024-07-04T17:27:28.025556+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:58:39.364744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.984795+0000","last_change":"2024-07-04T17:27:07.883042+0000","last_active":"2024-07-04T17:27:27.984795+0000","last_peered":"2024-07-04T17:27:27.984795+0000","last_clean":"2024-07-04T17:27:27.984795+0000","last_became_active":"2024-07-04T17:27:07.882772+0000","last_became_peered":"2024-07-04T17:27:07.882772+0000","last_unstale":"2024-07-04T17:27:27.984795+0000","last_undegraded":"2024-07-04T17:27:27.984795+0000","last_fullsized":"2024-07-04T17:27:27.984795+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:03:21.103650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.026879+0000","last_change":"2024-07-04T17:27:28.009005+0000","last_active":"2024-07-04T17:27:28.026879+0000","last_peered":"2024-07-04T17:27:28.026879+0000","last_clean":"2024-07-04T17:27:28.026879+0000","last_became_active":"2024-07-04T17:27:28.008680+0000","last_became_peered":"2024-07-04T17:27:28.008680+0000","last_unstale":"2024-07-04T17:27:28.026879+0000","last_undegraded":"2024-07-04T17:27:28.026879+0000","last_fullsized":"2024-07-04T17:27:28.026879+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:48:23.096987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.980368+0000","last_change":"2024-07-04T17:27:07.911625+0000","last_active":"2024-07-04T17:27:27.980368+0000","last_peered":"2024-07-04T17:27:27.980368+0000","last_clean":"2024-07-04T17:27:27.980368+0000","last_became_active":"2024-07-04T17:27:07.910841+0000","last_became_peered":"2024-07-04T17:27:07.910841+0000","last_unstale":"2024-07-04T17:27:27.980368+0000","last_undegraded":"2024-07-04T17:27:27.980368+0000","last_fullsized":"2024-07-04T17:27:27.980368+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:43:39.860474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.034776+0000","last_change":"2024-07-04T17:27:28.032931+0000","last_active":"2024-07-04T17:27:28.034776+0000","last_peered":"2024-07-04T17:27:28.034776+0000","last_clean":"2024-07-04T17:27:28.034776+0000","last_became_active":"2024-07-04T17:27:28.032254+0000","last_became_peered":"2024-07-04T17:27:28.032254+0000","last_unstale":"2024-07-04T17:27:28.034776+0000","last_undegraded":"2024-07-04T17:27:28.034776+0000","last_fullsized":"2024-07-04T17:27:28.034776+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:17:43.391316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.975930+0000","last_change":"2024-07-04T17:27:07.901813+0000","last_active":"2024-07-04T17:27:27.975930+0000","last_peered":"2024-07-04T17:27:27.975930+0000","last_clean":"2024-07-04T17:27:27.975930+0000","last_became_active":"2024-07-04T17:27:07.901333+0000","last_became_peered":"2024-07-04T17:27:07.901333+0000","last_unstale":"2024-07-04T17:27:27.975930+0000","last_undegraded":"2024-07-04T17:27:27.975930+0000","last_fullsized":"2024-07-04T17:27:27.975930+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:24:58.267453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.030595+0000","last_change":"2024-07-04T17:27:28.021463+0000","last_active":"2024-07-04T17:27:28.030595+0000","last_peered":"2024-07-04T17:27:28.030595+0000","last_clean":"2024-07-04T17:27:28.030595+0000","last_became_active":"2024-07-04T17:27:28.021168+0000","last_became_peered":"2024-07-04T17:27:28.021168+0000","last_unstale":"2024-07-04T17:27:28.030595+0000","last_undegraded":"2024-07-04T17:27:28.030595+0000","last_fullsized":"2024-07-04T17:27:28.030595+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:57:25.963098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.976889+0000","last_change":"2024-07-04T17:27:07.877946+0000","last_active":"2024-07-04T17:27:27.976889+0000","last_peered":"2024-07-04T17:27:27.976889+0000","last_clean":"2024-07-04T17:27:27.976889+0000","last_became_active":"2024-07-04T17:27:07.877581+0000","last_became_peered":"2024-07-04T17:27:07.877581+0000","last_unstale":"2024-07-04T17:27:27.976889+0000","last_undegraded":"2024-07-04T17:27:27.976889+0000","last_fullsized":"2024-07-04T17:27:27.976889+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:38:45.217394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.015649+0000","last_change":"2024-07-04T17:27:28.000527+0000","last_active":"2024-07-04T17:27:28.015649+0000","last_peered":"2024-07-04T17:27:28.015649+0000","last_clean":"2024-07-04T17:27:28.015649+0000","last_became_active":"2024-07-04T17:27:28.000096+0000","last_became_peered":"2024-07-04T17:27:28.000096+0000","last_unstale":"2024-07-04T17:27:28.015649+0000","last_undegraded":"2024-07-04T17:27:28.015649+0000","last_fullsized":"2024-07-04T17:27:28.015649+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:26:43.565949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.977428+0000","last_change":"2024-07-04T17:27:07.896753+0000","last_active":"2024-07-04T17:27:27.977428+0000","last_peered":"2024-07-04T17:27:27.977428+0000","last_clean":"2024-07-04T17:27:27.977428+0000","last_became_active":"2024-07-04T17:27:07.896053+0000","last_became_peered":"2024-07-04T17:27:07.896053+0000","last_unstale":"2024-07-04T17:27:27.977428+0000","last_undegraded":"2024-07-04T17:27:27.977428+0000","last_fullsized":"2024-07-04T17:27:27.977428+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:56:08.436905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.009332+0000","last_change":"2024-07-04T17:27:27.997002+0000","last_active":"2024-07-04T17:27:28.009332+0000","last_peered":"2024-07-04T17:27:28.009332+0000","last_clean":"2024-07-04T17:27:28.009332+0000","last_became_active":"2024-07-04T17:27:27.996658+0000","last_became_peered":"2024-07-04T17:27:27.996658+0000","last_unstale":"2024-07-04T17:27:28.009332+0000","last_undegraded":"2024-07-04T17:27:28.009332+0000","last_fullsized":"2024-07-04T17:27:28.009332+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:11:13.050081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.969297+0000","last_change":"2024-07-04T17:27:07.898354+0000","last_active":"2024-07-04T17:27:27.969297+0000","last_peered":"2024-07-04T17:27:27.969297+0000","last_clean":"2024-07-04T17:27:27.969297+0000","last_became_active":"2024-07-04T17:27:07.897988+0000","last_became_peered":"2024-07-04T17:27:07.897988+0000","last_unstale":"2024-07-04T17:27:27.969297+0000","last_undegraded":"2024-07-04T17:27:27.969297+0000","last_fullsized":"2024-07-04T17:27:27.969297+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:25:52.643999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.031300+0000","last_change":"2024-07-04T17:27:28.025891+0000","last_active":"2024-07-04T17:27:28.031300+0000","last_peered":"2024-07-04T17:27:28.031300+0000","last_clean":"2024-07-04T17:27:28.031300+0000","last_became_active":"2024-07-04T17:27:28.024996+0000","last_became_peered":"2024-07-04T17:27:28.024996+0000","last_unstale":"2024-07-04T17:27:28.031300+0000","last_undegraded":"2024-07-04T17:27:28.031300+0000","last_fullsized":"2024-07-04T17:27:28.031300+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:58:39.364744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.967770+0000","last_change":"2024-07-04T17:27:07.883846+0000","last_active":"2024-07-04T17:27:27.967770+0000","last_peered":"2024-07-04T17:27:27.967770+0000","last_clean":"2024-07-04T17:27:27.967770+0000","last_became_active":"2024-07-04T17:27:07.881609+0000","last_became_peered":"2024-07-04T17:27:07.881609+0000","last_unstale":"2024-07-04T17:27:27.967770+0000","last_undegraded":"2024-07-04T17:27:27.967770+0000","last_fullsized":"2024-07-04T17:27:27.967770+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:41:01.912287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.006486+0000","last_change":"2024-07-04T17:27:27.999062+0000","last_active":"2024-07-04T17:27:28.006486+0000","last_peered":"2024-07-04T17:27:28.006486+0000","last_clean":"2024-07-04T17:27:28.006486+0000","last_became_active":"2024-07-04T17:27:27.998482+0000","last_became_peered":"2024-07-04T17:27:27.998482+0000","last_unstale":"2024-07-04T17:27:28.006486+0000","last_undegraded":"2024-07-04T17:27:28.006486+0000","last_fullsized":"2024-07-04T17:27:28.006486+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:44:51.178776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.974277+0000","last_change":"2024-07-04T17:27:07.904537+0000","last_active":"2024-07-04T17:27:27.974277+0000","last_peered":"2024-07-04T17:27:27.974277+0000","last_clean":"2024-07-04T17:27:27.974277+0000","last_became_active":"2024-07-04T17:27:07.903476+0000","last_became_peered":"2024-07-04T17:27:07.903476+0000","last_unstale":"2024-07-04T17:27:27.974277+0000","last_undegraded":"2024-07-04T17:27:27.974277+0000","last_fullsized":"2024-07-04T17:27:27.974277+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:24:58.267453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.009961+0000","last_change":"2024-07-04T17:27:27.994553+0000","last_active":"2024-07-04T17:27:28.009961+0000","last_peered":"2024-07-04T17:27:28.009961+0000","last_clean":"2024-07-04T17:27:28.009961+0000","last_became_active":"2024-07-04T17:27:27.991603+0000","last_became_peered":"2024-07-04T17:27:27.991603+0000","last_unstale":"2024-07-04T17:27:28.009961+0000","last_undegraded":"2024-07-04T17:27:28.009961+0000","last_fullsized":"2024-07-04T17:27:28.009961+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:20:48.237599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.982718+0000","last_change":"2024-07-04T17:27:07.887157+0000","last_active":"2024-07-04T17:27:27.982718+0000","last_peered":"2024-07-04T17:27:27.982718+0000","last_clean":"2024-07-04T17:27:27.982718+0000","last_became_active":"2024-07-04T17:27:07.886783+0000","last_became_peered":"2024-07-04T17:27:07.886783+0000","last_unstale":"2024-07-04T17:27:27.982718+0000","last_undegraded":"2024-07-04T17:27:27.982718+0000","last_fullsized":"2024-07-04T17:27:27.982718+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:18:17.970386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.027260+0000","last_change":"2024-07-04T17:27:28.016402+0000","last_active":"2024-07-04T17:27:28.027260+0000","last_peered":"2024-07-04T17:27:28.027260+0000","last_clean":"2024-07-04T17:27:28.027260+0000","last_became_active":"2024-07-04T17:27:28.016008+0000","last_became_peered":"2024-07-04T17:27:28.016008+0000","last_unstale":"2024-07-04T17:27:28.027260+0000","last_undegraded":"2024-07-04T17:27:28.027260+0000","last_fullsized":"2024-07-04T17:27:28.027260+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:31:34.667696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.980487+0000","last_change":"2024-07-04T17:27:07.916151+0000","last_active":"2024-07-04T17:27:27.980487+0000","last_peered":"2024-07-04T17:27:27.980487+0000","last_clean":"2024-07-04T17:27:27.980487+0000","last_became_active":"2024-07-04T17:27:07.909999+0000","last_became_peered":"2024-07-04T17:27:07.909999+0000","last_unstale":"2024-07-04T17:27:27.980487+0000","last_undegraded":"2024-07-04T17:27:27.980487+0000","last_fullsized":"2024-07-04T17:27:27.980487+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:38:38.667113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.030945+0000","last_change":"2024-07-04T17:27:28.022637+0000","last_active":"2024-07-04T17:27:28.030945+0000","last_peered":"2024-07-04T17:27:28.030945+0000","last_clean":"2024-07-04T17:27:28.030945+0000","last_became_active":"2024-07-04T17:27:28.022337+0000","last_became_peered":"2024-07-04T17:27:28.022337+0000","last_unstale":"2024-07-04T17:27:28.030945+0000","last_undegraded":"2024-07-04T17:27:28.030945+0000","last_fullsized":"2024-07-04T17:27:28.030945+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:04:48.577717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.974892+0000","last_change":"2024-07-04T17:27:07.893065+0000","last_active":"2024-07-04T17:27:27.974892+0000","last_peered":"2024-07-04T17:27:27.974892+0000","last_clean":"2024-07-04T17:27:27.974892+0000","last_became_active":"2024-07-04T17:27:07.892736+0000","last_became_peered":"2024-07-04T17:27:07.892736+0000","last_unstale":"2024-07-04T17:27:27.974892+0000","last_undegraded":"2024-07-04T17:27:27.974892+0000","last_fullsized":"2024-07-04T17:27:27.974892+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:18:57.498497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.030572+0000","last_change":"2024-07-04T17:27:28.020357+0000","last_active":"2024-07-04T17:27:28.030572+0000","last_peered":"2024-07-04T17:27:28.030572+0000","last_clean":"2024-07-04T17:27:28.030572+0000","last_became_active":"2024-07-04T17:27:28.019814+0000","last_became_peered":"2024-07-04T17:27:28.019814+0000","last_unstale":"2024-07-04T17:27:28.030572+0000","last_undegraded":"2024-07-04T17:27:28.030572+0000","last_fullsized":"2024-07-04T17:27:28.030572+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:53:07.282774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.965488+0000","last_change":"2024-07-04T17:27:07.883614+0000","last_active":"2024-07-04T17:27:27.965488+0000","last_peered":"2024-07-04T17:27:27.965488+0000","last_clean":"2024-07-04T17:27:27.965488+0000","last_became_active":"2024-07-04T17:27:07.883289+0000","last_became_peered":"2024-07-04T17:27:07.883289+0000","last_unstale":"2024-07-04T17:27:27.965488+0000","last_undegraded":"2024-07-04T17:27:27.965488+0000","last_fullsized":"2024-07-04T17:27:27.965488+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:54:45.341155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.015735+0000","last_change":"2024-07-04T17:27:28.011027+0000","last_active":"2024-07-04T17:27:28.015735+0000","last_peered":"2024-07-04T17:27:28.015735+0000","last_clean":"2024-07-04T17:27:28.015735+0000","last_became_active":"2024-07-04T17:27:28.010612+0000","last_became_peered":"2024-07-04T17:27:28.010612+0000","last_unstale":"2024-07-04T17:27:28.015735+0000","last_undegraded":"2024-07-04T17:27:28.015735+0000","last_fullsized":"2024-07-04T17:27:28.015735+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:41:06.228068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.978309+0000","last_change":"2024-07-04T17:27:07.877401+0000","last_active":"2024-07-04T17:27:27.978309+0000","last_peered":"2024-07-04T17:27:27.978309+0000","last_clean":"2024-07-04T17:27:27.978309+0000","last_became_active":"2024-07-04T17:27:07.877014+0000","last_became_peered":"2024-07-04T17:27:07.877014+0000","last_unstale":"2024-07-04T17:27:27.978309+0000","last_undegraded":"2024-07-04T17:27:27.978309+0000","last_fullsized":"2024-07-04T17:27:27.978309+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:35:30.543101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.009265+0000","last_change":"2024-07-04T17:27:27.995614+0000","last_active":"2024-07-04T17:27:28.009265+0000","last_peered":"2024-07-04T17:27:28.009265+0000","last_clean":"2024-07-04T17:27:28.009265+0000","last_became_active":"2024-07-04T17:27:27.994228+0000","last_became_peered":"2024-07-04T17:27:27.994228+0000","last_unstale":"2024-07-04T17:27:28.009265+0000","last_undegraded":"2024-07-04T17:27:28.009265+0000","last_fullsized":"2024-07-04T17:27:28.009265+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:54:06.893081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.971609+0000","last_change":"2024-07-04T17:27:07.884482+0000","last_active":"2024-07-04T17:27:27.971609+0000","last_peered":"2024-07-04T17:27:27.971609+0000","last_clean":"2024-07-04T17:27:27.971609+0000","last_became_active":"2024-07-04T17:27:07.882978+0000","last_became_peered":"2024-07-04T17:27:07.882978+0000","last_unstale":"2024-07-04T17:27:27.971609+0000","last_undegraded":"2024-07-04T17:27:27.971609+0000","last_fullsized":"2024-07-04T17:27:27.971609+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:40:02.282743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.026970+0000","last_change":"2024-07-04T17:27:28.001068+0000","last_active":"2024-07-04T17:27:28.026970+0000","last_peered":"2024-07-04T17:27:28.026970+0000","last_clean":"2024-07-04T17:27:28.026970+0000","last_became_active":"2024-07-04T17:27:27.998408+0000","last_became_peered":"2024-07-04T17:27:27.998408+0000","last_unstale":"2024-07-04T17:27:28.026970+0000","last_undegraded":"2024-07-04T17:27:28.026970+0000","last_fullsized":"2024-07-04T17:27:28.026970+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:55:07.289164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.967196+0000","last_change":"2024-07-04T17:27:07.873017+0000","last_active":"2024-07-04T17:27:27.967196+0000","last_peered":"2024-07-04T17:27:27.967196+0000","last_clean":"2024-07-04T17:27:27.967196+0000","last_became_active":"2024-07-04T17:27:07.872715+0000","last_became_peered":"2024-07-04T17:27:07.872715+0000","last_unstale":"2024-07-04T17:27:27.967196+0000","last_undegraded":"2024-07-04T17:27:27.967196+0000","last_fullsized":"2024-07-04T17:27:27.967196+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:09:07.885707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.021412+0000","last_change":"2024-07-04T17:27:28.016351+0000","last_active":"2024-07-04T17:27:28.021412+0000","last_peered":"2024-07-04T17:27:28.021412+0000","last_clean":"2024-07-04T17:27:28.021412+0000","last_became_active":"2024-07-04T17:27:28.011415+0000","last_became_peered":"2024-07-04T17:27:28.011415+0000","last_unstale":"2024-07-04T17:27:28.021412+0000","last_undegraded":"2024-07-04T17:27:28.021412+0000","last_fullsized":"2024-07-04T17:27:28.021412+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:16:54.856497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.958657+0000","last_change":"2024-07-04T17:27:07.879943+0000","last_active":"2024-07-04T17:27:27.958657+0000","last_peered":"2024-07-04T17:27:27.958657+0000","last_clean":"2024-07-04T17:27:27.958657+0000","last_became_active":"2024-07-04T17:27:07.875927+0000","last_became_peered":"2024-07-04T17:27:07.875927+0000","last_unstale":"2024-07-04T17:27:27.958657+0000","last_undegraded":"2024-07-04T17:27:27.958657+0000","last_fullsized":"2024-07-04T17:27:27.958657+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:38:48.110832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.006292+0000","last_change":"2024-07-04T17:27:27.999385+0000","last_active":"2024-07-04T17:27:28.006292+0000","last_peered":"2024-07-04T17:27:28.006292+0000","last_clean":"2024-07-04T17:27:28.006292+0000","last_became_active":"2024-07-04T17:27:27.998592+0000","last_became_peered":"2024-07-04T17:27:27.998592+0000","last_unstale":"2024-07-04T17:27:28.006292+0000","last_undegraded":"2024-07-04T17:27:28.006292+0000","last_fullsized":"2024-07-04T17:27:28.006292+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:55:05.451797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.978100+0000","last_change":"2024-07-04T17:27:07.896119+0000","last_active":"2024-07-04T17:27:27.978100+0000","last_peered":"2024-07-04T17:27:27.978100+0000","last_clean":"2024-07-04T17:27:27.978100+0000","last_became_active":"2024-07-04T17:27:07.895583+0000","last_became_peered":"2024-07-04T17:27:07.895583+0000","last_unstale":"2024-07-04T17:27:27.978100+0000","last_undegraded":"2024-07-04T17:27:27.978100+0000","last_fullsized":"2024-07-04T17:27:27.978100+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:49:26.007232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.033363+0000","last_change":"2024-07-04T17:27:28.031337+0000","last_active":"2024-07-04T17:27:28.033363+0000","last_peered":"2024-07-04T17:27:28.033363+0000","last_clean":"2024-07-04T17:27:28.033363+0000","last_became_active":"2024-07-04T17:27:28.029954+0000","last_became_peered":"2024-07-04T17:27:28.029954+0000","last_unstale":"2024-07-04T17:27:28.033363+0000","last_undegraded":"2024-07-04T17:27:28.033363+0000","last_fullsized":"2024-07-04T17:27:28.033363+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:07:18.483120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.970596+0000","last_change":"2024-07-04T17:27:07.886093+0000","last_active":"2024-07-04T17:27:27.970596+0000","last_peered":"2024-07-04T17:27:27.970596+0000","last_clean":"2024-07-04T17:27:27.970596+0000","last_became_active":"2024-07-04T17:27:07.885701+0000","last_became_peered":"2024-07-04T17:27:07.885701+0000","last_unstale":"2024-07-04T17:27:27.970596+0000","last_undegraded":"2024-07-04T17:27:27.970596+0000","last_fullsized":"2024-07-04T17:27:27.970596+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:43:39.860474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.027029+0000","last_change":"2024-07-04T17:27:28.023068+0000","last_active":"2024-07-04T17:27:28.027029+0000","last_peered":"2024-07-04T17:27:28.027029+0000","last_clean":"2024-07-04T17:27:28.027029+0000","last_became_active":"2024-07-04T17:27:28.022773+0000","last_became_peered":"2024-07-04T17:27:28.022773+0000","last_unstale":"2024-07-04T17:27:28.027029+0000","last_undegraded":"2024-07-04T17:27:28.027029+0000","last_fullsized":"2024-07-04T17:27:28.027029+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:50:15.117673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.984101+0000","last_change":"2024-07-04T17:27:07.893125+0000","last_active":"2024-07-04T17:27:27.984101+0000","last_peered":"2024-07-04T17:27:27.984101+0000","last_clean":"2024-07-04T17:27:27.984101+0000","last_became_active":"2024-07-04T17:27:07.892921+0000","last_became_peered":"2024-07-04T17:27:07.892921+0000","last_unstale":"2024-07-04T17:27:27.984101+0000","last_undegraded":"2024-07-04T17:27:27.984101+0000","last_fullsized":"2024-07-04T17:27:27.984101+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:53:39.500774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.030545+0000","last_change":"2024-07-04T17:27:28.020064+0000","last_active":"2024-07-04T17:27:28.030545+0000","last_peered":"2024-07-04T17:27:28.030545+0000","last_clean":"2024-07-04T17:27:28.030545+0000","last_became_active":"2024-07-04T17:27:28.019528+0000","last_became_peered":"2024-07-04T17:27:28.019528+0000","last_unstale":"2024-07-04T17:27:28.030545+0000","last_undegraded":"2024-07-04T17:27:28.030545+0000","last_fullsized":"2024-07-04T17:27:28.030545+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:47:31.123344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.976999+0000","last_change":"2024-07-04T17:27:07.871832+0000","last_active":"2024-07-04T17:27:27.976999+0000","last_peered":"2024-07-04T17:27:27.976999+0000","last_clean":"2024-07-04T17:27:27.976999+0000","last_became_active":"2024-07-04T17:27:07.870928+0000","last_became_peered":"2024-07-04T17:27:07.870928+0000","last_unstale":"2024-07-04T17:27:27.976999+0000","last_undegraded":"2024-07-04T17:27:27.976999+0000","last_fullsized":"2024-07-04T17:27:27.976999+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:06:36.067329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.010197+0000","last_change":"2024-07-04T17:27:27.993394+0000","last_active":"2024-07-04T17:27:28.010197+0000","last_peered":"2024-07-04T17:27:28.010197+0000","last_clean":"2024-07-04T17:27:28.010197+0000","last_became_active":"2024-07-04T17:27:27.993050+0000","last_became_peered":"2024-07-04T17:27:27.993050+0000","last_unstale":"2024-07-04T17:27:28.010197+0000","last_undegraded":"2024-07-04T17:27:28.010197+0000","last_fullsized":"2024-07-04T17:27:28.010197+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:33:10.722625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.976542+0000","last_change":"2024-07-04T17:27:07.897001+0000","last_active":"2024-07-04T17:27:27.976542+0000","last_peered":"2024-07-04T17:27:27.976542+0000","last_clean":"2024-07-04T17:27:27.976542+0000","last_became_active":"2024-07-04T17:27:07.896453+0000","last_became_peered":"2024-07-04T17:27:07.896453+0000","last_unstale":"2024-07-04T17:27:27.976542+0000","last_undegraded":"2024-07-04T17:27:27.976542+0000","last_fullsized":"2024-07-04T17:27:27.976542+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:20:22.623733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.009309+0000","last_change":"2024-07-04T17:27:27.996306+0000","last_active":"2024-07-04T17:27:28.009309+0000","last_peered":"2024-07-04T17:27:28.009309+0000","last_clean":"2024-07-04T17:27:28.009309+0000","last_became_active":"2024-07-04T17:27:27.993929+0000","last_became_peered":"2024-07-04T17:27:27.993929+0000","last_unstale":"2024-07-04T17:27:28.009309+0000","last_undegraded":"2024-07-04T17:27:28.009309+0000","last_fullsized":"2024-07-04T17:27:28.009309+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:08:46.562654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.969405+0000","last_change":"2024-07-04T17:27:07.899412+0000","last_active":"2024-07-04T17:27:27.969405+0000","last_peered":"2024-07-04T17:27:27.969405+0000","last_clean":"2024-07-04T17:27:27.969405+0000","last_became_active":"2024-07-04T17:27:07.898386+0000","last_became_peered":"2024-07-04T17:27:07.898386+0000","last_unstale":"2024-07-04T17:27:27.969405+0000","last_undegraded":"2024-07-04T17:27:27.969405+0000","last_fullsized":"2024-07-04T17:27:27.969405+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:11:22.527978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.016382+0000","last_change":"2024-07-04T17:27:28.002162+0000","last_active":"2024-07-04T17:27:28.016382+0000","last_peered":"2024-07-04T17:27:28.016382+0000","last_clean":"2024-07-04T17:27:28.016382+0000","last_became_active":"2024-07-04T17:27:28.001587+0000","last_became_peered":"2024-07-04T17:27:28.001587+0000","last_unstale":"2024-07-04T17:27:28.016382+0000","last_undegraded":"2024-07-04T17:27:28.016382+0000","last_fullsized":"2024-07-04T17:27:28.016382+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:24:46.523771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.971601+0000","last_change":"2024-07-04T17:27:07.894901+0000","last_active":"2024-07-04T17:27:27.971601+0000","last_peered":"2024-07-04T17:27:27.971601+0000","last_clean":"2024-07-04T17:27:27.971601+0000","last_became_active":"2024-07-04T17:27:07.894469+0000","last_became_peered":"2024-07-04T17:27:07.894469+0000","last_unstale":"2024-07-04T17:27:27.971601+0000","last_undegraded":"2024-07-04T17:27:27.971601+0000","last_fullsized":"2024-07-04T17:27:27.971601+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:13:09.667367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.007066+0000","last_change":"2024-07-04T17:27:27.991828+0000","last_active":"2024-07-04T17:27:28.007066+0000","last_peered":"2024-07-04T17:27:28.007066+0000","last_clean":"2024-07-04T17:27:28.007066+0000","last_became_active":"2024-07-04T17:27:27.991514+0000","last_became_peered":"2024-07-04T17:27:27.991514+0000","last_unstale":"2024-07-04T17:27:28.007066+0000","last_undegraded":"2024-07-04T17:27:28.007066+0000","last_fullsized":"2024-07-04T17:27:28.007066+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:29:39.649214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.969117+0000","last_change":"2024-07-04T17:27:07.883415+0000","last_active":"2024-07-04T17:27:27.969117+0000","last_peered":"2024-07-04T17:27:27.969117+0000","last_clean":"2024-07-04T17:27:27.969117+0000","last_became_active":"2024-07-04T17:27:07.878729+0000","last_became_peered":"2024-07-04T17:27:07.878729+0000","last_unstale":"2024-07-04T17:27:27.969117+0000","last_undegraded":"2024-07-04T17:27:27.969117+0000","last_fullsized":"2024-07-04T17:27:27.969117+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:47:16.318473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.015982+0000","last_change":"2024-07-04T17:27:28.000905+0000","last_active":"2024-07-04T17:27:28.015982+0000","last_peered":"2024-07-04T17:27:28.015982+0000","last_clean":"2024-07-04T17:27:28.015982+0000","last_became_active":"2024-07-04T17:27:28.000592+0000","last_became_peered":"2024-07-04T17:27:28.000592+0000","last_unstale":"2024-07-04T17:27:28.015982+0000","last_undegraded":"2024-07-04T17:27:28.015982+0000","last_fullsized":"2024-07-04T17:27:28.015982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:05:16.540103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.977161+0000","last_change":"2024-07-04T17:27:07.905295+0000","last_active":"2024-07-04T17:27:27.977161+0000","last_peered":"2024-07-04T17:27:27.977161+0000","last_clean":"2024-07-04T17:27:27.977161+0000","last_became_active":"2024-07-04T17:27:07.904234+0000","last_became_peered":"2024-07-04T17:27:07.904234+0000","last_unstale":"2024-07-04T17:27:27.977161+0000","last_undegraded":"2024-07-04T17:27:27.977161+0000","last_fullsized":"2024-07-04T17:27:27.977161+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:27:04.910602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.031512+0000","last_change":"2024-07-04T17:27:28.024141+0000","last_active":"2024-07-04T17:27:28.031512+0000","last_peered":"2024-07-04T17:27:28.031512+0000","last_clean":"2024-07-04T17:27:28.031512+0000","last_became_active":"2024-07-04T17:27:28.023845+0000","last_became_peered":"2024-07-04T17:27:28.023845+0000","last_unstale":"2024-07-04T17:27:28.031512+0000","last_undegraded":"2024-07-04T17:27:28.031512+0000","last_fullsized":"2024-07-04T17:27:28.031512+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:25:29.884373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.970911+0000","last_change":"2024-07-04T17:27:07.889861+0000","last_active":"2024-07-04T17:27:27.970911+0000","last_peered":"2024-07-04T17:27:27.970911+0000","last_clean":"2024-07-04T17:27:27.970911+0000","last_became_active":"2024-07-04T17:27:07.889003+0000","last_became_peered":"2024-07-04T17:27:07.889003+0000","last_unstale":"2024-07-04T17:27:27.970911+0000","last_undegraded":"2024-07-04T17:27:27.970911+0000","last_fullsized":"2024-07-04T17:27:27.970911+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:38:38.667113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.034412+0000","last_change":"2024-07-04T17:27:28.031644+0000","last_active":"2024-07-04T17:27:28.034412+0000","last_peered":"2024-07-04T17:27:28.034412+0000","last_clean":"2024-07-04T17:27:28.034412+0000","last_became_active":"2024-07-04T17:27:28.031351+0000","last_became_peered":"2024-07-04T17:27:28.031351+0000","last_unstale":"2024-07-04T17:27:28.034412+0000","last_undegraded":"2024-07-04T17:27:28.034412+0000","last_fullsized":"2024-07-04T17:27:28.034412+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:48:02.847129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.975314+0000","last_change":"2024-07-04T17:27:07.891626+0000","last_active":"2024-07-04T17:27:27.975314+0000","last_peered":"2024-07-04T17:27:27.975314+0000","last_clean":"2024-07-04T17:27:27.975314+0000","last_became_active":"2024-07-04T17:27:07.891070+0000","last_became_peered":"2024-07-04T17:27:07.891070+0000","last_unstale":"2024-07-04T17:27:27.975314+0000","last_undegraded":"2024-07-04T17:27:27.975314+0000","last_fullsized":"2024-07-04T17:27:27.975314+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:12:04.631448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.033426+0000","last_change":"2024-07-04T17:27:28.031048+0000","last_active":"2024-07-04T17:27:28.033426+0000","last_peered":"2024-07-04T17:27:28.033426+0000","last_clean":"2024-07-04T17:27:28.033426+0000","last_became_active":"2024-07-04T17:27:28.030694+0000","last_became_peered":"2024-07-04T17:27:28.030694+0000","last_unstale":"2024-07-04T17:27:28.033426+0000","last_undegraded":"2024-07-04T17:27:28.033426+0000","last_fullsized":"2024-07-04T17:27:28.033426+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:01:00.610669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.983130+0000","last_change":"2024-07-04T17:27:07.884739+0000","last_active":"2024-07-04T17:27:27.983130+0000","last_peered":"2024-07-04T17:27:27.983130+0000","last_clean":"2024-07-04T17:27:27.983130+0000","last_became_active":"2024-07-04T17:27:07.884425+0000","last_became_peered":"2024-07-04T17:27:07.884425+0000","last_unstale":"2024-07-04T17:27:27.983130+0000","last_undegraded":"2024-07-04T17:27:27.983130+0000","last_fullsized":"2024-07-04T17:27:27.983130+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:20:28.358639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.999011+0000","last_change":"2024-07-04T17:27:27.988801+0000","last_active":"2024-07-04T17:27:27.999011+0000","last_peered":"2024-07-04T17:27:27.999011+0000","last_clean":"2024-07-04T17:27:27.999011+0000","last_became_active":"2024-07-04T17:27:27.988456+0000","last_became_peered":"2024-07-04T17:27:27.988456+0000","last_unstale":"2024-07-04T17:27:27.999011+0000","last_undegraded":"2024-07-04T17:27:27.999011+0000","last_fullsized":"2024-07-04T17:27:27.999011+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:04:48.577717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.982863+0000","last_change":"2024-07-04T17:27:07.916245+0000","last_active":"2024-07-04T17:27:27.982863+0000","last_peered":"2024-07-04T17:27:27.982863+0000","last_clean":"2024-07-04T17:27:27.982863+0000","last_became_active":"2024-07-04T17:27:07.912310+0000","last_became_peered":"2024-07-04T17:27:07.912310+0000","last_unstale":"2024-07-04T17:27:27.982863+0000","last_undegraded":"2024-07-04T17:27:27.982863+0000","last_fullsized":"2024-07-04T17:27:27.982863+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:23.798405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":3},{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":4107,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738388,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31468,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340372,"statfs":{"total":96636764160,"available":96604540928,"internally_reserved":0,"allocated":286720,"data_stored":96246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738389,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":909312,"data_stored":690631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738387,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":872448,"data_stored":685044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:27:32.761 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:27:33.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.092+0000 7ff8b8c00640 1 -- 172.21.15.42:0/3209967630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8b406d580 msgr2=0x7ff8b406c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:33.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.092+0000 7ff8b8c00640 1 --2- 172.21.15.42:0/3209967630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8b406d580 0x7ff8b406c2c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4005fd0 tx=0x7ff8a402bb60 comp rx=0 tx=0).stop 2024-07-04T17:27:33.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.092+0000 7ff8b8c00640 1 -- 172.21.15.42:0/3209967630 shutdown_connections 2024-07-04T17:27:33.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.092+0000 7ff8b8c00640 1 --2- 172.21.15.42:0/3209967630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8b406d580 0x7ff8b406c2c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:33.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.092+0000 7ff8b8c00640 1 -- 172.21.15.42:0/3209967630 >> 172.21.15.42:0/3209967630 conn(0x7ff8b40bebc0 msgr2=0x7ff8b40befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:33.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.092+0000 7ff8b8c00640 1 -- 172.21.15.42:0/3209967630 shutdown_connections 2024-07-04T17:27:33.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.092+0000 7ff8b8c00640 1 -- 172.21.15.42:0/3209967630 wait complete. 2024-07-04T17:27:33.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.095+0000 7ff8b8c00640 1 Processor -- start 2024-07-04T17:27:33.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.095+0000 7ff8b8c00640 1 -- start start 2024-07-04T17:27:33.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.097+0000 7ff8b8c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8b406d580 0x7ff8b41d35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:33.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.097+0000 7ff8b8c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b40b2810 con 0x7ff8b406d580 2024-07-04T17:27:33.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.097+0000 7ff8b3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8b406d580 0x7ff8b41d35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:33.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.097+0000 7ff8b3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8b406d580 0x7ff8b41d35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36740/0 (socket says 172.21.15.42:36740) 2024-07-04T17:27:33.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.097+0000 7ff8b3400640 1 -- 172.21.15.42:0/3062482643 learned_addr learned my addr 172.21.15.42:0/3062482643 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:33.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.098+0000 7ff8b3400640 1 -- 172.21.15.42:0/3062482643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b40b8fe0 con 0x7ff8b406d580 2024-07-04T17:27:33.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.098+0000 7ff8b3400640 1 --2- 172.21.15.42:0/3062482643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8b406d580 0x7ff8b41d35d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4006010 tx=0x7ff8a402ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:33.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.098+0000 7ff8ab400640 1 -- 172.21.15.42:0/3062482643 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8a4041020 con 0x7ff8b406d580 2024-07-04T17:27:33.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.099+0000 7ff8ab400640 1 -- 172.21.15.42:0/3062482643 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8a4004260 con 0x7ff8b406d580 2024-07-04T17:27:33.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.099+0000 7ff8ab400640 1 -- 172.21.15.42:0/3062482643 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8a40045e0 con 0x7ff8b406d580 2024-07-04T17:27:33.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.099+0000 7ff8b8c00640 1 -- 172.21.15.42:0/3062482643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b40adca0 con 0x7ff8b406d580 2024-07-04T17:27:33.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.099+0000 7ff8b8c00640 1 -- 172.21.15.42:0/3062482643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b40ad8e0 con 0x7ff8b406d580 2024-07-04T17:27:33.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.100+0000 7ff8b8c00640 1 -- 172.21.15.42:0/3062482643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8b406c2c0 con 0x7ff8b406d580 2024-07-04T17:27:33.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.100+0000 7ff8ab400640 1 -- 172.21.15.42:0/3062482643 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7ff8a4044020 con 0x7ff8b406d580 2024-07-04T17:27:33.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.101+0000 7ff8ab400640 1 --2- 172.21.15.42:0/3062482643 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff89403d2b0 0x7ff89403f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:33.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.101+0000 7ff8ab400640 1 -- 172.21.15.42:0/3062482643 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4303+0+0 (secure 0 0 0) 0x7ff8a403a040 con 0x7ff8b406d580 2024-07-04T17:27:33.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.101+0000 7ff8b2a00640 1 --2- 172.21.15.42:0/3062482643 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff89403d2b0 0x7ff89403f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:33.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.102+0000 7ff8b2a00640 1 --2- 172.21.15.42:0/3062482643 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff89403d2b0 0x7ff89403f7c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0001560 tx=0x7ff8a0007a90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:33.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.106+0000 7ff8ab400640 1 -- 172.21.15.42:0/3062482643 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7ff8a404b070 con 0x7ff8b406d580 2024-07-04T17:27:33.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.312+0000 7ff8b8c00640 1 -- 172.21.15.42:0/3062482643 --> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff8b41d4250 con 0x7ff89403d2b0 2024-07-04T17:27:33.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.335+0000 7ff8ab400640 1 -- 172.21.15.42:0/3062482643 <== mgr.4103 v2:172.21.15.42:6824/3523793950 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+482153 (secure 0 0 0) 0x7ff8b41d4250 con 0x7ff89403d2b0 2024-07-04T17:27:33.339 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:27:33.347 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:27:33.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.343+0000 7ff8b8c00640 1 -- 172.21.15.42:0/3062482643 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff89403d2b0 msgr2=0x7ff89403f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:33.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.343+0000 7ff8b8c00640 1 --2- 172.21.15.42:0/3062482643 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff89403d2b0 0x7ff89403f7c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0001560 tx=0x7ff8a0007a90 comp rx=0 tx=0).stop 2024-07-04T17:27:33.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.343+0000 7ff8b8c00640 1 -- 172.21.15.42:0/3062482643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8b406d580 msgr2=0x7ff8b41d35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:33.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.343+0000 7ff8b8c00640 1 --2- 172.21.15.42:0/3062482643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8b406d580 0x7ff8b41d35d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4006010 tx=0x7ff8a402ef90 comp rx=0 tx=0).stop 2024-07-04T17:27:33.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.344+0000 7ff8b8c00640 1 -- 172.21.15.42:0/3062482643 shutdown_connections 2024-07-04T17:27:33.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.344+0000 7ff8b8c00640 1 --2- 172.21.15.42:0/3062482643 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7ff89403d2b0 0x7ff89403f7c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:33.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.344+0000 7ff8b8c00640 1 --2- 172.21.15.42:0/3062482643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8b406d580 0x7ff8b41d35d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:33.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.344+0000 7ff8b8c00640 1 -- 172.21.15.42:0/3062482643 >> 172.21.15.42:0/3062482643 conn(0x7ff8b40bebc0 msgr2=0x7ff8b40b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:33.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.344+0000 7ff8b8c00640 1 -- 172.21.15.42:0/3062482643 shutdown_connections 2024-07-04T17:27:33.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.344+0000 7ff8b8c00640 1 -- 172.21.15.42:0/3062482643 wait complete. 2024-07-04T17:27:33.374 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-07-04T17:27:33.287554+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596885,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":171,"num_write_kb":1137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":206,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":95572,"kb_used_data":2020,"kb_used_omap":4,"kb_used_meta":93499,"kb_avail":283019948,"statfs":{"total":289910292480,"available":289812426752,"internally_reserved":0,"allocated":2068480,"data_stored":1471921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":95743311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.354416"},"pg_stats":[{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.016871+0000","last_change":"2024-07-04T17:27:28.000192+0000","last_active":"2024-07-04T17:27:28.016871+0000","last_peered":"2024-07-04T17:27:28.016871+0000","last_clean":"2024-07-04T17:27:28.016871+0000","last_became_active":"2024-07-04T17:27:27.999880+0000","last_became_peered":"2024-07-04T17:27:27.999880+0000","last_unstale":"2024-07-04T17:27:28.016871+0000","last_undegraded":"2024-07-04T17:27:28.016871+0000","last_fullsized":"2024-07-04T17:27:28.016871+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:32:32.138712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"10'52","reported_seq":115,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.971855+0000","last_change":"2024-07-04T17:26:04.953559+0000","last_active":"2024-07-04T17:27:27.971855+0000","last_peered":"2024-07-04T17:27:27.971855+0000","last_clean":"2024-07-04T17:27:27.971855+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:27:27.971855+0000","last_undegraded":"2024-07-04T17:27:27.971855+0000","last_fullsized":"2024-07-04T17:27:27.971855+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_clean_scrub_stamp":"2024-07-04T17:26:03.933432+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.064181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.974655+0000","last_change":"2024-07-04T17:27:06.874596+0000","last_active":"2024-07-04T17:27:27.974655+0000","last_peered":"2024-07-04T17:27:27.974655+0000","last_clean":"2024-07-04T17:27:27.974655+0000","last_became_active":"2024-07-04T17:27:06.874252+0000","last_became_peered":"2024-07-04T17:27:06.874252+0000","last_unstale":"2024-07-04T17:27:27.974655+0000","last_undegraded":"2024-07-04T17:27:27.974655+0000","last_fullsized":"2024-07-04T17:27:27.974655+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:17:38.211735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.968322+0000","last_change":"2024-07-04T17:27:07.891018+0000","last_active":"2024-07-04T17:27:27.968322+0000","last_peered":"2024-07-04T17:27:27.968322+0000","last_clean":"2024-07-04T17:27:27.968322+0000","last_became_active":"2024-07-04T17:27:07.889885+0000","last_became_peered":"2024-07-04T17:27:07.889885+0000","last_unstale":"2024-07-04T17:27:27.968322+0000","last_undegraded":"2024-07-04T17:27:27.968322+0000","last_fullsized":"2024-07-04T17:27:27.968322+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:59:08.231146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"30'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.032442+0000","last_change":"2024-07-04T17:27:26.992615+0000","last_active":"2024-07-04T17:27:28.032442+0000","last_peered":"2024-07-04T17:27:28.032442+0000","last_clean":"2024-07-04T17:27:28.032442+0000","last_became_active":"2024-07-04T17:27:26.992123+0000","last_became_peered":"2024-07-04T17:27:26.992123+0000","last_unstale":"2024-07-04T17:27:28.032442+0000","last_undegraded":"2024-07-04T17:27:28.032442+0000","last_fullsized":"2024-07-04T17:27:28.032442+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:11:50.417822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.016336+0000","last_change":"2024-07-04T17:27:27.999411+0000","last_active":"2024-07-04T17:27:28.016336+0000","last_peered":"2024-07-04T17:27:28.016336+0000","last_clean":"2024-07-04T17:27:28.016336+0000","last_became_active":"2024-07-04T17:27:27.994185+0000","last_became_peered":"2024-07-04T17:27:27.994185+0000","last_unstale":"2024-07-04T17:27:28.016336+0000","last_undegraded":"2024-07-04T17:27:28.016336+0000","last_fullsized":"2024-07-04T17:27:28.016336+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:56:44.529403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.968251+0000","last_change":"2024-07-04T17:27:06.859760+0000","last_active":"2024-07-04T17:27:27.968251+0000","last_peered":"2024-07-04T17:27:27.968251+0000","last_clean":"2024-07-04T17:27:27.968251+0000","last_became_active":"2024-07-04T17:27:06.859396+0000","last_became_peered":"2024-07-04T17:27:06.859396+0000","last_unstale":"2024-07-04T17:27:27.968251+0000","last_undegraded":"2024-07-04T17:27:27.968251+0000","last_fullsized":"2024-07-04T17:27:27.968251+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:48:12.553882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.964861+0000","last_change":"2024-07-04T17:27:07.871918+0000","last_active":"2024-07-04T17:27:27.964861+0000","last_peered":"2024-07-04T17:27:27.964861+0000","last_clean":"2024-07-04T17:27:27.964861+0000","last_became_active":"2024-07-04T17:27:07.871229+0000","last_became_peered":"2024-07-04T17:27:07.871229+0000","last_unstale":"2024-07-04T17:27:27.964861+0000","last_undegraded":"2024-07-04T17:27:27.964861+0000","last_fullsized":"2024-07-04T17:27:27.964861+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:00:11.500712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.969486+0000","last_change":"2024-07-04T17:27:26.989371+0000","last_active":"2024-07-04T17:27:27.969486+0000","last_peered":"2024-07-04T17:27:27.969486+0000","last_clean":"2024-07-04T17:27:27.969486+0000","last_became_active":"2024-07-04T17:27:26.988886+0000","last_became_peered":"2024-07-04T17:27:26.988886+0000","last_unstale":"2024-07-04T17:27:27.969486+0000","last_undegraded":"2024-07-04T17:27:27.969486+0000","last_fullsized":"2024-07-04T17:27:27.969486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:59:27.331368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.026664+0000","last_change":"2024-07-04T17:27:28.003755+0000","last_active":"2024-07-04T17:27:28.026664+0000","last_peered":"2024-07-04T17:27:28.026664+0000","last_clean":"2024-07-04T17:27:28.026664+0000","last_became_active":"2024-07-04T17:27:28.002942+0000","last_became_peered":"2024-07-04T17:27:28.002942+0000","last_unstale":"2024-07-04T17:27:28.026664+0000","last_undegraded":"2024-07-04T17:27:28.026664+0000","last_fullsized":"2024-07-04T17:27:28.026664+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:53:22.051231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.971736+0000","last_change":"2024-07-04T17:27:06.883045+0000","last_active":"2024-07-04T17:27:27.971736+0000","last_peered":"2024-07-04T17:27:27.971736+0000","last_clean":"2024-07-04T17:27:27.971736+0000","last_became_active":"2024-07-04T17:27:06.882363+0000","last_became_peered":"2024-07-04T17:27:06.882363+0000","last_unstale":"2024-07-04T17:27:27.971736+0000","last_undegraded":"2024-07-04T17:27:27.971736+0000","last_fullsized":"2024-07-04T17:27:27.971736+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:05:41.513762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.967990+0000","last_change":"2024-07-04T17:27:07.909121+0000","last_active":"2024-07-04T17:27:27.967990+0000","last_peered":"2024-07-04T17:27:27.967990+0000","last_clean":"2024-07-04T17:27:27.967990+0000","last_became_active":"2024-07-04T17:27:07.908646+0000","last_became_peered":"2024-07-04T17:27:07.908646+0000","last_unstale":"2024-07-04T17:27:27.967990+0000","last_undegraded":"2024-07-04T17:27:27.967990+0000","last_fullsized":"2024-07-04T17:27:27.967990+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:34:46.000005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.991395+0000","last_change":"2024-07-04T17:27:26.992253+0000","last_active":"2024-07-04T17:27:27.991395+0000","last_peered":"2024-07-04T17:27:27.991395+0000","last_clean":"2024-07-04T17:27:27.991395+0000","last_became_active":"2024-07-04T17:27:26.991917+0000","last_became_peered":"2024-07-04T17:27:26.991917+0000","last_unstale":"2024-07-04T17:27:27.991395+0000","last_undegraded":"2024-07-04T17:27:27.991395+0000","last_fullsized":"2024-07-04T17:27:27.991395+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:08:43.921506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.017355+0000","last_change":"2024-07-04T17:27:28.002245+0000","last_active":"2024-07-04T17:27:28.017355+0000","last_peered":"2024-07-04T17:27:28.017355+0000","last_clean":"2024-07-04T17:27:28.017355+0000","last_became_active":"2024-07-04T17:27:28.000808+0000","last_became_peered":"2024-07-04T17:27:28.000808+0000","last_unstale":"2024-07-04T17:27:28.017355+0000","last_undegraded":"2024-07-04T17:27:28.017355+0000","last_fullsized":"2024-07-04T17:27:28.017355+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:24:20.430216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.966797+0000","last_change":"2024-07-04T17:27:06.861673+0000","last_active":"2024-07-04T17:27:27.966797+0000","last_peered":"2024-07-04T17:27:27.966797+0000","last_clean":"2024-07-04T17:27:27.966797+0000","last_became_active":"2024-07-04T17:27:06.861297+0000","last_became_peered":"2024-07-04T17:27:06.861297+0000","last_unstale":"2024-07-04T17:27:27.966797+0000","last_undegraded":"2024-07-04T17:27:27.966797+0000","last_fullsized":"2024-07-04T17:27:27.966797+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:10:34.165745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.982964+0000","last_change":"2024-07-04T17:27:07.882615+0000","last_active":"2024-07-04T17:27:27.982964+0000","last_peered":"2024-07-04T17:27:27.982964+0000","last_clean":"2024-07-04T17:27:27.982964+0000","last_became_active":"2024-07-04T17:27:07.882132+0000","last_became_peered":"2024-07-04T17:27:07.882132+0000","last_unstale":"2024-07-04T17:27:27.982964+0000","last_undegraded":"2024-07-04T17:27:27.982964+0000","last_fullsized":"2024-07-04T17:27:27.982964+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:45:52.813695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.958026+0000","last_change":"2024-07-04T17:27:26.986473+0000","last_active":"2024-07-04T17:27:27.958026+0000","last_peered":"2024-07-04T17:27:27.958026+0000","last_clean":"2024-07-04T17:27:27.958026+0000","last_became_active":"2024-07-04T17:27:26.986137+0000","last_became_peered":"2024-07-04T17:27:26.986137+0000","last_unstale":"2024-07-04T17:27:27.958026+0000","last_undegraded":"2024-07-04T17:27:27.958026+0000","last_fullsized":"2024-07-04T17:27:27.958026+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:31:33.657276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.998766+0000","last_change":"2024-07-04T17:27:27.984855+0000","last_active":"2024-07-04T17:27:27.998766+0000","last_peered":"2024-07-04T17:27:27.998766+0000","last_clean":"2024-07-04T17:27:27.998766+0000","last_became_active":"2024-07-04T17:27:27.984390+0000","last_became_peered":"2024-07-04T17:27:27.984390+0000","last_unstale":"2024-07-04T17:27:27.998766+0000","last_undegraded":"2024-07-04T17:27:27.998766+0000","last_fullsized":"2024-07-04T17:27:27.998766+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:53:07.282774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.982594+0000","last_change":"2024-07-04T17:27:06.870890+0000","last_active":"2024-07-04T17:27:27.982594+0000","last_peered":"2024-07-04T17:27:27.982594+0000","last_clean":"2024-07-04T17:27:27.982594+0000","last_became_active":"2024-07-04T17:27:06.870581+0000","last_became_peered":"2024-07-04T17:27:06.870581+0000","last_unstale":"2024-07-04T17:27:27.982594+0000","last_undegraded":"2024-07-04T17:27:27.982594+0000","last_fullsized":"2024-07-04T17:27:27.982594+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:55:15.415040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.957418+0000","last_change":"2024-07-04T17:27:07.878014+0000","last_active":"2024-07-04T17:27:27.957418+0000","last_peered":"2024-07-04T17:27:27.957418+0000","last_clean":"2024-07-04T17:27:27.957418+0000","last_became_active":"2024-07-04T17:27:07.874485+0000","last_became_peered":"2024-07-04T17:27:07.874485+0000","last_unstale":"2024-07-04T17:27:27.957418+0000","last_undegraded":"2024-07-04T17:27:27.957418+0000","last_fullsized":"2024-07-04T17:27:27.957418+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:55:10.737185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.967712+0000","last_change":"2024-07-04T17:27:26.983805+0000","last_active":"2024-07-04T17:27:27.967712+0000","last_peered":"2024-07-04T17:27:27.967712+0000","last_clean":"2024-07-04T17:27:27.967712+0000","last_became_active":"2024-07-04T17:27:26.982536+0000","last_became_peered":"2024-07-04T17:27:26.982536+0000","last_unstale":"2024-07-04T17:27:27.967712+0000","last_undegraded":"2024-07-04T17:27:27.967712+0000","last_fullsized":"2024-07-04T17:27:27.967712+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:44:50.168355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.000829+0000","last_change":"2024-07-04T17:27:27.992927+0000","last_active":"2024-07-04T17:27:28.000829+0000","last_peered":"2024-07-04T17:27:28.000829+0000","last_clean":"2024-07-04T17:27:28.000829+0000","last_became_active":"2024-07-04T17:27:27.992276+0000","last_became_peered":"2024-07-04T17:27:27.992276+0000","last_unstale":"2024-07-04T17:27:28.000829+0000","last_undegraded":"2024-07-04T17:27:28.000829+0000","last_fullsized":"2024-07-04T17:27:28.000829+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:25:29.884373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.969998+0000","last_change":"2024-07-04T17:27:06.872489+0000","last_active":"2024-07-04T17:27:27.969998+0000","last_peered":"2024-07-04T17:27:27.969998+0000","last_clean":"2024-07-04T17:27:27.969998+0000","last_became_active":"2024-07-04T17:27:06.872125+0000","last_became_peered":"2024-07-04T17:27:06.872125+0000","last_unstale":"2024-07-04T17:27:27.969998+0000","last_undegraded":"2024-07-04T17:27:27.969998+0000","last_fullsized":"2024-07-04T17:27:27.969998+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:10:29.469978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.974496+0000","last_change":"2024-07-04T17:27:07.883459+0000","last_active":"2024-07-04T17:27:27.974496+0000","last_peered":"2024-07-04T17:27:27.974496+0000","last_clean":"2024-07-04T17:27:27.974496+0000","last_became_active":"2024-07-04T17:27:07.883202+0000","last_became_peered":"2024-07-04T17:27:07.883202+0000","last_unstale":"2024-07-04T17:27:27.974496+0000","last_undegraded":"2024-07-04T17:27:27.974496+0000","last_fullsized":"2024-07-04T17:27:27.974496+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:29:49.821609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.976708+0000","last_change":"2024-07-04T17:27:26.982200+0000","last_active":"2024-07-04T17:27:27.976708+0000","last_peered":"2024-07-04T17:27:27.976708+0000","last_clean":"2024-07-04T17:27:27.976708+0000","last_became_active":"2024-07-04T17:27:26.981617+0000","last_became_peered":"2024-07-04T17:27:26.981617+0000","last_unstale":"2024-07-04T17:27:27.976708+0000","last_undegraded":"2024-07-04T17:27:27.976708+0000","last_fullsized":"2024-07-04T17:27:27.976708+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:48:22.086567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.031197+0000","last_change":"2024-07-04T17:27:28.026525+0000","last_active":"2024-07-04T17:27:28.031197+0000","last_peered":"2024-07-04T17:27:28.031197+0000","last_clean":"2024-07-04T17:27:28.031197+0000","last_became_active":"2024-07-04T17:27:28.026192+0000","last_became_peered":"2024-07-04T17:27:28.026192+0000","last_unstale":"2024-07-04T17:27:28.031197+0000","last_undegraded":"2024-07-04T17:27:28.031197+0000","last_fullsized":"2024-07-04T17:27:28.031197+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:44:15.882179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"27'1","reported_seq":21,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.959431+0000","last_change":"2024-07-04T17:27:06.866198+0000","last_active":"2024-07-04T17:27:27.959431+0000","last_peered":"2024-07-04T17:27:27.959431+0000","last_clean":"2024-07-04T17:27:27.959431+0000","last_became_active":"2024-07-04T17:27:06.865891+0000","last_became_peered":"2024-07-04T17:27:06.865891+0000","last_unstale":"2024-07-04T17:27:27.959431+0000","last_undegraded":"2024-07-04T17:27:27.959431+0000","last_fullsized":"2024-07-04T17:27:27.959431+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:59:47.703820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.978168+0000","last_change":"2024-07-04T17:27:07.882402+0000","last_active":"2024-07-04T17:27:27.978168+0000","last_peered":"2024-07-04T17:27:27.978168+0000","last_clean":"2024-07-04T17:27:27.978168+0000","last_became_active":"2024-07-04T17:27:07.882061+0000","last_became_peered":"2024-07-04T17:27:07.882061+0000","last_unstale":"2024-07-04T17:27:27.978168+0000","last_undegraded":"2024-07-04T17:27:27.978168+0000","last_fullsized":"2024-07-04T17:27:27.978168+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:49:59.324212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.971233+0000","last_change":"2024-07-04T17:27:26.986939+0000","last_active":"2024-07-04T17:27:27.971233+0000","last_peered":"2024-07-04T17:27:27.971233+0000","last_clean":"2024-07-04T17:27:27.971233+0000","last_became_active":"2024-07-04T17:27:26.986582+0000","last_became_peered":"2024-07-04T17:27:26.986582+0000","last_unstale":"2024-07-04T17:27:27.971233+0000","last_undegraded":"2024-07-04T17:27:27.971233+0000","last_fullsized":"2024-07-04T17:27:27.971233+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:55:06.278743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.015152+0000","last_change":"2024-07-04T17:27:27.998065+0000","last_active":"2024-07-04T17:27:28.015152+0000","last_peered":"2024-07-04T17:27:28.015152+0000","last_clean":"2024-07-04T17:27:28.015152+0000","last_became_active":"2024-07-04T17:27:27.997759+0000","last_became_peered":"2024-07-04T17:27:27.997759+0000","last_unstale":"2024-07-04T17:27:28.015152+0000","last_undegraded":"2024-07-04T17:27:28.015152+0000","last_fullsized":"2024-07-04T17:27:28.015152+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:06:51.782470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"26'2","reported_seq":22,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.976250+0000","last_change":"2024-07-04T17:27:06.865834+0000","last_active":"2024-07-04T17:27:27.976250+0000","last_peered":"2024-07-04T17:27:27.976250+0000","last_clean":"2024-07-04T17:27:27.976250+0000","last_became_active":"2024-07-04T17:27:06.865463+0000","last_became_peered":"2024-07-04T17:27:06.865463+0000","last_unstale":"2024-07-04T17:27:27.976250+0000","last_undegraded":"2024-07-04T17:27:27.976250+0000","last_fullsized":"2024-07-04T17:27:27.976250+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:35:43.457323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.982344+0000","last_change":"2024-07-04T17:27:07.911944+0000","last_active":"2024-07-04T17:27:27.982344+0000","last_peered":"2024-07-04T17:27:27.982344+0000","last_clean":"2024-07-04T17:27:27.982344+0000","last_became_active":"2024-07-04T17:27:07.911639+0000","last_became_peered":"2024-07-04T17:27:07.911639+0000","last_unstale":"2024-07-04T17:27:27.982344+0000","last_undegraded":"2024-07-04T17:27:27.982344+0000","last_fullsized":"2024-07-04T17:27:27.982344+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:56:08.436905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.975139+0000","last_change":"2024-07-04T17:27:26.988445+0000","last_active":"2024-07-04T17:27:27.975139+0000","last_peered":"2024-07-04T17:27:27.975139+0000","last_clean":"2024-07-04T17:27:27.975139+0000","last_became_active":"2024-07-04T17:27:26.986965+0000","last_became_peered":"2024-07-04T17:27:26.986965+0000","last_unstale":"2024-07-04T17:27:27.975139+0000","last_undegraded":"2024-07-04T17:27:27.975139+0000","last_fullsized":"2024-07-04T17:27:27.975139+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:37:50.824328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.999020+0000","last_change":"2024-07-04T17:27:27.983773+0000","last_active":"2024-07-04T17:27:27.999020+0000","last_peered":"2024-07-04T17:27:27.999020+0000","last_clean":"2024-07-04T17:27:27.999020+0000","last_became_active":"2024-07-04T17:27:27.983392+0000","last_became_peered":"2024-07-04T17:27:27.983392+0000","last_unstale":"2024-07-04T17:27:27.999020+0000","last_undegraded":"2024-07-04T17:27:27.999020+0000","last_fullsized":"2024-07-04T17:27:27.999020+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:47:31.123344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.971657+0000","last_change":"2024-07-04T17:27:06.871847+0000","last_active":"2024-07-04T17:27:27.971657+0000","last_peered":"2024-07-04T17:27:27.971657+0000","last_clean":"2024-07-04T17:27:27.971657+0000","last_became_active":"2024-07-04T17:27:06.871475+0000","last_became_peered":"2024-07-04T17:27:06.871475+0000","last_unstale":"2024-07-04T17:27:27.971657+0000","last_undegraded":"2024-07-04T17:27:27.971657+0000","last_fullsized":"2024-07-04T17:27:27.971657+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:09:28.414254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.965007+0000","last_change":"2024-07-04T17:27:07.872007+0000","last_active":"2024-07-04T17:27:27.965007+0000","last_peered":"2024-07-04T17:27:27.965007+0000","last_clean":"2024-07-04T17:27:27.965007+0000","last_became_active":"2024-07-04T17:27:07.871647+0000","last_became_peered":"2024-07-04T17:27:07.871647+0000","last_unstale":"2024-07-04T17:27:27.965007+0000","last_undegraded":"2024-07-04T17:27:27.965007+0000","last_fullsized":"2024-07-04T17:27:27.965007+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:17:48.892420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.982841+0000","last_change":"2024-07-04T17:27:26.985765+0000","last_active":"2024-07-04T17:27:27.982841+0000","last_peered":"2024-07-04T17:27:27.982841+0000","last_clean":"2024-07-04T17:27:27.982841+0000","last_became_active":"2024-07-04T17:27:26.984981+0000","last_became_peered":"2024-07-04T17:27:26.984981+0000","last_unstale":"2024-07-04T17:27:27.982841+0000","last_undegraded":"2024-07-04T17:27:27.982841+0000","last_fullsized":"2024-07-04T17:27:27.982841+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:45:58.822013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.031124+0000","last_change":"2024-07-04T17:27:28.022666+0000","last_active":"2024-07-04T17:27:28.031124+0000","last_peered":"2024-07-04T17:27:28.031124+0000","last_clean":"2024-07-04T17:27:28.031124+0000","last_became_active":"2024-07-04T17:27:28.021315+0000","last_became_peered":"2024-07-04T17:27:28.021315+0000","last_unstale":"2024-07-04T17:27:28.031124+0000","last_undegraded":"2024-07-04T17:27:28.031124+0000","last_fullsized":"2024-07-04T17:27:28.031124+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:37:40.507481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.964714+0000","last_change":"2024-07-04T17:27:06.868310+0000","last_active":"2024-07-04T17:27:27.964714+0000","last_peered":"2024-07-04T17:27:27.964714+0000","last_clean":"2024-07-04T17:27:27.964714+0000","last_became_active":"2024-07-04T17:27:06.866928+0000","last_became_peered":"2024-07-04T17:27:06.866928+0000","last_unstale":"2024-07-04T17:27:27.964714+0000","last_undegraded":"2024-07-04T17:27:27.964714+0000","last_fullsized":"2024-07-04T17:27:27.964714+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:31:45.030996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.968135+0000","last_change":"2024-07-04T17:27:07.905869+0000","last_active":"2024-07-04T17:27:27.968135+0000","last_peered":"2024-07-04T17:27:27.968135+0000","last_clean":"2024-07-04T17:27:27.968135+0000","last_became_active":"2024-07-04T17:27:07.905559+0000","last_became_peered":"2024-07-04T17:27:07.905559+0000","last_unstale":"2024-07-04T17:27:27.968135+0000","last_undegraded":"2024-07-04T17:27:27.968135+0000","last_fullsized":"2024-07-04T17:27:27.968135+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:00:11.500712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"30'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.014114+0000","last_change":"2024-07-04T17:27:26.990435+0000","last_active":"2024-07-04T17:27:28.014114+0000","last_peered":"2024-07-04T17:27:28.014114+0000","last_clean":"2024-07-04T17:27:28.014114+0000","last_became_active":"2024-07-04T17:27:26.989929+0000","last_became_peered":"2024-07-04T17:27:26.989929+0000","last_unstale":"2024-07-04T17:27:28.014114+0000","last_undegraded":"2024-07-04T17:27:28.014114+0000","last_fullsized":"2024-07-04T17:27:28.014114+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:55:04.441377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.020983+0000","last_change":"2024-07-04T17:27:28.013631+0000","last_active":"2024-07-04T17:27:28.020983+0000","last_peered":"2024-07-04T17:27:28.020983+0000","last_clean":"2024-07-04T17:27:28.020983+0000","last_became_active":"2024-07-04T17:27:28.011264+0000","last_became_peered":"2024-07-04T17:27:28.011264+0000","last_unstale":"2024-07-04T17:27:28.020983+0000","last_undegraded":"2024-07-04T17:27:28.020983+0000","last_fullsized":"2024-07-04T17:27:28.020983+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:41:35.479418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.971470+0000","last_change":"2024-07-04T17:27:06.885598+0000","last_active":"2024-07-04T17:27:27.971470+0000","last_peered":"2024-07-04T17:27:27.971470+0000","last_clean":"2024-07-04T17:27:27.971470+0000","last_became_active":"2024-07-04T17:27:06.885220+0000","last_became_peered":"2024-07-04T17:27:06.885220+0000","last_unstale":"2024-07-04T17:27:27.971470+0000","last_undegraded":"2024-07-04T17:27:27.971470+0000","last_fullsized":"2024-07-04T17:27:27.971470+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:49:35.335628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.977849+0000","last_change":"2024-07-04T17:27:07.904396+0000","last_active":"2024-07-04T17:27:27.977849+0000","last_peered":"2024-07-04T17:27:27.977849+0000","last_clean":"2024-07-04T17:27:27.977849+0000","last_became_active":"2024-07-04T17:27:07.903961+0000","last_became_peered":"2024-07-04T17:27:07.903961+0000","last_unstale":"2024-07-04T17:27:27.977849+0000","last_undegraded":"2024-07-04T17:27:27.977849+0000","last_fullsized":"2024-07-04T17:27:27.977849+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:59:48.713747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.969758+0000","last_change":"2024-07-04T17:27:26.980557+0000","last_active":"2024-07-04T17:27:27.969758+0000","last_peered":"2024-07-04T17:27:27.969758+0000","last_clean":"2024-07-04T17:27:27.969758+0000","last_became_active":"2024-07-04T17:27:26.979421+0000","last_became_peered":"2024-07-04T17:27:26.979421+0000","last_unstale":"2024-07-04T17:27:27.969758+0000","last_undegraded":"2024-07-04T17:27:27.969758+0000","last_fullsized":"2024-07-04T17:27:27.969758+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:53:58.600996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.003470+0000","last_change":"2024-07-04T17:27:27.998798+0000","last_active":"2024-07-04T17:27:28.003470+0000","last_peered":"2024-07-04T17:27:28.003470+0000","last_clean":"2024-07-04T17:27:28.003470+0000","last_became_active":"2024-07-04T17:27:27.998444+0000","last_became_peered":"2024-07-04T17:27:27.998444+0000","last_unstale":"2024-07-04T17:27:28.003470+0000","last_undegraded":"2024-07-04T17:27:28.003470+0000","last_fullsized":"2024-07-04T17:27:28.003470+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:20:48.469282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"26'2","reported_seq":22,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.982842+0000","last_change":"2024-07-04T17:27:06.859869+0000","last_active":"2024-07-04T17:27:27.982842+0000","last_peered":"2024-07-04T17:27:27.982842+0000","last_clean":"2024-07-04T17:27:27.982842+0000","last_became_active":"2024-07-04T17:27:06.859475+0000","last_became_peered":"2024-07-04T17:27:06.859475+0000","last_unstale":"2024-07-04T17:27:27.982842+0000","last_undegraded":"2024-07-04T17:27:27.982842+0000","last_fullsized":"2024-07-04T17:27:27.982842+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:00:10.490785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.974216+0000","last_change":"2024-07-04T17:27:07.905531+0000","last_active":"2024-07-04T17:27:27.974216+0000","last_peered":"2024-07-04T17:27:27.974216+0000","last_clean":"2024-07-04T17:27:27.974216+0000","last_became_active":"2024-07-04T17:27:07.903880+0000","last_became_peered":"2024-07-04T17:27:07.903880+0000","last_unstale":"2024-07-04T17:27:27.974216+0000","last_undegraded":"2024-07-04T17:27:27.974216+0000","last_fullsized":"2024-07-04T17:27:27.974216+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:21:38.559811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.968059+0000","last_change":"2024-07-04T17:27:26.993149+0000","last_active":"2024-07-04T17:27:27.968059+0000","last_peered":"2024-07-04T17:27:27.968059+0000","last_clean":"2024-07-04T17:27:27.968059+0000","last_became_active":"2024-07-04T17:27:26.987822+0000","last_became_peered":"2024-07-04T17:27:26.987822+0000","last_unstale":"2024-07-04T17:27:27.968059+0000","last_undegraded":"2024-07-04T17:27:27.968059+0000","last_fullsized":"2024-07-04T17:27:27.968059+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:54:26.165578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.016651+0000","last_change":"2024-07-04T17:27:28.010844+0000","last_active":"2024-07-04T17:27:28.016651+0000","last_peered":"2024-07-04T17:27:28.016651+0000","last_clean":"2024-07-04T17:27:28.016651+0000","last_became_active":"2024-07-04T17:27:28.010550+0000","last_became_peered":"2024-07-04T17:27:28.010550+0000","last_unstale":"2024-07-04T17:27:28.016651+0000","last_undegraded":"2024-07-04T17:27:28.016651+0000","last_fullsized":"2024-07-04T17:27:28.016651+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:15:30.121052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.958711+0000","last_change":"2024-07-04T17:27:06.859888+0000","last_active":"2024-07-04T17:27:27.958711+0000","last_peered":"2024-07-04T17:27:27.958711+0000","last_clean":"2024-07-04T17:27:27.958711+0000","last_became_active":"2024-07-04T17:27:06.859565+0000","last_became_peered":"2024-07-04T17:27:06.859565+0000","last_unstale":"2024-07-04T17:27:27.958711+0000","last_undegraded":"2024-07-04T17:27:27.958711+0000","last_fullsized":"2024-07-04T17:27:27.958711+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:25:51.634073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.980198+0000","last_change":"2024-07-04T17:27:07.912224+0000","last_active":"2024-07-04T17:27:27.980198+0000","last_peered":"2024-07-04T17:27:27.980198+0000","last_clean":"2024-07-04T17:27:27.980198+0000","last_became_active":"2024-07-04T17:27:07.911921+0000","last_became_peered":"2024-07-04T17:27:07.911921+0000","last_unstale":"2024-07-04T17:27:27.980198+0000","last_undegraded":"2024-07-04T17:27:27.980198+0000","last_fullsized":"2024-07-04T17:27:27.980198+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:12:04.631448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"30'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.020350+0000","last_change":"2024-07-04T17:27:26.981903+0000","last_active":"2024-07-04T17:27:28.020350+0000","last_peered":"2024-07-04T17:27:28.020350+0000","last_clean":"2024-07-04T17:27:28.020350+0000","last_became_active":"2024-07-04T17:27:26.981211+0000","last_became_peered":"2024-07-04T17:27:26.981211+0000","last_unstale":"2024-07-04T17:27:28.020350+0000","last_undegraded":"2024-07-04T17:27:28.020350+0000","last_fullsized":"2024-07-04T17:27:28.020350+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:32.422119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.026775+0000","last_change":"2024-07-04T17:27:28.007392+0000","last_active":"2024-07-04T17:27:28.026775+0000","last_peered":"2024-07-04T17:27:28.026775+0000","last_clean":"2024-07-04T17:27:28.026775+0000","last_became_active":"2024-07-04T17:27:28.003205+0000","last_became_peered":"2024-07-04T17:27:28.003205+0000","last_unstale":"2024-07-04T17:27:28.026775+0000","last_undegraded":"2024-07-04T17:27:28.026775+0000","last_fullsized":"2024-07-04T17:27:28.026775+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:18:35.514937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"26'1","reported_seq":21,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.974785+0000","last_change":"2024-07-04T17:27:06.877855+0000","last_active":"2024-07-04T17:27:27.974785+0000","last_peered":"2024-07-04T17:27:27.974785+0000","last_clean":"2024-07-04T17:27:27.974785+0000","last_became_active":"2024-07-04T17:27:06.876852+0000","last_became_peered":"2024-07-04T17:27:06.876852+0000","last_unstale":"2024-07-04T17:27:27.974785+0000","last_undegraded":"2024-07-04T17:27:27.974785+0000","last_fullsized":"2024-07-04T17:27:27.974785+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:19:43.481377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.970474+0000","last_change":"2024-07-04T17:27:07.904773+0000","last_active":"2024-07-04T17:27:27.970474+0000","last_peered":"2024-07-04T17:27:27.970474+0000","last_clean":"2024-07-04T17:27:27.970474+0000","last_became_active":"2024-07-04T17:27:07.900622+0000","last_became_peered":"2024-07-04T17:27:07.900622+0000","last_unstale":"2024-07-04T17:27:27.970474+0000","last_undegraded":"2024-07-04T17:27:27.970474+0000","last_fullsized":"2024-07-04T17:27:27.970474+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:21:38.559811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"30'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.017767+0000","last_change":"2024-07-04T17:27:26.966387+0000","last_active":"2024-07-04T17:27:28.017767+0000","last_peered":"2024-07-04T17:27:28.017767+0000","last_clean":"2024-07-04T17:27:28.017767+0000","last_became_active":"2024-07-04T17:27:26.966045+0000","last_became_peered":"2024-07-04T17:27:26.966045+0000","last_unstale":"2024-07-04T17:27:28.017767+0000","last_undegraded":"2024-07-04T17:27:28.017767+0000","last_fullsized":"2024-07-04T17:27:28.017767+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:51:48.147252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.026321+0000","last_change":"2024-07-04T17:27:28.015465+0000","last_active":"2024-07-04T17:27:28.026321+0000","last_peered":"2024-07-04T17:27:28.026321+0000","last_clean":"2024-07-04T17:27:28.026321+0000","last_became_active":"2024-07-04T17:27:28.013215+0000","last_became_peered":"2024-07-04T17:27:28.013215+0000","last_unstale":"2024-07-04T17:27:28.026321+0000","last_undegraded":"2024-07-04T17:27:28.026321+0000","last_fullsized":"2024-07-04T17:27:28.026321+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:25:12.049628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"26'2","reported_seq":22,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.980551+0000","last_change":"2024-07-04T17:27:06.885939+0000","last_active":"2024-07-04T17:27:27.980551+0000","last_peered":"2024-07-04T17:27:27.980551+0000","last_clean":"2024-07-04T17:27:27.980551+0000","last_became_active":"2024-07-04T17:27:06.885638+0000","last_became_peered":"2024-07-04T17:27:06.885638+0000","last_unstale":"2024-07-04T17:27:27.980551+0000","last_undegraded":"2024-07-04T17:27:27.980551+0000","last_fullsized":"2024-07-04T17:27:27.980551+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:31:45.030996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.977299+0000","last_change":"2024-07-04T17:27:07.899041+0000","last_active":"2024-07-04T17:27:27.977299+0000","last_peered":"2024-07-04T17:27:27.977299+0000","last_clean":"2024-07-04T17:27:27.977299+0000","last_became_active":"2024-07-04T17:27:07.895875+0000","last_became_peered":"2024-07-04T17:27:07.895875+0000","last_unstale":"2024-07-04T17:27:27.977299+0000","last_undegraded":"2024-07-04T17:27:27.977299+0000","last_fullsized":"2024-07-04T17:27:27.977299+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:34:35.587534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.970840+0000","last_change":"2024-07-04T17:27:26.981319+0000","last_active":"2024-07-04T17:27:27.970840+0000","last_peered":"2024-07-04T17:27:27.970840+0000","last_clean":"2024-07-04T17:27:27.970840+0000","last_became_active":"2024-07-04T17:27:26.980593+0000","last_became_peered":"2024-07-04T17:27:26.980593+0000","last_unstale":"2024-07-04T17:27:27.970840+0000","last_undegraded":"2024-07-04T17:27:27.970840+0000","last_fullsized":"2024-07-04T17:27:27.970840+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:11:05.732872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.030472+0000","last_change":"2024-07-04T17:27:28.010746+0000","last_active":"2024-07-04T17:27:28.030472+0000","last_peered":"2024-07-04T17:27:28.030472+0000","last_clean":"2024-07-04T17:27:28.030472+0000","last_became_active":"2024-07-04T17:27:28.009340+0000","last_became_peered":"2024-07-04T17:27:28.009340+0000","last_unstale":"2024-07-04T17:27:28.030472+0000","last_undegraded":"2024-07-04T17:27:28.030472+0000","last_fullsized":"2024-07-04T17:27:28.030472+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:17:51.739014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"26'1","reported_seq":21,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.976403+0000","last_change":"2024-07-04T17:27:06.877349+0000","last_active":"2024-07-04T17:27:27.976403+0000","last_peered":"2024-07-04T17:27:27.976403+0000","last_clean":"2024-07-04T17:27:27.976403+0000","last_became_active":"2024-07-04T17:27:06.876814+0000","last_became_peered":"2024-07-04T17:27:06.876814+0000","last_unstale":"2024-07-04T17:27:27.976403+0000","last_undegraded":"2024-07-04T17:27:27.976403+0000","last_fullsized":"2024-07-04T17:27:27.976403+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:52:15.359670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.982195+0000","last_change":"2024-07-04T17:27:07.915949+0000","last_active":"2024-07-04T17:27:27.982195+0000","last_peered":"2024-07-04T17:27:27.982195+0000","last_clean":"2024-07-04T17:27:27.982195+0000","last_became_active":"2024-07-04T17:27:07.915603+0000","last_became_peered":"2024-07-04T17:27:07.915603+0000","last_unstale":"2024-07-04T17:27:27.982195+0000","last_undegraded":"2024-07-04T17:27:27.982195+0000","last_fullsized":"2024-07-04T17:27:27.982195+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:27:08.653236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"30'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.013749+0000","last_change":"2024-07-04T17:27:26.988800+0000","last_active":"2024-07-04T17:27:28.013749+0000","last_peered":"2024-07-04T17:27:28.013749+0000","last_clean":"2024-07-04T17:27:28.013749+0000","last_became_active":"2024-07-04T17:27:26.988563+0000","last_became_peered":"2024-07-04T17:27:26.988563+0000","last_unstale":"2024-07-04T17:27:28.013749+0000","last_undegraded":"2024-07-04T17:27:28.013749+0000","last_fullsized":"2024-07-04T17:27:28.013749+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:18:34.504516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.034456+0000","last_change":"2024-07-04T17:27:28.032546+0000","last_active":"2024-07-04T17:27:28.034456+0000","last_peered":"2024-07-04T17:27:28.034456+0000","last_clean":"2024-07-04T17:27:28.034456+0000","last_became_active":"2024-07-04T17:27:28.032101+0000","last_became_peered":"2024-07-04T17:27:28.032101+0000","last_unstale":"2024-07-04T17:27:28.034456+0000","last_undegraded":"2024-07-04T17:27:28.034456+0000","last_fullsized":"2024-07-04T17:27:28.034456+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:47:32.101334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"26'8","reported_seq":28,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.972064+0000","last_change":"2024-07-04T17:27:06.865126+0000","last_active":"2024-07-04T17:27:27.972064+0000","last_peered":"2024-07-04T17:27:27.972064+0000","last_clean":"2024-07-04T17:27:27.972064+0000","last_became_active":"2024-07-04T17:27:06.864781+0000","last_became_peered":"2024-07-04T17:27:06.864781+0000","last_unstale":"2024-07-04T17:27:27.972064+0000","last_undegraded":"2024-07-04T17:27:27.972064+0000","last_fullsized":"2024-07-04T17:27:27.972064+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:44:59.048384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.967458+0000","last_change":"2024-07-04T17:27:07.886267+0000","last_active":"2024-07-04T17:27:27.967458+0000","last_peered":"2024-07-04T17:27:27.967458+0000","last_clean":"2024-07-04T17:27:27.967458+0000","last_became_active":"2024-07-04T17:27:07.885526+0000","last_became_peered":"2024-07-04T17:27:07.885526+0000","last_unstale":"2024-07-04T17:27:27.967458+0000","last_undegraded":"2024-07-04T17:27:27.967458+0000","last_fullsized":"2024-07-04T17:27:27.967458+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:42:03.682646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.958076+0000","last_change":"2024-07-04T17:27:26.992067+0000","last_active":"2024-07-04T17:27:27.958076+0000","last_peered":"2024-07-04T17:27:27.958076+0000","last_clean":"2024-07-04T17:27:27.958076+0000","last_became_active":"2024-07-04T17:27:26.985396+0000","last_became_peered":"2024-07-04T17:27:26.985396+0000","last_unstale":"2024-07-04T17:27:27.958076+0000","last_undegraded":"2024-07-04T17:27:27.958076+0000","last_fullsized":"2024-07-04T17:27:27.958076+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:49:35.175636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.021963+0000","last_change":"2024-07-04T17:27:28.010519+0000","last_active":"2024-07-04T17:27:28.021963+0000","last_peered":"2024-07-04T17:27:28.021963+0000","last_clean":"2024-07-04T17:27:28.021963+0000","last_became_active":"2024-07-04T17:27:28.010200+0000","last_became_peered":"2024-07-04T17:27:28.010200+0000","last_unstale":"2024-07-04T17:27:28.021963+0000","last_undegraded":"2024-07-04T17:27:28.021963+0000","last_fullsized":"2024-07-04T17:27:28.021963+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:37:40.507481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.971449+0000","last_change":"2024-07-04T17:27:06.883046+0000","last_active":"2024-07-04T17:27:27.971449+0000","last_peered":"2024-07-04T17:27:27.971449+0000","last_clean":"2024-07-04T17:27:27.971449+0000","last_became_active":"2024-07-04T17:27:06.881811+0000","last_became_peered":"2024-07-04T17:27:06.881811+0000","last_unstale":"2024-07-04T17:27:27.971449+0000","last_undegraded":"2024-07-04T17:27:27.971449+0000","last_fullsized":"2024-07-04T17:27:27.971449+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:40:58.357800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.968665+0000","last_change":"2024-07-04T17:27:07.909034+0000","last_active":"2024-07-04T17:27:27.968665+0000","last_peered":"2024-07-04T17:27:27.968665+0000","last_clean":"2024-07-04T17:27:27.968665+0000","last_became_active":"2024-07-04T17:27:07.908392+0000","last_became_peered":"2024-07-04T17:27:07.908392+0000","last_unstale":"2024-07-04T17:27:27.968665+0000","last_undegraded":"2024-07-04T17:27:27.968665+0000","last_fullsized":"2024-07-04T17:27:27.968665+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:09:16.307994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"30'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.018119+0000","last_change":"2024-07-04T17:27:26.982698+0000","last_active":"2024-07-04T17:27:28.018119+0000","last_peered":"2024-07-04T17:27:28.018119+0000","last_clean":"2024-07-04T17:27:28.018119+0000","last_became_active":"2024-07-04T17:27:26.982363+0000","last_became_peered":"2024-07-04T17:27:26.982363+0000","last_unstale":"2024-07-04T17:27:28.018119+0000","last_undegraded":"2024-07-04T17:27:28.018119+0000","last_fullsized":"2024-07-04T17:27:28.018119+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:30:08.921831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.030635+0000","last_change":"2024-07-04T17:27:28.020047+0000","last_active":"2024-07-04T17:27:28.030635+0000","last_peered":"2024-07-04T17:27:28.030635+0000","last_clean":"2024-07-04T17:27:28.030635+0000","last_became_active":"2024-07-04T17:27:28.019697+0000","last_became_peered":"2024-07-04T17:27:28.019697+0000","last_unstale":"2024-07-04T17:27:28.030635+0000","last_undegraded":"2024-07-04T17:27:28.030635+0000","last_fullsized":"2024-07-04T17:27:28.030635+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:15:24.893281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.966923+0000","last_change":"2024-07-04T17:27:06.873153+0000","last_active":"2024-07-04T17:27:27.966923+0000","last_peered":"2024-07-04T17:27:27.966923+0000","last_clean":"2024-07-04T17:27:27.966923+0000","last_became_active":"2024-07-04T17:27:06.872701+0000","last_became_peered":"2024-07-04T17:27:06.872701+0000","last_unstale":"2024-07-04T17:27:27.966923+0000","last_undegraded":"2024-07-04T17:27:27.966923+0000","last_fullsized":"2024-07-04T17:27:27.966923+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:23:40.790450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.969976+0000","last_change":"2024-07-04T17:27:07.886433+0000","last_active":"2024-07-04T17:27:27.969976+0000","last_peered":"2024-07-04T17:27:27.969976+0000","last_clean":"2024-07-04T17:27:27.969976+0000","last_became_active":"2024-07-04T17:27:07.878497+0000","last_became_peered":"2024-07-04T17:27:07.878497+0000","last_unstale":"2024-07-04T17:27:27.969976+0000","last_undegraded":"2024-07-04T17:27:27.969976+0000","last_fullsized":"2024-07-04T17:27:27.969976+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:59:48.713747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.10","version":"30'8","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.034024+0000","last_change":"2024-07-04T17:27:26.993836+0000","last_active":"2024-07-04T17:27:28.034024+0000","last_peered":"2024-07-04T17:27:28.034024+0000","last_clean":"2024-07-04T17:27:28.034024+0000","last_became_active":"2024-07-04T17:27:26.993545+0000","last_became_peered":"2024-07-04T17:27:26.993545+0000","last_unstale":"2024-07-04T17:27:28.034024+0000","last_undegraded":"2024-07-04T17:27:28.034024+0000","last_fullsized":"2024-07-04T17:27:28.034024+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:25:11.039208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.998793+0000","last_change":"2024-07-04T17:27:27.985575+0000","last_active":"2024-07-04T17:27:27.998793+0000","last_peered":"2024-07-04T17:27:27.998793+0000","last_clean":"2024-07-04T17:27:27.998793+0000","last_became_active":"2024-07-04T17:27:27.985158+0000","last_became_peered":"2024-07-04T17:27:27.985158+0000","last_unstale":"2024-07-04T17:27:27.998793+0000","last_undegraded":"2024-07-04T17:27:27.998793+0000","last_fullsized":"2024-07-04T17:27:27.998793+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:57:25.963098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"26'2","reported_seq":22,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.978430+0000","last_change":"2024-07-04T17:27:06.883778+0000","last_active":"2024-07-04T17:27:27.978430+0000","last_peered":"2024-07-04T17:27:27.978430+0000","last_clean":"2024-07-04T17:27:27.978430+0000","last_became_active":"2024-07-04T17:27:06.883449+0000","last_became_peered":"2024-07-04T17:27:06.883449+0000","last_unstale":"2024-07-04T17:27:27.978430+0000","last_undegraded":"2024-07-04T17:27:27.978430+0000","last_fullsized":"2024-07-04T17:27:27.978430+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:05:09.285237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.968707+0000","last_change":"2024-07-04T17:27:07.878309+0000","last_active":"2024-07-04T17:27:27.968707+0000","last_peered":"2024-07-04T17:27:27.968707+0000","last_clean":"2024-07-04T17:27:27.968707+0000","last_became_active":"2024-07-04T17:27:07.877975+0000","last_became_peered":"2024-07-04T17:27:07.877975+0000","last_unstale":"2024-07-04T17:27:27.968707+0000","last_undegraded":"2024-07-04T17:27:27.968707+0000","last_fullsized":"2024-07-04T17:27:27.968707+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:23.798405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.973070+0000","last_change":"2024-07-04T17:27:26.979760+0000","last_active":"2024-07-04T17:27:27.973070+0000","last_peered":"2024-07-04T17:27:27.973070+0000","last_clean":"2024-07-04T17:27:27.973070+0000","last_became_active":"2024-07-04T17:27:26.979431+0000","last_became_peered":"2024-07-04T17:27:26.979431+0000","last_unstale":"2024-07-04T17:27:27.973070+0000","last_undegraded":"2024-07-04T17:27:27.973070+0000","last_fullsized":"2024-07-04T17:27:27.973070+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:19:16.598719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.025987+0000","last_change":"2024-07-04T17:27:28.014511+0000","last_active":"2024-07-04T17:27:28.025987+0000","last_peered":"2024-07-04T17:27:28.025987+0000","last_clean":"2024-07-04T17:27:28.025987+0000","last_became_active":"2024-07-04T17:27:28.014044+0000","last_became_peered":"2024-07-04T17:27:28.014044+0000","last_unstale":"2024-07-04T17:27:28.025987+0000","last_undegraded":"2024-07-04T17:27:28.025987+0000","last_fullsized":"2024-07-04T17:27:28.025987+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:44:15.882179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"27'9","reported_seq":29,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.970340+0000","last_change":"2024-07-04T17:27:06.872191+0000","last_active":"2024-07-04T17:27:27.970340+0000","last_peered":"2024-07-04T17:27:27.970340+0000","last_clean":"2024-07-04T17:27:27.970340+0000","last_became_active":"2024-07-04T17:27:06.871594+0000","last_became_peered":"2024-07-04T17:27:06.871594+0000","last_unstale":"2024-07-04T17:27:27.970340+0000","last_undegraded":"2024-07-04T17:27:27.970340+0000","last_fullsized":"2024-07-04T17:27:27.970340+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:05:09.285237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3309,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.974606+0000","last_change":"2024-07-04T17:27:07.889067+0000","last_active":"2024-07-04T17:27:27.974606+0000","last_peered":"2024-07-04T17:27:27.974606+0000","last_clean":"2024-07-04T17:27:27.974606+0000","last_became_active":"2024-07-04T17:27:07.888715+0000","last_became_peered":"2024-07-04T17:27:07.888715+0000","last_unstale":"2024-07-04T17:27:27.974606+0000","last_undegraded":"2024-07-04T17:27:27.974606+0000","last_fullsized":"2024-07-04T17:27:27.974606+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:51:30.952338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.983593+0000","last_change":"2024-07-04T17:27:26.985620+0000","last_active":"2024-07-04T17:27:27.983593+0000","last_peered":"2024-07-04T17:27:27.983593+0000","last_clean":"2024-07-04T17:27:27.983593+0000","last_became_active":"2024-07-04T17:27:26.985096+0000","last_became_peered":"2024-07-04T17:27:26.985096+0000","last_unstale":"2024-07-04T17:27:27.983593+0000","last_undegraded":"2024-07-04T17:27:27.983593+0000","last_fullsized":"2024-07-04T17:27:27.983593+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:05:29.349573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.029366+0000","last_change":"2024-07-04T17:27:28.020943+0000","last_active":"2024-07-04T17:27:28.029366+0000","last_peered":"2024-07-04T17:27:28.029366+0000","last_clean":"2024-07-04T17:27:28.029366+0000","last_became_active":"2024-07-04T17:27:28.020616+0000","last_became_peered":"2024-07-04T17:27:28.020616+0000","last_unstale":"2024-07-04T17:27:28.029366+0000","last_undegraded":"2024-07-04T17:27:28.029366+0000","last_fullsized":"2024-07-04T17:27:28.029366+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:49:36.186056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"26'3","reported_seq":23,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.968300+0000","last_change":"2024-07-04T17:27:06.868145+0000","last_active":"2024-07-04T17:27:27.968300+0000","last_peered":"2024-07-04T17:27:27.968300+0000","last_clean":"2024-07-04T17:27:27.968300+0000","last_became_active":"2024-07-04T17:27:06.867751+0000","last_became_peered":"2024-07-04T17:27:06.867751+0000","last_unstale":"2024-07-04T17:27:27.968300+0000","last_undegraded":"2024-07-04T17:27:27.968300+0000","last_fullsized":"2024-07-04T17:27:27.968300+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:13:08.657441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.958997+0000","last_change":"2024-07-04T17:27:07.891116+0000","last_active":"2024-07-04T17:27:27.958997+0000","last_peered":"2024-07-04T17:27:27.958997+0000","last_clean":"2024-07-04T17:27:27.958997+0000","last_became_active":"2024-07-04T17:27:07.890572+0000","last_became_peered":"2024-07-04T17:27:07.890572+0000","last_unstale":"2024-07-04T17:27:27.958997+0000","last_undegraded":"2024-07-04T17:27:27.958997+0000","last_fullsized":"2024-07-04T17:27:27.958997+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:51:29.047031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.977916+0000","last_change":"2024-07-04T17:27:26.977555+0000","last_active":"2024-07-04T17:27:27.977916+0000","last_peered":"2024-07-04T17:27:27.977916+0000","last_clean":"2024-07-04T17:27:27.977916+0000","last_became_active":"2024-07-04T17:27:26.976808+0000","last_became_peered":"2024-07-04T17:27:26.976808+0000","last_unstale":"2024-07-04T17:27:27.977916+0000","last_undegraded":"2024-07-04T17:27:27.977916+0000","last_fullsized":"2024-07-04T17:27:27.977916+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:40:10.805520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.017334+0000","last_change":"2024-07-04T17:27:28.002590+0000","last_active":"2024-07-04T17:27:28.017334+0000","last_peered":"2024-07-04T17:27:28.017334+0000","last_clean":"2024-07-04T17:27:28.017334+0000","last_became_active":"2024-07-04T17:27:28.000398+0000","last_became_peered":"2024-07-04T17:27:28.000398+0000","last_unstale":"2024-07-04T17:27:28.017334+0000","last_undegraded":"2024-07-04T17:27:28.017334+0000","last_fullsized":"2024-07-04T17:27:28.017334+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:11:06.743293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.959761+0000","last_change":"2024-07-04T17:27:06.873231+0000","last_active":"2024-07-04T17:27:27.959761+0000","last_peered":"2024-07-04T17:27:27.959761+0000","last_clean":"2024-07-04T17:27:27.959761+0000","last_became_active":"2024-07-04T17:27:06.872934+0000","last_became_peered":"2024-07-04T17:27:06.872934+0000","last_unstale":"2024-07-04T17:27:27.959761+0000","last_undegraded":"2024-07-04T17:27:27.959761+0000","last_fullsized":"2024-07-04T17:27:27.959761+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:42:02.672719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.983055+0000","last_change":"2024-07-04T17:27:07.898440+0000","last_active":"2024-07-04T17:27:27.983055+0000","last_peered":"2024-07-04T17:27:27.983055+0000","last_clean":"2024-07-04T17:27:27.983055+0000","last_became_active":"2024-07-04T17:27:07.897813+0000","last_became_peered":"2024-07-04T17:27:07.897813+0000","last_unstale":"2024-07-04T17:27:27.983055+0000","last_undegraded":"2024-07-04T17:27:27.983055+0000","last_fullsized":"2024-07-04T17:27:27.983055+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:38:45.217394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.14","version":"30'6","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.036663+0000","last_change":"2024-07-04T17:27:26.979315+0000","last_active":"2024-07-04T17:27:28.036663+0000","last_peered":"2024-07-04T17:27:28.036663+0000","last_clean":"2024-07-04T17:27:28.036663+0000","last_became_active":"2024-07-04T17:27:26.978520+0000","last_became_peered":"2024-07-04T17:27:26.978520+0000","last_unstale":"2024-07-04T17:27:28.036663+0000","last_undegraded":"2024-07-04T17:27:28.036663+0000","last_fullsized":"2024-07-04T17:27:28.036663+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:35:49.643323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.999104+0000","last_change":"2024-07-04T17:27:27.987156+0000","last_active":"2024-07-04T17:27:27.999104+0000","last_peered":"2024-07-04T17:27:27.999104+0000","last_clean":"2024-07-04T17:27:27.999104+0000","last_became_active":"2024-07-04T17:27:27.986834+0000","last_became_peered":"2024-07-04T17:27:27.986834+0000","last_unstale":"2024-07-04T17:27:27.999104+0000","last_undegraded":"2024-07-04T17:27:27.999104+0000","last_fullsized":"2024-07-04T17:27:27.999104+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:01:26.219741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.977201+0000","last_change":"2024-07-04T17:27:06.861151+0000","last_active":"2024-07-04T17:27:27.977201+0000","last_peered":"2024-07-04T17:27:27.977201+0000","last_clean":"2024-07-04T17:27:27.977201+0000","last_became_active":"2024-07-04T17:27:06.860975+0000","last_became_peered":"2024-07-04T17:27:06.860975+0000","last_unstale":"2024-07-04T17:27:27.977201+0000","last_undegraded":"2024-07-04T17:27:27.977201+0000","last_fullsized":"2024-07-04T17:27:27.977201+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:17:47.882494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.976776+0000","last_change":"2024-07-04T17:27:07.877680+0000","last_active":"2024-07-04T17:27:27.976776+0000","last_peered":"2024-07-04T17:27:27.976776+0000","last_clean":"2024-07-04T17:27:27.976776+0000","last_became_active":"2024-07-04T17:27:07.877486+0000","last_became_peered":"2024-07-04T17:27:07.877486+0000","last_unstale":"2024-07-04T17:27:27.976776+0000","last_undegraded":"2024-07-04T17:27:27.976776+0000","last_fullsized":"2024-07-04T17:27:27.976776+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:27:08.653236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.15","version":"30'3","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.023632+0000","last_change":"2024-07-04T17:27:26.980595+0000","last_active":"2024-07-04T17:27:28.023632+0000","last_peered":"2024-07-04T17:27:28.023632+0000","last_clean":"2024-07-04T17:27:28.023632+0000","last_became_active":"2024-07-04T17:27:26.978486+0000","last_became_peered":"2024-07-04T17:27:26.978486+0000","last_unstale":"2024-07-04T17:27:28.023632+0000","last_undegraded":"2024-07-04T17:27:28.023632+0000","last_fullsized":"2024-07-04T17:27:28.023632+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:51:50.052560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.016362+0000","last_change":"2024-07-04T17:27:28.001911+0000","last_active":"2024-07-04T17:27:28.016362+0000","last_peered":"2024-07-04T17:27:28.016362+0000","last_clean":"2024-07-04T17:27:28.016362+0000","last_became_active":"2024-07-04T17:27:28.001234+0000","last_became_peered":"2024-07-04T17:27:28.001234+0000","last_unstale":"2024-07-04T17:27:28.016362+0000","last_undegraded":"2024-07-04T17:27:28.016362+0000","last_fullsized":"2024-07-04T17:27:28.016362+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:09:53.789876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.967489+0000","last_change":"2024-07-04T17:27:06.883828+0000","last_active":"2024-07-04T17:27:27.967489+0000","last_peered":"2024-07-04T17:27:27.967489+0000","last_clean":"2024-07-04T17:27:27.967489+0000","last_became_active":"2024-07-04T17:27:06.883508+0000","last_became_peered":"2024-07-04T17:27:06.883508+0000","last_unstale":"2024-07-04T17:27:27.967489+0000","last_undegraded":"2024-07-04T17:27:27.967489+0000","last_fullsized":"2024-07-04T17:27:27.967489+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:44:59.048384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.967778+0000","last_change":"2024-07-04T17:27:07.885943+0000","last_active":"2024-07-04T17:27:27.967778+0000","last_peered":"2024-07-04T17:27:27.967778+0000","last_clean":"2024-07-04T17:27:27.967778+0000","last_became_active":"2024-07-04T17:27:07.876919+0000","last_became_peered":"2024-07-04T17:27:07.876919+0000","last_unstale":"2024-07-04T17:27:27.967778+0000","last_undegraded":"2024-07-04T17:27:27.967778+0000","last_fullsized":"2024-07-04T17:27:27.967778+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:09:16.307994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"30'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.029663+0000","last_change":"2024-07-04T17:27:26.987170+0000","last_active":"2024-07-04T17:27:28.029663+0000","last_peered":"2024-07-04T17:27:28.029663+0000","last_clean":"2024-07-04T17:27:28.029663+0000","last_became_active":"2024-07-04T17:27:26.986855+0000","last_became_peered":"2024-07-04T17:27:26.986855+0000","last_unstale":"2024-07-04T17:27:28.029663+0000","last_undegraded":"2024-07-04T17:27:28.029663+0000","last_fullsized":"2024-07-04T17:27:28.029663+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:44:29.281558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.030774+0000","last_change":"2024-07-04T17:27:28.021858+0000","last_active":"2024-07-04T17:27:28.030774+0000","last_peered":"2024-07-04T17:27:28.030774+0000","last_clean":"2024-07-04T17:27:28.030774+0000","last_became_active":"2024-07-04T17:27:28.021553+0000","last_became_peered":"2024-07-04T17:27:28.021553+0000","last_unstale":"2024-07-04T17:27:28.030774+0000","last_undegraded":"2024-07-04T17:27:28.030774+0000","last_fullsized":"2024-07-04T17:27:28.030774+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:01:26.219741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.955437+0000","last_change":"2024-07-04T17:27:06.872335+0000","last_active":"2024-07-04T17:27:27.955437+0000","last_peered":"2024-07-04T17:27:27.955437+0000","last_clean":"2024-07-04T17:27:27.955437+0000","last_became_active":"2024-07-04T17:27:06.871960+0000","last_became_peered":"2024-07-04T17:27:06.871960+0000","last_unstale":"2024-07-04T17:27:27.955437+0000","last_undegraded":"2024-07-04T17:27:27.955437+0000","last_fullsized":"2024-07-04T17:27:27.955437+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:57:28.966215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.972195+0000","last_change":"2024-07-04T17:27:07.866107+0000","last_active":"2024-07-04T17:27:27.972195+0000","last_peered":"2024-07-04T17:27:27.972195+0000","last_clean":"2024-07-04T17:27:27.972195+0000","last_became_active":"2024-07-04T17:27:07.865761+0000","last_became_peered":"2024-07-04T17:27:07.865761+0000","last_unstale":"2024-07-04T17:27:27.972195+0000","last_undegraded":"2024-07-04T17:27:27.972195+0000","last_fullsized":"2024-07-04T17:27:27.972195+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:49:26.007232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"30'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.033379+0000","last_change":"2024-07-04T17:27:26.989568+0000","last_active":"2024-07-04T17:27:28.033379+0000","last_peered":"2024-07-04T17:27:28.033379+0000","last_clean":"2024-07-04T17:27:28.033379+0000","last_became_active":"2024-07-04T17:27:26.989241+0000","last_became_peered":"2024-07-04T17:27:26.989241+0000","last_unstale":"2024-07-04T17:27:28.033379+0000","last_undegraded":"2024-07-04T17:27:28.033379+0000","last_fullsized":"2024-07-04T17:27:28.033379+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:34:39.046205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.998717+0000","last_change":"2024-07-04T17:27:27.983473+0000","last_active":"2024-07-04T17:27:27.998717+0000","last_peered":"2024-07-04T17:27:27.998717+0000","last_clean":"2024-07-04T17:27:27.998717+0000","last_became_active":"2024-07-04T17:27:27.983015+0000","last_became_peered":"2024-07-04T17:27:27.983015+0000","last_unstale":"2024-07-04T17:27:27.998717+0000","last_undegraded":"2024-07-04T17:27:27.998717+0000","last_fullsized":"2024-07-04T17:27:27.998717+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:15:24.893281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"26'1","reported_seq":21,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.973406+0000","last_change":"2024-07-04T17:27:06.881468+0000","last_active":"2024-07-04T17:27:27.973406+0000","last_peered":"2024-07-04T17:27:27.973406+0000","last_clean":"2024-07-04T17:27:27.973406+0000","last_became_active":"2024-07-04T17:27:06.881150+0000","last_became_peered":"2024-07-04T17:27:06.881150+0000","last_unstale":"2024-07-04T17:27:27.973406+0000","last_undegraded":"2024-07-04T17:27:27.973406+0000","last_fullsized":"2024-07-04T17:27:27.973406+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:49:39.642772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.977979+0000","last_change":"2024-07-04T17:27:07.904214+0000","last_active":"2024-07-04T17:27:27.977979+0000","last_peered":"2024-07-04T17:27:27.977979+0000","last_clean":"2024-07-04T17:27:27.977979+0000","last_became_active":"2024-07-04T17:27:07.903782+0000","last_became_peered":"2024-07-04T17:27:07.903782+0000","last_unstale":"2024-07-04T17:27:27.977979+0000","last_undegraded":"2024-07-04T17:27:27.977979+0000","last_fullsized":"2024-07-04T17:27:27.977979+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:42:03.682646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.968185+0000","last_change":"2024-07-04T17:27:26.970514+0000","last_active":"2024-07-04T17:27:27.968185+0000","last_peered":"2024-07-04T17:27:27.968185+0000","last_clean":"2024-07-04T17:27:27.968185+0000","last_became_active":"2024-07-04T17:27:26.970160+0000","last_became_peered":"2024-07-04T17:27:26.970160+0000","last_unstale":"2024-07-04T17:27:27.968185+0000","last_undegraded":"2024-07-04T17:27:27.968185+0000","last_fullsized":"2024-07-04T17:27:27.968185+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:13:57.406777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.016540+0000","last_change":"2024-07-04T17:27:27.999689+0000","last_active":"2024-07-04T17:27:28.016540+0000","last_peered":"2024-07-04T17:27:28.016540+0000","last_clean":"2024-07-04T17:27:28.016540+0000","last_became_active":"2024-07-04T17:27:27.999348+0000","last_became_peered":"2024-07-04T17:27:27.999348+0000","last_unstale":"2024-07-04T17:27:28.016540+0000","last_undegraded":"2024-07-04T17:27:28.016540+0000","last_fullsized":"2024-07-04T17:27:28.016540+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:31:17.246893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"26'1","reported_seq":21,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.977306+0000","last_change":"2024-07-04T17:27:06.883716+0000","last_active":"2024-07-04T17:27:27.977306+0000","last_peered":"2024-07-04T17:27:27.977306+0000","last_clean":"2024-07-04T17:27:27.977306+0000","last_became_active":"2024-07-04T17:27:06.882604+0000","last_became_peered":"2024-07-04T17:27:06.882604+0000","last_unstale":"2024-07-04T17:27:27.977306+0000","last_undegraded":"2024-07-04T17:27:27.977306+0000","last_fullsized":"2024-07-04T17:27:27.977306+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:04:44.886174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.968914+0000","last_change":"2024-07-04T17:27:07.872169+0000","last_active":"2024-07-04T17:27:27.968914+0000","last_peered":"2024-07-04T17:27:27.968914+0000","last_clean":"2024-07-04T17:27:27.968914+0000","last_became_active":"2024-07-04T17:27:07.870986+0000","last_became_peered":"2024-07-04T17:27:07.870986+0000","last_unstale":"2024-07-04T17:27:27.968914+0000","last_undegraded":"2024-07-04T17:27:27.968914+0000","last_fullsized":"2024-07-04T17:27:27.968914+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:34:46.000005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.956997+0000","last_change":"2024-07-04T17:27:26.988054+0000","last_active":"2024-07-04T17:27:27.956997+0000","last_peered":"2024-07-04T17:27:27.956997+0000","last_clean":"2024-07-04T17:27:27.956997+0000","last_became_active":"2024-07-04T17:27:26.985155+0000","last_became_peered":"2024-07-04T17:27:26.985155+0000","last_unstale":"2024-07-04T17:27:27.956997+0000","last_undegraded":"2024-07-04T17:27:27.956997+0000","last_fullsized":"2024-07-04T17:27:27.956997+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:36:05.554366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.021704+0000","last_change":"2024-07-04T17:27:28.018532+0000","last_active":"2024-07-04T17:27:28.021704+0000","last_peered":"2024-07-04T17:27:28.021704+0000","last_clean":"2024-07-04T17:27:28.021704+0000","last_became_active":"2024-07-04T17:27:28.018166+0000","last_became_peered":"2024-07-04T17:27:28.018166+0000","last_unstale":"2024-07-04T17:27:28.021704+0000","last_undegraded":"2024-07-04T17:27:28.021704+0000","last_fullsized":"2024-07-04T17:27:28.021704+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:38:51.351144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"30'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.016248+0000","last_change":"2024-07-04T17:27:26.989355+0000","last_active":"2024-07-04T17:27:28.016248+0000","last_peered":"2024-07-04T17:27:28.016248+0000","last_clean":"2024-07-04T17:27:28.016248+0000","last_became_active":"2024-07-04T17:27:26.988955+0000","last_became_peered":"2024-07-04T17:27:26.988955+0000","last_unstale":"2024-07-04T17:27:28.016248+0000","last_undegraded":"2024-07-04T17:27:28.016248+0000","last_fullsized":"2024-07-04T17:27:28.016248+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:17:50.728594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"26'2","reported_seq":22,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.979708+0000","last_change":"2024-07-04T17:27:06.885584+0000","last_active":"2024-07-04T17:27:27.979708+0000","last_peered":"2024-07-04T17:27:27.979708+0000","last_clean":"2024-07-04T17:27:27.979708+0000","last_became_active":"2024-07-04T17:27:06.885226+0000","last_became_peered":"2024-07-04T17:27:06.885226+0000","last_unstale":"2024-07-04T17:27:27.979708+0000","last_undegraded":"2024-07-04T17:27:27.979708+0000","last_fullsized":"2024-07-04T17:27:27.979708+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:10:29.469978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.975437+0000","last_change":"2024-07-04T17:27:07.901586+0000","last_active":"2024-07-04T17:27:27.975437+0000","last_peered":"2024-07-04T17:27:27.975437+0000","last_clean":"2024-07-04T17:27:27.975437+0000","last_became_active":"2024-07-04T17:27:07.900901+0000","last_became_peered":"2024-07-04T17:27:07.900901+0000","last_unstale":"2024-07-04T17:27:27.975437+0000","last_undegraded":"2024-07-04T17:27:27.975437+0000","last_fullsized":"2024-07-04T17:27:27.975437+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:27:04.910602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.026098+0000","last_change":"2024-07-04T17:27:28.015072+0000","last_active":"2024-07-04T17:27:28.026098+0000","last_peered":"2024-07-04T17:27:28.026098+0000","last_clean":"2024-07-04T17:27:28.026098+0000","last_became_active":"2024-07-04T17:27:28.014401+0000","last_became_peered":"2024-07-04T17:27:28.014401+0000","last_unstale":"2024-07-04T17:27:28.026098+0000","last_undegraded":"2024-07-04T17:27:28.026098+0000","last_fullsized":"2024-07-04T17:27:28.026098+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:28:51.661977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"30'4","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.023736+0000","last_change":"2024-07-04T17:27:26.980348+0000","last_active":"2024-07-04T17:27:28.023736+0000","last_peered":"2024-07-04T17:27:28.023736+0000","last_clean":"2024-07-04T17:27:28.023736+0000","last_became_active":"2024-07-04T17:27:26.979958+0000","last_became_peered":"2024-07-04T17:27:26.979958+0000","last_unstale":"2024-07-04T17:27:28.023736+0000","last_undegraded":"2024-07-04T17:27:28.023736+0000","last_fullsized":"2024-07-04T17:27:28.023736+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:50:18.424434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"26'2","reported_seq":22,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.974359+0000","last_change":"2024-07-04T17:27:06.860195+0000","last_active":"2024-07-04T17:27:27.974359+0000","last_peered":"2024-07-04T17:27:27.974359+0000","last_clean":"2024-07-04T17:27:27.974359+0000","last_became_active":"2024-07-04T17:27:06.859440+0000","last_became_peered":"2024-07-04T17:27:06.859440+0000","last_unstale":"2024-07-04T17:27:27.974359+0000","last_undegraded":"2024-07-04T17:27:27.974359+0000","last_fullsized":"2024-07-04T17:27:27.974359+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:11:21.518052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.980236+0000","last_change":"2024-07-04T17:27:07.911112+0000","last_active":"2024-07-04T17:27:27.980236+0000","last_peered":"2024-07-04T17:27:27.980236+0000","last_clean":"2024-07-04T17:27:27.980236+0000","last_became_active":"2024-07-04T17:27:07.910473+0000","last_became_peered":"2024-07-04T17:27:07.910473+0000","last_unstale":"2024-07-04T17:27:27.980236+0000","last_undegraded":"2024-07-04T17:27:27.980236+0000","last_fullsized":"2024-07-04T17:27:27.980236+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:47:16.318473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.026349+0000","last_change":"2024-07-04T17:27:28.015745+0000","last_active":"2024-07-04T17:27:28.026349+0000","last_peered":"2024-07-04T17:27:28.026349+0000","last_clean":"2024-07-04T17:27:28.026349+0000","last_became_active":"2024-07-04T17:27:28.014012+0000","last_became_peered":"2024-07-04T17:27:28.014012+0000","last_unstale":"2024-07-04T17:27:28.026349+0000","last_undegraded":"2024-07-04T17:27:28.026349+0000","last_fullsized":"2024-07-04T17:27:28.026349+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:36:06.564787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"26'1","reported_seq":21,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.985428+0000","last_change":"2024-07-04T17:27:06.863457+0000","last_active":"2024-07-04T17:27:27.985428+0000","last_peered":"2024-07-04T17:27:27.985428+0000","last_clean":"2024-07-04T17:27:27.985428+0000","last_became_active":"2024-07-04T17:27:06.862649+0000","last_became_peered":"2024-07-04T17:27:06.862649+0000","last_unstale":"2024-07-04T17:27:27.985428+0000","last_undegraded":"2024-07-04T17:27:27.985428+0000","last_fullsized":"2024-07-04T17:27:27.985428+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:49:24.997305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.982983+0000","last_change":"2024-07-04T17:27:07.896653+0000","last_active":"2024-07-04T17:27:27.982983+0000","last_peered":"2024-07-04T17:27:27.982983+0000","last_clean":"2024-07-04T17:27:27.982983+0000","last_became_active":"2024-07-04T17:27:07.895759+0000","last_became_peered":"2024-07-04T17:27:07.895759+0000","last_unstale":"2024-07-04T17:27:27.982983+0000","last_undegraded":"2024-07-04T17:27:27.982983+0000","last_fullsized":"2024-07-04T17:27:27.982983+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:06:36.067329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"30'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.019678+0000","last_change":"2024-07-04T17:27:26.981603+0000","last_active":"2024-07-04T17:27:28.019678+0000","last_peered":"2024-07-04T17:27:28.019678+0000","last_clean":"2024-07-04T17:27:28.019678+0000","last_became_active":"2024-07-04T17:27:26.980997+0000","last_became_peered":"2024-07-04T17:27:26.980997+0000","last_unstale":"2024-07-04T17:27:28.019678+0000","last_undegraded":"2024-07-04T17:27:28.019678+0000","last_fullsized":"2024-07-04T17:27:28.019678+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:24:19.419795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.028462+0000","last_change":"2024-07-04T17:27:28.010451+0000","last_active":"2024-07-04T17:27:28.028462+0000","last_peered":"2024-07-04T17:27:28.028462+0000","last_clean":"2024-07-04T17:27:28.028462+0000","last_became_active":"2024-07-04T17:27:28.003814+0000","last_became_peered":"2024-07-04T17:27:28.003814+0000","last_unstale":"2024-07-04T17:27:28.028462+0000","last_undegraded":"2024-07-04T17:27:28.028462+0000","last_fullsized":"2024-07-04T17:27:28.028462+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:33.432540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"26'4","reported_seq":24,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.977841+0000","last_change":"2024-07-04T17:27:06.866478+0000","last_active":"2024-07-04T17:27:27.977841+0000","last_peered":"2024-07-04T17:27:27.977841+0000","last_clean":"2024-07-04T17:27:27.977841+0000","last_became_active":"2024-07-04T17:27:06.865591+0000","last_became_peered":"2024-07-04T17:27:06.865591+0000","last_unstale":"2024-07-04T17:27:27.977841+0000","last_undegraded":"2024-07-04T17:27:27.977841+0000","last_fullsized":"2024-07-04T17:27:27.977841+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:05.805963+0000","last_clean_scrub_stamp":"2024-07-04T17:27:05.805963+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:09:15.298068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.965337+0000","last_change":"2024-07-04T17:27:07.888430+0000","last_active":"2024-07-04T17:27:27.965337+0000","last_peered":"2024-07-04T17:27:27.965337+0000","last_clean":"2024-07-04T17:27:27.965337+0000","last_became_active":"2024-07-04T17:27:07.886722+0000","last_became_peered":"2024-07-04T17:27:07.886722+0000","last_unstale":"2024-07-04T17:27:27.965337+0000","last_undegraded":"2024-07-04T17:27:27.965337+0000","last_fullsized":"2024-07-04T17:27:27.965337+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:44:31.068133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"30'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.015268+0000","last_change":"2024-07-04T17:27:26.986270+0000","last_active":"2024-07-04T17:27:28.015268+0000","last_peered":"2024-07-04T17:27:28.015268+0000","last_clean":"2024-07-04T17:27:28.015268+0000","last_became_active":"2024-07-04T17:27:26.985927+0000","last_became_peered":"2024-07-04T17:27:26.985927+0000","last_unstale":"2024-07-04T17:27:28.015268+0000","last_undegraded":"2024-07-04T17:27:28.015268+0000","last_fullsized":"2024-07-04T17:27:28.015268+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:25.916111+0000","last_clean_scrub_stamp":"2024-07-04T17:27:25.916111+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:53:21.040811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.028435+0000","last_change":"2024-07-04T17:27:28.014596+0000","last_active":"2024-07-04T17:27:28.028435+0000","last_peered":"2024-07-04T17:27:28.028435+0000","last_clean":"2024-07-04T17:27:28.028435+0000","last_became_active":"2024-07-04T17:27:28.014261+0000","last_became_peered":"2024-07-04T17:27:28.014261+0000","last_unstale":"2024-07-04T17:27:28.028435+0000","last_undegraded":"2024-07-04T17:27:28.028435+0000","last_fullsized":"2024-07-04T17:27:28.028435+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:40:11.815941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.969165+0000","last_change":"2024-07-04T17:27:07.910603+0000","last_active":"2024-07-04T17:27:27.969165+0000","last_peered":"2024-07-04T17:27:27.969165+0000","last_clean":"2024-07-04T17:27:27.969165+0000","last_became_active":"2024-07-04T17:27:07.908845+0000","last_became_peered":"2024-07-04T17:27:07.908845+0000","last_unstale":"2024-07-04T17:27:27.969165+0000","last_undegraded":"2024-07-04T17:27:27.969165+0000","last_fullsized":"2024-07-04T17:27:27.969165+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:17:48.892420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.029698+0000","last_change":"2024-07-04T17:27:28.027868+0000","last_active":"2024-07-04T17:27:28.029698+0000","last_peered":"2024-07-04T17:27:28.029698+0000","last_clean":"2024-07-04T17:27:28.029698+0000","last_became_active":"2024-07-04T17:27:28.026823+0000","last_became_peered":"2024-07-04T17:27:28.026823+0000","last_unstale":"2024-07-04T17:27:28.029698+0000","last_undegraded":"2024-07-04T17:27:28.029698+0000","last_fullsized":"2024-07-04T17:27:28.029698+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:21:02.597642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.971962+0000","last_change":"2024-07-04T17:27:07.906719+0000","last_active":"2024-07-04T17:27:27.971962+0000","last_peered":"2024-07-04T17:27:27.971962+0000","last_clean":"2024-07-04T17:27:27.971962+0000","last_became_active":"2024-07-04T17:27:07.903370+0000","last_became_peered":"2024-07-04T17:27:07.903370+0000","last_unstale":"2024-07-04T17:27:27.971962+0000","last_undegraded":"2024-07-04T17:27:27.971962+0000","last_fullsized":"2024-07-04T17:27:27.971962+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:55:16.424967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.006519+0000","last_change":"2024-07-04T17:27:27.999222+0000","last_active":"2024-07-04T17:27:28.006519+0000","last_peered":"2024-07-04T17:27:28.006519+0000","last_clean":"2024-07-04T17:27:28.006519+0000","last_became_active":"2024-07-04T17:27:27.998897+0000","last_became_peered":"2024-07-04T17:27:27.998897+0000","last_unstale":"2024-07-04T17:27:28.006519+0000","last_undegraded":"2024-07-04T17:27:28.006519+0000","last_fullsized":"2024-07-04T17:27:28.006519+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:59:28.341788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.958272+0000","last_change":"2024-07-04T17:27:07.878292+0000","last_active":"2024-07-04T17:27:27.958272+0000","last_peered":"2024-07-04T17:27:27.958272+0000","last_clean":"2024-07-04T17:27:27.958272+0000","last_became_active":"2024-07-04T17:27:07.874906+0000","last_became_peered":"2024-07-04T17:27:07.874906+0000","last_unstale":"2024-07-04T17:27:27.958272+0000","last_undegraded":"2024-07-04T17:27:27.958272+0000","last_fullsized":"2024-07-04T17:27:27.958272+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:20:57.602937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.025556+0000","last_change":"2024-07-04T17:27:28.013502+0000","last_active":"2024-07-04T17:27:28.025556+0000","last_peered":"2024-07-04T17:27:28.025556+0000","last_clean":"2024-07-04T17:27:28.025556+0000","last_became_active":"2024-07-04T17:27:28.013162+0000","last_became_peered":"2024-07-04T17:27:28.013162+0000","last_unstale":"2024-07-04T17:27:28.025556+0000","last_undegraded":"2024-07-04T17:27:28.025556+0000","last_fullsized":"2024-07-04T17:27:28.025556+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:58:39.364744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.984795+0000","last_change":"2024-07-04T17:27:07.883042+0000","last_active":"2024-07-04T17:27:27.984795+0000","last_peered":"2024-07-04T17:27:27.984795+0000","last_clean":"2024-07-04T17:27:27.984795+0000","last_became_active":"2024-07-04T17:27:07.882772+0000","last_became_peered":"2024-07-04T17:27:07.882772+0000","last_unstale":"2024-07-04T17:27:27.984795+0000","last_undegraded":"2024-07-04T17:27:27.984795+0000","last_fullsized":"2024-07-04T17:27:27.984795+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:03:21.103650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.026879+0000","last_change":"2024-07-04T17:27:28.009005+0000","last_active":"2024-07-04T17:27:28.026879+0000","last_peered":"2024-07-04T17:27:28.026879+0000","last_clean":"2024-07-04T17:27:28.026879+0000","last_became_active":"2024-07-04T17:27:28.008680+0000","last_became_peered":"2024-07-04T17:27:28.008680+0000","last_unstale":"2024-07-04T17:27:28.026879+0000","last_undegraded":"2024-07-04T17:27:28.026879+0000","last_fullsized":"2024-07-04T17:27:28.026879+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:48:23.096987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.980368+0000","last_change":"2024-07-04T17:27:07.911625+0000","last_active":"2024-07-04T17:27:27.980368+0000","last_peered":"2024-07-04T17:27:27.980368+0000","last_clean":"2024-07-04T17:27:27.980368+0000","last_became_active":"2024-07-04T17:27:07.910841+0000","last_became_peered":"2024-07-04T17:27:07.910841+0000","last_unstale":"2024-07-04T17:27:27.980368+0000","last_undegraded":"2024-07-04T17:27:27.980368+0000","last_fullsized":"2024-07-04T17:27:27.980368+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:43:39.860474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.034776+0000","last_change":"2024-07-04T17:27:28.032931+0000","last_active":"2024-07-04T17:27:28.034776+0000","last_peered":"2024-07-04T17:27:28.034776+0000","last_clean":"2024-07-04T17:27:28.034776+0000","last_became_active":"2024-07-04T17:27:28.032254+0000","last_became_peered":"2024-07-04T17:27:28.032254+0000","last_unstale":"2024-07-04T17:27:28.034776+0000","last_undegraded":"2024-07-04T17:27:28.034776+0000","last_fullsized":"2024-07-04T17:27:28.034776+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:17:43.391316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.975930+0000","last_change":"2024-07-04T17:27:07.901813+0000","last_active":"2024-07-04T17:27:27.975930+0000","last_peered":"2024-07-04T17:27:27.975930+0000","last_clean":"2024-07-04T17:27:27.975930+0000","last_became_active":"2024-07-04T17:27:07.901333+0000","last_became_peered":"2024-07-04T17:27:07.901333+0000","last_unstale":"2024-07-04T17:27:27.975930+0000","last_undegraded":"2024-07-04T17:27:27.975930+0000","last_fullsized":"2024-07-04T17:27:27.975930+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:24:58.267453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.030595+0000","last_change":"2024-07-04T17:27:28.021463+0000","last_active":"2024-07-04T17:27:28.030595+0000","last_peered":"2024-07-04T17:27:28.030595+0000","last_clean":"2024-07-04T17:27:28.030595+0000","last_became_active":"2024-07-04T17:27:28.021168+0000","last_became_peered":"2024-07-04T17:27:28.021168+0000","last_unstale":"2024-07-04T17:27:28.030595+0000","last_undegraded":"2024-07-04T17:27:28.030595+0000","last_fullsized":"2024-07-04T17:27:28.030595+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:57:25.963098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.976889+0000","last_change":"2024-07-04T17:27:07.877946+0000","last_active":"2024-07-04T17:27:27.976889+0000","last_peered":"2024-07-04T17:27:27.976889+0000","last_clean":"2024-07-04T17:27:27.976889+0000","last_became_active":"2024-07-04T17:27:07.877581+0000","last_became_peered":"2024-07-04T17:27:07.877581+0000","last_unstale":"2024-07-04T17:27:27.976889+0000","last_undegraded":"2024-07-04T17:27:27.976889+0000","last_fullsized":"2024-07-04T17:27:27.976889+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:38:45.217394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.015649+0000","last_change":"2024-07-04T17:27:28.000527+0000","last_active":"2024-07-04T17:27:28.015649+0000","last_peered":"2024-07-04T17:27:28.015649+0000","last_clean":"2024-07-04T17:27:28.015649+0000","last_became_active":"2024-07-04T17:27:28.000096+0000","last_became_peered":"2024-07-04T17:27:28.000096+0000","last_unstale":"2024-07-04T17:27:28.015649+0000","last_undegraded":"2024-07-04T17:27:28.015649+0000","last_fullsized":"2024-07-04T17:27:28.015649+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:26:43.565949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.977428+0000","last_change":"2024-07-04T17:27:07.896753+0000","last_active":"2024-07-04T17:27:27.977428+0000","last_peered":"2024-07-04T17:27:27.977428+0000","last_clean":"2024-07-04T17:27:27.977428+0000","last_became_active":"2024-07-04T17:27:07.896053+0000","last_became_peered":"2024-07-04T17:27:07.896053+0000","last_unstale":"2024-07-04T17:27:27.977428+0000","last_undegraded":"2024-07-04T17:27:27.977428+0000","last_fullsized":"2024-07-04T17:27:27.977428+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:56:08.436905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.009332+0000","last_change":"2024-07-04T17:27:27.997002+0000","last_active":"2024-07-04T17:27:28.009332+0000","last_peered":"2024-07-04T17:27:28.009332+0000","last_clean":"2024-07-04T17:27:28.009332+0000","last_became_active":"2024-07-04T17:27:27.996658+0000","last_became_peered":"2024-07-04T17:27:27.996658+0000","last_unstale":"2024-07-04T17:27:28.009332+0000","last_undegraded":"2024-07-04T17:27:28.009332+0000","last_fullsized":"2024-07-04T17:27:28.009332+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:11:13.050081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.969297+0000","last_change":"2024-07-04T17:27:07.898354+0000","last_active":"2024-07-04T17:27:27.969297+0000","last_peered":"2024-07-04T17:27:27.969297+0000","last_clean":"2024-07-04T17:27:27.969297+0000","last_became_active":"2024-07-04T17:27:07.897988+0000","last_became_peered":"2024-07-04T17:27:07.897988+0000","last_unstale":"2024-07-04T17:27:27.969297+0000","last_undegraded":"2024-07-04T17:27:27.969297+0000","last_fullsized":"2024-07-04T17:27:27.969297+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:25:52.643999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.031300+0000","last_change":"2024-07-04T17:27:28.025891+0000","last_active":"2024-07-04T17:27:28.031300+0000","last_peered":"2024-07-04T17:27:28.031300+0000","last_clean":"2024-07-04T17:27:28.031300+0000","last_became_active":"2024-07-04T17:27:28.024996+0000","last_became_peered":"2024-07-04T17:27:28.024996+0000","last_unstale":"2024-07-04T17:27:28.031300+0000","last_undegraded":"2024-07-04T17:27:28.031300+0000","last_fullsized":"2024-07-04T17:27:28.031300+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:58:39.364744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.967770+0000","last_change":"2024-07-04T17:27:07.883846+0000","last_active":"2024-07-04T17:27:27.967770+0000","last_peered":"2024-07-04T17:27:27.967770+0000","last_clean":"2024-07-04T17:27:27.967770+0000","last_became_active":"2024-07-04T17:27:07.881609+0000","last_became_peered":"2024-07-04T17:27:07.881609+0000","last_unstale":"2024-07-04T17:27:27.967770+0000","last_undegraded":"2024-07-04T17:27:27.967770+0000","last_fullsized":"2024-07-04T17:27:27.967770+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:41:01.912287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.006486+0000","last_change":"2024-07-04T17:27:27.999062+0000","last_active":"2024-07-04T17:27:28.006486+0000","last_peered":"2024-07-04T17:27:28.006486+0000","last_clean":"2024-07-04T17:27:28.006486+0000","last_became_active":"2024-07-04T17:27:27.998482+0000","last_became_peered":"2024-07-04T17:27:27.998482+0000","last_unstale":"2024-07-04T17:27:28.006486+0000","last_undegraded":"2024-07-04T17:27:28.006486+0000","last_fullsized":"2024-07-04T17:27:28.006486+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:44:51.178776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.974277+0000","last_change":"2024-07-04T17:27:07.904537+0000","last_active":"2024-07-04T17:27:27.974277+0000","last_peered":"2024-07-04T17:27:27.974277+0000","last_clean":"2024-07-04T17:27:27.974277+0000","last_became_active":"2024-07-04T17:27:07.903476+0000","last_became_peered":"2024-07-04T17:27:07.903476+0000","last_unstale":"2024-07-04T17:27:27.974277+0000","last_undegraded":"2024-07-04T17:27:27.974277+0000","last_fullsized":"2024-07-04T17:27:27.974277+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:24:58.267453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.009961+0000","last_change":"2024-07-04T17:27:27.994553+0000","last_active":"2024-07-04T17:27:28.009961+0000","last_peered":"2024-07-04T17:27:28.009961+0000","last_clean":"2024-07-04T17:27:28.009961+0000","last_became_active":"2024-07-04T17:27:27.991603+0000","last_became_peered":"2024-07-04T17:27:27.991603+0000","last_unstale":"2024-07-04T17:27:28.009961+0000","last_undegraded":"2024-07-04T17:27:28.009961+0000","last_fullsized":"2024-07-04T17:27:28.009961+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:20:48.237599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.982718+0000","last_change":"2024-07-04T17:27:07.887157+0000","last_active":"2024-07-04T17:27:27.982718+0000","last_peered":"2024-07-04T17:27:27.982718+0000","last_clean":"2024-07-04T17:27:27.982718+0000","last_became_active":"2024-07-04T17:27:07.886783+0000","last_became_peered":"2024-07-04T17:27:07.886783+0000","last_unstale":"2024-07-04T17:27:27.982718+0000","last_undegraded":"2024-07-04T17:27:27.982718+0000","last_fullsized":"2024-07-04T17:27:27.982718+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:18:17.970386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.027260+0000","last_change":"2024-07-04T17:27:28.016402+0000","last_active":"2024-07-04T17:27:28.027260+0000","last_peered":"2024-07-04T17:27:28.027260+0000","last_clean":"2024-07-04T17:27:28.027260+0000","last_became_active":"2024-07-04T17:27:28.016008+0000","last_became_peered":"2024-07-04T17:27:28.016008+0000","last_unstale":"2024-07-04T17:27:28.027260+0000","last_undegraded":"2024-07-04T17:27:28.027260+0000","last_fullsized":"2024-07-04T17:27:28.027260+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:31:34.667696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.980487+0000","last_change":"2024-07-04T17:27:07.916151+0000","last_active":"2024-07-04T17:27:27.980487+0000","last_peered":"2024-07-04T17:27:27.980487+0000","last_clean":"2024-07-04T17:27:27.980487+0000","last_became_active":"2024-07-04T17:27:07.909999+0000","last_became_peered":"2024-07-04T17:27:07.909999+0000","last_unstale":"2024-07-04T17:27:27.980487+0000","last_undegraded":"2024-07-04T17:27:27.980487+0000","last_fullsized":"2024-07-04T17:27:27.980487+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:38:38.667113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.030945+0000","last_change":"2024-07-04T17:27:28.022637+0000","last_active":"2024-07-04T17:27:28.030945+0000","last_peered":"2024-07-04T17:27:28.030945+0000","last_clean":"2024-07-04T17:27:28.030945+0000","last_became_active":"2024-07-04T17:27:28.022337+0000","last_became_peered":"2024-07-04T17:27:28.022337+0000","last_unstale":"2024-07-04T17:27:28.030945+0000","last_undegraded":"2024-07-04T17:27:28.030945+0000","last_fullsized":"2024-07-04T17:27:28.030945+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:04:48.577717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.974892+0000","last_change":"2024-07-04T17:27:07.893065+0000","last_active":"2024-07-04T17:27:27.974892+0000","last_peered":"2024-07-04T17:27:27.974892+0000","last_clean":"2024-07-04T17:27:27.974892+0000","last_became_active":"2024-07-04T17:27:07.892736+0000","last_became_peered":"2024-07-04T17:27:07.892736+0000","last_unstale":"2024-07-04T17:27:27.974892+0000","last_undegraded":"2024-07-04T17:27:27.974892+0000","last_fullsized":"2024-07-04T17:27:27.974892+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:18:57.498497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.030572+0000","last_change":"2024-07-04T17:27:28.020357+0000","last_active":"2024-07-04T17:27:28.030572+0000","last_peered":"2024-07-04T17:27:28.030572+0000","last_clean":"2024-07-04T17:27:28.030572+0000","last_became_active":"2024-07-04T17:27:28.019814+0000","last_became_peered":"2024-07-04T17:27:28.019814+0000","last_unstale":"2024-07-04T17:27:28.030572+0000","last_undegraded":"2024-07-04T17:27:28.030572+0000","last_fullsized":"2024-07-04T17:27:28.030572+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:53:07.282774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.965488+0000","last_change":"2024-07-04T17:27:07.883614+0000","last_active":"2024-07-04T17:27:27.965488+0000","last_peered":"2024-07-04T17:27:27.965488+0000","last_clean":"2024-07-04T17:27:27.965488+0000","last_became_active":"2024-07-04T17:27:07.883289+0000","last_became_peered":"2024-07-04T17:27:07.883289+0000","last_unstale":"2024-07-04T17:27:27.965488+0000","last_undegraded":"2024-07-04T17:27:27.965488+0000","last_fullsized":"2024-07-04T17:27:27.965488+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:54:45.341155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.015735+0000","last_change":"2024-07-04T17:27:28.011027+0000","last_active":"2024-07-04T17:27:28.015735+0000","last_peered":"2024-07-04T17:27:28.015735+0000","last_clean":"2024-07-04T17:27:28.015735+0000","last_became_active":"2024-07-04T17:27:28.010612+0000","last_became_peered":"2024-07-04T17:27:28.010612+0000","last_unstale":"2024-07-04T17:27:28.015735+0000","last_undegraded":"2024-07-04T17:27:28.015735+0000","last_fullsized":"2024-07-04T17:27:28.015735+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:41:06.228068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.978309+0000","last_change":"2024-07-04T17:27:07.877401+0000","last_active":"2024-07-04T17:27:27.978309+0000","last_peered":"2024-07-04T17:27:27.978309+0000","last_clean":"2024-07-04T17:27:27.978309+0000","last_became_active":"2024-07-04T17:27:07.877014+0000","last_became_peered":"2024-07-04T17:27:07.877014+0000","last_unstale":"2024-07-04T17:27:27.978309+0000","last_undegraded":"2024-07-04T17:27:27.978309+0000","last_fullsized":"2024-07-04T17:27:27.978309+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:35:30.543101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.009265+0000","last_change":"2024-07-04T17:27:27.995614+0000","last_active":"2024-07-04T17:27:28.009265+0000","last_peered":"2024-07-04T17:27:28.009265+0000","last_clean":"2024-07-04T17:27:28.009265+0000","last_became_active":"2024-07-04T17:27:27.994228+0000","last_became_peered":"2024-07-04T17:27:27.994228+0000","last_unstale":"2024-07-04T17:27:28.009265+0000","last_undegraded":"2024-07-04T17:27:28.009265+0000","last_fullsized":"2024-07-04T17:27:28.009265+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:54:06.893081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.971609+0000","last_change":"2024-07-04T17:27:07.884482+0000","last_active":"2024-07-04T17:27:27.971609+0000","last_peered":"2024-07-04T17:27:27.971609+0000","last_clean":"2024-07-04T17:27:27.971609+0000","last_became_active":"2024-07-04T17:27:07.882978+0000","last_became_peered":"2024-07-04T17:27:07.882978+0000","last_unstale":"2024-07-04T17:27:27.971609+0000","last_undegraded":"2024-07-04T17:27:27.971609+0000","last_fullsized":"2024-07-04T17:27:27.971609+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:40:02.282743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.026970+0000","last_change":"2024-07-04T17:27:28.001068+0000","last_active":"2024-07-04T17:27:28.026970+0000","last_peered":"2024-07-04T17:27:28.026970+0000","last_clean":"2024-07-04T17:27:28.026970+0000","last_became_active":"2024-07-04T17:27:27.998408+0000","last_became_peered":"2024-07-04T17:27:27.998408+0000","last_unstale":"2024-07-04T17:27:28.026970+0000","last_undegraded":"2024-07-04T17:27:28.026970+0000","last_fullsized":"2024-07-04T17:27:28.026970+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:55:07.289164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.967196+0000","last_change":"2024-07-04T17:27:07.873017+0000","last_active":"2024-07-04T17:27:27.967196+0000","last_peered":"2024-07-04T17:27:27.967196+0000","last_clean":"2024-07-04T17:27:27.967196+0000","last_became_active":"2024-07-04T17:27:07.872715+0000","last_became_peered":"2024-07-04T17:27:07.872715+0000","last_unstale":"2024-07-04T17:27:27.967196+0000","last_undegraded":"2024-07-04T17:27:27.967196+0000","last_fullsized":"2024-07-04T17:27:27.967196+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:09:07.885707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.021412+0000","last_change":"2024-07-04T17:27:28.016351+0000","last_active":"2024-07-04T17:27:28.021412+0000","last_peered":"2024-07-04T17:27:28.021412+0000","last_clean":"2024-07-04T17:27:28.021412+0000","last_became_active":"2024-07-04T17:27:28.011415+0000","last_became_peered":"2024-07-04T17:27:28.011415+0000","last_unstale":"2024-07-04T17:27:28.021412+0000","last_undegraded":"2024-07-04T17:27:28.021412+0000","last_fullsized":"2024-07-04T17:27:28.021412+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:16:54.856497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.958657+0000","last_change":"2024-07-04T17:27:07.879943+0000","last_active":"2024-07-04T17:27:27.958657+0000","last_peered":"2024-07-04T17:27:27.958657+0000","last_clean":"2024-07-04T17:27:27.958657+0000","last_became_active":"2024-07-04T17:27:07.875927+0000","last_became_peered":"2024-07-04T17:27:07.875927+0000","last_unstale":"2024-07-04T17:27:27.958657+0000","last_undegraded":"2024-07-04T17:27:27.958657+0000","last_fullsized":"2024-07-04T17:27:27.958657+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:38:48.110832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.006292+0000","last_change":"2024-07-04T17:27:27.999385+0000","last_active":"2024-07-04T17:27:28.006292+0000","last_peered":"2024-07-04T17:27:28.006292+0000","last_clean":"2024-07-04T17:27:28.006292+0000","last_became_active":"2024-07-04T17:27:27.998592+0000","last_became_peered":"2024-07-04T17:27:27.998592+0000","last_unstale":"2024-07-04T17:27:28.006292+0000","last_undegraded":"2024-07-04T17:27:28.006292+0000","last_fullsized":"2024-07-04T17:27:28.006292+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:55:05.451797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.978100+0000","last_change":"2024-07-04T17:27:07.896119+0000","last_active":"2024-07-04T17:27:27.978100+0000","last_peered":"2024-07-04T17:27:27.978100+0000","last_clean":"2024-07-04T17:27:27.978100+0000","last_became_active":"2024-07-04T17:27:07.895583+0000","last_became_peered":"2024-07-04T17:27:07.895583+0000","last_unstale":"2024-07-04T17:27:27.978100+0000","last_undegraded":"2024-07-04T17:27:27.978100+0000","last_fullsized":"2024-07-04T17:27:27.978100+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:49:26.007232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.033363+0000","last_change":"2024-07-04T17:27:28.031337+0000","last_active":"2024-07-04T17:27:28.033363+0000","last_peered":"2024-07-04T17:27:28.033363+0000","last_clean":"2024-07-04T17:27:28.033363+0000","last_became_active":"2024-07-04T17:27:28.029954+0000","last_became_peered":"2024-07-04T17:27:28.029954+0000","last_unstale":"2024-07-04T17:27:28.033363+0000","last_undegraded":"2024-07-04T17:27:28.033363+0000","last_fullsized":"2024-07-04T17:27:28.033363+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:07:18.483120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.970596+0000","last_change":"2024-07-04T17:27:07.886093+0000","last_active":"2024-07-04T17:27:27.970596+0000","last_peered":"2024-07-04T17:27:27.970596+0000","last_clean":"2024-07-04T17:27:27.970596+0000","last_became_active":"2024-07-04T17:27:07.885701+0000","last_became_peered":"2024-07-04T17:27:07.885701+0000","last_unstale":"2024-07-04T17:27:27.970596+0000","last_undegraded":"2024-07-04T17:27:27.970596+0000","last_fullsized":"2024-07-04T17:27:27.970596+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:43:39.860474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.027029+0000","last_change":"2024-07-04T17:27:28.023068+0000","last_active":"2024-07-04T17:27:28.027029+0000","last_peered":"2024-07-04T17:27:28.027029+0000","last_clean":"2024-07-04T17:27:28.027029+0000","last_became_active":"2024-07-04T17:27:28.022773+0000","last_became_peered":"2024-07-04T17:27:28.022773+0000","last_unstale":"2024-07-04T17:27:28.027029+0000","last_undegraded":"2024-07-04T17:27:28.027029+0000","last_fullsized":"2024-07-04T17:27:28.027029+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:50:15.117673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.984101+0000","last_change":"2024-07-04T17:27:07.893125+0000","last_active":"2024-07-04T17:27:27.984101+0000","last_peered":"2024-07-04T17:27:27.984101+0000","last_clean":"2024-07-04T17:27:27.984101+0000","last_became_active":"2024-07-04T17:27:07.892921+0000","last_became_peered":"2024-07-04T17:27:07.892921+0000","last_unstale":"2024-07-04T17:27:27.984101+0000","last_undegraded":"2024-07-04T17:27:27.984101+0000","last_fullsized":"2024-07-04T17:27:27.984101+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:53:39.500774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.030545+0000","last_change":"2024-07-04T17:27:28.020064+0000","last_active":"2024-07-04T17:27:28.030545+0000","last_peered":"2024-07-04T17:27:28.030545+0000","last_clean":"2024-07-04T17:27:28.030545+0000","last_became_active":"2024-07-04T17:27:28.019528+0000","last_became_peered":"2024-07-04T17:27:28.019528+0000","last_unstale":"2024-07-04T17:27:28.030545+0000","last_undegraded":"2024-07-04T17:27:28.030545+0000","last_fullsized":"2024-07-04T17:27:28.030545+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:47:31.123344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.976999+0000","last_change":"2024-07-04T17:27:07.871832+0000","last_active":"2024-07-04T17:27:27.976999+0000","last_peered":"2024-07-04T17:27:27.976999+0000","last_clean":"2024-07-04T17:27:27.976999+0000","last_became_active":"2024-07-04T17:27:07.870928+0000","last_became_peered":"2024-07-04T17:27:07.870928+0000","last_unstale":"2024-07-04T17:27:27.976999+0000","last_undegraded":"2024-07-04T17:27:27.976999+0000","last_fullsized":"2024-07-04T17:27:27.976999+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:06:36.067329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.010197+0000","last_change":"2024-07-04T17:27:27.993394+0000","last_active":"2024-07-04T17:27:28.010197+0000","last_peered":"2024-07-04T17:27:28.010197+0000","last_clean":"2024-07-04T17:27:28.010197+0000","last_became_active":"2024-07-04T17:27:27.993050+0000","last_became_peered":"2024-07-04T17:27:27.993050+0000","last_unstale":"2024-07-04T17:27:28.010197+0000","last_undegraded":"2024-07-04T17:27:28.010197+0000","last_fullsized":"2024-07-04T17:27:28.010197+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:33:10.722625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.976542+0000","last_change":"2024-07-04T17:27:07.897001+0000","last_active":"2024-07-04T17:27:27.976542+0000","last_peered":"2024-07-04T17:27:27.976542+0000","last_clean":"2024-07-04T17:27:27.976542+0000","last_became_active":"2024-07-04T17:27:07.896453+0000","last_became_peered":"2024-07-04T17:27:07.896453+0000","last_unstale":"2024-07-04T17:27:27.976542+0000","last_undegraded":"2024-07-04T17:27:27.976542+0000","last_fullsized":"2024-07-04T17:27:27.976542+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:20:22.623733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.009309+0000","last_change":"2024-07-04T17:27:27.996306+0000","last_active":"2024-07-04T17:27:28.009309+0000","last_peered":"2024-07-04T17:27:28.009309+0000","last_clean":"2024-07-04T17:27:28.009309+0000","last_became_active":"2024-07-04T17:27:27.993929+0000","last_became_peered":"2024-07-04T17:27:27.993929+0000","last_unstale":"2024-07-04T17:27:28.009309+0000","last_undegraded":"2024-07-04T17:27:28.009309+0000","last_fullsized":"2024-07-04T17:27:28.009309+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:08:46.562654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.969405+0000","last_change":"2024-07-04T17:27:07.899412+0000","last_active":"2024-07-04T17:27:27.969405+0000","last_peered":"2024-07-04T17:27:27.969405+0000","last_clean":"2024-07-04T17:27:27.969405+0000","last_became_active":"2024-07-04T17:27:07.898386+0000","last_became_peered":"2024-07-04T17:27:07.898386+0000","last_unstale":"2024-07-04T17:27:27.969405+0000","last_undegraded":"2024-07-04T17:27:27.969405+0000","last_fullsized":"2024-07-04T17:27:27.969405+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:11:22.527978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.016382+0000","last_change":"2024-07-04T17:27:28.002162+0000","last_active":"2024-07-04T17:27:28.016382+0000","last_peered":"2024-07-04T17:27:28.016382+0000","last_clean":"2024-07-04T17:27:28.016382+0000","last_became_active":"2024-07-04T17:27:28.001587+0000","last_became_peered":"2024-07-04T17:27:28.001587+0000","last_unstale":"2024-07-04T17:27:28.016382+0000","last_undegraded":"2024-07-04T17:27:28.016382+0000","last_fullsized":"2024-07-04T17:27:28.016382+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:24:46.523771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.971601+0000","last_change":"2024-07-04T17:27:07.894901+0000","last_active":"2024-07-04T17:27:27.971601+0000","last_peered":"2024-07-04T17:27:27.971601+0000","last_clean":"2024-07-04T17:27:27.971601+0000","last_became_active":"2024-07-04T17:27:07.894469+0000","last_became_peered":"2024-07-04T17:27:07.894469+0000","last_unstale":"2024-07-04T17:27:27.971601+0000","last_undegraded":"2024-07-04T17:27:27.971601+0000","last_fullsized":"2024-07-04T17:27:27.971601+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:13:09.667367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.007066+0000","last_change":"2024-07-04T17:27:27.991828+0000","last_active":"2024-07-04T17:27:28.007066+0000","last_peered":"2024-07-04T17:27:28.007066+0000","last_clean":"2024-07-04T17:27:28.007066+0000","last_became_active":"2024-07-04T17:27:27.991514+0000","last_became_peered":"2024-07-04T17:27:27.991514+0000","last_unstale":"2024-07-04T17:27:28.007066+0000","last_undegraded":"2024-07-04T17:27:28.007066+0000","last_fullsized":"2024-07-04T17:27:28.007066+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:29:39.649214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.969117+0000","last_change":"2024-07-04T17:27:07.883415+0000","last_active":"2024-07-04T17:27:27.969117+0000","last_peered":"2024-07-04T17:27:27.969117+0000","last_clean":"2024-07-04T17:27:27.969117+0000","last_became_active":"2024-07-04T17:27:07.878729+0000","last_became_peered":"2024-07-04T17:27:07.878729+0000","last_unstale":"2024-07-04T17:27:27.969117+0000","last_undegraded":"2024-07-04T17:27:27.969117+0000","last_fullsized":"2024-07-04T17:27:27.969117+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:47:16.318473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.015982+0000","last_change":"2024-07-04T17:27:28.000905+0000","last_active":"2024-07-04T17:27:28.015982+0000","last_peered":"2024-07-04T17:27:28.015982+0000","last_clean":"2024-07-04T17:27:28.015982+0000","last_became_active":"2024-07-04T17:27:28.000592+0000","last_became_peered":"2024-07-04T17:27:28.000592+0000","last_unstale":"2024-07-04T17:27:28.015982+0000","last_undegraded":"2024-07-04T17:27:28.015982+0000","last_fullsized":"2024-07-04T17:27:28.015982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:05:16.540103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.977161+0000","last_change":"2024-07-04T17:27:07.905295+0000","last_active":"2024-07-04T17:27:27.977161+0000","last_peered":"2024-07-04T17:27:27.977161+0000","last_clean":"2024-07-04T17:27:27.977161+0000","last_became_active":"2024-07-04T17:27:07.904234+0000","last_became_peered":"2024-07-04T17:27:07.904234+0000","last_unstale":"2024-07-04T17:27:27.977161+0000","last_undegraded":"2024-07-04T17:27:27.977161+0000","last_fullsized":"2024-07-04T17:27:27.977161+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:27:04.910602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.031512+0000","last_change":"2024-07-04T17:27:28.024141+0000","last_active":"2024-07-04T17:27:28.031512+0000","last_peered":"2024-07-04T17:27:28.031512+0000","last_clean":"2024-07-04T17:27:28.031512+0000","last_became_active":"2024-07-04T17:27:28.023845+0000","last_became_peered":"2024-07-04T17:27:28.023845+0000","last_unstale":"2024-07-04T17:27:28.031512+0000","last_undegraded":"2024-07-04T17:27:28.031512+0000","last_fullsized":"2024-07-04T17:27:28.031512+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:25:29.884373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.970911+0000","last_change":"2024-07-04T17:27:07.889861+0000","last_active":"2024-07-04T17:27:27.970911+0000","last_peered":"2024-07-04T17:27:27.970911+0000","last_clean":"2024-07-04T17:27:27.970911+0000","last_became_active":"2024-07-04T17:27:07.889003+0000","last_became_peered":"2024-07-04T17:27:07.889003+0000","last_unstale":"2024-07-04T17:27:27.970911+0000","last_undegraded":"2024-07-04T17:27:27.970911+0000","last_fullsized":"2024-07-04T17:27:27.970911+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:38:38.667113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.034412+0000","last_change":"2024-07-04T17:27:28.031644+0000","last_active":"2024-07-04T17:27:28.034412+0000","last_peered":"2024-07-04T17:27:28.034412+0000","last_clean":"2024-07-04T17:27:28.034412+0000","last_became_active":"2024-07-04T17:27:28.031351+0000","last_became_peered":"2024-07-04T17:27:28.031351+0000","last_unstale":"2024-07-04T17:27:28.034412+0000","last_undegraded":"2024-07-04T17:27:28.034412+0000","last_fullsized":"2024-07-04T17:27:28.034412+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:48:02.847129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.975314+0000","last_change":"2024-07-04T17:27:07.891626+0000","last_active":"2024-07-04T17:27:27.975314+0000","last_peered":"2024-07-04T17:27:27.975314+0000","last_clean":"2024-07-04T17:27:27.975314+0000","last_became_active":"2024-07-04T17:27:07.891070+0000","last_became_peered":"2024-07-04T17:27:07.891070+0000","last_unstale":"2024-07-04T17:27:27.975314+0000","last_undegraded":"2024-07-04T17:27:27.975314+0000","last_fullsized":"2024-07-04T17:27:27.975314+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:12:04.631448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:28.033426+0000","last_change":"2024-07-04T17:27:28.031048+0000","last_active":"2024-07-04T17:27:28.033426+0000","last_peered":"2024-07-04T17:27:28.033426+0000","last_clean":"2024-07-04T17:27:28.033426+0000","last_became_active":"2024-07-04T17:27:28.030694+0000","last_became_peered":"2024-07-04T17:27:28.030694+0000","last_unstale":"2024-07-04T17:27:28.033426+0000","last_undegraded":"2024-07-04T17:27:28.033426+0000","last_fullsized":"2024-07-04T17:27:28.033426+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:01:00.610669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.983130+0000","last_change":"2024-07-04T17:27:07.884739+0000","last_active":"2024-07-04T17:27:27.983130+0000","last_peered":"2024-07-04T17:27:27.983130+0000","last_clean":"2024-07-04T17:27:27.983130+0000","last_became_active":"2024-07-04T17:27:07.884425+0000","last_became_peered":"2024-07-04T17:27:07.884425+0000","last_unstale":"2024-07-04T17:27:27.983130+0000","last_undegraded":"2024-07-04T17:27:27.983130+0000","last_fullsized":"2024-07-04T17:27:27.983130+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:20:28.358639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.999011+0000","last_change":"2024-07-04T17:27:27.988801+0000","last_active":"2024-07-04T17:27:27.999011+0000","last_peered":"2024-07-04T17:27:27.999011+0000","last_clean":"2024-07-04T17:27:27.999011+0000","last_became_active":"2024-07-04T17:27:27.988456+0000","last_became_peered":"2024-07-04T17:27:27.988456+0000","last_unstale":"2024-07-04T17:27:27.999011+0000","last_undegraded":"2024-07-04T17:27:27.999011+0000","last_fullsized":"2024-07-04T17:27:27.999011+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:26.926532+0000","last_clean_scrub_stamp":"2024-07-04T17:27:26.926532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:04:48.577717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3f","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-07-04T17:27:27.982863+0000","last_change":"2024-07-04T17:27:07.916245+0000","last_active":"2024-07-04T17:27:27.982863+0000","last_peered":"2024-07-04T17:27:27.982863+0000","last_clean":"2024-07-04T17:27:27.982863+0000","last_became_active":"2024-07-04T17:27:07.912310+0000","last_became_peered":"2024-07-04T17:27:07.912310+0000","last_unstale":"2024-07-04T17:27:27.982863+0000","last_undegraded":"2024-07-04T17:27:27.982863+0000","last_fullsized":"2024-07-04T17:27:27.982863+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:27:06.815890+0000","last_clean_scrub_stamp":"2024-07-04T17:27:06.815890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:23.798405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":3},{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":4107,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738388,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31468,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340372,"statfs":{"total":96636764160,"available":96604540928,"internally_reserved":0,"allocated":286720,"data_stored":96246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738389,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":909312,"data_stored":690631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738387,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":872448,"data_stored":685044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:27:33.381 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-04T17:27:33.381 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:27:33.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.712+0000 7fe4b1a00640 1 -- 172.21.15.42:0/3143170662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4ac0b2d20 msgr2=0x7fe4ac0bb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:33.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.712+0000 7fe4b1a00640 1 --2- 172.21.15.42:0/3143170662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4ac0b2d20 0x7fe4ac0bb1f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe498005fd0 tx=0x7fe49802a560 comp rx=0 tx=0).stop 2024-07-04T17:27:33.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.712+0000 7fe4b1a00640 1 -- 172.21.15.42:0/3143170662 shutdown_connections 2024-07-04T17:27:33.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.712+0000 7fe4b1a00640 1 --2- 172.21.15.42:0/3143170662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4ac0b2d20 0x7fe4ac0bb1f0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:33.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.712+0000 7fe4b1a00640 1 -- 172.21.15.42:0/3143170662 >> 172.21.15.42:0/3143170662 conn(0x7fe4ac0bf700 msgr2=0x7fe4ac0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:33.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.712+0000 7fe4b1a00640 1 -- 172.21.15.42:0/3143170662 shutdown_connections 2024-07-04T17:27:33.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.712+0000 7fe4b1a00640 1 -- 172.21.15.42:0/3143170662 wait complete. 2024-07-04T17:27:33.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.714+0000 7fe4b1a00640 1 Processor -- start 2024-07-04T17:27:33.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.715+0000 7fe4b1a00640 1 -- start start 2024-07-04T17:27:33.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.716+0000 7fe4b1a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4ac0b2d20 0x7fe4ac0aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:33.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.716+0000 7fe4b1a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac06c020 con 0x7fe4ac0b2d20 2024-07-04T17:27:33.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.716+0000 7fe4aaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4ac0b2d20 0x7fe4ac0aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:33.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.716+0000 7fe4aaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4ac0b2d20 0x7fe4ac0aebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36766/0 (socket says 172.21.15.42:36766) 2024-07-04T17:27:33.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.716+0000 7fe4aaa00640 1 -- 172.21.15.42:0/517229919 learned_addr learned my addr 172.21.15.42:0/517229919 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:33.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.717+0000 7fe4aaa00640 1 -- 172.21.15.42:0/517229919 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4ac071440 con 0x7fe4ac0b2d20 2024-07-04T17:27:33.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.717+0000 7fe4aaa00640 1 --2- 172.21.15.42:0/517229919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4ac0b2d20 0x7fe4ac0aebb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe49c00c380 tx=0x7fe49c00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:33.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.717+0000 7fe4a2a00640 1 -- 172.21.15.42:0/517229919 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe49c017070 con 0x7fe4ac0b2d20 2024-07-04T17:27:33.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.717+0000 7fe4a2a00640 1 -- 172.21.15.42:0/517229919 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe49c0052f0 con 0x7fe4ac0b2d20 2024-07-04T17:27:33.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.717+0000 7fe4a2a00640 1 -- 172.21.15.42:0/517229919 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe49c0035f0 con 0x7fe4ac0b2d20 2024-07-04T17:27:33.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.717+0000 7fe4b1a00640 1 -- 172.21.15.42:0/517229919 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4ac1e43f0 con 0x7fe4ac0b2d20 2024-07-04T17:27:33.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.717+0000 7fe4b1a00640 1 -- 172.21.15.42:0/517229919 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4ac137100 con 0x7fe4ac0b2d20 2024-07-04T17:27:33.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.718+0000 7fe4b1a00640 1 -- 172.21.15.42:0/517229919 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4ac1e4620 con 0x7fe4ac0b2d20 2024-07-04T17:27:33.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.719+0000 7fe4a2a00640 1 -- 172.21.15.42:0/517229919 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fe49c01a070 con 0x7fe4ac0b2d20 2024-07-04T17:27:33.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.719+0000 7fe4a2a00640 1 --2- 172.21.15.42:0/517229919 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fe48c03d2b0 0x7fe48c03f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:33.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.719+0000 7fe4a2a00640 1 -- 172.21.15.42:0/517229919 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4303+0+0 (secure 0 0 0) 0x7fe49c050c50 con 0x7fe4ac0b2d20 2024-07-04T17:27:33.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.719+0000 7fe4aa000640 1 --2- 172.21.15.42:0/517229919 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fe48c03d2b0 0x7fe48c03f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:33.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.720+0000 7fe4aa000640 1 --2- 172.21.15.42:0/517229919 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fe48c03d2b0 0x7fe48c03f7c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe498033000 tx=0x7fe498004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:33.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.724+0000 7fe4a2a00640 1 -- 172.21.15.42:0/517229919 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fe49c016170 con 0x7fe4ac0b2d20 2024-07-04T17:27:33.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.980+0000 7fe4b1a00640 1 -- 172.21.15.42:0/517229919 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4ac06c710 con 0x7fe4ac0b2d20 2024-07-04T17:27:33.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.981+0000 7fe4a2a00640 1 -- 172.21.15.42:0/517229919 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+7359 (secure 0 0 0) 0x7fe49c0192c0 con 0x7fe4ac0b2d20 2024-07-04T17:27:33.983 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:27:33.983 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":23,"btime":"2024-07-04T17:27:30:978140+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/399707138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":399707138},{"type":"v1","addr":"172.21.15.42:6827","nonce":399707138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4414,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1510714231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1510714231},{"type":"v1","addr":"172.21.15.42:6829","nonce":1510714231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/3069680926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":3069680926},{"type":"v1","addr":"172.21.15.42:6831","nonce":3069680926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:27:07.402007+0000","modified":"2024-07-04T17:27:10.462342+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4423":{"gid":4423,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.42:6835/807364821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":807364821},{"type":"v1","addr":"172.21.15.42:6835","nonce":807364821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4423,"qdb_cluster":[4423]},"id":2},{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:27:27.513977+0000","modified":"2024-07-04T17:27:30.569774+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4422":{"gid":4422,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.42:6833/1348446958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1348446958},{"type":"v1","addr":"172.21.15.42:6833","nonce":1348446958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4422,"qdb_cluster":[4422]},"id":3}]} 2024-07-04T17:27:33.984 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 23 2024-07-04T17:27:33.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.984+0000 7fe4b1a00640 1 -- 172.21.15.42:0/517229919 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fe48c03d2b0 msgr2=0x7fe48c03f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:33.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.984+0000 7fe4b1a00640 1 --2- 172.21.15.42:0/517229919 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fe48c03d2b0 0x7fe48c03f7c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe498033000 tx=0x7fe498004730 comp rx=0 tx=0).stop 2024-07-04T17:27:33.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.984+0000 7fe4b1a00640 1 -- 172.21.15.42:0/517229919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4ac0b2d20 msgr2=0x7fe4ac0aebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:33.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.984+0000 7fe4b1a00640 1 --2- 172.21.15.42:0/517229919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4ac0b2d20 0x7fe4ac0aebb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe49c00c380 tx=0x7fe49c00c840 comp rx=0 tx=0).stop 2024-07-04T17:27:33.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.985+0000 7fe4b1a00640 1 -- 172.21.15.42:0/517229919 shutdown_connections 2024-07-04T17:27:33.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.985+0000 7fe4b1a00640 1 --2- 172.21.15.42:0/517229919 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fe48c03d2b0 0x7fe48c03f7c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:33.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.985+0000 7fe4b1a00640 1 --2- 172.21.15.42:0/517229919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4ac0b2d20 0x7fe4ac0aebb0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:33.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.985+0000 7fe4b1a00640 1 -- 172.21.15.42:0/517229919 >> 172.21.15.42:0/517229919 conn(0x7fe4ac0bf700 msgr2=0x7fe4ac0bd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:33.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.985+0000 7fe4b1a00640 1 -- 172.21.15.42:0/517229919 shutdown_connections 2024-07-04T17:27:33.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:33.985+0000 7fe4b1a00640 1 -- 172.21.15.42:0/517229919 wait complete. 2024-07-04T17:27:34.513 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-04T17:27:27.513977+0000', 'modified': '2024-07-04T17:27:30.569774+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4422}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4422': {'gid': 4422, 'name': 'a', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.42:6833/1348446958', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6832', 'nonce': 1348446958}, {'type': 'v1', 'addr': '172.21.15.42:6833', 'nonce': 1348446958}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'backup_fs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4422, 'qdb_cluster': [4422]} 2024-07-04T17:27:34.513 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-04T17:27:34.514 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-07-04T17:27:34.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.808+0000 7f5bc6400640 1 -- 172.21.15.42:0/508713986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bc00b37d0 msgr2=0x7f5bc00b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:34.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.808+0000 7f5bc6400640 1 --2- 172.21.15.42:0/508713986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bc00b37d0 0x7f5bc00b01f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8005fd0 tx=0x7f5ba802a5f0 comp rx=0 tx=0).stop 2024-07-04T17:27:34.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.809+0000 7f5bc6400640 1 -- 172.21.15.42:0/508713986 shutdown_connections 2024-07-04T17:27:34.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.809+0000 7f5bc6400640 1 --2- 172.21.15.42:0/508713986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bc00b37d0 0x7f5bc00b01f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:34.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.809+0000 7f5bc6400640 1 -- 172.21.15.42:0/508713986 >> 172.21.15.42:0/508713986 conn(0x7f5bc00bf700 msgr2=0x7f5bc00af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:34.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.809+0000 7f5bc6400640 1 -- 172.21.15.42:0/508713986 shutdown_connections 2024-07-04T17:27:34.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.809+0000 7f5bc6400640 1 -- 172.21.15.42:0/508713986 wait complete. 2024-07-04T17:27:34.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.811+0000 7f5bc6400640 1 Processor -- start 2024-07-04T17:27:34.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.811+0000 7f5bc6400640 1 -- start start 2024-07-04T17:27:34.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.812+0000 7f5bc6400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bc00b2d20 0x7f5bc01d4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:34.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.812+0000 7f5bc6400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc006ce30 con 0x7f5bc00b2d20 2024-07-04T17:27:34.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.812+0000 7f5bbf400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bc00b2d20 0x7f5bc01d4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:34.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.812+0000 7f5bbf400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bc00b2d20 0x7f5bc01d4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57004/0 (socket says 172.21.15.42:57004) 2024-07-04T17:27:34.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.812+0000 7f5bbf400640 1 -- 172.21.15.42:0/904918884 learned_addr learned my addr 172.21.15.42:0/904918884 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:34.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.813+0000 7f5bbf400640 1 -- 172.21.15.42:0/904918884 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc01d5f60 con 0x7f5bc00b2d20 2024-07-04T17:27:34.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.813+0000 7f5bbf400640 1 --2- 172.21.15.42:0/904918884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bc00b2d20 0x7f5bc01d4c00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0001eb0 tx=0x7f5bb000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:34.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.813+0000 7f5bb7400640 1 -- 172.21.15.42:0/904918884 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bb0017070 con 0x7f5bc00b2d20 2024-07-04T17:27:34.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.813+0000 7f5bb7400640 1 -- 172.21.15.42:0/904918884 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5bb00052f0 con 0x7f5bc00b2d20 2024-07-04T17:27:34.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.813+0000 7f5bb7400640 1 -- 172.21.15.42:0/904918884 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bb00035f0 con 0x7f5bc00b2d20 2024-07-04T17:27:34.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.813+0000 7f5bc6400640 1 -- 172.21.15.42:0/904918884 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bc01d52b0 con 0x7f5bc00b2d20 2024-07-04T17:27:34.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.813+0000 7f5bc6400640 1 -- 172.21.15.42:0/904918884 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bc013cad0 con 0x7f5bc00b2d20 2024-07-04T17:27:34.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.815+0000 7f5bb7400640 1 -- 172.21.15.42:0/904918884 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f5bb001a070 con 0x7f5bc00b2d20 2024-07-04T17:27:34.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.815+0000 7f5bb7400640 1 --2- 172.21.15.42:0/904918884 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f5b9c03d2b0 0x7f5b9c03f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:34.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.815+0000 7f5bb7400640 1 -- 172.21.15.42:0/904918884 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4303+0+0 (secure 0 0 0) 0x7f5bb0050b80 con 0x7f5bc00b2d20 2024-07-04T17:27:34.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.816+0000 7f5bb7e00640 1 --2- 172.21.15.42:0/904918884 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f5b9c03d2b0 0x7f5b9c03f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:34.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.816+0000 7f5bb7e00640 1 --2- 172.21.15.42:0/904918884 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f5b9c03d2b0 0x7f5b9c03f7c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8033000 tx=0x7f5ba8004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:34.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.817+0000 7f5bcd3d5740 1 Processor -- start 2024-07-04T17:27:34.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.817+0000 7f5bcd3d5740 1 -- start start 2024-07-04T17:27:34.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.818+0000 7f5bcd3d5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b459c02270 0x55b459fa8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:34.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.818+0000 7f5bcd3d5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b459f195b0 con 0x55b459c02270 2024-07-04T17:27:34.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.819+0000 7f5bbf400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b459c02270 0x55b459fa8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:34.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.819+0000 7f5bbf400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b459c02270 0x55b459fa8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57008/0 (socket says 172.21.15.42:57008) 2024-07-04T17:27:34.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.819+0000 7f5bbf400640 1 -- 172.21.15.42:0/755450159 learned_addr learned my addr 172.21.15.42:0/755450159 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:34.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.819+0000 7f5bbf400640 1 -- 172.21.15.42:0/755450159 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b459c6d850 con 0x55b459c02270 2024-07-04T17:27:34.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.819+0000 7f5bbf400640 1 --2- 172.21.15.42:0/755450159 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b459c02270 0x55b459fa8b70 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0005040 tx=0x7f5bb000e3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a62f74ea52b01af server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:34.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.820+0000 7f5bafe00640 1 -- 172.21.15.42:0/755450159 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bb001be20 con 0x55b459c02270 2024-07-04T17:27:34.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.820+0000 7f5bafe00640 1 -- 172.21.15.42:0/755450159 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5bb00102e0 con 0x55b459c02270 2024-07-04T17:27:34.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.820+0000 7f5bafe00640 1 -- 172.21.15.42:0/755450159 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bb0048900 con 0x55b459c02270 2024-07-04T17:27:34.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.820+0000 7f5bcd3d5740 1 -- 172.21.15.42:0/755450159 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b459c02270 msgr2=0x55b459fa8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:34.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.820+0000 7f5bcd3d5740 1 --2- 172.21.15.42:0/755450159 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b459c02270 0x55b459fa8b70 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0005040 tx=0x7f5bb000e3b0 comp rx=0 tx=0).stop 2024-07-04T17:27:34.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.820+0000 7f5bcd3d5740 1 -- 172.21.15.42:0/755450159 shutdown_connections 2024-07-04T17:27:34.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.820+0000 7f5bcd3d5740 1 --2- 172.21.15.42:0/755450159 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b459c02270 0x55b459fa8b70 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:34.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.820+0000 7f5bcd3d5740 1 -- 172.21.15.42:0/755450159 >> 172.21.15.42:0/755450159 conn(0x55b459bf63c0 msgr2=0x55b459fa7310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:34.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.821+0000 7f5bcd3d5740 1 -- 172.21.15.42:0/755450159 shutdown_connections 2024-07-04T17:27:34.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.821+0000 7f5bcd3d5740 1 -- 172.21.15.42:0/755450159 wait complete. 2024-07-04T17:27:34.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.822+0000 7f5bcd3d5740 1 -- start start 2024-07-04T17:27:34.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.822+0000 7f5bcd3d5740 1 Processor -- start 2024-07-04T17:27:34.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.823+0000 7f5bcd3d5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b459c02270 0x55b459feb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:34.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.823+0000 7f5bcd3d5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b459f0cb90 con 0x55b459c02270 2024-07-04T17:27:34.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.823+0000 7f5bcd3d5740 10 client.0 _ll_register_callbacks cb 0x55b459eb18a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T17:27:34.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.823+0000 7f5bbf400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b459c02270 0x55b459feb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:34.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.823+0000 7f5bbf400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b459c02270 0x55b459feb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57020/0 (socket says 172.21.15.42:57020) 2024-07-04T17:27:34.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.823+0000 7f5bbf400640 1 -- 172.21.15.42:0/558028238 learned_addr learned my addr 172.21.15.42:0/558028238 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:34.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.824+0000 7f5bbf400640 1 -- 172.21.15.42:0/558028238 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b459ce46b0 con 0x55b459c02270 2024-07-04T17:27:34.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.824+0000 7f5bbf400640 1 --2- 172.21.15.42:0/558028238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b459c02270 0x55b459feb7f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5bb001bc00 tx=0x7f5bb0048b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6261b50470130dd server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:34.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.824+0000 7f5bacc00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-07-04T17:27:34.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.824+0000 7f5bacc00640 1 -- 172.21.15.42:0/558028238 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bb004e2b0 con 0x55b459c02270 2024-07-04T17:27:34.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.824+0000 7f5bcd3d5740 1 -- 172.21.15.42:0/558028238 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b459ea96c0 con 0x55b459c02270 2024-07-04T17:27:34.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.824+0000 7f5bacc00640 1 -- 172.21.15.42:0/558028238 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(31..31 src has 1..31) ==== 4303+0+0 (secure 0 0 0) 0x7f5bb00482e0 con 0x55b459c02270 2024-07-04T17:27:34.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.825+0000 7f5bacc00640 1 -- 172.21.15.42:0/558028238 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f5bb0015120 con 0x55b459c02270 2024-07-04T17:27:34.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.825+0000 7f5bcd3d5740 10 client.4555 fetch_fsmap learned FSMap version 23 2024-07-04T17:27:34.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.825+0000 7f5bcd3d5740 1 -- 172.21.15.42:0/558028238 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55b459febe20 con 0x55b459c02270 2024-07-04T17:27:34.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.825+0000 7f5bacc00640 1 -- 172.21.15.42:0/558028238 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55b459ea96c0 con 0x55b459c02270 2024-07-04T17:27:34.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.825+0000 7f5bacc00640 1 -- 172.21.15.42:0/558028238 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 23) ==== 4754+0+0 (secure 0 0 0) 0x7f5bb0015340 con 0x55b459c02270 2024-07-04T17:27:34.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.825+0000 7f5bcd3d5740 10 client.4555 fetch_fsmap finished waiting for FSMap version 23 2024-07-04T17:27:34.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.825+0000 7f5bcd3d5740 10 client.4555 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-04T17:27:34.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.825+0000 7f5bcd3d5740 20 client.4555 populate_metadata read hostname 'smithi042' 2024-07-04T17:27:34.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.825+0000 7f5bcd3d5740 1 --2- 172.21.15.42:0/558028238 >> [v2:172.21.15.42:6832/1348446958,v1:172.21.15.42:6833/1348446958] conn(0x55b459bd93b0 0x55b459fcbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:34.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.825+0000 7f5bcd3d5740 4 client.4555 mds_command: new command op to 4422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T17:27:34.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.825+0000 7f5bcd3d5740 1 -- 172.21.15.42:0/558028238 --> [v2:172.21.15.42:6832/1348446958,v1:172.21.15.42:6833/1348446958] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b459f1b500 con 0x55b459bd93b0 2024-07-04T17:27:34.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.827+0000 7f5bb7e00640 1 --2- 172.21.15.42:0/558028238 >> [v2:172.21.15.42:6832/1348446958,v1:172.21.15.42:6833/1348446958] conn(0x55b459bd93b0 0x55b459fcbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:34.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.827+0000 7f5bb7e00640 1 --2- 172.21.15.42:0/558028238 >> [v2:172.21.15.42:6832/1348446958,v1:172.21.15.42:6833/1348446958] conn(0x55b459bd93b0 0x55b459fcbed0 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f7a227358efd44c server_cookie=2ca8a238b3c27ef0 in_seq=0 out_seq=0 2024-07-04T17:27:34.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.827+0000 7f5bacc00640 10 client.4555 ms_handle_connect on v2:172.21.15.42:6832/1348446958 2024-07-04T17:27:34.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.827+0000 7f5bacc00640 1 -- 172.21.15.42:0/558028238 <== mds.0 v2:172.21.15.42:6832/1348446958 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b459f1b500 con 0x55b459bd93b0 2024-07-04T17:27:34.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.827+0000 7f5bacc00640 10 client.4555 handle_command_reply: tid=0 2024-07-04T17:27:34.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.827+0000 7f5bcd3d5740 1 client.4555 shutdown 2024-07-04T17:27:34.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.827+0000 7f5bcd3d5740 1 -- 172.21.15.42:0/558028238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b459c02270 msgr2=0x55b459feb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:34.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.827+0000 7f5bcd3d5740 1 --2- 172.21.15.42:0/558028238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b459c02270 0x55b459feb7f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5bb001bc00 tx=0x7f5bb0048b30 comp rx=0 tx=0).stop 2024-07-04T17:27:34.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.828+0000 7f5bcd3d5740 1 -- 172.21.15.42:0/558028238 shutdown_connections 2024-07-04T17:27:34.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.828+0000 7f5bcd3d5740 1 --2- 172.21.15.42:0/558028238 >> [v2:172.21.15.42:6832/1348446958,v1:172.21.15.42:6833/1348446958] conn(0x55b459bd93b0 0x55b459fcbed0 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:34.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.828+0000 7f5bcd3d5740 1 --2- 172.21.15.42:0/558028238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b459c02270 0x55b459feb7f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:34.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.828+0000 7f5bcd3d5740 1 -- 172.21.15.42:0/558028238 >> 172.21.15.42:0/558028238 conn(0x55b459bf63c0 msgr2=0x55b459fa2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:34.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.828+0000 7f5bacc00640 0 client.4555 ms_handle_reset on v2:172.21.15.42:6832/1348446958 2024-07-04T17:27:34.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.828+0000 7f5bcd3d5740 1 -- 172.21.15.42:0/558028238 shutdown_connections 2024-07-04T17:27:34.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.828+0000 7f5bcd3d5740 1 -- 172.21.15.42:0/558028238 wait complete. 2024-07-04T17:27:34.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.828+0000 7f5bcd3d5740 20 client.4555 trim_cache size 0 max 16384 2024-07-04T17:27:34.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.849+0000 7f5bcd3d5740 1 Processor -- start 2024-07-04T17:27:34.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.850+0000 7f5bcd3d5740 1 -- start start 2024-07-04T17:27:34.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.851+0000 7f5bcd3d5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b459c02270 0x55b459feb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:34.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.851+0000 7f5bcd3d5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc01e7750 con 0x55b459c02270 2024-07-04T17:27:34.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.851+0000 7f5bbf400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b459c02270 0x55b459feb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:34.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.851+0000 7f5bbf400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b459c02270 0x55b459feb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57028/0 (socket says 172.21.15.42:57028) 2024-07-04T17:27:34.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.851+0000 7f5bbf400640 1 -- 172.21.15.42:0/482052611 learned_addr learned my addr 172.21.15.42:0/482052611 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:34.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.852+0000 7f5bbf400640 1 -- 172.21.15.42:0/482052611 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b459fa8ee0 con 0x55b459c02270 2024-07-04T17:27:34.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.852+0000 7f5bbf400640 1 --2- 172.21.15.42:0/482052611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b459c02270 0x55b459feb6c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0000c00 tx=0x7f5bb0048470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6caed7d4e477d7 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:34.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.852+0000 7f5baf400640 1 -- 172.21.15.42:0/482052611 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bb004f330 con 0x55b459c02270 2024-07-04T17:27:34.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.852+0000 7f5baf400640 1 -- 172.21.15.42:0/482052611 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5bb0033070 con 0x55b459c02270 2024-07-04T17:27:34.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.852+0000 7f5baf400640 1 -- 172.21.15.42:0/482052611 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bb00152c0 con 0x55b459c02270 2024-07-04T17:27:34.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.853+0000 7f5bcd3d5740 1 -- 172.21.15.42:0/482052611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b459c02270 msgr2=0x55b459feb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:34.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.853+0000 7f5bcd3d5740 1 --2- 172.21.15.42:0/482052611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b459c02270 0x55b459feb6c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0000c00 tx=0x7f5bb0048470 comp rx=0 tx=0).stop 2024-07-04T17:27:34.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.853+0000 7f5bcd3d5740 1 -- 172.21.15.42:0/482052611 shutdown_connections 2024-07-04T17:27:34.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.853+0000 7f5bcd3d5740 1 --2- 172.21.15.42:0/482052611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b459c02270 0x55b459feb6c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:34.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.853+0000 7f5bcd3d5740 1 -- 172.21.15.42:0/482052611 >> 172.21.15.42:0/482052611 conn(0x55b459bf63c0 msgr2=0x55b459fd0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:34.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.853+0000 7f5bcd3d5740 1 -- 172.21.15.42:0/482052611 shutdown_connections 2024-07-04T17:27:34.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.853+0000 7f5bcd3d5740 1 -- 172.21.15.42:0/482052611 wait complete. 2024-07-04T17:27:34.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.854+0000 7f5bcd3d5740 1 -- start start 2024-07-04T17:27:34.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.854+0000 7f5bcd3d5740 1 Processor -- start 2024-07-04T17:27:34.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.856+0000 7f5bcd3d5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b459bc0e40 0x55b459ffa780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:34.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.856+0000 7f5bcd3d5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b459f08b50 con 0x55b459bc0e40 2024-07-04T17:27:34.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.856+0000 7f5bcd3d5740 10 client.0 _ll_register_callbacks cb 0x7f5bb0011590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T17:27:34.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.856+0000 7f5bbf400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b459bc0e40 0x55b459ffa780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:34.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.856+0000 7f5bbf400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b459bc0e40 0x55b459ffa780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57038/0 (socket says 172.21.15.42:57038) 2024-07-04T17:27:34.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.856+0000 7f5bbf400640 1 -- 172.21.15.42:0/2395587141 learned_addr learned my addr 172.21.15.42:0/2395587141 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:34.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.856+0000 7f5bbf400640 1 -- 172.21.15.42:0/2395587141 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b459fc4c70 con 0x55b459bc0e40 2024-07-04T17:27:34.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.857+0000 7f5bbf400640 1 --2- 172.21.15.42:0/2395587141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b459bc0e40 0x55b459ffa780 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0019c80 tx=0x7f5bb000e280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69bb8d6b89969846 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:34.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.857+0000 7f5bacc00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-07-04T17:27:34.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.857+0000 7f5bacc00640 1 -- 172.21.15.42:0/2395587141 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bb004f330 con 0x55b459bc0e40 2024-07-04T17:27:34.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.857+0000 7f5bacc00640 1 -- 172.21.15.42:0/2395587141 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(31..31 src has 1..31) ==== 4303+0+0 (secure 0 0 0) 0x7f5bb0003ae0 con 0x55b459bc0e40 2024-07-04T17:27:34.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.857+0000 7f5bcd3d5740 1 -- 172.21.15.42:0/2395587141 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bb0033070 con 0x55b459bc0e40 2024-07-04T17:27:34.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.857+0000 7f5bacc00640 1 -- 172.21.15.42:0/2395587141 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f5bb001be20 con 0x55b459bc0e40 2024-07-04T17:27:34.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.857+0000 7f5bcd3d5740 10 client.4557 fetch_fsmap learned FSMap version 23 2024-07-04T17:27:34.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.857+0000 7f5bcd3d5740 1 -- 172.21.15.42:0/2395587141 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55b459fc7c80 con 0x55b459bc0e40 2024-07-04T17:27:34.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.857+0000 7f5bacc00640 1 -- 172.21.15.42:0/2395587141 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5bb0033070 con 0x55b459bc0e40 2024-07-04T17:27:34.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.857+0000 7f5bacc00640 1 -- 172.21.15.42:0/2395587141 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 23) ==== 4754+0+0 (secure 0 0 0) 0x7f5bb0003ce0 con 0x55b459bc0e40 2024-07-04T17:27:34.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.858+0000 7f5bcd3d5740 10 client.4557 fetch_fsmap finished waiting for FSMap version 23 2024-07-04T17:27:34.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.858+0000 7f5bcd3d5740 10 client.4557 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-04T17:27:34.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.858+0000 7f5bcd3d5740 20 client.4557 populate_metadata read hostname 'smithi042' 2024-07-04T17:27:34.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.858+0000 7f5bcd3d5740 1 --2- 172.21.15.42:0/2395587141 >> [v2:172.21.15.42:6832/1348446958,v1:172.21.15.42:6833/1348446958] conn(0x55b459b96280 0x55b45a04daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:34.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.858+0000 7f5bcd3d5740 4 client.4557 mds_command: new command op to 4422 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-04T17:27:34.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.858+0000 7f5bcd3d5740 1 -- 172.21.15.42:0/2395587141 --> [v2:172.21.15.42:6832/1348446958,v1:172.21.15.42:6833/1348446958] -- command(tid 0: {"prefix": "status"}) -- 0x55b459f16fa0 con 0x55b459b96280 2024-07-04T17:27:34.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.858+0000 7f5bb7e00640 1 --2- 172.21.15.42:0/2395587141 >> [v2:172.21.15.42:6832/1348446958,v1:172.21.15.42:6833/1348446958] conn(0x55b459b96280 0x55b45a04daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:34.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.858+0000 7f5bb7e00640 1 --2- 172.21.15.42:0/2395587141 >> [v2:172.21.15.42:6832/1348446958,v1:172.21.15.42:6833/1348446958] conn(0x55b459b96280 0x55b45a04daf0 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eebaaec0896cf3c server_cookie=2465ea752c704d74 in_seq=0 out_seq=0 2024-07-04T17:27:34.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.858+0000 7f5bacc00640 10 client.4557 ms_handle_connect on v2:172.21.15.42:6832/1348446958 2024-07-04T17:27:34.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.859+0000 7f5bacc00640 1 -- 172.21.15.42:0/2395587141 <== mds.0 v2:172.21.15.42:6832/1348446958 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55b459f16fa0 con 0x55b459b96280 2024-07-04T17:27:34.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.859+0000 7f5bacc00640 10 client.4557 handle_command_reply: tid=0 2024-07-04T17:27:34.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.859+0000 7f5bcd3d5740 1 client.4557 shutdown 2024-07-04T17:27:34.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.859+0000 7f5bcd3d5740 1 -- 172.21.15.42:0/2395587141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b459bc0e40 msgr2=0x55b459ffa780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:34.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.859+0000 7f5bcd3d5740 1 --2- 172.21.15.42:0/2395587141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b459bc0e40 0x55b459ffa780 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0019c80 tx=0x7f5bb000e280 comp rx=0 tx=0).stop 2024-07-04T17:27:34.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.859+0000 7f5bcd3d5740 1 -- 172.21.15.42:0/2395587141 shutdown_connections 2024-07-04T17:27:34.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.859+0000 7f5bcd3d5740 1 --2- 172.21.15.42:0/2395587141 >> [v2:172.21.15.42:6832/1348446958,v1:172.21.15.42:6833/1348446958] conn(0x55b459b96280 0x55b45a04daf0 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:34.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.859+0000 7f5bcd3d5740 1 --2- 172.21.15.42:0/2395587141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b459bc0e40 0x55b459ffa780 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:34.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.859+0000 7f5bcd3d5740 1 -- 172.21.15.42:0/2395587141 >> 172.21.15.42:0/2395587141 conn(0x55b459bf63c0 msgr2=0x55b459fef260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:34.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.859+0000 7f5bacc00640 0 client.4557 ms_handle_reset on v2:172.21.15.42:6832/1348446958 2024-07-04T17:27:34.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.859+0000 7f5bcd3d5740 1 -- 172.21.15.42:0/2395587141 shutdown_connections 2024-07-04T17:27:34.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.859+0000 7f5bcd3d5740 1 -- 172.21.15.42:0/2395587141 wait complete. 2024-07-04T17:27:34.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.859+0000 7f5bcd3d5740 20 client.4557 trim_cache size 0 max 16384 2024-07-04T17:27:34.863 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:27:34.863 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "be4f300a-93ef-4489-b540-37ece39b0aac", 2024-07-04T17:27:34.864 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-07-04T17:27:34.864 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 4422, 2024-07-04T17:27:34.864 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-07-04T17:27:34.864 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-07-04T17:27:34.864 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "backup_fs", 2024-07-04T17:27:34.864 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 6.8980087130000003, 2024-07-04T17:27:34.864 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 23, 2024-07-04T17:27:34.864 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 31, 2024-07-04T17:27:34.864 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 31, 2024-07-04T17:27:34.864 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 35.908780729 2024-07-04T17:27:34.864 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:27:34.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.860+0000 7f5bc6400640 1 -- 172.21.15.42:0/904918884 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f5b9c03d2b0 msgr2=0x7f5b9c03f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:34.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.860+0000 7f5bc6400640 1 --2- 172.21.15.42:0/904918884 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f5b9c03d2b0 0x7f5b9c03f7c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8033000 tx=0x7f5ba8004730 comp rx=0 tx=0).stop 2024-07-04T17:27:34.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.860+0000 7f5bc6400640 1 -- 172.21.15.42:0/904918884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bc00b2d20 msgr2=0x7f5bc01d4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:34.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.860+0000 7f5bc6400640 1 --2- 172.21.15.42:0/904918884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bc00b2d20 0x7f5bc01d4c00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0001eb0 tx=0x7f5bb000c810 comp rx=0 tx=0).stop 2024-07-04T17:27:34.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.860+0000 7f5bc6400640 1 -- 172.21.15.42:0/904918884 shutdown_connections 2024-07-04T17:27:34.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.860+0000 7f5bc6400640 1 --2- 172.21.15.42:0/904918884 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7f5b9c03d2b0 0x7f5b9c03f7c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:34.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.860+0000 7f5bc6400640 1 --2- 172.21.15.42:0/904918884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bc00b2d20 0x7f5bc01d4c00 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:34.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.860+0000 7f5bc6400640 1 -- 172.21.15.42:0/904918884 >> 172.21.15.42:0/904918884 conn(0x7f5bc00bf700 msgr2=0x7f5bc00a94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:34.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.860+0000 7f5bc6400640 1 -- 172.21.15.42:0/904918884 shutdown_connections 2024-07-04T17:27:34.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:34.860+0000 7f5bc6400640 1 -- 172.21.15.42:0/904918884 wait complete. 2024-07-04T17:27:34.886 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable mirroring 2024-07-04T17:27:35.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:35.167+0000 7fbda3e00640 1 -- 172.21.15.42:0/2429773975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbda40ac5b0 msgr2=0x7fbda40b0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:35.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:35.167+0000 7fbda3e00640 1 --2- 172.21.15.42:0/2429773975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbda40ac5b0 0x7fbda40b0db0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c005fd0 tx=0x7fbd8c02a5f0 comp rx=0 tx=0).stop 2024-07-04T17:27:35.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:35.167+0000 7fbda3e00640 1 -- 172.21.15.42:0/2429773975 shutdown_connections 2024-07-04T17:27:35.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:35.167+0000 7fbda3e00640 1 --2- 172.21.15.42:0/2429773975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbda40ac5b0 0x7fbda40b0db0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:35.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:35.167+0000 7fbda3e00640 1 -- 172.21.15.42:0/2429773975 >> 172.21.15.42:0/2429773975 conn(0x7fbda40afaa0 msgr2=0x7fbda40afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:35.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:35.167+0000 7fbda3e00640 1 -- 172.21.15.42:0/2429773975 shutdown_connections 2024-07-04T17:27:35.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:35.167+0000 7fbda3e00640 1 -- 172.21.15.42:0/2429773975 wait complete. 2024-07-04T17:27:35.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:35.169+0000 7fbda3e00640 1 Processor -- start 2024-07-04T17:27:35.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:35.169+0000 7fbda3e00640 1 -- start start 2024-07-04T17:27:35.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:35.170+0000 7fbda3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbda406da60 0x7fbda41d4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:35.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:35.170+0000 7fbda3e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbda40ae9d0 con 0x7fbda406da60 2024-07-04T17:27:35.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:35.170+0000 7fbda2a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbda406da60 0x7fbda41d4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:35.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:35.171+0000 7fbda2a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbda406da60 0x7fbda41d4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57056/0 (socket says 172.21.15.42:57056) 2024-07-04T17:27:35.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:35.171+0000 7fbda2a00640 1 -- 172.21.15.42:0/4239488450 learned_addr learned my addr 172.21.15.42:0/4239488450 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:35.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:35.171+0000 7fbda2a00640 1 -- 172.21.15.42:0/4239488450 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda4071100 con 0x7fbda406da60 2024-07-04T17:27:35.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:35.171+0000 7fbda2a00640 1 --2- 172.21.15.42:0/4239488450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbda406da60 0x7fbda41d4260 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400c320 tx=0x7fbd9400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:35.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:35.171+0000 7fbd9b400640 1 -- 172.21.15.42:0/4239488450 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd94017070 con 0x7fbda406da60 2024-07-04T17:27:35.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:35.171+0000 7fbd9b400640 1 -- 172.21.15.42:0/4239488450 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd940052f0 con 0x7fbda406da60 2024-07-04T17:27:35.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:35.171+0000 7fbd9b400640 1 -- 172.21.15.42:0/4239488450 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd940035f0 con 0x7fbda406da60 2024-07-04T17:27:35.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:35.171+0000 7fbda3e00640 1 -- 172.21.15.42:0/4239488450 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbda41dd550 con 0x7fbda406da60 2024-07-04T17:27:35.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:35.172+0000 7fbda3e00640 1 -- 172.21.15.42:0/4239488450 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbda4138280 con 0x7fbda406da60 2024-07-04T17:27:35.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:35.173+0000 7fbd9b400640 1 -- 172.21.15.42:0/4239488450 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fbd9401a070 con 0x7fbda406da60 2024-07-04T17:27:35.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:35.173+0000 7fbda3e00640 1 -- 172.21.15.42:0/4239488450 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbda40af100 con 0x7fbda406da60 2024-07-04T17:27:35.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:35.173+0000 7fbd9b400640 1 --2- 172.21.15.42:0/4239488450 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fbd8003d2b0 0x7fbd8003f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:35.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:35.173+0000 7fbd9b400640 1 -- 172.21.15.42:0/4239488450 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4303+0+0 (secure 0 0 0) 0x7fbd94050bf0 con 0x7fbda406da60 2024-07-04T17:27:35.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:35.173+0000 7fbd9be00640 1 --2- 172.21.15.42:0/4239488450 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fbd8003d2b0 0x7fbd8003f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:35.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:35.174+0000 7fbd9be00640 1 --2- 172.21.15.42:0/4239488450 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fbd8003d2b0 0x7fbd8003f7c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c033000 tx=0x7fbd8c004730 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:35.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:35.178+0000 7fbd9b400640 1 -- 172.21.15.42:0/4239488450 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fbd940102e0 con 0x7fbda406da60 2024-07-04T17:27:35.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:35.441+0000 7fbda3e00640 1 -- 172.21.15.42:0/4239488450 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "mirroring"} v 0) -- 0x7fbda4072860 con 0x7fbda406da60 2024-07-04T17:27:36.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.012+0000 7fbd9b400640 1 -- 172.21.15.42:0/4239488450 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "mirroring"}]=0 v6) ==== 88+0+0 (secure 0 0 0) 0x7fbd94011910 con 0x7fbda406da60 2024-07-04T17:27:36.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.016+0000 7fbda3e00640 1 -- 172.21.15.42:0/4239488450 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fbd8003d2b0 msgr2=0x7fbd8003f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:36.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.016+0000 7fbda3e00640 1 --2- 172.21.15.42:0/4239488450 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fbd8003d2b0 0x7fbd8003f7c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c033000 tx=0x7fbd8c004730 comp rx=0 tx=0).stop 2024-07-04T17:27:36.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.016+0000 7fbda3e00640 1 -- 172.21.15.42:0/4239488450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbda406da60 msgr2=0x7fbda41d4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:36.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.016+0000 7fbda3e00640 1 --2- 172.21.15.42:0/4239488450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbda406da60 0x7fbda41d4260 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400c320 tx=0x7fbd9400c7e0 comp rx=0 tx=0).stop 2024-07-04T17:27:36.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.016+0000 7fbda3e00640 1 -- 172.21.15.42:0/4239488450 shutdown_connections 2024-07-04T17:27:36.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.016+0000 7fbda3e00640 1 --2- 172.21.15.42:0/4239488450 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fbd8003d2b0 0x7fbd8003f7c0 secure :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c033000 tx=0x7fbd8c004730 comp rx=0 tx=0).stop 2024-07-04T17:27:36.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.016+0000 7fbda3e00640 1 --2- 172.21.15.42:0/4239488450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbda406da60 0x7fbda41d4260 secure :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400c320 tx=0x7fbd9400c7e0 comp rx=0 tx=0).stop 2024-07-04T17:27:36.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.016+0000 7fbda3e00640 1 -- 172.21.15.42:0/4239488450 >> 172.21.15.42:0/4239488450 conn(0x7fbda40afaa0 msgr2=0x7fbda4073740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:36.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.018+0000 7fbda3e00640 1 -- 172.21.15.42:0/4239488450 shutdown_connections 2024-07-04T17:27:36.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.019+0000 7fbda3e00640 1 -- 172.21.15.42:0/4239488450 wait complete. 2024-07-04T17:27:36.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:27:36.053 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3) 2024-07-04T17:27:36.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:27:36.112 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d4) 2024-07-04T17:27:36.159 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:27:36.159 DEBUG:teuthology.orchestra.run.smithi042:counter dump for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok counter dump 2024-07-04T17:27:36.366 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:27:36.366 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker": [ 2024-07-04T17:27:36.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:36.366 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:27:36.366 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "0" 2024-07-04T17:27:36.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:36.366 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:36.366 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:27:36.367 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:27:36.367 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:36.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:36.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:36.367 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:27:36.367 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "1" 2024-07-04T17:27:36.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:36.367 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:36.367 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:27:36.367 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:27:36.367 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:36.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:36.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:36.368 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:27:36.368 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "2" 2024-07-04T17:27:36.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:36.368 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:36.368 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:27:36.368 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:27:36.368 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:36.368 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:36.368 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:36.368 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": [ 2024-07-04T17:27:36.369 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:36.369 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:36.369 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:36.369 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 19, 2024-07-04T17:27:36.369 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 3, 2024-07-04T17:27:36.369 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 69520, 2024-07-04T17:27:36.369 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 464, 2024-07-04T17:27:36.369 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:27:36.369 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:27:36.370 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.007330361, 2024-07-04T17:27:36.370 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.000677651, 2024-07-04T17:27:36.370 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.005077146, 2024-07-04T17:27:36.370 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.001471948, 2024-07-04T17:27:36.370 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:27:36.370 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 3, 2024-07-04T17:27:36.370 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.004312719, 2024-07-04T17:27:36.371 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.001437573 2024-07-04T17:27:36.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:36.371 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:27:36.371 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:27:36.371 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:27:36.371 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:27:36.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:36.371 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 69520, 2024-07-04T17:27:36.372 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 464 2024-07-04T17:27:36.372 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:36.372 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:36.372 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:36.372 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": [ 2024-07-04T17:27:36.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:36.372 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:36.373 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:36.373 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 1, 2024-07-04T17:27:36.373 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 15, 2024-07-04T17:27:36.373 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 144, 2024-07-04T17:27:36.373 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 85280, 2024-07-04T17:27:36.373 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 4, 2024-07-04T17:27:36.373 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 0, 2024-07-04T17:27:36.373 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.004740068, 2024-07-04T17:27:36.373 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.002201468, 2024-07-04T17:27:36.373 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.001459162, 2024-07-04T17:27:36.373 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:27:36.374 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:27:36.374 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 14, 2024-07-04T17:27:36.374 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.001036643, 2024-07-04T17:27:36.374 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000074045 2024-07-04T17:27:36.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:36.374 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:27:36.374 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:27:36.374 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:27:36.374 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:27:36.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:36.374 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 144, 2024-07-04T17:27:36.375 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 85280 2024-07-04T17:27:36.375 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:36.375 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:36.375 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:36.375 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": [ 2024-07-04T17:27:36.375 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:36.375 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:36.375 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:36.375 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 9, 2024-07-04T17:27:36.375 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 6, 2024-07-04T17:27:36.375 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 104560, 2024-07-04T17:27:36.376 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 928, 2024-07-04T17:27:36.376 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:27:36.376 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:27:36.376 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.004565713, 2024-07-04T17:27:36.376 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.000695954, 2024-07-04T17:27:36.376 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.005742584, 2024-07-04T17:27:36.376 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:27:36.376 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:27:36.376 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 6, 2024-07-04T17:27:36.376 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.003172173, 2024-07-04T17:27:36.376 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000528695 2024-07-04T17:27:36.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:36.377 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:27:36.377 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:27:36.377 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:27:36.377 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:27:36.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:36.377 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 104560, 2024-07-04T17:27:36.377 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 928 2024-07-04T17:27:36.377 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:36.377 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:36.377 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:36.377 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": [ 2024-07-04T17:27:36.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:36.378 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:36.378 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:36.378 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-07-04T17:27:36.378 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-07-04T17:27:36.378 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:36.378 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:36.378 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:36.378 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror": [ 2024-07-04T17:27:36.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:36.378 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:36.378 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:36.378 INFO:teuthology.orchestra.run.smithi042.stdout: "mirrored_filesystems": 0, 2024-07-04T17:27:36.379 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_enable_failures": 0 2024-07-04T17:27:36.379 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:36.379 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:36.379 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:36.379 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": [ 2024-07-04T17:27:36.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:36.379 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:36.379 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:36.379 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-07-04T17:27:36.379 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-07-04T17:27:36.379 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T17:27:36.380 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-07-04T17:27:36.380 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T17:27:36.380 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-07-04T17:27:36.380 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T17:27:36.380 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T17:27:36.380 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T17:27:36.380 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T17:27:36.380 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T17:27:36.380 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-07-04T17:27:36.380 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T17:27:36.380 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T17:27:36.380 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-07-04T17:27:36.381 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-07-04T17:27:36.381 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-07-04T17:27:36.381 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-07-04T17:27:36.381 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T17:27:36.381 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T17:27:36.381 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T17:27:36.381 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T17:27:36.381 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T17:27:36.381 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-07-04T17:27:36.381 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-07-04T17:27:36.381 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-07-04T17:27:36.381 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T17:27:36.382 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T17:27:36.382 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-07-04T17:27:36.382 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-07-04T17:27:36.382 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-07-04T17:27:36.382 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-07-04T17:27:36.382 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T17:27:36.382 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-07-04T17:27:36.382 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T17:27:36.382 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-07-04T17:27:36.382 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 148338, 2024-07-04T17:27:36.382 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 24, 2024-07-04T17:27:36.383 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-07-04T17:27:36.383 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-07-04T17:27:36.383 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-07-04T17:27:36.383 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-07-04T17:27:36.383 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-07-04T17:27:36.383 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-07-04T17:27:36.383 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-07-04T17:27:36.383 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-07-04T17:27:36.383 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 4592, 2024-07-04T17:27:36.383 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 37, 2024-07-04T17:27:36.383 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T17:27:36.383 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-07-04T17:27:36.384 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-07-04T17:27:36.384 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-07-04T17:27:36.384 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 0, 2024-07-04T17:27:36.384 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 0, 2024-07-04T17:27:36.384 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-07-04T17:27:36.384 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-07-04T17:27:36.384 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-07-04T17:27:36.384 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-07-04T17:27:36.384 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:36.384 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:36.384 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:36.384 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter": [ 2024-07-04T17:27:36.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:36.385 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:36.385 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:36.385 INFO:teuthology.orchestra.run.smithi042.stdout: "op_active": 0, 2024-07-04T17:27:36.385 INFO:teuthology.orchestra.run.smithi042.stdout: "op_laggy": 0, 2024-07-04T17:27:36.385 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send": 0, 2024-07-04T17:27:36.385 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send_bytes": 0, 2024-07-04T17:27:36.385 INFO:teuthology.orchestra.run.smithi042.stdout: "op_resend": 0, 2024-07-04T17:27:36.385 INFO:teuthology.orchestra.run.smithi042.stdout: "op_reply": 0, 2024-07-04T17:27:36.385 INFO:teuthology.orchestra.run.smithi042.stdout: "op_latency": { 2024-07-04T17:27:36.385 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:27:36.385 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:27:36.385 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:27:36.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:36.386 INFO:teuthology.orchestra.run.smithi042.stdout: "op_inflight": 0, 2024-07-04T17:27:36.386 INFO:teuthology.orchestra.run.smithi042.stdout: "oplen_avg": { 2024-07-04T17:27:36.386 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:27:36.386 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0 2024-07-04T17:27:36.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:36.386 INFO:teuthology.orchestra.run.smithi042.stdout: "op": 0, 2024-07-04T17:27:36.386 INFO:teuthology.orchestra.run.smithi042.stdout: "op_r": 0, 2024-07-04T17:27:36.386 INFO:teuthology.orchestra.run.smithi042.stdout: "op_w": 0, 2024-07-04T17:27:36.386 INFO:teuthology.orchestra.run.smithi042.stdout: "op_rmw": 0, 2024-07-04T17:27:36.386 INFO:teuthology.orchestra.run.smithi042.stdout: "op_pg": 0, 2024-07-04T17:27:36.387 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_stat": 0, 2024-07-04T17:27:36.387 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_create": 0, 2024-07-04T17:27:36.387 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_read": 0, 2024-07-04T17:27:36.387 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_write": 0, 2024-07-04T17:27:36.387 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writefull": 0, 2024-07-04T17:27:36.387 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writesame": 0, 2024-07-04T17:27:36.387 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_append": 0, 2024-07-04T17:27:36.387 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_zero": 0, 2024-07-04T17:27:36.387 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_truncate": 0, 2024-07-04T17:27:36.387 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_delete": 0, 2024-07-04T17:27:36.387 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_mapext": 0, 2024-07-04T17:27:36.387 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_sparse_read": 0, 2024-07-04T17:27:36.387 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_clonerange": 0, 2024-07-04T17:27:36.388 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_getxattr": 0, 2024-07-04T17:27:36.388 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_setxattr": 0, 2024-07-04T17:27:36.388 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_cmpxattr": 0, 2024-07-04T17:27:36.388 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_rmxattr": 0, 2024-07-04T17:27:36.388 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_resetxattrs": 0, 2024-07-04T17:27:36.388 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_call": 0, 2024-07-04T17:27:36.388 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_watch": 0, 2024-07-04T17:27:36.388 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_notify": 0, 2024-07-04T17:27:36.388 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T17:27:36.388 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls": 0, 2024-07-04T17:27:36.388 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls_filter": 0, 2024-07-04T17:27:36.389 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_other": 0, 2024-07-04T17:27:36.389 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_active": 0, 2024-07-04T17:27:36.389 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_send": 0, 2024-07-04T17:27:36.389 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_resend": 0, 2024-07-04T17:27:36.389 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_ping": 0, 2024-07-04T17:27:36.389 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_active": 0, 2024-07-04T17:27:36.389 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_send": 0, 2024-07-04T17:27:36.389 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_resend": 0, 2024-07-04T17:27:36.389 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_active": 0, 2024-07-04T17:27:36.389 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_send": 0, 2024-07-04T17:27:36.389 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_resend": 0, 2024-07-04T17:27:36.389 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_active": 0, 2024-07-04T17:27:36.390 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_send": 0, 2024-07-04T17:27:36.390 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_resend": 0, 2024-07-04T17:27:36.390 INFO:teuthology.orchestra.run.smithi042.stdout: "command_active": 0, 2024-07-04T17:27:36.390 INFO:teuthology.orchestra.run.smithi042.stdout: "command_send": 0, 2024-07-04T17:27:36.390 INFO:teuthology.orchestra.run.smithi042.stdout: "command_resend": 0, 2024-07-04T17:27:36.390 INFO:teuthology.orchestra.run.smithi042.stdout: "map_epoch": 0, 2024-07-04T17:27:36.390 INFO:teuthology.orchestra.run.smithi042.stdout: "map_full": 0, 2024-07-04T17:27:36.390 INFO:teuthology.orchestra.run.smithi042.stdout: "map_inc": 0, 2024-07-04T17:27:36.390 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_sessions": 0, 2024-07-04T17:27:36.390 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_open": 0, 2024-07-04T17:27:36.390 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_close": 0, 2024-07-04T17:27:36.390 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_laggy": 0, 2024-07-04T17:27:36.391 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_wr": 0, 2024-07-04T17:27:36.391 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_rd": 0, 2024-07-04T17:27:36.391 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_del": 0 2024-07-04T17:27:36.391 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:36.391 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:36.391 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:36.391 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-client": [ 2024-07-04T17:27:36.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:36.391 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:36.391 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:36.391 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:27:36.391 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:27:36.392 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:27:36.392 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 19, 2024-07-04T17:27:36.392 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 66941, 2024-07-04T17:27:36.392 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:27:36.392 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 19, 2024-07-04T17:27:36.392 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:27:36.392 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:27:36.392 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 19, 2024-07-04T17:27:36.392 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 66941, 2024-07-04T17:27:36.392 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:27:36.392 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:27:36.393 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:27:36.393 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:27:36.393 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:36.393 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:36.393 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:36.393 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:36.393 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-radosclient": [ 2024-07-04T17:27:36.393 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:36.393 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:36.393 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:36.393 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:27:36.393 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:27:36.394 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:27:36.394 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 7, 2024-07-04T17:27:36.394 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 102838, 2024-07-04T17:27:36.394 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:27:36.394 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 7, 2024-07-04T17:27:36.394 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:27:36.394 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:27:36.394 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 7, 2024-07-04T17:27:36.394 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 102838, 2024-07-04T17:27:36.394 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:27:36.394 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:27:36.394 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:27:36.395 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:27:36.395 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:36.395 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:36.395 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:36.395 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:36.395 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_bytes": [ 2024-07-04T17:27:36.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:36.395 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:36.395 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:36.395 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:27:36.395 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:27:36.395 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:27:36.396 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:27:36.396 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:27:36.396 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:27:36.396 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:27:36.396 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:27:36.396 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:27:36.396 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:27:36.396 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:27:36.396 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:27:36.396 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:27:36.396 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:27:36.396 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:27:36.397 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:36.397 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:36.397 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:36.397 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:36.397 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_ops": [ 2024-07-04T17:27:36.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:36.397 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:36.397 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:36.397 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:27:36.397 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1024, 2024-07-04T17:27:36.397 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:27:36.397 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:27:36.398 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:27:36.398 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:27:36.398 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:27:36.398 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:27:36.398 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:27:36.398 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:27:36.398 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:27:36.398 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:27:36.398 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:27:36.398 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:27:36.398 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:27:36.398 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:36.399 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:36.399 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:36.399 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-07-04T17:27:36.399 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:27:36.400 DEBUG:tasks.cephfs.test_mirroring:command returned={ "AsyncMessenger::Worker": [ { "labels": { "id": "0" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "1" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "2" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } } ], "AsyncMessenger::Worker-0": [ { "labels": {}, "counters": { "msgr_recv_messages": 19, "msgr_send_messages": 3, "msgr_recv_bytes": 69520, "msgr_send_bytes": 464, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.007330361, "msgr_running_send_time": 0.000677651, "msgr_running_recv_time": 0.005077146, "msgr_running_fast_dispatch_time": 0.001471948, "msgr_send_messages_queue_lat": { "avgcount": 3, "sum": 0.004312719, "avgtime": 0.001437573 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 69520, "msgr_send_encrypted_bytes": 464 } } ], "AsyncMessenger::Worker-1": [ { "labels": {}, "counters": { "msgr_recv_messages": 1, "msgr_send_messages": 15, "msgr_recv_bytes": 144, "msgr_send_bytes": 85280, "msgr_created_connections": 4, "msgr_active_connections": 0, "msgr_running_total_time": 0.004740068, "msgr_running_send_time": 0.002201468, "msgr_running_recv_time": 0.001459162, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 14, "sum": 0.001036643, "avgtime": 0.000074045 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 144, "msgr_send_encrypted_bytes": 85280 } } ], "AsyncMessenger::Worker-2": [ { "labels": {}, "counters": { "msgr_recv_messages": 9, "msgr_send_messages": 6, "msgr_recv_bytes": 104560, "msgr_send_bytes": 928, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.004565713, "msgr_running_send_time": 0.000695954, "msgr_running_recv_time": 0.005742584, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 6, "sum": 0.003172173, "avgtime": 0.000528695 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 104560, "msgr_send_encrypted_bytes": 928 } } ], "cct": [ { "labels": {}, "counters": { "total_workers": 0, "unhealthy_workers": 0 } } ], "cephfs_mirror": [ { "labels": {}, "counters": { "mirrored_filesystems": 0, "mirror_enable_failures": 0 } } ], "mempool": [ { "labels": {}, "counters": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_items": 0, "bluefs_bytes": 0, "bluefs_items": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "buffer_anon_bytes": 148338, "buffer_anon_items": 24, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 4592, "osdmap_items": 37, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "mds_co_bytes": 0, "mds_co_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 } } ], "objecter": [ { "labels": {}, "counters": { "op_active": 0, "op_laggy": 0, "op_send": 0, "op_send_bytes": 0, "op_resend": 0, "op_reply": 0, "op_latency": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "op_inflight": 0, "oplen_avg": { "avgcount": 0, "sum": 0 }, "op": 0, "op_r": 0, "op_w": 0, "op_rmw": 0, "op_pg": 0, "osdop_stat": 0, "osdop_create": 0, "osdop_read": 0, "osdop_write": 0, "osdop_writefull": 0, "osdop_writesame": 0, "osdop_append": 0, "osdop_zero": 0, "osdop_truncate": 0, "osdop_delete": 0, "osdop_mapext": 0, "osdop_sparse_read": 0, "osdop_clonerange": 0, "osdop_getxattr": 0, "osdop_setxattr": 0, "osdop_cmpxattr": 0, "osdop_rmxattr": 0, "osdop_resetxattrs": 0, "osdop_call": 0, "osdop_watch": 0, "osdop_notify": 0, "osdop_src_cmpxattr": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_other": 0, "linger_active": 0, "linger_send": 0, "linger_resend": 0, "linger_ping": 0, "poolop_active": 0, "poolop_send": 0, "poolop_resend": 0, "poolstat_active": 0, "poolstat_send": 0, "poolstat_resend": 0, "statfs_active": 0, "statfs_send": 0, "statfs_resend": 0, "command_active": 0, "command_send": 0, "command_resend": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "osd_sessions": 0, "osd_session_open": 0, "osd_session_close": 0, "osd_laggy": 0, "omap_wr": 0, "omap_rd": 0, "omap_del": 0 } } ], "throttle-msgr_dispatch_throttler-client": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 19, "get_sum": 66941, "get_or_fail_fail": 0, "get_or_fail_success": 19, "take": 0, "take_sum": 0, "put": 19, "put_sum": 66941, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-msgr_dispatch_throttler-radosclient": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 7, "get_sum": 102838, "get_or_fail_fail": 0, "get_or_fail_success": 7, "take": 0, "take_sum": 0, "put": 7, "put_sum": 102838, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_bytes": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_ops": [ { "labels": {}, "counters": { "val": 0, "max": 1024, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ] } 2024-07-04T17:27:36.400 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snapshot mirror enable cephfs 2024-07-04T17:27:36.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.689+0000 7fb192800640 1 -- 172.21.15.42:0/3133873658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb18c0b7e40 msgr2=0x7fb18c0be0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:36.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.689+0000 7fb192800640 1 --2- 172.21.15.42:0/3133873658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb18c0b7e40 0x7fb18c0be0d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb17c005fd0 tx=0x7fb17c02a5f0 comp rx=0 tx=0).stop 2024-07-04T17:27:36.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.689+0000 7fb192800640 1 -- 172.21.15.42:0/3133873658 shutdown_connections 2024-07-04T17:27:36.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.689+0000 7fb192800640 1 --2- 172.21.15.42:0/3133873658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb18c0b7e40 0x7fb18c0be0d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:36.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.689+0000 7fb192800640 1 -- 172.21.15.42:0/3133873658 >> 172.21.15.42:0/3133873658 conn(0x7fb18c06e6d0 msgr2=0x7fb18c073960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:36.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.689+0000 7fb192800640 1 -- 172.21.15.42:0/3133873658 shutdown_connections 2024-07-04T17:27:36.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.689+0000 7fb192800640 1 -- 172.21.15.42:0/3133873658 wait complete. 2024-07-04T17:27:36.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.692+0000 7fb192800640 1 Processor -- start 2024-07-04T17:27:36.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.692+0000 7fb192800640 1 -- start start 2024-07-04T17:27:36.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.694+0000 7fb192800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb18c0b3570 0x7fb18c1d9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:36.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.694+0000 7fb192800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb18c0b3d10 con 0x7fb18c0b3570 2024-07-04T17:27:36.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.694+0000 7fb18b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb18c0b3570 0x7fb18c1d9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:36.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.694+0000 7fb18b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb18c0b3570 0x7fb18c1d9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57092/0 (socket says 172.21.15.42:57092) 2024-07-04T17:27:36.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.694+0000 7fb18b400640 1 -- 172.21.15.42:0/3378265577 learned_addr learned my addr 172.21.15.42:0/3378265577 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:36.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.695+0000 7fb18b400640 1 -- 172.21.15.42:0/3378265577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb18c1378c0 con 0x7fb18c0b3570 2024-07-04T17:27:36.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.695+0000 7fb18b400640 1 --2- 172.21.15.42:0/3378265577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb18c0b3570 0x7fb18c1d9870 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb17c009850 tx=0x7fb17c0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:36.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.695+0000 7fb183400640 1 -- 172.21.15.42:0/3378265577 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb17c03e030 con 0x7fb18c0b3570 2024-07-04T17:27:36.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.695+0000 7fb183400640 1 -- 172.21.15.42:0/3378265577 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb17c0035f0 con 0x7fb18c0b3570 2024-07-04T17:27:36.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.695+0000 7fb183400640 1 -- 172.21.15.42:0/3378265577 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb17c035040 con 0x7fb18c0b3570 2024-07-04T17:27:36.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.695+0000 7fb192800640 1 -- 172.21.15.42:0/3378265577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb18c136650 con 0x7fb18c0b3570 2024-07-04T17:27:36.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.696+0000 7fb192800640 1 -- 172.21.15.42:0/3378265577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb18c1dd820 con 0x7fb18c0b3570 2024-07-04T17:27:36.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.697+0000 7fb192800640 1 -- 172.21.15.42:0/3378265577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb18c1d9db0 con 0x7fb18c0b3570 2024-07-04T17:27:36.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.697+0000 7fb183400640 1 -- 172.21.15.42:0/3378265577 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 6) ==== 49424+0+0 (secure 0 0 0) 0x7fb17c041030 con 0x7fb18c0b3570 2024-07-04T17:27:36.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.697+0000 7fb183400640 1 --2- 172.21.15.42:0/3378265577 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb16c03d300 0x7fb16c03f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:36.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.697+0000 7fb183400640 1 -- 172.21.15.42:0/3378265577 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4303+0+0 (secure 0 0 0) 0x7fb17c03f070 con 0x7fb18c0b3570 2024-07-04T17:27:36.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.698+0000 7fb18aa00640 1 -- 172.21.15.42:0/3378265577 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb16c03d300 msgr2=0x7fb16c03f810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/3523793950 2024-07-04T17:27:36.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.698+0000 7fb18aa00640 1 --2- 172.21.15.42:0/3378265577 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb16c03d300 0x7fb16c03f810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:27:36.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.703+0000 7fb183400640 1 -- 172.21.15.42:0/3378265577 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fb18c1dd820 con 0x7fb18c0b3570 2024-07-04T17:27:36.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.899+0000 7fb18aa00640 1 -- 172.21.15.42:0/3378265577 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb16c03d300 msgr2=0x7fb16c03f810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/3523793950 2024-07-04T17:27:36.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.899+0000 7fb18aa00640 1 --2- 172.21.15.42:0/3378265577 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb16c03d300 0x7fb16c03f810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:27:36.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:36.932+0000 7fb192800640 1 -- 172.21.15.42:0/3378265577 --> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] -- mgr_command(tid 0: {"prefix": "fs snapshot mirror enable", "fs_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7fb18c137a90 con 0x7fb16c03d300 2024-07-04T17:27:37.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:37.301+0000 7fb18aa00640 1 -- 172.21.15.42:0/3378265577 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb16c03d300 msgr2=0x7fb16c03f810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/3523793950 2024-07-04T17:27:37.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:37.301+0000 7fb18aa00640 1 --2- 172.21.15.42:0/3378265577 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb16c03d300 0x7fb16c03f810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-07-04T17:27:38.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:38.102+0000 7fb18aa00640 1 -- 172.21.15.42:0/3378265577 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb16c03d300 msgr2=0x7fb16c03f810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/3523793950 2024-07-04T17:27:38.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:38.102+0000 7fb18aa00640 1 --2- 172.21.15.42:0/3378265577 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb16c03d300 0x7fb16c03f810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-07-04T17:27:39.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:39.705+0000 7fb18aa00640 1 -- 172.21.15.42:0/3378265577 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb16c03d300 msgr2=0x7fb16c03f810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/3523793950 2024-07-04T17:27:39.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:39.705+0000 7fb18aa00640 1 --2- 172.21.15.42:0/3378265577 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb16c03d300 0x7fb16c03f810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-07-04T17:27:40.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:40.667+0000 7fb183400640 1 -- 172.21.15.42:0/3378265577 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 7) ==== 49135+0+0 (secure 0 0 0) 0x7fb17c030c90 con 0x7fb18c0b3570 2024-07-04T17:27:40.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:40.668+0000 7fb183400640 1 -- 172.21.15.42:0/3378265577 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb16c03d300 msgr2=0x7fb16c03f810 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:27:40.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:40.668+0000 7fb183400640 1 --2- 172.21.15.42:0/3378265577 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb16c03d300 0x7fb16c03f810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:40.753 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:27:40.753+0000 7fcd67a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:27:40.753 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:27:40.753+0000 7fcd67a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:27:40.753 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:27:40.753+0000 7fcd67a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:27:40.754 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:27:40.753+0000 7fcd67a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:27:40.754 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:27:40.753+0000 7fcd67a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:27:40.754 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:27:40.753+0000 7fcd6c000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:27:40.754 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:27:40.753+0000 7fcd6c000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:27:40.754 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:27:40.753+0000 7fcd6c000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:27:40.754 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:27:40.753+0000 7fcd6c000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:27:40.754 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:27:40.753+0000 7fcd6c000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:27:40.777 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:27:40.776+0000 7fcd67a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:27:40.777 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:27:40.776+0000 7fcd67a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:27:40.777 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:27:40.776+0000 7fcd67a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:27:40.777 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:27:40.776+0000 7fcd67a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:27:40.777 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:27:40.776+0000 7fcd67a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:27:40.778 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:27:40.777+0000 7fcd6c000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:27:40.778 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:27:40.777+0000 7fcd6c000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:27:40.778 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:27:40.777+0000 7fcd6c000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:27:40.778 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:27:40.777+0000 7fcd6c000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:27:40.778 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:27:40.777+0000 7fcd6c000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:27:41.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:41.672+0000 7fb183400640 1 -- 172.21.15.42:0/3378265577 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mgrmap(e 8) ==== 49262+0+0 (secure 0 0 0) 0x7fb17c058ba0 con 0x7fb18c0b3570 2024-07-04T17:27:41.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:41.673+0000 7fb183400640 1 --2- 172.21.15.42:0/3378265577 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7fb16c046940 0x7fb16c048d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:41.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:41.673+0000 7fb183400640 1 -- 172.21.15.42:0/3378265577 --> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] -- mgr_command(tid 0: {"prefix": "fs snapshot mirror enable", "fs_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7fb18c137a90 con 0x7fb16c046940 2024-07-04T17:27:41.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:41.681+0000 7fb18aa00640 1 --2- 172.21.15.42:0/3378265577 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7fb16c046940 0x7fb16c048d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:41.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:41.688+0000 7fb18aa00640 1 --2- 172.21.15.42:0/3378265577 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7fb16c046940 0x7fb16c048d60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb178001180 tx=0x7fb178007a10 comp rx=0 tx=0).ready entity=mgr.4563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:42.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:42.678+0000 7fb183400640 1 -- 172.21.15.42:0/3378265577 <== mgr.4563 v2:172.21.15.42:6824/488387844 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fb18c137a90 con 0x7fb16c046940 2024-07-04T17:27:42.679 INFO:teuthology.orchestra.run.smithi042.stdout:{} 2024-07-04T17:27:42.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:42.682+0000 7fb192800640 1 -- 172.21.15.42:0/3378265577 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7fb16c046940 msgr2=0x7fb16c048d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:42.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:42.682+0000 7fb192800640 1 --2- 172.21.15.42:0/3378265577 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7fb16c046940 0x7fb16c048d60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb178001180 tx=0x7fb178007a10 comp rx=0 tx=0).stop 2024-07-04T17:27:42.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:42.682+0000 7fb192800640 1 -- 172.21.15.42:0/3378265577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb18c0b3570 msgr2=0x7fb18c1d9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:42.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:42.682+0000 7fb192800640 1 --2- 172.21.15.42:0/3378265577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb18c0b3570 0x7fb18c1d9870 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb17c009850 tx=0x7fb17c0049d0 comp rx=0 tx=0).stop 2024-07-04T17:27:42.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:42.682+0000 7fb192800640 1 -- 172.21.15.42:0/3378265577 shutdown_connections 2024-07-04T17:27:42.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:42.682+0000 7fb192800640 1 --2- 172.21.15.42:0/3378265577 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7fb16c046940 0x7fb16c048d60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:42.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:42.682+0000 7fb192800640 1 --2- 172.21.15.42:0/3378265577 >> [v2:172.21.15.42:6824/3523793950,v1:172.21.15.42:6825/3523793950] conn(0x7fb16c03d300 0x7fb16c03f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:42.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:42.682+0000 7fb192800640 1 --2- 172.21.15.42:0/3378265577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb18c0b3570 0x7fb18c1d9870 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:42.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:42.682+0000 7fb192800640 1 -- 172.21.15.42:0/3378265577 >> 172.21.15.42:0/3378265577 conn(0x7fb18c06e6d0 msgr2=0x7fb18c075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:42.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:42.682+0000 7fb192800640 1 -- 172.21.15.42:0/3378265577 shutdown_connections 2024-07-04T17:27:42.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:42.682+0000 7fb192800640 1 -- 172.21.15.42:0/3378265577 wait complete. 2024-07-04T17:27:52.714 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:27:52.715 DEBUG:teuthology.orchestra.run.smithi042:mirror status for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@2 2024-07-04T17:27:52.948 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:27:52.949 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_inst": "172.21.15.42:0/487218478", 2024-07-04T17:27:52.949 INFO:teuthology.orchestra.run.smithi042.stdout: "peers": {}, 2024-07-04T17:27:52.949 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_dirs": { 2024-07-04T17:27:52.949 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_count": 0 2024-07-04T17:27:52.949 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:52.949 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:27:52.965 DEBUG:tasks.cephfs.test_mirroring:command returned={ "rados_inst": "172.21.15.42:0/487218478", "peers": {}, "snap_dirs": { "dir_count": 0 } } 2024-07-04T17:27:52.966 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:27:52.966 DEBUG:teuthology.orchestra.run.smithi042:counter dump for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok counter dump 2024-07-04T17:27:53.208 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:27:53.209 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker": [ 2024-07-04T17:27:53.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:53.209 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:27:53.209 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "0" 2024-07-04T17:27:53.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:53.209 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:53.209 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:27:53.209 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:27:53.209 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:53.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:53.209 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:27:53.209 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "1" 2024-07-04T17:27:53.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:53.210 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:53.210 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:27:53.210 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:27:53.210 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:53.210 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:53.210 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:27:53.210 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "2" 2024-07-04T17:27:53.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:53.210 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:53.210 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:27:53.210 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:27:53.210 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.211 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.211 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:53.211 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": [ 2024-07-04T17:27:53.211 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:53.211 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:53.211 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:53.211 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 20, 2024-07-04T17:27:53.211 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 3, 2024-07-04T17:27:53.211 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 74416, 2024-07-04T17:27:53.211 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 464, 2024-07-04T17:27:53.211 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:27:53.211 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:27:53.212 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.007916628, 2024-07-04T17:27:53.212 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.000760014, 2024-07-04T17:27:53.212 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.005337197, 2024-07-04T17:27:53.212 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.001671330, 2024-07-04T17:27:53.212 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:27:53.212 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 3, 2024-07-04T17:27:53.212 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.004312719, 2024-07-04T17:27:53.212 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.001437573 2024-07-04T17:27:53.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:53.212 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:27:53.212 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:27:53.213 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:27:53.213 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:27:53.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:53.213 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 74416, 2024-07-04T17:27:53.213 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 464 2024-07-04T17:27:53.213 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.213 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.213 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:53.213 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": [ 2024-07-04T17:27:53.213 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:53.213 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:53.213 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:53.213 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 2, 2024-07-04T17:27:53.213 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 19, 2024-07-04T17:27:53.214 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 288, 2024-07-04T17:27:53.214 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 166400, 2024-07-04T17:27:53.214 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 5, 2024-07-04T17:27:53.214 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:27:53.214 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.009524650, 2024-07-04T17:27:53.214 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.003642952, 2024-07-04T17:27:53.214 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.003527254, 2024-07-04T17:27:53.214 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:27:53.214 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:27:53.214 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 18, 2024-07-04T17:27:53.214 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.016382422, 2024-07-04T17:27:53.214 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000910134 2024-07-04T17:27:53.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:53.214 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:27:53.214 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:27:53.215 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:27:53.215 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:27:53.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:53.215 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 288, 2024-07-04T17:27:53.215 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 166400 2024-07-04T17:27:53.215 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.215 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.215 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:53.215 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": [ 2024-07-04T17:27:53.215 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:53.215 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:53.215 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:53.215 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 13, 2024-07-04T17:27:53.215 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 6, 2024-07-04T17:27:53.215 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 302800, 2024-07-04T17:27:53.216 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 928, 2024-07-04T17:27:53.216 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:27:53.216 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:27:53.216 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.010592410, 2024-07-04T17:27:53.216 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.000778310, 2024-07-04T17:27:53.216 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.017163663, 2024-07-04T17:27:53.216 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:27:53.216 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:27:53.216 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 6, 2024-07-04T17:27:53.216 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.003172173, 2024-07-04T17:27:53.216 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000528695 2024-07-04T17:27:53.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:53.216 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:27:53.216 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:27:53.217 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:27:53.217 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:27:53.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:53.217 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 302800, 2024-07-04T17:27:53.217 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 928 2024-07-04T17:27:53.217 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.217 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.217 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:53.217 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": [ 2024-07-04T17:27:53.217 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:53.217 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:53.217 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:53.217 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-07-04T17:27:53.217 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-07-04T17:27:53.218 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.218 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.218 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:53.218 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror": [ 2024-07-04T17:27:53.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:53.218 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:53.218 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:53.218 INFO:teuthology.orchestra.run.smithi042.stdout: "mirrored_filesystems": 1, 2024-07-04T17:27:53.218 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_enable_failures": 0 2024-07-04T17:27:53.218 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.218 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.218 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:53.218 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_mirrored_filesystems": [ 2024-07-04T17:27:53.218 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:53.219 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:27:53.219 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem": "cephfs" 2024-07-04T17:27:53.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:53.219 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:53.219 INFO:teuthology.orchestra.run.smithi042.stdout: "mirroring_peers": 0, 2024-07-04T17:27:53.219 INFO:teuthology.orchestra.run.smithi042.stdout: "directory_count": 0 2024-07-04T17:27:53.219 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.219 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.219 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:53.219 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": [ 2024-07-04T17:27:53.219 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:53.220 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:53.220 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:53.220 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-07-04T17:27:53.220 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-07-04T17:27:53.220 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T17:27:53.221 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-07-04T17:27:53.221 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T17:27:53.221 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-07-04T17:27:53.221 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T17:27:53.221 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T17:27:53.221 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T17:27:53.221 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T17:27:53.221 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T17:27:53.221 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-07-04T17:27:53.222 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T17:27:53.222 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T17:27:53.222 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-07-04T17:27:53.222 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-07-04T17:27:53.222 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-07-04T17:27:53.222 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-07-04T17:27:53.222 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T17:27:53.223 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T17:27:53.223 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T17:27:53.223 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T17:27:53.223 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T17:27:53.223 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-07-04T17:27:53.223 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-07-04T17:27:53.223 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-07-04T17:27:53.223 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T17:27:53.223 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T17:27:53.223 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-07-04T17:27:53.223 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-07-04T17:27:53.223 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-07-04T17:27:53.224 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-07-04T17:27:53.224 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T17:27:53.224 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-07-04T17:27:53.224 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T17:27:53.224 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-07-04T17:27:53.224 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 182671, 2024-07-04T17:27:53.224 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 53, 2024-07-04T17:27:53.224 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-07-04T17:27:53.224 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-07-04T17:27:53.224 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-07-04T17:27:53.224 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-07-04T17:27:53.224 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-07-04T17:27:53.224 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-07-04T17:27:53.224 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-07-04T17:27:53.224 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-07-04T17:27:53.225 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 18208, 2024-07-04T17:27:53.225 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 161, 2024-07-04T17:27:53.225 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T17:27:53.225 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-07-04T17:27:53.225 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-07-04T17:27:53.225 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-07-04T17:27:53.225 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 0, 2024-07-04T17:27:53.225 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 0, 2024-07-04T17:27:53.225 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-07-04T17:27:53.225 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-07-04T17:27:53.225 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-07-04T17:27:53.225 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-07-04T17:27:53.225 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.225 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.225 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:53.226 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter": [ 2024-07-04T17:27:53.226 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:53.226 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:53.226 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:53.226 INFO:teuthology.orchestra.run.smithi042.stdout: "op_active": 0, 2024-07-04T17:27:53.226 INFO:teuthology.orchestra.run.smithi042.stdout: "op_laggy": 0, 2024-07-04T17:27:53.226 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send": 0, 2024-07-04T17:27:53.227 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send_bytes": 0, 2024-07-04T17:27:53.227 INFO:teuthology.orchestra.run.smithi042.stdout: "op_resend": 0, 2024-07-04T17:27:53.227 INFO:teuthology.orchestra.run.smithi042.stdout: "op_reply": 0, 2024-07-04T17:27:53.227 INFO:teuthology.orchestra.run.smithi042.stdout: "op_latency": { 2024-07-04T17:27:53.227 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:27:53.228 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:27:53.228 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:27:53.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:53.228 INFO:teuthology.orchestra.run.smithi042.stdout: "op_inflight": 0, 2024-07-04T17:27:53.228 INFO:teuthology.orchestra.run.smithi042.stdout: "oplen_avg": { 2024-07-04T17:27:53.228 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:27:53.229 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0 2024-07-04T17:27:53.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:53.229 INFO:teuthology.orchestra.run.smithi042.stdout: "op": 0, 2024-07-04T17:27:53.229 INFO:teuthology.orchestra.run.smithi042.stdout: "op_r": 0, 2024-07-04T17:27:53.229 INFO:teuthology.orchestra.run.smithi042.stdout: "op_w": 0, 2024-07-04T17:27:53.230 INFO:teuthology.orchestra.run.smithi042.stdout: "op_rmw": 0, 2024-07-04T17:27:53.230 INFO:teuthology.orchestra.run.smithi042.stdout: "op_pg": 0, 2024-07-04T17:27:53.230 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_stat": 0, 2024-07-04T17:27:53.230 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_create": 0, 2024-07-04T17:27:53.230 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_read": 0, 2024-07-04T17:27:53.230 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_write": 0, 2024-07-04T17:27:53.231 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writefull": 0, 2024-07-04T17:27:53.231 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writesame": 0, 2024-07-04T17:27:53.231 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_append": 0, 2024-07-04T17:27:53.231 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_zero": 0, 2024-07-04T17:27:53.231 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_truncate": 0, 2024-07-04T17:27:53.232 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_delete": 0, 2024-07-04T17:27:53.232 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_mapext": 0, 2024-07-04T17:27:53.232 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_sparse_read": 0, 2024-07-04T17:27:53.232 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_clonerange": 0, 2024-07-04T17:27:53.232 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_getxattr": 0, 2024-07-04T17:27:53.233 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_setxattr": 0, 2024-07-04T17:27:53.233 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_cmpxattr": 0, 2024-07-04T17:27:53.233 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_rmxattr": 0, 2024-07-04T17:27:53.233 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_resetxattrs": 0, 2024-07-04T17:27:53.233 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_call": 0, 2024-07-04T17:27:53.234 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_watch": 0, 2024-07-04T17:27:53.234 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_notify": 0, 2024-07-04T17:27:53.234 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T17:27:53.234 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls": 0, 2024-07-04T17:27:53.234 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls_filter": 0, 2024-07-04T17:27:53.235 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_other": 0, 2024-07-04T17:27:53.235 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_active": 0, 2024-07-04T17:27:53.235 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_send": 0, 2024-07-04T17:27:53.235 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_resend": 0, 2024-07-04T17:27:53.235 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_ping": 0, 2024-07-04T17:27:53.235 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_active": 0, 2024-07-04T17:27:53.236 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_send": 0, 2024-07-04T17:27:53.236 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_resend": 0, 2024-07-04T17:27:53.236 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_active": 0, 2024-07-04T17:27:53.236 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_send": 0, 2024-07-04T17:27:53.236 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_resend": 0, 2024-07-04T17:27:53.237 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_active": 0, 2024-07-04T17:27:53.237 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_send": 0, 2024-07-04T17:27:53.237 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_resend": 0, 2024-07-04T17:27:53.237 INFO:teuthology.orchestra.run.smithi042.stdout: "command_active": 0, 2024-07-04T17:27:53.237 INFO:teuthology.orchestra.run.smithi042.stdout: "command_send": 0, 2024-07-04T17:27:53.237 INFO:teuthology.orchestra.run.smithi042.stdout: "command_resend": 0, 2024-07-04T17:27:53.238 INFO:teuthology.orchestra.run.smithi042.stdout: "map_epoch": 0, 2024-07-04T17:27:53.238 INFO:teuthology.orchestra.run.smithi042.stdout: "map_full": 0, 2024-07-04T17:27:53.238 INFO:teuthology.orchestra.run.smithi042.stdout: "map_inc": 0, 2024-07-04T17:27:53.238 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_sessions": 0, 2024-07-04T17:27:53.238 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_open": 0, 2024-07-04T17:27:53.239 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_close": 0, 2024-07-04T17:27:53.239 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_laggy": 0, 2024-07-04T17:27:53.239 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_wr": 0, 2024-07-04T17:27:53.239 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_rd": 0, 2024-07-04T17:27:53.239 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_del": 0 2024-07-04T17:27:53.239 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.240 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.240 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:53.240 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-client": [ 2024-07-04T17:27:53.240 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:53.240 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:53.241 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:53.241 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:27:53.241 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:27:53.241 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:27:53.241 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 20, 2024-07-04T17:27:53.242 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 71695, 2024-07-04T17:27:53.242 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:27:53.242 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 20, 2024-07-04T17:27:53.242 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:27:53.242 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:27:53.242 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 20, 2024-07-04T17:27:53.243 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 71695, 2024-07-04T17:27:53.243 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:27:53.243 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:27:53.243 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:27:53.243 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:27:53.244 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.244 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.244 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.244 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:53.244 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-radosclient": [ 2024-07-04T17:27:53.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:53.245 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:53.245 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:53.245 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:27:53.245 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:27:53.246 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:27:53.246 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 12, 2024-07-04T17:27:53.246 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 300554, 2024-07-04T17:27:53.246 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:27:53.246 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 12, 2024-07-04T17:27:53.247 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:27:53.247 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:27:53.247 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 12, 2024-07-04T17:27:53.247 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 300554, 2024-07-04T17:27:53.247 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:27:53.247 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:27:53.248 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:27:53.248 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:27:53.248 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.248 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.248 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.249 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:53.249 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_bytes": [ 2024-07-04T17:27:53.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:53.249 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:53.249 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:53.250 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:27:53.250 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:27:53.250 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:27:53.250 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:27:53.250 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:27:53.251 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:27:53.251 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:27:53.251 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:27:53.251 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:27:53.251 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:27:53.251 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:27:53.252 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:27:53.252 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:27:53.252 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:27:53.252 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:27:53.252 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.253 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.253 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.253 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:53.253 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_ops": [ 2024-07-04T17:27:53.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:53.254 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:53.254 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:53.254 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:27:53.254 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1024, 2024-07-04T17:27:53.254 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:27:53.255 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:27:53.255 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:27:53.255 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:27:53.255 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:27:53.255 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:27:53.255 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:27:53.256 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:27:53.256 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:27:53.256 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:27:53.256 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:27:53.256 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:27:53.257 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:27:53.257 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.257 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.257 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.257 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-07-04T17:27:53.258 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:27:53.259 DEBUG:tasks.cephfs.test_mirroring:command returned={ "AsyncMessenger::Worker": [ { "labels": { "id": "0" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "1" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "2" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } } ], "AsyncMessenger::Worker-0": [ { "labels": {}, "counters": { "msgr_recv_messages": 20, "msgr_send_messages": 3, "msgr_recv_bytes": 74416, "msgr_send_bytes": 464, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.007916628, "msgr_running_send_time": 0.000760014, "msgr_running_recv_time": 0.005337197, "msgr_running_fast_dispatch_time": 0.001671330, "msgr_send_messages_queue_lat": { "avgcount": 3, "sum": 0.004312719, "avgtime": 0.001437573 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 74416, "msgr_send_encrypted_bytes": 464 } } ], "AsyncMessenger::Worker-1": [ { "labels": {}, "counters": { "msgr_recv_messages": 2, "msgr_send_messages": 19, "msgr_recv_bytes": 288, "msgr_send_bytes": 166400, "msgr_created_connections": 5, "msgr_active_connections": 1, "msgr_running_total_time": 0.009524650, "msgr_running_send_time": 0.003642952, "msgr_running_recv_time": 0.003527254, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 18, "sum": 0.016382422, "avgtime": 0.000910134 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 288, "msgr_send_encrypted_bytes": 166400 } } ], "AsyncMessenger::Worker-2": [ { "labels": {}, "counters": { "msgr_recv_messages": 13, "msgr_send_messages": 6, "msgr_recv_bytes": 302800, "msgr_send_bytes": 928, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.010592410, "msgr_running_send_time": 0.000778310, "msgr_running_recv_time": 0.017163663, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 6, "sum": 0.003172173, "avgtime": 0.000528695 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 302800, "msgr_send_encrypted_bytes": 928 } } ], "cct": [ { "labels": {}, "counters": { "total_workers": 0, "unhealthy_workers": 0 } } ], "cephfs_mirror": [ { "labels": {}, "counters": { "mirrored_filesystems": 1, "mirror_enable_failures": 0 } } ], "cephfs_mirror_mirrored_filesystems": [ { "labels": { "filesystem": "cephfs" }, "counters": { "mirroring_peers": 0, "directory_count": 0 } } ], "mempool": [ { "labels": {}, "counters": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_items": 0, "bluefs_bytes": 0, "bluefs_items": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "buffer_anon_bytes": 182671, "buffer_anon_items": 53, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 18208, "osdmap_items": 161, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "mds_co_bytes": 0, "mds_co_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 } } ], "objecter": [ { "labels": {}, "counters": { "op_active": 0, "op_laggy": 0, "op_send": 0, "op_send_bytes": 0, "op_resend": 0, "op_reply": 0, "op_latency": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "op_inflight": 0, "oplen_avg": { "avgcount": 0, "sum": 0 }, "op": 0, "op_r": 0, "op_w": 0, "op_rmw": 0, "op_pg": 0, "osdop_stat": 0, "osdop_create": 0, "osdop_read": 0, "osdop_write": 0, "osdop_writefull": 0, "osdop_writesame": 0, "osdop_append": 0, "osdop_zero": 0, "osdop_truncate": 0, "osdop_delete": 0, "osdop_mapext": 0, "osdop_sparse_read": 0, "osdop_clonerange": 0, "osdop_getxattr": 0, "osdop_setxattr": 0, "osdop_cmpxattr": 0, "osdop_rmxattr": 0, "osdop_resetxattrs": 0, "osdop_call": 0, "osdop_watch": 0, "osdop_notify": 0, "osdop_src_cmpxattr": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_other": 0, "linger_active": 0, "linger_send": 0, "linger_resend": 0, "linger_ping": 0, "poolop_active": 0, "poolop_send": 0, "poolop_resend": 0, "poolstat_active": 0, "poolstat_send": 0, "poolstat_resend": 0, "statfs_active": 0, "statfs_send": 0, "statfs_resend": 0, "command_active": 0, "command_send": 0, "command_resend": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "osd_sessions": 0, "osd_session_open": 0, "osd_session_close": 0, "osd_laggy": 0, "omap_wr": 0, "omap_rd": 0, "omap_del": 0 } } ], "throttle-msgr_dispatch_throttler-client": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 20, "get_sum": 71695, "get_or_fail_fail": 0, "get_or_fail_success": 20, "take": 0, "take_sum": 0, "put": 20, "put_sum": 71695, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-msgr_dispatch_throttler-radosclient": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 12, "get_sum": 300554, "get_or_fail_fail": 0, "get_or_fail_success": 12, "take": 0, "take_sum": 0, "put": 12, "put_sum": 300554, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_bytes": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_ops": [ { "labels": {}, "counters": { "val": 0, "max": 1024, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ] } 2024-07-04T17:27:53.260 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:27:53.260 DEBUG:teuthology.orchestra.run.smithi042:counter dump for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok counter dump 2024-07-04T17:27:53.510 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:27:53.511 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker": [ 2024-07-04T17:27:53.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:53.511 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:27:53.511 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "0" 2024-07-04T17:27:53.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:53.511 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:53.511 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:27:53.511 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:27:53.511 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:53.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:53.511 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:27:53.511 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "1" 2024-07-04T17:27:53.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:53.512 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:53.512 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:27:53.512 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:27:53.512 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:53.512 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:53.512 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:27:53.512 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "2" 2024-07-04T17:27:53.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:53.512 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:53.512 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:27:53.512 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:27:53.512 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.512 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.513 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:53.513 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": [ 2024-07-04T17:27:53.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:53.513 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:53.513 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:53.513 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 20, 2024-07-04T17:27:53.513 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 3, 2024-07-04T17:27:53.513 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 74416, 2024-07-04T17:27:53.513 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 464, 2024-07-04T17:27:53.513 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:27:53.513 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:27:53.513 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.007916628, 2024-07-04T17:27:53.513 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.000760014, 2024-07-04T17:27:53.514 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.005337197, 2024-07-04T17:27:53.514 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.001671330, 2024-07-04T17:27:53.514 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:27:53.514 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 3, 2024-07-04T17:27:53.514 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.004312719, 2024-07-04T17:27:53.514 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.001437573 2024-07-04T17:27:53.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:53.514 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:27:53.514 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:27:53.514 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:27:53.514 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:27:53.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:53.515 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 74416, 2024-07-04T17:27:53.515 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 464 2024-07-04T17:27:53.515 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.515 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.515 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:53.515 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": [ 2024-07-04T17:27:53.515 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:53.515 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:53.515 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:53.515 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 2, 2024-07-04T17:27:53.515 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 19, 2024-07-04T17:27:53.515 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 288, 2024-07-04T17:27:53.515 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 166400, 2024-07-04T17:27:53.516 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 5, 2024-07-04T17:27:53.516 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:27:53.516 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.009524650, 2024-07-04T17:27:53.516 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.003642952, 2024-07-04T17:27:53.516 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.003527254, 2024-07-04T17:27:53.516 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:27:53.516 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:27:53.516 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 18, 2024-07-04T17:27:53.516 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.016382422, 2024-07-04T17:27:53.516 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000910134 2024-07-04T17:27:53.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:53.516 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:27:53.517 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:27:53.517 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:27:53.517 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:27:53.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:53.517 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 288, 2024-07-04T17:27:53.517 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 166400 2024-07-04T17:27:53.517 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.517 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.517 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:53.517 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": [ 2024-07-04T17:27:53.517 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:53.517 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:53.517 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:53.518 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 13, 2024-07-04T17:27:53.518 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 6, 2024-07-04T17:27:53.518 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 302800, 2024-07-04T17:27:53.518 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 928, 2024-07-04T17:27:53.518 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:27:53.518 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:27:53.518 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.010592410, 2024-07-04T17:27:53.518 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.000778310, 2024-07-04T17:27:53.518 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.017163663, 2024-07-04T17:27:53.518 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:27:53.518 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:27:53.518 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 6, 2024-07-04T17:27:53.518 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.003172173, 2024-07-04T17:27:53.519 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000528695 2024-07-04T17:27:53.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:53.519 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:27:53.519 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:27:53.519 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:27:53.519 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:27:53.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:53.519 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 302800, 2024-07-04T17:27:53.519 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 928 2024-07-04T17:27:53.519 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.519 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.519 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:53.519 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": [ 2024-07-04T17:27:53.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:53.520 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:53.520 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:53.520 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-07-04T17:27:53.520 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-07-04T17:27:53.520 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.520 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.520 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:53.520 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror": [ 2024-07-04T17:27:53.520 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:53.520 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:53.521 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:53.521 INFO:teuthology.orchestra.run.smithi042.stdout: "mirrored_filesystems": 1, 2024-07-04T17:27:53.521 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_enable_failures": 0 2024-07-04T17:27:53.521 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.521 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.521 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:53.521 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_mirrored_filesystems": [ 2024-07-04T17:27:53.521 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:53.521 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:27:53.521 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem": "cephfs" 2024-07-04T17:27:53.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:53.521 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:53.521 INFO:teuthology.orchestra.run.smithi042.stdout: "mirroring_peers": 0, 2024-07-04T17:27:53.522 INFO:teuthology.orchestra.run.smithi042.stdout: "directory_count": 0 2024-07-04T17:27:53.522 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.522 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.522 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:53.522 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": [ 2024-07-04T17:27:53.522 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:53.522 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:53.522 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:53.522 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-07-04T17:27:53.522 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-07-04T17:27:53.522 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T17:27:53.522 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-07-04T17:27:53.522 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T17:27:53.523 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-07-04T17:27:53.523 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T17:27:53.523 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T17:27:53.523 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T17:27:53.523 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T17:27:53.523 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T17:27:53.523 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-07-04T17:27:53.523 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T17:27:53.523 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T17:27:53.523 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-07-04T17:27:53.523 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-07-04T17:27:53.523 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-07-04T17:27:53.523 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-07-04T17:27:53.523 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T17:27:53.524 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T17:27:53.524 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T17:27:53.524 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T17:27:53.524 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T17:27:53.524 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-07-04T17:27:53.524 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-07-04T17:27:53.524 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-07-04T17:27:53.524 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T17:27:53.524 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T17:27:53.524 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-07-04T17:27:53.524 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-07-04T17:27:53.524 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-07-04T17:27:53.524 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-07-04T17:27:53.524 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T17:27:53.525 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-07-04T17:27:53.525 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T17:27:53.525 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-07-04T17:27:53.525 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 182671, 2024-07-04T17:27:53.525 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 53, 2024-07-04T17:27:53.525 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-07-04T17:27:53.525 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-07-04T17:27:53.525 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-07-04T17:27:53.525 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-07-04T17:27:53.525 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-07-04T17:27:53.525 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-07-04T17:27:53.525 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-07-04T17:27:53.525 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-07-04T17:27:53.526 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 18208, 2024-07-04T17:27:53.526 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 161, 2024-07-04T17:27:53.526 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T17:27:53.526 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-07-04T17:27:53.526 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-07-04T17:27:53.526 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-07-04T17:27:53.526 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 0, 2024-07-04T17:27:53.526 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 0, 2024-07-04T17:27:53.526 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-07-04T17:27:53.526 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-07-04T17:27:53.526 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-07-04T17:27:53.527 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-07-04T17:27:53.527 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.527 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.527 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:53.527 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter": [ 2024-07-04T17:27:53.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:53.527 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:53.527 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:53.527 INFO:teuthology.orchestra.run.smithi042.stdout: "op_active": 0, 2024-07-04T17:27:53.527 INFO:teuthology.orchestra.run.smithi042.stdout: "op_laggy": 0, 2024-07-04T17:27:53.527 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send": 0, 2024-07-04T17:27:53.527 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send_bytes": 0, 2024-07-04T17:27:53.527 INFO:teuthology.orchestra.run.smithi042.stdout: "op_resend": 0, 2024-07-04T17:27:53.527 INFO:teuthology.orchestra.run.smithi042.stdout: "op_reply": 0, 2024-07-04T17:27:53.528 INFO:teuthology.orchestra.run.smithi042.stdout: "op_latency": { 2024-07-04T17:27:53.528 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:27:53.528 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:27:53.528 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:27:53.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:53.528 INFO:teuthology.orchestra.run.smithi042.stdout: "op_inflight": 0, 2024-07-04T17:27:53.528 INFO:teuthology.orchestra.run.smithi042.stdout: "oplen_avg": { 2024-07-04T17:27:53.528 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:27:53.528 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0 2024-07-04T17:27:53.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:27:53.528 INFO:teuthology.orchestra.run.smithi042.stdout: "op": 0, 2024-07-04T17:27:53.528 INFO:teuthology.orchestra.run.smithi042.stdout: "op_r": 0, 2024-07-04T17:27:53.528 INFO:teuthology.orchestra.run.smithi042.stdout: "op_w": 0, 2024-07-04T17:27:53.528 INFO:teuthology.orchestra.run.smithi042.stdout: "op_rmw": 0, 2024-07-04T17:27:53.528 INFO:teuthology.orchestra.run.smithi042.stdout: "op_pg": 0, 2024-07-04T17:27:53.529 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_stat": 0, 2024-07-04T17:27:53.529 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_create": 0, 2024-07-04T17:27:53.529 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_read": 0, 2024-07-04T17:27:53.529 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_write": 0, 2024-07-04T17:27:53.529 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writefull": 0, 2024-07-04T17:27:53.529 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writesame": 0, 2024-07-04T17:27:53.529 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_append": 0, 2024-07-04T17:27:53.529 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_zero": 0, 2024-07-04T17:27:53.529 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_truncate": 0, 2024-07-04T17:27:53.529 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_delete": 0, 2024-07-04T17:27:53.529 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_mapext": 0, 2024-07-04T17:27:53.529 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_sparse_read": 0, 2024-07-04T17:27:53.529 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_clonerange": 0, 2024-07-04T17:27:53.529 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_getxattr": 0, 2024-07-04T17:27:53.530 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_setxattr": 0, 2024-07-04T17:27:53.530 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_cmpxattr": 0, 2024-07-04T17:27:53.530 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_rmxattr": 0, 2024-07-04T17:27:53.530 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_resetxattrs": 0, 2024-07-04T17:27:53.530 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_call": 0, 2024-07-04T17:27:53.530 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_watch": 0, 2024-07-04T17:27:53.530 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_notify": 0, 2024-07-04T17:27:53.530 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T17:27:53.530 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls": 0, 2024-07-04T17:27:53.530 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls_filter": 0, 2024-07-04T17:27:53.530 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_other": 0, 2024-07-04T17:27:53.530 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_active": 0, 2024-07-04T17:27:53.530 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_send": 0, 2024-07-04T17:27:53.530 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_resend": 0, 2024-07-04T17:27:53.531 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_ping": 0, 2024-07-04T17:27:53.531 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_active": 0, 2024-07-04T17:27:53.531 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_send": 0, 2024-07-04T17:27:53.531 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_resend": 0, 2024-07-04T17:27:53.531 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_active": 0, 2024-07-04T17:27:53.531 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_send": 0, 2024-07-04T17:27:53.531 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_resend": 0, 2024-07-04T17:27:53.531 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_active": 0, 2024-07-04T17:27:53.531 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_send": 0, 2024-07-04T17:27:53.531 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_resend": 0, 2024-07-04T17:27:53.531 INFO:teuthology.orchestra.run.smithi042.stdout: "command_active": 0, 2024-07-04T17:27:53.531 INFO:teuthology.orchestra.run.smithi042.stdout: "command_send": 0, 2024-07-04T17:27:53.531 INFO:teuthology.orchestra.run.smithi042.stdout: "command_resend": 0, 2024-07-04T17:27:53.531 INFO:teuthology.orchestra.run.smithi042.stdout: "map_epoch": 0, 2024-07-04T17:27:53.531 INFO:teuthology.orchestra.run.smithi042.stdout: "map_full": 0, 2024-07-04T17:27:53.532 INFO:teuthology.orchestra.run.smithi042.stdout: "map_inc": 0, 2024-07-04T17:27:53.532 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_sessions": 0, 2024-07-04T17:27:53.532 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_open": 0, 2024-07-04T17:27:53.532 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_close": 0, 2024-07-04T17:27:53.532 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_laggy": 0, 2024-07-04T17:27:53.532 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_wr": 0, 2024-07-04T17:27:53.532 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_rd": 0, 2024-07-04T17:27:53.532 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_del": 0 2024-07-04T17:27:53.532 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.532 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.532 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:53.532 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-client": [ 2024-07-04T17:27:53.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:53.532 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:53.532 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:53.533 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:27:53.533 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:27:53.533 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:27:53.533 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 20, 2024-07-04T17:27:53.533 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 71695, 2024-07-04T17:27:53.533 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:27:53.533 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 20, 2024-07-04T17:27:53.533 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:27:53.533 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:27:53.533 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 20, 2024-07-04T17:27:53.533 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 71695, 2024-07-04T17:27:53.533 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:27:53.533 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:27:53.533 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:27:53.534 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:27:53.534 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.534 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.534 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.534 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:53.534 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-radosclient": [ 2024-07-04T17:27:53.534 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:53.534 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:53.534 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:53.534 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:27:53.534 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:27:53.534 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:27:53.535 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 12, 2024-07-04T17:27:53.535 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 300554, 2024-07-04T17:27:53.535 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:27:53.535 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 12, 2024-07-04T17:27:53.535 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:27:53.535 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:27:53.535 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 12, 2024-07-04T17:27:53.535 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 300554, 2024-07-04T17:27:53.535 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:27:53.535 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:27:53.535 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:27:53.535 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:27:53.535 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.535 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.535 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.536 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:53.536 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_bytes": [ 2024-07-04T17:27:53.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:53.536 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:53.536 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:53.536 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:27:53.536 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:27:53.536 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:27:53.536 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:27:53.536 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:27:53.536 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:27:53.536 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:27:53.536 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:27:53.537 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:27:53.537 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:27:53.537 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:27:53.537 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:27:53.537 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:27:53.537 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:27:53.537 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:27:53.537 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.537 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.537 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.537 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:27:53.537 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_ops": [ 2024-07-04T17:27:53.537 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:27:53.537 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:27:53.537 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:27:53.538 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:27:53.538 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1024, 2024-07-04T17:27:53.538 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:27:53.538 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:27:53.538 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:27:53.538 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:27:53.538 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:27:53.538 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:27:53.538 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:27:53.538 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:27:53.538 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:27:53.538 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:27:53.538 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:27:53.539 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:27:53.539 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:27:53.539 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.539 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.539 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.539 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-07-04T17:27:53.539 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:27:53.540 DEBUG:tasks.cephfs.test_mirroring:command returned={ "AsyncMessenger::Worker": [ { "labels": { "id": "0" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "1" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "2" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } } ], "AsyncMessenger::Worker-0": [ { "labels": {}, "counters": { "msgr_recv_messages": 20, "msgr_send_messages": 3, "msgr_recv_bytes": 74416, "msgr_send_bytes": 464, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.007916628, "msgr_running_send_time": 0.000760014, "msgr_running_recv_time": 0.005337197, "msgr_running_fast_dispatch_time": 0.001671330, "msgr_send_messages_queue_lat": { "avgcount": 3, "sum": 0.004312719, "avgtime": 0.001437573 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 74416, "msgr_send_encrypted_bytes": 464 } } ], "AsyncMessenger::Worker-1": [ { "labels": {}, "counters": { "msgr_recv_messages": 2, "msgr_send_messages": 19, "msgr_recv_bytes": 288, "msgr_send_bytes": 166400, "msgr_created_connections": 5, "msgr_active_connections": 1, "msgr_running_total_time": 0.009524650, "msgr_running_send_time": 0.003642952, "msgr_running_recv_time": 0.003527254, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 18, "sum": 0.016382422, "avgtime": 0.000910134 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 288, "msgr_send_encrypted_bytes": 166400 } } ], "AsyncMessenger::Worker-2": [ { "labels": {}, "counters": { "msgr_recv_messages": 13, "msgr_send_messages": 6, "msgr_recv_bytes": 302800, "msgr_send_bytes": 928, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.010592410, "msgr_running_send_time": 0.000778310, "msgr_running_recv_time": 0.017163663, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 6, "sum": 0.003172173, "avgtime": 0.000528695 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 302800, "msgr_send_encrypted_bytes": 928 } } ], "cct": [ { "labels": {}, "counters": { "total_workers": 0, "unhealthy_workers": 0 } } ], "cephfs_mirror": [ { "labels": {}, "counters": { "mirrored_filesystems": 1, "mirror_enable_failures": 0 } } ], "cephfs_mirror_mirrored_filesystems": [ { "labels": { "filesystem": "cephfs" }, "counters": { "mirroring_peers": 0, "directory_count": 0 } } ], "mempool": [ { "labels": {}, "counters": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_items": 0, "bluefs_bytes": 0, "bluefs_items": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "buffer_anon_bytes": 182671, "buffer_anon_items": 53, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 18208, "osdmap_items": 161, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "mds_co_bytes": 0, "mds_co_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 } } ], "objecter": [ { "labels": {}, "counters": { "op_active": 0, "op_laggy": 0, "op_send": 0, "op_send_bytes": 0, "op_resend": 0, "op_reply": 0, "op_latency": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "op_inflight": 0, "oplen_avg": { "avgcount": 0, "sum": 0 }, "op": 0, "op_r": 0, "op_w": 0, "op_rmw": 0, "op_pg": 0, "osdop_stat": 0, "osdop_create": 0, "osdop_read": 0, "osdop_write": 0, "osdop_writefull": 0, "osdop_writesame": 0, "osdop_append": 0, "osdop_zero": 0, "osdop_truncate": 0, "osdop_delete": 0, "osdop_mapext": 0, "osdop_sparse_read": 0, "osdop_clonerange": 0, "osdop_getxattr": 0, "osdop_setxattr": 0, "osdop_cmpxattr": 0, "osdop_rmxattr": 0, "osdop_resetxattrs": 0, "osdop_call": 0, "osdop_watch": 0, "osdop_notify": 0, "osdop_src_cmpxattr": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_other": 0, "linger_active": 0, "linger_send": 0, "linger_resend": 0, "linger_ping": 0, "poolop_active": 0, "poolop_send": 0, "poolop_resend": 0, "poolstat_active": 0, "poolstat_send": 0, "poolstat_resend": 0, "statfs_active": 0, "statfs_send": 0, "statfs_resend": 0, "command_active": 0, "command_send": 0, "command_resend": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "osd_sessions": 0, "osd_session_open": 0, "osd_session_close": 0, "osd_laggy": 0, "omap_wr": 0, "omap_rd": 0, "omap_del": 0 } } ], "throttle-msgr_dispatch_throttler-client": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 20, "get_sum": 71695, "get_or_fail_fail": 0, "get_or_fail_success": 20, "take": 0, "take_sum": 0, "put": 20, "put_sum": 71695, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-msgr_dispatch_throttler-radosclient": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 12, "get_sum": 300554, "get_or_fail_fail": 0, "get_or_fail_success": 12, "take": 0, "take_sum": 0, "put": 12, "put_sum": 300554, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_bytes": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_ops": [ { "labels": {}, "counters": { "val": 0, "max": 1024, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ] } 2024-07-04T17:27:53.540 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:27:53.540 DEBUG:teuthology.orchestra.run.smithi042:mirror status for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@2 2024-07-04T17:27:53.788 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:27:53.788 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_inst": "172.21.15.42:0/487218478", 2024-07-04T17:27:53.788 INFO:teuthology.orchestra.run.smithi042.stdout: "peers": {}, 2024-07-04T17:27:53.789 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_dirs": { 2024-07-04T17:27:53.789 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_count": 0 2024-07-04T17:27:53.789 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:27:53.789 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:27:53.805 DEBUG:tasks.cephfs.test_mirroring:command returned={ "rados_inst": "172.21.15.42:0/487218478", "peers": {}, "snap_dirs": { "dir_count": 0 } } 2024-07-04T17:27:53.805 DEBUG:tasks.cephfs.test_mirroring:initial dir_count=0 2024-07-04T17:27:53.805 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snapshot mirror add cephfs /d1/d2/ 2024-07-04T17:27:54.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.111+0000 7f2ebc000640 1 -- 172.21.15.42:0/1580804525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2eb406dc80 msgr2=0x7f2eb40a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:54.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.111+0000 7f2ebc000640 1 --2- 172.21.15.42:0/1580804525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2eb406dc80 0x7f2eb40a94d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8005fd0 tx=0x7f2ea802a8b0 comp rx=0 tx=0).stop 2024-07-04T17:27:54.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.112+0000 7f2ebc000640 1 -- 172.21.15.42:0/1580804525 shutdown_connections 2024-07-04T17:27:54.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.112+0000 7f2ebc000640 1 --2- 172.21.15.42:0/1580804525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2eb406dc80 0x7f2eb40a94d0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:54.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.112+0000 7f2ebc000640 1 -- 172.21.15.42:0/1580804525 >> 172.21.15.42:0/1580804525 conn(0x7f2eb406c020 msgr2=0x7f2eb406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:54.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.112+0000 7f2ebc000640 1 -- 172.21.15.42:0/1580804525 shutdown_connections 2024-07-04T17:27:54.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.112+0000 7f2ebc000640 1 -- 172.21.15.42:0/1580804525 wait complete. 2024-07-04T17:27:54.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.114+0000 7f2ebc000640 1 Processor -- start 2024-07-04T17:27:54.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.114+0000 7f2ebc000640 1 -- start start 2024-07-04T17:27:54.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.115+0000 7f2ebc000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2eb406dc80 0x7f2eb41d4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:54.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.115+0000 7f2ebc000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb4072740 con 0x7f2eb406dc80 2024-07-04T17:27:54.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.116+0000 7f2eb9600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2eb406dc80 0x7f2eb41d4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:54.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.116+0000 7f2eb9600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2eb406dc80 0x7f2eb41d4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46880/0 (socket says 172.21.15.42:46880) 2024-07-04T17:27:54.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.116+0000 7f2eb9600640 1 -- 172.21.15.42:0/1946392272 learned_addr learned my addr 172.21.15.42:0/1946392272 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:27:54.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.116+0000 7f2eb9600640 1 -- 172.21.15.42:0/1946392272 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb41d6a70 con 0x7f2eb406dc80 2024-07-04T17:27:54.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.117+0000 7f2eb9600640 1 --2- 172.21.15.42:0/1946392272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2eb406dc80 0x7f2eb41d4ae0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8009c50 tx=0x7f2ea8031670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:54.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.117+0000 7f2eb1600640 1 -- 172.21.15.42:0/1946392272 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ea80318c0 con 0x7f2eb406dc80 2024-07-04T17:27:54.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.117+0000 7f2eb1600640 1 -- 172.21.15.42:0/1946392272 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ea8031b50 con 0x7f2eb406dc80 2024-07-04T17:27:54.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.117+0000 7f2ebc000640 1 -- 172.21.15.42:0/1946392272 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eb41e4760 con 0x7f2eb406dc80 2024-07-04T17:27:54.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.117+0000 7f2eb1600640 1 -- 172.21.15.42:0/1946392272 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ea80330d0 con 0x7f2eb406dc80 2024-07-04T17:27:54.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.117+0000 7f2ebc000640 1 -- 172.21.15.42:0/1946392272 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eb413ba80 con 0x7f2eb406dc80 2024-07-04T17:27:54.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.118+0000 7f2ebc000640 1 -- 172.21.15.42:0/1946392272 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eb40a94d0 con 0x7f2eb406dc80 2024-07-04T17:27:54.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.119+0000 7f2eb1600640 1 -- 172.21.15.42:0/1946392272 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 10) ==== 49681+0+0 (secure 0 0 0) 0x7f2ea8033270 con 0x7f2eb406dc80 2024-07-04T17:27:54.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.119+0000 7f2eb1600640 1 --2- 172.21.15.42:0/1946392272 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f2e9403d5d0 0x7f2e9403fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:27:54.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.119+0000 7f2eb1600640 1 -- 172.21.15.42:0/1946392272 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4561+0+0 (secure 0 0 0) 0x7f2ea80734a0 con 0x7f2eb406dc80 2024-07-04T17:27:54.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.120+0000 7f2eb8c00640 1 --2- 172.21.15.42:0/1946392272 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f2e9403d5d0 0x7f2e9403fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:27:54.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.121+0000 7f2eb8c00640 1 --2- 172.21.15.42:0/1946392272 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f2e9403d5d0 0x7f2e9403fae0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2ea00015e0 tx=0x7f2ea0001290 comp rx=0 tx=0).ready entity=mgr.4563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:27:54.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.125+0000 7f2eb1600640 1 -- 172.21.15.42:0/1946392272 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f2ea803f030 con 0x7f2eb406dc80 2024-07-04T17:27:54.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.354+0000 7f2ebc000640 1 -- 172.21.15.42:0/1946392272 --> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] -- mgr_command(tid 0: {"prefix": "fs snapshot mirror add", "fs_name": "cephfs", "path": "/d1/d2/", "target": ["mon-mgr", ""]}) -- 0x7f2eb41e3770 con 0x7f2e9403d5d0 2024-07-04T17:27:54.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.361+0000 7f2eb1600640 1 -- 172.21.15.42:0/1946392272 <== mgr.4563 v2:172.21.15.42:6824/488387844 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f2eb41e3770 con 0x7f2e9403d5d0 2024-07-04T17:27:54.362 INFO:teuthology.orchestra.run.smithi042.stdout:{} 2024-07-04T17:27:54.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.364+0000 7f2ebc000640 1 -- 172.21.15.42:0/1946392272 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f2e9403d5d0 msgr2=0x7f2e9403fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:54.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.364+0000 7f2ebc000640 1 --2- 172.21.15.42:0/1946392272 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f2e9403d5d0 0x7f2e9403fae0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2ea00015e0 tx=0x7f2ea0001290 comp rx=0 tx=0).stop 2024-07-04T17:27:54.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.364+0000 7f2ebc000640 1 -- 172.21.15.42:0/1946392272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2eb406dc80 msgr2=0x7f2eb41d4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:27:54.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.364+0000 7f2ebc000640 1 --2- 172.21.15.42:0/1946392272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2eb406dc80 0x7f2eb41d4ae0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8009c50 tx=0x7f2ea8031670 comp rx=0 tx=0).stop 2024-07-04T17:27:54.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.364+0000 7f2ebc000640 1 -- 172.21.15.42:0/1946392272 shutdown_connections 2024-07-04T17:27:54.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.364+0000 7f2ebc000640 1 --2- 172.21.15.42:0/1946392272 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f2e9403d5d0 0x7f2e9403fae0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:54.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.364+0000 7f2ebc000640 1 --2- 172.21.15.42:0/1946392272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2eb406dc80 0x7f2eb41d4ae0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:27:54.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.364+0000 7f2ebc000640 1 -- 172.21.15.42:0/1946392272 >> 172.21.15.42:0/1946392272 conn(0x7f2eb406c020 msgr2=0x7f2eb413e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:27:54.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.364+0000 7f2ebc000640 1 -- 172.21.15.42:0/1946392272 shutdown_connections 2024-07-04T17:27:54.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:27:54.364+0000 7f2ebc000640 1 -- 172.21.15.42:0/1946392272 wait complete. 2024-07-04T17:27:59.772 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-04T17:28:04.391 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:28:04.392 DEBUG:teuthology.orchestra.run.smithi042:mirror status for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@2 2024-07-04T17:28:04.640 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:28:04.640 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_inst": "172.21.15.42:0/487218478", 2024-07-04T17:28:04.641 INFO:teuthology.orchestra.run.smithi042.stdout: "peers": {}, 2024-07-04T17:28:04.641 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_dirs": { 2024-07-04T17:28:04.641 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_count": 1 2024-07-04T17:28:04.641 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.641 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:28:04.657 DEBUG:tasks.cephfs.test_mirroring:command returned={ "rados_inst": "172.21.15.42:0/487218478", "peers": {}, "snap_dirs": { "dir_count": 1 } } 2024-07-04T17:28:04.658 DEBUG:tasks.cephfs.test_mirroring:new dir_count=1 2024-07-04T17:28:04.658 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:28:04.658 DEBUG:teuthology.orchestra.run.smithi042:counter dump for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok counter dump 2024-07-04T17:28:04.902 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:28:04.902 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker": [ 2024-07-04T17:28:04.902 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:04.902 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:28:04.902 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "0" 2024-07-04T17:28:04.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:04.902 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:04.902 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:28:04.902 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:28:04.902 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:04.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:04.903 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:28:04.903 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "1" 2024-07-04T17:28:04.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:04.903 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:04.903 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:28:04.903 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:28:04.903 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:04.903 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:04.903 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:28:04.903 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "2" 2024-07-04T17:28:04.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:04.904 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:04.904 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:28:04.904 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:28:04.904 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.904 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.904 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:04.904 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": [ 2024-07-04T17:28:04.904 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:04.904 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:04.904 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:04.904 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 20, 2024-07-04T17:28:04.904 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 3, 2024-07-04T17:28:04.904 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 74416, 2024-07-04T17:28:04.904 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 464, 2024-07-04T17:28:04.905 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:28:04.905 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:28:04.905 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.008159046, 2024-07-04T17:28:04.905 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.000843327, 2024-07-04T17:28:04.905 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.005383483, 2024-07-04T17:28:04.905 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.001671330, 2024-07-04T17:28:04.905 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:28:04.905 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 3, 2024-07-04T17:28:04.905 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.004312719, 2024-07-04T17:28:04.905 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.001437573 2024-07-04T17:28:04.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:04.905 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:28:04.905 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:04.905 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:04.906 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:04.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:04.906 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 74416, 2024-07-04T17:28:04.906 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 464 2024-07-04T17:28:04.906 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.906 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.906 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:04.906 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": [ 2024-07-04T17:28:04.906 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:04.906 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:04.906 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:04.906 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 2, 2024-07-04T17:28:04.906 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 21, 2024-07-04T17:28:04.907 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 288, 2024-07-04T17:28:04.907 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 166960, 2024-07-04T17:28:04.907 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 5, 2024-07-04T17:28:04.907 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:28:04.907 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.009966010, 2024-07-04T17:28:04.907 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.004031544, 2024-07-04T17:28:04.907 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.003527254, 2024-07-04T17:28:04.907 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:28:04.907 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:28:04.907 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 20, 2024-07-04T17:28:04.907 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.016584089, 2024-07-04T17:28:04.907 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000829204 2024-07-04T17:28:04.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:04.907 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:28:04.908 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:04.908 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:04.908 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:04.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:04.908 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 288, 2024-07-04T17:28:04.908 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 166960 2024-07-04T17:28:04.908 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.908 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.908 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:04.908 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": [ 2024-07-04T17:28:04.908 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:04.908 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:04.908 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:04.908 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 13, 2024-07-04T17:28:04.909 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 6, 2024-07-04T17:28:04.909 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 302800, 2024-07-04T17:28:04.909 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 928, 2024-07-04T17:28:04.909 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:28:04.909 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:28:04.909 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.010757335, 2024-07-04T17:28:04.909 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.000862072, 2024-07-04T17:28:04.909 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.017202050, 2024-07-04T17:28:04.909 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:28:04.909 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:28:04.909 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 6, 2024-07-04T17:28:04.909 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.003172173, 2024-07-04T17:28:04.909 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000528695 2024-07-04T17:28:04.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:04.909 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:28:04.909 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:04.910 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:04.910 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:04.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:04.910 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 302800, 2024-07-04T17:28:04.910 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 928 2024-07-04T17:28:04.910 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.910 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.910 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:04.910 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": [ 2024-07-04T17:28:04.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:04.911 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:04.911 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:04.911 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-07-04T17:28:04.911 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-07-04T17:28:04.911 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.911 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.911 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:04.911 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror": [ 2024-07-04T17:28:04.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:04.911 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:04.911 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:04.911 INFO:teuthology.orchestra.run.smithi042.stdout: "mirrored_filesystems": 1, 2024-07-04T17:28:04.911 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_enable_failures": 0 2024-07-04T17:28:04.911 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.912 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.912 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:04.912 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_mirrored_filesystems": [ 2024-07-04T17:28:04.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:04.912 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:28:04.912 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem": "cephfs" 2024-07-04T17:28:04.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:04.912 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:04.912 INFO:teuthology.orchestra.run.smithi042.stdout: "mirroring_peers": 0, 2024-07-04T17:28:04.912 INFO:teuthology.orchestra.run.smithi042.stdout: "directory_count": 1 2024-07-04T17:28:04.912 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.912 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.912 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:04.912 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": [ 2024-07-04T17:28:04.912 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:04.913 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:04.913 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:04.913 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-07-04T17:28:04.913 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-07-04T17:28:04.913 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T17:28:04.913 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-07-04T17:28:04.913 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T17:28:04.913 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-07-04T17:28:04.913 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T17:28:04.913 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T17:28:04.913 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T17:28:04.913 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T17:28:04.913 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T17:28:04.914 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-07-04T17:28:04.914 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T17:28:04.914 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T17:28:04.914 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-07-04T17:28:04.914 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-07-04T17:28:04.914 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-07-04T17:28:04.914 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-07-04T17:28:04.914 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T17:28:04.914 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T17:28:04.914 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T17:28:04.914 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T17:28:04.914 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T17:28:04.914 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-07-04T17:28:04.914 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-07-04T17:28:04.915 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-07-04T17:28:04.915 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T17:28:04.915 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T17:28:04.915 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-07-04T17:28:04.915 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-07-04T17:28:04.915 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-07-04T17:28:04.915 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-07-04T17:28:04.915 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T17:28:04.915 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-07-04T17:28:04.915 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T17:28:04.915 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-07-04T17:28:04.915 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 182671, 2024-07-04T17:28:04.915 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 53, 2024-07-04T17:28:04.915 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-07-04T17:28:04.916 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-07-04T17:28:04.916 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-07-04T17:28:04.916 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-07-04T17:28:04.916 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-07-04T17:28:04.916 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-07-04T17:28:04.916 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-07-04T17:28:04.916 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-07-04T17:28:04.916 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 18208, 2024-07-04T17:28:04.916 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 161, 2024-07-04T17:28:04.916 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T17:28:04.916 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-07-04T17:28:04.916 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-07-04T17:28:04.916 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-07-04T17:28:04.916 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 0, 2024-07-04T17:28:04.916 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 0, 2024-07-04T17:28:04.917 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-07-04T17:28:04.917 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-07-04T17:28:04.917 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-07-04T17:28:04.917 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-07-04T17:28:04.917 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.917 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.917 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:04.917 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter": [ 2024-07-04T17:28:04.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:04.917 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:04.917 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:04.917 INFO:teuthology.orchestra.run.smithi042.stdout: "op_active": 0, 2024-07-04T17:28:04.917 INFO:teuthology.orchestra.run.smithi042.stdout: "op_laggy": 0, 2024-07-04T17:28:04.917 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send": 0, 2024-07-04T17:28:04.918 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send_bytes": 0, 2024-07-04T17:28:04.918 INFO:teuthology.orchestra.run.smithi042.stdout: "op_resend": 0, 2024-07-04T17:28:04.918 INFO:teuthology.orchestra.run.smithi042.stdout: "op_reply": 0, 2024-07-04T17:28:04.918 INFO:teuthology.orchestra.run.smithi042.stdout: "op_latency": { 2024-07-04T17:28:04.918 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:04.918 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:04.918 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:04.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:04.918 INFO:teuthology.orchestra.run.smithi042.stdout: "op_inflight": 0, 2024-07-04T17:28:04.918 INFO:teuthology.orchestra.run.smithi042.stdout: "oplen_avg": { 2024-07-04T17:28:04.918 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:04.918 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0 2024-07-04T17:28:04.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:04.919 INFO:teuthology.orchestra.run.smithi042.stdout: "op": 0, 2024-07-04T17:28:04.919 INFO:teuthology.orchestra.run.smithi042.stdout: "op_r": 0, 2024-07-04T17:28:04.919 INFO:teuthology.orchestra.run.smithi042.stdout: "op_w": 0, 2024-07-04T17:28:04.919 INFO:teuthology.orchestra.run.smithi042.stdout: "op_rmw": 0, 2024-07-04T17:28:04.919 INFO:teuthology.orchestra.run.smithi042.stdout: "op_pg": 0, 2024-07-04T17:28:04.919 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_stat": 0, 2024-07-04T17:28:04.919 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_create": 0, 2024-07-04T17:28:04.919 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_read": 0, 2024-07-04T17:28:04.919 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_write": 0, 2024-07-04T17:28:04.919 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writefull": 0, 2024-07-04T17:28:04.919 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writesame": 0, 2024-07-04T17:28:04.919 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_append": 0, 2024-07-04T17:28:04.919 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_zero": 0, 2024-07-04T17:28:04.919 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_truncate": 0, 2024-07-04T17:28:04.919 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_delete": 0, 2024-07-04T17:28:04.919 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_mapext": 0, 2024-07-04T17:28:04.920 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_sparse_read": 0, 2024-07-04T17:28:04.920 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_clonerange": 0, 2024-07-04T17:28:04.920 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_getxattr": 0, 2024-07-04T17:28:04.920 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_setxattr": 0, 2024-07-04T17:28:04.920 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_cmpxattr": 0, 2024-07-04T17:28:04.920 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_rmxattr": 0, 2024-07-04T17:28:04.920 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_resetxattrs": 0, 2024-07-04T17:28:04.920 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_call": 0, 2024-07-04T17:28:04.920 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_watch": 0, 2024-07-04T17:28:04.920 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_notify": 0, 2024-07-04T17:28:04.920 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T17:28:04.920 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls": 0, 2024-07-04T17:28:04.920 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls_filter": 0, 2024-07-04T17:28:04.920 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_other": 0, 2024-07-04T17:28:04.920 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_active": 0, 2024-07-04T17:28:04.921 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_send": 0, 2024-07-04T17:28:04.921 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_resend": 0, 2024-07-04T17:28:04.921 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_ping": 0, 2024-07-04T17:28:04.921 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_active": 0, 2024-07-04T17:28:04.921 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_send": 0, 2024-07-04T17:28:04.921 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_resend": 0, 2024-07-04T17:28:04.921 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_active": 0, 2024-07-04T17:28:04.921 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_send": 0, 2024-07-04T17:28:04.921 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_resend": 0, 2024-07-04T17:28:04.921 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_active": 0, 2024-07-04T17:28:04.921 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_send": 0, 2024-07-04T17:28:04.921 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_resend": 0, 2024-07-04T17:28:04.921 INFO:teuthology.orchestra.run.smithi042.stdout: "command_active": 0, 2024-07-04T17:28:04.921 INFO:teuthology.orchestra.run.smithi042.stdout: "command_send": 0, 2024-07-04T17:28:04.921 INFO:teuthology.orchestra.run.smithi042.stdout: "command_resend": 0, 2024-07-04T17:28:04.922 INFO:teuthology.orchestra.run.smithi042.stdout: "map_epoch": 0, 2024-07-04T17:28:04.922 INFO:teuthology.orchestra.run.smithi042.stdout: "map_full": 0, 2024-07-04T17:28:04.922 INFO:teuthology.orchestra.run.smithi042.stdout: "map_inc": 0, 2024-07-04T17:28:04.922 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_sessions": 0, 2024-07-04T17:28:04.922 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_open": 0, 2024-07-04T17:28:04.922 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_close": 0, 2024-07-04T17:28:04.922 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_laggy": 0, 2024-07-04T17:28:04.922 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_wr": 0, 2024-07-04T17:28:04.922 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_rd": 0, 2024-07-04T17:28:04.922 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_del": 0 2024-07-04T17:28:04.922 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.922 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.923 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:04.923 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-client": [ 2024-07-04T17:28:04.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:04.923 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:04.923 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:04.923 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:28:04.923 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:28:04.923 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:28:04.923 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 20, 2024-07-04T17:28:04.923 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 71695, 2024-07-04T17:28:04.923 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:28:04.923 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 20, 2024-07-04T17:28:04.923 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:28:04.923 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:28:04.924 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 20, 2024-07-04T17:28:04.924 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 71695, 2024-07-04T17:28:04.924 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:28:04.924 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:04.924 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:04.924 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:04.924 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.924 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.924 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.924 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:04.924 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-radosclient": [ 2024-07-04T17:28:04.924 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:04.924 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:04.924 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:04.925 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:28:04.925 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:28:04.925 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:28:04.925 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 12, 2024-07-04T17:28:04.925 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 300554, 2024-07-04T17:28:04.925 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:28:04.925 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 12, 2024-07-04T17:28:04.925 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:28:04.925 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:28:04.925 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 12, 2024-07-04T17:28:04.925 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 300554, 2024-07-04T17:28:04.925 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:28:04.925 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:04.925 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:04.925 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:04.925 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.926 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.926 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.926 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:04.926 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_bytes": [ 2024-07-04T17:28:04.926 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:04.926 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:04.926 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:04.926 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:28:04.926 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:28:04.926 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:28:04.926 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:28:04.926 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:28:04.926 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:28:04.927 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:28:04.927 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:28:04.927 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:28:04.927 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:28:04.927 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:28:04.927 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:28:04.927 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:04.927 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:04.927 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:04.927 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.927 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.927 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.927 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:04.927 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_ops": [ 2024-07-04T17:28:04.927 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:04.928 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:04.928 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:04.928 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:28:04.928 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1024, 2024-07-04T17:28:04.928 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:28:04.928 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:28:04.928 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:28:04.928 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:28:04.928 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:28:04.928 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:28:04.928 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:28:04.928 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:28:04.928 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:28:04.928 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:28:04.928 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:04.929 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:04.929 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:04.929 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.929 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.929 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:04.929 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-07-04T17:28:04.929 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:28:04.930 DEBUG:tasks.cephfs.test_mirroring:command returned={ "AsyncMessenger::Worker": [ { "labels": { "id": "0" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "1" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "2" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } } ], "AsyncMessenger::Worker-0": [ { "labels": {}, "counters": { "msgr_recv_messages": 20, "msgr_send_messages": 3, "msgr_recv_bytes": 74416, "msgr_send_bytes": 464, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.008159046, "msgr_running_send_time": 0.000843327, "msgr_running_recv_time": 0.005383483, "msgr_running_fast_dispatch_time": 0.001671330, "msgr_send_messages_queue_lat": { "avgcount": 3, "sum": 0.004312719, "avgtime": 0.001437573 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 74416, "msgr_send_encrypted_bytes": 464 } } ], "AsyncMessenger::Worker-1": [ { "labels": {}, "counters": { "msgr_recv_messages": 2, "msgr_send_messages": 21, "msgr_recv_bytes": 288, "msgr_send_bytes": 166960, "msgr_created_connections": 5, "msgr_active_connections": 1, "msgr_running_total_time": 0.009966010, "msgr_running_send_time": 0.004031544, "msgr_running_recv_time": 0.003527254, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 20, "sum": 0.016584089, "avgtime": 0.000829204 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 288, "msgr_send_encrypted_bytes": 166960 } } ], "AsyncMessenger::Worker-2": [ { "labels": {}, "counters": { "msgr_recv_messages": 13, "msgr_send_messages": 6, "msgr_recv_bytes": 302800, "msgr_send_bytes": 928, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.010757335, "msgr_running_send_time": 0.000862072, "msgr_running_recv_time": 0.017202050, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 6, "sum": 0.003172173, "avgtime": 0.000528695 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 302800, "msgr_send_encrypted_bytes": 928 } } ], "cct": [ { "labels": {}, "counters": { "total_workers": 0, "unhealthy_workers": 0 } } ], "cephfs_mirror": [ { "labels": {}, "counters": { "mirrored_filesystems": 1, "mirror_enable_failures": 0 } } ], "cephfs_mirror_mirrored_filesystems": [ { "labels": { "filesystem": "cephfs" }, "counters": { "mirroring_peers": 0, "directory_count": 1 } } ], "mempool": [ { "labels": {}, "counters": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_items": 0, "bluefs_bytes": 0, "bluefs_items": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "buffer_anon_bytes": 182671, "buffer_anon_items": 53, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 18208, "osdmap_items": 161, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "mds_co_bytes": 0, "mds_co_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 } } ], "objecter": [ { "labels": {}, "counters": { "op_active": 0, "op_laggy": 0, "op_send": 0, "op_send_bytes": 0, "op_resend": 0, "op_reply": 0, "op_latency": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "op_inflight": 0, "oplen_avg": { "avgcount": 0, "sum": 0 }, "op": 0, "op_r": 0, "op_w": 0, "op_rmw": 0, "op_pg": 0, "osdop_stat": 0, "osdop_create": 0, "osdop_read": 0, "osdop_write": 0, "osdop_writefull": 0, "osdop_writesame": 0, "osdop_append": 0, "osdop_zero": 0, "osdop_truncate": 0, "osdop_delete": 0, "osdop_mapext": 0, "osdop_sparse_read": 0, "osdop_clonerange": 0, "osdop_getxattr": 0, "osdop_setxattr": 0, "osdop_cmpxattr": 0, "osdop_rmxattr": 0, "osdop_resetxattrs": 0, "osdop_call": 0, "osdop_watch": 0, "osdop_notify": 0, "osdop_src_cmpxattr": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_other": 0, "linger_active": 0, "linger_send": 0, "linger_resend": 0, "linger_ping": 0, "poolop_active": 0, "poolop_send": 0, "poolop_resend": 0, "poolstat_active": 0, "poolstat_send": 0, "poolstat_resend": 0, "statfs_active": 0, "statfs_send": 0, "statfs_resend": 0, "command_active": 0, "command_send": 0, "command_resend": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "osd_sessions": 0, "osd_session_open": 0, "osd_session_close": 0, "osd_laggy": 0, "omap_wr": 0, "omap_rd": 0, "omap_del": 0 } } ], "throttle-msgr_dispatch_throttler-client": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 20, "get_sum": 71695, "get_or_fail_fail": 0, "get_or_fail_success": 20, "take": 0, "take_sum": 0, "put": 20, "put_sum": 71695, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-msgr_dispatch_throttler-radosclient": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 12, "get_sum": 300554, "get_or_fail_fail": 0, "get_or_fail_success": 12, "take": 0, "take_sum": 0, "put": 12, "put_sum": 300554, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_bytes": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_ops": [ { "labels": {}, "counters": { "val": 0, "max": 1024, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ] } 2024-07-04T17:28:04.930 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:28:04.930 DEBUG:teuthology.orchestra.run.smithi042:mirror status for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@2 2024-07-04T17:28:05.174 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:28:05.175 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_inst": "172.21.15.42:0/487218478", 2024-07-04T17:28:05.175 INFO:teuthology.orchestra.run.smithi042.stdout: "peers": {}, 2024-07-04T17:28:05.175 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_dirs": { 2024-07-04T17:28:05.175 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_count": 1 2024-07-04T17:28:05.175 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:05.175 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:28:05.193 DEBUG:tasks.cephfs.test_mirroring:command returned={ "rados_inst": "172.21.15.42:0/487218478", "peers": {}, "snap_dirs": { "dir_count": 1 } } 2024-07-04T17:28:05.193 DEBUG:tasks.cephfs.test_mirroring:initial dir_count=1 2024-07-04T17:28:05.193 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snapshot mirror add cephfs / 2024-07-04T17:28:05.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:05.483+0000 7fb376400640 1 -- 172.21.15.42:0/3573897505 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb370073470 msgr2=0x7fb3700a9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:05.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:05.483+0000 7fb376400640 1 --2- 172.21.15.42:0/3573897505 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb370073470 0x7fb3700a9990 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb35c0097b0 tx=0x7fb35c02a540 comp rx=0 tx=0).stop 2024-07-04T17:28:05.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:05.483+0000 7fb376400640 1 -- 172.21.15.42:0/3573897505 shutdown_connections 2024-07-04T17:28:05.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:05.483+0000 7fb376400640 1 --2- 172.21.15.42:0/3573897505 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb370073470 0x7fb3700a9990 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:05.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:05.483+0000 7fb376400640 1 -- 172.21.15.42:0/3573897505 >> 172.21.15.42:0/3573897505 conn(0x7fb37006c020 msgr2=0x7fb37006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:05.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:05.483+0000 7fb376400640 1 -- 172.21.15.42:0/3573897505 shutdown_connections 2024-07-04T17:28:05.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:05.483+0000 7fb376400640 1 -- 172.21.15.42:0/3573897505 wait complete. 2024-07-04T17:28:05.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:05.485+0000 7fb376400640 1 Processor -- start 2024-07-04T17:28:05.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:05.485+0000 7fb376400640 1 -- start start 2024-07-04T17:28:05.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:05.487+0000 7fb376400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb370073470 0x7fb3700ac2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:05.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:05.487+0000 7fb376400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb37013f430 con 0x7fb370073470 2024-07-04T17:28:05.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:05.487+0000 7fb36f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb370073470 0x7fb3700ac2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:05.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:05.487+0000 7fb36f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb370073470 0x7fb3700ac2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58136/0 (socket says 172.21.15.42:58136) 2024-07-04T17:28:05.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:05.487+0000 7fb36f400640 1 -- 172.21.15.42:0/1434229639 learned_addr learned my addr 172.21.15.42:0/1434229639 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:05.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:05.487+0000 7fb36f400640 1 -- 172.21.15.42:0/1434229639 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3700ac910 con 0x7fb370073470 2024-07-04T17:28:05.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:05.488+0000 7fb36f400640 1 --2- 172.21.15.42:0/1434229639 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb370073470 0x7fb3700ac2f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb36000cb40 tx=0x7fb36000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:05.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:05.488+0000 7fb367400640 1 -- 172.21.15.42:0/1434229639 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb360016020 con 0x7fb370073470 2024-07-04T17:28:05.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:05.488+0000 7fb367400640 1 -- 172.21.15.42:0/1434229639 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb36000ac30 con 0x7fb370073470 2024-07-04T17:28:05.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:05.488+0000 7fb376400640 1 -- 172.21.15.42:0/1434229639 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3701e1f30 con 0x7fb370073470 2024-07-04T17:28:05.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:05.488+0000 7fb367400640 1 -- 172.21.15.42:0/1434229639 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb360012420 con 0x7fb370073470 2024-07-04T17:28:05.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:05.488+0000 7fb376400640 1 -- 172.21.15.42:0/1434229639 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3700aef80 con 0x7fb370073470 2024-07-04T17:28:05.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:05.490+0000 7fb367400640 1 -- 172.21.15.42:0/1434229639 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 10) ==== 49681+0+0 (secure 0 0 0) 0x7fb360019020 con 0x7fb370073470 2024-07-04T17:28:05.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:05.490+0000 7fb376400640 1 -- 172.21.15.42:0/1434229639 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb37013fb00 con 0x7fb370073470 2024-07-04T17:28:05.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:05.490+0000 7fb367400640 1 --2- 172.21.15.42:0/1434229639 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7fb35003d580 0x7fb35003fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:05.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:05.490+0000 7fb367400640 1 -- 172.21.15.42:0/1434229639 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4561+0+0 (secure 0 0 0) 0x7fb3600521a0 con 0x7fb370073470 2024-07-04T17:28:05.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:05.491+0000 7fb36ea00640 1 --2- 172.21.15.42:0/1434229639 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7fb35003d580 0x7fb35003fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:05.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:05.492+0000 7fb36ea00640 1 --2- 172.21.15.42:0/1434229639 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7fb35003d580 0x7fb35003fa90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb35c034000 tx=0x7fb35c004730 comp rx=0 tx=0).ready entity=mgr.4563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:05.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:05.496+0000 7fb367400640 1 -- 172.21.15.42:0/1434229639 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fb360020030 con 0x7fb370073470 2024-07-04T17:28:05.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:05.729+0000 7fb376400640 1 -- 172.21.15.42:0/1434229639 --> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] -- mgr_command(tid 0: {"prefix": "fs snapshot mirror add", "fs_name": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fb370075aa0 con 0x7fb35003d580 2024-07-04T17:28:05.731 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:28:05.730+0000 7fcd83c00640 -1 mgr.server reply reply (22) Invalid argument / is a ancestor of tracked path /d1/d2 2024-07-04T17:28:05.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:05.731+0000 7fb367400640 1 -- 172.21.15.42:0/1434229639 <== mgr.4563 v2:172.21.15.42:6824/488387844 1 ==== mgr_command_reply(tid 0: -22 / is a ancestor of tracked path /d1/d2) ==== 46+0+0 (secure 0 0 0) 0x7fb370075aa0 con 0x7fb35003d580 2024-07-04T17:28:05.731 INFO:teuthology.orchestra.run.smithi042.stderr:Error EINVAL: / is a ancestor of tracked path /d1/d2 2024-07-04T17:28:05.758 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-07-04T17:28:05.759 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:28:05.759 DEBUG:teuthology.orchestra.run.smithi042:mirror status for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@2 2024-07-04T17:28:05.998 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:28:05.998 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_inst": "172.21.15.42:0/487218478", 2024-07-04T17:28:05.998 INFO:teuthology.orchestra.run.smithi042.stdout: "peers": {}, 2024-07-04T17:28:05.998 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_dirs": { 2024-07-04T17:28:05.998 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_count": 1 2024-07-04T17:28:05.998 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:05.998 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:28:06.015 DEBUG:tasks.cephfs.test_mirroring:command returned={ "rados_inst": "172.21.15.42:0/487218478", "peers": {}, "snap_dirs": { "dir_count": 1 } } 2024-07-04T17:28:06.015 DEBUG:tasks.cephfs.test_mirroring:initial dir_count=1 2024-07-04T17:28:06.015 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snapshot mirror add cephfs /d1 2024-07-04T17:28:06.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:06.305+0000 7fb337e00640 1 -- 172.21.15.42:0/1227851002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3380abe40 msgr2=0x7fb3380aa900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:06.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:06.305+0000 7fb337e00640 1 --2- 172.21.15.42:0/1227851002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3380abe40 0x7fb3380aa900 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb3200097b0 tx=0x7fb32002a530 comp rx=0 tx=0).stop 2024-07-04T17:28:06.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:06.305+0000 7fb337e00640 1 -- 172.21.15.42:0/1227851002 shutdown_connections 2024-07-04T17:28:06.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:06.305+0000 7fb337e00640 1 --2- 172.21.15.42:0/1227851002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3380abe40 0x7fb3380aa900 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:06.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:06.305+0000 7fb337e00640 1 -- 172.21.15.42:0/1227851002 >> 172.21.15.42:0/1227851002 conn(0x7fb3380afaa0 msgr2=0x7fb3380afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:06.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:06.306+0000 7fb337e00640 1 -- 172.21.15.42:0/1227851002 shutdown_connections 2024-07-04T17:28:06.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:06.306+0000 7fb337e00640 1 -- 172.21.15.42:0/1227851002 wait complete. 2024-07-04T17:28:06.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:06.308+0000 7fb337e00640 1 Processor -- start 2024-07-04T17:28:06.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:06.308+0000 7fb337e00640 1 -- start start 2024-07-04T17:28:06.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:06.309+0000 7fb337e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3380701d0 0x7fb33814bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:06.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:06.309+0000 7fb337e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb338073ed0 con 0x7fb3380701d0 2024-07-04T17:28:06.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:06.309+0000 7fb336a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3380701d0 0x7fb33814bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:06.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:06.309+0000 7fb336a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3380701d0 0x7fb33814bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58158/0 (socket says 172.21.15.42:58158) 2024-07-04T17:28:06.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:06.309+0000 7fb336a00640 1 -- 172.21.15.42:0/896899058 learned_addr learned my addr 172.21.15.42:0/896899058 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:06.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:06.310+0000 7fb336a00640 1 -- 172.21.15.42:0/896899058 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb338072dc0 con 0x7fb3380701d0 2024-07-04T17:28:06.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:06.310+0000 7fb336a00640 1 --2- 172.21.15.42:0/896899058 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3380701d0 0x7fb33814bae0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb32800cb30 tx=0x7fb3280021c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:06.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:06.310+0000 7fb32f400640 1 -- 172.21.15.42:0/896899058 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb328016020 con 0x7fb3380701d0 2024-07-04T17:28:06.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:06.310+0000 7fb32f400640 1 -- 172.21.15.42:0/896899058 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb32800aba0 con 0x7fb3380701d0 2024-07-04T17:28:06.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:06.310+0000 7fb337e00640 1 -- 172.21.15.42:0/896899058 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb33813e140 con 0x7fb3380701d0 2024-07-04T17:28:06.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:06.310+0000 7fb32f400640 1 -- 172.21.15.42:0/896899058 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb328012420 con 0x7fb3380701d0 2024-07-04T17:28:06.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:06.311+0000 7fb337e00640 1 -- 172.21.15.42:0/896899058 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3381daef0 con 0x7fb3380701d0 2024-07-04T17:28:06.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:06.312+0000 7fb337e00640 1 -- 172.21.15.42:0/896899058 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb33813dc10 con 0x7fb3380701d0 2024-07-04T17:28:06.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:06.312+0000 7fb32f400640 1 -- 172.21.15.42:0/896899058 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 10) ==== 49681+0+0 (secure 0 0 0) 0x7fb328019020 con 0x7fb3380701d0 2024-07-04T17:28:06.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:06.313+0000 7fb32f400640 1 --2- 172.21.15.42:0/896899058 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7fb31403d580 0x7fb31403fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:06.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:06.313+0000 7fb32f400640 1 -- 172.21.15.42:0/896899058 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4561+0+0 (secure 0 0 0) 0x7fb3280525d0 con 0x7fb3380701d0 2024-07-04T17:28:06.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:06.313+0000 7fb32e000640 1 --2- 172.21.15.42:0/896899058 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7fb31403d580 0x7fb31403fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:06.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:06.314+0000 7fb32e000640 1 --2- 172.21.15.42:0/896899058 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7fb31403d580 0x7fb31403fa90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb320034000 tx=0x7fb320004730 comp rx=0 tx=0).ready entity=mgr.4563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:06.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:06.318+0000 7fb32f400640 1 -- 172.21.15.42:0/896899058 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fb328020020 con 0x7fb3380701d0 2024-07-04T17:28:06.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:06.557+0000 7fb337e00640 1 -- 172.21.15.42:0/896899058 --> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] -- mgr_command(tid 0: {"prefix": "fs snapshot mirror add", "fs_name": "cephfs", "path": "/d1", "target": ["mon-mgr", ""]}) -- 0x7fb33814b910 con 0x7fb31403d580 2024-07-04T17:28:06.559 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:28:06.558+0000 7fcd83c00640 -1 mgr.server reply reply (22) Invalid argument /d1 is a ancestor of tracked path /d1/d2 2024-07-04T17:28:06.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:06.559+0000 7fb32f400640 1 -- 172.21.15.42:0/896899058 <== mgr.4563 v2:172.21.15.42:6824/488387844 1 ==== mgr_command_reply(tid 0: -22 /d1 is a ancestor of tracked path /d1/d2) ==== 48+0+0 (secure 0 0 0) 0x7fb33814b910 con 0x7fb31403d580 2024-07-04T17:28:06.559 INFO:teuthology.orchestra.run.smithi042.stderr:Error EINVAL: /d1 is a ancestor of tracked path /d1/d2 2024-07-04T17:28:06.586 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-07-04T17:28:06.586 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:28:06.587 DEBUG:teuthology.orchestra.run.smithi042:mirror status for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@2 2024-07-04T17:28:06.826 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:28:06.826 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_inst": "172.21.15.42:0/487218478", 2024-07-04T17:28:06.826 INFO:teuthology.orchestra.run.smithi042.stdout: "peers": {}, 2024-07-04T17:28:06.826 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_dirs": { 2024-07-04T17:28:06.826 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_count": 1 2024-07-04T17:28:06.827 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:06.827 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:28:06.842 DEBUG:tasks.cephfs.test_mirroring:command returned={ "rados_inst": "172.21.15.42:0/487218478", "peers": {}, "snap_dirs": { "dir_count": 1 } } 2024-07-04T17:28:06.843 DEBUG:tasks.cephfs.test_mirroring:initial dir_count=1 2024-07-04T17:28:06.843 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snapshot mirror add cephfs /d1/d2/d3 2024-07-04T17:28:07.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:07.149+0000 7f6ce3200640 1 -- 172.21.15.42:0/720998835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ce40ac5b0 msgr2=0x7f6ce40b0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:07.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:07.149+0000 7f6ce3200640 1 --2- 172.21.15.42:0/720998835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ce40ac5b0 0x7f6ce40b0db0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6ce40ac560 tx=0x7f6cd402ad10 comp rx=0 tx=0).stop 2024-07-04T17:28:07.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:07.149+0000 7f6ce3200640 1 -- 172.21.15.42:0/720998835 shutdown_connections 2024-07-04T17:28:07.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:07.149+0000 7f6ce3200640 1 --2- 172.21.15.42:0/720998835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ce40ac5b0 0x7f6ce40b0db0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:07.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:07.149+0000 7f6ce3200640 1 -- 172.21.15.42:0/720998835 >> 172.21.15.42:0/720998835 conn(0x7f6ce40afaa0 msgr2=0x7f6ce40afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:07.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:07.149+0000 7f6ce3200640 1 -- 172.21.15.42:0/720998835 shutdown_connections 2024-07-04T17:28:07.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:07.149+0000 7f6ce3200640 1 -- 172.21.15.42:0/720998835 wait complete. 2024-07-04T17:28:07.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:07.151+0000 7f6ce3200640 1 Processor -- start 2024-07-04T17:28:07.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:07.152+0000 7f6ce3200640 1 -- start start 2024-07-04T17:28:07.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:07.153+0000 7f6ce3200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ce40ac5b0 0x7f6ce413a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:07.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:07.153+0000 7f6ce3200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce40ae590 con 0x7f6ce40ac5b0 2024-07-04T17:28:07.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:07.153+0000 7f6ce1e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ce40ac5b0 0x7f6ce413a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:07.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:07.153+0000 7f6ce1e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ce40ac5b0 0x7f6ce413a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58172/0 (socket says 172.21.15.42:58172) 2024-07-04T17:28:07.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:07.153+0000 7f6ce1e00640 1 -- 172.21.15.42:0/572977163 learned_addr learned my addr 172.21.15.42:0/572977163 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:07.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:07.153+0000 7f6ce1e00640 1 -- 172.21.15.42:0/572977163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce413b000 con 0x7f6ce40ac5b0 2024-07-04T17:28:07.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:07.154+0000 7f6ce1e00640 1 --2- 172.21.15.42:0/572977163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ce40ac5b0 0x7f6ce413a960 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6cd400b280 tx=0x7f6cd4009460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:07.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:07.154+0000 7f6cda000640 1 -- 172.21.15.42:0/572977163 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cd4038040 con 0x7f6ce40ac5b0 2024-07-04T17:28:07.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:07.154+0000 7f6cda000640 1 -- 172.21.15.42:0/572977163 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6cd4041070 con 0x7f6ce40ac5b0 2024-07-04T17:28:07.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:07.154+0000 7f6cda000640 1 -- 172.21.15.42:0/572977163 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cd4009840 con 0x7f6ce40ac5b0 2024-07-04T17:28:07.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:07.154+0000 7f6ce3200640 1 -- 172.21.15.42:0/572977163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ce40b2660 con 0x7f6ce40ac5b0 2024-07-04T17:28:07.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:07.154+0000 7f6ce3200640 1 -- 172.21.15.42:0/572977163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ce41dd750 con 0x7f6ce40ac5b0 2024-07-04T17:28:07.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:07.156+0000 7f6cda000640 1 -- 172.21.15.42:0/572977163 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 10) ==== 49681+0+0 (secure 0 0 0) 0x7f6cd40035f0 con 0x7f6ce40ac5b0 2024-07-04T17:28:07.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:07.155+0000 7f6ce3200640 1 -- 172.21.15.42:0/572977163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ce40b0db0 con 0x7f6ce40ac5b0 2024-07-04T17:28:07.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:07.156+0000 7f6cda000640 1 --2- 172.21.15.42:0/572977163 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f6cc003d580 0x7f6cc003fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:07.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:07.156+0000 7f6cda000640 1 -- 172.21.15.42:0/572977163 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4561+0+0 (secure 0 0 0) 0x7f6cd4072480 con 0x7f6ce40ac5b0 2024-07-04T17:28:07.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:07.156+0000 7f6ce1400640 1 --2- 172.21.15.42:0/572977163 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f6cc003d580 0x7f6cc003fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:07.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:07.157+0000 7f6ce1400640 1 --2- 172.21.15.42:0/572977163 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f6cc003d580 0x7f6cc003fa90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc0015c0 tx=0x7f6ccc007a90 comp rx=0 tx=0).ready entity=mgr.4563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:07.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:07.162+0000 7f6cda000640 1 -- 172.21.15.42:0/572977163 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f6cd40038e0 con 0x7f6ce40ac5b0 2024-07-04T17:28:07.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:07.395+0000 7f6ce3200640 1 -- 172.21.15.42:0/572977163 --> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] -- mgr_command(tid 0: {"prefix": "fs snapshot mirror add", "fs_name": "cephfs", "path": "/d1/d2/d3", "target": ["mon-mgr", ""]}) -- 0x7f6ce413ba00 con 0x7f6cc003d580 2024-07-04T17:28:07.397 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:28:07.397+0000 7fcd83c00640 -1 mgr.server reply reply (22) Invalid argument /d1/d2/d3 is a subtree of tracked path /d1/d2 2024-07-04T17:28:07.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:07.397+0000 7f6cda000640 1 -- 172.21.15.42:0/572977163 <== mgr.4563 v2:172.21.15.42:6824/488387844 1 ==== mgr_command_reply(tid 0: -22 /d1/d2/d3 is a subtree of tracked path /d1/d2) ==== 53+0+0 (secure 0 0 0) 0x7f6ce413ba00 con 0x7f6cc003d580 2024-07-04T17:28:07.397 INFO:teuthology.orchestra.run.smithi042.stderr:Error EINVAL: /d1/d2/d3 is a subtree of tracked path /d1/d2 2024-07-04T17:28:07.430 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-07-04T17:28:07.431 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:28:07.431 DEBUG:teuthology.orchestra.run.smithi042:counter dump for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok counter dump 2024-07-04T17:28:07.672 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:28:07.672 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker": [ 2024-07-04T17:28:07.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:07.672 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:28:07.672 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "0" 2024-07-04T17:28:07.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:07.672 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:07.672 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:28:07.673 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:28:07.673 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:07.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:07.673 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:28:07.673 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "1" 2024-07-04T17:28:07.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:07.673 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:07.673 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:28:07.673 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:28:07.673 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:07.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:07.673 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:28:07.673 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "2" 2024-07-04T17:28:07.674 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:07.674 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:07.674 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:28:07.674 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:28:07.674 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.674 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.674 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:07.674 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": [ 2024-07-04T17:28:07.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:07.674 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:07.674 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:07.674 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 20, 2024-07-04T17:28:07.674 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 3, 2024-07-04T17:28:07.675 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 74416, 2024-07-04T17:28:07.675 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 464, 2024-07-04T17:28:07.675 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:28:07.675 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:28:07.675 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.008313014, 2024-07-04T17:28:07.675 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.000922423, 2024-07-04T17:28:07.675 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.005418089, 2024-07-04T17:28:07.675 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.001671330, 2024-07-04T17:28:07.675 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:28:07.675 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 3, 2024-07-04T17:28:07.675 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.004312719, 2024-07-04T17:28:07.675 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.001437573 2024-07-04T17:28:07.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:07.676 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:28:07.676 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:07.676 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:07.676 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:07.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:07.676 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 74416, 2024-07-04T17:28:07.676 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 464 2024-07-04T17:28:07.676 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.676 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.676 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:07.676 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": [ 2024-07-04T17:28:07.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:07.676 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:07.677 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:07.677 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 2, 2024-07-04T17:28:07.677 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 22, 2024-07-04T17:28:07.677 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 288, 2024-07-04T17:28:07.677 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 167200, 2024-07-04T17:28:07.677 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 5, 2024-07-04T17:28:07.677 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:28:07.677 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.010156982, 2024-07-04T17:28:07.677 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.004197010, 2024-07-04T17:28:07.677 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.003527254, 2024-07-04T17:28:07.677 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:28:07.677 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:28:07.677 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 21, 2024-07-04T17:28:07.677 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.016680479, 2024-07-04T17:28:07.677 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000794308 2024-07-04T17:28:07.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:07.678 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:28:07.678 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:07.678 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:07.678 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:07.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:07.678 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 288, 2024-07-04T17:28:07.678 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 167200 2024-07-04T17:28:07.678 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.678 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.678 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:07.678 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": [ 2024-07-04T17:28:07.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:07.678 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:07.679 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:07.679 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 13, 2024-07-04T17:28:07.679 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 6, 2024-07-04T17:28:07.679 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 302800, 2024-07-04T17:28:07.679 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 928, 2024-07-04T17:28:07.679 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:28:07.679 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:28:07.679 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.010953481, 2024-07-04T17:28:07.679 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.000968241, 2024-07-04T17:28:07.679 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.017241644, 2024-07-04T17:28:07.679 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:28:07.679 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:28:07.679 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 6, 2024-07-04T17:28:07.679 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.003172173, 2024-07-04T17:28:07.680 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000528695 2024-07-04T17:28:07.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:07.680 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:28:07.680 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:07.680 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:07.680 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:07.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:07.680 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 302800, 2024-07-04T17:28:07.680 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 928 2024-07-04T17:28:07.680 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.680 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.680 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:07.680 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": [ 2024-07-04T17:28:07.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:07.680 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:07.681 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:07.681 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-07-04T17:28:07.681 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-07-04T17:28:07.681 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.681 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.681 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:07.681 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror": [ 2024-07-04T17:28:07.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:07.681 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:07.681 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:07.681 INFO:teuthology.orchestra.run.smithi042.stdout: "mirrored_filesystems": 1, 2024-07-04T17:28:07.681 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_enable_failures": 0 2024-07-04T17:28:07.681 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.681 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.681 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:07.681 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_mirrored_filesystems": [ 2024-07-04T17:28:07.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:07.682 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:28:07.682 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem": "cephfs" 2024-07-04T17:28:07.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:07.682 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:07.682 INFO:teuthology.orchestra.run.smithi042.stdout: "mirroring_peers": 0, 2024-07-04T17:28:07.682 INFO:teuthology.orchestra.run.smithi042.stdout: "directory_count": 1 2024-07-04T17:28:07.682 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.682 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.682 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:07.682 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": [ 2024-07-04T17:28:07.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:07.682 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:07.682 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:07.683 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-07-04T17:28:07.683 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-07-04T17:28:07.683 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T17:28:07.683 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-07-04T17:28:07.683 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T17:28:07.683 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-07-04T17:28:07.683 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T17:28:07.683 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T17:28:07.683 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T17:28:07.683 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T17:28:07.683 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T17:28:07.683 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-07-04T17:28:07.683 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T17:28:07.684 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T17:28:07.684 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-07-04T17:28:07.684 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-07-04T17:28:07.684 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-07-04T17:28:07.684 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-07-04T17:28:07.684 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T17:28:07.684 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T17:28:07.684 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T17:28:07.684 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T17:28:07.684 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T17:28:07.684 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-07-04T17:28:07.684 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-07-04T17:28:07.684 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-07-04T17:28:07.684 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T17:28:07.684 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T17:28:07.684 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-07-04T17:28:07.685 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-07-04T17:28:07.685 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-07-04T17:28:07.685 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-07-04T17:28:07.685 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T17:28:07.685 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-07-04T17:28:07.685 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T17:28:07.685 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-07-04T17:28:07.685 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 182671, 2024-07-04T17:28:07.685 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 53, 2024-07-04T17:28:07.685 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-07-04T17:28:07.685 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-07-04T17:28:07.685 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-07-04T17:28:07.685 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-07-04T17:28:07.685 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-07-04T17:28:07.685 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-07-04T17:28:07.686 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-07-04T17:28:07.686 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-07-04T17:28:07.686 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 18208, 2024-07-04T17:28:07.686 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 161, 2024-07-04T17:28:07.686 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T17:28:07.686 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-07-04T17:28:07.686 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-07-04T17:28:07.686 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-07-04T17:28:07.686 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 0, 2024-07-04T17:28:07.686 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 0, 2024-07-04T17:28:07.686 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-07-04T17:28:07.686 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-07-04T17:28:07.686 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-07-04T17:28:07.686 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-07-04T17:28:07.687 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.687 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.687 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:07.687 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter": [ 2024-07-04T17:28:07.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:07.687 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:07.687 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:07.687 INFO:teuthology.orchestra.run.smithi042.stdout: "op_active": 0, 2024-07-04T17:28:07.687 INFO:teuthology.orchestra.run.smithi042.stdout: "op_laggy": 0, 2024-07-04T17:28:07.687 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send": 0, 2024-07-04T17:28:07.687 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send_bytes": 0, 2024-07-04T17:28:07.687 INFO:teuthology.orchestra.run.smithi042.stdout: "op_resend": 0, 2024-07-04T17:28:07.687 INFO:teuthology.orchestra.run.smithi042.stdout: "op_reply": 0, 2024-07-04T17:28:07.687 INFO:teuthology.orchestra.run.smithi042.stdout: "op_latency": { 2024-07-04T17:28:07.688 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:07.688 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:07.688 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:07.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:07.688 INFO:teuthology.orchestra.run.smithi042.stdout: "op_inflight": 0, 2024-07-04T17:28:07.688 INFO:teuthology.orchestra.run.smithi042.stdout: "oplen_avg": { 2024-07-04T17:28:07.688 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:07.688 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0 2024-07-04T17:28:07.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:07.688 INFO:teuthology.orchestra.run.smithi042.stdout: "op": 0, 2024-07-04T17:28:07.688 INFO:teuthology.orchestra.run.smithi042.stdout: "op_r": 0, 2024-07-04T17:28:07.688 INFO:teuthology.orchestra.run.smithi042.stdout: "op_w": 0, 2024-07-04T17:28:07.688 INFO:teuthology.orchestra.run.smithi042.stdout: "op_rmw": 0, 2024-07-04T17:28:07.688 INFO:teuthology.orchestra.run.smithi042.stdout: "op_pg": 0, 2024-07-04T17:28:07.688 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_stat": 0, 2024-07-04T17:28:07.689 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_create": 0, 2024-07-04T17:28:07.689 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_read": 0, 2024-07-04T17:28:07.689 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_write": 0, 2024-07-04T17:28:07.689 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writefull": 0, 2024-07-04T17:28:07.689 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writesame": 0, 2024-07-04T17:28:07.689 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_append": 0, 2024-07-04T17:28:07.689 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_zero": 0, 2024-07-04T17:28:07.689 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_truncate": 0, 2024-07-04T17:28:07.689 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_delete": 0, 2024-07-04T17:28:07.689 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_mapext": 0, 2024-07-04T17:28:07.689 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_sparse_read": 0, 2024-07-04T17:28:07.689 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_clonerange": 0, 2024-07-04T17:28:07.689 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_getxattr": 0, 2024-07-04T17:28:07.689 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_setxattr": 0, 2024-07-04T17:28:07.689 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_cmpxattr": 0, 2024-07-04T17:28:07.690 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_rmxattr": 0, 2024-07-04T17:28:07.690 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_resetxattrs": 0, 2024-07-04T17:28:07.690 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_call": 0, 2024-07-04T17:28:07.690 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_watch": 0, 2024-07-04T17:28:07.690 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_notify": 0, 2024-07-04T17:28:07.690 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T17:28:07.690 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls": 0, 2024-07-04T17:28:07.690 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls_filter": 0, 2024-07-04T17:28:07.690 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_other": 0, 2024-07-04T17:28:07.690 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_active": 0, 2024-07-04T17:28:07.690 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_send": 0, 2024-07-04T17:28:07.690 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_resend": 0, 2024-07-04T17:28:07.690 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_ping": 0, 2024-07-04T17:28:07.691 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_active": 0, 2024-07-04T17:28:07.691 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_send": 0, 2024-07-04T17:28:07.691 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_resend": 0, 2024-07-04T17:28:07.691 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_active": 0, 2024-07-04T17:28:07.691 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_send": 0, 2024-07-04T17:28:07.691 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_resend": 0, 2024-07-04T17:28:07.691 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_active": 0, 2024-07-04T17:28:07.691 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_send": 0, 2024-07-04T17:28:07.691 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_resend": 0, 2024-07-04T17:28:07.691 INFO:teuthology.orchestra.run.smithi042.stdout: "command_active": 0, 2024-07-04T17:28:07.691 INFO:teuthology.orchestra.run.smithi042.stdout: "command_send": 0, 2024-07-04T17:28:07.691 INFO:teuthology.orchestra.run.smithi042.stdout: "command_resend": 0, 2024-07-04T17:28:07.691 INFO:teuthology.orchestra.run.smithi042.stdout: "map_epoch": 0, 2024-07-04T17:28:07.691 INFO:teuthology.orchestra.run.smithi042.stdout: "map_full": 0, 2024-07-04T17:28:07.691 INFO:teuthology.orchestra.run.smithi042.stdout: "map_inc": 0, 2024-07-04T17:28:07.692 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_sessions": 0, 2024-07-04T17:28:07.692 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_open": 0, 2024-07-04T17:28:07.692 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_close": 0, 2024-07-04T17:28:07.692 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_laggy": 0, 2024-07-04T17:28:07.692 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_wr": 0, 2024-07-04T17:28:07.692 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_rd": 0, 2024-07-04T17:28:07.692 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_del": 0 2024-07-04T17:28:07.692 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.692 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.692 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:07.692 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-client": [ 2024-07-04T17:28:07.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:07.692 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:07.693 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:07.693 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:28:07.693 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:28:07.693 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:28:07.693 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 20, 2024-07-04T17:28:07.693 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 71695, 2024-07-04T17:28:07.693 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:28:07.693 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 20, 2024-07-04T17:28:07.693 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:28:07.693 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:28:07.693 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 20, 2024-07-04T17:28:07.693 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 71695, 2024-07-04T17:28:07.693 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:28:07.693 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:07.694 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:07.694 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:07.694 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.694 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.694 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.694 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:07.694 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-radosclient": [ 2024-07-04T17:28:07.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:07.694 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:07.694 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:07.694 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:28:07.694 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:28:07.694 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:28:07.694 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 12, 2024-07-04T17:28:07.695 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 300554, 2024-07-04T17:28:07.695 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:28:07.695 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 12, 2024-07-04T17:28:07.695 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:28:07.695 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:28:07.695 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 12, 2024-07-04T17:28:07.695 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 300554, 2024-07-04T17:28:07.695 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:28:07.695 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:07.695 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:07.695 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:07.695 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.695 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.695 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.696 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:07.696 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_bytes": [ 2024-07-04T17:28:07.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:07.696 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:07.696 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:07.696 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:28:07.696 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:28:07.696 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:28:07.696 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:28:07.696 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:28:07.696 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:28:07.696 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:28:07.696 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:28:07.696 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:28:07.696 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:28:07.697 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:28:07.697 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:28:07.697 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:07.697 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:07.697 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:07.697 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.697 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.697 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.697 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:07.697 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_ops": [ 2024-07-04T17:28:07.697 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:07.697 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:07.697 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:07.697 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:28:07.698 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1024, 2024-07-04T17:28:07.698 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:28:07.698 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:28:07.698 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:28:07.698 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:28:07.698 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:28:07.698 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:28:07.698 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:28:07.698 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:28:07.698 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:28:07.698 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:28:07.698 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:07.698 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:07.699 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:07.699 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.699 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.699 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.699 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-07-04T17:28:07.699 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:28:07.700 DEBUG:tasks.cephfs.test_mirroring:command returned={ "AsyncMessenger::Worker": [ { "labels": { "id": "0" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "1" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "2" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } } ], "AsyncMessenger::Worker-0": [ { "labels": {}, "counters": { "msgr_recv_messages": 20, "msgr_send_messages": 3, "msgr_recv_bytes": 74416, "msgr_send_bytes": 464, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.008313014, "msgr_running_send_time": 0.000922423, "msgr_running_recv_time": 0.005418089, "msgr_running_fast_dispatch_time": 0.001671330, "msgr_send_messages_queue_lat": { "avgcount": 3, "sum": 0.004312719, "avgtime": 0.001437573 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 74416, "msgr_send_encrypted_bytes": 464 } } ], "AsyncMessenger::Worker-1": [ { "labels": {}, "counters": { "msgr_recv_messages": 2, "msgr_send_messages": 22, "msgr_recv_bytes": 288, "msgr_send_bytes": 167200, "msgr_created_connections": 5, "msgr_active_connections": 1, "msgr_running_total_time": 0.010156982, "msgr_running_send_time": 0.004197010, "msgr_running_recv_time": 0.003527254, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 21, "sum": 0.016680479, "avgtime": 0.000794308 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 288, "msgr_send_encrypted_bytes": 167200 } } ], "AsyncMessenger::Worker-2": [ { "labels": {}, "counters": { "msgr_recv_messages": 13, "msgr_send_messages": 6, "msgr_recv_bytes": 302800, "msgr_send_bytes": 928, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.010953481, "msgr_running_send_time": 0.000968241, "msgr_running_recv_time": 0.017241644, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 6, "sum": 0.003172173, "avgtime": 0.000528695 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 302800, "msgr_send_encrypted_bytes": 928 } } ], "cct": [ { "labels": {}, "counters": { "total_workers": 0, "unhealthy_workers": 0 } } ], "cephfs_mirror": [ { "labels": {}, "counters": { "mirrored_filesystems": 1, "mirror_enable_failures": 0 } } ], "cephfs_mirror_mirrored_filesystems": [ { "labels": { "filesystem": "cephfs" }, "counters": { "mirroring_peers": 0, "directory_count": 1 } } ], "mempool": [ { "labels": {}, "counters": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_items": 0, "bluefs_bytes": 0, "bluefs_items": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "buffer_anon_bytes": 182671, "buffer_anon_items": 53, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 18208, "osdmap_items": 161, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "mds_co_bytes": 0, "mds_co_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 } } ], "objecter": [ { "labels": {}, "counters": { "op_active": 0, "op_laggy": 0, "op_send": 0, "op_send_bytes": 0, "op_resend": 0, "op_reply": 0, "op_latency": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "op_inflight": 0, "oplen_avg": { "avgcount": 0, "sum": 0 }, "op": 0, "op_r": 0, "op_w": 0, "op_rmw": 0, "op_pg": 0, "osdop_stat": 0, "osdop_create": 0, "osdop_read": 0, "osdop_write": 0, "osdop_writefull": 0, "osdop_writesame": 0, "osdop_append": 0, "osdop_zero": 0, "osdop_truncate": 0, "osdop_delete": 0, "osdop_mapext": 0, "osdop_sparse_read": 0, "osdop_clonerange": 0, "osdop_getxattr": 0, "osdop_setxattr": 0, "osdop_cmpxattr": 0, "osdop_rmxattr": 0, "osdop_resetxattrs": 0, "osdop_call": 0, "osdop_watch": 0, "osdop_notify": 0, "osdop_src_cmpxattr": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_other": 0, "linger_active": 0, "linger_send": 0, "linger_resend": 0, "linger_ping": 0, "poolop_active": 0, "poolop_send": 0, "poolop_resend": 0, "poolstat_active": 0, "poolstat_send": 0, "poolstat_resend": 0, "statfs_active": 0, "statfs_send": 0, "statfs_resend": 0, "command_active": 0, "command_send": 0, "command_resend": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "osd_sessions": 0, "osd_session_open": 0, "osd_session_close": 0, "osd_laggy": 0, "omap_wr": 0, "omap_rd": 0, "omap_del": 0 } } ], "throttle-msgr_dispatch_throttler-client": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 20, "get_sum": 71695, "get_or_fail_fail": 0, "get_or_fail_success": 20, "take": 0, "take_sum": 0, "put": 20, "put_sum": 71695, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-msgr_dispatch_throttler-radosclient": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 12, "get_sum": 300554, "get_or_fail_fail": 0, "get_or_fail_success": 12, "take": 0, "take_sum": 0, "put": 12, "put_sum": 300554, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_bytes": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_ops": [ { "labels": {}, "counters": { "val": 0, "max": 1024, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ] } 2024-07-04T17:28:07.700 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:28:07.700 DEBUG:teuthology.orchestra.run.smithi042:mirror status for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@2 2024-07-04T17:28:07.950 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:28:07.950 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_inst": "172.21.15.42:0/487218478", 2024-07-04T17:28:07.950 INFO:teuthology.orchestra.run.smithi042.stdout: "peers": {}, 2024-07-04T17:28:07.950 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_dirs": { 2024-07-04T17:28:07.950 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_count": 1 2024-07-04T17:28:07.950 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:07.950 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:28:07.966 DEBUG:tasks.cephfs.test_mirroring:command returned={ "rados_inst": "172.21.15.42:0/487218478", "peers": {}, "snap_dirs": { "dir_count": 1 } } 2024-07-04T17:28:07.967 DEBUG:tasks.cephfs.test_mirroring:initial dir_count=1 2024-07-04T17:28:07.967 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snapshot mirror add cephfs /d1/d4/ 2024-07-04T17:28:08.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.263+0000 7f34fa200640 1 -- 172.21.15.42:0/3184533969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34f40af8d0 msgr2=0x7f34f4068720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:08.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.263+0000 7f34fa200640 1 --2- 172.21.15.42:0/3184533969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34f40af8d0 0x7f34f4068720 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f34e4005fd0 tx=0x7f34e402a940 comp rx=0 tx=0).stop 2024-07-04T17:28:08.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.263+0000 7f34fa200640 1 -- 172.21.15.42:0/3184533969 shutdown_connections 2024-07-04T17:28:08.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.263+0000 7f34fa200640 1 --2- 172.21.15.42:0/3184533969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34f40af8d0 0x7f34f4068720 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:08.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.263+0000 7f34fa200640 1 -- 172.21.15.42:0/3184533969 >> 172.21.15.42:0/3184533969 conn(0x7f34f40b2930 msgr2=0x7f34f40b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:08.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.263+0000 7f34fa200640 1 -- 172.21.15.42:0/3184533969 shutdown_connections 2024-07-04T17:28:08.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.263+0000 7f34fa200640 1 -- 172.21.15.42:0/3184533969 wait complete. 2024-07-04T17:28:08.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.265+0000 7f34fa200640 1 Processor -- start 2024-07-04T17:28:08.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.266+0000 7f34fa200640 1 -- start start 2024-07-04T17:28:08.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.267+0000 7f34fa200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34f406c020 0x7f34f414c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:08.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.267+0000 7f34fa200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f34f406d090 con 0x7f34f406c020 2024-07-04T17:28:08.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.267+0000 7f34f8e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34f406c020 0x7f34f414c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:08.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.267+0000 7f34f8e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34f406c020 0x7f34f414c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58188/0 (socket says 172.21.15.42:58188) 2024-07-04T17:28:08.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.267+0000 7f34f8e00640 1 -- 172.21.15.42:0/329941587 learned_addr learned my addr 172.21.15.42:0/329941587 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:08.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.268+0000 7f34f8e00640 1 -- 172.21.15.42:0/329941587 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f41d77d0 con 0x7f34f406c020 2024-07-04T17:28:08.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.268+0000 7f34f8e00640 1 --2- 172.21.15.42:0/329941587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34f406c020 0x7f34f414c9e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f34e4006010 tx=0x7f34e4003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:08.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.268+0000 7f34f1600640 1 -- 172.21.15.42:0/329941587 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34e4030ce0 con 0x7f34f406c020 2024-07-04T17:28:08.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.268+0000 7f34f1600640 1 -- 172.21.15.42:0/329941587 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34e403e070 con 0x7f34f406c020 2024-07-04T17:28:08.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.268+0000 7f34f1600640 1 -- 172.21.15.42:0/329941587 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34e4032070 con 0x7f34f406c020 2024-07-04T17:28:08.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.268+0000 7f34fa200640 1 -- 172.21.15.42:0/329941587 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f414d0f0 con 0x7f34f406c020 2024-07-04T17:28:08.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.269+0000 7f34fa200640 1 -- 172.21.15.42:0/329941587 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f414d610 con 0x7f34f406c020 2024-07-04T17:28:08.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.270+0000 7f34fa200640 1 -- 172.21.15.42:0/329941587 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34f414d840 con 0x7f34f406c020 2024-07-04T17:28:08.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.270+0000 7f34f1600640 1 -- 172.21.15.42:0/329941587 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 10) ==== 49681+0+0 (secure 0 0 0) 0x7f34e4032210 con 0x7f34f406c020 2024-07-04T17:28:08.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.271+0000 7f34f1600640 1 --2- 172.21.15.42:0/329941587 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f34d003d5d0 0x7f34d003fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:08.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.271+0000 7f34f1600640 1 -- 172.21.15.42:0/329941587 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4561+0+0 (secure 0 0 0) 0x7f34e402d070 con 0x7f34f406c020 2024-07-04T17:28:08.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.271+0000 7f34ebe00640 1 --2- 172.21.15.42:0/329941587 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f34d003d5d0 0x7f34d003fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:08.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.272+0000 7f34ebe00640 1 --2- 172.21.15.42:0/329941587 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f34d003d5d0 0x7f34d003fae0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f34dc0015c0 tx=0x7f34dc007a90 comp rx=0 tx=0).ready entity=mgr.4563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:08.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.276+0000 7f34f1600640 1 -- 172.21.15.42:0/329941587 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f34e4004040 con 0x7f34f406c020 2024-07-04T17:28:08.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.507+0000 7f34fa200640 1 -- 172.21.15.42:0/329941587 --> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] -- mgr_command(tid 0: {"prefix": "fs snapshot mirror add", "fs_name": "cephfs", "path": "/d1/d4/", "target": ["mon-mgr", ""]}) -- 0x7f34f414c830 con 0x7f34d003d5d0 2024-07-04T17:28:08.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.514+0000 7f34f1600640 1 -- 172.21.15.42:0/329941587 <== mgr.4563 v2:172.21.15.42:6824/488387844 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f34f414c830 con 0x7f34d003d5d0 2024-07-04T17:28:08.515 INFO:teuthology.orchestra.run.smithi042.stdout:{} 2024-07-04T17:28:08.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.517+0000 7f34fa200640 1 -- 172.21.15.42:0/329941587 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f34d003d5d0 msgr2=0x7f34d003fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:08.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.517+0000 7f34fa200640 1 --2- 172.21.15.42:0/329941587 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f34d003d5d0 0x7f34d003fae0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f34dc0015c0 tx=0x7f34dc007a90 comp rx=0 tx=0).stop 2024-07-04T17:28:08.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.517+0000 7f34fa200640 1 -- 172.21.15.42:0/329941587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34f406c020 msgr2=0x7f34f414c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:08.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.517+0000 7f34fa200640 1 --2- 172.21.15.42:0/329941587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34f406c020 0x7f34f414c9e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f34e4006010 tx=0x7f34e4003d00 comp rx=0 tx=0).stop 2024-07-04T17:28:08.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.517+0000 7f34fa200640 1 -- 172.21.15.42:0/329941587 shutdown_connections 2024-07-04T17:28:08.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.517+0000 7f34fa200640 1 --2- 172.21.15.42:0/329941587 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f34d003d5d0 0x7f34d003fae0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:08.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.517+0000 7f34fa200640 1 --2- 172.21.15.42:0/329941587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34f406c020 0x7f34f414c9e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:08.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.517+0000 7f34fa200640 1 -- 172.21.15.42:0/329941587 >> 172.21.15.42:0/329941587 conn(0x7f34f40b2930 msgr2=0x7f34f40728a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:08.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.518+0000 7f34fa200640 1 -- 172.21.15.42:0/329941587 shutdown_connections 2024-07-04T17:28:08.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:08.518+0000 7f34fa200640 1 -- 172.21.15.42:0/329941587 wait complete. 2024-07-04T17:28:18.544 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:28:18.545 DEBUG:teuthology.orchestra.run.smithi042:mirror status for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@2 2024-07-04T17:28:18.796 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:28:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_inst": "172.21.15.42:0/487218478", 2024-07-04T17:28:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: "peers": {}, 2024-07-04T17:28:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_dirs": { 2024-07-04T17:28:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_count": 2 2024-07-04T17:28:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:18.796 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:28:18.815 DEBUG:tasks.cephfs.test_mirroring:command returned={ "rados_inst": "172.21.15.42:0/487218478", "peers": {}, "snap_dirs": { "dir_count": 2 } } 2024-07-04T17:28:18.815 DEBUG:tasks.cephfs.test_mirroring:new dir_count=2 2024-07-04T17:28:18.815 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:28:18.815 DEBUG:teuthology.orchestra.run.smithi042:counter dump for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok counter dump 2024-07-04T17:28:19.046 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:28:19.046 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker": [ 2024-07-04T17:28:19.046 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:19.046 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:28:19.046 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "0" 2024-07-04T17:28:19.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:19.046 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:19.046 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:28:19.046 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:28:19.047 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:19.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:19.047 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:28:19.047 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "1" 2024-07-04T17:28:19.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:19.047 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:19.047 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:28:19.047 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:28:19.047 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:19.047 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:19.047 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:28:19.047 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "2" 2024-07-04T17:28:19.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:19.048 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:19.048 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:28:19.048 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:28:19.048 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.048 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.048 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:19.048 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": [ 2024-07-04T17:28:19.048 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:19.048 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:19.048 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:19.048 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 20, 2024-07-04T17:28:19.048 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 3, 2024-07-04T17:28:19.048 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 74416, 2024-07-04T17:28:19.049 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 464, 2024-07-04T17:28:19.049 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:28:19.049 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:28:19.049 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.008505389, 2024-07-04T17:28:19.049 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.001034692, 2024-07-04T17:28:19.049 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.005455686, 2024-07-04T17:28:19.049 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.001671330, 2024-07-04T17:28:19.049 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:28:19.049 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 3, 2024-07-04T17:28:19.049 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.004312719, 2024-07-04T17:28:19.049 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.001437573 2024-07-04T17:28:19.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:19.050 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:28:19.050 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:19.050 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:19.050 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:19.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:19.050 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 74416, 2024-07-04T17:28:19.050 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 464 2024-07-04T17:28:19.050 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.050 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.050 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:19.050 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": [ 2024-07-04T17:28:19.050 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:19.050 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:19.050 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:19.051 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 2, 2024-07-04T17:28:19.051 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 24, 2024-07-04T17:28:19.051 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 288, 2024-07-04T17:28:19.051 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 167760, 2024-07-04T17:28:19.051 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 5, 2024-07-04T17:28:19.051 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:28:19.051 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.010615228, 2024-07-04T17:28:19.051 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.004599616, 2024-07-04T17:28:19.051 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.003527254, 2024-07-04T17:28:19.051 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:28:19.051 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:28:19.051 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 23, 2024-07-04T17:28:19.051 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.016886915, 2024-07-04T17:28:19.051 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000734213 2024-07-04T17:28:19.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:19.052 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:28:19.052 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:19.052 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:19.052 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:19.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:19.052 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 288, 2024-07-04T17:28:19.052 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 167760 2024-07-04T17:28:19.052 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.052 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.052 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:19.052 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": [ 2024-07-04T17:28:19.052 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:19.052 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:19.052 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:19.052 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 13, 2024-07-04T17:28:19.053 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 6, 2024-07-04T17:28:19.053 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 302800, 2024-07-04T17:28:19.053 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 928, 2024-07-04T17:28:19.053 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:28:19.053 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:28:19.053 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.011135330, 2024-07-04T17:28:19.053 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.001062987, 2024-07-04T17:28:19.053 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.017280314, 2024-07-04T17:28:19.053 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:28:19.053 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:28:19.053 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 6, 2024-07-04T17:28:19.053 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.003172173, 2024-07-04T17:28:19.053 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000528695 2024-07-04T17:28:19.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:19.054 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:28:19.054 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:19.054 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:19.054 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:19.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:19.054 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 302800, 2024-07-04T17:28:19.054 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 928 2024-07-04T17:28:19.054 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.054 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.054 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:19.054 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": [ 2024-07-04T17:28:19.054 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:19.055 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:19.055 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:19.055 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-07-04T17:28:19.055 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-07-04T17:28:19.055 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.055 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.055 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:19.055 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror": [ 2024-07-04T17:28:19.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:19.055 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:19.055 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:19.055 INFO:teuthology.orchestra.run.smithi042.stdout: "mirrored_filesystems": 1, 2024-07-04T17:28:19.056 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_enable_failures": 0 2024-07-04T17:28:19.056 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.056 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.056 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:19.056 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_mirrored_filesystems": [ 2024-07-04T17:28:19.056 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:19.056 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:28:19.056 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem": "cephfs" 2024-07-04T17:28:19.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:19.056 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:19.057 INFO:teuthology.orchestra.run.smithi042.stdout: "mirroring_peers": 0, 2024-07-04T17:28:19.057 INFO:teuthology.orchestra.run.smithi042.stdout: "directory_count": 2 2024-07-04T17:28:19.057 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.057 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.057 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:19.057 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": [ 2024-07-04T17:28:19.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:19.057 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:19.057 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:19.058 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-07-04T17:28:19.058 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-07-04T17:28:19.058 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T17:28:19.058 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-07-04T17:28:19.058 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T17:28:19.058 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-07-04T17:28:19.058 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T17:28:19.058 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T17:28:19.058 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T17:28:19.058 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T17:28:19.058 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T17:28:19.058 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-07-04T17:28:19.058 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T17:28:19.059 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T17:28:19.059 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-07-04T17:28:19.059 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-07-04T17:28:19.059 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-07-04T17:28:19.059 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-07-04T17:28:19.059 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T17:28:19.059 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T17:28:19.059 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T17:28:19.059 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T17:28:19.060 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T17:28:19.060 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-07-04T17:28:19.060 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-07-04T17:28:19.060 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-07-04T17:28:19.060 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T17:28:19.060 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T17:28:19.060 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-07-04T17:28:19.060 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-07-04T17:28:19.060 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-07-04T17:28:19.060 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-07-04T17:28:19.060 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T17:28:19.060 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-07-04T17:28:19.060 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T17:28:19.061 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-07-04T17:28:19.061 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 182671, 2024-07-04T17:28:19.061 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 53, 2024-07-04T17:28:19.061 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-07-04T17:28:19.061 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-07-04T17:28:19.061 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-07-04T17:28:19.061 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-07-04T17:28:19.061 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-07-04T17:28:19.061 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-07-04T17:28:19.061 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-07-04T17:28:19.061 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-07-04T17:28:19.061 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 18208, 2024-07-04T17:28:19.061 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 161, 2024-07-04T17:28:19.061 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T17:28:19.062 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-07-04T17:28:19.062 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-07-04T17:28:19.062 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-07-04T17:28:19.062 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 0, 2024-07-04T17:28:19.062 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 0, 2024-07-04T17:28:19.062 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-07-04T17:28:19.062 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-07-04T17:28:19.062 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-07-04T17:28:19.062 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-07-04T17:28:19.062 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.062 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.062 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:19.063 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter": [ 2024-07-04T17:28:19.063 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:19.063 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:19.063 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:19.063 INFO:teuthology.orchestra.run.smithi042.stdout: "op_active": 0, 2024-07-04T17:28:19.063 INFO:teuthology.orchestra.run.smithi042.stdout: "op_laggy": 0, 2024-07-04T17:28:19.063 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send": 0, 2024-07-04T17:28:19.063 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send_bytes": 0, 2024-07-04T17:28:19.063 INFO:teuthology.orchestra.run.smithi042.stdout: "op_resend": 0, 2024-07-04T17:28:19.063 INFO:teuthology.orchestra.run.smithi042.stdout: "op_reply": 0, 2024-07-04T17:28:19.063 INFO:teuthology.orchestra.run.smithi042.stdout: "op_latency": { 2024-07-04T17:28:19.063 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:19.064 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:19.064 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:19.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:19.064 INFO:teuthology.orchestra.run.smithi042.stdout: "op_inflight": 0, 2024-07-04T17:28:19.064 INFO:teuthology.orchestra.run.smithi042.stdout: "oplen_avg": { 2024-07-04T17:28:19.064 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:19.064 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0 2024-07-04T17:28:19.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:19.064 INFO:teuthology.orchestra.run.smithi042.stdout: "op": 0, 2024-07-04T17:28:19.064 INFO:teuthology.orchestra.run.smithi042.stdout: "op_r": 0, 2024-07-04T17:28:19.064 INFO:teuthology.orchestra.run.smithi042.stdout: "op_w": 0, 2024-07-04T17:28:19.064 INFO:teuthology.orchestra.run.smithi042.stdout: "op_rmw": 0, 2024-07-04T17:28:19.064 INFO:teuthology.orchestra.run.smithi042.stdout: "op_pg": 0, 2024-07-04T17:28:19.064 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_stat": 0, 2024-07-04T17:28:19.065 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_create": 0, 2024-07-04T17:28:19.065 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_read": 0, 2024-07-04T17:28:19.065 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_write": 0, 2024-07-04T17:28:19.065 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writefull": 0, 2024-07-04T17:28:19.065 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writesame": 0, 2024-07-04T17:28:19.065 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_append": 0, 2024-07-04T17:28:19.065 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_zero": 0, 2024-07-04T17:28:19.065 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_truncate": 0, 2024-07-04T17:28:19.065 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_delete": 0, 2024-07-04T17:28:19.065 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_mapext": 0, 2024-07-04T17:28:19.065 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_sparse_read": 0, 2024-07-04T17:28:19.065 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_clonerange": 0, 2024-07-04T17:28:19.065 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_getxattr": 0, 2024-07-04T17:28:19.065 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_setxattr": 0, 2024-07-04T17:28:19.066 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_cmpxattr": 0, 2024-07-04T17:28:19.066 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_rmxattr": 0, 2024-07-04T17:28:19.066 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_resetxattrs": 0, 2024-07-04T17:28:19.066 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_call": 0, 2024-07-04T17:28:19.066 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_watch": 0, 2024-07-04T17:28:19.066 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_notify": 0, 2024-07-04T17:28:19.066 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T17:28:19.066 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls": 0, 2024-07-04T17:28:19.066 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls_filter": 0, 2024-07-04T17:28:19.066 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_other": 0, 2024-07-04T17:28:19.066 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_active": 0, 2024-07-04T17:28:19.066 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_send": 0, 2024-07-04T17:28:19.066 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_resend": 0, 2024-07-04T17:28:19.066 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_ping": 0, 2024-07-04T17:28:19.067 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_active": 0, 2024-07-04T17:28:19.067 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_send": 0, 2024-07-04T17:28:19.067 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_resend": 0, 2024-07-04T17:28:19.067 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_active": 0, 2024-07-04T17:28:19.067 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_send": 0, 2024-07-04T17:28:19.067 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_resend": 0, 2024-07-04T17:28:19.067 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_active": 0, 2024-07-04T17:28:19.067 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_send": 0, 2024-07-04T17:28:19.067 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_resend": 0, 2024-07-04T17:28:19.067 INFO:teuthology.orchestra.run.smithi042.stdout: "command_active": 0, 2024-07-04T17:28:19.067 INFO:teuthology.orchestra.run.smithi042.stdout: "command_send": 0, 2024-07-04T17:28:19.067 INFO:teuthology.orchestra.run.smithi042.stdout: "command_resend": 0, 2024-07-04T17:28:19.068 INFO:teuthology.orchestra.run.smithi042.stdout: "map_epoch": 0, 2024-07-04T17:28:19.068 INFO:teuthology.orchestra.run.smithi042.stdout: "map_full": 0, 2024-07-04T17:28:19.068 INFO:teuthology.orchestra.run.smithi042.stdout: "map_inc": 0, 2024-07-04T17:28:19.068 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_sessions": 0, 2024-07-04T17:28:19.068 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_open": 0, 2024-07-04T17:28:19.068 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_close": 0, 2024-07-04T17:28:19.068 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_laggy": 0, 2024-07-04T17:28:19.068 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_wr": 0, 2024-07-04T17:28:19.068 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_rd": 0, 2024-07-04T17:28:19.068 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_del": 0 2024-07-04T17:28:19.068 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.068 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.068 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:19.068 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-client": [ 2024-07-04T17:28:19.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:19.069 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:19.069 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:19.069 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:28:19.069 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:28:19.069 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:28:19.069 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 20, 2024-07-04T17:28:19.069 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 71695, 2024-07-04T17:28:19.069 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:28:19.069 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 20, 2024-07-04T17:28:19.069 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:28:19.069 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:28:19.069 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 20, 2024-07-04T17:28:19.069 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 71695, 2024-07-04T17:28:19.069 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:28:19.069 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:19.070 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:19.070 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:19.070 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.070 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.070 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.070 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:19.070 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-radosclient": [ 2024-07-04T17:28:19.070 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:19.070 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:19.070 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:19.070 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:28:19.070 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:28:19.070 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:28:19.070 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 12, 2024-07-04T17:28:19.070 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 300554, 2024-07-04T17:28:19.071 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:28:19.071 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 12, 2024-07-04T17:28:19.071 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:28:19.071 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:28:19.071 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 12, 2024-07-04T17:28:19.071 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 300554, 2024-07-04T17:28:19.071 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:28:19.071 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:19.071 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:19.071 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:19.071 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.071 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.071 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.071 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:19.072 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_bytes": [ 2024-07-04T17:28:19.072 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:19.072 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:19.072 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:19.072 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:28:19.072 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:28:19.072 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:28:19.072 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:28:19.072 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:28:19.072 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:28:19.072 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:28:19.072 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:28:19.072 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:28:19.072 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:28:19.072 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:28:19.072 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:28:19.073 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:19.073 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:19.073 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:19.073 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.073 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.073 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.073 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:19.073 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_ops": [ 2024-07-04T17:28:19.073 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:19.073 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:19.073 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:19.073 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:28:19.073 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1024, 2024-07-04T17:28:19.073 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:28:19.073 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:28:19.074 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:28:19.074 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:28:19.074 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:28:19.074 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:28:19.074 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:28:19.074 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:28:19.074 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:28:19.074 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:28:19.074 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:19.074 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:19.074 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:19.074 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.074 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.075 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.075 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-07-04T17:28:19.075 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:28:19.075 DEBUG:tasks.cephfs.test_mirroring:command returned={ "AsyncMessenger::Worker": [ { "labels": { "id": "0" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "1" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "2" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } } ], "AsyncMessenger::Worker-0": [ { "labels": {}, "counters": { "msgr_recv_messages": 20, "msgr_send_messages": 3, "msgr_recv_bytes": 74416, "msgr_send_bytes": 464, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.008505389, "msgr_running_send_time": 0.001034692, "msgr_running_recv_time": 0.005455686, "msgr_running_fast_dispatch_time": 0.001671330, "msgr_send_messages_queue_lat": { "avgcount": 3, "sum": 0.004312719, "avgtime": 0.001437573 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 74416, "msgr_send_encrypted_bytes": 464 } } ], "AsyncMessenger::Worker-1": [ { "labels": {}, "counters": { "msgr_recv_messages": 2, "msgr_send_messages": 24, "msgr_recv_bytes": 288, "msgr_send_bytes": 167760, "msgr_created_connections": 5, "msgr_active_connections": 1, "msgr_running_total_time": 0.010615228, "msgr_running_send_time": 0.004599616, "msgr_running_recv_time": 0.003527254, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 23, "sum": 0.016886915, "avgtime": 0.000734213 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 288, "msgr_send_encrypted_bytes": 167760 } } ], "AsyncMessenger::Worker-2": [ { "labels": {}, "counters": { "msgr_recv_messages": 13, "msgr_send_messages": 6, "msgr_recv_bytes": 302800, "msgr_send_bytes": 928, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.011135330, "msgr_running_send_time": 0.001062987, "msgr_running_recv_time": 0.017280314, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 6, "sum": 0.003172173, "avgtime": 0.000528695 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 302800, "msgr_send_encrypted_bytes": 928 } } ], "cct": [ { "labels": {}, "counters": { "total_workers": 0, "unhealthy_workers": 0 } } ], "cephfs_mirror": [ { "labels": {}, "counters": { "mirrored_filesystems": 1, "mirror_enable_failures": 0 } } ], "cephfs_mirror_mirrored_filesystems": [ { "labels": { "filesystem": "cephfs" }, "counters": { "mirroring_peers": 0, "directory_count": 2 } } ], "mempool": [ { "labels": {}, "counters": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_items": 0, "bluefs_bytes": 0, "bluefs_items": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "buffer_anon_bytes": 182671, "buffer_anon_items": 53, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 18208, "osdmap_items": 161, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "mds_co_bytes": 0, "mds_co_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 } } ], "objecter": [ { "labels": {}, "counters": { "op_active": 0, "op_laggy": 0, "op_send": 0, "op_send_bytes": 0, "op_resend": 0, "op_reply": 0, "op_latency": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "op_inflight": 0, "oplen_avg": { "avgcount": 0, "sum": 0 }, "op": 0, "op_r": 0, "op_w": 0, "op_rmw": 0, "op_pg": 0, "osdop_stat": 0, "osdop_create": 0, "osdop_read": 0, "osdop_write": 0, "osdop_writefull": 0, "osdop_writesame": 0, "osdop_append": 0, "osdop_zero": 0, "osdop_truncate": 0, "osdop_delete": 0, "osdop_mapext": 0, "osdop_sparse_read": 0, "osdop_clonerange": 0, "osdop_getxattr": 0, "osdop_setxattr": 0, "osdop_cmpxattr": 0, "osdop_rmxattr": 0, "osdop_resetxattrs": 0, "osdop_call": 0, "osdop_watch": 0, "osdop_notify": 0, "osdop_src_cmpxattr": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_other": 0, "linger_active": 0, "linger_send": 0, "linger_resend": 0, "linger_ping": 0, "poolop_active": 0, "poolop_send": 0, "poolop_resend": 0, "poolstat_active": 0, "poolstat_send": 0, "poolstat_resend": 0, "statfs_active": 0, "statfs_send": 0, "statfs_resend": 0, "command_active": 0, "command_send": 0, "command_resend": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "osd_sessions": 0, "osd_session_open": 0, "osd_session_close": 0, "osd_laggy": 0, "omap_wr": 0, "omap_rd": 0, "omap_del": 0 } } ], "throttle-msgr_dispatch_throttler-client": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 20, "get_sum": 71695, "get_or_fail_fail": 0, "get_or_fail_success": 20, "take": 0, "take_sum": 0, "put": 20, "put_sum": 71695, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-msgr_dispatch_throttler-radosclient": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 12, "get_sum": 300554, "get_or_fail_fail": 0, "get_or_fail_success": 12, "take": 0, "take_sum": 0, "put": 12, "put_sum": 300554, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_bytes": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_ops": [ { "labels": {}, "counters": { "val": 0, "max": 1024, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ] } 2024-07-04T17:28:19.076 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:28:19.076 DEBUG:teuthology.orchestra.run.smithi042:counter dump for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok counter dump 2024-07-04T17:28:19.322 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:28:19.322 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker": [ 2024-07-04T17:28:19.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:19.322 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:28:19.322 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "0" 2024-07-04T17:28:19.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:19.322 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:19.322 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:28:19.323 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:28:19.323 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:19.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:19.323 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:28:19.323 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "1" 2024-07-04T17:28:19.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:19.323 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:19.323 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:28:19.323 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:28:19.323 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:19.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:19.324 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:28:19.324 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "2" 2024-07-04T17:28:19.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:19.324 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:19.324 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:28:19.324 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:28:19.324 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.324 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.324 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:19.324 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": [ 2024-07-04T17:28:19.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:19.324 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:19.324 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:19.325 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 20, 2024-07-04T17:28:19.325 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 3, 2024-07-04T17:28:19.325 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 74416, 2024-07-04T17:28:19.325 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 464, 2024-07-04T17:28:19.325 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:28:19.325 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:28:19.325 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.008505389, 2024-07-04T17:28:19.325 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.001034692, 2024-07-04T17:28:19.325 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.005455686, 2024-07-04T17:28:19.325 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.001671330, 2024-07-04T17:28:19.325 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:28:19.325 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 3, 2024-07-04T17:28:19.325 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.004312719, 2024-07-04T17:28:19.326 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.001437573 2024-07-04T17:28:19.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:19.326 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:28:19.326 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:19.326 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:19.326 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:19.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:19.326 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 74416, 2024-07-04T17:28:19.326 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 464 2024-07-04T17:28:19.326 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.326 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.326 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:19.327 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": [ 2024-07-04T17:28:19.327 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:19.327 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:19.327 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:19.327 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 2, 2024-07-04T17:28:19.327 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 24, 2024-07-04T17:28:19.327 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 288, 2024-07-04T17:28:19.327 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 167760, 2024-07-04T17:28:19.327 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 5, 2024-07-04T17:28:19.327 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:28:19.327 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.010615228, 2024-07-04T17:28:19.327 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.004599616, 2024-07-04T17:28:19.327 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.003527254, 2024-07-04T17:28:19.328 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:28:19.328 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:28:19.328 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 23, 2024-07-04T17:28:19.328 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.016886915, 2024-07-04T17:28:19.328 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000734213 2024-07-04T17:28:19.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:19.328 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:28:19.328 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:19.328 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:19.328 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:19.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:19.328 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 288, 2024-07-04T17:28:19.329 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 167760 2024-07-04T17:28:19.329 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.329 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.329 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:19.329 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": [ 2024-07-04T17:28:19.329 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:19.329 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:19.329 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:19.329 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 13, 2024-07-04T17:28:19.329 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 6, 2024-07-04T17:28:19.329 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 302800, 2024-07-04T17:28:19.329 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 928, 2024-07-04T17:28:19.329 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:28:19.329 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:28:19.329 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.011135330, 2024-07-04T17:28:19.330 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.001062987, 2024-07-04T17:28:19.330 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.017280314, 2024-07-04T17:28:19.330 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:28:19.330 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:28:19.330 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 6, 2024-07-04T17:28:19.330 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.003172173, 2024-07-04T17:28:19.330 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000528695 2024-07-04T17:28:19.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:19.330 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:28:19.330 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:19.330 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:19.330 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:19.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:19.331 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 302800, 2024-07-04T17:28:19.331 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 928 2024-07-04T17:28:19.331 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.331 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.331 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:19.331 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": [ 2024-07-04T17:28:19.331 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:19.331 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:19.331 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:19.331 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-07-04T17:28:19.331 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-07-04T17:28:19.331 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.331 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.332 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:19.332 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror": [ 2024-07-04T17:28:19.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:19.332 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:19.332 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:19.332 INFO:teuthology.orchestra.run.smithi042.stdout: "mirrored_filesystems": 1, 2024-07-04T17:28:19.332 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_enable_failures": 0 2024-07-04T17:28:19.332 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.332 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.332 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:19.332 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_mirrored_filesystems": [ 2024-07-04T17:28:19.332 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:19.332 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:28:19.332 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem": "cephfs" 2024-07-04T17:28:19.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:19.333 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:19.333 INFO:teuthology.orchestra.run.smithi042.stdout: "mirroring_peers": 0, 2024-07-04T17:28:19.333 INFO:teuthology.orchestra.run.smithi042.stdout: "directory_count": 2 2024-07-04T17:28:19.333 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.333 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.333 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:19.333 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": [ 2024-07-04T17:28:19.333 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:19.333 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:19.333 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:19.333 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-07-04T17:28:19.333 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-07-04T17:28:19.333 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T17:28:19.334 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-07-04T17:28:19.334 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T17:28:19.334 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-07-04T17:28:19.334 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T17:28:19.334 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T17:28:19.334 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T17:28:19.334 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T17:28:19.334 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T17:28:19.334 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-07-04T17:28:19.334 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T17:28:19.334 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T17:28:19.334 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-07-04T17:28:19.335 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-07-04T17:28:19.335 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-07-04T17:28:19.335 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-07-04T17:28:19.335 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T17:28:19.335 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T17:28:19.335 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T17:28:19.335 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T17:28:19.335 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T17:28:19.335 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-07-04T17:28:19.335 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-07-04T17:28:19.335 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-07-04T17:28:19.335 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T17:28:19.336 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T17:28:19.336 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-07-04T17:28:19.336 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-07-04T17:28:19.336 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-07-04T17:28:19.336 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-07-04T17:28:19.336 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T17:28:19.336 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-07-04T17:28:19.336 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T17:28:19.336 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-07-04T17:28:19.336 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 182671, 2024-07-04T17:28:19.336 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 53, 2024-07-04T17:28:19.336 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-07-04T17:28:19.337 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-07-04T17:28:19.337 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-07-04T17:28:19.337 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-07-04T17:28:19.337 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-07-04T17:28:19.337 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-07-04T17:28:19.337 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-07-04T17:28:19.337 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-07-04T17:28:19.337 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 18208, 2024-07-04T17:28:19.337 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 161, 2024-07-04T17:28:19.337 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T17:28:19.337 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-07-04T17:28:19.337 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-07-04T17:28:19.337 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-07-04T17:28:19.337 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 0, 2024-07-04T17:28:19.338 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 0, 2024-07-04T17:28:19.338 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-07-04T17:28:19.338 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-07-04T17:28:19.338 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-07-04T17:28:19.338 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-07-04T17:28:19.338 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.338 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.338 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:19.338 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter": [ 2024-07-04T17:28:19.338 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:19.338 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:19.338 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:19.338 INFO:teuthology.orchestra.run.smithi042.stdout: "op_active": 0, 2024-07-04T17:28:19.338 INFO:teuthology.orchestra.run.smithi042.stdout: "op_laggy": 0, 2024-07-04T17:28:19.338 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send": 0, 2024-07-04T17:28:19.339 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send_bytes": 0, 2024-07-04T17:28:19.339 INFO:teuthology.orchestra.run.smithi042.stdout: "op_resend": 0, 2024-07-04T17:28:19.339 INFO:teuthology.orchestra.run.smithi042.stdout: "op_reply": 0, 2024-07-04T17:28:19.339 INFO:teuthology.orchestra.run.smithi042.stdout: "op_latency": { 2024-07-04T17:28:19.339 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:19.339 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:19.339 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:19.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:19.339 INFO:teuthology.orchestra.run.smithi042.stdout: "op_inflight": 0, 2024-07-04T17:28:19.339 INFO:teuthology.orchestra.run.smithi042.stdout: "oplen_avg": { 2024-07-04T17:28:19.339 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:19.339 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0 2024-07-04T17:28:19.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:19.340 INFO:teuthology.orchestra.run.smithi042.stdout: "op": 0, 2024-07-04T17:28:19.340 INFO:teuthology.orchestra.run.smithi042.stdout: "op_r": 0, 2024-07-04T17:28:19.340 INFO:teuthology.orchestra.run.smithi042.stdout: "op_w": 0, 2024-07-04T17:28:19.340 INFO:teuthology.orchestra.run.smithi042.stdout: "op_rmw": 0, 2024-07-04T17:28:19.340 INFO:teuthology.orchestra.run.smithi042.stdout: "op_pg": 0, 2024-07-04T17:28:19.340 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_stat": 0, 2024-07-04T17:28:19.340 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_create": 0, 2024-07-04T17:28:19.340 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_read": 0, 2024-07-04T17:28:19.340 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_write": 0, 2024-07-04T17:28:19.340 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writefull": 0, 2024-07-04T17:28:19.340 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writesame": 0, 2024-07-04T17:28:19.340 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_append": 0, 2024-07-04T17:28:19.340 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_zero": 0, 2024-07-04T17:28:19.341 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_truncate": 0, 2024-07-04T17:28:19.341 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_delete": 0, 2024-07-04T17:28:19.341 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_mapext": 0, 2024-07-04T17:28:19.341 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_sparse_read": 0, 2024-07-04T17:28:19.341 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_clonerange": 0, 2024-07-04T17:28:19.341 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_getxattr": 0, 2024-07-04T17:28:19.341 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_setxattr": 0, 2024-07-04T17:28:19.341 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_cmpxattr": 0, 2024-07-04T17:28:19.341 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_rmxattr": 0, 2024-07-04T17:28:19.341 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_resetxattrs": 0, 2024-07-04T17:28:19.341 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_call": 0, 2024-07-04T17:28:19.341 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_watch": 0, 2024-07-04T17:28:19.341 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_notify": 0, 2024-07-04T17:28:19.341 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T17:28:19.341 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls": 0, 2024-07-04T17:28:19.342 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls_filter": 0, 2024-07-04T17:28:19.342 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_other": 0, 2024-07-04T17:28:19.342 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_active": 0, 2024-07-04T17:28:19.342 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_send": 0, 2024-07-04T17:28:19.342 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_resend": 0, 2024-07-04T17:28:19.342 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_ping": 0, 2024-07-04T17:28:19.342 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_active": 0, 2024-07-04T17:28:19.342 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_send": 0, 2024-07-04T17:28:19.342 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_resend": 0, 2024-07-04T17:28:19.342 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_active": 0, 2024-07-04T17:28:19.342 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_send": 0, 2024-07-04T17:28:19.342 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_resend": 0, 2024-07-04T17:28:19.343 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_active": 0, 2024-07-04T17:28:19.343 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_send": 0, 2024-07-04T17:28:19.343 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_resend": 0, 2024-07-04T17:28:19.343 INFO:teuthology.orchestra.run.smithi042.stdout: "command_active": 0, 2024-07-04T17:28:19.343 INFO:teuthology.orchestra.run.smithi042.stdout: "command_send": 0, 2024-07-04T17:28:19.343 INFO:teuthology.orchestra.run.smithi042.stdout: "command_resend": 0, 2024-07-04T17:28:19.343 INFO:teuthology.orchestra.run.smithi042.stdout: "map_epoch": 0, 2024-07-04T17:28:19.343 INFO:teuthology.orchestra.run.smithi042.stdout: "map_full": 0, 2024-07-04T17:28:19.343 INFO:teuthology.orchestra.run.smithi042.stdout: "map_inc": 0, 2024-07-04T17:28:19.343 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_sessions": 0, 2024-07-04T17:28:19.343 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_open": 0, 2024-07-04T17:28:19.343 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_close": 0, 2024-07-04T17:28:19.343 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_laggy": 0, 2024-07-04T17:28:19.343 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_wr": 0, 2024-07-04T17:28:19.344 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_rd": 0, 2024-07-04T17:28:19.344 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_del": 0 2024-07-04T17:28:19.344 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.344 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.344 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:19.344 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-client": [ 2024-07-04T17:28:19.344 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:19.344 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:19.344 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:19.344 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:28:19.344 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:28:19.344 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:28:19.344 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 20, 2024-07-04T17:28:19.345 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 71695, 2024-07-04T17:28:19.345 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:28:19.345 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 20, 2024-07-04T17:28:19.345 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:28:19.345 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:28:19.345 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 20, 2024-07-04T17:28:19.345 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 71695, 2024-07-04T17:28:19.345 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:28:19.345 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:19.345 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:19.345 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:19.345 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.345 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.345 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.346 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:19.346 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-radosclient": [ 2024-07-04T17:28:19.346 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:19.346 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:19.346 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:19.346 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:28:19.346 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:28:19.346 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:28:19.346 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 12, 2024-07-04T17:28:19.346 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 300554, 2024-07-04T17:28:19.346 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:28:19.346 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 12, 2024-07-04T17:28:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:28:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:28:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 12, 2024-07-04T17:28:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 300554, 2024-07-04T17:28:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:28:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_bytes": [ 2024-07-04T17:28:19.348 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:19.348 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:19.348 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:19.348 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:28:19.348 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:28:19.348 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:28:19.348 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:28:19.348 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:28:19.348 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:28:19.348 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:28:19.348 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:28:19.348 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:28:19.348 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:28:19.348 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:28:19.349 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:28:19.349 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:19.349 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:19.349 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:19.349 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.349 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.349 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.349 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:19.349 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_ops": [ 2024-07-04T17:28:19.349 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:19.349 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:19.349 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:19.349 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:28:19.349 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1024, 2024-07-04T17:28:19.349 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:28:19.350 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:28:19.350 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:28:19.350 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:28:19.350 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:28:19.350 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:28:19.350 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:28:19.350 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:28:19.350 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:28:19.350 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:28:19.350 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:19.350 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:19.350 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:19.350 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.351 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.351 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:19.351 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-07-04T17:28:19.351 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:28:19.352 DEBUG:tasks.cephfs.test_mirroring:command returned={ "AsyncMessenger::Worker": [ { "labels": { "id": "0" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "1" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "2" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } } ], "AsyncMessenger::Worker-0": [ { "labels": {}, "counters": { "msgr_recv_messages": 20, "msgr_send_messages": 3, "msgr_recv_bytes": 74416, "msgr_send_bytes": 464, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.008505389, "msgr_running_send_time": 0.001034692, "msgr_running_recv_time": 0.005455686, "msgr_running_fast_dispatch_time": 0.001671330, "msgr_send_messages_queue_lat": { "avgcount": 3, "sum": 0.004312719, "avgtime": 0.001437573 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 74416, "msgr_send_encrypted_bytes": 464 } } ], "AsyncMessenger::Worker-1": [ { "labels": {}, "counters": { "msgr_recv_messages": 2, "msgr_send_messages": 24, "msgr_recv_bytes": 288, "msgr_send_bytes": 167760, "msgr_created_connections": 5, "msgr_active_connections": 1, "msgr_running_total_time": 0.010615228, "msgr_running_send_time": 0.004599616, "msgr_running_recv_time": 0.003527254, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 23, "sum": 0.016886915, "avgtime": 0.000734213 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 288, "msgr_send_encrypted_bytes": 167760 } } ], "AsyncMessenger::Worker-2": [ { "labels": {}, "counters": { "msgr_recv_messages": 13, "msgr_send_messages": 6, "msgr_recv_bytes": 302800, "msgr_send_bytes": 928, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.011135330, "msgr_running_send_time": 0.001062987, "msgr_running_recv_time": 0.017280314, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 6, "sum": 0.003172173, "avgtime": 0.000528695 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 302800, "msgr_send_encrypted_bytes": 928 } } ], "cct": [ { "labels": {}, "counters": { "total_workers": 0, "unhealthy_workers": 0 } } ], "cephfs_mirror": [ { "labels": {}, "counters": { "mirrored_filesystems": 1, "mirror_enable_failures": 0 } } ], "cephfs_mirror_mirrored_filesystems": [ { "labels": { "filesystem": "cephfs" }, "counters": { "mirroring_peers": 0, "directory_count": 2 } } ], "mempool": [ { "labels": {}, "counters": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_items": 0, "bluefs_bytes": 0, "bluefs_items": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "buffer_anon_bytes": 182671, "buffer_anon_items": 53, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 18208, "osdmap_items": 161, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "mds_co_bytes": 0, "mds_co_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 } } ], "objecter": [ { "labels": {}, "counters": { "op_active": 0, "op_laggy": 0, "op_send": 0, "op_send_bytes": 0, "op_resend": 0, "op_reply": 0, "op_latency": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "op_inflight": 0, "oplen_avg": { "avgcount": 0, "sum": 0 }, "op": 0, "op_r": 0, "op_w": 0, "op_rmw": 0, "op_pg": 0, "osdop_stat": 0, "osdop_create": 0, "osdop_read": 0, "osdop_write": 0, "osdop_writefull": 0, "osdop_writesame": 0, "osdop_append": 0, "osdop_zero": 0, "osdop_truncate": 0, "osdop_delete": 0, "osdop_mapext": 0, "osdop_sparse_read": 0, "osdop_clonerange": 0, "osdop_getxattr": 0, "osdop_setxattr": 0, "osdop_cmpxattr": 0, "osdop_rmxattr": 0, "osdop_resetxattrs": 0, "osdop_call": 0, "osdop_watch": 0, "osdop_notify": 0, "osdop_src_cmpxattr": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_other": 0, "linger_active": 0, "linger_send": 0, "linger_resend": 0, "linger_ping": 0, "poolop_active": 0, "poolop_send": 0, "poolop_resend": 0, "poolstat_active": 0, "poolstat_send": 0, "poolstat_resend": 0, "statfs_active": 0, "statfs_send": 0, "statfs_resend": 0, "command_active": 0, "command_send": 0, "command_resend": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "osd_sessions": 0, "osd_session_open": 0, "osd_session_close": 0, "osd_laggy": 0, "omap_wr": 0, "omap_rd": 0, "omap_del": 0 } } ], "throttle-msgr_dispatch_throttler-client": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 20, "get_sum": 71695, "get_or_fail_fail": 0, "get_or_fail_success": 20, "take": 0, "take_sum": 0, "put": 20, "put_sum": 71695, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-msgr_dispatch_throttler-radosclient": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 12, "get_sum": 300554, "get_or_fail_fail": 0, "get_or_fail_success": 12, "take": 0, "take_sum": 0, "put": 12, "put_sum": 300554, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_bytes": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_ops": [ { "labels": {}, "counters": { "val": 0, "max": 1024, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ] } 2024-07-04T17:28:19.352 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snapshot mirror disable cephfs 2024-07-04T17:28:19.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:19.656+0000 7f6b75c00640 1 -- 172.21.15.42:0/4291260414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b7006d070 msgr2=0x7f6b70073a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:19.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:19.656+0000 7f6b75c00640 1 --2- 172.21.15.42:0/4291260414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b7006d070 0x7f6b70073a20 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6b600097b0 tx=0x7f6b6002a540 comp rx=0 tx=0).stop 2024-07-04T17:28:19.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:19.656+0000 7f6b75c00640 1 -- 172.21.15.42:0/4291260414 shutdown_connections 2024-07-04T17:28:19.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:19.656+0000 7f6b75c00640 1 --2- 172.21.15.42:0/4291260414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b7006d070 0x7f6b70073a20 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:19.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:19.656+0000 7f6b75c00640 1 -- 172.21.15.42:0/4291260414 >> 172.21.15.42:0/4291260414 conn(0x7f6b7006c020 msgr2=0x7f6b7006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:19.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:19.657+0000 7f6b75c00640 1 -- 172.21.15.42:0/4291260414 shutdown_connections 2024-07-04T17:28:19.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:19.657+0000 7f6b75c00640 1 -- 172.21.15.42:0/4291260414 wait complete. 2024-07-04T17:28:19.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:19.659+0000 7f6b75c00640 1 Processor -- start 2024-07-04T17:28:19.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:19.659+0000 7f6b75c00640 1 -- start start 2024-07-04T17:28:19.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:19.660+0000 7f6b75c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b7006d070 0x7f6b7013afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:19.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:19.660+0000 7f6b75c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6b70071c40 con 0x7f6b7006d070 2024-07-04T17:28:19.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:19.661+0000 7f6b6ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b7006d070 0x7f6b7013afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:19.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:19.661+0000 7f6b6ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b7006d070 0x7f6b7013afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46190/0 (socket says 172.21.15.42:46190) 2024-07-04T17:28:19.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:19.661+0000 7f6b6ea00640 1 -- 172.21.15.42:0/878711324 learned_addr learned my addr 172.21.15.42:0/878711324 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:19.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:19.661+0000 7f6b6ea00640 1 -- 172.21.15.42:0/878711324 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b700b6f20 con 0x7f6b7006d070 2024-07-04T17:28:19.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:19.662+0000 7f6b6ea00640 1 --2- 172.21.15.42:0/878711324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b7006d070 0x7f6b7013afd0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6b600098d0 tx=0x7f6b60033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:19.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:19.662+0000 7f6b66a00640 1 -- 172.21.15.42:0/878711324 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b60031880 con 0x7f6b7006d070 2024-07-04T17:28:19.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:19.662+0000 7f6b66a00640 1 -- 172.21.15.42:0/878711324 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b60031b10 con 0x7f6b7006d070 2024-07-04T17:28:19.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:19.662+0000 7f6b66a00640 1 -- 172.21.15.42:0/878711324 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b60031df0 con 0x7f6b7006d070 2024-07-04T17:28:19.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:19.662+0000 7f6b75c00640 1 -- 172.21.15.42:0/878711324 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b700b6a60 con 0x7f6b7006d070 2024-07-04T17:28:19.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:19.662+0000 7f6b75c00640 1 -- 172.21.15.42:0/878711324 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b700aed30 con 0x7f6b7006d070 2024-07-04T17:28:19.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:19.663+0000 7f6b75c00640 1 -- 172.21.15.42:0/878711324 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b70073a20 con 0x7f6b7006d070 2024-07-04T17:28:19.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:19.664+0000 7f6b66a00640 1 -- 172.21.15.42:0/878711324 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 10) ==== 49681+0+0 (secure 0 0 0) 0x7f6b60033070 con 0x7f6b7006d070 2024-07-04T17:28:19.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:19.664+0000 7f6b66a00640 1 --2- 172.21.15.42:0/878711324 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f6b5003d5d0 0x7f6b5003fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:19.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:19.664+0000 7f6b66a00640 1 -- 172.21.15.42:0/878711324 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4561+0+0 (secure 0 0 0) 0x7f6b60073290 con 0x7f6b7006d070 2024-07-04T17:28:19.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:19.665+0000 7f6b6e000640 1 --2- 172.21.15.42:0/878711324 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f6b5003d5d0 0x7f6b5003fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:19.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:19.666+0000 7f6b6e000640 1 --2- 172.21.15.42:0/878711324 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f6b5003d5d0 0x7f6b5003fae0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c0015e0 tx=0x7f6b5c001290 comp rx=0 tx=0).ready entity=mgr.4563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:19.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:19.670+0000 7f6b66a00640 1 -- 172.21.15.42:0/878711324 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f6b6003f030 con 0x7f6b7006d070 2024-07-04T17:28:19.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:19.912+0000 7f6b75c00640 1 -- 172.21.15.42:0/878711324 --> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] -- mgr_command(tid 0: {"prefix": "fs snapshot mirror disable", "fs_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7f6b700b6c30 con 0x7f6b5003d5d0 2024-07-04T17:28:20.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:20.800+0000 7f6b66a00640 1 -- 172.21.15.42:0/878711324 <== mgr.4563 v2:172.21.15.42:6824/488387844 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f6b700b6c30 con 0x7f6b5003d5d0 2024-07-04T17:28:20.802 INFO:teuthology.orchestra.run.smithi042.stdout:{} 2024-07-04T17:28:20.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:20.804+0000 7f6b75c00640 1 -- 172.21.15.42:0/878711324 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f6b5003d5d0 msgr2=0x7f6b5003fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:20.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:20.804+0000 7f6b75c00640 1 --2- 172.21.15.42:0/878711324 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f6b5003d5d0 0x7f6b5003fae0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c0015e0 tx=0x7f6b5c001290 comp rx=0 tx=0).stop 2024-07-04T17:28:20.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:20.804+0000 7f6b75c00640 1 -- 172.21.15.42:0/878711324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b7006d070 msgr2=0x7f6b7013afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:20.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:20.804+0000 7f6b75c00640 1 --2- 172.21.15.42:0/878711324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b7006d070 0x7f6b7013afd0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6b600098d0 tx=0x7f6b60033ad0 comp rx=0 tx=0).stop 2024-07-04T17:28:20.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:20.804+0000 7f6b75c00640 1 -- 172.21.15.42:0/878711324 shutdown_connections 2024-07-04T17:28:20.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:20.804+0000 7f6b75c00640 1 --2- 172.21.15.42:0/878711324 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f6b5003d5d0 0x7f6b5003fae0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:20.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:20.804+0000 7f6b75c00640 1 --2- 172.21.15.42:0/878711324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b7006d070 0x7f6b7013afd0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:20.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:20.804+0000 7f6b75c00640 1 -- 172.21.15.42:0/878711324 >> 172.21.15.42:0/878711324 conn(0x7f6b7006c020 msgr2=0x7f6b70140ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:20.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:20.804+0000 7f6b75c00640 1 -- 172.21.15.42:0/878711324 shutdown_connections 2024-07-04T17:28:20.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:20.804+0000 7f6b75c00640 1 -- 172.21.15.42:0/878711324 wait complete. 2024-07-04T17:28:29.898 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-04T17:28:30.842 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:28:30.842 DEBUG:teuthology.orchestra.run.smithi042:mirror status for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@2 2024-07-04T17:28:31.097 INFO:teuthology.orchestra.run.smithi042.stderr:no valid command found; 10 closest matches: 2024-07-04T17:28:31.098 INFO:teuthology.orchestra.run.smithi042.stderr:0 2024-07-04T17:28:31.098 INFO:teuthology.orchestra.run.smithi042.stderr:1 2024-07-04T17:28:31.098 INFO:teuthology.orchestra.run.smithi042.stderr:2 2024-07-04T17:28:31.098 INFO:teuthology.orchestra.run.smithi042.stderr:abort 2024-07-04T17:28:31.098 INFO:teuthology.orchestra.run.smithi042.stderr:assert 2024-07-04T17:28:31.098 INFO:teuthology.orchestra.run.smithi042.stderr:config diff 2024-07-04T17:28:31.098 INFO:teuthology.orchestra.run.smithi042.stderr:config diff get 2024-07-04T17:28:31.098 INFO:teuthology.orchestra.run.smithi042.stderr:config get 2024-07-04T17:28:31.098 INFO:teuthology.orchestra.run.smithi042.stderr:config help [] 2024-07-04T17:28:31.098 INFO:teuthology.orchestra.run.smithi042.stderr:config set ... 2024-07-04T17:28:31.099 INFO:teuthology.orchestra.run.smithi042.stderr:admin_socket: invalid command 2024-07-04T17:28:31.107 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-07-04T17:28:31.108 WARNING:tasks.cephfs.test_mirroring:mirror daemon command with label "mirror status for fs: cephfs" failed: Command failed (mirror status for fs: cephfs) on smithi042 with status 22: 'ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@2' 2024-07-04T17:28:31.108 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:28:31.108 DEBUG:teuthology.orchestra.run.smithi042:counter dump for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok counter dump 2024-07-04T17:28:31.350 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:28:31.351 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker": [ 2024-07-04T17:28:31.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:31.351 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:28:31.351 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "0" 2024-07-04T17:28:31.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:31.351 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:31.351 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:28:31.351 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:28:31.351 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:31.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:31.351 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:31.351 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:28:31.351 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "1" 2024-07-04T17:28:31.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:31.352 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:31.352 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:28:31.352 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:28:31.352 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:31.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:31.352 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:31.352 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:28:31.352 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "2" 2024-07-04T17:28:31.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:31.352 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:31.352 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:28:31.352 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:28:31.353 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:31.353 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:31.353 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:31.353 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": [ 2024-07-04T17:28:31.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:31.353 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:31.353 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:31.353 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 21, 2024-07-04T17:28:31.353 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 3, 2024-07-04T17:28:31.353 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 79312, 2024-07-04T17:28:31.353 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 464, 2024-07-04T17:28:31.353 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:28:31.353 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:28:31.353 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.009053496, 2024-07-04T17:28:31.354 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.001116215, 2024-07-04T17:28:31.354 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.005700475, 2024-07-04T17:28:31.354 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.001846862, 2024-07-04T17:28:31.354 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:28:31.354 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 3, 2024-07-04T17:28:31.354 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.004312719, 2024-07-04T17:28:31.354 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.001437573 2024-07-04T17:28:31.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:31.354 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:28:31.354 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:31.354 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:31.354 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:31.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:31.355 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 79312, 2024-07-04T17:28:31.355 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 464 2024-07-04T17:28:31.355 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:31.355 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:31.355 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:31.355 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": [ 2024-07-04T17:28:31.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:31.355 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:31.355 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:31.355 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 2, 2024-07-04T17:28:31.355 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 26, 2024-07-04T17:28:31.355 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 288, 2024-07-04T17:28:31.355 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 168272, 2024-07-04T17:28:31.355 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 5, 2024-07-04T17:28:31.356 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:28:31.356 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.010943112, 2024-07-04T17:28:31.356 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.004880198, 2024-07-04T17:28:31.356 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.003527254, 2024-07-04T17:28:31.356 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:28:31.356 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:28:31.356 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 25, 2024-07-04T17:28:31.356 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.017119963, 2024-07-04T17:28:31.356 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000684798 2024-07-04T17:28:31.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:31.356 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:28:31.356 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:31.356 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:31.357 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:31.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:31.357 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 288, 2024-07-04T17:28:31.357 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 168272 2024-07-04T17:28:31.357 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:31.357 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:31.357 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:31.357 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": [ 2024-07-04T17:28:31.357 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:31.357 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:31.357 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:31.357 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 13, 2024-07-04T17:28:31.358 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 6, 2024-07-04T17:28:31.358 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 302800, 2024-07-04T17:28:31.358 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 928, 2024-07-04T17:28:31.358 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:28:31.358 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:28:31.358 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.011393171, 2024-07-04T17:28:31.358 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.001171413, 2024-07-04T17:28:31.358 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.017320721, 2024-07-04T17:28:31.358 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:28:31.358 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:28:31.358 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 6, 2024-07-04T17:28:31.358 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.003172173, 2024-07-04T17:28:31.358 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000528695 2024-07-04T17:28:31.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:31.359 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:28:31.359 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:31.359 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:31.359 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:31.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:31.359 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 302800, 2024-07-04T17:28:31.359 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 928 2024-07-04T17:28:31.359 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:31.359 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:31.359 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:31.359 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": [ 2024-07-04T17:28:31.359 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:31.359 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:31.360 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:31.360 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-07-04T17:28:31.360 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-07-04T17:28:31.360 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:31.360 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:31.360 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:31.360 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror": [ 2024-07-04T17:28:31.360 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:31.360 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:31.360 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:31.360 INFO:teuthology.orchestra.run.smithi042.stdout: "mirrored_filesystems": 0, 2024-07-04T17:28:31.360 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_enable_failures": 0 2024-07-04T17:28:31.360 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:31.361 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:31.361 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:31.361 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": [ 2024-07-04T17:28:31.361 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:31.361 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:31.361 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:31.361 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-07-04T17:28:31.361 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-07-04T17:28:31.361 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T17:28:31.361 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-07-04T17:28:31.361 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T17:28:31.361 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-07-04T17:28:31.361 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T17:28:31.362 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T17:28:31.362 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T17:28:31.362 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T17:28:31.362 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T17:28:31.362 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-07-04T17:28:31.362 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T17:28:31.362 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T17:28:31.362 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-07-04T17:28:31.362 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-07-04T17:28:31.362 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-07-04T17:28:31.362 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-07-04T17:28:31.363 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T17:28:31.363 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T17:28:31.363 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T17:28:31.363 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T17:28:31.363 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T17:28:31.363 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-07-04T17:28:31.363 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-07-04T17:28:31.363 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-07-04T17:28:31.363 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T17:28:31.363 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T17:28:31.363 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-07-04T17:28:31.363 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-07-04T17:28:31.364 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-07-04T17:28:31.364 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-07-04T17:28:31.364 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T17:28:31.364 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-07-04T17:28:31.364 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T17:28:31.364 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-07-04T17:28:31.364 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 148338, 2024-07-04T17:28:31.364 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 24, 2024-07-04T17:28:31.364 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-07-04T17:28:31.364 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-07-04T17:28:31.364 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-07-04T17:28:31.364 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-07-04T17:28:31.364 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-07-04T17:28:31.365 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-07-04T17:28:31.365 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-07-04T17:28:31.365 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-07-04T17:28:31.365 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 4592, 2024-07-04T17:28:31.365 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 37, 2024-07-04T17:28:31.365 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T17:28:31.365 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-07-04T17:28:31.365 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-07-04T17:28:31.365 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-07-04T17:28:31.365 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 0, 2024-07-04T17:28:31.365 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 0, 2024-07-04T17:28:31.365 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-07-04T17:28:31.365 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-07-04T17:28:31.366 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-07-04T17:28:31.366 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-07-04T17:28:31.366 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:31.366 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:31.366 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:31.366 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter": [ 2024-07-04T17:28:31.366 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:31.366 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:31.366 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:31.366 INFO:teuthology.orchestra.run.smithi042.stdout: "op_active": 0, 2024-07-04T17:28:31.366 INFO:teuthology.orchestra.run.smithi042.stdout: "op_laggy": 0, 2024-07-04T17:28:31.366 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send": 0, 2024-07-04T17:28:31.367 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send_bytes": 0, 2024-07-04T17:28:31.367 INFO:teuthology.orchestra.run.smithi042.stdout: "op_resend": 0, 2024-07-04T17:28:31.367 INFO:teuthology.orchestra.run.smithi042.stdout: "op_reply": 0, 2024-07-04T17:28:31.367 INFO:teuthology.orchestra.run.smithi042.stdout: "op_latency": { 2024-07-04T17:28:31.367 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:31.367 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:31.367 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:31.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:31.367 INFO:teuthology.orchestra.run.smithi042.stdout: "op_inflight": 0, 2024-07-04T17:28:31.367 INFO:teuthology.orchestra.run.smithi042.stdout: "oplen_avg": { 2024-07-04T17:28:31.367 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:31.368 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0 2024-07-04T17:28:31.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:28:31.368 INFO:teuthology.orchestra.run.smithi042.stdout: "op": 0, 2024-07-04T17:28:31.368 INFO:teuthology.orchestra.run.smithi042.stdout: "op_r": 0, 2024-07-04T17:28:31.368 INFO:teuthology.orchestra.run.smithi042.stdout: "op_w": 0, 2024-07-04T17:28:31.368 INFO:teuthology.orchestra.run.smithi042.stdout: "op_rmw": 0, 2024-07-04T17:28:31.368 INFO:teuthology.orchestra.run.smithi042.stdout: "op_pg": 0, 2024-07-04T17:28:31.368 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_stat": 0, 2024-07-04T17:28:31.368 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_create": 0, 2024-07-04T17:28:31.368 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_read": 0, 2024-07-04T17:28:31.369 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_write": 0, 2024-07-04T17:28:31.369 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writefull": 0, 2024-07-04T17:28:31.369 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writesame": 0, 2024-07-04T17:28:31.369 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_append": 0, 2024-07-04T17:28:31.369 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_zero": 0, 2024-07-04T17:28:31.369 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_truncate": 0, 2024-07-04T17:28:31.369 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_delete": 0, 2024-07-04T17:28:31.369 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_mapext": 0, 2024-07-04T17:28:31.369 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_sparse_read": 0, 2024-07-04T17:28:31.369 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_clonerange": 0, 2024-07-04T17:28:31.369 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_getxattr": 0, 2024-07-04T17:28:31.369 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_setxattr": 0, 2024-07-04T17:28:31.369 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_cmpxattr": 0, 2024-07-04T17:28:31.370 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_rmxattr": 0, 2024-07-04T17:28:31.370 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_resetxattrs": 0, 2024-07-04T17:28:31.370 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_call": 0, 2024-07-04T17:28:31.370 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_watch": 0, 2024-07-04T17:28:31.370 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_notify": 0, 2024-07-04T17:28:31.370 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T17:28:31.370 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls": 0, 2024-07-04T17:28:31.370 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls_filter": 0, 2024-07-04T17:28:31.370 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_other": 0, 2024-07-04T17:28:31.370 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_active": 0, 2024-07-04T17:28:31.370 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_send": 0, 2024-07-04T17:28:31.370 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_resend": 0, 2024-07-04T17:28:31.371 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_ping": 0, 2024-07-04T17:28:31.371 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_active": 0, 2024-07-04T17:28:31.371 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_send": 0, 2024-07-04T17:28:31.371 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_resend": 0, 2024-07-04T17:28:31.371 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_active": 0, 2024-07-04T17:28:31.371 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_send": 0, 2024-07-04T17:28:31.371 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_resend": 0, 2024-07-04T17:28:31.371 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_active": 0, 2024-07-04T17:28:31.371 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_send": 0, 2024-07-04T17:28:31.371 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_resend": 0, 2024-07-04T17:28:31.371 INFO:teuthology.orchestra.run.smithi042.stdout: "command_active": 0, 2024-07-04T17:28:31.371 INFO:teuthology.orchestra.run.smithi042.stdout: "command_send": 0, 2024-07-04T17:28:31.371 INFO:teuthology.orchestra.run.smithi042.stdout: "command_resend": 0, 2024-07-04T17:28:31.372 INFO:teuthology.orchestra.run.smithi042.stdout: "map_epoch": 0, 2024-07-04T17:28:31.372 INFO:teuthology.orchestra.run.smithi042.stdout: "map_full": 0, 2024-07-04T17:28:31.372 INFO:teuthology.orchestra.run.smithi042.stdout: "map_inc": 0, 2024-07-04T17:28:31.372 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_sessions": 0, 2024-07-04T17:28:31.372 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_open": 0, 2024-07-04T17:28:31.372 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_close": 0, 2024-07-04T17:28:31.372 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_laggy": 0, 2024-07-04T17:28:31.372 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_wr": 0, 2024-07-04T17:28:31.372 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_rd": 0, 2024-07-04T17:28:31.372 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_del": 0 2024-07-04T17:28:31.372 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:31.372 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:31.372 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:31.373 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-client": [ 2024-07-04T17:28:31.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:31.373 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:31.373 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:31.373 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:28:31.373 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:28:31.373 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:28:31.373 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 21, 2024-07-04T17:28:31.373 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 76449, 2024-07-04T17:28:31.373 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:28:31.373 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 21, 2024-07-04T17:28:31.373 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:28:31.373 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:28:31.374 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 21, 2024-07-04T17:28:31.374 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 76449, 2024-07-04T17:28:31.374 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:28:31.374 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:31.374 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:31.374 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:31.374 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:31.374 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:31.374 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:31.374 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:31.374 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-radosclient": [ 2024-07-04T17:28:31.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:31.374 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:31.374 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:31.375 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:28:31.375 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:28:31.375 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:28:31.375 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 12, 2024-07-04T17:28:31.375 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 300554, 2024-07-04T17:28:31.375 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:28:31.375 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 12, 2024-07-04T17:28:31.375 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:28:31.375 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:28:31.375 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 12, 2024-07-04T17:28:31.375 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 300554, 2024-07-04T17:28:31.375 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:28:31.375 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:31.375 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:31.375 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:31.375 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:31.376 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:31.376 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:31.376 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:31.376 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_bytes": [ 2024-07-04T17:28:31.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:31.376 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:31.376 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:31.376 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:28:31.376 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:28:31.376 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:28:31.376 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:28:31.376 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:28:31.376 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:28:31.376 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:28:31.376 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:28:31.376 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:28:31.377 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:28:31.377 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:28:31.377 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:28:31.377 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:31.377 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:31.377 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:31.377 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:31.377 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:31.377 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:31.377 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:28:31.377 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_ops": [ 2024-07-04T17:28:31.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:28:31.377 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:28:31.377 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:28:31.377 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:28:31.378 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1024, 2024-07-04T17:28:31.378 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:28:31.378 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:28:31.378 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:28:31.378 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:28:31.378 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:28:31.378 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:28:31.378 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:28:31.378 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:28:31.378 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:28:31.378 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:28:31.378 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:28:31.378 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:28:31.378 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:28:31.378 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:31.379 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:31.379 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:28:31.379 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-07-04T17:28:31.379 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:28:31.379 DEBUG:tasks.cephfs.test_mirroring:command returned={ "AsyncMessenger::Worker": [ { "labels": { "id": "0" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "1" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "2" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } } ], "AsyncMessenger::Worker-0": [ { "labels": {}, "counters": { "msgr_recv_messages": 21, "msgr_send_messages": 3, "msgr_recv_bytes": 79312, "msgr_send_bytes": 464, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.009053496, "msgr_running_send_time": 0.001116215, "msgr_running_recv_time": 0.005700475, "msgr_running_fast_dispatch_time": 0.001846862, "msgr_send_messages_queue_lat": { "avgcount": 3, "sum": 0.004312719, "avgtime": 0.001437573 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 79312, "msgr_send_encrypted_bytes": 464 } } ], "AsyncMessenger::Worker-1": [ { "labels": {}, "counters": { "msgr_recv_messages": 2, "msgr_send_messages": 26, "msgr_recv_bytes": 288, "msgr_send_bytes": 168272, "msgr_created_connections": 5, "msgr_active_connections": 1, "msgr_running_total_time": 0.010943112, "msgr_running_send_time": 0.004880198, "msgr_running_recv_time": 0.003527254, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 25, "sum": 0.017119963, "avgtime": 0.000684798 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 288, "msgr_send_encrypted_bytes": 168272 } } ], "AsyncMessenger::Worker-2": [ { "labels": {}, "counters": { "msgr_recv_messages": 13, "msgr_send_messages": 6, "msgr_recv_bytes": 302800, "msgr_send_bytes": 928, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.011393171, "msgr_running_send_time": 0.001171413, "msgr_running_recv_time": 0.017320721, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 6, "sum": 0.003172173, "avgtime": 0.000528695 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 302800, "msgr_send_encrypted_bytes": 928 } } ], "cct": [ { "labels": {}, "counters": { "total_workers": 0, "unhealthy_workers": 0 } } ], "cephfs_mirror": [ { "labels": {}, "counters": { "mirrored_filesystems": 0, "mirror_enable_failures": 0 } } ], "mempool": [ { "labels": {}, "counters": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_items": 0, "bluefs_bytes": 0, "bluefs_items": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "buffer_anon_bytes": 148338, "buffer_anon_items": 24, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 4592, "osdmap_items": 37, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "mds_co_bytes": 0, "mds_co_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 } } ], "objecter": [ { "labels": {}, "counters": { "op_active": 0, "op_laggy": 0, "op_send": 0, "op_send_bytes": 0, "op_resend": 0, "op_reply": 0, "op_latency": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "op_inflight": 0, "oplen_avg": { "avgcount": 0, "sum": 0 }, "op": 0, "op_r": 0, "op_w": 0, "op_rmw": 0, "op_pg": 0, "osdop_stat": 0, "osdop_create": 0, "osdop_read": 0, "osdop_write": 0, "osdop_writefull": 0, "osdop_writesame": 0, "osdop_append": 0, "osdop_zero": 0, "osdop_truncate": 0, "osdop_delete": 0, "osdop_mapext": 0, "osdop_sparse_read": 0, "osdop_clonerange": 0, "osdop_getxattr": 0, "osdop_setxattr": 0, "osdop_cmpxattr": 0, "osdop_rmxattr": 0, "osdop_resetxattrs": 0, "osdop_call": 0, "osdop_watch": 0, "osdop_notify": 0, "osdop_src_cmpxattr": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_other": 0, "linger_active": 0, "linger_send": 0, "linger_resend": 0, "linger_ping": 0, "poolop_active": 0, "poolop_send": 0, "poolop_resend": 0, "poolstat_active": 0, "poolstat_send": 0, "poolstat_resend": 0, "statfs_active": 0, "statfs_send": 0, "statfs_resend": 0, "command_active": 0, "command_send": 0, "command_resend": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "osd_sessions": 0, "osd_session_open": 0, "osd_session_close": 0, "osd_laggy": 0, "omap_wr": 0, "omap_rd": 0, "omap_del": 0 } } ], "throttle-msgr_dispatch_throttler-client": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 21, "get_sum": 76449, "get_or_fail_fail": 0, "get_or_fail_success": 21, "take": 0, "take_sum": 0, "put": 21, "put_sum": 76449, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-msgr_dispatch_throttler-radosclient": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 12, "get_sum": 300554, "get_or_fail_fail": 0, "get_or_fail_success": 12, "take": 0, "take_sum": 0, "put": 12, "put_sum": 300554, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_bytes": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_ops": [ { "labels": {}, "counters": { "val": 0, "max": 1024, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ] } 2024-07-04T17:28:31.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:28:31.380 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -rf d1) 2024-07-04T17:28:31.533 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable mirroring 2024-07-04T17:28:31.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:31.823+0000 7f6f73200640 1 -- 172.21.15.42:0/3849079481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f6c0bdbf0 msgr2=0x7f6f6c0c36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:31.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:31.823+0000 7f6f73200640 1 --2- 172.21.15.42:0/3849079481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f6c0bdbf0 0x7f6f6c0c36d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0097e0 tx=0x7f6f5c02a560 comp rx=0 tx=0).stop 2024-07-04T17:28:31.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:31.824+0000 7f6f73200640 1 -- 172.21.15.42:0/3849079481 shutdown_connections 2024-07-04T17:28:31.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:31.824+0000 7f6f73200640 1 --2- 172.21.15.42:0/3849079481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f6c0bdbf0 0x7f6f6c0c36d0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:31.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:31.824+0000 7f6f73200640 1 -- 172.21.15.42:0/3849079481 >> 172.21.15.42:0/3849079481 conn(0x7f6f6c06e210 msgr2=0x7f6f6c06e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:31.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:31.824+0000 7f6f73200640 1 -- 172.21.15.42:0/3849079481 shutdown_connections 2024-07-04T17:28:31.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:31.824+0000 7f6f73200640 1 -- 172.21.15.42:0/3849079481 wait complete. 2024-07-04T17:28:31.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:31.826+0000 7f6f73200640 1 Processor -- start 2024-07-04T17:28:31.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:31.827+0000 7f6f73200640 1 -- start start 2024-07-04T17:28:31.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:31.828+0000 7f6f73200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f6c0bc500 0x7f6f6c0c2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:31.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:31.828+0000 7f6f73200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c0b7700 con 0x7f6f6c0bc500 2024-07-04T17:28:31.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:31.828+0000 7f6f71e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f6c0bc500 0x7f6f6c0c2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:31.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:31.829+0000 7f6f71e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f6c0bc500 0x7f6f6c0c2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56674/0 (socket says 172.21.15.42:56674) 2024-07-04T17:28:31.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:31.829+0000 7f6f71e00640 1 -- 172.21.15.42:0/227760025 learned_addr learned my addr 172.21.15.42:0/227760025 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:31.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:31.829+0000 7f6f71e00640 1 -- 172.21.15.42:0/227760025 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f6c0aa380 con 0x7f6f6c0bc500 2024-07-04T17:28:31.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:31.829+0000 7f6f71e00640 1 --2- 172.21.15.42:0/227760025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f6c0bc500 0x7f6f6c0c2f90 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6f60001eb0 tx=0x7f6f6000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:31.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:31.830+0000 7f6f66000640 1 -- 172.21.15.42:0/227760025 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f600116c0 con 0x7f6f6c0bc500 2024-07-04T17:28:31.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:31.830+0000 7f6f66000640 1 -- 172.21.15.42:0/227760025 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f60017070 con 0x7f6f6c0bc500 2024-07-04T17:28:31.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:31.830+0000 7f6f66000640 1 -- 172.21.15.42:0/227760025 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f60011ae0 con 0x7f6f6c0bc500 2024-07-04T17:28:31.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:31.830+0000 7f6f73200640 1 -- 172.21.15.42:0/227760025 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f6c0abe20 con 0x7f6f6c0bc500 2024-07-04T17:28:31.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:31.830+0000 7f6f73200640 1 -- 172.21.15.42:0/227760025 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f6c0aa860 con 0x7f6f6c0bc500 2024-07-04T17:28:31.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:31.831+0000 7f6f66000640 1 -- 172.21.15.42:0/227760025 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 10) ==== 49681+0+0 (secure 0 0 0) 0x7f6f60011c80 con 0x7f6f6c0bc500 2024-07-04T17:28:31.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:31.831+0000 7f6f73200640 1 -- 172.21.15.42:0/227760025 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f6c0b7e30 con 0x7f6f6c0bc500 2024-07-04T17:28:31.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:31.832+0000 7f6f66000640 1 --2- 172.21.15.42:0/227760025 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f6f5003d5d0 0x7f6f5003fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:31.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:31.832+0000 7f6f66000640 1 -- 172.21.15.42:0/227760025 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4604+0+0 (secure 0 0 0) 0x7f6f60051b30 con 0x7f6f6c0bc500 2024-07-04T17:28:31.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:31.832+0000 7f6f71400640 1 --2- 172.21.15.42:0/227760025 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f6f5003d5d0 0x7f6f5003fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:31.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:31.833+0000 7f6f71400640 1 --2- 172.21.15.42:0/227760025 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f6f5003d5d0 0x7f6f5003fae0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c034000 tx=0x7f6f5c004730 comp rx=0 tx=0).ready entity=mgr.4563 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:31.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:31.838+0000 7f6f66000640 1 -- 172.21.15.42:0/227760025 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f6f6c0bc910 con 0x7f6f6c0bc500 2024-07-04T17:28:32.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:32.114+0000 7f6f73200640 1 -- 172.21.15.42:0/227760025 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "mirroring"} v 0) -- 0x7f6f6c068e30 con 0x7f6f6c0bc500 2024-07-04T17:28:32.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:32.826+0000 7f6f66000640 1 -- 172.21.15.42:0/227760025 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "mirroring"}]=0 v11) ==== 89+0+0 (secure 0 0 0) 0x7f6f6000a140 con 0x7f6f6c0bc500 2024-07-04T17:28:32.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:32.830+0000 7f6f73200640 1 -- 172.21.15.42:0/227760025 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f6f5003d5d0 msgr2=0x7f6f5003fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:32.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:32.830+0000 7f6f73200640 1 --2- 172.21.15.42:0/227760025 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f6f5003d5d0 0x7f6f5003fae0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c034000 tx=0x7f6f5c004730 comp rx=0 tx=0).stop 2024-07-04T17:28:32.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:32.830+0000 7f6f73200640 1 -- 172.21.15.42:0/227760025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f6c0bc500 msgr2=0x7f6f6c0c2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:32.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:32.830+0000 7f6f73200640 1 --2- 172.21.15.42:0/227760025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f6c0bc500 0x7f6f6c0c2f90 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6f60001eb0 tx=0x7f6f6000c810 comp rx=0 tx=0).stop 2024-07-04T17:28:32.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:32.831+0000 7f6f73200640 1 -- 172.21.15.42:0/227760025 shutdown_connections 2024-07-04T17:28:32.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:32.831+0000 7f6f73200640 1 --2- 172.21.15.42:0/227760025 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f6f5003d5d0 0x7f6f5003fae0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:32.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:32.831+0000 7f6f73200640 1 --2- 172.21.15.42:0/227760025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f6c0bc500 0x7f6f6c0c2f90 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:32.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:32.831+0000 7f6f73200640 1 -- 172.21.15.42:0/227760025 >> 172.21.15.42:0/227760025 conn(0x7f6f6c06e210 msgr2=0x7f6f6c0740b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:32.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:32.832+0000 7f6f73200640 1 -- 172.21.15.42:0/227760025 shutdown_connections 2024-07-04T17:28:32.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:32.832+0000 7f6f73200640 1 -- 172.21.15.42:0/227760025 wait complete. 2024-07-04T17:28:32.867 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-04T17:28:32.867 DEBUG:teuthology.orchestra.run.smithi042:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-04T17:28:32.962 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-04T17:28:32.962 INFO:tasks.cephfs.mount:Terminating background process 2024-07-04T17:28:32.962 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:28:32.962 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:28:33.013 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:28:33.013 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:28:33.059 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-04T17:28:33.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.367+0000 7f6715e00640 1 -- 172.21.15.42:0/49078447 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67100bbef0 msgr2=0x7f67101414a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:33.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.367+0000 7f6715e00640 1 --2- 172.21.15.42:0/49078447 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67100bbef0 0x7f67101414a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f67040097b0 tx=0x7f670402a540 comp rx=0 tx=0).stop 2024-07-04T17:28:33.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.367+0000 7f6715e00640 1 -- 172.21.15.42:0/49078447 shutdown_connections 2024-07-04T17:28:33.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.367+0000 7f6715e00640 1 --2- 172.21.15.42:0/49078447 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67100bbef0 0x7f67101414a0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:33.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.367+0000 7f6715e00640 1 -- 172.21.15.42:0/49078447 >> 172.21.15.42:0/49078447 conn(0x7f67100688a0 msgr2=0x7f671006db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:33.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.367+0000 7f6715e00640 1 -- 172.21.15.42:0/49078447 shutdown_connections 2024-07-04T17:28:33.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.367+0000 7f6715e00640 1 -- 172.21.15.42:0/49078447 wait complete. 2024-07-04T17:28:33.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.370+0000 7f6715e00640 1 Processor -- start 2024-07-04T17:28:33.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.370+0000 7f6715e00640 1 -- start start 2024-07-04T17:28:33.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.371+0000 7f6715e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67100bbef0 0x7f671013c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:33.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.371+0000 7f6715e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f67100b79a0 con 0x7f67100bbef0 2024-07-04T17:28:33.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.371+0000 7f6714a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67100bbef0 0x7f671013c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:33.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.371+0000 7f6714a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67100bbef0 0x7f671013c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56728/0 (socket says 172.21.15.42:56728) 2024-07-04T17:28:33.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.371+0000 7f6714a00640 1 -- 172.21.15.42:0/4252533568 learned_addr learned my addr 172.21.15.42:0/4252533568 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:33.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.372+0000 7f6714a00640 1 -- 172.21.15.42:0/4252533568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671013c0a0 con 0x7f67100bbef0 2024-07-04T17:28:33.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.372+0000 7f6714a00640 1 --2- 172.21.15.42:0/4252533568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67100bbef0 0x7f671013c5c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f67040098d0 tx=0x7f6704033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:33.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.372+0000 7f670cc00640 1 -- 172.21.15.42:0/4252533568 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6704031bc0 con 0x7f67100bbef0 2024-07-04T17:28:33.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.372+0000 7f670cc00640 1 -- 172.21.15.42:0/4252533568 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6704031e50 con 0x7f67100bbef0 2024-07-04T17:28:33.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.372+0000 7f670cc00640 1 -- 172.21.15.42:0/4252533568 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6704033260 con 0x7f67100bbef0 2024-07-04T17:28:33.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.373+0000 7f6715e00640 1 -- 172.21.15.42:0/4252533568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67101e38e0 con 0x7f67100bbef0 2024-07-04T17:28:33.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.373+0000 7f6715e00640 1 -- 172.21.15.42:0/4252533568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f671013cce0 con 0x7f67100bbef0 2024-07-04T17:28:33.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.374+0000 7f670cc00640 1 -- 172.21.15.42:0/4252533568 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 11) ==== 49668+0+0 (secure 0 0 0) 0x7f6704033400 con 0x7f67100bbef0 2024-07-04T17:28:33.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.374+0000 7f6715e00640 1 -- 172.21.15.42:0/4252533568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67100b8020 con 0x7f67100bbef0 2024-07-04T17:28:33.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.375+0000 7f670cc00640 1 --2- 172.21.15.42:0/4252533568 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f66f003d580 0x7f66f003fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:33.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.375+0000 7f670cc00640 1 -- 172.21.15.42:0/4252533568 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4604+0+0 (secure 0 0 0) 0x7f6704073400 con 0x7f67100bbef0 2024-07-04T17:28:33.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.375+0000 7f670fe00640 1 -- 172.21.15.42:0/4252533568 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f66f003d580 msgr2=0x7f66f003fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/488387844 2024-07-04T17:28:33.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.375+0000 7f670fe00640 1 --2- 172.21.15.42:0/4252533568 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f66f003d580 0x7f66f003fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:28:33.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.384+0000 7f670cc00640 1 -- 172.21.15.42:0/4252533568 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f670403f030 con 0x7f67100bbef0 2024-07-04T17:28:33.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.576+0000 7f670fe00640 1 -- 172.21.15.42:0/4252533568 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f66f003d580 msgr2=0x7f66f003fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/488387844 2024-07-04T17:28:33.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.577+0000 7f670fe00640 1 --2- 172.21.15.42:0/4252533568 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f66f003d580 0x7f66f003fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:28:33.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.607+0000 7f6715e00640 1 -- 172.21.15.42:0/4252533568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6710068e30 con 0x7f67100bbef0 2024-07-04T17:28:33.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.608+0000 7f670cc00640 1 -- 172.21.15.42:0/4252533568 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 7 entries v33) ==== 80+0+408 (secure 0 0 0) 0x7f67040035f0 con 0x7f67100bbef0 2024-07-04T17:28:33.609 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3526015921 2024-07-05T17:27:40.654396+0000 2024-07-04T17:28:33.609 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/83074206 2024-07-05T17:27:40.654396+0000 2024-07-04T17:28:33.609 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/880857241 2024-07-05T17:27:40.654396+0000 2024-07-04T17:28:33.609 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2974639683 2024-07-05T17:27:40.654396+0000 2024-07-04T17:28:33.610 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/487218478 2024-07-04T18:28:21.818727+0000 2024-07-04T17:28:33.610 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6825/3523793950 2024-07-05T17:27:40.654396+0000 2024-07-04T17:28:33.610 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6824/3523793950 2024-07-05T17:27:40.654396+0000 2024-07-04T17:28:33.610 INFO:teuthology.orchestra.run.smithi042.stderr:listed 7 entries 2024-07-04T17:28:33.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.610+0000 7f6715e00640 1 -- 172.21.15.42:0/4252533568 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f66f003d580 msgr2=0x7f66f003fa90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:28:33.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.610+0000 7f6715e00640 1 --2- 172.21.15.42:0/4252533568 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f66f003d580 0x7f66f003fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:33.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.610+0000 7f6715e00640 1 -- 172.21.15.42:0/4252533568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67100bbef0 msgr2=0x7f671013c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:33.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.610+0000 7f6715e00640 1 --2- 172.21.15.42:0/4252533568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67100bbef0 0x7f671013c5c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f67040098d0 tx=0x7f6704033e40 comp rx=0 tx=0).stop 2024-07-04T17:28:33.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.611+0000 7f6715e00640 1 -- 172.21.15.42:0/4252533568 shutdown_connections 2024-07-04T17:28:33.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.611+0000 7f6715e00640 1 --2- 172.21.15.42:0/4252533568 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f66f003d580 0x7f66f003fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:33.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.611+0000 7f6715e00640 1 --2- 172.21.15.42:0/4252533568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67100bbef0 0x7f671013c5c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:33.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.611+0000 7f6715e00640 1 -- 172.21.15.42:0/4252533568 >> 172.21.15.42:0/4252533568 conn(0x7f67100688a0 msgr2=0x7f6710075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:33.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.611+0000 7f6715e00640 1 -- 172.21.15.42:0/4252533568 shutdown_connections 2024-07-04T17:28:33.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:33.611+0000 7f6715e00640 1 -- 172.21.15.42:0/4252533568 wait complete. 2024-07-04T17:28:33.636 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi042.front.sepia.ceph.com... 2024-07-04T17:28:33.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:28:33.637 DEBUG:teuthology.orchestra.run.smithi042:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-07-04T17:28:33.706 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[46040]: fuse finished with error 0 and tester_r 0 2024-07-04T17:28:33.710 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-07-04T17:28:33.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:28:33.710 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-04T17:28:33.778 INFO:tasks.cephfs.mount:Terminating background process 2024-07-04T17:28:33.778 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:28:33.778 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:28:33.870 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:28:33.870 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:28:33.964 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-04T17:28:34.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.232+0000 7fc5b5600640 1 -- 172.21.15.42:0/2538563108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5b00b37b0 msgr2=0x7fc5b00b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:34.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.232+0000 7fc5b5600640 1 --2- 172.21.15.42:0/2538563108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5b00b37b0 0x7fc5b00b01f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc5980097b0 tx=0x7fc59802a530 comp rx=0 tx=0).stop 2024-07-04T17:28:34.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.232+0000 7fc5b5600640 1 -- 172.21.15.42:0/2538563108 shutdown_connections 2024-07-04T17:28:34.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.232+0000 7fc5b5600640 1 --2- 172.21.15.42:0/2538563108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5b00b37b0 0x7fc5b00b01f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:34.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.232+0000 7fc5b5600640 1 -- 172.21.15.42:0/2538563108 >> 172.21.15.42:0/2538563108 conn(0x7fc5b00bf700 msgr2=0x7fc5b00af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:34.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.232+0000 7fc5b5600640 1 -- 172.21.15.42:0/2538563108 shutdown_connections 2024-07-04T17:28:34.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.232+0000 7fc5b5600640 1 -- 172.21.15.42:0/2538563108 wait complete. 2024-07-04T17:28:34.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.234+0000 7fc5b5600640 1 Processor -- start 2024-07-04T17:28:34.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.234+0000 7fc5b5600640 1 -- start start 2024-07-04T17:28:34.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.236+0000 7fc5b5600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5b00b2e00 0x7fc5b0141700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:34.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.236+0000 7fc5b5600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b006ce30 con 0x7fc5b00b2e00 2024-07-04T17:28:34.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.236+0000 7fc5aea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5b00b2e00 0x7fc5b0141700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:34.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.236+0000 7fc5aea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5b00b2e00 0x7fc5b0141700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37414/0 (socket says 172.21.15.42:37414) 2024-07-04T17:28:34.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.236+0000 7fc5aea00640 1 -- 172.21.15.42:0/1715163202 learned_addr learned my addr 172.21.15.42:0/1715163202 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:34.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.236+0000 7fc5aea00640 1 -- 172.21.15.42:0/1715163202 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b01d5440 con 0x7fc5b00b2e00 2024-07-04T17:28:34.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.237+0000 7fc5aea00640 1 --2- 172.21.15.42:0/1715163202 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5b00b2e00 0x7fc5b0141700 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc5a000c340 tx=0x7fc5a000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:34.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.237+0000 7fc5a6a00640 1 -- 172.21.15.42:0/1715163202 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5a00116c0 con 0x7fc5b00b2e00 2024-07-04T17:28:34.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.237+0000 7fc5a6a00640 1 -- 172.21.15.42:0/1715163202 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5a0017070 con 0x7fc5b00b2e00 2024-07-04T17:28:34.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.237+0000 7fc5b5600640 1 -- 172.21.15.42:0/1715163202 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5b01d6e90 con 0x7fc5b00b2e00 2024-07-04T17:28:34.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.237+0000 7fc5a6a00640 1 -- 172.21.15.42:0/1715163202 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5a0011b80 con 0x7fc5b00b2e00 2024-07-04T17:28:34.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.237+0000 7fc5b5600640 1 -- 172.21.15.42:0/1715163202 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5b01d79f0 con 0x7fc5b00b2e00 2024-07-04T17:28:34.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.239+0000 7fc5b5600640 1 -- 172.21.15.42:0/1715163202 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5b00b22b0 con 0x7fc5b00b2e00 2024-07-04T17:28:34.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.239+0000 7fc5a6a00640 1 -- 172.21.15.42:0/1715163202 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 11) ==== 49668+0+0 (secure 0 0 0) 0x7fc5a001c460 con 0x7fc5b00b2e00 2024-07-04T17:28:34.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.239+0000 7fc5a6a00640 1 --2- 172.21.15.42:0/1715163202 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7fc58c03d530 0x7fc58c03fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:34.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.239+0000 7fc5a6a00640 1 -- 172.21.15.42:0/1715163202 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4604+0+0 (secure 0 0 0) 0x7fc5a0051b30 con 0x7fc5b00b2e00 2024-07-04T17:28:34.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.239+0000 7fc5a7e00640 1 -- 172.21.15.42:0/1715163202 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7fc58c03d530 msgr2=0x7fc58c03fa40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/488387844 2024-07-04T17:28:34.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.239+0000 7fc5a7e00640 1 --2- 172.21.15.42:0/1715163202 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7fc58c03d530 0x7fc58c03fa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:28:34.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.249+0000 7fc5a6a00640 1 -- 172.21.15.42:0/1715163202 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fc5a001c800 con 0x7fc5b00b2e00 2024-07-04T17:28:34.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.440+0000 7fc5a7e00640 1 -- 172.21.15.42:0/1715163202 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7fc58c03d530 msgr2=0x7fc58c03fa40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/488387844 2024-07-04T17:28:34.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.440+0000 7fc5a7e00640 1 --2- 172.21.15.42:0/1715163202 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7fc58c03d530 0x7fc58c03fa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:28:34.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.471+0000 7fc5b5600640 1 -- 172.21.15.42:0/1715163202 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc5b006d440 con 0x7fc5b00b2e00 2024-07-04T17:28:34.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.472+0000 7fc5a6a00640 1 -- 172.21.15.42:0/1715163202 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 7 entries v33) ==== 80+0+408 (secure 0 0 0) 0x7fc5a0011d20 con 0x7fc5b00b2e00 2024-07-04T17:28:34.474 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3526015921 2024-07-05T17:27:40.654396+0000 2024-07-04T17:28:34.474 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/83074206 2024-07-05T17:27:40.654396+0000 2024-07-04T17:28:34.474 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/880857241 2024-07-05T17:27:40.654396+0000 2024-07-04T17:28:34.474 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2974639683 2024-07-05T17:27:40.654396+0000 2024-07-04T17:28:34.474 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/487218478 2024-07-04T18:28:21.818727+0000 2024-07-04T17:28:34.474 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6825/3523793950 2024-07-05T17:27:40.654396+0000 2024-07-04T17:28:34.474 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6824/3523793950 2024-07-05T17:27:40.654396+0000 2024-07-04T17:28:34.474 INFO:teuthology.orchestra.run.smithi042.stderr:listed 7 entries 2024-07-04T17:28:34.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.475+0000 7fc5b5600640 1 -- 172.21.15.42:0/1715163202 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7fc58c03d530 msgr2=0x7fc58c03fa40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:28:34.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.475+0000 7fc5b5600640 1 --2- 172.21.15.42:0/1715163202 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7fc58c03d530 0x7fc58c03fa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:34.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.475+0000 7fc5b5600640 1 -- 172.21.15.42:0/1715163202 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5b00b2e00 msgr2=0x7fc5b0141700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:34.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.475+0000 7fc5b5600640 1 --2- 172.21.15.42:0/1715163202 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5b00b2e00 0x7fc5b0141700 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc5a000c340 tx=0x7fc5a000c800 comp rx=0 tx=0).stop 2024-07-04T17:28:34.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.475+0000 7fc5b5600640 1 -- 172.21.15.42:0/1715163202 shutdown_connections 2024-07-04T17:28:34.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.475+0000 7fc5b5600640 1 --2- 172.21.15.42:0/1715163202 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7fc58c03d530 0x7fc58c03fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:34.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.475+0000 7fc5b5600640 1 --2- 172.21.15.42:0/1715163202 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5b00b2e00 0x7fc5b0141700 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:34.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.475+0000 7fc5b5600640 1 -- 172.21.15.42:0/1715163202 >> 172.21.15.42:0/1715163202 conn(0x7fc5b00bf700 msgr2=0x7fc5b00b1180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:34.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.475+0000 7fc5b5600640 1 -- 172.21.15.42:0/1715163202 shutdown_connections 2024-07-04T17:28:34.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.475+0000 7fc5b5600640 1 -- 172.21.15.42:0/1715163202 wait complete. 2024-07-04T17:28:34.504 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi042.front.sepia.ceph.com... 2024-07-04T17:28:34.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:28:34.504 DEBUG:teuthology.orchestra.run.smithi042:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-07-04T17:28:34.584 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi042.stderr:ceph-fuse[46509]: fuse finished with error 0 and tester_r 0 2024-07-04T17:28:34.589 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-07-04T17:28:34.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:28:34.589 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-07-04T17:28:34.634 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:28:34.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.953+0000 7f8e13400640 1 -- 172.21.15.42:0/1813854094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e0c06cea0 msgr2=0x7f8e0c0b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:34.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.953+0000 7f8e13400640 1 --2- 172.21.15.42:0/1813854094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e0c06cea0 0x7f8e0c0b3030 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8e00005fd0 tx=0x7f8e0002ae80 comp rx=0 tx=0).stop 2024-07-04T17:28:34.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.953+0000 7f8e13400640 1 -- 172.21.15.42:0/1813854094 shutdown_connections 2024-07-04T17:28:34.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.953+0000 7f8e13400640 1 --2- 172.21.15.42:0/1813854094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e0c06cea0 0x7f8e0c0b3030 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:34.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.953+0000 7f8e13400640 1 -- 172.21.15.42:0/1813854094 >> 172.21.15.42:0/1813854094 conn(0x7f8e0c0bf700 msgr2=0x7f8e0c0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:34.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.953+0000 7f8e13400640 1 -- 172.21.15.42:0/1813854094 shutdown_connections 2024-07-04T17:28:34.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.953+0000 7f8e13400640 1 -- 172.21.15.42:0/1813854094 wait complete. 2024-07-04T17:28:34.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.955+0000 7f8e13400640 1 Processor -- start 2024-07-04T17:28:34.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.955+0000 7f8e13400640 1 -- start start 2024-07-04T17:28:34.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.956+0000 7f8e13400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e0c06cea0 0x7f8e0c1d72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:34.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.956+0000 7f8e13400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c06ef50 con 0x7f8e0c06cea0 2024-07-04T17:28:34.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.957+0000 7f8e10a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e0c06cea0 0x7f8e0c1d72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:34.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.957+0000 7f8e10a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e0c06cea0 0x7f8e0c1d72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37434/0 (socket says 172.21.15.42:37434) 2024-07-04T17:28:34.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.957+0000 7f8e10a00640 1 -- 172.21.15.42:0/3388768188 learned_addr learned my addr 172.21.15.42:0/3388768188 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:34.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.957+0000 7f8e10a00640 1 -- 172.21.15.42:0/3388768188 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0c0ae3e0 con 0x7f8e0c06cea0 2024-07-04T17:28:34.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.957+0000 7f8e10a00640 1 --2- 172.21.15.42:0/3388768188 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e0c06cea0 0x7f8e0c1d72e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8e00035040 tx=0x7f8e00004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:34.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.958+0000 7f8e08c00640 1 -- 172.21.15.42:0/3388768188 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e00041020 con 0x7f8e0c06cea0 2024-07-04T17:28:34.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.958+0000 7f8e08c00640 1 -- 172.21.15.42:0/3388768188 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e000094c0 con 0x7f8e0c06cea0 2024-07-04T17:28:34.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.958+0000 7f8e13400640 1 -- 172.21.15.42:0/3388768188 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e0c071890 con 0x7f8e0c06cea0 2024-07-04T17:28:34.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.958+0000 7f8e08c00640 1 -- 172.21.15.42:0/3388768188 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e00009840 con 0x7f8e0c06cea0 2024-07-04T17:28:34.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.958+0000 7f8e13400640 1 -- 172.21.15.42:0/3388768188 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e0c071f10 con 0x7f8e0c06cea0 2024-07-04T17:28:34.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.959+0000 7f8e13400640 1 -- 172.21.15.42:0/3388768188 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e0c0b3030 con 0x7f8e0c06cea0 2024-07-04T17:28:34.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.959+0000 7f8e08c00640 1 -- 172.21.15.42:0/3388768188 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 11) ==== 49668+0+0 (secure 0 0 0) 0x7f8e00044020 con 0x7f8e0c06cea0 2024-07-04T17:28:34.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.960+0000 7f8e08c00640 1 --2- 172.21.15.42:0/3388768188 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f8dec03d580 0x7f8dec03fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:34.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.960+0000 7f8e08c00640 1 -- 172.21.15.42:0/3388768188 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4604+0+0 (secure 0 0 0) 0x7f8e00039040 con 0x7f8e0c06cea0 2024-07-04T17:28:34.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.960+0000 7f8e0be00640 1 -- 172.21.15.42:0/3388768188 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f8dec03d580 msgr2=0x7f8dec03fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/488387844 2024-07-04T17:28:34.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.960+0000 7f8e0be00640 1 --2- 172.21.15.42:0/3388768188 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f8dec03d580 0x7f8dec03fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:28:34.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:34.969+0000 7f8e08c00640 1 -- 172.21.15.42:0/3388768188 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f8e0004b030 con 0x7f8e0c06cea0 2024-07-04T17:28:35.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.161+0000 7f8e0be00640 1 -- 172.21.15.42:0/3388768188 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f8dec03d580 msgr2=0x7f8dec03fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/488387844 2024-07-04T17:28:35.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.161+0000 7f8e0be00640 1 --2- 172.21.15.42:0/3388768188 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f8dec03d580 0x7f8dec03fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:28:35.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.218+0000 7f8e13400640 1 -- 172.21.15.42:0/3388768188 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e0c071a60 con 0x7f8e0c06cea0 2024-07-04T17:28:35.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.219+0000 7f8e08c00640 1 -- 172.21.15.42:0/3388768188 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+7359 (secure 0 0 0) 0x7f8e0c0ae3e0 con 0x7f8e0c06cea0 2024-07-04T17:28:35.221 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:28:35.221 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":25,"btime":"2024-07-04T17:28:20:786366+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/399707138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":399707138},{"type":"v1","addr":"172.21.15.42:6827","nonce":399707138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4414,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1510714231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1510714231},{"type":"v1","addr":"172.21.15.42:6829","nonce":1510714231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/3069680926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":3069680926},{"type":"v1","addr":"172.21.15.42:6831","nonce":3069680926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:27:07.402007+0000","modified":"2024-07-04T17:28:19.916603+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4423":{"gid":4423,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.42:6835/807364821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":807364821},{"type":"v1","addr":"172.21.15.42:6835","nonce":807364821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4423,"qdb_cluster":[4423]},"id":2},{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:27:27.513977+0000","modified":"2024-07-04T17:27:30.569774+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4422":{"gid":4422,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.42:6833/1348446958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1348446958},{"type":"v1","addr":"172.21.15.42:6833","nonce":1348446958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4422,"qdb_cluster":[4422]},"id":3}]} 2024-07-04T17:28:35.222 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 25 2024-07-04T17:28:35.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.222+0000 7f8e13400640 1 -- 172.21.15.42:0/3388768188 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f8dec03d580 msgr2=0x7f8dec03fa90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:28:35.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.222+0000 7f8e13400640 1 --2- 172.21.15.42:0/3388768188 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f8dec03d580 0x7f8dec03fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:35.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.222+0000 7f8e13400640 1 -- 172.21.15.42:0/3388768188 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e0c06cea0 msgr2=0x7f8e0c1d72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:35.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.222+0000 7f8e13400640 1 --2- 172.21.15.42:0/3388768188 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e0c06cea0 0x7f8e0c1d72e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8e00035040 tx=0x7f8e00004250 comp rx=0 tx=0).stop 2024-07-04T17:28:35.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.222+0000 7f8e13400640 1 -- 172.21.15.42:0/3388768188 shutdown_connections 2024-07-04T17:28:35.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.222+0000 7f8e13400640 1 --2- 172.21.15.42:0/3388768188 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f8dec03d580 0x7f8dec03fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:35.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.222+0000 7f8e13400640 1 --2- 172.21.15.42:0/3388768188 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e0c06cea0 0x7f8e0c1d72e0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:35.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.222+0000 7f8e13400640 1 -- 172.21.15.42:0/3388768188 >> 172.21.15.42:0/3388768188 conn(0x7f8e0c0bf700 msgr2=0x7f8e0c0aa3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:35.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.222+0000 7f8e13400640 1 -- 172.21.15.42:0/3388768188 shutdown_connections 2024-07-04T17:28:35.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.222+0000 7f8e13400640 1 -- 172.21.15.42:0/3388768188 wait complete. 2024-07-04T17:28:35.248 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:28:35.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.521+0000 7f6360e00640 1 -- 172.21.15.42:0/2596490168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f635c0b2e00 msgr2=0x7f635c0b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:35.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.521+0000 7f6360e00640 1 --2- 172.21.15.42:0/2596490168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f635c0b2e00 0x7f635c0b37b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f634c005fd0 tx=0x7f634c02a5f0 comp rx=0 tx=0).stop 2024-07-04T17:28:35.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.521+0000 7f6360e00640 1 -- 172.21.15.42:0/2596490168 shutdown_connections 2024-07-04T17:28:35.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.521+0000 7f6360e00640 1 --2- 172.21.15.42:0/2596490168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f635c0b2e00 0x7f635c0b37b0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:35.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.521+0000 7f6360e00640 1 -- 172.21.15.42:0/2596490168 >> 172.21.15.42:0/2596490168 conn(0x7f635c0bf700 msgr2=0x7f635c0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:35.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.522+0000 7f6360e00640 1 -- 172.21.15.42:0/2596490168 shutdown_connections 2024-07-04T17:28:35.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.522+0000 7f6360e00640 1 -- 172.21.15.42:0/2596490168 wait complete. 2024-07-04T17:28:35.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.525+0000 7f6360e00640 1 Processor -- start 2024-07-04T17:28:35.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.525+0000 7f6360e00640 1 -- start start 2024-07-04T17:28:35.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.527+0000 7f6360e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f635c0b01f0 0x7f635c0ac9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:35.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.527+0000 7f6360e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f635c06ce30 con 0x7f635c0b01f0 2024-07-04T17:28:35.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.528+0000 7f6359e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f635c0b01f0 0x7f635c0ac9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:35.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.528+0000 7f6359e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f635c0b01f0 0x7f635c0ac9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37458/0 (socket says 172.21.15.42:37458) 2024-07-04T17:28:35.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.528+0000 7f6359e00640 1 -- 172.21.15.42:0/2523115984 learned_addr learned my addr 172.21.15.42:0/2523115984 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:35.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.528+0000 7f6359e00640 1 -- 172.21.15.42:0/2523115984 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f635c1d6f60 con 0x7f635c0b01f0 2024-07-04T17:28:35.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.528+0000 7f6359e00640 1 --2- 172.21.15.42:0/2523115984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f635c0b01f0 0x7f635c0ac9e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f634c034040 tx=0x7f634c0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:35.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.529+0000 7f6352000640 1 -- 172.21.15.42:0/2523115984 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f634c03e030 con 0x7f635c0b01f0 2024-07-04T17:28:35.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.529+0000 7f6352000640 1 -- 172.21.15.42:0/2523115984 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f634c0035f0 con 0x7f635c0b01f0 2024-07-04T17:28:35.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.529+0000 7f6352000640 1 -- 172.21.15.42:0/2523115984 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f634c035040 con 0x7f635c0b01f0 2024-07-04T17:28:35.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.529+0000 7f6360e00640 1 -- 172.21.15.42:0/2523115984 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f635c1d6bd0 con 0x7f635c0b01f0 2024-07-04T17:28:35.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.529+0000 7f6360e00640 1 -- 172.21.15.42:0/2523115984 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f635c0ad100 con 0x7f635c0b01f0 2024-07-04T17:28:35.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.530+0000 7f6360e00640 1 -- 172.21.15.42:0/2523115984 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f635c06d4e0 con 0x7f635c0b01f0 2024-07-04T17:28:35.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.530+0000 7f6352000640 1 -- 172.21.15.42:0/2523115984 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 11) ==== 49668+0+0 (secure 0 0 0) 0x7f634c041030 con 0x7f635c0b01f0 2024-07-04T17:28:35.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.531+0000 7f6352000640 1 --2- 172.21.15.42:0/2523115984 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f633c03d580 0x7f633c03fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:35.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.531+0000 7f6352000640 1 -- 172.21.15.42:0/2523115984 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4604+0+0 (secure 0 0 0) 0x7f634c03f070 con 0x7f635c0b01f0 2024-07-04T17:28:35.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.531+0000 7f6359400640 1 -- 172.21.15.42:0/2523115984 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f633c03d580 msgr2=0x7f633c03fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/488387844 2024-07-04T17:28:35.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.531+0000 7f6359400640 1 --2- 172.21.15.42:0/2523115984 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f633c03d580 0x7f633c03fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:28:35.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.536+0000 7f6352000640 1 -- 172.21.15.42:0/2523115984 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f634c04bd60 con 0x7f635c0b01f0 2024-07-04T17:28:35.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.732+0000 7f6359400640 1 -- 172.21.15.42:0/2523115984 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f633c03d580 msgr2=0x7f633c03fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/488387844 2024-07-04T17:28:35.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.732+0000 7f6359400640 1 --2- 172.21.15.42:0/2523115984 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f633c03d580 0x7f633c03fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:28:35.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.780+0000 7f6360e00640 1 -- 172.21.15.42:0/2523115984 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f635c142240 con 0x7f635c0b01f0 2024-07-04T17:28:35.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.781+0000 7f6352000640 1 -- 172.21.15.42:0/2523115984 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+7359 (secure 0 0 0) 0x7f634c031400 con 0x7f635c0b01f0 2024-07-04T17:28:35.783 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:28:35.783 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":25,"btime":"2024-07-04T17:28:20:786366+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/399707138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":399707138},{"type":"v1","addr":"172.21.15.42:6827","nonce":399707138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4414,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1510714231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1510714231},{"type":"v1","addr":"172.21.15.42:6829","nonce":1510714231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/3069680926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":3069680926},{"type":"v1","addr":"172.21.15.42:6831","nonce":3069680926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:27:07.402007+0000","modified":"2024-07-04T17:28:19.916603+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4423":{"gid":4423,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.42:6835/807364821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":807364821},{"type":"v1","addr":"172.21.15.42:6835","nonce":807364821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4423,"qdb_cluster":[4423]},"id":2},{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:27:27.513977+0000","modified":"2024-07-04T17:27:30.569774+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4422":{"gid":4422,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.42:6833/1348446958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1348446958},{"type":"v1","addr":"172.21.15.42:6833","nonce":1348446958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4422,"qdb_cluster":[4422]},"id":3}]} 2024-07-04T17:28:35.783 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 25 2024-07-04T17:28:35.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.784+0000 7f6360e00640 1 -- 172.21.15.42:0/2523115984 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f633c03d580 msgr2=0x7f633c03fa90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:28:35.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.784+0000 7f6360e00640 1 --2- 172.21.15.42:0/2523115984 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f633c03d580 0x7f633c03fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:35.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.784+0000 7f6360e00640 1 -- 172.21.15.42:0/2523115984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f635c0b01f0 msgr2=0x7f635c0ac9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:35.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.784+0000 7f6360e00640 1 --2- 172.21.15.42:0/2523115984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f635c0b01f0 0x7f635c0ac9e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f634c034040 tx=0x7f634c0049d0 comp rx=0 tx=0).stop 2024-07-04T17:28:35.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.784+0000 7f6360e00640 1 -- 172.21.15.42:0/2523115984 shutdown_connections 2024-07-04T17:28:35.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.784+0000 7f6360e00640 1 --2- 172.21.15.42:0/2523115984 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f633c03d580 0x7f633c03fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:35.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.784+0000 7f6360e00640 1 --2- 172.21.15.42:0/2523115984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f635c0b01f0 0x7f635c0ac9e0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:35.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.784+0000 7f6360e00640 1 -- 172.21.15.42:0/2523115984 >> 172.21.15.42:0/2523115984 conn(0x7f635c0bf700 msgr2=0x7f635c0be730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:35.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.784+0000 7f6360e00640 1 -- 172.21.15.42:0/2523115984 shutdown_connections 2024-07-04T17:28:35.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:35.784+0000 7f6360e00640 1 -- 172.21.15.42:0/2523115984 wait complete. 2024-07-04T17:28:35.811 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:28:36.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.079+0000 7f5d8a200640 1 -- 172.21.15.42:0/1457693459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d8406cc80 msgr2=0x7f5d840b3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:36.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.079+0000 7f5d8a200640 1 --2- 172.21.15.42:0/1457693459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d8406cc80 0x7f5d840b3650 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5d70005fd0 tx=0x7f5d7002a5f0 comp rx=0 tx=0).stop 2024-07-04T17:28:36.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.079+0000 7f5d8a200640 1 -- 172.21.15.42:0/1457693459 shutdown_connections 2024-07-04T17:28:36.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.079+0000 7f5d8a200640 1 --2- 172.21.15.42:0/1457693459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d8406cc80 0x7f5d840b3650 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:36.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.079+0000 7f5d8a200640 1 -- 172.21.15.42:0/1457693459 >> 172.21.15.42:0/1457693459 conn(0x7f5d840bf700 msgr2=0x7f5d840af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:36.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.080+0000 7f5d8a200640 1 -- 172.21.15.42:0/1457693459 shutdown_connections 2024-07-04T17:28:36.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.080+0000 7f5d8a200640 1 -- 172.21.15.42:0/1457693459 wait complete. 2024-07-04T17:28:36.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.082+0000 7f5d8a200640 1 Processor -- start 2024-07-04T17:28:36.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.082+0000 7f5d8a200640 1 -- start start 2024-07-04T17:28:36.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.084+0000 7f5d8a200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d8406cc80 0x7f5d841da9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:36.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.084+0000 7f5d8a200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5d841463b0 con 0x7f5d8406cc80 2024-07-04T17:28:36.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.084+0000 7f5d83400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d8406cc80 0x7f5d841da9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:36.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.084+0000 7f5d83400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d8406cc80 0x7f5d841da9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37488/0 (socket says 172.21.15.42:37488) 2024-07-04T17:28:36.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.084+0000 7f5d83400640 1 -- 172.21.15.42:0/2562451018 learned_addr learned my addr 172.21.15.42:0/2562451018 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:36.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.085+0000 7f5d83400640 1 -- 172.21.15.42:0/2562451018 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d840b6580 con 0x7f5d8406cc80 2024-07-04T17:28:36.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.085+0000 7f5d83400640 1 --2- 172.21.15.42:0/2562451018 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d8406cc80 0x7f5d841da9c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5d7400c380 tx=0x7f5d7400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:36.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.085+0000 7f5d7b400640 1 -- 172.21.15.42:0/2562451018 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d74017070 con 0x7f5d8406cc80 2024-07-04T17:28:36.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.085+0000 7f5d7b400640 1 -- 172.21.15.42:0/2562451018 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d740052f0 con 0x7f5d8406cc80 2024-07-04T17:28:36.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.085+0000 7f5d7b400640 1 -- 172.21.15.42:0/2562451018 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d740035f0 con 0x7f5d8406cc80 2024-07-04T17:28:36.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.085+0000 7f5d8a200640 1 -- 172.21.15.42:0/2562451018 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d84143810 con 0x7f5d8406cc80 2024-07-04T17:28:36.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.086+0000 7f5d8a200640 1 -- 172.21.15.42:0/2562451018 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d84143310 con 0x7f5d8406cc80 2024-07-04T17:28:36.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.087+0000 7f5d7b400640 1 -- 172.21.15.42:0/2562451018 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 11) ==== 49668+0+0 (secure 0 0 0) 0x7f5d7401a070 con 0x7f5d8406cc80 2024-07-04T17:28:36.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.087+0000 7f5d8a200640 1 -- 172.21.15.42:0/2562451018 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d84143a40 con 0x7f5d8406cc80 2024-07-04T17:28:36.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.087+0000 7f5d7b400640 1 --2- 172.21.15.42:0/2562451018 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f5d6403d580 0x7f5d6403fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:36.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.088+0000 7f5d7b400640 1 -- 172.21.15.42:0/2562451018 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4604+0+0 (secure 0 0 0) 0x7f5d74051190 con 0x7f5d8406cc80 2024-07-04T17:28:36.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.088+0000 7f5d82a00640 1 -- 172.21.15.42:0/2562451018 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f5d6403d580 msgr2=0x7f5d6403fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/488387844 2024-07-04T17:28:36.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.088+0000 7f5d82a00640 1 --2- 172.21.15.42:0/2562451018 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f5d6403d580 0x7f5d6403fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:28:36.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.093+0000 7f5d7b400640 1 -- 172.21.15.42:0/2562451018 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f5d840b06c0 con 0x7f5d8406cc80 2024-07-04T17:28:36.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.289+0000 7f5d82a00640 1 -- 172.21.15.42:0/2562451018 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f5d6403d580 msgr2=0x7f5d6403fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/488387844 2024-07-04T17:28:36.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.289+0000 7f5d82a00640 1 --2- 172.21.15.42:0/2562451018 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f5d6403d580 0x7f5d6403fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:28:36.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.303+0000 7f5d8a200640 1 -- 172.21.15.42:0/2562451018 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d840b3650 con 0x7f5d8406cc80 2024-07-04T17:28:36.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.306+0000 7f5d7b400640 1 -- 172.21.15.42:0/2562451018 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+14884 (secure 0 0 0) 0x7f5d74011630 con 0x7f5d8406cc80 2024-07-04T17:28:36.307 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:28:36.308 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":33,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:28:22.796980+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:27:04.932839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-07-04T17:27:06.393713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.0800000429153442,"score_stable":1.0800000429153442,"optimal_score":1,"raw_score_acting":1.0800000429153442,"raw_score_stable":1.0800000429153442,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:27:25.040056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:27:26.487236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/3526015921":"2024-07-05T17:27:40.654396+0000","172.21.15.42:0/83074206":"2024-07-05T17:27:40.654396+0000","172.21.15.42:0/880857241":"2024-07-05T17:27:40.654396+0000","172.21.15.42:0/2974639683":"2024-07-05T17:27:40.654396+0000","172.21.15.42:0/487218478":"2024-07-04T18:28:21.818727+0000","172.21.15.42:6825/3523793950":"2024-07-05T17:27:40.654396+0000","172.21.15.42:6824/3523793950":"2024-07-05T17:27:40.654396+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:28:36.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.309+0000 7f5d8a200640 1 -- 172.21.15.42:0/2562451018 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f5d6403d580 msgr2=0x7f5d6403fa90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:28:36.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.309+0000 7f5d8a200640 1 --2- 172.21.15.42:0/2562451018 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f5d6403d580 0x7f5d6403fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:36.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.309+0000 7f5d8a200640 1 -- 172.21.15.42:0/2562451018 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d8406cc80 msgr2=0x7f5d841da9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:36.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.309+0000 7f5d8a200640 1 --2- 172.21.15.42:0/2562451018 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d8406cc80 0x7f5d841da9c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5d7400c380 tx=0x7f5d7400c840 comp rx=0 tx=0).stop 2024-07-04T17:28:36.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.309+0000 7f5d8a200640 1 -- 172.21.15.42:0/2562451018 shutdown_connections 2024-07-04T17:28:36.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.309+0000 7f5d8a200640 1 --2- 172.21.15.42:0/2562451018 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f5d6403d580 0x7f5d6403fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:36.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.309+0000 7f5d8a200640 1 --2- 172.21.15.42:0/2562451018 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d8406cc80 0x7f5d841da9c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:36.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.309+0000 7f5d8a200640 1 -- 172.21.15.42:0/2562451018 >> 172.21.15.42:0/2562451018 conn(0x7f5d840bf700 msgr2=0x7f5d84075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:36.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.309+0000 7f5d8a200640 1 -- 172.21.15.42:0/2562451018 shutdown_connections 2024-07-04T17:28:36.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.309+0000 7f5d8a200640 1 -- 172.21.15.42:0/2562451018 wait complete. 2024-07-04T17:28:36.336 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-04T17:28:36.336 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:28:36.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.595+0000 7f40d4000640 1 -- 172.21.15.42:0/3742337932 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40cc0b3030 msgr2=0x7f40cc0b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:36.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.595+0000 7f40d4000640 1 --2- 172.21.15.42:0/3742337932 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40cc0b3030 0x7f40cc0b01f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f40bc005fd0 tx=0x7f40bc02ae80 comp rx=0 tx=0).stop 2024-07-04T17:28:36.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.596+0000 7f40d4000640 1 -- 172.21.15.42:0/3742337932 shutdown_connections 2024-07-04T17:28:36.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.596+0000 7f40d4000640 1 --2- 172.21.15.42:0/3742337932 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40cc0b3030 0x7f40cc0b01f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:36.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.596+0000 7f40d4000640 1 -- 172.21.15.42:0/3742337932 >> 172.21.15.42:0/3742337932 conn(0x7f40cc0bf700 msgr2=0x7f40cc0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:36.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.596+0000 7f40d4000640 1 -- 172.21.15.42:0/3742337932 shutdown_connections 2024-07-04T17:28:36.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.596+0000 7f40d4000640 1 -- 172.21.15.42:0/3742337932 wait complete. 2024-07-04T17:28:36.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.598+0000 7f40d4000640 1 Processor -- start 2024-07-04T17:28:36.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.598+0000 7f40d4000640 1 -- start start 2024-07-04T17:28:36.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.600+0000 7f40d4000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40cc06cea0 0x7f40cc146030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:36.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.600+0000 7f40d4000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc06ef50 con 0x7f40cc06cea0 2024-07-04T17:28:36.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.600+0000 7f40d1600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40cc06cea0 0x7f40cc146030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:36.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.600+0000 7f40d1600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40cc06cea0 0x7f40cc146030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37508/0 (socket says 172.21.15.42:37508) 2024-07-04T17:28:36.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.600+0000 7f40d1600640 1 -- 172.21.15.42:0/921004027 learned_addr learned my addr 172.21.15.42:0/921004027 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:36.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.601+0000 7f40d1600640 1 -- 172.21.15.42:0/921004027 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40cc144c70 con 0x7f40cc06cea0 2024-07-04T17:28:36.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.601+0000 7f40d1600640 1 --2- 172.21.15.42:0/921004027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40cc06cea0 0x7f40cc146030 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f40c000c380 tx=0x7f40c000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:36.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.601+0000 7f40c9600640 1 -- 172.21.15.42:0/921004027 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40c0017070 con 0x7f40cc06cea0 2024-07-04T17:28:36.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.602+0000 7f40c9600640 1 -- 172.21.15.42:0/921004027 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40c00052f0 con 0x7f40cc06cea0 2024-07-04T17:28:36.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.602+0000 7f40d4000640 1 -- 172.21.15.42:0/921004027 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40cc0ae200 con 0x7f40cc06cea0 2024-07-04T17:28:36.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.602+0000 7f40c9600640 1 -- 172.21.15.42:0/921004027 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40c00035f0 con 0x7f40cc06cea0 2024-07-04T17:28:36.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.602+0000 7f40d4000640 1 -- 172.21.15.42:0/921004027 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40cc0ada80 con 0x7f40cc06cea0 2024-07-04T17:28:36.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.603+0000 7f40d4000640 1 -- 172.21.15.42:0/921004027 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40cc144080 con 0x7f40cc06cea0 2024-07-04T17:28:36.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.603+0000 7f40c9600640 1 -- 172.21.15.42:0/921004027 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 11) ==== 49668+0+0 (secure 0 0 0) 0x7f40c001a070 con 0x7f40cc06cea0 2024-07-04T17:28:36.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.604+0000 7f40c9600640 1 --2- 172.21.15.42:0/921004027 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f40b003d5d0 0x7f40b003fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:36.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.604+0000 7f40c9600640 1 -- 172.21.15.42:0/921004027 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4604+0+0 (secure 0 0 0) 0x7f40c00511e0 con 0x7f40cc06cea0 2024-07-04T17:28:36.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.604+0000 7f40d0c00640 1 -- 172.21.15.42:0/921004027 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f40b003d5d0 msgr2=0x7f40b003fae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/488387844 2024-07-04T17:28:36.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.604+0000 7f40d0c00640 1 --2- 172.21.15.42:0/921004027 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f40b003d5d0 0x7f40b003fae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:28:36.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.609+0000 7f40c9600640 1 -- 172.21.15.42:0/921004027 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f40cc144080 con 0x7f40cc06cea0 2024-07-04T17:28:36.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.806+0000 7f40d0c00640 1 -- 172.21.15.42:0/921004027 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f40b003d5d0 msgr2=0x7f40b003fae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/488387844 2024-07-04T17:28:36.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.806+0000 7f40d0c00640 1 --2- 172.21.15.42:0/921004027 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f40b003d5d0 0x7f40b003fae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:28:36.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.885+0000 7f40d4000640 1 -- 172.21.15.42:0/921004027 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40cc0b01f0 con 0x7f40cc06cea0 2024-07-04T17:28:36.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.887+0000 7f40c9600640 1 -- 172.21.15.42:0/921004027 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+7359 (secure 0 0 0) 0x7f40c0016170 con 0x7f40cc06cea0 2024-07-04T17:28:36.888 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:28:36.888 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":25,"btime":"2024-07-04T17:28:20:786366+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/399707138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":399707138},{"type":"v1","addr":"172.21.15.42:6827","nonce":399707138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4414,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1510714231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1510714231},{"type":"v1","addr":"172.21.15.42:6829","nonce":1510714231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/3069680926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":3069680926},{"type":"v1","addr":"172.21.15.42:6831","nonce":3069680926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:27:07.402007+0000","modified":"2024-07-04T17:28:19.916603+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4423":{"gid":4423,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.42:6835/807364821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":807364821},{"type":"v1","addr":"172.21.15.42:6835","nonce":807364821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4423,"qdb_cluster":[4423]},"id":2},{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:27:27.513977+0000","modified":"2024-07-04T17:27:30.569774+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4422":{"gid":4422,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.42:6833/1348446958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1348446958},{"type":"v1","addr":"172.21.15.42:6833","nonce":1348446958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4422,"qdb_cluster":[4422]},"id":3}]} 2024-07-04T17:28:36.888 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 25 2024-07-04T17:28:36.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.890+0000 7f40d4000640 1 -- 172.21.15.42:0/921004027 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f40b003d5d0 msgr2=0x7f40b003fae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:28:36.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.890+0000 7f40d4000640 1 --2- 172.21.15.42:0/921004027 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f40b003d5d0 0x7f40b003fae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:36.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.890+0000 7f40d4000640 1 -- 172.21.15.42:0/921004027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40cc06cea0 msgr2=0x7f40cc146030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:36.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.890+0000 7f40d4000640 1 --2- 172.21.15.42:0/921004027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40cc06cea0 0x7f40cc146030 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f40c000c380 tx=0x7f40c000c840 comp rx=0 tx=0).stop 2024-07-04T17:28:36.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.890+0000 7f40d4000640 1 -- 172.21.15.42:0/921004027 shutdown_connections 2024-07-04T17:28:36.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.890+0000 7f40d4000640 1 --2- 172.21.15.42:0/921004027 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f40b003d5d0 0x7f40b003fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:36.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.890+0000 7f40d4000640 1 --2- 172.21.15.42:0/921004027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40cc06cea0 0x7f40cc146030 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:36.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.890+0000 7f40d4000640 1 -- 172.21.15.42:0/921004027 >> 172.21.15.42:0/921004027 conn(0x7f40cc0bf700 msgr2=0x7f40cc0aee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:36.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.890+0000 7f40d4000640 1 -- 172.21.15.42:0/921004027 shutdown_connections 2024-07-04T17:28:36.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:36.890+0000 7f40d4000640 1 -- 172.21.15.42:0/921004027 wait complete. 2024-07-04T17:28:36.919 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:28:37.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.190+0000 7f89ebe00640 1 -- 172.21.15.42:0/2994799329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89ec0ba830 msgr2=0x7f89ec0b7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:37.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.190+0000 7f89ebe00640 1 --2- 172.21.15.42:0/2994799329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89ec0ba830 0x7f89ec0b7cc0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f89dc005fd0 tx=0x7f89dc02a5f0 comp rx=0 tx=0).stop 2024-07-04T17:28:37.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.190+0000 7f89ebe00640 1 -- 172.21.15.42:0/2994799329 shutdown_connections 2024-07-04T17:28:37.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.190+0000 7f89ebe00640 1 --2- 172.21.15.42:0/2994799329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89ec0ba830 0x7f89ec0b7cc0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:37.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.190+0000 7f89ebe00640 1 -- 172.21.15.42:0/2994799329 >> 172.21.15.42:0/2994799329 conn(0x7f89ec0bebc0 msgr2=0x7f89ec0befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:37.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.190+0000 7f89ebe00640 1 -- 172.21.15.42:0/2994799329 shutdown_connections 2024-07-04T17:28:37.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.190+0000 7f89ebe00640 1 -- 172.21.15.42:0/2994799329 wait complete. 2024-07-04T17:28:37.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.192+0000 7f89ebe00640 1 Processor -- start 2024-07-04T17:28:37.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.192+0000 7f89ebe00640 1 -- start start 2024-07-04T17:28:37.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.194+0000 7f89ebe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89ec0ba830 0x7f89ec1d8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:37.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.194+0000 7f89ebe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f89ec0b1e30 con 0x7f89ec0ba830 2024-07-04T17:28:37.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.194+0000 7f89eaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89ec0ba830 0x7f89ec1d8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:37.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.194+0000 7f89eaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89ec0ba830 0x7f89ec1d8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37528/0 (socket says 172.21.15.42:37528) 2024-07-04T17:28:37.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.194+0000 7f89eaa00640 1 -- 172.21.15.42:0/1881395451 learned_addr learned my addr 172.21.15.42:0/1881395451 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:37.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.194+0000 7f89eaa00640 1 -- 172.21.15.42:0/1881395451 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89ec1d9160 con 0x7f89ec0ba830 2024-07-04T17:28:37.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.195+0000 7f89eaa00640 1 --2- 172.21.15.42:0/1881395451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89ec0ba830 0x7f89ec1d8050 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f89dc009850 tx=0x7f89dc0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:37.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.195+0000 7f89e2a00640 1 -- 172.21.15.42:0/1881395451 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89dc03e030 con 0x7f89ec0ba830 2024-07-04T17:28:37.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.195+0000 7f89e2a00640 1 -- 172.21.15.42:0/1881395451 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89dc0035f0 con 0x7f89ec0ba830 2024-07-04T17:28:37.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.195+0000 7f89ebe00640 1 -- 172.21.15.42:0/1881395451 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89ec1e3ba0 con 0x7f89ec0ba830 2024-07-04T17:28:37.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.195+0000 7f89e2a00640 1 -- 172.21.15.42:0/1881395451 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89dc035040 con 0x7f89ec0ba830 2024-07-04T17:28:37.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.195+0000 7f89ebe00640 1 -- 172.21.15.42:0/1881395451 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89ec13e630 con 0x7f89ec0ba830 2024-07-04T17:28:37.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.196+0000 7f89e2a00640 1 -- 172.21.15.42:0/1881395451 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 11) ==== 49668+0+0 (secure 0 0 0) 0x7f89dc041030 con 0x7f89ec0ba830 2024-07-04T17:28:37.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.196+0000 7f89ebe00640 1 -- 172.21.15.42:0/1881395451 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89ec0b1730 con 0x7f89ec0ba830 2024-07-04T17:28:37.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.197+0000 7f89e2a00640 1 --2- 172.21.15.42:0/1881395451 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f89c803d530 0x7f89c803fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:37.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.197+0000 7f89e2a00640 1 -- 172.21.15.42:0/1881395451 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4604+0+0 (secure 0 0 0) 0x7f89dc03f070 con 0x7f89ec0ba830 2024-07-04T17:28:37.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.197+0000 7f89ea000640 1 -- 172.21.15.42:0/1881395451 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f89c803d530 msgr2=0x7f89c803fa40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/488387844 2024-07-04T17:28:37.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.197+0000 7f89ea000640 1 --2- 172.21.15.42:0/1881395451 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f89c803d530 0x7f89c803fa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:28:37.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.202+0000 7f89e2a00640 1 -- 172.21.15.42:0/1881395451 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f89ec0b1730 con 0x7f89ec0ba830 2024-07-04T17:28:37.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.398+0000 7f89ea000640 1 -- 172.21.15.42:0/1881395451 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f89c803d530 msgr2=0x7f89c803fa40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/488387844 2024-07-04T17:28:37.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.398+0000 7f89ea000640 1 --2- 172.21.15.42:0/1881395451 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f89c803d530 0x7f89c803fa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:28:37.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.448+0000 7f89ebe00640 1 -- 172.21.15.42:0/1881395451 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89ec0b7cc0 con 0x7f89ec0ba830 2024-07-04T17:28:37.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.449+0000 7f89e2a00640 1 -- 172.21.15.42:0/1881395451 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+7359 (secure 0 0 0) 0x7f89dc042ce0 con 0x7f89ec0ba830 2024-07-04T17:28:37.451 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:28:37.451 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":25,"btime":"2024-07-04T17:28:20:786366+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/399707138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":399707138},{"type":"v1","addr":"172.21.15.42:6827","nonce":399707138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4414,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1510714231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1510714231},{"type":"v1","addr":"172.21.15.42:6829","nonce":1510714231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/3069680926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":3069680926},{"type":"v1","addr":"172.21.15.42:6831","nonce":3069680926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:27:07.402007+0000","modified":"2024-07-04T17:28:19.916603+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4423":{"gid":4423,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.42:6835/807364821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":807364821},{"type":"v1","addr":"172.21.15.42:6835","nonce":807364821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4423,"qdb_cluster":[4423]},"id":2},{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:27:27.513977+0000","modified":"2024-07-04T17:27:30.569774+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4422":{"gid":4422,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.42:6833/1348446958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1348446958},{"type":"v1","addr":"172.21.15.42:6833","nonce":1348446958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4422,"qdb_cluster":[4422]},"id":3}]} 2024-07-04T17:28:37.451 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 25 2024-07-04T17:28:37.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.452+0000 7f89ebe00640 1 -- 172.21.15.42:0/1881395451 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f89c803d530 msgr2=0x7f89c803fa40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:28:37.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.452+0000 7f89ebe00640 1 --2- 172.21.15.42:0/1881395451 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f89c803d530 0x7f89c803fa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:37.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.452+0000 7f89ebe00640 1 -- 172.21.15.42:0/1881395451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89ec0ba830 msgr2=0x7f89ec1d8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:37.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.452+0000 7f89ebe00640 1 --2- 172.21.15.42:0/1881395451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89ec0ba830 0x7f89ec1d8050 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f89dc009850 tx=0x7f89dc0049d0 comp rx=0 tx=0).stop 2024-07-04T17:28:37.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.452+0000 7f89ebe00640 1 -- 172.21.15.42:0/1881395451 shutdown_connections 2024-07-04T17:28:37.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.452+0000 7f89ebe00640 1 --2- 172.21.15.42:0/1881395451 >> [v2:172.21.15.42:6824/488387844,v1:172.21.15.42:6825/488387844] conn(0x7f89c803d530 0x7f89c803fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:37.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.452+0000 7f89ebe00640 1 --2- 172.21.15.42:0/1881395451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89ec0ba830 0x7f89ec1d8050 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:37.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.452+0000 7f89ebe00640 1 -- 172.21.15.42:0/1881395451 >> 172.21.15.42:0/1881395451 conn(0x7f89ec0bebc0 msgr2=0x7f89ec143d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:37.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.453+0000 7f89ebe00640 1 -- 172.21.15.42:0/1881395451 shutdown_connections 2024-07-04T17:28:37.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.453+0000 7f89ebe00640 1 -- 172.21.15.42:0/1881395451 wait complete. 2024-07-04T17:28:37.478 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:28:37.703 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:28:37.701+0000 7f7dee000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:28:37.703 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:28:37.701+0000 7f7dee000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:28:37.703 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:28:37.701+0000 7f7dee000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:28:37.703 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:28:37.701+0000 7f7dee000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:28:37.703 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:28:37.701+0000 7f7dee000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:28:37.704 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:28:37.702+0000 7f7de8600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:28:37.704 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:28:37.702+0000 7f7de8600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:28:37.704 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:28:37.702+0000 7f7de8600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:28:37.704 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:28:37.702+0000 7f7de8600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:28:37.704 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:28:37.702+0000 7f7de8600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:28:37.717 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:28:37.716+0000 7f7dee000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:28:37.717 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:28:37.716+0000 7f7dee000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:28:37.717 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:28:37.716+0000 7f7dee000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:28:37.717 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:28:37.716+0000 7f7dee000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:28:37.717 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:28:37.716+0000 7f7dee000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:28:37.718 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:28:37.717+0000 7f7de8600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:28:37.718 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:28:37.717+0000 7f7de8600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:28:37.718 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:28:37.717+0000 7f7de8600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:28:37.719 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:28:37.717+0000 7f7de8600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:28:37.719 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:28:37.717+0000 7f7de8600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:28:37.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.768+0000 7fabcce00640 1 -- 172.21.15.42:0/538713597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc80b2d20 msgr2=0x7fabc80bb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:37.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.768+0000 7fabcce00640 1 --2- 172.21.15.42:0/538713597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc80b2d20 0x7fabc80bb1f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fabb40097b0 tx=0x7fabb402a4b0 comp rx=0 tx=0).stop 2024-07-04T17:28:37.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.768+0000 7fabcce00640 1 -- 172.21.15.42:0/538713597 shutdown_connections 2024-07-04T17:28:37.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.768+0000 7fabcce00640 1 --2- 172.21.15.42:0/538713597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc80b2d20 0x7fabc80bb1f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:37.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.768+0000 7fabcce00640 1 -- 172.21.15.42:0/538713597 >> 172.21.15.42:0/538713597 conn(0x7fabc80bf700 msgr2=0x7fabc80af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:37.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.768+0000 7fabcce00640 1 -- 172.21.15.42:0/538713597 shutdown_connections 2024-07-04T17:28:37.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.768+0000 7fabcce00640 1 -- 172.21.15.42:0/538713597 wait complete. 2024-07-04T17:28:37.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.770+0000 7fabcce00640 1 Processor -- start 2024-07-04T17:28:37.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.770+0000 7fabcce00640 1 -- start start 2024-07-04T17:28:37.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.771+0000 7fabcce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc80b2d20 0x7fabc80aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:37.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.771+0000 7fabcce00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fabc806c020 con 0x7fabc80b2d20 2024-07-04T17:28:37.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.771+0000 7fabc5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc80b2d20 0x7fabc80aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:37.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.771+0000 7fabc5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc80b2d20 0x7fabc80aebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37688/0 (socket says 172.21.15.42:37688) 2024-07-04T17:28:37.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.771+0000 7fabc5e00640 1 -- 172.21.15.42:0/3265935321 learned_addr learned my addr 172.21.15.42:0/3265935321 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:37.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.772+0000 7fabc5e00640 1 -- 172.21.15.42:0/3265935321 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc8071440 con 0x7fabc80b2d20 2024-07-04T17:28:37.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.772+0000 7fabc5e00640 1 --2- 172.21.15.42:0/3265935321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc80b2d20 0x7fabc80aebb0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fabb800c380 tx=0x7fabb800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:37.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.772+0000 7fabbe000640 1 -- 172.21.15.42:0/3265935321 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabb80116c0 con 0x7fabc80b2d20 2024-07-04T17:28:37.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.772+0000 7fabbe000640 1 -- 172.21.15.42:0/3265935321 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fabb8017070 con 0x7fabc80b2d20 2024-07-04T17:28:37.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.772+0000 7fabbe000640 1 -- 172.21.15.42:0/3265935321 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabb8011b20 con 0x7fabc80b2d20 2024-07-04T17:28:37.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.772+0000 7fabcce00640 1 -- 172.21.15.42:0/3265935321 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc81e43f0 con 0x7fabc80b2d20 2024-07-04T17:28:37.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.773+0000 7fabcce00640 1 -- 172.21.15.42:0/3265935321 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc8137100 con 0x7fabc80b2d20 2024-07-04T17:28:37.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.774+0000 7fabcce00640 1 -- 172.21.15.42:0/3265935321 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabc81e4620 con 0x7fabc80b2d20 2024-07-04T17:28:37.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.774+0000 7fabbe000640 1 -- 172.21.15.42:0/3265935321 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 12) ==== 49122+0+0 (secure 0 0 0) 0x7fabb8011cc0 con 0x7fabc80b2d20 2024-07-04T17:28:37.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.774+0000 7fabbe000640 1 -- 172.21.15.42:0/3265935321 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5077+0+0 (secure 0 0 0) 0x7fabb8051510 con 0x7fabc80b2d20 2024-07-04T17:28:37.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.780+0000 7fabbe000640 1 -- 172.21.15.42:0/3265935321 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fabb8010cf0 con 0x7fabc80b2d20 2024-07-04T17:28:37.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.988+0000 7fabcce00640 1 -- 172.21.15.42:0/3265935321 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fabc806c710 con 0x7fabc80b2d20 2024-07-04T17:28:37.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.990+0000 7fabbe000640 1 -- 172.21.15.42:0/3265935321 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+15568 (secure 0 0 0) 0x7fabb8012420 con 0x7fabc80b2d20 2024-07-04T17:28:37.992 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:28:37.992 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":34,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:28:37.604559+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:27:04.932839+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-07-04T17:27:06.393713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.0800000429153442,"score_stable":1.0800000429153442,"optimal_score":1,"raw_score_acting":1.0800000429153442,"raw_score_stable":1.0800000429153442,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:27:25.040056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:27:26.487236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/429805019":"2024-07-05T17:28:37.604461+0000","172.21.15.42:0/3791890040":"2024-07-05T17:28:37.604461+0000","172.21.15.42:0/1994443123":"2024-07-05T17:28:37.604461+0000","172.21.15.42:0/2983064181":"2024-07-05T17:28:37.604461+0000","172.21.15.42:6824/3523793950":"2024-07-05T17:27:40.654396+0000","172.21.15.42:6824/488387844":"2024-07-05T17:28:37.604461+0000","172.21.15.42:6825/3523793950":"2024-07-05T17:27:40.654396+0000","172.21.15.42:0/487218478":"2024-07-04T18:28:21.818727+0000","172.21.15.42:0/2974639683":"2024-07-05T17:27:40.654396+0000","172.21.15.42:0/880857241":"2024-07-05T17:27:40.654396+0000","172.21.15.42:0/2267375347":"2024-07-05T17:28:37.604461+0000","172.21.15.42:6825/488387844":"2024-07-05T17:28:37.604461+0000","172.21.15.42:0/83074206":"2024-07-05T17:27:40.654396+0000","172.21.15.42:0/3526015921":"2024-07-05T17:27:40.654396+0000","172.21.15.42:0/1537192714":"2024-07-05T17:28:37.604461+0000","172.21.15.42:0/435904326":"2024-07-05T17:28:37.604461+0000","172.21.15.42:0/3378322019":"2024-07-05T17:28:37.604461+0000","172.21.15.42:0/2362095151":"2024-07-05T17:28:37.604461+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:28:37.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.993+0000 7fabcce00640 1 -- 172.21.15.42:0/3265935321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc80b2d20 msgr2=0x7fabc80aebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:37.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.993+0000 7fabcce00640 1 --2- 172.21.15.42:0/3265935321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc80b2d20 0x7fabc80aebb0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fabb800c380 tx=0x7fabb800c840 comp rx=0 tx=0).stop 2024-07-04T17:28:37.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.994+0000 7fabcce00640 1 -- 172.21.15.42:0/3265935321 shutdown_connections 2024-07-04T17:28:37.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.994+0000 7fabcce00640 1 --2- 172.21.15.42:0/3265935321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc80b2d20 0x7fabc80aebb0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:37.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.994+0000 7fabcce00640 1 -- 172.21.15.42:0/3265935321 >> 172.21.15.42:0/3265935321 conn(0x7fabc80bf700 msgr2=0x7fabc80bd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:37.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.994+0000 7fabcce00640 1 -- 172.21.15.42:0/3265935321 shutdown_connections 2024-07-04T17:28:37.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:37.994+0000 7fabcce00640 1 -- 172.21.15.42:0/3265935321 wait complete. 2024-07-04T17:28:38.022 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-07-04T17:28:38.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.312+0000 7f6f09a00640 1 -- 172.21.15.42:0/2236854602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f040b2d20 msgr2=0x7f6f040bb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:38.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.312+0000 7f6f09a00640 1 --2- 172.21.15.42:0/2236854602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f040b2d20 0x7f6f040bb1f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6eec005fd0 tx=0x7f6eec02a560 comp rx=0 tx=0).stop 2024-07-04T17:28:38.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.312+0000 7f6f09a00640 1 -- 172.21.15.42:0/2236854602 shutdown_connections 2024-07-04T17:28:38.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.312+0000 7f6f09a00640 1 --2- 172.21.15.42:0/2236854602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f040b2d20 0x7f6f040bb1f0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:38.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.312+0000 7f6f09a00640 1 -- 172.21.15.42:0/2236854602 >> 172.21.15.42:0/2236854602 conn(0x7f6f040bf700 msgr2=0x7f6f040af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:38.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.312+0000 7f6f09a00640 1 -- 172.21.15.42:0/2236854602 shutdown_connections 2024-07-04T17:28:38.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.312+0000 7f6f09a00640 1 -- 172.21.15.42:0/2236854602 wait complete. 2024-07-04T17:28:38.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.314+0000 7f6f09a00640 1 Processor -- start 2024-07-04T17:28:38.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.314+0000 7f6f09a00640 1 -- start start 2024-07-04T17:28:38.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.316+0000 7f6f09a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f040b2d20 0x7f6f040ab0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:38.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.316+0000 7f6f09a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0406c020 con 0x7f6f040b2d20 2024-07-04T17:28:38.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.316+0000 7f6f02a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f040b2d20 0x7f6f040ab0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:38.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.316+0000 7f6f02a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f040b2d20 0x7f6f040ab0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37706/0 (socket says 172.21.15.42:37706) 2024-07-04T17:28:38.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.316+0000 7f6f02a00640 1 -- 172.21.15.42:0/3584533121 learned_addr learned my addr 172.21.15.42:0/3584533121 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:38.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.316+0000 7f6f02a00640 1 -- 172.21.15.42:0/3584533121 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f04070e30 con 0x7f6f040b2d20 2024-07-04T17:28:38.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.317+0000 7f6f02a00640 1 --2- 172.21.15.42:0/3584533121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f040b2d20 0x7f6f040ab0c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6ef400c380 tx=0x7f6ef400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:38.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.317+0000 7f6efaa00640 1 -- 172.21.15.42:0/3584533121 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ef4017070 con 0x7f6f040b2d20 2024-07-04T17:28:38.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.317+0000 7f6efaa00640 1 -- 172.21.15.42:0/3584533121 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ef40052f0 con 0x7f6f040b2d20 2024-07-04T17:28:38.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.317+0000 7f6efaa00640 1 -- 172.21.15.42:0/3584533121 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ef40035f0 con 0x7f6f040b2d20 2024-07-04T17:28:38.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.317+0000 7f6f09a00640 1 -- 172.21.15.42:0/3584533121 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f041e7040 con 0x7f6f040b2d20 2024-07-04T17:28:38.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.317+0000 7f6f09a00640 1 -- 172.21.15.42:0/3584533121 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f041e6b90 con 0x7f6f040b2d20 2024-07-04T17:28:38.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.318+0000 7f6f09a00640 1 -- 172.21.15.42:0/3584533121 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f0406c710 con 0x7f6f040b2d20 2024-07-04T17:28:38.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.318+0000 7f6efaa00640 1 -- 172.21.15.42:0/3584533121 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 12) ==== 49122+0+0 (secure 0 0 0) 0x7f6ef401a070 con 0x7f6f040b2d20 2024-07-04T17:28:38.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.319+0000 7f6efaa00640 1 -- 172.21.15.42:0/3584533121 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5077+0+0 (secure 0 0 0) 0x7f6ef40509e0 con 0x7f6f040b2d20 2024-07-04T17:28:38.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.324+0000 7f6efaa00640 1 -- 172.21.15.42:0/3584533121 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f6ef4016170 con 0x7f6f040b2d20 2024-07-04T17:28:38.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.568+0000 7f6f09a00640 1 -- 172.21.15.42:0/3584533121 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6f041e6dc0 con 0x7f6f040b2d20 2024-07-04T17:28:38.571 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-07-04T17:28:38.570+0000 7f8616e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-04T17:28:38.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.571+0000 7f6efaa00640 1 -- 172.21.15.42:0/3584533121 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) ==== 161+0+0 (secure 0 0 0) 0x7f6ef4003c60 con 0x7f6f040b2d20 2024-07-04T17:28:38.573 INFO:teuthology.orchestra.run.smithi042.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-04T17:28:38.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.575+0000 7f6f09a00640 1 -- 172.21.15.42:0/3584533121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f040b2d20 msgr2=0x7f6f040ab0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:38.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.575+0000 7f6f09a00640 1 --2- 172.21.15.42:0/3584533121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f040b2d20 0x7f6f040ab0c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6ef400c380 tx=0x7f6ef400c840 comp rx=0 tx=0).stop 2024-07-04T17:28:38.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.575+0000 7f6f09a00640 1 -- 172.21.15.42:0/3584533121 shutdown_connections 2024-07-04T17:28:38.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.575+0000 7f6f09a00640 1 --2- 172.21.15.42:0/3584533121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f040b2d20 0x7f6f040ab0c0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:38.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.575+0000 7f6f09a00640 1 -- 172.21.15.42:0/3584533121 >> 172.21.15.42:0/3584533121 conn(0x7f6f040bf700 msgr2=0x7f6f04141080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:38.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.575+0000 7f6f09a00640 1 -- 172.21.15.42:0/3584533121 shutdown_connections 2024-07-04T17:28:38.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.576+0000 7f6f09a00640 1 -- 172.21.15.42:0/3584533121 wait complete. 2024-07-04T17:28:38.617 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-07-04T17:28:38.749 INFO:tasks.ceph.mds.b.smithi042.stdout:starting mds.b at 2024-07-04T17:28:38.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.878+0000 7fa735a00640 1 -- 172.21.15.42:0/3861388078 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7300bc150 msgr2=0x7fa7300bd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:38.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.878+0000 7fa735a00640 1 --2- 172.21.15.42:0/3861388078 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7300bc150 0x7fa7300bd8c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa7300be180 tx=0x7fa72002a4a0 comp rx=0 tx=0).stop 2024-07-04T17:28:38.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.878+0000 7fa735a00640 1 -- 172.21.15.42:0/3861388078 shutdown_connections 2024-07-04T17:28:38.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.878+0000 7fa735a00640 1 --2- 172.21.15.42:0/3861388078 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7300bc150 0x7fa7300bd8c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:38.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.878+0000 7fa735a00640 1 -- 172.21.15.42:0/3861388078 >> 172.21.15.42:0/3861388078 conn(0x7fa73006e240 msgr2=0x7fa73006e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:38.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.878+0000 7fa735a00640 1 -- 172.21.15.42:0/3861388078 shutdown_connections 2024-07-04T17:28:38.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.878+0000 7fa735a00640 1 -- 172.21.15.42:0/3861388078 wait complete. 2024-07-04T17:28:38.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.880+0000 7fa735a00640 1 Processor -- start 2024-07-04T17:28:38.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.880+0000 7fa735a00640 1 -- start start 2024-07-04T17:28:38.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.881+0000 7fa735a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7300bc150 0x7fa7300aa280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:38.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.881+0000 7fa735a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa7300b7470 con 0x7fa7300bc150 2024-07-04T17:28:38.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.882+0000 7fa72fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7300bc150 0x7fa7300aa280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:38.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.882+0000 7fa72fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7300bc150 0x7fa7300aa280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37748/0 (socket says 172.21.15.42:37748) 2024-07-04T17:28:38.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.882+0000 7fa72fe00640 1 -- 172.21.15.42:0/3238432918 learned_addr learned my addr 172.21.15.42:0/3238432918 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:38.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.882+0000 7fa72fe00640 1 -- 172.21.15.42:0/3238432918 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7300ab830 con 0x7fa7300bc150 2024-07-04T17:28:38.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.883+0000 7fa72fe00640 1 --2- 172.21.15.42:0/3238432918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7300bc150 0x7fa7300aa280 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa72002aa90 tx=0x7fa720033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:38.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.883+0000 7fa727e00640 1 -- 172.21.15.42:0/3238432918 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa720031880 con 0x7fa7300bc150 2024-07-04T17:28:38.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.883+0000 7fa727e00640 1 -- 172.21.15.42:0/3238432918 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa720031b10 con 0x7fa7300bc150 2024-07-04T17:28:38.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.883+0000 7fa727e00640 1 -- 172.21.15.42:0/3238432918 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa720033070 con 0x7fa7300bc150 2024-07-04T17:28:38.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.883+0000 7fa735a00640 1 -- 172.21.15.42:0/3238432918 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7300a9800 con 0x7fa7300bc150 2024-07-04T17:28:38.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.883+0000 7fa735a00640 1 -- 172.21.15.42:0/3238432918 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa730151560 con 0x7fa7300bc150 2024-07-04T17:28:38.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.884+0000 7fa735a00640 1 -- 172.21.15.42:0/3238432918 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7300b7b80 con 0x7fa7300bc150 2024-07-04T17:28:38.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.884+0000 7fa727e00640 1 -- 172.21.15.42:0/3238432918 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 49249+0+0 (secure 0 0 0) 0x7fa720033210 con 0x7fa7300bc150 2024-07-04T17:28:38.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.885+0000 7fa727e00640 1 --2- 172.21.15.42:0/3238432918 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fa70c03d100 0x7fa70c03f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:38.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.885+0000 7fa727e00640 1 -- 172.21.15.42:0/3238432918 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5163+0+0 (secure 0 0 0) 0x7fa720072c50 con 0x7fa7300bc150 2024-07-04T17:28:38.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.885+0000 7fa72f400640 1 --2- 172.21.15.42:0/3238432918 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fa70c03d100 0x7fa70c03f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:38.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.886+0000 7fa72f400640 1 --2- 172.21.15.42:0/3238432918 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fa70c03d100 0x7fa70c03f610 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa7300b8d30 tx=0x7fa718001290 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:38.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:38.890+0000 7fa727e00640 1 -- 172.21.15.42:0/3238432918 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fa72003f030 con 0x7fa7300bc150 2024-07-04T17:28:39.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.143+0000 7fa735a00640 1 -- 172.21.15.42:0/3238432918 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa730068dc0 con 0x7fa7300bc150 2024-07-04T17:28:39.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.619+0000 7fa727e00640 1 -- 172.21.15.42:0/3238432918 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) ==== 104+0+0 (secure 0 0 0) 0x7fa7200335b0 con 0x7fa7300bc150 2024-07-04T17:28:39.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.623+0000 7fa735a00640 1 -- 172.21.15.42:0/3238432918 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fa70c03d100 msgr2=0x7fa70c03f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:39.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.623+0000 7fa735a00640 1 --2- 172.21.15.42:0/3238432918 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fa70c03d100 0x7fa70c03f610 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa7300b8d30 tx=0x7fa718001290 comp rx=0 tx=0).stop 2024-07-04T17:28:39.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.623+0000 7fa735a00640 1 -- 172.21.15.42:0/3238432918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7300bc150 msgr2=0x7fa7300aa280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:39.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.623+0000 7fa735a00640 1 --2- 172.21.15.42:0/3238432918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7300bc150 0x7fa7300aa280 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa72002aa90 tx=0x7fa720033ad0 comp rx=0 tx=0).stop 2024-07-04T17:28:39.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.623+0000 7fa735a00640 1 -- 172.21.15.42:0/3238432918 shutdown_connections 2024-07-04T17:28:39.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.623+0000 7fa735a00640 1 --2- 172.21.15.42:0/3238432918 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fa70c03d100 0x7fa70c03f610 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:39.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.623+0000 7fa735a00640 1 --2- 172.21.15.42:0/3238432918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7300bc150 0x7fa7300aa280 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:39.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.623+0000 7fa735a00640 1 -- 172.21.15.42:0/3238432918 >> 172.21.15.42:0/3238432918 conn(0x7fa73006e240 msgr2=0x7fa7300b4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:39.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.623+0000 7fa735a00640 1 -- 172.21.15.42:0/3238432918 shutdown_connections 2024-07-04T17:28:39.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.624+0000 7fa735a00640 1 -- 172.21.15.42:0/3238432918 wait complete. 2024-07-04T17:28:39.657 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-04T17:28:39.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.941+0000 7f89d4200640 1 -- 172.21.15.42:0/747310650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89cc06d070 msgr2=0x7f89cc073a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:39.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.941+0000 7f89d4200640 1 --2- 172.21.15.42:0/747310650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89cc06d070 0x7f89cc073a20 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f89c00097b0 tx=0x7f89c002a540 comp rx=0 tx=0).stop 2024-07-04T17:28:39.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.941+0000 7f89d4200640 1 -- 172.21.15.42:0/747310650 shutdown_connections 2024-07-04T17:28:39.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.941+0000 7f89d4200640 1 --2- 172.21.15.42:0/747310650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89cc06d070 0x7f89cc073a20 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:39.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.941+0000 7f89d4200640 1 -- 172.21.15.42:0/747310650 >> 172.21.15.42:0/747310650 conn(0x7f89cc06c020 msgr2=0x7f89cc06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:39.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.941+0000 7f89d4200640 1 -- 172.21.15.42:0/747310650 shutdown_connections 2024-07-04T17:28:39.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.941+0000 7f89d4200640 1 -- 172.21.15.42:0/747310650 wait complete. 2024-07-04T17:28:39.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.944+0000 7f89d4200640 1 Processor -- start 2024-07-04T17:28:39.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.944+0000 7f89d4200640 1 -- start start 2024-07-04T17:28:39.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.946+0000 7f89d4200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89cc06d070 0x7f89cc0aed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:39.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.946+0000 7f89d4200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc071c40 con 0x7f89cc06d070 2024-07-04T17:28:39.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.946+0000 7f89d1800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89cc06d070 0x7f89cc0aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:39.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.946+0000 7f89d1800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89cc06d070 0x7f89cc0aed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37784/0 (socket says 172.21.15.42:37784) 2024-07-04T17:28:39.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.946+0000 7f89d1800640 1 -- 172.21.15.42:0/4068526312 learned_addr learned my addr 172.21.15.42:0/4068526312 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:39.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.947+0000 7f89d1800640 1 -- 172.21.15.42:0/4068526312 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89cc1d7590 con 0x7f89cc06d070 2024-07-04T17:28:39.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.947+0000 7f89d1800640 1 --2- 172.21.15.42:0/4068526312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89cc06d070 0x7f89cc0aed30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f89c00098d0 tx=0x7f89c0033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:39.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.947+0000 7f89c9600640 1 -- 172.21.15.42:0/4068526312 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89c0031bc0 con 0x7f89cc06d070 2024-07-04T17:28:39.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.947+0000 7f89c9600640 1 -- 172.21.15.42:0/4068526312 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89c0031e50 con 0x7f89cc06d070 2024-07-04T17:28:39.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.947+0000 7f89c9600640 1 -- 172.21.15.42:0/4068526312 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89c0033200 con 0x7f89cc06d070 2024-07-04T17:28:39.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.947+0000 7f89d4200640 1 -- 172.21.15.42:0/4068526312 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89cc1d4450 con 0x7f89cc06d070 2024-07-04T17:28:39.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.947+0000 7f89d4200640 1 -- 172.21.15.42:0/4068526312 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89cc1d39e0 con 0x7f89cc06d070 2024-07-04T17:28:39.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.948+0000 7f89d4200640 1 -- 172.21.15.42:0/4068526312 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89cc1d3cd0 con 0x7f89cc06d070 2024-07-04T17:28:39.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.949+0000 7f89c9600640 1 -- 172.21.15.42:0/4068526312 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 14) ==== 49559+0+0 (secure 0 0 0) 0x7f89c00333a0 con 0x7f89cc06d070 2024-07-04T17:28:39.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.949+0000 7f89c9600640 1 --2- 172.21.15.42:0/4068526312 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f89b003d460 0x7f89b003f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:39.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.949+0000 7f89c9600640 1 -- 172.21.15.42:0/4068526312 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5163+0+0 (secure 0 0 0) 0x7f89c0073530 con 0x7f89cc06d070 2024-07-04T17:28:39.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.950+0000 7f89d0e00640 1 --2- 172.21.15.42:0/4068526312 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f89b003d460 0x7f89b003f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:39.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.951+0000 7f89d0e00640 1 --2- 172.21.15.42:0/4068526312 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f89b003d460 0x7f89b003f970 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f89bc0015c0 tx=0x7f89bc007a90 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:39.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:39.954+0000 7f89c9600640 1 -- 172.21.15.42:0/4068526312 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f89c003f030 con 0x7f89cc06d070 2024-07-04T17:28:40.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:40.183+0000 7f89d4200640 1 -- 172.21.15.42:0/4068526312 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f89cc073a20 con 0x7f89cc06d070 2024-07-04T17:28:40.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:40.851+0000 7f89c9600640 1 -- 172.21.15.42:0/4068526312 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v36) ==== 181+0+0 (secure 0 0 0) 0x7f89c0033690 con 0x7f89cc06d070 2024-07-04T17:28:40.853 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' removed 2024-07-04T17:28:40.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:40.855+0000 7f89d4200640 1 -- 172.21.15.42:0/4068526312 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f89b003d460 msgr2=0x7f89b003f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:40.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:40.855+0000 7f89d4200640 1 --2- 172.21.15.42:0/4068526312 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f89b003d460 0x7f89b003f970 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f89bc0015c0 tx=0x7f89bc007a90 comp rx=0 tx=0).stop 2024-07-04T17:28:40.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:40.855+0000 7f89d4200640 1 -- 172.21.15.42:0/4068526312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89cc06d070 msgr2=0x7f89cc0aed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:40.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:40.855+0000 7f89d4200640 1 --2- 172.21.15.42:0/4068526312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89cc06d070 0x7f89cc0aed30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f89c00098d0 tx=0x7f89c0033e40 comp rx=0 tx=0).stop 2024-07-04T17:28:40.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:40.855+0000 7f89d4200640 1 -- 172.21.15.42:0/4068526312 shutdown_connections 2024-07-04T17:28:40.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:40.855+0000 7f89d4200640 1 --2- 172.21.15.42:0/4068526312 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f89b003d460 0x7f89b003f970 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:40.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:40.855+0000 7f89d4200640 1 --2- 172.21.15.42:0/4068526312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89cc06d070 0x7f89cc0aed30 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:40.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:40.855+0000 7f89d4200640 1 -- 172.21.15.42:0/4068526312 >> 172.21.15.42:0/4068526312 conn(0x7f89cc06c020 msgr2=0x7f89cc140ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:40.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:40.856+0000 7f89d4200640 1 -- 172.21.15.42:0/4068526312 shutdown_connections 2024-07-04T17:28:40.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:40.856+0000 7f89d4200640 1 -- 172.21.15.42:0/4068526312 wait complete. 2024-07-04T17:28:40.895 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-04T17:28:41.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.171+0000 7f91ace00640 1 -- 172.21.15.42:0/3258029096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91a806da60 msgr2=0x7f91a80b0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:41.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.171+0000 7f91ace00640 1 --2- 172.21.15.42:0/3258029096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91a806da60 0x7f91a80b0db0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9198005fd0 tx=0x7f919802b910 comp rx=0 tx=0).stop 2024-07-04T17:28:41.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.172+0000 7f91ace00640 1 -- 172.21.15.42:0/3258029096 shutdown_connections 2024-07-04T17:28:41.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.172+0000 7f91ace00640 1 --2- 172.21.15.42:0/3258029096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91a806da60 0x7f91a80b0db0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:41.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.172+0000 7f91ace00640 1 -- 172.21.15.42:0/3258029096 >> 172.21.15.42:0/3258029096 conn(0x7f91a80afaa0 msgr2=0x7f91a80afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:41.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.172+0000 7f91ace00640 1 -- 172.21.15.42:0/3258029096 shutdown_connections 2024-07-04T17:28:41.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.172+0000 7f91ace00640 1 -- 172.21.15.42:0/3258029096 wait complete. 2024-07-04T17:28:41.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.174+0000 7f91ace00640 1 Processor -- start 2024-07-04T17:28:41.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.174+0000 7f91ace00640 1 -- start start 2024-07-04T17:28:41.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.175+0000 7f91ace00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91a806da60 0x7f91a81d2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:41.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.175+0000 7f91ace00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f91a80ae590 con 0x7f91a806da60 2024-07-04T17:28:41.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.176+0000 7f91a7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91a806da60 0x7f91a81d2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:41.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.176+0000 7f91a7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91a806da60 0x7f91a81d2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37802/0 (socket says 172.21.15.42:37802) 2024-07-04T17:28:41.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.176+0000 7f91a7400640 1 -- 172.21.15.42:0/914067271 learned_addr learned my addr 172.21.15.42:0/914067271 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:41.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.176+0000 7f91a7400640 1 -- 172.21.15.42:0/914067271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a81d9dd0 con 0x7f91a806da60 2024-07-04T17:28:41.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.176+0000 7f91a7400640 1 --2- 172.21.15.42:0/914067271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91a806da60 0x7f91a81d2cc0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f919800ad00 tx=0x7f9198003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:41.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.177+0000 7f919f400640 1 -- 172.21.15.42:0/914067271 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9198035050 con 0x7f91a806da60 2024-07-04T17:28:41.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.177+0000 7f919f400640 1 -- 172.21.15.42:0/914067271 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91980044c0 con 0x7f91a806da60 2024-07-04T17:28:41.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.177+0000 7f919f400640 1 -- 172.21.15.42:0/914067271 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91980047e0 con 0x7f91a806da60 2024-07-04T17:28:41.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.177+0000 7f91ace00640 1 -- 172.21.15.42:0/914067271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91a81d4a70 con 0x7f91a806da60 2024-07-04T17:28:41.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.177+0000 7f91ace00640 1 -- 172.21.15.42:0/914067271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91a81d3200 con 0x7f91a806da60 2024-07-04T17:28:41.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.178+0000 7f91ace00640 1 -- 172.21.15.42:0/914067271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91a80b0db0 con 0x7f91a806da60 2024-07-04T17:28:41.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.178+0000 7f919f400640 1 -- 172.21.15.42:0/914067271 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 14) ==== 49559+0+0 (secure 0 0 0) 0x7f9198038020 con 0x7f91a806da60 2024-07-04T17:28:41.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.179+0000 7f919f400640 1 --2- 172.21.15.42:0/914067271 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f918803d490 0x7f918803f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:41.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.179+0000 7f919f400640 1 -- 172.21.15.42:0/914067271 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4725+0+0 (secure 0 0 0) 0x7f91980732b0 con 0x7f91a806da60 2024-07-04T17:28:41.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.179+0000 7f91a6a00640 1 --2- 172.21.15.42:0/914067271 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f918803d490 0x7f918803f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:41.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.180+0000 7f91a6a00640 1 --2- 172.21.15.42:0/914067271 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f918803d490 0x7f918803f9a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9194001560 tx=0x7f91940079f0 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:41.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.185+0000 7f919f400640 1 -- 172.21.15.42:0/914067271 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f91a81d9dd0 con 0x7f91a806da60 2024-07-04T17:28:41.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.399+0000 7f91ace00640 1 -- 172.21.15.42:0/914067271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f91a813c6a0 con 0x7f91a806da60 2024-07-04T17:28:41.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.870+0000 7f919f400640 1 -- 172.21.15.42:0/914067271 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v37) ==== 169+0+0 (secure 0 0 0) 0x7f9198045e20 con 0x7f91a806da60 2024-07-04T17:28:41.872 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' removed 2024-07-04T17:28:41.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.874+0000 7f91ace00640 1 -- 172.21.15.42:0/914067271 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f918803d490 msgr2=0x7f918803f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:41.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.874+0000 7f91ace00640 1 --2- 172.21.15.42:0/914067271 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f918803d490 0x7f918803f9a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9194001560 tx=0x7f91940079f0 comp rx=0 tx=0).stop 2024-07-04T17:28:41.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.874+0000 7f91ace00640 1 -- 172.21.15.42:0/914067271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91a806da60 msgr2=0x7f91a81d2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:41.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.874+0000 7f91ace00640 1 --2- 172.21.15.42:0/914067271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91a806da60 0x7f91a81d2cc0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f919800ad00 tx=0x7f9198003f30 comp rx=0 tx=0).stop 2024-07-04T17:28:41.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.875+0000 7f91ace00640 1 -- 172.21.15.42:0/914067271 shutdown_connections 2024-07-04T17:28:41.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.875+0000 7f91ace00640 1 --2- 172.21.15.42:0/914067271 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f918803d490 0x7f918803f9a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:41.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.875+0000 7f91ace00640 1 --2- 172.21.15.42:0/914067271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91a806da60 0x7f91a81d2cc0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:41.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.875+0000 7f91ace00640 1 -- 172.21.15.42:0/914067271 >> 172.21.15.42:0/914067271 conn(0x7f91a80afaa0 msgr2=0x7f91a813f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:41.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.878+0000 7f91ace00640 1 -- 172.21.15.42:0/914067271 shutdown_connections 2024-07-04T17:28:41.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:41.878+0000 7f91ace00640 1 -- 172.21.15.42:0/914067271 wait complete. 2024-07-04T17:28:41.915 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:28:42.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.190+0000 7f4c40200640 1 -- 172.21.15.42:0/4122183049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c380b38b0 msgr2=0x7f4c380b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:42.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.190+0000 7f4c40200640 1 --2- 172.21.15.42:0/4122183049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c380b38b0 0x7f4c380b2d20 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c005fd0 tx=0x7f4c2c02a5f0 comp rx=0 tx=0).stop 2024-07-04T17:28:42.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.190+0000 7f4c40200640 1 -- 172.21.15.42:0/4122183049 shutdown_connections 2024-07-04T17:28:42.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.190+0000 7f4c40200640 1 --2- 172.21.15.42:0/4122183049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c380b38b0 0x7f4c380b2d20 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:42.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.190+0000 7f4c40200640 1 -- 172.21.15.42:0/4122183049 >> 172.21.15.42:0/4122183049 conn(0x7f4c380bf700 msgr2=0x7f4c380af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:42.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.190+0000 7f4c40200640 1 -- 172.21.15.42:0/4122183049 shutdown_connections 2024-07-04T17:28:42.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.190+0000 7f4c40200640 1 -- 172.21.15.42:0/4122183049 wait complete. 2024-07-04T17:28:42.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.192+0000 7f4c40200640 1 Processor -- start 2024-07-04T17:28:42.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.193+0000 7f4c40200640 1 -- start start 2024-07-04T17:28:42.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.194+0000 7f4c40200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c380b1e00 0x7f4c381d4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:42.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.194+0000 7f4c40200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4c380b06e0 con 0x7f4c380b1e00 2024-07-04T17:28:42.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.194+0000 7f4c3d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c380b1e00 0x7f4c381d4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:42.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.194+0000 7f4c3d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c380b1e00 0x7f4c381d4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37818/0 (socket says 172.21.15.42:37818) 2024-07-04T17:28:42.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.194+0000 7f4c3d800640 1 -- 172.21.15.42:0/845318129 learned_addr learned my addr 172.21.15.42:0/845318129 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:42.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.195+0000 7f4c3d800640 1 -- 172.21.15.42:0/845318129 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c381d6740 con 0x7f4c380b1e00 2024-07-04T17:28:42.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.195+0000 7f4c3d800640 1 --2- 172.21.15.42:0/845318129 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c380b1e00 0x7f4c381d4df0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c034040 tx=0x7f4c2c0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:42.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.195+0000 7f4c35600640 1 -- 172.21.15.42:0/845318129 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c2c03e030 con 0x7f4c380b1e00 2024-07-04T17:28:42.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.195+0000 7f4c35600640 1 -- 172.21.15.42:0/845318129 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c2c0035f0 con 0x7f4c380b1e00 2024-07-04T17:28:42.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.195+0000 7f4c40200640 1 -- 172.21.15.42:0/845318129 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c380aa540 con 0x7f4c380b1e00 2024-07-04T17:28:42.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.195+0000 7f4c35600640 1 -- 172.21.15.42:0/845318129 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c2c035040 con 0x7f4c380b1e00 2024-07-04T17:28:42.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.195+0000 7f4c40200640 1 -- 172.21.15.42:0/845318129 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c381d5330 con 0x7f4c380b1e00 2024-07-04T17:28:42.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.197+0000 7f4c35600640 1 -- 172.21.15.42:0/845318129 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f4c2c041030 con 0x7f4c380b1e00 2024-07-04T17:28:42.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.197+0000 7f4c40200640 1 -- 172.21.15.42:0/845318129 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c3806c590 con 0x7f4c380b1e00 2024-07-04T17:28:42.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.197+0000 7f4c35600640 1 --2- 172.21.15.42:0/845318129 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f4c1803d4f0 0x7f4c1803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:42.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.197+0000 7f4c35600640 1 -- 172.21.15.42:0/845318129 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4311+0+0 (secure 0 0 0) 0x7f4c2c03f070 con 0x7f4c380b1e00 2024-07-04T17:28:42.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.198+0000 7f4c3ce00640 1 --2- 172.21.15.42:0/845318129 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f4c1803d4f0 0x7f4c1803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:42.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.198+0000 7f4c3ce00640 1 --2- 172.21.15.42:0/845318129 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f4c1803d4f0 0x7f4c1803fa00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4c3806ed80 tx=0x7f4c24007a30 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:42.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.203+0000 7f4c35600640 1 -- 172.21.15.42:0/845318129 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f4c2c04bd60 con 0x7f4c380b1e00 2024-07-04T17:28:42.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.453+0000 7f4c40200640 1 -- 172.21.15.42:0/845318129 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c38069b70 con 0x7f4c380b1e00 2024-07-04T17:28:42.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.454+0000 7f4c35600640 1 -- 172.21.15.42:0/845318129 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+6049 (secure 0 0 0) 0x7f4c2c030cc0 con 0x7f4c380b1e00 2024-07-04T17:28:42.456 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:28:42.456 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"btime":"2024-07-04T17:28:39:614520+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/399707138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":399707138},{"type":"v1","addr":"172.21.15.42:6827","nonce":399707138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4414,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1510714231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1510714231},{"type":"v1","addr":"172.21.15.42:6829","nonce":1510714231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/3069680926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":3069680926},{"type":"v1","addr":"172.21.15.42:6831","nonce":3069680926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4652,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2211433974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2211433974},{"type":"v1","addr":"172.21.15.42:6835","nonce":2211433974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:27:27.513977+0000","modified":"2024-07-04T17:27:30.569774+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4422":{"gid":4422,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.42:6833/1348446958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1348446958},{"type":"v1","addr":"172.21.15.42:6833","nonce":1348446958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4422,"qdb_cluster":[4422]},"id":3}]} 2024-07-04T17:28:42.456 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2024-07-04T17:28:42.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.457+0000 7f4c40200640 1 -- 172.21.15.42:0/845318129 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f4c1803d4f0 msgr2=0x7f4c1803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:42.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.457+0000 7f4c40200640 1 --2- 172.21.15.42:0/845318129 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f4c1803d4f0 0x7f4c1803fa00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4c3806ed80 tx=0x7f4c24007a30 comp rx=0 tx=0).stop 2024-07-04T17:28:42.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.457+0000 7f4c40200640 1 -- 172.21.15.42:0/845318129 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c380b1e00 msgr2=0x7f4c381d4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:42.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.457+0000 7f4c40200640 1 --2- 172.21.15.42:0/845318129 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c380b1e00 0x7f4c381d4df0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c034040 tx=0x7f4c2c0049d0 comp rx=0 tx=0).stop 2024-07-04T17:28:42.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.457+0000 7f4c40200640 1 -- 172.21.15.42:0/845318129 shutdown_connections 2024-07-04T17:28:42.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.457+0000 7f4c40200640 1 --2- 172.21.15.42:0/845318129 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f4c1803d4f0 0x7f4c1803fa00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:42.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.457+0000 7f4c40200640 1 --2- 172.21.15.42:0/845318129 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c380b1e00 0x7f4c381d4df0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:42.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.457+0000 7f4c40200640 1 -- 172.21.15.42:0/845318129 >> 172.21.15.42:0/845318129 conn(0x7f4c380bf700 msgr2=0x7f4c381412f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:42.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.457+0000 7f4c40200640 1 -- 172.21.15.42:0/845318129 shutdown_connections 2024-07-04T17:28:42.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.457+0000 7f4c40200640 1 -- 172.21.15.42:0/845318129 wait complete. 2024-07-04T17:28:42.485 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:28:42.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.770+0000 7f1617200640 1 -- 172.21.15.42:0/494371533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16180ba8b0 msgr2=0x7f16180b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:42.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.770+0000 7f1617200640 1 --2- 172.21.15.42:0/494371533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16180ba8b0 0x7f16180b3550 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f16080097b0 tx=0x7f160802a540 comp rx=0 tx=0).stop 2024-07-04T17:28:42.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.770+0000 7f1617200640 1 -- 172.21.15.42:0/494371533 shutdown_connections 2024-07-04T17:28:42.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.770+0000 7f1617200640 1 --2- 172.21.15.42:0/494371533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16180ba8b0 0x7f16180b3550 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:42.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.770+0000 7f1617200640 1 -- 172.21.15.42:0/494371533 >> 172.21.15.42:0/494371533 conn(0x7f16180bebc0 msgr2=0x7f16180befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:42.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.770+0000 7f1617200640 1 -- 172.21.15.42:0/494371533 shutdown_connections 2024-07-04T17:28:42.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.770+0000 7f1617200640 1 -- 172.21.15.42:0/494371533 wait complete. 2024-07-04T17:28:42.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.772+0000 7f1617200640 1 Processor -- start 2024-07-04T17:28:42.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.773+0000 7f1617200640 1 -- start start 2024-07-04T17:28:42.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.774+0000 7f1617200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16180b1d20 0x7f16180a9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:42.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.774+0000 7f1617200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f16180b8b40 con 0x7f16180b1d20 2024-07-04T17:28:42.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.774+0000 7f1615e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16180b1d20 0x7f16180a9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:42.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.774+0000 7f1615e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16180b1d20 0x7f16180a9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37838/0 (socket says 172.21.15.42:37838) 2024-07-04T17:28:42.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.774+0000 7f1615e00640 1 -- 172.21.15.42:0/1976681994 learned_addr learned my addr 172.21.15.42:0/1976681994 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:42.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.775+0000 7f1615e00640 1 -- 172.21.15.42:0/1976681994 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1618073160 con 0x7f16180b1d20 2024-07-04T17:28:42.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.775+0000 7f1615e00640 1 --2- 172.21.15.42:0/1976681994 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16180b1d20 0x7f16180a9800 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f16080098d0 tx=0x7f1608033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:42.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.775+0000 7f160e000640 1 -- 172.21.15.42:0/1976681994 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1608031bc0 con 0x7f16180b1d20 2024-07-04T17:28:42.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.775+0000 7f160e000640 1 -- 172.21.15.42:0/1976681994 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1608031e50 con 0x7f16180b1d20 2024-07-04T17:28:42.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.775+0000 7f1617200640 1 -- 172.21.15.42:0/1976681994 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1618073b00 con 0x7f16180b1d20 2024-07-04T17:28:42.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.775+0000 7f160e000640 1 -- 172.21.15.42:0/1976681994 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1608033200 con 0x7f16180b1d20 2024-07-04T17:28:42.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.776+0000 7f1617200640 1 -- 172.21.15.42:0/1976681994 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f161813e630 con 0x7f16180b1d20 2024-07-04T17:28:42.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.777+0000 7f1617200640 1 -- 172.21.15.42:0/1976681994 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1618143460 con 0x7f16180b1d20 2024-07-04T17:28:42.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.777+0000 7f160e000640 1 -- 172.21.15.42:0/1976681994 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f16080333a0 con 0x7f16180b1d20 2024-07-04T17:28:42.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.778+0000 7f160e000640 1 --2- 172.21.15.42:0/1976681994 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f15f403d4f0 0x7f15f403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:42.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.778+0000 7f160e000640 1 -- 172.21.15.42:0/1976681994 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4311+0+0 (secure 0 0 0) 0x7f16080732b0 con 0x7f16180b1d20 2024-07-04T17:28:42.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.778+0000 7f1615400640 1 --2- 172.21.15.42:0/1976681994 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f15f403d4f0 0x7f15f403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:42.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.779+0000 7f1615400640 1 --2- 172.21.15.42:0/1976681994 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f15f403d4f0 0x7f15f403fa00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f16181442c0 tx=0x7f1600007a90 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:42.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.783+0000 7f160e000640 1 -- 172.21.15.42:0/1976681994 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f1618143460 con 0x7f16180b1d20 2024-07-04T17:28:42.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.989+0000 7f1617200640 1 -- 172.21.15.42:0/1976681994 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f16180b9240 con 0x7f16180b1d20 2024-07-04T17:28:42.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.991+0000 7f160e000640 1 -- 172.21.15.42:0/1976681994 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+11862 (secure 0 0 0) 0x7f160803f030 con 0x7f16180b1d20 2024-07-04T17:28:42.992 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:28:42.992 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":37,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:28:41.861483+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:27:25.040056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:27:26.487236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6834/807364821":"2024-07-05T17:28:38.571021+0000","172.21.15.42:0/429805019":"2024-07-05T17:28:37.604461+0000","172.21.15.42:0/3791890040":"2024-07-05T17:28:37.604461+0000","172.21.15.42:0/1994443123":"2024-07-05T17:28:37.604461+0000","172.21.15.42:0/2983064181":"2024-07-05T17:28:37.604461+0000","172.21.15.42:6824/3523793950":"2024-07-05T17:27:40.654396+0000","172.21.15.42:6824/488387844":"2024-07-05T17:28:37.604461+0000","172.21.15.42:6825/3523793950":"2024-07-05T17:27:40.654396+0000","172.21.15.42:0/487218478":"2024-07-04T18:28:21.818727+0000","172.21.15.42:0/2974639683":"2024-07-05T17:27:40.654396+0000","172.21.15.42:0/880857241":"2024-07-05T17:27:40.654396+0000","172.21.15.42:6835/807364821":"2024-07-05T17:28:38.571021+0000","172.21.15.42:0/2267375347":"2024-07-05T17:28:37.604461+0000","172.21.15.42:6825/488387844":"2024-07-05T17:28:37.604461+0000","172.21.15.42:0/83074206":"2024-07-05T17:27:40.654396+0000","172.21.15.42:0/3526015921":"2024-07-05T17:27:40.654396+0000","172.21.15.42:0/1537192714":"2024-07-05T17:28:37.604461+0000","172.21.15.42:0/435904326":"2024-07-05T17:28:37.604461+0000","172.21.15.42:0/3378322019":"2024-07-05T17:28:37.604461+0000","172.21.15.42:0/2362095151":"2024-07-05T17:28:37.604461+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:28:42.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.994+0000 7f1617200640 1 -- 172.21.15.42:0/1976681994 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f15f403d4f0 msgr2=0x7f15f403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:42.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.994+0000 7f1617200640 1 --2- 172.21.15.42:0/1976681994 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f15f403d4f0 0x7f15f403fa00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f16181442c0 tx=0x7f1600007a90 comp rx=0 tx=0).stop 2024-07-04T17:28:42.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.994+0000 7f1617200640 1 -- 172.21.15.42:0/1976681994 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16180b1d20 msgr2=0x7f16180a9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:42.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.994+0000 7f1617200640 1 --2- 172.21.15.42:0/1976681994 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16180b1d20 0x7f16180a9800 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f16080098d0 tx=0x7f1608033e40 comp rx=0 tx=0).stop 2024-07-04T17:28:42.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.994+0000 7f1617200640 1 -- 172.21.15.42:0/1976681994 shutdown_connections 2024-07-04T17:28:42.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.994+0000 7f1617200640 1 --2- 172.21.15.42:0/1976681994 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f15f403d4f0 0x7f15f403fa00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:42.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.994+0000 7f1617200640 1 --2- 172.21.15.42:0/1976681994 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16180b1d20 0x7f16180a9800 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:42.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.994+0000 7f1617200640 1 -- 172.21.15.42:0/1976681994 >> 172.21.15.42:0/1976681994 conn(0x7f16180bebc0 msgr2=0x7f1618075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:42.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.994+0000 7f1617200640 1 -- 172.21.15.42:0/1976681994 shutdown_connections 2024-07-04T17:28:42.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:42.994+0000 7f1617200640 1 -- 172.21.15.42:0/1976681994 wait complete. 2024-07-04T17:28:43.021 INFO:tasks.cephfs.filesystem:Destroying file system backup_fs and related pools 2024-07-04T17:28:43.021 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:28:43.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.309+0000 7efda5400640 1 -- 172.21.15.42:0/1936267610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efda006cf90 msgr2=0x7efda0141dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:43.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.309+0000 7efda5400640 1 --2- 172.21.15.42:0/1936267610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efda006cf90 0x7efda0141dd0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7efd90005fd0 tx=0x7efd9002a5f0 comp rx=0 tx=0).stop 2024-07-04T17:28:43.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.309+0000 7efda5400640 1 -- 172.21.15.42:0/1936267610 shutdown_connections 2024-07-04T17:28:43.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.309+0000 7efda5400640 1 --2- 172.21.15.42:0/1936267610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efda006cf90 0x7efda0141dd0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:43.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.309+0000 7efda5400640 1 -- 172.21.15.42:0/1936267610 >> 172.21.15.42:0/1936267610 conn(0x7efda00bebc0 msgr2=0x7efda00befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:43.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.310+0000 7efda5400640 1 -- 172.21.15.42:0/1936267610 shutdown_connections 2024-07-04T17:28:43.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.310+0000 7efda5400640 1 -- 172.21.15.42:0/1936267610 wait complete. 2024-07-04T17:28:43.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.312+0000 7efda5400640 1 Processor -- start 2024-07-04T17:28:43.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.313+0000 7efda5400640 1 -- start start 2024-07-04T17:28:43.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.314+0000 7efda5400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efda006cf90 0x7efda00b83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:43.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.314+0000 7efda5400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efda00babe0 con 0x7efda006cf90 2024-07-04T17:28:43.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.315+0000 7efd9fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efda006cf90 0x7efda00b83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:43.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.315+0000 7efd9fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efda006cf90 0x7efda00b83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37856/0 (socket says 172.21.15.42:37856) 2024-07-04T17:28:43.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.315+0000 7efd9fe00640 1 -- 172.21.15.42:0/3862563515 learned_addr learned my addr 172.21.15.42:0/3862563515 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:43.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.315+0000 7efd9fe00640 1 -- 172.21.15.42:0/3862563515 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda00b7220 con 0x7efda006cf90 2024-07-04T17:28:43.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.315+0000 7efd9fe00640 1 --2- 172.21.15.42:0/3862563515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efda006cf90 0x7efda00b83a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7efd90009850 tx=0x7efd900049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:43.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.316+0000 7efd9cc00640 1 -- 172.21.15.42:0/3862563515 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd9003e030 con 0x7efda006cf90 2024-07-04T17:28:43.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.316+0000 7efda5400640 1 -- 172.21.15.42:0/3862563515 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda00b7990 con 0x7efda006cf90 2024-07-04T17:28:43.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.316+0000 7efd9cc00640 1 -- 172.21.15.42:0/3862563515 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd900035f0 con 0x7efda006cf90 2024-07-04T17:28:43.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.316+0000 7efd9cc00640 1 -- 172.21.15.42:0/3862563515 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd90035040 con 0x7efda006cf90 2024-07-04T17:28:43.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.316+0000 7efda5400640 1 -- 172.21.15.42:0/3862563515 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda0142ab0 con 0x7efda006cf90 2024-07-04T17:28:43.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.317+0000 7efda5400640 1 -- 172.21.15.42:0/3862563515 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efda0142da0 con 0x7efda006cf90 2024-07-04T17:28:43.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.318+0000 7efd9cc00640 1 -- 172.21.15.42:0/3862563515 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7efd90041030 con 0x7efda006cf90 2024-07-04T17:28:43.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.318+0000 7efd9cc00640 1 --2- 172.21.15.42:0/3862563515 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7efd7c03d4a0 0x7efd7c03f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:43.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.318+0000 7efd9cc00640 1 -- 172.21.15.42:0/3862563515 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4311+0+0 (secure 0 0 0) 0x7efd9003f070 con 0x7efda006cf90 2024-07-04T17:28:43.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.319+0000 7efd97e00640 1 --2- 172.21.15.42:0/3862563515 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7efd7c03d4a0 0x7efd7c03f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:43.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.319+0000 7efd97e00640 1 --2- 172.21.15.42:0/3862563515 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7efd7c03d4a0 0x7efd7c03f9b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7efd88001560 tx=0x7efd880079f0 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:43.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.323+0000 7efd9cc00640 1 -- 172.21.15.42:0/3862563515 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7efda0142da0 con 0x7efda006cf90 2024-07-04T17:28:43.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.584+0000 7efda5400640 1 -- 172.21.15.42:0/3862563515 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efda00bb1b0 con 0x7efda006cf90 2024-07-04T17:28:43.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.586+0000 7efd9cc00640 1 -- 172.21.15.42:0/3862563515 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+6049 (secure 0 0 0) 0x7efd90042ce0 con 0x7efda006cf90 2024-07-04T17:28:43.587 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:28:43.587 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"btime":"2024-07-04T17:28:39:614520+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/399707138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":399707138},{"type":"v1","addr":"172.21.15.42:6827","nonce":399707138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4414,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1510714231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1510714231},{"type":"v1","addr":"172.21.15.42:6829","nonce":1510714231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/3069680926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":3069680926},{"type":"v1","addr":"172.21.15.42:6831","nonce":3069680926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4652,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2211433974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2211433974},{"type":"v1","addr":"172.21.15.42:6835","nonce":2211433974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:27:27.513977+0000","modified":"2024-07-04T17:27:30.569774+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4422":{"gid":4422,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.42:6833/1348446958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1348446958},{"type":"v1","addr":"172.21.15.42:6833","nonce":1348446958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4422,"qdb_cluster":[4422]},"id":3}]} 2024-07-04T17:28:43.588 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2024-07-04T17:28:43.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.588+0000 7efda5400640 1 -- 172.21.15.42:0/3862563515 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7efd7c03d4a0 msgr2=0x7efd7c03f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:43.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.588+0000 7efda5400640 1 --2- 172.21.15.42:0/3862563515 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7efd7c03d4a0 0x7efd7c03f9b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7efd88001560 tx=0x7efd880079f0 comp rx=0 tx=0).stop 2024-07-04T17:28:43.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.588+0000 7efda5400640 1 -- 172.21.15.42:0/3862563515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efda006cf90 msgr2=0x7efda00b83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:43.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.588+0000 7efda5400640 1 --2- 172.21.15.42:0/3862563515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efda006cf90 0x7efda00b83a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7efd90009850 tx=0x7efd900049d0 comp rx=0 tx=0).stop 2024-07-04T17:28:43.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.588+0000 7efda5400640 1 -- 172.21.15.42:0/3862563515 shutdown_connections 2024-07-04T17:28:43.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.588+0000 7efda5400640 1 --2- 172.21.15.42:0/3862563515 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7efd7c03d4a0 0x7efd7c03f9b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:43.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.588+0000 7efda5400640 1 --2- 172.21.15.42:0/3862563515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efda006cf90 0x7efda00b83a0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:43.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.588+0000 7efda5400640 1 -- 172.21.15.42:0/3862563515 >> 172.21.15.42:0/3862563515 conn(0x7efda00bebc0 msgr2=0x7efda00b9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:43.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.588+0000 7efda5400640 1 -- 172.21.15.42:0/3862563515 shutdown_connections 2024-07-04T17:28:43.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.588+0000 7efda5400640 1 -- 172.21.15.42:0/3862563515 wait complete. 2024-07-04T17:28:43.615 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:28:43.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.886+0000 7f6c98000640 1 -- 172.21.15.42:0/2484802287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c900b38b0 msgr2=0x7f6c900b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:43.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.886+0000 7f6c98000640 1 --2- 172.21.15.42:0/2484802287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c900b38b0 0x7f6c900b2d20 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6c84005fd0 tx=0x7f6c8402a8b0 comp rx=0 tx=0).stop 2024-07-04T17:28:43.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.886+0000 7f6c98000640 1 -- 172.21.15.42:0/2484802287 shutdown_connections 2024-07-04T17:28:43.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.886+0000 7f6c98000640 1 --2- 172.21.15.42:0/2484802287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c900b38b0 0x7f6c900b2d20 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:43.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.886+0000 7f6c98000640 1 -- 172.21.15.42:0/2484802287 >> 172.21.15.42:0/2484802287 conn(0x7f6c900bf700 msgr2=0x7f6c900af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:43.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.886+0000 7f6c98000640 1 -- 172.21.15.42:0/2484802287 shutdown_connections 2024-07-04T17:28:43.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.886+0000 7f6c98000640 1 -- 172.21.15.42:0/2484802287 wait complete. 2024-07-04T17:28:43.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.888+0000 7f6c98000640 1 Processor -- start 2024-07-04T17:28:43.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.888+0000 7f6c98000640 1 -- start start 2024-07-04T17:28:43.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.889+0000 7f6c98000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c900b1e00 0x7f6c901d4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:43.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.889+0000 7f6c98000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6c900b06e0 con 0x7f6c900b1e00 2024-07-04T17:28:43.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.890+0000 7f6c95600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c900b1e00 0x7f6c901d4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:43.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.890+0000 7f6c95600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c900b1e00 0x7f6c901d4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54416/0 (socket says 172.21.15.42:54416) 2024-07-04T17:28:43.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.890+0000 7f6c95600640 1 -- 172.21.15.42:0/3928523031 learned_addr learned my addr 172.21.15.42:0/3928523031 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:43.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.890+0000 7f6c95600640 1 -- 172.21.15.42:0/3928523031 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c901d6740 con 0x7f6c900b1e00 2024-07-04T17:28:43.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.890+0000 7f6c95600640 1 --2- 172.21.15.42:0/3928523031 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c900b1e00 0x7f6c901d4df0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6c84006010 tx=0x7f6c84033b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:43.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.891+0000 7f6c8d600640 1 -- 172.21.15.42:0/3928523031 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c840319f0 con 0x7f6c900b1e00 2024-07-04T17:28:43.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.891+0000 7f6c8d600640 1 -- 172.21.15.42:0/3928523031 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c84031b90 con 0x7f6c900b1e00 2024-07-04T17:28:43.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.891+0000 7f6c8d600640 1 -- 172.21.15.42:0/3928523031 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c84033070 con 0x7f6c900b1e00 2024-07-04T17:28:43.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.891+0000 7f6c98000640 1 -- 172.21.15.42:0/3928523031 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c900aa540 con 0x7f6c900b1e00 2024-07-04T17:28:43.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.891+0000 7f6c98000640 1 -- 172.21.15.42:0/3928523031 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c901d5330 con 0x7f6c900b1e00 2024-07-04T17:28:43.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.892+0000 7f6c8d600640 1 -- 172.21.15.42:0/3928523031 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f6c84033210 con 0x7f6c900b1e00 2024-07-04T17:28:43.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.892+0000 7f6c98000640 1 -- 172.21.15.42:0/3928523031 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c9006c590 con 0x7f6c900b1e00 2024-07-04T17:28:43.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.893+0000 7f6c8d600640 1 --2- 172.21.15.42:0/3928523031 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f6c7003d4f0 0x7f6c7003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:43.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.893+0000 7f6c8d600640 1 -- 172.21.15.42:0/3928523031 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4311+0+0 (secure 0 0 0) 0x7f6c84073450 con 0x7f6c900b1e00 2024-07-04T17:28:43.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.893+0000 7f6c94c00640 1 --2- 172.21.15.42:0/3928523031 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f6c7003d4f0 0x7f6c7003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:43.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.894+0000 7f6c94c00640 1 --2- 172.21.15.42:0/3928523031 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f6c7003d4f0 0x7f6c7003fa00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6c9006ed80 tx=0x7f6c7c007a90 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:43.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:43.898+0000 7f6c8d600640 1 -- 172.21.15.42:0/3928523031 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f6c8403f030 con 0x7f6c900b1e00 2024-07-04T17:28:44.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.141+0000 7f6c98000640 1 -- 172.21.15.42:0/3928523031 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c90069b70 con 0x7f6c900b1e00 2024-07-04T17:28:44.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.142+0000 7f6c8d600640 1 -- 172.21.15.42:0/3928523031 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+6049 (secure 0 0 0) 0x7f6c84033500 con 0x7f6c900b1e00 2024-07-04T17:28:44.144 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:28:44.144 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"btime":"2024-07-04T17:28:39:614520+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/399707138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":399707138},{"type":"v1","addr":"172.21.15.42:6827","nonce":399707138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4414,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1510714231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1510714231},{"type":"v1","addr":"172.21.15.42:6829","nonce":1510714231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/3069680926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":3069680926},{"type":"v1","addr":"172.21.15.42:6831","nonce":3069680926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4652,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2211433974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2211433974},{"type":"v1","addr":"172.21.15.42:6835","nonce":2211433974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:27:27.513977+0000","modified":"2024-07-04T17:27:30.569774+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4422":{"gid":4422,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.42:6833/1348446958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1348446958},{"type":"v1","addr":"172.21.15.42:6833","nonce":1348446958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4422,"qdb_cluster":[4422]},"id":3}]} 2024-07-04T17:28:44.144 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2024-07-04T17:28:44.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.145+0000 7f6c98000640 1 -- 172.21.15.42:0/3928523031 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f6c7003d4f0 msgr2=0x7f6c7003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:44.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.145+0000 7f6c98000640 1 --2- 172.21.15.42:0/3928523031 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f6c7003d4f0 0x7f6c7003fa00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6c9006ed80 tx=0x7f6c7c007a90 comp rx=0 tx=0).stop 2024-07-04T17:28:44.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.145+0000 7f6c98000640 1 -- 172.21.15.42:0/3928523031 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c900b1e00 msgr2=0x7f6c901d4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:44.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.145+0000 7f6c98000640 1 --2- 172.21.15.42:0/3928523031 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c900b1e00 0x7f6c901d4df0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6c84006010 tx=0x7f6c84033b10 comp rx=0 tx=0).stop 2024-07-04T17:28:44.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.145+0000 7f6c98000640 1 -- 172.21.15.42:0/3928523031 shutdown_connections 2024-07-04T17:28:44.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.145+0000 7f6c98000640 1 --2- 172.21.15.42:0/3928523031 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f6c7003d4f0 0x7f6c7003fa00 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:44.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.145+0000 7f6c98000640 1 --2- 172.21.15.42:0/3928523031 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c900b1e00 0x7f6c901d4df0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:44.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.145+0000 7f6c98000640 1 -- 172.21.15.42:0/3928523031 >> 172.21.15.42:0/3928523031 conn(0x7f6c900bf700 msgr2=0x7f6c901412f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:44.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.146+0000 7f6c98000640 1 -- 172.21.15.42:0/3928523031 shutdown_connections 2024-07-04T17:28:44.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.146+0000 7f6c98000640 1 -- 172.21.15.42:0/3928523031 wait complete. 2024-07-04T17:28:44.174 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:28:44.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.460+0000 7f5110000640 1 -- 172.21.15.42:0/1143189287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51080b38b0 msgr2=0x7f51080b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:44.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.460+0000 7f5110000640 1 --2- 172.21.15.42:0/1143189287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51080b38b0 0x7f51080b2d20 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f50fc0097b0 tx=0x7f50fc02a540 comp rx=0 tx=0).stop 2024-07-04T17:28:44.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.460+0000 7f5110000640 1 -- 172.21.15.42:0/1143189287 shutdown_connections 2024-07-04T17:28:44.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.460+0000 7f5110000640 1 --2- 172.21.15.42:0/1143189287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51080b38b0 0x7f51080b2d20 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:44.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.460+0000 7f5110000640 1 -- 172.21.15.42:0/1143189287 >> 172.21.15.42:0/1143189287 conn(0x7f51080bf700 msgr2=0x7f51080af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:44.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.460+0000 7f5110000640 1 -- 172.21.15.42:0/1143189287 shutdown_connections 2024-07-04T17:28:44.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.460+0000 7f5110000640 1 -- 172.21.15.42:0/1143189287 wait complete. 2024-07-04T17:28:44.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.463+0000 7f5110000640 1 Processor -- start 2024-07-04T17:28:44.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.463+0000 7f5110000640 1 -- start start 2024-07-04T17:28:44.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.464+0000 7f5110000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51080b1e00 0x7f51080ad7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:44.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.464+0000 7f5110000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f51080b06e0 con 0x7f51080b1e00 2024-07-04T17:28:44.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.464+0000 7f510d600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51080b1e00 0x7f51080ad7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:44.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.464+0000 7f510d600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51080b1e00 0x7f51080ad7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54442/0 (socket says 172.21.15.42:54442) 2024-07-04T17:28:44.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.464+0000 7f510d600640 1 -- 172.21.15.42:0/524647566 learned_addr learned my addr 172.21.15.42:0/524647566 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:44.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.465+0000 7f510d600640 1 -- 172.21.15.42:0/524647566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51080add70 con 0x7f51080b1e00 2024-07-04T17:28:44.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.465+0000 7f510d600640 1 --2- 172.21.15.42:0/524647566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51080b1e00 0x7f51080ad7e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f50fc0098d0 tx=0x7f50fc033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:44.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.465+0000 7f5105600640 1 -- 172.21.15.42:0/524647566 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50fc031bc0 con 0x7f51080b1e00 2024-07-04T17:28:44.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.465+0000 7f5105600640 1 -- 172.21.15.42:0/524647566 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50fc031e50 con 0x7f51080b1e00 2024-07-04T17:28:44.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.465+0000 7f5105600640 1 -- 172.21.15.42:0/524647566 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50fc033200 con 0x7f51080b1e00 2024-07-04T17:28:44.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.466+0000 7f5110000640 1 -- 172.21.15.42:0/524647566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51080aecd0 con 0x7f51080b1e00 2024-07-04T17:28:44.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.466+0000 7f5110000640 1 -- 172.21.15.42:0/524647566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f510813f2f0 con 0x7f51080b1e00 2024-07-04T17:28:44.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.467+0000 7f5105600640 1 -- 172.21.15.42:0/524647566 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f50fc0333a0 con 0x7f51080b1e00 2024-07-04T17:28:44.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.467+0000 7f5110000640 1 -- 172.21.15.42:0/524647566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f510806c590 con 0x7f51080b1e00 2024-07-04T17:28:44.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.467+0000 7f5105600640 1 --2- 172.21.15.42:0/524647566 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f50e803d4f0 0x7f50e803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:44.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.468+0000 7f5105600640 1 -- 172.21.15.42:0/524647566 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4311+0+0 (secure 0 0 0) 0x7f50fc073220 con 0x7f51080b1e00 2024-07-04T17:28:44.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.468+0000 7f510cc00640 1 --2- 172.21.15.42:0/524647566 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f50e803d4f0 0x7f50e803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:44.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.469+0000 7f510cc00640 1 --2- 172.21.15.42:0/524647566 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f50e803d4f0 0x7f50e803fa00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f510806ed80 tx=0x7f50f4007a90 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:44.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.473+0000 7f5105600640 1 -- 172.21.15.42:0/524647566 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f50fc03f030 con 0x7f51080b1e00 2024-07-04T17:28:44.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.688+0000 7f5110000640 1 -- 172.21.15.42:0/524647566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5108069b70 con 0x7f51080b1e00 2024-07-04T17:28:44.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.690+0000 7f5105600640 1 -- 172.21.15.42:0/524647566 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+11862 (secure 0 0 0) 0x7f50fc033690 con 0x7f51080b1e00 2024-07-04T17:28:44.692 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:28:44.692 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":37,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:28:41.861483+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:27:25.040056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:27:26.487236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6834/807364821":"2024-07-05T17:28:38.571021+0000","172.21.15.42:0/429805019":"2024-07-05T17:28:37.604461+0000","172.21.15.42:0/3791890040":"2024-07-05T17:28:37.604461+0000","172.21.15.42:0/1994443123":"2024-07-05T17:28:37.604461+0000","172.21.15.42:0/2983064181":"2024-07-05T17:28:37.604461+0000","172.21.15.42:6824/3523793950":"2024-07-05T17:27:40.654396+0000","172.21.15.42:6824/488387844":"2024-07-05T17:28:37.604461+0000","172.21.15.42:6825/3523793950":"2024-07-05T17:27:40.654396+0000","172.21.15.42:0/487218478":"2024-07-04T18:28:21.818727+0000","172.21.15.42:0/2974639683":"2024-07-05T17:27:40.654396+0000","172.21.15.42:0/880857241":"2024-07-05T17:27:40.654396+0000","172.21.15.42:6835/807364821":"2024-07-05T17:28:38.571021+0000","172.21.15.42:0/2267375347":"2024-07-05T17:28:37.604461+0000","172.21.15.42:6825/488387844":"2024-07-05T17:28:37.604461+0000","172.21.15.42:0/83074206":"2024-07-05T17:27:40.654396+0000","172.21.15.42:0/3526015921":"2024-07-05T17:27:40.654396+0000","172.21.15.42:0/1537192714":"2024-07-05T17:28:37.604461+0000","172.21.15.42:0/435904326":"2024-07-05T17:28:37.604461+0000","172.21.15.42:0/3378322019":"2024-07-05T17:28:37.604461+0000","172.21.15.42:0/2362095151":"2024-07-05T17:28:37.604461+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:28:44.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.693+0000 7f5110000640 1 -- 172.21.15.42:0/524647566 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f50e803d4f0 msgr2=0x7f50e803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:44.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.693+0000 7f5110000640 1 --2- 172.21.15.42:0/524647566 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f50e803d4f0 0x7f50e803fa00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f510806ed80 tx=0x7f50f4007a90 comp rx=0 tx=0).stop 2024-07-04T17:28:44.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.693+0000 7f5110000640 1 -- 172.21.15.42:0/524647566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51080b1e00 msgr2=0x7f51080ad7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:44.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.693+0000 7f5110000640 1 --2- 172.21.15.42:0/524647566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51080b1e00 0x7f51080ad7e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f50fc0098d0 tx=0x7f50fc033e40 comp rx=0 tx=0).stop 2024-07-04T17:28:44.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.693+0000 7f5110000640 1 -- 172.21.15.42:0/524647566 shutdown_connections 2024-07-04T17:28:44.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.693+0000 7f5110000640 1 --2- 172.21.15.42:0/524647566 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f50e803d4f0 0x7f50e803fa00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:44.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.693+0000 7f5110000640 1 --2- 172.21.15.42:0/524647566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51080b1e00 0x7f51080ad7e0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:44.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.693+0000 7f5110000640 1 -- 172.21.15.42:0/524647566 >> 172.21.15.42:0/524647566 conn(0x7f51080bf700 msgr2=0x7f51080bd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:44.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.693+0000 7f5110000640 1 -- 172.21.15.42:0/524647566 shutdown_connections 2024-07-04T17:28:44.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.693+0000 7f5110000640 1 -- 172.21.15.42:0/524647566 wait complete. 2024-07-04T17:28:44.722 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail backup_fs 2024-07-04T17:28:44.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.997+0000 7ff839000640 1 -- 172.21.15.42:0/3262396631 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff83406c020 msgr2=0x7ff83414a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:44.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.997+0000 7ff839000640 1 --2- 172.21.15.42:0/3262396631 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff83406c020 0x7ff83414a9e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff81c005fd0 tx=0x7ff81c02a560 comp rx=0 tx=0).stop 2024-07-04T17:28:44.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.998+0000 7ff839000640 1 -- 172.21.15.42:0/3262396631 shutdown_connections 2024-07-04T17:28:45.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.998+0000 7ff839000640 1 --2- 172.21.15.42:0/3262396631 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff83406c020 0x7ff83414a9e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:45.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.998+0000 7ff839000640 1 -- 172.21.15.42:0/3262396631 >> 172.21.15.42:0/3262396631 conn(0x7ff8340bebc0 msgr2=0x7ff8340befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:45.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.998+0000 7ff839000640 1 -- 172.21.15.42:0/3262396631 shutdown_connections 2024-07-04T17:28:45.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:44.998+0000 7ff839000640 1 -- 172.21.15.42:0/3262396631 wait complete. 2024-07-04T17:28:45.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.000+0000 7ff839000640 1 Processor -- start 2024-07-04T17:28:45.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.000+0000 7ff839000640 1 -- start start 2024-07-04T17:28:45.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.001+0000 7ff839000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff83406c020 0x7ff8341d5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:45.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.001+0000 7ff839000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff83414b500 con 0x7ff83406c020 2024-07-04T17:28:45.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.001+0000 7ff833400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff83406c020 0x7ff8341d5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:45.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.001+0000 7ff833400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff83406c020 0x7ff8341d5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54458/0 (socket says 172.21.15.42:54458) 2024-07-04T17:28:45.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.001+0000 7ff833400640 1 -- 172.21.15.42:0/2492511464 learned_addr learned my addr 172.21.15.42:0/2492511464 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:45.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.002+0000 7ff833400640 1 -- 172.21.15.42:0/2492511464 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8340a9da0 con 0x7ff83406c020 2024-07-04T17:28:45.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.002+0000 7ff833400640 1 --2- 172.21.15.42:0/2492511464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff83406c020 0x7ff8341d5620 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff824001eb0 tx=0x7ff82400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:45.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.002+0000 7ff82b400640 1 -- 172.21.15.42:0/2492511464 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff824017070 con 0x7ff83406c020 2024-07-04T17:28:45.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.002+0000 7ff82b400640 1 -- 172.21.15.42:0/2492511464 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8240052f0 con 0x7ff83406c020 2024-07-04T17:28:45.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.002+0000 7ff82b400640 1 -- 172.21.15.42:0/2492511464 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8240035f0 con 0x7ff83406c020 2024-07-04T17:28:45.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.002+0000 7ff839000640 1 -- 172.21.15.42:0/2492511464 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8340a9990 con 0x7ff83406c020 2024-07-04T17:28:45.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.003+0000 7ff839000640 1 -- 172.21.15.42:0/2492511464 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff834149980 con 0x7ff83406c020 2024-07-04T17:28:45.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.004+0000 7ff839000640 1 -- 172.21.15.42:0/2492511464 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8340a9bc0 con 0x7ff83406c020 2024-07-04T17:28:45.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.004+0000 7ff82b400640 1 -- 172.21.15.42:0/2492511464 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7ff82401a070 con 0x7ff83406c020 2024-07-04T17:28:45.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.004+0000 7ff82b400640 1 --2- 172.21.15.42:0/2492511464 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7ff81003d4f0 0x7ff81003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:45.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.004+0000 7ff82b400640 1 -- 172.21.15.42:0/2492511464 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4311+0+0 (secure 0 0 0) 0x7ff824050ed0 con 0x7ff83406c020 2024-07-04T17:28:45.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.005+0000 7ff82be00640 1 --2- 172.21.15.42:0/2492511464 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7ff81003d4f0 0x7ff81003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:45.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.005+0000 7ff82be00640 1 --2- 172.21.15.42:0/2492511464 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7ff81003d4f0 0x7ff81003fa00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff81c005fd0 tx=0x7ff81c004730 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:45.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.009+0000 7ff82b400640 1 -- 172.21.15.42:0/2492511464 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7ff8340a9bc0 con 0x7ff83406c020 2024-07-04T17:28:45.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.256+0000 7ff839000640 1 -- 172.21.15.42:0/2492511464 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "backup_fs"} v 0) -- 0x7ff83414bb40 con 0x7ff83406c020 2024-07-04T17:28:45.259 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-07-04T17:28:45.258+0000 7f8616e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-04T17:28:45.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.260+0000 7ff82b400640 1 -- 172.21.15.42:0/2492511464 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "backup_fs"}]=0 backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) ==== 167+0+0 (secure 0 0 0) 0x7ff834149980 con 0x7ff83406c020 2024-07-04T17:28:45.261 INFO:teuthology.orchestra.run.smithi042.stderr:backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-04T17:28:45.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.264+0000 7ff839000640 1 -- 172.21.15.42:0/2492511464 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7ff81003d4f0 msgr2=0x7ff81003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:45.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.264+0000 7ff839000640 1 --2- 172.21.15.42:0/2492511464 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7ff81003d4f0 0x7ff81003fa00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff81c005fd0 tx=0x7ff81c004730 comp rx=0 tx=0).stop 2024-07-04T17:28:45.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.264+0000 7ff839000640 1 -- 172.21.15.42:0/2492511464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff83406c020 msgr2=0x7ff8341d5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:45.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.264+0000 7ff839000640 1 --2- 172.21.15.42:0/2492511464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff83406c020 0x7ff8341d5620 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff824001eb0 tx=0x7ff82400c810 comp rx=0 tx=0).stop 2024-07-04T17:28:45.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.264+0000 7ff839000640 1 -- 172.21.15.42:0/2492511464 shutdown_connections 2024-07-04T17:28:45.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.264+0000 7ff839000640 1 --2- 172.21.15.42:0/2492511464 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7ff81003d4f0 0x7ff81003fa00 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:45.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.264+0000 7ff839000640 1 --2- 172.21.15.42:0/2492511464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff83406c020 0x7ff8341d5620 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:45.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.264+0000 7ff839000640 1 -- 172.21.15.42:0/2492511464 >> 172.21.15.42:0/2492511464 conn(0x7ff8340bebc0 msgr2=0x7ff8340b2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:45.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.265+0000 7ff839000640 1 -- 172.21.15.42:0/2492511464 shutdown_connections 2024-07-04T17:28:45.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.265+0000 7ff839000640 1 -- 172.21.15.42:0/2492511464 wait complete. 2024-07-04T17:28:45.276 INFO:tasks.ceph.mds.a.smithi042.stderr: -4606> 2024-07-04T17:27:30.559+0000 7f63cf400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-04T17:28:45.303 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm backup_fs --yes-i-really-mean-it 2024-07-04T17:28:45.420 INFO:tasks.ceph.mds.a.smithi042.stdout:starting mds.a at 2024-07-04T17:28:45.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.580+0000 7f5882a00640 1 -- 172.21.15.42:0/1827007315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f587c0b02b0 msgr2=0x7f587c073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:45.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.580+0000 7f5882a00640 1 --2- 172.21.15.42:0/1827007315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f587c0b02b0 0x7f587c073c90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f586c005fd0 tx=0x7f586c02a5f0 comp rx=0 tx=0).stop 2024-07-04T17:28:45.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.580+0000 7f5882a00640 1 -- 172.21.15.42:0/1827007315 shutdown_connections 2024-07-04T17:28:45.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.580+0000 7f5882a00640 1 --2- 172.21.15.42:0/1827007315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f587c0b02b0 0x7f587c073c90 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:45.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.580+0000 7f5882a00640 1 -- 172.21.15.42:0/1827007315 >> 172.21.15.42:0/1827007315 conn(0x7f587c06fe40 msgr2=0x7f587c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:45.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.580+0000 7f5882a00640 1 -- 172.21.15.42:0/1827007315 shutdown_connections 2024-07-04T17:28:45.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.580+0000 7f5882a00640 1 -- 172.21.15.42:0/1827007315 wait complete. 2024-07-04T17:28:45.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.582+0000 7f5882a00640 1 Processor -- start 2024-07-04T17:28:45.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.582+0000 7f5882a00640 1 -- start start 2024-07-04T17:28:45.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.583+0000 7f5882a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f587c0741d0 0x7f587c1e6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:45.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.583+0000 7f5882a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f587c141790 con 0x7f587c0741d0 2024-07-04T17:28:45.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.584+0000 7f587be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f587c0741d0 0x7f587c1e6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:45.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.584+0000 7f587be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f587c0741d0 0x7f587c1e6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54484/0 (socket says 172.21.15.42:54484) 2024-07-04T17:28:45.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.584+0000 7f587be00640 1 -- 172.21.15.42:0/2989397995 learned_addr learned my addr 172.21.15.42:0/2989397995 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:45.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.584+0000 7f587be00640 1 -- 172.21.15.42:0/2989397995 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587c0b5db0 con 0x7f587c0741d0 2024-07-04T17:28:45.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.584+0000 7f587be00640 1 --2- 172.21.15.42:0/2989397995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f587c0741d0 0x7f587c1e6b70 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f586c009850 tx=0x7f586c0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:45.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.585+0000 7f5873e00640 1 -- 172.21.15.42:0/2989397995 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f586c03e030 con 0x7f587c0741d0 2024-07-04T17:28:45.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.585+0000 7f5873e00640 1 -- 172.21.15.42:0/2989397995 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f586c0035f0 con 0x7f587c0741d0 2024-07-04T17:28:45.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.585+0000 7f5882a00640 1 -- 172.21.15.42:0/2989397995 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f587c0b6050 con 0x7f587c0741d0 2024-07-04T17:28:45.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.585+0000 7f5873e00640 1 -- 172.21.15.42:0/2989397995 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f586c035040 con 0x7f587c0741d0 2024-07-04T17:28:45.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.585+0000 7f5882a00640 1 -- 172.21.15.42:0/2989397995 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f587c0bffa0 con 0x7f587c0741d0 2024-07-04T17:28:45.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.586+0000 7f5882a00640 1 -- 172.21.15.42:0/2989397995 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f587c073c90 con 0x7f587c0741d0 2024-07-04T17:28:45.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.586+0000 7f5873e00640 1 -- 172.21.15.42:0/2989397995 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f586c041030 con 0x7f587c0741d0 2024-07-04T17:28:45.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.587+0000 7f5873e00640 1 --2- 172.21.15.42:0/2989397995 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f585c03d4f0 0x7f585c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:45.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.587+0000 7f5873e00640 1 -- 172.21.15.42:0/2989397995 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4397+0+0 (secure 0 0 0) 0x7f586c03f070 con 0x7f587c0741d0 2024-07-04T17:28:45.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.587+0000 7f587b400640 1 --2- 172.21.15.42:0/2989397995 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f585c03d4f0 0x7f585c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:45.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.588+0000 7f587b400640 1 --2- 172.21.15.42:0/2989397995 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f585c03d4f0 0x7f585c03fa00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5868001560 tx=0x7f58680079f0 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:45.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.592+0000 7f5873e00640 1 -- 172.21.15.42:0/2989397995 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f586c04bd60 con 0x7f587c0741d0 2024-07-04T17:28:45.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:45.853+0000 7f5882a00640 1 -- 172.21.15.42:0/2989397995 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true} v 0) -- 0x7f587c144750 con 0x7f587c0741d0 2024-07-04T17:28:46.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.262+0000 7f5873e00640 1 -- 172.21.15.42:0/2989397995 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true}]=0 v29) ==== 107+0+0 (secure 0 0 0) 0x7f586c030cc0 con 0x7f587c0741d0 2024-07-04T17:28:46.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.266+0000 7f5882a00640 1 -- 172.21.15.42:0/2989397995 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f585c03d4f0 msgr2=0x7f585c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:46.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.266+0000 7f5882a00640 1 --2- 172.21.15.42:0/2989397995 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f585c03d4f0 0x7f585c03fa00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5868001560 tx=0x7f58680079f0 comp rx=0 tx=0).stop 2024-07-04T17:28:46.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.266+0000 7f5882a00640 1 -- 172.21.15.42:0/2989397995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f587c0741d0 msgr2=0x7f587c1e6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:46.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.266+0000 7f5882a00640 1 --2- 172.21.15.42:0/2989397995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f587c0741d0 0x7f587c1e6b70 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f586c009850 tx=0x7f586c0049d0 comp rx=0 tx=0).stop 2024-07-04T17:28:46.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.266+0000 7f5882a00640 1 -- 172.21.15.42:0/2989397995 shutdown_connections 2024-07-04T17:28:46.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.266+0000 7f5882a00640 1 --2- 172.21.15.42:0/2989397995 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f585c03d4f0 0x7f585c03fa00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:46.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.266+0000 7f5882a00640 1 --2- 172.21.15.42:0/2989397995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f587c0741d0 0x7f587c1e6b70 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:46.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.266+0000 7f5882a00640 1 -- 172.21.15.42:0/2989397995 >> 172.21.15.42:0/2989397995 conn(0x7f587c06fe40 msgr2=0x7f587c140ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:46.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.266+0000 7f5882a00640 1 -- 172.21.15.42:0/2989397995 shutdown_connections 2024-07-04T17:28:46.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.266+0000 7f5882a00640 1 -- 172.21.15.42:0/2989397995 wait complete. 2024-07-04T17:28:46.301 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_metadata backup_fs_metadata --yes-i-really-really-mean-it 2024-07-04T17:28:46.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.598+0000 7f6b42a00640 1 -- 172.21.15.42:0/4148467869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b3c073a20 msgr2=0x7f6b3c13e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:46.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.598+0000 7f6b42a00640 1 --2- 172.21.15.42:0/4148467869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b3c073a20 0x7f6b3c13e1a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6b24005fd0 tx=0x7f6b2402a5f0 comp rx=0 tx=0).stop 2024-07-04T17:28:46.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.599+0000 7f6b42a00640 1 -- 172.21.15.42:0/4148467869 shutdown_connections 2024-07-04T17:28:46.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.599+0000 7f6b42a00640 1 --2- 172.21.15.42:0/4148467869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b3c073a20 0x7f6b3c13e1a0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:46.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.599+0000 7f6b42a00640 1 -- 172.21.15.42:0/4148467869 >> 172.21.15.42:0/4148467869 conn(0x7f6b3c06c020 msgr2=0x7f6b3c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:46.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.599+0000 7f6b42a00640 1 -- 172.21.15.42:0/4148467869 shutdown_connections 2024-07-04T17:28:46.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.599+0000 7f6b42a00640 1 -- 172.21.15.42:0/4148467869 wait complete. 2024-07-04T17:28:46.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.601+0000 7f6b42a00640 1 Processor -- start 2024-07-04T17:28:46.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.601+0000 7f6b42a00640 1 -- start start 2024-07-04T17:28:46.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.602+0000 7f6b42a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b3c06d070 0x7f6b3c1d44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:46.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.602+0000 7f6b42a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c071c40 con 0x7f6b3c06d070 2024-07-04T17:28:46.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.602+0000 7f6b3be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b3c06d070 0x7f6b3c1d44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:46.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.602+0000 7f6b3be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b3c06d070 0x7f6b3c1d44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54504/0 (socket says 172.21.15.42:54504) 2024-07-04T17:28:46.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.603+0000 7f6b3be00640 1 -- 172.21.15.42:0/1083774722 learned_addr learned my addr 172.21.15.42:0/1083774722 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:46.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.603+0000 7f6b3be00640 1 -- 172.21.15.42:0/1083774722 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b3c1d5a70 con 0x7f6b3c06d070 2024-07-04T17:28:46.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.603+0000 7f6b3be00640 1 --2- 172.21.15.42:0/1083774722 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b3c06d070 0x7f6b3c1d44e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c00c320 tx=0x7f6b2c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:46.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.603+0000 7f6b38c00640 1 -- 172.21.15.42:0/1083774722 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b2c017070 con 0x7f6b3c06d070 2024-07-04T17:28:46.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.604+0000 7f6b38c00640 1 -- 172.21.15.42:0/1083774722 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b2c005310 con 0x7f6b3c06d070 2024-07-04T17:28:46.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.604+0000 7f6b42a00640 1 -- 172.21.15.42:0/1083774722 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b3c0aed60 con 0x7f6b3c06d070 2024-07-04T17:28:46.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.604+0000 7f6b38c00640 1 -- 172.21.15.42:0/1083774722 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b2c0035f0 con 0x7f6b3c06d070 2024-07-04T17:28:46.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.604+0000 7f6b42a00640 1 -- 172.21.15.42:0/1083774722 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b3c0af230 con 0x7f6b3c06d070 2024-07-04T17:28:46.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.605+0000 7f6b42a00640 1 -- 172.21.15.42:0/1083774722 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b3c13e1a0 con 0x7f6b3c06d070 2024-07-04T17:28:46.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.605+0000 7f6b38c00640 1 -- 172.21.15.42:0/1083774722 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f6b2c01a070 con 0x7f6b3c06d070 2024-07-04T17:28:46.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.606+0000 7f6b38c00640 1 --2- 172.21.15.42:0/1083774722 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f6b1803d4f0 0x7f6b1803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:46.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.606+0000 7f6b38c00640 1 -- 172.21.15.42:0/1083774722 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4397+0+0 (secure 0 0 0) 0x7f6b2c050ed0 con 0x7f6b3c06d070 2024-07-04T17:28:46.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.606+0000 7f6b33400640 1 --2- 172.21.15.42:0/1083774722 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f6b1803d4f0 0x7f6b1803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:46.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.607+0000 7f6b33400640 1 --2- 172.21.15.42:0/1083774722 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f6b1803d4f0 0x7f6b1803fa00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6b24033000 tx=0x7f6b24004730 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:46.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.612+0000 7f6b38c00640 1 -- 172.21.15.42:0/1083774722 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f6b3c0af230 con 0x7f6b3c06d070 2024-07-04T17:28:46.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:46.822+0000 7f6b42a00640 1 -- 172.21.15.42:0/1083774722 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6b3c069b70 con 0x7f6b3c06d070 2024-07-04T17:28:47.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.271+0000 7f6b38c00640 1 -- 172.21.15.42:0/1083774722 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_metadata' removed v39) ==== 190+0+0 (secure 0 0 0) 0x7f6b3c06d480 con 0x7f6b3c06d070 2024-07-04T17:28:47.273 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'backup_fs_metadata' removed 2024-07-04T17:28:47.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.276+0000 7f6b42a00640 1 -- 172.21.15.42:0/1083774722 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f6b1803d4f0 msgr2=0x7f6b1803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:47.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.276+0000 7f6b42a00640 1 --2- 172.21.15.42:0/1083774722 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f6b1803d4f0 0x7f6b1803fa00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6b24033000 tx=0x7f6b24004730 comp rx=0 tx=0).stop 2024-07-04T17:28:47.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.276+0000 7f6b42a00640 1 -- 172.21.15.42:0/1083774722 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b3c06d070 msgr2=0x7f6b3c1d44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:47.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.276+0000 7f6b42a00640 1 --2- 172.21.15.42:0/1083774722 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b3c06d070 0x7f6b3c1d44e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c00c320 tx=0x7f6b2c00c7e0 comp rx=0 tx=0).stop 2024-07-04T17:28:47.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.277+0000 7f6b42a00640 1 -- 172.21.15.42:0/1083774722 shutdown_connections 2024-07-04T17:28:47.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.277+0000 7f6b42a00640 1 --2- 172.21.15.42:0/1083774722 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f6b1803d4f0 0x7f6b1803fa00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:47.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.277+0000 7f6b42a00640 1 --2- 172.21.15.42:0/1083774722 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b3c06d070 0x7f6b3c1d44e0 secure :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c00c320 tx=0x7f6b2c00c7e0 comp rx=0 tx=0).stop 2024-07-04T17:28:47.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.277+0000 7f6b42a00640 1 -- 172.21.15.42:0/1083774722 >> 172.21.15.42:0/1083774722 conn(0x7f6b3c06c020 msgr2=0x7f6b3c13c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:47.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.278+0000 7f6b42a00640 1 -- 172.21.15.42:0/1083774722 shutdown_connections 2024-07-04T17:28:47.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.278+0000 7f6b42a00640 1 -- 172.21.15.42:0/1083774722 wait complete. 2024-07-04T17:28:47.321 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_data backup_fs_data --yes-i-really-really-mean-it 2024-07-04T17:28:47.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.610+0000 7f36dfe00640 1 -- 172.21.15.42:0/2454040218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36e00701d0 msgr2=0x7f36e00abe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:47.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.610+0000 7f36dfe00640 1 --2- 172.21.15.42:0/2454040218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36e00701d0 0x7f36e00abe40 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f36d00097b0 tx=0x7f36d002a540 comp rx=0 tx=0).stop 2024-07-04T17:28:47.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.610+0000 7f36dfe00640 1 -- 172.21.15.42:0/2454040218 shutdown_connections 2024-07-04T17:28:47.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.610+0000 7f36dfe00640 1 --2- 172.21.15.42:0/2454040218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36e00701d0 0x7f36e00abe40 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:47.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.610+0000 7f36dfe00640 1 -- 172.21.15.42:0/2454040218 >> 172.21.15.42:0/2454040218 conn(0x7f36e00afaa0 msgr2=0x7f36e00afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:47.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.610+0000 7f36dfe00640 1 -- 172.21.15.42:0/2454040218 shutdown_connections 2024-07-04T17:28:47.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.610+0000 7f36dfe00640 1 -- 172.21.15.42:0/2454040218 wait complete. 2024-07-04T17:28:47.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.612+0000 7f36dfe00640 1 Processor -- start 2024-07-04T17:28:47.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.613+0000 7f36dfe00640 1 -- start start 2024-07-04T17:28:47.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.614+0000 7f36dfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36e00701d0 0x7f36e014c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:47.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.614+0000 7f36dfe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f36e00ab190 con 0x7f36e00701d0 2024-07-04T17:28:47.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.614+0000 7f36dea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36e00701d0 0x7f36e014c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:47.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.614+0000 7f36dea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36e00701d0 0x7f36e014c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54530/0 (socket says 172.21.15.42:54530) 2024-07-04T17:28:47.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.614+0000 7f36dea00640 1 -- 172.21.15.42:0/2111999030 learned_addr learned my addr 172.21.15.42:0/2111999030 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:47.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.615+0000 7f36dea00640 1 -- 172.21.15.42:0/2111999030 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e014ce60 con 0x7f36e00701d0 2024-07-04T17:28:47.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.615+0000 7f36dea00640 1 --2- 172.21.15.42:0/2111999030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36e00701d0 0x7f36e014c870 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f36d00098d0 tx=0x7f36d0033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:47.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.615+0000 7f36d7400640 1 -- 172.21.15.42:0/2111999030 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36d0031bc0 con 0x7f36e00701d0 2024-07-04T17:28:47.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.615+0000 7f36dfe00640 1 -- 172.21.15.42:0/2111999030 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36e0144a20 con 0x7f36e00701d0 2024-07-04T17:28:47.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.615+0000 7f36d7400640 1 -- 172.21.15.42:0/2111999030 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36d0031e50 con 0x7f36e00701d0 2024-07-04T17:28:47.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.615+0000 7f36d7400640 1 -- 172.21.15.42:0/2111999030 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36d0033200 con 0x7f36e00701d0 2024-07-04T17:28:47.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.615+0000 7f36dfe00640 1 -- 172.21.15.42:0/2111999030 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36e014f3c0 con 0x7f36e00701d0 2024-07-04T17:28:47.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.617+0000 7f36dfe00640 1 -- 172.21.15.42:0/2111999030 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36e00abe40 con 0x7f36e00701d0 2024-07-04T17:28:47.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.617+0000 7f36d7400640 1 -- 172.21.15.42:0/2111999030 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f36d00333a0 con 0x7f36e00701d0 2024-07-04T17:28:47.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.617+0000 7f36d7400640 1 --2- 172.21.15.42:0/2111999030 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f36bc03d4a0 0x7f36bc03f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:47.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.618+0000 7f36d7400640 1 -- 172.21.15.42:0/2111999030 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 3953+0+0 (secure 0 0 0) 0x7f36d0073110 con 0x7f36e00701d0 2024-07-04T17:28:47.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.618+0000 7f36d7e00640 1 --2- 172.21.15.42:0/2111999030 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f36bc03d4a0 0x7f36bc03f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:47.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.619+0000 7f36d7e00640 1 --2- 172.21.15.42:0/2111999030 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f36bc03d4a0 0x7f36bc03f9b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f36c80015c0 tx=0x7f36c8007a90 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:47.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.623+0000 7f36d7400640 1 -- 172.21.15.42:0/2111999030 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f36d003f030 con 0x7f36e00701d0 2024-07-04T17:28:47.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:47.832+0000 7f36dfe00640 1 -- 172.21.15.42:0/2111999030 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f36e0147810 con 0x7f36e00701d0 2024-07-04T17:28:48.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.279+0000 7f36d7400640 1 -- 172.21.15.42:0/2111999030 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_data' removed v40) ==== 178+0+0 (secure 0 0 0) 0x7f36d003a3a0 con 0x7f36e00701d0 2024-07-04T17:28:48.281 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'backup_fs_data' removed 2024-07-04T17:28:48.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.283+0000 7f36dfe00640 1 -- 172.21.15.42:0/2111999030 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f36bc03d4a0 msgr2=0x7f36bc03f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:48.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.283+0000 7f36dfe00640 1 --2- 172.21.15.42:0/2111999030 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f36bc03d4a0 0x7f36bc03f9b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f36c80015c0 tx=0x7f36c8007a90 comp rx=0 tx=0).stop 2024-07-04T17:28:48.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.283+0000 7f36dfe00640 1 -- 172.21.15.42:0/2111999030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36e00701d0 msgr2=0x7f36e014c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:48.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.283+0000 7f36dfe00640 1 --2- 172.21.15.42:0/2111999030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36e00701d0 0x7f36e014c870 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f36d00098d0 tx=0x7f36d0033e40 comp rx=0 tx=0).stop 2024-07-04T17:28:48.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.283+0000 7f36dfe00640 1 -- 172.21.15.42:0/2111999030 shutdown_connections 2024-07-04T17:28:48.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.283+0000 7f36dfe00640 1 --2- 172.21.15.42:0/2111999030 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f36bc03d4a0 0x7f36bc03f9b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:48.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.283+0000 7f36dfe00640 1 --2- 172.21.15.42:0/2111999030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36e00701d0 0x7f36e014c870 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:48.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.284+0000 7f36dfe00640 1 -- 172.21.15.42:0/2111999030 >> 172.21.15.42:0/2111999030 conn(0x7f36e00afaa0 msgr2=0x7f36e00b36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:48.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.284+0000 7f36dfe00640 1 -- 172.21.15.42:0/2111999030 shutdown_connections 2024-07-04T17:28:48.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.284+0000 7f36dfe00640 1 -- 172.21.15.42:0/2111999030 wait complete. 2024-07-04T17:28:48.332 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 22 2024-07-04T17:28:48.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.606+0000 7f52fb800640 1 -- 172.21.15.42:0/2749951292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52f40bc040 msgr2=0x7f52f40bd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:48.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.606+0000 7f52fb800640 1 --2- 172.21.15.42:0/2749951292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52f40bc040 0x7f52f40bd2d0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f52ec005fd0 tx=0x7f52ec02aa10 comp rx=0 tx=0).stop 2024-07-04T17:28:48.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.607+0000 7f52fb800640 1 -- 172.21.15.42:0/2749951292 shutdown_connections 2024-07-04T17:28:48.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.607+0000 7f52fb800640 1 --2- 172.21.15.42:0/2749951292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52f40bc040 0x7f52f40bd2d0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:48.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.607+0000 7f52fb800640 1 -- 172.21.15.42:0/2749951292 >> 172.21.15.42:0/2749951292 conn(0x7f52f406c940 msgr2=0x7f52f406ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:48.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.607+0000 7f52fb800640 1 -- 172.21.15.42:0/2749951292 shutdown_connections 2024-07-04T17:28:48.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.607+0000 7f52fb800640 1 -- 172.21.15.42:0/2749951292 wait complete. 2024-07-04T17:28:48.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.609+0000 7f52fb800640 1 Processor -- start 2024-07-04T17:28:48.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.609+0000 7f52fb800640 1 -- start start 2024-07-04T17:28:48.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.611+0000 7f52fb800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52f40bc040 0x7f52f40bf130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:48.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.611+0000 7f52fb800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f52f40b7530 con 0x7f52f40bc040 2024-07-04T17:28:48.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.611+0000 7f52f8e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52f40bc040 0x7f52f40bf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:48.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.611+0000 7f52f8e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52f40bc040 0x7f52f40bf130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54560/0 (socket says 172.21.15.42:54560) 2024-07-04T17:28:48.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.611+0000 7f52f8e00640 1 -- 172.21.15.42:0/3748415384 learned_addr learned my addr 172.21.15.42:0/3748415384 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:48.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.611+0000 7f52f8e00640 1 -- 172.21.15.42:0/3748415384 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52f41e38d0 con 0x7f52f40bc040 2024-07-04T17:28:48.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.612+0000 7f52f8e00640 1 --2- 172.21.15.42:0/3748415384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52f40bc040 0x7f52f40bf130 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f52ec009bf0 tx=0x7f52ec003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:48.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.612+0000 7f52f0c00640 1 -- 172.21.15.42:0/3748415384 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52ec046070 con 0x7f52f40bc040 2024-07-04T17:28:48.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.612+0000 7f52f0c00640 1 -- 172.21.15.42:0/3748415384 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52ec0044f0 con 0x7f52f40bc040 2024-07-04T17:28:48.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.612+0000 7f52f0c00640 1 -- 172.21.15.42:0/3748415384 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52ec033070 con 0x7f52f40bc040 2024-07-04T17:28:48.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.612+0000 7f52fb800640 1 -- 172.21.15.42:0/3748415384 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52f40bea20 con 0x7f52f40bc040 2024-07-04T17:28:48.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.612+0000 7f52fb800640 1 -- 172.21.15.42:0/3748415384 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52f40bf670 con 0x7f52f40bc040 2024-07-04T17:28:48.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.613+0000 7f52fb800640 1 -- 172.21.15.42:0/3748415384 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52f40b7c60 con 0x7f52f40bc040 2024-07-04T17:28:48.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.614+0000 7f52f0c00640 1 -- 172.21.15.42:0/3748415384 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f52ec049070 con 0x7f52f40bc040 2024-07-04T17:28:48.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.614+0000 7f52f0c00640 1 --2- 172.21.15.42:0/3748415384 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f52d003d4f0 0x7f52d003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:48.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.614+0000 7f52f0c00640 1 -- 172.21.15.42:0/3748415384 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 3533+0+0 (secure 0 0 0) 0x7f52ec072d00 con 0x7f52f40bc040 2024-07-04T17:28:48.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.615+0000 7f52f3e00640 1 --2- 172.21.15.42:0/3748415384 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f52d003d4f0 0x7f52d003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:48.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.615+0000 7f52f3e00640 1 --2- 172.21.15.42:0/3748415384 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f52d003d4f0 0x7f52d003fa00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009670 tx=0x7f52dc006bd0 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:48.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.619+0000 7f52f0c00640 1 -- 172.21.15.42:0/3748415384 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f52ec03e070 con 0x7f52f40bc040 2024-07-04T17:28:48.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.830+0000 7f52fb800640 1 -- 172.21.15.42:0/3748415384 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config reset", "num": 22} v 0) -- 0x7f52f40c09b0 con 0x7f52f40bc040 2024-07-04T17:28:48.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.835+0000 7f52f0c00640 1 -- 172.21.15.42:0/3748415384 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 22}]=0 v24) ==== 71+0+0 (secure 0 0 0) 0x7f52ec004810 con 0x7f52f40bc040 2024-07-04T17:28:48.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.838+0000 7f52fb800640 1 -- 172.21.15.42:0/3748415384 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f52d003d4f0 msgr2=0x7f52d003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:48.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.838+0000 7f52fb800640 1 --2- 172.21.15.42:0/3748415384 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f52d003d4f0 0x7f52d003fa00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009670 tx=0x7f52dc006bd0 comp rx=0 tx=0).stop 2024-07-04T17:28:48.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.838+0000 7f52fb800640 1 -- 172.21.15.42:0/3748415384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52f40bc040 msgr2=0x7f52f40bf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:48.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.838+0000 7f52fb800640 1 --2- 172.21.15.42:0/3748415384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52f40bc040 0x7f52f40bf130 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f52ec009bf0 tx=0x7f52ec003f30 comp rx=0 tx=0).stop 2024-07-04T17:28:48.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.838+0000 7f52fb800640 1 -- 172.21.15.42:0/3748415384 shutdown_connections 2024-07-04T17:28:48.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.838+0000 7f52fb800640 1 --2- 172.21.15.42:0/3748415384 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f52d003d4f0 0x7f52d003fa00 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:48.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.838+0000 7f52fb800640 1 --2- 172.21.15.42:0/3748415384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52f40bc040 0x7f52f40bf130 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:48.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.838+0000 7f52fb800640 1 -- 172.21.15.42:0/3748415384 >> 172.21.15.42:0/3748415384 conn(0x7f52f406c940 msgr2=0x7f52f4145bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:48.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.839+0000 7f52fb800640 1 -- 172.21.15.42:0/3748415384 shutdown_connections 2024-07-04T17:28:48.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:48.839+0000 7f52fb800640 1 -- 172.21.15.42:0/3748415384 wait complete. 2024-07-04T17:28:48.868 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_mirroring.TestMirroring.test_add_ancestor_and_child_directory' 2024-07-04T17:28:49.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:49.158+0000 7f1f6ec00640 1 -- 172.21.15.42:0/4290860954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f680b38f0 msgr2=0x7f1f680b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:49.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:49.158+0000 7f1f6ec00640 1 --2- 172.21.15.42:0/4290860954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f680b38f0 0x7f1f680b2d20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1f58005fd0 tx=0x7f1f5802a5f0 comp rx=0 tx=0).stop 2024-07-04T17:28:49.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:49.159+0000 7f1f6ec00640 1 -- 172.21.15.42:0/4290860954 shutdown_connections 2024-07-04T17:28:49.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:49.159+0000 7f1f6ec00640 1 --2- 172.21.15.42:0/4290860954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f680b38f0 0x7f1f680b2d20 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:49.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:49.159+0000 7f1f6ec00640 1 -- 172.21.15.42:0/4290860954 >> 172.21.15.42:0/4290860954 conn(0x7f1f680bf700 msgr2=0x7f1f680af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:49.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:49.159+0000 7f1f6ec00640 1 -- 172.21.15.42:0/4290860954 shutdown_connections 2024-07-04T17:28:49.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:49.159+0000 7f1f6ec00640 1 -- 172.21.15.42:0/4290860954 wait complete. 2024-07-04T17:28:49.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:49.161+0000 7f1f6ec00640 1 Processor -- start 2024-07-04T17:28:49.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:49.161+0000 7f1f6ec00640 1 -- start start 2024-07-04T17:28:49.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:49.162+0000 7f1f6ec00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f680b01f0 0x7f1f68143960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:49.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:49.162+0000 7f1f6ec00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6806cd40 con 0x7f1f680b01f0 2024-07-04T17:28:49.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:49.162+0000 7f1f67e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f680b01f0 0x7f1f68143960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:49.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:49.162+0000 7f1f67e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f680b01f0 0x7f1f68143960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54566/0 (socket says 172.21.15.42:54566) 2024-07-04T17:28:49.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:49.162+0000 7f1f67e00640 1 -- 172.21.15.42:0/804107396 learned_addr learned my addr 172.21.15.42:0/804107396 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:49.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:49.163+0000 7f1f67e00640 1 -- 172.21.15.42:0/804107396 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f6813dc20 con 0x7f1f680b01f0 2024-07-04T17:28:49.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:49.163+0000 7f1f67e00640 1 --2- 172.21.15.42:0/804107396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f680b01f0 0x7f1f68143960 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1f58034040 tx=0x7f1f580049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:49.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:49.163+0000 7f1f5fe00640 1 -- 172.21.15.42:0/804107396 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f5803e030 con 0x7f1f680b01f0 2024-07-04T17:28:49.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:49.163+0000 7f1f5fe00640 1 -- 172.21.15.42:0/804107396 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f580035f0 con 0x7f1f680b01f0 2024-07-04T17:28:49.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:49.164+0000 7f1f5fe00640 1 -- 172.21.15.42:0/804107396 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f58035040 con 0x7f1f680b01f0 2024-07-04T17:28:49.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:49.164+0000 7f1f6ec00640 1 -- 172.21.15.42:0/804107396 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f6813c000 con 0x7f1f680b01f0 2024-07-04T17:28:49.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:49.164+0000 7f1f6ec00640 1 -- 172.21.15.42:0/804107396 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f681e7650 con 0x7f1f680b01f0 2024-07-04T17:28:49.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:49.165+0000 7f1f6ec00640 1 -- 172.21.15.42:0/804107396 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f6806d400 con 0x7f1f680b01f0 2024-07-04T17:28:49.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:49.165+0000 7f1f5fe00640 1 -- 172.21.15.42:0/804107396 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f1f58041030 con 0x7f1f680b01f0 2024-07-04T17:28:49.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:49.165+0000 7f1f5fe00640 1 --2- 172.21.15.42:0/804107396 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f1f4403d4f0 0x7f1f4403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:49.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:49.165+0000 7f1f5fe00640 1 -- 172.21.15.42:0/804107396 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 3533+0+0 (secure 0 0 0) 0x7f1f5803f070 con 0x7f1f680b01f0 2024-07-04T17:28:49.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:49.166+0000 7f1f67400640 1 --2- 172.21.15.42:0/804107396 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f1f4403d4f0 0x7f1f4403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:49.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:49.167+0000 7f1f67400640 1 --2- 172.21.15.42:0/804107396 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f1f4403d4f0 0x7f1f4403fa00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1f68144ef0 tx=0x7f1f500079f0 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:49.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:49.171+0000 7f1f5fe00640 1 -- 172.21.15.42:0/804107396 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f1f58003df0 con 0x7f1f680b01f0 2024-07-04T17:28:49.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:49.472+0000 7f1f6ec00640 1 -- 172.21.15.42:0/804107396 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_mirroring.TestMirroring.test_add_ancestor_and_child_directory"]} v 0) -- 0x7f1f680becf0 con 0x7f1f680b01f0 2024-07-04T17:28:50.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.287+0000 7f1f5fe00640 1 -- 172.21.15.42:0/804107396 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_mirroring.TestMirroring.test_add_ancestor_and_child_directory"]}]=0 v147) ==== 158+0+0 (secure 0 0 0) 0x7f1f5804bd60 con 0x7f1f680b01f0 2024-07-04T17:28:50.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.290+0000 7f1f6ec00640 1 -- 172.21.15.42:0/804107396 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f1f4403d4f0 msgr2=0x7f1f4403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:50.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.290+0000 7f1f6ec00640 1 --2- 172.21.15.42:0/804107396 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f1f4403d4f0 0x7f1f4403fa00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1f68144ef0 tx=0x7f1f500079f0 comp rx=0 tx=0).stop 2024-07-04T17:28:50.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.290+0000 7f1f6ec00640 1 -- 172.21.15.42:0/804107396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f680b01f0 msgr2=0x7f1f68143960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:50.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.290+0000 7f1f6ec00640 1 --2- 172.21.15.42:0/804107396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f680b01f0 0x7f1f68143960 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1f58034040 tx=0x7f1f580049d0 comp rx=0 tx=0).stop 2024-07-04T17:28:50.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.291+0000 7f1f6ec00640 1 -- 172.21.15.42:0/804107396 shutdown_connections 2024-07-04T17:28:50.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.291+0000 7f1f6ec00640 1 --2- 172.21.15.42:0/804107396 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f1f4403d4f0 0x7f1f4403fa00 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:50.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.291+0000 7f1f6ec00640 1 --2- 172.21.15.42:0/804107396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f680b01f0 0x7f1f68143960 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:50.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.291+0000 7f1f6ec00640 1 -- 172.21.15.42:0/804107396 >> 172.21.15.42:0/804107396 conn(0x7f1f680bf700 msgr2=0x7f1f68075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:50.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.291+0000 7f1f6ec00640 1 -- 172.21.15.42:0/804107396 shutdown_connections 2024-07-04T17:28:50.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.291+0000 7f1f6ec00640 1 -- 172.21.15.42:0/804107396 wait complete. 2024-07-04T17:28:50.320 INFO:tasks.cephfs_test_runner:test_add_ancestor_and_child_directory (tasks.cephfs.test_mirroring.TestMirroring) ... ok 2024-07-04T17:28:50.320 INFO:tasks.cephfs_test_runner:Starting test: test_add_directory_path_normalization (tasks.cephfs.test_mirroring.TestMirroring) 2024-07-04T17:28:50.321 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:28:50.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.620+0000 7ff413c00640 1 -- 172.21.15.42:0/2857803451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff40c0b38b0 msgr2=0x7ff40c0b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:50.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.620+0000 7ff413c00640 1 --2- 172.21.15.42:0/2857803451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff40c0b38b0 0x7ff40c0b2d20 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0097b0 tx=0x7ff3fc02a540 comp rx=0 tx=0).stop 2024-07-04T17:28:50.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.620+0000 7ff413c00640 1 -- 172.21.15.42:0/2857803451 shutdown_connections 2024-07-04T17:28:50.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.620+0000 7ff413c00640 1 --2- 172.21.15.42:0/2857803451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff40c0b38b0 0x7ff40c0b2d20 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:50.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.620+0000 7ff413c00640 1 -- 172.21.15.42:0/2857803451 >> 172.21.15.42:0/2857803451 conn(0x7ff40c0bf700 msgr2=0x7ff40c0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:50.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.620+0000 7ff413c00640 1 -- 172.21.15.42:0/2857803451 shutdown_connections 2024-07-04T17:28:50.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.621+0000 7ff413c00640 1 -- 172.21.15.42:0/2857803451 wait complete. 2024-07-04T17:28:50.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.623+0000 7ff413c00640 1 Processor -- start 2024-07-04T17:28:50.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.623+0000 7ff413c00640 1 -- start start 2024-07-04T17:28:50.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.625+0000 7ff413c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff40c0b1e00 0x7ff40c0aac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:50.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.625+0000 7ff413c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff40c0b06e0 con 0x7ff40c0b1e00 2024-07-04T17:28:50.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.625+0000 7ff411200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff40c0b1e00 0x7ff40c0aac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:50.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.626+0000 7ff411200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff40c0b1e00 0x7ff40c0aac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54586/0 (socket says 172.21.15.42:54586) 2024-07-04T17:28:50.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.626+0000 7ff411200640 1 -- 172.21.15.42:0/3928393811 learned_addr learned my addr 172.21.15.42:0/3928393811 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:50.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.626+0000 7ff411200640 1 -- 172.21.15.42:0/3928393811 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff40c0ac800 con 0x7ff40c0b1e00 2024-07-04T17:28:50.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.626+0000 7ff411200640 1 --2- 172.21.15.42:0/3928393811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff40c0b1e00 0x7ff40c0aac90 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0098d0 tx=0x7ff3fc033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:50.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.627+0000 7ff409600640 1 -- 172.21.15.42:0/3928393811 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3fc031bc0 con 0x7ff40c0b1e00 2024-07-04T17:28:50.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.627+0000 7ff409600640 1 -- 172.21.15.42:0/3928393811 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3fc033070 con 0x7ff40c0b1e00 2024-07-04T17:28:50.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.627+0000 7ff413c00640 1 -- 172.21.15.42:0/3928393811 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff40c1d8280 con 0x7ff40c0b1e00 2024-07-04T17:28:50.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.627+0000 7ff409600640 1 -- 172.21.15.42:0/3928393811 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3fc0333f0 con 0x7ff40c0b1e00 2024-07-04T17:28:50.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.627+0000 7ff413c00640 1 -- 172.21.15.42:0/3928393811 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff40c0ac410 con 0x7ff40c0b1e00 2024-07-04T17:28:50.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.629+0000 7ff413c00640 1 -- 172.21.15.42:0/3928393811 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff40c06c590 con 0x7ff40c0b1e00 2024-07-04T17:28:50.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.629+0000 7ff409600640 1 -- 172.21.15.42:0/3928393811 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7ff3fc033590 con 0x7ff40c0b1e00 2024-07-04T17:28:50.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.630+0000 7ff409600640 1 --2- 172.21.15.42:0/3928393811 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7ff3e803d4f0 0x7ff3e803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:50.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.630+0000 7ff409600640 1 -- 172.21.15.42:0/3928393811 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 3533+0+0 (secure 0 0 0) 0x7ff3fc072ef0 con 0x7ff40c0b1e00 2024-07-04T17:28:50.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.630+0000 7ff403e00640 1 --2- 172.21.15.42:0/3928393811 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7ff3e803d4f0 0x7ff3e803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:50.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.631+0000 7ff403e00640 1 --2- 172.21.15.42:0/3928393811 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7ff3e803d4f0 0x7ff3e803fa00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff40c06ed80 tx=0x7ff3f4007a90 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:50.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.635+0000 7ff409600640 1 -- 172.21.15.42:0/3928393811 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7ff3fc03f030 con 0x7ff40c0b1e00 2024-07-04T17:28:50.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.848+0000 7ff413c00640 1 -- 172.21.15.42:0/3928393811 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff40c1d8450 con 0x7ff40c0b1e00 2024-07-04T17:28:50.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.849+0000 7ff409600640 1 -- 172.21.15.42:0/3928393811 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+8146 (secure 0 0 0) 0x7ff3fc031610 con 0x7ff40c0b1e00 2024-07-04T17:28:50.851 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:28:50.851 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":40,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:28:48.273468+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6833/1348446958":"2024-07-05T17:28:45.259094+0000","172.21.15.42:6832/1348446958":"2024-07-05T17:28:45.259094+0000","172.21.15.42:6834/807364821":"2024-07-05T17:28:38.571021+0000","172.21.15.42:0/429805019":"2024-07-05T17:28:37.604461+0000","172.21.15.42:0/3791890040":"2024-07-05T17:28:37.604461+0000","172.21.15.42:0/1994443123":"2024-07-05T17:28:37.604461+0000","172.21.15.42:0/2983064181":"2024-07-05T17:28:37.604461+0000","172.21.15.42:6824/3523793950":"2024-07-05T17:27:40.654396+0000","172.21.15.42:6824/488387844":"2024-07-05T17:28:37.604461+0000","172.21.15.42:6825/3523793950":"2024-07-05T17:27:40.654396+0000","172.21.15.42:0/487218478":"2024-07-04T18:28:21.818727+0000","172.21.15.42:0/2974639683":"2024-07-05T17:27:40.654396+0000","172.21.15.42:0/880857241":"2024-07-05T17:27:40.654396+0000","172.21.15.42:6835/807364821":"2024-07-05T17:28:38.571021+0000","172.21.15.42:0/2267375347":"2024-07-05T17:28:37.604461+0000","172.21.15.42:6825/488387844":"2024-07-05T17:28:37.604461+0000","172.21.15.42:0/83074206":"2024-07-05T17:27:40.654396+0000","172.21.15.42:0/3526015921":"2024-07-05T17:27:40.654396+0000","172.21.15.42:0/1537192714":"2024-07-05T17:28:37.604461+0000","172.21.15.42:0/435904326":"2024-07-05T17:28:37.604461+0000","172.21.15.42:0/3378322019":"2024-07-05T17:28:37.604461+0000","172.21.15.42:0/2362095151":"2024-07-05T17:28:37.604461+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:28:50.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.852+0000 7ff413c00640 1 -- 172.21.15.42:0/3928393811 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7ff3e803d4f0 msgr2=0x7ff3e803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:50.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.853+0000 7ff413c00640 1 --2- 172.21.15.42:0/3928393811 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7ff3e803d4f0 0x7ff3e803fa00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff40c06ed80 tx=0x7ff3f4007a90 comp rx=0 tx=0).stop 2024-07-04T17:28:50.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.853+0000 7ff413c00640 1 -- 172.21.15.42:0/3928393811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff40c0b1e00 msgr2=0x7ff40c0aac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:50.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.853+0000 7ff413c00640 1 --2- 172.21.15.42:0/3928393811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff40c0b1e00 0x7ff40c0aac90 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0098d0 tx=0x7ff3fc033e40 comp rx=0 tx=0).stop 2024-07-04T17:28:50.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.853+0000 7ff413c00640 1 -- 172.21.15.42:0/3928393811 shutdown_connections 2024-07-04T17:28:50.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.853+0000 7ff413c00640 1 --2- 172.21.15.42:0/3928393811 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7ff3e803d4f0 0x7ff3e803fa00 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:50.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.853+0000 7ff413c00640 1 --2- 172.21.15.42:0/3928393811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff40c0b1e00 0x7ff40c0aac90 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:50.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.853+0000 7ff413c00640 1 -- 172.21.15.42:0/3928393811 >> 172.21.15.42:0/3928393811 conn(0x7ff40c0bf700 msgr2=0x7ff40c0b0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:50.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.853+0000 7ff413c00640 1 -- 172.21.15.42:0/3928393811 shutdown_connections 2024-07-04T17:28:50.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:50.853+0000 7ff413c00640 1 -- 172.21.15.42:0/3928393811 wait complete. 2024-07-04T17:28:50.879 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-04T17:26:03.297630+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-04T17:28:50.879 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-04T17:28:51.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.168+0000 7fb9bfe00640 1 -- 172.21.15.42:0/1813404387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9c00b1ac0 msgr2=0x7fb9c00b2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:51.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.168+0000 7fb9bfe00640 1 --2- 172.21.15.42:0/1813404387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9c00b1ac0 0x7fb9c00b2c80 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb9c00b34b0 tx=0x7fb9b002a480 comp rx=0 tx=0).stop 2024-07-04T17:28:51.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.168+0000 7fb9bfe00640 1 -- 172.21.15.42:0/1813404387 shutdown_connections 2024-07-04T17:28:51.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.168+0000 7fb9bfe00640 1 --2- 172.21.15.42:0/1813404387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9c00b1ac0 0x7fb9c00b2c80 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:51.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.168+0000 7fb9bfe00640 1 -- 172.21.15.42:0/1813404387 >> 172.21.15.42:0/1813404387 conn(0x7fb9c00bf260 msgr2=0x7fb9c00af5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:51.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.168+0000 7fb9bfe00640 1 -- 172.21.15.42:0/1813404387 shutdown_connections 2024-07-04T17:28:51.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.168+0000 7fb9bfe00640 1 -- 172.21.15.42:0/1813404387 wait complete. 2024-07-04T17:28:51.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.170+0000 7fb9bfe00640 1 Processor -- start 2024-07-04T17:28:51.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.170+0000 7fb9bfe00640 1 -- start start 2024-07-04T17:28:51.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.172+0000 7fb9bfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9c00afeb0 0x7fb9c01d2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:51.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.172+0000 7fb9bfe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c014c7a0 con 0x7fb9c00afeb0 2024-07-04T17:28:51.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.172+0000 7fb9bea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9c00afeb0 0x7fb9c01d2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:51.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.172+0000 7fb9bea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9c00afeb0 0x7fb9c01d2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54598/0 (socket says 172.21.15.42:54598) 2024-07-04T17:28:51.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.172+0000 7fb9bea00640 1 -- 172.21.15.42:0/3286825118 learned_addr learned my addr 172.21.15.42:0/3286825118 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:51.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.172+0000 7fb9bea00640 1 -- 172.21.15.42:0/3286825118 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c01d3420 con 0x7fb9c00afeb0 2024-07-04T17:28:51.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.173+0000 7fb9bea00640 1 --2- 172.21.15.42:0/3286825118 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9c00afeb0 0x7fb9c01d2e30 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb9b002aa70 tx=0x7fb9b0032c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:51.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.173+0000 7fb9b7e00640 1 -- 172.21.15.42:0/3286825118 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9b003f030 con 0x7fb9c00afeb0 2024-07-04T17:28:51.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.173+0000 7fb9b7e00640 1 -- 172.21.15.42:0/3286825118 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9b0003a20 con 0x7fb9c00afeb0 2024-07-04T17:28:51.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.173+0000 7fb9b7e00640 1 -- 172.21.15.42:0/3286825118 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9b0003d40 con 0x7fb9c00afeb0 2024-07-04T17:28:51.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.173+0000 7fb9bfe00640 1 -- 172.21.15.42:0/3286825118 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9c01d4e80 con 0x7fb9c00afeb0 2024-07-04T17:28:51.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.173+0000 7fb9bfe00640 1 -- 172.21.15.42:0/3286825118 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9c01dcb20 con 0x7fb9c00afeb0 2024-07-04T17:28:51.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.175+0000 7fb9bfe00640 1 -- 172.21.15.42:0/3286825118 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9c00bd940 con 0x7fb9c00afeb0 2024-07-04T17:28:51.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.175+0000 7fb9b7e00640 1 -- 172.21.15.42:0/3286825118 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7fb9b0042030 con 0x7fb9c00afeb0 2024-07-04T17:28:51.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.175+0000 7fb9b7e00640 1 --2- 172.21.15.42:0/3286825118 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fb99c03d4f0 0x7fb99c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:51.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.176+0000 7fb9b7e00640 1 -- 172.21.15.42:0/3286825118 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 3533+0+0 (secure 0 0 0) 0x7fb9b0040070 con 0x7fb9c00afeb0 2024-07-04T17:28:51.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.176+0000 7fb9b6000640 1 --2- 172.21.15.42:0/3286825118 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fb99c03d4f0 0x7fb99c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:51.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.177+0000 7fb9b6000640 1 --2- 172.21.15.42:0/3286825118 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fb99c03d4f0 0x7fb99c03fa00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8007fd0 tx=0x7fb9a8007920 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:51.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.181+0000 7fb9b7e00640 1 -- 172.21.15.42:0/3286825118 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fb9b0049e30 con 0x7fb9c00afeb0 2024-07-04T17:28:51.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.389+0000 7fb9bfe00640 1 -- 172.21.15.42:0/3286825118 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb9c0068e30 con 0x7fb9c00afeb0 2024-07-04T17:28:51.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.390+0000 7fb9b7e00640 1 -- 172.21.15.42:0/3286825118 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) ==== 93+0+10 (secure 0 0 0) 0x7fb9b003bd90 con 0x7fb9c00afeb0 2024-07-04T17:28:51.392 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-07-04T17:28:51.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.393+0000 7fb9bfe00640 1 -- 172.21.15.42:0/3286825118 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fb99c03d4f0 msgr2=0x7fb99c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:51.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.393+0000 7fb9bfe00640 1 --2- 172.21.15.42:0/3286825118 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fb99c03d4f0 0x7fb99c03fa00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8007fd0 tx=0x7fb9a8007920 comp rx=0 tx=0).stop 2024-07-04T17:28:51.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.393+0000 7fb9bfe00640 1 -- 172.21.15.42:0/3286825118 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9c00afeb0 msgr2=0x7fb9c01d2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:51.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.393+0000 7fb9bfe00640 1 --2- 172.21.15.42:0/3286825118 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9c00afeb0 0x7fb9c01d2e30 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb9b002aa70 tx=0x7fb9b0032c00 comp rx=0 tx=0).stop 2024-07-04T17:28:51.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.393+0000 7fb9bfe00640 1 -- 172.21.15.42:0/3286825118 shutdown_connections 2024-07-04T17:28:51.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.393+0000 7fb9bfe00640 1 --2- 172.21.15.42:0/3286825118 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fb99c03d4f0 0x7fb99c03fa00 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:51.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.393+0000 7fb9bfe00640 1 --2- 172.21.15.42:0/3286825118 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9c00afeb0 0x7fb9c01d2e30 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:51.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.393+0000 7fb9bfe00640 1 -- 172.21.15.42:0/3286825118 >> 172.21.15.42:0/3286825118 conn(0x7fb9c00bf260 msgr2=0x7fb9c00bde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:51.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.394+0000 7fb9bfe00640 1 -- 172.21.15.42:0/3286825118 shutdown_connections 2024-07-04T17:28:51.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.394+0000 7fb9bfe00640 1 -- 172.21.15.42:0/3286825118 wait complete. 2024-07-04T17:28:51.422 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_mirroring.TestMirroring.test_add_directory_path_normalization' 2024-07-04T17:28:51.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.714+0000 7f2812600640 1 -- 172.21.15.42:0/2832576652 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f280c0bad70 msgr2=0x7f280c06c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:51.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.714+0000 7f2812600640 1 --2- 172.21.15.42:0/2832576652 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f280c0bad70 0x7f280c06c2c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00a8c0 tx=0x7f27fc02b7d0 comp rx=0 tx=0).stop 2024-07-04T17:28:51.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.714+0000 7f2812600640 1 -- 172.21.15.42:0/2832576652 shutdown_connections 2024-07-04T17:28:51.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.714+0000 7f2812600640 1 --2- 172.21.15.42:0/2832576652 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f280c0bad70 0x7f280c06c2c0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:51.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.714+0000 7f2812600640 1 -- 172.21.15.42:0/2832576652 >> 172.21.15.42:0/2832576652 conn(0x7f280c0bebc0 msgr2=0x7f280c0befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:51.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.714+0000 7f2812600640 1 -- 172.21.15.42:0/2832576652 shutdown_connections 2024-07-04T17:28:51.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.714+0000 7f2812600640 1 -- 172.21.15.42:0/2832576652 wait complete. 2024-07-04T17:28:51.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.716+0000 7f2812600640 1 Processor -- start 2024-07-04T17:28:51.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.716+0000 7f2812600640 1 -- start start 2024-07-04T17:28:51.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.718+0000 7f2812600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f280c06d580 0x7f280c0bdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:51.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.718+0000 7f2812600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f280c0b2810 con 0x7f280c06d580 2024-07-04T17:28:51.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.718+0000 7f2811200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f280c06d580 0x7f280c0bdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:51.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.718+0000 7f2811200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f280c06d580 0x7f280c0bdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54622/0 (socket says 172.21.15.42:54622) 2024-07-04T17:28:51.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.718+0000 7f2811200640 1 -- 172.21.15.42:0/3145913152 learned_addr learned my addr 172.21.15.42:0/3145913152 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:51.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.719+0000 7f2811200640 1 -- 172.21.15.42:0/3145913152 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280c0708b0 con 0x7f280c06d580 2024-07-04T17:28:51.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.719+0000 7f2811200640 1 --2- 172.21.15.42:0/3145913152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f280c06d580 0x7f280c0bdc00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f280000c380 tx=0x7f280000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:51.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.719+0000 7f2808c00640 1 -- 172.21.15.42:0/3145913152 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28000116c0 con 0x7f280c06d580 2024-07-04T17:28:51.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.719+0000 7f2808c00640 1 -- 172.21.15.42:0/3145913152 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2800017070 con 0x7f280c06d580 2024-07-04T17:28:51.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.719+0000 7f2812600640 1 -- 172.21.15.42:0/3145913152 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f280c071530 con 0x7f280c06d580 2024-07-04T17:28:51.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.719+0000 7f2808c00640 1 -- 172.21.15.42:0/3145913152 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2800011b20 con 0x7f280c06d580 2024-07-04T17:28:51.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.720+0000 7f2812600640 1 -- 172.21.15.42:0/3145913152 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f280c142e60 con 0x7f280c06d580 2024-07-04T17:28:51.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.721+0000 7f2812600640 1 -- 172.21.15.42:0/3145913152 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f280c0be140 con 0x7f280c06d580 2024-07-04T17:28:51.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.721+0000 7f2808c00640 1 -- 172.21.15.42:0/3145913152 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f2800011cc0 con 0x7f280c06d580 2024-07-04T17:28:51.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.721+0000 7f2808c00640 1 --2- 172.21.15.42:0/3145913152 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f27f003d4f0 0x7f27f003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:51.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.721+0000 7f2808c00640 1 -- 172.21.15.42:0/3145913152 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 3533+0+0 (secure 0 0 0) 0x7f2800051690 con 0x7f280c06d580 2024-07-04T17:28:51.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.722+0000 7f280be00640 1 --2- 172.21.15.42:0/3145913152 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f27f003d4f0 0x7f27f003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:51.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.722+0000 7f280be00640 1 --2- 172.21.15.42:0/3145913152 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f27f003d4f0 0x7f27f003fa00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f27fc036000 tx=0x7f27fc004730 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:51.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:51.726+0000 7f2808c00640 1 -- 172.21.15.42:0/3145913152 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f2800010cf0 con 0x7f280c06d580 2024-07-04T17:28:52.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.011+0000 7f2812600640 1 -- 172.21.15.42:0/3145913152 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_mirroring.TestMirroring.test_add_directory_path_normalization"]} v 0) -- 0x7f280c06c2c0 con 0x7f280c06d580 2024-07-04T17:28:52.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.295+0000 7f2808c00640 1 -- 172.21.15.42:0/3145913152 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_mirroring.TestMirroring.test_add_directory_path_normalization"]}]=0 v149) ==== 161+0+0 (secure 0 0 0) 0x7f2800051340 con 0x7f280c06d580 2024-07-04T17:28:52.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.298+0000 7f2812600640 1 -- 172.21.15.42:0/3145913152 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f27f003d4f0 msgr2=0x7f27f003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:52.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.298+0000 7f2812600640 1 --2- 172.21.15.42:0/3145913152 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f27f003d4f0 0x7f27f003fa00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f27fc036000 tx=0x7f27fc004730 comp rx=0 tx=0).stop 2024-07-04T17:28:52.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.298+0000 7f2812600640 1 -- 172.21.15.42:0/3145913152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f280c06d580 msgr2=0x7f280c0bdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:52.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.298+0000 7f2812600640 1 --2- 172.21.15.42:0/3145913152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f280c06d580 0x7f280c0bdc00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f280000c380 tx=0x7f280000c840 comp rx=0 tx=0).stop 2024-07-04T17:28:52.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.298+0000 7f2812600640 1 -- 172.21.15.42:0/3145913152 shutdown_connections 2024-07-04T17:28:52.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.298+0000 7f2812600640 1 --2- 172.21.15.42:0/3145913152 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f27f003d4f0 0x7f27f003fa00 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:52.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.298+0000 7f2812600640 1 --2- 172.21.15.42:0/3145913152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f280c06d580 0x7f280c0bdc00 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:52.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.298+0000 7f2812600640 1 -- 172.21.15.42:0/3145913152 >> 172.21.15.42:0/3145913152 conn(0x7f280c0bebc0 msgr2=0x7f280c0bbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:52.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.298+0000 7f2812600640 1 -- 172.21.15.42:0/3145913152 shutdown_connections 2024-07-04T17:28:52.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.298+0000 7f2812600640 1 -- 172.21.15.42:0/3145913152 wait complete. 2024-07-04T17:28:52.327 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-07-04T17:28:52.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.616+0000 7fa8ee000640 1 -- 172.21.15.42:0/4114278942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8e80b3030 msgr2=0x7fa8e80b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:52.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.616+0000 7fa8ee000640 1 --2- 172.21.15.42:0/4114278942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8e80b3030 0x7fa8e80b01f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4005fd0 tx=0x7fa8d402a5f0 comp rx=0 tx=0).stop 2024-07-04T17:28:52.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.616+0000 7fa8ee000640 1 -- 172.21.15.42:0/4114278942 shutdown_connections 2024-07-04T17:28:52.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.616+0000 7fa8ee000640 1 --2- 172.21.15.42:0/4114278942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8e80b3030 0x7fa8e80b01f0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:52.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.616+0000 7fa8ee000640 1 -- 172.21.15.42:0/4114278942 >> 172.21.15.42:0/4114278942 conn(0x7fa8e80bf700 msgr2=0x7fa8e80af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:52.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.617+0000 7fa8ee000640 1 -- 172.21.15.42:0/4114278942 shutdown_connections 2024-07-04T17:28:52.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.617+0000 7fa8ee000640 1 -- 172.21.15.42:0/4114278942 wait complete. 2024-07-04T17:28:52.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.619+0000 7fa8ee000640 1 Processor -- start 2024-07-04T17:28:52.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.619+0000 7fa8ee000640 1 -- start start 2024-07-04T17:28:52.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.620+0000 7fa8ee000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8e806cea0 0x7fa8e8070540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:52.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.620+0000 7fa8ee000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e806ef50 con 0x7fa8e806cea0 2024-07-04T17:28:52.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.620+0000 7fa8e7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8e806cea0 0x7fa8e8070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:52.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.620+0000 7fa8e7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8e806cea0 0x7fa8e8070540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54640/0 (socket says 172.21.15.42:54640) 2024-07-04T17:28:52.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.620+0000 7fa8e7400640 1 -- 172.21.15.42:0/747596190 learned_addr learned my addr 172.21.15.42:0/747596190 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:52.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.621+0000 7fa8e7400640 1 -- 172.21.15.42:0/747596190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e8071670 con 0x7fa8e806cea0 2024-07-04T17:28:52.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.621+0000 7fa8e7400640 1 --2- 172.21.15.42:0/747596190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8e806cea0 0x7fa8e8070540 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa8d800c320 tx=0x7fa8d800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:52.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.621+0000 7fa8df400640 1 -- 172.21.15.42:0/747596190 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8d8017070 con 0x7fa8e806cea0 2024-07-04T17:28:52.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.621+0000 7fa8df400640 1 -- 172.21.15.42:0/747596190 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8d8005310 con 0x7fa8e806cea0 2024-07-04T17:28:52.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.621+0000 7fa8df400640 1 -- 172.21.15.42:0/747596190 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8d80035f0 con 0x7fa8e806cea0 2024-07-04T17:28:52.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.621+0000 7fa8ee000640 1 -- 172.21.15.42:0/747596190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8e80713a0 con 0x7fa8e806cea0 2024-07-04T17:28:52.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.622+0000 7fa8ee000640 1 -- 172.21.15.42:0/747596190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8e813e300 con 0x7fa8e806cea0 2024-07-04T17:28:52.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.623+0000 7fa8ee000640 1 -- 172.21.15.42:0/747596190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8e80b01f0 con 0x7fa8e806cea0 2024-07-04T17:28:52.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.623+0000 7fa8df400640 1 -- 172.21.15.42:0/747596190 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7fa8d801a070 con 0x7fa8e806cea0 2024-07-04T17:28:52.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.624+0000 7fa8df400640 1 --2- 172.21.15.42:0/747596190 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fa8c803d4f0 0x7fa8c803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:52.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.624+0000 7fa8df400640 1 -- 172.21.15.42:0/747596190 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 3533+0+0 (secure 0 0 0) 0x7fa8d8050c30 con 0x7fa8e806cea0 2024-07-04T17:28:52.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.624+0000 7fa8e6a00640 1 --2- 172.21.15.42:0/747596190 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fa8c803d4f0 0x7fa8c803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:52.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.625+0000 7fa8e6a00640 1 --2- 172.21.15.42:0/747596190 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fa8c803d4f0 0x7fa8c803fa00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4033000 tx=0x7fa8d4004730 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:52.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.629+0000 7fa8df400640 1 -- 172.21.15.42:0/747596190 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fa8d801ace0 con 0x7fa8e806cea0 2024-07-04T17:28:52.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.831+0000 7fa8ee000640 1 -- 172.21.15.42:0/747596190 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fa8e806d430 con 0x7fa8e806cea0 2024-07-04T17:28:52.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.832+0000 7fa8df400640 1 -- 172.21.15.42:0/747596190 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v24) ==== 59+0+1628 (secure 0 0 0) 0x7fa8d801e0d0 con 0x7fa8e806cea0 2024-07-04T17:28:52.834 INFO:teuthology.orchestra.run.smithi042.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-07-04T17:28:52.834 INFO:teuthology.orchestra.run.smithi042.stdout:mon advanced mon_op_complaint_time 120 2024-07-04T17:28:52.834 INFO:teuthology.orchestra.run.smithi042.stdout:mon dev mon_warn_on_pool_no_app false 2024-07-04T17:28:52.834 INFO:teuthology.orchestra.run.smithi042.stdout:osd advanced osd_op_complaint_time 180.000000 2024-07-04T17:28:52.834 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3541.919687 2024-07-04T17:28:52.834 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3558.529833 2024-07-04T17:28:52.834 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3688.441103 2024-07-04T17:28:52.834 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced debug_mds 20/20 2024-07-04T17:28:52.834 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced debug_mds_balancer 20/20 2024-07-04T17:28:52.834 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced debug_ms 1/1 2024-07-04T17:28:52.834 INFO:teuthology.orchestra.run.smithi042.stdout:mds dev mds_debug_frag true 2024-07-04T17:28:52.834 INFO:teuthology.orchestra.run.smithi042.stdout:mds dev mds_debug_scatterstat true 2024-07-04T17:28:52.835 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced mds_op_complaint_time 180.000000 2024-07-04T17:28:52.835 INFO:teuthology.orchestra.run.smithi042.stdout:mds dev mds_verify_scatter true 2024-07-04T17:28:52.835 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced osd_op_complaint_time 180.000000 2024-07-04T17:28:52.835 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced rados_mon_op_timeout 900 2024-07-04T17:28:52.835 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced rados_osd_op_timeout 900 2024-07-04T17:28:52.835 INFO:teuthology.orchestra.run.smithi042.stdout:client advanced client_mount_timeout 600 2024-07-04T17:28:52.835 INFO:teuthology.orchestra.run.smithi042.stdout:client advanced debug_client 20/20 2024-07-04T17:28:52.835 INFO:teuthology.orchestra.run.smithi042.stdout:client advanced debug_ms 1/1 2024-07-04T17:28:52.835 INFO:teuthology.orchestra.run.smithi042.stdout:client advanced rados_mon_op_timeout 900 2024-07-04T17:28:52.835 INFO:teuthology.orchestra.run.smithi042.stdout:client advanced rados_osd_op_timeout 900 2024-07-04T17:28:52.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.834+0000 7fa8ee000640 1 -- 172.21.15.42:0/747596190 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fa8c803d4f0 msgr2=0x7fa8c803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:52.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.834+0000 7fa8ee000640 1 --2- 172.21.15.42:0/747596190 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fa8c803d4f0 0x7fa8c803fa00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4033000 tx=0x7fa8d4004730 comp rx=0 tx=0).stop 2024-07-04T17:28:52.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.834+0000 7fa8ee000640 1 -- 172.21.15.42:0/747596190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8e806cea0 msgr2=0x7fa8e8070540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:52.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.834+0000 7fa8ee000640 1 --2- 172.21.15.42:0/747596190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8e806cea0 0x7fa8e8070540 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa8d800c320 tx=0x7fa8d800c7e0 comp rx=0 tx=0).stop 2024-07-04T17:28:52.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.835+0000 7fa8ee000640 1 -- 172.21.15.42:0/747596190 shutdown_connections 2024-07-04T17:28:52.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.835+0000 7fa8ee000640 1 --2- 172.21.15.42:0/747596190 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fa8c803d4f0 0x7fa8c803fa00 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:52.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.835+0000 7fa8ee000640 1 --2- 172.21.15.42:0/747596190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8e806cea0 0x7fa8e8070540 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:52.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.835+0000 7fa8ee000640 1 -- 172.21.15.42:0/747596190 >> 172.21.15.42:0/747596190 conn(0x7fa8e80bf700 msgr2=0x7fa8e8152e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:52.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.835+0000 7fa8ee000640 1 -- 172.21.15.42:0/747596190 shutdown_connections 2024-07-04T17:28:52.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:52.835+0000 7fa8ee000640 1 -- 172.21.15.42:0/747596190 wait complete. 2024-07-04T17:28:52.861 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-07-04T17:28:53.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.142+0000 7f44cd200640 1 -- 172.21.15.42:0/3960190041 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44c80ba640 msgr2=0x7f44c80aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:53.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.142+0000 7f44cd200640 1 --2- 172.21.15.42:0/3960190041 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44c80ba640 0x7f44c80aff70 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f44b8005fd0 tx=0x7f44b802aac0 comp rx=0 tx=0).stop 2024-07-04T17:28:53.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.142+0000 7f44cd200640 1 -- 172.21.15.42:0/3960190041 shutdown_connections 2024-07-04T17:28:53.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.142+0000 7f44cd200640 1 --2- 172.21.15.42:0/3960190041 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44c80ba640 0x7f44c80aff70 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:53.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.142+0000 7f44cd200640 1 -- 172.21.15.42:0/3960190041 >> 172.21.15.42:0/3960190041 conn(0x7f44c80bf260 msgr2=0x7f44c80af5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:53.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.142+0000 7f44cd200640 1 -- 172.21.15.42:0/3960190041 shutdown_connections 2024-07-04T17:28:53.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.142+0000 7f44cd200640 1 -- 172.21.15.42:0/3960190041 wait complete. 2024-07-04T17:28:53.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.145+0000 7f44cd200640 1 Processor -- start 2024-07-04T17:28:53.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.145+0000 7f44cd200640 1 -- start start 2024-07-04T17:28:53.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.147+0000 7f44cd200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44c80b76f0 0x7f44c8144040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:53.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.147+0000 7f44cd200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f44c806c5a0 con 0x7f44c80b76f0 2024-07-04T17:28:53.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.147+0000 7f44c7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44c80b76f0 0x7f44c8144040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:53.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.147+0000 7f44c7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44c80b76f0 0x7f44c8144040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54662/0 (socket says 172.21.15.42:54662) 2024-07-04T17:28:53.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.147+0000 7f44c7400640 1 -- 172.21.15.42:0/3900051462 learned_addr learned my addr 172.21.15.42:0/3900051462 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:53.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.148+0000 7f44c7400640 1 -- 172.21.15.42:0/3900051462 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c8149370 con 0x7f44c80b76f0 2024-07-04T17:28:53.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.148+0000 7f44c7400640 1 --2- 172.21.15.42:0/3900051462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44c80b76f0 0x7f44c8144040 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f44b8006010 tx=0x7f44b8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:53.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.148+0000 7f44bf400640 1 -- 172.21.15.42:0/3900051462 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44b8044070 con 0x7f44c80b76f0 2024-07-04T17:28:53.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.148+0000 7f44bf400640 1 -- 172.21.15.42:0/3900051462 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44b8004380 con 0x7f44c80b76f0 2024-07-04T17:28:53.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.148+0000 7f44cd200640 1 -- 172.21.15.42:0/3900051462 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44c81447e0 con 0x7f44c80b76f0 2024-07-04T17:28:53.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.148+0000 7f44bf400640 1 -- 172.21.15.42:0/3900051462 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44b80046c0 con 0x7f44c80b76f0 2024-07-04T17:28:53.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.148+0000 7f44cd200640 1 -- 172.21.15.42:0/3900051462 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44c8148df0 con 0x7f44c80b76f0 2024-07-04T17:28:53.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.150+0000 7f44bf400640 1 -- 172.21.15.42:0/3900051462 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f44b8047070 con 0x7f44c80b76f0 2024-07-04T17:28:53.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.150+0000 7f44cd200640 1 -- 172.21.15.42:0/3900051462 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44c806cc50 con 0x7f44c80b76f0 2024-07-04T17:28:53.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.151+0000 7f44bf400640 1 --2- 172.21.15.42:0/3900051462 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f44a803d4f0 0x7f44a803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:53.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.151+0000 7f44bf400640 1 -- 172.21.15.42:0/3900051462 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 3533+0+0 (secure 0 0 0) 0x7f44b803c070 con 0x7f44c80b76f0 2024-07-04T17:28:53.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.151+0000 7f44c6a00640 1 --2- 172.21.15.42:0/3900051462 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f44a803d4f0 0x7f44a803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:53.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.152+0000 7f44c6a00640 1 --2- 172.21.15.42:0/3900051462 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f44a803d4f0 0x7f44a803fa00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f44c814f850 tx=0x7f44b40079f0 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:53.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.157+0000 7f44bf400640 1 -- 172.21.15.42:0/3900051462 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f44b8032030 con 0x7f44c80b76f0 2024-07-04T17:28:53.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.367+0000 7f44cd200640 1 -- 172.21.15.42:0/3900051462 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f44c80008d0 con 0x7f44c80b76f0 2024-07-04T17:28:53.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.373+0000 7f44bf400640 1 -- 172.21.15.42:0/3900051462 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v25)=0 v25) ==== 122+0+0 (secure 0 0 0) 0x7f44b8047360 con 0x7f44c80b76f0 2024-07-04T17:28:53.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.376+0000 7f44cd200640 1 -- 172.21.15.42:0/3900051462 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f44a803d4f0 msgr2=0x7f44a803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:53.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.376+0000 7f44cd200640 1 --2- 172.21.15.42:0/3900051462 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f44a803d4f0 0x7f44a803fa00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f44c814f850 tx=0x7f44b40079f0 comp rx=0 tx=0).stop 2024-07-04T17:28:53.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.376+0000 7f44cd200640 1 -- 172.21.15.42:0/3900051462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44c80b76f0 msgr2=0x7f44c8144040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:53.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.376+0000 7f44cd200640 1 --2- 172.21.15.42:0/3900051462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44c80b76f0 0x7f44c8144040 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f44b8006010 tx=0x7f44b8003f10 comp rx=0 tx=0).stop 2024-07-04T17:28:53.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.376+0000 7f44cd200640 1 -- 172.21.15.42:0/3900051462 shutdown_connections 2024-07-04T17:28:53.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.376+0000 7f44cd200640 1 --2- 172.21.15.42:0/3900051462 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f44a803d4f0 0x7f44a803fa00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:53.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.376+0000 7f44cd200640 1 --2- 172.21.15.42:0/3900051462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44c80b76f0 0x7f44c8144040 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:53.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.376+0000 7f44cd200640 1 -- 172.21.15.42:0/3900051462 >> 172.21.15.42:0/3900051462 conn(0x7f44c80bf260 msgr2=0x7f44c8075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:53.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.377+0000 7f44cd200640 1 -- 172.21.15.42:0/3900051462 shutdown_connections 2024-07-04T17:28:53.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.377+0000 7f44cd200640 1 -- 172.21.15.42:0/3900051462 wait complete. 2024-07-04T17:28:53.403 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-04T17:28:53.404 DEBUG:teuthology.orchestra.run.smithi042:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-04T17:28:53.515 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-04T17:28:53.515 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:28:53.515 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:28:53.570 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-07-04T17:28:53.570 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:28:53.571 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:28:53.626 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-07-04T17:28:53.626 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-07-04T17:28:53.626 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:28:53.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.922+0000 7f8bfd000640 1 -- 172.21.15.42:0/1062114516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bf806edd0 msgr2=0x7f8bf80b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:53.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.922+0000 7f8bfd000640 1 --2- 172.21.15.42:0/1062114516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bf806edd0 0x7f8bf80b3030 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8be8005fd0 tx=0x7f8be802b930 comp rx=0 tx=0).stop 2024-07-04T17:28:53.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.923+0000 7f8bfd000640 1 -- 172.21.15.42:0/1062114516 shutdown_connections 2024-07-04T17:28:53.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.923+0000 7f8bfd000640 1 --2- 172.21.15.42:0/1062114516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bf806edd0 0x7f8bf80b3030 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:53.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.923+0000 7f8bfd000640 1 -- 172.21.15.42:0/1062114516 >> 172.21.15.42:0/1062114516 conn(0x7f8bf80bf700 msgr2=0x7f8bf80af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:53.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.923+0000 7f8bfd000640 1 -- 172.21.15.42:0/1062114516 shutdown_connections 2024-07-04T17:28:53.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.923+0000 7f8bfd000640 1 -- 172.21.15.42:0/1062114516 wait complete. 2024-07-04T17:28:53.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.925+0000 7f8bfd000640 1 Processor -- start 2024-07-04T17:28:53.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.925+0000 7f8bfd000640 1 -- start start 2024-07-04T17:28:53.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.926+0000 7f8bfd000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bf806cea0 0x7f8bf80bdec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:53.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.926+0000 7f8bfd000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf80b0370 con 0x7f8bf806cea0 2024-07-04T17:28:53.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.927+0000 7f8bf5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bf806cea0 0x7f8bf80bdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:53.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.927+0000 7f8bf5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bf806cea0 0x7f8bf80bdec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37280/0 (socket says 172.21.15.42:37280) 2024-07-04T17:28:53.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.927+0000 7f8bf5e00640 1 -- 172.21.15.42:0/579292046 learned_addr learned my addr 172.21.15.42:0/579292046 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:28:53.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.927+0000 7f8bf5e00640 1 -- 172.21.15.42:0/579292046 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf80716c0 con 0x7f8bf806cea0 2024-07-04T17:28:53.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.927+0000 7f8bf5e00640 1 --2- 172.21.15.42:0/579292046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bf806cea0 0x7f8bf80bdec0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8be8034040 tx=0x7f8be80043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:53.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.928+0000 7f8beea00640 1 -- 172.21.15.42:0/579292046 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8be8004ae0 con 0x7f8bf806cea0 2024-07-04T17:28:53.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.928+0000 7f8beea00640 1 -- 172.21.15.42:0/579292046 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8be8004d90 con 0x7f8bf806cea0 2024-07-04T17:28:53.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.928+0000 7f8beea00640 1 -- 172.21.15.42:0/579292046 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8be8031630 con 0x7f8bf806cea0 2024-07-04T17:28:53.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.928+0000 7f8bfd000640 1 -- 172.21.15.42:0/579292046 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bf80ae140 con 0x7f8bf806cea0 2024-07-04T17:28:53.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.928+0000 7f8bfd000640 1 -- 172.21.15.42:0/579292046 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bf81dcbf0 con 0x7f8bf806cea0 2024-07-04T17:28:53.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.929+0000 7f8bfd000640 1 -- 172.21.15.42:0/579292046 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bf80b39f0 con 0x7f8bf806cea0 2024-07-04T17:28:53.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.929+0000 7f8beea00640 1 -- 172.21.15.42:0/579292046 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f8be8042020 con 0x7f8bf806cea0 2024-07-04T17:28:53.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.930+0000 7f8beea00640 1 --2- 172.21.15.42:0/579292046 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f8bd403d4f0 0x7f8bd403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:28:53.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.930+0000 7f8beea00640 1 -- 172.21.15.42:0/579292046 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 3533+0+0 (secure 0 0 0) 0x7f8be8036070 con 0x7f8bf806cea0 2024-07-04T17:28:53.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.930+0000 7f8bed400640 1 --2- 172.21.15.42:0/579292046 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f8bd403d4f0 0x7f8bd403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:28:53.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.931+0000 7f8bed400640 1 --2- 172.21.15.42:0/579292046 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f8bd403d4f0 0x7f8bd403fa00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8be0007fd0 tx=0x7f8be00078e0 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:28:53.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:53.935+0000 7f8beea00640 1 -- 172.21.15.42:0/579292046 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f8bf80b39f0 con 0x7f8bf806cea0 2024-07-04T17:28:54.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:54.184+0000 7f8bfd000640 1 -- 172.21.15.42:0/579292046 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8bf80b3030 con 0x7f8bf806cea0 2024-07-04T17:28:54.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:54.185+0000 7f8beea00640 1 -- 172.21.15.42:0/579292046 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+4733 (secure 0 0 0) 0x7f8be804dcf0 con 0x7f8bf806cea0 2024-07-04T17:28:54.187 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:28:54.187 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":29,"btime":"2024-07-04T17:28:46:261488+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/399707138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":399707138},{"type":"v1","addr":"172.21.15.42:6827","nonce":399707138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4414,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1510714231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1510714231},{"type":"v1","addr":"172.21.15.42:6829","nonce":1510714231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/3069680926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":3069680926},{"type":"v1","addr":"172.21.15.42:6831","nonce":3069680926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4652,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2211433974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2211433974},{"type":"v1","addr":"172.21.15.42:6835","nonce":2211433974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":4684,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6833/1828290628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1828290628},{"type":"v1","addr":"172.21.15.42:6833","nonce":1828290628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[]} 2024-07-04T17:28:54.187 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-07-04T17:28:54.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:54.188+0000 7f8bfd000640 1 -- 172.21.15.42:0/579292046 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f8bd403d4f0 msgr2=0x7f8bd403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:54.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:54.188+0000 7f8bfd000640 1 --2- 172.21.15.42:0/579292046 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f8bd403d4f0 0x7f8bd403fa00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8be0007fd0 tx=0x7f8be00078e0 comp rx=0 tx=0).stop 2024-07-04T17:28:54.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:54.188+0000 7f8bfd000640 1 -- 172.21.15.42:0/579292046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bf806cea0 msgr2=0x7f8bf80bdec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:28:54.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:54.188+0000 7f8bfd000640 1 --2- 172.21.15.42:0/579292046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bf806cea0 0x7f8bf80bdec0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8be8034040 tx=0x7f8be80043b0 comp rx=0 tx=0).stop 2024-07-04T17:28:54.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:54.188+0000 7f8bfd000640 1 -- 172.21.15.42:0/579292046 shutdown_connections 2024-07-04T17:28:54.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:54.188+0000 7f8bfd000640 1 --2- 172.21.15.42:0/579292046 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f8bd403d4f0 0x7f8bd403fa00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:54.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:54.188+0000 7f8bfd000640 1 --2- 172.21.15.42:0/579292046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bf806cea0 0x7f8bf80bdec0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:28:54.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:54.188+0000 7f8bfd000640 1 -- 172.21.15.42:0/579292046 >> 172.21.15.42:0/579292046 conn(0x7f8bf80bf700 msgr2=0x7f8bf81495c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:28:54.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:54.188+0000 7f8bfd000640 1 -- 172.21.15.42:0/579292046 shutdown_connections 2024-07-04T17:28:54.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:28:54.188+0000 7f8bfd000640 1 -- 172.21.15.42:0/579292046 wait complete. 2024-07-04T17:28:54.217 INFO:tasks.ceph.mds.a:Restarting daemon 2024-07-04T17:28:54.217 INFO:tasks.ceph.mds.a:Stopping old one... 2024-07-04T17:28:54.217 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-07-04T17:28:54.217 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:28:54.217 INFO:tasks.ceph.mds.b:Restarting daemon 2024-07-04T17:28:54.217 INFO:tasks.ceph.mds.b:Stopping old one... 2024-07-04T17:28:54.217 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-07-04T17:28:54.218 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:28:54.218 INFO:tasks.ceph.mds.c:Restarting daemon 2024-07-04T17:28:54.218 INFO:tasks.ceph.mds.c:Stopping old one... 2024-07-04T17:28:54.218 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-07-04T17:28:54.218 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:28:54.218 INFO:tasks.ceph.mds.d:Restarting daemon 2024-07-04T17:28:54.218 INFO:tasks.ceph.mds.d:Stopping old one... 2024-07-04T17:28:54.219 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-07-04T17:28:54.219 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:28:54.219 INFO:tasks.ceph.mds.e:Restarting daemon 2024-07-04T17:28:54.219 INFO:tasks.ceph.mds.e:Stopping old one... 2024-07-04T17:28:54.219 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-07-04T17:28:54.219 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:28:54.247 INFO:tasks.ceph.mds.e:Stopped 2024-07-04T17:28:54.247 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-07-04T17:28:54.248 INFO:tasks.ceph.mds.a:Stopped 2024-07-04T17:28:54.248 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-07-04T17:28:54.257 INFO:tasks.ceph.mds.e:Started 2024-07-04T17:28:54.258 DEBUG:teuthology.parallel:result is None 2024-07-04T17:28:54.258 INFO:tasks.ceph.mds.b:Stopped 2024-07-04T17:28:54.258 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-07-04T17:28:54.261 INFO:tasks.ceph.mds.a:Started 2024-07-04T17:28:54.261 DEBUG:teuthology.parallel:result is None 2024-07-04T17:28:54.262 INFO:tasks.ceph.mds.c:Stopped 2024-07-04T17:28:54.262 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-07-04T17:28:54.307 INFO:tasks.ceph.mds.b:Started 2024-07-04T17:28:54.308 DEBUG:teuthology.parallel:result is None 2024-07-04T17:28:54.315 INFO:tasks.ceph.mds.c:Started 2024-07-04T17:28:54.315 DEBUG:teuthology.parallel:result is None 2024-07-04T17:28:54.480 INFO:tasks.ceph.mds.a.smithi042.stdout:starting mds.a at 2024-07-04T17:28:54.484 INFO:tasks.ceph.mds.e.smithi042.stdout:starting mds.e at 2024-07-04T17:28:54.532 INFO:tasks.ceph.mds.b.smithi042.stdout:starting mds.b at 2024-07-04T17:28:54.540 INFO:tasks.ceph.mds.c.smithi042.stdout:starting mds.c at 2024-07-04T17:28:58.401 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-07-04T17:29:00.021 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-04T17:29:00.253 INFO:tasks.ceph.mds.d:Stopped 2024-07-04T17:29:00.253 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-07-04T17:29:00.261 INFO:tasks.ceph.mds.d:Started 2024-07-04T17:29:00.261 DEBUG:teuthology.parallel:result is None 2024-07-04T17:29:00.261 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-07-04T17:29:00.504 INFO:tasks.ceph.mds.d.smithi042.stdout:starting mds.d at 2024-07-04T17:29:00.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:00.534+0000 7fad89200640 1 -- 172.21.15.42:0/2894296819 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad8406cf90 msgr2=0x7fad84141dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:00.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:00.534+0000 7fad89200640 1 --2- 172.21.15.42:0/2894296819 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad8406cf90 0x7fad84141dd0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fad74005fd0 tx=0x7fad7402a5f0 comp rx=0 tx=0).stop 2024-07-04T17:29:00.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:00.534+0000 7fad89200640 1 -- 172.21.15.42:0/2894296819 shutdown_connections 2024-07-04T17:29:00.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:00.534+0000 7fad89200640 1 --2- 172.21.15.42:0/2894296819 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad8406cf90 0x7fad84141dd0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:00.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:00.534+0000 7fad89200640 1 -- 172.21.15.42:0/2894296819 >> 172.21.15.42:0/2894296819 conn(0x7fad840bebc0 msgr2=0x7fad840befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:00.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:00.535+0000 7fad89200640 1 -- 172.21.15.42:0/2894296819 shutdown_connections 2024-07-04T17:29:00.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:00.535+0000 7fad89200640 1 -- 172.21.15.42:0/2894296819 wait complete. 2024-07-04T17:29:00.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:00.537+0000 7fad89200640 1 Processor -- start 2024-07-04T17:29:00.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:00.537+0000 7fad89200640 1 -- start start 2024-07-04T17:29:00.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:00.538+0000 7fad89200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad8406cf90 0x7fad84139f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:00.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:00.538+0000 7fad89200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fad840babe0 con 0x7fad8406cf90 2024-07-04T17:29:00.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:00.538+0000 7fad83400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad8406cf90 0x7fad84139f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:00.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:00.538+0000 7fad83400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad8406cf90 0x7fad84139f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37388/0 (socket says 172.21.15.42:37388) 2024-07-04T17:29:00.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:00.538+0000 7fad83400640 1 -- 172.21.15.42:0/4160414829 learned_addr learned my addr 172.21.15.42:0/4160414829 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:00.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:00.539+0000 7fad83400640 1 -- 172.21.15.42:0/4160414829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad8413bc90 con 0x7fad8406cf90 2024-07-04T17:29:00.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:00.539+0000 7fad83400640 1 --2- 172.21.15.42:0/4160414829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad8406cf90 0x7fad84139f60 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fad74009850 tx=0x7fad740049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:00.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:00.539+0000 7fad7b400640 1 -- 172.21.15.42:0/4160414829 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad7403e030 con 0x7fad8406cf90 2024-07-04T17:29:00.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:00.539+0000 7fad7b400640 1 -- 172.21.15.42:0/4160414829 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad740035f0 con 0x7fad8406cf90 2024-07-04T17:29:00.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:00.539+0000 7fad89200640 1 -- 172.21.15.42:0/4160414829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad84138d10 con 0x7fad8406cf90 2024-07-04T17:29:00.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:00.539+0000 7fad7b400640 1 -- 172.21.15.42:0/4160414829 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad74035040 con 0x7fad8406cf90 2024-07-04T17:29:00.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:00.540+0000 7fad89200640 1 -- 172.21.15.42:0/4160414829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad84139230 con 0x7fad8406cf90 2024-07-04T17:29:00.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:00.541+0000 7fad7b400640 1 -- 172.21.15.42:0/4160414829 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7fad74041030 con 0x7fad8406cf90 2024-07-04T17:29:00.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:00.541+0000 7fad89200640 1 -- 172.21.15.42:0/4160414829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad8413bab0 con 0x7fad8406cf90 2024-07-04T17:29:00.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:00.541+0000 7fad7b400640 1 --2- 172.21.15.42:0/4160414829 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fad6003d4a0 0x7fad6003f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:00.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:00.541+0000 7fad7b400640 1 -- 172.21.15.42:0/4160414829 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 3533+0+0 (secure 0 0 0) 0x7fad7403f070 con 0x7fad8406cf90 2024-07-04T17:29:00.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:00.542+0000 7fad82a00640 1 --2- 172.21.15.42:0/4160414829 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fad6003d4a0 0x7fad6003f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:00.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:00.542+0000 7fad82a00640 1 --2- 172.21.15.42:0/4160414829 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fad6003d4a0 0x7fad6003f9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fad6c001560 tx=0x7fad6c0079f0 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:00.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:00.547+0000 7fad7b400640 1 -- 172.21.15.42:0/4160414829 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fad8413bab0 con 0x7fad8406cf90 2024-07-04T17:29:00.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:00.773+0000 7fad89200640 1 -- 172.21.15.42:0/4160414829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fad840bb1b0 con 0x7fad8406cf90 2024-07-04T17:29:01.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:01.711+0000 7fad7b400640 1 -- 172.21.15.42:0/4160414829 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v46) ==== 97+0+0 (secure 0 0 0) 0x7fad740720f0 con 0x7fad8406cf90 2024-07-04T17:29:01.713 INFO:teuthology.orchestra.run.smithi042.stderr: removed all blocklist entries 2024-07-04T17:29:01.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:01.715+0000 7fad89200640 1 -- 172.21.15.42:0/4160414829 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fad6003d4a0 msgr2=0x7fad6003f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:01.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:01.715+0000 7fad89200640 1 --2- 172.21.15.42:0/4160414829 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fad6003d4a0 0x7fad6003f9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fad6c001560 tx=0x7fad6c0079f0 comp rx=0 tx=0).stop 2024-07-04T17:29:01.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:01.715+0000 7fad89200640 1 -- 172.21.15.42:0/4160414829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad8406cf90 msgr2=0x7fad84139f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:01.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:01.715+0000 7fad89200640 1 --2- 172.21.15.42:0/4160414829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad8406cf90 0x7fad84139f60 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fad74009850 tx=0x7fad740049d0 comp rx=0 tx=0).stop 2024-07-04T17:29:01.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:01.715+0000 7fad89200640 1 -- 172.21.15.42:0/4160414829 shutdown_connections 2024-07-04T17:29:01.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:01.715+0000 7fad89200640 1 --2- 172.21.15.42:0/4160414829 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fad6003d4a0 0x7fad6003f9b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:01.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:01.715+0000 7fad89200640 1 --2- 172.21.15.42:0/4160414829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fad8406cf90 0x7fad84139f60 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:01.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:01.715+0000 7fad89200640 1 -- 172.21.15.42:0/4160414829 >> 172.21.15.42:0/4160414829 conn(0x7fad840bebc0 msgr2=0x7fad840b9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:01.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:01.715+0000 7fad89200640 1 -- 172.21.15.42:0/4160414829 shutdown_connections 2024-07-04T17:29:01.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:01.715+0000 7fad89200640 1 -- 172.21.15.42:0/4160414829 wait complete. 2024-07-04T17:29:01.744 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-07-04T17:29:02.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.025+0000 7f15df200640 1 -- 172.21.15.42:0/758261787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15e0144fe0 msgr2=0x7f15e006c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:02.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.025+0000 7f15df200640 1 --2- 172.21.15.42:0/758261787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15e0144fe0 0x7f15e006c020 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f15c8005fd0 tx=0x7f15c802a5f0 comp rx=0 tx=0).stop 2024-07-04T17:29:02.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.025+0000 7f15df200640 1 -- 172.21.15.42:0/758261787 shutdown_connections 2024-07-04T17:29:02.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.025+0000 7f15df200640 1 --2- 172.21.15.42:0/758261787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15e0144fe0 0x7f15e006c020 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:02.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.026+0000 7f15df200640 1 -- 172.21.15.42:0/758261787 >> 172.21.15.42:0/758261787 conn(0x7f15e00bebc0 msgr2=0x7f15e00befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:02.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.026+0000 7f15df200640 1 -- 172.21.15.42:0/758261787 shutdown_connections 2024-07-04T17:29:02.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.026+0000 7f15df200640 1 -- 172.21.15.42:0/758261787 wait complete. 2024-07-04T17:29:02.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.028+0000 7f15df200640 1 Processor -- start 2024-07-04T17:29:02.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.028+0000 7f15df200640 1 -- start start 2024-07-04T17:29:02.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.029+0000 7f15df200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15e00ba830 0x7f15e00ad250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:02.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.029+0000 7f15df200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f15e00b1e30 con 0x7f15e00ba830 2024-07-04T17:29:02.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.030+0000 7f15dde00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15e00ba830 0x7f15e00ad250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:02.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.030+0000 7f15dde00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15e00ba830 0x7f15e00ad250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37412/0 (socket says 172.21.15.42:37412) 2024-07-04T17:29:02.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.030+0000 7f15dde00640 1 -- 172.21.15.42:0/2306115610 learned_addr learned my addr 172.21.15.42:0/2306115610 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:02.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.030+0000 7f15dde00640 1 -- 172.21.15.42:0/2306115610 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e00acd70 con 0x7f15e00ba830 2024-07-04T17:29:02.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.030+0000 7f15dde00640 1 --2- 172.21.15.42:0/2306115610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15e00ba830 0x7f15e00ad250 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f15d000c2f0 tx=0x7f15d000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:02.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.031+0000 7f15d6000640 1 -- 172.21.15.42:0/2306115610 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15d0017070 con 0x7f15e00ba830 2024-07-04T17:29:02.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.031+0000 7f15d6000640 1 -- 172.21.15.42:0/2306115610 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15d0005410 con 0x7f15e00ba830 2024-07-04T17:29:02.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.031+0000 7f15d6000640 1 -- 172.21.15.42:0/2306115610 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15d0003780 con 0x7f15e00ba830 2024-07-04T17:29:02.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.031+0000 7f15df200640 1 -- 172.21.15.42:0/2306115610 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e013fd40 con 0x7f15e00ba830 2024-07-04T17:29:02.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.031+0000 7f15df200640 1 -- 172.21.15.42:0/2306115610 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e00ac9d0 con 0x7f15e00ba830 2024-07-04T17:29:02.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.032+0000 7f15d6000640 1 -- 172.21.15.42:0/2306115610 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f15d001a070 con 0x7f15e00ba830 2024-07-04T17:29:02.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.032+0000 7f15df200640 1 -- 172.21.15.42:0/2306115610 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15e00bce60 con 0x7f15e00ba830 2024-07-04T17:29:02.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.033+0000 7f15d6000640 1 --2- 172.21.15.42:0/2306115610 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f15bc03d4f0 0x7f15bc03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:02.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.033+0000 7f15d6000640 1 -- 172.21.15.42:0/2306115610 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 2587+0+0 (secure 0 0 0) 0x7f15d00508c0 con 0x7f15e00ba830 2024-07-04T17:29:02.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.033+0000 7f15d7e00640 1 --2- 172.21.15.42:0/2306115610 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f15bc03d4f0 0x7f15bc03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:02.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.034+0000 7f15d7e00640 1 --2- 172.21.15.42:0/2306115610 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f15bc03d4f0 0x7f15bc03fa00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f15c8033000 tx=0x7f15c8004730 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:02.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.038+0000 7f15d6000640 1 -- 172.21.15.42:0/2306115610 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f15e00bce60 con 0x7f15e00ba830 2024-07-04T17:29:02.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.314+0000 7f15df200640 1 -- 172.21.15.42:0/2306115610 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f15e006c020 con 0x7f15e00ba830 2024-07-04T17:29:02.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.314+0000 7f15d6000640 1 -- 172.21.15.42:0/2306115610 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+4064 (secure 0 0 0) 0x7f15d0010300 con 0x7f15e00ba830 2024-07-04T17:29:02.316 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:29:02.316 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:29:02.316 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_dump": [ 2024-07-04T17:29:02.316 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:02.316 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.a", 2024-07-04T17:29:02.316 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAM24Zm2VMEFxAAQqdcGux6Ker7jEr1dGAfzA==", 2024-07-04T17:29:02.317 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:29:02.317 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:29:02.317 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:29:02.317 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:29:02.317 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:29:02.317 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:02.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:02.317 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:02.317 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.b", 2024-07-04T17:29:02.317 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAM24ZmS/S7KxAAfjTZrLYR9qLxuTvd2zsCcQ==", 2024-07-04T17:29:02.317 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:29:02.317 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:29:02.317 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:29:02.317 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:29:02.317 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:29:02.318 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:02.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:02.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:02.318 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.c", 2024-07-04T17:29:02.318 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAN24Zm2JrRBBAAFMYIM2fLsr9EOfcyh3tZpA==", 2024-07-04T17:29:02.318 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:29:02.318 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:29:02.318 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:29:02.318 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:29:02.318 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:29:02.318 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:02.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:02.318 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:02.319 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.d", 2024-07-04T17:29:02.319 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAN24ZmPCGLFxAAhsh3XaAHCyauCGxh/3DFsw==", 2024-07-04T17:29:02.319 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:29:02.319 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:29:02.319 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:29:02.319 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:29:02.319 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:29:02.319 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:02.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:02.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:02.319 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.e", 2024-07-04T17:29:02.319 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAN24ZmVIaSKhAAhoBdDalwQ4eoo0rqb6p0Vw==", 2024-07-04T17:29:02.319 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:29:02.319 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:29:02.320 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:29:02.320 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:29:02.320 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:29:02.320 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:02.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:02.320 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:02.320 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.0", 2024-07-04T17:29:02.320 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAQ24Zm4spcCBAA4kTvvR7IOVa01K5ZCrJ35w==", 2024-07-04T17:29:02.320 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:29:02.320 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-07-04T17:29:02.320 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-07-04T17:29:02.320 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:29:02.320 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:02.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:02.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:02.321 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.1", 2024-07-04T17:29:02.321 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAT24Zmz1sfGRAAgfH5eIiQrufnPq6F94rHWg==", 2024-07-04T17:29:02.321 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:29:02.321 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-07-04T17:29:02.321 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-07-04T17:29:02.321 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:29:02.321 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:02.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:02.321 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:02.321 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.2", 2024-07-04T17:29:02.321 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAW24ZmJdQHORAAEjAM9OBqSIRhX148GC6pcw==", 2024-07-04T17:29:02.321 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:29:02.322 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-07-04T17:29:02.322 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-07-04T17:29:02.322 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:29:02.322 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:02.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:02.322 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:02.322 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.0", 2024-07-04T17:29:02.322 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAO24ZmUMmYABAAh4pNlkb9JsHQtK8uCJFQCA==", 2024-07-04T17:29:02.322 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:29:02.322 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:29:02.322 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow r", 2024-07-04T17:29:02.322 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-07-04T17:29:02.322 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:02.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:02.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:02.323 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.1", 2024-07-04T17:29:02.323 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAO24Zm0pAJEBAAntDeQ/oWDlJ+3xC7q5EQnQ==", 2024-07-04T17:29:02.323 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:29:02.323 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:29:02.323 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-07-04T17:29:02.323 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow rw", 2024-07-04T17:29:02.323 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rwx" 2024-07-04T17:29:02.323 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:02.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:02.323 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:02.323 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.admin", 2024-07-04T17:29:02.323 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAL24ZmMe3iHhAAFciH9cbUAIHLSAJvXy30EQ==", 2024-07-04T17:29:02.323 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:29:02.323 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-07-04T17:29:02.323 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:29:02.324 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:29:02.324 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:29:02.324 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:02.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:02.324 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:02.324 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.mirror", 2024-07-04T17:29:02.324 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAO24ZmBJzBHhAAR4Zg36YTtVizxm/g/Cdk0w==", 2024-07-04T17:29:02.324 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:29:02.324 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow r", 2024-07-04T17:29:02.324 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-07-04T17:29:02.324 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "profile cephfs-mirror", 2024-07-04T17:29:02.324 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rw tag cephfs metadata=*, allow r tag cephfs data=*" 2024-07-04T17:29:02.324 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:02.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:02.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:02.325 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.mirror_remote", 2024-07-04T17:29:02.325 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAO24Zm+BsqLhAA19GVuEjfWQeMe/fSc+dmoA==", 2024-07-04T17:29:02.325 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:29:02.325 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow rwps", 2024-07-04T17:29:02.325 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-07-04T17:29:02.325 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow r", 2024-07-04T17:29:02.325 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rw tag cephfs *=*" 2024-07-04T17:29:02.325 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:02.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:02.325 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:02.325 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mgr.x", 2024-07-04T17:29:02.325 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAM24Zm47tDCBAAjszudEEB4QHLlWqPXyZc2g==", 2024-07-04T17:29:02.326 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:29:02.326 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-07-04T17:29:02.326 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile mgr", 2024-07-04T17:29:02.326 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:29:02.326 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:02.326 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:02.326 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-07-04T17:29:02.326 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:29:02.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.317+0000 7f15df200640 1 -- 172.21.15.42:0/2306115610 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f15bc03d4f0 msgr2=0x7f15bc03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:02.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.317+0000 7f15df200640 1 --2- 172.21.15.42:0/2306115610 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f15bc03d4f0 0x7f15bc03fa00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f15c8033000 tx=0x7f15c8004730 comp rx=0 tx=0).stop 2024-07-04T17:29:02.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.317+0000 7f15df200640 1 -- 172.21.15.42:0/2306115610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15e00ba830 msgr2=0x7f15e00ad250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:02.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.317+0000 7f15df200640 1 --2- 172.21.15.42:0/2306115610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15e00ba830 0x7f15e00ad250 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f15d000c2f0 tx=0x7f15d000c7b0 comp rx=0 tx=0).stop 2024-07-04T17:29:02.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.317+0000 7f15df200640 1 -- 172.21.15.42:0/2306115610 shutdown_connections 2024-07-04T17:29:02.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.317+0000 7f15df200640 1 --2- 172.21.15.42:0/2306115610 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f15bc03d4f0 0x7f15bc03fa00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:02.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.317+0000 7f15df200640 1 --2- 172.21.15.42:0/2306115610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15e00ba830 0x7f15e00ad250 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:02.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.317+0000 7f15df200640 1 -- 172.21.15.42:0/2306115610 >> 172.21.15.42:0/2306115610 conn(0x7f15e00bebc0 msgr2=0x7f15e0142d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:02.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.317+0000 7f15df200640 1 -- 172.21.15.42:0/2306115610 shutdown_connections 2024-07-04T17:29:02.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.317+0000 7f15df200640 1 -- 172.21.15.42:0/2306115610 wait complete. 2024-07-04T17:29:02.344 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-04T17:29:02.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.616+0000 7f3787200640 1 -- 172.21.15.42:0/2385560369 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f378806c2c0 msgr2=0x7f37880b5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:02.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.616+0000 7f3787200640 1 --2- 172.21.15.42:0/2385560369 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f378806c2c0 0x7f37880b5650 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3774005fd0 tx=0x7f377402ae00 comp rx=0 tx=0).stop 2024-07-04T17:29:02.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.617+0000 7f3787200640 1 -- 172.21.15.42:0/2385560369 shutdown_connections 2024-07-04T17:29:02.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.617+0000 7f3787200640 1 --2- 172.21.15.42:0/2385560369 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f378806c2c0 0x7f37880b5650 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:02.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.617+0000 7f3787200640 1 -- 172.21.15.42:0/2385560369 >> 172.21.15.42:0/2385560369 conn(0x7f37880bebc0 msgr2=0x7f37880befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:02.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.617+0000 7f3787200640 1 -- 172.21.15.42:0/2385560369 shutdown_connections 2024-07-04T17:29:02.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.617+0000 7f3787200640 1 -- 172.21.15.42:0/2385560369 wait complete. 2024-07-04T17:29:02.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.619+0000 7f3787200640 1 Processor -- start 2024-07-04T17:29:02.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.619+0000 7f3787200640 1 -- start start 2024-07-04T17:29:02.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.620+0000 7f3787200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f378806c2c0 0x7f378806fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:02.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.620+0000 7f3787200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f37880b27b0 con 0x7f378806c2c0 2024-07-04T17:29:02.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.620+0000 7f3785e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f378806c2c0 0x7f378806fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:02.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.620+0000 7f3785e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f378806c2c0 0x7f378806fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37426/0 (socket says 172.21.15.42:37426) 2024-07-04T17:29:02.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.620+0000 7f3785e00640 1 -- 172.21.15.42:0/3179513717 learned_addr learned my addr 172.21.15.42:0/3179513717 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:02.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.621+0000 7f3785e00640 1 -- 172.21.15.42:0/3179513717 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378813d560 con 0x7f378806c2c0 2024-07-04T17:29:02.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.621+0000 7f3785e00640 1 --2- 172.21.15.42:0/3179513717 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f378806c2c0 0x7f378806fe70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f377800c380 tx=0x7f377800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:02.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.621+0000 7f377e000640 1 -- 172.21.15.42:0/3179513717 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3778017070 con 0x7f378806c2c0 2024-07-04T17:29:02.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.621+0000 7f377e000640 1 -- 172.21.15.42:0/3179513717 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37780052f0 con 0x7f378806c2c0 2024-07-04T17:29:02.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.621+0000 7f377e000640 1 -- 172.21.15.42:0/3179513717 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37780035f0 con 0x7f378806c2c0 2024-07-04T17:29:02.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.621+0000 7f3787200640 1 -- 172.21.15.42:0/3179513717 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3788070f60 con 0x7f378806c2c0 2024-07-04T17:29:02.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.621+0000 7f3787200640 1 -- 172.21.15.42:0/3179513717 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3788070b90 con 0x7f378806c2c0 2024-07-04T17:29:02.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.622+0000 7f3787200640 1 -- 172.21.15.42:0/3179513717 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37880b5650 con 0x7f378806c2c0 2024-07-04T17:29:02.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.623+0000 7f377e000640 1 -- 172.21.15.42:0/3179513717 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f377801a070 con 0x7f378806c2c0 2024-07-04T17:29:02.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.623+0000 7f377e000640 1 --2- 172.21.15.42:0/3179513717 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f376803d4f0 0x7f376803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:02.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.623+0000 7f377e000640 1 -- 172.21.15.42:0/3179513717 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 2587+0+0 (secure 0 0 0) 0x7f3778050840 con 0x7f378806c2c0 2024-07-04T17:29:02.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.623+0000 7f3785400640 1 --2- 172.21.15.42:0/3179513717 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f376803d4f0 0x7f376803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:02.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.624+0000 7f3785400640 1 --2- 172.21.15.42:0/3179513717 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f376803d4f0 0x7f376803fa00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3774034000 tx=0x7f3774004730 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:02.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.628+0000 7f377e000640 1 -- 172.21.15.42:0/3179513717 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f378806c6d0 con 0x7f378806c2c0 2024-07-04T17:29:02.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.843+0000 7f3787200640 1 -- 172.21.15.42:0/3179513717 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f3788068e30 con 0x7f378806c2c0 2024-07-04T17:29:02.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.844+0000 7f377e000640 1 -- 172.21.15.42:0/3179513717 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v46) ==== 84+0+65 (secure 0 0 0) 0x7f3778016170 con 0x7f378806c2c0 2024-07-04T17:29:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:29:02.846 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-07-04T17:29:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 1, 2024-07-04T17:29:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": ".mgr" 2024-07-04T17:29:02.846 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:02.846 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-07-04T17:29:02.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.847+0000 7f3787200640 1 -- 172.21.15.42:0/3179513717 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f376803d4f0 msgr2=0x7f376803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:02.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.847+0000 7f3787200640 1 --2- 172.21.15.42:0/3179513717 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f376803d4f0 0x7f376803fa00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3774034000 tx=0x7f3774004730 comp rx=0 tx=0).stop 2024-07-04T17:29:02.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.847+0000 7f3787200640 1 -- 172.21.15.42:0/3179513717 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f378806c2c0 msgr2=0x7f378806fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:02.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.847+0000 7f3787200640 1 --2- 172.21.15.42:0/3179513717 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f378806c2c0 0x7f378806fe70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f377800c380 tx=0x7f377800c840 comp rx=0 tx=0).stop 2024-07-04T17:29:02.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.847+0000 7f3787200640 1 -- 172.21.15.42:0/3179513717 shutdown_connections 2024-07-04T17:29:02.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.847+0000 7f3787200640 1 --2- 172.21.15.42:0/3179513717 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f376803d4f0 0x7f376803fa00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:02.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.847+0000 7f3787200640 1 --2- 172.21.15.42:0/3179513717 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f378806c2c0 0x7f378806fe70 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:02.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.848+0000 7f3787200640 1 -- 172.21.15.42:0/3179513717 >> 172.21.15.42:0/3179513717 conn(0x7f37880bebc0 msgr2=0x7f37881527c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:02.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.848+0000 7f3787200640 1 -- 172.21.15.42:0/3179513717 shutdown_connections 2024-07-04T17:29:02.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:02.848+0000 7f3787200640 1 -- 172.21.15.42:0/3179513717 wait complete. 2024-07-04T17:29:02.877 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-04T17:29:02.877 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-04T17:29:03.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.166+0000 7ff0f8000640 1 -- 172.21.15.42:0/2900497970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0f00aa050 msgr2=0x7ff0f013f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:03.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.166+0000 7ff0f8000640 1 --2- 172.21.15.42:0/2900497970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0f00aa050 0x7ff0f013f2b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc0097b0 tx=0x7ff0dc02a5d0 comp rx=0 tx=0).stop 2024-07-04T17:29:03.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.166+0000 7ff0f8000640 1 -- 172.21.15.42:0/2900497970 shutdown_connections 2024-07-04T17:29:03.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.166+0000 7ff0f8000640 1 --2- 172.21.15.42:0/2900497970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0f00aa050 0x7ff0f013f2b0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:03.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.166+0000 7ff0f8000640 1 -- 172.21.15.42:0/2900497970 >> 172.21.15.42:0/2900497970 conn(0x7ff0f006c020 msgr2=0x7ff0f006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:03.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.166+0000 7ff0f8000640 1 -- 172.21.15.42:0/2900497970 shutdown_connections 2024-07-04T17:29:03.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.166+0000 7ff0f8000640 1 -- 172.21.15.42:0/2900497970 wait complete. 2024-07-04T17:29:03.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.168+0000 7ff0f8000640 1 Processor -- start 2024-07-04T17:29:03.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.168+0000 7ff0f8000640 1 -- start start 2024-07-04T17:29:03.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.169+0000 7ff0f8000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0f00733c0 0x7ff0f00b7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:03.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.170+0000 7ff0f8000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f0140880 con 0x7ff0f00733c0 2024-07-04T17:29:03.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.170+0000 7ff0f5600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0f00733c0 0x7ff0f00b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:03.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.170+0000 7ff0f5600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0f00733c0 0x7ff0f00b7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37454/0 (socket says 172.21.15.42:37454) 2024-07-04T17:29:03.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.170+0000 7ff0f5600640 1 -- 172.21.15.42:0/1747643649 learned_addr learned my addr 172.21.15.42:0/1747643649 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:03.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.170+0000 7ff0f5600640 1 -- 172.21.15.42:0/1747643649 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f00b7d30 con 0x7ff0f00733c0 2024-07-04T17:29:03.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.170+0000 7ff0f5600640 1 --2- 172.21.15.42:0/1747643649 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0f00733c0 0x7ff0f00b7660 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000cb40 tx=0x7ff0e000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:03.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.171+0000 7ff0ed600640 1 -- 172.21.15.42:0/1747643649 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0e0016020 con 0x7ff0f00733c0 2024-07-04T17:29:03.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.171+0000 7ff0ed600640 1 -- 172.21.15.42:0/1747643649 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0e000abf0 con 0x7ff0f00733c0 2024-07-04T17:29:03.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.171+0000 7ff0ed600640 1 -- 172.21.15.42:0/1747643649 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0e0012420 con 0x7ff0f00733c0 2024-07-04T17:29:03.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.171+0000 7ff0f8000640 1 -- 172.21.15.42:0/1747643649 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0f01e2340 con 0x7ff0f00733c0 2024-07-04T17:29:03.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.171+0000 7ff0f8000640 1 -- 172.21.15.42:0/1747643649 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0f013e990 con 0x7ff0f00733c0 2024-07-04T17:29:03.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.172+0000 7ff0f8000640 1 -- 172.21.15.42:0/1747643649 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0f013ec20 con 0x7ff0f00733c0 2024-07-04T17:29:03.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.172+0000 7ff0ed600640 1 -- 172.21.15.42:0/1747643649 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7ff0e0019020 con 0x7ff0f00733c0 2024-07-04T17:29:03.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.173+0000 7ff0ed600640 1 --2- 172.21.15.42:0/1747643649 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7ff0d003d4f0 0x7ff0d003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:03.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.173+0000 7ff0ed600640 1 -- 172.21.15.42:0/1747643649 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 2587+0+0 (secure 0 0 0) 0x7ff0e0051940 con 0x7ff0f00733c0 2024-07-04T17:29:03.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.173+0000 7ff0f4c00640 1 --2- 172.21.15.42:0/1747643649 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7ff0d003d4f0 0x7ff0d003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:03.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.174+0000 7ff0f4c00640 1 --2- 172.21.15.42:0/1747643649 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7ff0d003d4f0 0x7ff0d003fa00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc034000 tx=0x7ff0dc004730 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:03.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.178+0000 7ff0ed600640 1 -- 172.21.15.42:0/1747643649 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7ff0e0020030 con 0x7ff0f00733c0 2024-07-04T17:29:03.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.384+0000 7ff0f8000640 1 -- 172.21.15.42:0/1747643649 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff0f0140f30 con 0x7ff0f00733c0 2024-07-04T17:29:03.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.730+0000 7ff0ed600640 1 -- 172.21.15.42:0/1747643649 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v47) ==== 138+0+0 (secure 0 0 0) 0x7ff0e000d150 con 0x7ff0f00733c0 2024-07-04T17:29:03.731 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' created 2024-07-04T17:29:03.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.733+0000 7ff0f8000640 1 -- 172.21.15.42:0/1747643649 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7ff0d003d4f0 msgr2=0x7ff0d003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:03.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.733+0000 7ff0f8000640 1 --2- 172.21.15.42:0/1747643649 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7ff0d003d4f0 0x7ff0d003fa00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc034000 tx=0x7ff0dc004730 comp rx=0 tx=0).stop 2024-07-04T17:29:03.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.734+0000 7ff0f8000640 1 -- 172.21.15.42:0/1747643649 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0f00733c0 msgr2=0x7ff0f00b7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:03.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.734+0000 7ff0f8000640 1 --2- 172.21.15.42:0/1747643649 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0f00733c0 0x7ff0f00b7660 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000cb40 tx=0x7ff0e000cf40 comp rx=0 tx=0).stop 2024-07-04T17:29:03.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.734+0000 7ff0f8000640 1 -- 172.21.15.42:0/1747643649 shutdown_connections 2024-07-04T17:29:03.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.734+0000 7ff0f8000640 1 --2- 172.21.15.42:0/1747643649 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7ff0d003d4f0 0x7ff0d003fa00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:03.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.734+0000 7ff0f8000640 1 --2- 172.21.15.42:0/1747643649 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0f00733c0 0x7ff0f00b7660 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:03.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.734+0000 7ff0f8000640 1 -- 172.21.15.42:0/1747643649 >> 172.21.15.42:0/1747643649 conn(0x7ff0f006c020 msgr2=0x7ff0f00b26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:03.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.734+0000 7ff0f8000640 1 -- 172.21.15.42:0/1747643649 shutdown_connections 2024-07-04T17:29:03.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:03.734+0000 7ff0f8000640 1 -- 172.21.15.42:0/1747643649 wait complete. 2024-07-04T17:29:03.774 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-04T17:29:04.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.061+0000 7fbd74a00640 1 -- 172.21.15.42:0/2025778660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd700b15c0 msgr2=0x7fbd700b0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:04.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.061+0000 7fbd74a00640 1 --2- 172.21.15.42:0/2025778660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd700b15c0 0x7fbd700b0210 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c0097b0 tx=0x7fbd5c02a540 comp rx=0 tx=0).stop 2024-07-04T17:29:04.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.061+0000 7fbd74a00640 1 -- 172.21.15.42:0/2025778660 shutdown_connections 2024-07-04T17:29:04.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.061+0000 7fbd74a00640 1 --2- 172.21.15.42:0/2025778660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd700b15c0 0x7fbd700b0210 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:04.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.061+0000 7fbd74a00640 1 -- 172.21.15.42:0/2025778660 >> 172.21.15.42:0/2025778660 conn(0x7fbd7006fdf0 msgr2=0x7fbd7006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:04.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.061+0000 7fbd74a00640 1 -- 172.21.15.42:0/2025778660 shutdown_connections 2024-07-04T17:29:04.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.061+0000 7fbd74a00640 1 -- 172.21.15.42:0/2025778660 wait complete. 2024-07-04T17:29:04.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.063+0000 7fbd74a00640 1 Processor -- start 2024-07-04T17:29:04.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.063+0000 7fbd74a00640 1 -- start start 2024-07-04T17:29:04.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.065+0000 7fbd74a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd700b15c0 0x7fbd700b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:04.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.065+0000 7fbd74a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbd701439c0 con 0x7fbd700b15c0 2024-07-04T17:29:04.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.065+0000 7fbd6e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd700b15c0 0x7fbd700b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:04.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.065+0000 7fbd6e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd700b15c0 0x7fbd700b6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37378/0 (socket says 172.21.15.42:37378) 2024-07-04T17:29:04.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.065+0000 7fbd6e800640 1 -- 172.21.15.42:0/3283651641 learned_addr learned my addr 172.21.15.42:0/3283651641 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:04.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.066+0000 7fbd6e800640 1 -- 172.21.15.42:0/3283651641 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd70071970 con 0x7fbd700b15c0 2024-07-04T17:29:04.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.066+0000 7fbd6e800640 1 --2- 172.21.15.42:0/3283651641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd700b15c0 0x7fbd700b6120 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fbd6000cb40 tx=0x7fbd6000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:04.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.066+0000 7fbd66a00640 1 -- 172.21.15.42:0/3283651641 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd60016020 con 0x7fbd700b15c0 2024-07-04T17:29:04.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.066+0000 7fbd66a00640 1 -- 172.21.15.42:0/3283651641 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd6000ac30 con 0x7fbd700b15c0 2024-07-04T17:29:04.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.066+0000 7fbd66a00640 1 -- 172.21.15.42:0/3283651641 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd60012420 con 0x7fbd700b15c0 2024-07-04T17:29:04.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.066+0000 7fbd74a00640 1 -- 172.21.15.42:0/3283651641 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd700b5ce0 con 0x7fbd700b15c0 2024-07-04T17:29:04.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.066+0000 7fbd74a00640 1 -- 172.21.15.42:0/3283651641 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd701dcf00 con 0x7fbd700b15c0 2024-07-04T17:29:04.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.068+0000 7fbd74a00640 1 -- 172.21.15.42:0/3283651641 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd70144070 con 0x7fbd700b15c0 2024-07-04T17:29:04.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.068+0000 7fbd66a00640 1 -- 172.21.15.42:0/3283651641 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7fbd60019020 con 0x7fbd700b15c0 2024-07-04T17:29:04.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.069+0000 7fbd66a00640 1 --2- 172.21.15.42:0/3283651641 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fbd5003d4f0 0x7fbd5003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:04.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.069+0000 7fbd66a00640 1 -- 172.21.15.42:0/3283651641 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 2957+0+0 (secure 0 0 0) 0x7fbd60051a10 con 0x7fbd700b15c0 2024-07-04T17:29:04.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.069+0000 7fbd6de00640 1 --2- 172.21.15.42:0/3283651641 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fbd5003d4f0 0x7fbd5003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:04.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.070+0000 7fbd6de00640 1 --2- 172.21.15.42:0/3283651641 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fbd5003d4f0 0x7fbd5003fa00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c034000 tx=0x7fbd5c004730 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:04.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.074+0000 7fbd66a00640 1 -- 172.21.15.42:0/3283651641 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fbd60020030 con 0x7fbd700b15c0 2024-07-04T17:29:04.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.284+0000 7fbd74a00640 1 -- 172.21.15.42:0/3283651641 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fbd70068e30 con 0x7fbd700b15c0 2024-07-04T17:29:04.408 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-07-04T17:29:04.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.734+0000 7fbd66a00640 1 -- 172.21.15.42:0/3283651641 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v48) ==== 170+0+0 (secure 0 0 0) 0x7fbd60011380 con 0x7fbd700b15c0 2024-07-04T17:29:04.736 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' created 2024-07-04T17:29:04.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.738+0000 7fbd74a00640 1 -- 172.21.15.42:0/3283651641 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fbd5003d4f0 msgr2=0x7fbd5003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:04.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.738+0000 7fbd74a00640 1 --2- 172.21.15.42:0/3283651641 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fbd5003d4f0 0x7fbd5003fa00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c034000 tx=0x7fbd5c004730 comp rx=0 tx=0).stop 2024-07-04T17:29:04.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.738+0000 7fbd74a00640 1 -- 172.21.15.42:0/3283651641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd700b15c0 msgr2=0x7fbd700b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:04.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.738+0000 7fbd74a00640 1 --2- 172.21.15.42:0/3283651641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd700b15c0 0x7fbd700b6120 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fbd6000cb40 tx=0x7fbd6000a060 comp rx=0 tx=0).stop 2024-07-04T17:29:04.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.738+0000 7fbd74a00640 1 -- 172.21.15.42:0/3283651641 shutdown_connections 2024-07-04T17:29:04.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.738+0000 7fbd74a00640 1 --2- 172.21.15.42:0/3283651641 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fbd5003d4f0 0x7fbd5003fa00 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:04.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.738+0000 7fbd74a00640 1 --2- 172.21.15.42:0/3283651641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd700b15c0 0x7fbd700b6120 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:04.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.738+0000 7fbd74a00640 1 -- 172.21.15.42:0/3283651641 >> 172.21.15.42:0/3283651641 conn(0x7fbd7006fdf0 msgr2=0x7fbd7013fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:04.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.739+0000 7fbd74a00640 1 -- 172.21.15.42:0/3283651641 shutdown_connections 2024-07-04T17:29:04.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:04.739+0000 7fbd74a00640 1 -- 172.21.15.42:0/3283651641 wait complete. 2024-07-04T17:29:04.788 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-04T17:29:05.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.052+0000 7fd160000640 1 -- 172.21.15.42:0/127365579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1580bd770 msgr2=0x7fd1580bdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:05.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.052+0000 7fd160000640 1 --2- 172.21.15.42:0/127365579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1580bd770 0x7fd1580bdb80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd1440097b0 tx=0x7fd14402a540 comp rx=0 tx=0).stop 2024-07-04T17:29:05.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.052+0000 7fd160000640 1 -- 172.21.15.42:0/127365579 shutdown_connections 2024-07-04T17:29:05.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.052+0000 7fd160000640 1 --2- 172.21.15.42:0/127365579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1580bd770 0x7fd1580bdb80 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:05.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.052+0000 7fd160000640 1 -- 172.21.15.42:0/127365579 >> 172.21.15.42:0/127365579 conn(0x7fd15806e2f0 msgr2=0x7fd15806e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:05.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.052+0000 7fd160000640 1 -- 172.21.15.42:0/127365579 shutdown_connections 2024-07-04T17:29:05.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.052+0000 7fd160000640 1 -- 172.21.15.42:0/127365579 wait complete. 2024-07-04T17:29:05.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.054+0000 7fd160000640 1 Processor -- start 2024-07-04T17:29:05.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.055+0000 7fd160000640 1 -- start start 2024-07-04T17:29:05.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.056+0000 7fd160000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1580bc4e0 0x7fd1580c0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:05.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.056+0000 7fd160000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd1580b7810 con 0x7fd1580bc4e0 2024-07-04T17:29:05.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.056+0000 7fd15d600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1580bc4e0 0x7fd1580c0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:05.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.056+0000 7fd15d600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1580bc4e0 0x7fd1580c0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37390/0 (socket says 172.21.15.42:37390) 2024-07-04T17:29:05.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.056+0000 7fd15d600640 1 -- 172.21.15.42:0/3342352279 learned_addr learned my addr 172.21.15.42:0/3342352279 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:05.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.057+0000 7fd15d600640 1 -- 172.21.15.42:0/3342352279 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1581e3280 con 0x7fd1580bc4e0 2024-07-04T17:29:05.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.057+0000 7fd15d600640 1 --2- 172.21.15.42:0/3342352279 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1580bc4e0 0x7fd1580c0500 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd14800c340 tx=0x7fd14800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:05.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.057+0000 7fd155600640 1 -- 172.21.15.42:0/3342352279 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd148005410 con 0x7fd1580bc4e0 2024-07-04T17:29:05.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.057+0000 7fd155600640 1 -- 172.21.15.42:0/3342352279 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd148007030 con 0x7fd1580bc4e0 2024-07-04T17:29:05.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.057+0000 7fd155600640 1 -- 172.21.15.42:0/3342352279 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1480073f0 con 0x7fd1580bc4e0 2024-07-04T17:29:05.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.057+0000 7fd160000640 1 -- 172.21.15.42:0/3342352279 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd15813fda0 con 0x7fd1580bc4e0 2024-07-04T17:29:05.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.058+0000 7fd160000640 1 -- 172.21.15.42:0/3342352279 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1581e3c80 con 0x7fd1580bc4e0 2024-07-04T17:29:05.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.059+0000 7fd160000640 1 -- 172.21.15.42:0/3342352279 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1580b7ef0 con 0x7fd1580bc4e0 2024-07-04T17:29:05.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.059+0000 7fd155600640 1 -- 172.21.15.42:0/3342352279 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7fd14801a020 con 0x7fd1580bc4e0 2024-07-04T17:29:05.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.060+0000 7fd155600640 1 --2- 172.21.15.42:0/3342352279 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fd13803d4f0 0x7fd13803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:05.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.060+0000 7fd155600640 1 -- 172.21.15.42:0/3342352279 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 3339+0+0 (secure 0 0 0) 0x7fd148050fc0 con 0x7fd1580bc4e0 2024-07-04T17:29:05.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.060+0000 7fd15cc00640 1 --2- 172.21.15.42:0/3342352279 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fd13803d4f0 0x7fd13803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:05.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.061+0000 7fd15cc00640 1 --2- 172.21.15.42:0/3342352279 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fd13803d4f0 0x7fd13803fa00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd144034000 tx=0x7fd144004730 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:05.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.065+0000 7fd155600640 1 -- 172.21.15.42:0/3342352279 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fd148007590 con 0x7fd1580bc4e0 2024-07-04T17:29:05.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.310+0000 7fd160000640 1 -- 172.21.15.42:0/3342352279 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd158068ea0 con 0x7fd1580bc4e0 2024-07-04T17:29:05.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.761+0000 7fd155600640 1 -- 172.21.15.42:0/3342352279 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v33) ==== 172+0+0 (secure 0 0 0) 0x7fd14800dce0 con 0x7fd1580bc4e0 2024-07-04T17:29:05.765 INFO:teuthology.orchestra.run.smithi042.stderr:new fs with metadata pool 8 and data pool 9 2024-07-04T17:29:05.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.768+0000 7fd160000640 1 -- 172.21.15.42:0/3342352279 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fd13803d4f0 msgr2=0x7fd13803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:05.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.768+0000 7fd160000640 1 --2- 172.21.15.42:0/3342352279 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fd13803d4f0 0x7fd13803fa00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd144034000 tx=0x7fd144004730 comp rx=0 tx=0).stop 2024-07-04T17:29:05.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.768+0000 7fd160000640 1 -- 172.21.15.42:0/3342352279 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1580bc4e0 msgr2=0x7fd1580c0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:05.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.768+0000 7fd160000640 1 --2- 172.21.15.42:0/3342352279 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1580bc4e0 0x7fd1580c0500 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd14800c340 tx=0x7fd14800c800 comp rx=0 tx=0).stop 2024-07-04T17:29:05.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.770+0000 7fd160000640 1 -- 172.21.15.42:0/3342352279 shutdown_connections 2024-07-04T17:29:05.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.770+0000 7fd160000640 1 --2- 172.21.15.42:0/3342352279 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fd13803d4f0 0x7fd13803fa00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:05.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.770+0000 7fd160000640 1 --2- 172.21.15.42:0/3342352279 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1580bc4e0 0x7fd1580c0500 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:05.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.770+0000 7fd160000640 1 -- 172.21.15.42:0/3342352279 >> 172.21.15.42:0/3342352279 conn(0x7fd15806e2f0 msgr2=0x7fd158075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:05.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.771+0000 7fd160000640 1 -- 172.21.15.42:0/3342352279 shutdown_connections 2024-07-04T17:29:05.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:05.771+0000 7fd160000640 1 -- 172.21.15.42:0/3342352279 wait complete. 2024-07-04T17:29:05.827 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:29:06.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.079+0000 7f3942400640 1 -- 172.21.15.42:0/1038337629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f393c0b17d0 msgr2=0x7f393c0b1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:06.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.079+0000 7f3942400640 1 --2- 172.21.15.42:0/1038337629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f393c0b17d0 0x7f393c0b1be0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f3928005fd0 tx=0x7f392802a820 comp rx=0 tx=0).stop 2024-07-04T17:29:06.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.079+0000 7f3942400640 1 -- 172.21.15.42:0/1038337629 shutdown_connections 2024-07-04T17:29:06.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.079+0000 7f3942400640 1 --2- 172.21.15.42:0/1038337629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f393c0b17d0 0x7f393c0b1be0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:06.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.079+0000 7f3942400640 1 -- 172.21.15.42:0/1038337629 >> 172.21.15.42:0/1038337629 conn(0x7f393c0bebc0 msgr2=0x7f393c0befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:06.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.079+0000 7f3942400640 1 -- 172.21.15.42:0/1038337629 shutdown_connections 2024-07-04T17:29:06.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.079+0000 7f3942400640 1 -- 172.21.15.42:0/1038337629 wait complete. 2024-07-04T17:29:06.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.081+0000 7f3942400640 1 Processor -- start 2024-07-04T17:29:06.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.081+0000 7f3942400640 1 -- start start 2024-07-04T17:29:06.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.083+0000 7f3942400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f393c06c200 0x7f393c0b6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:06.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.083+0000 7f3942400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f393c14ae80 con 0x7f393c06c200 2024-07-04T17:29:06.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.083+0000 7f3941000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f393c06c200 0x7f393c0b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:06.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.083+0000 7f3941000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f393c06c200 0x7f393c0b6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37416/0 (socket says 172.21.15.42:37416) 2024-07-04T17:29:06.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.083+0000 7f3941000640 1 -- 172.21.15.42:0/1900314055 learned_addr learned my addr 172.21.15.42:0/1900314055 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:06.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.083+0000 7f3941000640 1 -- 172.21.15.42:0/1900314055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393c0adbf0 con 0x7f393c06c200 2024-07-04T17:29:06.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.084+0000 7f3941000640 1 --2- 172.21.15.42:0/1900314055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f393c06c200 0x7f393c0b6960 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f392c00c320 tx=0x7f392c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:06.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.084+0000 7f3938c00640 1 -- 172.21.15.42:0/1900314055 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f392c0116c0 con 0x7f393c06c200 2024-07-04T17:29:06.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.084+0000 7f3938c00640 1 -- 172.21.15.42:0/1900314055 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f392c017070 con 0x7f393c06c200 2024-07-04T17:29:06.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.084+0000 7f3938c00640 1 -- 172.21.15.42:0/1900314055 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f392c011b40 con 0x7f393c06c200 2024-07-04T17:29:06.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.084+0000 7f3942400640 1 -- 172.21.15.42:0/1900314055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f393c1e4090 con 0x7f393c06c200 2024-07-04T17:29:06.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.084+0000 7f3942400640 1 -- 172.21.15.42:0/1900314055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f393c140400 con 0x7f393c06c200 2024-07-04T17:29:06.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.085+0000 7f3942400640 1 -- 172.21.15.42:0/1900314055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f393c14b4c0 con 0x7f393c06c200 2024-07-04T17:29:06.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.086+0000 7f3938c00640 1 -- 172.21.15.42:0/1900314055 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f392c011ce0 con 0x7f393c06c200 2024-07-04T17:29:06.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.086+0000 7f3938c00640 1 --2- 172.21.15.42:0/1900314055 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f391c03d4f0 0x7f391c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:06.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.086+0000 7f3938c00640 1 -- 172.21.15.42:0/1900314055 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 3439+0+0 (secure 0 0 0) 0x7f392c051690 con 0x7f393c06c200 2024-07-04T17:29:06.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.086+0000 7f393be00640 1 --2- 172.21.15.42:0/1900314055 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f391c03d4f0 0x7f391c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:06.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.087+0000 7f393be00640 1 --2- 172.21.15.42:0/1900314055 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f391c03d4f0 0x7f391c03fa00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3928034000 tx=0x7f3928004ab0 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:06.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.092+0000 7f3938c00640 1 -- 172.21.15.42:0/1900314055 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f392c00a140 con 0x7f393c06c200 2024-07-04T17:29:06.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.295+0000 7f3942400640 1 -- 172.21.15.42:0/1900314055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f393c0addc0 con 0x7f393c06c200 2024-07-04T17:29:06.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.297+0000 7f3938c00640 1 -- 172.21.15.42:0/1900314055 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+10616 (secure 0 0 0) 0x7f392c01ac60 con 0x7f393c06c200 2024-07-04T17:29:06.299 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:29:06.299 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":49,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:29:05.313134+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:29:03.403851+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-07-04T17:29:04.303537+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:29:06.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.300+0000 7f3942400640 1 -- 172.21.15.42:0/1900314055 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f391c03d4f0 msgr2=0x7f391c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:06.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.300+0000 7f3942400640 1 --2- 172.21.15.42:0/1900314055 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f391c03d4f0 0x7f391c03fa00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3928034000 tx=0x7f3928004ab0 comp rx=0 tx=0).stop 2024-07-04T17:29:06.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.300+0000 7f3942400640 1 -- 172.21.15.42:0/1900314055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f393c06c200 msgr2=0x7f393c0b6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:06.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.300+0000 7f3942400640 1 --2- 172.21.15.42:0/1900314055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f393c06c200 0x7f393c0b6960 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f392c00c320 tx=0x7f392c00c7e0 comp rx=0 tx=0).stop 2024-07-04T17:29:06.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.300+0000 7f3942400640 1 -- 172.21.15.42:0/1900314055 shutdown_connections 2024-07-04T17:29:06.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.300+0000 7f3942400640 1 --2- 172.21.15.42:0/1900314055 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f391c03d4f0 0x7f391c03fa00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:06.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.300+0000 7f3942400640 1 --2- 172.21.15.42:0/1900314055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f393c06c200 0x7f393c0b6960 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:06.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.300+0000 7f3942400640 1 -- 172.21.15.42:0/1900314055 >> 172.21.15.42:0/1900314055 conn(0x7f393c0bebc0 msgr2=0x7f393c0b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:06.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.300+0000 7f3942400640 1 -- 172.21.15.42:0/1900314055 shutdown_connections 2024-07-04T17:29:06.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.300+0000 7f3942400640 1 -- 172.21.15.42:0/1900314055 wait complete. 2024-07-04T17:29:06.327 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:29:06.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.610+0000 7f1792600640 1 -- 172.21.15.42:0/2222160795 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f178c06c2c0 msgr2=0x7f178c0b5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:06.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.610+0000 7f1792600640 1 --2- 172.21.15.42:0/2222160795 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f178c06c2c0 0x7f178c0b5650 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f1778005fd0 tx=0x7f177802b670 comp rx=0 tx=0).stop 2024-07-04T17:29:06.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.610+0000 7f1792600640 1 -- 172.21.15.42:0/2222160795 shutdown_connections 2024-07-04T17:29:06.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.610+0000 7f1792600640 1 --2- 172.21.15.42:0/2222160795 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f178c06c2c0 0x7f178c0b5650 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:06.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.610+0000 7f1792600640 1 -- 172.21.15.42:0/2222160795 >> 172.21.15.42:0/2222160795 conn(0x7f178c0bebc0 msgr2=0x7f178c0befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:06.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.610+0000 7f1792600640 1 -- 172.21.15.42:0/2222160795 shutdown_connections 2024-07-04T17:29:06.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.610+0000 7f1792600640 1 -- 172.21.15.42:0/2222160795 wait complete. 2024-07-04T17:29:06.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.612+0000 7f1792600640 1 Processor -- start 2024-07-04T17:29:06.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.612+0000 7f1792600640 1 -- start start 2024-07-04T17:29:06.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.613+0000 7f1792600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f178c06c2c0 0x7f178c13af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:06.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.613+0000 7f1792600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f178c0b2810 con 0x7f178c06c2c0 2024-07-04T17:29:06.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.614+0000 7f1791200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f178c06c2c0 0x7f178c13af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:06.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.614+0000 7f1791200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f178c06c2c0 0x7f178c13af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37436/0 (socket says 172.21.15.42:37436) 2024-07-04T17:29:06.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.614+0000 7f1791200640 1 -- 172.21.15.42:0/4169370106 learned_addr learned my addr 172.21.15.42:0/4169370106 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:06.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.614+0000 7f1791200640 1 -- 172.21.15.42:0/4169370106 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f178c13e000 con 0x7f178c06c2c0 2024-07-04T17:29:06.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.614+0000 7f1791200640 1 --2- 172.21.15.42:0/4169370106 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f178c06c2c0 0x7f178c13af80 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f178000c380 tx=0x7f178000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:06.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.615+0000 7f1788c00640 1 -- 172.21.15.42:0/4169370106 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1780017070 con 0x7f178c06c2c0 2024-07-04T17:29:06.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.615+0000 7f1788c00640 1 -- 172.21.15.42:0/4169370106 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17800052f0 con 0x7f178c06c2c0 2024-07-04T17:29:06.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.615+0000 7f1788c00640 1 -- 172.21.15.42:0/4169370106 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17800035f0 con 0x7f178c06c2c0 2024-07-04T17:29:06.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.615+0000 7f1792600640 1 -- 172.21.15.42:0/4169370106 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f178c13b550 con 0x7f178c06c2c0 2024-07-04T17:29:06.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.615+0000 7f1792600640 1 -- 172.21.15.42:0/4169370106 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f178c139500 con 0x7f178c06c2c0 2024-07-04T17:29:06.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.616+0000 7f1792600640 1 -- 172.21.15.42:0/4169370106 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f178c0b5650 con 0x7f178c06c2c0 2024-07-04T17:29:06.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.616+0000 7f1788c00640 1 -- 172.21.15.42:0/4169370106 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f178001a070 con 0x7f178c06c2c0 2024-07-04T17:29:06.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.617+0000 7f1788c00640 1 --2- 172.21.15.42:0/4169370106 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f176c041900 0x7f176c043e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:06.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.617+0000 7f1788c00640 1 -- 172.21.15.42:0/4169370106 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 3439+0+0 (secure 0 0 0) 0x7f1780050b80 con 0x7f178c06c2c0 2024-07-04T17:29:06.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.618+0000 7f178be00640 1 --2- 172.21.15.42:0/4169370106 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f176c041900 0x7f176c043e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:06.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.618+0000 7f178be00640 1 --2- 172.21.15.42:0/4169370106 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f176c041900 0x7f176c043e10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1778034000 tx=0x7f1778004730 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:06.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.622+0000 7f1788c00640 1 -- 172.21.15.42:0/4169370106 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f178c06c6d0 con 0x7f178c06c2c0 2024-07-04T17:29:06.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.830+0000 7f1792600640 1 -- 172.21.15.42:0/4169370106 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f178c13e220 con 0x7f178c06c2c0 2024-07-04T17:29:06.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.832+0000 7f1788c00640 1 -- 172.21.15.42:0/4169370106 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+10603 (secure 0 0 0) 0x7f1780011630 con 0x7f178c06c2c0 2024-07-04T17:29:06.833 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:29:06.833 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":50,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:29:06.762003+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:29:03.403851+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-07-04T17:29:04.303537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:29:06.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.835+0000 7f1792600640 1 -- 172.21.15.42:0/4169370106 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f176c041900 msgr2=0x7f176c043e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:06.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.835+0000 7f1792600640 1 --2- 172.21.15.42:0/4169370106 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f176c041900 0x7f176c043e10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1778034000 tx=0x7f1778004730 comp rx=0 tx=0).stop 2024-07-04T17:29:06.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.835+0000 7f1792600640 1 -- 172.21.15.42:0/4169370106 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f178c06c2c0 msgr2=0x7f178c13af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:06.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.835+0000 7f1792600640 1 --2- 172.21.15.42:0/4169370106 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f178c06c2c0 0x7f178c13af80 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f178000c380 tx=0x7f178000c840 comp rx=0 tx=0).stop 2024-07-04T17:29:06.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.835+0000 7f1792600640 1 -- 172.21.15.42:0/4169370106 shutdown_connections 2024-07-04T17:29:06.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.835+0000 7f1792600640 1 --2- 172.21.15.42:0/4169370106 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f176c041900 0x7f176c043e10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:06.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.835+0000 7f1792600640 1 --2- 172.21.15.42:0/4169370106 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f178c06c2c0 0x7f178c13af80 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:06.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.835+0000 7f1792600640 1 -- 172.21.15.42:0/4169370106 >> 172.21.15.42:0/4169370106 conn(0x7f178c0bebc0 msgr2=0x7f178c0b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:06.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.835+0000 7f1792600640 1 -- 172.21.15.42:0/4169370106 shutdown_connections 2024-07-04T17:29:06.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:06.835+0000 7f1792600640 1 -- 172.21.15.42:0/4169370106 wait complete. 2024-07-04T17:29:06.860 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-07-04T17:29:07.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.138+0000 7ff76f200640 1 -- 172.21.15.42:0/29758035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7700ac5b0 msgr2=0x7ff7700b0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:07.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.138+0000 7ff76f200640 1 --2- 172.21.15.42:0/29758035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7700ac5b0 0x7ff7700b0db0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff75c0097b0 tx=0x7ff75c02a4a0 comp rx=0 tx=0).stop 2024-07-04T17:29:07.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.138+0000 7ff76f200640 1 -- 172.21.15.42:0/29758035 shutdown_connections 2024-07-04T17:29:07.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.138+0000 7ff76f200640 1 --2- 172.21.15.42:0/29758035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7700ac5b0 0x7ff7700b0db0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:07.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.138+0000 7ff76f200640 1 -- 172.21.15.42:0/29758035 >> 172.21.15.42:0/29758035 conn(0x7ff7700afaa0 msgr2=0x7ff7700afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:07.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.138+0000 7ff76f200640 1 -- 172.21.15.42:0/29758035 shutdown_connections 2024-07-04T17:29:07.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.138+0000 7ff76f200640 1 -- 172.21.15.42:0/29758035 wait complete. 2024-07-04T17:29:07.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.141+0000 7ff76f200640 1 Processor -- start 2024-07-04T17:29:07.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.141+0000 7ff76f200640 1 -- start start 2024-07-04T17:29:07.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.143+0000 7ff76f200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff77006da60 0x7ff7701d5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:07.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.143+0000 7ff76f200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff7700ae590 con 0x7ff77006da60 2024-07-04T17:29:07.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.143+0000 7ff76de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff77006da60 0x7ff7701d5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:07.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.144+0000 7ff76de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff77006da60 0x7ff7701d5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37460/0 (socket says 172.21.15.42:37460) 2024-07-04T17:29:07.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.144+0000 7ff76de00640 1 -- 172.21.15.42:0/2516401065 learned_addr learned my addr 172.21.15.42:0/2516401065 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:07.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.144+0000 7ff76de00640 1 -- 172.21.15.42:0/2516401065 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7701d5570 con 0x7ff77006da60 2024-07-04T17:29:07.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.144+0000 7ff76de00640 1 --2- 172.21.15.42:0/2516401065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff77006da60 0x7ff7701d5a70 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff76000c3b0 tx=0x7ff76000c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:07.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.144+0000 7ff766000640 1 -- 172.21.15.42:0/2516401065 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff760011780 con 0x7ff77006da60 2024-07-04T17:29:07.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.144+0000 7ff766000640 1 -- 172.21.15.42:0/2516401065 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff760017070 con 0x7ff77006da60 2024-07-04T17:29:07.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.144+0000 7ff766000640 1 -- 172.21.15.42:0/2516401065 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff760011c00 con 0x7ff77006da60 2024-07-04T17:29:07.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.145+0000 7ff76f200640 1 -- 172.21.15.42:0/2516401065 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff77013c0f0 con 0x7ff77006da60 2024-07-04T17:29:07.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.145+0000 7ff76f200640 1 -- 172.21.15.42:0/2516401065 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff77013ccf0 con 0x7ff77006da60 2024-07-04T17:29:07.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.146+0000 7ff76f200640 1 -- 172.21.15.42:0/2516401065 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7700b0db0 con 0x7ff77006da60 2024-07-04T17:29:07.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.146+0000 7ff766000640 1 -- 172.21.15.42:0/2516401065 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7ff76001c460 con 0x7ff77006da60 2024-07-04T17:29:07.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.147+0000 7ff766000640 1 --2- 172.21.15.42:0/2516401065 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7ff75003d4f0 0x7ff75003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:07.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.147+0000 7ff766000640 1 -- 172.21.15.42:0/2516401065 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 3439+0+0 (secure 0 0 0) 0x7ff760051600 con 0x7ff77006da60 2024-07-04T17:29:07.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.147+0000 7ff76d400640 1 --2- 172.21.15.42:0/2516401065 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7ff75003d4f0 0x7ff75003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:07.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.148+0000 7ff76d400640 1 --2- 172.21.15.42:0/2516401065 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7ff75003d4f0 0x7ff75003fa00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff75c034000 tx=0x7ff75c004730 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:07.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.152+0000 7ff766000640 1 -- 172.21.15.42:0/2516401065 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7ff77013ccf0 con 0x7ff77006da60 2024-07-04T17:29:07.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.407+0000 7ff76f200640 1 -- 172.21.15.42:0/2516401065 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff7701403b0 con 0x7ff77006da60 2024-07-04T17:29:07.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.770+0000 7ff766000640 1 -- 172.21.15.42:0/2516401065 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v35) ==== 118+0+0 (secure 0 0 0) 0x7ff760010c60 con 0x7ff77006da60 2024-07-04T17:29:07.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.774+0000 7ff76f200640 1 -- 172.21.15.42:0/2516401065 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7ff75003d4f0 msgr2=0x7ff75003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:07.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.774+0000 7ff76f200640 1 --2- 172.21.15.42:0/2516401065 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7ff75003d4f0 0x7ff75003fa00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff75c034000 tx=0x7ff75c004730 comp rx=0 tx=0).stop 2024-07-04T17:29:07.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.774+0000 7ff76f200640 1 -- 172.21.15.42:0/2516401065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff77006da60 msgr2=0x7ff7701d5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:07.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.774+0000 7ff76f200640 1 --2- 172.21.15.42:0/2516401065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff77006da60 0x7ff7701d5a70 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff76000c3b0 tx=0x7ff76000c870 comp rx=0 tx=0).stop 2024-07-04T17:29:07.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.775+0000 7ff76f200640 1 -- 172.21.15.42:0/2516401065 shutdown_connections 2024-07-04T17:29:07.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.775+0000 7ff76f200640 1 --2- 172.21.15.42:0/2516401065 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7ff75003d4f0 0x7ff75003fa00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:07.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.775+0000 7ff76f200640 1 --2- 172.21.15.42:0/2516401065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff77006da60 0x7ff7701d5a70 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:07.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.775+0000 7ff76f200640 1 -- 172.21.15.42:0/2516401065 >> 172.21.15.42:0/2516401065 conn(0x7ff7700afaa0 msgr2=0x7ff7700b2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:07.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.775+0000 7ff76f200640 1 -- 172.21.15.42:0/2516401065 shutdown_connections 2024-07-04T17:29:07.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:07.775+0000 7ff76f200640 1 -- 172.21.15.42:0/2516401065 wait complete. 2024-07-04T17:29:07.805 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-04T17:29:07.805 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-07-04T17:29:08.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.101+0000 7f97c5200640 1 -- 172.21.15.42:0/989782997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97c006d0f0 msgr2=0x7f97c00712d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:08.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.101+0000 7f97c5200640 1 --2- 172.21.15.42:0/989782997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97c006d0f0 0x7f97c00712d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f97ac0097b0 tx=0x7f97ac02a540 comp rx=0 tx=0).stop 2024-07-04T17:29:08.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.101+0000 7f97c5200640 1 -- 172.21.15.42:0/989782997 shutdown_connections 2024-07-04T17:29:08.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.101+0000 7f97c5200640 1 --2- 172.21.15.42:0/989782997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97c006d0f0 0x7f97c00712d0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:08.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.101+0000 7f97c5200640 1 -- 172.21.15.42:0/989782997 >> 172.21.15.42:0/989782997 conn(0x7f97c006c020 msgr2=0x7f97c006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:08.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.101+0000 7f97c5200640 1 -- 172.21.15.42:0/989782997 shutdown_connections 2024-07-04T17:29:08.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.101+0000 7f97c5200640 1 -- 172.21.15.42:0/989782997 wait complete. 2024-07-04T17:29:08.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.103+0000 7f97c5200640 1 Processor -- start 2024-07-04T17:29:08.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.103+0000 7f97c5200640 1 -- start start 2024-07-04T17:29:08.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.105+0000 7f97c5200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97c006d0f0 0x7f97c0137dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:08.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.105+0000 7f97c5200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f97c0141600 con 0x7f97c006d0f0 2024-07-04T17:29:08.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.105+0000 7f97bde00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97c006d0f0 0x7f97c0137dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:08.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.105+0000 7f97bde00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97c006d0f0 0x7f97c0137dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37466/0 (socket says 172.21.15.42:37466) 2024-07-04T17:29:08.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.105+0000 7f97bde00640 1 -- 172.21.15.42:0/1756037493 learned_addr learned my addr 172.21.15.42:0/1756037493 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:08.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.106+0000 7f97bde00640 1 -- 172.21.15.42:0/1756037493 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c0145240 con 0x7f97c006d0f0 2024-07-04T17:29:08.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.106+0000 7f97bde00640 1 --2- 172.21.15.42:0/1756037493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97c006d0f0 0x7f97c0137dc0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f97b000c6b0 tx=0x7f97b000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:08.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.106+0000 7f97b6000640 1 -- 172.21.15.42:0/1756037493 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97b0016020 con 0x7f97c006d0f0 2024-07-04T17:29:08.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.106+0000 7f97b6000640 1 -- 172.21.15.42:0/1756037493 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f97b000a5c0 con 0x7f97c006d0f0 2024-07-04T17:29:08.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.106+0000 7f97b6000640 1 -- 172.21.15.42:0/1756037493 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97b000a960 con 0x7f97c006d0f0 2024-07-04T17:29:08.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.106+0000 7f97c5200640 1 -- 172.21.15.42:0/1756037493 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97c01e4a40 con 0x7f97c006d0f0 2024-07-04T17:29:08.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.106+0000 7f97c5200640 1 -- 172.21.15.42:0/1756037493 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97c0136a60 con 0x7f97c006d0f0 2024-07-04T17:29:08.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.108+0000 7f97c5200640 1 -- 172.21.15.42:0/1756037493 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97c01384e0 con 0x7f97c006d0f0 2024-07-04T17:29:08.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.108+0000 7f97b6000640 1 -- 172.21.15.42:0/1756037493 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f97b0019020 con 0x7f97c006d0f0 2024-07-04T17:29:08.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.109+0000 7f97b6000640 1 --2- 172.21.15.42:0/1756037493 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f97a003d4f0 0x7f97a003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:08.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.109+0000 7f97b6000640 1 -- 172.21.15.42:0/1756037493 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 3439+0+0 (secure 0 0 0) 0x7f97b00517b0 con 0x7f97c006d0f0 2024-07-04T17:29:08.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.109+0000 7f97bd400640 1 --2- 172.21.15.42:0/1756037493 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f97a003d4f0 0x7f97a003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:08.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.110+0000 7f97bd400640 1 --2- 172.21.15.42:0/1756037493 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f97a003d4f0 0x7f97a003fa00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f97ac034000 tx=0x7f97ac004730 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:08.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.114+0000 7f97b6000640 1 -- 172.21.15.42:0/1756037493 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f97b0020020 con 0x7f97c006d0f0 2024-07-04T17:29:08.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.368+0000 7f97c5200640 1 -- 172.21.15.42:0/1756037493 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f97c0141ca0 con 0x7f97c006d0f0 2024-07-04T17:29:08.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.788+0000 7f97b6000640 1 -- 172.21.15.42:0/1756037493 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7f97c006d500 con 0x7f97c006d0f0 2024-07-04T17:29:08.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.792+0000 7f97c5200640 1 -- 172.21.15.42:0/1756037493 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f97a003d4f0 msgr2=0x7f97a003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:08.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.792+0000 7f97c5200640 1 --2- 172.21.15.42:0/1756037493 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f97a003d4f0 0x7f97a003fa00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f97ac034000 tx=0x7f97ac004730 comp rx=0 tx=0).stop 2024-07-04T17:29:08.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.792+0000 7f97c5200640 1 -- 172.21.15.42:0/1756037493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97c006d0f0 msgr2=0x7f97c0137dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:08.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.792+0000 7f97c5200640 1 --2- 172.21.15.42:0/1756037493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97c006d0f0 0x7f97c0137dc0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f97b000c6b0 tx=0x7f97b000cb70 comp rx=0 tx=0).stop 2024-07-04T17:29:08.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.792+0000 7f97c5200640 1 -- 172.21.15.42:0/1756037493 shutdown_connections 2024-07-04T17:29:08.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.792+0000 7f97c5200640 1 --2- 172.21.15.42:0/1756037493 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f97a003d4f0 0x7f97a003fa00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:08.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.792+0000 7f97c5200640 1 --2- 172.21.15.42:0/1756037493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97c006d0f0 0x7f97c0137dc0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:08.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.792+0000 7f97c5200640 1 -- 172.21.15.42:0/1756037493 >> 172.21.15.42:0/1756037493 conn(0x7f97c006c020 msgr2=0x7f97c0140770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:08.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.792+0000 7f97c5200640 1 -- 172.21.15.42:0/1756037493 shutdown_connections 2024-07-04T17:29:08.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:08.792+0000 7f97c5200640 1 -- 172.21.15.42:0/1756037493 wait complete. 2024-07-04T17:29:08.821 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-07-04T17:29:08.821 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:29:09.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.110+0000 7f7d45200640 1 -- 172.21.15.42:0/3498340360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d400bbef0 msgr2=0x7f7d401414a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:09.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.110+0000 7f7d45200640 1 --2- 172.21.15.42:0/3498340360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d400bbef0 0x7f7d401414a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7d280097b0 tx=0x7f7d2802a540 comp rx=0 tx=0).stop 2024-07-04T17:29:09.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.110+0000 7f7d45200640 1 -- 172.21.15.42:0/3498340360 shutdown_connections 2024-07-04T17:29:09.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.110+0000 7f7d45200640 1 --2- 172.21.15.42:0/3498340360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d400bbef0 0x7f7d401414a0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:09.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.110+0000 7f7d45200640 1 -- 172.21.15.42:0/3498340360 >> 172.21.15.42:0/3498340360 conn(0x7f7d400688a0 msgr2=0x7f7d4006db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:09.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.110+0000 7f7d45200640 1 -- 172.21.15.42:0/3498340360 shutdown_connections 2024-07-04T17:29:09.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.110+0000 7f7d45200640 1 -- 172.21.15.42:0/3498340360 wait complete. 2024-07-04T17:29:09.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.112+0000 7f7d45200640 1 Processor -- start 2024-07-04T17:29:09.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.112+0000 7f7d45200640 1 -- start start 2024-07-04T17:29:09.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.114+0000 7f7d45200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d400bbef0 0x7f7d400bee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:09.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.114+0000 7f7d45200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7d400b79a0 con 0x7f7d400bbef0 2024-07-04T17:29:09.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.114+0000 7f7d3f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d400bbef0 0x7f7d400bee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:09.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.114+0000 7f7d3f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d400bbef0 0x7f7d400bee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37474/0 (socket says 172.21.15.42:37474) 2024-07-04T17:29:09.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.114+0000 7f7d3f400640 1 -- 172.21.15.42:0/3385403336 learned_addr learned my addr 172.21.15.42:0/3385403336 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:09.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.114+0000 7f7d3f400640 1 -- 172.21.15.42:0/3385403336 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d401dd2a0 con 0x7f7d400bbef0 2024-07-04T17:29:09.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.115+0000 7f7d3f400640 1 --2- 172.21.15.42:0/3385403336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d400bbef0 0x7f7d400bee00 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7d280098d0 tx=0x7f7d28033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:09.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.115+0000 7f7d37400640 1 -- 172.21.15.42:0/3385403336 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d28031bc0 con 0x7f7d400bbef0 2024-07-04T17:29:09.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.115+0000 7f7d37400640 1 -- 172.21.15.42:0/3385403336 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d28031e50 con 0x7f7d400bbef0 2024-07-04T17:29:09.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.115+0000 7f7d37400640 1 -- 172.21.15.42:0/3385403336 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d28033260 con 0x7f7d400bbef0 2024-07-04T17:29:09.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.115+0000 7f7d45200640 1 -- 172.21.15.42:0/3385403336 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d4013f350 con 0x7f7d400bbef0 2024-07-04T17:29:09.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.115+0000 7f7d45200640 1 -- 172.21.15.42:0/3385403336 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d40145280 con 0x7f7d400bbef0 2024-07-04T17:29:09.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.116+0000 7f7d45200640 1 -- 172.21.15.42:0/3385403336 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d400b8020 con 0x7f7d400bbef0 2024-07-04T17:29:09.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.117+0000 7f7d37400640 1 -- 172.21.15.42:0/3385403336 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f7d28033400 con 0x7f7d400bbef0 2024-07-04T17:29:09.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.117+0000 7f7d37400640 1 --2- 172.21.15.42:0/3385403336 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f7d1c03d4f0 0x7f7d1c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:09.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.117+0000 7f7d37400640 1 -- 172.21.15.42:0/3385403336 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 3439+0+0 (secure 0 0 0) 0x7f7d28072eb0 con 0x7f7d400bbef0 2024-07-04T17:29:09.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.118+0000 7f7d3ea00640 1 --2- 172.21.15.42:0/3385403336 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f7d1c03d4f0 0x7f7d1c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:09.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.118+0000 7f7d3ea00640 1 --2- 172.21.15.42:0/3385403336 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f7d1c03d4f0 0x7f7d1c03fa00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7d4006db00 tx=0x7f7d30001650 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:09.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.122+0000 7f7d37400640 1 -- 172.21.15.42:0/3385403336 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f7d401dd2a0 con 0x7f7d400bbef0 2024-07-04T17:29:09.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.371+0000 7f7d45200640 1 -- 172.21.15.42:0/3385403336 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d400c01a0 con 0x7f7d400bbef0 2024-07-04T17:29:09.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.372+0000 7f7d37400640 1 -- 172.21.15.42:0/3385403336 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+6045 (secure 0 0 0) 0x7f7d2803f030 con 0x7f7d400bbef0 2024-07-04T17:29:09.374 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:29:09.374 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":36,"btime":"2024-07-04T17:29:08:780650+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/343791977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":343791977},{"type":"v1","addr":"172.21.15.42:6827","nonce":343791977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":4722,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/3172446281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":3172446281},{"type":"v1","addr":"172.21.15.42:6829","nonce":3172446281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":4727,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/1866310217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1866310217},{"type":"v1","addr":"172.21.15.42:6831","nonce":1866310217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":4728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6833/1072723036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1072723036},{"type":"v1","addr":"172.21.15.42:6833","nonce":1072723036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:29:05.313061+0000","modified":"2024-07-04T17:29:08.370956+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4731},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4731":{"gid":4731,"name":"d","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6835/1327681666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1327681666},{"type":"v1","addr":"172.21.15.42:6835","nonce":1327681666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4731,"qdb_cluster":[4731]},"id":4}]} 2024-07-04T17:29:09.374 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 36 2024-07-04T17:29:09.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.375+0000 7f7d45200640 1 -- 172.21.15.42:0/3385403336 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f7d1c03d4f0 msgr2=0x7f7d1c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:09.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.375+0000 7f7d45200640 1 --2- 172.21.15.42:0/3385403336 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f7d1c03d4f0 0x7f7d1c03fa00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7d4006db00 tx=0x7f7d30001650 comp rx=0 tx=0).stop 2024-07-04T17:29:09.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.375+0000 7f7d45200640 1 -- 172.21.15.42:0/3385403336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d400bbef0 msgr2=0x7f7d400bee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:09.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.375+0000 7f7d45200640 1 --2- 172.21.15.42:0/3385403336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d400bbef0 0x7f7d400bee00 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7d280098d0 tx=0x7f7d28033e40 comp rx=0 tx=0).stop 2024-07-04T17:29:09.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.375+0000 7f7d45200640 1 -- 172.21.15.42:0/3385403336 shutdown_connections 2024-07-04T17:29:09.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.375+0000 7f7d45200640 1 --2- 172.21.15.42:0/3385403336 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f7d1c03d4f0 0x7f7d1c03fa00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:09.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.375+0000 7f7d45200640 1 --2- 172.21.15.42:0/3385403336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d400bbef0 0x7f7d400bee00 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:09.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.375+0000 7f7d45200640 1 -- 172.21.15.42:0/3385403336 >> 172.21.15.42:0/3385403336 conn(0x7f7d400688a0 msgr2=0x7f7d400b6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:09.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.375+0000 7f7d45200640 1 -- 172.21.15.42:0/3385403336 shutdown_connections 2024-07-04T17:29:09.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.375+0000 7f7d45200640 1 -- 172.21.15.42:0/3385403336 wait complete. 2024-07-04T17:29:09.402 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:29:09.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.677+0000 7f2ed9800640 1 -- 172.21.15.42:0/3613436481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ed406c020 msgr2=0x7f2ed414a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:09.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.677+0000 7f2ed9800640 1 --2- 172.21.15.42:0/3613436481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ed406c020 0x7f2ed414a9e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc0097b0 tx=0x7f2ebc02a540 comp rx=0 tx=0).stop 2024-07-04T17:29:09.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.677+0000 7f2ed9800640 1 -- 172.21.15.42:0/3613436481 shutdown_connections 2024-07-04T17:29:09.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.677+0000 7f2ed9800640 1 --2- 172.21.15.42:0/3613436481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ed406c020 0x7f2ed414a9e0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:09.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.677+0000 7f2ed9800640 1 -- 172.21.15.42:0/3613436481 >> 172.21.15.42:0/3613436481 conn(0x7f2ed40bebc0 msgr2=0x7f2ed40befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:09.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.677+0000 7f2ed9800640 1 -- 172.21.15.42:0/3613436481 shutdown_connections 2024-07-04T17:29:09.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.677+0000 7f2ed9800640 1 -- 172.21.15.42:0/3613436481 wait complete. 2024-07-04T17:29:09.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.679+0000 7f2ed9800640 1 Processor -- start 2024-07-04T17:29:09.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.679+0000 7f2ed9800640 1 -- start start 2024-07-04T17:29:09.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.680+0000 7f2ed9800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ed406c020 0x7f2ed41d5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:09.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.680+0000 7f2ed9800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed414b550 con 0x7f2ed406c020 2024-07-04T17:29:09.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.681+0000 7f2ed3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ed406c020 0x7f2ed41d5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:09.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.681+0000 7f2ed3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ed406c020 0x7f2ed41d5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37494/0 (socket says 172.21.15.42:37494) 2024-07-04T17:29:09.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.681+0000 7f2ed3e00640 1 -- 172.21.15.42:0/2195682843 learned_addr learned my addr 172.21.15.42:0/2195682843 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:09.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.681+0000 7f2ed3e00640 1 -- 172.21.15.42:0/2195682843 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed41d8fc0 con 0x7f2ed406c020 2024-07-04T17:29:09.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.682+0000 7f2ed3e00640 1 --2- 172.21.15.42:0/2195682843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ed406c020 0x7f2ed41d5a60 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4001eb0 tx=0x7f2ec400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:09.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.682+0000 7f2ed0c00640 1 -- 172.21.15.42:0/2195682843 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ec40116c0 con 0x7f2ed406c020 2024-07-04T17:29:09.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.682+0000 7f2ed0c00640 1 -- 172.21.15.42:0/2195682843 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ec4017070 con 0x7f2ed406c020 2024-07-04T17:29:09.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.682+0000 7f2ed0c00640 1 -- 172.21.15.42:0/2195682843 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ec4011ae0 con 0x7f2ed406c020 2024-07-04T17:29:09.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.682+0000 7f2ed9800640 1 -- 172.21.15.42:0/2195682843 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed40b1070 con 0x7f2ed406c020 2024-07-04T17:29:09.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.682+0000 7f2ed9800640 1 -- 172.21.15.42:0/2195682843 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed41d7c80 con 0x7f2ed406c020 2024-07-04T17:29:09.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.683+0000 7f2ed9800640 1 -- 172.21.15.42:0/2195682843 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ed414bc80 con 0x7f2ed406c020 2024-07-04T17:29:09.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.683+0000 7f2ed0c00640 1 -- 172.21.15.42:0/2195682843 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f2ec4011c80 con 0x7f2ed406c020 2024-07-04T17:29:09.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.684+0000 7f2ed0c00640 1 --2- 172.21.15.42:0/2195682843 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f2eb003d4f0 0x7f2eb003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:09.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.684+0000 7f2ed0c00640 1 -- 172.21.15.42:0/2195682843 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 3439+0+0 (secure 0 0 0) 0x7f2ec40515b0 con 0x7f2ed406c020 2024-07-04T17:29:09.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.684+0000 7f2ecbe00640 1 --2- 172.21.15.42:0/2195682843 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f2eb003d4f0 0x7f2eb003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:09.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.685+0000 7f2ecbe00640 1 --2- 172.21.15.42:0/2195682843 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f2eb003d4f0 0x7f2eb003fa00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc034000 tx=0x7f2ebc004730 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:09.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.689+0000 7f2ed0c00640 1 -- 172.21.15.42:0/2195682843 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f2ec400a140 con 0x7f2ed406c020 2024-07-04T17:29:09.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.902+0000 7f2ed9800640 1 -- 172.21.15.42:0/2195682843 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ed414f540 con 0x7f2ed406c020 2024-07-04T17:29:09.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.905+0000 7f2ed0c00640 1 -- 172.21.15.42:0/2195682843 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+10603 (secure 0 0 0) 0x7f2ec4010cf0 con 0x7f2ed406c020 2024-07-04T17:29:09.907 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:29:09.907 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":50,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:29:06.762003+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:29:03.403851+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-07-04T17:29:04.303537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:29:09.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.908+0000 7f2ed9800640 1 -- 172.21.15.42:0/2195682843 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f2eb003d4f0 msgr2=0x7f2eb003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:09.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.908+0000 7f2ed9800640 1 --2- 172.21.15.42:0/2195682843 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f2eb003d4f0 0x7f2eb003fa00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc034000 tx=0x7f2ebc004730 comp rx=0 tx=0).stop 2024-07-04T17:29:09.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.908+0000 7f2ed9800640 1 -- 172.21.15.42:0/2195682843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ed406c020 msgr2=0x7f2ed41d5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:09.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.908+0000 7f2ed9800640 1 --2- 172.21.15.42:0/2195682843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ed406c020 0x7f2ed41d5a60 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4001eb0 tx=0x7f2ec400c810 comp rx=0 tx=0).stop 2024-07-04T17:29:09.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.908+0000 7f2ed9800640 1 -- 172.21.15.42:0/2195682843 shutdown_connections 2024-07-04T17:29:09.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.908+0000 7f2ed9800640 1 --2- 172.21.15.42:0/2195682843 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f2eb003d4f0 0x7f2eb003fa00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:09.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.908+0000 7f2ed9800640 1 --2- 172.21.15.42:0/2195682843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ed406c020 0x7f2ed41d5a60 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:09.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.908+0000 7f2ed9800640 1 -- 172.21.15.42:0/2195682843 >> 172.21.15.42:0/2195682843 conn(0x7f2ed40bebc0 msgr2=0x7f2ed40b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:09.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.908+0000 7f2ed9800640 1 -- 172.21.15.42:0/2195682843 shutdown_connections 2024-07-04T17:29:09.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:09.908+0000 7f2ed9800640 1 -- 172.21.15.42:0/2195682843 wait complete. 2024-07-04T17:29:09.934 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-04T17:29:09.934 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:29:10.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.222+0000 7f4d8e000640 1 -- 172.21.15.42:0/275989262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d880b17d0 msgr2=0x7f4d880b1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:10.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.222+0000 7f4d8e000640 1 --2- 172.21.15.42:0/275989262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d880b17d0 0x7f4d880b1be0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f4d740097b0 tx=0x7f4d7402a4b0 comp rx=0 tx=0).stop 2024-07-04T17:29:10.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.222+0000 7f4d8e000640 1 -- 172.21.15.42:0/275989262 shutdown_connections 2024-07-04T17:29:10.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.222+0000 7f4d8e000640 1 --2- 172.21.15.42:0/275989262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d880b17d0 0x7f4d880b1be0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:10.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.222+0000 7f4d8e000640 1 -- 172.21.15.42:0/275989262 >> 172.21.15.42:0/275989262 conn(0x7f4d880bebc0 msgr2=0x7f4d880befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:10.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.222+0000 7f4d8e000640 1 -- 172.21.15.42:0/275989262 shutdown_connections 2024-07-04T17:29:10.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.222+0000 7f4d8e000640 1 -- 172.21.15.42:0/275989262 wait complete. 2024-07-04T17:29:10.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.224+0000 7f4d8e000640 1 Processor -- start 2024-07-04T17:29:10.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.224+0000 7f4d8e000640 1 -- start start 2024-07-04T17:29:10.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.226+0000 7f4d8e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d8806c200 0x7f4d880b6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:10.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.226+0000 7f4d8e000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8814ae80 con 0x7f4d8806c200 2024-07-04T17:29:10.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.226+0000 7f4d8cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d8806c200 0x7f4d880b6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:10.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.226+0000 7f4d8cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d8806c200 0x7f4d880b6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37514/0 (socket says 172.21.15.42:37514) 2024-07-04T17:29:10.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.226+0000 7f4d8cc00640 1 -- 172.21.15.42:0/983231467 learned_addr learned my addr 172.21.15.42:0/983231467 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:10.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.226+0000 7f4d8cc00640 1 -- 172.21.15.42:0/983231467 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d88147fd0 con 0x7f4d8806c200 2024-07-04T17:29:10.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.227+0000 7f4d8cc00640 1 --2- 172.21.15.42:0/983231467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d8806c200 0x7f4d880b6b90 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c00c320 tx=0x7f4d7c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:10.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.227+0000 7f4d84c00640 1 -- 172.21.15.42:0/983231467 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d7c0116c0 con 0x7f4d8806c200 2024-07-04T17:29:10.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.227+0000 7f4d84c00640 1 -- 172.21.15.42:0/983231467 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d7c017070 con 0x7f4d8806c200 2024-07-04T17:29:10.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.227+0000 7f4d84c00640 1 -- 172.21.15.42:0/983231467 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d7c011b40 con 0x7f4d8806c200 2024-07-04T17:29:10.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.227+0000 7f4d8e000640 1 -- 172.21.15.42:0/983231467 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d880b70d0 con 0x7f4d8806c200 2024-07-04T17:29:10.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.227+0000 7f4d8e000640 1 -- 172.21.15.42:0/983231467 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d88147330 con 0x7f4d8806c200 2024-07-04T17:29:10.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.228+0000 7f4d8e000640 1 -- 172.21.15.42:0/983231467 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d880b44b0 con 0x7f4d8806c200 2024-07-04T17:29:10.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.228+0000 7f4d84c00640 1 -- 172.21.15.42:0/983231467 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f4d7c011ce0 con 0x7f4d8806c200 2024-07-04T17:29:10.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.229+0000 7f4d84c00640 1 --2- 172.21.15.42:0/983231467 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f4d6803d4f0 0x7f4d6803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:10.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.229+0000 7f4d84c00640 1 -- 172.21.15.42:0/983231467 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 3439+0+0 (secure 0 0 0) 0x7f4d7c051510 con 0x7f4d8806c200 2024-07-04T17:29:10.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.229+0000 7f4d87e00640 1 --2- 172.21.15.42:0/983231467 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f4d6803d4f0 0x7f4d6803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:10.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.230+0000 7f4d87e00640 1 --2- 172.21.15.42:0/983231467 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f4d6803d4f0 0x7f4d6803fa00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4d74034000 tx=0x7f4d74004730 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:10.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.234+0000 7f4d84c00640 1 -- 172.21.15.42:0/983231467 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f4d880b44b0 con 0x7f4d8806c200 2024-07-04T17:29:10.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.443+0000 7f4d8e000640 1 -- 172.21.15.42:0/983231467 --> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4d881481a0 con 0x7f4d6803d4f0 2024-07-04T17:29:10.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.451+0000 7f4d84c00640 1 -- 172.21.15.42:0/983231467 <== mgr.4613 v2:172.21.15.42:6824/2638271195 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245972 (secure 0 0 0) 0x7f4d881481a0 con 0x7f4d6803d4f0 2024-07-04T17:29:10.455 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:29:10.457 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:29:10.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.457+0000 7f4d8e000640 1 -- 172.21.15.42:0/983231467 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f4d6803d4f0 msgr2=0x7f4d6803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:10.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.457+0000 7f4d8e000640 1 --2- 172.21.15.42:0/983231467 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f4d6803d4f0 0x7f4d6803fa00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4d74034000 tx=0x7f4d74004730 comp rx=0 tx=0).stop 2024-07-04T17:29:10.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.457+0000 7f4d8e000640 1 -- 172.21.15.42:0/983231467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d8806c200 msgr2=0x7f4d880b6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:10.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.457+0000 7f4d8e000640 1 --2- 172.21.15.42:0/983231467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d8806c200 0x7f4d880b6b90 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c00c320 tx=0x7f4d7c00c7e0 comp rx=0 tx=0).stop 2024-07-04T17:29:10.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.457+0000 7f4d8e000640 1 -- 172.21.15.42:0/983231467 shutdown_connections 2024-07-04T17:29:10.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.457+0000 7f4d8e000640 1 --2- 172.21.15.42:0/983231467 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f4d6803d4f0 0x7f4d6803fa00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:10.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.457+0000 7f4d8e000640 1 --2- 172.21.15.42:0/983231467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d8806c200 0x7f4d880b6b90 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:10.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.457+0000 7f4d8e000640 1 -- 172.21.15.42:0/983231467 >> 172.21.15.42:0/983231467 conn(0x7f4d880bebc0 msgr2=0x7f4d881d72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:10.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.458+0000 7f4d8e000640 1 -- 172.21.15.42:0/983231467 shutdown_connections 2024-07-04T17:29:10.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.458+0000 7f4d8e000640 1 -- 172.21.15.42:0/983231467 wait complete. 2024-07-04T17:29:10.487 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-07-04T17:29:09.651909+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":176,"num_read_kb":398,"num_write":148,"num_write_kb":1120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":115,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":96188,"kb_used_data":2632,"kb_used_omap":4,"kb_used_meta":93499,"kb_avail":283019332,"statfs":{"total":289910292480,"available":289811795968,"internally_reserved":0,"allocated":2695168,"data_stored":1733448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":95743311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.919578"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.789264+0000","last_change":"2024-07-04T17:29:05.832523+0000","last_active":"2024-07-04T17:29:06.789264+0000","last_peered":"2024-07-04T17:29:06.789264+0000","last_clean":"2024-07-04T17:29:06.789264+0000","last_became_active":"2024-07-04T17:29:05.831208+0000","last_became_peered":"2024-07-04T17:29:05.831208+0000","last_unstale":"2024-07-04T17:29:06.789264+0000","last_undegraded":"2024-07-04T17:29:06.789264+0000","last_fullsized":"2024-07-04T17:29:06.789264+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:52:48.071452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.788398+0000","last_change":"2024-07-04T17:29:05.826928+0000","last_active":"2024-07-04T17:29:06.788398+0000","last_peered":"2024-07-04T17:29:06.788398+0000","last_clean":"2024-07-04T17:29:06.788398+0000","last_became_active":"2024-07-04T17:29:05.826320+0000","last_became_peered":"2024-07-04T17:29:05.826320+0000","last_unstale":"2024-07-04T17:29:06.788398+0000","last_undegraded":"2024-07-04T17:29:06.788398+0000","last_fullsized":"2024-07-04T17:29:06.788398+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:06:24.983072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.788359+0000","last_change":"2024-07-04T17:29:05.825968+0000","last_active":"2024-07-04T17:29:06.788359+0000","last_peered":"2024-07-04T17:29:06.788359+0000","last_clean":"2024-07-04T17:29:06.788359+0000","last_became_active":"2024-07-04T17:29:05.825203+0000","last_became_peered":"2024-07-04T17:29:05.825203+0000","last_unstale":"2024-07-04T17:29:06.788359+0000","last_undegraded":"2024-07-04T17:29:06.788359+0000","last_fullsized":"2024-07-04T17:29:06.788359+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:19:50.013783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.779269+0000","last_change":"2024-07-04T17:29:05.783206+0000","last_active":"2024-07-04T17:29:06.779269+0000","last_peered":"2024-07-04T17:29:06.779269+0000","last_clean":"2024-07-04T17:29:06.779269+0000","last_became_active":"2024-07-04T17:29:05.782870+0000","last_became_peered":"2024-07-04T17:29:05.782870+0000","last_unstale":"2024-07-04T17:29:06.779269+0000","last_undegraded":"2024-07-04T17:29:06.779269+0000","last_fullsized":"2024-07-04T17:29:06.779269+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:25:13.149991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.786048+0000","last_change":"2024-07-04T17:29:05.810625+0000","last_active":"2024-07-04T17:29:06.786048+0000","last_peered":"2024-07-04T17:29:06.786048+0000","last_clean":"2024-07-04T17:29:06.786048+0000","last_became_active":"2024-07-04T17:29:05.810316+0000","last_became_peered":"2024-07-04T17:29:05.810316+0000","last_unstale":"2024-07-04T17:29:06.786048+0000","last_undegraded":"2024-07-04T17:29:06.786048+0000","last_fullsized":"2024-07-04T17:29:06.786048+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:48:26.486742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.778745+0000","last_change":"2024-07-04T17:29:05.799666+0000","last_active":"2024-07-04T17:29:06.778745+0000","last_peered":"2024-07-04T17:29:06.778745+0000","last_clean":"2024-07-04T17:29:06.778745+0000","last_became_active":"2024-07-04T17:29:05.788794+0000","last_became_peered":"2024-07-04T17:29:05.788794+0000","last_unstale":"2024-07-04T17:29:06.778745+0000","last_undegraded":"2024-07-04T17:29:06.778745+0000","last_fullsized":"2024-07-04T17:29:06.778745+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:33:48.215079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.777572+0000","last_change":"2024-07-04T17:29:05.803716+0000","last_active":"2024-07-04T17:29:06.777572+0000","last_peered":"2024-07-04T17:29:06.777572+0000","last_clean":"2024-07-04T17:29:06.777572+0000","last_became_active":"2024-07-04T17:29:05.803425+0000","last_became_peered":"2024-07-04T17:29:05.803425+0000","last_unstale":"2024-07-04T17:29:06.777572+0000","last_undegraded":"2024-07-04T17:29:06.777572+0000","last_fullsized":"2024-07-04T17:29:06.777572+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:51:34.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.785750+0000","last_change":"2024-07-04T17:29:05.802356+0000","last_active":"2024-07-04T17:29:06.785750+0000","last_peered":"2024-07-04T17:29:06.785750+0000","last_clean":"2024-07-04T17:29:06.785750+0000","last_became_active":"2024-07-04T17:29:05.798614+0000","last_became_peered":"2024-07-04T17:29:05.798614+0000","last_unstale":"2024-07-04T17:29:06.785750+0000","last_undegraded":"2024-07-04T17:29:06.785750+0000","last_fullsized":"2024-07-04T17:29:06.785750+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:33:04.621287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.496012+0000","last_change":"2024-07-04T17:29:05.804142+0000","last_active":"2024-07-04T17:29:08.496012+0000","last_peered":"2024-07-04T17:29:08.496012+0000","last_clean":"2024-07-04T17:29:08.496012+0000","last_became_active":"2024-07-04T17:29:05.799194+0000","last_became_peered":"2024-07-04T17:29:05.799194+0000","last_unstale":"2024-07-04T17:29:08.496012+0000","last_undegraded":"2024-07-04T17:29:08.496012+0000","last_fullsized":"2024-07-04T17:29:08.496012+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:44:51.271924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.495860+0000","last_change":"2024-07-04T17:29:05.815840+0000","last_active":"2024-07-04T17:29:08.495860+0000","last_peered":"2024-07-04T17:29:08.495860+0000","last_clean":"2024-07-04T17:29:08.495860+0000","last_became_active":"2024-07-04T17:29:05.810912+0000","last_became_peered":"2024-07-04T17:29:05.810912+0000","last_unstale":"2024-07-04T17:29:08.495860+0000","last_undegraded":"2024-07-04T17:29:08.495860+0000","last_fullsized":"2024-07-04T17:29:08.495860+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:31:51.703656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.495566+0000","last_change":"2024-07-04T17:29:05.785152+0000","last_active":"2024-07-04T17:29:08.495566+0000","last_peered":"2024-07-04T17:29:08.495566+0000","last_clean":"2024-07-04T17:29:08.495566+0000","last_became_active":"2024-07-04T17:29:05.784449+0000","last_became_peered":"2024-07-04T17:29:05.784449+0000","last_unstale":"2024-07-04T17:29:08.495566+0000","last_undegraded":"2024-07-04T17:29:08.495566+0000","last_fullsized":"2024-07-04T17:29:08.495566+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:35:30.393302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.493910+0000","last_change":"2024-07-04T17:29:05.823239+0000","last_active":"2024-07-04T17:29:08.493910+0000","last_peered":"2024-07-04T17:29:08.493910+0000","last_clean":"2024-07-04T17:29:08.493910+0000","last_became_active":"2024-07-04T17:29:05.814164+0000","last_became_peered":"2024-07-04T17:29:05.814164+0000","last_unstale":"2024-07-04T17:29:08.493910+0000","last_undegraded":"2024-07-04T17:29:08.493910+0000","last_fullsized":"2024-07-04T17:29:08.493910+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:03:20.072994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.778454+0000","last_change":"2024-07-04T17:29:05.796770+0000","last_active":"2024-07-04T17:29:06.778454+0000","last_peered":"2024-07-04T17:29:06.778454+0000","last_clean":"2024-07-04T17:29:06.778454+0000","last_became_active":"2024-07-04T17:29:05.789644+0000","last_became_peered":"2024-07-04T17:29:05.789644+0000","last_unstale":"2024-07-04T17:29:06.778454+0000","last_undegraded":"2024-07-04T17:29:06.778454+0000","last_fullsized":"2024-07-04T17:29:06.778454+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:58:03.269316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.786860+0000","last_change":"2024-07-04T17:29:05.808774+0000","last_active":"2024-07-04T17:29:06.786860+0000","last_peered":"2024-07-04T17:29:06.786860+0000","last_clean":"2024-07-04T17:29:06.786860+0000","last_became_active":"2024-07-04T17:29:05.808459+0000","last_became_peered":"2024-07-04T17:29:05.808459+0000","last_unstale":"2024-07-04T17:29:06.786860+0000","last_undegraded":"2024-07-04T17:29:06.786860+0000","last_fullsized":"2024-07-04T17:29:06.786860+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:13:24.537766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.779503+0000","last_change":"2024-07-04T17:29:05.800412+0000","last_active":"2024-07-04T17:29:06.779503+0000","last_peered":"2024-07-04T17:29:06.779503+0000","last_clean":"2024-07-04T17:29:06.779503+0000","last_became_active":"2024-07-04T17:29:05.799152+0000","last_became_peered":"2024-07-04T17:29:05.799152+0000","last_unstale":"2024-07-04T17:29:06.779503+0000","last_undegraded":"2024-07-04T17:29:06.779503+0000","last_fullsized":"2024-07-04T17:29:06.779503+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:48:07.191680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.776833+0000","last_change":"2024-07-04T17:29:05.795773+0000","last_active":"2024-07-04T17:29:06.776833+0000","last_peered":"2024-07-04T17:29:06.776833+0000","last_clean":"2024-07-04T17:29:06.776833+0000","last_became_active":"2024-07-04T17:29:05.795470+0000","last_became_peered":"2024-07-04T17:29:05.795470+0000","last_unstale":"2024-07-04T17:29:06.776833+0000","last_undegraded":"2024-07-04T17:29:06.776833+0000","last_fullsized":"2024-07-04T17:29:06.776833+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:32:34.271916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.789186+0000","last_change":"2024-07-04T17:29:05.820276+0000","last_active":"2024-07-04T17:29:06.789186+0000","last_peered":"2024-07-04T17:29:06.789186+0000","last_clean":"2024-07-04T17:29:06.789186+0000","last_became_active":"2024-07-04T17:29:05.819959+0000","last_became_peered":"2024-07-04T17:29:05.819959+0000","last_unstale":"2024-07-04T17:29:06.789186+0000","last_undegraded":"2024-07-04T17:29:06.789186+0000","last_fullsized":"2024-07-04T17:29:06.789186+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:50:43.333430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.788642+0000","last_change":"2024-07-04T17:29:05.817114+0000","last_active":"2024-07-04T17:29:06.788642+0000","last_peered":"2024-07-04T17:29:06.788642+0000","last_clean":"2024-07-04T17:29:06.788642+0000","last_became_active":"2024-07-04T17:29:05.816724+0000","last_became_peered":"2024-07-04T17:29:05.816724+0000","last_unstale":"2024-07-04T17:29:06.788642+0000","last_undegraded":"2024-07-04T17:29:06.788642+0000","last_fullsized":"2024-07-04T17:29:06.788642+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:44:51.271924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.788642+0000","last_change":"2024-07-04T17:29:05.816646+0000","last_active":"2024-07-04T17:29:06.788642+0000","last_peered":"2024-07-04T17:29:06.788642+0000","last_clean":"2024-07-04T17:29:06.788642+0000","last_became_active":"2024-07-04T17:29:05.816345+0000","last_became_peered":"2024-07-04T17:29:05.816345+0000","last_unstale":"2024-07-04T17:29:06.788642+0000","last_undegraded":"2024-07-04T17:29:06.788642+0000","last_fullsized":"2024-07-04T17:29:06.788642+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:31:51.703656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.494347+0000","last_change":"2024-07-04T17:29:05.792444+0000","last_active":"2024-07-04T17:29:08.494347+0000","last_peered":"2024-07-04T17:29:08.494347+0000","last_clean":"2024-07-04T17:29:08.494347+0000","last_became_active":"2024-07-04T17:29:05.792058+0000","last_became_peered":"2024-07-04T17:29:05.792058+0000","last_unstale":"2024-07-04T17:29:08.494347+0000","last_undegraded":"2024-07-04T17:29:08.494347+0000","last_fullsized":"2024-07-04T17:29:08.494347+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:13:24.537766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.778960+0000","last_change":"2024-07-04T17:29:05.799399+0000","last_active":"2024-07-04T17:29:06.778960+0000","last_peered":"2024-07-04T17:29:06.778960+0000","last_clean":"2024-07-04T17:29:06.778960+0000","last_became_active":"2024-07-04T17:29:05.789221+0000","last_became_peered":"2024-07-04T17:29:05.789221+0000","last_unstale":"2024-07-04T17:29:06.778960+0000","last_undegraded":"2024-07-04T17:29:06.778960+0000","last_fullsized":"2024-07-04T17:29:06.778960+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:23:53.530646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.787390+0000","last_change":"2024-07-04T17:29:05.813751+0000","last_active":"2024-07-04T17:29:06.787390+0000","last_peered":"2024-07-04T17:29:06.787390+0000","last_clean":"2024-07-04T17:29:06.787390+0000","last_became_active":"2024-07-04T17:29:05.813362+0000","last_became_peered":"2024-07-04T17:29:05.813362+0000","last_unstale":"2024-07-04T17:29:06.787390+0000","last_undegraded":"2024-07-04T17:29:06.787390+0000","last_fullsized":"2024-07-04T17:29:06.787390+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:58:35.184348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.493677+0000","last_change":"2024-07-04T17:29:05.809788+0000","last_active":"2024-07-04T17:29:08.493677+0000","last_peered":"2024-07-04T17:29:08.493677+0000","last_clean":"2024-07-04T17:29:08.493677+0000","last_became_active":"2024-07-04T17:29:05.809201+0000","last_became_peered":"2024-07-04T17:29:05.809201+0000","last_unstale":"2024-07-04T17:29:08.493677+0000","last_undegraded":"2024-07-04T17:29:08.493677+0000","last_fullsized":"2024-07-04T17:29:08.493677+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:48:26.486742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.493137+0000","last_change":"2024-07-04T17:29:05.820075+0000","last_active":"2024-07-04T17:29:08.493137+0000","last_peered":"2024-07-04T17:29:08.493137+0000","last_clean":"2024-07-04T17:29:08.493137+0000","last_became_active":"2024-07-04T17:29:05.819738+0000","last_became_peered":"2024-07-04T17:29:05.819738+0000","last_unstale":"2024-07-04T17:29:08.493137+0000","last_undegraded":"2024-07-04T17:29:08.493137+0000","last_fullsized":"2024-07-04T17:29:08.493137+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:47:41.453377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.789805+0000","last_change":"2024-07-04T17:29:05.832254+0000","last_active":"2024-07-04T17:29:06.789805+0000","last_peered":"2024-07-04T17:29:06.789805+0000","last_clean":"2024-07-04T17:29:06.789805+0000","last_became_active":"2024-07-04T17:29:05.830771+0000","last_became_peered":"2024-07-04T17:29:05.830771+0000","last_unstale":"2024-07-04T17:29:06.789805+0000","last_undegraded":"2024-07-04T17:29:06.789805+0000","last_fullsized":"2024-07-04T17:29:06.789805+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:01:45.227217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.779110+0000","last_change":"2024-07-04T17:29:05.787671+0000","last_active":"2024-07-04T17:29:06.779110+0000","last_peered":"2024-07-04T17:29:06.779110+0000","last_clean":"2024-07-04T17:29:06.779110+0000","last_became_active":"2024-07-04T17:29:05.783612+0000","last_became_peered":"2024-07-04T17:29:05.783612+0000","last_unstale":"2024-07-04T17:29:06.779110+0000","last_undegraded":"2024-07-04T17:29:06.779110+0000","last_fullsized":"2024-07-04T17:29:06.779110+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:17:31.565173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.777973+0000","last_change":"2024-07-04T17:29:05.785801+0000","last_active":"2024-07-04T17:29:06.777973+0000","last_peered":"2024-07-04T17:29:06.777973+0000","last_clean":"2024-07-04T17:29:06.777973+0000","last_became_active":"2024-07-04T17:29:05.784924+0000","last_became_peered":"2024-07-04T17:29:05.784924+0000","last_unstale":"2024-07-04T17:29:06.777973+0000","last_undegraded":"2024-07-04T17:29:06.777973+0000","last_fullsized":"2024-07-04T17:29:06.777973+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:52:10.436668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.786933+0000","last_change":"2024-07-04T17:29:05.817096+0000","last_active":"2024-07-04T17:29:06.786933+0000","last_peered":"2024-07-04T17:29:06.786933+0000","last_clean":"2024-07-04T17:29:06.786933+0000","last_became_active":"2024-07-04T17:29:05.816304+0000","last_became_peered":"2024-07-04T17:29:05.816304+0000","last_unstale":"2024-07-04T17:29:06.786933+0000","last_undegraded":"2024-07-04T17:29:06.786933+0000","last_fullsized":"2024-07-04T17:29:06.786933+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:44:09.505235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.786232+0000","last_change":"2024-07-04T17:29:05.825187+0000","last_active":"2024-07-04T17:29:06.786232+0000","last_peered":"2024-07-04T17:29:06.786232+0000","last_clean":"2024-07-04T17:29:06.786232+0000","last_became_active":"2024-07-04T17:29:05.824857+0000","last_became_peered":"2024-07-04T17:29:05.824857+0000","last_unstale":"2024-07-04T17:29:06.786232+0000","last_undegraded":"2024-07-04T17:29:06.786232+0000","last_fullsized":"2024-07-04T17:29:06.786232+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:14:24.515173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.493231+0000","last_change":"2024-07-04T17:29:05.815028+0000","last_active":"2024-07-04T17:29:08.493231+0000","last_peered":"2024-07-04T17:29:08.493231+0000","last_clean":"2024-07-04T17:29:08.493231+0000","last_became_active":"2024-07-04T17:29:05.814094+0000","last_became_peered":"2024-07-04T17:29:05.814094+0000","last_unstale":"2024-07-04T17:29:08.493231+0000","last_undegraded":"2024-07-04T17:29:08.493231+0000","last_fullsized":"2024-07-04T17:29:08.493231+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:48:33.874991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.786110+0000","last_change":"2024-07-04T17:29:05.813654+0000","last_active":"2024-07-04T17:29:06.786110+0000","last_peered":"2024-07-04T17:29:06.786110+0000","last_clean":"2024-07-04T17:29:06.786110+0000","last_became_active":"2024-07-04T17:29:05.813139+0000","last_became_peered":"2024-07-04T17:29:05.813139+0000","last_unstale":"2024-07-04T17:29:06.786110+0000","last_undegraded":"2024-07-04T17:29:06.786110+0000","last_fullsized":"2024-07-04T17:29:06.786110+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:36:15.241406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.777172+0000","last_change":"2024-07-04T17:29:05.799841+0000","last_active":"2024-07-04T17:29:06.777172+0000","last_peered":"2024-07-04T17:29:06.777172+0000","last_clean":"2024-07-04T17:29:06.777172+0000","last_became_active":"2024-07-04T17:29:05.799478+0000","last_became_peered":"2024-07-04T17:29:05.799478+0000","last_unstale":"2024-07-04T17:29:06.777172+0000","last_undegraded":"2024-07-04T17:29:06.777172+0000","last_fullsized":"2024-07-04T17:29:06.777172+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:25:37.804442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.779497+0000","last_change":"2024-07-04T17:29:05.795312+0000","last_active":"2024-07-04T17:29:06.779497+0000","last_peered":"2024-07-04T17:29:06.779497+0000","last_clean":"2024-07-04T17:29:06.779497+0000","last_became_active":"2024-07-04T17:29:05.794446+0000","last_became_peered":"2024-07-04T17:29:05.794446+0000","last_unstale":"2024-07-04T17:29:06.779497+0000","last_undegraded":"2024-07-04T17:29:06.779497+0000","last_fullsized":"2024-07-04T17:29:06.779497+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:54:42.187784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"49'4","reported_seq":18,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.779652+0000","last_change":"2024-07-04T17:29:04.768776+0000","last_active":"2024-07-04T17:29:06.779652+0000","last_peered":"2024-07-04T17:29:06.779652+0000","last_clean":"2024-07-04T17:29:06.779652+0000","last_became_active":"2024-07-04T17:29:04.766498+0000","last_became_peered":"2024-07-04T17:29:04.766498+0000","last_unstale":"2024-07-04T17:29:06.779652+0000","last_undegraded":"2024-07-04T17:29:06.779652+0000","last_fullsized":"2024-07-04T17:29:06.779652+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:03:44.115014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.495718+0000","last_change":"2024-07-04T17:29:05.790122+0000","last_active":"2024-07-04T17:29:08.495718+0000","last_peered":"2024-07-04T17:29:08.495718+0000","last_clean":"2024-07-04T17:29:08.495718+0000","last_became_active":"2024-07-04T17:29:05.789804+0000","last_became_peered":"2024-07-04T17:29:05.789804+0000","last_unstale":"2024-07-04T17:29:08.495718+0000","last_undegraded":"2024-07-04T17:29:08.495718+0000","last_fullsized":"2024-07-04T17:29:08.495718+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:06:24.983072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.495869+0000","last_change":"2024-07-04T17:29:04.779844+0000","last_active":"2024-07-04T17:29:08.495869+0000","last_peered":"2024-07-04T17:29:08.495869+0000","last_clean":"2024-07-04T17:29:08.495869+0000","last_became_active":"2024-07-04T17:29:04.779212+0000","last_became_peered":"2024-07-04T17:29:04.779212+0000","last_unstale":"2024-07-04T17:29:08.495869+0000","last_undegraded":"2024-07-04T17:29:08.495869+0000","last_fullsized":"2024-07-04T17:29:08.495869+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:06:45.499488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.780222+0000","last_change":"2024-07-04T17:29:05.799208+0000","last_active":"2024-07-04T17:29:06.780222+0000","last_peered":"2024-07-04T17:29:06.780222+0000","last_clean":"2024-07-04T17:29:06.780222+0000","last_became_active":"2024-07-04T17:29:05.796533+0000","last_became_peered":"2024-07-04T17:29:05.796533+0000","last_unstale":"2024-07-04T17:29:06.780222+0000","last_undegraded":"2024-07-04T17:29:06.780222+0000","last_fullsized":"2024-07-04T17:29:06.780222+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:28:33.331115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.778686+0000","last_change":"2024-07-04T17:29:04.767288+0000","last_active":"2024-07-04T17:29:06.778686+0000","last_peered":"2024-07-04T17:29:06.778686+0000","last_clean":"2024-07-04T17:29:06.778686+0000","last_became_active":"2024-07-04T17:29:04.765876+0000","last_became_peered":"2024-07-04T17:29:04.765876+0000","last_unstale":"2024-07-04T17:29:06.778686+0000","last_undegraded":"2024-07-04T17:29:06.778686+0000","last_fullsized":"2024-07-04T17:29:06.778686+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:54:49.684096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.778162+0000","last_change":"2024-07-04T17:29:05.785226+0000","last_active":"2024-07-04T17:29:06.778162+0000","last_peered":"2024-07-04T17:29:06.778162+0000","last_clean":"2024-07-04T17:29:06.778162+0000","last_became_active":"2024-07-04T17:29:05.784069+0000","last_became_peered":"2024-07-04T17:29:05.784069+0000","last_unstale":"2024-07-04T17:29:06.778162+0000","last_undegraded":"2024-07-04T17:29:06.778162+0000","last_fullsized":"2024-07-04T17:29:06.778162+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:27:43.064532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.787982+0000","last_change":"2024-07-04T17:29:04.793187+0000","last_active":"2024-07-04T17:29:06.787982+0000","last_peered":"2024-07-04T17:29:06.787982+0000","last_clean":"2024-07-04T17:29:06.787982+0000","last_became_active":"2024-07-04T17:29:04.792877+0000","last_became_peered":"2024-07-04T17:29:04.792877+0000","last_unstale":"2024-07-04T17:29:06.787982+0000","last_undegraded":"2024-07-04T17:29:06.787982+0000","last_fullsized":"2024-07-04T17:29:06.787982+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:16:45.088529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.494440+0000","last_change":"2024-07-04T17:29:05.796869+0000","last_active":"2024-07-04T17:29:08.494440+0000","last_peered":"2024-07-04T17:29:08.494440+0000","last_clean":"2024-07-04T17:29:08.494440+0000","last_became_active":"2024-07-04T17:29:05.796573+0000","last_became_peered":"2024-07-04T17:29:05.796573+0000","last_unstale":"2024-07-04T17:29:08.494440+0000","last_undegraded":"2024-07-04T17:29:08.494440+0000","last_fullsized":"2024-07-04T17:29:08.494440+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:41:01.850387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1b","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.786620+0000","last_change":"2024-07-04T17:29:04.793835+0000","last_active":"2024-07-04T17:29:06.786620+0000","last_peered":"2024-07-04T17:29:06.786620+0000","last_clean":"2024-07-04T17:29:06.786620+0000","last_became_active":"2024-07-04T17:29:04.792785+0000","last_became_peered":"2024-07-04T17:29:04.792785+0000","last_unstale":"2024-07-04T17:29:06.786620+0000","last_undegraded":"2024-07-04T17:29:06.786620+0000","last_fullsized":"2024-07-04T17:29:06.786620+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:21:48.040633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.779463+0000","last_change":"2024-07-04T17:29:05.796490+0000","last_active":"2024-07-04T17:29:06.779463+0000","last_peered":"2024-07-04T17:29:06.779463+0000","last_clean":"2024-07-04T17:29:06.779463+0000","last_became_active":"2024-07-04T17:29:05.788778+0000","last_became_peered":"2024-07-04T17:29:05.788778+0000","last_unstale":"2024-07-04T17:29:06.779463+0000","last_undegraded":"2024-07-04T17:29:06.779463+0000","last_fullsized":"2024-07-04T17:29:06.779463+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:23:59.134121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1a","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.779091+0000","last_change":"2024-07-04T17:29:04.756095+0000","last_active":"2024-07-04T17:29:06.779091+0000","last_peered":"2024-07-04T17:29:06.779091+0000","last_clean":"2024-07-04T17:29:06.779091+0000","last_became_active":"2024-07-04T17:29:04.755679+0000","last_became_peered":"2024-07-04T17:29:04.755679+0000","last_unstale":"2024-07-04T17:29:06.779091+0000","last_undegraded":"2024-07-04T17:29:06.779091+0000","last_fullsized":"2024-07-04T17:29:06.779091+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:30:46.500676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.786319+0000","last_change":"2024-07-04T17:29:05.809480+0000","last_active":"2024-07-04T17:29:06.786319+0000","last_peered":"2024-07-04T17:29:06.786319+0000","last_clean":"2024-07-04T17:29:06.786319+0000","last_became_active":"2024-07-04T17:29:05.809138+0000","last_became_peered":"2024-07-04T17:29:05.809138+0000","last_unstale":"2024-07-04T17:29:06.786319+0000","last_undegraded":"2024-07-04T17:29:06.786319+0000","last_fullsized":"2024-07-04T17:29:06.786319+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:03:20.072994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.493322+0000","last_change":"2024-07-04T17:29:04.776840+0000","last_active":"2024-07-04T17:29:08.493322+0000","last_peered":"2024-07-04T17:29:08.493322+0000","last_clean":"2024-07-04T17:29:08.493322+0000","last_became_active":"2024-07-04T17:29:04.775729+0000","last_became_peered":"2024-07-04T17:29:04.775729+0000","last_unstale":"2024-07-04T17:29:08.493322+0000","last_undegraded":"2024-07-04T17:29:08.493322+0000","last_fullsized":"2024-07-04T17:29:08.493322+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:28:41.894029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.786904+0000","last_change":"2024-07-04T17:29:05.818108+0000","last_active":"2024-07-04T17:29:06.786904+0000","last_peered":"2024-07-04T17:29:06.786904+0000","last_clean":"2024-07-04T17:29:06.786904+0000","last_became_active":"2024-07-04T17:29:05.816963+0000","last_became_peered":"2024-07-04T17:29:05.816963+0000","last_unstale":"2024-07-04T17:29:06.786904+0000","last_undegraded":"2024-07-04T17:29:06.786904+0000","last_fullsized":"2024-07-04T17:29:06.786904+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:07:08.833691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.777125+0000","last_change":"2024-07-04T17:29:04.774790+0000","last_active":"2024-07-04T17:29:06.777125+0000","last_peered":"2024-07-04T17:29:06.777125+0000","last_clean":"2024-07-04T17:29:06.777125+0000","last_became_active":"2024-07-04T17:29:04.774427+0000","last_became_peered":"2024-07-04T17:29:04.774427+0000","last_unstale":"2024-07-04T17:29:06.777125+0000","last_undegraded":"2024-07-04T17:29:06.777125+0000","last_fullsized":"2024-07-04T17:29:06.777125+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:28:48.914599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.779685+0000","last_change":"2024-07-04T17:29:05.795423+0000","last_active":"2024-07-04T17:29:06.779685+0000","last_peered":"2024-07-04T17:29:06.779685+0000","last_clean":"2024-07-04T17:29:06.779685+0000","last_became_active":"2024-07-04T17:29:05.794868+0000","last_became_peered":"2024-07-04T17:29:05.794868+0000","last_unstale":"2024-07-04T17:29:06.779685+0000","last_undegraded":"2024-07-04T17:29:06.779685+0000","last_fullsized":"2024-07-04T17:29:06.779685+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:36:01.857174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.496077+0000","last_change":"2024-07-04T17:29:04.772998+0000","last_active":"2024-07-04T17:29:08.496077+0000","last_peered":"2024-07-04T17:29:08.496077+0000","last_clean":"2024-07-04T17:29:08.496077+0000","last_became_active":"2024-07-04T17:29:04.772635+0000","last_became_peered":"2024-07-04T17:29:04.772635+0000","last_unstale":"2024-07-04T17:29:08.496077+0000","last_undegraded":"2024-07-04T17:29:08.496077+0000","last_fullsized":"2024-07-04T17:29:08.496077+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:59:48.689604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.789520+0000","last_change":"2024-07-04T17:29:05.832809+0000","last_active":"2024-07-04T17:29:06.789520+0000","last_peered":"2024-07-04T17:29:06.789520+0000","last_clean":"2024-07-04T17:29:06.789520+0000","last_became_active":"2024-07-04T17:29:05.829582+0000","last_became_peered":"2024-07-04T17:29:05.829582+0000","last_unstale":"2024-07-04T17:29:06.789520+0000","last_undegraded":"2024-07-04T17:29:06.789520+0000","last_fullsized":"2024-07-04T17:29:06.789520+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:50:42.737526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.496204+0000","last_change":"2024-07-04T17:29:04.780571+0000","last_active":"2024-07-04T17:29:08.496204+0000","last_peered":"2024-07-04T17:29:08.496204+0000","last_clean":"2024-07-04T17:29:08.496204+0000","last_became_active":"2024-07-04T17:29:04.780236+0000","last_became_peered":"2024-07-04T17:29:04.780236+0000","last_unstale":"2024-07-04T17:29:08.496204+0000","last_undegraded":"2024-07-04T17:29:08.496204+0000","last_fullsized":"2024-07-04T17:29:08.496204+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:55:21.724770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.494460+0000","last_change":"2024-07-04T17:29:05.783421+0000","last_active":"2024-07-04T17:29:08.494460+0000","last_peered":"2024-07-04T17:29:08.494460+0000","last_clean":"2024-07-04T17:29:08.494460+0000","last_became_active":"2024-07-04T17:29:05.783057+0000","last_became_peered":"2024-07-04T17:29:05.783057+0000","last_unstale":"2024-07-04T17:29:08.494460+0000","last_undegraded":"2024-07-04T17:29:08.494460+0000","last_fullsized":"2024-07-04T17:29:08.494460+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:44:09.505235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"49'3","reported_seq":17,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.789045+0000","last_change":"2024-07-04T17:29:04.794454+0000","last_active":"2024-07-04T17:29:06.789045+0000","last_peered":"2024-07-04T17:29:06.789045+0000","last_clean":"2024-07-04T17:29:06.789045+0000","last_became_active":"2024-07-04T17:29:04.793939+0000","last_became_peered":"2024-07-04T17:29:04.793939+0000","last_unstale":"2024-07-04T17:29:06.789045+0000","last_undegraded":"2024-07-04T17:29:06.789045+0000","last_fullsized":"2024-07-04T17:29:06.789045+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:10:23.616021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.495405+0000","last_change":"2024-07-04T17:29:05.792689+0000","last_active":"2024-07-04T17:29:08.495405+0000","last_peered":"2024-07-04T17:29:08.495405+0000","last_clean":"2024-07-04T17:29:08.495405+0000","last_became_active":"2024-07-04T17:29:05.792368+0000","last_became_peered":"2024-07-04T17:29:05.792368+0000","last_unstale":"2024-07-04T17:29:08.495405+0000","last_undegraded":"2024-07-04T17:29:08.495405+0000","last_fullsized":"2024-07-04T17:29:08.495405+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:58:35.184348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.14","version":"49'6","reported_seq":20,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.780205+0000","last_change":"2024-07-04T17:29:04.778462+0000","last_active":"2024-07-04T17:29:06.780205+0000","last_peered":"2024-07-04T17:29:06.780205+0000","last_clean":"2024-07-04T17:29:06.780205+0000","last_became_active":"2024-07-04T17:29:04.775998+0000","last_became_peered":"2024-07-04T17:29:04.775998+0000","last_unstale":"2024-07-04T17:29:06.780205+0000","last_undegraded":"2024-07-04T17:29:06.780205+0000","last_fullsized":"2024-07-04T17:29:06.780205+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:16:55.504943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.493827+0000","last_change":"2024-07-04T17:29:05.815269+0000","last_active":"2024-07-04T17:29:08.493827+0000","last_peered":"2024-07-04T17:29:08.493827+0000","last_clean":"2024-07-04T17:29:08.493827+0000","last_became_active":"2024-07-04T17:29:05.814468+0000","last_became_peered":"2024-07-04T17:29:05.814468+0000","last_unstale":"2024-07-04T17:29:08.493827+0000","last_undegraded":"2024-07-04T17:29:08.493827+0000","last_fullsized":"2024-07-04T17:29:08.493827+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:14:24.515173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.493881+0000","last_change":"2024-07-04T17:29:04.782215+0000","last_active":"2024-07-04T17:29:08.493881+0000","last_peered":"2024-07-04T17:29:08.493881+0000","last_clean":"2024-07-04T17:29:08.493881+0000","last_became_active":"2024-07-04T17:29:04.781891+0000","last_became_peered":"2024-07-04T17:29:04.781891+0000","last_unstale":"2024-07-04T17:29:08.493881+0000","last_undegraded":"2024-07-04T17:29:08.493881+0000","last_fullsized":"2024-07-04T17:29:08.493881+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:56:44.314464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.786924+0000","last_change":"2024-07-04T17:29:05.830879+0000","last_active":"2024-07-04T17:29:06.786924+0000","last_peered":"2024-07-04T17:29:06.786924+0000","last_clean":"2024-07-04T17:29:06.786924+0000","last_became_active":"2024-07-04T17:29:05.830572+0000","last_became_peered":"2024-07-04T17:29:05.830572+0000","last_unstale":"2024-07-04T17:29:06.786924+0000","last_undegraded":"2024-07-04T17:29:06.786924+0000","last_fullsized":"2024-07-04T17:29:06.786924+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:41:01.850387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.787695+0000","last_change":"2024-07-04T17:29:04.793440+0000","last_active":"2024-07-04T17:29:06.787695+0000","last_peered":"2024-07-04T17:29:06.787695+0000","last_clean":"2024-07-04T17:29:06.787695+0000","last_became_active":"2024-07-04T17:29:04.792810+0000","last_became_peered":"2024-07-04T17:29:04.792810+0000","last_unstale":"2024-07-04T17:29:06.787695+0000","last_undegraded":"2024-07-04T17:29:06.787695+0000","last_fullsized":"2024-07-04T17:29:06.787695+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:28:41.894029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.493534+0000","last_change":"2024-07-04T17:29:05.822812+0000","last_active":"2024-07-04T17:29:08.493534+0000","last_peered":"2024-07-04T17:29:08.493534+0000","last_clean":"2024-07-04T17:29:08.493534+0000","last_became_active":"2024-07-04T17:29:05.822519+0000","last_became_peered":"2024-07-04T17:29:05.822519+0000","last_unstale":"2024-07-04T17:29:08.493534+0000","last_undegraded":"2024-07-04T17:29:08.493534+0000","last_fullsized":"2024-07-04T17:29:08.493534+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:33:04.621287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.787191+0000","last_change":"2024-07-04T17:29:04.793535+0000","last_active":"2024-07-04T17:29:06.787191+0000","last_peered":"2024-07-04T17:29:06.787191+0000","last_clean":"2024-07-04T17:29:06.787191+0000","last_became_active":"2024-07-04T17:29:04.793160+0000","last_became_peered":"2024-07-04T17:29:04.793160+0000","last_unstale":"2024-07-04T17:29:06.787191+0000","last_undegraded":"2024-07-04T17:29:06.787191+0000","last_fullsized":"2024-07-04T17:29:06.787191+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:29:48.755500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.780421+0000","last_change":"2024-07-04T17:29:05.793691+0000","last_active":"2024-07-04T17:29:06.780421+0000","last_peered":"2024-07-04T17:29:06.780421+0000","last_clean":"2024-07-04T17:29:06.780421+0000","last_became_active":"2024-07-04T17:29:05.792767+0000","last_became_peered":"2024-07-04T17:29:05.792767+0000","last_unstale":"2024-07-04T17:29:06.780421+0000","last_undegraded":"2024-07-04T17:29:06.780421+0000","last_fullsized":"2024-07-04T17:29:06.780421+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:25:53.656021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.10","version":"49'8","reported_seq":22,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.777268+0000","last_change":"2024-07-04T17:29:04.767026+0000","last_active":"2024-07-04T17:29:06.777268+0000","last_peered":"2024-07-04T17:29:06.777268+0000","last_clean":"2024-07-04T17:29:06.777268+0000","last_became_active":"2024-07-04T17:29:04.764888+0000","last_became_peered":"2024-07-04T17:29:04.764888+0000","last_unstale":"2024-07-04T17:29:06.777268+0000","last_undegraded":"2024-07-04T17:29:06.777268+0000","last_fullsized":"2024-07-04T17:29:06.777268+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:08:09.031069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.788757+0000","last_change":"2024-07-04T17:29:05.826630+0000","last_active":"2024-07-04T17:29:06.788757+0000","last_peered":"2024-07-04T17:29:06.788757+0000","last_clean":"2024-07-04T17:29:06.788757+0000","last_became_active":"2024-07-04T17:29:05.826180+0000","last_became_peered":"2024-07-04T17:29:05.826180+0000","last_unstale":"2024-07-04T17:29:06.788757+0000","last_undegraded":"2024-07-04T17:29:06.788757+0000","last_fullsized":"2024-07-04T17:29:06.788757+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:42:34.680103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.779700+0000","last_change":"2024-07-04T17:29:04.778986+0000","last_active":"2024-07-04T17:29:06.779700+0000","last_peered":"2024-07-04T17:29:06.779700+0000","last_clean":"2024-07-04T17:29:06.779700+0000","last_became_active":"2024-07-04T17:29:04.775326+0000","last_became_peered":"2024-07-04T17:29:04.775326+0000","last_unstale":"2024-07-04T17:29:06.779700+0000","last_undegraded":"2024-07-04T17:29:06.779700+0000","last_fullsized":"2024-07-04T17:29:06.779700+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:01:44.218523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.789569+0000","last_change":"2024-07-04T17:29:05.825845+0000","last_active":"2024-07-04T17:29:06.789569+0000","last_peered":"2024-07-04T17:29:06.789569+0000","last_clean":"2024-07-04T17:29:06.789569+0000","last_became_active":"2024-07-04T17:29:05.824923+0000","last_became_peered":"2024-07-04T17:29:05.824923+0000","last_unstale":"2024-07-04T17:29:06.789569+0000","last_undegraded":"2024-07-04T17:29:06.789569+0000","last_fullsized":"2024-07-04T17:29:06.789569+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:30:47.509371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.779858+0000","last_change":"2024-07-04T17:29:04.787322+0000","last_active":"2024-07-04T17:29:06.779858+0000","last_peered":"2024-07-04T17:29:06.779858+0000","last_clean":"2024-07-04T17:29:06.779858+0000","last_became_active":"2024-07-04T17:29:04.786939+0000","last_became_peered":"2024-07-04T17:29:04.786939+0000","last_unstale":"2024-07-04T17:29:06.779858+0000","last_undegraded":"2024-07-04T17:29:06.779858+0000","last_fullsized":"2024-07-04T17:29:06.779858+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:31:08.982204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.787421+0000","last_change":"2024-07-04T17:29:05.817332+0000","last_active":"2024-07-04T17:29:06.787421+0000","last_peered":"2024-07-04T17:29:06.787421+0000","last_clean":"2024-07-04T17:29:06.787421+0000","last_became_active":"2024-07-04T17:29:05.816980+0000","last_became_peered":"2024-07-04T17:29:05.816980+0000","last_unstale":"2024-07-04T17:29:06.787421+0000","last_undegraded":"2024-07-04T17:29:06.787421+0000","last_fullsized":"2024-07-04T17:29:06.787421+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:35:30.393302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.789405+0000","last_change":"2024-07-04T17:29:04.793950+0000","last_active":"2024-07-04T17:29:06.789405+0000","last_peered":"2024-07-04T17:29:06.789405+0000","last_clean":"2024-07-04T17:29:06.789405+0000","last_became_active":"2024-07-04T17:29:04.793520+0000","last_became_peered":"2024-07-04T17:29:04.793520+0000","last_unstale":"2024-07-04T17:29:06.789405+0000","last_undegraded":"2024-07-04T17:29:06.789405+0000","last_fullsized":"2024-07-04T17:29:06.789405+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:43:57.308367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.780680+0000","last_change":"2024-07-04T17:29:05.796377+0000","last_active":"2024-07-04T17:29:06.780680+0000","last_peered":"2024-07-04T17:29:06.780680+0000","last_clean":"2024-07-04T17:29:06.780680+0000","last_became_active":"2024-07-04T17:29:05.796069+0000","last_became_peered":"2024-07-04T17:29:05.796069+0000","last_unstale":"2024-07-04T17:29:06.780680+0000","last_undegraded":"2024-07-04T17:29:06.780680+0000","last_fullsized":"2024-07-04T17:29:06.780680+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:36:28.462582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.780553+0000","last_change":"2024-07-04T17:29:04.775241+0000","last_active":"2024-07-04T17:29:06.780553+0000","last_peered":"2024-07-04T17:29:06.780553+0000","last_clean":"2024-07-04T17:29:06.780553+0000","last_became_active":"2024-07-04T17:29:04.774292+0000","last_became_peered":"2024-07-04T17:29:04.774292+0000","last_unstale":"2024-07-04T17:29:06.780553+0000","last_undegraded":"2024-07-04T17:29:06.780553+0000","last_fullsized":"2024-07-04T17:29:06.780553+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:48:39.886887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.494364+0000","last_change":"2024-07-04T17:29:05.814789+0000","last_active":"2024-07-04T17:29:08.494364+0000","last_peered":"2024-07-04T17:29:08.494364+0000","last_clean":"2024-07-04T17:29:08.494364+0000","last_became_active":"2024-07-04T17:29:05.813725+0000","last_became_peered":"2024-07-04T17:29:05.813725+0000","last_unstale":"2024-07-04T17:29:08.494364+0000","last_undegraded":"2024-07-04T17:29:08.494364+0000","last_fullsized":"2024-07-04T17:29:08.494364+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:07:08.833691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.780830+0000","last_change":"2024-07-04T17:29:04.768448+0000","last_active":"2024-07-04T17:29:06.780830+0000","last_peered":"2024-07-04T17:29:06.780830+0000","last_clean":"2024-07-04T17:29:06.780830+0000","last_became_active":"2024-07-04T17:29:04.765316+0000","last_became_peered":"2024-07-04T17:29:04.765316+0000","last_unstale":"2024-07-04T17:29:06.780830+0000","last_undegraded":"2024-07-04T17:29:06.780830+0000","last_fullsized":"2024-07-04T17:29:06.780830+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:54:53.983424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.779961+0000","last_change":"2024-07-04T17:29:05.784868+0000","last_active":"2024-07-04T17:29:06.779961+0000","last_peered":"2024-07-04T17:29:06.779961+0000","last_clean":"2024-07-04T17:29:06.779961+0000","last_became_active":"2024-07-04T17:29:05.784419+0000","last_became_peered":"2024-07-04T17:29:05.784419+0000","last_unstale":"2024-07-04T17:29:06.779961+0000","last_undegraded":"2024-07-04T17:29:06.779961+0000","last_fullsized":"2024-07-04T17:29:06.779961+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:33:47.592614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.495868+0000","last_change":"2024-07-04T17:29:04.780522+0000","last_active":"2024-07-04T17:29:08.495868+0000","last_peered":"2024-07-04T17:29:08.495868+0000","last_clean":"2024-07-04T17:29:08.495868+0000","last_became_active":"2024-07-04T17:29:04.780161+0000","last_became_peered":"2024-07-04T17:29:04.780161+0000","last_unstale":"2024-07-04T17:29:08.495868+0000","last_undegraded":"2024-07-04T17:29:08.495868+0000","last_fullsized":"2024-07-04T17:29:08.495868+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:29:48.755500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.780310+0000","last_change":"2024-07-04T17:29:05.793437+0000","last_active":"2024-07-04T17:29:06.780310+0000","last_peered":"2024-07-04T17:29:06.780310+0000","last_clean":"2024-07-04T17:29:06.780310+0000","last_became_active":"2024-07-04T17:29:05.792368+0000","last_became_peered":"2024-07-04T17:29:05.792368+0000","last_unstale":"2024-07-04T17:29:06.780310+0000","last_undegraded":"2024-07-04T17:29:06.780310+0000","last_fullsized":"2024-07-04T17:29:06.780310+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:17:35.033747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.494033+0000","last_change":"2024-07-04T17:29:04.774550+0000","last_active":"2024-07-04T17:29:08.494033+0000","last_peered":"2024-07-04T17:29:08.494033+0000","last_clean":"2024-07-04T17:29:08.494033+0000","last_became_active":"2024-07-04T17:29:04.774168+0000","last_became_peered":"2024-07-04T17:29:04.774168+0000","last_unstale":"2024-07-04T17:29:08.494033+0000","last_undegraded":"2024-07-04T17:29:08.494033+0000","last_fullsized":"2024-07-04T17:29:08.494033+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:43:57.308367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.788825+0000","last_change":"2024-07-04T17:29:05.826827+0000","last_active":"2024-07-04T17:29:06.788825+0000","last_peered":"2024-07-04T17:29:06.788825+0000","last_clean":"2024-07-04T17:29:06.788825+0000","last_became_active":"2024-07-04T17:29:05.825730+0000","last_became_peered":"2024-07-04T17:29:05.825730+0000","last_unstale":"2024-07-04T17:29:06.788825+0000","last_undegraded":"2024-07-04T17:29:06.788825+0000","last_fullsized":"2024-07-04T17:29:06.788825+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:14:31.441713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.496220+0000","last_change":"2024-07-04T17:29:04.780892+0000","last_active":"2024-07-04T17:29:08.496220+0000","last_peered":"2024-07-04T17:29:08.496220+0000","last_clean":"2024-07-04T17:29:08.496220+0000","last_became_active":"2024-07-04T17:29:04.780559+0000","last_became_peered":"2024-07-04T17:29:04.780559+0000","last_unstale":"2024-07-04T17:29:08.496220+0000","last_undegraded":"2024-07-04T17:29:08.496220+0000","last_fullsized":"2024-07-04T17:29:08.496220+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:46:37.056095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.789765+0000","last_change":"2024-07-04T17:29:05.831979+0000","last_active":"2024-07-04T17:29:06.789765+0000","last_peered":"2024-07-04T17:29:06.789765+0000","last_clean":"2024-07-04T17:29:06.789765+0000","last_became_active":"2024-07-04T17:29:05.829983+0000","last_became_peered":"2024-07-04T17:29:05.829983+0000","last_unstale":"2024-07-04T17:29:06.789765+0000","last_undegraded":"2024-07-04T17:29:06.789765+0000","last_fullsized":"2024-07-04T17:29:06.789765+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:03:45.123708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"50'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.783682+0000","last_change":"2024-07-04T17:29:04.784413+0000","last_active":"2024-07-04T17:29:06.783682+0000","last_peered":"2024-07-04T17:29:06.783682+0000","last_clean":"2024-07-04T17:29:06.783682+0000","last_became_active":"2024-07-04T17:29:04.784058+0000","last_became_peered":"2024-07-04T17:29:04.784058+0000","last_unstale":"2024-07-04T17:29:06.783682+0000","last_undegraded":"2024-07-04T17:29:06.783682+0000","last_fullsized":"2024-07-04T17:29:06.783682+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:35:32.227491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.495077+0000","last_change":"2024-07-04T17:29:05.821127+0000","last_active":"2024-07-04T17:29:08.495077+0000","last_peered":"2024-07-04T17:29:08.495077+0000","last_clean":"2024-07-04T17:29:08.495077+0000","last_became_active":"2024-07-04T17:29:05.820787+0000","last_became_peered":"2024-07-04T17:29:05.820787+0000","last_unstale":"2024-07-04T17:29:08.495077+0000","last_undegraded":"2024-07-04T17:29:08.495077+0000","last_fullsized":"2024-07-04T17:29:08.495077+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:33:26.836346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.495846+0000","last_change":"2024-07-04T17:29:04.786173+0000","last_active":"2024-07-04T17:29:08.495846+0000","last_peered":"2024-07-04T17:29:08.495846+0000","last_clean":"2024-07-04T17:29:08.495846+0000","last_became_active":"2024-07-04T17:29:04.783241+0000","last_became_peered":"2024-07-04T17:29:04.783241+0000","last_unstale":"2024-07-04T17:29:08.495846+0000","last_undegraded":"2024-07-04T17:29:08.495846+0000","last_fullsized":"2024-07-04T17:29:08.495846+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:16:45.088529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.778525+0000","last_change":"2024-07-04T17:29:05.785545+0000","last_active":"2024-07-04T17:29:06.778525+0000","last_peered":"2024-07-04T17:29:06.778525+0000","last_clean":"2024-07-04T17:29:06.778525+0000","last_became_active":"2024-07-04T17:29:05.784520+0000","last_became_peered":"2024-07-04T17:29:05.784520+0000","last_unstale":"2024-07-04T17:29:06.778525+0000","last_undegraded":"2024-07-04T17:29:06.778525+0000","last_fullsized":"2024-07-04T17:29:06.778525+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:31:15.745622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.787945+0000","last_change":"2024-07-04T17:29:04.794330+0000","last_active":"2024-07-04T17:29:06.787945+0000","last_peered":"2024-07-04T17:29:06.787945+0000","last_clean":"2024-07-04T17:29:06.787945+0000","last_became_active":"2024-07-04T17:29:04.793980+0000","last_became_peered":"2024-07-04T17:29:04.793980+0000","last_unstale":"2024-07-04T17:29:06.787945+0000","last_undegraded":"2024-07-04T17:29:06.787945+0000","last_fullsized":"2024-07-04T17:29:06.787945+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:06:13.018475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.495701+0000","last_change":"2024-07-04T17:29:05.802494+0000","last_active":"2024-07-04T17:29:08.495701+0000","last_peered":"2024-07-04T17:29:08.495701+0000","last_clean":"2024-07-04T17:29:08.495701+0000","last_became_active":"2024-07-04T17:29:05.795642+0000","last_became_peered":"2024-07-04T17:29:05.795642+0000","last_unstale":"2024-07-04T17:29:08.495701+0000","last_undegraded":"2024-07-04T17:29:08.495701+0000","last_fullsized":"2024-07-04T17:29:08.495701+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:19:50.013783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.494710+0000","last_change":"2024-07-04T17:29:04.774431+0000","last_active":"2024-07-04T17:29:08.494710+0000","last_peered":"2024-07-04T17:29:08.494710+0000","last_clean":"2024-07-04T17:29:08.494710+0000","last_became_active":"2024-07-04T17:29:04.774034+0000","last_became_peered":"2024-07-04T17:29:04.774034+0000","last_unstale":"2024-07-04T17:29:08.494710+0000","last_undegraded":"2024-07-04T17:29:08.494710+0000","last_fullsized":"2024-07-04T17:29:08.494710+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:10:23.616021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.780478+0000","last_change":"2024-07-04T17:29:05.784396+0000","last_active":"2024-07-04T17:29:06.780478+0000","last_peered":"2024-07-04T17:29:06.780478+0000","last_clean":"2024-07-04T17:29:06.780478+0000","last_became_active":"2024-07-04T17:29:05.783953+0000","last_became_peered":"2024-07-04T17:29:05.783953+0000","last_unstale":"2024-07-04T17:29:06.780478+0000","last_undegraded":"2024-07-04T17:29:06.780478+0000","last_fullsized":"2024-07-04T17:29:06.780478+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:53:09.757420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.787796+0000","last_change":"2024-07-04T17:29:04.795160+0000","last_active":"2024-07-04T17:29:06.787796+0000","last_peered":"2024-07-04T17:29:06.787796+0000","last_clean":"2024-07-04T17:29:06.787796+0000","last_became_active":"2024-07-04T17:29:04.793328+0000","last_became_peered":"2024-07-04T17:29:04.793328+0000","last_unstale":"2024-07-04T17:29:06.787796+0000","last_undegraded":"2024-07-04T17:29:06.787796+0000","last_fullsized":"2024-07-04T17:29:06.787796+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:56:44.314464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.493704+0000","last_change":"2024-07-04T17:29:05.796165+0000","last_active":"2024-07-04T17:29:08.493704+0000","last_peered":"2024-07-04T17:29:08.493704+0000","last_clean":"2024-07-04T17:29:08.493704+0000","last_became_active":"2024-07-04T17:29:05.795837+0000","last_became_peered":"2024-07-04T17:29:05.795837+0000","last_unstale":"2024-07-04T17:29:08.493704+0000","last_undegraded":"2024-07-04T17:29:08.493704+0000","last_fullsized":"2024-07-04T17:29:08.493704+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:36:15.241406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.493658+0000","last_change":"2024-07-04T17:29:04.790173+0000","last_active":"2024-07-04T17:29:08.493658+0000","last_peered":"2024-07-04T17:29:08.493658+0000","last_clean":"2024-07-04T17:29:08.493658+0000","last_became_active":"2024-07-04T17:29:04.789042+0000","last_became_peered":"2024-07-04T17:29:04.789042+0000","last_unstale":"2024-07-04T17:29:08.493658+0000","last_undegraded":"2024-07-04T17:29:08.493658+0000","last_fullsized":"2024-07-04T17:29:08.493658+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:06:13.018475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.780953+0000","last_change":"2024-07-04T17:29:05.793083+0000","last_active":"2024-07-04T17:29:06.780953+0000","last_peered":"2024-07-04T17:29:06.780953+0000","last_clean":"2024-07-04T17:29:06.780953+0000","last_became_active":"2024-07-04T17:29:05.790165+0000","last_became_peered":"2024-07-04T17:29:05.790165+0000","last_unstale":"2024-07-04T17:29:06.780953+0000","last_undegraded":"2024-07-04T17:29:06.780953+0000","last_fullsized":"2024-07-04T17:29:06.780953+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:20:46.692294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.494163+0000","last_change":"2024-07-04T17:29:04.770011+0000","last_active":"2024-07-04T17:29:08.494163+0000","last_peered":"2024-07-04T17:29:08.494163+0000","last_clean":"2024-07-04T17:29:08.494163+0000","last_became_active":"2024-07-04T17:29:04.769666+0000","last_became_peered":"2024-07-04T17:29:04.769666+0000","last_unstale":"2024-07-04T17:29:08.494163+0000","last_undegraded":"2024-07-04T17:29:08.494163+0000","last_fullsized":"2024-07-04T17:29:08.494163+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:21:48.040633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.787341+0000","last_change":"2024-07-04T17:29:05.816496+0000","last_active":"2024-07-04T17:29:06.787341+0000","last_peered":"2024-07-04T17:29:06.787341+0000","last_clean":"2024-07-04T17:29:06.787341+0000","last_became_active":"2024-07-04T17:29:05.816147+0000","last_became_peered":"2024-07-04T17:29:05.816147+0000","last_unstale":"2024-07-04T17:29:06.787341+0000","last_undegraded":"2024-07-04T17:29:06.787341+0000","last_fullsized":"2024-07-04T17:29:06.787341+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:33:26.836346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.780719+0000","last_change":"2024-07-04T17:29:04.770392+0000","last_active":"2024-07-04T17:29:06.780719+0000","last_peered":"2024-07-04T17:29:06.780719+0000","last_clean":"2024-07-04T17:29:06.780719+0000","last_became_active":"2024-07-04T17:29:04.758815+0000","last_became_peered":"2024-07-04T17:29:04.758815+0000","last_unstale":"2024-07-04T17:29:06.780719+0000","last_undegraded":"2024-07-04T17:29:06.780719+0000","last_fullsized":"2024-07-04T17:29:06.780719+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:35:08.872682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"35'66","reported_seq":203,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.494249+0000","last_change":"2024-07-04T17:26:04.953559+0000","last_active":"2024-07-04T17:29:08.494249+0000","last_peered":"2024-07-04T17:29:08.494249+0000","last_clean":"2024-07-04T17:29:08.494249+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:29:08.494249+0000","last_undegraded":"2024-07-04T17:29:08.494249+0000","last_fullsized":"2024-07-04T17:29:08.494249+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_clean_scrub_stamp":"2024-07-04T17:26:03.933432+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.064181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":398,"num_write":103,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":398,"num_write":103,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738407,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":512000,"data_stored":185802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738408,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32248,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339592,"statfs":{"total":96636764160,"available":96603742208,"internally_reserved":0,"allocated":1085440,"data_stored":771088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738406,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1097728,"data_stored":776558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:29:10.493 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:29:10.723 INFO:tasks.ceph.mds.d.smithi042.stderr:2024-07-04T17:29:10.722+0000 7fb57b000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-04T17:29:10.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.826+0000 7f3447a00640 1 -- 172.21.15.42:0/3354258072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34400b37b0 msgr2=0x7f34400b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:10.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.826+0000 7f3447a00640 1 --2- 172.21.15.42:0/3354258072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34400b37b0 0x7f34400b01f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f3430005fd0 tx=0x7f343002aac0 comp rx=0 tx=0).stop 2024-07-04T17:29:10.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.827+0000 7f3447a00640 1 -- 172.21.15.42:0/3354258072 shutdown_connections 2024-07-04T17:29:10.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.827+0000 7f3447a00640 1 --2- 172.21.15.42:0/3354258072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34400b37b0 0x7f34400b01f0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:10.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.827+0000 7f3447a00640 1 -- 172.21.15.42:0/3354258072 >> 172.21.15.42:0/3354258072 conn(0x7f34400bf700 msgr2=0x7f34400af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:10.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.827+0000 7f3447a00640 1 -- 172.21.15.42:0/3354258072 shutdown_connections 2024-07-04T17:29:10.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.827+0000 7f3447a00640 1 -- 172.21.15.42:0/3354258072 wait complete. 2024-07-04T17:29:10.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.829+0000 7f3447a00640 1 Processor -- start 2024-07-04T17:29:10.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.829+0000 7f3447a00640 1 -- start start 2024-07-04T17:29:10.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.830+0000 7f3447a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34400b2e00 0x7f34400ac5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:10.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.830+0000 7f3447a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f344006ce30 con 0x7f34400b2e00 2024-07-04T17:29:10.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.831+0000 7f3445000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34400b2e00 0x7f34400ac5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:10.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.831+0000 7f3445000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34400b2e00 0x7f34400ac5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37532/0 (socket says 172.21.15.42:37532) 2024-07-04T17:29:10.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.831+0000 7f3445000640 1 -- 172.21.15.42:0/4191963239 learned_addr learned my addr 172.21.15.42:0/4191963239 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:10.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.831+0000 7f3445000640 1 -- 172.21.15.42:0/4191963239 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34400af0e0 con 0x7f34400b2e00 2024-07-04T17:29:10.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.832+0000 7f3445000640 1 --2- 172.21.15.42:0/4191963239 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34400b2e00 0x7f34400ac5f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f342800c340 tx=0x7f342800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:10.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.832+0000 7f343cc00640 1 -- 172.21.15.42:0/4191963239 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3428017070 con 0x7f34400b2e00 2024-07-04T17:29:10.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.832+0000 7f343cc00640 1 -- 172.21.15.42:0/4191963239 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34280052f0 con 0x7f34400b2e00 2024-07-04T17:29:10.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.832+0000 7f343cc00640 1 -- 172.21.15.42:0/4191963239 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34280035f0 con 0x7f34400b2e00 2024-07-04T17:29:10.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.832+0000 7f3447a00640 1 -- 172.21.15.42:0/4191963239 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34400ada50 con 0x7f34400b2e00 2024-07-04T17:29:10.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.832+0000 7f3447a00640 1 -- 172.21.15.42:0/4191963239 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f344013f7b0 con 0x7f34400b2e00 2024-07-04T17:29:10.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.833+0000 7f3447a00640 1 -- 172.21.15.42:0/4191963239 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34400af2b0 con 0x7f34400b2e00 2024-07-04T17:29:10.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.833+0000 7f343cc00640 1 -- 172.21.15.42:0/4191963239 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f342801a070 con 0x7f34400b2e00 2024-07-04T17:29:10.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.834+0000 7f343cc00640 1 --2- 172.21.15.42:0/4191963239 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f341c03d4f0 0x7f341c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:10.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.834+0000 7f343cc00640 1 -- 172.21.15.42:0/4191963239 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 3439+0+0 (secure 0 0 0) 0x7f3428050bb0 con 0x7f34400b2e00 2024-07-04T17:29:10.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.834+0000 7f343fe00640 1 --2- 172.21.15.42:0/4191963239 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f341c03d4f0 0x7f341c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:10.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.835+0000 7f343fe00640 1 --2- 172.21.15.42:0/4191963239 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f341c03d4f0 0x7f341c03fa00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3430035000 tx=0x7f3430004ab0 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:10.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:10.839+0000 7f343cc00640 1 -- 172.21.15.42:0/4191963239 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f34280102e0 con 0x7f34400b2e00 2024-07-04T17:29:11.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.047+0000 7f3447a00640 1 -- 172.21.15.42:0/4191963239 --> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f344006d440 con 0x7f341c03d4f0 2024-07-04T17:29:11.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.055+0000 7f343cc00640 1 -- 172.21.15.42:0/4191963239 <== mgr.4613 v2:172.21.15.42:6824/2638271195 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245972 (secure 0 0 0) 0x7f344006d440 con 0x7f341c03d4f0 2024-07-04T17:29:11.058 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:29:11.062 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:29:11.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.062+0000 7f3447a00640 1 -- 172.21.15.42:0/4191963239 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f341c03d4f0 msgr2=0x7f341c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:11.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.062+0000 7f3447a00640 1 --2- 172.21.15.42:0/4191963239 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f341c03d4f0 0x7f341c03fa00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3430035000 tx=0x7f3430004ab0 comp rx=0 tx=0).stop 2024-07-04T17:29:11.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.062+0000 7f3447a00640 1 -- 172.21.15.42:0/4191963239 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34400b2e00 msgr2=0x7f34400ac5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:11.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.062+0000 7f3447a00640 1 --2- 172.21.15.42:0/4191963239 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34400b2e00 0x7f34400ac5f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f342800c340 tx=0x7f342800c800 comp rx=0 tx=0).stop 2024-07-04T17:29:11.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.062+0000 7f3447a00640 1 -- 172.21.15.42:0/4191963239 shutdown_connections 2024-07-04T17:29:11.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.062+0000 7f3447a00640 1 --2- 172.21.15.42:0/4191963239 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f341c03d4f0 0x7f341c03fa00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:11.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.062+0000 7f3447a00640 1 --2- 172.21.15.42:0/4191963239 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34400b2e00 0x7f34400ac5f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:11.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.062+0000 7f3447a00640 1 -- 172.21.15.42:0/4191963239 >> 172.21.15.42:0/4191963239 conn(0x7f34400bf700 msgr2=0x7f34400b1180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:11.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.063+0000 7f3447a00640 1 -- 172.21.15.42:0/4191963239 shutdown_connections 2024-07-04T17:29:11.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.063+0000 7f3447a00640 1 -- 172.21.15.42:0/4191963239 wait complete. 2024-07-04T17:29:11.090 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-07-04T17:29:09.651909+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":176,"num_read_kb":398,"num_write":148,"num_write_kb":1120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":115,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":96188,"kb_used_data":2632,"kb_used_omap":4,"kb_used_meta":93499,"kb_avail":283019332,"statfs":{"total":289910292480,"available":289811795968,"internally_reserved":0,"allocated":2695168,"data_stored":1733448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":95743311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.919578"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.789264+0000","last_change":"2024-07-04T17:29:05.832523+0000","last_active":"2024-07-04T17:29:06.789264+0000","last_peered":"2024-07-04T17:29:06.789264+0000","last_clean":"2024-07-04T17:29:06.789264+0000","last_became_active":"2024-07-04T17:29:05.831208+0000","last_became_peered":"2024-07-04T17:29:05.831208+0000","last_unstale":"2024-07-04T17:29:06.789264+0000","last_undegraded":"2024-07-04T17:29:06.789264+0000","last_fullsized":"2024-07-04T17:29:06.789264+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:52:48.071452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.788398+0000","last_change":"2024-07-04T17:29:05.826928+0000","last_active":"2024-07-04T17:29:06.788398+0000","last_peered":"2024-07-04T17:29:06.788398+0000","last_clean":"2024-07-04T17:29:06.788398+0000","last_became_active":"2024-07-04T17:29:05.826320+0000","last_became_peered":"2024-07-04T17:29:05.826320+0000","last_unstale":"2024-07-04T17:29:06.788398+0000","last_undegraded":"2024-07-04T17:29:06.788398+0000","last_fullsized":"2024-07-04T17:29:06.788398+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:06:24.983072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.788359+0000","last_change":"2024-07-04T17:29:05.825968+0000","last_active":"2024-07-04T17:29:06.788359+0000","last_peered":"2024-07-04T17:29:06.788359+0000","last_clean":"2024-07-04T17:29:06.788359+0000","last_became_active":"2024-07-04T17:29:05.825203+0000","last_became_peered":"2024-07-04T17:29:05.825203+0000","last_unstale":"2024-07-04T17:29:06.788359+0000","last_undegraded":"2024-07-04T17:29:06.788359+0000","last_fullsized":"2024-07-04T17:29:06.788359+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:19:50.013783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.779269+0000","last_change":"2024-07-04T17:29:05.783206+0000","last_active":"2024-07-04T17:29:06.779269+0000","last_peered":"2024-07-04T17:29:06.779269+0000","last_clean":"2024-07-04T17:29:06.779269+0000","last_became_active":"2024-07-04T17:29:05.782870+0000","last_became_peered":"2024-07-04T17:29:05.782870+0000","last_unstale":"2024-07-04T17:29:06.779269+0000","last_undegraded":"2024-07-04T17:29:06.779269+0000","last_fullsized":"2024-07-04T17:29:06.779269+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:25:13.149991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.786048+0000","last_change":"2024-07-04T17:29:05.810625+0000","last_active":"2024-07-04T17:29:06.786048+0000","last_peered":"2024-07-04T17:29:06.786048+0000","last_clean":"2024-07-04T17:29:06.786048+0000","last_became_active":"2024-07-04T17:29:05.810316+0000","last_became_peered":"2024-07-04T17:29:05.810316+0000","last_unstale":"2024-07-04T17:29:06.786048+0000","last_undegraded":"2024-07-04T17:29:06.786048+0000","last_fullsized":"2024-07-04T17:29:06.786048+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:48:26.486742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.778745+0000","last_change":"2024-07-04T17:29:05.799666+0000","last_active":"2024-07-04T17:29:06.778745+0000","last_peered":"2024-07-04T17:29:06.778745+0000","last_clean":"2024-07-04T17:29:06.778745+0000","last_became_active":"2024-07-04T17:29:05.788794+0000","last_became_peered":"2024-07-04T17:29:05.788794+0000","last_unstale":"2024-07-04T17:29:06.778745+0000","last_undegraded":"2024-07-04T17:29:06.778745+0000","last_fullsized":"2024-07-04T17:29:06.778745+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:33:48.215079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.777572+0000","last_change":"2024-07-04T17:29:05.803716+0000","last_active":"2024-07-04T17:29:06.777572+0000","last_peered":"2024-07-04T17:29:06.777572+0000","last_clean":"2024-07-04T17:29:06.777572+0000","last_became_active":"2024-07-04T17:29:05.803425+0000","last_became_peered":"2024-07-04T17:29:05.803425+0000","last_unstale":"2024-07-04T17:29:06.777572+0000","last_undegraded":"2024-07-04T17:29:06.777572+0000","last_fullsized":"2024-07-04T17:29:06.777572+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:51:34.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.785750+0000","last_change":"2024-07-04T17:29:05.802356+0000","last_active":"2024-07-04T17:29:06.785750+0000","last_peered":"2024-07-04T17:29:06.785750+0000","last_clean":"2024-07-04T17:29:06.785750+0000","last_became_active":"2024-07-04T17:29:05.798614+0000","last_became_peered":"2024-07-04T17:29:05.798614+0000","last_unstale":"2024-07-04T17:29:06.785750+0000","last_undegraded":"2024-07-04T17:29:06.785750+0000","last_fullsized":"2024-07-04T17:29:06.785750+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:33:04.621287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.496012+0000","last_change":"2024-07-04T17:29:05.804142+0000","last_active":"2024-07-04T17:29:08.496012+0000","last_peered":"2024-07-04T17:29:08.496012+0000","last_clean":"2024-07-04T17:29:08.496012+0000","last_became_active":"2024-07-04T17:29:05.799194+0000","last_became_peered":"2024-07-04T17:29:05.799194+0000","last_unstale":"2024-07-04T17:29:08.496012+0000","last_undegraded":"2024-07-04T17:29:08.496012+0000","last_fullsized":"2024-07-04T17:29:08.496012+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:44:51.271924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.495860+0000","last_change":"2024-07-04T17:29:05.815840+0000","last_active":"2024-07-04T17:29:08.495860+0000","last_peered":"2024-07-04T17:29:08.495860+0000","last_clean":"2024-07-04T17:29:08.495860+0000","last_became_active":"2024-07-04T17:29:05.810912+0000","last_became_peered":"2024-07-04T17:29:05.810912+0000","last_unstale":"2024-07-04T17:29:08.495860+0000","last_undegraded":"2024-07-04T17:29:08.495860+0000","last_fullsized":"2024-07-04T17:29:08.495860+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:31:51.703656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.495566+0000","last_change":"2024-07-04T17:29:05.785152+0000","last_active":"2024-07-04T17:29:08.495566+0000","last_peered":"2024-07-04T17:29:08.495566+0000","last_clean":"2024-07-04T17:29:08.495566+0000","last_became_active":"2024-07-04T17:29:05.784449+0000","last_became_peered":"2024-07-04T17:29:05.784449+0000","last_unstale":"2024-07-04T17:29:08.495566+0000","last_undegraded":"2024-07-04T17:29:08.495566+0000","last_fullsized":"2024-07-04T17:29:08.495566+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:35:30.393302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.493910+0000","last_change":"2024-07-04T17:29:05.823239+0000","last_active":"2024-07-04T17:29:08.493910+0000","last_peered":"2024-07-04T17:29:08.493910+0000","last_clean":"2024-07-04T17:29:08.493910+0000","last_became_active":"2024-07-04T17:29:05.814164+0000","last_became_peered":"2024-07-04T17:29:05.814164+0000","last_unstale":"2024-07-04T17:29:08.493910+0000","last_undegraded":"2024-07-04T17:29:08.493910+0000","last_fullsized":"2024-07-04T17:29:08.493910+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:03:20.072994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.778454+0000","last_change":"2024-07-04T17:29:05.796770+0000","last_active":"2024-07-04T17:29:06.778454+0000","last_peered":"2024-07-04T17:29:06.778454+0000","last_clean":"2024-07-04T17:29:06.778454+0000","last_became_active":"2024-07-04T17:29:05.789644+0000","last_became_peered":"2024-07-04T17:29:05.789644+0000","last_unstale":"2024-07-04T17:29:06.778454+0000","last_undegraded":"2024-07-04T17:29:06.778454+0000","last_fullsized":"2024-07-04T17:29:06.778454+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:58:03.269316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.786860+0000","last_change":"2024-07-04T17:29:05.808774+0000","last_active":"2024-07-04T17:29:06.786860+0000","last_peered":"2024-07-04T17:29:06.786860+0000","last_clean":"2024-07-04T17:29:06.786860+0000","last_became_active":"2024-07-04T17:29:05.808459+0000","last_became_peered":"2024-07-04T17:29:05.808459+0000","last_unstale":"2024-07-04T17:29:06.786860+0000","last_undegraded":"2024-07-04T17:29:06.786860+0000","last_fullsized":"2024-07-04T17:29:06.786860+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:13:24.537766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.779503+0000","last_change":"2024-07-04T17:29:05.800412+0000","last_active":"2024-07-04T17:29:06.779503+0000","last_peered":"2024-07-04T17:29:06.779503+0000","last_clean":"2024-07-04T17:29:06.779503+0000","last_became_active":"2024-07-04T17:29:05.799152+0000","last_became_peered":"2024-07-04T17:29:05.799152+0000","last_unstale":"2024-07-04T17:29:06.779503+0000","last_undegraded":"2024-07-04T17:29:06.779503+0000","last_fullsized":"2024-07-04T17:29:06.779503+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:48:07.191680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.776833+0000","last_change":"2024-07-04T17:29:05.795773+0000","last_active":"2024-07-04T17:29:06.776833+0000","last_peered":"2024-07-04T17:29:06.776833+0000","last_clean":"2024-07-04T17:29:06.776833+0000","last_became_active":"2024-07-04T17:29:05.795470+0000","last_became_peered":"2024-07-04T17:29:05.795470+0000","last_unstale":"2024-07-04T17:29:06.776833+0000","last_undegraded":"2024-07-04T17:29:06.776833+0000","last_fullsized":"2024-07-04T17:29:06.776833+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:32:34.271916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.789186+0000","last_change":"2024-07-04T17:29:05.820276+0000","last_active":"2024-07-04T17:29:06.789186+0000","last_peered":"2024-07-04T17:29:06.789186+0000","last_clean":"2024-07-04T17:29:06.789186+0000","last_became_active":"2024-07-04T17:29:05.819959+0000","last_became_peered":"2024-07-04T17:29:05.819959+0000","last_unstale":"2024-07-04T17:29:06.789186+0000","last_undegraded":"2024-07-04T17:29:06.789186+0000","last_fullsized":"2024-07-04T17:29:06.789186+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:50:43.333430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.788642+0000","last_change":"2024-07-04T17:29:05.817114+0000","last_active":"2024-07-04T17:29:06.788642+0000","last_peered":"2024-07-04T17:29:06.788642+0000","last_clean":"2024-07-04T17:29:06.788642+0000","last_became_active":"2024-07-04T17:29:05.816724+0000","last_became_peered":"2024-07-04T17:29:05.816724+0000","last_unstale":"2024-07-04T17:29:06.788642+0000","last_undegraded":"2024-07-04T17:29:06.788642+0000","last_fullsized":"2024-07-04T17:29:06.788642+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:44:51.271924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.788642+0000","last_change":"2024-07-04T17:29:05.816646+0000","last_active":"2024-07-04T17:29:06.788642+0000","last_peered":"2024-07-04T17:29:06.788642+0000","last_clean":"2024-07-04T17:29:06.788642+0000","last_became_active":"2024-07-04T17:29:05.816345+0000","last_became_peered":"2024-07-04T17:29:05.816345+0000","last_unstale":"2024-07-04T17:29:06.788642+0000","last_undegraded":"2024-07-04T17:29:06.788642+0000","last_fullsized":"2024-07-04T17:29:06.788642+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:31:51.703656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.494347+0000","last_change":"2024-07-04T17:29:05.792444+0000","last_active":"2024-07-04T17:29:08.494347+0000","last_peered":"2024-07-04T17:29:08.494347+0000","last_clean":"2024-07-04T17:29:08.494347+0000","last_became_active":"2024-07-04T17:29:05.792058+0000","last_became_peered":"2024-07-04T17:29:05.792058+0000","last_unstale":"2024-07-04T17:29:08.494347+0000","last_undegraded":"2024-07-04T17:29:08.494347+0000","last_fullsized":"2024-07-04T17:29:08.494347+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:13:24.537766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.778960+0000","last_change":"2024-07-04T17:29:05.799399+0000","last_active":"2024-07-04T17:29:06.778960+0000","last_peered":"2024-07-04T17:29:06.778960+0000","last_clean":"2024-07-04T17:29:06.778960+0000","last_became_active":"2024-07-04T17:29:05.789221+0000","last_became_peered":"2024-07-04T17:29:05.789221+0000","last_unstale":"2024-07-04T17:29:06.778960+0000","last_undegraded":"2024-07-04T17:29:06.778960+0000","last_fullsized":"2024-07-04T17:29:06.778960+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:23:53.530646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.787390+0000","last_change":"2024-07-04T17:29:05.813751+0000","last_active":"2024-07-04T17:29:06.787390+0000","last_peered":"2024-07-04T17:29:06.787390+0000","last_clean":"2024-07-04T17:29:06.787390+0000","last_became_active":"2024-07-04T17:29:05.813362+0000","last_became_peered":"2024-07-04T17:29:05.813362+0000","last_unstale":"2024-07-04T17:29:06.787390+0000","last_undegraded":"2024-07-04T17:29:06.787390+0000","last_fullsized":"2024-07-04T17:29:06.787390+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:58:35.184348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.493677+0000","last_change":"2024-07-04T17:29:05.809788+0000","last_active":"2024-07-04T17:29:08.493677+0000","last_peered":"2024-07-04T17:29:08.493677+0000","last_clean":"2024-07-04T17:29:08.493677+0000","last_became_active":"2024-07-04T17:29:05.809201+0000","last_became_peered":"2024-07-04T17:29:05.809201+0000","last_unstale":"2024-07-04T17:29:08.493677+0000","last_undegraded":"2024-07-04T17:29:08.493677+0000","last_fullsized":"2024-07-04T17:29:08.493677+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:48:26.486742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.493137+0000","last_change":"2024-07-04T17:29:05.820075+0000","last_active":"2024-07-04T17:29:08.493137+0000","last_peered":"2024-07-04T17:29:08.493137+0000","last_clean":"2024-07-04T17:29:08.493137+0000","last_became_active":"2024-07-04T17:29:05.819738+0000","last_became_peered":"2024-07-04T17:29:05.819738+0000","last_unstale":"2024-07-04T17:29:08.493137+0000","last_undegraded":"2024-07-04T17:29:08.493137+0000","last_fullsized":"2024-07-04T17:29:08.493137+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:47:41.453377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.789805+0000","last_change":"2024-07-04T17:29:05.832254+0000","last_active":"2024-07-04T17:29:06.789805+0000","last_peered":"2024-07-04T17:29:06.789805+0000","last_clean":"2024-07-04T17:29:06.789805+0000","last_became_active":"2024-07-04T17:29:05.830771+0000","last_became_peered":"2024-07-04T17:29:05.830771+0000","last_unstale":"2024-07-04T17:29:06.789805+0000","last_undegraded":"2024-07-04T17:29:06.789805+0000","last_fullsized":"2024-07-04T17:29:06.789805+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:01:45.227217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.779110+0000","last_change":"2024-07-04T17:29:05.787671+0000","last_active":"2024-07-04T17:29:06.779110+0000","last_peered":"2024-07-04T17:29:06.779110+0000","last_clean":"2024-07-04T17:29:06.779110+0000","last_became_active":"2024-07-04T17:29:05.783612+0000","last_became_peered":"2024-07-04T17:29:05.783612+0000","last_unstale":"2024-07-04T17:29:06.779110+0000","last_undegraded":"2024-07-04T17:29:06.779110+0000","last_fullsized":"2024-07-04T17:29:06.779110+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:17:31.565173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.777973+0000","last_change":"2024-07-04T17:29:05.785801+0000","last_active":"2024-07-04T17:29:06.777973+0000","last_peered":"2024-07-04T17:29:06.777973+0000","last_clean":"2024-07-04T17:29:06.777973+0000","last_became_active":"2024-07-04T17:29:05.784924+0000","last_became_peered":"2024-07-04T17:29:05.784924+0000","last_unstale":"2024-07-04T17:29:06.777973+0000","last_undegraded":"2024-07-04T17:29:06.777973+0000","last_fullsized":"2024-07-04T17:29:06.777973+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:52:10.436668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.786933+0000","last_change":"2024-07-04T17:29:05.817096+0000","last_active":"2024-07-04T17:29:06.786933+0000","last_peered":"2024-07-04T17:29:06.786933+0000","last_clean":"2024-07-04T17:29:06.786933+0000","last_became_active":"2024-07-04T17:29:05.816304+0000","last_became_peered":"2024-07-04T17:29:05.816304+0000","last_unstale":"2024-07-04T17:29:06.786933+0000","last_undegraded":"2024-07-04T17:29:06.786933+0000","last_fullsized":"2024-07-04T17:29:06.786933+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:44:09.505235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.786232+0000","last_change":"2024-07-04T17:29:05.825187+0000","last_active":"2024-07-04T17:29:06.786232+0000","last_peered":"2024-07-04T17:29:06.786232+0000","last_clean":"2024-07-04T17:29:06.786232+0000","last_became_active":"2024-07-04T17:29:05.824857+0000","last_became_peered":"2024-07-04T17:29:05.824857+0000","last_unstale":"2024-07-04T17:29:06.786232+0000","last_undegraded":"2024-07-04T17:29:06.786232+0000","last_fullsized":"2024-07-04T17:29:06.786232+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:14:24.515173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.493231+0000","last_change":"2024-07-04T17:29:05.815028+0000","last_active":"2024-07-04T17:29:08.493231+0000","last_peered":"2024-07-04T17:29:08.493231+0000","last_clean":"2024-07-04T17:29:08.493231+0000","last_became_active":"2024-07-04T17:29:05.814094+0000","last_became_peered":"2024-07-04T17:29:05.814094+0000","last_unstale":"2024-07-04T17:29:08.493231+0000","last_undegraded":"2024-07-04T17:29:08.493231+0000","last_fullsized":"2024-07-04T17:29:08.493231+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:48:33.874991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.786110+0000","last_change":"2024-07-04T17:29:05.813654+0000","last_active":"2024-07-04T17:29:06.786110+0000","last_peered":"2024-07-04T17:29:06.786110+0000","last_clean":"2024-07-04T17:29:06.786110+0000","last_became_active":"2024-07-04T17:29:05.813139+0000","last_became_peered":"2024-07-04T17:29:05.813139+0000","last_unstale":"2024-07-04T17:29:06.786110+0000","last_undegraded":"2024-07-04T17:29:06.786110+0000","last_fullsized":"2024-07-04T17:29:06.786110+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:36:15.241406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.777172+0000","last_change":"2024-07-04T17:29:05.799841+0000","last_active":"2024-07-04T17:29:06.777172+0000","last_peered":"2024-07-04T17:29:06.777172+0000","last_clean":"2024-07-04T17:29:06.777172+0000","last_became_active":"2024-07-04T17:29:05.799478+0000","last_became_peered":"2024-07-04T17:29:05.799478+0000","last_unstale":"2024-07-04T17:29:06.777172+0000","last_undegraded":"2024-07-04T17:29:06.777172+0000","last_fullsized":"2024-07-04T17:29:06.777172+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:25:37.804442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.779497+0000","last_change":"2024-07-04T17:29:05.795312+0000","last_active":"2024-07-04T17:29:06.779497+0000","last_peered":"2024-07-04T17:29:06.779497+0000","last_clean":"2024-07-04T17:29:06.779497+0000","last_became_active":"2024-07-04T17:29:05.794446+0000","last_became_peered":"2024-07-04T17:29:05.794446+0000","last_unstale":"2024-07-04T17:29:06.779497+0000","last_undegraded":"2024-07-04T17:29:06.779497+0000","last_fullsized":"2024-07-04T17:29:06.779497+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:54:42.187784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"49'4","reported_seq":18,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.779652+0000","last_change":"2024-07-04T17:29:04.768776+0000","last_active":"2024-07-04T17:29:06.779652+0000","last_peered":"2024-07-04T17:29:06.779652+0000","last_clean":"2024-07-04T17:29:06.779652+0000","last_became_active":"2024-07-04T17:29:04.766498+0000","last_became_peered":"2024-07-04T17:29:04.766498+0000","last_unstale":"2024-07-04T17:29:06.779652+0000","last_undegraded":"2024-07-04T17:29:06.779652+0000","last_fullsized":"2024-07-04T17:29:06.779652+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:03:44.115014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.495718+0000","last_change":"2024-07-04T17:29:05.790122+0000","last_active":"2024-07-04T17:29:08.495718+0000","last_peered":"2024-07-04T17:29:08.495718+0000","last_clean":"2024-07-04T17:29:08.495718+0000","last_became_active":"2024-07-04T17:29:05.789804+0000","last_became_peered":"2024-07-04T17:29:05.789804+0000","last_unstale":"2024-07-04T17:29:08.495718+0000","last_undegraded":"2024-07-04T17:29:08.495718+0000","last_fullsized":"2024-07-04T17:29:08.495718+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:06:24.983072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.495869+0000","last_change":"2024-07-04T17:29:04.779844+0000","last_active":"2024-07-04T17:29:08.495869+0000","last_peered":"2024-07-04T17:29:08.495869+0000","last_clean":"2024-07-04T17:29:08.495869+0000","last_became_active":"2024-07-04T17:29:04.779212+0000","last_became_peered":"2024-07-04T17:29:04.779212+0000","last_unstale":"2024-07-04T17:29:08.495869+0000","last_undegraded":"2024-07-04T17:29:08.495869+0000","last_fullsized":"2024-07-04T17:29:08.495869+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:06:45.499488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.780222+0000","last_change":"2024-07-04T17:29:05.799208+0000","last_active":"2024-07-04T17:29:06.780222+0000","last_peered":"2024-07-04T17:29:06.780222+0000","last_clean":"2024-07-04T17:29:06.780222+0000","last_became_active":"2024-07-04T17:29:05.796533+0000","last_became_peered":"2024-07-04T17:29:05.796533+0000","last_unstale":"2024-07-04T17:29:06.780222+0000","last_undegraded":"2024-07-04T17:29:06.780222+0000","last_fullsized":"2024-07-04T17:29:06.780222+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:28:33.331115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.778686+0000","last_change":"2024-07-04T17:29:04.767288+0000","last_active":"2024-07-04T17:29:06.778686+0000","last_peered":"2024-07-04T17:29:06.778686+0000","last_clean":"2024-07-04T17:29:06.778686+0000","last_became_active":"2024-07-04T17:29:04.765876+0000","last_became_peered":"2024-07-04T17:29:04.765876+0000","last_unstale":"2024-07-04T17:29:06.778686+0000","last_undegraded":"2024-07-04T17:29:06.778686+0000","last_fullsized":"2024-07-04T17:29:06.778686+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:54:49.684096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.778162+0000","last_change":"2024-07-04T17:29:05.785226+0000","last_active":"2024-07-04T17:29:06.778162+0000","last_peered":"2024-07-04T17:29:06.778162+0000","last_clean":"2024-07-04T17:29:06.778162+0000","last_became_active":"2024-07-04T17:29:05.784069+0000","last_became_peered":"2024-07-04T17:29:05.784069+0000","last_unstale":"2024-07-04T17:29:06.778162+0000","last_undegraded":"2024-07-04T17:29:06.778162+0000","last_fullsized":"2024-07-04T17:29:06.778162+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:27:43.064532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.787982+0000","last_change":"2024-07-04T17:29:04.793187+0000","last_active":"2024-07-04T17:29:06.787982+0000","last_peered":"2024-07-04T17:29:06.787982+0000","last_clean":"2024-07-04T17:29:06.787982+0000","last_became_active":"2024-07-04T17:29:04.792877+0000","last_became_peered":"2024-07-04T17:29:04.792877+0000","last_unstale":"2024-07-04T17:29:06.787982+0000","last_undegraded":"2024-07-04T17:29:06.787982+0000","last_fullsized":"2024-07-04T17:29:06.787982+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:16:45.088529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.494440+0000","last_change":"2024-07-04T17:29:05.796869+0000","last_active":"2024-07-04T17:29:08.494440+0000","last_peered":"2024-07-04T17:29:08.494440+0000","last_clean":"2024-07-04T17:29:08.494440+0000","last_became_active":"2024-07-04T17:29:05.796573+0000","last_became_peered":"2024-07-04T17:29:05.796573+0000","last_unstale":"2024-07-04T17:29:08.494440+0000","last_undegraded":"2024-07-04T17:29:08.494440+0000","last_fullsized":"2024-07-04T17:29:08.494440+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:41:01.850387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1b","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.786620+0000","last_change":"2024-07-04T17:29:04.793835+0000","last_active":"2024-07-04T17:29:06.786620+0000","last_peered":"2024-07-04T17:29:06.786620+0000","last_clean":"2024-07-04T17:29:06.786620+0000","last_became_active":"2024-07-04T17:29:04.792785+0000","last_became_peered":"2024-07-04T17:29:04.792785+0000","last_unstale":"2024-07-04T17:29:06.786620+0000","last_undegraded":"2024-07-04T17:29:06.786620+0000","last_fullsized":"2024-07-04T17:29:06.786620+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:21:48.040633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.779463+0000","last_change":"2024-07-04T17:29:05.796490+0000","last_active":"2024-07-04T17:29:06.779463+0000","last_peered":"2024-07-04T17:29:06.779463+0000","last_clean":"2024-07-04T17:29:06.779463+0000","last_became_active":"2024-07-04T17:29:05.788778+0000","last_became_peered":"2024-07-04T17:29:05.788778+0000","last_unstale":"2024-07-04T17:29:06.779463+0000","last_undegraded":"2024-07-04T17:29:06.779463+0000","last_fullsized":"2024-07-04T17:29:06.779463+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:23:59.134121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1a","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.779091+0000","last_change":"2024-07-04T17:29:04.756095+0000","last_active":"2024-07-04T17:29:06.779091+0000","last_peered":"2024-07-04T17:29:06.779091+0000","last_clean":"2024-07-04T17:29:06.779091+0000","last_became_active":"2024-07-04T17:29:04.755679+0000","last_became_peered":"2024-07-04T17:29:04.755679+0000","last_unstale":"2024-07-04T17:29:06.779091+0000","last_undegraded":"2024-07-04T17:29:06.779091+0000","last_fullsized":"2024-07-04T17:29:06.779091+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:30:46.500676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.786319+0000","last_change":"2024-07-04T17:29:05.809480+0000","last_active":"2024-07-04T17:29:06.786319+0000","last_peered":"2024-07-04T17:29:06.786319+0000","last_clean":"2024-07-04T17:29:06.786319+0000","last_became_active":"2024-07-04T17:29:05.809138+0000","last_became_peered":"2024-07-04T17:29:05.809138+0000","last_unstale":"2024-07-04T17:29:06.786319+0000","last_undegraded":"2024-07-04T17:29:06.786319+0000","last_fullsized":"2024-07-04T17:29:06.786319+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:03:20.072994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.493322+0000","last_change":"2024-07-04T17:29:04.776840+0000","last_active":"2024-07-04T17:29:08.493322+0000","last_peered":"2024-07-04T17:29:08.493322+0000","last_clean":"2024-07-04T17:29:08.493322+0000","last_became_active":"2024-07-04T17:29:04.775729+0000","last_became_peered":"2024-07-04T17:29:04.775729+0000","last_unstale":"2024-07-04T17:29:08.493322+0000","last_undegraded":"2024-07-04T17:29:08.493322+0000","last_fullsized":"2024-07-04T17:29:08.493322+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:28:41.894029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.786904+0000","last_change":"2024-07-04T17:29:05.818108+0000","last_active":"2024-07-04T17:29:06.786904+0000","last_peered":"2024-07-04T17:29:06.786904+0000","last_clean":"2024-07-04T17:29:06.786904+0000","last_became_active":"2024-07-04T17:29:05.816963+0000","last_became_peered":"2024-07-04T17:29:05.816963+0000","last_unstale":"2024-07-04T17:29:06.786904+0000","last_undegraded":"2024-07-04T17:29:06.786904+0000","last_fullsized":"2024-07-04T17:29:06.786904+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:07:08.833691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.777125+0000","last_change":"2024-07-04T17:29:04.774790+0000","last_active":"2024-07-04T17:29:06.777125+0000","last_peered":"2024-07-04T17:29:06.777125+0000","last_clean":"2024-07-04T17:29:06.777125+0000","last_became_active":"2024-07-04T17:29:04.774427+0000","last_became_peered":"2024-07-04T17:29:04.774427+0000","last_unstale":"2024-07-04T17:29:06.777125+0000","last_undegraded":"2024-07-04T17:29:06.777125+0000","last_fullsized":"2024-07-04T17:29:06.777125+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:28:48.914599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.779685+0000","last_change":"2024-07-04T17:29:05.795423+0000","last_active":"2024-07-04T17:29:06.779685+0000","last_peered":"2024-07-04T17:29:06.779685+0000","last_clean":"2024-07-04T17:29:06.779685+0000","last_became_active":"2024-07-04T17:29:05.794868+0000","last_became_peered":"2024-07-04T17:29:05.794868+0000","last_unstale":"2024-07-04T17:29:06.779685+0000","last_undegraded":"2024-07-04T17:29:06.779685+0000","last_fullsized":"2024-07-04T17:29:06.779685+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:36:01.857174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.496077+0000","last_change":"2024-07-04T17:29:04.772998+0000","last_active":"2024-07-04T17:29:08.496077+0000","last_peered":"2024-07-04T17:29:08.496077+0000","last_clean":"2024-07-04T17:29:08.496077+0000","last_became_active":"2024-07-04T17:29:04.772635+0000","last_became_peered":"2024-07-04T17:29:04.772635+0000","last_unstale":"2024-07-04T17:29:08.496077+0000","last_undegraded":"2024-07-04T17:29:08.496077+0000","last_fullsized":"2024-07-04T17:29:08.496077+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:59:48.689604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.789520+0000","last_change":"2024-07-04T17:29:05.832809+0000","last_active":"2024-07-04T17:29:06.789520+0000","last_peered":"2024-07-04T17:29:06.789520+0000","last_clean":"2024-07-04T17:29:06.789520+0000","last_became_active":"2024-07-04T17:29:05.829582+0000","last_became_peered":"2024-07-04T17:29:05.829582+0000","last_unstale":"2024-07-04T17:29:06.789520+0000","last_undegraded":"2024-07-04T17:29:06.789520+0000","last_fullsized":"2024-07-04T17:29:06.789520+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:50:42.737526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.496204+0000","last_change":"2024-07-04T17:29:04.780571+0000","last_active":"2024-07-04T17:29:08.496204+0000","last_peered":"2024-07-04T17:29:08.496204+0000","last_clean":"2024-07-04T17:29:08.496204+0000","last_became_active":"2024-07-04T17:29:04.780236+0000","last_became_peered":"2024-07-04T17:29:04.780236+0000","last_unstale":"2024-07-04T17:29:08.496204+0000","last_undegraded":"2024-07-04T17:29:08.496204+0000","last_fullsized":"2024-07-04T17:29:08.496204+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:55:21.724770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.494460+0000","last_change":"2024-07-04T17:29:05.783421+0000","last_active":"2024-07-04T17:29:08.494460+0000","last_peered":"2024-07-04T17:29:08.494460+0000","last_clean":"2024-07-04T17:29:08.494460+0000","last_became_active":"2024-07-04T17:29:05.783057+0000","last_became_peered":"2024-07-04T17:29:05.783057+0000","last_unstale":"2024-07-04T17:29:08.494460+0000","last_undegraded":"2024-07-04T17:29:08.494460+0000","last_fullsized":"2024-07-04T17:29:08.494460+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:44:09.505235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"49'3","reported_seq":17,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.789045+0000","last_change":"2024-07-04T17:29:04.794454+0000","last_active":"2024-07-04T17:29:06.789045+0000","last_peered":"2024-07-04T17:29:06.789045+0000","last_clean":"2024-07-04T17:29:06.789045+0000","last_became_active":"2024-07-04T17:29:04.793939+0000","last_became_peered":"2024-07-04T17:29:04.793939+0000","last_unstale":"2024-07-04T17:29:06.789045+0000","last_undegraded":"2024-07-04T17:29:06.789045+0000","last_fullsized":"2024-07-04T17:29:06.789045+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:10:23.616021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.495405+0000","last_change":"2024-07-04T17:29:05.792689+0000","last_active":"2024-07-04T17:29:08.495405+0000","last_peered":"2024-07-04T17:29:08.495405+0000","last_clean":"2024-07-04T17:29:08.495405+0000","last_became_active":"2024-07-04T17:29:05.792368+0000","last_became_peered":"2024-07-04T17:29:05.792368+0000","last_unstale":"2024-07-04T17:29:08.495405+0000","last_undegraded":"2024-07-04T17:29:08.495405+0000","last_fullsized":"2024-07-04T17:29:08.495405+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:58:35.184348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.14","version":"49'6","reported_seq":20,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.780205+0000","last_change":"2024-07-04T17:29:04.778462+0000","last_active":"2024-07-04T17:29:06.780205+0000","last_peered":"2024-07-04T17:29:06.780205+0000","last_clean":"2024-07-04T17:29:06.780205+0000","last_became_active":"2024-07-04T17:29:04.775998+0000","last_became_peered":"2024-07-04T17:29:04.775998+0000","last_unstale":"2024-07-04T17:29:06.780205+0000","last_undegraded":"2024-07-04T17:29:06.780205+0000","last_fullsized":"2024-07-04T17:29:06.780205+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:16:55.504943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.493827+0000","last_change":"2024-07-04T17:29:05.815269+0000","last_active":"2024-07-04T17:29:08.493827+0000","last_peered":"2024-07-04T17:29:08.493827+0000","last_clean":"2024-07-04T17:29:08.493827+0000","last_became_active":"2024-07-04T17:29:05.814468+0000","last_became_peered":"2024-07-04T17:29:05.814468+0000","last_unstale":"2024-07-04T17:29:08.493827+0000","last_undegraded":"2024-07-04T17:29:08.493827+0000","last_fullsized":"2024-07-04T17:29:08.493827+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:14:24.515173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.493881+0000","last_change":"2024-07-04T17:29:04.782215+0000","last_active":"2024-07-04T17:29:08.493881+0000","last_peered":"2024-07-04T17:29:08.493881+0000","last_clean":"2024-07-04T17:29:08.493881+0000","last_became_active":"2024-07-04T17:29:04.781891+0000","last_became_peered":"2024-07-04T17:29:04.781891+0000","last_unstale":"2024-07-04T17:29:08.493881+0000","last_undegraded":"2024-07-04T17:29:08.493881+0000","last_fullsized":"2024-07-04T17:29:08.493881+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:56:44.314464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.786924+0000","last_change":"2024-07-04T17:29:05.830879+0000","last_active":"2024-07-04T17:29:06.786924+0000","last_peered":"2024-07-04T17:29:06.786924+0000","last_clean":"2024-07-04T17:29:06.786924+0000","last_became_active":"2024-07-04T17:29:05.830572+0000","last_became_peered":"2024-07-04T17:29:05.830572+0000","last_unstale":"2024-07-04T17:29:06.786924+0000","last_undegraded":"2024-07-04T17:29:06.786924+0000","last_fullsized":"2024-07-04T17:29:06.786924+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:41:01.850387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.787695+0000","last_change":"2024-07-04T17:29:04.793440+0000","last_active":"2024-07-04T17:29:06.787695+0000","last_peered":"2024-07-04T17:29:06.787695+0000","last_clean":"2024-07-04T17:29:06.787695+0000","last_became_active":"2024-07-04T17:29:04.792810+0000","last_became_peered":"2024-07-04T17:29:04.792810+0000","last_unstale":"2024-07-04T17:29:06.787695+0000","last_undegraded":"2024-07-04T17:29:06.787695+0000","last_fullsized":"2024-07-04T17:29:06.787695+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:28:41.894029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.493534+0000","last_change":"2024-07-04T17:29:05.822812+0000","last_active":"2024-07-04T17:29:08.493534+0000","last_peered":"2024-07-04T17:29:08.493534+0000","last_clean":"2024-07-04T17:29:08.493534+0000","last_became_active":"2024-07-04T17:29:05.822519+0000","last_became_peered":"2024-07-04T17:29:05.822519+0000","last_unstale":"2024-07-04T17:29:08.493534+0000","last_undegraded":"2024-07-04T17:29:08.493534+0000","last_fullsized":"2024-07-04T17:29:08.493534+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:33:04.621287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.787191+0000","last_change":"2024-07-04T17:29:04.793535+0000","last_active":"2024-07-04T17:29:06.787191+0000","last_peered":"2024-07-04T17:29:06.787191+0000","last_clean":"2024-07-04T17:29:06.787191+0000","last_became_active":"2024-07-04T17:29:04.793160+0000","last_became_peered":"2024-07-04T17:29:04.793160+0000","last_unstale":"2024-07-04T17:29:06.787191+0000","last_undegraded":"2024-07-04T17:29:06.787191+0000","last_fullsized":"2024-07-04T17:29:06.787191+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:29:48.755500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.780421+0000","last_change":"2024-07-04T17:29:05.793691+0000","last_active":"2024-07-04T17:29:06.780421+0000","last_peered":"2024-07-04T17:29:06.780421+0000","last_clean":"2024-07-04T17:29:06.780421+0000","last_became_active":"2024-07-04T17:29:05.792767+0000","last_became_peered":"2024-07-04T17:29:05.792767+0000","last_unstale":"2024-07-04T17:29:06.780421+0000","last_undegraded":"2024-07-04T17:29:06.780421+0000","last_fullsized":"2024-07-04T17:29:06.780421+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:25:53.656021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.10","version":"49'8","reported_seq":22,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.777268+0000","last_change":"2024-07-04T17:29:04.767026+0000","last_active":"2024-07-04T17:29:06.777268+0000","last_peered":"2024-07-04T17:29:06.777268+0000","last_clean":"2024-07-04T17:29:06.777268+0000","last_became_active":"2024-07-04T17:29:04.764888+0000","last_became_peered":"2024-07-04T17:29:04.764888+0000","last_unstale":"2024-07-04T17:29:06.777268+0000","last_undegraded":"2024-07-04T17:29:06.777268+0000","last_fullsized":"2024-07-04T17:29:06.777268+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:08:09.031069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.788757+0000","last_change":"2024-07-04T17:29:05.826630+0000","last_active":"2024-07-04T17:29:06.788757+0000","last_peered":"2024-07-04T17:29:06.788757+0000","last_clean":"2024-07-04T17:29:06.788757+0000","last_became_active":"2024-07-04T17:29:05.826180+0000","last_became_peered":"2024-07-04T17:29:05.826180+0000","last_unstale":"2024-07-04T17:29:06.788757+0000","last_undegraded":"2024-07-04T17:29:06.788757+0000","last_fullsized":"2024-07-04T17:29:06.788757+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:42:34.680103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.779700+0000","last_change":"2024-07-04T17:29:04.778986+0000","last_active":"2024-07-04T17:29:06.779700+0000","last_peered":"2024-07-04T17:29:06.779700+0000","last_clean":"2024-07-04T17:29:06.779700+0000","last_became_active":"2024-07-04T17:29:04.775326+0000","last_became_peered":"2024-07-04T17:29:04.775326+0000","last_unstale":"2024-07-04T17:29:06.779700+0000","last_undegraded":"2024-07-04T17:29:06.779700+0000","last_fullsized":"2024-07-04T17:29:06.779700+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:01:44.218523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.789569+0000","last_change":"2024-07-04T17:29:05.825845+0000","last_active":"2024-07-04T17:29:06.789569+0000","last_peered":"2024-07-04T17:29:06.789569+0000","last_clean":"2024-07-04T17:29:06.789569+0000","last_became_active":"2024-07-04T17:29:05.824923+0000","last_became_peered":"2024-07-04T17:29:05.824923+0000","last_unstale":"2024-07-04T17:29:06.789569+0000","last_undegraded":"2024-07-04T17:29:06.789569+0000","last_fullsized":"2024-07-04T17:29:06.789569+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:30:47.509371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.779858+0000","last_change":"2024-07-04T17:29:04.787322+0000","last_active":"2024-07-04T17:29:06.779858+0000","last_peered":"2024-07-04T17:29:06.779858+0000","last_clean":"2024-07-04T17:29:06.779858+0000","last_became_active":"2024-07-04T17:29:04.786939+0000","last_became_peered":"2024-07-04T17:29:04.786939+0000","last_unstale":"2024-07-04T17:29:06.779858+0000","last_undegraded":"2024-07-04T17:29:06.779858+0000","last_fullsized":"2024-07-04T17:29:06.779858+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:31:08.982204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.787421+0000","last_change":"2024-07-04T17:29:05.817332+0000","last_active":"2024-07-04T17:29:06.787421+0000","last_peered":"2024-07-04T17:29:06.787421+0000","last_clean":"2024-07-04T17:29:06.787421+0000","last_became_active":"2024-07-04T17:29:05.816980+0000","last_became_peered":"2024-07-04T17:29:05.816980+0000","last_unstale":"2024-07-04T17:29:06.787421+0000","last_undegraded":"2024-07-04T17:29:06.787421+0000","last_fullsized":"2024-07-04T17:29:06.787421+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:35:30.393302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"49'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.789405+0000","last_change":"2024-07-04T17:29:04.793950+0000","last_active":"2024-07-04T17:29:06.789405+0000","last_peered":"2024-07-04T17:29:06.789405+0000","last_clean":"2024-07-04T17:29:06.789405+0000","last_became_active":"2024-07-04T17:29:04.793520+0000","last_became_peered":"2024-07-04T17:29:04.793520+0000","last_unstale":"2024-07-04T17:29:06.789405+0000","last_undegraded":"2024-07-04T17:29:06.789405+0000","last_fullsized":"2024-07-04T17:29:06.789405+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:43:57.308367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.780680+0000","last_change":"2024-07-04T17:29:05.796377+0000","last_active":"2024-07-04T17:29:06.780680+0000","last_peered":"2024-07-04T17:29:06.780680+0000","last_clean":"2024-07-04T17:29:06.780680+0000","last_became_active":"2024-07-04T17:29:05.796069+0000","last_became_peered":"2024-07-04T17:29:05.796069+0000","last_unstale":"2024-07-04T17:29:06.780680+0000","last_undegraded":"2024-07-04T17:29:06.780680+0000","last_fullsized":"2024-07-04T17:29:06.780680+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:36:28.462582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.780553+0000","last_change":"2024-07-04T17:29:04.775241+0000","last_active":"2024-07-04T17:29:06.780553+0000","last_peered":"2024-07-04T17:29:06.780553+0000","last_clean":"2024-07-04T17:29:06.780553+0000","last_became_active":"2024-07-04T17:29:04.774292+0000","last_became_peered":"2024-07-04T17:29:04.774292+0000","last_unstale":"2024-07-04T17:29:06.780553+0000","last_undegraded":"2024-07-04T17:29:06.780553+0000","last_fullsized":"2024-07-04T17:29:06.780553+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:48:39.886887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.494364+0000","last_change":"2024-07-04T17:29:05.814789+0000","last_active":"2024-07-04T17:29:08.494364+0000","last_peered":"2024-07-04T17:29:08.494364+0000","last_clean":"2024-07-04T17:29:08.494364+0000","last_became_active":"2024-07-04T17:29:05.813725+0000","last_became_peered":"2024-07-04T17:29:05.813725+0000","last_unstale":"2024-07-04T17:29:08.494364+0000","last_undegraded":"2024-07-04T17:29:08.494364+0000","last_fullsized":"2024-07-04T17:29:08.494364+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:07:08.833691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.780830+0000","last_change":"2024-07-04T17:29:04.768448+0000","last_active":"2024-07-04T17:29:06.780830+0000","last_peered":"2024-07-04T17:29:06.780830+0000","last_clean":"2024-07-04T17:29:06.780830+0000","last_became_active":"2024-07-04T17:29:04.765316+0000","last_became_peered":"2024-07-04T17:29:04.765316+0000","last_unstale":"2024-07-04T17:29:06.780830+0000","last_undegraded":"2024-07-04T17:29:06.780830+0000","last_fullsized":"2024-07-04T17:29:06.780830+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:54:53.983424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.779961+0000","last_change":"2024-07-04T17:29:05.784868+0000","last_active":"2024-07-04T17:29:06.779961+0000","last_peered":"2024-07-04T17:29:06.779961+0000","last_clean":"2024-07-04T17:29:06.779961+0000","last_became_active":"2024-07-04T17:29:05.784419+0000","last_became_peered":"2024-07-04T17:29:05.784419+0000","last_unstale":"2024-07-04T17:29:06.779961+0000","last_undegraded":"2024-07-04T17:29:06.779961+0000","last_fullsized":"2024-07-04T17:29:06.779961+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:33:47.592614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.495868+0000","last_change":"2024-07-04T17:29:04.780522+0000","last_active":"2024-07-04T17:29:08.495868+0000","last_peered":"2024-07-04T17:29:08.495868+0000","last_clean":"2024-07-04T17:29:08.495868+0000","last_became_active":"2024-07-04T17:29:04.780161+0000","last_became_peered":"2024-07-04T17:29:04.780161+0000","last_unstale":"2024-07-04T17:29:08.495868+0000","last_undegraded":"2024-07-04T17:29:08.495868+0000","last_fullsized":"2024-07-04T17:29:08.495868+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:29:48.755500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.780310+0000","last_change":"2024-07-04T17:29:05.793437+0000","last_active":"2024-07-04T17:29:06.780310+0000","last_peered":"2024-07-04T17:29:06.780310+0000","last_clean":"2024-07-04T17:29:06.780310+0000","last_became_active":"2024-07-04T17:29:05.792368+0000","last_became_peered":"2024-07-04T17:29:05.792368+0000","last_unstale":"2024-07-04T17:29:06.780310+0000","last_undegraded":"2024-07-04T17:29:06.780310+0000","last_fullsized":"2024-07-04T17:29:06.780310+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:17:35.033747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.494033+0000","last_change":"2024-07-04T17:29:04.774550+0000","last_active":"2024-07-04T17:29:08.494033+0000","last_peered":"2024-07-04T17:29:08.494033+0000","last_clean":"2024-07-04T17:29:08.494033+0000","last_became_active":"2024-07-04T17:29:04.774168+0000","last_became_peered":"2024-07-04T17:29:04.774168+0000","last_unstale":"2024-07-04T17:29:08.494033+0000","last_undegraded":"2024-07-04T17:29:08.494033+0000","last_fullsized":"2024-07-04T17:29:08.494033+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:43:57.308367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.788825+0000","last_change":"2024-07-04T17:29:05.826827+0000","last_active":"2024-07-04T17:29:06.788825+0000","last_peered":"2024-07-04T17:29:06.788825+0000","last_clean":"2024-07-04T17:29:06.788825+0000","last_became_active":"2024-07-04T17:29:05.825730+0000","last_became_peered":"2024-07-04T17:29:05.825730+0000","last_unstale":"2024-07-04T17:29:06.788825+0000","last_undegraded":"2024-07-04T17:29:06.788825+0000","last_fullsized":"2024-07-04T17:29:06.788825+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:14:31.441713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"49'2","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.496220+0000","last_change":"2024-07-04T17:29:04.780892+0000","last_active":"2024-07-04T17:29:08.496220+0000","last_peered":"2024-07-04T17:29:08.496220+0000","last_clean":"2024-07-04T17:29:08.496220+0000","last_became_active":"2024-07-04T17:29:04.780559+0000","last_became_peered":"2024-07-04T17:29:04.780559+0000","last_unstale":"2024-07-04T17:29:08.496220+0000","last_undegraded":"2024-07-04T17:29:08.496220+0000","last_fullsized":"2024-07-04T17:29:08.496220+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:46:37.056095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.789765+0000","last_change":"2024-07-04T17:29:05.831979+0000","last_active":"2024-07-04T17:29:06.789765+0000","last_peered":"2024-07-04T17:29:06.789765+0000","last_clean":"2024-07-04T17:29:06.789765+0000","last_became_active":"2024-07-04T17:29:05.829983+0000","last_became_peered":"2024-07-04T17:29:05.829983+0000","last_unstale":"2024-07-04T17:29:06.789765+0000","last_undegraded":"2024-07-04T17:29:06.789765+0000","last_fullsized":"2024-07-04T17:29:06.789765+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:03:45.123708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"50'1","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.783682+0000","last_change":"2024-07-04T17:29:04.784413+0000","last_active":"2024-07-04T17:29:06.783682+0000","last_peered":"2024-07-04T17:29:06.783682+0000","last_clean":"2024-07-04T17:29:06.783682+0000","last_became_active":"2024-07-04T17:29:04.784058+0000","last_became_peered":"2024-07-04T17:29:04.784058+0000","last_unstale":"2024-07-04T17:29:06.783682+0000","last_undegraded":"2024-07-04T17:29:06.783682+0000","last_fullsized":"2024-07-04T17:29:06.783682+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:35:32.227491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.495077+0000","last_change":"2024-07-04T17:29:05.821127+0000","last_active":"2024-07-04T17:29:08.495077+0000","last_peered":"2024-07-04T17:29:08.495077+0000","last_clean":"2024-07-04T17:29:08.495077+0000","last_became_active":"2024-07-04T17:29:05.820787+0000","last_became_peered":"2024-07-04T17:29:05.820787+0000","last_unstale":"2024-07-04T17:29:08.495077+0000","last_undegraded":"2024-07-04T17:29:08.495077+0000","last_fullsized":"2024-07-04T17:29:08.495077+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:33:26.836346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.495846+0000","last_change":"2024-07-04T17:29:04.786173+0000","last_active":"2024-07-04T17:29:08.495846+0000","last_peered":"2024-07-04T17:29:08.495846+0000","last_clean":"2024-07-04T17:29:08.495846+0000","last_became_active":"2024-07-04T17:29:04.783241+0000","last_became_peered":"2024-07-04T17:29:04.783241+0000","last_unstale":"2024-07-04T17:29:08.495846+0000","last_undegraded":"2024-07-04T17:29:08.495846+0000","last_fullsized":"2024-07-04T17:29:08.495846+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:16:45.088529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.778525+0000","last_change":"2024-07-04T17:29:05.785545+0000","last_active":"2024-07-04T17:29:06.778525+0000","last_peered":"2024-07-04T17:29:06.778525+0000","last_clean":"2024-07-04T17:29:06.778525+0000","last_became_active":"2024-07-04T17:29:05.784520+0000","last_became_peered":"2024-07-04T17:29:05.784520+0000","last_unstale":"2024-07-04T17:29:06.778525+0000","last_undegraded":"2024-07-04T17:29:06.778525+0000","last_fullsized":"2024-07-04T17:29:06.778525+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:31:15.745622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.787945+0000","last_change":"2024-07-04T17:29:04.794330+0000","last_active":"2024-07-04T17:29:06.787945+0000","last_peered":"2024-07-04T17:29:06.787945+0000","last_clean":"2024-07-04T17:29:06.787945+0000","last_became_active":"2024-07-04T17:29:04.793980+0000","last_became_peered":"2024-07-04T17:29:04.793980+0000","last_unstale":"2024-07-04T17:29:06.787945+0000","last_undegraded":"2024-07-04T17:29:06.787945+0000","last_fullsized":"2024-07-04T17:29:06.787945+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:06:13.018475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.495701+0000","last_change":"2024-07-04T17:29:05.802494+0000","last_active":"2024-07-04T17:29:08.495701+0000","last_peered":"2024-07-04T17:29:08.495701+0000","last_clean":"2024-07-04T17:29:08.495701+0000","last_became_active":"2024-07-04T17:29:05.795642+0000","last_became_peered":"2024-07-04T17:29:05.795642+0000","last_unstale":"2024-07-04T17:29:08.495701+0000","last_undegraded":"2024-07-04T17:29:08.495701+0000","last_fullsized":"2024-07-04T17:29:08.495701+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:19:50.013783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.494710+0000","last_change":"2024-07-04T17:29:04.774431+0000","last_active":"2024-07-04T17:29:08.494710+0000","last_peered":"2024-07-04T17:29:08.494710+0000","last_clean":"2024-07-04T17:29:08.494710+0000","last_became_active":"2024-07-04T17:29:04.774034+0000","last_became_peered":"2024-07-04T17:29:04.774034+0000","last_unstale":"2024-07-04T17:29:08.494710+0000","last_undegraded":"2024-07-04T17:29:08.494710+0000","last_fullsized":"2024-07-04T17:29:08.494710+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:10:23.616021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.780478+0000","last_change":"2024-07-04T17:29:05.784396+0000","last_active":"2024-07-04T17:29:06.780478+0000","last_peered":"2024-07-04T17:29:06.780478+0000","last_clean":"2024-07-04T17:29:06.780478+0000","last_became_active":"2024-07-04T17:29:05.783953+0000","last_became_peered":"2024-07-04T17:29:05.783953+0000","last_unstale":"2024-07-04T17:29:06.780478+0000","last_undegraded":"2024-07-04T17:29:06.780478+0000","last_fullsized":"2024-07-04T17:29:06.780478+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:53:09.757420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.787796+0000","last_change":"2024-07-04T17:29:04.795160+0000","last_active":"2024-07-04T17:29:06.787796+0000","last_peered":"2024-07-04T17:29:06.787796+0000","last_clean":"2024-07-04T17:29:06.787796+0000","last_became_active":"2024-07-04T17:29:04.793328+0000","last_became_peered":"2024-07-04T17:29:04.793328+0000","last_unstale":"2024-07-04T17:29:06.787796+0000","last_undegraded":"2024-07-04T17:29:06.787796+0000","last_fullsized":"2024-07-04T17:29:06.787796+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:56:44.314464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.493704+0000","last_change":"2024-07-04T17:29:05.796165+0000","last_active":"2024-07-04T17:29:08.493704+0000","last_peered":"2024-07-04T17:29:08.493704+0000","last_clean":"2024-07-04T17:29:08.493704+0000","last_became_active":"2024-07-04T17:29:05.795837+0000","last_became_peered":"2024-07-04T17:29:05.795837+0000","last_unstale":"2024-07-04T17:29:08.493704+0000","last_undegraded":"2024-07-04T17:29:08.493704+0000","last_fullsized":"2024-07-04T17:29:08.493704+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:36:15.241406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.493658+0000","last_change":"2024-07-04T17:29:04.790173+0000","last_active":"2024-07-04T17:29:08.493658+0000","last_peered":"2024-07-04T17:29:08.493658+0000","last_clean":"2024-07-04T17:29:08.493658+0000","last_became_active":"2024-07-04T17:29:04.789042+0000","last_became_peered":"2024-07-04T17:29:04.789042+0000","last_unstale":"2024-07-04T17:29:08.493658+0000","last_undegraded":"2024-07-04T17:29:08.493658+0000","last_fullsized":"2024-07-04T17:29:08.493658+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:06:13.018475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.780953+0000","last_change":"2024-07-04T17:29:05.793083+0000","last_active":"2024-07-04T17:29:06.780953+0000","last_peered":"2024-07-04T17:29:06.780953+0000","last_clean":"2024-07-04T17:29:06.780953+0000","last_became_active":"2024-07-04T17:29:05.790165+0000","last_became_peered":"2024-07-04T17:29:05.790165+0000","last_unstale":"2024-07-04T17:29:06.780953+0000","last_undegraded":"2024-07-04T17:29:06.780953+0000","last_fullsized":"2024-07-04T17:29:06.780953+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:20:46.692294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.494163+0000","last_change":"2024-07-04T17:29:04.770011+0000","last_active":"2024-07-04T17:29:08.494163+0000","last_peered":"2024-07-04T17:29:08.494163+0000","last_clean":"2024-07-04T17:29:08.494163+0000","last_became_active":"2024-07-04T17:29:04.769666+0000","last_became_peered":"2024-07-04T17:29:04.769666+0000","last_unstale":"2024-07-04T17:29:08.494163+0000","last_undegraded":"2024-07-04T17:29:08.494163+0000","last_fullsized":"2024-07-04T17:29:08.494163+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:21:48.040633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.787341+0000","last_change":"2024-07-04T17:29:05.816496+0000","last_active":"2024-07-04T17:29:06.787341+0000","last_peered":"2024-07-04T17:29:06.787341+0000","last_clean":"2024-07-04T17:29:06.787341+0000","last_became_active":"2024-07-04T17:29:05.816147+0000","last_became_peered":"2024-07-04T17:29:05.816147+0000","last_unstale":"2024-07-04T17:29:06.787341+0000","last_undegraded":"2024-07-04T17:29:06.787341+0000","last_fullsized":"2024-07-04T17:29:06.787341+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:33:26.836346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:06.780719+0000","last_change":"2024-07-04T17:29:04.770392+0000","last_active":"2024-07-04T17:29:06.780719+0000","last_peered":"2024-07-04T17:29:06.780719+0000","last_clean":"2024-07-04T17:29:06.780719+0000","last_became_active":"2024-07-04T17:29:04.758815+0000","last_became_peered":"2024-07-04T17:29:04.758815+0000","last_unstale":"2024-07-04T17:29:06.780719+0000","last_undegraded":"2024-07-04T17:29:06.780719+0000","last_fullsized":"2024-07-04T17:29:06.780719+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:35:08.872682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"35'66","reported_seq":203,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-07-04T17:29:08.494249+0000","last_change":"2024-07-04T17:26:04.953559+0000","last_active":"2024-07-04T17:29:08.494249+0000","last_peered":"2024-07-04T17:29:08.494249+0000","last_clean":"2024-07-04T17:29:08.494249+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:29:08.494249+0000","last_undegraded":"2024-07-04T17:29:08.494249+0000","last_fullsized":"2024-07-04T17:29:08.494249+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_clean_scrub_stamp":"2024-07-04T17:26:03.933432+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.064181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":398,"num_write":103,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":398,"num_write":103,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738407,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":512000,"data_stored":185802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738408,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32248,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339592,"statfs":{"total":96636764160,"available":96603742208,"internally_reserved":0,"allocated":1085440,"data_stored":771088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738406,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1097728,"data_stored":776558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:29:11.093 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-04T17:29:11.094 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-07-04T17:29:11.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.436+0000 7fc620800640 1 -- 172.21.15.42:0/1865047917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6180a99b0 msgr2=0x7fc6180ab1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:11.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.436+0000 7fc620800640 1 --2- 172.21.15.42:0/1865047917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6180a99b0 0x7fc6180ab1c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc60c0097b0 tx=0x7fc60c02a530 comp rx=0 tx=0).stop 2024-07-04T17:29:11.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.436+0000 7fc620800640 1 -- 172.21.15.42:0/1865047917 shutdown_connections 2024-07-04T17:29:11.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.436+0000 7fc620800640 1 --2- 172.21.15.42:0/1865047917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6180a99b0 0x7fc6180ab1c0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:11.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.436+0000 7fc620800640 1 -- 172.21.15.42:0/1865047917 >> 172.21.15.42:0/1865047917 conn(0x7fc61806c020 msgr2=0x7fc61806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:11.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.436+0000 7fc620800640 1 -- 172.21.15.42:0/1865047917 shutdown_connections 2024-07-04T17:29:11.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.436+0000 7fc620800640 1 -- 172.21.15.42:0/1865047917 wait complete. 2024-07-04T17:29:11.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.438+0000 7fc620800640 1 Processor -- start 2024-07-04T17:29:11.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.439+0000 7fc620800640 1 -- start start 2024-07-04T17:29:11.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.440+0000 7fc620800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6180a99b0 0x7fc61813be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:11.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.440+0000 7fc620800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc61813f450 con 0x7fc6180a99b0 2024-07-04T17:29:11.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.440+0000 7fc61de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6180a99b0 0x7fc61813be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:11.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.440+0000 7fc61de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6180a99b0 0x7fc61813be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37552/0 (socket says 172.21.15.42:37552) 2024-07-04T17:29:11.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.440+0000 7fc61de00640 1 -- 172.21.15.42:0/2076927269 learned_addr learned my addr 172.21.15.42:0/2076927269 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:11.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.441+0000 7fc61de00640 1 -- 172.21.15.42:0/2076927269 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6181e3b50 con 0x7fc6180a99b0 2024-07-04T17:29:11.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.441+0000 7fc61de00640 1 --2- 172.21.15.42:0/2076927269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6180a99b0 0x7fc61813be00 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc60c0098d0 tx=0x7fc60c033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:11.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.441+0000 7fc612000640 1 -- 172.21.15.42:0/2076927269 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc60c031880 con 0x7fc6180a99b0 2024-07-04T17:29:11.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.441+0000 7fc612000640 1 -- 172.21.15.42:0/2076927269 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc60c031b10 con 0x7fc6180a99b0 2024-07-04T17:29:11.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.442+0000 7fc612000640 1 -- 172.21.15.42:0/2076927269 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc60c031df0 con 0x7fc6180a99b0 2024-07-04T17:29:11.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.442+0000 7fc620800640 1 -- 172.21.15.42:0/2076927269 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc61813a6d0 con 0x7fc6180a99b0 2024-07-04T17:29:11.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.442+0000 7fc620800640 1 -- 172.21.15.42:0/2076927269 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc61813d510 con 0x7fc6180a99b0 2024-07-04T17:29:11.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.443+0000 7fc612000640 1 -- 172.21.15.42:0/2076927269 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7fc60c033070 con 0x7fc6180a99b0 2024-07-04T17:29:11.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.443+0000 7fc620800640 1 -- 172.21.15.42:0/2076927269 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc61813ba20 con 0x7fc6180a99b0 2024-07-04T17:29:11.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.443+0000 7fc612000640 1 --2- 172.21.15.42:0/2076927269 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fc5fc03d4f0 0x7fc5fc03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:11.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.443+0000 7fc612000640 1 -- 172.21.15.42:0/2076927269 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 3439+0+0 (secure 0 0 0) 0x7fc60c072d60 con 0x7fc6180a99b0 2024-07-04T17:29:11.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.444+0000 7fc61d400640 1 --2- 172.21.15.42:0/2076927269 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fc5fc03d4f0 0x7fc5fc03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:11.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.445+0000 7fc61d400640 1 --2- 172.21.15.42:0/2076927269 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fc5fc03d4f0 0x7fc5fc03fa00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc608007fd0 tx=0x7fc608007920 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:11.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.449+0000 7fc612000640 1 -- 172.21.15.42:0/2076927269 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fc61813ba20 con 0x7fc6180a99b0 2024-07-04T17:29:11.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.736+0000 7fc620800640 1 -- 172.21.15.42:0/2076927269 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fc61813fb60 con 0x7fc6180a99b0 2024-07-04T17:29:11.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.740+0000 7fc612000640 1 -- 172.21.15.42:0/2076927269 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7fc60c03f030 con 0x7fc6180a99b0 2024-07-04T17:29:11.742 INFO:teuthology.orchestra.run.smithi042.stdout:[client.0] 2024-07-04T17:29:11.742 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAO24ZmUMmYABAAh4pNlkb9JsHQtK8uCJFQCA== 2024-07-04T17:29:11.742 INFO:teuthology.orchestra.run.smithi042.stdout: caps mds = "allow" 2024-07-04T17:29:11.742 INFO:teuthology.orchestra.run.smithi042.stdout: caps mon = "allow r" 2024-07-04T17:29:11.742 INFO:teuthology.orchestra.run.smithi042.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-07-04T17:29:11.742 INFO:teuthology.orchestra.run.smithi042.stderr:updated caps for client.0 2024-07-04T17:29:11.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.743+0000 7fc620800640 1 -- 172.21.15.42:0/2076927269 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fc5fc03d4f0 msgr2=0x7fc5fc03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:11.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.743+0000 7fc620800640 1 --2- 172.21.15.42:0/2076927269 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fc5fc03d4f0 0x7fc5fc03fa00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc608007fd0 tx=0x7fc608007920 comp rx=0 tx=0).stop 2024-07-04T17:29:11.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.743+0000 7fc620800640 1 -- 172.21.15.42:0/2076927269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6180a99b0 msgr2=0x7fc61813be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:11.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.743+0000 7fc620800640 1 --2- 172.21.15.42:0/2076927269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6180a99b0 0x7fc61813be00 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc60c0098d0 tx=0x7fc60c033ad0 comp rx=0 tx=0).stop 2024-07-04T17:29:11.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.743+0000 7fc620800640 1 -- 172.21.15.42:0/2076927269 shutdown_connections 2024-07-04T17:29:11.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.743+0000 7fc620800640 1 --2- 172.21.15.42:0/2076927269 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fc5fc03d4f0 0x7fc5fc03fa00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:11.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.743+0000 7fc620800640 1 --2- 172.21.15.42:0/2076927269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6180a99b0 0x7fc61813be00 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:11.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.743+0000 7fc620800640 1 -- 172.21.15.42:0/2076927269 >> 172.21.15.42:0/2076927269 conn(0x7fc61806c020 msgr2=0x7fc61813e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:11.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.744+0000 7fc620800640 1 -- 172.21.15.42:0/2076927269 shutdown_connections 2024-07-04T17:29:11.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:11.744+0000 7fc620800640 1 -- 172.21.15.42:0/2076927269 wait complete. 2024-07-04T17:29:11.771 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:29:12.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.061+0000 7f4fda400640 1 -- 172.21.15.42:0/1728033824 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fd40b38b0 msgr2=0x7f4fd40b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:12.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.061+0000 7f4fda400640 1 --2- 172.21.15.42:0/1728033824 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fd40b38b0 0x7f4fd40b2d20 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4005fd0 tx=0x7f4fc402a5f0 comp rx=0 tx=0).stop 2024-07-04T17:29:12.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.061+0000 7f4fda400640 1 -- 172.21.15.42:0/1728033824 shutdown_connections 2024-07-04T17:29:12.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.061+0000 7f4fda400640 1 --2- 172.21.15.42:0/1728033824 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fd40b38b0 0x7f4fd40b2d20 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:12.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.061+0000 7f4fda400640 1 -- 172.21.15.42:0/1728033824 >> 172.21.15.42:0/1728033824 conn(0x7f4fd40bf700 msgr2=0x7f4fd40af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:12.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.061+0000 7f4fda400640 1 -- 172.21.15.42:0/1728033824 shutdown_connections 2024-07-04T17:29:12.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.061+0000 7f4fda400640 1 -- 172.21.15.42:0/1728033824 wait complete. 2024-07-04T17:29:12.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.064+0000 7f4fda400640 1 Processor -- start 2024-07-04T17:29:12.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.064+0000 7f4fda400640 1 -- start start 2024-07-04T17:29:12.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.066+0000 7f4fda400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fd40b1e00 0x7f4fd41d4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:12.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.066+0000 7f4fda400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd40b06e0 con 0x7f4fd40b1e00 2024-07-04T17:29:12.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.066+0000 7f4fd3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fd40b1e00 0x7f4fd41d4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:12.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.066+0000 7f4fd3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fd40b1e00 0x7f4fd41d4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37570/0 (socket says 172.21.15.42:37570) 2024-07-04T17:29:12.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.066+0000 7f4fd3400640 1 -- 172.21.15.42:0/1651330816 learned_addr learned my addr 172.21.15.42:0/1651330816 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:12.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.067+0000 7f4fd3400640 1 -- 172.21.15.42:0/1651330816 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd41d6740 con 0x7f4fd40b1e00 2024-07-04T17:29:12.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.067+0000 7f4fd3400640 1 --2- 172.21.15.42:0/1651330816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fd40b1e00 0x7f4fd41d4df0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4009850 tx=0x7f4fc40049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:12.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.067+0000 7f4fcb400640 1 -- 172.21.15.42:0/1651330816 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fc403e030 con 0x7f4fd40b1e00 2024-07-04T17:29:12.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.067+0000 7f4fcb400640 1 -- 172.21.15.42:0/1651330816 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4fc40035f0 con 0x7f4fd40b1e00 2024-07-04T17:29:12.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.067+0000 7f4fda400640 1 -- 172.21.15.42:0/1651330816 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd40aa540 con 0x7f4fd40b1e00 2024-07-04T17:29:12.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.068+0000 7f4fcb400640 1 -- 172.21.15.42:0/1651330816 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fc4035040 con 0x7f4fd40b1e00 2024-07-04T17:29:12.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.068+0000 7f4fda400640 1 -- 172.21.15.42:0/1651330816 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd41d5330 con 0x7f4fd40b1e00 2024-07-04T17:29:12.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.069+0000 7f4fda400640 1 -- 172.21.15.42:0/1651330816 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fd406c590 con 0x7f4fd40b1e00 2024-07-04T17:29:12.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.069+0000 7f4fcb400640 1 -- 172.21.15.42:0/1651330816 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f4fc4041030 con 0x7f4fd40b1e00 2024-07-04T17:29:12.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.070+0000 7f4fcb400640 1 --2- 172.21.15.42:0/1651330816 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f4fb003d4f0 0x7f4fb003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:12.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.070+0000 7f4fcb400640 1 -- 172.21.15.42:0/1651330816 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 3439+0+0 (secure 0 0 0) 0x7f4fc403f070 con 0x7f4fd40b1e00 2024-07-04T17:29:12.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.070+0000 7f4fd2a00640 1 --2- 172.21.15.42:0/1651330816 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f4fb003d4f0 0x7f4fb003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:12.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.071+0000 7f4fd2a00640 1 --2- 172.21.15.42:0/1651330816 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f4fb003d4f0 0x7f4fb003fa00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4fd406ed80 tx=0x7f4fbc0079f0 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:12.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.075+0000 7f4fcb400640 1 -- 172.21.15.42:0/1651330816 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f4fc404bd60 con 0x7f4fd40b1e00 2024-07-04T17:29:12.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.319+0000 7f4fda400640 1 -- 172.21.15.42:0/1651330816 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4fd4069b70 con 0x7f4fd40b1e00 2024-07-04T17:29:12.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.320+0000 7f4fcb400640 1 -- 172.21.15.42:0/1651330816 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+6045 (secure 0 0 0) 0x7f4fc40720b0 con 0x7f4fd40b1e00 2024-07-04T17:29:12.321 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:29:12.322 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":36,"btime":"2024-07-04T17:29:08:780650+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/343791977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":343791977},{"type":"v1","addr":"172.21.15.42:6827","nonce":343791977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":4722,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/3172446281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":3172446281},{"type":"v1","addr":"172.21.15.42:6829","nonce":3172446281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":4727,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/1866310217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1866310217},{"type":"v1","addr":"172.21.15.42:6831","nonce":1866310217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":4728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6833/1072723036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1072723036},{"type":"v1","addr":"172.21.15.42:6833","nonce":1072723036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:29:05.313061+0000","modified":"2024-07-04T17:29:08.370956+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4731},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4731":{"gid":4731,"name":"d","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6835/1327681666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1327681666},{"type":"v1","addr":"172.21.15.42:6835","nonce":1327681666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4731,"qdb_cluster":[4731]},"id":4}]} 2024-07-04T17:29:12.322 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 36 2024-07-04T17:29:12.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.323+0000 7f4fda400640 1 -- 172.21.15.42:0/1651330816 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f4fb003d4f0 msgr2=0x7f4fb003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:12.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.323+0000 7f4fda400640 1 --2- 172.21.15.42:0/1651330816 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f4fb003d4f0 0x7f4fb003fa00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4fd406ed80 tx=0x7f4fbc0079f0 comp rx=0 tx=0).stop 2024-07-04T17:29:12.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.323+0000 7f4fda400640 1 -- 172.21.15.42:0/1651330816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fd40b1e00 msgr2=0x7f4fd41d4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:12.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.323+0000 7f4fda400640 1 --2- 172.21.15.42:0/1651330816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fd40b1e00 0x7f4fd41d4df0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4009850 tx=0x7f4fc40049d0 comp rx=0 tx=0).stop 2024-07-04T17:29:12.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.323+0000 7f4fda400640 1 -- 172.21.15.42:0/1651330816 shutdown_connections 2024-07-04T17:29:12.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.323+0000 7f4fda400640 1 --2- 172.21.15.42:0/1651330816 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f4fb003d4f0 0x7f4fb003fa00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:12.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.323+0000 7f4fda400640 1 --2- 172.21.15.42:0/1651330816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fd40b1e00 0x7f4fd41d4df0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:12.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.323+0000 7f4fda400640 1 -- 172.21.15.42:0/1651330816 >> 172.21.15.42:0/1651330816 conn(0x7f4fd40bf700 msgr2=0x7f4fd41412f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:12.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.323+0000 7f4fda400640 1 -- 172.21.15.42:0/1651330816 shutdown_connections 2024-07-04T17:29:12.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:12.323+0000 7f4fda400640 1 -- 172.21.15.42:0/1651330816 wait complete. 2024-07-04T17:29:12.853 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-04T17:29:05.313061+0000', 'modified': '2024-07-04T17:29:08.370956+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4731}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4731': {'gid': 4731, 'name': 'd', 'rank': 0, 'incarnation': 33, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.42:6835/1327681666', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6834', 'nonce': 1327681666}, {'type': 'v1', 'addr': '172.21.15.42:6835', 'nonce': 1327681666}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4731, 'qdb_cluster': [4731]} 2024-07-04T17:29:12.853 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-04T17:29:12.854 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-07-04T17:29:13.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.153+0000 7f2673e00640 1 -- 172.21.15.42:0/562168725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26740b1760 msgr2=0x7f26740b8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:13.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.153+0000 7f2673e00640 1 --2- 172.21.15.42:0/562168725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26740b1760 0x7f26740b8da0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f26640097b0 tx=0x7f266402a540 comp rx=0 tx=0).stop 2024-07-04T17:29:13.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.153+0000 7f2673e00640 1 -- 172.21.15.42:0/562168725 shutdown_connections 2024-07-04T17:29:13.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.153+0000 7f2673e00640 1 --2- 172.21.15.42:0/562168725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26740b1760 0x7f26740b8da0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:13.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.153+0000 7f2673e00640 1 -- 172.21.15.42:0/562168725 >> 172.21.15.42:0/562168725 conn(0x7f26740bebc0 msgr2=0x7f26740befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:13.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.153+0000 7f2673e00640 1 -- 172.21.15.42:0/562168725 shutdown_connections 2024-07-04T17:29:13.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.153+0000 7f2673e00640 1 -- 172.21.15.42:0/562168725 wait complete. 2024-07-04T17:29:13.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.155+0000 7f2673e00640 1 Processor -- start 2024-07-04T17:29:13.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.155+0000 7f2673e00640 1 -- start start 2024-07-04T17:29:13.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.157+0000 7f2673e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f267406c020 0x7f26741d96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:13.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.157+0000 7f2673e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f267414eed0 con 0x7f267406c020 2024-07-04T17:29:13.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.157+0000 7f2672a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f267406c020 0x7f26741d96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:13.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.157+0000 7f2672a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f267406c020 0x7f26741d96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37578/0 (socket says 172.21.15.42:37578) 2024-07-04T17:29:13.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.157+0000 7f2672a00640 1 -- 172.21.15.42:0/2202336426 learned_addr learned my addr 172.21.15.42:0/2202336426 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:13.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.157+0000 7f2672a00640 1 -- 172.21.15.42:0/2202336426 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26741d7fa0 con 0x7f267406c020 2024-07-04T17:29:13.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.158+0000 7f2672a00640 1 --2- 172.21.15.42:0/2202336426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f267406c020 0x7f26741d96a0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f26640098d0 tx=0x7f2664033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:13.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.158+0000 7f266aa00640 1 -- 172.21.15.42:0/2202336426 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2664031bc0 con 0x7f267406c020 2024-07-04T17:29:13.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.158+0000 7f266aa00640 1 -- 172.21.15.42:0/2202336426 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2664031e50 con 0x7f267406c020 2024-07-04T17:29:13.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.158+0000 7f266aa00640 1 -- 172.21.15.42:0/2202336426 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2664033200 con 0x7f267406c020 2024-07-04T17:29:13.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.158+0000 7f2673e00640 1 -- 172.21.15.42:0/2202336426 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26741d8230 con 0x7f267406c020 2024-07-04T17:29:13.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.158+0000 7f2673e00640 1 -- 172.21.15.42:0/2202336426 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26741d84c0 con 0x7f267406c020 2024-07-04T17:29:13.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.160+0000 7f266aa00640 1 -- 172.21.15.42:0/2202336426 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f26640333a0 con 0x7f267406c020 2024-07-04T17:29:13.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.160+0000 7f266aa00640 1 --2- 172.21.15.42:0/2202336426 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f265003d4f0 0x7f265003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:13.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.160+0000 7f266aa00640 1 -- 172.21.15.42:0/2202336426 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 3439+0+0 (secure 0 0 0) 0x7f2664072eb0 con 0x7f267406c020 2024-07-04T17:29:13.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.161+0000 7f2672000640 1 --2- 172.21.15.42:0/2202336426 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f265003d4f0 0x7f265003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:13.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.161+0000 7f2672000640 1 --2- 172.21.15.42:0/2202336426 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f265003d4f0 0x7f265003fa00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f26740b8760 tx=0x7f265c007a90 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:13.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.162+0000 7f26805c3740 1 Processor -- start 2024-07-04T17:29:13.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.162+0000 7f26805c3740 1 -- start start 2024-07-04T17:29:13.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.164+0000 7f26805c3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558cbed2c270 0x558cbf0d2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:13.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.164+0000 7f26805c3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x558cbf0435b0 con 0x558cbed2c270 2024-07-04T17:29:13.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.164+0000 7f2672a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558cbed2c270 0x558cbf0d2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:13.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.164+0000 7f2672a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558cbed2c270 0x558cbf0d2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37594/0 (socket says 172.21.15.42:37594) 2024-07-04T17:29:13.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.164+0000 7f2672a00640 1 -- 172.21.15.42:0/3946462501 learned_addr learned my addr 172.21.15.42:0/3946462501 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:13.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.164+0000 7f2672a00640 1 -- 172.21.15.42:0/3946462501 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cbed97850 con 0x558cbed2c270 2024-07-04T17:29:13.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.165+0000 7f2672a00640 1 --2- 172.21.15.42:0/3946462501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558cbed2c270 0x558cbf0d2d30 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f2664033f20 tx=0x7f2664033690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa7db70bbd05e08c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:13.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.165+0000 7f2663000640 1 -- 172.21.15.42:0/3946462501 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2664003720 con 0x558cbed2c270 2024-07-04T17:29:13.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.165+0000 7f2663000640 1 -- 172.21.15.42:0/3946462501 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f266406e070 con 0x558cbed2c270 2024-07-04T17:29:13.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.165+0000 7f2663000640 1 -- 172.21.15.42:0/3946462501 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f266404b510 con 0x558cbed2c270 2024-07-04T17:29:13.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.166+0000 7f26805c3740 1 -- 172.21.15.42:0/3946462501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558cbed2c270 msgr2=0x558cbf0d2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:13.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.166+0000 7f26805c3740 1 --2- 172.21.15.42:0/3946462501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558cbed2c270 0x558cbf0d2d30 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f2664033f20 tx=0x7f2664033690 comp rx=0 tx=0).stop 2024-07-04T17:29:13.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.166+0000 7f26805c3740 1 -- 172.21.15.42:0/3946462501 shutdown_connections 2024-07-04T17:29:13.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.166+0000 7f26805c3740 1 --2- 172.21.15.42:0/3946462501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558cbed2c270 0x558cbf0d2d30 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:13.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.166+0000 7f26805c3740 1 -- 172.21.15.42:0/3946462501 >> 172.21.15.42:0/3946462501 conn(0x558cbed203c0 msgr2=0x558cbf0d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:13.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.166+0000 7f26805c3740 1 -- 172.21.15.42:0/3946462501 shutdown_connections 2024-07-04T17:29:13.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.166+0000 7f26805c3740 1 -- 172.21.15.42:0/3946462501 wait complete. 2024-07-04T17:29:13.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.167+0000 7f26805c3740 1 -- start start 2024-07-04T17:29:13.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.167+0000 7f26805c3740 1 Processor -- start 2024-07-04T17:29:13.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.168+0000 7f26805c3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558cbed2c270 0x558cbf0f5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:13.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.168+0000 7f26805c3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x558cbf036b90 con 0x558cbed2c270 2024-07-04T17:29:13.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.168+0000 7f26805c3740 10 client.0 _ll_register_callbacks cb 0x558cbefdb8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T17:29:13.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.169+0000 7f2672a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558cbed2c270 0x558cbf0f5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:13.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.169+0000 7f2672a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558cbed2c270 0x558cbf0f5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37606/0 (socket says 172.21.15.42:37606) 2024-07-04T17:29:13.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.169+0000 7f2672a00640 1 -- 172.21.15.42:0/2763946957 learned_addr learned my addr 172.21.15.42:0/2763946957 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:13.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.169+0000 7f2672a00640 1 -- 172.21.15.42:0/2763946957 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cbee0e6b0 con 0x558cbed2c270 2024-07-04T17:29:13.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.169+0000 7f2672a00640 1 --2- 172.21.15.42:0/2763946957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558cbed2c270 0x558cbf0f5a20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f266403bfd0 tx=0x7f266406ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b5dd6c10e058692 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:13.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.169+0000 7f2657400640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-07-04T17:29:13.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.170+0000 7f2657400640 1 -- 172.21.15.42:0/2763946957 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2664037e30 con 0x558cbed2c270 2024-07-04T17:29:13.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.170+0000 7f2657400640 1 -- 172.21.15.42:0/2763946957 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(50..50 src has 1..50) ==== 3439+0+0 (secure 0 0 0) 0x7f2664071b70 con 0x558cbed2c270 2024-07-04T17:29:13.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.170+0000 7f26805c3740 1 -- 172.21.15.42:0/2763946957 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558cbefd36c0 con 0x558cbed2c270 2024-07-04T17:29:13.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.170+0000 7f2657400640 1 -- 172.21.15.42:0/2763946957 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2664071d70 con 0x558cbed2c270 2024-07-04T17:29:13.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.170+0000 7f26805c3740 10 client.4784 fetch_fsmap learned FSMap version 36 2024-07-04T17:29:13.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.170+0000 7f26805c3740 1 -- 172.21.15.42:0/2763946957 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x558cbf0f7260 con 0x558cbed2c270 2024-07-04T17:29:13.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.170+0000 7f2657400640 1 -- 172.21.15.42:0/2763946957 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x558cbefd36c0 con 0x558cbed2c270 2024-07-04T17:29:13.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.170+0000 7f2657400640 1 -- 172.21.15.42:0/2763946957 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 36) ==== 4107+0+0 (secure 0 0 0) 0x7f266404b720 con 0x558cbed2c270 2024-07-04T17:29:13.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.170+0000 7f26805c3740 10 client.4784 fetch_fsmap finished waiting for FSMap version 36 2024-07-04T17:29:13.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.170+0000 7f26805c3740 10 client.4784 resolve_mds: resolved name 'd' to daemon mds.d 2024-07-04T17:29:13.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.170+0000 7f26805c3740 20 client.4784 populate_metadata read hostname 'smithi042' 2024-07-04T17:29:13.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.170+0000 7f26805c3740 1 --2- 172.21.15.42:0/2763946957 >> [v2:172.21.15.42:6834/1327681666,v1:172.21.15.42:6835/1327681666] conn(0x558cbed033b0 0x558cbf115d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:13.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.170+0000 7f26805c3740 4 client.4784 mds_command: new command op to 4731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T17:29:13.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.170+0000 7f26805c3740 1 -- 172.21.15.42:0/2763946957 --> [v2:172.21.15.42:6834/1327681666,v1:172.21.15.42:6835/1327681666] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cbf045500 con 0x558cbed033b0 2024-07-04T17:29:13.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.171+0000 7f2672000640 1 --2- 172.21.15.42:0/2763946957 >> [v2:172.21.15.42:6834/1327681666,v1:172.21.15.42:6835/1327681666] conn(0x558cbed033b0 0x558cbf115d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:13.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.171+0000 7f2672000640 1 --2- 172.21.15.42:0/2763946957 >> [v2:172.21.15.42:6834/1327681666,v1:172.21.15.42:6835/1327681666] conn(0x558cbed033b0 0x558cbf115d30 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efc8e1ea9bb01561 server_cookie=dde859f60b3698de in_seq=0 out_seq=0 2024-07-04T17:29:13.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.171+0000 7f2657400640 10 client.4784 ms_handle_connect on v2:172.21.15.42:6834/1327681666 2024-07-04T17:29:13.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.172+0000 7f2657400640 1 -- 172.21.15.42:0/2763946957 <== mds.0 v2:172.21.15.42:6834/1327681666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558cbf045500 con 0x558cbed033b0 2024-07-04T17:29:13.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.172+0000 7f2657400640 10 client.4784 handle_command_reply: tid=0 2024-07-04T17:29:13.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.172+0000 7f26805c3740 1 client.4784 shutdown 2024-07-04T17:29:13.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.173+0000 7f26805c3740 1 -- 172.21.15.42:0/2763946957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558cbed2c270 msgr2=0x558cbf0f5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:13.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.173+0000 7f26805c3740 1 --2- 172.21.15.42:0/2763946957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558cbed2c270 0x558cbf0f5a20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f266403bfd0 tx=0x7f266406ad60 comp rx=0 tx=0).stop 2024-07-04T17:29:13.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.173+0000 7f26805c3740 1 -- 172.21.15.42:0/2763946957 shutdown_connections 2024-07-04T17:29:13.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.173+0000 7f26805c3740 1 --2- 172.21.15.42:0/2763946957 >> [v2:172.21.15.42:6834/1327681666,v1:172.21.15.42:6835/1327681666] conn(0x558cbed033b0 0x558cbf115d30 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:13.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.173+0000 7f26805c3740 1 --2- 172.21.15.42:0/2763946957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558cbed2c270 0x558cbf0f5a20 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:13.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.173+0000 7f26805c3740 1 -- 172.21.15.42:0/2763946957 >> 172.21.15.42:0/2763946957 conn(0x558cbed203c0 msgr2=0x558cbf0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:13.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.173+0000 7f2657400640 0 client.4784 ms_handle_reset on v2:172.21.15.42:6834/1327681666 2024-07-04T17:29:13.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.173+0000 7f26805c3740 1 -- 172.21.15.42:0/2763946957 shutdown_connections 2024-07-04T17:29:13.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.173+0000 7f26805c3740 1 -- 172.21.15.42:0/2763946957 wait complete. 2024-07-04T17:29:13.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.173+0000 7f26805c3740 20 client.4784 trim_cache size 0 max 16384 2024-07-04T17:29:13.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.194+0000 7f26805c3740 1 Processor -- start 2024-07-04T17:29:13.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.194+0000 7f26805c3740 1 -- start start 2024-07-04T17:29:13.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.195+0000 7f26805c3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558cbed2c270 0x558cbf0c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:13.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.195+0000 7f26805c3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f26740be7f0 con 0x558cbed2c270 2024-07-04T17:29:13.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.195+0000 7f2672a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558cbed2c270 0x558cbf0c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:13.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.196+0000 7f2672a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558cbed2c270 0x558cbf0c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37620/0 (socket says 172.21.15.42:37620) 2024-07-04T17:29:13.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.196+0000 7f2672a00640 1 -- 172.21.15.42:0/3617928833 learned_addr learned my addr 172.21.15.42:0/3617928833 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:13.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.196+0000 7f2672a00640 1 -- 172.21.15.42:0/3617928833 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cbf0f64e0 con 0x558cbed2c270 2024-07-04T17:29:13.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.196+0000 7f2672a00640 1 --2- 172.21.15.42:0/3617928833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558cbed2c270 0x558cbf0c8830 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2664033f20 tx=0x7f26640038c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=543bd50d7752db67 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:13.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.197+0000 7f2662600640 1 -- 172.21.15.42:0/3617928833 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2664037e30 con 0x558cbed2c270 2024-07-04T17:29:13.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.197+0000 7f2662600640 1 -- 172.21.15.42:0/3617928833 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f266406e050 con 0x558cbed2c270 2024-07-04T17:29:13.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.197+0000 7f2662600640 1 -- 172.21.15.42:0/3617928833 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26640303a0 con 0x558cbed2c270 2024-07-04T17:29:13.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.197+0000 7f26805c3740 1 -- 172.21.15.42:0/3617928833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558cbed2c270 msgr2=0x558cbf0c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:13.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.197+0000 7f26805c3740 1 --2- 172.21.15.42:0/3617928833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558cbed2c270 0x558cbf0c8830 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2664033f20 tx=0x7f26640038c0 comp rx=0 tx=0).stop 2024-07-04T17:29:13.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.197+0000 7f26805c3740 1 -- 172.21.15.42:0/3617928833 shutdown_connections 2024-07-04T17:29:13.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.197+0000 7f26805c3740 1 --2- 172.21.15.42:0/3617928833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558cbed2c270 0x558cbf0c8830 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:13.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.197+0000 7f26805c3740 1 -- 172.21.15.42:0/3617928833 >> 172.21.15.42:0/3617928833 conn(0x558cbed203c0 msgr2=0x558cbf0edda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:13.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.198+0000 7f26805c3740 1 -- 172.21.15.42:0/3617928833 shutdown_connections 2024-07-04T17:29:13.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.198+0000 7f26805c3740 1 -- 172.21.15.42:0/3617928833 wait complete. 2024-07-04T17:29:13.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.198+0000 7f26805c3740 1 -- start start 2024-07-04T17:29:13.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.199+0000 7f26805c3740 1 Processor -- start 2024-07-04T17:29:13.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.200+0000 7f26805c3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558cbed2c270 0x558cbf120570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:13.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.200+0000 7f26805c3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x558cbf032b50 con 0x558cbed2c270 2024-07-04T17:29:13.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.200+0000 7f26805c3740 10 client.0 _ll_register_callbacks cb 0x7f26640035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T17:29:13.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.200+0000 7f2672a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558cbed2c270 0x558cbf120570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:13.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.200+0000 7f2672a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558cbed2c270 0x558cbf120570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37628/0 (socket says 172.21.15.42:37628) 2024-07-04T17:29:13.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.200+0000 7f2672a00640 1 -- 172.21.15.42:0/485346380 learned_addr learned my addr 172.21.15.42:0/485346380 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:13.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.201+0000 7f2672a00640 1 -- 172.21.15.42:0/485346380 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cbf0d1c50 con 0x558cbed2c270 2024-07-04T17:29:13.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.201+0000 7f2672a00640 1 --2- 172.21.15.42:0/485346380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558cbed2c270 0x558cbf120570 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f2664030560 tx=0x7f2664003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e75da31bcd3839 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:13.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.201+0000 7f2657400640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-07-04T17:29:13.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.201+0000 7f2657400640 1 -- 172.21.15.42:0/485346380 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f266407f060 con 0x558cbed2c270 2024-07-04T17:29:13.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.201+0000 7f2657400640 1 -- 172.21.15.42:0/485346380 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(50..50 src has 1..50) ==== 3439+0+0 (secure 0 0 0) 0x7f2664071b70 con 0x558cbed2c270 2024-07-04T17:29:13.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.201+0000 7f26805c3740 1 -- 172.21.15.42:0/485346380 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f266406e050 con 0x558cbed2c270 2024-07-04T17:29:13.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.201+0000 7f2657400640 1 -- 172.21.15.42:0/485346380 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f26640305d0 con 0x558cbed2c270 2024-07-04T17:29:13.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.202+0000 7f26805c3740 10 client.4786 fetch_fsmap learned FSMap version 36 2024-07-04T17:29:13.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.202+0000 7f26805c3740 1 -- 172.21.15.42:0/485346380 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x558cbf0d1c50 con 0x558cbed2c270 2024-07-04T17:29:13.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.202+0000 7f2657400640 1 -- 172.21.15.42:0/485346380 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f266406e050 con 0x558cbed2c270 2024-07-04T17:29:13.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.202+0000 7f2657400640 1 -- 172.21.15.42:0/485346380 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 36) ==== 4107+0+0 (secure 0 0 0) 0x7f266406a3e0 con 0x558cbed2c270 2024-07-04T17:29:13.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.202+0000 7f26805c3740 10 client.4786 fetch_fsmap finished waiting for FSMap version 36 2024-07-04T17:29:13.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.202+0000 7f26805c3740 10 client.4786 resolve_mds: resolved name 'd' to daemon mds.d 2024-07-04T17:29:13.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.202+0000 7f26805c3740 20 client.4786 populate_metadata read hostname 'smithi042' 2024-07-04T17:29:13.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.202+0000 7f26805c3740 1 --2- 172.21.15.42:0/485346380 >> [v2:172.21.15.42:6834/1327681666,v1:172.21.15.42:6835/1327681666] conn(0x558cbeceae40 0x558cbf17bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:13.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.202+0000 7f26805c3740 4 client.4786 mds_command: new command op to 4731 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-04T17:29:13.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.202+0000 7f26805c3740 1 -- 172.21.15.42:0/485346380 --> [v2:172.21.15.42:6834/1327681666,v1:172.21.15.42:6835/1327681666] -- command(tid 0: {"prefix": "status"}) -- 0x558cbf1156b0 con 0x558cbeceae40 2024-07-04T17:29:13.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.202+0000 7f2672000640 1 --2- 172.21.15.42:0/485346380 >> [v2:172.21.15.42:6834/1327681666,v1:172.21.15.42:6835/1327681666] conn(0x558cbeceae40 0x558cbf17bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:13.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.203+0000 7f2657400640 10 client.4786 ms_handle_connect on v2:172.21.15.42:6834/1327681666 2024-07-04T17:29:13.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.203+0000 7f2672000640 1 --2- 172.21.15.42:0/485346380 >> [v2:172.21.15.42:6834/1327681666,v1:172.21.15.42:6835/1327681666] conn(0x558cbeceae40 0x558cbf17bf40 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c352690da1abe18 server_cookie=40de71841464a54b in_seq=0 out_seq=0 2024-07-04T17:29:13.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.203+0000 7f2657400640 1 -- 172.21.15.42:0/485346380 <== mds.0 v2:172.21.15.42:6834/1327681666 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x558cbf1156b0 con 0x558cbeceae40 2024-07-04T17:29:13.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.203+0000 7f2657400640 10 client.4786 handle_command_reply: tid=0 2024-07-04T17:29:13.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.203+0000 7f26805c3740 1 client.4786 shutdown 2024-07-04T17:29:13.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.203+0000 7f26805c3740 1 -- 172.21.15.42:0/485346380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558cbed2c270 msgr2=0x558cbf120570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:13.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.203+0000 7f26805c3740 1 --2- 172.21.15.42:0/485346380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558cbed2c270 0x558cbf120570 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f2664030560 tx=0x7f2664003d40 comp rx=0 tx=0).stop 2024-07-04T17:29:13.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.203+0000 7f26805c3740 1 -- 172.21.15.42:0/485346380 shutdown_connections 2024-07-04T17:29:13.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.203+0000 7f26805c3740 1 --2- 172.21.15.42:0/485346380 >> [v2:172.21.15.42:6834/1327681666,v1:172.21.15.42:6835/1327681666] conn(0x558cbeceae40 0x558cbf17bf40 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:13.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.203+0000 7f26805c3740 1 --2- 172.21.15.42:0/485346380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558cbed2c270 0x558cbf120570 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:13.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.204+0000 7f26805c3740 1 -- 172.21.15.42:0/485346380 >> 172.21.15.42:0/485346380 conn(0x558cbed203c0 msgr2=0x558cbf0fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:13.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.204+0000 7f2657400640 0 client.4786 ms_handle_reset on v2:172.21.15.42:6834/1327681666 2024-07-04T17:29:13.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.204+0000 7f26805c3740 1 -- 172.21.15.42:0/485346380 shutdown_connections 2024-07-04T17:29:13.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.204+0000 7f26805c3740 1 -- 172.21.15.42:0/485346380 wait complete. 2024-07-04T17:29:13.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.204+0000 7f26805c3740 20 client.4786 trim_cache size 0 max 16384 2024-07-04T17:29:13.207 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:29:13.207 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "be4f300a-93ef-4489-b540-37ece39b0aac", 2024-07-04T17:29:13.207 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-07-04T17:29:13.207 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 4731, 2024-07-04T17:29:13.207 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-07-04T17:29:13.207 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-07-04T17:29:13.207 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-07-04T17:29:13.207 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 7.4399778510000001, 2024-07-04T17:29:13.208 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 36, 2024-07-04T17:29:13.208 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 50, 2024-07-04T17:29:13.208 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 50, 2024-07-04T17:29:13.208 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 12.696530728999999 2024-07-04T17:29:13.208 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:29:13.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.204+0000 7f2673e00640 1 -- 172.21.15.42:0/2202336426 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f265003d4f0 msgr2=0x7f265003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:13.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.204+0000 7f2673e00640 1 --2- 172.21.15.42:0/2202336426 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f265003d4f0 0x7f265003fa00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f26740b8760 tx=0x7f265c007a90 comp rx=0 tx=0).stop 2024-07-04T17:29:13.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.204+0000 7f2673e00640 1 -- 172.21.15.42:0/2202336426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f267406c020 msgr2=0x7f26741d96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:13.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.204+0000 7f2673e00640 1 --2- 172.21.15.42:0/2202336426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f267406c020 0x7f26741d96a0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f26640098d0 tx=0x7f2664033e40 comp rx=0 tx=0).stop 2024-07-04T17:29:13.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.205+0000 7f2673e00640 1 -- 172.21.15.42:0/2202336426 shutdown_connections 2024-07-04T17:29:13.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.205+0000 7f2673e00640 1 --2- 172.21.15.42:0/2202336426 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f265003d4f0 0x7f265003fa00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:13.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.205+0000 7f2673e00640 1 --2- 172.21.15.42:0/2202336426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f267406c020 0x7f26741d96a0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:13.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.205+0000 7f2673e00640 1 -- 172.21.15.42:0/2202336426 >> 172.21.15.42:0/2202336426 conn(0x7f26740bebc0 msgr2=0x7f26740befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:13.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.205+0000 7f2673e00640 1 -- 172.21.15.42:0/2202336426 shutdown_connections 2024-07-04T17:29:13.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.205+0000 7f2673e00640 1 -- 172.21.15.42:0/2202336426 wait complete. 2024-07-04T17:29:13.233 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-04T17:29:13.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.518+0000 7f814be00640 1 -- 172.21.15.42:0/381178967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f814c0b1760 msgr2=0x7f814c0b8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:13.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.518+0000 7f814be00640 1 --2- 172.21.15.42:0/381178967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f814c0b1760 0x7f814c0b8da0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f813c005fd0 tx=0x7f813c02a5f0 comp rx=0 tx=0).stop 2024-07-04T17:29:13.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.518+0000 7f814be00640 1 -- 172.21.15.42:0/381178967 shutdown_connections 2024-07-04T17:29:13.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.518+0000 7f814be00640 1 --2- 172.21.15.42:0/381178967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f814c0b1760 0x7f814c0b8da0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:13.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.518+0000 7f814be00640 1 -- 172.21.15.42:0/381178967 >> 172.21.15.42:0/381178967 conn(0x7f814c0bebc0 msgr2=0x7f814c0befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:13.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.519+0000 7f814be00640 1 -- 172.21.15.42:0/381178967 shutdown_connections 2024-07-04T17:29:13.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.519+0000 7f814be00640 1 -- 172.21.15.42:0/381178967 wait complete. 2024-07-04T17:29:13.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.521+0000 7f814be00640 1 Processor -- start 2024-07-04T17:29:13.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.521+0000 7f814be00640 1 -- start start 2024-07-04T17:29:13.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.522+0000 7f814be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f814c06c020 0x7f814c1d96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:13.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.522+0000 7f814be00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f814c14eed0 con 0x7f814c06c020 2024-07-04T17:29:13.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.522+0000 7f814aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f814c06c020 0x7f814c1d96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:13.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.522+0000 7f814aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f814c06c020 0x7f814c1d96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37650/0 (socket says 172.21.15.42:37650) 2024-07-04T17:29:13.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.522+0000 7f814aa00640 1 -- 172.21.15.42:0/1423339918 learned_addr learned my addr 172.21.15.42:0/1423339918 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:13.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.523+0000 7f814aa00640 1 -- 172.21.15.42:0/1423339918 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f814c1d7fa0 con 0x7f814c06c020 2024-07-04T17:29:13.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.523+0000 7f814aa00640 1 --2- 172.21.15.42:0/1423339918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f814c06c020 0x7f814c1d96a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f813c009850 tx=0x7f813c0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:13.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.523+0000 7f8142a00640 1 -- 172.21.15.42:0/1423339918 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f813c03e030 con 0x7f814c06c020 2024-07-04T17:29:13.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.523+0000 7f8142a00640 1 -- 172.21.15.42:0/1423339918 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f813c0035f0 con 0x7f814c06c020 2024-07-04T17:29:13.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.523+0000 7f8142a00640 1 -- 172.21.15.42:0/1423339918 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f813c035040 con 0x7f814c06c020 2024-07-04T17:29:13.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.523+0000 7f814be00640 1 -- 172.21.15.42:0/1423339918 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f814c1d8230 con 0x7f814c06c020 2024-07-04T17:29:13.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.524+0000 7f814be00640 1 -- 172.21.15.42:0/1423339918 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f814c1d84c0 con 0x7f814c06c020 2024-07-04T17:29:13.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.525+0000 7f814be00640 1 -- 172.21.15.42:0/1423339918 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f814c0b8da0 con 0x7f814c06c020 2024-07-04T17:29:13.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.525+0000 7f8142a00640 1 -- 172.21.15.42:0/1423339918 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f813c041030 con 0x7f814c06c020 2024-07-04T17:29:13.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.526+0000 7f8142a00640 1 --2- 172.21.15.42:0/1423339918 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f812803d4f0 0x7f812803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:13.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.526+0000 7f8142a00640 1 -- 172.21.15.42:0/1423339918 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 3439+0+0 (secure 0 0 0) 0x7f813c03f070 con 0x7f814c06c020 2024-07-04T17:29:13.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.526+0000 7f8143e00640 1 --2- 172.21.15.42:0/1423339918 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f812803d4f0 0x7f812803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:13.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.527+0000 7f8143e00640 1 --2- 172.21.15.42:0/1423339918 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f812803d4f0 0x7f812803fa00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f814c0b8760 tx=0x7f81340079f0 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:13.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.530+0000 7f8142a00640 1 -- 172.21.15.42:0/1423339918 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f813c04bd60 con 0x7f814c06c020 2024-07-04T17:29:13.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.771+0000 7f814be00640 1 -- 172.21.15.42:0/1423339918 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f814c149460 con 0x7f814c06c020 2024-07-04T17:29:13.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.772+0000 7f8142a00640 1 -- 172.21.15.42:0/1423339918 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) ==== 53+0+73 (secure 0 0 0) 0x7f813c0720b0 con 0x7f814c06c020 2024-07-04T17:29:13.773 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-04T17:29:13.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.774+0000 7f814be00640 1 -- 172.21.15.42:0/1423339918 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f812803d4f0 msgr2=0x7f812803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:13.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.774+0000 7f814be00640 1 --2- 172.21.15.42:0/1423339918 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f812803d4f0 0x7f812803fa00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f814c0b8760 tx=0x7f81340079f0 comp rx=0 tx=0).stop 2024-07-04T17:29:13.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.774+0000 7f814be00640 1 -- 172.21.15.42:0/1423339918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f814c06c020 msgr2=0x7f814c1d96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:13.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.774+0000 7f814be00640 1 --2- 172.21.15.42:0/1423339918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f814c06c020 0x7f814c1d96a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f813c009850 tx=0x7f813c0049d0 comp rx=0 tx=0).stop 2024-07-04T17:29:13.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.774+0000 7f814be00640 1 -- 172.21.15.42:0/1423339918 shutdown_connections 2024-07-04T17:29:13.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.774+0000 7f814be00640 1 --2- 172.21.15.42:0/1423339918 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f812803d4f0 0x7f812803fa00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:13.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.774+0000 7f814be00640 1 --2- 172.21.15.42:0/1423339918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f814c06c020 0x7f814c1d96a0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:13.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.774+0000 7f814be00640 1 -- 172.21.15.42:0/1423339918 >> 172.21.15.42:0/1423339918 conn(0x7f814c0bebc0 msgr2=0x7f814c0befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:13.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.774+0000 7f814be00640 1 -- 172.21.15.42:0/1423339918 shutdown_connections 2024-07-04T17:29:13.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:13.774+0000 7f814be00640 1 -- 172.21.15.42:0/1423339918 wait complete. 2024-07-04T17:29:13.804 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-07-04T17:29:13.804 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-04T17:29:13.804 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2024-07-04T17:29:13.805 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-04T17:29:13.805 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-04T17:29:13.805 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-07-04T17:29:13.805 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-07-04T17:29:13.805 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-04T17:29:13.805 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-04T17:29:13.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:29:13.805 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2024-07-04T17:29:13.855 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-04T17:29:13.855 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-04T17:29:13.855 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-07-04T17:29:13.855 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:29:13.855 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-07-04T17:29:13.855 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:29:13.855 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:29:13.855 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2024-07-04T17:29:13.855 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f0 2024-07-04T17:29:13.856 INFO:teuthology.orchestra.run.smithi042.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:29:13.856 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2024-07-04T17:29:13.856 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f1 2024-07-04T17:29:13.856 INFO:teuthology.orchestra.run.smithi042.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-04T17:29:13.856 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:29:13.856 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f0 2024-07-04T17:29:13.856 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-07-04T17:29:13.856 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 42416sec preferred_lft 42416sec 2024-07-04T17:29:13.856 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::ec4:7aff:febd:1440/64 scope link noprefixroute 2024-07-04T17:29:13.856 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:29:13.856 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:29:13.856 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:29:13.857 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f1 2024-07-04T17:29:13.857 INFO:teuthology.orchestra.run.smithi042.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-04T17:29:13.857 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 32:75:d6:f1:31:e1 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:29:13.857 INFO:teuthology.orchestra.run.smithi042.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-04T17:29:13.857 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:29:13.857 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::8839:a9ff:fe74:17f5/64 scope link 2024-07-04T17:29:13.857 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:29:13.857 INFO:teuthology.orchestra.run.smithi042.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-04T17:29:13.857 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether e6:ab:57:fb:ca:9e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:29:13.857 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::e4ab:57ff:fefb:ca9e/64 scope link 2024-07-04T17:29:13.857 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:29:13.857 INFO:teuthology.orchestra.run.smithi042.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-04T17:29:13.857 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 32:75:d6:f1:31:e1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:29:13.857 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::3075:d6ff:fef1:31e1/64 scope link 2024-07-04T17:29:13.858 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:29:13.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:29:13.858 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-04T17:29:13.931 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-07-04T17:29:13.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:29:13.935 DEBUG:teuthology.orchestra.run.smithi042:> route 2024-07-04T17:29:13.987 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2024-07-04T17:29:13.987 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-04T17:29:13.988 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-04T17:29:13.988 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-04T17:29:13.988 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-04T17:29:13.988 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-04T17:29:13.988 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-07-04T17:29:13.988 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-04T17:29:13.988 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-04T17:29:13.988 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-04T17:29:13.988 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-07-04T17:29:14.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-04T17:29:14.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:29:14.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:29:14.212 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-07-04T17:29:14.260 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-04T17:29:14.260 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2024-07-04T17:29:14.261 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-04T17:29:14.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:29:14.261 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.0 up 2024-07-04T17:29:14.338 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-07-04T17:29:14.339 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-04T17:29:14.339 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-04T17:29:14.390 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-04T17:29:14.390 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-04T17:29:14.391 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-04T17:29:14.443 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:29:14.532 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:29:14.591 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:29:14.591 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:29:14.591 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:29:14.591 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:29:14.591 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:29:14.591 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:29:14.591 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:29:14.592 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:29:14.592 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:29:14.592 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:29:14.592 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:29:14.592 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:29:14.592 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:29:14.592 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:29:14.592 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:29:14.592 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:29:14.592 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:29:14.592 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:29:14.592 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:29:14.593 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:29:14.593 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:29:14.593 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:29:14.593 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:29:14.593 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:29:14.593 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:29:14.593 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:29:14.593 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:29:14.593 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:29:14.593 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:29:14.593 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:29:14.593 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:29:14.593 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:29:14.594 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:29:14.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:29:14.595 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:29:14.654 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-07-04T17:29:14.654 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-07-04T17:29:14.659 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:29:14.731 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:29:14.772 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:29:14.772 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:29:14.772 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:29:14.772 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:29:14.772 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:29:14.772 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:29:14.772 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:29:14.772 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:29:14.772 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:29:14.773 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:29:14.773 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:29:14.773 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:29:14.773 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:29:14.773 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:29:14.773 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:29:14.773 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:29:14.773 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:29:14.773 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:29:14.773 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:29:14.773 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:29:14.773 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:29:14.773 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:29:14.773 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:29:14.774 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:29:14.774 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:29:14.774 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:29:14.774 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:29:14.774 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:29:14.774 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:29:14.774 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:29:14.774 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:29:14.774 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:29:14.774 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:29:14.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:29:14.775 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:29:14.917 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:2024-07-04T17:29:14.916+0000 7ff4bd71f480 -1 init, newargv = 0x7ff4a0004640 newargc=15 2024-07-04T17:29:14.917 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[51270]: starting ceph client 2024-07-04T17:29:14.924 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[51270]: starting fuse 2024-07-04T17:29:15.965 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:29:16.064 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:29:16.125 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:29:16.125 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:29:16.125 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:29:16.126 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:29:16.126 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:29:16.126 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:29:16.126 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:29:16.126 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:29:16.126 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:29:16.126 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:29:16.126 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:29:16.126 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:29:16.126 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:29:16.127 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:29:16.127 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:29:16.127 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:29:16.127 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:29:16.127 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:29:16.127 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:29:16.127 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:29:16.127 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:29:16.127 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:29:16.128 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:29:16.128 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:29:16.128 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:29:16.128 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:29:16.128 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:29:16.128 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:29:16.128 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:29:16.128 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:29:16.128 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:29:16.128 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:29:16.128 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:29:16.128 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-07-04T17:29:16.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:29:16.129 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:29:16.185 INFO:teuthology.orchestra.run.smithi042.stdout:42 2024-07-04T17:29:16.185 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-07-04T17:29:16.186 DEBUG:teuthology.orchestra.run.smithi042:> sudo stdin-killer -- python3 -c ' 2024-07-04T17:29:16.186 DEBUG:teuthology.orchestra.run.smithi042:> import glob 2024-07-04T17:29:16.186 DEBUG:teuthology.orchestra.run.smithi042:> import re 2024-07-04T17:29:16.186 DEBUG:teuthology.orchestra.run.smithi042:> import os 2024-07-04T17:29:16.186 DEBUG:teuthology.orchestra.run.smithi042:> import subprocess 2024-07-04T17:29:16.186 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:29:16.186 DEBUG:teuthology.orchestra.run.smithi042:> def _find_admin_socket(client_name): 2024-07-04T17:29:16.186 DEBUG:teuthology.orchestra.run.smithi042:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-07-04T17:29:16.186 DEBUG:teuthology.orchestra.run.smithi042:> files = glob.glob(asok_path) 2024-07-04T17:29:16.186 DEBUG:teuthology.orchestra.run.smithi042:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-07-04T17:29:16.186 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:29:16.187 DEBUG:teuthology.orchestra.run.smithi042:> # Given a non-glob path, it better be there 2024-07-04T17:29:16.187 DEBUG:teuthology.orchestra.run.smithi042:> if "*" not in asok_path: 2024-07-04T17:29:16.187 DEBUG:teuthology.orchestra.run.smithi042:> assert(len(files) == 1) 2024-07-04T17:29:16.187 DEBUG:teuthology.orchestra.run.smithi042:> return files[0] 2024-07-04T17:29:16.187 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:29:16.187 DEBUG:teuthology.orchestra.run.smithi042:> for f in files: 2024-07-04T17:29:16.187 DEBUG:teuthology.orchestra.run.smithi042:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-07-04T17:29:16.187 DEBUG:teuthology.orchestra.run.smithi042:> if os.path.exists("/proc/{0}".format(pid)): 2024-07-04T17:29:16.188 DEBUG:teuthology.orchestra.run.smithi042:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-07-04T17:29:16.188 DEBUG:teuthology.orchestra.run.smithi042:> contents = proc_f.read() 2024-07-04T17:29:16.188 DEBUG:teuthology.orchestra.run.smithi042:> if mountpoint in contents: 2024-07-04T17:29:16.188 DEBUG:teuthology.orchestra.run.smithi042:> return f 2024-07-04T17:29:16.188 DEBUG:teuthology.orchestra.run.smithi042:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-07-04T17:29:16.188 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:29:16.188 DEBUG:teuthology.orchestra.run.smithi042:> print(_find_admin_socket("client.0")) 2024-07-04T17:29:16.188 DEBUG:teuthology.orchestra.run.smithi042:> ' 2024-07-04T17:29:16.367 INFO:teuthology.orchestra.run.smithi042.stdout:/var/run/ceph/ceph-client.0.51270.asok 2024-07-04T17:29:16.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:29:16.385 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.51270.asok 2024-07-04T17:29:16.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:29:16.385 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.51270.asok status 2024-07-04T17:29:16.692 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:29:16.692 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": { 2024-07-04T17:29:16.693 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_sha1": "9dba1e836b4756bf3a32be2a4936f371dff55008", 2024-07-04T17:29:16.693 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.0.0-4667-g9dba1e83 (9dba1e836b4756bf3a32be2a4936f371dff55008) squid (dev)", 2024-07-04T17:29:16.693 INFO:teuthology.orchestra.run.smithi042.stdout: "entity_id": "0", 2024-07-04T17:29:16.693 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": "smithi042", 2024-07-04T17:29:16.693 INFO:teuthology.orchestra.run.smithi042.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-07-04T17:29:16.693 INFO:teuthology.orchestra.run.smithi042.stdout: "pid": "51270", 2024-07-04T17:29:16.693 INFO:teuthology.orchestra.run.smithi042.stdout: "root": "/" 2024-07-04T17:29:16.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:16.693 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_count": 0, 2024-07-04T17:29:16.693 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_pinned_count": 0, 2024-07-04T17:29:16.693 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 4792, 2024-07-04T17:29:16.694 INFO:teuthology.orchestra.run.smithi042.stdout: "inst": { 2024-07-04T17:29:16.694 INFO:teuthology.orchestra.run.smithi042.stdout: "name": { 2024-07-04T17:29:16.694 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "client", 2024-07-04T17:29:16.694 INFO:teuthology.orchestra.run.smithi042.stdout: "num": 4792 2024-07-04T17:29:16.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:16.694 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-07-04T17:29:16.694 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-07-04T17:29:16.694 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-07-04T17:29:16.694 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2517053341 2024-07-04T17:29:16.694 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:16.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:16.695 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-07-04T17:29:16.695 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-07-04T17:29:16.695 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-07-04T17:29:16.695 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2517053341 2024-07-04T17:29:16.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:16.695 INFO:teuthology.orchestra.run.smithi042.stdout: "inst_str": "client.4792 192.168.0.1:0/2517053341", 2024-07-04T17:29:16.695 INFO:teuthology.orchestra.run.smithi042.stdout: "addr_str": "192.168.0.1:0/2517053341", 2024-07-04T17:29:16.695 INFO:teuthology.orchestra.run.smithi042.stdout: "inode_count": 1, 2024-07-04T17:29:16.695 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_epoch": 36, 2024-07-04T17:29:16.695 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch": 50, 2024-07-04T17:29:16.695 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch_barrier": 0, 2024-07-04T17:29:16.696 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklisted": false, 2024-07-04T17:29:16.696 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs" 2024-07-04T17:29:16.696 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:29:16.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:29:16.713 DEBUG:teuthology.orchestra.run.smithi042:> stat --file-system '--printf=%T 2024-07-04T17:29:16.714 DEBUG:teuthology.orchestra.run.smithi042:> ' -- /home/ubuntu/cephtest/mnt.0 2024-07-04T17:29:16.765 INFO:teuthology.orchestra.run.smithi042.stdout:fuseblk 2024-07-04T17:29:16.765 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-07-04T17:29:16.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:29:16.766 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-04T17:29:16.859 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-04T17:29:17.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.150+0000 7fadf2800640 1 -- 172.21.15.42:0/1054161054 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadec144fe0 msgr2=0x7fadec06c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:17.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.150+0000 7fadf2800640 1 --2- 172.21.15.42:0/1054161054 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadec144fe0 0x7fadec06c020 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fadd8005fd0 tx=0x7fadd802a5f0 comp rx=0 tx=0).stop 2024-07-04T17:29:17.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.150+0000 7fadf2800640 1 -- 172.21.15.42:0/1054161054 shutdown_connections 2024-07-04T17:29:17.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.150+0000 7fadf2800640 1 --2- 172.21.15.42:0/1054161054 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadec144fe0 0x7fadec06c020 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:17.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.150+0000 7fadf2800640 1 -- 172.21.15.42:0/1054161054 >> 172.21.15.42:0/1054161054 conn(0x7fadec0bebc0 msgr2=0x7fadec0befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:17.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.150+0000 7fadf2800640 1 -- 172.21.15.42:0/1054161054 shutdown_connections 2024-07-04T17:29:17.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.150+0000 7fadf2800640 1 -- 172.21.15.42:0/1054161054 wait complete. 2024-07-04T17:29:17.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.153+0000 7fadf2800640 1 Processor -- start 2024-07-04T17:29:17.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.153+0000 7fadf2800640 1 -- start start 2024-07-04T17:29:17.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.154+0000 7fadf2800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadec0ba830 0x7fadec143160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:17.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.154+0000 7fadf2800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fadec0b1e30 con 0x7fadec0ba830 2024-07-04T17:29:17.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.154+0000 7fadf1400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadec0ba830 0x7fadec143160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:17.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.154+0000 7fadf1400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadec0ba830 0x7fadec143160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44072/0 (socket says 172.21.15.42:44072) 2024-07-04T17:29:17.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.154+0000 7fadf1400640 1 -- 172.21.15.42:0/3262126428 learned_addr learned my addr 172.21.15.42:0/3262126428 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:17.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.155+0000 7fadf1400640 1 -- 172.21.15.42:0/3262126428 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadec142c70 con 0x7fadec0ba830 2024-07-04T17:29:17.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.155+0000 7fadf1400640 1 --2- 172.21.15.42:0/3262126428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadec0ba830 0x7fadec143160 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fade000c700 tx=0x7fade000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:17.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.155+0000 7fade9600640 1 -- 172.21.15.42:0/3262126428 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fade0003970 con 0x7fadec0ba830 2024-07-04T17:29:17.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.155+0000 7fade9600640 1 -- 172.21.15.42:0/3262126428 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fade0003b10 con 0x7fadec0ba830 2024-07-04T17:29:17.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.155+0000 7fade9600640 1 -- 172.21.15.42:0/3262126428 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fade000a820 con 0x7fadec0ba830 2024-07-04T17:29:17.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.155+0000 7fadf2800640 1 -- 172.21.15.42:0/3262126428 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadec1401c0 con 0x7fadec0ba830 2024-07-04T17:29:17.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.156+0000 7fadf2800640 1 -- 172.21.15.42:0/3262126428 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadec13e2e0 con 0x7fadec0ba830 2024-07-04T17:29:17.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.157+0000 7fadf2800640 1 -- 172.21.15.42:0/3262126428 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadec06c020 con 0x7fadec0ba830 2024-07-04T17:29:17.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.157+0000 7fade9600640 1 -- 172.21.15.42:0/3262126428 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7fade000a9c0 con 0x7fadec0ba830 2024-07-04T17:29:17.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.158+0000 7fade9600640 1 --2- 172.21.15.42:0/3262126428 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fadcc03d570 0x7fadcc03fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:17.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.158+0000 7fade9600640 1 -- 172.21.15.42:0/3262126428 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 3439+0+0 (secure 0 0 0) 0x7fade00517e0 con 0x7fadec0ba830 2024-07-04T17:29:17.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.158+0000 7fadf0a00640 1 --2- 172.21.15.42:0/3262126428 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fadcc03d570 0x7fadcc03fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:17.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.159+0000 7fadf0a00640 1 --2- 172.21.15.42:0/3262126428 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fadcc03d570 0x7fadcc03fa80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fadd8033000 tx=0x7fadd8004730 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:17.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.163+0000 7fade9600640 1 -- 172.21.15.42:0/3262126428 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fadec13e2e0 con 0x7fadec0ba830 2024-07-04T17:29:17.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.412+0000 7fadf2800640 1 -- 172.21.15.42:0/3262126428 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fadec068e30 con 0x7fadec0ba830 2024-07-04T17:29:17.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.412+0000 7fade9600640 1 -- 172.21.15.42:0/3262126428 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) ==== 53+0+73 (secure 0 0 0) 0x7fade000d150 con 0x7fadec0ba830 2024-07-04T17:29:17.414 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-04T17:29:17.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.415+0000 7fadf2800640 1 -- 172.21.15.42:0/3262126428 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fadcc03d570 msgr2=0x7fadcc03fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:17.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.415+0000 7fadf2800640 1 --2- 172.21.15.42:0/3262126428 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fadcc03d570 0x7fadcc03fa80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fadd8033000 tx=0x7fadd8004730 comp rx=0 tx=0).stop 2024-07-04T17:29:17.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.415+0000 7fadf2800640 1 -- 172.21.15.42:0/3262126428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadec0ba830 msgr2=0x7fadec143160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:17.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.415+0000 7fadf2800640 1 --2- 172.21.15.42:0/3262126428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadec0ba830 0x7fadec143160 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fade000c700 tx=0x7fade000cbc0 comp rx=0 tx=0).stop 2024-07-04T17:29:17.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.416+0000 7fadf2800640 1 -- 172.21.15.42:0/3262126428 shutdown_connections 2024-07-04T17:29:17.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.416+0000 7fadf2800640 1 --2- 172.21.15.42:0/3262126428 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fadcc03d570 0x7fadcc03fa80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:17.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.416+0000 7fadf2800640 1 --2- 172.21.15.42:0/3262126428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadec0ba830 0x7fadec143160 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:17.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.416+0000 7fadf2800640 1 -- 172.21.15.42:0/3262126428 >> 172.21.15.42:0/3262126428 conn(0x7fadec0bebc0 msgr2=0x7fadec0b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:17.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.416+0000 7fadf2800640 1 -- 172.21.15.42:0/3262126428 shutdown_connections 2024-07-04T17:29:17.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17.416+0000 7fadf2800640 1 -- 172.21.15.42:0/3262126428 wait complete. 2024-07-04T17:29:17.443 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-07-04T17:29:17.443 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-04T17:29:17.443 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2024-07-04T17:29:17.443 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-07-04T17:29:17.443 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-07-04T17:29:17.443 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-07-04T17:29:17.443 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-07-04T17:29:17.443 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-04T17:29:17.443 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-07-04T17:29:17.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:29:17.444 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2024-07-04T17:29:17.496 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-04T17:29:17.496 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-04T17:29:17.496 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-07-04T17:29:17.496 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:29:17.496 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-07-04T17:29:17.497 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:29:17.497 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:29:17.497 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2024-07-04T17:29:17.497 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f0 2024-07-04T17:29:17.497 INFO:teuthology.orchestra.run.smithi042.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:29:17.497 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2024-07-04T17:29:17.497 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f1 2024-07-04T17:29:17.497 INFO:teuthology.orchestra.run.smithi042.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-04T17:29:17.497 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:29:17.497 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f0 2024-07-04T17:29:17.497 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-07-04T17:29:17.497 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 42413sec preferred_lft 42413sec 2024-07-04T17:29:17.497 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::ec4:7aff:febd:1440/64 scope link noprefixroute 2024-07-04T17:29:17.497 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:29:17.498 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:29:17.498 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:29:17.498 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f1 2024-07-04T17:29:17.498 INFO:teuthology.orchestra.run.smithi042.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-04T17:29:17.498 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 32:75:d6:f1:31:e1 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:29:17.498 INFO:teuthology.orchestra.run.smithi042.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-04T17:29:17.498 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:29:17.498 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::8839:a9ff:fe74:17f5/64 scope link 2024-07-04T17:29:17.498 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:29:17.498 INFO:teuthology.orchestra.run.smithi042.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-04T17:29:17.498 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether e6:ab:57:fb:ca:9e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:29:17.498 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::e4ab:57ff:fefb:ca9e/64 scope link 2024-07-04T17:29:17.499 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:29:17.499 INFO:teuthology.orchestra.run.smithi042.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-04T17:29:17.499 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 32:75:d6:f1:31:e1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:29:17.499 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::3075:d6ff:fef1:31e1/64 scope link 2024-07-04T17:29:17.499 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:29:17.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:29:17.500 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-04T17:29:17.578 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-07-04T17:29:17.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:29:17.583 DEBUG:teuthology.orchestra.run.smithi042:> route 2024-07-04T17:29:17.634 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2024-07-04T17:29:17.635 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-04T17:29:17.635 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-04T17:29:17.635 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-04T17:29:17.635 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-04T17:29:17.636 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-04T17:29:17.636 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-07-04T17:29:17.636 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-04T17:29:17.636 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-04T17:29:17.636 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-04T17:29:17.636 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-07-04T17:29:17.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-04T17:29:17.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:29:17.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:29:17.863 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-07-04T17:29:17.909 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-04T17:29:17.910 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2024-07-04T17:29:17.911 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-07-04T17:29:17.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:29:17.911 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.1 up 2024-07-04T17:29:18.045 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-07-04T17:29:18.045 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-04T17:29:18.045 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-07-04T17:29:18.106 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-07-04T17:29:18.107 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-04T17:29:18.107 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-07-04T17:29:18.160 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:29:18.247 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:29:18.308 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:29:18.308 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:29:18.308 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:29:18.308 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:29:18.308 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:29:18.308 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:29:18.308 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:29:18.308 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:29:18.308 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:29:18.309 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:29:18.309 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:29:18.309 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:29:18.309 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:29:18.309 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:29:18.309 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:29:18.309 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:29:18.309 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:29:18.309 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:29:18.309 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:29:18.309 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:29:18.309 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:29:18.309 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:29:18.309 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:29:18.310 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:29:18.310 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:29:18.310 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:29:18.310 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:29:18.310 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:29:18.310 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:29:18.310 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:29:18.310 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:29:18.310 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:29:18.310 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:29:18.310 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-07-04T17:29:18.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:29:18.311 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:29:18.368 INFO:teuthology.orchestra.run.smithi042.stdout:42 2024-07-04T17:29:18.368 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-07-04T17:29:18.369 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-07-04T17:29:18.374 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:29:18.448 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:29:18.493 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:29:18.493 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:29:18.493 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:29:18.493 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:29:18.493 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:29:18.493 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:29:18.493 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:29:18.493 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:29:18.493 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:29:18.494 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:29:18.494 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:29:18.494 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:29:18.494 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:29:18.494 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:29:18.494 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:29:18.494 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:29:18.494 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:29:18.494 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:29:18.494 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:29:18.494 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:29:18.494 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:29:18.494 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:29:18.495 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:29:18.495 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:29:18.495 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:29:18.495 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:29:18.495 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:29:18.495 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:29:18.495 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:29:18.495 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:29:18.495 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:29:18.495 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:29:18.495 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:29:18.495 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-07-04T17:29:18.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:29:18.496 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:29:18.590 INFO:teuthology.orchestra.run.smithi042.stdout:42 2024-07-04T17:29:18.625 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi042.stderr:2024-07-04T17:29:18.624+0000 7f992400e480 -1 init, newargv = 0x7f9908001080 newargc=15 2024-07-04T17:29:18.625 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi042.stderr:ceph-fuse[51739]: starting ceph client 2024-07-04T17:29:18.633 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi042.stderr:ceph-fuse[51739]: starting fuse 2024-07-04T17:29:19.691 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:29:19.788 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:29:19.850 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:29:19.850 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:29:19.850 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:29:19.850 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:29:19.850 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:29:19.850 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:29:19.851 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:29:19.851 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:29:19.851 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:29:19.851 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:29:19.851 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:29:19.851 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:29:19.851 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:29:19.851 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:29:19.851 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:29:19.851 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:29:19.851 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:29:19.851 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:29:19.851 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:29:19.851 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:29:19.852 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:29:19.852 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:29:19.852 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:29:19.852 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:29:19.852 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:29:19.852 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:29:19.852 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:29:19.852 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:29:19.852 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:29:19.852 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:29:19.852 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:29:19.852 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:29:19.853 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:29:19.853 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-07-04T17:29:19.853 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.1 2024-07-04T17:29:19.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:29:19.854 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:29:19.909 INFO:teuthology.orchestra.run.smithi042.stdout:42 2024-07-04T17:29:19.909 INFO:teuthology.orchestra.run.smithi042.stdout:43 2024-07-04T17:29:19.909 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 43] 2024-07-04T17:29:19.910 DEBUG:teuthology.orchestra.run.smithi042:> sudo stdin-killer -- python3 -c ' 2024-07-04T17:29:19.910 DEBUG:teuthology.orchestra.run.smithi042:> import glob 2024-07-04T17:29:19.910 DEBUG:teuthology.orchestra.run.smithi042:> import re 2024-07-04T17:29:19.910 DEBUG:teuthology.orchestra.run.smithi042:> import os 2024-07-04T17:29:19.910 DEBUG:teuthology.orchestra.run.smithi042:> import subprocess 2024-07-04T17:29:19.910 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:29:19.910 DEBUG:teuthology.orchestra.run.smithi042:> def _find_admin_socket(client_name): 2024-07-04T17:29:19.910 DEBUG:teuthology.orchestra.run.smithi042:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-07-04T17:29:19.910 DEBUG:teuthology.orchestra.run.smithi042:> files = glob.glob(asok_path) 2024-07-04T17:29:19.910 DEBUG:teuthology.orchestra.run.smithi042:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-07-04T17:29:19.910 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:29:19.910 DEBUG:teuthology.orchestra.run.smithi042:> # Given a non-glob path, it better be there 2024-07-04T17:29:19.911 DEBUG:teuthology.orchestra.run.smithi042:> if "*" not in asok_path: 2024-07-04T17:29:19.911 DEBUG:teuthology.orchestra.run.smithi042:> assert(len(files) == 1) 2024-07-04T17:29:19.911 DEBUG:teuthology.orchestra.run.smithi042:> return files[0] 2024-07-04T17:29:19.911 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:29:19.911 DEBUG:teuthology.orchestra.run.smithi042:> for f in files: 2024-07-04T17:29:19.911 DEBUG:teuthology.orchestra.run.smithi042:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-07-04T17:29:19.911 DEBUG:teuthology.orchestra.run.smithi042:> if os.path.exists("/proc/{0}".format(pid)): 2024-07-04T17:29:19.911 DEBUG:teuthology.orchestra.run.smithi042:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-07-04T17:29:19.911 DEBUG:teuthology.orchestra.run.smithi042:> contents = proc_f.read() 2024-07-04T17:29:19.911 DEBUG:teuthology.orchestra.run.smithi042:> if mountpoint in contents: 2024-07-04T17:29:19.911 DEBUG:teuthology.orchestra.run.smithi042:> return f 2024-07-04T17:29:19.911 DEBUG:teuthology.orchestra.run.smithi042:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-07-04T17:29:19.911 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:29:19.911 DEBUG:teuthology.orchestra.run.smithi042:> print(_find_admin_socket("client.1")) 2024-07-04T17:29:19.912 DEBUG:teuthology.orchestra.run.smithi042:> ' 2024-07-04T17:29:20.089 INFO:teuthology.orchestra.run.smithi042.stdout:/var/run/ceph/ceph-client.1.51739.asok 2024-07-04T17:29:20.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:29:20.108 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.51739.asok 2024-07-04T17:29:20.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:29:20.109 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.51739.asok status 2024-07-04T17:29:20.416 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:29:20.416 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": { 2024-07-04T17:29:20.417 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_sha1": "9dba1e836b4756bf3a32be2a4936f371dff55008", 2024-07-04T17:29:20.417 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.0.0-4667-g9dba1e83 (9dba1e836b4756bf3a32be2a4936f371dff55008) squid (dev)", 2024-07-04T17:29:20.417 INFO:teuthology.orchestra.run.smithi042.stdout: "entity_id": "1", 2024-07-04T17:29:20.417 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": "smithi042", 2024-07-04T17:29:20.417 INFO:teuthology.orchestra.run.smithi042.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-07-04T17:29:20.417 INFO:teuthology.orchestra.run.smithi042.stdout: "pid": "51739", 2024-07-04T17:29:20.417 INFO:teuthology.orchestra.run.smithi042.stdout: "root": "/" 2024-07-04T17:29:20.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:20.417 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_count": 0, 2024-07-04T17:29:20.417 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_pinned_count": 0, 2024-07-04T17:29:20.417 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 4798, 2024-07-04T17:29:20.417 INFO:teuthology.orchestra.run.smithi042.stdout: "inst": { 2024-07-04T17:29:20.418 INFO:teuthology.orchestra.run.smithi042.stdout: "name": { 2024-07-04T17:29:20.418 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "client", 2024-07-04T17:29:20.418 INFO:teuthology.orchestra.run.smithi042.stdout: "num": 4798 2024-07-04T17:29:20.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:20.418 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-07-04T17:29:20.418 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-07-04T17:29:20.418 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.2:0", 2024-07-04T17:29:20.418 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 952593627 2024-07-04T17:29:20.418 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:20.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:20.418 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-07-04T17:29:20.418 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-07-04T17:29:20.418 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.2:0", 2024-07-04T17:29:20.419 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 952593627 2024-07-04T17:29:20.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:20.419 INFO:teuthology.orchestra.run.smithi042.stdout: "inst_str": "client.4798 192.168.0.2:0/952593627", 2024-07-04T17:29:20.419 INFO:teuthology.orchestra.run.smithi042.stdout: "addr_str": "192.168.0.2:0/952593627", 2024-07-04T17:29:20.419 INFO:teuthology.orchestra.run.smithi042.stdout: "inode_count": 1, 2024-07-04T17:29:20.419 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_epoch": 36, 2024-07-04T17:29:20.419 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch": 50, 2024-07-04T17:29:20.419 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch_barrier": 0, 2024-07-04T17:29:20.419 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklisted": false, 2024-07-04T17:29:20.419 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs" 2024-07-04T17:29:20.419 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:29:20.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:29:20.439 DEBUG:teuthology.orchestra.run.smithi042:> stat --file-system '--printf=%T 2024-07-04T17:29:20.439 DEBUG:teuthology.orchestra.run.smithi042:> ' -- /home/ubuntu/cephtest/mnt.1 2024-07-04T17:29:20.494 INFO:teuthology.orchestra.run.smithi042.stdout:fuseblk 2024-07-04T17:29:20.494 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-07-04T17:29:20.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:29:20.494 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-07-04T17:29:20.584 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-07-04T17:29:20.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:20.886+0000 7f26d8e00640 1 -- 172.21.15.42:0/2688179906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26d40ac5b0 msgr2=0x7f26d40b0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:20.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:20.886+0000 7f26d8e00640 1 --2- 172.21.15.42:0/2688179906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26d40ac5b0 0x7f26d40b0db0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f26d40ac560 tx=0x7f26c402a480 comp rx=0 tx=0).stop 2024-07-04T17:29:20.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:20.886+0000 7f26d8e00640 1 -- 172.21.15.42:0/2688179906 shutdown_connections 2024-07-04T17:29:20.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:20.886+0000 7f26d8e00640 1 --2- 172.21.15.42:0/2688179906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26d40ac5b0 0x7f26d40b0db0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:20.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:20.886+0000 7f26d8e00640 1 -- 172.21.15.42:0/2688179906 >> 172.21.15.42:0/2688179906 conn(0x7f26d40afaa0 msgr2=0x7f26d40afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:20.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:20.886+0000 7f26d8e00640 1 -- 172.21.15.42:0/2688179906 shutdown_connections 2024-07-04T17:29:20.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:20.886+0000 7f26d8e00640 1 -- 172.21.15.42:0/2688179906 wait complete. 2024-07-04T17:29:20.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:20.889+0000 7f26d8e00640 1 Processor -- start 2024-07-04T17:29:20.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:20.889+0000 7f26d8e00640 1 -- start start 2024-07-04T17:29:20.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:20.890+0000 7f26d8e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26d40ac5b0 0x7f26d41d9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:20.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:20.890+0000 7f26d8e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f26d40ae590 con 0x7f26d40ac5b0 2024-07-04T17:29:20.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:20.890+0000 7f26d3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26d40ac5b0 0x7f26d41d9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:20.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:20.890+0000 7f26d3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26d40ac5b0 0x7f26d41d9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44088/0 (socket says 172.21.15.42:44088) 2024-07-04T17:29:20.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:20.890+0000 7f26d3400640 1 -- 172.21.15.42:0/982929469 learned_addr learned my addr 172.21.15.42:0/982929469 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:20.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:20.891+0000 7f26d3400640 1 -- 172.21.15.42:0/982929469 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d4139cd0 con 0x7f26d40ac5b0 2024-07-04T17:29:20.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:20.891+0000 7f26d3400640 1 --2- 172.21.15.42:0/982929469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26d40ac5b0 0x7f26d41d9410 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f26c402aa70 tx=0x7f26c4033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:20.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:20.891+0000 7f26cbe00640 1 -- 172.21.15.42:0/982929469 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26c40319b0 con 0x7f26d40ac5b0 2024-07-04T17:29:20.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:20.892+0000 7f26cbe00640 1 -- 172.21.15.42:0/982929469 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26c4031c40 con 0x7f26d40ac5b0 2024-07-04T17:29:20.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:20.892+0000 7f26cbe00640 1 -- 172.21.15.42:0/982929469 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26c4033070 con 0x7f26d40ac5b0 2024-07-04T17:29:20.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:20.892+0000 7f26d8e00640 1 -- 172.21.15.42:0/982929469 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26d41d82d0 con 0x7f26d40ac5b0 2024-07-04T17:29:20.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:20.892+0000 7f26d8e00640 1 -- 172.21.15.42:0/982929469 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26d41438c0 con 0x7f26d40ac5b0 2024-07-04T17:29:20.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:20.893+0000 7f26d8e00640 1 -- 172.21.15.42:0/982929469 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26d40b0db0 con 0x7f26d40ac5b0 2024-07-04T17:29:20.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:20.893+0000 7f26cbe00640 1 -- 172.21.15.42:0/982929469 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f26c4033210 con 0x7f26d40ac5b0 2024-07-04T17:29:20.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:20.894+0000 7f26cbe00640 1 --2- 172.21.15.42:0/982929469 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f26b003d4f0 0x7f26b003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:20.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:20.894+0000 7f26cbe00640 1 -- 172.21.15.42:0/982929469 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 3439+0+0 (secure 0 0 0) 0x7f26c4072dc0 con 0x7f26d40ac5b0 2024-07-04T17:29:20.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:20.894+0000 7f26caa00640 1 --2- 172.21.15.42:0/982929469 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f26b003d4f0 0x7f26b003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:20.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:20.895+0000 7f26caa00640 1 --2- 172.21.15.42:0/982929469 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f26b003d4f0 0x7f26b003fa00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f26bc002020 tx=0x7f26bc001670 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:20.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:20.899+0000 7f26cbe00640 1 -- 172.21.15.42:0/982929469 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f26c403f030 con 0x7f26d40ac5b0 2024-07-04T17:29:21.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:21.151+0000 7f26d8e00640 1 -- 172.21.15.42:0/982929469 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f26d4140200 con 0x7f26d40ac5b0 2024-07-04T17:29:21.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:21.826+0000 7f26cbe00640 1 -- 172.21.15.42:0/982929469 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v37) ==== 136+0+0 (secure 0 0 0) 0x7f26c40335b0 con 0x7f26d40ac5b0 2024-07-04T17:29:21.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:21.831+0000 7f26d8e00640 1 -- 172.21.15.42:0/982929469 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f26b003d4f0 msgr2=0x7f26b003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:21.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:21.831+0000 7f26d8e00640 1 --2- 172.21.15.42:0/982929469 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f26b003d4f0 0x7f26b003fa00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f26bc002020 tx=0x7f26bc001670 comp rx=0 tx=0).stop 2024-07-04T17:29:21.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:21.831+0000 7f26d8e00640 1 -- 172.21.15.42:0/982929469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26d40ac5b0 msgr2=0x7f26d41d9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:21.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:21.831+0000 7f26d8e00640 1 --2- 172.21.15.42:0/982929469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26d40ac5b0 0x7f26d41d9410 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f26c402aa70 tx=0x7f26c4033ad0 comp rx=0 tx=0).stop 2024-07-04T17:29:21.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:21.831+0000 7f26d8e00640 1 -- 172.21.15.42:0/982929469 shutdown_connections 2024-07-04T17:29:21.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:21.831+0000 7f26d8e00640 1 --2- 172.21.15.42:0/982929469 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f26b003d4f0 0x7f26b003fa00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:21.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:21.831+0000 7f26d8e00640 1 --2- 172.21.15.42:0/982929469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26d40ac5b0 0x7f26d41d9410 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:21.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:21.831+0000 7f26d8e00640 1 -- 172.21.15.42:0/982929469 >> 172.21.15.42:0/982929469 conn(0x7f26d40afaa0 msgr2=0x7f26d41420b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:21.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:21.831+0000 7f26d8e00640 1 -- 172.21.15.42:0/982929469 shutdown_connections 2024-07-04T17:29:21.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:21.831+0000 7f26d8e00640 1 -- 172.21.15.42:0/982929469 wait complete. 2024-07-04T17:29:21.861 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-04T17:29:22.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.143+0000 7faddb800640 1 -- 172.21.15.42:0/1544969249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadd406cea0 msgr2=0x7fadd40b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:22.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.143+0000 7faddb800640 1 --2- 172.21.15.42:0/1544969249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadd406cea0 0x7fadd40b3030 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fadc4005fd0 tx=0x7fadc402a8b0 comp rx=0 tx=0).stop 2024-07-04T17:29:22.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.143+0000 7faddb800640 1 -- 172.21.15.42:0/1544969249 shutdown_connections 2024-07-04T17:29:22.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.143+0000 7faddb800640 1 --2- 172.21.15.42:0/1544969249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadd406cea0 0x7fadd40b3030 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:22.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.143+0000 7faddb800640 1 -- 172.21.15.42:0/1544969249 >> 172.21.15.42:0/1544969249 conn(0x7fadd40bf700 msgr2=0x7fadd40af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:22.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.143+0000 7faddb800640 1 -- 172.21.15.42:0/1544969249 shutdown_connections 2024-07-04T17:29:22.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.143+0000 7faddb800640 1 -- 172.21.15.42:0/1544969249 wait complete. 2024-07-04T17:29:22.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.145+0000 7faddb800640 1 Processor -- start 2024-07-04T17:29:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.146+0000 7faddb800640 1 -- start start 2024-07-04T17:29:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.147+0000 7faddb800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadd406cea0 0x7fadd4073a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.147+0000 7faddb800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fadd406ef50 con 0x7fadd406cea0 2024-07-04T17:29:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.147+0000 7fadd8e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadd406cea0 0x7fadd4073a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.147+0000 7fadd8e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadd406cea0 0x7fadd4073a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44110/0 (socket says 172.21.15.42:44110) 2024-07-04T17:29:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.147+0000 7fadd8e00640 1 -- 172.21.15.42:0/3470355687 learned_addr learned my addr 172.21.15.42:0/3470355687 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.148+0000 7fadd8e00640 1 -- 172.21.15.42:0/3470355687 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd4072dc0 con 0x7fadd406cea0 2024-07-04T17:29:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.148+0000 7fadd8e00640 1 --2- 172.21.15.42:0/3470355687 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadd406cea0 0x7fadd4073a10 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fadc4009c50 tx=0x7fadc4031650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.148+0000 7fadd0c00640 1 -- 172.21.15.42:0/3470355687 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadc4004250 con 0x7fadd406cea0 2024-07-04T17:29:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.148+0000 7fadd0c00640 1 -- 172.21.15.42:0/3470355687 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fadc40044e0 con 0x7fadd406cea0 2024-07-04T17:29:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.148+0000 7faddb800640 1 -- 172.21.15.42:0/3470355687 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadd4072a80 con 0x7fadd406cea0 2024-07-04T17:29:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.148+0000 7fadd0c00640 1 -- 172.21.15.42:0/3470355687 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadc4004860 con 0x7fadd406cea0 2024-07-04T17:29:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.148+0000 7faddb800640 1 -- 172.21.15.42:0/3470355687 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadd40732c0 con 0x7fadd406cea0 2024-07-04T17:29:22.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.150+0000 7faddb800640 1 -- 172.21.15.42:0/3470355687 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadd40b3030 con 0x7fadd406cea0 2024-07-04T17:29:22.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.150+0000 7fadd0c00640 1 -- 172.21.15.42:0/3470355687 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7fadc403b410 con 0x7fadd406cea0 2024-07-04T17:29:22.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.150+0000 7fadd0c00640 1 --2- 172.21.15.42:0/3470355687 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fadb003d4f0 0x7fadb003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:22.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.151+0000 7fadd0c00640 1 -- 172.21.15.42:0/3470355687 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 3439+0+0 (secure 0 0 0) 0x7fadc4073230 con 0x7fadd406cea0 2024-07-04T17:29:22.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.151+0000 7fadd3e00640 1 --2- 172.21.15.42:0/3470355687 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fadb003d4f0 0x7fadb003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:22.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.152+0000 7fadd3e00640 1 --2- 172.21.15.42:0/3470355687 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fadb003d4f0 0x7fadb003fa00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fadbc007fd0 tx=0x7fadbc007920 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:22.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.156+0000 7fadd0c00640 1 -- 172.21.15.42:0/3470355687 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fadc403f030 con 0x7fadd406cea0 2024-07-04T17:29:22.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.368+0000 7faddb800640 1 -- 172.21.15.42:0/3470355687 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fadd4072fe0 con 0x7fadd406cea0 2024-07-04T17:29:22.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.368+0000 7fadd0c00640 1 -- 172.21.15.42:0/3470355687 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v50) ==== 84+0+207 (secure 0 0 0) 0x7fadc4008220 con 0x7fadd406cea0 2024-07-04T17:29:22.370 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:29:22.370 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-07-04T17:29:22.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:22.370 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 1, 2024-07-04T17:29:22.370 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": ".mgr" 2024-07-04T17:29:22.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:22.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:22.370 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 8, 2024-07-04T17:29:22.370 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": "cephfs_metadata" 2024-07-04T17:29:22.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:22.371 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:22.371 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 9, 2024-07-04T17:29:22.371 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": "cephfs_data" 2024-07-04T17:29:22.371 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:22.371 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-07-04T17:29:22.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.371+0000 7faddb800640 1 -- 172.21.15.42:0/3470355687 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fadb003d4f0 msgr2=0x7fadb003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:22.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.371+0000 7faddb800640 1 --2- 172.21.15.42:0/3470355687 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fadb003d4f0 0x7fadb003fa00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fadbc007fd0 tx=0x7fadbc007920 comp rx=0 tx=0).stop 2024-07-04T17:29:22.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.371+0000 7faddb800640 1 -- 172.21.15.42:0/3470355687 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadd406cea0 msgr2=0x7fadd4073a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:22.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.371+0000 7faddb800640 1 --2- 172.21.15.42:0/3470355687 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadd406cea0 0x7fadd4073a10 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fadc4009c50 tx=0x7fadc4031650 comp rx=0 tx=0).stop 2024-07-04T17:29:22.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.371+0000 7faddb800640 1 -- 172.21.15.42:0/3470355687 shutdown_connections 2024-07-04T17:29:22.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.371+0000 7faddb800640 1 --2- 172.21.15.42:0/3470355687 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fadb003d4f0 0x7fadb003fa00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:22.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.371+0000 7faddb800640 1 --2- 172.21.15.42:0/3470355687 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadd406cea0 0x7fadd4073a10 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:22.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.371+0000 7faddb800640 1 -- 172.21.15.42:0/3470355687 >> 172.21.15.42:0/3470355687 conn(0x7fadd40bf700 msgr2=0x7fadd40aa3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:22.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.372+0000 7faddb800640 1 -- 172.21.15.42:0/3470355687 shutdown_connections 2024-07-04T17:29:22.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.372+0000 7faddb800640 1 -- 172.21.15.42:0/3470355687 wait complete. 2024-07-04T17:29:22.399 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'backup_fs' 2024-07-04T17:29:22.399 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_metadata --pg_num_min 64 2024-07-04T17:29:22.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.696+0000 7fb578000640 1 -- 172.21.15.42:0/929328012 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5700b3040 msgr2=0x7fb5700b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:22.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.696+0000 7fb578000640 1 --2- 172.21.15.42:0/929328012 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5700b3040 0x7fb5700b01f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb56400b2b0 tx=0x7fb56402ad10 comp rx=0 tx=0).stop 2024-07-04T17:29:22.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.697+0000 7fb578000640 1 -- 172.21.15.42:0/929328012 shutdown_connections 2024-07-04T17:29:22.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.697+0000 7fb578000640 1 --2- 172.21.15.42:0/929328012 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5700b3040 0x7fb5700b01f0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:22.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.697+0000 7fb578000640 1 -- 172.21.15.42:0/929328012 >> 172.21.15.42:0/929328012 conn(0x7fb5700bf700 msgr2=0x7fb5700af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:22.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.697+0000 7fb578000640 1 -- 172.21.15.42:0/929328012 shutdown_connections 2024-07-04T17:29:22.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.697+0000 7fb578000640 1 -- 172.21.15.42:0/929328012 wait complete. 2024-07-04T17:29:22.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.699+0000 7fb578000640 1 Processor -- start 2024-07-04T17:29:22.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.699+0000 7fb578000640 1 -- start start 2024-07-04T17:29:22.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.700+0000 7fb578000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5700a94d0 0x7fb5701d73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:22.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.700+0000 7fb578000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb5700a9c70 con 0x7fb5700a94d0 2024-07-04T17:29:22.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.701+0000 7fb575600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5700a94d0 0x7fb5701d73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:22.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.701+0000 7fb575600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5700a94d0 0x7fb5701d73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44128/0 (socket says 172.21.15.42:44128) 2024-07-04T17:29:22.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.701+0000 7fb575600640 1 -- 172.21.15.42:0/1219388741 learned_addr learned my addr 172.21.15.42:0/1219388741 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:22.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.701+0000 7fb575600640 1 -- 172.21.15.42:0/1219388741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5700ac7a0 con 0x7fb5700a94d0 2024-07-04T17:29:22.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.701+0000 7fb575600640 1 --2- 172.21.15.42:0/1219388741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5700a94d0 0x7fb5701d73a0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb56402b750 tx=0x7fb5640097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:22.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.702+0000 7fb56d600640 1 -- 172.21.15.42:0/1219388741 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb564038040 con 0x7fb5700a94d0 2024-07-04T17:29:22.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.702+0000 7fb56d600640 1 -- 172.21.15.42:0/1219388741 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb564004180 con 0x7fb5700a94d0 2024-07-04T17:29:22.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.702+0000 7fb56d600640 1 -- 172.21.15.42:0/1219388741 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5640044a0 con 0x7fb5700a94d0 2024-07-04T17:29:22.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.702+0000 7fb578000640 1 -- 172.21.15.42:0/1219388741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb570145d40 con 0x7fb5700a94d0 2024-07-04T17:29:22.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.702+0000 7fb578000640 1 -- 172.21.15.42:0/1219388741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5700ade90 con 0x7fb5700a94d0 2024-07-04T17:29:22.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.703+0000 7fb578000640 1 -- 172.21.15.42:0/1219388741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5700b01f0 con 0x7fb5700a94d0 2024-07-04T17:29:22.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.703+0000 7fb56d600640 1 -- 172.21.15.42:0/1219388741 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7fb5640035f0 con 0x7fb5700a94d0 2024-07-04T17:29:22.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.704+0000 7fb56d600640 1 --2- 172.21.15.42:0/1219388741 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fb55003d4f0 0x7fb55003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:22.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.704+0000 7fb56d600640 1 -- 172.21.15.42:0/1219388741 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 3439+0+0 (secure 0 0 0) 0x7fb564072360 con 0x7fb5700a94d0 2024-07-04T17:29:22.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.704+0000 7fb574c00640 1 --2- 172.21.15.42:0/1219388741 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fb55003d4f0 0x7fb55003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:22.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.705+0000 7fb574c00640 1 --2- 172.21.15.42:0/1219388741 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fb55003d4f0 0x7fb55003fa00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb55c001560 tx=0x7fb55c007a90 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:22.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.709+0000 7fb56d600640 1 -- 172.21.15.42:0/1219388741 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fb56404c020 con 0x7fb5700a94d0 2024-07-04T17:29:22.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:22.920+0000 7fb578000640 1 -- 172.21.15.42:0/1219388741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64} v 0) -- 0x7fb5700acef0 con 0x7fb5700a94d0 2024-07-04T17:29:23.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:23.838+0000 7fb56d600640 1 -- 172.21.15.42:0/1219388741 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64}]=0 pool 'backup_fs_metadata' created v51) ==== 144+0+0 (secure 0 0 0) 0x7fb564046360 con 0x7fb5700a94d0 2024-07-04T17:29:23.840 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'backup_fs_metadata' created 2024-07-04T17:29:23.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:23.843+0000 7fb578000640 1 -- 172.21.15.42:0/1219388741 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fb55003d4f0 msgr2=0x7fb55003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:23.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:23.843+0000 7fb578000640 1 --2- 172.21.15.42:0/1219388741 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fb55003d4f0 0x7fb55003fa00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb55c001560 tx=0x7fb55c007a90 comp rx=0 tx=0).stop 2024-07-04T17:29:23.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:23.843+0000 7fb578000640 1 -- 172.21.15.42:0/1219388741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5700a94d0 msgr2=0x7fb5701d73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:23.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:23.843+0000 7fb578000640 1 --2- 172.21.15.42:0/1219388741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5700a94d0 0x7fb5701d73a0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb56402b750 tx=0x7fb5640097d0 comp rx=0 tx=0).stop 2024-07-04T17:29:23.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:23.844+0000 7fb578000640 1 -- 172.21.15.42:0/1219388741 shutdown_connections 2024-07-04T17:29:23.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:23.844+0000 7fb578000640 1 --2- 172.21.15.42:0/1219388741 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fb55003d4f0 0x7fb55003fa00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:23.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:23.844+0000 7fb578000640 1 --2- 172.21.15.42:0/1219388741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5700a94d0 0x7fb5701d73a0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:23.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:23.844+0000 7fb578000640 1 -- 172.21.15.42:0/1219388741 >> 172.21.15.42:0/1219388741 conn(0x7fb5700bf700 msgr2=0x7fb5700ae7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:23.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:23.846+0000 7fb578000640 1 -- 172.21.15.42:0/1219388741 shutdown_connections 2024-07-04T17:29:23.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:23.847+0000 7fb578000640 1 -- 172.21.15.42:0/1219388741 wait complete. 2024-07-04T17:29:23.891 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-04T17:29:24.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.161+0000 7f3c54200640 1 -- 172.21.15.42:0/1576005330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c4c0a94d0 msgr2=0x7f3c4c0b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:24.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.161+0000 7f3c54200640 1 --2- 172.21.15.42:0/1576005330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c4c0a94d0 0x7f3c4c0b1090 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3c380097b0 tx=0x7f3c3802a540 comp rx=0 tx=0).stop 2024-07-04T17:29:24.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.161+0000 7f3c54200640 1 -- 172.21.15.42:0/1576005330 shutdown_connections 2024-07-04T17:29:24.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.161+0000 7f3c54200640 1 --2- 172.21.15.42:0/1576005330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c4c0a94d0 0x7f3c4c0b1090 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:24.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.161+0000 7f3c54200640 1 -- 172.21.15.42:0/1576005330 >> 172.21.15.42:0/1576005330 conn(0x7f3c4c06c020 msgr2=0x7f3c4c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:24.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.161+0000 7f3c54200640 1 -- 172.21.15.42:0/1576005330 shutdown_connections 2024-07-04T17:29:24.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.161+0000 7f3c54200640 1 -- 172.21.15.42:0/1576005330 wait complete. 2024-07-04T17:29:24.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.163+0000 7f3c54200640 1 Processor -- start 2024-07-04T17:29:24.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.163+0000 7f3c54200640 1 -- start start 2024-07-04T17:29:24.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.165+0000 7f3c54200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c4c06dcf0 0x7f3c4c1406d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:24.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.165+0000 7f3c54200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c13da50 con 0x7f3c4c06dcf0 2024-07-04T17:29:24.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.165+0000 7f3c51800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c4c06dcf0 0x7f3c4c1406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:24.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.165+0000 7f3c51800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c4c06dcf0 0x7f3c4c1406d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45194/0 (socket says 172.21.15.42:45194) 2024-07-04T17:29:24.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.165+0000 7f3c51800640 1 -- 172.21.15.42:0/3435300493 learned_addr learned my addr 172.21.15.42:0/3435300493 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:24.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.165+0000 7f3c51800640 1 -- 172.21.15.42:0/3435300493 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c4c140d30 con 0x7f3c4c06dcf0 2024-07-04T17:29:24.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.166+0000 7f3c51800640 1 --2- 172.21.15.42:0/3435300493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c4c06dcf0 0x7f3c4c1406d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3c4000c320 tx=0x7f3c4000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:24.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.166+0000 7f3c49600640 1 -- 172.21.15.42:0/3435300493 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c400116c0 con 0x7f3c4c06dcf0 2024-07-04T17:29:24.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.166+0000 7f3c49600640 1 -- 172.21.15.42:0/3435300493 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c40017070 con 0x7f3c4c06dcf0 2024-07-04T17:29:24.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.166+0000 7f3c54200640 1 -- 172.21.15.42:0/3435300493 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c4c13aba0 con 0x7f3c4c06dcf0 2024-07-04T17:29:24.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.166+0000 7f3c49600640 1 -- 172.21.15.42:0/3435300493 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c40011b00 con 0x7f3c4c06dcf0 2024-07-04T17:29:24.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.166+0000 7f3c54200640 1 -- 172.21.15.42:0/3435300493 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c4c141250 con 0x7f3c4c06dcf0 2024-07-04T17:29:24.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.167+0000 7f3c54200640 1 -- 172.21.15.42:0/3435300493 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c4c0b1090 con 0x7f3c4c06dcf0 2024-07-04T17:29:24.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.168+0000 7f3c49600640 1 -- 172.21.15.42:0/3435300493 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f3c40011ca0 con 0x7f3c4c06dcf0 2024-07-04T17:29:24.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.168+0000 7f3c49600640 1 --2- 172.21.15.42:0/3435300493 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f3c2c03d4f0 0x7f3c2c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:24.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.168+0000 7f3c49600640 1 -- 172.21.15.42:0/3435300493 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 3812+0+0 (secure 0 0 0) 0x7f3c40051770 con 0x7f3c4c06dcf0 2024-07-04T17:29:24.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.169+0000 7f3c50e00640 1 --2- 172.21.15.42:0/3435300493 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f3c2c03d4f0 0x7f3c2c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:24.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.169+0000 7f3c50e00640 1 --2- 172.21.15.42:0/3435300493 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f3c2c03d4f0 0x7f3c2c03fa00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3c380097b0 tx=0x7f3c38004730 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:24.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.174+0000 7f3c49600640 1 -- 172.21.15.42:0/3435300493 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f3c4000a140 con 0x7f3c4c06dcf0 2024-07-04T17:29:24.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.387+0000 7f3c54200640 1 -- 172.21.15.42:0/3435300493 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f3c4c14db20 con 0x7f3c4c06dcf0 2024-07-04T17:29:24.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.843+0000 7f3c49600640 1 -- 172.21.15.42:0/3435300493 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'backup_fs_data' created v52) ==== 176+0+0 (secure 0 0 0) 0x7f3c40010cf0 con 0x7f3c4c06dcf0 2024-07-04T17:29:24.845 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'backup_fs_data' created 2024-07-04T17:29:24.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.847+0000 7f3c54200640 1 -- 172.21.15.42:0/3435300493 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f3c2c03d4f0 msgr2=0x7f3c2c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:24.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.847+0000 7f3c54200640 1 --2- 172.21.15.42:0/3435300493 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f3c2c03d4f0 0x7f3c2c03fa00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3c380097b0 tx=0x7f3c38004730 comp rx=0 tx=0).stop 2024-07-04T17:29:24.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.847+0000 7f3c54200640 1 -- 172.21.15.42:0/3435300493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c4c06dcf0 msgr2=0x7f3c4c1406d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:24.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.847+0000 7f3c54200640 1 --2- 172.21.15.42:0/3435300493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c4c06dcf0 0x7f3c4c1406d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3c4000c320 tx=0x7f3c4000c7e0 comp rx=0 tx=0).stop 2024-07-04T17:29:24.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.847+0000 7f3c54200640 1 -- 172.21.15.42:0/3435300493 shutdown_connections 2024-07-04T17:29:24.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.847+0000 7f3c54200640 1 --2- 172.21.15.42:0/3435300493 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f3c2c03d4f0 0x7f3c2c03fa00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:24.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.847+0000 7f3c54200640 1 --2- 172.21.15.42:0/3435300493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c4c06dcf0 0x7f3c4c1406d0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:24.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.847+0000 7f3c54200640 1 -- 172.21.15.42:0/3435300493 >> 172.21.15.42:0/3435300493 conn(0x7f3c4c06c020 msgr2=0x7f3c4c071a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:24.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.848+0000 7f3c54200640 1 -- 172.21.15.42:0/3435300493 shutdown_connections 2024-07-04T17:29:24.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:24.848+0000 7f3c54200640 1 -- 172.21.15.42:0/3435300493 wait complete. 2024-07-04T17:29:24.907 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new backup_fs backup_fs_metadata backup_fs_data 2024-07-04T17:29:25.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.165+0000 7f5f1b200640 1 -- 172.21.15.42:0/3731859676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f140aff80 msgr2=0x7f5f1406ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:25.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.165+0000 7f5f1b200640 1 --2- 172.21.15.42:0/3731859676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f140aff80 0x7f5f1406ff60 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f5f0400b2b0 tx=0x7f5f0402ad10 comp rx=0 tx=0).stop 2024-07-04T17:29:25.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.165+0000 7f5f1b200640 1 -- 172.21.15.42:0/3731859676 shutdown_connections 2024-07-04T17:29:25.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.165+0000 7f5f1b200640 1 --2- 172.21.15.42:0/3731859676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f140aff80 0x7f5f1406ff60 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:25.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.166+0000 7f5f1b200640 1 -- 172.21.15.42:0/3731859676 >> 172.21.15.42:0/3731859676 conn(0x7f5f140bf4d0 msgr2=0x7f5f140af870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:25.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.166+0000 7f5f1b200640 1 -- 172.21.15.42:0/3731859676 shutdown_connections 2024-07-04T17:29:25.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.166+0000 7f5f1b200640 1 -- 172.21.15.42:0/3731859676 wait complete. 2024-07-04T17:29:25.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.168+0000 7f5f1b200640 1 Processor -- start 2024-07-04T17:29:25.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.168+0000 7f5f1b200640 1 -- start start 2024-07-04T17:29:25.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.169+0000 7f5f1b200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f140aff80 0x7f5f14142040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:25.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.169+0000 7f5f1b200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5f140b2e60 con 0x7f5f140aff80 2024-07-04T17:29:25.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.169+0000 7f5f13e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f140aff80 0x7f5f14142040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:25.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.169+0000 7f5f13e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f140aff80 0x7f5f14142040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45198/0 (socket says 172.21.15.42:45198) 2024-07-04T17:29:25.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.169+0000 7f5f13e00640 1 -- 172.21.15.42:0/3444169368 learned_addr learned my addr 172.21.15.42:0/3444169368 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:25.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.170+0000 7f5f13e00640 1 -- 172.21.15.42:0/3444169368 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f141e6f50 con 0x7f5f140aff80 2024-07-04T17:29:25.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.170+0000 7f5f13e00640 1 --2- 172.21.15.42:0/3444169368 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f140aff80 0x7f5f14142040 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5f04000c00 tx=0x7f5f040097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:25.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.170+0000 7f5f0be00640 1 -- 172.21.15.42:0/3444169368 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f04038040 con 0x7f5f140aff80 2024-07-04T17:29:25.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.170+0000 7f5f0be00640 1 -- 172.21.15.42:0/3444169368 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f04041070 con 0x7f5f140aff80 2024-07-04T17:29:25.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.170+0000 7f5f0be00640 1 -- 172.21.15.42:0/3444169368 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f04004420 con 0x7f5f140aff80 2024-07-04T17:29:25.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.170+0000 7f5f1b200640 1 -- 172.21.15.42:0/3444169368 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f141e6c20 con 0x7f5f140aff80 2024-07-04T17:29:25.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.171+0000 7f5f1b200640 1 -- 172.21.15.42:0/3444169368 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f14072680 con 0x7f5f140aff80 2024-07-04T17:29:25.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.172+0000 7f5f1b200640 1 -- 172.21.15.42:0/3444169368 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f1406ff60 con 0x7f5f140aff80 2024-07-04T17:29:25.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.172+0000 7f5f0be00640 1 -- 172.21.15.42:0/3444169368 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f5f040035f0 con 0x7f5f140aff80 2024-07-04T17:29:25.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.173+0000 7f5f0be00640 1 --2- 172.21.15.42:0/3444169368 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f5ef403d4f0 0x7f5ef403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:25.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.173+0000 7f5f0be00640 1 -- 172.21.15.42:0/3444169368 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4197+0+0 (secure 0 0 0) 0x7f5f040724a0 con 0x7f5f140aff80 2024-07-04T17:29:25.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.173+0000 7f5f13400640 1 --2- 172.21.15.42:0/3444169368 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f5ef403d4f0 0x7f5ef403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:25.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.174+0000 7f5f13400640 1 --2- 172.21.15.42:0/3444169368 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f5ef403d4f0 0x7f5ef403fa00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5f000015c0 tx=0x7f5f00007a90 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:25.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.179+0000 7f5f0be00640 1 -- 172.21.15.42:0/3444169368 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f5f040453c0 con 0x7f5f140aff80 2024-07-04T17:29:25.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.422+0000 7f5f1b200640 1 -- 172.21.15.42:0/3444169368 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"} v 0) -- 0x7f5f14069b70 con 0x7f5f140aff80 2024-07-04T17:29:25.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.875+0000 7f5f0be00640 1 -- 172.21.15.42:0/3444169368 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"}]=0 new fs with metadata pool 10 and data pool 11 v38) ==== 183+0+0 (secure 0 0 0) 0x7f5f04033cf0 con 0x7f5f140aff80 2024-07-04T17:29:25.880 INFO:teuthology.orchestra.run.smithi042.stderr:new fs with metadata pool 10 and data pool 11 2024-07-04T17:29:25.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.885+0000 7f5f1b200640 1 -- 172.21.15.42:0/3444169368 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f5ef403d4f0 msgr2=0x7f5ef403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:25.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.885+0000 7f5f1b200640 1 --2- 172.21.15.42:0/3444169368 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f5ef403d4f0 0x7f5ef403fa00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5f000015c0 tx=0x7f5f00007a90 comp rx=0 tx=0).stop 2024-07-04T17:29:25.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.885+0000 7f5f1b200640 1 -- 172.21.15.42:0/3444169368 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f140aff80 msgr2=0x7f5f14142040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:25.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.885+0000 7f5f1b200640 1 --2- 172.21.15.42:0/3444169368 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f140aff80 0x7f5f14142040 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5f04000c00 tx=0x7f5f040097d0 comp rx=0 tx=0).stop 2024-07-04T17:29:25.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.886+0000 7f5f1b200640 1 -- 172.21.15.42:0/3444169368 shutdown_connections 2024-07-04T17:29:25.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.886+0000 7f5f1b200640 1 --2- 172.21.15.42:0/3444169368 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f5ef403d4f0 0x7f5ef403fa00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:25.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.886+0000 7f5f1b200640 1 --2- 172.21.15.42:0/3444169368 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f140aff80 0x7f5f14142040 secure :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5f04000c00 tx=0x7f5f040097d0 comp rx=0 tx=0).stop 2024-07-04T17:29:25.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.886+0000 7f5f1b200640 1 -- 172.21.15.42:0/3444169368 >> 172.21.15.42:0/3444169368 conn(0x7f5f140bf4d0 msgr2=0x7f5f140bdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:25.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.890+0000 7f5f1b200640 1 -- 172.21.15.42:0/3444169368 shutdown_connections 2024-07-04T17:29:25.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:25.892+0000 7f5f1b200640 1 -- 172.21.15.42:0/3444169368 wait complete. 2024-07-04T17:29:25.945 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:29:26.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.199+0000 7f313a000640 1 -- 172.21.15.42:0/335520627 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f313406cea0 msgr2=0x7f31340b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:26.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.199+0000 7f313a000640 1 --2- 172.21.15.42:0/335520627 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f313406cea0 0x7f31340b3030 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f3124009fd0 tx=0x7f312402a8b0 comp rx=0 tx=0).stop 2024-07-04T17:29:26.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.200+0000 7f313a000640 1 -- 172.21.15.42:0/335520627 shutdown_connections 2024-07-04T17:29:26.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.200+0000 7f313a000640 1 --2- 172.21.15.42:0/335520627 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f313406cea0 0x7f31340b3030 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:26.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.200+0000 7f313a000640 1 -- 172.21.15.42:0/335520627 >> 172.21.15.42:0/335520627 conn(0x7f31340bf700 msgr2=0x7f31340af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:26.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.200+0000 7f313a000640 1 -- 172.21.15.42:0/335520627 shutdown_connections 2024-07-04T17:29:26.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.200+0000 7f313a000640 1 -- 172.21.15.42:0/335520627 wait complete. 2024-07-04T17:29:26.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.202+0000 7f313a000640 1 Processor -- start 2024-07-04T17:29:26.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.202+0000 7f313a000640 1 -- start start 2024-07-04T17:29:26.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.203+0000 7f313a000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f313406cea0 0x7f31341d9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:26.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.203+0000 7f313a000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f313406ef50 con 0x7f313406cea0 2024-07-04T17:29:26.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.204+0000 7f3133400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f313406cea0 0x7f31341d9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:26.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.204+0000 7f3133400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f313406cea0 0x7f31341d9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45212/0 (socket says 172.21.15.42:45212) 2024-07-04T17:29:26.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.204+0000 7f3133400640 1 -- 172.21.15.42:0/4056467884 learned_addr learned my addr 172.21.15.42:0/4056467884 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:26.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.204+0000 7f3133400640 1 -- 172.21.15.42:0/4056467884 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31341dcb60 con 0x7f313406cea0 2024-07-04T17:29:26.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.204+0000 7f3133400640 1 --2- 172.21.15.42:0/4056467884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f313406cea0 0x7f31341d9170 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f3124034040 tx=0x7f312402f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:26.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.205+0000 7f312b400640 1 -- 172.21.15.42:0/4056467884 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3124041030 con 0x7f313406cea0 2024-07-04T17:29:26.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.205+0000 7f312b400640 1 -- 172.21.15.42:0/4056467884 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3124037040 con 0x7f313406cea0 2024-07-04T17:29:26.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.205+0000 7f313a000640 1 -- 172.21.15.42:0/4056467884 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3134070c00 con 0x7f313406cea0 2024-07-04T17:29:26.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.205+0000 7f312b400640 1 -- 172.21.15.42:0/4056467884 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3124036070 con 0x7f313406cea0 2024-07-04T17:29:26.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.205+0000 7f313a000640 1 -- 172.21.15.42:0/4056467884 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31341d96d0 con 0x7f313406cea0 2024-07-04T17:29:26.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.206+0000 7f313a000640 1 -- 172.21.15.42:0/4056467884 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3134063f70 con 0x7f313406cea0 2024-07-04T17:29:26.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.206+0000 7f312b400640 1 -- 172.21.15.42:0/4056467884 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f3124044070 con 0x7f313406cea0 2024-07-04T17:29:26.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.207+0000 7f312b400640 1 --2- 172.21.15.42:0/4056467884 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f311403d4f0 0x7f311403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:26.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.207+0000 7f312b400640 1 -- 172.21.15.42:0/4056467884 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4303+0+0 (secure 0 0 0) 0x7f3124042070 con 0x7f313406cea0 2024-07-04T17:29:26.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.207+0000 7f3132a00640 1 --2- 172.21.15.42:0/4056467884 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f311403d4f0 0x7f311403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:26.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.208+0000 7f3132a00640 1 --2- 172.21.15.42:0/4056467884 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f311403d4f0 0x7f311403fa00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3120007fd0 tx=0x7f3120007920 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:26.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.212+0000 7f312b400640 1 -- 172.21.15.42:0/4056467884 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f3134063f70 con 0x7f313406cea0 2024-07-04T17:29:26.401 INFO:tasks.ceph.mds.c.smithi042.stderr:2024-07-04T17:29:26.400+0000 7f0c8ee00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-04T17:29:26.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.413+0000 7f313a000640 1 -- 172.21.15.42:0/4056467884 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f31340b3030 con 0x7f313406cea0 2024-07-04T17:29:26.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.415+0000 7f312b400640 1 -- 172.21.15.42:0/4056467884 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+14465 (secure 0 0 0) 0x7f312404b070 con 0x7f313406cea0 2024-07-04T17:29:26.417 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:29:26.417 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":53,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:29:25.424823+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:29:03.403851+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-07-04T17:29:04.303537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:29:22.942399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.0299999713897705,"score_stable":1.0299999713897705,"optimal_score":1,"raw_score_acting":1.0299999713897705,"raw_score_stable":1.0299999713897705,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:29:24.406438+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:29:26.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.417+0000 7f313a000640 1 -- 172.21.15.42:0/4056467884 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f311403d4f0 msgr2=0x7f311403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:26.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.417+0000 7f313a000640 1 --2- 172.21.15.42:0/4056467884 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f311403d4f0 0x7f311403fa00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3120007fd0 tx=0x7f3120007920 comp rx=0 tx=0).stop 2024-07-04T17:29:26.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.418+0000 7f313a000640 1 -- 172.21.15.42:0/4056467884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f313406cea0 msgr2=0x7f31341d9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:26.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.418+0000 7f313a000640 1 --2- 172.21.15.42:0/4056467884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f313406cea0 0x7f31341d9170 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f3124034040 tx=0x7f312402f610 comp rx=0 tx=0).stop 2024-07-04T17:29:26.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.418+0000 7f313a000640 1 -- 172.21.15.42:0/4056467884 shutdown_connections 2024-07-04T17:29:26.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.418+0000 7f313a000640 1 --2- 172.21.15.42:0/4056467884 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f311403d4f0 0x7f311403fa00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:26.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.418+0000 7f313a000640 1 --2- 172.21.15.42:0/4056467884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f313406cea0 0x7f31341d9170 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:26.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.418+0000 7f313a000640 1 -- 172.21.15.42:0/4056467884 >> 172.21.15.42:0/4056467884 conn(0x7f31340bf700 msgr2=0x7f31341490c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:26.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.418+0000 7f313a000640 1 -- 172.21.15.42:0/4056467884 shutdown_connections 2024-07-04T17:29:26.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.418+0000 7f313a000640 1 -- 172.21.15.42:0/4056467884 wait complete. 2024-07-04T17:29:26.446 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:29:26.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.726+0000 7f0caec00640 1 -- 172.21.15.42:0/1333797451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ca80bc730 msgr2=0x7f0ca80bcb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:26.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.726+0000 7f0caec00640 1 --2- 172.21.15.42:0/1333797451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ca80bc730 0x7f0ca80bcb40 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f0c940097b0 tx=0x7f0c9402a540 comp rx=0 tx=0).stop 2024-07-04T17:29:26.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.726+0000 7f0caec00640 1 -- 172.21.15.42:0/1333797451 shutdown_connections 2024-07-04T17:29:26.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.726+0000 7f0caec00640 1 --2- 172.21.15.42:0/1333797451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ca80bc730 0x7f0ca80bcb40 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:26.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.726+0000 7f0caec00640 1 -- 172.21.15.42:0/1333797451 >> 172.21.15.42:0/1333797451 conn(0x7f0ca8070830 msgr2=0x7f0ca80bfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:26.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.726+0000 7f0caec00640 1 -- 172.21.15.42:0/1333797451 shutdown_connections 2024-07-04T17:29:26.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.726+0000 7f0caec00640 1 -- 172.21.15.42:0/1333797451 wait complete. 2024-07-04T17:29:26.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.728+0000 7f0caec00640 1 Processor -- start 2024-07-04T17:29:26.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.728+0000 7f0caec00640 1 -- start start 2024-07-04T17:29:26.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.729+0000 7f0caec00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ca806dc30 0x7f0ca80bedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:26.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.729+0000 7f0caec00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca806c5a0 con 0x7f0ca806dc30 2024-07-04T17:29:26.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.730+0000 7f0cad800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ca806dc30 0x7f0ca80bedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:26.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.730+0000 7f0cad800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ca806dc30 0x7f0ca80bedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45234/0 (socket says 172.21.15.42:45234) 2024-07-04T17:29:26.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.730+0000 7f0cad800640 1 -- 172.21.15.42:0/894495206 learned_addr learned my addr 172.21.15.42:0/894495206 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:26.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.730+0000 7f0cad800640 1 -- 172.21.15.42:0/894495206 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca81d6040 con 0x7f0ca806dc30 2024-07-04T17:29:26.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.730+0000 7f0cad800640 1 --2- 172.21.15.42:0/894495206 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ca806dc30 0x7f0ca80bedc0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c00c340 tx=0x7f0c9c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:26.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.731+0000 7f0ca5600640 1 -- 172.21.15.42:0/894495206 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c9c0116c0 con 0x7f0ca806dc30 2024-07-04T17:29:26.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.731+0000 7f0ca5600640 1 -- 172.21.15.42:0/894495206 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c9c017070 con 0x7f0ca806dc30 2024-07-04T17:29:26.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.731+0000 7f0ca5600640 1 -- 172.21.15.42:0/894495206 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c9c011b80 con 0x7f0ca806dc30 2024-07-04T17:29:26.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.731+0000 7f0caec00640 1 -- 172.21.15.42:0/894495206 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ca8072720 con 0x7f0ca806dc30 2024-07-04T17:29:26.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.731+0000 7f0caec00640 1 -- 172.21.15.42:0/894495206 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ca8072bf0 con 0x7f0ca806dc30 2024-07-04T17:29:26.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.732+0000 7f0caec00640 1 -- 172.21.15.42:0/894495206 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ca81d6210 con 0x7f0ca806dc30 2024-07-04T17:29:26.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.732+0000 7f0ca5600640 1 -- 172.21.15.42:0/894495206 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f0c9c01c460 con 0x7f0ca806dc30 2024-07-04T17:29:26.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.733+0000 7f0ca5600640 1 --2- 172.21.15.42:0/894495206 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f0c8803d4a0 0x7f0c8803f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:26.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.733+0000 7f0ca5600640 1 -- 172.21.15.42:0/894495206 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4303+0+0 (secure 0 0 0) 0x7f0c9c0518c0 con 0x7f0ca806dc30 2024-07-04T17:29:26.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.733+0000 7f0cace00640 1 --2- 172.21.15.42:0/894495206 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f0c8803d4a0 0x7f0c8803f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:26.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.734+0000 7f0cace00640 1 --2- 172.21.15.42:0/894495206 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f0c8803d4a0 0x7f0c8803f9b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0c94034000 tx=0x7f0c94004730 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:26.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.738+0000 7f0ca5600640 1 -- 172.21.15.42:0/894495206 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f0ca806e040 con 0x7f0ca806dc30 2024-07-04T17:29:26.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.955+0000 7f0caec00640 1 -- 172.21.15.42:0/894495206 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0ca81d4bd0 con 0x7f0ca806dc30 2024-07-04T17:29:26.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.957+0000 7f0ca5600640 1 -- 172.21.15.42:0/894495206 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+14452 (secure 0 0 0) 0x7f0c9c01c800 con 0x7f0ca806dc30 2024-07-04T17:29:26.959 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:29:26.959 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":54,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:29:26.875523+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:29:03.403851+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-07-04T17:29:04.303537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:29:22.942399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.0299999713897705,"score_stable":1.0299999713897705,"optimal_score":1,"raw_score_acting":1.0299999713897705,"raw_score_stable":1.0299999713897705,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:29:24.406438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:29:26.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.960+0000 7f0caec00640 1 -- 172.21.15.42:0/894495206 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f0c8803d4a0 msgr2=0x7f0c8803f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:26.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.960+0000 7f0caec00640 1 --2- 172.21.15.42:0/894495206 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f0c8803d4a0 0x7f0c8803f9b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0c94034000 tx=0x7f0c94004730 comp rx=0 tx=0).stop 2024-07-04T17:29:26.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.960+0000 7f0caec00640 1 -- 172.21.15.42:0/894495206 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ca806dc30 msgr2=0x7f0ca80bedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:26.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.960+0000 7f0caec00640 1 --2- 172.21.15.42:0/894495206 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ca806dc30 0x7f0ca80bedc0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c00c340 tx=0x7f0c9c00c800 comp rx=0 tx=0).stop 2024-07-04T17:29:26.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.960+0000 7f0caec00640 1 -- 172.21.15.42:0/894495206 shutdown_connections 2024-07-04T17:29:26.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.960+0000 7f0caec00640 1 --2- 172.21.15.42:0/894495206 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f0c8803d4a0 0x7f0c8803f9b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:26.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.960+0000 7f0caec00640 1 --2- 172.21.15.42:0/894495206 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ca806dc30 0x7f0ca80bedc0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:26.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.960+0000 7f0caec00640 1 -- 172.21.15.42:0/894495206 >> 172.21.15.42:0/894495206 conn(0x7f0ca8070830 msgr2=0x7f0ca80bfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:26.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.960+0000 7f0caec00640 1 -- 172.21.15.42:0/894495206 shutdown_connections 2024-07-04T17:29:26.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:26.960+0000 7f0caec00640 1 -- 172.21.15.42:0/894495206 wait complete. 2024-07-04T17:29:26.985 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs standby_count_wanted 0 2024-07-04T17:29:27.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.269+0000 7f753d000640 1 -- 172.21.15.42:0/1279830694 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75380b1100 msgr2=0x7f753813f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:27.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.269+0000 7f753d000640 1 --2- 172.21.15.42:0/1279830694 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75380b1100 0x7f753813f800 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f7520005fd0 tx=0x7f752002ae80 comp rx=0 tx=0).stop 2024-07-04T17:29:27.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.269+0000 7f753d000640 1 -- 172.21.15.42:0/1279830694 shutdown_connections 2024-07-04T17:29:27.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.269+0000 7f753d000640 1 --2- 172.21.15.42:0/1279830694 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75380b1100 0x7f753813f800 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:27.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.269+0000 7f753d000640 1 -- 172.21.15.42:0/1279830694 >> 172.21.15.42:0/1279830694 conn(0x7f753806c020 msgr2=0x7f753806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:27.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.269+0000 7f753d000640 1 -- 172.21.15.42:0/1279830694 shutdown_connections 2024-07-04T17:29:27.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.269+0000 7f753d000640 1 -- 172.21.15.42:0/1279830694 wait complete. 2024-07-04T17:29:27.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.271+0000 7f753d000640 1 Processor -- start 2024-07-04T17:29:27.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.271+0000 7f753d000640 1 -- start start 2024-07-04T17:29:27.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.272+0000 7f753d000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75380a94d0 0x7f7538135800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:27.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.272+0000 7f753d000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f75381401e0 con 0x7f75380a94d0 2024-07-04T17:29:27.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.273+0000 7f7535e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75380a94d0 0x7f7538135800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:27.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.273+0000 7f7535e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75380a94d0 0x7f7538135800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45246/0 (socket says 172.21.15.42:45246) 2024-07-04T17:29:27.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.273+0000 7f7535e00640 1 -- 172.21.15.42:0/251118938 learned_addr learned my addr 172.21.15.42:0/251118938 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:27.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.273+0000 7f7535e00640 1 -- 172.21.15.42:0/251118938 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7538135340 con 0x7f75380a94d0 2024-07-04T17:29:27.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.274+0000 7f7535e00640 1 --2- 172.21.15.42:0/251118938 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75380a94d0 0x7f7538135800 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7528001eb0 tx=0x7f752800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:27.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.274+0000 7f752e000640 1 -- 172.21.15.42:0/251118938 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7528017070 con 0x7f75380a94d0 2024-07-04T17:29:27.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.274+0000 7f752e000640 1 -- 172.21.15.42:0/251118938 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75280052f0 con 0x7f75380a94d0 2024-07-04T17:29:27.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.274+0000 7f752e000640 1 -- 172.21.15.42:0/251118938 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75280035f0 con 0x7f75380a94d0 2024-07-04T17:29:27.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.274+0000 7f753d000640 1 -- 172.21.15.42:0/251118938 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7538134230 con 0x7f75380a94d0 2024-07-04T17:29:27.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.274+0000 7f753d000640 1 -- 172.21.15.42:0/251118938 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75381d77e0 con 0x7f75380a94d0 2024-07-04T17:29:27.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.275+0000 7f753d000640 1 -- 172.21.15.42:0/251118938 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7538139b00 con 0x7f75380a94d0 2024-07-04T17:29:27.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.275+0000 7f752e000640 1 -- 172.21.15.42:0/251118938 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f752801a070 con 0x7f75380a94d0 2024-07-04T17:29:27.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.276+0000 7f752e000640 1 --2- 172.21.15.42:0/251118938 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f751403d4f0 0x7f751403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:27.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.276+0000 7f752e000640 1 -- 172.21.15.42:0/251118938 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4303+0+0 (secure 0 0 0) 0x7f7528050f80 con 0x7f75380a94d0 2024-07-04T17:29:27.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.276+0000 7f752fe00640 1 --2- 172.21.15.42:0/251118938 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f751403d4f0 0x7f751403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:27.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.277+0000 7f752fe00640 1 --2- 172.21.15.42:0/251118938 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f751403d4f0 0x7f751403fa00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7520034000 tx=0x7f7520004730 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:27.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.281+0000 7f752e000640 1 -- 172.21.15.42:0/251118938 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f7538139b00 con 0x7f75380a94d0 2024-07-04T17:29:27.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.534+0000 7f753d000640 1 -- 172.21.15.42:0/251118938 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f753813f800 con 0x7f75380a94d0 2024-07-04T17:29:27.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.893+0000 7f752e000640 1 -- 172.21.15.42:0/251118938 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"}]=0 v40) ==== 121+0+0 (secure 0 0 0) 0x7f75380b1510 con 0x7f75380a94d0 2024-07-04T17:29:27.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.897+0000 7f753d000640 1 -- 172.21.15.42:0/251118938 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f751403d4f0 msgr2=0x7f751403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:27.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.897+0000 7f753d000640 1 --2- 172.21.15.42:0/251118938 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f751403d4f0 0x7f751403fa00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7520034000 tx=0x7f7520004730 comp rx=0 tx=0).stop 2024-07-04T17:29:27.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.897+0000 7f753d000640 1 -- 172.21.15.42:0/251118938 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75380a94d0 msgr2=0x7f7538135800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:27.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.897+0000 7f753d000640 1 --2- 172.21.15.42:0/251118938 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75380a94d0 0x7f7538135800 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7528001eb0 tx=0x7f752800c810 comp rx=0 tx=0).stop 2024-07-04T17:29:27.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.897+0000 7f753d000640 1 -- 172.21.15.42:0/251118938 shutdown_connections 2024-07-04T17:29:27.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.897+0000 7f753d000640 1 --2- 172.21.15.42:0/251118938 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f751403d4f0 0x7f751403fa00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:27.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.897+0000 7f753d000640 1 --2- 172.21.15.42:0/251118938 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75380a94d0 0x7f7538135800 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:27.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.897+0000 7f753d000640 1 -- 172.21.15.42:0/251118938 >> 172.21.15.42:0/251118938 conn(0x7f753806c020 msgr2=0x7f753813bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:27.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.898+0000 7f753d000640 1 -- 172.21.15.42:0/251118938 shutdown_connections 2024-07-04T17:29:27.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:27.898+0000 7f753d000640 1 -- 172.21.15.42:0/251118938 wait complete. 2024-07-04T17:29:27.934 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-04T17:29:27.934 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs allow_standby_replay false 2024-07-04T17:29:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.231+0000 7f90b6200640 1 -- 172.21.15.42:0/1493888613 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90b00a94d0 msgr2=0x7f90b00b0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.231+0000 7f90b6200640 1 --2- 172.21.15.42:0/1493888613 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90b00a94d0 0x7f90b00b0ed0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f909800a8c0 tx=0x7f909802b860 comp rx=0 tx=0).stop 2024-07-04T17:29:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.231+0000 7f90b6200640 1 -- 172.21.15.42:0/1493888613 shutdown_connections 2024-07-04T17:29:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.231+0000 7f90b6200640 1 --2- 172.21.15.42:0/1493888613 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90b00a94d0 0x7f90b00b0ed0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.231+0000 7f90b6200640 1 -- 172.21.15.42:0/1493888613 >> 172.21.15.42:0/1493888613 conn(0x7f90b006c020 msgr2=0x7f90b006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.231+0000 7f90b6200640 1 -- 172.21.15.42:0/1493888613 shutdown_connections 2024-07-04T17:29:28.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.231+0000 7f90b6200640 1 -- 172.21.15.42:0/1493888613 wait complete. 2024-07-04T17:29:28.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.233+0000 7f90b6200640 1 Processor -- start 2024-07-04T17:29:28.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.234+0000 7f90b6200640 1 -- start start 2024-07-04T17:29:28.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.235+0000 7f90b6200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90b00732e0 0x7f90b01d3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:28.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.235+0000 7f90b6200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f90b006d050 con 0x7f90b00732e0 2024-07-04T17:29:28.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.235+0000 7f90af400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90b00732e0 0x7f90b01d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:28.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.235+0000 7f90af400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90b00732e0 0x7f90b01d3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45268/0 (socket says 172.21.15.42:45268) 2024-07-04T17:29:28.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.235+0000 7f90af400640 1 -- 172.21.15.42:0/1050264621 learned_addr learned my addr 172.21.15.42:0/1050264621 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:28.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.236+0000 7f90af400640 1 -- 172.21.15.42:0/1050264621 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b01d3860 con 0x7f90b00732e0 2024-07-04T17:29:28.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.236+0000 7f90af400640 1 --2- 172.21.15.42:0/1050264621 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90b00732e0 0x7f90b01d3320 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f90a000cb40 tx=0x7f90a000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:28.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.236+0000 7f90a7e00640 1 -- 172.21.15.42:0/1050264621 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90a0016020 con 0x7f90b00732e0 2024-07-04T17:29:28.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.236+0000 7f90a7e00640 1 -- 172.21.15.42:0/1050264621 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90a000ac30 con 0x7f90b00732e0 2024-07-04T17:29:28.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.236+0000 7f90a7e00640 1 -- 172.21.15.42:0/1050264621 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90a0012420 con 0x7f90b00732e0 2024-07-04T17:29:28.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.236+0000 7f90b6200640 1 -- 172.21.15.42:0/1050264621 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b00ae870 con 0x7f90b00732e0 2024-07-04T17:29:28.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.237+0000 7f90b6200640 1 -- 172.21.15.42:0/1050264621 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b00ae4d0 con 0x7f90b00732e0 2024-07-04T17:29:28.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.238+0000 7f90b6200640 1 -- 172.21.15.42:0/1050264621 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90b00b0ed0 con 0x7f90b00732e0 2024-07-04T17:29:28.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.238+0000 7f90a7e00640 1 -- 172.21.15.42:0/1050264621 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f90a0019020 con 0x7f90b00732e0 2024-07-04T17:29:28.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.238+0000 7f90a7e00640 1 --2- 172.21.15.42:0/1050264621 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f908c03d4f0 0x7f908c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:28.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.238+0000 7f90a7e00640 1 -- 172.21.15.42:0/1050264621 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4303+0+0 (secure 0 0 0) 0x7f90a00520a0 con 0x7f90b00732e0 2024-07-04T17:29:28.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.239+0000 7f90a6a00640 1 --2- 172.21.15.42:0/1050264621 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f908c03d4f0 0x7f908c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:28.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.240+0000 7f90a6a00640 1 --2- 172.21.15.42:0/1050264621 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f908c03d4f0 0x7f908c03fa00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9098036000 tx=0x7f9098004730 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:28.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.244+0000 7f90a7e00640 1 -- 172.21.15.42:0/1050264621 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f90a0020030 con 0x7f90b00732e0 2024-07-04T17:29:28.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.494+0000 7f90b6200640 1 -- 172.21.15.42:0/1050264621 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f90b0074520 con 0x7f90b00732e0 2024-07-04T17:29:28.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.902+0000 7f90a7e00640 1 -- 172.21.15.42:0/1050264621 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"}]=0 v41) ==== 125+0+0 (secure 0 0 0) 0x7f90a000d150 con 0x7f90b00732e0 2024-07-04T17:29:28.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.906+0000 7f90b6200640 1 -- 172.21.15.42:0/1050264621 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f908c03d4f0 msgr2=0x7f908c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:28.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.906+0000 7f90b6200640 1 --2- 172.21.15.42:0/1050264621 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f908c03d4f0 0x7f908c03fa00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9098036000 tx=0x7f9098004730 comp rx=0 tx=0).stop 2024-07-04T17:29:28.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.906+0000 7f90b6200640 1 -- 172.21.15.42:0/1050264621 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90b00732e0 msgr2=0x7f90b01d3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:28.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.906+0000 7f90b6200640 1 --2- 172.21.15.42:0/1050264621 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90b00732e0 0x7f90b01d3320 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f90a000cb40 tx=0x7f90a000a060 comp rx=0 tx=0).stop 2024-07-04T17:29:28.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.906+0000 7f90b6200640 1 -- 172.21.15.42:0/1050264621 shutdown_connections 2024-07-04T17:29:28.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.906+0000 7f90b6200640 1 --2- 172.21.15.42:0/1050264621 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f908c03d4f0 0x7f908c03fa00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:28.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.906+0000 7f90b6200640 1 --2- 172.21.15.42:0/1050264621 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90b00732e0 0x7f90b01d3320 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:28.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.906+0000 7f90b6200640 1 -- 172.21.15.42:0/1050264621 >> 172.21.15.42:0/1050264621 conn(0x7f90b006c020 msgr2=0x7f90b013cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:28.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.906+0000 7f90b6200640 1 -- 172.21.15.42:0/1050264621 shutdown_connections 2024-07-04T17:29:28.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:28.907+0000 7f90b6200640 1 -- 172.21.15.42:0/1050264621 wait complete. 2024-07-04T17:29:28.936 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'backup_fs' 2024-07-04T17:29:28.936 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:29:29.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.229+0000 7f2c42800640 1 -- 172.21.15.42:0/6570040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c3c0be540 msgr2=0x7f2c3c0adac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:29.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.229+0000 7f2c42800640 1 --2- 172.21.15.42:0/6570040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c3c0be540 0x7f2c3c0adac0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f2c30005fd0 tx=0x7f2c3002a8b0 comp rx=0 tx=0).stop 2024-07-04T17:29:29.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.229+0000 7f2c42800640 1 -- 172.21.15.42:0/6570040 shutdown_connections 2024-07-04T17:29:29.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.229+0000 7f2c42800640 1 --2- 172.21.15.42:0/6570040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c3c0be540 0x7f2c3c0adac0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:29.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.229+0000 7f2c42800640 1 -- 172.21.15.42:0/6570040 >> 172.21.15.42:0/6570040 conn(0x7f2c3c071960 msgr2=0x7f2c3c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:29.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.229+0000 7f2c42800640 1 -- 172.21.15.42:0/6570040 shutdown_connections 2024-07-04T17:29:29.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.229+0000 7f2c42800640 1 -- 172.21.15.42:0/6570040 wait complete. 2024-07-04T17:29:29.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.231+0000 7f2c42800640 1 Processor -- start 2024-07-04T17:29:29.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.231+0000 7f2c42800640 1 -- start start 2024-07-04T17:29:29.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.232+0000 7f2c42800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c3c0be540 0x7f2c3c1476e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:29.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.232+0000 7f2c42800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c14e540 con 0x7f2c3c0be540 2024-07-04T17:29:29.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.233+0000 7f2c41400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c3c0be540 0x7f2c3c1476e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:29.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.233+0000 7f2c41400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c3c0be540 0x7f2c3c1476e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45294/0 (socket says 172.21.15.42:45294) 2024-07-04T17:29:29.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.233+0000 7f2c41400640 1 -- 172.21.15.42:0/3607393747 learned_addr learned my addr 172.21.15.42:0/3607393747 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:29.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.233+0000 7f2c41400640 1 -- 172.21.15.42:0/3607393747 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c3c14fa00 con 0x7f2c3c0be540 2024-07-04T17:29:29.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.233+0000 7f2c41400640 1 --2- 172.21.15.42:0/3607393747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c3c0be540 0x7f2c3c1476e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2c30009c50 tx=0x7f2c30031670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:29.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.234+0000 7f2c39600640 1 -- 172.21.15.42:0/3607393747 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c300318c0 con 0x7f2c3c0be540 2024-07-04T17:29:29.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.234+0000 7f2c39600640 1 -- 172.21.15.42:0/3607393747 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c30031b50 con 0x7f2c3c0be540 2024-07-04T17:29:29.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.234+0000 7f2c39600640 1 -- 172.21.15.42:0/3607393747 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c300330d0 con 0x7f2c3c0be540 2024-07-04T17:29:29.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.234+0000 7f2c42800640 1 -- 172.21.15.42:0/3607393747 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c3c1453c0 con 0x7f2c3c0be540 2024-07-04T17:29:29.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.234+0000 7f2c42800640 1 -- 172.21.15.42:0/3607393747 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c3c14f0d0 con 0x7f2c3c0be540 2024-07-04T17:29:29.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.235+0000 7f2c39600640 1 -- 172.21.15.42:0/3607393747 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f2c30033270 con 0x7f2c3c0be540 2024-07-04T17:29:29.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.235+0000 7f2c42800640 1 -- 172.21.15.42:0/3607393747 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c3c06ecb0 con 0x7f2c3c0be540 2024-07-04T17:29:29.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.236+0000 7f2c39600640 1 --2- 172.21.15.42:0/3607393747 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f2c1c03d4f0 0x7f2c1c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:29.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.236+0000 7f2c39600640 1 -- 172.21.15.42:0/3607393747 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4303+0+0 (secure 0 0 0) 0x7f2c30073240 con 0x7f2c3c0be540 2024-07-04T17:29:29.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.236+0000 7f2c40a00640 1 --2- 172.21.15.42:0/3607393747 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f2c1c03d4f0 0x7f2c1c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:29.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.237+0000 7f2c40a00640 1 --2- 172.21.15.42:0/3607393747 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f2c1c03d4f0 0x7f2c1c03fa00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2c28007fd0 tx=0x7f2c28007920 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:29.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.241+0000 7f2c39600640 1 -- 172.21.15.42:0/3607393747 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f2c3003f030 con 0x7f2c3c0be540 2024-07-04T17:29:29.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.490+0000 7f2c42800640 1 -- 172.21.15.42:0/3607393747 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c3c068d10 con 0x7f2c3c0be540 2024-07-04T17:29:29.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.491+0000 7f2c39600640 1 -- 172.21.15.42:0/3607393747 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+7364 (secure 0 0 0) 0x7f2c30043d00 con 0x7f2c3c0be540 2024-07-04T17:29:29.493 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:29:29.493 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":41,"btime":"2024-07-04T17:29:28:895382+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/343791977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":343791977},{"type":"v1","addr":"172.21.15.42:6827","nonce":343791977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":4722,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/3172446281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":3172446281},{"type":"v1","addr":"172.21.15.42:6829","nonce":3172446281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":4727,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/1866310217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1866310217},{"type":"v1","addr":"172.21.15.42:6831","nonce":1866310217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:29:05.313061+0000","modified":"2024-07-04T17:29:08.370956+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4731},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4731":{"gid":4731,"name":"d","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6835/1327681666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1327681666},{"type":"v1","addr":"172.21.15.42:6835","nonce":1327681666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4731,"qdb_cluster":[4731]},"id":4},{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:29:25.424728+0000","modified":"2024-07-04T17:29:28.496007+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4728":{"gid":4728,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":9,"addr":"172.21.15.42:6833/1072723036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1072723036},{"type":"v1","addr":"172.21.15.42:6833","nonce":1072723036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4728,"qdb_cluster":[4728]},"id":5}]} 2024-07-04T17:29:29.493 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 41 2024-07-04T17:29:29.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.494+0000 7f2c42800640 1 -- 172.21.15.42:0/3607393747 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f2c1c03d4f0 msgr2=0x7f2c1c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:29.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.494+0000 7f2c42800640 1 --2- 172.21.15.42:0/3607393747 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f2c1c03d4f0 0x7f2c1c03fa00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2c28007fd0 tx=0x7f2c28007920 comp rx=0 tx=0).stop 2024-07-04T17:29:29.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.494+0000 7f2c42800640 1 -- 172.21.15.42:0/3607393747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c3c0be540 msgr2=0x7f2c3c1476e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:29.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.494+0000 7f2c42800640 1 --2- 172.21.15.42:0/3607393747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c3c0be540 0x7f2c3c1476e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2c30009c50 tx=0x7f2c30031670 comp rx=0 tx=0).stop 2024-07-04T17:29:29.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.494+0000 7f2c42800640 1 -- 172.21.15.42:0/3607393747 shutdown_connections 2024-07-04T17:29:29.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.494+0000 7f2c42800640 1 --2- 172.21.15.42:0/3607393747 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f2c1c03d4f0 0x7f2c1c03fa00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:29.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.494+0000 7f2c42800640 1 --2- 172.21.15.42:0/3607393747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c3c0be540 0x7f2c3c1476e0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:29.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.494+0000 7f2c42800640 1 -- 172.21.15.42:0/3607393747 >> 172.21.15.42:0/3607393747 conn(0x7f2c3c071960 msgr2=0x7f2c3c072d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:29.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.494+0000 7f2c42800640 1 -- 172.21.15.42:0/3607393747 shutdown_connections 2024-07-04T17:29:29.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.495+0000 7f2c42800640 1 -- 172.21.15.42:0/3607393747 wait complete. 2024-07-04T17:29:29.524 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:29:29.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.818+0000 7f643a600640 1 -- 172.21.15.42:0/374931568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64340b9840 msgr2=0x7f643406dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:29.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.818+0000 7f643a600640 1 --2- 172.21.15.42:0/374931568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64340b9840 0x7f643406dca0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f641c0097b0 tx=0x7f641c02a530 comp rx=0 tx=0).stop 2024-07-04T17:29:29.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.818+0000 7f643a600640 1 -- 172.21.15.42:0/374931568 shutdown_connections 2024-07-04T17:29:29.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.818+0000 7f643a600640 1 --2- 172.21.15.42:0/374931568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64340b9840 0x7f643406dca0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:29.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.818+0000 7f643a600640 1 -- 172.21.15.42:0/374931568 >> 172.21.15.42:0/374931568 conn(0x7f6434070830 msgr2=0x7f64340bfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:29.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.818+0000 7f643a600640 1 -- 172.21.15.42:0/374931568 shutdown_connections 2024-07-04T17:29:29.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.818+0000 7f643a600640 1 -- 172.21.15.42:0/374931568 wait complete. 2024-07-04T17:29:29.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.821+0000 7f643a600640 1 Processor -- start 2024-07-04T17:29:29.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.821+0000 7f643a600640 1 -- start start 2024-07-04T17:29:29.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.822+0000 7f643a600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64340b9840 0x7f64340bc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:29.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.822+0000 7f643a600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6434155170 con 0x7f64340b9840 2024-07-04T17:29:29.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.822+0000 7f6439200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64340b9840 0x7f64340bc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:29.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.823+0000 7f6439200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64340b9840 0x7f64340bc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45324/0 (socket says 172.21.15.42:45324) 2024-07-04T17:29:29.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.823+0000 7f6439200640 1 -- 172.21.15.42:0/1039647704 learned_addr learned my addr 172.21.15.42:0/1039647704 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:29.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.823+0000 7f6439200640 1 -- 172.21.15.42:0/1039647704 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64340bc980 con 0x7f64340b9840 2024-07-04T17:29:29.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.823+0000 7f6439200640 1 --2- 172.21.15.42:0/1039647704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64340b9840 0x7f64340bc240 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f642400ca90 tx=0x7f642400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:29.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.823+0000 7f6431600640 1 -- 172.21.15.42:0/1039647704 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6424016020 con 0x7f64340b9840 2024-07-04T17:29:29.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.824+0000 7f6431600640 1 -- 172.21.15.42:0/1039647704 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f642400abf0 con 0x7f64340b9840 2024-07-04T17:29:29.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.824+0000 7f643a600640 1 -- 172.21.15.42:0/1039647704 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64340bedb0 con 0x7f64340b9840 2024-07-04T17:29:29.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.824+0000 7f6431600640 1 -- 172.21.15.42:0/1039647704 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6424012420 con 0x7f64340b9840 2024-07-04T17:29:29.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.824+0000 7f643a600640 1 -- 172.21.15.42:0/1039647704 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64340bd1d0 con 0x7f64340b9840 2024-07-04T17:29:29.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.825+0000 7f643a600640 1 -- 172.21.15.42:0/1039647704 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64340bd4c0 con 0x7f64340b9840 2024-07-04T17:29:29.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.826+0000 7f6431600640 1 -- 172.21.15.42:0/1039647704 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f6424019020 con 0x7f64340b9840 2024-07-04T17:29:29.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.826+0000 7f6431600640 1 --2- 172.21.15.42:0/1039647704 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f641003d4f0 0x7f641003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:29.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.826+0000 7f6431600640 1 -- 172.21.15.42:0/1039647704 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4303+0+0 (secure 0 0 0) 0x7f64240521e0 con 0x7f64340b9840 2024-07-04T17:29:29.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.826+0000 7f642be00640 1 --2- 172.21.15.42:0/1039647704 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f641003d4f0 0x7f641003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:29.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.827+0000 7f642be00640 1 --2- 172.21.15.42:0/1039647704 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f641003d4f0 0x7f641003fa00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f641c034000 tx=0x7f641c004730 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:29.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:29.831+0000 7f6431600640 1 -- 172.21.15.42:0/1039647704 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f6424020020 con 0x7f64340b9840 2024-07-04T17:29:30.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.044+0000 7f643a600640 1 -- 172.21.15.42:0/1039647704 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f64340bc780 con 0x7f64340b9840 2024-07-04T17:29:30.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.046+0000 7f6431600640 1 -- 172.21.15.42:0/1039647704 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+14452 (secure 0 0 0) 0x7f64340b9c50 con 0x7f64340b9840 2024-07-04T17:29:30.048 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:29:30.048 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":54,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:29:26.875523+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:29:03.403851+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-07-04T17:29:04.303537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:29:22.942399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.0299999713897705,"score_stable":1.0299999713897705,"optimal_score":1,"raw_score_acting":1.0299999713897705,"raw_score_stable":1.0299999713897705,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:29:24.406438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:29:30.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.049+0000 7f643a600640 1 -- 172.21.15.42:0/1039647704 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f641003d4f0 msgr2=0x7f641003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:30.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.049+0000 7f643a600640 1 --2- 172.21.15.42:0/1039647704 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f641003d4f0 0x7f641003fa00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f641c034000 tx=0x7f641c004730 comp rx=0 tx=0).stop 2024-07-04T17:29:30.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.049+0000 7f643a600640 1 -- 172.21.15.42:0/1039647704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64340b9840 msgr2=0x7f64340bc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:30.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.049+0000 7f643a600640 1 --2- 172.21.15.42:0/1039647704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64340b9840 0x7f64340bc240 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f642400ca90 tx=0x7f642400cf50 comp rx=0 tx=0).stop 2024-07-04T17:29:30.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.049+0000 7f643a600640 1 -- 172.21.15.42:0/1039647704 shutdown_connections 2024-07-04T17:29:30.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.049+0000 7f643a600640 1 --2- 172.21.15.42:0/1039647704 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f641003d4f0 0x7f641003fa00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:30.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.049+0000 7f643a600640 1 --2- 172.21.15.42:0/1039647704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64340b9840 0x7f64340bc240 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:30.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.049+0000 7f643a600640 1 -- 172.21.15.42:0/1039647704 >> 172.21.15.42:0/1039647704 conn(0x7f6434070830 msgr2=0x7f64341458f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:30.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.050+0000 7f643a600640 1 -- 172.21.15.42:0/1039647704 shutdown_connections 2024-07-04T17:29:30.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.050+0000 7f643a600640 1 -- 172.21.15.42:0/1039647704 wait complete. 2024-07-04T17:29:30.079 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-04T17:29:30.080 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:29:30.149 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-04T17:29:30.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.354+0000 7f1a39c00640 1 -- 172.21.15.42:0/513490795 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a3406c200 msgr2=0x7f1a3414ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:30.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.354+0000 7f1a39c00640 1 --2- 172.21.15.42:0/513490795 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a3406c200 0x7f1a3414ef10 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f1a24005fd0 tx=0x7f1a2402a820 comp rx=0 tx=0).stop 2024-07-04T17:29:30.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.354+0000 7f1a39c00640 1 -- 172.21.15.42:0/513490795 shutdown_connections 2024-07-04T17:29:30.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.354+0000 7f1a39c00640 1 --2- 172.21.15.42:0/513490795 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a3406c200 0x7f1a3414ef10 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:30.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.354+0000 7f1a39c00640 1 -- 172.21.15.42:0/513490795 >> 172.21.15.42:0/513490795 conn(0x7f1a340bebc0 msgr2=0x7f1a340befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:30.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.354+0000 7f1a39c00640 1 -- 172.21.15.42:0/513490795 shutdown_connections 2024-07-04T17:29:30.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.354+0000 7f1a39c00640 1 -- 172.21.15.42:0/513490795 wait complete. 2024-07-04T17:29:30.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.356+0000 7f1a39c00640 1 Processor -- start 2024-07-04T17:29:30.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.356+0000 7f1a39c00640 1 -- start start 2024-07-04T17:29:30.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.358+0000 7f1a39c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a3406c200 0x7f1a341d6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:30.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.358+0000 7f1a39c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3414ae50 con 0x7f1a3406c200 2024-07-04T17:29:30.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.358+0000 7f1a33e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a3406c200 0x7f1a341d6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:30.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.358+0000 7f1a33e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a3406c200 0x7f1a341d6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45348/0 (socket says 172.21.15.42:45348) 2024-07-04T17:29:30.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.358+0000 7f1a33e00640 1 -- 172.21.15.42:0/3853419363 learned_addr learned my addr 172.21.15.42:0/3853419363 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:30.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.358+0000 7f1a33e00640 1 -- 172.21.15.42:0/3853419363 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a340b6ec0 con 0x7f1a3406c200 2024-07-04T17:29:30.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.359+0000 7f1a33e00640 1 --2- 172.21.15.42:0/3853419363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a3406c200 0x7f1a341d6ac0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009c50 tx=0x7f1a24032dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:30.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.359+0000 7f1a2be00640 1 -- 172.21.15.42:0/3853419363 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a24037050 con 0x7f1a3406c200 2024-07-04T17:29:30.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.359+0000 7f1a2be00640 1 -- 172.21.15.42:0/3853419363 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a24043070 con 0x7f1a3406c200 2024-07-04T17:29:30.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.359+0000 7f1a2be00640 1 -- 172.21.15.42:0/3853419363 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a240043e0 con 0x7f1a3406c200 2024-07-04T17:29:30.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.359+0000 7f1a39c00640 1 -- 172.21.15.42:0/3853419363 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a340b56b0 con 0x7f1a3406c200 2024-07-04T17:29:30.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.359+0000 7f1a39c00640 1 -- 172.21.15.42:0/3853419363 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a340b13f0 con 0x7f1a3406c200 2024-07-04T17:29:30.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.360+0000 7f1a39c00640 1 -- 172.21.15.42:0/3853419363 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a341d4e30 con 0x7f1a3406c200 2024-07-04T17:29:30.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.360+0000 7f1a2be00640 1 -- 172.21.15.42:0/3853419363 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f1a24004580 con 0x7f1a3406c200 2024-07-04T17:29:30.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.361+0000 7f1a2be00640 1 --2- 172.21.15.42:0/3853419363 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f1a1403d4f0 0x7f1a1403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:30.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.361+0000 7f1a2be00640 1 -- 172.21.15.42:0/3853419363 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4303+0+0 (secure 0 0 0) 0x7f1a2402f070 con 0x7f1a3406c200 2024-07-04T17:29:30.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.361+0000 7f1a33400640 1 --2- 172.21.15.42:0/3853419363 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f1a1403d4f0 0x7f1a1403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:30.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.362+0000 7f1a33400640 1 --2- 172.21.15.42:0/3853419363 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f1a1403d4f0 0x7f1a1403fa00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1a20007fd0 tx=0x7f1a20007920 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:30.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.367+0000 7f1a2be00640 1 -- 172.21.15.42:0/3853419363 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f1a341d4e30 con 0x7f1a3406c200 2024-07-04T17:29:30.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.572+0000 7f1a39c00640 1 -- 172.21.15.42:0/3853419363 --> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1a3414b490 con 0x7f1a1403d4f0 2024-07-04T17:29:30.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.604+0000 7f1a2be00640 1 -- 172.21.15.42:0/3853419363 <== mgr.4613 v2:172.21.15.42:6824/2638271195 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+482337 (secure 0 0 0) 0x7f1a3414b490 con 0x7f1a1403d4f0 2024-07-04T17:29:30.614 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:29:30.618 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:29:30.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.615+0000 7f1a39c00640 1 -- 172.21.15.42:0/3853419363 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f1a1403d4f0 msgr2=0x7f1a1403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:30.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.615+0000 7f1a39c00640 1 --2- 172.21.15.42:0/3853419363 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f1a1403d4f0 0x7f1a1403fa00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1a20007fd0 tx=0x7f1a20007920 comp rx=0 tx=0).stop 2024-07-04T17:29:30.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.615+0000 7f1a39c00640 1 -- 172.21.15.42:0/3853419363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a3406c200 msgr2=0x7f1a341d6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:30.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.615+0000 7f1a39c00640 1 --2- 172.21.15.42:0/3853419363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a3406c200 0x7f1a341d6ac0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009c50 tx=0x7f1a24032dc0 comp rx=0 tx=0).stop 2024-07-04T17:29:30.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.615+0000 7f1a39c00640 1 -- 172.21.15.42:0/3853419363 shutdown_connections 2024-07-04T17:29:30.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.615+0000 7f1a39c00640 1 --2- 172.21.15.42:0/3853419363 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f1a1403d4f0 0x7f1a1403fa00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:30.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.615+0000 7f1a39c00640 1 --2- 172.21.15.42:0/3853419363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a3406c200 0x7f1a341d6ac0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:30.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.615+0000 7f1a39c00640 1 -- 172.21.15.42:0/3853419363 >> 172.21.15.42:0/3853419363 conn(0x7f1a340bebc0 msgr2=0x7f1a340bbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:30.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.615+0000 7f1a39c00640 1 -- 172.21.15.42:0/3853419363 shutdown_connections 2024-07-04T17:29:30.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.615+0000 7f1a39c00640 1 -- 172.21.15.42:0/3853419363 wait complete. 2024-07-04T17:29:30.646 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":48,"stamp":"2024-07-04T17:29:29.660059+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596885,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":176,"num_read_kb":398,"num_write":195,"num_write_kb":1137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":364,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":96396,"kb_used_data":2836,"kb_used_omap":4,"kb_used_meta":93499,"kb_avail":283019124,"statfs":{"total":289910292480,"available":289811582976,"internally_reserved":0,"allocated":2904064,"data_stored":1804081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":95743311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.819331"},"pg_stats":[{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.883229+0000","last_change":"2024-07-04T17:29:25.937624+0000","last_active":"2024-07-04T17:29:26.883229+0000","last_peered":"2024-07-04T17:29:26.883229+0000","last_clean":"2024-07-04T17:29:26.883229+0000","last_became_active":"2024-07-04T17:29:25.937236+0000","last_became_peered":"2024-07-04T17:29:25.937236+0000","last_unstale":"2024-07-04T17:29:26.883229+0000","last_undegraded":"2024-07-04T17:29:26.883229+0000","last_fullsized":"2024-07-04T17:29:26.883229+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:35:54.403127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3f","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.893787+0000","last_change":"2024-07-04T17:29:05.832523+0000","last_active":"2024-07-04T17:29:26.893787+0000","last_peered":"2024-07-04T17:29:26.893787+0000","last_clean":"2024-07-04T17:29:26.893787+0000","last_became_active":"2024-07-04T17:29:05.831208+0000","last_became_peered":"2024-07-04T17:29:05.831208+0000","last_unstale":"2024-07-04T17:29:26.893787+0000","last_undegraded":"2024-07-04T17:29:26.893787+0000","last_fullsized":"2024-07-04T17:29:26.893787+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:52:48.071452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.904845+0000","last_change":"2024-07-04T17:29:25.921896+0000","last_active":"2024-07-04T17:29:26.904845+0000","last_peered":"2024-07-04T17:29:26.904845+0000","last_clean":"2024-07-04T17:29:26.904845+0000","last_became_active":"2024-07-04T17:29:25.921426+0000","last_became_peered":"2024-07-04T17:29:25.921426+0000","last_unstale":"2024-07-04T17:29:26.904845+0000","last_undegraded":"2024-07-04T17:29:26.904845+0000","last_fullsized":"2024-07-04T17:29:26.904845+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:28:53.439362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.894664+0000","last_change":"2024-07-04T17:29:05.826928+0000","last_active":"2024-07-04T17:29:26.894664+0000","last_peered":"2024-07-04T17:29:26.894664+0000","last_clean":"2024-07-04T17:29:26.894664+0000","last_became_active":"2024-07-04T17:29:05.826320+0000","last_became_peered":"2024-07-04T17:29:05.826320+0000","last_unstale":"2024-07-04T17:29:26.894664+0000","last_undegraded":"2024-07-04T17:29:26.894664+0000","last_fullsized":"2024-07-04T17:29:26.894664+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:06:24.983072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.911801+0000","last_change":"2024-07-04T17:29:25.942153+0000","last_active":"2024-07-04T17:29:26.911801+0000","last_peered":"2024-07-04T17:29:26.911801+0000","last_clean":"2024-07-04T17:29:26.911801+0000","last_became_active":"2024-07-04T17:29:25.941720+0000","last_became_peered":"2024-07-04T17:29:25.941720+0000","last_unstale":"2024-07-04T17:29:26.911801+0000","last_undegraded":"2024-07-04T17:29:26.911801+0000","last_fullsized":"2024-07-04T17:29:26.911801+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:03:24.301593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.884373+0000","last_change":"2024-07-04T17:29:05.825968+0000","last_active":"2024-07-04T17:29:26.884373+0000","last_peered":"2024-07-04T17:29:26.884373+0000","last_clean":"2024-07-04T17:29:26.884373+0000","last_became_active":"2024-07-04T17:29:05.825203+0000","last_became_peered":"2024-07-04T17:29:05.825203+0000","last_unstale":"2024-07-04T17:29:26.884373+0000","last_undegraded":"2024-07-04T17:29:26.884373+0000","last_fullsized":"2024-07-04T17:29:26.884373+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:19:50.013783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.919795+0000","last_change":"2024-07-04T17:29:25.909129+0000","last_active":"2024-07-04T17:29:26.919795+0000","last_peered":"2024-07-04T17:29:26.919795+0000","last_clean":"2024-07-04T17:29:26.919795+0000","last_became_active":"2024-07-04T17:29:25.908807+0000","last_became_peered":"2024-07-04T17:29:25.908807+0000","last_unstale":"2024-07-04T17:29:26.919795+0000","last_undegraded":"2024-07-04T17:29:26.919795+0000","last_fullsized":"2024-07-04T17:29:26.919795+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:46:57.481302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.921004+0000","last_change":"2024-07-04T17:29:05.783206+0000","last_active":"2024-07-04T17:29:26.921004+0000","last_peered":"2024-07-04T17:29:26.921004+0000","last_clean":"2024-07-04T17:29:26.921004+0000","last_became_active":"2024-07-04T17:29:05.782870+0000","last_became_peered":"2024-07-04T17:29:05.782870+0000","last_unstale":"2024-07-04T17:29:26.921004+0000","last_undegraded":"2024-07-04T17:29:26.921004+0000","last_fullsized":"2024-07-04T17:29:26.921004+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:25:13.149991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.903109+0000","last_change":"2024-07-04T17:29:25.921875+0000","last_active":"2024-07-04T17:29:26.903109+0000","last_peered":"2024-07-04T17:29:26.903109+0000","last_clean":"2024-07-04T17:29:26.903109+0000","last_became_active":"2024-07-04T17:29:25.921525+0000","last_became_peered":"2024-07-04T17:29:25.921525+0000","last_unstale":"2024-07-04T17:29:26.903109+0000","last_undegraded":"2024-07-04T17:29:26.903109+0000","last_fullsized":"2024-07-04T17:29:26.903109+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:24:19.242368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.895145+0000","last_change":"2024-07-04T17:29:05.810625+0000","last_active":"2024-07-04T17:29:26.895145+0000","last_peered":"2024-07-04T17:29:26.895145+0000","last_clean":"2024-07-04T17:29:26.895145+0000","last_became_active":"2024-07-04T17:29:05.810316+0000","last_became_peered":"2024-07-04T17:29:05.810316+0000","last_unstale":"2024-07-04T17:29:26.895145+0000","last_undegraded":"2024-07-04T17:29:26.895145+0000","last_fullsized":"2024-07-04T17:29:26.895145+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:48:26.486742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.915700+0000","last_change":"2024-07-04T17:29:25.905083+0000","last_active":"2024-07-04T17:29:26.915700+0000","last_peered":"2024-07-04T17:29:26.915700+0000","last_clean":"2024-07-04T17:29:26.915700+0000","last_became_active":"2024-07-04T17:29:25.904251+0000","last_became_peered":"2024-07-04T17:29:25.904251+0000","last_unstale":"2024-07-04T17:29:26.915700+0000","last_undegraded":"2024-07-04T17:29:26.915700+0000","last_fullsized":"2024-07-04T17:29:26.915700+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:49:56.055826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.917469+0000","last_change":"2024-07-04T17:29:05.799666+0000","last_active":"2024-07-04T17:29:26.917469+0000","last_peered":"2024-07-04T17:29:26.917469+0000","last_clean":"2024-07-04T17:29:26.917469+0000","last_became_active":"2024-07-04T17:29:05.788794+0000","last_became_peered":"2024-07-04T17:29:05.788794+0000","last_unstale":"2024-07-04T17:29:26.917469+0000","last_undegraded":"2024-07-04T17:29:26.917469+0000","last_fullsized":"2024-07-04T17:29:26.917469+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:33:48.215079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.898301+0000","last_change":"2024-07-04T17:29:25.937361+0000","last_active":"2024-07-04T17:29:26.898301+0000","last_peered":"2024-07-04T17:29:26.898301+0000","last_clean":"2024-07-04T17:29:26.898301+0000","last_became_active":"2024-07-04T17:29:25.935187+0000","last_became_peered":"2024-07-04T17:29:25.935187+0000","last_unstale":"2024-07-04T17:29:26.898301+0000","last_undegraded":"2024-07-04T17:29:26.898301+0000","last_fullsized":"2024-07-04T17:29:26.898301+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:57:46.547083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.916773+0000","last_change":"2024-07-04T17:29:05.803716+0000","last_active":"2024-07-04T17:29:26.916773+0000","last_peered":"2024-07-04T17:29:26.916773+0000","last_clean":"2024-07-04T17:29:26.916773+0000","last_became_active":"2024-07-04T17:29:05.803425+0000","last_became_peered":"2024-07-04T17:29:05.803425+0000","last_unstale":"2024-07-04T17:29:26.916773+0000","last_undegraded":"2024-07-04T17:29:26.916773+0000","last_fullsized":"2024-07-04T17:29:26.916773+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:51:34.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.882898+0000","last_change":"2024-07-04T17:29:25.936087+0000","last_active":"2024-07-04T17:29:26.882898+0000","last_peered":"2024-07-04T17:29:26.882898+0000","last_clean":"2024-07-04T17:29:26.882898+0000","last_became_active":"2024-07-04T17:29:25.925783+0000","last_became_peered":"2024-07-04T17:29:25.925783+0000","last_unstale":"2024-07-04T17:29:26.882898+0000","last_undegraded":"2024-07-04T17:29:26.882898+0000","last_fullsized":"2024-07-04T17:29:26.882898+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:21:06.800541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.886805+0000","last_change":"2024-07-04T17:29:05.802356+0000","last_active":"2024-07-04T17:29:26.886805+0000","last_peered":"2024-07-04T17:29:26.886805+0000","last_clean":"2024-07-04T17:29:26.886805+0000","last_became_active":"2024-07-04T17:29:05.798614+0000","last_became_peered":"2024-07-04T17:29:05.798614+0000","last_unstale":"2024-07-04T17:29:26.886805+0000","last_undegraded":"2024-07-04T17:29:26.886805+0000","last_fullsized":"2024-07-04T17:29:26.886805+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:33:04.621287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.919603+0000","last_change":"2024-07-04T17:29:25.924609+0000","last_active":"2024-07-04T17:29:26.919603+0000","last_peered":"2024-07-04T17:29:26.919603+0000","last_clean":"2024-07-04T17:29:26.919603+0000","last_became_active":"2024-07-04T17:29:25.923063+0000","last_became_peered":"2024-07-04T17:29:25.923063+0000","last_unstale":"2024-07-04T17:29:26.919603+0000","last_undegraded":"2024-07-04T17:29:26.919603+0000","last_fullsized":"2024-07-04T17:29:26.919603+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:43:24.491322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.910054+0000","last_change":"2024-07-04T17:29:05.804142+0000","last_active":"2024-07-04T17:29:26.910054+0000","last_peered":"2024-07-04T17:29:26.910054+0000","last_clean":"2024-07-04T17:29:26.910054+0000","last_became_active":"2024-07-04T17:29:05.799194+0000","last_became_peered":"2024-07-04T17:29:05.799194+0000","last_unstale":"2024-07-04T17:29:26.910054+0000","last_undegraded":"2024-07-04T17:29:26.910054+0000","last_fullsized":"2024-07-04T17:29:26.910054+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:44:51.271924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.905047+0000","last_change":"2024-07-04T17:29:25.921615+0000","last_active":"2024-07-04T17:29:26.905047+0000","last_peered":"2024-07-04T17:29:26.905047+0000","last_clean":"2024-07-04T17:29:26.905047+0000","last_became_active":"2024-07-04T17:29:25.919321+0000","last_became_peered":"2024-07-04T17:29:25.919321+0000","last_unstale":"2024-07-04T17:29:26.905047+0000","last_undegraded":"2024-07-04T17:29:26.905047+0000","last_fullsized":"2024-07-04T17:29:26.905047+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:17:55.141994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.909657+0000","last_change":"2024-07-04T17:29:05.815840+0000","last_active":"2024-07-04T17:29:26.909657+0000","last_peered":"2024-07-04T17:29:26.909657+0000","last_clean":"2024-07-04T17:29:26.909657+0000","last_became_active":"2024-07-04T17:29:05.810912+0000","last_became_peered":"2024-07-04T17:29:05.810912+0000","last_unstale":"2024-07-04T17:29:26.909657+0000","last_undegraded":"2024-07-04T17:29:26.909657+0000","last_fullsized":"2024-07-04T17:29:26.909657+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:31:51.703656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.897909+0000","last_change":"2024-07-04T17:29:25.934566+0000","last_active":"2024-07-04T17:29:26.897909+0000","last_peered":"2024-07-04T17:29:26.897909+0000","last_clean":"2024-07-04T17:29:26.897909+0000","last_became_active":"2024-07-04T17:29:25.931262+0000","last_became_peered":"2024-07-04T17:29:25.931262+0000","last_unstale":"2024-07-04T17:29:26.897909+0000","last_undegraded":"2024-07-04T17:29:26.897909+0000","last_fullsized":"2024-07-04T17:29:26.897909+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:15:54.308742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.908137+0000","last_change":"2024-07-04T17:29:05.785152+0000","last_active":"2024-07-04T17:29:26.908137+0000","last_peered":"2024-07-04T17:29:26.908137+0000","last_clean":"2024-07-04T17:29:26.908137+0000","last_became_active":"2024-07-04T17:29:05.784449+0000","last_became_peered":"2024-07-04T17:29:05.784449+0000","last_unstale":"2024-07-04T17:29:26.908137+0000","last_undegraded":"2024-07-04T17:29:26.908137+0000","last_fullsized":"2024-07-04T17:29:26.908137+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:35:30.393302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.908031+0000","last_change":"2024-07-04T17:29:25.940924+0000","last_active":"2024-07-04T17:29:26.908031+0000","last_peered":"2024-07-04T17:29:26.908031+0000","last_clean":"2024-07-04T17:29:26.908031+0000","last_became_active":"2024-07-04T17:29:25.940610+0000","last_became_peered":"2024-07-04T17:29:25.940610+0000","last_unstale":"2024-07-04T17:29:26.908031+0000","last_undegraded":"2024-07-04T17:29:26.908031+0000","last_fullsized":"2024-07-04T17:29:26.908031+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:33:52.833086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.905671+0000","last_change":"2024-07-04T17:29:05.823239+0000","last_active":"2024-07-04T17:29:26.905671+0000","last_peered":"2024-07-04T17:29:26.905671+0000","last_clean":"2024-07-04T17:29:26.905671+0000","last_became_active":"2024-07-04T17:29:05.814164+0000","last_became_peered":"2024-07-04T17:29:05.814164+0000","last_unstale":"2024-07-04T17:29:26.905671+0000","last_undegraded":"2024-07-04T17:29:26.905671+0000","last_fullsized":"2024-07-04T17:29:26.905671+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:03:20.072994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.917567+0000","last_change":"2024-07-04T17:29:25.923597+0000","last_active":"2024-07-04T17:29:26.917567+0000","last_peered":"2024-07-04T17:29:26.917567+0000","last_clean":"2024-07-04T17:29:26.917567+0000","last_became_active":"2024-07-04T17:29:25.922667+0000","last_became_peered":"2024-07-04T17:29:25.922667+0000","last_unstale":"2024-07-04T17:29:26.917567+0000","last_undegraded":"2024-07-04T17:29:26.917567+0000","last_fullsized":"2024-07-04T17:29:26.917567+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:17:13.242558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.918965+0000","last_change":"2024-07-04T17:29:05.796770+0000","last_active":"2024-07-04T17:29:26.918965+0000","last_peered":"2024-07-04T17:29:26.918965+0000","last_clean":"2024-07-04T17:29:26.918965+0000","last_became_active":"2024-07-04T17:29:05.789644+0000","last_became_peered":"2024-07-04T17:29:05.789644+0000","last_unstale":"2024-07-04T17:29:26.918965+0000","last_undegraded":"2024-07-04T17:29:26.918965+0000","last_fullsized":"2024-07-04T17:29:26.918965+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:58:03.269316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.895202+0000","last_change":"2024-07-04T17:29:25.934419+0000","last_active":"2024-07-04T17:29:26.895202+0000","last_peered":"2024-07-04T17:29:26.895202+0000","last_clean":"2024-07-04T17:29:26.895202+0000","last_became_active":"2024-07-04T17:29:25.932343+0000","last_became_peered":"2024-07-04T17:29:25.932343+0000","last_unstale":"2024-07-04T17:29:26.895202+0000","last_undegraded":"2024-07-04T17:29:26.895202+0000","last_fullsized":"2024-07-04T17:29:26.895202+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:56:51.847871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.888310+0000","last_change":"2024-07-04T17:29:05.808774+0000","last_active":"2024-07-04T17:29:26.888310+0000","last_peered":"2024-07-04T17:29:26.888310+0000","last_clean":"2024-07-04T17:29:26.888310+0000","last_became_active":"2024-07-04T17:29:05.808459+0000","last_became_peered":"2024-07-04T17:29:05.808459+0000","last_unstale":"2024-07-04T17:29:26.888310+0000","last_undegraded":"2024-07-04T17:29:26.888310+0000","last_fullsized":"2024-07-04T17:29:26.888310+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:13:24.537766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.918273+0000","last_change":"2024-07-04T17:29:25.906262+0000","last_active":"2024-07-04T17:29:26.918273+0000","last_peered":"2024-07-04T17:29:26.918273+0000","last_clean":"2024-07-04T17:29:26.918273+0000","last_became_active":"2024-07-04T17:29:25.904574+0000","last_became_peered":"2024-07-04T17:29:25.904574+0000","last_unstale":"2024-07-04T17:29:26.918273+0000","last_undegraded":"2024-07-04T17:29:26.918273+0000","last_fullsized":"2024-07-04T17:29:26.918273+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:55:05.827423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.917160+0000","last_change":"2024-07-04T17:29:05.800412+0000","last_active":"2024-07-04T17:29:26.917160+0000","last_peered":"2024-07-04T17:29:26.917160+0000","last_clean":"2024-07-04T17:29:26.917160+0000","last_became_active":"2024-07-04T17:29:05.799152+0000","last_became_peered":"2024-07-04T17:29:05.799152+0000","last_unstale":"2024-07-04T17:29:26.917160+0000","last_undegraded":"2024-07-04T17:29:26.917160+0000","last_fullsized":"2024-07-04T17:29:26.917160+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:48:07.191680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.883257+0000","last_change":"2024-07-04T17:29:25.944657+0000","last_active":"2024-07-04T17:29:26.883257+0000","last_peered":"2024-07-04T17:29:26.883257+0000","last_clean":"2024-07-04T17:29:26.883257+0000","last_became_active":"2024-07-04T17:29:25.943605+0000","last_became_peered":"2024-07-04T17:29:25.943605+0000","last_unstale":"2024-07-04T17:29:26.883257+0000","last_undegraded":"2024-07-04T17:29:26.883257+0000","last_fullsized":"2024-07-04T17:29:26.883257+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:05:18.667710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.917790+0000","last_change":"2024-07-04T17:29:05.795773+0000","last_active":"2024-07-04T17:29:26.917790+0000","last_peered":"2024-07-04T17:29:26.917790+0000","last_clean":"2024-07-04T17:29:26.917790+0000","last_became_active":"2024-07-04T17:29:05.795470+0000","last_became_peered":"2024-07-04T17:29:05.795470+0000","last_unstale":"2024-07-04T17:29:26.917790+0000","last_undegraded":"2024-07-04T17:29:26.917790+0000","last_fullsized":"2024-07-04T17:29:26.917790+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:32:34.271916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.883524+0000","last_change":"2024-07-04T17:29:25.947789+0000","last_active":"2024-07-04T17:29:26.883524+0000","last_peered":"2024-07-04T17:29:26.883524+0000","last_clean":"2024-07-04T17:29:26.883524+0000","last_became_active":"2024-07-04T17:29:25.943347+0000","last_became_peered":"2024-07-04T17:29:25.943347+0000","last_unstale":"2024-07-04T17:29:26.883524+0000","last_undegraded":"2024-07-04T17:29:26.883524+0000","last_fullsized":"2024-07-04T17:29:26.883524+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:03:24.301593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.893671+0000","last_change":"2024-07-04T17:29:05.820276+0000","last_active":"2024-07-04T17:29:26.893671+0000","last_peered":"2024-07-04T17:29:26.893671+0000","last_clean":"2024-07-04T17:29:26.893671+0000","last_became_active":"2024-07-04T17:29:05.819959+0000","last_became_peered":"2024-07-04T17:29:05.819959+0000","last_unstale":"2024-07-04T17:29:26.893671+0000","last_undegraded":"2024-07-04T17:29:26.893671+0000","last_fullsized":"2024-07-04T17:29:26.893671+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:50:43.333430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.918755+0000","last_change":"2024-07-04T17:29:25.924876+0000","last_active":"2024-07-04T17:29:26.918755+0000","last_peered":"2024-07-04T17:29:26.918755+0000","last_clean":"2024-07-04T17:29:26.918755+0000","last_became_active":"2024-07-04T17:29:25.923540+0000","last_became_peered":"2024-07-04T17:29:25.923540+0000","last_unstale":"2024-07-04T17:29:26.918755+0000","last_undegraded":"2024-07-04T17:29:26.918755+0000","last_fullsized":"2024-07-04T17:29:26.918755+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:05:14.499220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.894950+0000","last_change":"2024-07-04T17:29:05.817114+0000","last_active":"2024-07-04T17:29:26.894950+0000","last_peered":"2024-07-04T17:29:26.894950+0000","last_clean":"2024-07-04T17:29:26.894950+0000","last_became_active":"2024-07-04T17:29:05.816724+0000","last_became_peered":"2024-07-04T17:29:05.816724+0000","last_unstale":"2024-07-04T17:29:26.894950+0000","last_undegraded":"2024-07-04T17:29:26.894950+0000","last_fullsized":"2024-07-04T17:29:26.894950+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:44:51.271924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.895424+0000","last_change":"2024-07-04T17:29:25.948453+0000","last_active":"2024-07-04T17:29:26.895424+0000","last_peered":"2024-07-04T17:29:26.895424+0000","last_clean":"2024-07-04T17:29:26.895424+0000","last_became_active":"2024-07-04T17:29:25.943851+0000","last_became_peered":"2024-07-04T17:29:25.943851+0000","last_unstale":"2024-07-04T17:29:26.895424+0000","last_undegraded":"2024-07-04T17:29:26.895424+0000","last_fullsized":"2024-07-04T17:29:26.895424+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:32:09.090128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.884041+0000","last_change":"2024-07-04T17:29:05.816646+0000","last_active":"2024-07-04T17:29:26.884041+0000","last_peered":"2024-07-04T17:29:26.884041+0000","last_clean":"2024-07-04T17:29:26.884041+0000","last_became_active":"2024-07-04T17:29:05.816345+0000","last_became_peered":"2024-07-04T17:29:05.816345+0000","last_unstale":"2024-07-04T17:29:26.884041+0000","last_undegraded":"2024-07-04T17:29:26.884041+0000","last_fullsized":"2024-07-04T17:29:26.884041+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:31:51.703656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.908207+0000","last_change":"2024-07-04T17:29:25.943057+0000","last_active":"2024-07-04T17:29:26.908207+0000","last_peered":"2024-07-04T17:29:26.908207+0000","last_clean":"2024-07-04T17:29:26.908207+0000","last_became_active":"2024-07-04T17:29:25.938914+0000","last_became_peered":"2024-07-04T17:29:25.938914+0000","last_unstale":"2024-07-04T17:29:26.908207+0000","last_undegraded":"2024-07-04T17:29:26.908207+0000","last_fullsized":"2024-07-04T17:29:26.908207+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:09:11.948958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.905880+0000","last_change":"2024-07-04T17:29:05.792444+0000","last_active":"2024-07-04T17:29:26.905880+0000","last_peered":"2024-07-04T17:29:26.905880+0000","last_clean":"2024-07-04T17:29:26.905880+0000","last_became_active":"2024-07-04T17:29:05.792058+0000","last_became_peered":"2024-07-04T17:29:05.792058+0000","last_unstale":"2024-07-04T17:29:26.905880+0000","last_undegraded":"2024-07-04T17:29:26.905880+0000","last_fullsized":"2024-07-04T17:29:26.905880+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:13:24.537766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.903377+0000","last_change":"2024-07-04T17:29:25.909296+0000","last_active":"2024-07-04T17:29:26.903377+0000","last_peered":"2024-07-04T17:29:26.903377+0000","last_clean":"2024-07-04T17:29:26.903377+0000","last_became_active":"2024-07-04T17:29:25.908940+0000","last_became_peered":"2024-07-04T17:29:25.908940+0000","last_unstale":"2024-07-04T17:29:26.903377+0000","last_undegraded":"2024-07-04T17:29:26.903377+0000","last_fullsized":"2024-07-04T17:29:26.903377+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:48:27.299927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.919341+0000","last_change":"2024-07-04T17:29:05.799399+0000","last_active":"2024-07-04T17:29:26.919341+0000","last_peered":"2024-07-04T17:29:26.919341+0000","last_clean":"2024-07-04T17:29:26.919341+0000","last_became_active":"2024-07-04T17:29:05.789221+0000","last_became_peered":"2024-07-04T17:29:05.789221+0000","last_unstale":"2024-07-04T17:29:26.919341+0000","last_undegraded":"2024-07-04T17:29:26.919341+0000","last_fullsized":"2024-07-04T17:29:26.919341+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:23:53.530646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.915961+0000","last_change":"2024-07-04T17:29:25.922918+0000","last_active":"2024-07-04T17:29:26.915961+0000","last_peered":"2024-07-04T17:29:26.915961+0000","last_clean":"2024-07-04T17:29:26.915961+0000","last_became_active":"2024-07-04T17:29:25.921933+0000","last_became_peered":"2024-07-04T17:29:25.921933+0000","last_unstale":"2024-07-04T17:29:26.915961+0000","last_undegraded":"2024-07-04T17:29:26.915961+0000","last_fullsized":"2024-07-04T17:29:26.915961+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:39:31.441441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.888443+0000","last_change":"2024-07-04T17:29:05.813751+0000","last_active":"2024-07-04T17:29:26.888443+0000","last_peered":"2024-07-04T17:29:26.888443+0000","last_clean":"2024-07-04T17:29:26.888443+0000","last_became_active":"2024-07-04T17:29:05.813362+0000","last_became_peered":"2024-07-04T17:29:05.813362+0000","last_unstale":"2024-07-04T17:29:26.888443+0000","last_undegraded":"2024-07-04T17:29:26.888443+0000","last_fullsized":"2024-07-04T17:29:26.888443+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:58:35.184348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.921689+0000","last_change":"2024-07-04T17:29:25.920481+0000","last_active":"2024-07-04T17:29:26.921689+0000","last_peered":"2024-07-04T17:29:26.921689+0000","last_clean":"2024-07-04T17:29:26.921689+0000","last_became_active":"2024-07-04T17:29:25.920127+0000","last_became_peered":"2024-07-04T17:29:25.920127+0000","last_unstale":"2024-07-04T17:29:26.921689+0000","last_undegraded":"2024-07-04T17:29:26.921689+0000","last_fullsized":"2024-07-04T17:29:26.921689+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:59:24.190086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.903927+0000","last_change":"2024-07-04T17:29:05.809788+0000","last_active":"2024-07-04T17:29:26.903927+0000","last_peered":"2024-07-04T17:29:26.903927+0000","last_clean":"2024-07-04T17:29:26.903927+0000","last_became_active":"2024-07-04T17:29:05.809201+0000","last_became_peered":"2024-07-04T17:29:05.809201+0000","last_unstale":"2024-07-04T17:29:26.903927+0000","last_undegraded":"2024-07-04T17:29:26.903927+0000","last_fullsized":"2024-07-04T17:29:26.903927+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:48:26.486742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.884025+0000","last_change":"2024-07-04T17:29:25.944666+0000","last_active":"2024-07-04T17:29:26.884025+0000","last_peered":"2024-07-04T17:29:26.884025+0000","last_clean":"2024-07-04T17:29:26.884025+0000","last_became_active":"2024-07-04T17:29:25.943104+0000","last_became_peered":"2024-07-04T17:29:25.943104+0000","last_unstale":"2024-07-04T17:29:26.884025+0000","last_undegraded":"2024-07-04T17:29:26.884025+0000","last_fullsized":"2024-07-04T17:29:26.884025+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:39:38.751729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.902694+0000","last_change":"2024-07-04T17:29:05.820075+0000","last_active":"2024-07-04T17:29:26.902694+0000","last_peered":"2024-07-04T17:29:26.902694+0000","last_clean":"2024-07-04T17:29:26.902694+0000","last_became_active":"2024-07-04T17:29:05.819738+0000","last_became_peered":"2024-07-04T17:29:05.819738+0000","last_unstale":"2024-07-04T17:29:26.902694+0000","last_undegraded":"2024-07-04T17:29:26.902694+0000","last_fullsized":"2024-07-04T17:29:26.902694+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:47:41.453377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.918687+0000","last_change":"2024-07-04T17:29:25.905372+0000","last_active":"2024-07-04T17:29:26.918687+0000","last_peered":"2024-07-04T17:29:26.918687+0000","last_clean":"2024-07-04T17:29:26.918687+0000","last_became_active":"2024-07-04T17:29:25.904919+0000","last_became_peered":"2024-07-04T17:29:25.904919+0000","last_unstale":"2024-07-04T17:29:26.918687+0000","last_undegraded":"2024-07-04T17:29:26.918687+0000","last_fullsized":"2024-07-04T17:29:26.918687+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:21:49.074154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.888029+0000","last_change":"2024-07-04T17:29:05.832254+0000","last_active":"2024-07-04T17:29:26.888029+0000","last_peered":"2024-07-04T17:29:26.888029+0000","last_clean":"2024-07-04T17:29:26.888029+0000","last_became_active":"2024-07-04T17:29:05.830771+0000","last_became_peered":"2024-07-04T17:29:05.830771+0000","last_unstale":"2024-07-04T17:29:26.888029+0000","last_undegraded":"2024-07-04T17:29:26.888029+0000","last_fullsized":"2024-07-04T17:29:26.888029+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:01:45.227217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.918914+0000","last_change":"2024-07-04T17:29:25.930828+0000","last_active":"2024-07-04T17:29:26.918914+0000","last_peered":"2024-07-04T17:29:26.918914+0000","last_clean":"2024-07-04T17:29:26.918914+0000","last_became_active":"2024-07-04T17:29:25.930440+0000","last_became_peered":"2024-07-04T17:29:25.930440+0000","last_unstale":"2024-07-04T17:29:26.918914+0000","last_undegraded":"2024-07-04T17:29:26.918914+0000","last_fullsized":"2024-07-04T17:29:26.918914+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:47:01.162367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.921691+0000","last_change":"2024-07-04T17:29:05.787671+0000","last_active":"2024-07-04T17:29:26.921691+0000","last_peered":"2024-07-04T17:29:26.921691+0000","last_clean":"2024-07-04T17:29:26.921691+0000","last_became_active":"2024-07-04T17:29:05.783612+0000","last_became_peered":"2024-07-04T17:29:05.783612+0000","last_unstale":"2024-07-04T17:29:26.921691+0000","last_undegraded":"2024-07-04T17:29:26.921691+0000","last_fullsized":"2024-07-04T17:29:26.921691+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:17:31.565173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.912000+0000","last_change":"2024-07-04T17:29:25.911967+0000","last_active":"2024-07-04T17:29:26.912000+0000","last_peered":"2024-07-04T17:29:26.912000+0000","last_clean":"2024-07-04T17:29:26.912000+0000","last_became_active":"2024-07-04T17:29:25.909919+0000","last_became_peered":"2024-07-04T17:29:25.909919+0000","last_unstale":"2024-07-04T17:29:26.912000+0000","last_undegraded":"2024-07-04T17:29:26.912000+0000","last_fullsized":"2024-07-04T17:29:26.912000+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:39:38.751729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.919224+0000","last_change":"2024-07-04T17:29:05.785801+0000","last_active":"2024-07-04T17:29:26.919224+0000","last_peered":"2024-07-04T17:29:26.919224+0000","last_clean":"2024-07-04T17:29:26.919224+0000","last_became_active":"2024-07-04T17:29:05.784924+0000","last_became_peered":"2024-07-04T17:29:05.784924+0000","last_unstale":"2024-07-04T17:29:26.919224+0000","last_undegraded":"2024-07-04T17:29:26.919224+0000","last_fullsized":"2024-07-04T17:29:26.919224+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:52:10.436668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.908358+0000","last_change":"2024-07-04T17:29:25.920858+0000","last_active":"2024-07-04T17:29:26.908358+0000","last_peered":"2024-07-04T17:29:26.908358+0000","last_clean":"2024-07-04T17:29:26.908358+0000","last_became_active":"2024-07-04T17:29:25.920505+0000","last_became_peered":"2024-07-04T17:29:25.920505+0000","last_unstale":"2024-07-04T17:29:26.908358+0000","last_undegraded":"2024-07-04T17:29:26.908358+0000","last_fullsized":"2024-07-04T17:29:26.908358+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:01:26.547719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.894628+0000","last_change":"2024-07-04T17:29:05.817096+0000","last_active":"2024-07-04T17:29:26.894628+0000","last_peered":"2024-07-04T17:29:26.894628+0000","last_clean":"2024-07-04T17:29:26.894628+0000","last_became_active":"2024-07-04T17:29:05.816304+0000","last_became_peered":"2024-07-04T17:29:05.816304+0000","last_unstale":"2024-07-04T17:29:26.894628+0000","last_undegraded":"2024-07-04T17:29:26.894628+0000","last_fullsized":"2024-07-04T17:29:26.894628+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:44:09.505235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.915977+0000","last_change":"2024-07-04T17:29:25.925124+0000","last_active":"2024-07-04T17:29:26.915977+0000","last_peered":"2024-07-04T17:29:26.915977+0000","last_clean":"2024-07-04T17:29:26.915977+0000","last_became_active":"2024-07-04T17:29:25.923082+0000","last_became_peered":"2024-07-04T17:29:25.923082+0000","last_unstale":"2024-07-04T17:29:26.915977+0000","last_undegraded":"2024-07-04T17:29:26.915977+0000","last_fullsized":"2024-07-04T17:29:26.915977+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:58:53.811042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.895642+0000","last_change":"2024-07-04T17:29:05.825187+0000","last_active":"2024-07-04T17:29:26.895642+0000","last_peered":"2024-07-04T17:29:26.895642+0000","last_clean":"2024-07-04T17:29:26.895642+0000","last_became_active":"2024-07-04T17:29:05.824857+0000","last_became_peered":"2024-07-04T17:29:05.824857+0000","last_unstale":"2024-07-04T17:29:26.895642+0000","last_undegraded":"2024-07-04T17:29:26.895642+0000","last_fullsized":"2024-07-04T17:29:26.895642+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:14:24.515173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.916203+0000","last_change":"2024-07-04T17:29:25.919972+0000","last_active":"2024-07-04T17:29:26.916203+0000","last_peered":"2024-07-04T17:29:26.916203+0000","last_clean":"2024-07-04T17:29:26.916203+0000","last_became_active":"2024-07-04T17:29:25.919635+0000","last_became_peered":"2024-07-04T17:29:25.919635+0000","last_unstale":"2024-07-04T17:29:26.916203+0000","last_undegraded":"2024-07-04T17:29:26.916203+0000","last_fullsized":"2024-07-04T17:29:26.916203+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:39:45.543393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.903863+0000","last_change":"2024-07-04T17:29:05.815028+0000","last_active":"2024-07-04T17:29:26.903863+0000","last_peered":"2024-07-04T17:29:26.903863+0000","last_clean":"2024-07-04T17:29:26.903863+0000","last_became_active":"2024-07-04T17:29:05.814094+0000","last_became_peered":"2024-07-04T17:29:05.814094+0000","last_unstale":"2024-07-04T17:29:26.903863+0000","last_undegraded":"2024-07-04T17:29:26.903863+0000","last_fullsized":"2024-07-04T17:29:26.903863+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:48:33.874991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.905443+0000","last_change":"2024-07-04T17:29:25.928433+0000","last_active":"2024-07-04T17:29:26.905443+0000","last_peered":"2024-07-04T17:29:26.905443+0000","last_clean":"2024-07-04T17:29:26.905443+0000","last_became_active":"2024-07-04T17:29:25.928016+0000","last_became_peered":"2024-07-04T17:29:25.928016+0000","last_unstale":"2024-07-04T17:29:26.905443+0000","last_undegraded":"2024-07-04T17:29:26.905443+0000","last_fullsized":"2024-07-04T17:29:26.905443+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:53:29.865667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.883574+0000","last_change":"2024-07-04T17:29:05.813654+0000","last_active":"2024-07-04T17:29:26.883574+0000","last_peered":"2024-07-04T17:29:26.883574+0000","last_clean":"2024-07-04T17:29:26.883574+0000","last_became_active":"2024-07-04T17:29:05.813139+0000","last_became_peered":"2024-07-04T17:29:05.813139+0000","last_unstale":"2024-07-04T17:29:26.883574+0000","last_undegraded":"2024-07-04T17:29:26.883574+0000","last_fullsized":"2024-07-04T17:29:26.883574+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:36:15.241406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.903686+0000","last_change":"2024-07-04T17:29:25.900786+0000","last_active":"2024-07-04T17:29:26.903686+0000","last_peered":"2024-07-04T17:29:26.903686+0000","last_clean":"2024-07-04T17:29:26.903686+0000","last_became_active":"2024-07-04T17:29:25.900420+0000","last_became_peered":"2024-07-04T17:29:25.900420+0000","last_unstale":"2024-07-04T17:29:26.903686+0000","last_undegraded":"2024-07-04T17:29:26.903686+0000","last_fullsized":"2024-07-04T17:29:26.903686+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:36:21.965421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.917257+0000","last_change":"2024-07-04T17:29:05.799841+0000","last_active":"2024-07-04T17:29:26.917257+0000","last_peered":"2024-07-04T17:29:26.917257+0000","last_clean":"2024-07-04T17:29:26.917257+0000","last_became_active":"2024-07-04T17:29:05.799478+0000","last_became_peered":"2024-07-04T17:29:05.799478+0000","last_unstale":"2024-07-04T17:29:26.917257+0000","last_undegraded":"2024-07-04T17:29:26.917257+0000","last_fullsized":"2024-07-04T17:29:26.917257+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:25:37.804442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.887738+0000","last_change":"2024-07-04T17:29:25.929707+0000","last_active":"2024-07-04T17:29:26.887738+0000","last_peered":"2024-07-04T17:29:26.887738+0000","last_clean":"2024-07-04T17:29:26.887738+0000","last_became_active":"2024-07-04T17:29:25.926712+0000","last_became_peered":"2024-07-04T17:29:25.926712+0000","last_unstale":"2024-07-04T17:29:26.887738+0000","last_undegraded":"2024-07-04T17:29:26.887738+0000","last_fullsized":"2024-07-04T17:29:26.887738+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:00:32.585006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.884303+0000","last_change":"2024-07-04T17:29:24.901349+0000","last_active":"2024-07-04T17:29:26.884303+0000","last_peered":"2024-07-04T17:29:26.884303+0000","last_clean":"2024-07-04T17:29:26.884303+0000","last_became_active":"2024-07-04T17:29:24.900750+0000","last_became_peered":"2024-07-04T17:29:24.900750+0000","last_unstale":"2024-07-04T17:29:26.884303+0000","last_undegraded":"2024-07-04T17:29:26.884303+0000","last_fullsized":"2024-07-04T17:29:26.884303+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:01:47.208317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.921816+0000","last_change":"2024-07-04T17:29:05.795312+0000","last_active":"2024-07-04T17:29:26.921816+0000","last_peered":"2024-07-04T17:29:26.921816+0000","last_clean":"2024-07-04T17:29:26.921816+0000","last_became_active":"2024-07-04T17:29:05.794446+0000","last_became_peered":"2024-07-04T17:29:05.794446+0000","last_unstale":"2024-07-04T17:29:26.921816+0000","last_undegraded":"2024-07-04T17:29:26.921816+0000","last_fullsized":"2024-07-04T17:29:26.921816+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:54:42.187784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"49'4","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.920595+0000","last_change":"2024-07-04T17:29:04.768776+0000","last_active":"2024-07-04T17:29:26.920595+0000","last_peered":"2024-07-04T17:29:26.920595+0000","last_clean":"2024-07-04T17:29:26.920595+0000","last_became_active":"2024-07-04T17:29:04.766498+0000","last_became_peered":"2024-07-04T17:29:04.766498+0000","last_unstale":"2024-07-04T17:29:26.920595+0000","last_undegraded":"2024-07-04T17:29:26.920595+0000","last_fullsized":"2024-07-04T17:29:26.920595+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:03:44.115014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.907640+0000","last_change":"2024-07-04T17:29:25.912673+0000","last_active":"2024-07-04T17:29:26.907640+0000","last_peered":"2024-07-04T17:29:26.907640+0000","last_clean":"2024-07-04T17:29:26.907640+0000","last_became_active":"2024-07-04T17:29:25.912357+0000","last_became_peered":"2024-07-04T17:29:25.912357+0000","last_unstale":"2024-07-04T17:29:26.907640+0000","last_undegraded":"2024-07-04T17:29:26.907640+0000","last_fullsized":"2024-07-04T17:29:26.907640+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:36:48.570829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.887107+0000","last_change":"2024-07-04T17:29:24.900405+0000","last_active":"2024-07-04T17:29:26.887107+0000","last_peered":"2024-07-04T17:29:26.887107+0000","last_clean":"2024-07-04T17:29:26.887107+0000","last_became_active":"2024-07-04T17:29:24.899947+0000","last_became_peered":"2024-07-04T17:29:24.899947+0000","last_unstale":"2024-07-04T17:29:26.887107+0000","last_undegraded":"2024-07-04T17:29:26.887107+0000","last_fullsized":"2024-07-04T17:29:26.887107+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:09:43.577366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.910205+0000","last_change":"2024-07-04T17:29:05.790122+0000","last_active":"2024-07-04T17:29:26.910205+0000","last_peered":"2024-07-04T17:29:26.910205+0000","last_clean":"2024-07-04T17:29:26.910205+0000","last_became_active":"2024-07-04T17:29:05.789804+0000","last_became_peered":"2024-07-04T17:29:05.789804+0000","last_unstale":"2024-07-04T17:29:26.910205+0000","last_undegraded":"2024-07-04T17:29:26.910205+0000","last_fullsized":"2024-07-04T17:29:26.910205+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:06:24.983072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"49'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.909787+0000","last_change":"2024-07-04T17:29:04.779844+0000","last_active":"2024-07-04T17:29:26.909787+0000","last_peered":"2024-07-04T17:29:26.909787+0000","last_clean":"2024-07-04T17:29:26.909787+0000","last_became_active":"2024-07-04T17:29:04.779212+0000","last_became_peered":"2024-07-04T17:29:04.779212+0000","last_unstale":"2024-07-04T17:29:26.909787+0000","last_undegraded":"2024-07-04T17:29:26.909787+0000","last_fullsized":"2024-07-04T17:29:26.909787+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:06:45.499488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.894213+0000","last_change":"2024-07-04T17:29:25.935198+0000","last_active":"2024-07-04T17:29:26.894213+0000","last_peered":"2024-07-04T17:29:26.894213+0000","last_clean":"2024-07-04T17:29:26.894213+0000","last_became_active":"2024-07-04T17:29:25.931181+0000","last_became_peered":"2024-07-04T17:29:25.931181+0000","last_unstale":"2024-07-04T17:29:26.894213+0000","last_undegraded":"2024-07-04T17:29:26.894213+0000","last_fullsized":"2024-07-04T17:29:26.894213+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:08:25.013508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1f","version":"53'4","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.894113+0000","last_change":"2024-07-04T17:29:24.900038+0000","last_active":"2024-07-04T17:29:26.894113+0000","last_peered":"2024-07-04T17:29:26.894113+0000","last_clean":"2024-07-04T17:29:26.894113+0000","last_became_active":"2024-07-04T17:29:24.899634+0000","last_became_peered":"2024-07-04T17:29:24.899634+0000","last_unstale":"2024-07-04T17:29:26.894113+0000","last_undegraded":"2024-07-04T17:29:26.894113+0000","last_fullsized":"2024-07-04T17:29:26.894113+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:31:21.584062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.921100+0000","last_change":"2024-07-04T17:29:05.799208+0000","last_active":"2024-07-04T17:29:26.921100+0000","last_peered":"2024-07-04T17:29:26.921100+0000","last_clean":"2024-07-04T17:29:26.921100+0000","last_became_active":"2024-07-04T17:29:05.796533+0000","last_became_peered":"2024-07-04T17:29:05.796533+0000","last_unstale":"2024-07-04T17:29:26.921100+0000","last_undegraded":"2024-07-04T17:29:26.921100+0000","last_fullsized":"2024-07-04T17:29:26.921100+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:28:33.331115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.920078+0000","last_change":"2024-07-04T17:29:04.767288+0000","last_active":"2024-07-04T17:29:26.920078+0000","last_peered":"2024-07-04T17:29:26.920078+0000","last_clean":"2024-07-04T17:29:26.920078+0000","last_became_active":"2024-07-04T17:29:04.765876+0000","last_became_peered":"2024-07-04T17:29:04.765876+0000","last_unstale":"2024-07-04T17:29:26.920078+0000","last_undegraded":"2024-07-04T17:29:26.920078+0000","last_fullsized":"2024-07-04T17:29:26.920078+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:54:49.684096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.888275+0000","last_change":"2024-07-04T17:29:25.950238+0000","last_active":"2024-07-04T17:29:26.888275+0000","last_peered":"2024-07-04T17:29:26.888275+0000","last_clean":"2024-07-04T17:29:26.888275+0000","last_became_active":"2024-07-04T17:29:25.947467+0000","last_became_peered":"2024-07-04T17:29:25.947467+0000","last_unstale":"2024-07-04T17:29:26.888275+0000","last_undegraded":"2024-07-04T17:29:26.888275+0000","last_fullsized":"2024-07-04T17:29:26.888275+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:29:31.123493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.920256+0000","last_change":"2024-07-04T17:29:24.906305+0000","last_active":"2024-07-04T17:29:26.920256+0000","last_peered":"2024-07-04T17:29:26.920256+0000","last_clean":"2024-07-04T17:29:26.920256+0000","last_became_active":"2024-07-04T17:29:24.905901+0000","last_became_peered":"2024-07-04T17:29:24.905901+0000","last_unstale":"2024-07-04T17:29:26.920256+0000","last_undegraded":"2024-07-04T17:29:26.920256+0000","last_fullsized":"2024-07-04T17:29:26.920256+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:57:45.538215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.919388+0000","last_change":"2024-07-04T17:29:05.785226+0000","last_active":"2024-07-04T17:29:26.919388+0000","last_peered":"2024-07-04T17:29:26.919388+0000","last_clean":"2024-07-04T17:29:26.919388+0000","last_became_active":"2024-07-04T17:29:05.784069+0000","last_became_peered":"2024-07-04T17:29:05.784069+0000","last_unstale":"2024-07-04T17:29:26.919388+0000","last_undegraded":"2024-07-04T17:29:26.919388+0000","last_fullsized":"2024-07-04T17:29:26.919388+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:27:43.064532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.895965+0000","last_change":"2024-07-04T17:29:04.793187+0000","last_active":"2024-07-04T17:29:26.895965+0000","last_peered":"2024-07-04T17:29:26.895965+0000","last_clean":"2024-07-04T17:29:26.895965+0000","last_became_active":"2024-07-04T17:29:04.792877+0000","last_became_peered":"2024-07-04T17:29:04.792877+0000","last_unstale":"2024-07-04T17:29:26.895965+0000","last_undegraded":"2024-07-04T17:29:26.895965+0000","last_fullsized":"2024-07-04T17:29:26.895965+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:16:45.088529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.883127+0000","last_change":"2024-07-04T17:29:25.941792+0000","last_active":"2024-07-04T17:29:26.883127+0000","last_peered":"2024-07-04T17:29:26.883127+0000","last_clean":"2024-07-04T17:29:26.883127+0000","last_became_active":"2024-07-04T17:29:25.941112+0000","last_became_peered":"2024-07-04T17:29:25.941112+0000","last_unstale":"2024-07-04T17:29:26.883127+0000","last_undegraded":"2024-07-04T17:29:26.883127+0000","last_fullsized":"2024-07-04T17:29:26.883127+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:09:11.948958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.903528+0000","last_change":"2024-07-04T17:29:24.897427+0000","last_active":"2024-07-04T17:29:26.903528+0000","last_peered":"2024-07-04T17:29:26.903528+0000","last_clean":"2024-07-04T17:29:26.903528+0000","last_became_active":"2024-07-04T17:29:24.896153+0000","last_became_peered":"2024-07-04T17:29:24.896153+0000","last_unstale":"2024-07-04T17:29:26.903528+0000","last_undegraded":"2024-07-04T17:29:26.903528+0000","last_fullsized":"2024-07-04T17:29:26.903528+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:29:29.433683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.903987+0000","last_change":"2024-07-04T17:29:05.796869+0000","last_active":"2024-07-04T17:29:26.903987+0000","last_peered":"2024-07-04T17:29:26.903987+0000","last_clean":"2024-07-04T17:29:26.903987+0000","last_became_active":"2024-07-04T17:29:05.796573+0000","last_became_peered":"2024-07-04T17:29:05.796573+0000","last_unstale":"2024-07-04T17:29:26.903987+0000","last_undegraded":"2024-07-04T17:29:26.903987+0000","last_fullsized":"2024-07-04T17:29:26.903987+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:41:01.850387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1b","version":"49'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.898364+0000","last_change":"2024-07-04T17:29:04.793835+0000","last_active":"2024-07-04T17:29:26.898364+0000","last_peered":"2024-07-04T17:29:26.898364+0000","last_clean":"2024-07-04T17:29:26.898364+0000","last_became_active":"2024-07-04T17:29:04.792785+0000","last_became_peered":"2024-07-04T17:29:04.792785+0000","last_unstale":"2024-07-04T17:29:26.898364+0000","last_undegraded":"2024-07-04T17:29:26.898364+0000","last_fullsized":"2024-07-04T17:29:26.898364+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:21:48.040633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.916361+0000","last_change":"2024-07-04T17:29:25.905773+0000","last_active":"2024-07-04T17:29:26.916361+0000","last_peered":"2024-07-04T17:29:26.916361+0000","last_clean":"2024-07-04T17:29:26.916361+0000","last_became_active":"2024-07-04T17:29:25.905444+0000","last_became_peered":"2024-07-04T17:29:25.905444+0000","last_unstale":"2024-07-04T17:29:26.916361+0000","last_undegraded":"2024-07-04T17:29:26.916361+0000","last_fullsized":"2024-07-04T17:29:26.916361+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:27:15.519089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.884436+0000","last_change":"2024-07-04T17:29:24.897264+0000","last_active":"2024-07-04T17:29:26.884436+0000","last_peered":"2024-07-04T17:29:26.884436+0000","last_clean":"2024-07-04T17:29:26.884436+0000","last_became_active":"2024-07-04T17:29:24.896807+0000","last_became_peered":"2024-07-04T17:29:24.896807+0000","last_unstale":"2024-07-04T17:29:26.884436+0000","last_undegraded":"2024-07-04T17:29:26.884436+0000","last_fullsized":"2024-07-04T17:29:26.884436+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:21:29.893615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.919645+0000","last_change":"2024-07-04T17:29:05.796490+0000","last_active":"2024-07-04T17:29:26.919645+0000","last_peered":"2024-07-04T17:29:26.919645+0000","last_clean":"2024-07-04T17:29:26.919645+0000","last_became_active":"2024-07-04T17:29:05.788778+0000","last_became_peered":"2024-07-04T17:29:05.788778+0000","last_unstale":"2024-07-04T17:29:26.919645+0000","last_undegraded":"2024-07-04T17:29:26.919645+0000","last_fullsized":"2024-07-04T17:29:26.919645+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:23:59.134121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1a","version":"49'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.917632+0000","last_change":"2024-07-04T17:29:04.756095+0000","last_active":"2024-07-04T17:29:26.917632+0000","last_peered":"2024-07-04T17:29:26.917632+0000","last_clean":"2024-07-04T17:29:26.917632+0000","last_became_active":"2024-07-04T17:29:04.755679+0000","last_became_peered":"2024-07-04T17:29:04.755679+0000","last_unstale":"2024-07-04T17:29:26.917632+0000","last_undegraded":"2024-07-04T17:29:26.917632+0000","last_fullsized":"2024-07-04T17:29:26.917632+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:30:46.500676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.884386+0000","last_change":"2024-07-04T17:29:25.936840+0000","last_active":"2024-07-04T17:29:26.884386+0000","last_peered":"2024-07-04T17:29:26.884386+0000","last_clean":"2024-07-04T17:29:26.884386+0000","last_became_active":"2024-07-04T17:29:25.935770+0000","last_became_peered":"2024-07-04T17:29:25.935770+0000","last_unstale":"2024-07-04T17:29:26.884386+0000","last_undegraded":"2024-07-04T17:29:26.884386+0000","last_fullsized":"2024-07-04T17:29:26.884386+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:38:02.551875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1b","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.893735+0000","last_change":"2024-07-04T17:29:24.897558+0000","last_active":"2024-07-04T17:29:26.893735+0000","last_peered":"2024-07-04T17:29:26.893735+0000","last_clean":"2024-07-04T17:29:26.893735+0000","last_became_active":"2024-07-04T17:29:24.896987+0000","last_became_peered":"2024-07-04T17:29:24.896987+0000","last_unstale":"2024-07-04T17:29:26.893735+0000","last_undegraded":"2024-07-04T17:29:26.893735+0000","last_fullsized":"2024-07-04T17:29:26.893735+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:48:17.259024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.887343+0000","last_change":"2024-07-04T17:29:05.809480+0000","last_active":"2024-07-04T17:29:26.887343+0000","last_peered":"2024-07-04T17:29:26.887343+0000","last_clean":"2024-07-04T17:29:26.887343+0000","last_became_active":"2024-07-04T17:29:05.809138+0000","last_became_peered":"2024-07-04T17:29:05.809138+0000","last_unstale":"2024-07-04T17:29:26.887343+0000","last_undegraded":"2024-07-04T17:29:26.887343+0000","last_fullsized":"2024-07-04T17:29:26.887343+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:03:20.072994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.904091+0000","last_change":"2024-07-04T17:29:04.776840+0000","last_active":"2024-07-04T17:29:26.904091+0000","last_peered":"2024-07-04T17:29:26.904091+0000","last_clean":"2024-07-04T17:29:26.904091+0000","last_became_active":"2024-07-04T17:29:04.775729+0000","last_became_peered":"2024-07-04T17:29:04.775729+0000","last_unstale":"2024-07-04T17:29:26.904091+0000","last_undegraded":"2024-07-04T17:29:26.904091+0000","last_fullsized":"2024-07-04T17:29:26.904091+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:28:41.894029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.895965+0000","last_change":"2024-07-04T17:29:25.944549+0000","last_active":"2024-07-04T17:29:26.895965+0000","last_peered":"2024-07-04T17:29:26.895965+0000","last_clean":"2024-07-04T17:29:26.895965+0000","last_became_active":"2024-07-04T17:29:25.942180+0000","last_became_peered":"2024-07-04T17:29:25.942180+0000","last_unstale":"2024-07-04T17:29:26.895965+0000","last_undegraded":"2024-07-04T17:29:26.895965+0000","last_fullsized":"2024-07-04T17:29:26.895965+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:19:14.326622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.903619+0000","last_change":"2024-07-04T17:29:24.900033+0000","last_active":"2024-07-04T17:29:26.903619+0000","last_peered":"2024-07-04T17:29:26.903619+0000","last_clean":"2024-07-04T17:29:26.903619+0000","last_became_active":"2024-07-04T17:29:24.899184+0000","last_became_peered":"2024-07-04T17:29:24.899184+0000","last_unstale":"2024-07-04T17:29:26.903619+0000","last_undegraded":"2024-07-04T17:29:26.903619+0000","last_fullsized":"2024-07-04T17:29:26.903619+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:59:59.456695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.883891+0000","last_change":"2024-07-04T17:29:05.818108+0000","last_active":"2024-07-04T17:29:26.883891+0000","last_peered":"2024-07-04T17:29:26.883891+0000","last_clean":"2024-07-04T17:29:26.883891+0000","last_became_active":"2024-07-04T17:29:05.816963+0000","last_became_peered":"2024-07-04T17:29:05.816963+0000","last_unstale":"2024-07-04T17:29:26.883891+0000","last_undegraded":"2024-07-04T17:29:26.883891+0000","last_fullsized":"2024-07-04T17:29:26.883891+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:07:08.833691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.917101+0000","last_change":"2024-07-04T17:29:04.774790+0000","last_active":"2024-07-04T17:29:26.917101+0000","last_peered":"2024-07-04T17:29:26.917101+0000","last_clean":"2024-07-04T17:29:26.917101+0000","last_became_active":"2024-07-04T17:29:04.774427+0000","last_became_peered":"2024-07-04T17:29:04.774427+0000","last_unstale":"2024-07-04T17:29:26.917101+0000","last_undegraded":"2024-07-04T17:29:26.917101+0000","last_fullsized":"2024-07-04T17:29:26.917101+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:28:48.914599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.919267+0000","last_change":"2024-07-04T17:29:25.907093+0000","last_active":"2024-07-04T17:29:26.919267+0000","last_peered":"2024-07-04T17:29:26.919267+0000","last_clean":"2024-07-04T17:29:26.919267+0000","last_became_active":"2024-07-04T17:29:25.904489+0000","last_became_peered":"2024-07-04T17:29:25.904489+0000","last_unstale":"2024-07-04T17:29:26.919267+0000","last_undegraded":"2024-07-04T17:29:26.919267+0000","last_fullsized":"2024-07-04T17:29:26.919267+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:14:29.722776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.15","version":"53'3","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.918946+0000","last_change":"2024-07-04T17:29:24.905925+0000","last_active":"2024-07-04T17:29:26.918946+0000","last_peered":"2024-07-04T17:29:26.918946+0000","last_clean":"2024-07-04T17:29:26.918946+0000","last_became_active":"2024-07-04T17:29:24.905580+0000","last_became_peered":"2024-07-04T17:29:24.905580+0000","last_unstale":"2024-07-04T17:29:26.918946+0000","last_undegraded":"2024-07-04T17:29:26.918946+0000","last_fullsized":"2024-07-04T17:29:26.918946+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:15:53.299874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.921847+0000","last_change":"2024-07-04T17:29:05.795423+0000","last_active":"2024-07-04T17:29:26.921847+0000","last_peered":"2024-07-04T17:29:26.921847+0000","last_clean":"2024-07-04T17:29:26.921847+0000","last_became_active":"2024-07-04T17:29:05.794868+0000","last_became_peered":"2024-07-04T17:29:05.794868+0000","last_unstale":"2024-07-04T17:29:26.921847+0000","last_undegraded":"2024-07-04T17:29:26.921847+0000","last_fullsized":"2024-07-04T17:29:26.921847+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:36:01.857174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.911610+0000","last_change":"2024-07-04T17:29:04.772998+0000","last_active":"2024-07-04T17:29:26.911610+0000","last_peered":"2024-07-04T17:29:26.911610+0000","last_clean":"2024-07-04T17:29:26.911610+0000","last_became_active":"2024-07-04T17:29:04.772635+0000","last_became_peered":"2024-07-04T17:29:04.772635+0000","last_unstale":"2024-07-04T17:29:26.911610+0000","last_undegraded":"2024-07-04T17:29:26.911610+0000","last_fullsized":"2024-07-04T17:29:26.911610+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:59:48.689604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.894589+0000","last_change":"2024-07-04T17:29:25.950423+0000","last_active":"2024-07-04T17:29:26.894589+0000","last_peered":"2024-07-04T17:29:26.894589+0000","last_clean":"2024-07-04T17:29:26.894589+0000","last_became_active":"2024-07-04T17:29:25.948056+0000","last_became_peered":"2024-07-04T17:29:25.948056+0000","last_unstale":"2024-07-04T17:29:26.894589+0000","last_undegraded":"2024-07-04T17:29:26.894589+0000","last_fullsized":"2024-07-04T17:29:26.894589+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:36:00.670947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"53'6","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.905534+0000","last_change":"2024-07-04T17:29:24.894897+0000","last_active":"2024-07-04T17:29:26.905534+0000","last_peered":"2024-07-04T17:29:26.905534+0000","last_clean":"2024-07-04T17:29:26.905534+0000","last_became_active":"2024-07-04T17:29:24.894614+0000","last_became_peered":"2024-07-04T17:29:24.894614+0000","last_unstale":"2024-07-04T17:29:26.905534+0000","last_undegraded":"2024-07-04T17:29:26.905534+0000","last_fullsized":"2024-07-04T17:29:26.905534+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:58:52.867982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.893545+0000","last_change":"2024-07-04T17:29:05.832809+0000","last_active":"2024-07-04T17:29:26.893545+0000","last_peered":"2024-07-04T17:29:26.893545+0000","last_clean":"2024-07-04T17:29:26.893545+0000","last_became_active":"2024-07-04T17:29:05.829582+0000","last_became_peered":"2024-07-04T17:29:05.829582+0000","last_unstale":"2024-07-04T17:29:26.893545+0000","last_undegraded":"2024-07-04T17:29:26.893545+0000","last_fullsized":"2024-07-04T17:29:26.893545+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:50:42.737526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.909969+0000","last_change":"2024-07-04T17:29:04.780571+0000","last_active":"2024-07-04T17:29:26.909969+0000","last_peered":"2024-07-04T17:29:26.909969+0000","last_clean":"2024-07-04T17:29:26.909969+0000","last_became_active":"2024-07-04T17:29:04.780236+0000","last_became_peered":"2024-07-04T17:29:04.780236+0000","last_unstale":"2024-07-04T17:29:26.909969+0000","last_undegraded":"2024-07-04T17:29:26.909969+0000","last_fullsized":"2024-07-04T17:29:26.909969+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:55:21.724770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.908703+0000","last_change":"2024-07-04T17:29:25.911751+0000","last_active":"2024-07-04T17:29:26.908703+0000","last_peered":"2024-07-04T17:29:26.908703+0000","last_clean":"2024-07-04T17:29:26.908703+0000","last_became_active":"2024-07-04T17:29:25.909886+0000","last_became_peered":"2024-07-04T17:29:25.909886+0000","last_unstale":"2024-07-04T17:29:26.908703+0000","last_undegraded":"2024-07-04T17:29:26.908703+0000","last_fullsized":"2024-07-04T17:29:26.908703+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:35:54.403127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.919843+0000","last_change":"2024-07-04T17:29:24.900115+0000","last_active":"2024-07-04T17:29:26.919843+0000","last_peered":"2024-07-04T17:29:26.919843+0000","last_clean":"2024-07-04T17:29:26.919843+0000","last_became_active":"2024-07-04T17:29:24.899540+0000","last_became_peered":"2024-07-04T17:29:24.899540+0000","last_unstale":"2024-07-04T17:29:26.919843+0000","last_undegraded":"2024-07-04T17:29:26.919843+0000","last_fullsized":"2024-07-04T17:29:26.919843+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:26:10.444100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.905868+0000","last_change":"2024-07-04T17:29:05.783421+0000","last_active":"2024-07-04T17:29:26.905868+0000","last_peered":"2024-07-04T17:29:26.905868+0000","last_clean":"2024-07-04T17:29:26.905868+0000","last_became_active":"2024-07-04T17:29:05.783057+0000","last_became_peered":"2024-07-04T17:29:05.783057+0000","last_unstale":"2024-07-04T17:29:26.905868+0000","last_undegraded":"2024-07-04T17:29:26.905868+0000","last_fullsized":"2024-07-04T17:29:26.905868+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:44:09.505235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"49'3","reported_seq":25,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.883724+0000","last_change":"2024-07-04T17:29:04.794454+0000","last_active":"2024-07-04T17:29:26.883724+0000","last_peered":"2024-07-04T17:29:26.883724+0000","last_clean":"2024-07-04T17:29:26.883724+0000","last_became_active":"2024-07-04T17:29:04.793939+0000","last_became_peered":"2024-07-04T17:29:04.793939+0000","last_unstale":"2024-07-04T17:29:26.883724+0000","last_undegraded":"2024-07-04T17:29:26.883724+0000","last_fullsized":"2024-07-04T17:29:26.883724+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:10:23.616021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.920177+0000","last_change":"2024-07-04T17:29:25.929853+0000","last_active":"2024-07-04T17:29:26.920177+0000","last_peered":"2024-07-04T17:29:26.920177+0000","last_clean":"2024-07-04T17:29:26.920177+0000","last_became_active":"2024-07-04T17:29:25.929273+0000","last_became_peered":"2024-07-04T17:29:25.929273+0000","last_unstale":"2024-07-04T17:29:26.920177+0000","last_undegraded":"2024-07-04T17:29:26.920177+0000","last_fullsized":"2024-07-04T17:29:26.920177+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:56:42.644139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.920910+0000","last_change":"2024-07-04T17:29:24.905893+0000","last_active":"2024-07-04T17:29:26.920910+0000","last_peered":"2024-07-04T17:29:26.920910+0000","last_clean":"2024-07-04T17:29:26.920910+0000","last_became_active":"2024-07-04T17:29:24.903376+0000","last_became_peered":"2024-07-04T17:29:24.903376+0000","last_unstale":"2024-07-04T17:29:26.920910+0000","last_undegraded":"2024-07-04T17:29:26.920910+0000","last_fullsized":"2024-07-04T17:29:26.920910+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:57:14.138542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.908335+0000","last_change":"2024-07-04T17:29:05.792689+0000","last_active":"2024-07-04T17:29:26.908335+0000","last_peered":"2024-07-04T17:29:26.908335+0000","last_clean":"2024-07-04T17:29:26.908335+0000","last_became_active":"2024-07-04T17:29:05.792368+0000","last_became_peered":"2024-07-04T17:29:05.792368+0000","last_unstale":"2024-07-04T17:29:26.908335+0000","last_undegraded":"2024-07-04T17:29:26.908335+0000","last_fullsized":"2024-07-04T17:29:26.908335+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:58:35.184348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.14","version":"50'8","reported_seq":30,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.921334+0000","last_change":"2024-07-04T17:29:04.778462+0000","last_active":"2024-07-04T17:29:26.921334+0000","last_peered":"2024-07-04T17:29:26.921334+0000","last_clean":"2024-07-04T17:29:26.921334+0000","last_became_active":"2024-07-04T17:29:04.775998+0000","last_became_peered":"2024-07-04T17:29:04.775998+0000","last_unstale":"2024-07-04T17:29:26.921334+0000","last_undegraded":"2024-07-04T17:29:26.921334+0000","last_fullsized":"2024-07-04T17:29:26.921334+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:16:55.504943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3309,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.917089+0000","last_change":"2024-07-04T17:29:25.919884+0000","last_active":"2024-07-04T17:29:26.917089+0000","last_peered":"2024-07-04T17:29:26.917089+0000","last_clean":"2024-07-04T17:29:26.917089+0000","last_became_active":"2024-07-04T17:29:25.916603+0000","last_became_peered":"2024-07-04T17:29:25.916603+0000","last_unstale":"2024-07-04T17:29:26.917089+0000","last_undegraded":"2024-07-04T17:29:26.917089+0000","last_fullsized":"2024-07-04T17:29:26.917089+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:36:32.157304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.903835+0000","last_change":"2024-07-04T17:29:24.896781+0000","last_active":"2024-07-04T17:29:26.903835+0000","last_peered":"2024-07-04T17:29:26.903835+0000","last_clean":"2024-07-04T17:29:26.903835+0000","last_became_active":"2024-07-04T17:29:24.895390+0000","last_became_peered":"2024-07-04T17:29:24.895390+0000","last_unstale":"2024-07-04T17:29:26.903835+0000","last_undegraded":"2024-07-04T17:29:26.903835+0000","last_fullsized":"2024-07-04T17:29:26.903835+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:10:57.006780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.904108+0000","last_change":"2024-07-04T17:29:05.815269+0000","last_active":"2024-07-04T17:29:26.904108+0000","last_peered":"2024-07-04T17:29:26.904108+0000","last_clean":"2024-07-04T17:29:26.904108+0000","last_became_active":"2024-07-04T17:29:05.814468+0000","last_became_peered":"2024-07-04T17:29:05.814468+0000","last_unstale":"2024-07-04T17:29:26.904108+0000","last_undegraded":"2024-07-04T17:29:26.904108+0000","last_fullsized":"2024-07-04T17:29:26.904108+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:14:24.515173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.904555+0000","last_change":"2024-07-04T17:29:04.782215+0000","last_active":"2024-07-04T17:29:26.904555+0000","last_peered":"2024-07-04T17:29:26.904555+0000","last_clean":"2024-07-04T17:29:26.904555+0000","last_became_active":"2024-07-04T17:29:04.781891+0000","last_became_peered":"2024-07-04T17:29:04.781891+0000","last_unstale":"2024-07-04T17:29:26.904555+0000","last_undegraded":"2024-07-04T17:29:26.904555+0000","last_fullsized":"2024-07-04T17:29:26.904555+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:56:44.314464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.882969+0000","last_change":"2024-07-04T17:29:25.935820+0000","last_active":"2024-07-04T17:29:26.882969+0000","last_peered":"2024-07-04T17:29:26.882969+0000","last_clean":"2024-07-04T17:29:26.882969+0000","last_became_active":"2024-07-04T17:29:25.926927+0000","last_became_peered":"2024-07-04T17:29:25.926927+0000","last_unstale":"2024-07-04T17:29:26.882969+0000","last_undegraded":"2024-07-04T17:29:26.882969+0000","last_fullsized":"2024-07-04T17:29:26.882969+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:33:52.833086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"53'8","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.902531+0000","last_change":"2024-07-04T17:29:24.894996+0000","last_active":"2024-07-04T17:29:26.902531+0000","last_peered":"2024-07-04T17:29:26.902531+0000","last_clean":"2024-07-04T17:29:26.902531+0000","last_became_active":"2024-07-04T17:29:24.894682+0000","last_became_peered":"2024-07-04T17:29:24.894682+0000","last_unstale":"2024-07-04T17:29:26.902531+0000","last_undegraded":"2024-07-04T17:29:26.902531+0000","last_fullsized":"2024-07-04T17:29:26.902531+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:20:48.765120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.898196+0000","last_change":"2024-07-04T17:29:05.830879+0000","last_active":"2024-07-04T17:29:26.898196+0000","last_peered":"2024-07-04T17:29:26.898196+0000","last_clean":"2024-07-04T17:29:26.898196+0000","last_became_active":"2024-07-04T17:29:05.830572+0000","last_became_peered":"2024-07-04T17:29:05.830572+0000","last_unstale":"2024-07-04T17:29:26.898196+0000","last_undegraded":"2024-07-04T17:29:26.898196+0000","last_fullsized":"2024-07-04T17:29:26.898196+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:41:01.850387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.894239+0000","last_change":"2024-07-04T17:29:04.793440+0000","last_active":"2024-07-04T17:29:26.894239+0000","last_peered":"2024-07-04T17:29:26.894239+0000","last_clean":"2024-07-04T17:29:26.894239+0000","last_became_active":"2024-07-04T17:29:04.792810+0000","last_became_peered":"2024-07-04T17:29:04.792810+0000","last_unstale":"2024-07-04T17:29:26.894239+0000","last_undegraded":"2024-07-04T17:29:26.894239+0000","last_fullsized":"2024-07-04T17:29:26.894239+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:28:41.894029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.886658+0000","last_change":"2024-07-04T17:29:25.936383+0000","last_active":"2024-07-04T17:29:26.886658+0000","last_peered":"2024-07-04T17:29:26.886658+0000","last_clean":"2024-07-04T17:29:26.886658+0000","last_became_active":"2024-07-04T17:29:25.926218+0000","last_became_peered":"2024-07-04T17:29:25.926218+0000","last_unstale":"2024-07-04T17:29:26.886658+0000","last_undegraded":"2024-07-04T17:29:26.886658+0000","last_fullsized":"2024-07-04T17:29:26.886658+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:43:08.292816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.905850+0000","last_change":"2024-07-04T17:29:24.898810+0000","last_active":"2024-07-04T17:29:26.905850+0000","last_peered":"2024-07-04T17:29:26.905850+0000","last_clean":"2024-07-04T17:29:26.905850+0000","last_became_active":"2024-07-04T17:29:24.898434+0000","last_became_peered":"2024-07-04T17:29:24.898434+0000","last_unstale":"2024-07-04T17:29:26.905850+0000","last_undegraded":"2024-07-04T17:29:26.905850+0000","last_fullsized":"2024-07-04T17:29:26.905850+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:21:29.893615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.902784+0000","last_change":"2024-07-04T17:29:05.822812+0000","last_active":"2024-07-04T17:29:26.902784+0000","last_peered":"2024-07-04T17:29:26.902784+0000","last_clean":"2024-07-04T17:29:26.902784+0000","last_became_active":"2024-07-04T17:29:05.822519+0000","last_became_peered":"2024-07-04T17:29:05.822519+0000","last_unstale":"2024-07-04T17:29:26.902784+0000","last_undegraded":"2024-07-04T17:29:26.902784+0000","last_fullsized":"2024-07-04T17:29:26.902784+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:33:04.621287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.886564+0000","last_change":"2024-07-04T17:29:04.793535+0000","last_active":"2024-07-04T17:29:26.886564+0000","last_peered":"2024-07-04T17:29:26.886564+0000","last_clean":"2024-07-04T17:29:26.886564+0000","last_became_active":"2024-07-04T17:29:04.793160+0000","last_became_peered":"2024-07-04T17:29:04.793160+0000","last_unstale":"2024-07-04T17:29:26.886564+0000","last_undegraded":"2024-07-04T17:29:26.886564+0000","last_fullsized":"2024-07-04T17:29:26.886564+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:29:48.755500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.904278+0000","last_change":"2024-07-04T17:29:25.909449+0000","last_active":"2024-07-04T17:29:26.904278+0000","last_peered":"2024-07-04T17:29:26.904278+0000","last_clean":"2024-07-04T17:29:26.904278+0000","last_became_active":"2024-07-04T17:29:25.909092+0000","last_became_peered":"2024-07-04T17:29:25.909092+0000","last_unstale":"2024-07-04T17:29:26.904278+0000","last_undegraded":"2024-07-04T17:29:26.904278+0000","last_fullsized":"2024-07-04T17:29:26.904278+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:34:07.700861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.903748+0000","last_change":"2024-07-04T17:29:24.898515+0000","last_active":"2024-07-04T17:29:26.903748+0000","last_peered":"2024-07-04T17:29:26.903748+0000","last_clean":"2024-07-04T17:29:26.903748+0000","last_became_active":"2024-07-04T17:29:24.897717+0000","last_became_peered":"2024-07-04T17:29:24.897717+0000","last_unstale":"2024-07-04T17:29:26.903748+0000","last_undegraded":"2024-07-04T17:29:26.903748+0000","last_fullsized":"2024-07-04T17:29:26.903748+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:41:26.553241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.917197+0000","last_change":"2024-07-04T17:29:05.793691+0000","last_active":"2024-07-04T17:29:26.917197+0000","last_peered":"2024-07-04T17:29:26.917197+0000","last_clean":"2024-07-04T17:29:26.917197+0000","last_became_active":"2024-07-04T17:29:05.792767+0000","last_became_peered":"2024-07-04T17:29:05.792767+0000","last_unstale":"2024-07-04T17:29:26.917197+0000","last_undegraded":"2024-07-04T17:29:26.917197+0000","last_fullsized":"2024-07-04T17:29:26.917197+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:25:53.656021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.10","version":"49'8","reported_seq":30,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.916754+0000","last_change":"2024-07-04T17:29:04.767026+0000","last_active":"2024-07-04T17:29:26.916754+0000","last_peered":"2024-07-04T17:29:26.916754+0000","last_clean":"2024-07-04T17:29:26.916754+0000","last_became_active":"2024-07-04T17:29:04.764888+0000","last_became_peered":"2024-07-04T17:29:04.764888+0000","last_unstale":"2024-07-04T17:29:26.916754+0000","last_undegraded":"2024-07-04T17:29:26.916754+0000","last_fullsized":"2024-07-04T17:29:26.916754+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:08:09.031069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.905228+0000","last_change":"2024-07-04T17:29:25.921748+0000","last_active":"2024-07-04T17:29:26.905228+0000","last_peered":"2024-07-04T17:29:26.905228+0000","last_clean":"2024-07-04T17:29:26.905228+0000","last_became_active":"2024-07-04T17:29:25.920943+0000","last_became_peered":"2024-07-04T17:29:25.920943+0000","last_unstale":"2024-07-04T17:29:26.905228+0000","last_undegraded":"2024-07-04T17:29:26.905228+0000","last_fullsized":"2024-07-04T17:29:26.905228+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:26:13.764268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.918883+0000","last_change":"2024-07-04T17:29:24.905805+0000","last_active":"2024-07-04T17:29:26.918883+0000","last_peered":"2024-07-04T17:29:26.918883+0000","last_clean":"2024-07-04T17:29:26.918883+0000","last_became_active":"2024-07-04T17:29:24.903556+0000","last_became_peered":"2024-07-04T17:29:24.903556+0000","last_unstale":"2024-07-04T17:29:26.918883+0000","last_undegraded":"2024-07-04T17:29:26.918883+0000","last_fullsized":"2024-07-04T17:29:26.918883+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:19:13.317754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.895241+0000","last_change":"2024-07-04T17:29:05.826630+0000","last_active":"2024-07-04T17:29:26.895241+0000","last_peered":"2024-07-04T17:29:26.895241+0000","last_clean":"2024-07-04T17:29:26.895241+0000","last_became_active":"2024-07-04T17:29:05.826180+0000","last_became_peered":"2024-07-04T17:29:05.826180+0000","last_unstale":"2024-07-04T17:29:26.895241+0000","last_undegraded":"2024-07-04T17:29:26.895241+0000","last_fullsized":"2024-07-04T17:29:26.895241+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:42:34.680103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"49'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.920649+0000","last_change":"2024-07-04T17:29:04.778986+0000","last_active":"2024-07-04T17:29:26.920649+0000","last_peered":"2024-07-04T17:29:26.920649+0000","last_clean":"2024-07-04T17:29:26.920649+0000","last_became_active":"2024-07-04T17:29:04.775326+0000","last_became_peered":"2024-07-04T17:29:04.775326+0000","last_unstale":"2024-07-04T17:29:26.920649+0000","last_undegraded":"2024-07-04T17:29:26.920649+0000","last_fullsized":"2024-07-04T17:29:26.920649+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:01:44.218523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.887333+0000","last_change":"2024-07-04T17:29:25.948373+0000","last_active":"2024-07-04T17:29:26.887333+0000","last_peered":"2024-07-04T17:29:26.887333+0000","last_clean":"2024-07-04T17:29:26.887333+0000","last_became_active":"2024-07-04T17:29:25.944152+0000","last_became_peered":"2024-07-04T17:29:25.944152+0000","last_unstale":"2024-07-04T17:29:26.887333+0000","last_undegraded":"2024-07-04T17:29:26.887333+0000","last_fullsized":"2024-07-04T17:29:26.887333+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:06:13.998695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.887483+0000","last_change":"2024-07-04T17:29:24.899625+0000","last_active":"2024-07-04T17:29:26.887483+0000","last_peered":"2024-07-04T17:29:26.887483+0000","last_clean":"2024-07-04T17:29:26.887483+0000","last_became_active":"2024-07-04T17:29:24.899254+0000","last_became_peered":"2024-07-04T17:29:24.899254+0000","last_unstale":"2024-07-04T17:29:26.887483+0000","last_undegraded":"2024-07-04T17:29:26.887483+0000","last_fullsized":"2024-07-04T17:29:26.887483+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:25:30.548940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.888459+0000","last_change":"2024-07-04T17:29:05.825845+0000","last_active":"2024-07-04T17:29:26.888459+0000","last_peered":"2024-07-04T17:29:26.888459+0000","last_clean":"2024-07-04T17:29:26.888459+0000","last_became_active":"2024-07-04T17:29:05.824923+0000","last_became_peered":"2024-07-04T17:29:05.824923+0000","last_unstale":"2024-07-04T17:29:26.888459+0000","last_undegraded":"2024-07-04T17:29:26.888459+0000","last_fullsized":"2024-07-04T17:29:26.888459+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:30:47.509371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.921983+0000","last_change":"2024-07-04T17:29:04.787322+0000","last_active":"2024-07-04T17:29:26.921983+0000","last_peered":"2024-07-04T17:29:26.921983+0000","last_clean":"2024-07-04T17:29:26.921983+0000","last_became_active":"2024-07-04T17:29:04.786939+0000","last_became_peered":"2024-07-04T17:29:04.786939+0000","last_unstale":"2024-07-04T17:29:26.921983+0000","last_undegraded":"2024-07-04T17:29:26.921983+0000","last_fullsized":"2024-07-04T17:29:26.921983+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:31:08.982204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.921660+0000","last_change":"2024-07-04T17:29:25.907591+0000","last_active":"2024-07-04T17:29:26.921660+0000","last_peered":"2024-07-04T17:29:26.921660+0000","last_clean":"2024-07-04T17:29:26.921660+0000","last_became_active":"2024-07-04T17:29:25.905992+0000","last_became_peered":"2024-07-04T17:29:25.905992+0000","last_unstale":"2024-07-04T17:29:26.921660+0000","last_undegraded":"2024-07-04T17:29:26.921660+0000","last_fullsized":"2024-07-04T17:29:26.921660+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:42:27.033468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.909376+0000","last_change":"2024-07-04T17:29:24.896510+0000","last_active":"2024-07-04T17:29:26.909376+0000","last_peered":"2024-07-04T17:29:26.909376+0000","last_clean":"2024-07-04T17:29:26.909376+0000","last_became_active":"2024-07-04T17:29:24.896182+0000","last_became_peered":"2024-07-04T17:29:24.896182+0000","last_unstale":"2024-07-04T17:29:26.909376+0000","last_undegraded":"2024-07-04T17:29:26.909376+0000","last_fullsized":"2024-07-04T17:29:26.909376+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:24:16.866462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.895159+0000","last_change":"2024-07-04T17:29:05.817332+0000","last_active":"2024-07-04T17:29:26.895159+0000","last_peered":"2024-07-04T17:29:26.895159+0000","last_clean":"2024-07-04T17:29:26.895159+0000","last_became_active":"2024-07-04T17:29:05.816980+0000","last_became_peered":"2024-07-04T17:29:05.816980+0000","last_unstale":"2024-07-04T17:29:26.895159+0000","last_undegraded":"2024-07-04T17:29:26.895159+0000","last_fullsized":"2024-07-04T17:29:26.895159+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:35:30.393302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"49'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.883595+0000","last_change":"2024-07-04T17:29:04.793950+0000","last_active":"2024-07-04T17:29:26.883595+0000","last_peered":"2024-07-04T17:29:26.883595+0000","last_clean":"2024-07-04T17:29:26.883595+0000","last_became_active":"2024-07-04T17:29:04.793520+0000","last_became_peered":"2024-07-04T17:29:04.793520+0000","last_unstale":"2024-07-04T17:29:26.883595+0000","last_undegraded":"2024-07-04T17:29:26.883595+0000","last_fullsized":"2024-07-04T17:29:26.883595+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:43:57.308367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.920495+0000","last_change":"2024-07-04T17:29:25.905407+0000","last_active":"2024-07-04T17:29:26.920495+0000","last_peered":"2024-07-04T17:29:26.920495+0000","last_clean":"2024-07-04T17:29:26.920495+0000","last_became_active":"2024-07-04T17:29:25.904925+0000","last_became_peered":"2024-07-04T17:29:25.904925+0000","last_unstale":"2024-07-04T17:29:26.920495+0000","last_undegraded":"2024-07-04T17:29:26.920495+0000","last_fullsized":"2024-07-04T17:29:26.920495+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:46:17.384367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.921168+0000","last_change":"2024-07-04T17:29:24.902667+0000","last_active":"2024-07-04T17:29:26.921168+0000","last_peered":"2024-07-04T17:29:26.921168+0000","last_clean":"2024-07-04T17:29:26.921168+0000","last_became_active":"2024-07-04T17:29:24.902340+0000","last_became_peered":"2024-07-04T17:29:24.902340+0000","last_unstale":"2024-07-04T17:29:26.921168+0000","last_undegraded":"2024-07-04T17:29:26.921168+0000","last_fullsized":"2024-07-04T17:29:26.921168+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:41:08.756979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.919736+0000","last_change":"2024-07-04T17:29:05.796377+0000","last_active":"2024-07-04T17:29:26.919736+0000","last_peered":"2024-07-04T17:29:26.919736+0000","last_clean":"2024-07-04T17:29:26.919736+0000","last_became_active":"2024-07-04T17:29:05.796069+0000","last_became_peered":"2024-07-04T17:29:05.796069+0000","last_unstale":"2024-07-04T17:29:26.919736+0000","last_undegraded":"2024-07-04T17:29:26.919736+0000","last_fullsized":"2024-07-04T17:29:26.919736+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:36:28.462582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.921376+0000","last_change":"2024-07-04T17:29:04.775241+0000","last_active":"2024-07-04T17:29:26.921376+0000","last_peered":"2024-07-04T17:29:26.921376+0000","last_clean":"2024-07-04T17:29:26.921376+0000","last_became_active":"2024-07-04T17:29:04.774292+0000","last_became_peered":"2024-07-04T17:29:04.774292+0000","last_unstale":"2024-07-04T17:29:26.921376+0000","last_undegraded":"2024-07-04T17:29:26.921376+0000","last_fullsized":"2024-07-04T17:29:26.921376+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:48:39.886887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.884077+0000","last_change":"2024-07-04T17:29:25.948492+0000","last_active":"2024-07-04T17:29:26.884077+0000","last_peered":"2024-07-04T17:29:26.884077+0000","last_clean":"2024-07-04T17:29:26.884077+0000","last_became_active":"2024-07-04T17:29:25.943924+0000","last_became_peered":"2024-07-04T17:29:25.943924+0000","last_unstale":"2024-07-04T17:29:26.884077+0000","last_undegraded":"2024-07-04T17:29:26.884077+0000","last_fullsized":"2024-07-04T17:29:26.884077+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:36:05.779361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.916683+0000","last_change":"2024-07-04T17:29:24.888049+0000","last_active":"2024-07-04T17:29:26.916683+0000","last_peered":"2024-07-04T17:29:26.916683+0000","last_clean":"2024-07-04T17:29:26.916683+0000","last_became_active":"2024-07-04T17:29:24.887713+0000","last_became_peered":"2024-07-04T17:29:24.887713+0000","last_unstale":"2024-07-04T17:29:26.916683+0000","last_undegraded":"2024-07-04T17:29:26.916683+0000","last_fullsized":"2024-07-04T17:29:26.916683+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:24:18.914442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.904205+0000","last_change":"2024-07-04T17:29:05.814789+0000","last_active":"2024-07-04T17:29:26.904205+0000","last_peered":"2024-07-04T17:29:26.904205+0000","last_clean":"2024-07-04T17:29:26.904205+0000","last_became_active":"2024-07-04T17:29:05.813725+0000","last_became_peered":"2024-07-04T17:29:05.813725+0000","last_unstale":"2024-07-04T17:29:26.904205+0000","last_undegraded":"2024-07-04T17:29:26.904205+0000","last_fullsized":"2024-07-04T17:29:26.904205+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:07:08.833691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.920700+0000","last_change":"2024-07-04T17:29:04.768448+0000","last_active":"2024-07-04T17:29:26.920700+0000","last_peered":"2024-07-04T17:29:26.920700+0000","last_clean":"2024-07-04T17:29:26.920700+0000","last_became_active":"2024-07-04T17:29:04.765316+0000","last_became_peered":"2024-07-04T17:29:04.765316+0000","last_unstale":"2024-07-04T17:29:26.920700+0000","last_undegraded":"2024-07-04T17:29:26.920700+0000","last_fullsized":"2024-07-04T17:29:26.920700+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:54:53.983424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.883132+0000","last_change":"2024-07-04T17:29:25.943546+0000","last_active":"2024-07-04T17:29:26.883132+0000","last_peered":"2024-07-04T17:29:26.883132+0000","last_clean":"2024-07-04T17:29:26.883132+0000","last_became_active":"2024-07-04T17:29:25.942910+0000","last_became_peered":"2024-07-04T17:29:25.942910+0000","last_unstale":"2024-07-04T17:29:26.883132+0000","last_undegraded":"2024-07-04T17:29:26.883132+0000","last_fullsized":"2024-07-04T17:29:26.883132+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:01:26.547719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.917578+0000","last_change":"2024-07-04T17:29:24.889471+0000","last_active":"2024-07-04T17:29:26.917578+0000","last_peered":"2024-07-04T17:29:26.917578+0000","last_clean":"2024-07-04T17:29:26.917578+0000","last_became_active":"2024-07-04T17:29:24.889150+0000","last_became_peered":"2024-07-04T17:29:24.889150+0000","last_unstale":"2024-07-04T17:29:26.917578+0000","last_undegraded":"2024-07-04T17:29:26.917578+0000","last_fullsized":"2024-07-04T17:29:26.917578+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:56:50.839003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.919960+0000","last_change":"2024-07-04T17:29:05.784868+0000","last_active":"2024-07-04T17:29:26.919960+0000","last_peered":"2024-07-04T17:29:26.919960+0000","last_clean":"2024-07-04T17:29:26.919960+0000","last_became_active":"2024-07-04T17:29:05.784419+0000","last_became_peered":"2024-07-04T17:29:05.784419+0000","last_unstale":"2024-07-04T17:29:26.919960+0000","last_undegraded":"2024-07-04T17:29:26.919960+0000","last_fullsized":"2024-07-04T17:29:26.919960+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:33:47.592614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.904398+0000","last_change":"2024-07-04T17:29:04.780522+0000","last_active":"2024-07-04T17:29:26.904398+0000","last_peered":"2024-07-04T17:29:26.904398+0000","last_clean":"2024-07-04T17:29:26.904398+0000","last_became_active":"2024-07-04T17:29:04.780161+0000","last_became_peered":"2024-07-04T17:29:04.780161+0000","last_unstale":"2024-07-04T17:29:26.904398+0000","last_undegraded":"2024-07-04T17:29:26.904398+0000","last_fullsized":"2024-07-04T17:29:26.904398+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:29:48.755500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.894823+0000","last_change":"2024-07-04T17:29:25.946621+0000","last_active":"2024-07-04T17:29:26.894823+0000","last_peered":"2024-07-04T17:29:26.894823+0000","last_clean":"2024-07-04T17:29:26.894823+0000","last_became_active":"2024-07-04T17:29:25.943573+0000","last_became_peered":"2024-07-04T17:29:25.943573+0000","last_unstale":"2024-07-04T17:29:26.894823+0000","last_undegraded":"2024-07-04T17:29:26.894823+0000","last_fullsized":"2024-07-04T17:29:26.894823+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:24:19.923310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.887911+0000","last_change":"2024-07-04T17:29:24.892364+0000","last_active":"2024-07-04T17:29:26.887911+0000","last_peered":"2024-07-04T17:29:26.887911+0000","last_clean":"2024-07-04T17:29:26.887911+0000","last_became_active":"2024-07-04T17:29:24.892013+0000","last_became_peered":"2024-07-04T17:29:24.892013+0000","last_unstale":"2024-07-04T17:29:26.887911+0000","last_undegraded":"2024-07-04T17:29:26.887911+0000","last_fullsized":"2024-07-04T17:29:26.887911+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:09:46.797607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.917424+0000","last_change":"2024-07-04T17:29:05.793437+0000","last_active":"2024-07-04T17:29:26.917424+0000","last_peered":"2024-07-04T17:29:26.917424+0000","last_clean":"2024-07-04T17:29:26.917424+0000","last_became_active":"2024-07-04T17:29:05.792368+0000","last_became_peered":"2024-07-04T17:29:05.792368+0000","last_unstale":"2024-07-04T17:29:26.917424+0000","last_undegraded":"2024-07-04T17:29:26.917424+0000","last_fullsized":"2024-07-04T17:29:26.917424+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:17:35.033747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.903016+0000","last_change":"2024-07-04T17:29:04.774550+0000","last_active":"2024-07-04T17:29:26.903016+0000","last_peered":"2024-07-04T17:29:26.903016+0000","last_clean":"2024-07-04T17:29:26.903016+0000","last_became_active":"2024-07-04T17:29:04.774168+0000","last_became_peered":"2024-07-04T17:29:04.774168+0000","last_unstale":"2024-07-04T17:29:26.903016+0000","last_undegraded":"2024-07-04T17:29:26.903016+0000","last_fullsized":"2024-07-04T17:29:26.903016+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:43:57.308367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.919451+0000","last_change":"2024-07-04T17:29:25.924201+0000","last_active":"2024-07-04T17:29:26.919451+0000","last_peered":"2024-07-04T17:29:26.919451+0000","last_clean":"2024-07-04T17:29:26.919451+0000","last_became_active":"2024-07-04T17:29:25.923907+0000","last_became_peered":"2024-07-04T17:29:25.923907+0000","last_unstale":"2024-07-04T17:29:26.919451+0000","last_undegraded":"2024-07-04T17:29:26.919451+0000","last_fullsized":"2024-07-04T17:29:26.919451+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:44:57.717348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.887137+0000","last_change":"2024-07-04T17:29:24.892444+0000","last_active":"2024-07-04T17:29:26.887137+0000","last_peered":"2024-07-04T17:29:26.887137+0000","last_clean":"2024-07-04T17:29:26.887137+0000","last_became_active":"2024-07-04T17:29:24.892078+0000","last_became_peered":"2024-07-04T17:29:24.892078+0000","last_unstale":"2024-07-04T17:29:26.887137+0000","last_undegraded":"2024-07-04T17:29:26.887137+0000","last_fullsized":"2024-07-04T17:29:26.887137+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:31:25.400229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.895574+0000","last_change":"2024-07-04T17:29:05.826827+0000","last_active":"2024-07-04T17:29:26.895574+0000","last_peered":"2024-07-04T17:29:26.895574+0000","last_clean":"2024-07-04T17:29:26.895574+0000","last_became_active":"2024-07-04T17:29:05.825730+0000","last_became_peered":"2024-07-04T17:29:05.825730+0000","last_unstale":"2024-07-04T17:29:26.895574+0000","last_undegraded":"2024-07-04T17:29:26.895574+0000","last_fullsized":"2024-07-04T17:29:26.895574+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:14:31.441713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.911966+0000","last_change":"2024-07-04T17:29:04.780892+0000","last_active":"2024-07-04T17:29:26.911966+0000","last_peered":"2024-07-04T17:29:26.911966+0000","last_clean":"2024-07-04T17:29:26.911966+0000","last_became_active":"2024-07-04T17:29:04.780559+0000","last_became_peered":"2024-07-04T17:29:04.780559+0000","last_unstale":"2024-07-04T17:29:26.911966+0000","last_undegraded":"2024-07-04T17:29:26.911966+0000","last_fullsized":"2024-07-04T17:29:26.911966+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:46:37.056095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.907770+0000","last_change":"2024-07-04T17:29:25.940245+0000","last_active":"2024-07-04T17:29:26.907770+0000","last_peered":"2024-07-04T17:29:26.907770+0000","last_clean":"2024-07-04T17:29:26.907770+0000","last_became_active":"2024-07-04T17:29:25.938812+0000","last_became_peered":"2024-07-04T17:29:25.938812+0000","last_unstale":"2024-07-04T17:29:26.907770+0000","last_undegraded":"2024-07-04T17:29:26.907770+0000","last_fullsized":"2024-07-04T17:29:26.907770+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:21:06.800541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.919611+0000","last_change":"2024-07-04T17:29:24.902583+0000","last_active":"2024-07-04T17:29:26.919611+0000","last_peered":"2024-07-04T17:29:26.919611+0000","last_clean":"2024-07-04T17:29:26.919611+0000","last_became_active":"2024-07-04T17:29:24.902196+0000","last_became_peered":"2024-07-04T17:29:24.902196+0000","last_unstale":"2024-07-04T17:29:26.919611+0000","last_undegraded":"2024-07-04T17:29:26.919611+0000","last_fullsized":"2024-07-04T17:29:26.919611+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:32:08.081260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.888344+0000","last_change":"2024-07-04T17:29:05.831979+0000","last_active":"2024-07-04T17:29:26.888344+0000","last_peered":"2024-07-04T17:29:26.888344+0000","last_clean":"2024-07-04T17:29:26.888344+0000","last_became_active":"2024-07-04T17:29:05.829983+0000","last_became_peered":"2024-07-04T17:29:05.829983+0000","last_unstale":"2024-07-04T17:29:26.888344+0000","last_undegraded":"2024-07-04T17:29:26.888344+0000","last_fullsized":"2024-07-04T17:29:26.888344+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:03:45.123708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"50'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.921994+0000","last_change":"2024-07-04T17:29:04.784413+0000","last_active":"2024-07-04T17:29:26.921994+0000","last_peered":"2024-07-04T17:29:26.921994+0000","last_clean":"2024-07-04T17:29:26.921994+0000","last_became_active":"2024-07-04T17:29:04.784058+0000","last_became_peered":"2024-07-04T17:29:04.784058+0000","last_unstale":"2024-07-04T17:29:26.921994+0000","last_undegraded":"2024-07-04T17:29:26.921994+0000","last_fullsized":"2024-07-04T17:29:26.921994+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:35:32.227491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.920845+0000","last_change":"2024-07-04T17:29:25.924562+0000","last_active":"2024-07-04T17:29:26.920845+0000","last_peered":"2024-07-04T17:29:26.920845+0000","last_clean":"2024-07-04T17:29:26.920845+0000","last_became_active":"2024-07-04T17:29:25.922723+0000","last_became_peered":"2024-07-04T17:29:25.922723+0000","last_unstale":"2024-07-04T17:29:26.920845+0000","last_undegraded":"2024-07-04T17:29:26.920845+0000","last_fullsized":"2024-07-04T17:29:26.920845+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:55:19.068481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.920041+0000","last_change":"2024-07-04T17:29:24.899837+0000","last_active":"2024-07-04T17:29:26.920041+0000","last_peered":"2024-07-04T17:29:26.920041+0000","last_clean":"2024-07-04T17:29:26.920041+0000","last_became_active":"2024-07-04T17:29:24.898972+0000","last_became_peered":"2024-07-04T17:29:24.898972+0000","last_unstale":"2024-07-04T17:29:26.920041+0000","last_undegraded":"2024-07-04T17:29:26.920041+0000","last_fullsized":"2024-07-04T17:29:26.920041+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:20:36.161659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.907872+0000","last_change":"2024-07-04T17:29:05.821127+0000","last_active":"2024-07-04T17:29:26.907872+0000","last_peered":"2024-07-04T17:29:26.907872+0000","last_clean":"2024-07-04T17:29:26.907872+0000","last_became_active":"2024-07-04T17:29:05.820787+0000","last_became_peered":"2024-07-04T17:29:05.820787+0000","last_unstale":"2024-07-04T17:29:26.907872+0000","last_undegraded":"2024-07-04T17:29:26.907872+0000","last_fullsized":"2024-07-04T17:29:26.907872+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:33:26.836346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.908351+0000","last_change":"2024-07-04T17:29:04.786173+0000","last_active":"2024-07-04T17:29:26.908351+0000","last_peered":"2024-07-04T17:29:26.908351+0000","last_clean":"2024-07-04T17:29:26.908351+0000","last_became_active":"2024-07-04T17:29:04.783241+0000","last_became_peered":"2024-07-04T17:29:04.783241+0000","last_unstale":"2024-07-04T17:29:26.908351+0000","last_undegraded":"2024-07-04T17:29:26.908351+0000","last_fullsized":"2024-07-04T17:29:26.908351+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:16:45.088529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.909537+0000","last_change":"2024-07-04T17:29:25.919167+0000","last_active":"2024-07-04T17:29:26.909537+0000","last_peered":"2024-07-04T17:29:26.909537+0000","last_clean":"2024-07-04T17:29:26.909537+0000","last_became_active":"2024-07-04T17:29:25.918786+0000","last_became_peered":"2024-07-04T17:29:25.918786+0000","last_unstale":"2024-07-04T17:29:26.909537+0000","last_undegraded":"2024-07-04T17:29:26.909537+0000","last_fullsized":"2024-07-04T17:29:26.909537+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:05:18.667710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"54'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.914918+0000","last_change":"2024-07-04T17:29:24.897862+0000","last_active":"2024-07-04T17:29:26.914918+0000","last_peered":"2024-07-04T17:29:26.914918+0000","last_clean":"2024-07-04T17:29:26.914918+0000","last_became_active":"2024-07-04T17:29:24.897554+0000","last_became_peered":"2024-07-04T17:29:24.897554+0000","last_unstale":"2024-07-04T17:29:26.914918+0000","last_undegraded":"2024-07-04T17:29:26.914918+0000","last_fullsized":"2024-07-04T17:29:26.914918+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:54:00.256493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.919955+0000","last_change":"2024-07-04T17:29:05.785545+0000","last_active":"2024-07-04T17:29:26.919955+0000","last_peered":"2024-07-04T17:29:26.919955+0000","last_clean":"2024-07-04T17:29:26.919955+0000","last_became_active":"2024-07-04T17:29:05.784520+0000","last_became_peered":"2024-07-04T17:29:05.784520+0000","last_unstale":"2024-07-04T17:29:26.919955+0000","last_undegraded":"2024-07-04T17:29:26.919955+0000","last_fullsized":"2024-07-04T17:29:26.919955+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:31:15.745622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.897995+0000","last_change":"2024-07-04T17:29:04.794330+0000","last_active":"2024-07-04T17:29:26.897995+0000","last_peered":"2024-07-04T17:29:26.897995+0000","last_clean":"2024-07-04T17:29:26.897995+0000","last_became_active":"2024-07-04T17:29:04.793980+0000","last_became_peered":"2024-07-04T17:29:04.793980+0000","last_unstale":"2024-07-04T17:29:26.897995+0000","last_undegraded":"2024-07-04T17:29:26.897995+0000","last_fullsized":"2024-07-04T17:29:26.897995+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:06:13.018475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.902492+0000","last_change":"2024-07-04T17:29:25.919313+0000","last_active":"2024-07-04T17:29:26.902492+0000","last_peered":"2024-07-04T17:29:26.902492+0000","last_clean":"2024-07-04T17:29:26.902492+0000","last_became_active":"2024-07-04T17:29:25.911731+0000","last_became_peered":"2024-07-04T17:29:25.911731+0000","last_unstale":"2024-07-04T17:29:26.902492+0000","last_undegraded":"2024-07-04T17:29:26.902492+0000","last_fullsized":"2024-07-04T17:29:26.902492+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:25:33.258238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.916522+0000","last_change":"2024-07-04T17:29:24.906197+0000","last_active":"2024-07-04T17:29:26.916522+0000","last_peered":"2024-07-04T17:29:26.916522+0000","last_clean":"2024-07-04T17:29:26.916522+0000","last_became_active":"2024-07-04T17:29:24.905468+0000","last_became_peered":"2024-07-04T17:29:24.905468+0000","last_unstale":"2024-07-04T17:29:26.916522+0000","last_undegraded":"2024-07-04T17:29:26.916522+0000","last_fullsized":"2024-07-04T17:29:26.916522+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:35:59.662079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.904228+0000","last_change":"2024-07-04T17:29:05.802494+0000","last_active":"2024-07-04T17:29:26.904228+0000","last_peered":"2024-07-04T17:29:26.904228+0000","last_clean":"2024-07-04T17:29:26.904228+0000","last_became_active":"2024-07-04T17:29:05.795642+0000","last_became_peered":"2024-07-04T17:29:05.795642+0000","last_unstale":"2024-07-04T17:29:26.904228+0000","last_undegraded":"2024-07-04T17:29:26.904228+0000","last_fullsized":"2024-07-04T17:29:26.904228+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:19:50.013783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.904907+0000","last_change":"2024-07-04T17:29:04.774431+0000","last_active":"2024-07-04T17:29:26.904907+0000","last_peered":"2024-07-04T17:29:26.904907+0000","last_clean":"2024-07-04T17:29:26.904907+0000","last_became_active":"2024-07-04T17:29:04.774034+0000","last_became_peered":"2024-07-04T17:29:04.774034+0000","last_unstale":"2024-07-04T17:29:26.904907+0000","last_undegraded":"2024-07-04T17:29:26.904907+0000","last_fullsized":"2024-07-04T17:29:26.904907+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:10:23.616021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.916328+0000","last_change":"2024-07-04T17:29:25.905238+0000","last_active":"2024-07-04T17:29:26.916328+0000","last_peered":"2024-07-04T17:29:26.916328+0000","last_clean":"2024-07-04T17:29:26.916328+0000","last_became_active":"2024-07-04T17:29:25.904711+0000","last_became_peered":"2024-07-04T17:29:25.904711+0000","last_unstale":"2024-07-04T17:29:26.916328+0000","last_undegraded":"2024-07-04T17:29:26.916328+0000","last_fullsized":"2024-07-04T17:29:26.916328+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:26:00.911107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.886421+0000","last_change":"2024-07-04T17:29:24.896819+0000","last_active":"2024-07-04T17:29:26.886421+0000","last_peered":"2024-07-04T17:29:26.886421+0000","last_clean":"2024-07-04T17:29:26.886421+0000","last_became_active":"2024-07-04T17:29:24.896516+0000","last_became_peered":"2024-07-04T17:29:24.896516+0000","last_unstale":"2024-07-04T17:29:26.886421+0000","last_undegraded":"2024-07-04T17:29:26.886421+0000","last_fullsized":"2024-07-04T17:29:26.886421+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:54:00.256493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.920271+0000","last_change":"2024-07-04T17:29:05.784396+0000","last_active":"2024-07-04T17:29:26.920271+0000","last_peered":"2024-07-04T17:29:26.920271+0000","last_clean":"2024-07-04T17:29:26.920271+0000","last_became_active":"2024-07-04T17:29:05.783953+0000","last_became_peered":"2024-07-04T17:29:05.783953+0000","last_unstale":"2024-07-04T17:29:26.920271+0000","last_undegraded":"2024-07-04T17:29:26.920271+0000","last_fullsized":"2024-07-04T17:29:26.920271+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:53:09.757420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.894616+0000","last_change":"2024-07-04T17:29:04.795160+0000","last_active":"2024-07-04T17:29:26.894616+0000","last_peered":"2024-07-04T17:29:26.894616+0000","last_clean":"2024-07-04T17:29:26.894616+0000","last_became_active":"2024-07-04T17:29:04.793328+0000","last_became_peered":"2024-07-04T17:29:04.793328+0000","last_unstale":"2024-07-04T17:29:26.894616+0000","last_undegraded":"2024-07-04T17:29:26.894616+0000","last_fullsized":"2024-07-04T17:29:26.894616+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:56:44.314464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.886702+0000","last_change":"2024-07-04T17:29:25.937197+0000","last_active":"2024-07-04T17:29:26.886702+0000","last_peered":"2024-07-04T17:29:26.886702+0000","last_clean":"2024-07-04T17:29:26.886702+0000","last_became_active":"2024-07-04T17:29:25.936471+0000","last_became_peered":"2024-07-04T17:29:25.936471+0000","last_unstale":"2024-07-04T17:29:26.886702+0000","last_undegraded":"2024-07-04T17:29:26.886702+0000","last_fullsized":"2024-07-04T17:29:26.886702+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:32:38.852994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.894079+0000","last_change":"2024-07-04T17:29:24.897294+0000","last_active":"2024-07-04T17:29:26.894079+0000","last_peered":"2024-07-04T17:29:26.894079+0000","last_clean":"2024-07-04T17:29:26.894079+0000","last_became_active":"2024-07-04T17:29:24.896572+0000","last_became_peered":"2024-07-04T17:29:24.896572+0000","last_unstale":"2024-07-04T17:29:26.894079+0000","last_undegraded":"2024-07-04T17:29:26.894079+0000","last_fullsized":"2024-07-04T17:29:26.894079+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:24:16.866462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.902814+0000","last_change":"2024-07-04T17:29:05.796165+0000","last_active":"2024-07-04T17:29:26.902814+0000","last_peered":"2024-07-04T17:29:26.902814+0000","last_clean":"2024-07-04T17:29:26.902814+0000","last_became_active":"2024-07-04T17:29:05.795837+0000","last_became_peered":"2024-07-04T17:29:05.795837+0000","last_unstale":"2024-07-04T17:29:26.902814+0000","last_undegraded":"2024-07-04T17:29:26.902814+0000","last_fullsized":"2024-07-04T17:29:26.902814+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:36:15.241406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.904075+0000","last_change":"2024-07-04T17:29:04.790173+0000","last_active":"2024-07-04T17:29:26.904075+0000","last_peered":"2024-07-04T17:29:26.904075+0000","last_clean":"2024-07-04T17:29:26.904075+0000","last_became_active":"2024-07-04T17:29:04.789042+0000","last_became_peered":"2024-07-04T17:29:04.789042+0000","last_unstale":"2024-07-04T17:29:26.904075+0000","last_undegraded":"2024-07-04T17:29:26.904075+0000","last_fullsized":"2024-07-04T17:29:26.904075+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:06:13.018475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.887509+0000","last_change":"2024-07-04T17:29:25.930211+0000","last_active":"2024-07-04T17:29:26.887509+0000","last_peered":"2024-07-04T17:29:26.887509+0000","last_clean":"2024-07-04T17:29:26.887509+0000","last_became_active":"2024-07-04T17:29:25.927797+0000","last_became_peered":"2024-07-04T17:29:25.927797+0000","last_unstale":"2024-07-04T17:29:26.887509+0000","last_undegraded":"2024-07-04T17:29:26.887509+0000","last_fullsized":"2024-07-04T17:29:26.887509+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:31:17.186697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.904026+0000","last_change":"2024-07-04T17:29:24.898861+0000","last_active":"2024-07-04T17:29:26.904026+0000","last_peered":"2024-07-04T17:29:26.904026+0000","last_clean":"2024-07-04T17:29:26.904026+0000","last_became_active":"2024-07-04T17:29:24.898159+0000","last_became_peered":"2024-07-04T17:29:24.898159+0000","last_unstale":"2024-07-04T17:29:26.904026+0000","last_undegraded":"2024-07-04T17:29:26.904026+0000","last_fullsized":"2024-07-04T17:29:26.904026+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:03:14.973539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.917597+0000","last_change":"2024-07-04T17:29:05.793083+0000","last_active":"2024-07-04T17:29:26.917597+0000","last_peered":"2024-07-04T17:29:26.917597+0000","last_clean":"2024-07-04T17:29:26.917597+0000","last_became_active":"2024-07-04T17:29:05.790165+0000","last_became_peered":"2024-07-04T17:29:05.790165+0000","last_unstale":"2024-07-04T17:29:26.917597+0000","last_undegraded":"2024-07-04T17:29:26.917597+0000","last_fullsized":"2024-07-04T17:29:26.917597+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:20:46.692294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.903127+0000","last_change":"2024-07-04T17:29:04.770011+0000","last_active":"2024-07-04T17:29:26.903127+0000","last_peered":"2024-07-04T17:29:26.903127+0000","last_clean":"2024-07-04T17:29:26.903127+0000","last_became_active":"2024-07-04T17:29:04.769666+0000","last_became_peered":"2024-07-04T17:29:04.769666+0000","last_unstale":"2024-07-04T17:29:26.903127+0000","last_undegraded":"2024-07-04T17:29:26.903127+0000","last_fullsized":"2024-07-04T17:29:26.903127+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:21:48.040633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.903486+0000","last_change":"2024-07-04T17:29:25.921104+0000","last_active":"2024-07-04T17:29:26.903486+0000","last_peered":"2024-07-04T17:29:26.903486+0000","last_clean":"2024-07-04T17:29:26.903486+0000","last_became_active":"2024-07-04T17:29:25.920695+0000","last_became_peered":"2024-07-04T17:29:25.920695+0000","last_unstale":"2024-07-04T17:29:26.903486+0000","last_undegraded":"2024-07-04T17:29:26.903486+0000","last_fullsized":"2024-07-04T17:29:26.903486+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:55:02.296031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.904825+0000","last_change":"2024-07-04T17:29:24.898336+0000","last_active":"2024-07-04T17:29:26.904825+0000","last_peered":"2024-07-04T17:29:26.904825+0000","last_clean":"2024-07-04T17:29:26.904825+0000","last_became_active":"2024-07-04T17:29:24.897943+0000","last_became_peered":"2024-07-04T17:29:24.897943+0000","last_unstale":"2024-07-04T17:29:26.904825+0000","last_undegraded":"2024-07-04T17:29:26.904825+0000","last_fullsized":"2024-07-04T17:29:26.904825+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:15:04.231122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.893543+0000","last_change":"2024-07-04T17:29:05.816496+0000","last_active":"2024-07-04T17:29:26.893543+0000","last_peered":"2024-07-04T17:29:26.893543+0000","last_clean":"2024-07-04T17:29:26.893543+0000","last_became_active":"2024-07-04T17:29:05.816147+0000","last_became_peered":"2024-07-04T17:29:05.816147+0000","last_unstale":"2024-07-04T17:29:26.893543+0000","last_undegraded":"2024-07-04T17:29:26.893543+0000","last_fullsized":"2024-07-04T17:29:26.893543+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:33:26.836346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.917650+0000","last_change":"2024-07-04T17:29:04.770392+0000","last_active":"2024-07-04T17:29:26.917650+0000","last_peered":"2024-07-04T17:29:26.917650+0000","last_clean":"2024-07-04T17:29:26.917650+0000","last_became_active":"2024-07-04T17:29:04.758815+0000","last_became_peered":"2024-07-04T17:29:04.758815+0000","last_unstale":"2024-07-04T17:29:26.917650+0000","last_undegraded":"2024-07-04T17:29:26.917650+0000","last_fullsized":"2024-07-04T17:29:26.917650+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:35:08.872682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"35'66","reported_seq":211,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.903412+0000","last_change":"2024-07-04T17:26:04.953559+0000","last_active":"2024-07-04T17:29:26.903412+0000","last_peered":"2024-07-04T17:29:26.903412+0000","last_clean":"2024-07-04T17:29:26.903412+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:29:26.903412+0000","last_undegraded":"2024-07-04T17:29:26.903412+0000","last_fullsized":"2024-07-04T17:29:26.903412+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_clean_scrub_stamp":"2024-07-04T17:26:03.933432+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.064181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":398,"num_write":103,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":4107,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":398,"num_write":103,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738411,"num_pgs":116,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31760,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340080,"statfs":{"total":96636764160,"available":96604241920,"internally_reserved":0,"allocated":589824,"data_stored":209372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738412,"num_pgs":122,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1163264,"data_stored":796831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738410,"num_pgs":126,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32312,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1150976,"data_stored":797878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:29:30.654 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:29:30.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.987+0000 7fea4d000640 1 -- 172.21.15.42:0/586679632 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea480ba830 msgr2=0x7fea480b7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:30.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.987+0000 7fea4d000640 1 --2- 172.21.15.42:0/586679632 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea480ba830 0x7fea480b7cc0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fea380097b0 tx=0x7fea3802a530 comp rx=0 tx=0).stop 2024-07-04T17:29:30.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.987+0000 7fea4d000640 1 -- 172.21.15.42:0/586679632 shutdown_connections 2024-07-04T17:29:30.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.987+0000 7fea4d000640 1 --2- 172.21.15.42:0/586679632 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea480ba830 0x7fea480b7cc0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:30.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.987+0000 7fea4d000640 1 -- 172.21.15.42:0/586679632 >> 172.21.15.42:0/586679632 conn(0x7fea480bebc0 msgr2=0x7fea480befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:30.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.987+0000 7fea4d000640 1 -- 172.21.15.42:0/586679632 shutdown_connections 2024-07-04T17:29:30.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.987+0000 7fea4d000640 1 -- 172.21.15.42:0/586679632 wait complete. 2024-07-04T17:29:30.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.989+0000 7fea4d000640 1 Processor -- start 2024-07-04T17:29:30.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.990+0000 7fea4d000640 1 -- start start 2024-07-04T17:29:30.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.991+0000 7fea4d000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea480ba830 0x7fea480aa740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:30.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.991+0000 7fea4d000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fea480b1e30 con 0x7fea480ba830 2024-07-04T17:29:30.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.991+0000 7fea47400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea480ba830 0x7fea480aa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:30.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.991+0000 7fea47400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea480ba830 0x7fea480aa740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45354/0 (socket says 172.21.15.42:45354) 2024-07-04T17:29:30.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.991+0000 7fea47400640 1 -- 172.21.15.42:0/3306597271 learned_addr learned my addr 172.21.15.42:0/3306597271 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:30.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.992+0000 7fea47400640 1 -- 172.21.15.42:0/3306597271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea480a9970 con 0x7fea480ba830 2024-07-04T17:29:30.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.992+0000 7fea47400640 1 --2- 172.21.15.42:0/3306597271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea480ba830 0x7fea480aa740 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fea380098d0 tx=0x7fea38033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:30.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.992+0000 7fea3f400640 1 -- 172.21.15.42:0/3306597271 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea38031880 con 0x7fea480ba830 2024-07-04T17:29:30.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.992+0000 7fea3f400640 1 -- 172.21.15.42:0/3306597271 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea38031b10 con 0x7fea480ba830 2024-07-04T17:29:30.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.992+0000 7fea3f400640 1 -- 172.21.15.42:0/3306597271 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea38031df0 con 0x7fea480ba830 2024-07-04T17:29:30.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.992+0000 7fea4d000640 1 -- 172.21.15.42:0/3306597271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea481e4300 con 0x7fea480ba830 2024-07-04T17:29:30.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.992+0000 7fea4d000640 1 -- 172.21.15.42:0/3306597271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea480a96b0 con 0x7fea480ba830 2024-07-04T17:29:30.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.994+0000 7fea3f400640 1 -- 172.21.15.42:0/3306597271 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7fea380330d0 con 0x7fea480ba830 2024-07-04T17:29:30.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.994+0000 7fea4d000640 1 -- 172.21.15.42:0/3306597271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea480b7cc0 con 0x7fea480ba830 2024-07-04T17:29:30.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.994+0000 7fea3f400640 1 --2- 172.21.15.42:0/3306597271 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fea2403d4f0 0x7fea2403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:30.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.995+0000 7fea3f400640 1 -- 172.21.15.42:0/3306597271 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4303+0+0 (secure 0 0 0) 0x7fea380730b0 con 0x7fea480ba830 2024-07-04T17:29:30.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.995+0000 7fea3fe00640 1 --2- 172.21.15.42:0/3306597271 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fea2403d4f0 0x7fea2403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:30.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:30.996+0000 7fea3fe00640 1 --2- 172.21.15.42:0/3306597271 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fea2403d4f0 0x7fea2403fa00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fea30007fd0 tx=0x7fea30007920 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:31.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.000+0000 7fea3f400640 1 -- 172.21.15.42:0/3306597271 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fea3803f030 con 0x7fea480ba830 2024-07-04T17:29:31.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.207+0000 7fea4d000640 1 -- 172.21.15.42:0/3306597271 --> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fea48068e30 con 0x7fea2403d4f0 2024-07-04T17:29:31.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.227+0000 7fea3f400640 1 -- 172.21.15.42:0/3306597271 <== mgr.4613 v2:172.21.15.42:6824/2638271195 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+482337 (secure 0 0 0) 0x7fea48068e30 con 0x7fea2403d4f0 2024-07-04T17:29:31.231 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:29:31.238 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:29:31.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.237+0000 7fea4d000640 1 -- 172.21.15.42:0/3306597271 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fea2403d4f0 msgr2=0x7fea2403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:31.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.237+0000 7fea4d000640 1 --2- 172.21.15.42:0/3306597271 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fea2403d4f0 0x7fea2403fa00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fea30007fd0 tx=0x7fea30007920 comp rx=0 tx=0).stop 2024-07-04T17:29:31.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.237+0000 7fea4d000640 1 -- 172.21.15.42:0/3306597271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea480ba830 msgr2=0x7fea480aa740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:31.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.237+0000 7fea4d000640 1 --2- 172.21.15.42:0/3306597271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea480ba830 0x7fea480aa740 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fea380098d0 tx=0x7fea38033ad0 comp rx=0 tx=0).stop 2024-07-04T17:29:31.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.237+0000 7fea4d000640 1 -- 172.21.15.42:0/3306597271 shutdown_connections 2024-07-04T17:29:31.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.237+0000 7fea4d000640 1 --2- 172.21.15.42:0/3306597271 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fea2403d4f0 0x7fea2403fa00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:31.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.237+0000 7fea4d000640 1 --2- 172.21.15.42:0/3306597271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea480ba830 0x7fea480aa740 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:31.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.237+0000 7fea4d000640 1 -- 172.21.15.42:0/3306597271 >> 172.21.15.42:0/3306597271 conn(0x7fea480bebc0 msgr2=0x7fea48142f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:31.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.238+0000 7fea4d000640 1 -- 172.21.15.42:0/3306597271 shutdown_connections 2024-07-04T17:29:31.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.238+0000 7fea4d000640 1 -- 172.21.15.42:0/3306597271 wait complete. 2024-07-04T17:29:31.269 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":48,"stamp":"2024-07-04T17:29:29.660059+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596885,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":176,"num_read_kb":398,"num_write":195,"num_write_kb":1137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":364,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":96396,"kb_used_data":2836,"kb_used_omap":4,"kb_used_meta":93499,"kb_avail":283019124,"statfs":{"total":289910292480,"available":289811582976,"internally_reserved":0,"allocated":2904064,"data_stored":1804081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":95743311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.819331"},"pg_stats":[{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.883229+0000","last_change":"2024-07-04T17:29:25.937624+0000","last_active":"2024-07-04T17:29:26.883229+0000","last_peered":"2024-07-04T17:29:26.883229+0000","last_clean":"2024-07-04T17:29:26.883229+0000","last_became_active":"2024-07-04T17:29:25.937236+0000","last_became_peered":"2024-07-04T17:29:25.937236+0000","last_unstale":"2024-07-04T17:29:26.883229+0000","last_undegraded":"2024-07-04T17:29:26.883229+0000","last_fullsized":"2024-07-04T17:29:26.883229+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:35:54.403127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3f","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.893787+0000","last_change":"2024-07-04T17:29:05.832523+0000","last_active":"2024-07-04T17:29:26.893787+0000","last_peered":"2024-07-04T17:29:26.893787+0000","last_clean":"2024-07-04T17:29:26.893787+0000","last_became_active":"2024-07-04T17:29:05.831208+0000","last_became_peered":"2024-07-04T17:29:05.831208+0000","last_unstale":"2024-07-04T17:29:26.893787+0000","last_undegraded":"2024-07-04T17:29:26.893787+0000","last_fullsized":"2024-07-04T17:29:26.893787+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:52:48.071452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.904845+0000","last_change":"2024-07-04T17:29:25.921896+0000","last_active":"2024-07-04T17:29:26.904845+0000","last_peered":"2024-07-04T17:29:26.904845+0000","last_clean":"2024-07-04T17:29:26.904845+0000","last_became_active":"2024-07-04T17:29:25.921426+0000","last_became_peered":"2024-07-04T17:29:25.921426+0000","last_unstale":"2024-07-04T17:29:26.904845+0000","last_undegraded":"2024-07-04T17:29:26.904845+0000","last_fullsized":"2024-07-04T17:29:26.904845+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:28:53.439362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.894664+0000","last_change":"2024-07-04T17:29:05.826928+0000","last_active":"2024-07-04T17:29:26.894664+0000","last_peered":"2024-07-04T17:29:26.894664+0000","last_clean":"2024-07-04T17:29:26.894664+0000","last_became_active":"2024-07-04T17:29:05.826320+0000","last_became_peered":"2024-07-04T17:29:05.826320+0000","last_unstale":"2024-07-04T17:29:26.894664+0000","last_undegraded":"2024-07-04T17:29:26.894664+0000","last_fullsized":"2024-07-04T17:29:26.894664+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:06:24.983072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.911801+0000","last_change":"2024-07-04T17:29:25.942153+0000","last_active":"2024-07-04T17:29:26.911801+0000","last_peered":"2024-07-04T17:29:26.911801+0000","last_clean":"2024-07-04T17:29:26.911801+0000","last_became_active":"2024-07-04T17:29:25.941720+0000","last_became_peered":"2024-07-04T17:29:25.941720+0000","last_unstale":"2024-07-04T17:29:26.911801+0000","last_undegraded":"2024-07-04T17:29:26.911801+0000","last_fullsized":"2024-07-04T17:29:26.911801+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:03:24.301593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.884373+0000","last_change":"2024-07-04T17:29:05.825968+0000","last_active":"2024-07-04T17:29:26.884373+0000","last_peered":"2024-07-04T17:29:26.884373+0000","last_clean":"2024-07-04T17:29:26.884373+0000","last_became_active":"2024-07-04T17:29:05.825203+0000","last_became_peered":"2024-07-04T17:29:05.825203+0000","last_unstale":"2024-07-04T17:29:26.884373+0000","last_undegraded":"2024-07-04T17:29:26.884373+0000","last_fullsized":"2024-07-04T17:29:26.884373+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:19:50.013783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.919795+0000","last_change":"2024-07-04T17:29:25.909129+0000","last_active":"2024-07-04T17:29:26.919795+0000","last_peered":"2024-07-04T17:29:26.919795+0000","last_clean":"2024-07-04T17:29:26.919795+0000","last_became_active":"2024-07-04T17:29:25.908807+0000","last_became_peered":"2024-07-04T17:29:25.908807+0000","last_unstale":"2024-07-04T17:29:26.919795+0000","last_undegraded":"2024-07-04T17:29:26.919795+0000","last_fullsized":"2024-07-04T17:29:26.919795+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:46:57.481302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.921004+0000","last_change":"2024-07-04T17:29:05.783206+0000","last_active":"2024-07-04T17:29:26.921004+0000","last_peered":"2024-07-04T17:29:26.921004+0000","last_clean":"2024-07-04T17:29:26.921004+0000","last_became_active":"2024-07-04T17:29:05.782870+0000","last_became_peered":"2024-07-04T17:29:05.782870+0000","last_unstale":"2024-07-04T17:29:26.921004+0000","last_undegraded":"2024-07-04T17:29:26.921004+0000","last_fullsized":"2024-07-04T17:29:26.921004+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:25:13.149991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.903109+0000","last_change":"2024-07-04T17:29:25.921875+0000","last_active":"2024-07-04T17:29:26.903109+0000","last_peered":"2024-07-04T17:29:26.903109+0000","last_clean":"2024-07-04T17:29:26.903109+0000","last_became_active":"2024-07-04T17:29:25.921525+0000","last_became_peered":"2024-07-04T17:29:25.921525+0000","last_unstale":"2024-07-04T17:29:26.903109+0000","last_undegraded":"2024-07-04T17:29:26.903109+0000","last_fullsized":"2024-07-04T17:29:26.903109+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:24:19.242368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.895145+0000","last_change":"2024-07-04T17:29:05.810625+0000","last_active":"2024-07-04T17:29:26.895145+0000","last_peered":"2024-07-04T17:29:26.895145+0000","last_clean":"2024-07-04T17:29:26.895145+0000","last_became_active":"2024-07-04T17:29:05.810316+0000","last_became_peered":"2024-07-04T17:29:05.810316+0000","last_unstale":"2024-07-04T17:29:26.895145+0000","last_undegraded":"2024-07-04T17:29:26.895145+0000","last_fullsized":"2024-07-04T17:29:26.895145+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:48:26.486742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.915700+0000","last_change":"2024-07-04T17:29:25.905083+0000","last_active":"2024-07-04T17:29:26.915700+0000","last_peered":"2024-07-04T17:29:26.915700+0000","last_clean":"2024-07-04T17:29:26.915700+0000","last_became_active":"2024-07-04T17:29:25.904251+0000","last_became_peered":"2024-07-04T17:29:25.904251+0000","last_unstale":"2024-07-04T17:29:26.915700+0000","last_undegraded":"2024-07-04T17:29:26.915700+0000","last_fullsized":"2024-07-04T17:29:26.915700+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:49:56.055826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.917469+0000","last_change":"2024-07-04T17:29:05.799666+0000","last_active":"2024-07-04T17:29:26.917469+0000","last_peered":"2024-07-04T17:29:26.917469+0000","last_clean":"2024-07-04T17:29:26.917469+0000","last_became_active":"2024-07-04T17:29:05.788794+0000","last_became_peered":"2024-07-04T17:29:05.788794+0000","last_unstale":"2024-07-04T17:29:26.917469+0000","last_undegraded":"2024-07-04T17:29:26.917469+0000","last_fullsized":"2024-07-04T17:29:26.917469+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:33:48.215079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.898301+0000","last_change":"2024-07-04T17:29:25.937361+0000","last_active":"2024-07-04T17:29:26.898301+0000","last_peered":"2024-07-04T17:29:26.898301+0000","last_clean":"2024-07-04T17:29:26.898301+0000","last_became_active":"2024-07-04T17:29:25.935187+0000","last_became_peered":"2024-07-04T17:29:25.935187+0000","last_unstale":"2024-07-04T17:29:26.898301+0000","last_undegraded":"2024-07-04T17:29:26.898301+0000","last_fullsized":"2024-07-04T17:29:26.898301+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:57:46.547083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.916773+0000","last_change":"2024-07-04T17:29:05.803716+0000","last_active":"2024-07-04T17:29:26.916773+0000","last_peered":"2024-07-04T17:29:26.916773+0000","last_clean":"2024-07-04T17:29:26.916773+0000","last_became_active":"2024-07-04T17:29:05.803425+0000","last_became_peered":"2024-07-04T17:29:05.803425+0000","last_unstale":"2024-07-04T17:29:26.916773+0000","last_undegraded":"2024-07-04T17:29:26.916773+0000","last_fullsized":"2024-07-04T17:29:26.916773+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:51:34.763940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.882898+0000","last_change":"2024-07-04T17:29:25.936087+0000","last_active":"2024-07-04T17:29:26.882898+0000","last_peered":"2024-07-04T17:29:26.882898+0000","last_clean":"2024-07-04T17:29:26.882898+0000","last_became_active":"2024-07-04T17:29:25.925783+0000","last_became_peered":"2024-07-04T17:29:25.925783+0000","last_unstale":"2024-07-04T17:29:26.882898+0000","last_undegraded":"2024-07-04T17:29:26.882898+0000","last_fullsized":"2024-07-04T17:29:26.882898+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:21:06.800541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.886805+0000","last_change":"2024-07-04T17:29:05.802356+0000","last_active":"2024-07-04T17:29:26.886805+0000","last_peered":"2024-07-04T17:29:26.886805+0000","last_clean":"2024-07-04T17:29:26.886805+0000","last_became_active":"2024-07-04T17:29:05.798614+0000","last_became_peered":"2024-07-04T17:29:05.798614+0000","last_unstale":"2024-07-04T17:29:26.886805+0000","last_undegraded":"2024-07-04T17:29:26.886805+0000","last_fullsized":"2024-07-04T17:29:26.886805+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:33:04.621287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.919603+0000","last_change":"2024-07-04T17:29:25.924609+0000","last_active":"2024-07-04T17:29:26.919603+0000","last_peered":"2024-07-04T17:29:26.919603+0000","last_clean":"2024-07-04T17:29:26.919603+0000","last_became_active":"2024-07-04T17:29:25.923063+0000","last_became_peered":"2024-07-04T17:29:25.923063+0000","last_unstale":"2024-07-04T17:29:26.919603+0000","last_undegraded":"2024-07-04T17:29:26.919603+0000","last_fullsized":"2024-07-04T17:29:26.919603+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:43:24.491322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.910054+0000","last_change":"2024-07-04T17:29:05.804142+0000","last_active":"2024-07-04T17:29:26.910054+0000","last_peered":"2024-07-04T17:29:26.910054+0000","last_clean":"2024-07-04T17:29:26.910054+0000","last_became_active":"2024-07-04T17:29:05.799194+0000","last_became_peered":"2024-07-04T17:29:05.799194+0000","last_unstale":"2024-07-04T17:29:26.910054+0000","last_undegraded":"2024-07-04T17:29:26.910054+0000","last_fullsized":"2024-07-04T17:29:26.910054+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:44:51.271924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.905047+0000","last_change":"2024-07-04T17:29:25.921615+0000","last_active":"2024-07-04T17:29:26.905047+0000","last_peered":"2024-07-04T17:29:26.905047+0000","last_clean":"2024-07-04T17:29:26.905047+0000","last_became_active":"2024-07-04T17:29:25.919321+0000","last_became_peered":"2024-07-04T17:29:25.919321+0000","last_unstale":"2024-07-04T17:29:26.905047+0000","last_undegraded":"2024-07-04T17:29:26.905047+0000","last_fullsized":"2024-07-04T17:29:26.905047+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:17:55.141994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.909657+0000","last_change":"2024-07-04T17:29:05.815840+0000","last_active":"2024-07-04T17:29:26.909657+0000","last_peered":"2024-07-04T17:29:26.909657+0000","last_clean":"2024-07-04T17:29:26.909657+0000","last_became_active":"2024-07-04T17:29:05.810912+0000","last_became_peered":"2024-07-04T17:29:05.810912+0000","last_unstale":"2024-07-04T17:29:26.909657+0000","last_undegraded":"2024-07-04T17:29:26.909657+0000","last_fullsized":"2024-07-04T17:29:26.909657+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:31:51.703656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.897909+0000","last_change":"2024-07-04T17:29:25.934566+0000","last_active":"2024-07-04T17:29:26.897909+0000","last_peered":"2024-07-04T17:29:26.897909+0000","last_clean":"2024-07-04T17:29:26.897909+0000","last_became_active":"2024-07-04T17:29:25.931262+0000","last_became_peered":"2024-07-04T17:29:25.931262+0000","last_unstale":"2024-07-04T17:29:26.897909+0000","last_undegraded":"2024-07-04T17:29:26.897909+0000","last_fullsized":"2024-07-04T17:29:26.897909+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:15:54.308742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.908137+0000","last_change":"2024-07-04T17:29:05.785152+0000","last_active":"2024-07-04T17:29:26.908137+0000","last_peered":"2024-07-04T17:29:26.908137+0000","last_clean":"2024-07-04T17:29:26.908137+0000","last_became_active":"2024-07-04T17:29:05.784449+0000","last_became_peered":"2024-07-04T17:29:05.784449+0000","last_unstale":"2024-07-04T17:29:26.908137+0000","last_undegraded":"2024-07-04T17:29:26.908137+0000","last_fullsized":"2024-07-04T17:29:26.908137+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:35:30.393302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.908031+0000","last_change":"2024-07-04T17:29:25.940924+0000","last_active":"2024-07-04T17:29:26.908031+0000","last_peered":"2024-07-04T17:29:26.908031+0000","last_clean":"2024-07-04T17:29:26.908031+0000","last_became_active":"2024-07-04T17:29:25.940610+0000","last_became_peered":"2024-07-04T17:29:25.940610+0000","last_unstale":"2024-07-04T17:29:26.908031+0000","last_undegraded":"2024-07-04T17:29:26.908031+0000","last_fullsized":"2024-07-04T17:29:26.908031+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:33:52.833086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.905671+0000","last_change":"2024-07-04T17:29:05.823239+0000","last_active":"2024-07-04T17:29:26.905671+0000","last_peered":"2024-07-04T17:29:26.905671+0000","last_clean":"2024-07-04T17:29:26.905671+0000","last_became_active":"2024-07-04T17:29:05.814164+0000","last_became_peered":"2024-07-04T17:29:05.814164+0000","last_unstale":"2024-07-04T17:29:26.905671+0000","last_undegraded":"2024-07-04T17:29:26.905671+0000","last_fullsized":"2024-07-04T17:29:26.905671+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:03:20.072994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.917567+0000","last_change":"2024-07-04T17:29:25.923597+0000","last_active":"2024-07-04T17:29:26.917567+0000","last_peered":"2024-07-04T17:29:26.917567+0000","last_clean":"2024-07-04T17:29:26.917567+0000","last_became_active":"2024-07-04T17:29:25.922667+0000","last_became_peered":"2024-07-04T17:29:25.922667+0000","last_unstale":"2024-07-04T17:29:26.917567+0000","last_undegraded":"2024-07-04T17:29:26.917567+0000","last_fullsized":"2024-07-04T17:29:26.917567+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:17:13.242558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.918965+0000","last_change":"2024-07-04T17:29:05.796770+0000","last_active":"2024-07-04T17:29:26.918965+0000","last_peered":"2024-07-04T17:29:26.918965+0000","last_clean":"2024-07-04T17:29:26.918965+0000","last_became_active":"2024-07-04T17:29:05.789644+0000","last_became_peered":"2024-07-04T17:29:05.789644+0000","last_unstale":"2024-07-04T17:29:26.918965+0000","last_undegraded":"2024-07-04T17:29:26.918965+0000","last_fullsized":"2024-07-04T17:29:26.918965+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:58:03.269316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.895202+0000","last_change":"2024-07-04T17:29:25.934419+0000","last_active":"2024-07-04T17:29:26.895202+0000","last_peered":"2024-07-04T17:29:26.895202+0000","last_clean":"2024-07-04T17:29:26.895202+0000","last_became_active":"2024-07-04T17:29:25.932343+0000","last_became_peered":"2024-07-04T17:29:25.932343+0000","last_unstale":"2024-07-04T17:29:26.895202+0000","last_undegraded":"2024-07-04T17:29:26.895202+0000","last_fullsized":"2024-07-04T17:29:26.895202+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:56:51.847871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.888310+0000","last_change":"2024-07-04T17:29:05.808774+0000","last_active":"2024-07-04T17:29:26.888310+0000","last_peered":"2024-07-04T17:29:26.888310+0000","last_clean":"2024-07-04T17:29:26.888310+0000","last_became_active":"2024-07-04T17:29:05.808459+0000","last_became_peered":"2024-07-04T17:29:05.808459+0000","last_unstale":"2024-07-04T17:29:26.888310+0000","last_undegraded":"2024-07-04T17:29:26.888310+0000","last_fullsized":"2024-07-04T17:29:26.888310+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:13:24.537766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.918273+0000","last_change":"2024-07-04T17:29:25.906262+0000","last_active":"2024-07-04T17:29:26.918273+0000","last_peered":"2024-07-04T17:29:26.918273+0000","last_clean":"2024-07-04T17:29:26.918273+0000","last_became_active":"2024-07-04T17:29:25.904574+0000","last_became_peered":"2024-07-04T17:29:25.904574+0000","last_unstale":"2024-07-04T17:29:26.918273+0000","last_undegraded":"2024-07-04T17:29:26.918273+0000","last_fullsized":"2024-07-04T17:29:26.918273+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:55:05.827423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.917160+0000","last_change":"2024-07-04T17:29:05.800412+0000","last_active":"2024-07-04T17:29:26.917160+0000","last_peered":"2024-07-04T17:29:26.917160+0000","last_clean":"2024-07-04T17:29:26.917160+0000","last_became_active":"2024-07-04T17:29:05.799152+0000","last_became_peered":"2024-07-04T17:29:05.799152+0000","last_unstale":"2024-07-04T17:29:26.917160+0000","last_undegraded":"2024-07-04T17:29:26.917160+0000","last_fullsized":"2024-07-04T17:29:26.917160+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:48:07.191680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.883257+0000","last_change":"2024-07-04T17:29:25.944657+0000","last_active":"2024-07-04T17:29:26.883257+0000","last_peered":"2024-07-04T17:29:26.883257+0000","last_clean":"2024-07-04T17:29:26.883257+0000","last_became_active":"2024-07-04T17:29:25.943605+0000","last_became_peered":"2024-07-04T17:29:25.943605+0000","last_unstale":"2024-07-04T17:29:26.883257+0000","last_undegraded":"2024-07-04T17:29:26.883257+0000","last_fullsized":"2024-07-04T17:29:26.883257+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:05:18.667710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.917790+0000","last_change":"2024-07-04T17:29:05.795773+0000","last_active":"2024-07-04T17:29:26.917790+0000","last_peered":"2024-07-04T17:29:26.917790+0000","last_clean":"2024-07-04T17:29:26.917790+0000","last_became_active":"2024-07-04T17:29:05.795470+0000","last_became_peered":"2024-07-04T17:29:05.795470+0000","last_unstale":"2024-07-04T17:29:26.917790+0000","last_undegraded":"2024-07-04T17:29:26.917790+0000","last_fullsized":"2024-07-04T17:29:26.917790+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:32:34.271916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.883524+0000","last_change":"2024-07-04T17:29:25.947789+0000","last_active":"2024-07-04T17:29:26.883524+0000","last_peered":"2024-07-04T17:29:26.883524+0000","last_clean":"2024-07-04T17:29:26.883524+0000","last_became_active":"2024-07-04T17:29:25.943347+0000","last_became_peered":"2024-07-04T17:29:25.943347+0000","last_unstale":"2024-07-04T17:29:26.883524+0000","last_undegraded":"2024-07-04T17:29:26.883524+0000","last_fullsized":"2024-07-04T17:29:26.883524+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:03:24.301593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.893671+0000","last_change":"2024-07-04T17:29:05.820276+0000","last_active":"2024-07-04T17:29:26.893671+0000","last_peered":"2024-07-04T17:29:26.893671+0000","last_clean":"2024-07-04T17:29:26.893671+0000","last_became_active":"2024-07-04T17:29:05.819959+0000","last_became_peered":"2024-07-04T17:29:05.819959+0000","last_unstale":"2024-07-04T17:29:26.893671+0000","last_undegraded":"2024-07-04T17:29:26.893671+0000","last_fullsized":"2024-07-04T17:29:26.893671+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:50:43.333430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.918755+0000","last_change":"2024-07-04T17:29:25.924876+0000","last_active":"2024-07-04T17:29:26.918755+0000","last_peered":"2024-07-04T17:29:26.918755+0000","last_clean":"2024-07-04T17:29:26.918755+0000","last_became_active":"2024-07-04T17:29:25.923540+0000","last_became_peered":"2024-07-04T17:29:25.923540+0000","last_unstale":"2024-07-04T17:29:26.918755+0000","last_undegraded":"2024-07-04T17:29:26.918755+0000","last_fullsized":"2024-07-04T17:29:26.918755+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:05:14.499220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.894950+0000","last_change":"2024-07-04T17:29:05.817114+0000","last_active":"2024-07-04T17:29:26.894950+0000","last_peered":"2024-07-04T17:29:26.894950+0000","last_clean":"2024-07-04T17:29:26.894950+0000","last_became_active":"2024-07-04T17:29:05.816724+0000","last_became_peered":"2024-07-04T17:29:05.816724+0000","last_unstale":"2024-07-04T17:29:26.894950+0000","last_undegraded":"2024-07-04T17:29:26.894950+0000","last_fullsized":"2024-07-04T17:29:26.894950+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:44:51.271924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.895424+0000","last_change":"2024-07-04T17:29:25.948453+0000","last_active":"2024-07-04T17:29:26.895424+0000","last_peered":"2024-07-04T17:29:26.895424+0000","last_clean":"2024-07-04T17:29:26.895424+0000","last_became_active":"2024-07-04T17:29:25.943851+0000","last_became_peered":"2024-07-04T17:29:25.943851+0000","last_unstale":"2024-07-04T17:29:26.895424+0000","last_undegraded":"2024-07-04T17:29:26.895424+0000","last_fullsized":"2024-07-04T17:29:26.895424+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:32:09.090128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.884041+0000","last_change":"2024-07-04T17:29:05.816646+0000","last_active":"2024-07-04T17:29:26.884041+0000","last_peered":"2024-07-04T17:29:26.884041+0000","last_clean":"2024-07-04T17:29:26.884041+0000","last_became_active":"2024-07-04T17:29:05.816345+0000","last_became_peered":"2024-07-04T17:29:05.816345+0000","last_unstale":"2024-07-04T17:29:26.884041+0000","last_undegraded":"2024-07-04T17:29:26.884041+0000","last_fullsized":"2024-07-04T17:29:26.884041+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:31:51.703656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.908207+0000","last_change":"2024-07-04T17:29:25.943057+0000","last_active":"2024-07-04T17:29:26.908207+0000","last_peered":"2024-07-04T17:29:26.908207+0000","last_clean":"2024-07-04T17:29:26.908207+0000","last_became_active":"2024-07-04T17:29:25.938914+0000","last_became_peered":"2024-07-04T17:29:25.938914+0000","last_unstale":"2024-07-04T17:29:26.908207+0000","last_undegraded":"2024-07-04T17:29:26.908207+0000","last_fullsized":"2024-07-04T17:29:26.908207+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:09:11.948958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.905880+0000","last_change":"2024-07-04T17:29:05.792444+0000","last_active":"2024-07-04T17:29:26.905880+0000","last_peered":"2024-07-04T17:29:26.905880+0000","last_clean":"2024-07-04T17:29:26.905880+0000","last_became_active":"2024-07-04T17:29:05.792058+0000","last_became_peered":"2024-07-04T17:29:05.792058+0000","last_unstale":"2024-07-04T17:29:26.905880+0000","last_undegraded":"2024-07-04T17:29:26.905880+0000","last_fullsized":"2024-07-04T17:29:26.905880+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:13:24.537766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.903377+0000","last_change":"2024-07-04T17:29:25.909296+0000","last_active":"2024-07-04T17:29:26.903377+0000","last_peered":"2024-07-04T17:29:26.903377+0000","last_clean":"2024-07-04T17:29:26.903377+0000","last_became_active":"2024-07-04T17:29:25.908940+0000","last_became_peered":"2024-07-04T17:29:25.908940+0000","last_unstale":"2024-07-04T17:29:26.903377+0000","last_undegraded":"2024-07-04T17:29:26.903377+0000","last_fullsized":"2024-07-04T17:29:26.903377+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:48:27.299927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.919341+0000","last_change":"2024-07-04T17:29:05.799399+0000","last_active":"2024-07-04T17:29:26.919341+0000","last_peered":"2024-07-04T17:29:26.919341+0000","last_clean":"2024-07-04T17:29:26.919341+0000","last_became_active":"2024-07-04T17:29:05.789221+0000","last_became_peered":"2024-07-04T17:29:05.789221+0000","last_unstale":"2024-07-04T17:29:26.919341+0000","last_undegraded":"2024-07-04T17:29:26.919341+0000","last_fullsized":"2024-07-04T17:29:26.919341+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:23:53.530646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.915961+0000","last_change":"2024-07-04T17:29:25.922918+0000","last_active":"2024-07-04T17:29:26.915961+0000","last_peered":"2024-07-04T17:29:26.915961+0000","last_clean":"2024-07-04T17:29:26.915961+0000","last_became_active":"2024-07-04T17:29:25.921933+0000","last_became_peered":"2024-07-04T17:29:25.921933+0000","last_unstale":"2024-07-04T17:29:26.915961+0000","last_undegraded":"2024-07-04T17:29:26.915961+0000","last_fullsized":"2024-07-04T17:29:26.915961+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:39:31.441441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.888443+0000","last_change":"2024-07-04T17:29:05.813751+0000","last_active":"2024-07-04T17:29:26.888443+0000","last_peered":"2024-07-04T17:29:26.888443+0000","last_clean":"2024-07-04T17:29:26.888443+0000","last_became_active":"2024-07-04T17:29:05.813362+0000","last_became_peered":"2024-07-04T17:29:05.813362+0000","last_unstale":"2024-07-04T17:29:26.888443+0000","last_undegraded":"2024-07-04T17:29:26.888443+0000","last_fullsized":"2024-07-04T17:29:26.888443+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:58:35.184348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.921689+0000","last_change":"2024-07-04T17:29:25.920481+0000","last_active":"2024-07-04T17:29:26.921689+0000","last_peered":"2024-07-04T17:29:26.921689+0000","last_clean":"2024-07-04T17:29:26.921689+0000","last_became_active":"2024-07-04T17:29:25.920127+0000","last_became_peered":"2024-07-04T17:29:25.920127+0000","last_unstale":"2024-07-04T17:29:26.921689+0000","last_undegraded":"2024-07-04T17:29:26.921689+0000","last_fullsized":"2024-07-04T17:29:26.921689+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:59:24.190086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.903927+0000","last_change":"2024-07-04T17:29:05.809788+0000","last_active":"2024-07-04T17:29:26.903927+0000","last_peered":"2024-07-04T17:29:26.903927+0000","last_clean":"2024-07-04T17:29:26.903927+0000","last_became_active":"2024-07-04T17:29:05.809201+0000","last_became_peered":"2024-07-04T17:29:05.809201+0000","last_unstale":"2024-07-04T17:29:26.903927+0000","last_undegraded":"2024-07-04T17:29:26.903927+0000","last_fullsized":"2024-07-04T17:29:26.903927+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:48:26.486742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.884025+0000","last_change":"2024-07-04T17:29:25.944666+0000","last_active":"2024-07-04T17:29:26.884025+0000","last_peered":"2024-07-04T17:29:26.884025+0000","last_clean":"2024-07-04T17:29:26.884025+0000","last_became_active":"2024-07-04T17:29:25.943104+0000","last_became_peered":"2024-07-04T17:29:25.943104+0000","last_unstale":"2024-07-04T17:29:26.884025+0000","last_undegraded":"2024-07-04T17:29:26.884025+0000","last_fullsized":"2024-07-04T17:29:26.884025+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:39:38.751729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.902694+0000","last_change":"2024-07-04T17:29:05.820075+0000","last_active":"2024-07-04T17:29:26.902694+0000","last_peered":"2024-07-04T17:29:26.902694+0000","last_clean":"2024-07-04T17:29:26.902694+0000","last_became_active":"2024-07-04T17:29:05.819738+0000","last_became_peered":"2024-07-04T17:29:05.819738+0000","last_unstale":"2024-07-04T17:29:26.902694+0000","last_undegraded":"2024-07-04T17:29:26.902694+0000","last_fullsized":"2024-07-04T17:29:26.902694+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:47:41.453377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.918687+0000","last_change":"2024-07-04T17:29:25.905372+0000","last_active":"2024-07-04T17:29:26.918687+0000","last_peered":"2024-07-04T17:29:26.918687+0000","last_clean":"2024-07-04T17:29:26.918687+0000","last_became_active":"2024-07-04T17:29:25.904919+0000","last_became_peered":"2024-07-04T17:29:25.904919+0000","last_unstale":"2024-07-04T17:29:26.918687+0000","last_undegraded":"2024-07-04T17:29:26.918687+0000","last_fullsized":"2024-07-04T17:29:26.918687+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:21:49.074154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.888029+0000","last_change":"2024-07-04T17:29:05.832254+0000","last_active":"2024-07-04T17:29:26.888029+0000","last_peered":"2024-07-04T17:29:26.888029+0000","last_clean":"2024-07-04T17:29:26.888029+0000","last_became_active":"2024-07-04T17:29:05.830771+0000","last_became_peered":"2024-07-04T17:29:05.830771+0000","last_unstale":"2024-07-04T17:29:26.888029+0000","last_undegraded":"2024-07-04T17:29:26.888029+0000","last_fullsized":"2024-07-04T17:29:26.888029+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:01:45.227217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.918914+0000","last_change":"2024-07-04T17:29:25.930828+0000","last_active":"2024-07-04T17:29:26.918914+0000","last_peered":"2024-07-04T17:29:26.918914+0000","last_clean":"2024-07-04T17:29:26.918914+0000","last_became_active":"2024-07-04T17:29:25.930440+0000","last_became_peered":"2024-07-04T17:29:25.930440+0000","last_unstale":"2024-07-04T17:29:26.918914+0000","last_undegraded":"2024-07-04T17:29:26.918914+0000","last_fullsized":"2024-07-04T17:29:26.918914+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:47:01.162367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.921691+0000","last_change":"2024-07-04T17:29:05.787671+0000","last_active":"2024-07-04T17:29:26.921691+0000","last_peered":"2024-07-04T17:29:26.921691+0000","last_clean":"2024-07-04T17:29:26.921691+0000","last_became_active":"2024-07-04T17:29:05.783612+0000","last_became_peered":"2024-07-04T17:29:05.783612+0000","last_unstale":"2024-07-04T17:29:26.921691+0000","last_undegraded":"2024-07-04T17:29:26.921691+0000","last_fullsized":"2024-07-04T17:29:26.921691+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:17:31.565173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.912000+0000","last_change":"2024-07-04T17:29:25.911967+0000","last_active":"2024-07-04T17:29:26.912000+0000","last_peered":"2024-07-04T17:29:26.912000+0000","last_clean":"2024-07-04T17:29:26.912000+0000","last_became_active":"2024-07-04T17:29:25.909919+0000","last_became_peered":"2024-07-04T17:29:25.909919+0000","last_unstale":"2024-07-04T17:29:26.912000+0000","last_undegraded":"2024-07-04T17:29:26.912000+0000","last_fullsized":"2024-07-04T17:29:26.912000+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:39:38.751729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.919224+0000","last_change":"2024-07-04T17:29:05.785801+0000","last_active":"2024-07-04T17:29:26.919224+0000","last_peered":"2024-07-04T17:29:26.919224+0000","last_clean":"2024-07-04T17:29:26.919224+0000","last_became_active":"2024-07-04T17:29:05.784924+0000","last_became_peered":"2024-07-04T17:29:05.784924+0000","last_unstale":"2024-07-04T17:29:26.919224+0000","last_undegraded":"2024-07-04T17:29:26.919224+0000","last_fullsized":"2024-07-04T17:29:26.919224+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:52:10.436668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.908358+0000","last_change":"2024-07-04T17:29:25.920858+0000","last_active":"2024-07-04T17:29:26.908358+0000","last_peered":"2024-07-04T17:29:26.908358+0000","last_clean":"2024-07-04T17:29:26.908358+0000","last_became_active":"2024-07-04T17:29:25.920505+0000","last_became_peered":"2024-07-04T17:29:25.920505+0000","last_unstale":"2024-07-04T17:29:26.908358+0000","last_undegraded":"2024-07-04T17:29:26.908358+0000","last_fullsized":"2024-07-04T17:29:26.908358+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:01:26.547719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.894628+0000","last_change":"2024-07-04T17:29:05.817096+0000","last_active":"2024-07-04T17:29:26.894628+0000","last_peered":"2024-07-04T17:29:26.894628+0000","last_clean":"2024-07-04T17:29:26.894628+0000","last_became_active":"2024-07-04T17:29:05.816304+0000","last_became_peered":"2024-07-04T17:29:05.816304+0000","last_unstale":"2024-07-04T17:29:26.894628+0000","last_undegraded":"2024-07-04T17:29:26.894628+0000","last_fullsized":"2024-07-04T17:29:26.894628+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:44:09.505235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.915977+0000","last_change":"2024-07-04T17:29:25.925124+0000","last_active":"2024-07-04T17:29:26.915977+0000","last_peered":"2024-07-04T17:29:26.915977+0000","last_clean":"2024-07-04T17:29:26.915977+0000","last_became_active":"2024-07-04T17:29:25.923082+0000","last_became_peered":"2024-07-04T17:29:25.923082+0000","last_unstale":"2024-07-04T17:29:26.915977+0000","last_undegraded":"2024-07-04T17:29:26.915977+0000","last_fullsized":"2024-07-04T17:29:26.915977+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:58:53.811042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.895642+0000","last_change":"2024-07-04T17:29:05.825187+0000","last_active":"2024-07-04T17:29:26.895642+0000","last_peered":"2024-07-04T17:29:26.895642+0000","last_clean":"2024-07-04T17:29:26.895642+0000","last_became_active":"2024-07-04T17:29:05.824857+0000","last_became_peered":"2024-07-04T17:29:05.824857+0000","last_unstale":"2024-07-04T17:29:26.895642+0000","last_undegraded":"2024-07-04T17:29:26.895642+0000","last_fullsized":"2024-07-04T17:29:26.895642+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:14:24.515173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.916203+0000","last_change":"2024-07-04T17:29:25.919972+0000","last_active":"2024-07-04T17:29:26.916203+0000","last_peered":"2024-07-04T17:29:26.916203+0000","last_clean":"2024-07-04T17:29:26.916203+0000","last_became_active":"2024-07-04T17:29:25.919635+0000","last_became_peered":"2024-07-04T17:29:25.919635+0000","last_unstale":"2024-07-04T17:29:26.916203+0000","last_undegraded":"2024-07-04T17:29:26.916203+0000","last_fullsized":"2024-07-04T17:29:26.916203+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:39:45.543393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.903863+0000","last_change":"2024-07-04T17:29:05.815028+0000","last_active":"2024-07-04T17:29:26.903863+0000","last_peered":"2024-07-04T17:29:26.903863+0000","last_clean":"2024-07-04T17:29:26.903863+0000","last_became_active":"2024-07-04T17:29:05.814094+0000","last_became_peered":"2024-07-04T17:29:05.814094+0000","last_unstale":"2024-07-04T17:29:26.903863+0000","last_undegraded":"2024-07-04T17:29:26.903863+0000","last_fullsized":"2024-07-04T17:29:26.903863+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:48:33.874991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.905443+0000","last_change":"2024-07-04T17:29:25.928433+0000","last_active":"2024-07-04T17:29:26.905443+0000","last_peered":"2024-07-04T17:29:26.905443+0000","last_clean":"2024-07-04T17:29:26.905443+0000","last_became_active":"2024-07-04T17:29:25.928016+0000","last_became_peered":"2024-07-04T17:29:25.928016+0000","last_unstale":"2024-07-04T17:29:26.905443+0000","last_undegraded":"2024-07-04T17:29:26.905443+0000","last_fullsized":"2024-07-04T17:29:26.905443+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:53:29.865667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.883574+0000","last_change":"2024-07-04T17:29:05.813654+0000","last_active":"2024-07-04T17:29:26.883574+0000","last_peered":"2024-07-04T17:29:26.883574+0000","last_clean":"2024-07-04T17:29:26.883574+0000","last_became_active":"2024-07-04T17:29:05.813139+0000","last_became_peered":"2024-07-04T17:29:05.813139+0000","last_unstale":"2024-07-04T17:29:26.883574+0000","last_undegraded":"2024-07-04T17:29:26.883574+0000","last_fullsized":"2024-07-04T17:29:26.883574+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:36:15.241406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.903686+0000","last_change":"2024-07-04T17:29:25.900786+0000","last_active":"2024-07-04T17:29:26.903686+0000","last_peered":"2024-07-04T17:29:26.903686+0000","last_clean":"2024-07-04T17:29:26.903686+0000","last_became_active":"2024-07-04T17:29:25.900420+0000","last_became_peered":"2024-07-04T17:29:25.900420+0000","last_unstale":"2024-07-04T17:29:26.903686+0000","last_undegraded":"2024-07-04T17:29:26.903686+0000","last_fullsized":"2024-07-04T17:29:26.903686+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:36:21.965421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.917257+0000","last_change":"2024-07-04T17:29:05.799841+0000","last_active":"2024-07-04T17:29:26.917257+0000","last_peered":"2024-07-04T17:29:26.917257+0000","last_clean":"2024-07-04T17:29:26.917257+0000","last_became_active":"2024-07-04T17:29:05.799478+0000","last_became_peered":"2024-07-04T17:29:05.799478+0000","last_unstale":"2024-07-04T17:29:26.917257+0000","last_undegraded":"2024-07-04T17:29:26.917257+0000","last_fullsized":"2024-07-04T17:29:26.917257+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:25:37.804442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.887738+0000","last_change":"2024-07-04T17:29:25.929707+0000","last_active":"2024-07-04T17:29:26.887738+0000","last_peered":"2024-07-04T17:29:26.887738+0000","last_clean":"2024-07-04T17:29:26.887738+0000","last_became_active":"2024-07-04T17:29:25.926712+0000","last_became_peered":"2024-07-04T17:29:25.926712+0000","last_unstale":"2024-07-04T17:29:26.887738+0000","last_undegraded":"2024-07-04T17:29:26.887738+0000","last_fullsized":"2024-07-04T17:29:26.887738+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:00:32.585006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.884303+0000","last_change":"2024-07-04T17:29:24.901349+0000","last_active":"2024-07-04T17:29:26.884303+0000","last_peered":"2024-07-04T17:29:26.884303+0000","last_clean":"2024-07-04T17:29:26.884303+0000","last_became_active":"2024-07-04T17:29:24.900750+0000","last_became_peered":"2024-07-04T17:29:24.900750+0000","last_unstale":"2024-07-04T17:29:26.884303+0000","last_undegraded":"2024-07-04T17:29:26.884303+0000","last_fullsized":"2024-07-04T17:29:26.884303+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:01:47.208317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.921816+0000","last_change":"2024-07-04T17:29:05.795312+0000","last_active":"2024-07-04T17:29:26.921816+0000","last_peered":"2024-07-04T17:29:26.921816+0000","last_clean":"2024-07-04T17:29:26.921816+0000","last_became_active":"2024-07-04T17:29:05.794446+0000","last_became_peered":"2024-07-04T17:29:05.794446+0000","last_unstale":"2024-07-04T17:29:26.921816+0000","last_undegraded":"2024-07-04T17:29:26.921816+0000","last_fullsized":"2024-07-04T17:29:26.921816+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:54:42.187784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"49'4","reported_seq":26,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.920595+0000","last_change":"2024-07-04T17:29:04.768776+0000","last_active":"2024-07-04T17:29:26.920595+0000","last_peered":"2024-07-04T17:29:26.920595+0000","last_clean":"2024-07-04T17:29:26.920595+0000","last_became_active":"2024-07-04T17:29:04.766498+0000","last_became_peered":"2024-07-04T17:29:04.766498+0000","last_unstale":"2024-07-04T17:29:26.920595+0000","last_undegraded":"2024-07-04T17:29:26.920595+0000","last_fullsized":"2024-07-04T17:29:26.920595+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:03:44.115014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.907640+0000","last_change":"2024-07-04T17:29:25.912673+0000","last_active":"2024-07-04T17:29:26.907640+0000","last_peered":"2024-07-04T17:29:26.907640+0000","last_clean":"2024-07-04T17:29:26.907640+0000","last_became_active":"2024-07-04T17:29:25.912357+0000","last_became_peered":"2024-07-04T17:29:25.912357+0000","last_unstale":"2024-07-04T17:29:26.907640+0000","last_undegraded":"2024-07-04T17:29:26.907640+0000","last_fullsized":"2024-07-04T17:29:26.907640+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:36:48.570829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.887107+0000","last_change":"2024-07-04T17:29:24.900405+0000","last_active":"2024-07-04T17:29:26.887107+0000","last_peered":"2024-07-04T17:29:26.887107+0000","last_clean":"2024-07-04T17:29:26.887107+0000","last_became_active":"2024-07-04T17:29:24.899947+0000","last_became_peered":"2024-07-04T17:29:24.899947+0000","last_unstale":"2024-07-04T17:29:26.887107+0000","last_undegraded":"2024-07-04T17:29:26.887107+0000","last_fullsized":"2024-07-04T17:29:26.887107+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:09:43.577366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.910205+0000","last_change":"2024-07-04T17:29:05.790122+0000","last_active":"2024-07-04T17:29:26.910205+0000","last_peered":"2024-07-04T17:29:26.910205+0000","last_clean":"2024-07-04T17:29:26.910205+0000","last_became_active":"2024-07-04T17:29:05.789804+0000","last_became_peered":"2024-07-04T17:29:05.789804+0000","last_unstale":"2024-07-04T17:29:26.910205+0000","last_undegraded":"2024-07-04T17:29:26.910205+0000","last_fullsized":"2024-07-04T17:29:26.910205+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:06:24.983072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"49'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.909787+0000","last_change":"2024-07-04T17:29:04.779844+0000","last_active":"2024-07-04T17:29:26.909787+0000","last_peered":"2024-07-04T17:29:26.909787+0000","last_clean":"2024-07-04T17:29:26.909787+0000","last_became_active":"2024-07-04T17:29:04.779212+0000","last_became_peered":"2024-07-04T17:29:04.779212+0000","last_unstale":"2024-07-04T17:29:26.909787+0000","last_undegraded":"2024-07-04T17:29:26.909787+0000","last_fullsized":"2024-07-04T17:29:26.909787+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:06:45.499488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.894213+0000","last_change":"2024-07-04T17:29:25.935198+0000","last_active":"2024-07-04T17:29:26.894213+0000","last_peered":"2024-07-04T17:29:26.894213+0000","last_clean":"2024-07-04T17:29:26.894213+0000","last_became_active":"2024-07-04T17:29:25.931181+0000","last_became_peered":"2024-07-04T17:29:25.931181+0000","last_unstale":"2024-07-04T17:29:26.894213+0000","last_undegraded":"2024-07-04T17:29:26.894213+0000","last_fullsized":"2024-07-04T17:29:26.894213+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:08:25.013508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1f","version":"53'4","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.894113+0000","last_change":"2024-07-04T17:29:24.900038+0000","last_active":"2024-07-04T17:29:26.894113+0000","last_peered":"2024-07-04T17:29:26.894113+0000","last_clean":"2024-07-04T17:29:26.894113+0000","last_became_active":"2024-07-04T17:29:24.899634+0000","last_became_peered":"2024-07-04T17:29:24.899634+0000","last_unstale":"2024-07-04T17:29:26.894113+0000","last_undegraded":"2024-07-04T17:29:26.894113+0000","last_fullsized":"2024-07-04T17:29:26.894113+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:31:21.584062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.921100+0000","last_change":"2024-07-04T17:29:05.799208+0000","last_active":"2024-07-04T17:29:26.921100+0000","last_peered":"2024-07-04T17:29:26.921100+0000","last_clean":"2024-07-04T17:29:26.921100+0000","last_became_active":"2024-07-04T17:29:05.796533+0000","last_became_peered":"2024-07-04T17:29:05.796533+0000","last_unstale":"2024-07-04T17:29:26.921100+0000","last_undegraded":"2024-07-04T17:29:26.921100+0000","last_fullsized":"2024-07-04T17:29:26.921100+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:28:33.331115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.920078+0000","last_change":"2024-07-04T17:29:04.767288+0000","last_active":"2024-07-04T17:29:26.920078+0000","last_peered":"2024-07-04T17:29:26.920078+0000","last_clean":"2024-07-04T17:29:26.920078+0000","last_became_active":"2024-07-04T17:29:04.765876+0000","last_became_peered":"2024-07-04T17:29:04.765876+0000","last_unstale":"2024-07-04T17:29:26.920078+0000","last_undegraded":"2024-07-04T17:29:26.920078+0000","last_fullsized":"2024-07-04T17:29:26.920078+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:54:49.684096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.888275+0000","last_change":"2024-07-04T17:29:25.950238+0000","last_active":"2024-07-04T17:29:26.888275+0000","last_peered":"2024-07-04T17:29:26.888275+0000","last_clean":"2024-07-04T17:29:26.888275+0000","last_became_active":"2024-07-04T17:29:25.947467+0000","last_became_peered":"2024-07-04T17:29:25.947467+0000","last_unstale":"2024-07-04T17:29:26.888275+0000","last_undegraded":"2024-07-04T17:29:26.888275+0000","last_fullsized":"2024-07-04T17:29:26.888275+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:29:31.123493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.920256+0000","last_change":"2024-07-04T17:29:24.906305+0000","last_active":"2024-07-04T17:29:26.920256+0000","last_peered":"2024-07-04T17:29:26.920256+0000","last_clean":"2024-07-04T17:29:26.920256+0000","last_became_active":"2024-07-04T17:29:24.905901+0000","last_became_peered":"2024-07-04T17:29:24.905901+0000","last_unstale":"2024-07-04T17:29:26.920256+0000","last_undegraded":"2024-07-04T17:29:26.920256+0000","last_fullsized":"2024-07-04T17:29:26.920256+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:57:45.538215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.919388+0000","last_change":"2024-07-04T17:29:05.785226+0000","last_active":"2024-07-04T17:29:26.919388+0000","last_peered":"2024-07-04T17:29:26.919388+0000","last_clean":"2024-07-04T17:29:26.919388+0000","last_became_active":"2024-07-04T17:29:05.784069+0000","last_became_peered":"2024-07-04T17:29:05.784069+0000","last_unstale":"2024-07-04T17:29:26.919388+0000","last_undegraded":"2024-07-04T17:29:26.919388+0000","last_fullsized":"2024-07-04T17:29:26.919388+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:27:43.064532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.895965+0000","last_change":"2024-07-04T17:29:04.793187+0000","last_active":"2024-07-04T17:29:26.895965+0000","last_peered":"2024-07-04T17:29:26.895965+0000","last_clean":"2024-07-04T17:29:26.895965+0000","last_became_active":"2024-07-04T17:29:04.792877+0000","last_became_peered":"2024-07-04T17:29:04.792877+0000","last_unstale":"2024-07-04T17:29:26.895965+0000","last_undegraded":"2024-07-04T17:29:26.895965+0000","last_fullsized":"2024-07-04T17:29:26.895965+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:16:45.088529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.883127+0000","last_change":"2024-07-04T17:29:25.941792+0000","last_active":"2024-07-04T17:29:26.883127+0000","last_peered":"2024-07-04T17:29:26.883127+0000","last_clean":"2024-07-04T17:29:26.883127+0000","last_became_active":"2024-07-04T17:29:25.941112+0000","last_became_peered":"2024-07-04T17:29:25.941112+0000","last_unstale":"2024-07-04T17:29:26.883127+0000","last_undegraded":"2024-07-04T17:29:26.883127+0000","last_fullsized":"2024-07-04T17:29:26.883127+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:09:11.948958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.903528+0000","last_change":"2024-07-04T17:29:24.897427+0000","last_active":"2024-07-04T17:29:26.903528+0000","last_peered":"2024-07-04T17:29:26.903528+0000","last_clean":"2024-07-04T17:29:26.903528+0000","last_became_active":"2024-07-04T17:29:24.896153+0000","last_became_peered":"2024-07-04T17:29:24.896153+0000","last_unstale":"2024-07-04T17:29:26.903528+0000","last_undegraded":"2024-07-04T17:29:26.903528+0000","last_fullsized":"2024-07-04T17:29:26.903528+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:29:29.433683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.903987+0000","last_change":"2024-07-04T17:29:05.796869+0000","last_active":"2024-07-04T17:29:26.903987+0000","last_peered":"2024-07-04T17:29:26.903987+0000","last_clean":"2024-07-04T17:29:26.903987+0000","last_became_active":"2024-07-04T17:29:05.796573+0000","last_became_peered":"2024-07-04T17:29:05.796573+0000","last_unstale":"2024-07-04T17:29:26.903987+0000","last_undegraded":"2024-07-04T17:29:26.903987+0000","last_fullsized":"2024-07-04T17:29:26.903987+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:41:01.850387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1b","version":"49'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.898364+0000","last_change":"2024-07-04T17:29:04.793835+0000","last_active":"2024-07-04T17:29:26.898364+0000","last_peered":"2024-07-04T17:29:26.898364+0000","last_clean":"2024-07-04T17:29:26.898364+0000","last_became_active":"2024-07-04T17:29:04.792785+0000","last_became_peered":"2024-07-04T17:29:04.792785+0000","last_unstale":"2024-07-04T17:29:26.898364+0000","last_undegraded":"2024-07-04T17:29:26.898364+0000","last_fullsized":"2024-07-04T17:29:26.898364+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:21:48.040633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.916361+0000","last_change":"2024-07-04T17:29:25.905773+0000","last_active":"2024-07-04T17:29:26.916361+0000","last_peered":"2024-07-04T17:29:26.916361+0000","last_clean":"2024-07-04T17:29:26.916361+0000","last_became_active":"2024-07-04T17:29:25.905444+0000","last_became_peered":"2024-07-04T17:29:25.905444+0000","last_unstale":"2024-07-04T17:29:26.916361+0000","last_undegraded":"2024-07-04T17:29:26.916361+0000","last_fullsized":"2024-07-04T17:29:26.916361+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:27:15.519089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.884436+0000","last_change":"2024-07-04T17:29:24.897264+0000","last_active":"2024-07-04T17:29:26.884436+0000","last_peered":"2024-07-04T17:29:26.884436+0000","last_clean":"2024-07-04T17:29:26.884436+0000","last_became_active":"2024-07-04T17:29:24.896807+0000","last_became_peered":"2024-07-04T17:29:24.896807+0000","last_unstale":"2024-07-04T17:29:26.884436+0000","last_undegraded":"2024-07-04T17:29:26.884436+0000","last_fullsized":"2024-07-04T17:29:26.884436+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:21:29.893615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.919645+0000","last_change":"2024-07-04T17:29:05.796490+0000","last_active":"2024-07-04T17:29:26.919645+0000","last_peered":"2024-07-04T17:29:26.919645+0000","last_clean":"2024-07-04T17:29:26.919645+0000","last_became_active":"2024-07-04T17:29:05.788778+0000","last_became_peered":"2024-07-04T17:29:05.788778+0000","last_unstale":"2024-07-04T17:29:26.919645+0000","last_undegraded":"2024-07-04T17:29:26.919645+0000","last_fullsized":"2024-07-04T17:29:26.919645+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:23:59.134121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1a","version":"49'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.917632+0000","last_change":"2024-07-04T17:29:04.756095+0000","last_active":"2024-07-04T17:29:26.917632+0000","last_peered":"2024-07-04T17:29:26.917632+0000","last_clean":"2024-07-04T17:29:26.917632+0000","last_became_active":"2024-07-04T17:29:04.755679+0000","last_became_peered":"2024-07-04T17:29:04.755679+0000","last_unstale":"2024-07-04T17:29:26.917632+0000","last_undegraded":"2024-07-04T17:29:26.917632+0000","last_fullsized":"2024-07-04T17:29:26.917632+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:30:46.500676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.884386+0000","last_change":"2024-07-04T17:29:25.936840+0000","last_active":"2024-07-04T17:29:26.884386+0000","last_peered":"2024-07-04T17:29:26.884386+0000","last_clean":"2024-07-04T17:29:26.884386+0000","last_became_active":"2024-07-04T17:29:25.935770+0000","last_became_peered":"2024-07-04T17:29:25.935770+0000","last_unstale":"2024-07-04T17:29:26.884386+0000","last_undegraded":"2024-07-04T17:29:26.884386+0000","last_fullsized":"2024-07-04T17:29:26.884386+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:38:02.551875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1b","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.893735+0000","last_change":"2024-07-04T17:29:24.897558+0000","last_active":"2024-07-04T17:29:26.893735+0000","last_peered":"2024-07-04T17:29:26.893735+0000","last_clean":"2024-07-04T17:29:26.893735+0000","last_became_active":"2024-07-04T17:29:24.896987+0000","last_became_peered":"2024-07-04T17:29:24.896987+0000","last_unstale":"2024-07-04T17:29:26.893735+0000","last_undegraded":"2024-07-04T17:29:26.893735+0000","last_fullsized":"2024-07-04T17:29:26.893735+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:48:17.259024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.887343+0000","last_change":"2024-07-04T17:29:05.809480+0000","last_active":"2024-07-04T17:29:26.887343+0000","last_peered":"2024-07-04T17:29:26.887343+0000","last_clean":"2024-07-04T17:29:26.887343+0000","last_became_active":"2024-07-04T17:29:05.809138+0000","last_became_peered":"2024-07-04T17:29:05.809138+0000","last_unstale":"2024-07-04T17:29:26.887343+0000","last_undegraded":"2024-07-04T17:29:26.887343+0000","last_fullsized":"2024-07-04T17:29:26.887343+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:03:20.072994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.904091+0000","last_change":"2024-07-04T17:29:04.776840+0000","last_active":"2024-07-04T17:29:26.904091+0000","last_peered":"2024-07-04T17:29:26.904091+0000","last_clean":"2024-07-04T17:29:26.904091+0000","last_became_active":"2024-07-04T17:29:04.775729+0000","last_became_peered":"2024-07-04T17:29:04.775729+0000","last_unstale":"2024-07-04T17:29:26.904091+0000","last_undegraded":"2024-07-04T17:29:26.904091+0000","last_fullsized":"2024-07-04T17:29:26.904091+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:28:41.894029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.895965+0000","last_change":"2024-07-04T17:29:25.944549+0000","last_active":"2024-07-04T17:29:26.895965+0000","last_peered":"2024-07-04T17:29:26.895965+0000","last_clean":"2024-07-04T17:29:26.895965+0000","last_became_active":"2024-07-04T17:29:25.942180+0000","last_became_peered":"2024-07-04T17:29:25.942180+0000","last_unstale":"2024-07-04T17:29:26.895965+0000","last_undegraded":"2024-07-04T17:29:26.895965+0000","last_fullsized":"2024-07-04T17:29:26.895965+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:19:14.326622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.903619+0000","last_change":"2024-07-04T17:29:24.900033+0000","last_active":"2024-07-04T17:29:26.903619+0000","last_peered":"2024-07-04T17:29:26.903619+0000","last_clean":"2024-07-04T17:29:26.903619+0000","last_became_active":"2024-07-04T17:29:24.899184+0000","last_became_peered":"2024-07-04T17:29:24.899184+0000","last_unstale":"2024-07-04T17:29:26.903619+0000","last_undegraded":"2024-07-04T17:29:26.903619+0000","last_fullsized":"2024-07-04T17:29:26.903619+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:59:59.456695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.883891+0000","last_change":"2024-07-04T17:29:05.818108+0000","last_active":"2024-07-04T17:29:26.883891+0000","last_peered":"2024-07-04T17:29:26.883891+0000","last_clean":"2024-07-04T17:29:26.883891+0000","last_became_active":"2024-07-04T17:29:05.816963+0000","last_became_peered":"2024-07-04T17:29:05.816963+0000","last_unstale":"2024-07-04T17:29:26.883891+0000","last_undegraded":"2024-07-04T17:29:26.883891+0000","last_fullsized":"2024-07-04T17:29:26.883891+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:07:08.833691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.917101+0000","last_change":"2024-07-04T17:29:04.774790+0000","last_active":"2024-07-04T17:29:26.917101+0000","last_peered":"2024-07-04T17:29:26.917101+0000","last_clean":"2024-07-04T17:29:26.917101+0000","last_became_active":"2024-07-04T17:29:04.774427+0000","last_became_peered":"2024-07-04T17:29:04.774427+0000","last_unstale":"2024-07-04T17:29:26.917101+0000","last_undegraded":"2024-07-04T17:29:26.917101+0000","last_fullsized":"2024-07-04T17:29:26.917101+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:28:48.914599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.919267+0000","last_change":"2024-07-04T17:29:25.907093+0000","last_active":"2024-07-04T17:29:26.919267+0000","last_peered":"2024-07-04T17:29:26.919267+0000","last_clean":"2024-07-04T17:29:26.919267+0000","last_became_active":"2024-07-04T17:29:25.904489+0000","last_became_peered":"2024-07-04T17:29:25.904489+0000","last_unstale":"2024-07-04T17:29:26.919267+0000","last_undegraded":"2024-07-04T17:29:26.919267+0000","last_fullsized":"2024-07-04T17:29:26.919267+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:14:29.722776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.15","version":"53'3","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.918946+0000","last_change":"2024-07-04T17:29:24.905925+0000","last_active":"2024-07-04T17:29:26.918946+0000","last_peered":"2024-07-04T17:29:26.918946+0000","last_clean":"2024-07-04T17:29:26.918946+0000","last_became_active":"2024-07-04T17:29:24.905580+0000","last_became_peered":"2024-07-04T17:29:24.905580+0000","last_unstale":"2024-07-04T17:29:26.918946+0000","last_undegraded":"2024-07-04T17:29:26.918946+0000","last_fullsized":"2024-07-04T17:29:26.918946+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:15:53.299874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.921847+0000","last_change":"2024-07-04T17:29:05.795423+0000","last_active":"2024-07-04T17:29:26.921847+0000","last_peered":"2024-07-04T17:29:26.921847+0000","last_clean":"2024-07-04T17:29:26.921847+0000","last_became_active":"2024-07-04T17:29:05.794868+0000","last_became_peered":"2024-07-04T17:29:05.794868+0000","last_unstale":"2024-07-04T17:29:26.921847+0000","last_undegraded":"2024-07-04T17:29:26.921847+0000","last_fullsized":"2024-07-04T17:29:26.921847+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:36:01.857174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.911610+0000","last_change":"2024-07-04T17:29:04.772998+0000","last_active":"2024-07-04T17:29:26.911610+0000","last_peered":"2024-07-04T17:29:26.911610+0000","last_clean":"2024-07-04T17:29:26.911610+0000","last_became_active":"2024-07-04T17:29:04.772635+0000","last_became_peered":"2024-07-04T17:29:04.772635+0000","last_unstale":"2024-07-04T17:29:26.911610+0000","last_undegraded":"2024-07-04T17:29:26.911610+0000","last_fullsized":"2024-07-04T17:29:26.911610+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:59:48.689604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.894589+0000","last_change":"2024-07-04T17:29:25.950423+0000","last_active":"2024-07-04T17:29:26.894589+0000","last_peered":"2024-07-04T17:29:26.894589+0000","last_clean":"2024-07-04T17:29:26.894589+0000","last_became_active":"2024-07-04T17:29:25.948056+0000","last_became_peered":"2024-07-04T17:29:25.948056+0000","last_unstale":"2024-07-04T17:29:26.894589+0000","last_undegraded":"2024-07-04T17:29:26.894589+0000","last_fullsized":"2024-07-04T17:29:26.894589+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:36:00.670947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"53'6","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.905534+0000","last_change":"2024-07-04T17:29:24.894897+0000","last_active":"2024-07-04T17:29:26.905534+0000","last_peered":"2024-07-04T17:29:26.905534+0000","last_clean":"2024-07-04T17:29:26.905534+0000","last_became_active":"2024-07-04T17:29:24.894614+0000","last_became_peered":"2024-07-04T17:29:24.894614+0000","last_unstale":"2024-07-04T17:29:26.905534+0000","last_undegraded":"2024-07-04T17:29:26.905534+0000","last_fullsized":"2024-07-04T17:29:26.905534+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:58:52.867982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.893545+0000","last_change":"2024-07-04T17:29:05.832809+0000","last_active":"2024-07-04T17:29:26.893545+0000","last_peered":"2024-07-04T17:29:26.893545+0000","last_clean":"2024-07-04T17:29:26.893545+0000","last_became_active":"2024-07-04T17:29:05.829582+0000","last_became_peered":"2024-07-04T17:29:05.829582+0000","last_unstale":"2024-07-04T17:29:26.893545+0000","last_undegraded":"2024-07-04T17:29:26.893545+0000","last_fullsized":"2024-07-04T17:29:26.893545+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:50:42.737526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.909969+0000","last_change":"2024-07-04T17:29:04.780571+0000","last_active":"2024-07-04T17:29:26.909969+0000","last_peered":"2024-07-04T17:29:26.909969+0000","last_clean":"2024-07-04T17:29:26.909969+0000","last_became_active":"2024-07-04T17:29:04.780236+0000","last_became_peered":"2024-07-04T17:29:04.780236+0000","last_unstale":"2024-07-04T17:29:26.909969+0000","last_undegraded":"2024-07-04T17:29:26.909969+0000","last_fullsized":"2024-07-04T17:29:26.909969+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:55:21.724770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.908703+0000","last_change":"2024-07-04T17:29:25.911751+0000","last_active":"2024-07-04T17:29:26.908703+0000","last_peered":"2024-07-04T17:29:26.908703+0000","last_clean":"2024-07-04T17:29:26.908703+0000","last_became_active":"2024-07-04T17:29:25.909886+0000","last_became_peered":"2024-07-04T17:29:25.909886+0000","last_unstale":"2024-07-04T17:29:26.908703+0000","last_undegraded":"2024-07-04T17:29:26.908703+0000","last_fullsized":"2024-07-04T17:29:26.908703+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:35:54.403127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.919843+0000","last_change":"2024-07-04T17:29:24.900115+0000","last_active":"2024-07-04T17:29:26.919843+0000","last_peered":"2024-07-04T17:29:26.919843+0000","last_clean":"2024-07-04T17:29:26.919843+0000","last_became_active":"2024-07-04T17:29:24.899540+0000","last_became_peered":"2024-07-04T17:29:24.899540+0000","last_unstale":"2024-07-04T17:29:26.919843+0000","last_undegraded":"2024-07-04T17:29:26.919843+0000","last_fullsized":"2024-07-04T17:29:26.919843+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:26:10.444100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.905868+0000","last_change":"2024-07-04T17:29:05.783421+0000","last_active":"2024-07-04T17:29:26.905868+0000","last_peered":"2024-07-04T17:29:26.905868+0000","last_clean":"2024-07-04T17:29:26.905868+0000","last_became_active":"2024-07-04T17:29:05.783057+0000","last_became_peered":"2024-07-04T17:29:05.783057+0000","last_unstale":"2024-07-04T17:29:26.905868+0000","last_undegraded":"2024-07-04T17:29:26.905868+0000","last_fullsized":"2024-07-04T17:29:26.905868+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:44:09.505235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"49'3","reported_seq":25,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.883724+0000","last_change":"2024-07-04T17:29:04.794454+0000","last_active":"2024-07-04T17:29:26.883724+0000","last_peered":"2024-07-04T17:29:26.883724+0000","last_clean":"2024-07-04T17:29:26.883724+0000","last_became_active":"2024-07-04T17:29:04.793939+0000","last_became_peered":"2024-07-04T17:29:04.793939+0000","last_unstale":"2024-07-04T17:29:26.883724+0000","last_undegraded":"2024-07-04T17:29:26.883724+0000","last_fullsized":"2024-07-04T17:29:26.883724+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:10:23.616021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.920177+0000","last_change":"2024-07-04T17:29:25.929853+0000","last_active":"2024-07-04T17:29:26.920177+0000","last_peered":"2024-07-04T17:29:26.920177+0000","last_clean":"2024-07-04T17:29:26.920177+0000","last_became_active":"2024-07-04T17:29:25.929273+0000","last_became_peered":"2024-07-04T17:29:25.929273+0000","last_unstale":"2024-07-04T17:29:26.920177+0000","last_undegraded":"2024-07-04T17:29:26.920177+0000","last_fullsized":"2024-07-04T17:29:26.920177+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:56:42.644139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.920910+0000","last_change":"2024-07-04T17:29:24.905893+0000","last_active":"2024-07-04T17:29:26.920910+0000","last_peered":"2024-07-04T17:29:26.920910+0000","last_clean":"2024-07-04T17:29:26.920910+0000","last_became_active":"2024-07-04T17:29:24.903376+0000","last_became_peered":"2024-07-04T17:29:24.903376+0000","last_unstale":"2024-07-04T17:29:26.920910+0000","last_undegraded":"2024-07-04T17:29:26.920910+0000","last_fullsized":"2024-07-04T17:29:26.920910+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:57:14.138542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.908335+0000","last_change":"2024-07-04T17:29:05.792689+0000","last_active":"2024-07-04T17:29:26.908335+0000","last_peered":"2024-07-04T17:29:26.908335+0000","last_clean":"2024-07-04T17:29:26.908335+0000","last_became_active":"2024-07-04T17:29:05.792368+0000","last_became_peered":"2024-07-04T17:29:05.792368+0000","last_unstale":"2024-07-04T17:29:26.908335+0000","last_undegraded":"2024-07-04T17:29:26.908335+0000","last_fullsized":"2024-07-04T17:29:26.908335+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:58:35.184348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.14","version":"50'8","reported_seq":30,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.921334+0000","last_change":"2024-07-04T17:29:04.778462+0000","last_active":"2024-07-04T17:29:26.921334+0000","last_peered":"2024-07-04T17:29:26.921334+0000","last_clean":"2024-07-04T17:29:26.921334+0000","last_became_active":"2024-07-04T17:29:04.775998+0000","last_became_peered":"2024-07-04T17:29:04.775998+0000","last_unstale":"2024-07-04T17:29:26.921334+0000","last_undegraded":"2024-07-04T17:29:26.921334+0000","last_fullsized":"2024-07-04T17:29:26.921334+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:16:55.504943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3309,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.917089+0000","last_change":"2024-07-04T17:29:25.919884+0000","last_active":"2024-07-04T17:29:26.917089+0000","last_peered":"2024-07-04T17:29:26.917089+0000","last_clean":"2024-07-04T17:29:26.917089+0000","last_became_active":"2024-07-04T17:29:25.916603+0000","last_became_peered":"2024-07-04T17:29:25.916603+0000","last_unstale":"2024-07-04T17:29:26.917089+0000","last_undegraded":"2024-07-04T17:29:26.917089+0000","last_fullsized":"2024-07-04T17:29:26.917089+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:36:32.157304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.903835+0000","last_change":"2024-07-04T17:29:24.896781+0000","last_active":"2024-07-04T17:29:26.903835+0000","last_peered":"2024-07-04T17:29:26.903835+0000","last_clean":"2024-07-04T17:29:26.903835+0000","last_became_active":"2024-07-04T17:29:24.895390+0000","last_became_peered":"2024-07-04T17:29:24.895390+0000","last_unstale":"2024-07-04T17:29:26.903835+0000","last_undegraded":"2024-07-04T17:29:26.903835+0000","last_fullsized":"2024-07-04T17:29:26.903835+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:10:57.006780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.904108+0000","last_change":"2024-07-04T17:29:05.815269+0000","last_active":"2024-07-04T17:29:26.904108+0000","last_peered":"2024-07-04T17:29:26.904108+0000","last_clean":"2024-07-04T17:29:26.904108+0000","last_became_active":"2024-07-04T17:29:05.814468+0000","last_became_peered":"2024-07-04T17:29:05.814468+0000","last_unstale":"2024-07-04T17:29:26.904108+0000","last_undegraded":"2024-07-04T17:29:26.904108+0000","last_fullsized":"2024-07-04T17:29:26.904108+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:14:24.515173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.904555+0000","last_change":"2024-07-04T17:29:04.782215+0000","last_active":"2024-07-04T17:29:26.904555+0000","last_peered":"2024-07-04T17:29:26.904555+0000","last_clean":"2024-07-04T17:29:26.904555+0000","last_became_active":"2024-07-04T17:29:04.781891+0000","last_became_peered":"2024-07-04T17:29:04.781891+0000","last_unstale":"2024-07-04T17:29:26.904555+0000","last_undegraded":"2024-07-04T17:29:26.904555+0000","last_fullsized":"2024-07-04T17:29:26.904555+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:56:44.314464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.882969+0000","last_change":"2024-07-04T17:29:25.935820+0000","last_active":"2024-07-04T17:29:26.882969+0000","last_peered":"2024-07-04T17:29:26.882969+0000","last_clean":"2024-07-04T17:29:26.882969+0000","last_became_active":"2024-07-04T17:29:25.926927+0000","last_became_peered":"2024-07-04T17:29:25.926927+0000","last_unstale":"2024-07-04T17:29:26.882969+0000","last_undegraded":"2024-07-04T17:29:26.882969+0000","last_fullsized":"2024-07-04T17:29:26.882969+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:33:52.833086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"53'8","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.902531+0000","last_change":"2024-07-04T17:29:24.894996+0000","last_active":"2024-07-04T17:29:26.902531+0000","last_peered":"2024-07-04T17:29:26.902531+0000","last_clean":"2024-07-04T17:29:26.902531+0000","last_became_active":"2024-07-04T17:29:24.894682+0000","last_became_peered":"2024-07-04T17:29:24.894682+0000","last_unstale":"2024-07-04T17:29:26.902531+0000","last_undegraded":"2024-07-04T17:29:26.902531+0000","last_fullsized":"2024-07-04T17:29:26.902531+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:20:48.765120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.898196+0000","last_change":"2024-07-04T17:29:05.830879+0000","last_active":"2024-07-04T17:29:26.898196+0000","last_peered":"2024-07-04T17:29:26.898196+0000","last_clean":"2024-07-04T17:29:26.898196+0000","last_became_active":"2024-07-04T17:29:05.830572+0000","last_became_peered":"2024-07-04T17:29:05.830572+0000","last_unstale":"2024-07-04T17:29:26.898196+0000","last_undegraded":"2024-07-04T17:29:26.898196+0000","last_fullsized":"2024-07-04T17:29:26.898196+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:41:01.850387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.894239+0000","last_change":"2024-07-04T17:29:04.793440+0000","last_active":"2024-07-04T17:29:26.894239+0000","last_peered":"2024-07-04T17:29:26.894239+0000","last_clean":"2024-07-04T17:29:26.894239+0000","last_became_active":"2024-07-04T17:29:04.792810+0000","last_became_peered":"2024-07-04T17:29:04.792810+0000","last_unstale":"2024-07-04T17:29:26.894239+0000","last_undegraded":"2024-07-04T17:29:26.894239+0000","last_fullsized":"2024-07-04T17:29:26.894239+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:28:41.894029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.886658+0000","last_change":"2024-07-04T17:29:25.936383+0000","last_active":"2024-07-04T17:29:26.886658+0000","last_peered":"2024-07-04T17:29:26.886658+0000","last_clean":"2024-07-04T17:29:26.886658+0000","last_became_active":"2024-07-04T17:29:25.926218+0000","last_became_peered":"2024-07-04T17:29:25.926218+0000","last_unstale":"2024-07-04T17:29:26.886658+0000","last_undegraded":"2024-07-04T17:29:26.886658+0000","last_fullsized":"2024-07-04T17:29:26.886658+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:43:08.292816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.905850+0000","last_change":"2024-07-04T17:29:24.898810+0000","last_active":"2024-07-04T17:29:26.905850+0000","last_peered":"2024-07-04T17:29:26.905850+0000","last_clean":"2024-07-04T17:29:26.905850+0000","last_became_active":"2024-07-04T17:29:24.898434+0000","last_became_peered":"2024-07-04T17:29:24.898434+0000","last_unstale":"2024-07-04T17:29:26.905850+0000","last_undegraded":"2024-07-04T17:29:26.905850+0000","last_fullsized":"2024-07-04T17:29:26.905850+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:21:29.893615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.902784+0000","last_change":"2024-07-04T17:29:05.822812+0000","last_active":"2024-07-04T17:29:26.902784+0000","last_peered":"2024-07-04T17:29:26.902784+0000","last_clean":"2024-07-04T17:29:26.902784+0000","last_became_active":"2024-07-04T17:29:05.822519+0000","last_became_peered":"2024-07-04T17:29:05.822519+0000","last_unstale":"2024-07-04T17:29:26.902784+0000","last_undegraded":"2024-07-04T17:29:26.902784+0000","last_fullsized":"2024-07-04T17:29:26.902784+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:33:04.621287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.886564+0000","last_change":"2024-07-04T17:29:04.793535+0000","last_active":"2024-07-04T17:29:26.886564+0000","last_peered":"2024-07-04T17:29:26.886564+0000","last_clean":"2024-07-04T17:29:26.886564+0000","last_became_active":"2024-07-04T17:29:04.793160+0000","last_became_peered":"2024-07-04T17:29:04.793160+0000","last_unstale":"2024-07-04T17:29:26.886564+0000","last_undegraded":"2024-07-04T17:29:26.886564+0000","last_fullsized":"2024-07-04T17:29:26.886564+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:29:48.755500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.904278+0000","last_change":"2024-07-04T17:29:25.909449+0000","last_active":"2024-07-04T17:29:26.904278+0000","last_peered":"2024-07-04T17:29:26.904278+0000","last_clean":"2024-07-04T17:29:26.904278+0000","last_became_active":"2024-07-04T17:29:25.909092+0000","last_became_peered":"2024-07-04T17:29:25.909092+0000","last_unstale":"2024-07-04T17:29:26.904278+0000","last_undegraded":"2024-07-04T17:29:26.904278+0000","last_fullsized":"2024-07-04T17:29:26.904278+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:34:07.700861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.903748+0000","last_change":"2024-07-04T17:29:24.898515+0000","last_active":"2024-07-04T17:29:26.903748+0000","last_peered":"2024-07-04T17:29:26.903748+0000","last_clean":"2024-07-04T17:29:26.903748+0000","last_became_active":"2024-07-04T17:29:24.897717+0000","last_became_peered":"2024-07-04T17:29:24.897717+0000","last_unstale":"2024-07-04T17:29:26.903748+0000","last_undegraded":"2024-07-04T17:29:26.903748+0000","last_fullsized":"2024-07-04T17:29:26.903748+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:41:26.553241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.917197+0000","last_change":"2024-07-04T17:29:05.793691+0000","last_active":"2024-07-04T17:29:26.917197+0000","last_peered":"2024-07-04T17:29:26.917197+0000","last_clean":"2024-07-04T17:29:26.917197+0000","last_became_active":"2024-07-04T17:29:05.792767+0000","last_became_peered":"2024-07-04T17:29:05.792767+0000","last_unstale":"2024-07-04T17:29:26.917197+0000","last_undegraded":"2024-07-04T17:29:26.917197+0000","last_fullsized":"2024-07-04T17:29:26.917197+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:25:53.656021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.10","version":"49'8","reported_seq":30,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.916754+0000","last_change":"2024-07-04T17:29:04.767026+0000","last_active":"2024-07-04T17:29:26.916754+0000","last_peered":"2024-07-04T17:29:26.916754+0000","last_clean":"2024-07-04T17:29:26.916754+0000","last_became_active":"2024-07-04T17:29:04.764888+0000","last_became_peered":"2024-07-04T17:29:04.764888+0000","last_unstale":"2024-07-04T17:29:26.916754+0000","last_undegraded":"2024-07-04T17:29:26.916754+0000","last_fullsized":"2024-07-04T17:29:26.916754+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:08:09.031069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.905228+0000","last_change":"2024-07-04T17:29:25.921748+0000","last_active":"2024-07-04T17:29:26.905228+0000","last_peered":"2024-07-04T17:29:26.905228+0000","last_clean":"2024-07-04T17:29:26.905228+0000","last_became_active":"2024-07-04T17:29:25.920943+0000","last_became_peered":"2024-07-04T17:29:25.920943+0000","last_unstale":"2024-07-04T17:29:26.905228+0000","last_undegraded":"2024-07-04T17:29:26.905228+0000","last_fullsized":"2024-07-04T17:29:26.905228+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:26:13.764268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.918883+0000","last_change":"2024-07-04T17:29:24.905805+0000","last_active":"2024-07-04T17:29:26.918883+0000","last_peered":"2024-07-04T17:29:26.918883+0000","last_clean":"2024-07-04T17:29:26.918883+0000","last_became_active":"2024-07-04T17:29:24.903556+0000","last_became_peered":"2024-07-04T17:29:24.903556+0000","last_unstale":"2024-07-04T17:29:26.918883+0000","last_undegraded":"2024-07-04T17:29:26.918883+0000","last_fullsized":"2024-07-04T17:29:26.918883+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:19:13.317754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.895241+0000","last_change":"2024-07-04T17:29:05.826630+0000","last_active":"2024-07-04T17:29:26.895241+0000","last_peered":"2024-07-04T17:29:26.895241+0000","last_clean":"2024-07-04T17:29:26.895241+0000","last_became_active":"2024-07-04T17:29:05.826180+0000","last_became_peered":"2024-07-04T17:29:05.826180+0000","last_unstale":"2024-07-04T17:29:26.895241+0000","last_undegraded":"2024-07-04T17:29:26.895241+0000","last_fullsized":"2024-07-04T17:29:26.895241+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:42:34.680103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"49'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.920649+0000","last_change":"2024-07-04T17:29:04.778986+0000","last_active":"2024-07-04T17:29:26.920649+0000","last_peered":"2024-07-04T17:29:26.920649+0000","last_clean":"2024-07-04T17:29:26.920649+0000","last_became_active":"2024-07-04T17:29:04.775326+0000","last_became_peered":"2024-07-04T17:29:04.775326+0000","last_unstale":"2024-07-04T17:29:26.920649+0000","last_undegraded":"2024-07-04T17:29:26.920649+0000","last_fullsized":"2024-07-04T17:29:26.920649+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:01:44.218523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.887333+0000","last_change":"2024-07-04T17:29:25.948373+0000","last_active":"2024-07-04T17:29:26.887333+0000","last_peered":"2024-07-04T17:29:26.887333+0000","last_clean":"2024-07-04T17:29:26.887333+0000","last_became_active":"2024-07-04T17:29:25.944152+0000","last_became_peered":"2024-07-04T17:29:25.944152+0000","last_unstale":"2024-07-04T17:29:26.887333+0000","last_undegraded":"2024-07-04T17:29:26.887333+0000","last_fullsized":"2024-07-04T17:29:26.887333+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:06:13.998695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.887483+0000","last_change":"2024-07-04T17:29:24.899625+0000","last_active":"2024-07-04T17:29:26.887483+0000","last_peered":"2024-07-04T17:29:26.887483+0000","last_clean":"2024-07-04T17:29:26.887483+0000","last_became_active":"2024-07-04T17:29:24.899254+0000","last_became_peered":"2024-07-04T17:29:24.899254+0000","last_unstale":"2024-07-04T17:29:26.887483+0000","last_undegraded":"2024-07-04T17:29:26.887483+0000","last_fullsized":"2024-07-04T17:29:26.887483+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:25:30.548940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.888459+0000","last_change":"2024-07-04T17:29:05.825845+0000","last_active":"2024-07-04T17:29:26.888459+0000","last_peered":"2024-07-04T17:29:26.888459+0000","last_clean":"2024-07-04T17:29:26.888459+0000","last_became_active":"2024-07-04T17:29:05.824923+0000","last_became_peered":"2024-07-04T17:29:05.824923+0000","last_unstale":"2024-07-04T17:29:26.888459+0000","last_undegraded":"2024-07-04T17:29:26.888459+0000","last_fullsized":"2024-07-04T17:29:26.888459+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:30:47.509371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.921983+0000","last_change":"2024-07-04T17:29:04.787322+0000","last_active":"2024-07-04T17:29:26.921983+0000","last_peered":"2024-07-04T17:29:26.921983+0000","last_clean":"2024-07-04T17:29:26.921983+0000","last_became_active":"2024-07-04T17:29:04.786939+0000","last_became_peered":"2024-07-04T17:29:04.786939+0000","last_unstale":"2024-07-04T17:29:26.921983+0000","last_undegraded":"2024-07-04T17:29:26.921983+0000","last_fullsized":"2024-07-04T17:29:26.921983+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:31:08.982204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.921660+0000","last_change":"2024-07-04T17:29:25.907591+0000","last_active":"2024-07-04T17:29:26.921660+0000","last_peered":"2024-07-04T17:29:26.921660+0000","last_clean":"2024-07-04T17:29:26.921660+0000","last_became_active":"2024-07-04T17:29:25.905992+0000","last_became_peered":"2024-07-04T17:29:25.905992+0000","last_unstale":"2024-07-04T17:29:26.921660+0000","last_undegraded":"2024-07-04T17:29:26.921660+0000","last_fullsized":"2024-07-04T17:29:26.921660+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:42:27.033468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.909376+0000","last_change":"2024-07-04T17:29:24.896510+0000","last_active":"2024-07-04T17:29:26.909376+0000","last_peered":"2024-07-04T17:29:26.909376+0000","last_clean":"2024-07-04T17:29:26.909376+0000","last_became_active":"2024-07-04T17:29:24.896182+0000","last_became_peered":"2024-07-04T17:29:24.896182+0000","last_unstale":"2024-07-04T17:29:26.909376+0000","last_undegraded":"2024-07-04T17:29:26.909376+0000","last_fullsized":"2024-07-04T17:29:26.909376+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:24:16.866462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.895159+0000","last_change":"2024-07-04T17:29:05.817332+0000","last_active":"2024-07-04T17:29:26.895159+0000","last_peered":"2024-07-04T17:29:26.895159+0000","last_clean":"2024-07-04T17:29:26.895159+0000","last_became_active":"2024-07-04T17:29:05.816980+0000","last_became_peered":"2024-07-04T17:29:05.816980+0000","last_unstale":"2024-07-04T17:29:26.895159+0000","last_undegraded":"2024-07-04T17:29:26.895159+0000","last_fullsized":"2024-07-04T17:29:26.895159+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:35:30.393302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.d","version":"49'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.883595+0000","last_change":"2024-07-04T17:29:04.793950+0000","last_active":"2024-07-04T17:29:26.883595+0000","last_peered":"2024-07-04T17:29:26.883595+0000","last_clean":"2024-07-04T17:29:26.883595+0000","last_became_active":"2024-07-04T17:29:04.793520+0000","last_became_peered":"2024-07-04T17:29:04.793520+0000","last_unstale":"2024-07-04T17:29:26.883595+0000","last_undegraded":"2024-07-04T17:29:26.883595+0000","last_fullsized":"2024-07-04T17:29:26.883595+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:43:57.308367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.920495+0000","last_change":"2024-07-04T17:29:25.905407+0000","last_active":"2024-07-04T17:29:26.920495+0000","last_peered":"2024-07-04T17:29:26.920495+0000","last_clean":"2024-07-04T17:29:26.920495+0000","last_became_active":"2024-07-04T17:29:25.904925+0000","last_became_peered":"2024-07-04T17:29:25.904925+0000","last_unstale":"2024-07-04T17:29:26.920495+0000","last_undegraded":"2024-07-04T17:29:26.920495+0000","last_fullsized":"2024-07-04T17:29:26.920495+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:46:17.384367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.921168+0000","last_change":"2024-07-04T17:29:24.902667+0000","last_active":"2024-07-04T17:29:26.921168+0000","last_peered":"2024-07-04T17:29:26.921168+0000","last_clean":"2024-07-04T17:29:26.921168+0000","last_became_active":"2024-07-04T17:29:24.902340+0000","last_became_peered":"2024-07-04T17:29:24.902340+0000","last_unstale":"2024-07-04T17:29:26.921168+0000","last_undegraded":"2024-07-04T17:29:26.921168+0000","last_fullsized":"2024-07-04T17:29:26.921168+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:41:08.756979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.919736+0000","last_change":"2024-07-04T17:29:05.796377+0000","last_active":"2024-07-04T17:29:26.919736+0000","last_peered":"2024-07-04T17:29:26.919736+0000","last_clean":"2024-07-04T17:29:26.919736+0000","last_became_active":"2024-07-04T17:29:05.796069+0000","last_became_peered":"2024-07-04T17:29:05.796069+0000","last_unstale":"2024-07-04T17:29:26.919736+0000","last_undegraded":"2024-07-04T17:29:26.919736+0000","last_fullsized":"2024-07-04T17:29:26.919736+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:36:28.462582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.921376+0000","last_change":"2024-07-04T17:29:04.775241+0000","last_active":"2024-07-04T17:29:26.921376+0000","last_peered":"2024-07-04T17:29:26.921376+0000","last_clean":"2024-07-04T17:29:26.921376+0000","last_became_active":"2024-07-04T17:29:04.774292+0000","last_became_peered":"2024-07-04T17:29:04.774292+0000","last_unstale":"2024-07-04T17:29:26.921376+0000","last_undegraded":"2024-07-04T17:29:26.921376+0000","last_fullsized":"2024-07-04T17:29:26.921376+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:48:39.886887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.884077+0000","last_change":"2024-07-04T17:29:25.948492+0000","last_active":"2024-07-04T17:29:26.884077+0000","last_peered":"2024-07-04T17:29:26.884077+0000","last_clean":"2024-07-04T17:29:26.884077+0000","last_became_active":"2024-07-04T17:29:25.943924+0000","last_became_peered":"2024-07-04T17:29:25.943924+0000","last_unstale":"2024-07-04T17:29:26.884077+0000","last_undegraded":"2024-07-04T17:29:26.884077+0000","last_fullsized":"2024-07-04T17:29:26.884077+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:36:05.779361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.916683+0000","last_change":"2024-07-04T17:29:24.888049+0000","last_active":"2024-07-04T17:29:26.916683+0000","last_peered":"2024-07-04T17:29:26.916683+0000","last_clean":"2024-07-04T17:29:26.916683+0000","last_became_active":"2024-07-04T17:29:24.887713+0000","last_became_peered":"2024-07-04T17:29:24.887713+0000","last_unstale":"2024-07-04T17:29:26.916683+0000","last_undegraded":"2024-07-04T17:29:26.916683+0000","last_fullsized":"2024-07-04T17:29:26.916683+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:24:18.914442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.904205+0000","last_change":"2024-07-04T17:29:05.814789+0000","last_active":"2024-07-04T17:29:26.904205+0000","last_peered":"2024-07-04T17:29:26.904205+0000","last_clean":"2024-07-04T17:29:26.904205+0000","last_became_active":"2024-07-04T17:29:05.813725+0000","last_became_peered":"2024-07-04T17:29:05.813725+0000","last_unstale":"2024-07-04T17:29:26.904205+0000","last_undegraded":"2024-07-04T17:29:26.904205+0000","last_fullsized":"2024-07-04T17:29:26.904205+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:07:08.833691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.920700+0000","last_change":"2024-07-04T17:29:04.768448+0000","last_active":"2024-07-04T17:29:26.920700+0000","last_peered":"2024-07-04T17:29:26.920700+0000","last_clean":"2024-07-04T17:29:26.920700+0000","last_became_active":"2024-07-04T17:29:04.765316+0000","last_became_peered":"2024-07-04T17:29:04.765316+0000","last_unstale":"2024-07-04T17:29:26.920700+0000","last_undegraded":"2024-07-04T17:29:26.920700+0000","last_fullsized":"2024-07-04T17:29:26.920700+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:54:53.983424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.883132+0000","last_change":"2024-07-04T17:29:25.943546+0000","last_active":"2024-07-04T17:29:26.883132+0000","last_peered":"2024-07-04T17:29:26.883132+0000","last_clean":"2024-07-04T17:29:26.883132+0000","last_became_active":"2024-07-04T17:29:25.942910+0000","last_became_peered":"2024-07-04T17:29:25.942910+0000","last_unstale":"2024-07-04T17:29:26.883132+0000","last_undegraded":"2024-07-04T17:29:26.883132+0000","last_fullsized":"2024-07-04T17:29:26.883132+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:01:26.547719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.917578+0000","last_change":"2024-07-04T17:29:24.889471+0000","last_active":"2024-07-04T17:29:26.917578+0000","last_peered":"2024-07-04T17:29:26.917578+0000","last_clean":"2024-07-04T17:29:26.917578+0000","last_became_active":"2024-07-04T17:29:24.889150+0000","last_became_peered":"2024-07-04T17:29:24.889150+0000","last_unstale":"2024-07-04T17:29:26.917578+0000","last_undegraded":"2024-07-04T17:29:26.917578+0000","last_fullsized":"2024-07-04T17:29:26.917578+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:56:50.839003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.919960+0000","last_change":"2024-07-04T17:29:05.784868+0000","last_active":"2024-07-04T17:29:26.919960+0000","last_peered":"2024-07-04T17:29:26.919960+0000","last_clean":"2024-07-04T17:29:26.919960+0000","last_became_active":"2024-07-04T17:29:05.784419+0000","last_became_peered":"2024-07-04T17:29:05.784419+0000","last_unstale":"2024-07-04T17:29:26.919960+0000","last_undegraded":"2024-07-04T17:29:26.919960+0000","last_fullsized":"2024-07-04T17:29:26.919960+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:33:47.592614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.904398+0000","last_change":"2024-07-04T17:29:04.780522+0000","last_active":"2024-07-04T17:29:26.904398+0000","last_peered":"2024-07-04T17:29:26.904398+0000","last_clean":"2024-07-04T17:29:26.904398+0000","last_became_active":"2024-07-04T17:29:04.780161+0000","last_became_peered":"2024-07-04T17:29:04.780161+0000","last_unstale":"2024-07-04T17:29:26.904398+0000","last_undegraded":"2024-07-04T17:29:26.904398+0000","last_fullsized":"2024-07-04T17:29:26.904398+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:29:48.755500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.894823+0000","last_change":"2024-07-04T17:29:25.946621+0000","last_active":"2024-07-04T17:29:26.894823+0000","last_peered":"2024-07-04T17:29:26.894823+0000","last_clean":"2024-07-04T17:29:26.894823+0000","last_became_active":"2024-07-04T17:29:25.943573+0000","last_became_peered":"2024-07-04T17:29:25.943573+0000","last_unstale":"2024-07-04T17:29:26.894823+0000","last_undegraded":"2024-07-04T17:29:26.894823+0000","last_fullsized":"2024-07-04T17:29:26.894823+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:24:19.923310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.887911+0000","last_change":"2024-07-04T17:29:24.892364+0000","last_active":"2024-07-04T17:29:26.887911+0000","last_peered":"2024-07-04T17:29:26.887911+0000","last_clean":"2024-07-04T17:29:26.887911+0000","last_became_active":"2024-07-04T17:29:24.892013+0000","last_became_peered":"2024-07-04T17:29:24.892013+0000","last_unstale":"2024-07-04T17:29:26.887911+0000","last_undegraded":"2024-07-04T17:29:26.887911+0000","last_fullsized":"2024-07-04T17:29:26.887911+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:09:46.797607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.917424+0000","last_change":"2024-07-04T17:29:05.793437+0000","last_active":"2024-07-04T17:29:26.917424+0000","last_peered":"2024-07-04T17:29:26.917424+0000","last_clean":"2024-07-04T17:29:26.917424+0000","last_became_active":"2024-07-04T17:29:05.792368+0000","last_became_peered":"2024-07-04T17:29:05.792368+0000","last_unstale":"2024-07-04T17:29:26.917424+0000","last_undegraded":"2024-07-04T17:29:26.917424+0000","last_fullsized":"2024-07-04T17:29:26.917424+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:17:35.033747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.903016+0000","last_change":"2024-07-04T17:29:04.774550+0000","last_active":"2024-07-04T17:29:26.903016+0000","last_peered":"2024-07-04T17:29:26.903016+0000","last_clean":"2024-07-04T17:29:26.903016+0000","last_became_active":"2024-07-04T17:29:04.774168+0000","last_became_peered":"2024-07-04T17:29:04.774168+0000","last_unstale":"2024-07-04T17:29:26.903016+0000","last_undegraded":"2024-07-04T17:29:26.903016+0000","last_fullsized":"2024-07-04T17:29:26.903016+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:43:57.308367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.919451+0000","last_change":"2024-07-04T17:29:25.924201+0000","last_active":"2024-07-04T17:29:26.919451+0000","last_peered":"2024-07-04T17:29:26.919451+0000","last_clean":"2024-07-04T17:29:26.919451+0000","last_became_active":"2024-07-04T17:29:25.923907+0000","last_became_peered":"2024-07-04T17:29:25.923907+0000","last_unstale":"2024-07-04T17:29:26.919451+0000","last_undegraded":"2024-07-04T17:29:26.919451+0000","last_fullsized":"2024-07-04T17:29:26.919451+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:44:57.717348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.887137+0000","last_change":"2024-07-04T17:29:24.892444+0000","last_active":"2024-07-04T17:29:26.887137+0000","last_peered":"2024-07-04T17:29:26.887137+0000","last_clean":"2024-07-04T17:29:26.887137+0000","last_became_active":"2024-07-04T17:29:24.892078+0000","last_became_peered":"2024-07-04T17:29:24.892078+0000","last_unstale":"2024-07-04T17:29:26.887137+0000","last_undegraded":"2024-07-04T17:29:26.887137+0000","last_fullsized":"2024-07-04T17:29:26.887137+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:31:25.400229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.895574+0000","last_change":"2024-07-04T17:29:05.826827+0000","last_active":"2024-07-04T17:29:26.895574+0000","last_peered":"2024-07-04T17:29:26.895574+0000","last_clean":"2024-07-04T17:29:26.895574+0000","last_became_active":"2024-07-04T17:29:05.825730+0000","last_became_peered":"2024-07-04T17:29:05.825730+0000","last_unstale":"2024-07-04T17:29:26.895574+0000","last_undegraded":"2024-07-04T17:29:26.895574+0000","last_fullsized":"2024-07-04T17:29:26.895574+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:14:31.441713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"49'2","reported_seq":24,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.911966+0000","last_change":"2024-07-04T17:29:04.780892+0000","last_active":"2024-07-04T17:29:26.911966+0000","last_peered":"2024-07-04T17:29:26.911966+0000","last_clean":"2024-07-04T17:29:26.911966+0000","last_became_active":"2024-07-04T17:29:04.780559+0000","last_became_peered":"2024-07-04T17:29:04.780559+0000","last_unstale":"2024-07-04T17:29:26.911966+0000","last_undegraded":"2024-07-04T17:29:26.911966+0000","last_fullsized":"2024-07-04T17:29:26.911966+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:46:37.056095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.907770+0000","last_change":"2024-07-04T17:29:25.940245+0000","last_active":"2024-07-04T17:29:26.907770+0000","last_peered":"2024-07-04T17:29:26.907770+0000","last_clean":"2024-07-04T17:29:26.907770+0000","last_became_active":"2024-07-04T17:29:25.938812+0000","last_became_peered":"2024-07-04T17:29:25.938812+0000","last_unstale":"2024-07-04T17:29:26.907770+0000","last_undegraded":"2024-07-04T17:29:26.907770+0000","last_fullsized":"2024-07-04T17:29:26.907770+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:21:06.800541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.919611+0000","last_change":"2024-07-04T17:29:24.902583+0000","last_active":"2024-07-04T17:29:26.919611+0000","last_peered":"2024-07-04T17:29:26.919611+0000","last_clean":"2024-07-04T17:29:26.919611+0000","last_became_active":"2024-07-04T17:29:24.902196+0000","last_became_peered":"2024-07-04T17:29:24.902196+0000","last_unstale":"2024-07-04T17:29:26.919611+0000","last_undegraded":"2024-07-04T17:29:26.919611+0000","last_fullsized":"2024-07-04T17:29:26.919611+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:32:08.081260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.888344+0000","last_change":"2024-07-04T17:29:05.831979+0000","last_active":"2024-07-04T17:29:26.888344+0000","last_peered":"2024-07-04T17:29:26.888344+0000","last_clean":"2024-07-04T17:29:26.888344+0000","last_became_active":"2024-07-04T17:29:05.829983+0000","last_became_peered":"2024-07-04T17:29:05.829983+0000","last_unstale":"2024-07-04T17:29:26.888344+0000","last_undegraded":"2024-07-04T17:29:26.888344+0000","last_fullsized":"2024-07-04T17:29:26.888344+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:03:45.123708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"50'1","reported_seq":23,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.921994+0000","last_change":"2024-07-04T17:29:04.784413+0000","last_active":"2024-07-04T17:29:26.921994+0000","last_peered":"2024-07-04T17:29:26.921994+0000","last_clean":"2024-07-04T17:29:26.921994+0000","last_became_active":"2024-07-04T17:29:04.784058+0000","last_became_peered":"2024-07-04T17:29:04.784058+0000","last_unstale":"2024-07-04T17:29:26.921994+0000","last_undegraded":"2024-07-04T17:29:26.921994+0000","last_fullsized":"2024-07-04T17:29:26.921994+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:35:32.227491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.920845+0000","last_change":"2024-07-04T17:29:25.924562+0000","last_active":"2024-07-04T17:29:26.920845+0000","last_peered":"2024-07-04T17:29:26.920845+0000","last_clean":"2024-07-04T17:29:26.920845+0000","last_became_active":"2024-07-04T17:29:25.922723+0000","last_became_peered":"2024-07-04T17:29:25.922723+0000","last_unstale":"2024-07-04T17:29:26.920845+0000","last_undegraded":"2024-07-04T17:29:26.920845+0000","last_fullsized":"2024-07-04T17:29:26.920845+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:55:19.068481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.920041+0000","last_change":"2024-07-04T17:29:24.899837+0000","last_active":"2024-07-04T17:29:26.920041+0000","last_peered":"2024-07-04T17:29:26.920041+0000","last_clean":"2024-07-04T17:29:26.920041+0000","last_became_active":"2024-07-04T17:29:24.898972+0000","last_became_peered":"2024-07-04T17:29:24.898972+0000","last_unstale":"2024-07-04T17:29:26.920041+0000","last_undegraded":"2024-07-04T17:29:26.920041+0000","last_fullsized":"2024-07-04T17:29:26.920041+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:20:36.161659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.907872+0000","last_change":"2024-07-04T17:29:05.821127+0000","last_active":"2024-07-04T17:29:26.907872+0000","last_peered":"2024-07-04T17:29:26.907872+0000","last_clean":"2024-07-04T17:29:26.907872+0000","last_became_active":"2024-07-04T17:29:05.820787+0000","last_became_peered":"2024-07-04T17:29:05.820787+0000","last_unstale":"2024-07-04T17:29:26.907872+0000","last_undegraded":"2024-07-04T17:29:26.907872+0000","last_fullsized":"2024-07-04T17:29:26.907872+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:33:26.836346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.908351+0000","last_change":"2024-07-04T17:29:04.786173+0000","last_active":"2024-07-04T17:29:26.908351+0000","last_peered":"2024-07-04T17:29:26.908351+0000","last_clean":"2024-07-04T17:29:26.908351+0000","last_became_active":"2024-07-04T17:29:04.783241+0000","last_became_peered":"2024-07-04T17:29:04.783241+0000","last_unstale":"2024-07-04T17:29:26.908351+0000","last_undegraded":"2024-07-04T17:29:26.908351+0000","last_fullsized":"2024-07-04T17:29:26.908351+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:16:45.088529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.909537+0000","last_change":"2024-07-04T17:29:25.919167+0000","last_active":"2024-07-04T17:29:26.909537+0000","last_peered":"2024-07-04T17:29:26.909537+0000","last_clean":"2024-07-04T17:29:26.909537+0000","last_became_active":"2024-07-04T17:29:25.918786+0000","last_became_peered":"2024-07-04T17:29:25.918786+0000","last_unstale":"2024-07-04T17:29:26.909537+0000","last_undegraded":"2024-07-04T17:29:26.909537+0000","last_fullsized":"2024-07-04T17:29:26.909537+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:05:18.667710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"54'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.914918+0000","last_change":"2024-07-04T17:29:24.897862+0000","last_active":"2024-07-04T17:29:26.914918+0000","last_peered":"2024-07-04T17:29:26.914918+0000","last_clean":"2024-07-04T17:29:26.914918+0000","last_became_active":"2024-07-04T17:29:24.897554+0000","last_became_peered":"2024-07-04T17:29:24.897554+0000","last_unstale":"2024-07-04T17:29:26.914918+0000","last_undegraded":"2024-07-04T17:29:26.914918+0000","last_fullsized":"2024-07-04T17:29:26.914918+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:54:00.256493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.919955+0000","last_change":"2024-07-04T17:29:05.785545+0000","last_active":"2024-07-04T17:29:26.919955+0000","last_peered":"2024-07-04T17:29:26.919955+0000","last_clean":"2024-07-04T17:29:26.919955+0000","last_became_active":"2024-07-04T17:29:05.784520+0000","last_became_peered":"2024-07-04T17:29:05.784520+0000","last_unstale":"2024-07-04T17:29:26.919955+0000","last_undegraded":"2024-07-04T17:29:26.919955+0000","last_fullsized":"2024-07-04T17:29:26.919955+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:31:15.745622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.897995+0000","last_change":"2024-07-04T17:29:04.794330+0000","last_active":"2024-07-04T17:29:26.897995+0000","last_peered":"2024-07-04T17:29:26.897995+0000","last_clean":"2024-07-04T17:29:26.897995+0000","last_became_active":"2024-07-04T17:29:04.793980+0000","last_became_peered":"2024-07-04T17:29:04.793980+0000","last_unstale":"2024-07-04T17:29:26.897995+0000","last_undegraded":"2024-07-04T17:29:26.897995+0000","last_fullsized":"2024-07-04T17:29:26.897995+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:06:13.018475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.902492+0000","last_change":"2024-07-04T17:29:25.919313+0000","last_active":"2024-07-04T17:29:26.902492+0000","last_peered":"2024-07-04T17:29:26.902492+0000","last_clean":"2024-07-04T17:29:26.902492+0000","last_became_active":"2024-07-04T17:29:25.911731+0000","last_became_peered":"2024-07-04T17:29:25.911731+0000","last_unstale":"2024-07-04T17:29:26.902492+0000","last_undegraded":"2024-07-04T17:29:26.902492+0000","last_fullsized":"2024-07-04T17:29:26.902492+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:25:33.258238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.916522+0000","last_change":"2024-07-04T17:29:24.906197+0000","last_active":"2024-07-04T17:29:26.916522+0000","last_peered":"2024-07-04T17:29:26.916522+0000","last_clean":"2024-07-04T17:29:26.916522+0000","last_became_active":"2024-07-04T17:29:24.905468+0000","last_became_peered":"2024-07-04T17:29:24.905468+0000","last_unstale":"2024-07-04T17:29:26.916522+0000","last_undegraded":"2024-07-04T17:29:26.916522+0000","last_fullsized":"2024-07-04T17:29:26.916522+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:35:59.662079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.904228+0000","last_change":"2024-07-04T17:29:05.802494+0000","last_active":"2024-07-04T17:29:26.904228+0000","last_peered":"2024-07-04T17:29:26.904228+0000","last_clean":"2024-07-04T17:29:26.904228+0000","last_became_active":"2024-07-04T17:29:05.795642+0000","last_became_peered":"2024-07-04T17:29:05.795642+0000","last_unstale":"2024-07-04T17:29:26.904228+0000","last_undegraded":"2024-07-04T17:29:26.904228+0000","last_fullsized":"2024-07-04T17:29:26.904228+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:19:50.013783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.904907+0000","last_change":"2024-07-04T17:29:04.774431+0000","last_active":"2024-07-04T17:29:26.904907+0000","last_peered":"2024-07-04T17:29:26.904907+0000","last_clean":"2024-07-04T17:29:26.904907+0000","last_became_active":"2024-07-04T17:29:04.774034+0000","last_became_peered":"2024-07-04T17:29:04.774034+0000","last_unstale":"2024-07-04T17:29:26.904907+0000","last_undegraded":"2024-07-04T17:29:26.904907+0000","last_fullsized":"2024-07-04T17:29:26.904907+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:10:23.616021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.916328+0000","last_change":"2024-07-04T17:29:25.905238+0000","last_active":"2024-07-04T17:29:26.916328+0000","last_peered":"2024-07-04T17:29:26.916328+0000","last_clean":"2024-07-04T17:29:26.916328+0000","last_became_active":"2024-07-04T17:29:25.904711+0000","last_became_peered":"2024-07-04T17:29:25.904711+0000","last_unstale":"2024-07-04T17:29:26.916328+0000","last_undegraded":"2024-07-04T17:29:26.916328+0000","last_fullsized":"2024-07-04T17:29:26.916328+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:26:00.911107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.886421+0000","last_change":"2024-07-04T17:29:24.896819+0000","last_active":"2024-07-04T17:29:26.886421+0000","last_peered":"2024-07-04T17:29:26.886421+0000","last_clean":"2024-07-04T17:29:26.886421+0000","last_became_active":"2024-07-04T17:29:24.896516+0000","last_became_peered":"2024-07-04T17:29:24.896516+0000","last_unstale":"2024-07-04T17:29:26.886421+0000","last_undegraded":"2024-07-04T17:29:26.886421+0000","last_fullsized":"2024-07-04T17:29:26.886421+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:54:00.256493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.920271+0000","last_change":"2024-07-04T17:29:05.784396+0000","last_active":"2024-07-04T17:29:26.920271+0000","last_peered":"2024-07-04T17:29:26.920271+0000","last_clean":"2024-07-04T17:29:26.920271+0000","last_became_active":"2024-07-04T17:29:05.783953+0000","last_became_peered":"2024-07-04T17:29:05.783953+0000","last_unstale":"2024-07-04T17:29:26.920271+0000","last_undegraded":"2024-07-04T17:29:26.920271+0000","last_fullsized":"2024-07-04T17:29:26.920271+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:53:09.757420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.894616+0000","last_change":"2024-07-04T17:29:04.795160+0000","last_active":"2024-07-04T17:29:26.894616+0000","last_peered":"2024-07-04T17:29:26.894616+0000","last_clean":"2024-07-04T17:29:26.894616+0000","last_became_active":"2024-07-04T17:29:04.793328+0000","last_became_peered":"2024-07-04T17:29:04.793328+0000","last_unstale":"2024-07-04T17:29:26.894616+0000","last_undegraded":"2024-07-04T17:29:26.894616+0000","last_fullsized":"2024-07-04T17:29:26.894616+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:56:44.314464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.886702+0000","last_change":"2024-07-04T17:29:25.937197+0000","last_active":"2024-07-04T17:29:26.886702+0000","last_peered":"2024-07-04T17:29:26.886702+0000","last_clean":"2024-07-04T17:29:26.886702+0000","last_became_active":"2024-07-04T17:29:25.936471+0000","last_became_peered":"2024-07-04T17:29:25.936471+0000","last_unstale":"2024-07-04T17:29:26.886702+0000","last_undegraded":"2024-07-04T17:29:26.886702+0000","last_fullsized":"2024-07-04T17:29:26.886702+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:32:38.852994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.894079+0000","last_change":"2024-07-04T17:29:24.897294+0000","last_active":"2024-07-04T17:29:26.894079+0000","last_peered":"2024-07-04T17:29:26.894079+0000","last_clean":"2024-07-04T17:29:26.894079+0000","last_became_active":"2024-07-04T17:29:24.896572+0000","last_became_peered":"2024-07-04T17:29:24.896572+0000","last_unstale":"2024-07-04T17:29:26.894079+0000","last_undegraded":"2024-07-04T17:29:26.894079+0000","last_fullsized":"2024-07-04T17:29:26.894079+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:24:16.866462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.902814+0000","last_change":"2024-07-04T17:29:05.796165+0000","last_active":"2024-07-04T17:29:26.902814+0000","last_peered":"2024-07-04T17:29:26.902814+0000","last_clean":"2024-07-04T17:29:26.902814+0000","last_became_active":"2024-07-04T17:29:05.795837+0000","last_became_peered":"2024-07-04T17:29:05.795837+0000","last_unstale":"2024-07-04T17:29:26.902814+0000","last_undegraded":"2024-07-04T17:29:26.902814+0000","last_fullsized":"2024-07-04T17:29:26.902814+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:36:15.241406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.904075+0000","last_change":"2024-07-04T17:29:04.790173+0000","last_active":"2024-07-04T17:29:26.904075+0000","last_peered":"2024-07-04T17:29:26.904075+0000","last_clean":"2024-07-04T17:29:26.904075+0000","last_became_active":"2024-07-04T17:29:04.789042+0000","last_became_peered":"2024-07-04T17:29:04.789042+0000","last_unstale":"2024-07-04T17:29:26.904075+0000","last_undegraded":"2024-07-04T17:29:26.904075+0000","last_fullsized":"2024-07-04T17:29:26.904075+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:06:13.018475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.887509+0000","last_change":"2024-07-04T17:29:25.930211+0000","last_active":"2024-07-04T17:29:26.887509+0000","last_peered":"2024-07-04T17:29:26.887509+0000","last_clean":"2024-07-04T17:29:26.887509+0000","last_became_active":"2024-07-04T17:29:25.927797+0000","last_became_peered":"2024-07-04T17:29:25.927797+0000","last_unstale":"2024-07-04T17:29:26.887509+0000","last_undegraded":"2024-07-04T17:29:26.887509+0000","last_fullsized":"2024-07-04T17:29:26.887509+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:31:17.186697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.904026+0000","last_change":"2024-07-04T17:29:24.898861+0000","last_active":"2024-07-04T17:29:26.904026+0000","last_peered":"2024-07-04T17:29:26.904026+0000","last_clean":"2024-07-04T17:29:26.904026+0000","last_became_active":"2024-07-04T17:29:24.898159+0000","last_became_peered":"2024-07-04T17:29:24.898159+0000","last_unstale":"2024-07-04T17:29:26.904026+0000","last_undegraded":"2024-07-04T17:29:26.904026+0000","last_fullsized":"2024-07-04T17:29:26.904026+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:03:14.973539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.917597+0000","last_change":"2024-07-04T17:29:05.793083+0000","last_active":"2024-07-04T17:29:26.917597+0000","last_peered":"2024-07-04T17:29:26.917597+0000","last_clean":"2024-07-04T17:29:26.917597+0000","last_became_active":"2024-07-04T17:29:05.790165+0000","last_became_peered":"2024-07-04T17:29:05.790165+0000","last_unstale":"2024-07-04T17:29:26.917597+0000","last_undegraded":"2024-07-04T17:29:26.917597+0000","last_fullsized":"2024-07-04T17:29:26.917597+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:20:46.692294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.903127+0000","last_change":"2024-07-04T17:29:04.770011+0000","last_active":"2024-07-04T17:29:26.903127+0000","last_peered":"2024-07-04T17:29:26.903127+0000","last_clean":"2024-07-04T17:29:26.903127+0000","last_became_active":"2024-07-04T17:29:04.769666+0000","last_became_peered":"2024-07-04T17:29:04.769666+0000","last_unstale":"2024-07-04T17:29:26.903127+0000","last_undegraded":"2024-07-04T17:29:26.903127+0000","last_fullsized":"2024-07-04T17:29:26.903127+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:21:48.040633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.903486+0000","last_change":"2024-07-04T17:29:25.921104+0000","last_active":"2024-07-04T17:29:26.903486+0000","last_peered":"2024-07-04T17:29:26.903486+0000","last_clean":"2024-07-04T17:29:26.903486+0000","last_became_active":"2024-07-04T17:29:25.920695+0000","last_became_peered":"2024-07-04T17:29:25.920695+0000","last_unstale":"2024-07-04T17:29:26.903486+0000","last_undegraded":"2024-07-04T17:29:26.903486+0000","last_fullsized":"2024-07-04T17:29:26.903486+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:24.839056+0000","last_clean_scrub_stamp":"2024-07-04T17:29:24.839056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:55:02.296031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.904825+0000","last_change":"2024-07-04T17:29:24.898336+0000","last_active":"2024-07-04T17:29:26.904825+0000","last_peered":"2024-07-04T17:29:26.904825+0000","last_clean":"2024-07-04T17:29:26.904825+0000","last_became_active":"2024-07-04T17:29:24.897943+0000","last_became_peered":"2024-07-04T17:29:24.897943+0000","last_unstale":"2024-07-04T17:29:26.904825+0000","last_undegraded":"2024-07-04T17:29:26.904825+0000","last_fullsized":"2024-07-04T17:29:26.904825+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:23.830188+0000","last_clean_scrub_stamp":"2024-07-04T17:29:23.830188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:15:04.231122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.893543+0000","last_change":"2024-07-04T17:29:05.816496+0000","last_active":"2024-07-04T17:29:26.893543+0000","last_peered":"2024-07-04T17:29:26.893543+0000","last_clean":"2024-07-04T17:29:26.893543+0000","last_became_active":"2024-07-04T17:29:05.816147+0000","last_became_peered":"2024-07-04T17:29:05.816147+0000","last_unstale":"2024-07-04T17:29:26.893543+0000","last_undegraded":"2024-07-04T17:29:26.893543+0000","last_fullsized":"2024-07-04T17:29:26.893543+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:04.730809+0000","last_clean_scrub_stamp":"2024-07-04T17:29:04.730809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:33:26.836346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.917650+0000","last_change":"2024-07-04T17:29:04.770392+0000","last_active":"2024-07-04T17:29:26.917650+0000","last_peered":"2024-07-04T17:29:26.917650+0000","last_clean":"2024-07-04T17:29:26.917650+0000","last_became_active":"2024-07-04T17:29:04.758815+0000","last_became_peered":"2024-07-04T17:29:04.758815+0000","last_unstale":"2024-07-04T17:29:26.917650+0000","last_undegraded":"2024-07-04T17:29:26.917650+0000","last_fullsized":"2024-07-04T17:29:26.917650+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:29:03.722115+0000","last_clean_scrub_stamp":"2024-07-04T17:29:03.722115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:35:08.872682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"35'66","reported_seq":211,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-07-04T17:29:26.903412+0000","last_change":"2024-07-04T17:26:04.953559+0000","last_active":"2024-07-04T17:29:26.903412+0000","last_peered":"2024-07-04T17:29:26.903412+0000","last_clean":"2024-07-04T17:29:26.903412+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:29:26.903412+0000","last_undegraded":"2024-07-04T17:29:26.903412+0000","last_fullsized":"2024-07-04T17:29:26.903412+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_clean_scrub_stamp":"2024-07-04T17:26:03.933432+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.064181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":398,"num_write":103,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":4107,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":398,"num_write":103,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738411,"num_pgs":116,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31760,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340080,"statfs":{"total":96636764160,"available":96604241920,"internally_reserved":0,"allocated":589824,"data_stored":209372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738412,"num_pgs":122,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1163264,"data_stored":796831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738410,"num_pgs":126,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32312,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1150976,"data_stored":797878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:29:31.276 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-04T17:29:31.276 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:29:31.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.605+0000 7f6c88800640 1 -- 172.21.15.42:0/2626550326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c8006cea0 msgr2=0x7f6c800b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:31.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.605+0000 7f6c88800640 1 --2- 172.21.15.42:0/2626550326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c8006cea0 0x7f6c800b3030 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009230 tx=0x7f6c7402a480 comp rx=0 tx=0).stop 2024-07-04T17:29:31.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.605+0000 7f6c88800640 1 -- 172.21.15.42:0/2626550326 shutdown_connections 2024-07-04T17:29:31.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.605+0000 7f6c88800640 1 --2- 172.21.15.42:0/2626550326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c8006cea0 0x7f6c800b3030 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:31.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.605+0000 7f6c88800640 1 -- 172.21.15.42:0/2626550326 >> 172.21.15.42:0/2626550326 conn(0x7f6c800bf700 msgr2=0x7f6c800af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:31.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.605+0000 7f6c88800640 1 -- 172.21.15.42:0/2626550326 shutdown_connections 2024-07-04T17:29:31.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.605+0000 7f6c88800640 1 -- 172.21.15.42:0/2626550326 wait complete. 2024-07-04T17:29:31.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.607+0000 7f6c88800640 1 Processor -- start 2024-07-04T17:29:31.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.607+0000 7f6c88800640 1 -- start start 2024-07-04T17:29:31.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.609+0000 7f6c88800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c8006cea0 0x7f6c801d9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:31.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.609+0000 7f6c88800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8006ef50 con 0x7f6c8006cea0 2024-07-04T17:29:31.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.609+0000 7f6c85e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c8006cea0 0x7f6c801d9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:31.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.609+0000 7f6c85e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c8006cea0 0x7f6c801d9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45370/0 (socket says 172.21.15.42:45370) 2024-07-04T17:29:31.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.609+0000 7f6c85e00640 1 -- 172.21.15.42:0/2682238021 learned_addr learned my addr 172.21.15.42:0/2682238021 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:31.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.609+0000 7f6c85e00640 1 -- 172.21.15.42:0/2682238021 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c80070da0 con 0x7f6c8006cea0 2024-07-04T17:29:31.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.610+0000 7f6c85e00640 1 --2- 172.21.15.42:0/2682238021 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c8006cea0 0x7f6c801d9170 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009290 tx=0x7f6c74031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:31.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.610+0000 7f6c7a000640 1 -- 172.21.15.42:0/2682238021 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c74041070 con 0x7f6c8006cea0 2024-07-04T17:29:31.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.610+0000 7f6c7a000640 1 -- 172.21.15.42:0/2682238021 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c74003be0 con 0x7f6c8006cea0 2024-07-04T17:29:31.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.610+0000 7f6c7a000640 1 -- 172.21.15.42:0/2682238021 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c7403b440 con 0x7f6c8006cea0 2024-07-04T17:29:31.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.610+0000 7f6c88800640 1 -- 172.21.15.42:0/2682238021 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c801dc9d0 con 0x7f6c8006cea0 2024-07-04T17:29:31.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.610+0000 7f6c88800640 1 -- 172.21.15.42:0/2682238021 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c800af4d0 con 0x7f6c8006cea0 2024-07-04T17:29:31.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.611+0000 7f6c88800640 1 -- 172.21.15.42:0/2682238021 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c80063f70 con 0x7f6c8006cea0 2024-07-04T17:29:31.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.612+0000 7f6c7a000640 1 -- 172.21.15.42:0/2682238021 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f6c74044070 con 0x7f6c8006cea0 2024-07-04T17:29:31.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.612+0000 7f6c7a000640 1 --2- 172.21.15.42:0/2682238021 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f6c6403d4f0 0x7f6c6403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:31.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.612+0000 7f6c7a000640 1 -- 172.21.15.42:0/2682238021 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4303+0+0 (secure 0 0 0) 0x7f6c74073370 con 0x7f6c8006cea0 2024-07-04T17:29:31.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.613+0000 7f6c85400640 1 --2- 172.21.15.42:0/2682238021 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f6c6403d4f0 0x7f6c6403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:31.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.613+0000 7f6c85400640 1 --2- 172.21.15.42:0/2682238021 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f6c6403d4f0 0x7f6c6403fa00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6c70007fd0 tx=0x7f6c70007920 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:31.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.617+0000 7f6c7a000640 1 -- 172.21.15.42:0/2682238021 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f6c80063f70 con 0x7f6c8006cea0 2024-07-04T17:29:31.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.868+0000 7f6c88800640 1 -- 172.21.15.42:0/2682238021 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c800b3030 con 0x7f6c8006cea0 2024-07-04T17:29:31.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.869+0000 7f6c7a000640 1 -- 172.21.15.42:0/2682238021 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+7364 (secure 0 0 0) 0x7f6c740730d0 con 0x7f6c8006cea0 2024-07-04T17:29:31.871 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:29:31.871 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":41,"btime":"2024-07-04T17:29:28:895382+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/343791977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":343791977},{"type":"v1","addr":"172.21.15.42:6827","nonce":343791977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":4722,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/3172446281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":3172446281},{"type":"v1","addr":"172.21.15.42:6829","nonce":3172446281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":4727,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/1866310217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1866310217},{"type":"v1","addr":"172.21.15.42:6831","nonce":1866310217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:29:05.313061+0000","modified":"2024-07-04T17:29:08.370956+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4731},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4731":{"gid":4731,"name":"d","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6835/1327681666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1327681666},{"type":"v1","addr":"172.21.15.42:6835","nonce":1327681666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4731,"qdb_cluster":[4731]},"id":4},{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:29:25.424728+0000","modified":"2024-07-04T17:29:28.496007+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4728":{"gid":4728,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":9,"addr":"172.21.15.42:6833/1072723036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1072723036},{"type":"v1","addr":"172.21.15.42:6833","nonce":1072723036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4728,"qdb_cluster":[4728]},"id":5}]} 2024-07-04T17:29:31.871 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 41 2024-07-04T17:29:31.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.871+0000 7f6c88800640 1 -- 172.21.15.42:0/2682238021 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f6c6403d4f0 msgr2=0x7f6c6403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:31.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.871+0000 7f6c88800640 1 --2- 172.21.15.42:0/2682238021 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f6c6403d4f0 0x7f6c6403fa00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6c70007fd0 tx=0x7f6c70007920 comp rx=0 tx=0).stop 2024-07-04T17:29:31.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.871+0000 7f6c88800640 1 -- 172.21.15.42:0/2682238021 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c8006cea0 msgr2=0x7f6c801d9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:31.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.871+0000 7f6c88800640 1 --2- 172.21.15.42:0/2682238021 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c8006cea0 0x7f6c801d9170 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009290 tx=0x7f6c74031a90 comp rx=0 tx=0).stop 2024-07-04T17:29:31.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.872+0000 7f6c88800640 1 -- 172.21.15.42:0/2682238021 shutdown_connections 2024-07-04T17:29:31.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.872+0000 7f6c88800640 1 --2- 172.21.15.42:0/2682238021 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f6c6403d4f0 0x7f6c6403fa00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:31.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.872+0000 7f6c88800640 1 --2- 172.21.15.42:0/2682238021 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c8006cea0 0x7f6c801d9170 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:31.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.872+0000 7f6c88800640 1 -- 172.21.15.42:0/2682238021 >> 172.21.15.42:0/2682238021 conn(0x7f6c800bf700 msgr2=0x7f6c801490c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:31.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.872+0000 7f6c88800640 1 -- 172.21.15.42:0/2682238021 shutdown_connections 2024-07-04T17:29:31.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:31.872+0000 7f6c88800640 1 -- 172.21.15.42:0/2682238021 wait complete. 2024-07-04T17:29:32.402 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 41, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-04T17:29:25.424728+0000', 'modified': '2024-07-04T17:29:28.496007+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4728}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4728': {'gid': 4728, 'name': 'c', 'rank': 0, 'incarnation': 38, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.42:6833/1072723036', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6832', 'nonce': 1072723036}, {'type': 'v1', 'addr': '172.21.15.42:6833', 'nonce': 1072723036}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'backup_fs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4728, 'qdb_cluster': [4728]} 2024-07-04T17:29:32.403 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-04T17:29:32.403 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-07-04T17:29:32.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.704+0000 7fe77e000640 1 -- 172.21.15.42:0/156691742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7780b2d20 msgr2=0x7fe7780b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:32.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.704+0000 7fe77e000640 1 --2- 172.21.15.42:0/156691742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7780b2d20 0x7fe7780b01f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe7600097b0 tx=0x7fe76002a530 comp rx=0 tx=0).stop 2024-07-04T17:29:32.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.704+0000 7fe77e000640 1 -- 172.21.15.42:0/156691742 shutdown_connections 2024-07-04T17:29:32.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.704+0000 7fe77e000640 1 --2- 172.21.15.42:0/156691742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7780b2d20 0x7fe7780b01f0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:32.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.704+0000 7fe77e000640 1 -- 172.21.15.42:0/156691742 >> 172.21.15.42:0/156691742 conn(0x7fe7780bf700 msgr2=0x7fe7780af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:32.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.704+0000 7fe77e000640 1 -- 172.21.15.42:0/156691742 shutdown_connections 2024-07-04T17:29:32.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.704+0000 7fe77e000640 1 -- 172.21.15.42:0/156691742 wait complete. 2024-07-04T17:29:32.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.706+0000 7fe77e000640 1 Processor -- start 2024-07-04T17:29:32.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.706+0000 7fe77e000640 1 -- start start 2024-07-04T17:29:32.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.707+0000 7fe77e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7780b2d20 0x7fe7780aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:32.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.707+0000 7fe77e000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe77806cf50 con 0x7fe7780b2d20 2024-07-04T17:29:32.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.708+0000 7fe777400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7780b2d20 0x7fe7780aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:32.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.708+0000 7fe777400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7780b2d20 0x7fe7780aebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45388/0 (socket says 172.21.15.42:45388) 2024-07-04T17:29:32.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.708+0000 7fe777400640 1 -- 172.21.15.42:0/1818498135 learned_addr learned my addr 172.21.15.42:0/1818498135 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:32.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.708+0000 7fe777400640 1 -- 172.21.15.42:0/1818498135 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe778070ba0 con 0x7fe7780b2d20 2024-07-04T17:29:32.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.708+0000 7fe777400640 1 --2- 172.21.15.42:0/1818498135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7780b2d20 0x7fe7780aebb0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fe76800c3b0 tx=0x7fe76800c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:32.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.709+0000 7fe76f400640 1 -- 172.21.15.42:0/1818498135 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe768011780 con 0x7fe7780b2d20 2024-07-04T17:29:32.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.709+0000 7fe76f400640 1 -- 172.21.15.42:0/1818498135 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe768017070 con 0x7fe7780b2d20 2024-07-04T17:29:32.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.709+0000 7fe76f400640 1 -- 172.21.15.42:0/1818498135 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe768011c00 con 0x7fe7780b2d20 2024-07-04T17:29:32.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.709+0000 7fe77e000640 1 -- 172.21.15.42:0/1818498135 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7780b9950 con 0x7fe7780b2d20 2024-07-04T17:29:32.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.709+0000 7fe77e000640 1 -- 172.21.15.42:0/1818498135 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7780b9760 con 0x7fe7780b2d20 2024-07-04T17:29:32.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.710+0000 7fe76f400640 1 -- 172.21.15.42:0/1818498135 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7fe76801c460 con 0x7fe7780b2d20 2024-07-04T17:29:32.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.711+0000 7fe76f400640 1 --2- 172.21.15.42:0/1818498135 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fe75403d4f0 0x7fe75403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:32.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.711+0000 7fe76f400640 1 -- 172.21.15.42:0/1818498135 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4303+0+0 (secure 0 0 0) 0x7fe7680519b0 con 0x7fe7780b2d20 2024-07-04T17:29:32.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.711+0000 7fe76fe00640 1 --2- 172.21.15.42:0/1818498135 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fe75403d4f0 0x7fe75403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:32.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.712+0000 7fe76fe00640 1 --2- 172.21.15.42:0/1818498135 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fe75403d4f0 0x7fe75403fa00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe760034000 tx=0x7fe760004730 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:32.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.713+0000 7fe785023740 1 Processor -- start 2024-07-04T17:29:32.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.713+0000 7fe785023740 1 -- start start 2024-07-04T17:29:32.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.714+0000 7fe785023740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5568baa3e270 0x5568bade45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:32.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.714+0000 7fe785023740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5568bad555b0 con 0x5568baa3e270 2024-07-04T17:29:32.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.715+0000 7fe777400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5568baa3e270 0x5568bade45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:32.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.715+0000 7fe777400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5568baa3e270 0x5568bade45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45396/0 (socket says 172.21.15.42:45396) 2024-07-04T17:29:32.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.715+0000 7fe777400640 1 -- 172.21.15.42:0/1853198905 learned_addr learned my addr 172.21.15.42:0/1853198905 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:32.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.715+0000 7fe777400640 1 -- 172.21.15.42:0/1853198905 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568baaa9850 con 0x5568baa3e270 2024-07-04T17:29:32.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.715+0000 7fe777400640 1 --2- 172.21.15.42:0/1853198905 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5568baa3e270 0x5568bade45c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fe768050170 tx=0x7fe7680109a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec410901926c2368 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:32.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.716+0000 7fe767400640 1 -- 172.21.15.42:0/1853198905 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe76801cad0 con 0x5568baa3e270 2024-07-04T17:29:32.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.716+0000 7fe767400640 1 -- 172.21.15.42:0/1853198905 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe768019310 con 0x5568baa3e270 2024-07-04T17:29:32.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.716+0000 7fe767400640 1 -- 172.21.15.42:0/1853198905 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe76801cc70 con 0x5568baa3e270 2024-07-04T17:29:32.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.716+0000 7fe785023740 1 -- 172.21.15.42:0/1853198905 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5568baa3e270 msgr2=0x5568bade45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:32.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.716+0000 7fe785023740 1 --2- 172.21.15.42:0/1853198905 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5568baa3e270 0x5568bade45c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fe768050170 tx=0x7fe7680109a0 comp rx=0 tx=0).stop 2024-07-04T17:29:32.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.716+0000 7fe785023740 1 -- 172.21.15.42:0/1853198905 shutdown_connections 2024-07-04T17:29:32.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.716+0000 7fe785023740 1 --2- 172.21.15.42:0/1853198905 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5568baa3e270 0x5568bade45c0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:32.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.716+0000 7fe785023740 1 -- 172.21.15.42:0/1853198905 >> 172.21.15.42:0/1853198905 conn(0x5568baa323c0 msgr2=0x5568bade2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:32.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.717+0000 7fe785023740 1 -- 172.21.15.42:0/1853198905 shutdown_connections 2024-07-04T17:29:32.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.717+0000 7fe785023740 1 -- 172.21.15.42:0/1853198905 wait complete. 2024-07-04T17:29:32.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.717+0000 7fe785023740 1 -- start start 2024-07-04T17:29:32.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.718+0000 7fe785023740 1 Processor -- start 2024-07-04T17:29:32.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.719+0000 7fe785023740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5568baa3e270 0x5568badffa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:32.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.719+0000 7fe785023740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5568bad48b90 con 0x5568baa3e270 2024-07-04T17:29:32.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.719+0000 7fe785023740 10 client.0 _ll_register_callbacks cb 0x5568baced8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T17:29:32.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.719+0000 7fe777400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5568baa3e270 0x5568badffa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:32.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.719+0000 7fe777400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5568baa3e270 0x5568badffa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45400/0 (socket says 172.21.15.42:45400) 2024-07-04T17:29:32.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.719+0000 7fe777400640 1 -- 172.21.15.42:0/2941897856 learned_addr learned my addr 172.21.15.42:0/2941897856 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:32.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.720+0000 7fe777400640 1 -- 172.21.15.42:0/2941897856 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568bab206b0 con 0x5568baa3e270 2024-07-04T17:29:32.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.720+0000 7fe777400640 1 --2- 172.21.15.42:0/2941897856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5568baa3e270 0x5568badffa50 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe768040010 tx=0x7fe76801cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2263e44d0eb40bd5 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:32.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.720+0000 7fe75be00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-07-04T17:29:32.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.720+0000 7fe75be00640 1 -- 172.21.15.42:0/2941897856 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe768015120 con 0x5568baa3e270 2024-07-04T17:29:32.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.720+0000 7fe75be00640 1 -- 172.21.15.42:0/2941897856 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 4303+0+0 (secure 0 0 0) 0x7fe76801ac60 con 0x5568baa3e270 2024-07-04T17:29:32.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.720+0000 7fe785023740 1 -- 172.21.15.42:0/2941897856 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5568bace56c0 con 0x5568baa3e270 2024-07-04T17:29:32.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.720+0000 7fe75be00640 1 -- 172.21.15.42:0/2941897856 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe768032050 con 0x5568baa3e270 2024-07-04T17:29:32.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.720+0000 7fe785023740 10 client.4845 fetch_fsmap learned FSMap version 41 2024-07-04T17:29:32.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.720+0000 7fe785023740 1 -- 172.21.15.42:0/2941897856 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) -- 0x5568bae0bbe0 con 0x5568baa3e270 2024-07-04T17:29:32.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.721+0000 7fe75be00640 1 -- 172.21.15.42:0/2941897856 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5568bace56c0 con 0x5568baa3e270 2024-07-04T17:29:32.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.721+0000 7fe75be00640 1 -- 172.21.15.42:0/2941897856 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 41) ==== 4754+0+0 (secure 0 0 0) 0x7fe768035020 con 0x5568baa3e270 2024-07-04T17:29:32.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.721+0000 7fe785023740 10 client.4845 fetch_fsmap finished waiting for FSMap version 41 2024-07-04T17:29:32.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.721+0000 7fe785023740 10 client.4845 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-04T17:29:32.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.721+0000 7fe785023740 20 client.4845 populate_metadata read hostname 'smithi042' 2024-07-04T17:29:32.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.721+0000 7fe785023740 1 --2- 172.21.15.42:0/2941897856 >> [v2:172.21.15.42:6832/1072723036,v1:172.21.15.42:6833/1072723036] conn(0x5568baa153b0 0x5568bae0bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:32.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.721+0000 7fe785023740 4 client.4845 mds_command: new command op to 4728 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T17:29:32.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.721+0000 7fe785023740 1 -- 172.21.15.42:0/2941897856 --> [v2:172.21.15.42:6832/1072723036,v1:172.21.15.42:6833/1072723036] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568bad57500 con 0x5568baa153b0 2024-07-04T17:29:32.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.721+0000 7fe76fe00640 1 --2- 172.21.15.42:0/2941897856 >> [v2:172.21.15.42:6832/1072723036,v1:172.21.15.42:6833/1072723036] conn(0x5568baa153b0 0x5568bae0bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:32.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.722+0000 7fe76fe00640 1 --2- 172.21.15.42:0/2941897856 >> [v2:172.21.15.42:6832/1072723036,v1:172.21.15.42:6833/1072723036] conn(0x5568baa153b0 0x5568bae0bed0 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9fe9571ca13f90d server_cookie=8b348d83a19d8cd8 in_seq=0 out_seq=0 2024-07-04T17:29:32.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.722+0000 7fe75be00640 10 client.4845 ms_handle_connect on v2:172.21.15.42:6832/1072723036 2024-07-04T17:29:32.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.723+0000 7fe75be00640 1 -- 172.21.15.42:0/2941897856 <== mds.0 v2:172.21.15.42:6832/1072723036 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5568bad57500 con 0x5568baa153b0 2024-07-04T17:29:32.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.723+0000 7fe75be00640 10 client.4845 handle_command_reply: tid=0 2024-07-04T17:29:32.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.723+0000 7fe785023740 1 client.4845 shutdown 2024-07-04T17:29:32.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.723+0000 7fe785023740 1 -- 172.21.15.42:0/2941897856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5568baa3e270 msgr2=0x5568badffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:32.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.723+0000 7fe785023740 1 --2- 172.21.15.42:0/2941897856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5568baa3e270 0x5568badffa50 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe768040010 tx=0x7fe76801cd00 comp rx=0 tx=0).stop 2024-07-04T17:29:32.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.723+0000 7fe785023740 1 -- 172.21.15.42:0/2941897856 shutdown_connections 2024-07-04T17:29:32.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.723+0000 7fe785023740 1 --2- 172.21.15.42:0/2941897856 >> [v2:172.21.15.42:6832/1072723036,v1:172.21.15.42:6833/1072723036] conn(0x5568baa153b0 0x5568bae0bed0 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:32.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.723+0000 7fe785023740 1 --2- 172.21.15.42:0/2941897856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5568baa3e270 0x5568badffa50 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:32.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.723+0000 7fe785023740 1 -- 172.21.15.42:0/2941897856 >> 172.21.15.42:0/2941897856 conn(0x5568baa323c0 msgr2=0x5568bade4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:32.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.723+0000 7fe75be00640 0 client.4845 ms_handle_reset on v2:172.21.15.42:6832/1072723036 2024-07-04T17:29:32.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.723+0000 7fe785023740 1 -- 172.21.15.42:0/2941897856 shutdown_connections 2024-07-04T17:29:32.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.724+0000 7fe785023740 1 -- 172.21.15.42:0/2941897856 wait complete. 2024-07-04T17:29:32.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.724+0000 7fe785023740 20 client.4845 trim_cache size 0 max 16384 2024-07-04T17:29:32.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.745+0000 7fe785023740 1 Processor -- start 2024-07-04T17:29:32.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.745+0000 7fe785023740 1 -- start start 2024-07-04T17:29:32.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.746+0000 7fe785023740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5568baa1d590 0x5568bae2d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:32.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.746+0000 7fe785023740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe7780aa300 con 0x5568baa1d590 2024-07-04T17:29:32.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.747+0000 7fe777400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5568baa1d590 0x5568bae2d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:32.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.747+0000 7fe777400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5568baa1d590 0x5568bae2d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45402/0 (socket says 172.21.15.42:45402) 2024-07-04T17:29:32.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.747+0000 7fe777400640 1 -- 172.21.15.42:0/533493485 learned_addr learned my addr 172.21.15.42:0/533493485 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:32.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.747+0000 7fe777400640 1 -- 172.21.15.42:0/533493485 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568bacfac30 con 0x5568baa1d590 2024-07-04T17:29:32.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.747+0000 7fe777400640 1 --2- 172.21.15.42:0/533493485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5568baa1d590 0x5568bae2d150 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe76801b730 tx=0x7fe768011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb9d2d87a9d105b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:32.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.748+0000 7fe766a00640 1 -- 172.21.15.42:0/533493485 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe76803a030 con 0x5568baa1d590 2024-07-04T17:29:32.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.748+0000 7fe766a00640 1 -- 172.21.15.42:0/533493485 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe76802a070 con 0x5568baa1d590 2024-07-04T17:29:32.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.748+0000 7fe766a00640 1 -- 172.21.15.42:0/533493485 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe76801ac60 con 0x5568baa1d590 2024-07-04T17:29:32.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.748+0000 7fe785023740 1 -- 172.21.15.42:0/533493485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5568baa1d590 msgr2=0x5568bae2d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:32.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.748+0000 7fe785023740 1 --2- 172.21.15.42:0/533493485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5568baa1d590 0x5568bae2d150 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe76801b730 tx=0x7fe768011da0 comp rx=0 tx=0).stop 2024-07-04T17:29:32.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.749+0000 7fe785023740 1 -- 172.21.15.42:0/533493485 shutdown_connections 2024-07-04T17:29:32.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.749+0000 7fe785023740 1 --2- 172.21.15.42:0/533493485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5568baa1d590 0x5568bae2d150 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:32.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.749+0000 7fe785023740 1 -- 172.21.15.42:0/533493485 >> 172.21.15.42:0/533493485 conn(0x5568baa323c0 msgr2=0x5568bade2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:32.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.749+0000 7fe785023740 1 -- 172.21.15.42:0/533493485 shutdown_connections 2024-07-04T17:29:32.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.749+0000 7fe785023740 1 -- 172.21.15.42:0/533493485 wait complete. 2024-07-04T17:29:32.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.749+0000 7fe785023740 1 -- start start 2024-07-04T17:29:32.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.750+0000 7fe785023740 1 Processor -- start 2024-07-04T17:29:32.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.751+0000 7fe785023740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5568baa1d590 0x5568baddf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:32.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.751+0000 7fe785023740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5568bad44b50 con 0x5568baa1d590 2024-07-04T17:29:32.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.751+0000 7fe785023740 10 client.0 _ll_register_callbacks cb 0x7fe768010710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T17:29:32.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.751+0000 7fe777400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5568baa1d590 0x5568baddf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:32.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.751+0000 7fe777400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5568baa1d590 0x5568baddf9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45406/0 (socket says 172.21.15.42:45406) 2024-07-04T17:29:32.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.751+0000 7fe777400640 1 -- 172.21.15.42:0/2241761650 learned_addr learned my addr 172.21.15.42:0/2241761650 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:32.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.752+0000 7fe777400640 1 -- 172.21.15.42:0/2241761650 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568bae35a10 con 0x5568baa1d590 2024-07-04T17:29:32.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.752+0000 7fe777400640 1 --2- 172.21.15.42:0/2241761650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5568baa1d590 0x5568baddf9d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fe76801ca50 tx=0x7fe76800e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6e7ff6376d3628 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:32.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.752+0000 7fe75be00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-07-04T17:29:32.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.752+0000 7fe75be00640 1 -- 172.21.15.42:0/2241761650 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe76803a070 con 0x5568baa1d590 2024-07-04T17:29:32.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.752+0000 7fe75be00640 1 -- 172.21.15.42:0/2241761650 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 4303+0+0 (secure 0 0 0) 0x7fe76803b070 con 0x5568baa1d590 2024-07-04T17:29:32.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.752+0000 7fe785023740 1 -- 172.21.15.42:0/2241761650 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe76802a070 con 0x5568baa1d590 2024-07-04T17:29:32.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.753+0000 7fe75be00640 1 -- 172.21.15.42:0/2241761650 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe768010cb0 con 0x5568baa1d590 2024-07-04T17:29:32.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.753+0000 7fe785023740 10 client.4847 fetch_fsmap learned FSMap version 41 2024-07-04T17:29:32.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.753+0000 7fe785023740 1 -- 172.21.15.42:0/2241761650 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) -- 0x5568bae35a10 con 0x5568baa1d590 2024-07-04T17:29:32.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.753+0000 7fe75be00640 1 -- 172.21.15.42:0/2241761650 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe76802a070 con 0x5568baa1d590 2024-07-04T17:29:32.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.753+0000 7fe75be00640 1 -- 172.21.15.42:0/2241761650 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 41) ==== 4754+0+0 (secure 0 0 0) 0x7fe768035070 con 0x5568baa1d590 2024-07-04T17:29:32.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.753+0000 7fe785023740 10 client.4847 fetch_fsmap finished waiting for FSMap version 41 2024-07-04T17:29:32.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.753+0000 7fe785023740 10 client.4847 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-04T17:29:32.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.753+0000 7fe785023740 20 client.4847 populate_metadata read hostname 'smithi042' 2024-07-04T17:29:32.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.753+0000 7fe785023740 1 --2- 172.21.15.42:0/2241761650 >> [v2:172.21.15.42:6832/1072723036,v1:172.21.15.42:6833/1072723036] conn(0x5568ba9d2280 0x5568bae03680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:32.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.753+0000 7fe785023740 4 client.4847 mds_command: new command op to 4728 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-04T17:29:32.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.753+0000 7fe785023740 1 -- 172.21.15.42:0/2241761650 --> [v2:172.21.15.42:6832/1072723036,v1:172.21.15.42:6833/1072723036] -- command(tid 0: {"prefix": "status"}) -- 0x5568bad52fa0 con 0x5568ba9d2280 2024-07-04T17:29:32.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.753+0000 7fe76fe00640 1 --2- 172.21.15.42:0/2241761650 >> [v2:172.21.15.42:6832/1072723036,v1:172.21.15.42:6833/1072723036] conn(0x5568ba9d2280 0x5568bae03680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:32.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.754+0000 7fe76fe00640 1 --2- 172.21.15.42:0/2241761650 >> [v2:172.21.15.42:6832/1072723036,v1:172.21.15.42:6833/1072723036] conn(0x5568ba9d2280 0x5568bae03680 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b201664dabc3feae server_cookie=352d6b1d9cf28de in_seq=0 out_seq=0 2024-07-04T17:29:32.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.754+0000 7fe75be00640 10 client.4847 ms_handle_connect on v2:172.21.15.42:6832/1072723036 2024-07-04T17:29:32.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.754+0000 7fe75be00640 1 -- 172.21.15.42:0/2241761650 <== mds.0 v2:172.21.15.42:6832/1072723036 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x5568bad52fa0 con 0x5568ba9d2280 2024-07-04T17:29:32.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.754+0000 7fe75be00640 10 client.4847 handle_command_reply: tid=0 2024-07-04T17:29:32.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.754+0000 7fe785023740 1 client.4847 shutdown 2024-07-04T17:29:32.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.754+0000 7fe785023740 1 -- 172.21.15.42:0/2241761650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5568baa1d590 msgr2=0x5568baddf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:32.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.754+0000 7fe785023740 1 --2- 172.21.15.42:0/2241761650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5568baa1d590 0x5568baddf9d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fe76801ca50 tx=0x7fe76800e040 comp rx=0 tx=0).stop 2024-07-04T17:29:32.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.754+0000 7fe785023740 1 -- 172.21.15.42:0/2241761650 shutdown_connections 2024-07-04T17:29:32.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.755+0000 7fe785023740 1 --2- 172.21.15.42:0/2241761650 >> [v2:172.21.15.42:6832/1072723036,v1:172.21.15.42:6833/1072723036] conn(0x5568ba9d2280 0x5568bae03680 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:32.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.755+0000 7fe785023740 1 --2- 172.21.15.42:0/2241761650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5568baa1d590 0x5568baddf9d0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:32.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.755+0000 7fe785023740 1 -- 172.21.15.42:0/2241761650 >> 172.21.15.42:0/2241761650 conn(0x5568baa323c0 msgr2=0x5568bae2d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:32.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.755+0000 7fe75be00640 0 client.4847 ms_handle_reset on v2:172.21.15.42:6832/1072723036 2024-07-04T17:29:32.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.755+0000 7fe785023740 1 -- 172.21.15.42:0/2241761650 shutdown_connections 2024-07-04T17:29:32.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.755+0000 7fe785023740 1 -- 172.21.15.42:0/2241761650 wait complete. 2024-07-04T17:29:32.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.755+0000 7fe785023740 20 client.4847 trim_cache size 0 max 16384 2024-07-04T17:29:32.759 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:29:32.759 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "be4f300a-93ef-4489-b540-37ece39b0aac", 2024-07-04T17:29:32.759 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-07-04T17:29:32.759 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 4728, 2024-07-04T17:29:32.759 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-07-04T17:29:32.759 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-07-04T17:29:32.759 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "backup_fs", 2024-07-04T17:29:32.759 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 6.8750551519999998, 2024-07-04T17:29:32.760 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 41, 2024-07-04T17:29:32.760 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 54, 2024-07-04T17:29:32.760 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 54, 2024-07-04T17:29:32.760 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 38.210812965999999 2024-07-04T17:29:32.760 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:29:32.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.756+0000 7fe77e000640 1 -- 172.21.15.42:0/1818498135 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fe75403d4f0 msgr2=0x7fe75403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:32.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.756+0000 7fe77e000640 1 --2- 172.21.15.42:0/1818498135 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fe75403d4f0 0x7fe75403fa00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe760034000 tx=0x7fe760004730 comp rx=0 tx=0).stop 2024-07-04T17:29:32.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.756+0000 7fe77e000640 1 -- 172.21.15.42:0/1818498135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7780b2d20 msgr2=0x7fe7780aebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:32.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.756+0000 7fe77e000640 1 --2- 172.21.15.42:0/1818498135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7780b2d20 0x7fe7780aebb0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fe76800c3b0 tx=0x7fe76800c870 comp rx=0 tx=0).stop 2024-07-04T17:29:32.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.756+0000 7fe77e000640 1 -- 172.21.15.42:0/1818498135 shutdown_connections 2024-07-04T17:29:32.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.756+0000 7fe77e000640 1 --2- 172.21.15.42:0/1818498135 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7fe75403d4f0 0x7fe75403fa00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:32.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.756+0000 7fe77e000640 1 --2- 172.21.15.42:0/1818498135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7780b2d20 0x7fe7780aebb0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:32.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.756+0000 7fe77e000640 1 -- 172.21.15.42:0/1818498135 >> 172.21.15.42:0/1818498135 conn(0x7fe7780bf700 msgr2=0x7fe778138120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:32.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.756+0000 7fe77e000640 1 -- 172.21.15.42:0/1818498135 shutdown_connections 2024-07-04T17:29:32.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:32.756+0000 7fe77e000640 1 -- 172.21.15.42:0/1818498135 wait complete. 2024-07-04T17:29:32.782 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable mirroring 2024-07-04T17:29:33.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.074+0000 7f5e53e00640 1 -- 172.21.15.42:0/194245778 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e4c0b1d20 msgr2=0x7f5e4c0b2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:33.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.074+0000 7f5e53e00640 1 --2- 172.21.15.42:0/194245778 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e4c0b1d20 0x7f5e4c0b2630 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5e400097b0 tx=0x7f5e4002a540 comp rx=0 tx=0).stop 2024-07-04T17:29:33.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.074+0000 7f5e53e00640 1 -- 172.21.15.42:0/194245778 shutdown_connections 2024-07-04T17:29:33.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.074+0000 7f5e53e00640 1 --2- 172.21.15.42:0/194245778 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e4c0b1d20 0x7f5e4c0b2630 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:33.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.074+0000 7f5e53e00640 1 -- 172.21.15.42:0/194245778 >> 172.21.15.42:0/194245778 conn(0x7f5e4c06fe40 msgr2=0x7f5e4c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:33.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.074+0000 7f5e53e00640 1 -- 172.21.15.42:0/194245778 shutdown_connections 2024-07-04T17:29:33.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.075+0000 7f5e53e00640 1 -- 172.21.15.42:0/194245778 wait complete. 2024-07-04T17:29:33.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.077+0000 7f5e53e00640 1 Processor -- start 2024-07-04T17:29:33.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.077+0000 7f5e53e00640 1 -- start start 2024-07-04T17:29:33.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.079+0000 7f5e53e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e4c0b1d20 0x7f5e4c071360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:33.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.079+0000 7f5e53e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4c073a90 con 0x7f5e4c0b1d20 2024-07-04T17:29:33.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.079+0000 7f5e51400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e4c0b1d20 0x7f5e4c071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:33.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.079+0000 7f5e51400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e4c0b1d20 0x7f5e4c071360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45438/0 (socket says 172.21.15.42:45438) 2024-07-04T17:29:33.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.079+0000 7f5e51400640 1 -- 172.21.15.42:0/2393717730 learned_addr learned my addr 172.21.15.42:0/2393717730 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:33.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.080+0000 7f5e51400640 1 -- 172.21.15.42:0/2393717730 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e4c070d70 con 0x7f5e4c0b1d20 2024-07-04T17:29:33.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.080+0000 7f5e51400640 1 --2- 172.21.15.42:0/2393717730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e4c0b1d20 0x7f5e4c071360 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5e400098d0 tx=0x7f5e40033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:33.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.080+0000 7f5e49600640 1 -- 172.21.15.42:0/2393717730 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e40031bc0 con 0x7f5e4c0b1d20 2024-07-04T17:29:33.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.080+0000 7f5e49600640 1 -- 172.21.15.42:0/2393717730 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e40031e50 con 0x7f5e4c0b1d20 2024-07-04T17:29:33.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.080+0000 7f5e49600640 1 -- 172.21.15.42:0/2393717730 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e40033260 con 0x7f5e4c0b1d20 2024-07-04T17:29:33.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.080+0000 7f5e53e00640 1 -- 172.21.15.42:0/2393717730 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e4c1dcb50 con 0x7f5e4c0b1d20 2024-07-04T17:29:33.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.081+0000 7f5e53e00640 1 -- 172.21.15.42:0/2393717730 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e4c0709f0 con 0x7f5e4c0b1d20 2024-07-04T17:29:33.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.082+0000 7f5e53e00640 1 -- 172.21.15.42:0/2393717730 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e4c0af820 con 0x7f5e4c0b1d20 2024-07-04T17:29:33.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.082+0000 7f5e49600640 1 -- 172.21.15.42:0/2393717730 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) ==== 49615+0+0 (secure 0 0 0) 0x7f5e40033400 con 0x7f5e4c0b1d20 2024-07-04T17:29:33.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.083+0000 7f5e49600640 1 --2- 172.21.15.42:0/2393717730 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f5e2c03d4f0 0x7f5e2c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:33.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.083+0000 7f5e49600640 1 -- 172.21.15.42:0/2393717730 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4303+0+0 (secure 0 0 0) 0x7f5e400731f0 con 0x7f5e4c0b1d20 2024-07-04T17:29:33.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.083+0000 7f5e50a00640 1 --2- 172.21.15.42:0/2393717730 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f5e2c03d4f0 0x7f5e2c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:33.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.084+0000 7f5e50a00640 1 --2- 172.21.15.42:0/2393717730 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f5e2c03d4f0 0x7f5e2c03fa00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5e38001560 tx=0x7f5e38007a90 comp rx=0 tx=0).ready entity=mgr.4613 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:33.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.088+0000 7f5e49600640 1 -- 172.21.15.42:0/2393717730 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f5e4003f030 con 0x7f5e4c0b1d20 2024-07-04T17:29:33.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.352+0000 7f5e53e00640 1 -- 172.21.15.42:0/2393717730 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "mirroring"} v 0) -- 0x7f5e4c074060 con 0x7f5e4c0b1d20 2024-07-04T17:29:33.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.931+0000 7f5e49600640 1 -- 172.21.15.42:0/2393717730 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "mirroring"}]=0 v16) ==== 88+0+0 (secure 0 0 0) 0x7f5e4003a3a0 con 0x7f5e4c0b1d20 2024-07-04T17:29:33.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.936+0000 7f5e53e00640 1 -- 172.21.15.42:0/2393717730 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f5e2c03d4f0 msgr2=0x7f5e2c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:33.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.936+0000 7f5e53e00640 1 --2- 172.21.15.42:0/2393717730 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f5e2c03d4f0 0x7f5e2c03fa00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5e38001560 tx=0x7f5e38007a90 comp rx=0 tx=0).stop 2024-07-04T17:29:33.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.936+0000 7f5e53e00640 1 -- 172.21.15.42:0/2393717730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e4c0b1d20 msgr2=0x7f5e4c071360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:33.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.936+0000 7f5e53e00640 1 --2- 172.21.15.42:0/2393717730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e4c0b1d20 0x7f5e4c071360 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5e400098d0 tx=0x7f5e40033e40 comp rx=0 tx=0).stop 2024-07-04T17:29:33.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.938+0000 7f5e53e00640 1 -- 172.21.15.42:0/2393717730 shutdown_connections 2024-07-04T17:29:33.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.938+0000 7f5e53e00640 1 --2- 172.21.15.42:0/2393717730 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f5e2c03d4f0 0x7f5e2c03fa00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:33.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.938+0000 7f5e53e00640 1 --2- 172.21.15.42:0/2393717730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e4c0b1d20 0x7f5e4c071360 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:33.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.938+0000 7f5e53e00640 1 -- 172.21.15.42:0/2393717730 >> 172.21.15.42:0/2393717730 conn(0x7f5e4c06fe40 msgr2=0x7f5e4c1d7680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:33.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.939+0000 7f5e53e00640 1 -- 172.21.15.42:0/2393717730 shutdown_connections 2024-07-04T17:29:33.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:33.941+0000 7f5e53e00640 1 -- 172.21.15.42:0/2393717730 wait complete. 2024-07-04T17:29:33.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:29:33.972 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3) 2024-07-04T17:29:34.028 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:29:34.028 DEBUG:teuthology.orchestra.run.smithi042:counter dump for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok counter dump 2024-07-04T17:29:34.243 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:29:34.243 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker": [ 2024-07-04T17:29:34.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:34.244 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:29:34.244 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "0" 2024-07-04T17:29:34.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:34.244 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:34.244 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:29:34.244 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:29:34.244 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:34.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:34.244 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:34.244 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:29:34.244 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "1" 2024-07-04T17:29:34.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:34.244 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:34.244 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:29:34.245 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:29:34.245 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:34.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:34.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:34.245 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:29:34.245 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "2" 2024-07-04T17:29:34.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:34.245 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:34.245 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:29:34.245 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:29:34.245 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:34.245 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:34.245 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:34.245 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": [ 2024-07-04T17:29:34.245 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:34.246 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:34.246 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:34.246 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 38, 2024-07-04T17:29:34.246 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 4, 2024-07-04T17:29:34.246 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 146976, 2024-07-04T17:29:34.246 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 800, 2024-07-04T17:29:34.246 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:29:34.246 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:29:34.246 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.015510792, 2024-07-04T17:29:34.246 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.001625944, 2024-07-04T17:29:34.246 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.008582815, 2024-07-04T17:29:34.246 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.004681321, 2024-07-04T17:29:34.247 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:29:34.247 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 4, 2024-07-04T17:29:34.247 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.004367369, 2024-07-04T17:29:34.247 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.001091842 2024-07-04T17:29:34.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:34.247 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:29:34.247 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:29:34.247 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:29:34.247 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:29:34.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:34.247 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 146976, 2024-07-04T17:29:34.247 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 800 2024-07-04T17:29:34.247 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:34.247 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:34.247 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:34.248 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": [ 2024-07-04T17:29:34.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:34.248 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:34.248 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:34.248 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 3, 2024-07-04T17:29:34.248 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 42, 2024-07-04T17:29:34.248 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 432, 2024-07-04T17:29:34.248 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 251744, 2024-07-04T17:29:34.248 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 8, 2024-07-04T17:29:34.248 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 2, 2024-07-04T17:29:34.248 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.016354263, 2024-07-04T17:29:34.248 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.006518475, 2024-07-04T17:29:34.248 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.004935686, 2024-07-04T17:29:34.248 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:29:34.249 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:29:34.249 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 39, 2024-07-04T17:29:34.249 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.026347724, 2024-07-04T17:29:34.249 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000675582 2024-07-04T17:29:34.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:34.249 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:29:34.249 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:29:34.249 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:29:34.249 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:29:34.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:34.249 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 432, 2024-07-04T17:29:34.249 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 251744 2024-07-04T17:29:34.249 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:34.249 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:34.250 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:34.250 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": [ 2024-07-04T17:29:34.250 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:34.250 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:34.250 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:34.250 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 20, 2024-07-04T17:29:34.250 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 7, 2024-07-04T17:29:34.250 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 601232, 2024-07-04T17:29:34.250 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 1264, 2024-07-04T17:29:34.250 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:29:34.250 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:29:34.250 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.021876197, 2024-07-04T17:29:34.250 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.001719643, 2024-07-04T17:29:34.250 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.036085944, 2024-07-04T17:29:34.251 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:29:34.251 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:29:34.251 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 7, 2024-07-04T17:29:34.251 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.003249255, 2024-07-04T17:29:34.251 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000464179 2024-07-04T17:29:34.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:34.251 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:29:34.251 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:29:34.251 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:29:34.251 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:29:34.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:34.251 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 601232, 2024-07-04T17:29:34.251 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 1264 2024-07-04T17:29:34.251 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:34.251 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:34.252 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:34.252 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": [ 2024-07-04T17:29:34.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:34.252 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:34.252 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:34.252 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-07-04T17:29:34.252 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-07-04T17:29:34.252 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:34.252 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:34.252 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:34.252 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror": [ 2024-07-04T17:29:34.252 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:34.252 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:34.252 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:34.252 INFO:teuthology.orchestra.run.smithi042.stdout: "mirrored_filesystems": 0, 2024-07-04T17:29:34.253 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_enable_failures": 0 2024-07-04T17:29:34.253 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:34.253 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:34.253 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:34.253 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": [ 2024-07-04T17:29:34.253 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:34.253 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:34.253 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:34.253 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-07-04T17:29:34.253 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-07-04T17:29:34.253 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T17:29:34.253 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-07-04T17:29:34.253 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T17:29:34.253 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-07-04T17:29:34.254 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T17:29:34.254 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T17:29:34.254 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T17:29:34.254 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T17:29:34.254 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T17:29:34.254 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-07-04T17:29:34.254 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T17:29:34.254 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T17:29:34.254 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-07-04T17:29:34.254 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-07-04T17:29:34.254 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-07-04T17:29:34.254 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-07-04T17:29:34.254 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T17:29:34.254 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T17:29:34.255 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T17:29:34.255 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T17:29:34.255 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T17:29:34.255 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-07-04T17:29:34.255 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-07-04T17:29:34.255 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-07-04T17:29:34.255 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T17:29:34.255 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T17:29:34.255 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-07-04T17:29:34.255 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-07-04T17:29:34.255 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-07-04T17:29:34.255 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-07-04T17:29:34.255 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T17:29:34.255 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-07-04T17:29:34.255 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T17:29:34.256 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-07-04T17:29:34.256 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 148348, 2024-07-04T17:29:34.256 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 25, 2024-07-04T17:29:34.256 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-07-04T17:29:34.256 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-07-04T17:29:34.256 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-07-04T17:29:34.256 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-07-04T17:29:34.256 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-07-04T17:29:34.256 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-07-04T17:29:34.256 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-07-04T17:29:34.256 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-07-04T17:29:34.256 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 4592, 2024-07-04T17:29:34.256 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 37, 2024-07-04T17:29:34.256 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T17:29:34.256 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-07-04T17:29:34.257 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-07-04T17:29:34.257 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-07-04T17:29:34.257 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 0, 2024-07-04T17:29:34.257 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 0, 2024-07-04T17:29:34.257 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-07-04T17:29:34.257 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-07-04T17:29:34.257 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-07-04T17:29:34.257 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-07-04T17:29:34.257 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:34.257 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:34.257 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:34.257 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter": [ 2024-07-04T17:29:34.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:34.257 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:34.257 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:34.257 INFO:teuthology.orchestra.run.smithi042.stdout: "op_active": 0, 2024-07-04T17:29:34.258 INFO:teuthology.orchestra.run.smithi042.stdout: "op_laggy": 0, 2024-07-04T17:29:34.258 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send": 0, 2024-07-04T17:29:34.258 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send_bytes": 0, 2024-07-04T17:29:34.258 INFO:teuthology.orchestra.run.smithi042.stdout: "op_resend": 0, 2024-07-04T17:29:34.258 INFO:teuthology.orchestra.run.smithi042.stdout: "op_reply": 0, 2024-07-04T17:29:34.258 INFO:teuthology.orchestra.run.smithi042.stdout: "op_latency": { 2024-07-04T17:29:34.258 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:29:34.258 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:29:34.258 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:29:34.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:34.258 INFO:teuthology.orchestra.run.smithi042.stdout: "op_inflight": 0, 2024-07-04T17:29:34.258 INFO:teuthology.orchestra.run.smithi042.stdout: "oplen_avg": { 2024-07-04T17:29:34.258 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:29:34.258 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0 2024-07-04T17:29:34.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:34.259 INFO:teuthology.orchestra.run.smithi042.stdout: "op": 0, 2024-07-04T17:29:34.259 INFO:teuthology.orchestra.run.smithi042.stdout: "op_r": 0, 2024-07-04T17:29:34.259 INFO:teuthology.orchestra.run.smithi042.stdout: "op_w": 0, 2024-07-04T17:29:34.259 INFO:teuthology.orchestra.run.smithi042.stdout: "op_rmw": 0, 2024-07-04T17:29:34.259 INFO:teuthology.orchestra.run.smithi042.stdout: "op_pg": 0, 2024-07-04T17:29:34.259 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_stat": 0, 2024-07-04T17:29:34.259 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_create": 0, 2024-07-04T17:29:34.259 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_read": 0, 2024-07-04T17:29:34.259 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_write": 0, 2024-07-04T17:29:34.259 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writefull": 0, 2024-07-04T17:29:34.259 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writesame": 0, 2024-07-04T17:29:34.259 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_append": 0, 2024-07-04T17:29:34.259 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_zero": 0, 2024-07-04T17:29:34.260 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_truncate": 0, 2024-07-04T17:29:34.260 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_delete": 0, 2024-07-04T17:29:34.260 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_mapext": 0, 2024-07-04T17:29:34.260 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_sparse_read": 0, 2024-07-04T17:29:34.260 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_clonerange": 0, 2024-07-04T17:29:34.260 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_getxattr": 0, 2024-07-04T17:29:34.260 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_setxattr": 0, 2024-07-04T17:29:34.260 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_cmpxattr": 0, 2024-07-04T17:29:34.260 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_rmxattr": 0, 2024-07-04T17:29:34.260 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_resetxattrs": 0, 2024-07-04T17:29:34.260 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_call": 0, 2024-07-04T17:29:34.260 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_watch": 0, 2024-07-04T17:29:34.260 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_notify": 0, 2024-07-04T17:29:34.261 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T17:29:34.261 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls": 0, 2024-07-04T17:29:34.261 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls_filter": 0, 2024-07-04T17:29:34.261 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_other": 0, 2024-07-04T17:29:34.261 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_active": 0, 2024-07-04T17:29:34.261 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_send": 0, 2024-07-04T17:29:34.261 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_resend": 0, 2024-07-04T17:29:34.261 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_ping": 0, 2024-07-04T17:29:34.261 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_active": 0, 2024-07-04T17:29:34.261 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_send": 0, 2024-07-04T17:29:34.261 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_resend": 0, 2024-07-04T17:29:34.261 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_active": 0, 2024-07-04T17:29:34.261 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_send": 0, 2024-07-04T17:29:34.261 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_resend": 0, 2024-07-04T17:29:34.261 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_active": 0, 2024-07-04T17:29:34.262 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_send": 0, 2024-07-04T17:29:34.262 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_resend": 0, 2024-07-04T17:29:34.262 INFO:teuthology.orchestra.run.smithi042.stdout: "command_active": 0, 2024-07-04T17:29:34.262 INFO:teuthology.orchestra.run.smithi042.stdout: "command_send": 0, 2024-07-04T17:29:34.262 INFO:teuthology.orchestra.run.smithi042.stdout: "command_resend": 0, 2024-07-04T17:29:34.262 INFO:teuthology.orchestra.run.smithi042.stdout: "map_epoch": 0, 2024-07-04T17:29:34.262 INFO:teuthology.orchestra.run.smithi042.stdout: "map_full": 0, 2024-07-04T17:29:34.262 INFO:teuthology.orchestra.run.smithi042.stdout: "map_inc": 0, 2024-07-04T17:29:34.262 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_sessions": 0, 2024-07-04T17:29:34.263 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_open": 0, 2024-07-04T17:29:34.263 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_close": 0, 2024-07-04T17:29:34.263 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_laggy": 0, 2024-07-04T17:29:34.263 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_wr": 0, 2024-07-04T17:29:34.263 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_rd": 0, 2024-07-04T17:29:34.263 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_del": 0 2024-07-04T17:29:34.263 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:34.263 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:34.263 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:34.263 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-client": [ 2024-07-04T17:29:34.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:34.263 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:34.263 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:34.263 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:29:34.263 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:29:34.263 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:29:34.264 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 38, 2024-07-04T17:29:34.264 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 141798, 2024-07-04T17:29:34.264 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:29:34.264 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 38, 2024-07-04T17:29:34.264 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:29:34.264 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:29:34.264 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 38, 2024-07-04T17:29:34.264 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 141798, 2024-07-04T17:29:34.264 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:29:34.264 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:29:34.264 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:29:34.264 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:29:34.264 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:34.264 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:34.265 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:34.265 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:34.265 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-radosclient": [ 2024-07-04T17:29:34.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:34.265 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:34.265 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:34.265 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:29:34.265 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:29:34.265 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:29:34.265 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 20, 2024-07-04T17:29:34.265 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 598036, 2024-07-04T17:29:34.265 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:29:34.265 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 20, 2024-07-04T17:29:34.265 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:29:34.265 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:29:34.266 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 20, 2024-07-04T17:29:34.266 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 598036, 2024-07-04T17:29:34.266 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:29:34.266 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:29:34.266 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:29:34.266 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:29:34.266 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:34.266 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:34.266 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:34.266 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:34.266 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_bytes": [ 2024-07-04T17:29:34.266 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:34.266 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:34.266 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:34.267 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:29:34.267 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:29:34.267 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:29:34.267 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:29:34.267 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:29:34.267 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:29:34.267 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:29:34.267 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:29:34.267 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:29:34.267 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:29:34.267 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:29:34.267 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:29:34.267 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:29:34.267 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:29:34.267 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:29:34.268 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:34.268 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:34.268 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:34.268 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:34.268 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_ops": [ 2024-07-04T17:29:34.268 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:34.268 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:34.268 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:34.268 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:29:34.268 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1024, 2024-07-04T17:29:34.268 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:29:34.268 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:29:34.268 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:29:34.268 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:29:34.268 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:29:34.268 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:29:34.269 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:29:34.269 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:29:34.269 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:29:34.269 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:29:34.269 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:29:34.269 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:29:34.269 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:29:34.269 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:34.269 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:34.269 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:34.269 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-07-04T17:29:34.269 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:29:34.270 DEBUG:tasks.cephfs.test_mirroring:command returned={ "AsyncMessenger::Worker": [ { "labels": { "id": "0" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "1" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "2" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } } ], "AsyncMessenger::Worker-0": [ { "labels": {}, "counters": { "msgr_recv_messages": 38, "msgr_send_messages": 4, "msgr_recv_bytes": 146976, "msgr_send_bytes": 800, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.015510792, "msgr_running_send_time": 0.001625944, "msgr_running_recv_time": 0.008582815, "msgr_running_fast_dispatch_time": 0.004681321, "msgr_send_messages_queue_lat": { "avgcount": 4, "sum": 0.004367369, "avgtime": 0.001091842 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 146976, "msgr_send_encrypted_bytes": 800 } } ], "AsyncMessenger::Worker-1": [ { "labels": {}, "counters": { "msgr_recv_messages": 3, "msgr_send_messages": 42, "msgr_recv_bytes": 432, "msgr_send_bytes": 251744, "msgr_created_connections": 8, "msgr_active_connections": 2, "msgr_running_total_time": 0.016354263, "msgr_running_send_time": 0.006518475, "msgr_running_recv_time": 0.004935686, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 39, "sum": 0.026347724, "avgtime": 0.000675582 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 432, "msgr_send_encrypted_bytes": 251744 } } ], "AsyncMessenger::Worker-2": [ { "labels": {}, "counters": { "msgr_recv_messages": 20, "msgr_send_messages": 7, "msgr_recv_bytes": 601232, "msgr_send_bytes": 1264, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.021876197, "msgr_running_send_time": 0.001719643, "msgr_running_recv_time": 0.036085944, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 7, "sum": 0.003249255, "avgtime": 0.000464179 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 601232, "msgr_send_encrypted_bytes": 1264 } } ], "cct": [ { "labels": {}, "counters": { "total_workers": 0, "unhealthy_workers": 0 } } ], "cephfs_mirror": [ { "labels": {}, "counters": { "mirrored_filesystems": 0, "mirror_enable_failures": 0 } } ], "mempool": [ { "labels": {}, "counters": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_items": 0, "bluefs_bytes": 0, "bluefs_items": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "buffer_anon_bytes": 148348, "buffer_anon_items": 25, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 4592, "osdmap_items": 37, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "mds_co_bytes": 0, "mds_co_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 } } ], "objecter": [ { "labels": {}, "counters": { "op_active": 0, "op_laggy": 0, "op_send": 0, "op_send_bytes": 0, "op_resend": 0, "op_reply": 0, "op_latency": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "op_inflight": 0, "oplen_avg": { "avgcount": 0, "sum": 0 }, "op": 0, "op_r": 0, "op_w": 0, "op_rmw": 0, "op_pg": 0, "osdop_stat": 0, "osdop_create": 0, "osdop_read": 0, "osdop_write": 0, "osdop_writefull": 0, "osdop_writesame": 0, "osdop_append": 0, "osdop_zero": 0, "osdop_truncate": 0, "osdop_delete": 0, "osdop_mapext": 0, "osdop_sparse_read": 0, "osdop_clonerange": 0, "osdop_getxattr": 0, "osdop_setxattr": 0, "osdop_cmpxattr": 0, "osdop_rmxattr": 0, "osdop_resetxattrs": 0, "osdop_call": 0, "osdop_watch": 0, "osdop_notify": 0, "osdop_src_cmpxattr": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_other": 0, "linger_active": 0, "linger_send": 0, "linger_resend": 0, "linger_ping": 0, "poolop_active": 0, "poolop_send": 0, "poolop_resend": 0, "poolstat_active": 0, "poolstat_send": 0, "poolstat_resend": 0, "statfs_active": 0, "statfs_send": 0, "statfs_resend": 0, "command_active": 0, "command_send": 0, "command_resend": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "osd_sessions": 0, "osd_session_open": 0, "osd_session_close": 0, "osd_laggy": 0, "omap_wr": 0, "omap_rd": 0, "omap_del": 0 } } ], "throttle-msgr_dispatch_throttler-client": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 38, "get_sum": 141798, "get_or_fail_fail": 0, "get_or_fail_success": 38, "take": 0, "take_sum": 0, "put": 38, "put_sum": 141798, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-msgr_dispatch_throttler-radosclient": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 20, "get_sum": 598036, "get_or_fail_fail": 0, "get_or_fail_success": 20, "take": 0, "take_sum": 0, "put": 20, "put_sum": 598036, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_bytes": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_ops": [ { "labels": {}, "counters": { "val": 0, "max": 1024, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ] } 2024-07-04T17:29:34.271 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snapshot mirror enable cephfs 2024-07-04T17:29:34.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:34.548+0000 7f245d800640 1 -- 172.21.15.42:0/2600647768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2458073d10 msgr2=0x7f24580aa680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:34.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:34.548+0000 7f245d800640 1 --2- 172.21.15.42:0/2600647768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2458073d10 0x7f24580aa680 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2448005fd0 tx=0x7f244802a5f0 comp rx=0 tx=0).stop 2024-07-04T17:29:34.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:34.548+0000 7f245d800640 1 -- 172.21.15.42:0/2600647768 shutdown_connections 2024-07-04T17:29:34.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:34.548+0000 7f245d800640 1 --2- 172.21.15.42:0/2600647768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2458073d10 0x7f24580aa680 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:34.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:34.548+0000 7f245d800640 1 -- 172.21.15.42:0/2600647768 >> 172.21.15.42:0/2600647768 conn(0x7f24580b3410 msgr2=0x7f24580b3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:34.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:34.548+0000 7f245d800640 1 -- 172.21.15.42:0/2600647768 shutdown_connections 2024-07-04T17:29:34.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:34.548+0000 7f245d800640 1 -- 172.21.15.42:0/2600647768 wait complete. 2024-07-04T17:29:34.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:34.550+0000 7f245d800640 1 Processor -- start 2024-07-04T17:29:34.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:34.550+0000 7f245d800640 1 -- start start 2024-07-04T17:29:34.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:34.551+0000 7f245d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2458073d10 0x7f24580ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:34.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:34.551+0000 7f245d800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f24580a9b00 con 0x7f2458073d10 2024-07-04T17:29:34.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:34.552+0000 7f2456a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2458073d10 0x7f24580ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:34.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:34.552+0000 7f2456a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2458073d10 0x7f24580ac8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56698/0 (socket says 172.21.15.42:56698) 2024-07-04T17:29:34.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:34.552+0000 7f2456a00640 1 -- 172.21.15.42:0/2457201001 learned_addr learned my addr 172.21.15.42:0/2457201001 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:34.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:34.552+0000 7f2456a00640 1 -- 172.21.15.42:0/2457201001 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24580adfc0 con 0x7f2458073d10 2024-07-04T17:29:34.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:34.552+0000 7f2456a00640 1 --2- 172.21.15.42:0/2457201001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2458073d10 0x7f24580ac8c0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f2448009850 tx=0x7f24480049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:34.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:34.553+0000 7f244ea00640 1 -- 172.21.15.42:0/2457201001 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f244803e030 con 0x7f2458073d10 2024-07-04T17:29:34.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:34.553+0000 7f244ea00640 1 -- 172.21.15.42:0/2457201001 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24480035f0 con 0x7f2458073d10 2024-07-04T17:29:34.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:34.553+0000 7f245d800640 1 -- 172.21.15.42:0/2457201001 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24581e21a0 con 0x7f2458073d10 2024-07-04T17:29:34.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:34.553+0000 7f244ea00640 1 -- 172.21.15.42:0/2457201001 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2448035040 con 0x7f2458073d10 2024-07-04T17:29:34.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:34.553+0000 7f245d800640 1 -- 172.21.15.42:0/2457201001 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24580ae4e0 con 0x7f2458073d10 2024-07-04T17:29:34.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:34.554+0000 7f245d800640 1 -- 172.21.15.42:0/2457201001 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24580aa680 con 0x7f2458073d10 2024-07-04T17:29:34.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:34.554+0000 7f244ea00640 1 -- 172.21.15.42:0/2457201001 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 49628+0+0 (secure 0 0 0) 0x7f2448041030 con 0x7f2458073d10 2024-07-04T17:29:34.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:34.555+0000 7f244ea00640 1 --2- 172.21.15.42:0/2457201001 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f243803d540 0x7f243803fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:34.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:34.555+0000 7f244ea00640 1 -- 172.21.15.42:0/2457201001 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4303+0+0 (secure 0 0 0) 0x7f244803f070 con 0x7f2458073d10 2024-07-04T17:29:34.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:34.555+0000 7f2456000640 1 -- 172.21.15.42:0/2457201001 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f243803d540 msgr2=0x7f243803fa50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/2638271195 2024-07-04T17:29:34.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:34.555+0000 7f2456000640 1 --2- 172.21.15.42:0/2457201001 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f243803d540 0x7f243803fa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:29:34.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:34.560+0000 7f244ea00640 1 -- 172.21.15.42:0/2457201001 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f244804bd60 con 0x7f2458073d10 2024-07-04T17:29:34.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:34.757+0000 7f2456000640 1 -- 172.21.15.42:0/2457201001 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f243803d540 msgr2=0x7f243803fa50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/2638271195 2024-07-04T17:29:34.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:34.757+0000 7f2456000640 1 --2- 172.21.15.42:0/2457201001 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f243803d540 0x7f243803fa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:29:34.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:34.787+0000 7f245d800640 1 -- 172.21.15.42:0/2457201001 --> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] -- mgr_command(tid 0: {"prefix": "fs snapshot mirror enable", "fs_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7f24580ad870 con 0x7f243803d540 2024-07-04T17:29:35.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:35.158+0000 7f2456000640 1 -- 172.21.15.42:0/2457201001 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f243803d540 msgr2=0x7f243803fa50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/2638271195 2024-07-04T17:29:35.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:35.158+0000 7f2456000640 1 --2- 172.21.15.42:0/2457201001 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f243803d540 0x7f243803fa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-07-04T17:29:35.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:35.960+0000 7f2456000640 1 -- 172.21.15.42:0/2457201001 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f243803d540 msgr2=0x7f243803fa50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/2638271195 2024-07-04T17:29:35.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:35.960+0000 7f2456000640 1 --2- 172.21.15.42:0/2457201001 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f243803d540 0x7f243803fa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-07-04T17:29:37.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:37.562+0000 7f2456000640 1 -- 172.21.15.42:0/2457201001 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f243803d540 msgr2=0x7f243803fa50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/2638271195 2024-07-04T17:29:37.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:37.562+0000 7f2456000640 1 --2- 172.21.15.42:0/2457201001 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f243803d540 0x7f243803fa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-07-04T17:29:38.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:38.583+0000 7f244ea00640 1 -- 172.21.15.42:0/2457201001 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 17) ==== 49135+0+0 (secure 0 0 0) 0x7f2448031400 con 0x7f2458073d10 2024-07-04T17:29:38.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:38.583+0000 7f244ea00640 1 -- 172.21.15.42:0/2457201001 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f243803d540 msgr2=0x7f243803fa50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:29:38.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:38.583+0000 7f244ea00640 1 --2- 172.21.15.42:0/2457201001 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f243803d540 0x7f243803fa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:38.675 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:29:38.674+0000 7f6787800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:29:38.675 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:29:38.674+0000 7f6787800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:29:38.675 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:29:38.674+0000 7f6787800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:29:38.676 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:29:38.674+0000 7f6787800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:29:38.676 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:29:38.674+0000 7f6787800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:29:38.676 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:29:38.674+0000 7f6783c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:29:38.676 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:29:38.674+0000 7f6783c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:29:38.676 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:29:38.674+0000 7f6783c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:29:38.676 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:29:38.674+0000 7f6783c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:29:38.676 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:29:38.674+0000 7f6783c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:29:38.694 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:29:38.693+0000 7f6783c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:29:38.694 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:29:38.693+0000 7f6783c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:29:38.694 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:29:38.693+0000 7f6783c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:29:38.694 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:29:38.693+0000 7f6783c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:29:38.694 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:29:38.693+0000 7f6783c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:29:38.695 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:29:38.694+0000 7f6787800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:29:38.695 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:29:38.694+0000 7f6787800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:29:38.695 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:29:38.694+0000 7f6787800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:29:38.695 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:29:38.694+0000 7f6787800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:29:38.695 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:29:38.694+0000 7f6787800640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:29:39.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:39.590+0000 7f244ea00640 1 -- 172.21.15.42:0/2457201001 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mgrmap(e 18) ==== 49262+0+0 (secure 0 0 0) 0x7f2448039d00 con 0x7f2458073d10 2024-07-04T17:29:39.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:39.591+0000 7f244ea00640 1 --2- 172.21.15.42:0/2457201001 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f2438046b80 0x7f2438048fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:39.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:39.591+0000 7f244ea00640 1 -- 172.21.15.42:0/2457201001 --> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] -- mgr_command(tid 0: {"prefix": "fs snapshot mirror enable", "fs_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7f2448075e50 con 0x7f2438046b80 2024-07-04T17:29:39.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:39.595+0000 7f2456000640 1 --2- 172.21.15.42:0/2457201001 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f2438046b80 0x7f2438048fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:39.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:39.605+0000 7f2456000640 1 --2- 172.21.15.42:0/2457201001 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f2438046b80 0x7f2438048fa0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f24580b5e90 tx=0x7f2444007a10 comp rx=0 tx=0).ready entity=mgr.4853 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:40.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:40.591+0000 7f244ea00640 1 -- 172.21.15.42:0/2457201001 <== mgr.4853 v2:172.21.15.42:6824/3045931584 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f2448075e50 con 0x7f2438046b80 2024-07-04T17:29:40.593 INFO:teuthology.orchestra.run.smithi042.stdout:{} 2024-07-04T17:29:40.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:40.595+0000 7f245d800640 1 -- 172.21.15.42:0/2457201001 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f2438046b80 msgr2=0x7f2438048fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:40.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:40.596+0000 7f245d800640 1 --2- 172.21.15.42:0/2457201001 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f2438046b80 0x7f2438048fa0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f24580b5e90 tx=0x7f2444007a10 comp rx=0 tx=0).stop 2024-07-04T17:29:40.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:40.596+0000 7f245d800640 1 -- 172.21.15.42:0/2457201001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2458073d10 msgr2=0x7f24580ac8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:40.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:40.596+0000 7f245d800640 1 --2- 172.21.15.42:0/2457201001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2458073d10 0x7f24580ac8c0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f2448009850 tx=0x7f24480049d0 comp rx=0 tx=0).stop 2024-07-04T17:29:40.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:40.596+0000 7f245d800640 1 -- 172.21.15.42:0/2457201001 shutdown_connections 2024-07-04T17:29:40.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:40.596+0000 7f245d800640 1 --2- 172.21.15.42:0/2457201001 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f2438046b80 0x7f2438048fa0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:40.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:40.596+0000 7f245d800640 1 --2- 172.21.15.42:0/2457201001 >> [v2:172.21.15.42:6824/2638271195,v1:172.21.15.42:6825/2638271195] conn(0x7f243803d540 0x7f243803fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:40.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:40.596+0000 7f245d800640 1 --2- 172.21.15.42:0/2457201001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2458073d10 0x7f24580ac8c0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:40.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:40.596+0000 7f245d800640 1 -- 172.21.15.42:0/2457201001 >> 172.21.15.42:0/2457201001 conn(0x7f24580b3410 msgr2=0x7f2458140540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:40.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:40.596+0000 7f245d800640 1 -- 172.21.15.42:0/2457201001 shutdown_connections 2024-07-04T17:29:40.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:40.596+0000 7f245d800640 1 -- 172.21.15.42:0/2457201001 wait complete. 2024-07-04T17:29:50.633 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:29:50.633 DEBUG:teuthology.orchestra.run.smithi042:mirror status for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@4 2024-07-04T17:29:50.861 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:29:50.861 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_inst": "172.21.15.42:0/7072365", 2024-07-04T17:29:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: "peers": {}, 2024-07-04T17:29:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_dirs": { 2024-07-04T17:29:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_count": 0 2024-07-04T17:29:50.862 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:50.862 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:29:50.881 DEBUG:tasks.cephfs.test_mirroring:command returned={ "rados_inst": "172.21.15.42:0/7072365", "peers": {}, "snap_dirs": { "dir_count": 0 } } 2024-07-04T17:29:50.881 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:29:50.881 DEBUG:teuthology.orchestra.run.smithi042:counter dump for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok counter dump 2024-07-04T17:29:51.104 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:29:51.105 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker": [ 2024-07-04T17:29:51.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:51.105 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:29:51.105 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "0" 2024-07-04T17:29:51.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:51.105 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:51.105 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:29:51.105 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:29:51.105 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:51.105 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:51.105 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:29:51.106 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "1" 2024-07-04T17:29:51.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:51.106 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:51.106 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:29:51.106 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:29:51.106 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:51.106 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:51.106 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:29:51.106 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "2" 2024-07-04T17:29:51.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:51.106 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:51.106 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:29:51.107 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:29:51.107 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.107 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.107 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:51.107 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": [ 2024-07-04T17:29:51.107 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:51.107 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:51.107 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:51.107 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 39, 2024-07-04T17:29:51.107 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 4, 2024-07-04T17:29:51.107 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 151872, 2024-07-04T17:29:51.107 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 800, 2024-07-04T17:29:51.107 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:29:51.108 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:29:51.108 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.016258169, 2024-07-04T17:29:51.108 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.001786130, 2024-07-04T17:29:51.108 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.008875063, 2024-07-04T17:29:51.108 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.004893093, 2024-07-04T17:29:51.108 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:29:51.108 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 4, 2024-07-04T17:29:51.108 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.004367369, 2024-07-04T17:29:51.108 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.001091842 2024-07-04T17:29:51.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:51.108 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:29:51.108 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:29:51.108 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:29:51.108 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:29:51.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:51.109 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 151872, 2024-07-04T17:29:51.109 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 800 2024-07-04T17:29:51.109 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.109 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.109 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:51.109 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": [ 2024-07-04T17:29:51.109 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:51.109 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:51.109 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:51.109 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 4, 2024-07-04T17:29:51.109 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 46, 2024-07-04T17:29:51.109 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 576, 2024-07-04T17:29:51.109 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 332896, 2024-07-04T17:29:51.110 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 9, 2024-07-04T17:29:51.110 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 3, 2024-07-04T17:29:51.110 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.021189213, 2024-07-04T17:29:51.110 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.007532880, 2024-07-04T17:29:51.110 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.007379601, 2024-07-04T17:29:51.110 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:29:51.110 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:29:51.110 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 43, 2024-07-04T17:29:51.110 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.034969193, 2024-07-04T17:29:51.110 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000813237 2024-07-04T17:29:51.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:51.110 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:29:51.110 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:29:51.110 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:29:51.110 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:29:51.111 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:51.111 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 576, 2024-07-04T17:29:51.111 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 332896 2024-07-04T17:29:51.111 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.111 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.111 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:51.111 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": [ 2024-07-04T17:29:51.111 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:51.111 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:51.111 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:51.111 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 24, 2024-07-04T17:29:51.111 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 7, 2024-07-04T17:29:51.111 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 799472, 2024-07-04T17:29:51.111 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 1264, 2024-07-04T17:29:51.112 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:29:51.112 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:29:51.112 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.027859965, 2024-07-04T17:29:51.112 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.001903531, 2024-07-04T17:29:51.112 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.047236962, 2024-07-04T17:29:51.112 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:29:51.112 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:29:51.112 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 7, 2024-07-04T17:29:51.112 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.003249255, 2024-07-04T17:29:51.112 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000464179 2024-07-04T17:29:51.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:51.112 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:29:51.112 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:29:51.112 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:29:51.112 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:29:51.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:51.113 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 799472, 2024-07-04T17:29:51.113 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 1264 2024-07-04T17:29:51.113 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.113 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.113 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:51.113 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": [ 2024-07-04T17:29:51.113 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:51.113 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:51.113 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:51.113 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-07-04T17:29:51.113 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-07-04T17:29:51.113 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.114 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.114 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:51.114 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror": [ 2024-07-04T17:29:51.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:51.114 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:51.114 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:51.114 INFO:teuthology.orchestra.run.smithi042.stdout: "mirrored_filesystems": 1, 2024-07-04T17:29:51.114 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_enable_failures": 0 2024-07-04T17:29:51.114 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.114 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.114 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:51.114 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_mirrored_filesystems": [ 2024-07-04T17:29:51.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:51.115 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:29:51.115 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem": "cephfs" 2024-07-04T17:29:51.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:51.115 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:51.115 INFO:teuthology.orchestra.run.smithi042.stdout: "mirroring_peers": 0, 2024-07-04T17:29:51.115 INFO:teuthology.orchestra.run.smithi042.stdout: "directory_count": 0 2024-07-04T17:29:51.115 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.115 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.115 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:51.115 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": [ 2024-07-04T17:29:51.115 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:51.115 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:51.115 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:51.116 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-07-04T17:29:51.116 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-07-04T17:29:51.116 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T17:29:51.116 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-07-04T17:29:51.116 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T17:29:51.116 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-07-04T17:29:51.116 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T17:29:51.116 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T17:29:51.116 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T17:29:51.116 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T17:29:51.116 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T17:29:51.116 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-07-04T17:29:51.116 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T17:29:51.116 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T17:29:51.116 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-07-04T17:29:51.117 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-07-04T17:29:51.117 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-07-04T17:29:51.117 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-07-04T17:29:51.117 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T17:29:51.117 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T17:29:51.117 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T17:29:51.117 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T17:29:51.117 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T17:29:51.117 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-07-04T17:29:51.117 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-07-04T17:29:51.117 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-07-04T17:29:51.117 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T17:29:51.117 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T17:29:51.117 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-07-04T17:29:51.117 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-07-04T17:29:51.117 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-07-04T17:29:51.118 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-07-04T17:29:51.118 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T17:29:51.118 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-07-04T17:29:51.118 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T17:29:51.118 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-07-04T17:29:51.118 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 183529, 2024-07-04T17:29:51.118 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 56, 2024-07-04T17:29:51.118 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-07-04T17:29:51.118 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-07-04T17:29:51.121 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-07-04T17:29:51.121 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-07-04T17:29:51.121 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-07-04T17:29:51.121 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-07-04T17:29:51.121 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-07-04T17:29:51.122 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-07-04T17:29:51.122 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 18720, 2024-07-04T17:29:51.122 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 169, 2024-07-04T17:29:51.122 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T17:29:51.122 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-07-04T17:29:51.122 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-07-04T17:29:51.122 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-07-04T17:29:51.122 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 0, 2024-07-04T17:29:51.122 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 0, 2024-07-04T17:29:51.122 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-07-04T17:29:51.122 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-07-04T17:29:51.122 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-07-04T17:29:51.122 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-07-04T17:29:51.123 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.123 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.123 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:51.123 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter": [ 2024-07-04T17:29:51.123 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:51.123 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:51.123 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:51.123 INFO:teuthology.orchestra.run.smithi042.stdout: "op_active": 0, 2024-07-04T17:29:51.123 INFO:teuthology.orchestra.run.smithi042.stdout: "op_laggy": 0, 2024-07-04T17:29:51.123 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send": 0, 2024-07-04T17:29:51.123 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send_bytes": 0, 2024-07-04T17:29:51.123 INFO:teuthology.orchestra.run.smithi042.stdout: "op_resend": 0, 2024-07-04T17:29:51.123 INFO:teuthology.orchestra.run.smithi042.stdout: "op_reply": 0, 2024-07-04T17:29:51.123 INFO:teuthology.orchestra.run.smithi042.stdout: "op_latency": { 2024-07-04T17:29:51.123 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:29:51.124 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:29:51.124 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:29:51.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:51.124 INFO:teuthology.orchestra.run.smithi042.stdout: "op_inflight": 0, 2024-07-04T17:29:51.124 INFO:teuthology.orchestra.run.smithi042.stdout: "oplen_avg": { 2024-07-04T17:29:51.124 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:29:51.124 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0 2024-07-04T17:29:51.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:51.124 INFO:teuthology.orchestra.run.smithi042.stdout: "op": 0, 2024-07-04T17:29:51.124 INFO:teuthology.orchestra.run.smithi042.stdout: "op_r": 0, 2024-07-04T17:29:51.124 INFO:teuthology.orchestra.run.smithi042.stdout: "op_w": 0, 2024-07-04T17:29:51.124 INFO:teuthology.orchestra.run.smithi042.stdout: "op_rmw": 0, 2024-07-04T17:29:51.124 INFO:teuthology.orchestra.run.smithi042.stdout: "op_pg": 0, 2024-07-04T17:29:51.124 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_stat": 0, 2024-07-04T17:29:51.125 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_create": 0, 2024-07-04T17:29:51.125 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_read": 0, 2024-07-04T17:29:51.125 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_write": 0, 2024-07-04T17:29:51.125 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writefull": 0, 2024-07-04T17:29:51.125 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writesame": 0, 2024-07-04T17:29:51.125 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_append": 0, 2024-07-04T17:29:51.125 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_zero": 0, 2024-07-04T17:29:51.125 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_truncate": 0, 2024-07-04T17:29:51.125 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_delete": 0, 2024-07-04T17:29:51.125 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_mapext": 0, 2024-07-04T17:29:51.125 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_sparse_read": 0, 2024-07-04T17:29:51.125 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_clonerange": 0, 2024-07-04T17:29:51.125 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_getxattr": 0, 2024-07-04T17:29:51.125 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_setxattr": 0, 2024-07-04T17:29:51.126 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_cmpxattr": 0, 2024-07-04T17:29:51.126 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_rmxattr": 0, 2024-07-04T17:29:51.126 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_resetxattrs": 0, 2024-07-04T17:29:51.126 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_call": 0, 2024-07-04T17:29:51.126 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_watch": 0, 2024-07-04T17:29:51.126 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_notify": 0, 2024-07-04T17:29:51.126 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T17:29:51.126 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls": 0, 2024-07-04T17:29:51.126 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls_filter": 0, 2024-07-04T17:29:51.126 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_other": 0, 2024-07-04T17:29:51.126 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_active": 0, 2024-07-04T17:29:51.126 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_send": 0, 2024-07-04T17:29:51.126 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_resend": 0, 2024-07-04T17:29:51.126 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_ping": 0, 2024-07-04T17:29:51.126 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_active": 0, 2024-07-04T17:29:51.127 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_send": 0, 2024-07-04T17:29:51.127 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_resend": 0, 2024-07-04T17:29:51.127 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_active": 0, 2024-07-04T17:29:51.127 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_send": 0, 2024-07-04T17:29:51.127 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_resend": 0, 2024-07-04T17:29:51.127 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_active": 0, 2024-07-04T17:29:51.127 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_send": 0, 2024-07-04T17:29:51.127 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_resend": 0, 2024-07-04T17:29:51.127 INFO:teuthology.orchestra.run.smithi042.stdout: "command_active": 0, 2024-07-04T17:29:51.127 INFO:teuthology.orchestra.run.smithi042.stdout: "command_send": 0, 2024-07-04T17:29:51.127 INFO:teuthology.orchestra.run.smithi042.stdout: "command_resend": 0, 2024-07-04T17:29:51.127 INFO:teuthology.orchestra.run.smithi042.stdout: "map_epoch": 0, 2024-07-04T17:29:51.127 INFO:teuthology.orchestra.run.smithi042.stdout: "map_full": 0, 2024-07-04T17:29:51.127 INFO:teuthology.orchestra.run.smithi042.stdout: "map_inc": 0, 2024-07-04T17:29:51.128 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_sessions": 0, 2024-07-04T17:29:51.128 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_open": 0, 2024-07-04T17:29:51.128 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_close": 0, 2024-07-04T17:29:51.128 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_laggy": 0, 2024-07-04T17:29:51.128 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_wr": 0, 2024-07-04T17:29:51.128 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_rd": 0, 2024-07-04T17:29:51.128 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_del": 0 2024-07-04T17:29:51.128 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.128 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.128 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:51.128 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-client": [ 2024-07-04T17:29:51.128 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:51.128 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:51.128 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:51.128 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:29:51.128 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:29:51.129 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:29:51.129 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 39, 2024-07-04T17:29:51.129 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 146552, 2024-07-04T17:29:51.129 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:29:51.129 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 39, 2024-07-04T17:29:51.129 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:29:51.129 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:29:51.129 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 39, 2024-07-04T17:29:51.129 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 146552, 2024-07-04T17:29:51.129 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:29:51.129 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:29:51.129 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:29:51.129 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:29:51.129 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.129 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.129 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.130 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:51.130 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-radosclient": [ 2024-07-04T17:29:51.130 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:51.130 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:51.130 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:51.130 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:29:51.130 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:29:51.130 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:29:51.130 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 25, 2024-07-04T17:29:51.130 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 795752, 2024-07-04T17:29:51.130 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:29:51.130 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 25, 2024-07-04T17:29:51.130 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:29:51.130 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:29:51.131 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 25, 2024-07-04T17:29:51.131 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 795752, 2024-07-04T17:29:51.131 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:29:51.131 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:29:51.131 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:29:51.131 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:29:51.131 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.131 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.131 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.131 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:51.131 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_bytes": [ 2024-07-04T17:29:51.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:51.131 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:51.131 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:51.131 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:29:51.131 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:29:51.132 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:29:51.132 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:29:51.132 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:29:51.132 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:29:51.132 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:29:51.132 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:29:51.132 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:29:51.132 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:29:51.132 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:29:51.132 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:29:51.132 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:29:51.132 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:29:51.132 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:29:51.132 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.133 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.133 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.133 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:51.133 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_ops": [ 2024-07-04T17:29:51.133 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:51.133 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:51.133 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:51.133 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:29:51.133 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1024, 2024-07-04T17:29:51.133 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:29:51.133 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:29:51.133 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:29:51.133 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:29:51.133 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:29:51.134 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:29:51.134 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:29:51.134 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:29:51.134 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:29:51.134 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:29:51.134 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:29:51.134 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:29:51.134 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:29:51.134 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.134 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.134 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.134 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-07-04T17:29:51.134 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:29:51.135 DEBUG:tasks.cephfs.test_mirroring:command returned={ "AsyncMessenger::Worker": [ { "labels": { "id": "0" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "1" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "2" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } } ], "AsyncMessenger::Worker-0": [ { "labels": {}, "counters": { "msgr_recv_messages": 39, "msgr_send_messages": 4, "msgr_recv_bytes": 151872, "msgr_send_bytes": 800, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.016258169, "msgr_running_send_time": 0.001786130, "msgr_running_recv_time": 0.008875063, "msgr_running_fast_dispatch_time": 0.004893093, "msgr_send_messages_queue_lat": { "avgcount": 4, "sum": 0.004367369, "avgtime": 0.001091842 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 151872, "msgr_send_encrypted_bytes": 800 } } ], "AsyncMessenger::Worker-1": [ { "labels": {}, "counters": { "msgr_recv_messages": 4, "msgr_send_messages": 46, "msgr_recv_bytes": 576, "msgr_send_bytes": 332896, "msgr_created_connections": 9, "msgr_active_connections": 3, "msgr_running_total_time": 0.021189213, "msgr_running_send_time": 0.007532880, "msgr_running_recv_time": 0.007379601, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 43, "sum": 0.034969193, "avgtime": 0.000813237 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 576, "msgr_send_encrypted_bytes": 332896 } } ], "AsyncMessenger::Worker-2": [ { "labels": {}, "counters": { "msgr_recv_messages": 24, "msgr_send_messages": 7, "msgr_recv_bytes": 799472, "msgr_send_bytes": 1264, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.027859965, "msgr_running_send_time": 0.001903531, "msgr_running_recv_time": 0.047236962, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 7, "sum": 0.003249255, "avgtime": 0.000464179 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 799472, "msgr_send_encrypted_bytes": 1264 } } ], "cct": [ { "labels": {}, "counters": { "total_workers": 0, "unhealthy_workers": 0 } } ], "cephfs_mirror": [ { "labels": {}, "counters": { "mirrored_filesystems": 1, "mirror_enable_failures": 0 } } ], "cephfs_mirror_mirrored_filesystems": [ { "labels": { "filesystem": "cephfs" }, "counters": { "mirroring_peers": 0, "directory_count": 0 } } ], "mempool": [ { "labels": {}, "counters": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_items": 0, "bluefs_bytes": 0, "bluefs_items": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "buffer_anon_bytes": 183529, "buffer_anon_items": 56, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 18720, "osdmap_items": 169, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "mds_co_bytes": 0, "mds_co_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 } } ], "objecter": [ { "labels": {}, "counters": { "op_active": 0, "op_laggy": 0, "op_send": 0, "op_send_bytes": 0, "op_resend": 0, "op_reply": 0, "op_latency": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "op_inflight": 0, "oplen_avg": { "avgcount": 0, "sum": 0 }, "op": 0, "op_r": 0, "op_w": 0, "op_rmw": 0, "op_pg": 0, "osdop_stat": 0, "osdop_create": 0, "osdop_read": 0, "osdop_write": 0, "osdop_writefull": 0, "osdop_writesame": 0, "osdop_append": 0, "osdop_zero": 0, "osdop_truncate": 0, "osdop_delete": 0, "osdop_mapext": 0, "osdop_sparse_read": 0, "osdop_clonerange": 0, "osdop_getxattr": 0, "osdop_setxattr": 0, "osdop_cmpxattr": 0, "osdop_rmxattr": 0, "osdop_resetxattrs": 0, "osdop_call": 0, "osdop_watch": 0, "osdop_notify": 0, "osdop_src_cmpxattr": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_other": 0, "linger_active": 0, "linger_send": 0, "linger_resend": 0, "linger_ping": 0, "poolop_active": 0, "poolop_send": 0, "poolop_resend": 0, "poolstat_active": 0, "poolstat_send": 0, "poolstat_resend": 0, "statfs_active": 0, "statfs_send": 0, "statfs_resend": 0, "command_active": 0, "command_send": 0, "command_resend": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "osd_sessions": 0, "osd_session_open": 0, "osd_session_close": 0, "osd_laggy": 0, "omap_wr": 0, "omap_rd": 0, "omap_del": 0 } } ], "throttle-msgr_dispatch_throttler-client": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 39, "get_sum": 146552, "get_or_fail_fail": 0, "get_or_fail_success": 39, "take": 0, "take_sum": 0, "put": 39, "put_sum": 146552, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-msgr_dispatch_throttler-radosclient": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 25, "get_sum": 795752, "get_or_fail_fail": 0, "get_or_fail_success": 25, "take": 0, "take_sum": 0, "put": 25, "put_sum": 795752, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_bytes": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_ops": [ { "labels": {}, "counters": { "val": 0, "max": 1024, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ] } 2024-07-04T17:29:51.136 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:29:51.136 DEBUG:teuthology.orchestra.run.smithi042:counter dump for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok counter dump 2024-07-04T17:29:51.371 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:29:51.372 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker": [ 2024-07-04T17:29:51.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:51.372 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:29:51.372 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "0" 2024-07-04T17:29:51.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:51.372 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:51.372 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:29:51.372 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:29:51.372 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:51.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:51.373 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:29:51.373 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "1" 2024-07-04T17:29:51.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:51.373 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:51.373 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:29:51.373 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:29:51.373 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:51.373 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:51.373 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:29:51.373 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "2" 2024-07-04T17:29:51.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:51.373 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:51.373 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:29:51.373 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:29:51.374 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.374 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.374 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:51.374 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": [ 2024-07-04T17:29:51.374 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:51.374 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:51.374 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:51.374 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 39, 2024-07-04T17:29:51.374 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 4, 2024-07-04T17:29:51.374 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 151872, 2024-07-04T17:29:51.374 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 800, 2024-07-04T17:29:51.374 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:29:51.374 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:29:51.374 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.016258169, 2024-07-04T17:29:51.375 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.001786130, 2024-07-04T17:29:51.375 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.008875063, 2024-07-04T17:29:51.375 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.004893093, 2024-07-04T17:29:51.375 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:29:51.375 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 4, 2024-07-04T17:29:51.375 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.004367369, 2024-07-04T17:29:51.375 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.001091842 2024-07-04T17:29:51.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:51.375 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:29:51.375 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:29:51.375 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:29:51.375 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:29:51.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:51.375 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 151872, 2024-07-04T17:29:51.375 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 800 2024-07-04T17:29:51.376 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.376 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.376 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:51.376 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": [ 2024-07-04T17:29:51.376 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:51.376 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:51.376 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:51.376 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 4, 2024-07-04T17:29:51.376 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 46, 2024-07-04T17:29:51.376 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 576, 2024-07-04T17:29:51.376 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 332896, 2024-07-04T17:29:51.376 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 9, 2024-07-04T17:29:51.376 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 3, 2024-07-04T17:29:51.377 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.021189213, 2024-07-04T17:29:51.377 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.007532880, 2024-07-04T17:29:51.377 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.007379601, 2024-07-04T17:29:51.377 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:29:51.377 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:29:51.377 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 43, 2024-07-04T17:29:51.377 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.034969193, 2024-07-04T17:29:51.377 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000813237 2024-07-04T17:29:51.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:51.377 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:29:51.377 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:29:51.377 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:29:51.377 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:29:51.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:51.377 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 576, 2024-07-04T17:29:51.378 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 332896 2024-07-04T17:29:51.378 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.378 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.378 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:51.378 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": [ 2024-07-04T17:29:51.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:51.378 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:51.378 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:51.378 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 24, 2024-07-04T17:29:51.378 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 7, 2024-07-04T17:29:51.378 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 799472, 2024-07-04T17:29:51.378 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 1264, 2024-07-04T17:29:51.378 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:29:51.379 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:29:51.379 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.027859965, 2024-07-04T17:29:51.379 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.001903531, 2024-07-04T17:29:51.379 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.047236962, 2024-07-04T17:29:51.379 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:29:51.379 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:29:51.379 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 7, 2024-07-04T17:29:51.379 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.003249255, 2024-07-04T17:29:51.379 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000464179 2024-07-04T17:29:51.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:51.379 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:29:51.380 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:29:51.380 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:29:51.380 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:29:51.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:51.380 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 799472, 2024-07-04T17:29:51.380 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 1264 2024-07-04T17:29:51.380 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.380 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.380 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:51.380 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": [ 2024-07-04T17:29:51.380 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:51.380 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:51.380 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:51.380 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-07-04T17:29:51.380 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-07-04T17:29:51.381 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.381 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.381 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:51.381 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror": [ 2024-07-04T17:29:51.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:51.381 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:51.381 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:51.381 INFO:teuthology.orchestra.run.smithi042.stdout: "mirrored_filesystems": 1, 2024-07-04T17:29:51.381 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_enable_failures": 0 2024-07-04T17:29:51.381 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.381 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.381 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:51.381 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_mirrored_filesystems": [ 2024-07-04T17:29:51.381 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:51.382 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:29:51.382 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem": "cephfs" 2024-07-04T17:29:51.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:51.382 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:51.382 INFO:teuthology.orchestra.run.smithi042.stdout: "mirroring_peers": 0, 2024-07-04T17:29:51.382 INFO:teuthology.orchestra.run.smithi042.stdout: "directory_count": 0 2024-07-04T17:29:51.382 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.382 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.382 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:51.382 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": [ 2024-07-04T17:29:51.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:51.382 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:51.382 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:51.382 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-07-04T17:29:51.383 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-07-04T17:29:51.383 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T17:29:51.383 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-07-04T17:29:51.383 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T17:29:51.383 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-07-04T17:29:51.383 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T17:29:51.383 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T17:29:51.383 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T17:29:51.383 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T17:29:51.383 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T17:29:51.383 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-07-04T17:29:51.383 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T17:29:51.383 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T17:29:51.384 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-07-04T17:29:51.384 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-07-04T17:29:51.384 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-07-04T17:29:51.384 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-07-04T17:29:51.384 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T17:29:51.384 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T17:29:51.384 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T17:29:51.384 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T17:29:51.384 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T17:29:51.384 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-07-04T17:29:51.384 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-07-04T17:29:51.384 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-07-04T17:29:51.384 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T17:29:51.384 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T17:29:51.385 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-07-04T17:29:51.385 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-07-04T17:29:51.385 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-07-04T17:29:51.385 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-07-04T17:29:51.385 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T17:29:51.385 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-07-04T17:29:51.385 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T17:29:51.385 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-07-04T17:29:51.385 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 183529, 2024-07-04T17:29:51.385 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 56, 2024-07-04T17:29:51.385 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-07-04T17:29:51.385 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-07-04T17:29:51.385 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-07-04T17:29:51.386 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-07-04T17:29:51.386 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-07-04T17:29:51.386 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-07-04T17:29:51.386 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-07-04T17:29:51.386 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-07-04T17:29:51.386 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 18720, 2024-07-04T17:29:51.386 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 169, 2024-07-04T17:29:51.386 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T17:29:51.386 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-07-04T17:29:51.386 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-07-04T17:29:51.386 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-07-04T17:29:51.386 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 0, 2024-07-04T17:29:51.386 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 0, 2024-07-04T17:29:51.387 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-07-04T17:29:51.387 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-07-04T17:29:51.387 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-07-04T17:29:51.387 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-07-04T17:29:51.387 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.387 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.387 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:51.387 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter": [ 2024-07-04T17:29:51.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:51.387 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:51.387 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:51.387 INFO:teuthology.orchestra.run.smithi042.stdout: "op_active": 0, 2024-07-04T17:29:51.387 INFO:teuthology.orchestra.run.smithi042.stdout: "op_laggy": 0, 2024-07-04T17:29:51.387 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send": 0, 2024-07-04T17:29:51.387 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send_bytes": 0, 2024-07-04T17:29:51.388 INFO:teuthology.orchestra.run.smithi042.stdout: "op_resend": 0, 2024-07-04T17:29:51.388 INFO:teuthology.orchestra.run.smithi042.stdout: "op_reply": 0, 2024-07-04T17:29:51.388 INFO:teuthology.orchestra.run.smithi042.stdout: "op_latency": { 2024-07-04T17:29:51.388 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:29:51.388 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:29:51.388 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:29:51.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:51.388 INFO:teuthology.orchestra.run.smithi042.stdout: "op_inflight": 0, 2024-07-04T17:29:51.388 INFO:teuthology.orchestra.run.smithi042.stdout: "oplen_avg": { 2024-07-04T17:29:51.388 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:29:51.388 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0 2024-07-04T17:29:51.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:29:51.388 INFO:teuthology.orchestra.run.smithi042.stdout: "op": 0, 2024-07-04T17:29:51.388 INFO:teuthology.orchestra.run.smithi042.stdout: "op_r": 0, 2024-07-04T17:29:51.388 INFO:teuthology.orchestra.run.smithi042.stdout: "op_w": 0, 2024-07-04T17:29:51.389 INFO:teuthology.orchestra.run.smithi042.stdout: "op_rmw": 0, 2024-07-04T17:29:51.389 INFO:teuthology.orchestra.run.smithi042.stdout: "op_pg": 0, 2024-07-04T17:29:51.389 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_stat": 0, 2024-07-04T17:29:51.389 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_create": 0, 2024-07-04T17:29:51.389 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_read": 0, 2024-07-04T17:29:51.389 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_write": 0, 2024-07-04T17:29:51.389 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writefull": 0, 2024-07-04T17:29:51.389 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writesame": 0, 2024-07-04T17:29:51.389 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_append": 0, 2024-07-04T17:29:51.389 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_zero": 0, 2024-07-04T17:29:51.389 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_truncate": 0, 2024-07-04T17:29:51.389 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_delete": 0, 2024-07-04T17:29:51.389 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_mapext": 0, 2024-07-04T17:29:51.389 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_sparse_read": 0, 2024-07-04T17:29:51.389 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_clonerange": 0, 2024-07-04T17:29:51.390 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_getxattr": 0, 2024-07-04T17:29:51.390 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_setxattr": 0, 2024-07-04T17:29:51.390 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_cmpxattr": 0, 2024-07-04T17:29:51.390 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_rmxattr": 0, 2024-07-04T17:29:51.390 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_resetxattrs": 0, 2024-07-04T17:29:51.390 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_call": 0, 2024-07-04T17:29:51.390 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_watch": 0, 2024-07-04T17:29:51.390 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_notify": 0, 2024-07-04T17:29:51.390 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T17:29:51.390 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls": 0, 2024-07-04T17:29:51.390 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls_filter": 0, 2024-07-04T17:29:51.390 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_other": 0, 2024-07-04T17:29:51.390 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_active": 0, 2024-07-04T17:29:51.390 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_send": 0, 2024-07-04T17:29:51.391 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_resend": 0, 2024-07-04T17:29:51.391 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_ping": 0, 2024-07-04T17:29:51.391 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_active": 0, 2024-07-04T17:29:51.391 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_send": 0, 2024-07-04T17:29:51.391 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_resend": 0, 2024-07-04T17:29:51.391 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_active": 0, 2024-07-04T17:29:51.391 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_send": 0, 2024-07-04T17:29:51.391 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_resend": 0, 2024-07-04T17:29:51.391 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_active": 0, 2024-07-04T17:29:51.391 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_send": 0, 2024-07-04T17:29:51.391 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_resend": 0, 2024-07-04T17:29:51.391 INFO:teuthology.orchestra.run.smithi042.stdout: "command_active": 0, 2024-07-04T17:29:51.391 INFO:teuthology.orchestra.run.smithi042.stdout: "command_send": 0, 2024-07-04T17:29:51.391 INFO:teuthology.orchestra.run.smithi042.stdout: "command_resend": 0, 2024-07-04T17:29:51.391 INFO:teuthology.orchestra.run.smithi042.stdout: "map_epoch": 0, 2024-07-04T17:29:51.391 INFO:teuthology.orchestra.run.smithi042.stdout: "map_full": 0, 2024-07-04T17:29:51.392 INFO:teuthology.orchestra.run.smithi042.stdout: "map_inc": 0, 2024-07-04T17:29:51.392 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_sessions": 0, 2024-07-04T17:29:51.392 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_open": 0, 2024-07-04T17:29:51.392 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_close": 0, 2024-07-04T17:29:51.392 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_laggy": 0, 2024-07-04T17:29:51.392 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_wr": 0, 2024-07-04T17:29:51.392 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_rd": 0, 2024-07-04T17:29:51.392 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_del": 0 2024-07-04T17:29:51.392 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.392 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.392 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:51.392 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-client": [ 2024-07-04T17:29:51.392 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:51.392 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:51.393 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:51.393 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:29:51.393 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:29:51.393 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:29:51.393 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 39, 2024-07-04T17:29:51.393 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 146552, 2024-07-04T17:29:51.393 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:29:51.393 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 39, 2024-07-04T17:29:51.393 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:29:51.393 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:29:51.393 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 39, 2024-07-04T17:29:51.393 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 146552, 2024-07-04T17:29:51.393 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:29:51.393 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:29:51.393 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:29:51.394 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:29:51.394 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.394 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.394 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.394 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:51.394 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-radosclient": [ 2024-07-04T17:29:51.394 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:51.394 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:51.394 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:51.394 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:29:51.394 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:29:51.394 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:29:51.394 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 25, 2024-07-04T17:29:51.395 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 795752, 2024-07-04T17:29:51.395 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:29:51.395 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 25, 2024-07-04T17:29:51.395 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:29:51.395 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:29:51.395 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 25, 2024-07-04T17:29:51.395 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 795752, 2024-07-04T17:29:51.395 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:29:51.395 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:29:51.395 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:29:51.395 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:29:51.395 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.396 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.396 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.396 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:51.396 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_bytes": [ 2024-07-04T17:29:51.396 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:51.396 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:51.396 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:51.396 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:29:51.396 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:29:51.396 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:29:51.396 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:29:51.397 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:29:51.397 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:29:51.397 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:29:51.397 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:29:51.397 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:29:51.397 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:29:51.397 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:29:51.397 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:29:51.397 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:29:51.397 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:29:51.397 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:29:51.397 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.397 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.398 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.398 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:29:51.398 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_ops": [ 2024-07-04T17:29:51.398 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:29:51.398 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:29:51.398 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:29:51.398 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:29:51.398 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1024, 2024-07-04T17:29:51.398 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:29:51.398 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:29:51.398 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:29:51.398 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:29:51.398 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:29:51.399 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:29:51.399 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:29:51.399 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:29:51.399 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:29:51.399 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:29:51.399 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:29:51.399 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:29:51.399 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:29:51.399 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.399 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.399 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.399 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-07-04T17:29:51.399 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:29:51.400 DEBUG:tasks.cephfs.test_mirroring:command returned={ "AsyncMessenger::Worker": [ { "labels": { "id": "0" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "1" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "2" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } } ], "AsyncMessenger::Worker-0": [ { "labels": {}, "counters": { "msgr_recv_messages": 39, "msgr_send_messages": 4, "msgr_recv_bytes": 151872, "msgr_send_bytes": 800, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.016258169, "msgr_running_send_time": 0.001786130, "msgr_running_recv_time": 0.008875063, "msgr_running_fast_dispatch_time": 0.004893093, "msgr_send_messages_queue_lat": { "avgcount": 4, "sum": 0.004367369, "avgtime": 0.001091842 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 151872, "msgr_send_encrypted_bytes": 800 } } ], "AsyncMessenger::Worker-1": [ { "labels": {}, "counters": { "msgr_recv_messages": 4, "msgr_send_messages": 46, "msgr_recv_bytes": 576, "msgr_send_bytes": 332896, "msgr_created_connections": 9, "msgr_active_connections": 3, "msgr_running_total_time": 0.021189213, "msgr_running_send_time": 0.007532880, "msgr_running_recv_time": 0.007379601, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 43, "sum": 0.034969193, "avgtime": 0.000813237 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 576, "msgr_send_encrypted_bytes": 332896 } } ], "AsyncMessenger::Worker-2": [ { "labels": {}, "counters": { "msgr_recv_messages": 24, "msgr_send_messages": 7, "msgr_recv_bytes": 799472, "msgr_send_bytes": 1264, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.027859965, "msgr_running_send_time": 0.001903531, "msgr_running_recv_time": 0.047236962, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 7, "sum": 0.003249255, "avgtime": 0.000464179 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 799472, "msgr_send_encrypted_bytes": 1264 } } ], "cct": [ { "labels": {}, "counters": { "total_workers": 0, "unhealthy_workers": 0 } } ], "cephfs_mirror": [ { "labels": {}, "counters": { "mirrored_filesystems": 1, "mirror_enable_failures": 0 } } ], "cephfs_mirror_mirrored_filesystems": [ { "labels": { "filesystem": "cephfs" }, "counters": { "mirroring_peers": 0, "directory_count": 0 } } ], "mempool": [ { "labels": {}, "counters": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_items": 0, "bluefs_bytes": 0, "bluefs_items": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "buffer_anon_bytes": 183529, "buffer_anon_items": 56, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 18720, "osdmap_items": 169, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "mds_co_bytes": 0, "mds_co_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 } } ], "objecter": [ { "labels": {}, "counters": { "op_active": 0, "op_laggy": 0, "op_send": 0, "op_send_bytes": 0, "op_resend": 0, "op_reply": 0, "op_latency": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "op_inflight": 0, "oplen_avg": { "avgcount": 0, "sum": 0 }, "op": 0, "op_r": 0, "op_w": 0, "op_rmw": 0, "op_pg": 0, "osdop_stat": 0, "osdop_create": 0, "osdop_read": 0, "osdop_write": 0, "osdop_writefull": 0, "osdop_writesame": 0, "osdop_append": 0, "osdop_zero": 0, "osdop_truncate": 0, "osdop_delete": 0, "osdop_mapext": 0, "osdop_sparse_read": 0, "osdop_clonerange": 0, "osdop_getxattr": 0, "osdop_setxattr": 0, "osdop_cmpxattr": 0, "osdop_rmxattr": 0, "osdop_resetxattrs": 0, "osdop_call": 0, "osdop_watch": 0, "osdop_notify": 0, "osdop_src_cmpxattr": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_other": 0, "linger_active": 0, "linger_send": 0, "linger_resend": 0, "linger_ping": 0, "poolop_active": 0, "poolop_send": 0, "poolop_resend": 0, "poolstat_active": 0, "poolstat_send": 0, "poolstat_resend": 0, "statfs_active": 0, "statfs_send": 0, "statfs_resend": 0, "command_active": 0, "command_send": 0, "command_resend": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "osd_sessions": 0, "osd_session_open": 0, "osd_session_close": 0, "osd_laggy": 0, "omap_wr": 0, "omap_rd": 0, "omap_del": 0 } } ], "throttle-msgr_dispatch_throttler-client": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 39, "get_sum": 146552, "get_or_fail_fail": 0, "get_or_fail_success": 39, "take": 0, "take_sum": 0, "put": 39, "put_sum": 146552, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-msgr_dispatch_throttler-radosclient": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 25, "get_sum": 795752, "get_or_fail_fail": 0, "get_or_fail_success": 25, "take": 0, "take_sum": 0, "put": 25, "put_sum": 795752, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_bytes": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_ops": [ { "labels": {}, "counters": { "val": 0, "max": 1024, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ] } 2024-07-04T17:29:51.400 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:29:51.401 DEBUG:teuthology.orchestra.run.smithi042:mirror status for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@4 2024-07-04T17:29:51.636 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:29:51.636 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_inst": "172.21.15.42:0/7072365", 2024-07-04T17:29:51.636 INFO:teuthology.orchestra.run.smithi042.stdout: "peers": {}, 2024-07-04T17:29:51.636 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_dirs": { 2024-07-04T17:29:51.636 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_count": 0 2024-07-04T17:29:51.636 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:29:51.636 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:29:51.653 DEBUG:tasks.cephfs.test_mirroring:command returned={ "rados_inst": "172.21.15.42:0/7072365", "peers": {}, "snap_dirs": { "dir_count": 0 } } 2024-07-04T17:29:51.653 DEBUG:tasks.cephfs.test_mirroring:initial dir_count=0 2024-07-04T17:29:51.653 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snapshot mirror add cephfs /d1/d2/d3 2024-07-04T17:29:51.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:51.950+0000 7f19dfe00640 1 -- 172.21.15.42:0/1081816964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19e00ac9d0 msgr2=0x7f19e00aef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:51.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:51.950+0000 7f19dfe00640 1 --2- 172.21.15.42:0/1081816964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19e00ac9d0 0x7f19e00aef80 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0097b0 tx=0x7f19cc02a4b0 comp rx=0 tx=0).stop 2024-07-04T17:29:51.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:51.950+0000 7f19dfe00640 1 -- 172.21.15.42:0/1081816964 shutdown_connections 2024-07-04T17:29:51.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:51.950+0000 7f19dfe00640 1 --2- 172.21.15.42:0/1081816964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19e00ac9d0 0x7f19e00aef80 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:51.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:51.950+0000 7f19dfe00640 1 -- 172.21.15.42:0/1081816964 >> 172.21.15.42:0/1081816964 conn(0x7f19e00afaa0 msgr2=0x7f19e00afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:51.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:51.950+0000 7f19dfe00640 1 -- 172.21.15.42:0/1081816964 shutdown_connections 2024-07-04T17:29:51.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:51.950+0000 7f19dfe00640 1 -- 172.21.15.42:0/1081816964 wait complete. 2024-07-04T17:29:51.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:51.953+0000 7f19dfe00640 1 Processor -- start 2024-07-04T17:29:51.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:51.953+0000 7f19dfe00640 1 -- start start 2024-07-04T17:29:51.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:51.954+0000 7f19dfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19e00ac9d0 0x7f19e0143100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:51.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:51.954+0000 7f19dfe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f19e01415a0 con 0x7f19e00ac9d0 2024-07-04T17:29:51.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:51.954+0000 7f19dea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19e00ac9d0 0x7f19e0143100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:51.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:51.954+0000 7f19dea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19e00ac9d0 0x7f19e0143100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58596/0 (socket says 172.21.15.42:58596) 2024-07-04T17:29:51.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:51.955+0000 7f19dea00640 1 -- 172.21.15.42:0/2278046696 learned_addr learned my addr 172.21.15.42:0/2278046696 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:29:51.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:51.955+0000 7f19dea00640 1 -- 172.21.15.42:0/2278046696 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e00ab420 con 0x7f19e00ac9d0 2024-07-04T17:29:51.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:51.955+0000 7f19dea00640 1 --2- 172.21.15.42:0/2278046696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19e00ac9d0 0x7f19e0143100 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f19d0001eb0 tx=0x7f19d000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:51.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:51.956+0000 7f19d6a00640 1 -- 172.21.15.42:0/2278046696 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19d00116c0 con 0x7f19e00ac9d0 2024-07-04T17:29:51.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:51.956+0000 7f19d6a00640 1 -- 172.21.15.42:0/2278046696 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19d0017070 con 0x7f19e00ac9d0 2024-07-04T17:29:51.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:51.956+0000 7f19dfe00640 1 -- 172.21.15.42:0/2278046696 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e00b5bc0 con 0x7f19e00ac9d0 2024-07-04T17:29:51.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:51.956+0000 7f19d6a00640 1 -- 172.21.15.42:0/2278046696 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19d0011ae0 con 0x7f19e00ac9d0 2024-07-04T17:29:51.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:51.956+0000 7f19dfe00640 1 -- 172.21.15.42:0/2278046696 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e0144ca0 con 0x7f19e00ac9d0 2024-07-04T17:29:51.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:51.957+0000 7f19dfe00640 1 -- 172.21.15.42:0/2278046696 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19e013e960 con 0x7f19e00ac9d0 2024-07-04T17:29:51.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:51.957+0000 7f19d6a00640 1 -- 172.21.15.42:0/2278046696 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 20) ==== 49681+0+0 (secure 0 0 0) 0x7f19d0011c80 con 0x7f19e00ac9d0 2024-07-04T17:29:51.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:51.958+0000 7f19d6a00640 1 --2- 172.21.15.42:0/2278046696 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f19c003d5d0 0x7f19c003fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:29:51.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:51.958+0000 7f19d6a00640 1 -- 172.21.15.42:0/2278046696 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4733+0+0 (secure 0 0 0) 0x7f19d0051c30 con 0x7f19e00ac9d0 2024-07-04T17:29:51.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:51.958+0000 7f19de000640 1 --2- 172.21.15.42:0/2278046696 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f19c003d5d0 0x7f19c003fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:29:51.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:51.959+0000 7f19de000640 1 --2- 172.21.15.42:0/2278046696 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f19c003d5d0 0x7f19c003fae0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f19cc034000 tx=0x7f19cc004730 comp rx=0 tx=0).ready entity=mgr.4853 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:29:51.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:51.963+0000 7f19d6a00640 1 -- 172.21.15.42:0/2278046696 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f19e0144ca0 con 0x7f19e00ac9d0 2024-07-04T17:29:52.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:52.195+0000 7f19dfe00640 1 -- 172.21.15.42:0/2278046696 --> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] -- mgr_command(tid 0: {"prefix": "fs snapshot mirror add", "fs_name": "cephfs", "path": "/d1/d2/d3", "target": ["mon-mgr", ""]}) -- 0x7f19e01467d0 con 0x7f19c003d5d0 2024-07-04T17:29:52.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:52.202+0000 7f19d6a00640 1 -- 172.21.15.42:0/2278046696 <== mgr.4853 v2:172.21.15.42:6824/3045931584 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f19e01467d0 con 0x7f19c003d5d0 2024-07-04T17:29:52.202 INFO:teuthology.orchestra.run.smithi042.stdout:{} 2024-07-04T17:29:52.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:52.204+0000 7f19dfe00640 1 -- 172.21.15.42:0/2278046696 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f19c003d5d0 msgr2=0x7f19c003fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:52.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:52.204+0000 7f19dfe00640 1 --2- 172.21.15.42:0/2278046696 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f19c003d5d0 0x7f19c003fae0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f19cc034000 tx=0x7f19cc004730 comp rx=0 tx=0).stop 2024-07-04T17:29:52.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:52.205+0000 7f19dfe00640 1 -- 172.21.15.42:0/2278046696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19e00ac9d0 msgr2=0x7f19e0143100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:29:52.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:52.205+0000 7f19dfe00640 1 --2- 172.21.15.42:0/2278046696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19e00ac9d0 0x7f19e0143100 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f19d0001eb0 tx=0x7f19d000c810 comp rx=0 tx=0).stop 2024-07-04T17:29:52.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:52.205+0000 7f19dfe00640 1 -- 172.21.15.42:0/2278046696 shutdown_connections 2024-07-04T17:29:52.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:52.205+0000 7f19dfe00640 1 --2- 172.21.15.42:0/2278046696 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f19c003d5d0 0x7f19c003fae0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:52.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:52.205+0000 7f19dfe00640 1 --2- 172.21.15.42:0/2278046696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19e00ac9d0 0x7f19e0143100 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:29:52.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:52.205+0000 7f19dfe00640 1 -- 172.21.15.42:0/2278046696 >> 172.21.15.42:0/2278046696 conn(0x7f19e00afaa0 msgr2=0x7f19e013f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:29:52.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:52.205+0000 7f19dfe00640 1 -- 172.21.15.42:0/2278046696 shutdown_connections 2024-07-04T17:29:52.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:29:52.205+0000 7f19dfe00640 1 -- 172.21.15.42:0/2278046696 wait complete. 2024-07-04T17:30:00.304 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-04T17:30:02.232 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:30:02.232 DEBUG:teuthology.orchestra.run.smithi042:mirror status for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@4 2024-07-04T17:30:02.484 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:30:02.484 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_inst": "172.21.15.42:0/7072365", 2024-07-04T17:30:02.484 INFO:teuthology.orchestra.run.smithi042.stdout: "peers": {}, 2024-07-04T17:30:02.484 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_dirs": { 2024-07-04T17:30:02.484 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_count": 1 2024-07-04T17:30:02.484 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.484 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:30:02.502 DEBUG:tasks.cephfs.test_mirroring:command returned={ "rados_inst": "172.21.15.42:0/7072365", "peers": {}, "snap_dirs": { "dir_count": 1 } } 2024-07-04T17:30:02.502 DEBUG:tasks.cephfs.test_mirroring:new dir_count=1 2024-07-04T17:30:02.502 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:30:02.503 DEBUG:teuthology.orchestra.run.smithi042:counter dump for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok counter dump 2024-07-04T17:30:02.732 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:30:02.732 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker": [ 2024-07-04T17:30:02.732 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:02.732 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:30:02.732 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "0" 2024-07-04T17:30:02.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:02.732 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:02.733 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:30:02.733 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:30:02.733 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:02.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:02.733 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:30:02.733 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "1" 2024-07-04T17:30:02.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:02.733 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:02.733 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:30:02.733 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:30:02.733 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:02.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:02.734 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:30:02.734 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "2" 2024-07-04T17:30:02.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:02.734 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:02.734 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:30:02.734 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:30:02.734 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.734 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.734 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:02.734 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": [ 2024-07-04T17:30:02.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:02.735 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:02.735 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:02.735 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 39, 2024-07-04T17:30:02.735 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 4, 2024-07-04T17:30:02.735 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 151872, 2024-07-04T17:30:02.735 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 800, 2024-07-04T17:30:02.735 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:30:02.735 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:30:02.735 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.016421602, 2024-07-04T17:30:02.735 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.001867233, 2024-07-04T17:30:02.735 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.008914213, 2024-07-04T17:30:02.735 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.004893093, 2024-07-04T17:30:02.736 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:30:02.736 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 4, 2024-07-04T17:30:02.736 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.004367369, 2024-07-04T17:30:02.736 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.001091842 2024-07-04T17:30:02.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:02.736 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:30:02.736 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:30:02.736 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:30:02.736 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:30:02.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:02.736 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 151872, 2024-07-04T17:30:02.736 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 800 2024-07-04T17:30:02.736 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.736 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.737 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:02.737 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": [ 2024-07-04T17:30:02.737 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:02.737 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:02.737 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:02.737 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 4, 2024-07-04T17:30:02.737 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 48, 2024-07-04T17:30:02.737 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 576, 2024-07-04T17:30:02.737 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 333456, 2024-07-04T17:30:02.737 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 9, 2024-07-04T17:30:02.737 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 3, 2024-07-04T17:30:02.737 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.021526957, 2024-07-04T17:30:02.738 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.007831078, 2024-07-04T17:30:02.738 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.007379601, 2024-07-04T17:30:02.738 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:30:02.738 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:30:02.738 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 45, 2024-07-04T17:30:02.738 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.035141512, 2024-07-04T17:30:02.738 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000780922 2024-07-04T17:30:02.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:02.738 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:30:02.738 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:30:02.738 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:30:02.739 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:30:02.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:02.739 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 576, 2024-07-04T17:30:02.739 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 333456 2024-07-04T17:30:02.739 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.739 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.739 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:02.739 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": [ 2024-07-04T17:30:02.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:02.739 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:02.739 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:02.739 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 24, 2024-07-04T17:30:02.739 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 7, 2024-07-04T17:30:02.739 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 799472, 2024-07-04T17:30:02.740 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 1264, 2024-07-04T17:30:02.740 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:30:02.740 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:30:02.740 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.028033521, 2024-07-04T17:30:02.740 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.001989957, 2024-07-04T17:30:02.740 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.047279265, 2024-07-04T17:30:02.740 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:30:02.740 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:30:02.740 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 7, 2024-07-04T17:30:02.740 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.003249255, 2024-07-04T17:30:02.740 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000464179 2024-07-04T17:30:02.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:02.740 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:30:02.740 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:30:02.741 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:30:02.741 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:30:02.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:02.741 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 799472, 2024-07-04T17:30:02.741 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 1264 2024-07-04T17:30:02.741 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.741 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.741 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:02.741 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": [ 2024-07-04T17:30:02.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:02.741 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:02.741 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:02.741 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-07-04T17:30:02.741 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-07-04T17:30:02.742 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.742 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.742 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:02.742 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror": [ 2024-07-04T17:30:02.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:02.742 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:02.742 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:02.742 INFO:teuthology.orchestra.run.smithi042.stdout: "mirrored_filesystems": 1, 2024-07-04T17:30:02.742 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_enable_failures": 0 2024-07-04T17:30:02.742 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.742 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.742 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:02.742 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_mirrored_filesystems": [ 2024-07-04T17:30:02.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:02.743 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:30:02.743 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem": "cephfs" 2024-07-04T17:30:02.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:02.743 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:02.743 INFO:teuthology.orchestra.run.smithi042.stdout: "mirroring_peers": 0, 2024-07-04T17:30:02.743 INFO:teuthology.orchestra.run.smithi042.stdout: "directory_count": 1 2024-07-04T17:30:02.743 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.743 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.743 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:02.743 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": [ 2024-07-04T17:30:02.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:02.743 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:02.743 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:02.743 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-07-04T17:30:02.744 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-07-04T17:30:02.744 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T17:30:02.744 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-07-04T17:30:02.744 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T17:30:02.744 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-07-04T17:30:02.744 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T17:30:02.744 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T17:30:02.744 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T17:30:02.744 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T17:30:02.744 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T17:30:02.744 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-07-04T17:30:02.744 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T17:30:02.744 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T17:30:02.744 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-07-04T17:30:02.745 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-07-04T17:30:02.745 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-07-04T17:30:02.745 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-07-04T17:30:02.745 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T17:30:02.745 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T17:30:02.745 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T17:30:02.745 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T17:30:02.745 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T17:30:02.745 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-07-04T17:30:02.745 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-07-04T17:30:02.745 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-07-04T17:30:02.745 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T17:30:02.745 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T17:30:02.745 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-07-04T17:30:02.745 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-07-04T17:30:02.746 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-07-04T17:30:02.746 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-07-04T17:30:02.746 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T17:30:02.746 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-07-04T17:30:02.746 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T17:30:02.746 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-07-04T17:30:02.746 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 183529, 2024-07-04T17:30:02.746 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 56, 2024-07-04T17:30:02.746 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-07-04T17:30:02.746 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-07-04T17:30:02.746 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-07-04T17:30:02.746 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-07-04T17:30:02.746 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-07-04T17:30:02.747 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-07-04T17:30:02.747 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-07-04T17:30:02.747 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-07-04T17:30:02.747 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 18720, 2024-07-04T17:30:02.747 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 169, 2024-07-04T17:30:02.747 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T17:30:02.747 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-07-04T17:30:02.747 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-07-04T17:30:02.747 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-07-04T17:30:02.747 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 0, 2024-07-04T17:30:02.747 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 0, 2024-07-04T17:30:02.747 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-07-04T17:30:02.747 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-07-04T17:30:02.747 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-07-04T17:30:02.748 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-07-04T17:30:02.748 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.748 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.748 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:02.748 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter": [ 2024-07-04T17:30:02.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:02.748 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:02.748 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:02.748 INFO:teuthology.orchestra.run.smithi042.stdout: "op_active": 0, 2024-07-04T17:30:02.748 INFO:teuthology.orchestra.run.smithi042.stdout: "op_laggy": 0, 2024-07-04T17:30:02.748 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send": 0, 2024-07-04T17:30:02.748 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send_bytes": 0, 2024-07-04T17:30:02.748 INFO:teuthology.orchestra.run.smithi042.stdout: "op_resend": 0, 2024-07-04T17:30:02.748 INFO:teuthology.orchestra.run.smithi042.stdout: "op_reply": 0, 2024-07-04T17:30:02.748 INFO:teuthology.orchestra.run.smithi042.stdout: "op_latency": { 2024-07-04T17:30:02.749 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:30:02.749 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:30:02.749 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:30:02.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:02.749 INFO:teuthology.orchestra.run.smithi042.stdout: "op_inflight": 0, 2024-07-04T17:30:02.749 INFO:teuthology.orchestra.run.smithi042.stdout: "oplen_avg": { 2024-07-04T17:30:02.749 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:30:02.749 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0 2024-07-04T17:30:02.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:02.749 INFO:teuthology.orchestra.run.smithi042.stdout: "op": 0, 2024-07-04T17:30:02.749 INFO:teuthology.orchestra.run.smithi042.stdout: "op_r": 0, 2024-07-04T17:30:02.749 INFO:teuthology.orchestra.run.smithi042.stdout: "op_w": 0, 2024-07-04T17:30:02.749 INFO:teuthology.orchestra.run.smithi042.stdout: "op_rmw": 0, 2024-07-04T17:30:02.749 INFO:teuthology.orchestra.run.smithi042.stdout: "op_pg": 0, 2024-07-04T17:30:02.750 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_stat": 0, 2024-07-04T17:30:02.750 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_create": 0, 2024-07-04T17:30:02.750 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_read": 0, 2024-07-04T17:30:02.750 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_write": 0, 2024-07-04T17:30:02.750 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writefull": 0, 2024-07-04T17:30:02.750 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writesame": 0, 2024-07-04T17:30:02.750 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_append": 0, 2024-07-04T17:30:02.750 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_zero": 0, 2024-07-04T17:30:02.750 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_truncate": 0, 2024-07-04T17:30:02.750 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_delete": 0, 2024-07-04T17:30:02.750 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_mapext": 0, 2024-07-04T17:30:02.750 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_sparse_read": 0, 2024-07-04T17:30:02.750 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_clonerange": 0, 2024-07-04T17:30:02.750 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_getxattr": 0, 2024-07-04T17:30:02.750 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_setxattr": 0, 2024-07-04T17:30:02.751 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_cmpxattr": 0, 2024-07-04T17:30:02.751 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_rmxattr": 0, 2024-07-04T17:30:02.751 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_resetxattrs": 0, 2024-07-04T17:30:02.751 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_call": 0, 2024-07-04T17:30:02.751 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_watch": 0, 2024-07-04T17:30:02.751 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_notify": 0, 2024-07-04T17:30:02.751 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T17:30:02.751 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls": 0, 2024-07-04T17:30:02.751 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls_filter": 0, 2024-07-04T17:30:02.751 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_other": 0, 2024-07-04T17:30:02.751 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_active": 0, 2024-07-04T17:30:02.751 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_send": 0, 2024-07-04T17:30:02.751 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_resend": 0, 2024-07-04T17:30:02.752 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_ping": 0, 2024-07-04T17:30:02.752 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_active": 0, 2024-07-04T17:30:02.752 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_send": 0, 2024-07-04T17:30:02.752 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_resend": 0, 2024-07-04T17:30:02.752 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_active": 0, 2024-07-04T17:30:02.752 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_send": 0, 2024-07-04T17:30:02.752 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_resend": 0, 2024-07-04T17:30:02.752 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_active": 0, 2024-07-04T17:30:02.752 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_send": 0, 2024-07-04T17:30:02.752 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_resend": 0, 2024-07-04T17:30:02.752 INFO:teuthology.orchestra.run.smithi042.stdout: "command_active": 0, 2024-07-04T17:30:02.753 INFO:teuthology.orchestra.run.smithi042.stdout: "command_send": 0, 2024-07-04T17:30:02.753 INFO:teuthology.orchestra.run.smithi042.stdout: "command_resend": 0, 2024-07-04T17:30:02.753 INFO:teuthology.orchestra.run.smithi042.stdout: "map_epoch": 0, 2024-07-04T17:30:02.753 INFO:teuthology.orchestra.run.smithi042.stdout: "map_full": 0, 2024-07-04T17:30:02.753 INFO:teuthology.orchestra.run.smithi042.stdout: "map_inc": 0, 2024-07-04T17:30:02.753 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_sessions": 0, 2024-07-04T17:30:02.753 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_open": 0, 2024-07-04T17:30:02.753 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_close": 0, 2024-07-04T17:30:02.753 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_laggy": 0, 2024-07-04T17:30:02.753 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_wr": 0, 2024-07-04T17:30:02.753 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_rd": 0, 2024-07-04T17:30:02.753 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_del": 0 2024-07-04T17:30:02.753 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.753 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.754 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:02.754 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-client": [ 2024-07-04T17:30:02.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:02.754 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:02.754 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:02.754 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:30:02.754 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:30:02.754 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:30:02.754 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 39, 2024-07-04T17:30:02.754 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 146552, 2024-07-04T17:30:02.754 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:30:02.754 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 39, 2024-07-04T17:30:02.754 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:30:02.755 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:30:02.755 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 39, 2024-07-04T17:30:02.755 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 146552, 2024-07-04T17:30:02.755 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:30:02.755 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:30:02.755 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:30:02.755 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:30:02.755 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.755 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.755 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.755 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:02.755 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-radosclient": [ 2024-07-04T17:30:02.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:02.755 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:02.755 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:02.756 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:30:02.756 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:30:02.756 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:30:02.756 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 25, 2024-07-04T17:30:02.756 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 795752, 2024-07-04T17:30:02.756 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:30:02.756 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 25, 2024-07-04T17:30:02.756 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:30:02.756 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:30:02.756 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 25, 2024-07-04T17:30:02.756 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 795752, 2024-07-04T17:30:02.756 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:30:02.756 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:30:02.757 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:30:02.757 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:30:02.757 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.757 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.757 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.757 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:02.757 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_bytes": [ 2024-07-04T17:30:02.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:02.757 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:02.757 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:02.757 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:30:02.758 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:30:02.758 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:30:02.758 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:30:02.758 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:30:02.758 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:30:02.758 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:30:02.758 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:30:02.758 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:30:02.758 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:30:02.758 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:30:02.758 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:30:02.758 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:30:02.758 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:30:02.759 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:30:02.759 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.759 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.759 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.759 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:02.759 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_ops": [ 2024-07-04T17:30:02.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:02.759 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:02.759 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:02.759 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:30:02.759 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1024, 2024-07-04T17:30:02.759 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:30:02.759 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:30:02.759 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:30:02.759 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:30:02.760 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:30:02.760 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:30:02.760 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:30:02.760 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:30:02.760 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:30:02.760 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:30:02.760 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:30:02.760 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:30:02.760 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:30:02.760 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.760 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.760 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:02.760 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-07-04T17:30:02.760 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:30:02.761 DEBUG:tasks.cephfs.test_mirroring:command returned={ "AsyncMessenger::Worker": [ { "labels": { "id": "0" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "1" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "2" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } } ], "AsyncMessenger::Worker-0": [ { "labels": {}, "counters": { "msgr_recv_messages": 39, "msgr_send_messages": 4, "msgr_recv_bytes": 151872, "msgr_send_bytes": 800, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.016421602, "msgr_running_send_time": 0.001867233, "msgr_running_recv_time": 0.008914213, "msgr_running_fast_dispatch_time": 0.004893093, "msgr_send_messages_queue_lat": { "avgcount": 4, "sum": 0.004367369, "avgtime": 0.001091842 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 151872, "msgr_send_encrypted_bytes": 800 } } ], "AsyncMessenger::Worker-1": [ { "labels": {}, "counters": { "msgr_recv_messages": 4, "msgr_send_messages": 48, "msgr_recv_bytes": 576, "msgr_send_bytes": 333456, "msgr_created_connections": 9, "msgr_active_connections": 3, "msgr_running_total_time": 0.021526957, "msgr_running_send_time": 0.007831078, "msgr_running_recv_time": 0.007379601, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 45, "sum": 0.035141512, "avgtime": 0.000780922 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 576, "msgr_send_encrypted_bytes": 333456 } } ], "AsyncMessenger::Worker-2": [ { "labels": {}, "counters": { "msgr_recv_messages": 24, "msgr_send_messages": 7, "msgr_recv_bytes": 799472, "msgr_send_bytes": 1264, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.028033521, "msgr_running_send_time": 0.001989957, "msgr_running_recv_time": 0.047279265, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 7, "sum": 0.003249255, "avgtime": 0.000464179 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 799472, "msgr_send_encrypted_bytes": 1264 } } ], "cct": [ { "labels": {}, "counters": { "total_workers": 0, "unhealthy_workers": 0 } } ], "cephfs_mirror": [ { "labels": {}, "counters": { "mirrored_filesystems": 1, "mirror_enable_failures": 0 } } ], "cephfs_mirror_mirrored_filesystems": [ { "labels": { "filesystem": "cephfs" }, "counters": { "mirroring_peers": 0, "directory_count": 1 } } ], "mempool": [ { "labels": {}, "counters": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_items": 0, "bluefs_bytes": 0, "bluefs_items": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "buffer_anon_bytes": 183529, "buffer_anon_items": 56, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 18720, "osdmap_items": 169, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "mds_co_bytes": 0, "mds_co_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 } } ], "objecter": [ { "labels": {}, "counters": { "op_active": 0, "op_laggy": 0, "op_send": 0, "op_send_bytes": 0, "op_resend": 0, "op_reply": 0, "op_latency": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "op_inflight": 0, "oplen_avg": { "avgcount": 0, "sum": 0 }, "op": 0, "op_r": 0, "op_w": 0, "op_rmw": 0, "op_pg": 0, "osdop_stat": 0, "osdop_create": 0, "osdop_read": 0, "osdop_write": 0, "osdop_writefull": 0, "osdop_writesame": 0, "osdop_append": 0, "osdop_zero": 0, "osdop_truncate": 0, "osdop_delete": 0, "osdop_mapext": 0, "osdop_sparse_read": 0, "osdop_clonerange": 0, "osdop_getxattr": 0, "osdop_setxattr": 0, "osdop_cmpxattr": 0, "osdop_rmxattr": 0, "osdop_resetxattrs": 0, "osdop_call": 0, "osdop_watch": 0, "osdop_notify": 0, "osdop_src_cmpxattr": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_other": 0, "linger_active": 0, "linger_send": 0, "linger_resend": 0, "linger_ping": 0, "poolop_active": 0, "poolop_send": 0, "poolop_resend": 0, "poolstat_active": 0, "poolstat_send": 0, "poolstat_resend": 0, "statfs_active": 0, "statfs_send": 0, "statfs_resend": 0, "command_active": 0, "command_send": 0, "command_resend": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "osd_sessions": 0, "osd_session_open": 0, "osd_session_close": 0, "osd_laggy": 0, "omap_wr": 0, "omap_rd": 0, "omap_del": 0 } } ], "throttle-msgr_dispatch_throttler-client": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 39, "get_sum": 146552, "get_or_fail_fail": 0, "get_or_fail_success": 39, "take": 0, "take_sum": 0, "put": 39, "put_sum": 146552, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-msgr_dispatch_throttler-radosclient": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 25, "get_sum": 795752, "get_or_fail_fail": 0, "get_or_fail_success": 25, "take": 0, "take_sum": 0, "put": 25, "put_sum": 795752, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_bytes": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_ops": [ { "labels": {}, "counters": { "val": 0, "max": 1024, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ] } 2024-07-04T17:30:02.762 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:30:02.762 DEBUG:teuthology.orchestra.run.smithi042:mirror status for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@4 2024-07-04T17:30:03.007 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:30:03.007 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_inst": "172.21.15.42:0/7072365", 2024-07-04T17:30:03.007 INFO:teuthology.orchestra.run.smithi042.stdout: "peers": {}, 2024-07-04T17:30:03.007 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_dirs": { 2024-07-04T17:30:03.008 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_count": 1 2024-07-04T17:30:03.008 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:03.008 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:30:03.026 DEBUG:tasks.cephfs.test_mirroring:command returned={ "rados_inst": "172.21.15.42:0/7072365", "peers": {}, "snap_dirs": { "dir_count": 1 } } 2024-07-04T17:30:03.026 DEBUG:tasks.cephfs.test_mirroring:initial dir_count=1 2024-07-04T17:30:03.026 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snapshot mirror add cephfs /d1/d2/././././././d3 2024-07-04T17:30:03.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:03.319+0000 7f523a800640 1 -- 172.21.15.42:0/1919772659 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52340b1100 msgr2=0x7f523413f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:03.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:03.319+0000 7f523a800640 1 --2- 172.21.15.42:0/1919772659 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52340b1100 0x7f523413f800 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f5224009fd0 tx=0x7f522402b140 comp rx=0 tx=0).stop 2024-07-04T17:30:03.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:03.319+0000 7f523a800640 1 -- 172.21.15.42:0/1919772659 shutdown_connections 2024-07-04T17:30:03.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:03.319+0000 7f523a800640 1 --2- 172.21.15.42:0/1919772659 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52340b1100 0x7f523413f800 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:03.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:03.319+0000 7f523a800640 1 -- 172.21.15.42:0/1919772659 >> 172.21.15.42:0/1919772659 conn(0x7f523406c020 msgr2=0x7f523406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:03.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:03.320+0000 7f523a800640 1 -- 172.21.15.42:0/1919772659 shutdown_connections 2024-07-04T17:30:03.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:03.320+0000 7f523a800640 1 -- 172.21.15.42:0/1919772659 wait complete. 2024-07-04T17:30:03.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:03.322+0000 7f523a800640 1 Processor -- start 2024-07-04T17:30:03.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:03.322+0000 7f523a800640 1 -- start start 2024-07-04T17:30:03.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:03.323+0000 7f523a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52340a94d0 0x7f5234141de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:03.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:03.323+0000 7f523a800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f52341401e0 con 0x7f52340a94d0 2024-07-04T17:30:03.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:03.324+0000 7f5233400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52340a94d0 0x7f5234141de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:03.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:03.324+0000 7f5233400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52340a94d0 0x7f5234141de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48278/0 (socket says 172.21.15.42:48278) 2024-07-04T17:30:03.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:03.324+0000 7f5233400640 1 -- 172.21.15.42:0/3842503958 learned_addr learned my addr 172.21.15.42:0/3842503958 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:03.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:03.324+0000 7f5233400640 1 -- 172.21.15.42:0/3842503958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52341d37b0 con 0x7f52340a94d0 2024-07-04T17:30:03.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:03.325+0000 7f5233400640 1 --2- 172.21.15.42:0/3842503958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52340a94d0 0x7f5234141de0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f521c001eb0 tx=0x7f521c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:03.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:03.325+0000 7f522b400640 1 -- 172.21.15.42:0/3842503958 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f521c0116c0 con 0x7f52340a94d0 2024-07-04T17:30:03.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:03.325+0000 7f522b400640 1 -- 172.21.15.42:0/3842503958 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f521c017070 con 0x7f52340a94d0 2024-07-04T17:30:03.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:03.325+0000 7f522b400640 1 -- 172.21.15.42:0/3842503958 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f521c011ae0 con 0x7f52340a94d0 2024-07-04T17:30:03.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:03.325+0000 7f523a800640 1 -- 172.21.15.42:0/3842503958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52341d2c90 con 0x7f52340a94d0 2024-07-04T17:30:03.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:03.325+0000 7f523a800640 1 -- 172.21.15.42:0/3842503958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52341d3e30 con 0x7f52340a94d0 2024-07-04T17:30:03.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:03.326+0000 7f523a800640 1 -- 172.21.15.42:0/3842503958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f523413f800 con 0x7f52340a94d0 2024-07-04T17:30:03.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:03.326+0000 7f522b400640 1 -- 172.21.15.42:0/3842503958 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 20) ==== 49681+0+0 (secure 0 0 0) 0x7f521c011c80 con 0x7f52340a94d0 2024-07-04T17:30:03.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:03.327+0000 7f522b400640 1 --2- 172.21.15.42:0/3842503958 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f521003d5d0 0x7f521003fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:03.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:03.327+0000 7f522b400640 1 -- 172.21.15.42:0/3842503958 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4733+0+0 (secure 0 0 0) 0x7f521c051bb0 con 0x7f52340a94d0 2024-07-04T17:30:03.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:03.327+0000 7f5232a00640 1 --2- 172.21.15.42:0/3842503958 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f521003d5d0 0x7f521003fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:03.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:03.328+0000 7f5232a00640 1 --2- 172.21.15.42:0/3842503958 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f521003d5d0 0x7f521003fae0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5224033000 tx=0x7f5224004ab0 comp rx=0 tx=0).ready entity=mgr.4853 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:03.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:03.333+0000 7f522b400640 1 -- 172.21.15.42:0/3842503958 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f52340b1510 con 0x7f52340a94d0 2024-07-04T17:30:03.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:03.564+0000 7f523a800640 1 -- 172.21.15.42:0/3842503958 --> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] -- mgr_command(tid 0: {"prefix": "fs snapshot mirror add", "fs_name": "cephfs", "path": "/d1/d2/././././././d3", "target": ["mon-mgr", ""]}) -- 0x7f52341d3600 con 0x7f521003d5d0 2024-07-04T17:30:03.566 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:30:03.566+0000 7f679f400640 -1 mgr.server reply reply (17) File exists directory /d1/d2/d3 is already tracked 2024-07-04T17:30:03.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:03.567+0000 7f522b400640 1 -- 172.21.15.42:0/3842503958 <== mgr.4853 v2:172.21.15.42:6824/3045931584 1 ==== mgr_command_reply(tid 0: -17 directory /d1/d2/d3 is already tracked) ==== 46+0+0 (secure 0 0 0) 0x7f52341d3600 con 0x7f521003d5d0 2024-07-04T17:30:03.567 INFO:teuthology.orchestra.run.smithi042.stderr:Error EEXIST: directory /d1/d2/d3 is already tracked 2024-07-04T17:30:03.602 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-07-04T17:30:03.602 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:30:03.603 DEBUG:teuthology.orchestra.run.smithi042:mirror status for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@4 2024-07-04T17:30:03.852 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:30:03.852 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_inst": "172.21.15.42:0/7072365", 2024-07-04T17:30:03.852 INFO:teuthology.orchestra.run.smithi042.stdout: "peers": {}, 2024-07-04T17:30:03.852 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_dirs": { 2024-07-04T17:30:03.852 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_count": 1 2024-07-04T17:30:03.852 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:03.852 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:30:03.870 DEBUG:tasks.cephfs.test_mirroring:command returned={ "rados_inst": "172.21.15.42:0/7072365", "peers": {}, "snap_dirs": { "dir_count": 1 } } 2024-07-04T17:30:03.871 DEBUG:tasks.cephfs.test_mirroring:initial dir_count=1 2024-07-04T17:30:03.871 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snapshot mirror add cephfs /d1/d2/././././././d3////// 2024-07-04T17:30:04.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:04.173+0000 7fea71200640 1 -- 172.21.15.42:0/2624385117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea6c06d070 msgr2=0x7fea6c073a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:04.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:04.174+0000 7fea71200640 1 --2- 172.21.15.42:0/2624385117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea6c06d070 0x7fea6c073a20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fea5c005fd0 tx=0x7fea5c02a8b0 comp rx=0 tx=0).stop 2024-07-04T17:30:04.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:04.174+0000 7fea71200640 1 -- 172.21.15.42:0/2624385117 shutdown_connections 2024-07-04T17:30:04.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:04.174+0000 7fea71200640 1 --2- 172.21.15.42:0/2624385117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea6c06d070 0x7fea6c073a20 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:04.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:04.174+0000 7fea71200640 1 -- 172.21.15.42:0/2624385117 >> 172.21.15.42:0/2624385117 conn(0x7fea6c06c020 msgr2=0x7fea6c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:04.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:04.174+0000 7fea71200640 1 -- 172.21.15.42:0/2624385117 shutdown_connections 2024-07-04T17:30:04.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:04.174+0000 7fea71200640 1 -- 172.21.15.42:0/2624385117 wait complete. 2024-07-04T17:30:04.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:04.176+0000 7fea71200640 1 Processor -- start 2024-07-04T17:30:04.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:04.176+0000 7fea71200640 1 -- start start 2024-07-04T17:30:04.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:04.177+0000 7fea71200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea6c06d070 0x7fea6c1d48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:04.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:04.178+0000 7fea71200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fea6c071c40 con 0x7fea6c06d070 2024-07-04T17:30:04.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:04.178+0000 7fea69e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea6c06d070 0x7fea6c1d48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:04.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:04.178+0000 7fea69e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea6c06d070 0x7fea6c1d48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59926/0 (socket says 172.21.15.42:59926) 2024-07-04T17:30:04.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:04.178+0000 7fea69e00640 1 -- 172.21.15.42:0/1776765932 learned_addr learned my addr 172.21.15.42:0/1776765932 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:04.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:04.178+0000 7fea69e00640 1 -- 172.21.15.42:0/1776765932 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6c1d4f20 con 0x7fea6c06d070 2024-07-04T17:30:04.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:04.179+0000 7fea69e00640 1 --2- 172.21.15.42:0/1776765932 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea6c06d070 0x7fea6c1d48e0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fea5c006010 tx=0x7fea5c033b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:04.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:04.179+0000 7fea62000640 1 -- 172.21.15.42:0/1776765932 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea5c0319f0 con 0x7fea6c06d070 2024-07-04T17:30:04.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:04.179+0000 7fea62000640 1 -- 172.21.15.42:0/1776765932 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea5c031b90 con 0x7fea6c06d070 2024-07-04T17:30:04.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:04.179+0000 7fea62000640 1 -- 172.21.15.42:0/1776765932 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea5c033070 con 0x7fea6c06d070 2024-07-04T17:30:04.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:04.179+0000 7fea71200640 1 -- 172.21.15.42:0/1776765932 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea6c1dd4c0 con 0x7fea6c06d070 2024-07-04T17:30:04.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:04.179+0000 7fea71200640 1 -- 172.21.15.42:0/1776765932 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea6c1d6060 con 0x7fea6c06d070 2024-07-04T17:30:04.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:04.180+0000 7fea71200640 1 -- 172.21.15.42:0/1776765932 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea6c073a20 con 0x7fea6c06d070 2024-07-04T17:30:04.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:04.180+0000 7fea62000640 1 -- 172.21.15.42:0/1776765932 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 20) ==== 49681+0+0 (secure 0 0 0) 0x7fea5c033210 con 0x7fea6c06d070 2024-07-04T17:30:04.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:04.181+0000 7fea62000640 1 --2- 172.21.15.42:0/1776765932 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7fea4803d5d0 0x7fea4803fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:04.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:04.181+0000 7fea62000640 1 -- 172.21.15.42:0/1776765932 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4733+0+0 (secure 0 0 0) 0x7fea5c073700 con 0x7fea6c06d070 2024-07-04T17:30:04.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:04.181+0000 7fea69400640 1 --2- 172.21.15.42:0/1776765932 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7fea4803d5d0 0x7fea4803fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:04.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:04.182+0000 7fea69400640 1 --2- 172.21.15.42:0/1776765932 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7fea4803d5d0 0x7fea4803fae0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fea540015c0 tx=0x7fea54007a90 comp rx=0 tx=0).ready entity=mgr.4853 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:04.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:04.186+0000 7fea62000640 1 -- 172.21.15.42:0/1776765932 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fea5c03f030 con 0x7fea6c06d070 2024-07-04T17:30:04.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:04.416+0000 7fea71200640 1 -- 172.21.15.42:0/1776765932 --> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] -- mgr_command(tid 0: {"prefix": "fs snapshot mirror add", "fs_name": "cephfs", "path": "/d1/d2/././././././d3//////", "target": ["mon-mgr", ""]}) -- 0x7fea6c0b31b0 con 0x7fea4803d5d0 2024-07-04T17:30:04.418 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:30:04.418+0000 7f679f400640 -1 mgr.server reply reply (17) File exists directory /d1/d2/d3 is already tracked 2024-07-04T17:30:04.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:04.418+0000 7fea62000640 1 -- 172.21.15.42:0/1776765932 <== mgr.4853 v2:172.21.15.42:6824/3045931584 1 ==== mgr_command_reply(tid 0: -17 directory /d1/d2/d3 is already tracked) ==== 46+0+0 (secure 0 0 0) 0x7fea6c0b31b0 con 0x7fea4803d5d0 2024-07-04T17:30:04.419 INFO:teuthology.orchestra.run.smithi042.stderr:Error EEXIST: directory /d1/d2/d3 is already tracked 2024-07-04T17:30:04.461 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-07-04T17:30:04.462 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:30:04.462 DEBUG:teuthology.orchestra.run.smithi042:mirror status for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@4 2024-07-04T17:30:04.708 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:30:04.708 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_inst": "172.21.15.42:0/7072365", 2024-07-04T17:30:04.709 INFO:teuthology.orchestra.run.smithi042.stdout: "peers": {}, 2024-07-04T17:30:04.709 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_dirs": { 2024-07-04T17:30:04.709 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_count": 1 2024-07-04T17:30:04.709 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:04.709 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:30:04.726 DEBUG:tasks.cephfs.test_mirroring:command returned={ "rados_inst": "172.21.15.42:0/7072365", "peers": {}, "snap_dirs": { "dir_count": 1 } } 2024-07-04T17:30:04.726 DEBUG:tasks.cephfs.test_mirroring:initial dir_count=1 2024-07-04T17:30:04.726 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snapshot mirror add cephfs /d1/d2/../d2/././././d3 2024-07-04T17:30:05.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.030+0000 7fc457200640 1 -- 172.21.15.42:0/3720320027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4580701d0 msgr2=0x7fc4580abe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:05.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.030+0000 7fc457200640 1 --2- 172.21.15.42:0/3720320027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4580701d0 0x7fc4580abe40 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc4480097b0 tx=0x7fc44802a530 comp rx=0 tx=0).stop 2024-07-04T17:30:05.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.030+0000 7fc457200640 1 -- 172.21.15.42:0/3720320027 shutdown_connections 2024-07-04T17:30:05.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.030+0000 7fc457200640 1 --2- 172.21.15.42:0/3720320027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4580701d0 0x7fc4580abe40 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:05.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.030+0000 7fc457200640 1 -- 172.21.15.42:0/3720320027 >> 172.21.15.42:0/3720320027 conn(0x7fc4580afaa0 msgr2=0x7fc4580afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:05.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.030+0000 7fc457200640 1 -- 172.21.15.42:0/3720320027 shutdown_connections 2024-07-04T17:30:05.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.031+0000 7fc457200640 1 -- 172.21.15.42:0/3720320027 wait complete. 2024-07-04T17:30:05.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.033+0000 7fc457200640 1 Processor -- start 2024-07-04T17:30:05.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.033+0000 7fc457200640 1 -- start start 2024-07-04T17:30:05.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.034+0000 7fc457200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4580701d0 0x7fc45814f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:05.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.034+0000 7fc457200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc4580ab190 con 0x7fc4580701d0 2024-07-04T17:30:05.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.034+0000 7fc455e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4580701d0 0x7fc45814f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:05.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.034+0000 7fc455e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4580701d0 0x7fc45814f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59940/0 (socket says 172.21.15.42:59940) 2024-07-04T17:30:05.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.034+0000 7fc455e00640 1 -- 172.21.15.42:0/1743367831 learned_addr learned my addr 172.21.15.42:0/1743367831 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:05.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.035+0000 7fc455e00640 1 -- 172.21.15.42:0/1743367831 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc45814d220 con 0x7fc4580701d0 2024-07-04T17:30:05.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.035+0000 7fc455e00640 1 --2- 172.21.15.42:0/1743367831 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4580701d0 0x7fc45814f230 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fc4480098d0 tx=0x7fc448033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:05.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.035+0000 7fc44e000640 1 -- 172.21.15.42:0/1743367831 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc448031a90 con 0x7fc4580701d0 2024-07-04T17:30:05.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.035+0000 7fc44e000640 1 -- 172.21.15.42:0/1743367831 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc448031d20 con 0x7fc4580701d0 2024-07-04T17:30:05.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.035+0000 7fc44e000640 1 -- 172.21.15.42:0/1743367831 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc448033200 con 0x7fc4580701d0 2024-07-04T17:30:05.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.035+0000 7fc457200640 1 -- 172.21.15.42:0/1743367831 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc45814f770 con 0x7fc4580701d0 2024-07-04T17:30:05.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.036+0000 7fc457200640 1 -- 172.21.15.42:0/1743367831 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc458144540 con 0x7fc4580701d0 2024-07-04T17:30:05.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.037+0000 7fc457200640 1 -- 172.21.15.42:0/1743367831 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4580759c0 con 0x7fc4580701d0 2024-07-04T17:30:05.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.037+0000 7fc44e000640 1 -- 172.21.15.42:0/1743367831 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 20) ==== 49681+0+0 (secure 0 0 0) 0x7fc4480333a0 con 0x7fc4580701d0 2024-07-04T17:30:05.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.038+0000 7fc44e000640 1 --2- 172.21.15.42:0/1743367831 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7fc43803d580 0x7fc43803fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:05.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.038+0000 7fc44e000640 1 -- 172.21.15.42:0/1743367831 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4733+0+0 (secure 0 0 0) 0x7fc448073420 con 0x7fc4580701d0 2024-07-04T17:30:05.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.038+0000 7fc455400640 1 --2- 172.21.15.42:0/1743367831 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7fc43803d580 0x7fc43803fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:05.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.039+0000 7fc455400640 1 --2- 172.21.15.42:0/1743367831 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7fc43803d580 0x7fc43803fa90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc444007fd0 tx=0x7fc444007920 comp rx=0 tx=0).ready entity=mgr.4853 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:05.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.043+0000 7fc44e000640 1 -- 172.21.15.42:0/1743367831 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fc4580759c0 con 0x7fc4580701d0 2024-07-04T17:30:05.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.279+0000 7fc457200640 1 -- 172.21.15.42:0/1743367831 --> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] -- mgr_command(tid 0: {"prefix": "fs snapshot mirror add", "fs_name": "cephfs", "path": "/d1/d2/../d2/././././d3", "target": ["mon-mgr", ""]}) -- 0x7fc45814ee90 con 0x7fc43803d580 2024-07-04T17:30:05.281 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:30:05.281+0000 7f679f400640 -1 mgr.server reply reply (17) File exists directory /d1/d2/d3 is already tracked 2024-07-04T17:30:05.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.281+0000 7fc44e000640 1 -- 172.21.15.42:0/1743367831 <== mgr.4853 v2:172.21.15.42:6824/3045931584 1 ==== mgr_command_reply(tid 0: -17 directory /d1/d2/d3 is already tracked) ==== 46+0+0 (secure 0 0 0) 0x7fc444006890 con 0x7fc43803d580 2024-07-04T17:30:05.281 INFO:teuthology.orchestra.run.smithi042.stderr:Error EEXIST: directory /d1/d2/d3 is already tracked 2024-07-04T17:30:05.310 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-07-04T17:30:05.310 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:30:05.310 DEBUG:teuthology.orchestra.run.smithi042:mirror status for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@4 2024-07-04T17:30:05.552 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:30:05.553 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_inst": "172.21.15.42:0/7072365", 2024-07-04T17:30:05.553 INFO:teuthology.orchestra.run.smithi042.stdout: "peers": {}, 2024-07-04T17:30:05.553 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_dirs": { 2024-07-04T17:30:05.553 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_count": 1 2024-07-04T17:30:05.553 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:05.553 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:30:05.570 DEBUG:tasks.cephfs.test_mirroring:command returned={ "rados_inst": "172.21.15.42:0/7072365", "peers": {}, "snap_dirs": { "dir_count": 1 } } 2024-07-04T17:30:05.570 DEBUG:tasks.cephfs.test_mirroring:initial dir_count=1 2024-07-04T17:30:05.571 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snapshot mirror add cephfs /././././d1/./././d2/./././d3////// 2024-07-04T17:30:05.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.864+0000 7f1e2ce00640 1 -- 172.21.15.42:0/4061942392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e280abe40 msgr2=0x7f1e280aa900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:05.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.864+0000 7f1e2ce00640 1 --2- 172.21.15.42:0/4061942392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e280abe40 0x7f1e280aa900 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f1e100097b0 tx=0x7f1e1002a4d0 comp rx=0 tx=0).stop 2024-07-04T17:30:05.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.864+0000 7f1e2ce00640 1 -- 172.21.15.42:0/4061942392 shutdown_connections 2024-07-04T17:30:05.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.864+0000 7f1e2ce00640 1 --2- 172.21.15.42:0/4061942392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e280abe40 0x7f1e280aa900 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:05.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.864+0000 7f1e2ce00640 1 -- 172.21.15.42:0/4061942392 >> 172.21.15.42:0/4061942392 conn(0x7f1e280afaa0 msgr2=0x7f1e280afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:05.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.864+0000 7f1e2ce00640 1 -- 172.21.15.42:0/4061942392 shutdown_connections 2024-07-04T17:30:05.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.864+0000 7f1e2ce00640 1 -- 172.21.15.42:0/4061942392 wait complete. 2024-07-04T17:30:05.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.867+0000 7f1e2ce00640 1 Processor -- start 2024-07-04T17:30:05.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.867+0000 7f1e2ce00640 1 -- start start 2024-07-04T17:30:05.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.869+0000 7f1e2ce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e280701d0 0x7f1e2814ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:05.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.869+0000 7f1e2ce00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1e28073ed0 con 0x7f1e280701d0 2024-07-04T17:30:05.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.869+0000 7f1e27400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e280701d0 0x7f1e2814ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:05.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.869+0000 7f1e27400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e280701d0 0x7f1e2814ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59950/0 (socket says 172.21.15.42:59950) 2024-07-04T17:30:05.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.869+0000 7f1e27400640 1 -- 172.21.15.42:0/246781997 learned_addr learned my addr 172.21.15.42:0/246781997 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:05.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.869+0000 7f1e27400640 1 -- 172.21.15.42:0/246781997 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2814ef30 con 0x7f1e280701d0 2024-07-04T17:30:05.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.870+0000 7f1e27400640 1 --2- 172.21.15.42:0/246781997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e280701d0 0x7f1e2814ca50 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f1e1800c480 tx=0x7f1e1800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:05.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.870+0000 7f1e1f400640 1 -- 172.21.15.42:0/246781997 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e180116c0 con 0x7f1e280701d0 2024-07-04T17:30:05.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.870+0000 7f1e1f400640 1 -- 172.21.15.42:0/246781997 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e18017070 con 0x7f1e280701d0 2024-07-04T17:30:05.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.870+0000 7f1e1f400640 1 -- 172.21.15.42:0/246781997 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e18011b20 con 0x7f1e280701d0 2024-07-04T17:30:05.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.870+0000 7f1e2ce00640 1 -- 172.21.15.42:0/246781997 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e280ae240 con 0x7f1e280701d0 2024-07-04T17:30:05.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.870+0000 7f1e2ce00640 1 -- 172.21.15.42:0/246781997 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e28144220 con 0x7f1e280701d0 2024-07-04T17:30:05.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.872+0000 7f1e2ce00640 1 -- 172.21.15.42:0/246781997 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e280008d0 con 0x7f1e280701d0 2024-07-04T17:30:05.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.872+0000 7f1e1f400640 1 -- 172.21.15.42:0/246781997 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 20) ==== 49681+0+0 (secure 0 0 0) 0x7f1e18011cc0 con 0x7f1e280701d0 2024-07-04T17:30:05.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.873+0000 7f1e1f400640 1 --2- 172.21.15.42:0/246781997 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f1e0403d5d0 0x7f1e0403fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:05.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.873+0000 7f1e1f400640 1 -- 172.21.15.42:0/246781997 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4733+0+0 (secure 0 0 0) 0x7f1e18051dc0 con 0x7f1e280701d0 2024-07-04T17:30:05.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.873+0000 7f1e1fe00640 1 --2- 172.21.15.42:0/246781997 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f1e0403d5d0 0x7f1e0403fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:05.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.874+0000 7f1e1fe00640 1 --2- 172.21.15.42:0/246781997 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f1e0403d5d0 0x7f1e0403fae0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1e10034000 tx=0x7f1e10004730 comp rx=0 tx=0).ready entity=mgr.4853 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:05.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:05.878+0000 7f1e1f400640 1 -- 172.21.15.42:0/246781997 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f1e18015d50 con 0x7f1e280701d0 2024-07-04T17:30:06.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:06.119+0000 7f1e2ce00640 1 -- 172.21.15.42:0/246781997 --> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] -- mgr_command(tid 0: {"prefix": "fs snapshot mirror add", "fs_name": "cephfs", "path": "/././././d1/./././d2/./././d3//////", "target": ["mon-mgr", ""]}) -- 0x7f1e280abe40 con 0x7f1e0403d5d0 2024-07-04T17:30:06.121 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:30:06.121+0000 7f679f400640 -1 mgr.server reply reply (17) File exists directory /d1/d2/d3 is already tracked 2024-07-04T17:30:06.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:06.121+0000 7f1e1f400640 1 -- 172.21.15.42:0/246781997 <== mgr.4853 v2:172.21.15.42:6824/3045931584 1 ==== mgr_command_reply(tid 0: -17 directory /d1/d2/d3 is already tracked) ==== 46+0+0 (secure 0 0 0) 0x7f1e280abe40 con 0x7f1e0403d5d0 2024-07-04T17:30:06.122 INFO:teuthology.orchestra.run.smithi042.stderr:Error EEXIST: directory /d1/d2/d3 is already tracked 2024-07-04T17:30:06.148 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-07-04T17:30:06.148 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:30:06.149 DEBUG:teuthology.orchestra.run.smithi042:mirror status for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@4 2024-07-04T17:30:06.392 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:30:06.393 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_inst": "172.21.15.42:0/7072365", 2024-07-04T17:30:06.393 INFO:teuthology.orchestra.run.smithi042.stdout: "peers": {}, 2024-07-04T17:30:06.393 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_dirs": { 2024-07-04T17:30:06.393 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_count": 1 2024-07-04T17:30:06.393 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:06.393 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:30:06.411 DEBUG:tasks.cephfs.test_mirroring:command returned={ "rados_inst": "172.21.15.42:0/7072365", "peers": {}, "snap_dirs": { "dir_count": 1 } } 2024-07-04T17:30:06.411 DEBUG:tasks.cephfs.test_mirroring:initial dir_count=1 2024-07-04T17:30:06.412 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snapshot mirror add cephfs /./d1/./d2/./d3/../../../d1/d2/d3 2024-07-04T17:30:06.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:06.707+0000 7f878c800640 1 -- 172.21.15.42:0/2487293001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87840ab2f0 msgr2=0x7f87840a9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:06.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:06.707+0000 7f878c800640 1 --2- 172.21.15.42:0/2487293001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87840ab2f0 0x7f87840a9990 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f87780097b0 tx=0x7f877802a530 comp rx=0 tx=0).stop 2024-07-04T17:30:06.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:06.707+0000 7f878c800640 1 -- 172.21.15.42:0/2487293001 shutdown_connections 2024-07-04T17:30:06.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:06.707+0000 7f878c800640 1 --2- 172.21.15.42:0/2487293001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87840ab2f0 0x7f87840a9990 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:06.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:06.707+0000 7f878c800640 1 -- 172.21.15.42:0/2487293001 >> 172.21.15.42:0/2487293001 conn(0x7f878406c020 msgr2=0x7f878406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:06.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:06.707+0000 7f878c800640 1 -- 172.21.15.42:0/2487293001 shutdown_connections 2024-07-04T17:30:06.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:06.707+0000 7f878c800640 1 -- 172.21.15.42:0/2487293001 wait complete. 2024-07-04T17:30:06.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:06.709+0000 7f878c800640 1 Processor -- start 2024-07-04T17:30:06.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:06.710+0000 7f878c800640 1 -- start start 2024-07-04T17:30:06.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:06.711+0000 7f878c800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f878406d110 0x7f87840bca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:06.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:06.711+0000 7f878c800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f878413f430 con 0x7f878406d110 2024-07-04T17:30:06.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:06.711+0000 7f8789e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f878406d110 0x7f87840bca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:06.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:06.711+0000 7f8789e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f878406d110 0x7f87840bca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59966/0 (socket says 172.21.15.42:59966) 2024-07-04T17:30:06.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:06.711+0000 7f8789e00640 1 -- 172.21.15.42:0/219897088 learned_addr learned my addr 172.21.15.42:0/219897088 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:06.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:06.712+0000 7f8789e00640 1 -- 172.21.15.42:0/219897088 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87840ba7b0 con 0x7f878406d110 2024-07-04T17:30:06.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:06.712+0000 7f8789e00640 1 --2- 172.21.15.42:0/219897088 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f878406d110 0x7f87840bca90 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f87780098d0 tx=0x7f8778033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:06.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:06.712+0000 7f877e000640 1 -- 172.21.15.42:0/219897088 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8778031a90 con 0x7f878406d110 2024-07-04T17:30:06.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:06.712+0000 7f877e000640 1 -- 172.21.15.42:0/219897088 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8778031d20 con 0x7f878406d110 2024-07-04T17:30:06.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:06.712+0000 7f877e000640 1 -- 172.21.15.42:0/219897088 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8778033200 con 0x7f878406d110 2024-07-04T17:30:06.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:06.712+0000 7f878c800640 1 -- 172.21.15.42:0/219897088 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87840ba400 con 0x7f878406d110 2024-07-04T17:30:06.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:06.713+0000 7f878c800640 1 -- 172.21.15.42:0/219897088 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87840bac80 con 0x7f878406d110 2024-07-04T17:30:06.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:06.714+0000 7f877e000640 1 -- 172.21.15.42:0/219897088 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 20) ==== 49681+0+0 (secure 0 0 0) 0x7f87780333a0 con 0x7f878406d110 2024-07-04T17:30:06.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:06.714+0000 7f878c800640 1 -- 172.21.15.42:0/219897088 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8784144870 con 0x7f878406d110 2024-07-04T17:30:06.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:06.714+0000 7f877e000640 1 --2- 172.21.15.42:0/219897088 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f876803d5d0 0x7f876803fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:06.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:06.715+0000 7f877e000640 1 -- 172.21.15.42:0/219897088 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4733+0+0 (secure 0 0 0) 0x7f8778073410 con 0x7f878406d110 2024-07-04T17:30:06.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:06.715+0000 7f8789400640 1 --2- 172.21.15.42:0/219897088 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f876803d5d0 0x7f876803fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:06.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:06.716+0000 7f8789400640 1 --2- 172.21.15.42:0/219897088 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f876803d5d0 0x7f876803fae0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f87740015e0 tx=0x7f8774001290 comp rx=0 tx=0).ready entity=mgr.4853 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:06.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:06.721+0000 7f877e000640 1 -- 172.21.15.42:0/219897088 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f8784144870 con 0x7f878406d110 2024-07-04T17:30:06.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:06.952+0000 7f878c800640 1 -- 172.21.15.42:0/219897088 --> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] -- mgr_command(tid 0: {"prefix": "fs snapshot mirror add", "fs_name": "cephfs", "path": "/./d1/./d2/./d3/../../../d1/d2/d3", "target": ["mon-mgr", ""]}) -- 0x7f87840bc8e0 con 0x7f876803d5d0 2024-07-04T17:30:06.954 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:30:06.953+0000 7f679f400640 -1 mgr.server reply reply (17) File exists directory /d1/d2/d3 is already tracked 2024-07-04T17:30:06.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:06.954+0000 7f877e000640 1 -- 172.21.15.42:0/219897088 <== mgr.4853 v2:172.21.15.42:6824/3045931584 1 ==== mgr_command_reply(tid 0: -17 directory /d1/d2/d3 is already tracked) ==== 46+0+0 (secure 0 0 0) 0x7f87840bc8e0 con 0x7f876803d5d0 2024-07-04T17:30:06.954 INFO:teuthology.orchestra.run.smithi042.stderr:Error EEXIST: directory /d1/d2/d3 is already tracked 2024-07-04T17:30:06.982 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-07-04T17:30:06.983 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:30:06.983 DEBUG:teuthology.orchestra.run.smithi042:counter dump for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok counter dump 2024-07-04T17:30:07.226 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:30:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker": [ 2024-07-04T17:30:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:30:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "0" 2024-07-04T17:30:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:30:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:30:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:07.227 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:30:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "1" 2024-07-04T17:30:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:30:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:30:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:30:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "2" 2024-07-04T17:30:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:07.228 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:30:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:30:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": [ 2024-07-04T17:30:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 39, 2024-07-04T17:30:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 4, 2024-07-04T17:30:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 151872, 2024-07-04T17:30:07.229 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 800, 2024-07-04T17:30:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:30:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:30:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.016514154, 2024-07-04T17:30:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.001912425, 2024-07-04T17:30:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.008934797, 2024-07-04T17:30:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.004893093, 2024-07-04T17:30:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:30:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 4, 2024-07-04T17:30:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.004367369, 2024-07-04T17:30:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.001091842 2024-07-04T17:30:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:30:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:30:07.230 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:30:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:30:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 151872, 2024-07-04T17:30:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 800 2024-07-04T17:30:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": [ 2024-07-04T17:30:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 4, 2024-07-04T17:30:07.231 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 49, 2024-07-04T17:30:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 576, 2024-07-04T17:30:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 333696, 2024-07-04T17:30:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 9, 2024-07-04T17:30:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 3, 2024-07-04T17:30:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.021688356, 2024-07-04T17:30:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.007968014, 2024-07-04T17:30:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.007379601, 2024-07-04T17:30:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:30:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:30:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 46, 2024-07-04T17:30:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.035250221, 2024-07-04T17:30:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000766309 2024-07-04T17:30:07.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:30:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:30:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:30:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:30:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 576, 2024-07-04T17:30:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 333696 2024-07-04T17:30:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": [ 2024-07-04T17:30:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:07.233 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 24, 2024-07-04T17:30:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 7, 2024-07-04T17:30:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 799472, 2024-07-04T17:30:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 1264, 2024-07-04T17:30:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:30:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:30:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.028104911, 2024-07-04T17:30:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.002026325, 2024-07-04T17:30:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.047295370, 2024-07-04T17:30:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:30:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:30:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 7, 2024-07-04T17:30:07.234 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.003249255, 2024-07-04T17:30:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000464179 2024-07-04T17:30:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:30:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:30:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:30:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:30:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 799472, 2024-07-04T17:30:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 1264 2024-07-04T17:30:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:07.235 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": [ 2024-07-04T17:30:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-07-04T17:30:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-07-04T17:30:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror": [ 2024-07-04T17:30:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: "mirrored_filesystems": 1, 2024-07-04T17:30:07.236 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_enable_failures": 0 2024-07-04T17:30:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_mirrored_filesystems": [ 2024-07-04T17:30:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:30:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem": "cephfs" 2024-07-04T17:30:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: "mirroring_peers": 0, 2024-07-04T17:30:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: "directory_count": 1 2024-07-04T17:30:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:07.237 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": [ 2024-07-04T17:30:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-07-04T17:30:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-07-04T17:30:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T17:30:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-07-04T17:30:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T17:30:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-07-04T17:30:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T17:30:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T17:30:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T17:30:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T17:30:07.238 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T17:30:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-07-04T17:30:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T17:30:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T17:30:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-07-04T17:30:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-07-04T17:30:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-07-04T17:30:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-07-04T17:30:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T17:30:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T17:30:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T17:30:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T17:30:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T17:30:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-07-04T17:30:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-07-04T17:30:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-07-04T17:30:07.239 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T17:30:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T17:30:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-07-04T17:30:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-07-04T17:30:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-07-04T17:30:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-07-04T17:30:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T17:30:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-07-04T17:30:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T17:30:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-07-04T17:30:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 183529, 2024-07-04T17:30:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 56, 2024-07-04T17:30:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-07-04T17:30:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-07-04T17:30:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-07-04T17:30:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-07-04T17:30:07.240 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-07-04T17:30:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-07-04T17:30:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-07-04T17:30:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-07-04T17:30:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 18720, 2024-07-04T17:30:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 169, 2024-07-04T17:30:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T17:30:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-07-04T17:30:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-07-04T17:30:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-07-04T17:30:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 0, 2024-07-04T17:30:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 0, 2024-07-04T17:30:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-07-04T17:30:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-07-04T17:30:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-07-04T17:30:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-07-04T17:30:07.241 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter": [ 2024-07-04T17:30:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: "op_active": 0, 2024-07-04T17:30:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: "op_laggy": 0, 2024-07-04T17:30:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send": 0, 2024-07-04T17:30:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send_bytes": 0, 2024-07-04T17:30:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: "op_resend": 0, 2024-07-04T17:30:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: "op_reply": 0, 2024-07-04T17:30:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: "op_latency": { 2024-07-04T17:30:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:30:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:30:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:30:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: "op_inflight": 0, 2024-07-04T17:30:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: "oplen_avg": { 2024-07-04T17:30:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:30:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0 2024-07-04T17:30:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: "op": 0, 2024-07-04T17:30:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: "op_r": 0, 2024-07-04T17:30:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: "op_w": 0, 2024-07-04T17:30:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: "op_rmw": 0, 2024-07-04T17:30:07.243 INFO:teuthology.orchestra.run.smithi042.stdout: "op_pg": 0, 2024-07-04T17:30:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_stat": 0, 2024-07-04T17:30:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_create": 0, 2024-07-04T17:30:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_read": 0, 2024-07-04T17:30:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_write": 0, 2024-07-04T17:30:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writefull": 0, 2024-07-04T17:30:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writesame": 0, 2024-07-04T17:30:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_append": 0, 2024-07-04T17:30:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_zero": 0, 2024-07-04T17:30:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_truncate": 0, 2024-07-04T17:30:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_delete": 0, 2024-07-04T17:30:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_mapext": 0, 2024-07-04T17:30:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_sparse_read": 0, 2024-07-04T17:30:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_clonerange": 0, 2024-07-04T17:30:07.244 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_getxattr": 0, 2024-07-04T17:30:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_setxattr": 0, 2024-07-04T17:30:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_cmpxattr": 0, 2024-07-04T17:30:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_rmxattr": 0, 2024-07-04T17:30:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_resetxattrs": 0, 2024-07-04T17:30:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_call": 0, 2024-07-04T17:30:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_watch": 0, 2024-07-04T17:30:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_notify": 0, 2024-07-04T17:30:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T17:30:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls": 0, 2024-07-04T17:30:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls_filter": 0, 2024-07-04T17:30:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_other": 0, 2024-07-04T17:30:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_active": 0, 2024-07-04T17:30:07.245 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_send": 0, 2024-07-04T17:30:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_resend": 0, 2024-07-04T17:30:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_ping": 0, 2024-07-04T17:30:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_active": 0, 2024-07-04T17:30:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_send": 0, 2024-07-04T17:30:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_resend": 0, 2024-07-04T17:30:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_active": 0, 2024-07-04T17:30:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_send": 0, 2024-07-04T17:30:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_resend": 0, 2024-07-04T17:30:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_active": 0, 2024-07-04T17:30:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_send": 0, 2024-07-04T17:30:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_resend": 0, 2024-07-04T17:30:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: "command_active": 0, 2024-07-04T17:30:07.246 INFO:teuthology.orchestra.run.smithi042.stdout: "command_send": 0, 2024-07-04T17:30:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: "command_resend": 0, 2024-07-04T17:30:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: "map_epoch": 0, 2024-07-04T17:30:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: "map_full": 0, 2024-07-04T17:30:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: "map_inc": 0, 2024-07-04T17:30:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_sessions": 0, 2024-07-04T17:30:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_open": 0, 2024-07-04T17:30:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_close": 0, 2024-07-04T17:30:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_laggy": 0, 2024-07-04T17:30:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_wr": 0, 2024-07-04T17:30:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_rd": 0, 2024-07-04T17:30:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_del": 0 2024-07-04T17:30:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:07.247 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-client": [ 2024-07-04T17:30:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:30:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:30:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:30:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 39, 2024-07-04T17:30:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 146552, 2024-07-04T17:30:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:30:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 39, 2024-07-04T17:30:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:30:07.248 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:30:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 39, 2024-07-04T17:30:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 146552, 2024-07-04T17:30:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:30:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:30:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:30:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:30:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-radosclient": [ 2024-07-04T17:30:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:07.249 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:30:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:30:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:30:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 25, 2024-07-04T17:30:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 795752, 2024-07-04T17:30:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:30:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 25, 2024-07-04T17:30:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:30:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:30:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 25, 2024-07-04T17:30:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 795752, 2024-07-04T17:30:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:30:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:30:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:30:07.250 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:30:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_bytes": [ 2024-07-04T17:30:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:07.251 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:30:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:30:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:30:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:30:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:30:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:30:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:30:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:30:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:30:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:30:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:30:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:30:07.253 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:30:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:30:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:30:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_ops": [ 2024-07-04T17:30:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:30:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1024, 2024-07-04T17:30:07.254 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:30:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:30:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:30:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:30:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:30:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:30:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:30:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:30:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:30:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:30:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:30:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:30:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:30:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:07.255 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:07.256 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-07-04T17:30:07.256 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:30:07.256 DEBUG:tasks.cephfs.test_mirroring:command returned={ "AsyncMessenger::Worker": [ { "labels": { "id": "0" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "1" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "2" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } } ], "AsyncMessenger::Worker-0": [ { "labels": {}, "counters": { "msgr_recv_messages": 39, "msgr_send_messages": 4, "msgr_recv_bytes": 151872, "msgr_send_bytes": 800, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.016514154, "msgr_running_send_time": 0.001912425, "msgr_running_recv_time": 0.008934797, "msgr_running_fast_dispatch_time": 0.004893093, "msgr_send_messages_queue_lat": { "avgcount": 4, "sum": 0.004367369, "avgtime": 0.001091842 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 151872, "msgr_send_encrypted_bytes": 800 } } ], "AsyncMessenger::Worker-1": [ { "labels": {}, "counters": { "msgr_recv_messages": 4, "msgr_send_messages": 49, "msgr_recv_bytes": 576, "msgr_send_bytes": 333696, "msgr_created_connections": 9, "msgr_active_connections": 3, "msgr_running_total_time": 0.021688356, "msgr_running_send_time": 0.007968014, "msgr_running_recv_time": 0.007379601, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 46, "sum": 0.035250221, "avgtime": 0.000766309 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 576, "msgr_send_encrypted_bytes": 333696 } } ], "AsyncMessenger::Worker-2": [ { "labels": {}, "counters": { "msgr_recv_messages": 24, "msgr_send_messages": 7, "msgr_recv_bytes": 799472, "msgr_send_bytes": 1264, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.028104911, "msgr_running_send_time": 0.002026325, "msgr_running_recv_time": 0.047295370, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 7, "sum": 0.003249255, "avgtime": 0.000464179 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 799472, "msgr_send_encrypted_bytes": 1264 } } ], "cct": [ { "labels": {}, "counters": { "total_workers": 0, "unhealthy_workers": 0 } } ], "cephfs_mirror": [ { "labels": {}, "counters": { "mirrored_filesystems": 1, "mirror_enable_failures": 0 } } ], "cephfs_mirror_mirrored_filesystems": [ { "labels": { "filesystem": "cephfs" }, "counters": { "mirroring_peers": 0, "directory_count": 1 } } ], "mempool": [ { "labels": {}, "counters": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_items": 0, "bluefs_bytes": 0, "bluefs_items": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "buffer_anon_bytes": 183529, "buffer_anon_items": 56, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 18720, "osdmap_items": 169, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "mds_co_bytes": 0, "mds_co_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 } } ], "objecter": [ { "labels": {}, "counters": { "op_active": 0, "op_laggy": 0, "op_send": 0, "op_send_bytes": 0, "op_resend": 0, "op_reply": 0, "op_latency": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "op_inflight": 0, "oplen_avg": { "avgcount": 0, "sum": 0 }, "op": 0, "op_r": 0, "op_w": 0, "op_rmw": 0, "op_pg": 0, "osdop_stat": 0, "osdop_create": 0, "osdop_read": 0, "osdop_write": 0, "osdop_writefull": 0, "osdop_writesame": 0, "osdop_append": 0, "osdop_zero": 0, "osdop_truncate": 0, "osdop_delete": 0, "osdop_mapext": 0, "osdop_sparse_read": 0, "osdop_clonerange": 0, "osdop_getxattr": 0, "osdop_setxattr": 0, "osdop_cmpxattr": 0, "osdop_rmxattr": 0, "osdop_resetxattrs": 0, "osdop_call": 0, "osdop_watch": 0, "osdop_notify": 0, "osdop_src_cmpxattr": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_other": 0, "linger_active": 0, "linger_send": 0, "linger_resend": 0, "linger_ping": 0, "poolop_active": 0, "poolop_send": 0, "poolop_resend": 0, "poolstat_active": 0, "poolstat_send": 0, "poolstat_resend": 0, "statfs_active": 0, "statfs_send": 0, "statfs_resend": 0, "command_active": 0, "command_send": 0, "command_resend": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "osd_sessions": 0, "osd_session_open": 0, "osd_session_close": 0, "osd_laggy": 0, "omap_wr": 0, "omap_rd": 0, "omap_del": 0 } } ], "throttle-msgr_dispatch_throttler-client": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 39, "get_sum": 146552, "get_or_fail_fail": 0, "get_or_fail_success": 39, "take": 0, "take_sum": 0, "put": 39, "put_sum": 146552, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-msgr_dispatch_throttler-radosclient": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 25, "get_sum": 795752, "get_or_fail_fail": 0, "get_or_fail_success": 25, "take": 0, "take_sum": 0, "put": 25, "put_sum": 795752, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_bytes": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_ops": [ { "labels": {}, "counters": { "val": 0, "max": 1024, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ] } 2024-07-04T17:30:07.257 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snapshot mirror disable cephfs 2024-07-04T17:30:07.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:07.558+0000 7f6281c00640 1 -- 172.21.15.42:0/4292486502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f627c0ac9d0 msgr2=0x7f627c0aef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:07.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:07.558+0000 7f6281c00640 1 --2- 172.21.15.42:0/4292486502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f627c0ac9d0 0x7f627c0aef80 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f62640097b0 tx=0x7f626402a4a0 comp rx=0 tx=0).stop 2024-07-04T17:30:07.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:07.558+0000 7f6281c00640 1 -- 172.21.15.42:0/4292486502 shutdown_connections 2024-07-04T17:30:07.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:07.558+0000 7f6281c00640 1 --2- 172.21.15.42:0/4292486502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f627c0ac9d0 0x7f627c0aef80 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:07.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:07.558+0000 7f6281c00640 1 -- 172.21.15.42:0/4292486502 >> 172.21.15.42:0/4292486502 conn(0x7f627c0afaa0 msgr2=0x7f627c0afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:07.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:07.558+0000 7f6281c00640 1 -- 172.21.15.42:0/4292486502 shutdown_connections 2024-07-04T17:30:07.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:07.558+0000 7f6281c00640 1 -- 172.21.15.42:0/4292486502 wait complete. 2024-07-04T17:30:07.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:07.561+0000 7f6281c00640 1 Processor -- start 2024-07-04T17:30:07.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:07.561+0000 7f6281c00640 1 -- start start 2024-07-04T17:30:07.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:07.562+0000 7f6281c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f627c0ac9d0 0x7f627c141e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:07.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:07.562+0000 7f6281c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f627c1415a0 con 0x7f627c0ac9d0 2024-07-04T17:30:07.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:07.562+0000 7f627be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f627c0ac9d0 0x7f627c141e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:07.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:07.562+0000 7f627be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f627c0ac9d0 0x7f627c141e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59976/0 (socket says 172.21.15.42:59976) 2024-07-04T17:30:07.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:07.562+0000 7f627be00640 1 -- 172.21.15.42:0/1638830082 learned_addr learned my addr 172.21.15.42:0/1638830082 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:07.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:07.563+0000 7f627be00640 1 -- 172.21.15.42:0/1638830082 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f627c142590 con 0x7f627c0ac9d0 2024-07-04T17:30:07.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:07.563+0000 7f627be00640 1 --2- 172.21.15.42:0/1638830082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f627c0ac9d0 0x7f627c141e60 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f626c00c6b0 tx=0x7f626c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:07.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:07.563+0000 7f6278c00640 1 -- 172.21.15.42:0/1638830082 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f626c007e10 con 0x7f627c0ac9d0 2024-07-04T17:30:07.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:07.563+0000 7f6278c00640 1 -- 172.21.15.42:0/1638830082 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f626c017070 con 0x7f627c0ac9d0 2024-07-04T17:30:07.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:07.563+0000 7f6278c00640 1 -- 172.21.15.42:0/1638830082 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f626c006b50 con 0x7f627c0ac9d0 2024-07-04T17:30:07.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:07.564+0000 7f6281c00640 1 -- 172.21.15.42:0/1638830082 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f627c1461f0 con 0x7f627c0ac9d0 2024-07-04T17:30:07.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:07.564+0000 7f6281c00640 1 -- 172.21.15.42:0/1638830082 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f627c0aab20 con 0x7f627c0ac9d0 2024-07-04T17:30:07.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:07.565+0000 7f6281c00640 1 -- 172.21.15.42:0/1638830082 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f627c141c50 con 0x7f627c0ac9d0 2024-07-04T17:30:07.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:07.565+0000 7f6278c00640 1 -- 172.21.15.42:0/1638830082 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 20) ==== 49681+0+0 (secure 0 0 0) 0x7f626c006cf0 con 0x7f627c0ac9d0 2024-07-04T17:30:07.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:07.565+0000 7f6278c00640 1 --2- 172.21.15.42:0/1638830082 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f625803d5d0 0x7f625803fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:07.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:07.566+0000 7f6278c00640 1 -- 172.21.15.42:0/1638830082 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4733+0+0 (secure 0 0 0) 0x7f626c051ba0 con 0x7f627c0ac9d0 2024-07-04T17:30:07.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:07.566+0000 7f6273e00640 1 --2- 172.21.15.42:0/1638830082 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f625803d5d0 0x7f625803fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:07.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:07.567+0000 7f6273e00640 1 --2- 172.21.15.42:0/1638830082 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f625803d5d0 0x7f625803fae0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6264034000 tx=0x7f6264004730 comp rx=0 tx=0).ready entity=mgr.4853 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:07.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:07.571+0000 7f6278c00640 1 -- 172.21.15.42:0/1638830082 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f627c0b1280 con 0x7f627c0ac9d0 2024-07-04T17:30:07.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:07.805+0000 7f6281c00640 1 -- 172.21.15.42:0/1638830082 --> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] -- mgr_command(tid 0: {"prefix": "fs snapshot mirror disable", "fs_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7f627c1466c0 con 0x7f625803d5d0 2024-07-04T17:30:08.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:08.707+0000 7f6278c00640 1 -- 172.21.15.42:0/1638830082 <== mgr.4853 v2:172.21.15.42:6824/3045931584 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f627c1466c0 con 0x7f625803d5d0 2024-07-04T17:30:08.708 INFO:teuthology.orchestra.run.smithi042.stdout:{} 2024-07-04T17:30:08.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:08.710+0000 7f6281c00640 1 -- 172.21.15.42:0/1638830082 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f625803d5d0 msgr2=0x7f625803fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:08.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:08.710+0000 7f6281c00640 1 --2- 172.21.15.42:0/1638830082 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f625803d5d0 0x7f625803fae0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6264034000 tx=0x7f6264004730 comp rx=0 tx=0).stop 2024-07-04T17:30:08.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:08.710+0000 7f6281c00640 1 -- 172.21.15.42:0/1638830082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f627c0ac9d0 msgr2=0x7f627c141e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:08.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:08.710+0000 7f6281c00640 1 --2- 172.21.15.42:0/1638830082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f627c0ac9d0 0x7f627c141e60 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f626c00c6b0 tx=0x7f626c00cb70 comp rx=0 tx=0).stop 2024-07-04T17:30:08.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:08.711+0000 7f6281c00640 1 -- 172.21.15.42:0/1638830082 shutdown_connections 2024-07-04T17:30:08.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:08.711+0000 7f6281c00640 1 --2- 172.21.15.42:0/1638830082 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f625803d5d0 0x7f625803fae0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:08.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:08.711+0000 7f6281c00640 1 --2- 172.21.15.42:0/1638830082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f627c0ac9d0 0x7f627c141e60 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:08.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:08.711+0000 7f6281c00640 1 -- 172.21.15.42:0/1638830082 >> 172.21.15.42:0/1638830082 conn(0x7f627c0afaa0 msgr2=0x7f627c13f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:08.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:08.711+0000 7f6281c00640 1 -- 172.21.15.42:0/1638830082 shutdown_connections 2024-07-04T17:30:08.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:08.711+0000 7f6281c00640 1 -- 172.21.15.42:0/1638830082 wait complete. 2024-07-04T17:30:18.746 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:30:18.746 DEBUG:teuthology.orchestra.run.smithi042:mirror status for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@4 2024-07-04T17:30:18.999 INFO:teuthology.orchestra.run.smithi042.stderr:no valid command found; 10 closest matches: 2024-07-04T17:30:18.999 INFO:teuthology.orchestra.run.smithi042.stderr:0 2024-07-04T17:30:18.999 INFO:teuthology.orchestra.run.smithi042.stderr:1 2024-07-04T17:30:18.999 INFO:teuthology.orchestra.run.smithi042.stderr:2 2024-07-04T17:30:18.999 INFO:teuthology.orchestra.run.smithi042.stderr:abort 2024-07-04T17:30:18.999 INFO:teuthology.orchestra.run.smithi042.stderr:assert 2024-07-04T17:30:18.999 INFO:teuthology.orchestra.run.smithi042.stderr:config diff 2024-07-04T17:30:18.999 INFO:teuthology.orchestra.run.smithi042.stderr:config diff get 2024-07-04T17:30:18.999 INFO:teuthology.orchestra.run.smithi042.stderr:config get 2024-07-04T17:30:18.999 INFO:teuthology.orchestra.run.smithi042.stderr:config help [] 2024-07-04T17:30:19.000 INFO:teuthology.orchestra.run.smithi042.stderr:config set ... 2024-07-04T17:30:19.000 INFO:teuthology.orchestra.run.smithi042.stderr:admin_socket: invalid command 2024-07-04T17:30:19.009 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-07-04T17:30:19.010 WARNING:tasks.cephfs.test_mirroring:mirror daemon command with label "mirror status for fs: cephfs" failed: Command failed (mirror status for fs: cephfs) on smithi042 with status 22: 'ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@4' 2024-07-04T17:30:19.010 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:30:19.010 DEBUG:teuthology.orchestra.run.smithi042:counter dump for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok counter dump 2024-07-04T17:30:19.255 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:30:19.255 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker": [ 2024-07-04T17:30:19.255 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:19.255 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:30:19.255 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "0" 2024-07-04T17:30:19.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:19.255 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:19.255 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:30:19.256 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:30:19.256 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:19.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:19.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:19.256 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:30:19.256 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "1" 2024-07-04T17:30:19.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:19.256 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:19.256 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:30:19.256 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:30:19.256 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:19.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:19.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:19.256 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:30:19.257 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "2" 2024-07-04T17:30:19.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:19.257 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:19.257 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:30:19.257 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:30:19.257 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:19.257 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:19.257 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:19.257 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": [ 2024-07-04T17:30:19.257 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:19.257 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:19.257 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:19.257 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 40, 2024-07-04T17:30:19.257 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 4, 2024-07-04T17:30:19.258 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 156768, 2024-07-04T17:30:19.258 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 800, 2024-07-04T17:30:19.258 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:30:19.258 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:30:19.258 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.017134369, 2024-07-04T17:30:19.258 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.001999828, 2024-07-04T17:30:19.258 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.009257528, 2024-07-04T17:30:19.258 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.005051975, 2024-07-04T17:30:19.258 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:30:19.258 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 4, 2024-07-04T17:30:19.258 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.004367369, 2024-07-04T17:30:19.258 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.001091842 2024-07-04T17:30:19.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:19.259 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:30:19.259 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:30:19.259 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:30:19.259 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:30:19.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:19.259 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 156768, 2024-07-04T17:30:19.259 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 800 2024-07-04T17:30:19.259 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:19.259 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:19.259 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:19.259 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": [ 2024-07-04T17:30:19.259 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:19.259 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:19.259 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:19.259 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 4, 2024-07-04T17:30:19.260 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 51, 2024-07-04T17:30:19.260 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 576, 2024-07-04T17:30:19.260 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 334208, 2024-07-04T17:30:19.260 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 9, 2024-07-04T17:30:19.260 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 3, 2024-07-04T17:30:19.260 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.022062590, 2024-07-04T17:30:19.260 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.008298532, 2024-07-04T17:30:19.260 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.007379601, 2024-07-04T17:30:19.260 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:30:19.260 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:30:19.260 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 48, 2024-07-04T17:30:19.260 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.035406993, 2024-07-04T17:30:19.260 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000737645 2024-07-04T17:30:19.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:19.261 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:30:19.261 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:30:19.261 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:30:19.261 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:30:19.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:19.261 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 576, 2024-07-04T17:30:19.261 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 334208 2024-07-04T17:30:19.261 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:19.261 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:19.261 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:19.261 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": [ 2024-07-04T17:30:19.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:19.261 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:19.261 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:19.262 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 24, 2024-07-04T17:30:19.262 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 7, 2024-07-04T17:30:19.262 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 799472, 2024-07-04T17:30:19.262 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 1264, 2024-07-04T17:30:19.262 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:30:19.262 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:30:19.262 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.028285453, 2024-07-04T17:30:19.262 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.002115009, 2024-07-04T17:30:19.262 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.047344373, 2024-07-04T17:30:19.262 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:30:19.262 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:30:19.262 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 7, 2024-07-04T17:30:19.262 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.003249255, 2024-07-04T17:30:19.262 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000464179 2024-07-04T17:30:19.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:19.263 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:30:19.263 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:30:19.263 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:30:19.263 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:30:19.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:19.263 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 799472, 2024-07-04T17:30:19.263 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 1264 2024-07-04T17:30:19.263 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:19.263 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:19.263 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:19.263 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": [ 2024-07-04T17:30:19.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:19.263 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:19.264 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:19.264 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-07-04T17:30:19.264 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-07-04T17:30:19.264 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:19.264 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:19.264 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:19.264 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror": [ 2024-07-04T17:30:19.264 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:19.264 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:19.264 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:19.264 INFO:teuthology.orchestra.run.smithi042.stdout: "mirrored_filesystems": 0, 2024-07-04T17:30:19.264 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_enable_failures": 0 2024-07-04T17:30:19.264 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:19.264 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:19.264 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:19.265 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": [ 2024-07-04T17:30:19.265 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:19.265 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:19.265 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:19.265 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-07-04T17:30:19.265 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-07-04T17:30:19.265 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T17:30:19.265 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-07-04T17:30:19.265 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T17:30:19.265 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-07-04T17:30:19.265 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T17:30:19.265 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T17:30:19.265 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T17:30:19.265 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T17:30:19.266 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T17:30:19.266 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-07-04T17:30:19.266 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T17:30:19.266 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T17:30:19.266 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-07-04T17:30:19.266 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-07-04T17:30:19.266 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-07-04T17:30:19.266 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-07-04T17:30:19.266 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T17:30:19.266 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T17:30:19.266 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T17:30:19.266 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T17:30:19.266 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T17:30:19.267 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-07-04T17:30:19.267 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-07-04T17:30:19.267 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-07-04T17:30:19.267 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T17:30:19.267 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T17:30:19.267 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-07-04T17:30:19.267 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-07-04T17:30:19.267 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-07-04T17:30:19.267 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-07-04T17:30:19.267 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T17:30:19.267 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-07-04T17:30:19.267 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T17:30:19.267 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-07-04T17:30:19.268 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 149196, 2024-07-04T17:30:19.268 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 27, 2024-07-04T17:30:19.268 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-07-04T17:30:19.268 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-07-04T17:30:19.268 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-07-04T17:30:19.268 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-07-04T17:30:19.268 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-07-04T17:30:19.268 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-07-04T17:30:19.268 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-07-04T17:30:19.268 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-07-04T17:30:19.268 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 4592, 2024-07-04T17:30:19.269 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 37, 2024-07-04T17:30:19.269 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T17:30:19.269 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-07-04T17:30:19.269 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-07-04T17:30:19.269 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-07-04T17:30:19.269 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 0, 2024-07-04T17:30:19.269 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 0, 2024-07-04T17:30:19.269 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-07-04T17:30:19.269 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-07-04T17:30:19.269 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-07-04T17:30:19.269 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-07-04T17:30:19.269 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:19.269 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:19.270 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:19.270 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter": [ 2024-07-04T17:30:19.270 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:19.270 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:19.270 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:19.270 INFO:teuthology.orchestra.run.smithi042.stdout: "op_active": 0, 2024-07-04T17:30:19.270 INFO:teuthology.orchestra.run.smithi042.stdout: "op_laggy": 0, 2024-07-04T17:30:19.270 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send": 0, 2024-07-04T17:30:19.270 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send_bytes": 0, 2024-07-04T17:30:19.270 INFO:teuthology.orchestra.run.smithi042.stdout: "op_resend": 0, 2024-07-04T17:30:19.270 INFO:teuthology.orchestra.run.smithi042.stdout: "op_reply": 0, 2024-07-04T17:30:19.270 INFO:teuthology.orchestra.run.smithi042.stdout: "op_latency": { 2024-07-04T17:30:19.270 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:30:19.270 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:30:19.271 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:30:19.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:19.271 INFO:teuthology.orchestra.run.smithi042.stdout: "op_inflight": 0, 2024-07-04T17:30:19.271 INFO:teuthology.orchestra.run.smithi042.stdout: "oplen_avg": { 2024-07-04T17:30:19.271 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:30:19.271 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0 2024-07-04T17:30:19.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:19.271 INFO:teuthology.orchestra.run.smithi042.stdout: "op": 0, 2024-07-04T17:30:19.271 INFO:teuthology.orchestra.run.smithi042.stdout: "op_r": 0, 2024-07-04T17:30:19.271 INFO:teuthology.orchestra.run.smithi042.stdout: "op_w": 0, 2024-07-04T17:30:19.271 INFO:teuthology.orchestra.run.smithi042.stdout: "op_rmw": 0, 2024-07-04T17:30:19.271 INFO:teuthology.orchestra.run.smithi042.stdout: "op_pg": 0, 2024-07-04T17:30:19.271 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_stat": 0, 2024-07-04T17:30:19.271 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_create": 0, 2024-07-04T17:30:19.272 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_read": 0, 2024-07-04T17:30:19.272 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_write": 0, 2024-07-04T17:30:19.272 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writefull": 0, 2024-07-04T17:30:19.272 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writesame": 0, 2024-07-04T17:30:19.272 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_append": 0, 2024-07-04T17:30:19.272 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_zero": 0, 2024-07-04T17:30:19.272 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_truncate": 0, 2024-07-04T17:30:19.272 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_delete": 0, 2024-07-04T17:30:19.272 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_mapext": 0, 2024-07-04T17:30:19.272 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_sparse_read": 0, 2024-07-04T17:30:19.272 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_clonerange": 0, 2024-07-04T17:30:19.272 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_getxattr": 0, 2024-07-04T17:30:19.272 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_setxattr": 0, 2024-07-04T17:30:19.272 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_cmpxattr": 0, 2024-07-04T17:30:19.273 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_rmxattr": 0, 2024-07-04T17:30:19.273 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_resetxattrs": 0, 2024-07-04T17:30:19.273 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_call": 0, 2024-07-04T17:30:19.273 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_watch": 0, 2024-07-04T17:30:19.273 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_notify": 0, 2024-07-04T17:30:19.273 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T17:30:19.273 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls": 0, 2024-07-04T17:30:19.273 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls_filter": 0, 2024-07-04T17:30:19.273 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_other": 0, 2024-07-04T17:30:19.273 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_active": 0, 2024-07-04T17:30:19.273 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_send": 0, 2024-07-04T17:30:19.273 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_resend": 0, 2024-07-04T17:30:19.273 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_ping": 0, 2024-07-04T17:30:19.273 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_active": 0, 2024-07-04T17:30:19.273 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_send": 0, 2024-07-04T17:30:19.274 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_resend": 0, 2024-07-04T17:30:19.274 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_active": 0, 2024-07-04T17:30:19.274 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_send": 0, 2024-07-04T17:30:19.274 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_resend": 0, 2024-07-04T17:30:19.274 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_active": 0, 2024-07-04T17:30:19.274 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_send": 0, 2024-07-04T17:30:19.274 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_resend": 0, 2024-07-04T17:30:19.274 INFO:teuthology.orchestra.run.smithi042.stdout: "command_active": 0, 2024-07-04T17:30:19.274 INFO:teuthology.orchestra.run.smithi042.stdout: "command_send": 0, 2024-07-04T17:30:19.274 INFO:teuthology.orchestra.run.smithi042.stdout: "command_resend": 0, 2024-07-04T17:30:19.274 INFO:teuthology.orchestra.run.smithi042.stdout: "map_epoch": 0, 2024-07-04T17:30:19.274 INFO:teuthology.orchestra.run.smithi042.stdout: "map_full": 0, 2024-07-04T17:30:19.275 INFO:teuthology.orchestra.run.smithi042.stdout: "map_inc": 0, 2024-07-04T17:30:19.275 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_sessions": 0, 2024-07-04T17:30:19.275 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_open": 0, 2024-07-04T17:30:19.275 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_close": 0, 2024-07-04T17:30:19.275 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_laggy": 0, 2024-07-04T17:30:19.275 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_wr": 0, 2024-07-04T17:30:19.275 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_rd": 0, 2024-07-04T17:30:19.275 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_del": 0 2024-07-04T17:30:19.275 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:19.275 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:19.275 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:19.275 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-client": [ 2024-07-04T17:30:19.275 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:19.275 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:30:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:30:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:30:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 40, 2024-07-04T17:30:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 151306, 2024-07-04T17:30:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:30:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 40, 2024-07-04T17:30:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:30:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:30:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 40, 2024-07-04T17:30:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 151306, 2024-07-04T17:30:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:30:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:30:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:30:19.277 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:30:19.277 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:19.277 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:19.277 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:19.277 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:19.277 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-radosclient": [ 2024-07-04T17:30:19.277 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:19.277 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:19.277 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:19.277 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:30:19.277 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:30:19.277 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:30:19.277 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 25, 2024-07-04T17:30:19.277 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 795752, 2024-07-04T17:30:19.277 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:30:19.278 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 25, 2024-07-04T17:30:19.278 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:30:19.278 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:30:19.278 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 25, 2024-07-04T17:30:19.278 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 795752, 2024-07-04T17:30:19.278 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:30:19.278 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:30:19.278 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:30:19.278 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:30:19.278 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:19.278 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:19.278 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:19.278 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:19.278 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_bytes": [ 2024-07-04T17:30:19.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:19.279 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:19.279 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:19.279 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:30:19.279 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:30:19.279 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:30:19.279 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:30:19.279 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:30:19.279 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:30:19.279 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:30:19.279 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:30:19.279 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:30:19.279 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:30:19.279 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:30:19.279 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:30:19.280 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:30:19.280 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:30:19.280 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:30:19.280 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:19.280 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:19.280 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:19.280 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:30:19.280 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_ops": [ 2024-07-04T17:30:19.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:19.280 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:30:19.280 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:30:19.280 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:30:19.280 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1024, 2024-07-04T17:30:19.280 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:30:19.280 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:30:19.280 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:30:19.281 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:30:19.281 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:30:19.281 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:30:19.281 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:30:19.281 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:30:19.281 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:30:19.281 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:30:19.281 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:30:19.281 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:30:19.281 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:30:19.281 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:19.281 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:19.281 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:19.281 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-07-04T17:30:19.281 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:30:19.282 DEBUG:tasks.cephfs.test_mirroring:command returned={ "AsyncMessenger::Worker": [ { "labels": { "id": "0" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "1" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "2" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } } ], "AsyncMessenger::Worker-0": [ { "labels": {}, "counters": { "msgr_recv_messages": 40, "msgr_send_messages": 4, "msgr_recv_bytes": 156768, "msgr_send_bytes": 800, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.017134369, "msgr_running_send_time": 0.001999828, "msgr_running_recv_time": 0.009257528, "msgr_running_fast_dispatch_time": 0.005051975, "msgr_send_messages_queue_lat": { "avgcount": 4, "sum": 0.004367369, "avgtime": 0.001091842 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 156768, "msgr_send_encrypted_bytes": 800 } } ], "AsyncMessenger::Worker-1": [ { "labels": {}, "counters": { "msgr_recv_messages": 4, "msgr_send_messages": 51, "msgr_recv_bytes": 576, "msgr_send_bytes": 334208, "msgr_created_connections": 9, "msgr_active_connections": 3, "msgr_running_total_time": 0.022062590, "msgr_running_send_time": 0.008298532, "msgr_running_recv_time": 0.007379601, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 48, "sum": 0.035406993, "avgtime": 0.000737645 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 576, "msgr_send_encrypted_bytes": 334208 } } ], "AsyncMessenger::Worker-2": [ { "labels": {}, "counters": { "msgr_recv_messages": 24, "msgr_send_messages": 7, "msgr_recv_bytes": 799472, "msgr_send_bytes": 1264, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.028285453, "msgr_running_send_time": 0.002115009, "msgr_running_recv_time": 0.047344373, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 7, "sum": 0.003249255, "avgtime": 0.000464179 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 799472, "msgr_send_encrypted_bytes": 1264 } } ], "cct": [ { "labels": {}, "counters": { "total_workers": 0, "unhealthy_workers": 0 } } ], "cephfs_mirror": [ { "labels": {}, "counters": { "mirrored_filesystems": 0, "mirror_enable_failures": 0 } } ], "mempool": [ { "labels": {}, "counters": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_items": 0, "bluefs_bytes": 0, "bluefs_items": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "buffer_anon_bytes": 149196, "buffer_anon_items": 27, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 4592, "osdmap_items": 37, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "mds_co_bytes": 0, "mds_co_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 } } ], "objecter": [ { "labels": {}, "counters": { "op_active": 0, "op_laggy": 0, "op_send": 0, "op_send_bytes": 0, "op_resend": 0, "op_reply": 0, "op_latency": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "op_inflight": 0, "oplen_avg": { "avgcount": 0, "sum": 0 }, "op": 0, "op_r": 0, "op_w": 0, "op_rmw": 0, "op_pg": 0, "osdop_stat": 0, "osdop_create": 0, "osdop_read": 0, "osdop_write": 0, "osdop_writefull": 0, "osdop_writesame": 0, "osdop_append": 0, "osdop_zero": 0, "osdop_truncate": 0, "osdop_delete": 0, "osdop_mapext": 0, "osdop_sparse_read": 0, "osdop_clonerange": 0, "osdop_getxattr": 0, "osdop_setxattr": 0, "osdop_cmpxattr": 0, "osdop_rmxattr": 0, "osdop_resetxattrs": 0, "osdop_call": 0, "osdop_watch": 0, "osdop_notify": 0, "osdop_src_cmpxattr": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_other": 0, "linger_active": 0, "linger_send": 0, "linger_resend": 0, "linger_ping": 0, "poolop_active": 0, "poolop_send": 0, "poolop_resend": 0, "poolstat_active": 0, "poolstat_send": 0, "poolstat_resend": 0, "statfs_active": 0, "statfs_send": 0, "statfs_resend": 0, "command_active": 0, "command_send": 0, "command_resend": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "osd_sessions": 0, "osd_session_open": 0, "osd_session_close": 0, "osd_laggy": 0, "omap_wr": 0, "omap_rd": 0, "omap_del": 0 } } ], "throttle-msgr_dispatch_throttler-client": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 40, "get_sum": 151306, "get_or_fail_fail": 0, "get_or_fail_success": 40, "take": 0, "take_sum": 0, "put": 40, "put_sum": 151306, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-msgr_dispatch_throttler-radosclient": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 25, "get_sum": 795752, "get_or_fail_fail": 0, "get_or_fail_success": 25, "take": 0, "take_sum": 0, "put": 25, "put_sum": 795752, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_bytes": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_ops": [ { "labels": {}, "counters": { "val": 0, "max": 1024, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ] } 2024-07-04T17:30:19.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:30:19.283 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -rf d1) 2024-07-04T17:30:19.413 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable mirroring 2024-07-04T17:30:19.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:19.708+0000 7f96dc800640 1 -- 172.21.15.42:0/42740340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96d40b0c70 msgr2=0x7f96d40b1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:19.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:19.708+0000 7f96dc800640 1 --2- 172.21.15.42:0/42740340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96d40b0c70 0x7f96d40b1930 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f96c80097b0 tx=0x7f96c802a530 comp rx=0 tx=0).stop 2024-07-04T17:30:19.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:19.708+0000 7f96dc800640 1 -- 172.21.15.42:0/42740340 shutdown_connections 2024-07-04T17:30:19.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:19.708+0000 7f96dc800640 1 --2- 172.21.15.42:0/42740340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96d40b0c70 0x7f96d40b1930 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:19.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:19.708+0000 7f96dc800640 1 -- 172.21.15.42:0/42740340 >> 172.21.15.42:0/42740340 conn(0x7f96d406fe40 msgr2=0x7f96d406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:19.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:19.708+0000 7f96dc800640 1 -- 172.21.15.42:0/42740340 shutdown_connections 2024-07-04T17:30:19.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:19.708+0000 7f96dc800640 1 -- 172.21.15.42:0/42740340 wait complete. 2024-07-04T17:30:19.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:19.710+0000 7f96dc800640 1 Processor -- start 2024-07-04T17:30:19.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:19.710+0000 7f96dc800640 1 -- start start 2024-07-04T17:30:19.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:19.712+0000 7f96dc800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96d40b0c70 0x7f96d40bfb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:19.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:19.712+0000 7f96dc800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f96d4073bf0 con 0x7f96d40b0c70 2024-07-04T17:30:19.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:19.712+0000 7f96d9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96d40b0c70 0x7f96d40bfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:19.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:19.712+0000 7f96d9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96d40b0c70 0x7f96d40bfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51448/0 (socket says 172.21.15.42:51448) 2024-07-04T17:30:19.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:19.712+0000 7f96d9e00640 1 -- 172.21.15.42:0/1858034259 learned_addr learned my addr 172.21.15.42:0/1858034259 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:19.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:19.712+0000 7f96d9e00640 1 -- 172.21.15.42:0/1858034259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d41d5d70 con 0x7f96d40b0c70 2024-07-04T17:30:19.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:19.713+0000 7f96d9e00640 1 --2- 172.21.15.42:0/1858034259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96d40b0c70 0x7f96d40bfb20 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f96c80098d0 tx=0x7f96c8033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:19.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:19.713+0000 7f96ce000640 1 -- 172.21.15.42:0/1858034259 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96c8031880 con 0x7f96d40b0c70 2024-07-04T17:30:19.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:19.713+0000 7f96ce000640 1 -- 172.21.15.42:0/1858034259 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96c8031b10 con 0x7f96d40b0c70 2024-07-04T17:30:19.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:19.713+0000 7f96ce000640 1 -- 172.21.15.42:0/1858034259 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96c8031df0 con 0x7f96d40b0c70 2024-07-04T17:30:19.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:19.713+0000 7f96dc800640 1 -- 172.21.15.42:0/1858034259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96d41d7830 con 0x7f96d40b0c70 2024-07-04T17:30:19.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:19.713+0000 7f96dc800640 1 -- 172.21.15.42:0/1858034259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96d41d96d0 con 0x7f96d40b0c70 2024-07-04T17:30:19.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:19.715+0000 7f96ce000640 1 -- 172.21.15.42:0/1858034259 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 20) ==== 49681+0+0 (secure 0 0 0) 0x7f96c80330d0 con 0x7f96d40b0c70 2024-07-04T17:30:19.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:19.715+0000 7f96dc800640 1 -- 172.21.15.42:0/1858034259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96d4074270 con 0x7f96d40b0c70 2024-07-04T17:30:19.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:19.715+0000 7f96ce000640 1 --2- 172.21.15.42:0/1858034259 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f96b803d5d0 0x7f96b803fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:19.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:19.715+0000 7f96ce000640 1 -- 172.21.15.42:0/1858034259 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 4776+0+0 (secure 0 0 0) 0x7f96c8073380 con 0x7f96d40b0c70 2024-07-04T17:30:19.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:19.715+0000 7f96d9400640 1 --2- 172.21.15.42:0/1858034259 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f96b803d5d0 0x7f96b803fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:19.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:19.716+0000 7f96d9400640 1 --2- 172.21.15.42:0/1858034259 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f96b803d5d0 0x7f96b803fae0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f96c4007fd0 tx=0x7f96c4007920 comp rx=0 tx=0).ready entity=mgr.4853 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:19.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:19.721+0000 7f96ce000640 1 -- 172.21.15.42:0/1858034259 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f96c803f030 con 0x7f96d40b0c70 2024-07-04T17:30:19.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:19.989+0000 7f96dc800640 1 -- 172.21.15.42:0/1858034259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "mirroring"} v 0) -- 0x7f96d40afec0 con 0x7f96d40b0c70 2024-07-04T17:30:20.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:20.741+0000 7f96ce000640 1 -- 172.21.15.42:0/1858034259 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "mirroring"}]=0 v21) ==== 89+0+0 (secure 0 0 0) 0x7f96c80730d0 con 0x7f96d40b0c70 2024-07-04T17:30:20.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:20.745+0000 7f96dc800640 1 -- 172.21.15.42:0/1858034259 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f96b803d5d0 msgr2=0x7f96b803fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:20.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:20.745+0000 7f96dc800640 1 --2- 172.21.15.42:0/1858034259 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f96b803d5d0 0x7f96b803fae0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f96c4007fd0 tx=0x7f96c4007920 comp rx=0 tx=0).stop 2024-07-04T17:30:20.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:20.746+0000 7f96dc800640 1 -- 172.21.15.42:0/1858034259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96d40b0c70 msgr2=0x7f96d40bfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:20.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:20.746+0000 7f96dc800640 1 --2- 172.21.15.42:0/1858034259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96d40b0c70 0x7f96d40bfb20 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f96c80098d0 tx=0x7f96c8033ad0 comp rx=0 tx=0).stop 2024-07-04T17:30:20.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:20.746+0000 7f96dc800640 1 -- 172.21.15.42:0/1858034259 shutdown_connections 2024-07-04T17:30:20.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:20.746+0000 7f96dc800640 1 --2- 172.21.15.42:0/1858034259 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f96b803d5d0 0x7f96b803fae0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:20.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:20.746+0000 7f96dc800640 1 --2- 172.21.15.42:0/1858034259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96d40b0c70 0x7f96d40bfb20 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:20.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:20.746+0000 7f96dc800640 1 -- 172.21.15.42:0/1858034259 >> 172.21.15.42:0/1858034259 conn(0x7f96d406fe40 msgr2=0x7f96d4142c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:20.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:20.746+0000 7f96dc800640 1 -- 172.21.15.42:0/1858034259 shutdown_connections 2024-07-04T17:30:20.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:20.747+0000 7f96dc800640 1 -- 172.21.15.42:0/1858034259 wait complete. 2024-07-04T17:30:20.787 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-04T17:30:20.787 DEBUG:teuthology.orchestra.run.smithi042:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-04T17:30:20.873 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-04T17:30:20.873 INFO:tasks.cephfs.mount:Terminating background process 2024-07-04T17:30:20.874 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:30:20.874 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:30:20.921 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:30:20.921 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:30:21.002 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-04T17:30:21.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.318+0000 7fcc6be00640 1 -- 172.21.15.42:0/1413412639 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc64141590 msgr2=0x7fcc641419a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:21.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.318+0000 7fcc6be00640 1 --2- 172.21.15.42:0/1413412639 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc64141590 0x7fcc641419a0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fcc500097b0 tx=0x7fcc5002a530 comp rx=0 tx=0).stop 2024-07-04T17:30:21.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.318+0000 7fcc6be00640 1 -- 172.21.15.42:0/1413412639 shutdown_connections 2024-07-04T17:30:21.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.318+0000 7fcc6be00640 1 --2- 172.21.15.42:0/1413412639 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc64141590 0x7fcc641419a0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:21.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.318+0000 7fcc6be00640 1 -- 172.21.15.42:0/1413412639 >> 172.21.15.42:0/1413412639 conn(0x7fcc6406c940 msgr2=0x7fcc6406ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:21.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.318+0000 7fcc6be00640 1 -- 172.21.15.42:0/1413412639 shutdown_connections 2024-07-04T17:30:21.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.318+0000 7fcc6be00640 1 -- 172.21.15.42:0/1413412639 wait complete. 2024-07-04T17:30:21.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.320+0000 7fcc6be00640 1 Processor -- start 2024-07-04T17:30:21.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.320+0000 7fcc6be00640 1 -- start start 2024-07-04T17:30:21.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.322+0000 7fcc6be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc640bc040 0x7fcc641dd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:21.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.322+0000 7fcc6be00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcc640b7530 con 0x7fcc640bc040 2024-07-04T17:30:21.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.322+0000 7fcc69400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc640bc040 0x7fcc641dd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:21.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.322+0000 7fcc69400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc640bc040 0x7fcc641dd0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51482/0 (socket says 172.21.15.42:51482) 2024-07-04T17:30:21.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.322+0000 7fcc69400640 1 -- 172.21.15.42:0/1997318638 learned_addr learned my addr 172.21.15.42:0/1997318638 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:21.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.323+0000 7fcc69400640 1 -- 172.21.15.42:0/1997318638 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc641d81b0 con 0x7fcc640bc040 2024-07-04T17:30:21.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.323+0000 7fcc69400640 1 --2- 172.21.15.42:0/1997318638 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc640bc040 0x7fcc641dd0f0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800c3b0 tx=0x7fcc5800c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:21.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.323+0000 7fcc61600640 1 -- 172.21.15.42:0/1997318638 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc58011780 con 0x7fcc640bc040 2024-07-04T17:30:21.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.323+0000 7fcc61600640 1 -- 172.21.15.42:0/1997318638 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc58017070 con 0x7fcc640bc040 2024-07-04T17:30:21.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.323+0000 7fcc61600640 1 -- 172.21.15.42:0/1997318638 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc58011c00 con 0x7fcc640bc040 2024-07-04T17:30:21.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.323+0000 7fcc6be00640 1 -- 172.21.15.42:0/1997318638 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc641d7b30 con 0x7fcc640bc040 2024-07-04T17:30:21.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.323+0000 7fcc6be00640 1 -- 172.21.15.42:0/1997318638 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc641d7e30 con 0x7fcc640bc040 2024-07-04T17:30:21.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.325+0000 7fcc61600640 1 -- 172.21.15.42:0/1997318638 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 21) ==== 49668+0+0 (secure 0 0 0) 0x7fcc5801c460 con 0x7fcc640bc040 2024-07-04T17:30:21.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.325+0000 7fcc6be00640 1 -- 172.21.15.42:0/1997318638 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc640b7bb0 con 0x7fcc640bc040 2024-07-04T17:30:21.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.325+0000 7fcc61600640 1 --2- 172.21.15.42:0/1997318638 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7fcc4403d580 0x7fcc4403fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:21.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.325+0000 7fcc61600640 1 -- 172.21.15.42:0/1997318638 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 4776+0+0 (secure 0 0 0) 0x7fcc58051c00 con 0x7fcc640bc040 2024-07-04T17:30:21.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.325+0000 7fcc68a00640 1 -- 172.21.15.42:0/1997318638 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7fcc4403d580 msgr2=0x7fcc4403fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/3045931584 2024-07-04T17:30:21.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.325+0000 7fcc68a00640 1 --2- 172.21.15.42:0/1997318638 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7fcc4403d580 0x7fcc4403fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:30:21.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.331+0000 7fcc61600640 1 -- 172.21.15.42:0/1997318638 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fcc58010c60 con 0x7fcc640bc040 2024-07-04T17:30:21.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.527+0000 7fcc68a00640 1 -- 172.21.15.42:0/1997318638 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7fcc4403d580 msgr2=0x7fcc4403fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/3045931584 2024-07-04T17:30:21.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.527+0000 7fcc68a00640 1 --2- 172.21.15.42:0/1997318638 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7fcc4403d580 0x7fcc4403fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:30:21.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.572+0000 7fcc6be00640 1 -- 172.21.15.42:0/1997318638 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcc641d8380 con 0x7fcc640bc040 2024-07-04T17:30:21.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.573+0000 7fcc61600640 1 -- 172.21.15.42:0/1997318638 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 11 entries v56) ==== 81+0+639 (secure 0 0 0) 0x7fcc5801c800 con 0x7fcc640bc040 2024-07-04T17:30:21.573 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/7072365 2024-07-04T18:30:09.692358+0000 2024-07-04T17:30:21.573 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1810008304 2024-07-05T17:29:38.572697+0000 2024-07-04T17:30:21.573 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1534342117 2024-07-05T17:29:38.572697+0000 2024-07-04T17:30:21.573 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6825/2638271195 2024-07-05T17:29:38.572697+0000 2024-07-04T17:30:21.573 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6824/2638271195 2024-07-05T17:29:38.572697+0000 2024-07-04T17:30:21.573 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/591942290 2024-07-05T17:29:38.572697+0000 2024-07-04T17:30:21.573 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2926090870 2024-07-05T17:29:38.572697+0000 2024-07-04T17:30:21.573 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/136286797 2024-07-05T17:29:38.572697+0000 2024-07-04T17:30:21.573 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2800592161 2024-07-05T17:29:38.572697+0000 2024-07-04T17:30:21.574 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1891351459 2024-07-05T17:29:38.572697+0000 2024-07-04T17:30:21.574 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3114375174 2024-07-05T17:29:38.572697+0000 2024-07-04T17:30:21.574 INFO:teuthology.orchestra.run.smithi042.stderr:listed 11 entries 2024-07-04T17:30:21.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.575+0000 7fcc6be00640 1 -- 172.21.15.42:0/1997318638 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7fcc4403d580 msgr2=0x7fcc4403fa90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:30:21.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.575+0000 7fcc6be00640 1 --2- 172.21.15.42:0/1997318638 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7fcc4403d580 0x7fcc4403fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:21.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.575+0000 7fcc6be00640 1 -- 172.21.15.42:0/1997318638 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc640bc040 msgr2=0x7fcc641dd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:21.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.575+0000 7fcc6be00640 1 --2- 172.21.15.42:0/1997318638 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc640bc040 0x7fcc641dd0f0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800c3b0 tx=0x7fcc5800c870 comp rx=0 tx=0).stop 2024-07-04T17:30:21.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.575+0000 7fcc6be00640 1 -- 172.21.15.42:0/1997318638 shutdown_connections 2024-07-04T17:30:21.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.575+0000 7fcc6be00640 1 --2- 172.21.15.42:0/1997318638 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7fcc4403d580 0x7fcc4403fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:21.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.575+0000 7fcc6be00640 1 --2- 172.21.15.42:0/1997318638 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc640bc040 0x7fcc641dd0f0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:21.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.575+0000 7fcc6be00640 1 -- 172.21.15.42:0/1997318638 >> 172.21.15.42:0/1997318638 conn(0x7fcc6406c940 msgr2=0x7fcc64145880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:21.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.575+0000 7fcc6be00640 1 -- 172.21.15.42:0/1997318638 shutdown_connections 2024-07-04T17:30:21.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:21.575+0000 7fcc6be00640 1 -- 172.21.15.42:0/1997318638 wait complete. 2024-07-04T17:30:21.600 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi042.front.sepia.ceph.com... 2024-07-04T17:30:21.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:30:21.600 DEBUG:teuthology.orchestra.run.smithi042:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-07-04T17:30:21.671 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[51270]: fuse finished with error 0 and tester_r 0 2024-07-04T17:30:21.676 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-07-04T17:30:21.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:30:21.676 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-04T17:30:21.720 INFO:tasks.cephfs.mount:Terminating background process 2024-07-04T17:30:21.720 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:30:21.720 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:30:21.811 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:30:21.812 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:30:21.857 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-04T17:30:22.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.172+0000 7f5a5b200640 1 -- 172.21.15.42:0/2522616008 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a5406cc80 msgr2=0x7f5a540b3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:22.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.172+0000 7f5a5b200640 1 --2- 172.21.15.42:0/2522616008 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a5406cc80 0x7f5a540b3650 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0097b0 tx=0x7f5a3c02a530 comp rx=0 tx=0).stop 2024-07-04T17:30:22.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.172+0000 7f5a5b200640 1 -- 172.21.15.42:0/2522616008 shutdown_connections 2024-07-04T17:30:22.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.172+0000 7f5a5b200640 1 --2- 172.21.15.42:0/2522616008 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a5406cc80 0x7f5a540b3650 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:22.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.172+0000 7f5a5b200640 1 -- 172.21.15.42:0/2522616008 >> 172.21.15.42:0/2522616008 conn(0x7f5a540bf700 msgr2=0x7f5a540af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:22.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.173+0000 7f5a5b200640 1 -- 172.21.15.42:0/2522616008 shutdown_connections 2024-07-04T17:30:22.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.173+0000 7f5a5b200640 1 -- 172.21.15.42:0/2522616008 wait complete. 2024-07-04T17:30:22.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.174+0000 7f5a5b200640 1 Processor -- start 2024-07-04T17:30:22.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.175+0000 7f5a5b200640 1 -- start start 2024-07-04T17:30:22.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.176+0000 7f5a5b200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a5406cc80 0x7f5a540b5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:22.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.176+0000 7f5a5b200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5a541463b0 con 0x7f5a5406cc80 2024-07-04T17:30:22.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.176+0000 7f5a53e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a5406cc80 0x7f5a540b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:22.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.176+0000 7f5a53e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a5406cc80 0x7f5a540b5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51488/0 (socket says 172.21.15.42:51488) 2024-07-04T17:30:22.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.176+0000 7f5a53e00640 1 -- 172.21.15.42:0/2646499913 learned_addr learned my addr 172.21.15.42:0/2646499913 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:22.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.177+0000 7f5a53e00640 1 -- 172.21.15.42:0/2646499913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a540b5890 con 0x7f5a5406cc80 2024-07-04T17:30:22.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.177+0000 7f5a53e00640 1 --2- 172.21.15.42:0/2646499913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a5406cc80 0x7f5a540b5350 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f5a4400c320 tx=0x7f5a4400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:22.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.177+0000 7f5a50c00640 1 -- 172.21.15.42:0/2646499913 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a440116c0 con 0x7f5a5406cc80 2024-07-04T17:30:22.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.177+0000 7f5a50c00640 1 -- 172.21.15.42:0/2646499913 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a44017070 con 0x7f5a5406cc80 2024-07-04T17:30:22.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.177+0000 7f5a50c00640 1 -- 172.21.15.42:0/2646499913 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a44011b40 con 0x7f5a5406cc80 2024-07-04T17:30:22.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.177+0000 7f5a5b200640 1 -- 172.21.15.42:0/2646499913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a541da800 con 0x7f5a5406cc80 2024-07-04T17:30:22.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.177+0000 7f5a5b200640 1 -- 172.21.15.42:0/2646499913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a541432f0 con 0x7f5a5406cc80 2024-07-04T17:30:22.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.179+0000 7f5a50c00640 1 -- 172.21.15.42:0/2646499913 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 21) ==== 49668+0+0 (secure 0 0 0) 0x7f5a4401c460 con 0x7f5a5406cc80 2024-07-04T17:30:22.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.179+0000 7f5a5b200640 1 -- 172.21.15.42:0/2646499913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a540b3650 con 0x7f5a5406cc80 2024-07-04T17:30:22.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.179+0000 7f5a50c00640 1 --2- 172.21.15.42:0/2646499913 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f5a3003d530 0x7f5a3003fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:22.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.179+0000 7f5a50c00640 1 -- 172.21.15.42:0/2646499913 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 4776+0+0 (secure 0 0 0) 0x7f5a44051c70 con 0x7f5a5406cc80 2024-07-04T17:30:22.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.179+0000 7f5a4be00640 1 -- 172.21.15.42:0/2646499913 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f5a3003d530 msgr2=0x7f5a3003fa40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/3045931584 2024-07-04T17:30:22.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.179+0000 7f5a4be00640 1 --2- 172.21.15.42:0/2646499913 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f5a3003d530 0x7f5a3003fa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:30:22.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.189+0000 7f5a50c00640 1 -- 172.21.15.42:0/2646499913 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f5a44058080 con 0x7f5a5406cc80 2024-07-04T17:30:22.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.380+0000 7f5a4be00640 1 -- 172.21.15.42:0/2646499913 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f5a3003d530 msgr2=0x7f5a3003fa40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/3045931584 2024-07-04T17:30:22.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.380+0000 7f5a4be00640 1 --2- 172.21.15.42:0/2646499913 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f5a3003d530 0x7f5a3003fa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:30:22.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.409+0000 7f5a5b200640 1 -- 172.21.15.42:0/2646499913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5a54069b70 con 0x7f5a5406cc80 2024-07-04T17:30:22.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.410+0000 7f5a50c00640 1 -- 172.21.15.42:0/2646499913 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 11 entries v56) ==== 81+0+639 (secure 0 0 0) 0x7f5a540b06c0 con 0x7f5a5406cc80 2024-07-04T17:30:22.411 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/7072365 2024-07-04T18:30:09.692358+0000 2024-07-04T17:30:22.411 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1810008304 2024-07-05T17:29:38.572697+0000 2024-07-04T17:30:22.411 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1534342117 2024-07-05T17:29:38.572697+0000 2024-07-04T17:30:22.411 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6825/2638271195 2024-07-05T17:29:38.572697+0000 2024-07-04T17:30:22.411 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6824/2638271195 2024-07-05T17:29:38.572697+0000 2024-07-04T17:30:22.411 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/591942290 2024-07-05T17:29:38.572697+0000 2024-07-04T17:30:22.411 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2926090870 2024-07-05T17:29:38.572697+0000 2024-07-04T17:30:22.411 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/136286797 2024-07-05T17:29:38.572697+0000 2024-07-04T17:30:22.411 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2800592161 2024-07-05T17:29:38.572697+0000 2024-07-04T17:30:22.411 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1891351459 2024-07-05T17:29:38.572697+0000 2024-07-04T17:30:22.411 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3114375174 2024-07-05T17:29:38.572697+0000 2024-07-04T17:30:22.412 INFO:teuthology.orchestra.run.smithi042.stderr:listed 11 entries 2024-07-04T17:30:22.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.413+0000 7f5a5b200640 1 -- 172.21.15.42:0/2646499913 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f5a3003d530 msgr2=0x7f5a3003fa40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:30:22.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.413+0000 7f5a5b200640 1 --2- 172.21.15.42:0/2646499913 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f5a3003d530 0x7f5a3003fa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:22.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.413+0000 7f5a5b200640 1 -- 172.21.15.42:0/2646499913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a5406cc80 msgr2=0x7f5a540b5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:22.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.413+0000 7f5a5b200640 1 --2- 172.21.15.42:0/2646499913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a5406cc80 0x7f5a540b5350 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f5a4400c320 tx=0x7f5a4400c7e0 comp rx=0 tx=0).stop 2024-07-04T17:30:22.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.413+0000 7f5a5b200640 1 -- 172.21.15.42:0/2646499913 shutdown_connections 2024-07-04T17:30:22.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.413+0000 7f5a5b200640 1 --2- 172.21.15.42:0/2646499913 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f5a3003d530 0x7f5a3003fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:22.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.413+0000 7f5a5b200640 1 --2- 172.21.15.42:0/2646499913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a5406cc80 0x7f5a540b5350 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:22.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.413+0000 7f5a5b200640 1 -- 172.21.15.42:0/2646499913 >> 172.21.15.42:0/2646499913 conn(0x7f5a540bf700 msgr2=0x7f5a54075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:22.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.414+0000 7f5a5b200640 1 -- 172.21.15.42:0/2646499913 shutdown_connections 2024-07-04T17:30:22.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.414+0000 7f5a5b200640 1 -- 172.21.15.42:0/2646499913 wait complete. 2024-07-04T17:30:22.437 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi042.front.sepia.ceph.com... 2024-07-04T17:30:22.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:30:22.437 DEBUG:teuthology.orchestra.run.smithi042:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-07-04T17:30:22.503 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi042.stderr:ceph-fuse[51739]: fuse finished with error 0 and tester_r 0 2024-07-04T17:30:22.510 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-07-04T17:30:22.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:30:22.510 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-07-04T17:30:22.635 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:30:22.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.913+0000 7f7d3c000640 1 -- 172.21.15.42:0/1502249474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d340bbff0 msgr2=0x7f7d340bd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:22.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.913+0000 7f7d3c000640 1 --2- 172.21.15.42:0/1502249474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d340bbff0 0x7f7d340bd370 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f7d28005fd0 tx=0x7f7d2802a8b0 comp rx=0 tx=0).stop 2024-07-04T17:30:22.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.913+0000 7f7d3c000640 1 -- 172.21.15.42:0/1502249474 shutdown_connections 2024-07-04T17:30:22.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.913+0000 7f7d3c000640 1 --2- 172.21.15.42:0/1502249474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d340bbff0 0x7f7d340bd370 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:22.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.913+0000 7f7d3c000640 1 -- 172.21.15.42:0/1502249474 >> 172.21.15.42:0/1502249474 conn(0x7f7d3406c9a0 msgr2=0x7f7d3406de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:22.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.913+0000 7f7d3c000640 1 -- 172.21.15.42:0/1502249474 shutdown_connections 2024-07-04T17:30:22.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.913+0000 7f7d3c000640 1 -- 172.21.15.42:0/1502249474 wait complete. 2024-07-04T17:30:22.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.916+0000 7f7d3c000640 1 Processor -- start 2024-07-04T17:30:22.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.916+0000 7f7d3c000640 1 -- start start 2024-07-04T17:30:22.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.917+0000 7f7d3c000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d340bbff0 0x7f7d341e6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:22.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.917+0000 7f7d3c000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7d340b7530 con 0x7f7d340bbff0 2024-07-04T17:30:22.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.918+0000 7f7d39600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d340bbff0 0x7f7d341e6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:22.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.918+0000 7f7d39600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d340bbff0 0x7f7d341e6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51512/0 (socket says 172.21.15.42:51512) 2024-07-04T17:30:22.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.918+0000 7f7d39600640 1 -- 172.21.15.42:0/1041549674 learned_addr learned my addr 172.21.15.42:0/1041549674 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:22.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.918+0000 7f7d39600640 1 -- 172.21.15.42:0/1041549674 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d34145b80 con 0x7f7d340bbff0 2024-07-04T17:30:22.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.919+0000 7f7d39600640 1 --2- 172.21.15.42:0/1041549674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d340bbff0 0x7f7d341e6d90 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f7d28006010 tx=0x7f7d28033b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:22.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.919+0000 7f7d31600640 1 -- 172.21.15.42:0/1041549674 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d280319f0 con 0x7f7d340bbff0 2024-07-04T17:30:22.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.919+0000 7f7d31600640 1 -- 172.21.15.42:0/1041549674 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d28031b90 con 0x7f7d340bbff0 2024-07-04T17:30:22.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.919+0000 7f7d31600640 1 -- 172.21.15.42:0/1041549674 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d280330d0 con 0x7f7d340bbff0 2024-07-04T17:30:22.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.919+0000 7f7d3c000640 1 -- 172.21.15.42:0/1041549674 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d34145e20 con 0x7f7d340bbff0 2024-07-04T17:30:22.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.919+0000 7f7d3c000640 1 -- 172.21.15.42:0/1041549674 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d34137c40 con 0x7f7d340bbff0 2024-07-04T17:30:22.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.921+0000 7f7d3c000640 1 -- 172.21.15.42:0/1041549674 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d341e76a0 con 0x7f7d340bbff0 2024-07-04T17:30:22.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.921+0000 7f7d31600640 1 -- 172.21.15.42:0/1041549674 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 21) ==== 49668+0+0 (secure 0 0 0) 0x7f7d28033270 con 0x7f7d340bbff0 2024-07-04T17:30:22.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.921+0000 7f7d31600640 1 --2- 172.21.15.42:0/1041549674 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f7d1403d580 0x7f7d1403fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:22.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.921+0000 7f7d31600640 1 -- 172.21.15.42:0/1041549674 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 4776+0+0 (secure 0 0 0) 0x7f7d28073630 con 0x7f7d340bbff0 2024-07-04T17:30:22.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.921+0000 7f7d38c00640 1 -- 172.21.15.42:0/1041549674 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f7d1403d580 msgr2=0x7f7d1403fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/3045931584 2024-07-04T17:30:22.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.921+0000 7f7d38c00640 1 --2- 172.21.15.42:0/1041549674 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f7d1403d580 0x7f7d1403fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:30:22.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:22.926+0000 7f7d31600640 1 -- 172.21.15.42:0/1041549674 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f7d2803f030 con 0x7f7d340bbff0 2024-07-04T17:30:23.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.122+0000 7f7d38c00640 1 -- 172.21.15.42:0/1041549674 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f7d1403d580 msgr2=0x7f7d1403fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/3045931584 2024-07-04T17:30:23.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.122+0000 7f7d38c00640 1 --2- 172.21.15.42:0/1041549674 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f7d1403d580 0x7f7d1403fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:30:23.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.173+0000 7f7d3c000640 1 -- 172.21.15.42:0/1041549674 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d340b7b70 con 0x7f7d340bbff0 2024-07-04T17:30:23.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.174+0000 7f7d31600640 1 -- 172.21.15.42:0/1041549674 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+7364 (secure 0 0 0) 0x7f7d28033610 con 0x7f7d340bbff0 2024-07-04T17:30:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:30:23.174 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":43,"btime":"2024-07-04T17:30:08:691538+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/343791977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":343791977},{"type":"v1","addr":"172.21.15.42:6827","nonce":343791977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":4722,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/3172446281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":3172446281},{"type":"v1","addr":"172.21.15.42:6829","nonce":3172446281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":4727,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/1866310217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1866310217},{"type":"v1","addr":"172.21.15.42:6831","nonce":1866310217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:29:05.313061+0000","modified":"2024-07-04T17:30:07.807707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4731},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4731":{"gid":4731,"name":"d","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6835/1327681666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1327681666},{"type":"v1","addr":"172.21.15.42:6835","nonce":1327681666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4731,"qdb_cluster":[4731]},"id":4},{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:29:25.424728+0000","modified":"2024-07-04T17:29:28.496007+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4728":{"gid":4728,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":9,"addr":"172.21.15.42:6833/1072723036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1072723036},{"type":"v1","addr":"172.21.15.42:6833","nonce":1072723036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4728,"qdb_cluster":[4728]},"id":5}]} 2024-07-04T17:30:23.175 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 43 2024-07-04T17:30:23.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.176+0000 7f7d3c000640 1 -- 172.21.15.42:0/1041549674 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f7d1403d580 msgr2=0x7f7d1403fa90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:30:23.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.176+0000 7f7d3c000640 1 --2- 172.21.15.42:0/1041549674 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f7d1403d580 0x7f7d1403fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:23.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.176+0000 7f7d3c000640 1 -- 172.21.15.42:0/1041549674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d340bbff0 msgr2=0x7f7d341e6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:23.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.176+0000 7f7d3c000640 1 --2- 172.21.15.42:0/1041549674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d340bbff0 0x7f7d341e6d90 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f7d28006010 tx=0x7f7d28033b10 comp rx=0 tx=0).stop 2024-07-04T17:30:23.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.177+0000 7f7d3c000640 1 -- 172.21.15.42:0/1041549674 shutdown_connections 2024-07-04T17:30:23.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.177+0000 7f7d3c000640 1 --2- 172.21.15.42:0/1041549674 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f7d1403d580 0x7f7d1403fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:23.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.177+0000 7f7d3c000640 1 --2- 172.21.15.42:0/1041549674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d340bbff0 0x7f7d341e6d90 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:23.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.177+0000 7f7d3c000640 1 -- 172.21.15.42:0/1041549674 >> 172.21.15.42:0/1041549674 conn(0x7f7d3406c9a0 msgr2=0x7f7d340740b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:23.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.177+0000 7f7d3c000640 1 -- 172.21.15.42:0/1041549674 shutdown_connections 2024-07-04T17:30:23.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.177+0000 7f7d3c000640 1 -- 172.21.15.42:0/1041549674 wait complete. 2024-07-04T17:30:23.204 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:30:23.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.475+0000 7f69e2c00640 1 -- 172.21.15.42:0/204811540 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69dc0bad70 msgr2=0x7f69dc06c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:23.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.475+0000 7f69e2c00640 1 --2- 172.21.15.42:0/204811540 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69dc0bad70 0x7f69dc06c2c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f69cc00b2b0 tx=0x7f69cc02ac80 comp rx=0 tx=0).stop 2024-07-04T17:30:23.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.475+0000 7f69e2c00640 1 -- 172.21.15.42:0/204811540 shutdown_connections 2024-07-04T17:30:23.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.475+0000 7f69e2c00640 1 --2- 172.21.15.42:0/204811540 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69dc0bad70 0x7f69dc06c2c0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:23.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.475+0000 7f69e2c00640 1 -- 172.21.15.42:0/204811540 >> 172.21.15.42:0/204811540 conn(0x7f69dc0bebc0 msgr2=0x7f69dc0befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:23.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.475+0000 7f69e2c00640 1 -- 172.21.15.42:0/204811540 shutdown_connections 2024-07-04T17:30:23.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.475+0000 7f69e2c00640 1 -- 172.21.15.42:0/204811540 wait complete. 2024-07-04T17:30:23.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.477+0000 7f69e2c00640 1 Processor -- start 2024-07-04T17:30:23.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.477+0000 7f69e2c00640 1 -- start start 2024-07-04T17:30:23.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.479+0000 7f69e2c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69dc0b5650 0x7f69dc0707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:23.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.479+0000 7f69e2c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f69dc0b27c0 con 0x7f69dc0b5650 2024-07-04T17:30:23.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.479+0000 7f69e1800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69dc0b5650 0x7f69dc0707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:23.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.479+0000 7f69e1800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69dc0b5650 0x7f69dc0707e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51530/0 (socket says 172.21.15.42:51530) 2024-07-04T17:30:23.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.479+0000 7f69e1800640 1 -- 172.21.15.42:0/246762450 learned_addr learned my addr 172.21.15.42:0/246762450 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:23.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.479+0000 7f69e1800640 1 -- 172.21.15.42:0/246762450 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69dc0b06d0 con 0x7f69dc0b5650 2024-07-04T17:30:23.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.480+0000 7f69e1800640 1 --2- 172.21.15.42:0/246762450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69dc0b5650 0x7f69dc0707e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f69cc000c00 tx=0x7f69cc0097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:23.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.480+0000 7f69d9600640 1 -- 172.21.15.42:0/246762450 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69cc038040 con 0x7f69dc0b5650 2024-07-04T17:30:23.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.480+0000 7f69d9600640 1 -- 172.21.15.42:0/246762450 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69cc041070 con 0x7f69dc0b5650 2024-07-04T17:30:23.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.480+0000 7f69e2c00640 1 -- 172.21.15.42:0/246762450 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69dc072200 con 0x7f69dc0b5650 2024-07-04T17:30:23.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.480+0000 7f69d9600640 1 -- 172.21.15.42:0/246762450 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69cc0042f0 con 0x7f69dc0b5650 2024-07-04T17:30:23.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.480+0000 7f69e2c00640 1 -- 172.21.15.42:0/246762450 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69dc13f6d0 con 0x7f69dc0b5650 2024-07-04T17:30:23.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.481+0000 7f69e2c00640 1 -- 172.21.15.42:0/246762450 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69dc06c2c0 con 0x7f69dc0b5650 2024-07-04T17:30:23.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.481+0000 7f69d9600640 1 -- 172.21.15.42:0/246762450 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 21) ==== 49668+0+0 (secure 0 0 0) 0x7f69cc0035f0 con 0x7f69dc0b5650 2024-07-04T17:30:23.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.482+0000 7f69d9600640 1 --2- 172.21.15.42:0/246762450 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f69bc03d580 0x7f69bc03fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:23.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.482+0000 7f69d9600640 1 -- 172.21.15.42:0/246762450 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 4776+0+0 (secure 0 0 0) 0x7f69cc072580 con 0x7f69dc0b5650 2024-07-04T17:30:23.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.482+0000 7f69e0e00640 1 -- 172.21.15.42:0/246762450 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f69bc03d580 msgr2=0x7f69bc03fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/3045931584 2024-07-04T17:30:23.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.482+0000 7f69e0e00640 1 --2- 172.21.15.42:0/246762450 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f69bc03d580 0x7f69bc03fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:30:23.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.487+0000 7f69d9600640 1 -- 172.21.15.42:0/246762450 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f69cc031e20 con 0x7f69dc0b5650 2024-07-04T17:30:23.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.684+0000 7f69e0e00640 1 -- 172.21.15.42:0/246762450 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f69bc03d580 msgr2=0x7f69bc03fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/3045931584 2024-07-04T17:30:23.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.684+0000 7f69e0e00640 1 --2- 172.21.15.42:0/246762450 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f69bc03d580 0x7f69bc03fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:30:23.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.727+0000 7f69e2c00640 1 -- 172.21.15.42:0/246762450 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69dc068e30 con 0x7f69dc0b5650 2024-07-04T17:30:23.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.728+0000 7f69d9600640 1 -- 172.21.15.42:0/246762450 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+7364 (secure 0 0 0) 0x7f69cc003990 con 0x7f69dc0b5650 2024-07-04T17:30:23.728 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:30:23.728 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":43,"btime":"2024-07-04T17:30:08:691538+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/343791977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":343791977},{"type":"v1","addr":"172.21.15.42:6827","nonce":343791977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":4722,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/3172446281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":3172446281},{"type":"v1","addr":"172.21.15.42:6829","nonce":3172446281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":4727,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/1866310217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1866310217},{"type":"v1","addr":"172.21.15.42:6831","nonce":1866310217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:29:05.313061+0000","modified":"2024-07-04T17:30:07.807707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4731},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4731":{"gid":4731,"name":"d","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6835/1327681666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1327681666},{"type":"v1","addr":"172.21.15.42:6835","nonce":1327681666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4731,"qdb_cluster":[4731]},"id":4},{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:29:25.424728+0000","modified":"2024-07-04T17:29:28.496007+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4728":{"gid":4728,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":9,"addr":"172.21.15.42:6833/1072723036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1072723036},{"type":"v1","addr":"172.21.15.42:6833","nonce":1072723036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4728,"qdb_cluster":[4728]},"id":5}]} 2024-07-04T17:30:23.729 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 43 2024-07-04T17:30:23.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.731+0000 7f69e2c00640 1 -- 172.21.15.42:0/246762450 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f69bc03d580 msgr2=0x7f69bc03fa90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:30:23.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.731+0000 7f69e2c00640 1 --2- 172.21.15.42:0/246762450 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f69bc03d580 0x7f69bc03fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:23.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.731+0000 7f69e2c00640 1 -- 172.21.15.42:0/246762450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69dc0b5650 msgr2=0x7f69dc0707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:23.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.731+0000 7f69e2c00640 1 --2- 172.21.15.42:0/246762450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69dc0b5650 0x7f69dc0707e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f69cc000c00 tx=0x7f69cc0097d0 comp rx=0 tx=0).stop 2024-07-04T17:30:23.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.731+0000 7f69e2c00640 1 -- 172.21.15.42:0/246762450 shutdown_connections 2024-07-04T17:30:23.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.731+0000 7f69e2c00640 1 --2- 172.21.15.42:0/246762450 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f69bc03d580 0x7f69bc03fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:23.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.731+0000 7f69e2c00640 1 --2- 172.21.15.42:0/246762450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69dc0b5650 0x7f69dc0707e0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:23.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.731+0000 7f69e2c00640 1 -- 172.21.15.42:0/246762450 >> 172.21.15.42:0/246762450 conn(0x7f69dc0bebc0 msgr2=0x7f69dc13fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:23.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.731+0000 7f69e2c00640 1 -- 172.21.15.42:0/246762450 shutdown_connections 2024-07-04T17:30:23.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:23.731+0000 7f69e2c00640 1 -- 172.21.15.42:0/246762450 wait complete. 2024-07-04T17:30:23.754 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:30:24.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.033+0000 7f8620a00640 1 -- 172.21.15.42:0/58924278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f861c0b2d20 msgr2=0x7f861c0b37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:24.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.033+0000 7f8620a00640 1 --2- 172.21.15.42:0/58924278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f861c0b2d20 0x7f861c0b37d0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f860c005fd0 tx=0x7f860c02a5f0 comp rx=0 tx=0).stop 2024-07-04T17:30:24.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.033+0000 7f8620a00640 1 -- 172.21.15.42:0/58924278 shutdown_connections 2024-07-04T17:30:24.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.033+0000 7f8620a00640 1 --2- 172.21.15.42:0/58924278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f861c0b2d20 0x7f861c0b37d0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:24.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.033+0000 7f8620a00640 1 -- 172.21.15.42:0/58924278 >> 172.21.15.42:0/58924278 conn(0x7f861c0bf700 msgr2=0x7f861c0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:24.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.033+0000 7f8620a00640 1 -- 172.21.15.42:0/58924278 shutdown_connections 2024-07-04T17:30:24.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.033+0000 7f8620a00640 1 -- 172.21.15.42:0/58924278 wait complete. 2024-07-04T17:30:24.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.035+0000 7f8620a00640 1 Processor -- start 2024-07-04T17:30:24.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.035+0000 7f8620a00640 1 -- start start 2024-07-04T17:30:24.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.036+0000 7f8620a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f861c0b01f0 0x7f861c13d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:24.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.036+0000 7f8620a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f861c06cc20 con 0x7f861c0b01f0 2024-07-04T17:30:24.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.037+0000 7f8619e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f861c0b01f0 0x7f861c13d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:24.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.037+0000 7f8619e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f861c0b01f0 0x7f861c13d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39960/0 (socket says 172.21.15.42:39960) 2024-07-04T17:30:24.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.037+0000 7f8619e00640 1 -- 172.21.15.42:0/2663804267 learned_addr learned my addr 172.21.15.42:0/2663804267 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:24.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.037+0000 7f8619e00640 1 -- 172.21.15.42:0/2663804267 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f861c0ab2d0 con 0x7f861c0b01f0 2024-07-04T17:30:24.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.038+0000 7f8619e00640 1 --2- 172.21.15.42:0/2663804267 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f861c0b01f0 0x7f861c13d520 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f860c009850 tx=0x7f860c0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:24.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.038+0000 7f8612000640 1 -- 172.21.15.42:0/2663804267 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f860c03e030 con 0x7f861c0b01f0 2024-07-04T17:30:24.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.038+0000 7f8612000640 1 -- 172.21.15.42:0/2663804267 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f860c0035f0 con 0x7f861c0b01f0 2024-07-04T17:30:24.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.038+0000 7f8612000640 1 -- 172.21.15.42:0/2663804267 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f860c035040 con 0x7f861c0b01f0 2024-07-04T17:30:24.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.038+0000 7f8620a00640 1 -- 172.21.15.42:0/2663804267 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f861c0a95e0 con 0x7f861c0b01f0 2024-07-04T17:30:24.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.038+0000 7f8620a00640 1 -- 172.21.15.42:0/2663804267 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f861c1dcf80 con 0x7f861c0b01f0 2024-07-04T17:30:24.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.039+0000 7f8620a00640 1 -- 172.21.15.42:0/2663804267 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f861c06d2c0 con 0x7f861c0b01f0 2024-07-04T17:30:24.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.040+0000 7f8612000640 1 -- 172.21.15.42:0/2663804267 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 21) ==== 49668+0+0 (secure 0 0 0) 0x7f860c041030 con 0x7f861c0b01f0 2024-07-04T17:30:24.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.040+0000 7f8612000640 1 --2- 172.21.15.42:0/2663804267 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f85f803d580 0x7f85f803fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:24.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.040+0000 7f8612000640 1 -- 172.21.15.42:0/2663804267 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 4776+0+0 (secure 0 0 0) 0x7f860c03f070 con 0x7f861c0b01f0 2024-07-04T17:30:24.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.040+0000 7f8613e00640 1 -- 172.21.15.42:0/2663804267 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f85f803d580 msgr2=0x7f85f803fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/3045931584 2024-07-04T17:30:24.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.041+0000 7f8613e00640 1 --2- 172.21.15.42:0/2663804267 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f85f803d580 0x7f85f803fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:30:24.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.046+0000 7f8612000640 1 -- 172.21.15.42:0/2663804267 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f860c04bd20 con 0x7f861c0b01f0 2024-07-04T17:30:24.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.242+0000 7f8613e00640 1 -- 172.21.15.42:0/2663804267 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f85f803d580 msgr2=0x7f85f803fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/3045931584 2024-07-04T17:30:24.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.242+0000 7f8613e00640 1 --2- 172.21.15.42:0/2663804267 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f85f803d580 0x7f85f803fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:30:24.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.250+0000 7f8620a00640 1 -- 172.21.15.42:0/2663804267 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f861c0af6e0 con 0x7f861c0b01f0 2024-07-04T17:30:24.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.251+0000 7f8612000640 1 -- 172.21.15.42:0/2663804267 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+15134 (secure 0 0 0) 0x7f860c030c90 con 0x7f861c0b01f0 2024-07-04T17:30:24.252 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:30:24.252 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":56,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:30:09.700204+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:29:03.403851+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-07-04T17:29:04.303537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:29:22.942399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.0299999713897705,"score_stable":1.0299999713897705,"optimal_score":1,"raw_score_acting":1.0299999713897705,"raw_score_stable":1.0299999713897705,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:29:24.406438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/7072365":"2024-07-04T18:30:09.692358+0000","172.21.15.42:0/1810008304":"2024-07-05T17:29:38.572697+0000","172.21.15.42:0/1534342117":"2024-07-05T17:29:38.572697+0000","172.21.15.42:6825/2638271195":"2024-07-05T17:29:38.572697+0000","172.21.15.42:6824/2638271195":"2024-07-05T17:29:38.572697+0000","172.21.15.42:0/591942290":"2024-07-05T17:29:38.572697+0000","172.21.15.42:0/2926090870":"2024-07-05T17:29:38.572697+0000","172.21.15.42:0/136286797":"2024-07-05T17:29:38.572697+0000","172.21.15.42:0/2800592161":"2024-07-05T17:29:38.572697+0000","172.21.15.42:0/1891351459":"2024-07-05T17:29:38.572697+0000","172.21.15.42:0/3114375174":"2024-07-05T17:29:38.572697+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:30:24.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.254+0000 7f8620a00640 1 -- 172.21.15.42:0/2663804267 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f85f803d580 msgr2=0x7f85f803fa90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:30:24.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.254+0000 7f8620a00640 1 --2- 172.21.15.42:0/2663804267 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f85f803d580 0x7f85f803fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:24.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.254+0000 7f8620a00640 1 -- 172.21.15.42:0/2663804267 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f861c0b01f0 msgr2=0x7f861c13d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:24.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.254+0000 7f8620a00640 1 --2- 172.21.15.42:0/2663804267 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f861c0b01f0 0x7f861c13d520 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f860c009850 tx=0x7f860c0049d0 comp rx=0 tx=0).stop 2024-07-04T17:30:24.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.254+0000 7f8620a00640 1 -- 172.21.15.42:0/2663804267 shutdown_connections 2024-07-04T17:30:24.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.254+0000 7f8620a00640 1 --2- 172.21.15.42:0/2663804267 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f85f803d580 0x7f85f803fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:24.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.254+0000 7f8620a00640 1 --2- 172.21.15.42:0/2663804267 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f861c0b01f0 0x7f861c13d520 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:24.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.254+0000 7f8620a00640 1 -- 172.21.15.42:0/2663804267 >> 172.21.15.42:0/2663804267 conn(0x7f861c0bf700 msgr2=0x7f861c0be730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:24.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.255+0000 7f8620a00640 1 -- 172.21.15.42:0/2663804267 shutdown_connections 2024-07-04T17:30:24.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.255+0000 7f8620a00640 1 -- 172.21.15.42:0/2663804267 wait complete. 2024-07-04T17:30:24.280 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-04T17:30:24.280 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:30:24.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.552+0000 7f32e7e00640 1 -- 172.21.15.42:0/3259455550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32e806c020 msgr2=0x7f32e814a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:24.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.552+0000 7f32e7e00640 1 --2- 172.21.15.42:0/3259455550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32e806c020 0x7f32e814a9e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f32d00097b0 tx=0x7f32d002a4a0 comp rx=0 tx=0).stop 2024-07-04T17:30:24.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.552+0000 7f32e7e00640 1 -- 172.21.15.42:0/3259455550 shutdown_connections 2024-07-04T17:30:24.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.552+0000 7f32e7e00640 1 --2- 172.21.15.42:0/3259455550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32e806c020 0x7f32e814a9e0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:24.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.552+0000 7f32e7e00640 1 -- 172.21.15.42:0/3259455550 >> 172.21.15.42:0/3259455550 conn(0x7f32e80bebc0 msgr2=0x7f32e80befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:24.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.553+0000 7f32e7e00640 1 -- 172.21.15.42:0/3259455550 shutdown_connections 2024-07-04T17:30:24.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.553+0000 7f32e7e00640 1 -- 172.21.15.42:0/3259455550 wait complete. 2024-07-04T17:30:24.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.555+0000 7f32e7e00640 1 Processor -- start 2024-07-04T17:30:24.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.555+0000 7f32e7e00640 1 -- start start 2024-07-04T17:30:24.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.556+0000 7f32e7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32e806c020 0x7f32e8149590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:24.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.556+0000 7f32e7e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f32e814b500 con 0x7f32e806c020 2024-07-04T17:30:24.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.556+0000 7f32e6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32e806c020 0x7f32e8149590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:24.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.556+0000 7f32e6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32e806c020 0x7f32e8149590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39988/0 (socket says 172.21.15.42:39988) 2024-07-04T17:30:24.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.556+0000 7f32e6a00640 1 -- 172.21.15.42:0/3391144490 learned_addr learned my addr 172.21.15.42:0/3391144490 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:24.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.557+0000 7f32e6a00640 1 -- 172.21.15.42:0/3391144490 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e80ae290 con 0x7f32e806c020 2024-07-04T17:30:24.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.557+0000 7f32e6a00640 1 --2- 172.21.15.42:0/3391144490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32e806c020 0x7f32e8149590 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f32d8001eb0 tx=0x7f32d800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:24.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.557+0000 7f32dea00640 1 -- 172.21.15.42:0/3391144490 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32d80116c0 con 0x7f32e806c020 2024-07-04T17:30:24.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.557+0000 7f32dea00640 1 -- 172.21.15.42:0/3391144490 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32d8017070 con 0x7f32e806c020 2024-07-04T17:30:24.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.557+0000 7f32e7e00640 1 -- 172.21.15.42:0/3391144490 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32e81e7790 con 0x7f32e806c020 2024-07-04T17:30:24.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.557+0000 7f32dea00640 1 -- 172.21.15.42:0/3391144490 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32d8011ae0 con 0x7f32e806c020 2024-07-04T17:30:24.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.558+0000 7f32e7e00640 1 -- 172.21.15.42:0/3391144490 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32e80adda0 con 0x7f32e806c020 2024-07-04T17:30:24.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.559+0000 7f32dea00640 1 -- 172.21.15.42:0/3391144490 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 21) ==== 49668+0+0 (secure 0 0 0) 0x7f32d801c460 con 0x7f32e806c020 2024-07-04T17:30:24.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.559+0000 7f32e7e00640 1 -- 172.21.15.42:0/3391144490 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32e814bb40 con 0x7f32e806c020 2024-07-04T17:30:24.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.559+0000 7f32dea00640 1 --2- 172.21.15.42:0/3391144490 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f32c403d530 0x7f32c403fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:24.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.559+0000 7f32dea00640 1 -- 172.21.15.42:0/3391144490 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 4776+0+0 (secure 0 0 0) 0x7f32d8051c60 con 0x7f32e806c020 2024-07-04T17:30:24.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.559+0000 7f32dfe00640 1 -- 172.21.15.42:0/3391144490 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f32c403d530 msgr2=0x7f32c403fa40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/3045931584 2024-07-04T17:30:24.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.560+0000 7f32dfe00640 1 --2- 172.21.15.42:0/3391144490 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f32c403d530 0x7f32c403fa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:30:24.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.565+0000 7f32dea00640 1 -- 172.21.15.42:0/3391144490 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f32d800a140 con 0x7f32e806c020 2024-07-04T17:30:24.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.761+0000 7f32dfe00640 1 -- 172.21.15.42:0/3391144490 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f32c403d530 msgr2=0x7f32c403fa40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/3045931584 2024-07-04T17:30:24.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.761+0000 7f32dfe00640 1 --2- 172.21.15.42:0/3391144490 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f32c403d530 0x7f32c403fa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:30:24.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.835+0000 7f32e7e00640 1 -- 172.21.15.42:0/3391144490 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32e814a9c0 con 0x7f32e806c020 2024-07-04T17:30:24.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.836+0000 7f32dea00640 1 -- 172.21.15.42:0/3391144490 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+7364 (secure 0 0 0) 0x7f32d8011c80 con 0x7f32e806c020 2024-07-04T17:30:24.837 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:30:24.837 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":43,"btime":"2024-07-04T17:30:08:691538+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/343791977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":343791977},{"type":"v1","addr":"172.21.15.42:6827","nonce":343791977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":4722,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/3172446281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":3172446281},{"type":"v1","addr":"172.21.15.42:6829","nonce":3172446281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":4727,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/1866310217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1866310217},{"type":"v1","addr":"172.21.15.42:6831","nonce":1866310217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:29:05.313061+0000","modified":"2024-07-04T17:30:07.807707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4731},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4731":{"gid":4731,"name":"d","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6835/1327681666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1327681666},{"type":"v1","addr":"172.21.15.42:6835","nonce":1327681666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4731,"qdb_cluster":[4731]},"id":4},{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:29:25.424728+0000","modified":"2024-07-04T17:29:28.496007+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4728":{"gid":4728,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":9,"addr":"172.21.15.42:6833/1072723036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1072723036},{"type":"v1","addr":"172.21.15.42:6833","nonce":1072723036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4728,"qdb_cluster":[4728]},"id":5}]} 2024-07-04T17:30:24.837 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 43 2024-07-04T17:30:24.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.839+0000 7f32e7e00640 1 -- 172.21.15.42:0/3391144490 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f32c403d530 msgr2=0x7f32c403fa40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:30:24.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.839+0000 7f32e7e00640 1 --2- 172.21.15.42:0/3391144490 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f32c403d530 0x7f32c403fa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:24.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.839+0000 7f32e7e00640 1 -- 172.21.15.42:0/3391144490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32e806c020 msgr2=0x7f32e8149590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:24.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.839+0000 7f32e7e00640 1 --2- 172.21.15.42:0/3391144490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32e806c020 0x7f32e8149590 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f32d8001eb0 tx=0x7f32d800c810 comp rx=0 tx=0).stop 2024-07-04T17:30:24.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.839+0000 7f32e7e00640 1 -- 172.21.15.42:0/3391144490 shutdown_connections 2024-07-04T17:30:24.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.839+0000 7f32e7e00640 1 --2- 172.21.15.42:0/3391144490 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7f32c403d530 0x7f32c403fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:24.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.839+0000 7f32e7e00640 1 --2- 172.21.15.42:0/3391144490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32e806c020 0x7f32e8149590 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:24.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.839+0000 7f32e7e00640 1 -- 172.21.15.42:0/3391144490 >> 172.21.15.42:0/3391144490 conn(0x7f32e80bebc0 msgr2=0x7f32e80befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:24.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.839+0000 7f32e7e00640 1 -- 172.21.15.42:0/3391144490 shutdown_connections 2024-07-04T17:30:24.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:24.839+0000 7f32e7e00640 1 -- 172.21.15.42:0/3391144490 wait complete. 2024-07-04T17:30:24.863 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:30:25.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.136+0000 7fb2f8200640 1 -- 172.21.15.42:0/1892416591 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2f00b02b0 msgr2=0x7fb2f00b3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:25.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.136+0000 7fb2f8200640 1 --2- 172.21.15.42:0/1892416591 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2f00b02b0 0x7fb2f00b3650 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4005fd0 tx=0x7fb2e402a560 comp rx=0 tx=0).stop 2024-07-04T17:30:25.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.136+0000 7fb2f8200640 1 -- 172.21.15.42:0/1892416591 shutdown_connections 2024-07-04T17:30:25.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.136+0000 7fb2f8200640 1 --2- 172.21.15.42:0/1892416591 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2f00b02b0 0x7fb2f00b3650 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:25.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.136+0000 7fb2f8200640 1 -- 172.21.15.42:0/1892416591 >> 172.21.15.42:0/1892416591 conn(0x7fb2f00bf700 msgr2=0x7fb2f00af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:25.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.136+0000 7fb2f8200640 1 -- 172.21.15.42:0/1892416591 shutdown_connections 2024-07-04T17:30:25.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.136+0000 7fb2f8200640 1 -- 172.21.15.42:0/1892416591 wait complete. 2024-07-04T17:30:25.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.138+0000 7fb2f8200640 1 Processor -- start 2024-07-04T17:30:25.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.139+0000 7fb2f8200640 1 -- start start 2024-07-04T17:30:25.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.140+0000 7fb2f8200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2f00b02b0 0x7fb2f00b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:25.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.140+0000 7fb2f8200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f006c710 con 0x7fb2f00b02b0 2024-07-04T17:30:25.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.140+0000 7fb2f5800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2f00b02b0 0x7fb2f00b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:25.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.140+0000 7fb2f5800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2f00b02b0 0x7fb2f00b7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40008/0 (socket says 172.21.15.42:40008) 2024-07-04T17:30:25.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.140+0000 7fb2f5800640 1 -- 172.21.15.42:0/840325188 learned_addr learned my addr 172.21.15.42:0/840325188 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:25.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.141+0000 7fb2f5800640 1 -- 172.21.15.42:0/840325188 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f00b9420 con 0x7fb2f00b02b0 2024-07-04T17:30:25.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.141+0000 7fb2f5800640 1 --2- 172.21.15.42:0/840325188 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2f00b02b0 0x7fb2f00b7140 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009850 tx=0x7fb2e40049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:25.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.141+0000 7fb2ed600640 1 -- 172.21.15.42:0/840325188 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2e403e030 con 0x7fb2f00b02b0 2024-07-04T17:30:25.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.141+0000 7fb2ed600640 1 -- 172.21.15.42:0/840325188 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2e40035f0 con 0x7fb2f00b02b0 2024-07-04T17:30:25.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.141+0000 7fb2ed600640 1 -- 172.21.15.42:0/840325188 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2e4035040 con 0x7fb2f00b02b0 2024-07-04T17:30:25.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.141+0000 7fb2f8200640 1 -- 172.21.15.42:0/840325188 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2f01e4570 con 0x7fb2f00b02b0 2024-07-04T17:30:25.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.141+0000 7fb2f8200640 1 -- 172.21.15.42:0/840325188 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2f00ba900 con 0x7fb2f00b02b0 2024-07-04T17:30:25.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.143+0000 7fb2ed600640 1 -- 172.21.15.42:0/840325188 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 21) ==== 49668+0+0 (secure 0 0 0) 0x7fb2e4041030 con 0x7fb2f00b02b0 2024-07-04T17:30:25.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.143+0000 7fb2f8200640 1 -- 172.21.15.42:0/840325188 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2f006cd50 con 0x7fb2f00b02b0 2024-07-04T17:30:25.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.143+0000 7fb2ed600640 1 --2- 172.21.15.42:0/840325188 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7fb2d003d580 0x7fb2d003fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:25.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.143+0000 7fb2ed600640 1 -- 172.21.15.42:0/840325188 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 4776+0+0 (secure 0 0 0) 0x7fb2e403f070 con 0x7fb2f00b02b0 2024-07-04T17:30:25.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.143+0000 7fb2f4e00640 1 -- 172.21.15.42:0/840325188 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7fb2d003d580 msgr2=0x7fb2d003fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/3045931584 2024-07-04T17:30:25.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.143+0000 7fb2f4e00640 1 --2- 172.21.15.42:0/840325188 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7fb2d003d580 0x7fb2d003fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:30:25.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.148+0000 7fb2ed600640 1 -- 172.21.15.42:0/840325188 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fb2e404bd60 con 0x7fb2f00b02b0 2024-07-04T17:30:25.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.343+0000 7fb2f4e00640 1 -- 172.21.15.42:0/840325188 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7fb2d003d580 msgr2=0x7fb2d003fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/3045931584 2024-07-04T17:30:25.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.343+0000 7fb2f4e00640 1 --2- 172.21.15.42:0/840325188 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7fb2d003d580 0x7fb2d003fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:30:25.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.393+0000 7fb2f8200640 1 -- 172.21.15.42:0/840325188 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2f00be150 con 0x7fb2f00b02b0 2024-07-04T17:30:25.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.394+0000 7fb2ed600640 1 -- 172.21.15.42:0/840325188 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+7364 (secure 0 0 0) 0x7fb2e4072070 con 0x7fb2f00b02b0 2024-07-04T17:30:25.396 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:30:25.396 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":43,"btime":"2024-07-04T17:30:08:691538+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/343791977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":343791977},{"type":"v1","addr":"172.21.15.42:6827","nonce":343791977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":4722,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/3172446281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":3172446281},{"type":"v1","addr":"172.21.15.42:6829","nonce":3172446281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":4727,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/1866310217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1866310217},{"type":"v1","addr":"172.21.15.42:6831","nonce":1866310217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:29:05.313061+0000","modified":"2024-07-04T17:30:07.807707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4731},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4731":{"gid":4731,"name":"d","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6835/1327681666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1327681666},{"type":"v1","addr":"172.21.15.42:6835","nonce":1327681666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4731,"qdb_cluster":[4731]},"id":4},{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:29:25.424728+0000","modified":"2024-07-04T17:29:28.496007+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4728":{"gid":4728,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":9,"addr":"172.21.15.42:6833/1072723036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1072723036},{"type":"v1","addr":"172.21.15.42:6833","nonce":1072723036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4728,"qdb_cluster":[4728]},"id":5}]} 2024-07-04T17:30:25.396 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 43 2024-07-04T17:30:25.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.397+0000 7fb2f8200640 1 -- 172.21.15.42:0/840325188 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7fb2d003d580 msgr2=0x7fb2d003fa90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:30:25.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.397+0000 7fb2f8200640 1 --2- 172.21.15.42:0/840325188 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7fb2d003d580 0x7fb2d003fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:25.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.397+0000 7fb2f8200640 1 -- 172.21.15.42:0/840325188 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2f00b02b0 msgr2=0x7fb2f00b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:25.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.397+0000 7fb2f8200640 1 --2- 172.21.15.42:0/840325188 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2f00b02b0 0x7fb2f00b7140 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009850 tx=0x7fb2e40049d0 comp rx=0 tx=0).stop 2024-07-04T17:30:25.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.397+0000 7fb2f8200640 1 -- 172.21.15.42:0/840325188 shutdown_connections 2024-07-04T17:30:25.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.397+0000 7fb2f8200640 1 --2- 172.21.15.42:0/840325188 >> [v2:172.21.15.42:6824/3045931584,v1:172.21.15.42:6825/3045931584] conn(0x7fb2d003d580 0x7fb2d003fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:25.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.397+0000 7fb2f8200640 1 --2- 172.21.15.42:0/840325188 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2f00b02b0 0x7fb2f00b7140 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:25.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.397+0000 7fb2f8200640 1 -- 172.21.15.42:0/840325188 >> 172.21.15.42:0/840325188 conn(0x7fb2f00bf700 msgr2=0x7fb2f0145320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:25.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.397+0000 7fb2f8200640 1 -- 172.21.15.42:0/840325188 shutdown_connections 2024-07-04T17:30:25.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.397+0000 7fb2f8200640 1 -- 172.21.15.42:0/840325188 wait complete. 2024-07-04T17:30:25.424 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:30:25.680 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:30:25.679+0000 7fe703000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:30:25.680 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:30:25.679+0000 7fe703000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:30:25.681 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:30:25.679+0000 7fe703000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:30:25.681 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:30:25.679+0000 7fe703000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:30:25.681 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:30:25.679+0000 7fe703000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:30:25.682 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:30:25.681+0000 7fe6ffe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:30:25.682 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:30:25.681+0000 7fe6ffe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:30:25.682 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:30:25.681+0000 7fe6ffe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:30:25.682 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:30:25.681+0000 7fe6ffe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:30:25.682 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:30:25.681+0000 7fe6ffe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:30:25.696 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:30:25.695+0000 7fe703000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:30:25.696 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:30:25.695+0000 7fe703000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:30:25.696 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:30:25.695+0000 7fe703000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:30:25.696 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:30:25.695+0000 7fe703000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:30:25.696 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:30:25.695+0000 7fe703000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:30:25.697 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:30:25.695+0000 7fe6ffe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:30:25.697 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:30:25.695+0000 7fe6ffe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:30:25.697 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:30:25.695+0000 7fe6ffe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:30:25.697 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:30:25.695+0000 7fe6ffe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:30:25.697 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:30:25.695+0000 7fe6ffe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:30:25.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.718+0000 7fe486800640 1 -- 172.21.15.42:0/3598929268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe48006edd0 msgr2=0x7fe4800b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:25.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.718+0000 7fe486800640 1 --2- 172.21.15.42:0/3598929268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe48006edd0 0x7fe4800b3030 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe470005fd0 tx=0x7fe47002b930 comp rx=0 tx=0).stop 2024-07-04T17:30:25.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.718+0000 7fe486800640 1 -- 172.21.15.42:0/3598929268 shutdown_connections 2024-07-04T17:30:25.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.718+0000 7fe486800640 1 --2- 172.21.15.42:0/3598929268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe48006edd0 0x7fe4800b3030 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:25.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.718+0000 7fe486800640 1 -- 172.21.15.42:0/3598929268 >> 172.21.15.42:0/3598929268 conn(0x7fe4800bf700 msgr2=0x7fe4800af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:25.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.718+0000 7fe486800640 1 -- 172.21.15.42:0/3598929268 shutdown_connections 2024-07-04T17:30:25.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.718+0000 7fe486800640 1 -- 172.21.15.42:0/3598929268 wait complete. 2024-07-04T17:30:25.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.720+0000 7fe486800640 1 Processor -- start 2024-07-04T17:30:25.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.720+0000 7fe486800640 1 -- start start 2024-07-04T17:30:25.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.722+0000 7fe486800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe48006cea0 0x7fe4800b6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:25.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.722+0000 7fe486800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe4800b0370 con 0x7fe48006cea0 2024-07-04T17:30:25.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.722+0000 7fe47f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe48006cea0 0x7fe4800b6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:25.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.722+0000 7fe47f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe48006cea0 0x7fe4800b6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40148/0 (socket says 172.21.15.42:40148) 2024-07-04T17:30:25.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.722+0000 7fe47f400640 1 -- 172.21.15.42:0/915876048 learned_addr learned my addr 172.21.15.42:0/915876048 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:25.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.722+0000 7fe47f400640 1 -- 172.21.15.42:0/915876048 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4800b7260 con 0x7fe48006cea0 2024-07-04T17:30:25.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.723+0000 7fe47f400640 1 --2- 172.21.15.42:0/915876048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe48006cea0 0x7fe4800b6c30 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe470006010 tx=0x7fe4700045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:25.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.723+0000 7fe477400640 1 -- 172.21.15.42:0/915876048 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe470004c20 con 0x7fe48006cea0 2024-07-04T17:30:25.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.723+0000 7fe486800640 1 -- 172.21.15.42:0/915876048 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4800737a0 con 0x7fe48006cea0 2024-07-04T17:30:25.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.723+0000 7fe477400640 1 -- 172.21.15.42:0/915876048 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe470004eb0 con 0x7fe48006cea0 2024-07-04T17:30:25.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.723+0000 7fe477400640 1 -- 172.21.15.42:0/915876048 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4700317c0 con 0x7fe48006cea0 2024-07-04T17:30:25.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.723+0000 7fe486800640 1 -- 172.21.15.42:0/915876048 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe480149770 con 0x7fe48006cea0 2024-07-04T17:30:25.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.724+0000 7fe486800640 1 -- 172.21.15.42:0/915876048 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4800b3030 con 0x7fe48006cea0 2024-07-04T17:30:25.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.725+0000 7fe477400640 1 -- 172.21.15.42:0/915876048 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 22) ==== 49122+0+0 (secure 0 0 0) 0x7fe470042020 con 0x7fe48006cea0 2024-07-04T17:30:25.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.725+0000 7fe477400640 1 -- 172.21.15.42:0/915876048 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5249+0+0 (secure 0 0 0) 0x7fe470036070 con 0x7fe48006cea0 2024-07-04T17:30:25.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.730+0000 7fe477400640 1 -- 172.21.15.42:0/915876048 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fe470031960 con 0x7fe48006cea0 2024-07-04T17:30:25.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.946+0000 7fe486800640 1 -- 172.21.15.42:0/915876048 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe480069b70 con 0x7fe48006cea0 2024-07-04T17:30:25.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.948+0000 7fe477400640 1 -- 172.21.15.42:0/915876048 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+15819 (secure 0 0 0) 0x7fe47004dcf0 con 0x7fe48006cea0 2024-07-04T17:30:25.949 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:30:25.950 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":57,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:30:25.586890+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:29:03.403851+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-07-04T17:29:04.303537+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:29:22.942399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.0299999713897705,"score_stable":1.0299999713897705,"optimal_score":1,"raw_score_acting":1.0299999713897705,"raw_score_stable":1.0299999713897705,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:29:24.406438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/934943970":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/4215342546":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/3491363662":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/3597724339":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/1322461005":"2024-07-05T17:30:25.586824+0000","172.21.15.42:6825/3045931584":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/461490182":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/7072365":"2024-07-04T18:30:09.692358+0000","172.21.15.42:0/176559669":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/1810008304":"2024-07-05T17:29:38.572697+0000","172.21.15.42:0/1534342117":"2024-07-05T17:29:38.572697+0000","172.21.15.42:6825/2638271195":"2024-07-05T17:29:38.572697+0000","172.21.15.42:0/2056846029":"2024-07-05T17:30:25.586824+0000","172.21.15.42:6824/2638271195":"2024-07-05T17:29:38.572697+0000","172.21.15.42:0/591942290":"2024-07-05T17:29:38.572697+0000","172.21.15.42:6824/3045931584":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/2926090870":"2024-07-05T17:29:38.572697+0000","172.21.15.42:0/136286797":"2024-07-05T17:29:38.572697+0000","172.21.15.42:0/1793407260":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/2800592161":"2024-07-05T17:29:38.572697+0000","172.21.15.42:0/1891351459":"2024-07-05T17:29:38.572697+0000","172.21.15.42:0/3114375174":"2024-07-05T17:29:38.572697+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:30:25.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.950+0000 7fe486800640 1 -- 172.21.15.42:0/915876048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe48006cea0 msgr2=0x7fe4800b6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:25.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.950+0000 7fe486800640 1 --2- 172.21.15.42:0/915876048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe48006cea0 0x7fe4800b6c30 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe470006010 tx=0x7fe4700045e0 comp rx=0 tx=0).stop 2024-07-04T17:30:25.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.951+0000 7fe486800640 1 -- 172.21.15.42:0/915876048 shutdown_connections 2024-07-04T17:30:25.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.951+0000 7fe486800640 1 --2- 172.21.15.42:0/915876048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe48006cea0 0x7fe4800b6c30 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:25.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.951+0000 7fe486800640 1 -- 172.21.15.42:0/915876048 >> 172.21.15.42:0/915876048 conn(0x7fe4800bf700 msgr2=0x7fe48014ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:25.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.951+0000 7fe486800640 1 -- 172.21.15.42:0/915876048 shutdown_connections 2024-07-04T17:30:25.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:25.951+0000 7fe486800640 1 -- 172.21.15.42:0/915876048 wait complete. 2024-07-04T17:30:25.978 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-07-04T17:30:26.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.256+0000 7fbfb4a00640 1 -- 172.21.15.42:0/2560469232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbfb00b2d20 msgr2=0x7fbfb00bb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:26.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.256+0000 7fbfb4a00640 1 --2- 172.21.15.42:0/2560469232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbfb00b2d20 0x7fbfb00bb1f0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c005fd0 tx=0x7fbf9c02a560 comp rx=0 tx=0).stop 2024-07-04T17:30:26.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.256+0000 7fbfb4a00640 1 -- 172.21.15.42:0/2560469232 shutdown_connections 2024-07-04T17:30:26.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.256+0000 7fbfb4a00640 1 --2- 172.21.15.42:0/2560469232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbfb00b2d20 0x7fbfb00bb1f0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:26.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.256+0000 7fbfb4a00640 1 -- 172.21.15.42:0/2560469232 >> 172.21.15.42:0/2560469232 conn(0x7fbfb00bf700 msgr2=0x7fbfb00af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:26.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.256+0000 7fbfb4a00640 1 -- 172.21.15.42:0/2560469232 shutdown_connections 2024-07-04T17:30:26.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.256+0000 7fbfb4a00640 1 -- 172.21.15.42:0/2560469232 wait complete. 2024-07-04T17:30:26.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.258+0000 7fbfb4a00640 1 Processor -- start 2024-07-04T17:30:26.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.258+0000 7fbfb4a00640 1 -- start start 2024-07-04T17:30:26.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.260+0000 7fbfb4a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbfb00b2d20 0x7fbfb0141a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:26.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.260+0000 7fbfb4a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb006c020 con 0x7fbfb00b2d20 2024-07-04T17:30:26.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.260+0000 7fbfade00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbfb00b2d20 0x7fbfb0141a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:26.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.260+0000 7fbfade00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbfb00b2d20 0x7fbfb0141a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40164/0 (socket says 172.21.15.42:40164) 2024-07-04T17:30:26.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.260+0000 7fbfade00640 1 -- 172.21.15.42:0/2133327430 learned_addr learned my addr 172.21.15.42:0/2133327430 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:26.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.260+0000 7fbfade00640 1 -- 172.21.15.42:0/2133327430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb00acff0 con 0x7fbfb00b2d20 2024-07-04T17:30:26.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.261+0000 7fbfade00640 1 --2- 172.21.15.42:0/2133327430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbfb00b2d20 0x7fbfb0141a30 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fbfa000c380 tx=0x7fbfa000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:26.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.261+0000 7fbfa6000640 1 -- 172.21.15.42:0/2133327430 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfa0017070 con 0x7fbfb00b2d20 2024-07-04T17:30:26.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.261+0000 7fbfa6000640 1 -- 172.21.15.42:0/2133327430 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbfa00052f0 con 0x7fbfb00b2d20 2024-07-04T17:30:26.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.261+0000 7fbfb4a00640 1 -- 172.21.15.42:0/2133327430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfb01e6ce0 con 0x7fbfb00b2d20 2024-07-04T17:30:26.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.261+0000 7fbfa6000640 1 -- 172.21.15.42:0/2133327430 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbfa00035f0 con 0x7fbfb00b2d20 2024-07-04T17:30:26.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.261+0000 7fbfb4a00640 1 -- 172.21.15.42:0/2133327430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfb01e6a80 con 0x7fbfb00b2d20 2024-07-04T17:30:26.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.262+0000 7fbfb4a00640 1 -- 172.21.15.42:0/2133327430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfb006c710 con 0x7fbfb00b2d20 2024-07-04T17:30:26.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.262+0000 7fbfa6000640 1 -- 172.21.15.42:0/2133327430 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 22) ==== 49122+0+0 (secure 0 0 0) 0x7fbfa001a070 con 0x7fbfb00b2d20 2024-07-04T17:30:26.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.263+0000 7fbfa6000640 1 -- 172.21.15.42:0/2133327430 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5249+0+0 (secure 0 0 0) 0x7fbfa0050ae0 con 0x7fbfb00b2d20 2024-07-04T17:30:26.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.268+0000 7fbfa6000640 1 -- 172.21.15.42:0/2133327430 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fbfa0003a20 con 0x7fbfb00b2d20 2024-07-04T17:30:26.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.505+0000 7fbfb4a00640 1 -- 172.21.15.42:0/2133327430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fbfb00af4a0 con 0x7fbfb00b2d20 2024-07-04T17:30:26.508 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-07-04T17:30:26.507+0000 7f8616e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-04T17:30:26.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.509+0000 7fbfa6000640 1 -- 172.21.15.42:0/2133327430 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v44) ==== 161+0+0 (secure 0 0 0) 0x7fbfa0003ca0 con 0x7fbfb00b2d20 2024-07-04T17:30:26.510 INFO:teuthology.orchestra.run.smithi042.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-04T17:30:26.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.513+0000 7fbfb4a00640 1 -- 172.21.15.42:0/2133327430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbfb00b2d20 msgr2=0x7fbfb0141a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:26.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.513+0000 7fbfb4a00640 1 --2- 172.21.15.42:0/2133327430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbfb00b2d20 0x7fbfb0141a30 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fbfa000c380 tx=0x7fbfa000c840 comp rx=0 tx=0).stop 2024-07-04T17:30:26.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.513+0000 7fbfb4a00640 1 -- 172.21.15.42:0/2133327430 shutdown_connections 2024-07-04T17:30:26.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.513+0000 7fbfb4a00640 1 --2- 172.21.15.42:0/2133327430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbfb00b2d20 0x7fbfb0141a30 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:26.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.513+0000 7fbfb4a00640 1 -- 172.21.15.42:0/2133327430 >> 172.21.15.42:0/2133327430 conn(0x7fbfb00bf700 msgr2=0x7fbfb00aada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:26.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.513+0000 7fbfb4a00640 1 -- 172.21.15.42:0/2133327430 shutdown_connections 2024-07-04T17:30:26.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.513+0000 7fbfb4a00640 1 -- 172.21.15.42:0/2133327430 wait complete. 2024-07-04T17:30:26.551 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-07-04T17:30:26.678 INFO:tasks.ceph.mds.d.smithi042.stdout:starting mds.d at 2024-07-04T17:30:26.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.814+0000 7f6c60a00640 1 -- 172.21.15.42:0/3044009050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c40003a50 msgr2=0x7f6c40003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:26.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.814+0000 7f6c60a00640 1 --2- 172.21.15.42:0/3044009050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c40003a50 0x7f6c40003ea0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00c4b0 tx=0x7f6c40005830 comp rx=0 tx=0).stop 2024-07-04T17:30:26.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.814+0000 7f6c60a00640 1 -- 172.21.15.42:0/3044009050 shutdown_connections 2024-07-04T17:30:26.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.814+0000 7f6c60a00640 1 --2- 172.21.15.42:0/3044009050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c40003a50 0x7f6c40003ea0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:26.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.814+0000 7f6c60a00640 1 --2- 172.21.15.42:0/3044009050 >> v2:172.21.15.42:3300/0 conn(0x7f6c5c1414c0 0x7f6c5c1418d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:26.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.814+0000 7f6c60a00640 1 -- 172.21.15.42:0/3044009050 >> 172.21.15.42:0/3044009050 conn(0x7f6c5c0afaa0 msgr2=0x7f6c5c0afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:26.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.814+0000 7f6c60a00640 1 -- 172.21.15.42:0/3044009050 shutdown_connections 2024-07-04T17:30:26.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.814+0000 7f6c60a00640 1 -- 172.21.15.42:0/3044009050 wait complete. 2024-07-04T17:30:26.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.816+0000 7f6c60a00640 1 Processor -- start 2024-07-04T17:30:26.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.817+0000 7f6c60a00640 1 -- start start 2024-07-04T17:30:26.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.818+0000 7f6c60a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c40003a50 0x7f6c5c147840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:26.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.818+0000 7f6c60a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5c0aee20 con 0x7f6c40003a50 2024-07-04T17:30:26.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.818+0000 7f6c5a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c40003a50 0x7f6c5c147840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:26.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.818+0000 7f6c5a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c40003a50 0x7f6c5c147840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40224/0 (socket says 172.21.15.42:40224) 2024-07-04T17:30:26.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.818+0000 7f6c5a800640 1 -- 172.21.15.42:0/1552697341 learned_addr learned my addr 172.21.15.42:0/1552697341 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:26.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.819+0000 7f6c5a800640 1 -- 172.21.15.42:0/1552697341 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c5c1e3cf0 con 0x7f6c40003a50 2024-07-04T17:30:26.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.819+0000 7f6c5a800640 1 --2- 172.21.15.42:0/1552697341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c40003a50 0x7f6c5c147840 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00c5d0 tx=0x7f6c4c001a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:26.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.819+0000 7f6c52a00640 1 -- 172.21.15.42:0/1552697341 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c4c039070 con 0x7f6c40003a50 2024-07-04T17:30:26.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.819+0000 7f6c52a00640 1 -- 172.21.15.42:0/1552697341 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c4c0013e0 con 0x7f6c40003a50 2024-07-04T17:30:26.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.819+0000 7f6c60a00640 1 -- 172.21.15.42:0/1552697341 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c5c142b80 con 0x7f6c40003a50 2024-07-04T17:30:26.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.819+0000 7f6c52a00640 1 -- 172.21.15.42:0/1552697341 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c4c02bc20 con 0x7f6c40003a50 2024-07-04T17:30:26.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.820+0000 7f6c60a00640 1 -- 172.21.15.42:0/1552697341 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c5c1e43d0 con 0x7f6c40003a50 2024-07-04T17:30:26.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.821+0000 7f6c60a00640 1 -- 172.21.15.42:0/1552697341 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c5c0af4d0 con 0x7f6c40003a50 2024-07-04T17:30:26.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.821+0000 7f6c52a00640 1 -- 172.21.15.42:0/1552697341 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) ==== 49249+0+0 (secure 0 0 0) 0x7f6c4c006070 con 0x7f6c40003a50 2024-07-04T17:30:26.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.821+0000 7f6c52a00640 1 --2- 172.21.15.42:0/1552697341 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f6c3c03d0b0 0x7f6c3c03f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:26.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.821+0000 7f6c52a00640 1 -- 172.21.15.42:0/1552697341 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5335+0+0 (secure 0 0 0) 0x7f6c4c071cc0 con 0x7f6c40003a50 2024-07-04T17:30:26.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.821+0000 7f6c59e00640 1 --2- 172.21.15.42:0/1552697341 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f6c3c03d0b0 0x7f6c3c03f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:26.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.822+0000 7f6c59e00640 1 --2- 172.21.15.42:0/1552697341 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f6c3c03d0b0 0x7f6c3c03f5c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6c48005eb0 tx=0x7f6c48005f90 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:26.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:26.827+0000 7f6c52a00640 1 -- 172.21.15.42:0/1552697341 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f6c4c077020 con 0x7f6c40003a50 2024-07-04T17:30:27.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.074+0000 7f6c60a00640 1 -- 172.21.15.42:0/1552697341 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f6c5c140ee0 con 0x7f6c40003a50 2024-07-04T17:30:27.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.599+0000 7f6c52a00640 1 -- 172.21.15.42:0/1552697341 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v45) ==== 104+0+0 (secure 0 0 0) 0x7f6c4c039e40 con 0x7f6c40003a50 2024-07-04T17:30:27.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.602+0000 7f6c60a00640 1 -- 172.21.15.42:0/1552697341 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f6c3c03d0b0 msgr2=0x7f6c3c03f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:27.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.602+0000 7f6c60a00640 1 --2- 172.21.15.42:0/1552697341 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f6c3c03d0b0 0x7f6c3c03f5c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6c48005eb0 tx=0x7f6c48005f90 comp rx=0 tx=0).stop 2024-07-04T17:30:27.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.602+0000 7f6c60a00640 1 -- 172.21.15.42:0/1552697341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c40003a50 msgr2=0x7f6c5c147840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:27.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.603+0000 7f6c60a00640 1 --2- 172.21.15.42:0/1552697341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c40003a50 0x7f6c5c147840 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00c5d0 tx=0x7f6c4c001a60 comp rx=0 tx=0).stop 2024-07-04T17:30:27.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.603+0000 7f6c60a00640 1 -- 172.21.15.42:0/1552697341 shutdown_connections 2024-07-04T17:30:27.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.603+0000 7f6c60a00640 1 --2- 172.21.15.42:0/1552697341 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f6c3c03d0b0 0x7f6c3c03f5c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:27.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.603+0000 7f6c60a00640 1 --2- 172.21.15.42:0/1552697341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c40003a50 0x7f6c5c147840 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:27.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.603+0000 7f6c60a00640 1 -- 172.21.15.42:0/1552697341 >> 172.21.15.42:0/1552697341 conn(0x7f6c5c0afaa0 msgr2=0x7f6c5c0a94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:27.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.603+0000 7f6c60a00640 1 -- 172.21.15.42:0/1552697341 shutdown_connections 2024-07-04T17:30:27.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.603+0000 7f6c60a00640 1 -- 172.21.15.42:0/1552697341 wait complete. 2024-07-04T17:30:27.640 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-04T17:30:27.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.919+0000 7fc5d2800640 1 -- 172.21.15.42:0/2592955749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5cc073470 msgr2=0x7fc5cc0a9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:27.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.919+0000 7fc5d2800640 1 --2- 172.21.15.42:0/2592955749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5cc073470 0x7fc5cc0a9990 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80097b0 tx=0x7fc5b802a540 comp rx=0 tx=0).stop 2024-07-04T17:30:27.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.920+0000 7fc5d2800640 1 -- 172.21.15.42:0/2592955749 shutdown_connections 2024-07-04T17:30:27.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.920+0000 7fc5d2800640 1 --2- 172.21.15.42:0/2592955749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5cc073470 0x7fc5cc0a9990 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:27.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.920+0000 7fc5d2800640 1 -- 172.21.15.42:0/2592955749 >> 172.21.15.42:0/2592955749 conn(0x7fc5cc06c020 msgr2=0x7fc5cc06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:27.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.920+0000 7fc5d2800640 1 -- 172.21.15.42:0/2592955749 shutdown_connections 2024-07-04T17:30:27.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.920+0000 7fc5d2800640 1 -- 172.21.15.42:0/2592955749 wait complete. 2024-07-04T17:30:27.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.922+0000 7fc5d2800640 1 Processor -- start 2024-07-04T17:30:27.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.922+0000 7fc5d2800640 1 -- start start 2024-07-04T17:30:27.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.923+0000 7fc5d2800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5cc073470 0x7fc5cc1d67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:27.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.923+0000 7fc5d2800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc13f430 con 0x7fc5cc073470 2024-07-04T17:30:27.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.924+0000 7fc5cb400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5cc073470 0x7fc5cc1d67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:27.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.924+0000 7fc5cb400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5cc073470 0x7fc5cc1d67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40258/0 (socket says 172.21.15.42:40258) 2024-07-04T17:30:27.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.924+0000 7fc5cb400640 1 -- 172.21.15.42:0/3445756664 learned_addr learned my addr 172.21.15.42:0/3445756664 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:27.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.924+0000 7fc5cb400640 1 -- 172.21.15.42:0/3445756664 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5cc0ac4b0 con 0x7fc5cc073470 2024-07-04T17:30:27.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.925+0000 7fc5cb400640 1 --2- 172.21.15.42:0/3445756664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5cc073470 0x7fc5cc1d67c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc00c380 tx=0x7fc5bc00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:27.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.925+0000 7fc5c3400640 1 -- 172.21.15.42:0/3445756664 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5bc0116c0 con 0x7fc5cc073470 2024-07-04T17:30:27.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.925+0000 7fc5c3400640 1 -- 172.21.15.42:0/3445756664 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5bc017070 con 0x7fc5cc073470 2024-07-04T17:30:27.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.925+0000 7fc5c3400640 1 -- 172.21.15.42:0/3445756664 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5bc011b20 con 0x7fc5cc073470 2024-07-04T17:30:27.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.925+0000 7fc5d2800640 1 -- 172.21.15.42:0/3445756664 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5cc1e43b0 con 0x7fc5cc073470 2024-07-04T17:30:27.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.925+0000 7fc5d2800640 1 -- 172.21.15.42:0/3445756664 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5cc0ab9c0 con 0x7fc5cc073470 2024-07-04T17:30:27.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.926+0000 7fc5d2800640 1 -- 172.21.15.42:0/3445756664 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5cc1e3310 con 0x7fc5cc073470 2024-07-04T17:30:27.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.926+0000 7fc5c3400640 1 -- 172.21.15.42:0/3445756664 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) ==== 49559+0+0 (secure 0 0 0) 0x7fc5bc011cc0 con 0x7fc5cc073470 2024-07-04T17:30:27.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.927+0000 7fc5c3400640 1 --2- 172.21.15.42:0/3445756664 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fc5ac03d460 0x7fc5ac03f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:27.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.927+0000 7fc5c3400640 1 -- 172.21.15.42:0/3445756664 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5335+0+0 (secure 0 0 0) 0x7fc5bc051db0 con 0x7fc5cc073470 2024-07-04T17:30:27.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.927+0000 7fc5caa00640 1 --2- 172.21.15.42:0/3445756664 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fc5ac03d460 0x7fc5ac03f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:27.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.928+0000 7fc5caa00640 1 --2- 172.21.15.42:0/3445756664 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fc5ac03d460 0x7fc5ac03f970 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8034000 tx=0x7fc5b8004730 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:27.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:27.932+0000 7fc5c3400640 1 -- 172.21.15.42:0/3445756664 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fc5bc010cf0 con 0x7fc5cc073470 2024-07-04T17:30:28.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:28.138+0000 7fc5d2800640 1 -- 172.21.15.42:0/3445756664 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc5cc13fb00 con 0x7fc5cc073470 2024-07-04T17:30:28.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:28.762+0000 7fc5c3400640 1 -- 172.21.15.42:0/3445756664 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v59) ==== 181+0+0 (secure 0 0 0) 0x7fc5bc012420 con 0x7fc5cc073470 2024-07-04T17:30:28.764 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' removed 2024-07-04T17:30:28.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:28.768+0000 7fc5d2800640 1 -- 172.21.15.42:0/3445756664 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fc5ac03d460 msgr2=0x7fc5ac03f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:28.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:28.768+0000 7fc5d2800640 1 --2- 172.21.15.42:0/3445756664 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fc5ac03d460 0x7fc5ac03f970 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8034000 tx=0x7fc5b8004730 comp rx=0 tx=0).stop 2024-07-04T17:30:28.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:28.768+0000 7fc5d2800640 1 -- 172.21.15.42:0/3445756664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5cc073470 msgr2=0x7fc5cc1d67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:28.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:28.768+0000 7fc5d2800640 1 --2- 172.21.15.42:0/3445756664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5cc073470 0x7fc5cc1d67c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc00c380 tx=0x7fc5bc00c840 comp rx=0 tx=0).stop 2024-07-04T17:30:28.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:28.769+0000 7fc5d2800640 1 -- 172.21.15.42:0/3445756664 shutdown_connections 2024-07-04T17:30:28.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:28.769+0000 7fc5d2800640 1 --2- 172.21.15.42:0/3445756664 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fc5ac03d460 0x7fc5ac03f970 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:28.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:28.769+0000 7fc5d2800640 1 --2- 172.21.15.42:0/3445756664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc5cc073470 0x7fc5cc1d67c0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:28.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:28.769+0000 7fc5d2800640 1 -- 172.21.15.42:0/3445756664 >> 172.21.15.42:0/3445756664 conn(0x7fc5cc06c020 msgr2=0x7fc5cc06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:28.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:28.769+0000 7fc5d2800640 1 -- 172.21.15.42:0/3445756664 shutdown_connections 2024-07-04T17:30:28.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:28.769+0000 7fc5d2800640 1 -- 172.21.15.42:0/3445756664 wait complete. 2024-07-04T17:30:28.808 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-04T17:30:29.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.086+0000 7ff16b000640 1 -- 172.21.15.42:0/1651197260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1640b0db0 msgr2=0x7ff1640abc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:29.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.086+0000 7ff16b000640 1 --2- 172.21.15.42:0/1651197260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1640b0db0 0x7ff1640abc80 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff150005fd0 tx=0x7ff15002ae00 comp rx=0 tx=0).stop 2024-07-04T17:30:29.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.086+0000 7ff16b000640 1 -- 172.21.15.42:0/1651197260 shutdown_connections 2024-07-04T17:30:29.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.086+0000 7ff16b000640 1 --2- 172.21.15.42:0/1651197260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1640b0db0 0x7ff1640abc80 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:29.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.086+0000 7ff16b000640 1 -- 172.21.15.42:0/1651197260 >> 172.21.15.42:0/1651197260 conn(0x7ff1640afaa0 msgr2=0x7ff1640afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:29.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.086+0000 7ff16b000640 1 -- 172.21.15.42:0/1651197260 shutdown_connections 2024-07-04T17:30:29.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.086+0000 7ff16b000640 1 -- 172.21.15.42:0/1651197260 wait complete. 2024-07-04T17:30:29.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.088+0000 7ff16b000640 1 Processor -- start 2024-07-04T17:30:29.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.088+0000 7ff16b000640 1 -- start start 2024-07-04T17:30:29.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.089+0000 7ff16b000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1640ac5b0 0x7ff164073340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:29.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.089+0000 7ff16b000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff1640ae590 con 0x7ff1640ac5b0 2024-07-04T17:30:29.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.090+0000 7ff169c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1640ac5b0 0x7ff164073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:29.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.090+0000 7ff169c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1640ac5b0 0x7ff164073340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40274/0 (socket says 172.21.15.42:40274) 2024-07-04T17:30:29.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.090+0000 7ff169c00640 1 -- 172.21.15.42:0/1876085282 learned_addr learned my addr 172.21.15.42:0/1876085282 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:29.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.090+0000 7ff169c00640 1 -- 172.21.15.42:0/1876085282 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff164072e20 con 0x7ff1640ac5b0 2024-07-04T17:30:29.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.090+0000 7ff169c00640 1 --2- 172.21.15.42:0/1876085282 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1640ac5b0 0x7ff164073340 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff15800c380 tx=0x7ff15800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:29.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.090+0000 7ff161600640 1 -- 172.21.15.42:0/1876085282 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff158017070 con 0x7ff1640ac5b0 2024-07-04T17:30:29.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.091+0000 7ff161600640 1 -- 172.21.15.42:0/1876085282 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1580052f0 con 0x7ff1640ac5b0 2024-07-04T17:30:29.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.091+0000 7ff161600640 1 -- 172.21.15.42:0/1876085282 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1580035f0 con 0x7ff1640ac5b0 2024-07-04T17:30:29.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.091+0000 7ff16b000640 1 -- 172.21.15.42:0/1876085282 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1640708e0 con 0x7ff1640ac5b0 2024-07-04T17:30:29.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.091+0000 7ff16b000640 1 -- 172.21.15.42:0/1876085282 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff16413d590 con 0x7ff1640ac5b0 2024-07-04T17:30:29.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.092+0000 7ff16b000640 1 -- 172.21.15.42:0/1876085282 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1640743e0 con 0x7ff1640ac5b0 2024-07-04T17:30:29.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.092+0000 7ff161600640 1 -- 172.21.15.42:0/1876085282 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) ==== 49559+0+0 (secure 0 0 0) 0x7ff15801a070 con 0x7ff1640ac5b0 2024-07-04T17:30:29.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.092+0000 7ff161600640 1 --2- 172.21.15.42:0/1876085282 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ff14403d460 0x7ff14403f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:29.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.092+0000 7ff161600640 1 -- 172.21.15.42:0/1876085282 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4897+0+0 (secure 0 0 0) 0x7ff1580511a0 con 0x7ff1640ac5b0 2024-07-04T17:30:29.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.093+0000 7ff169200640 1 --2- 172.21.15.42:0/1876085282 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ff14403d460 0x7ff14403f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:29.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.094+0000 7ff169200640 1 --2- 172.21.15.42:0/1876085282 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ff14403d460 0x7ff14403f970 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff150034000 tx=0x7ff150004730 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:29.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.098+0000 7ff161600640 1 -- 172.21.15.42:0/1876085282 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7ff15801e0d0 con 0x7ff1640ac5b0 2024-07-04T17:30:29.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.308+0000 7ff16b000640 1 -- 172.21.15.42:0/1876085282 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff1640abc80 con 0x7ff1640ac5b0 2024-07-04T17:30:29.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.783+0000 7ff161600640 1 -- 172.21.15.42:0/1876085282 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v60) ==== 169+0+0 (secure 0 0 0) 0x7ff1580192c0 con 0x7ff1640ac5b0 2024-07-04T17:30:29.784 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' removed 2024-07-04T17:30:29.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.788+0000 7ff16b000640 1 -- 172.21.15.42:0/1876085282 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ff14403d460 msgr2=0x7ff14403f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:29.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.788+0000 7ff16b000640 1 --2- 172.21.15.42:0/1876085282 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ff14403d460 0x7ff14403f970 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff150034000 tx=0x7ff150004730 comp rx=0 tx=0).stop 2024-07-04T17:30:29.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.788+0000 7ff16b000640 1 -- 172.21.15.42:0/1876085282 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1640ac5b0 msgr2=0x7ff164073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:29.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.788+0000 7ff16b000640 1 --2- 172.21.15.42:0/1876085282 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1640ac5b0 0x7ff164073340 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff15800c380 tx=0x7ff15800c840 comp rx=0 tx=0).stop 2024-07-04T17:30:29.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.789+0000 7ff16b000640 1 -- 172.21.15.42:0/1876085282 shutdown_connections 2024-07-04T17:30:29.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.789+0000 7ff16b000640 1 --2- 172.21.15.42:0/1876085282 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ff14403d460 0x7ff14403f970 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:29.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.789+0000 7ff16b000640 1 --2- 172.21.15.42:0/1876085282 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1640ac5b0 0x7ff164073340 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:29.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.789+0000 7ff16b000640 1 -- 172.21.15.42:0/1876085282 >> 172.21.15.42:0/1876085282 conn(0x7ff1640afaa0 msgr2=0x7ff16413fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:29.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.790+0000 7ff16b000640 1 -- 172.21.15.42:0/1876085282 shutdown_connections 2024-07-04T17:30:29.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:29.791+0000 7ff16b000640 1 -- 172.21.15.42:0/1876085282 wait complete. 2024-07-04T17:30:29.841 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:30:30.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.105+0000 7f0ada200640 1 -- 172.21.15.42:0/2686959700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ad406cea0 msgr2=0x7f0ad40b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:30.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.105+0000 7f0ada200640 1 --2- 172.21.15.42:0/2686959700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ad406cea0 0x7f0ad40b3030 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4007350 tx=0x7f0ac402b230 comp rx=0 tx=0).stop 2024-07-04T17:30:30.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.105+0000 7f0ada200640 1 -- 172.21.15.42:0/2686959700 shutdown_connections 2024-07-04T17:30:30.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.105+0000 7f0ada200640 1 --2- 172.21.15.42:0/2686959700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ad406cea0 0x7f0ad40b3030 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:30.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.105+0000 7f0ada200640 1 -- 172.21.15.42:0/2686959700 >> 172.21.15.42:0/2686959700 conn(0x7f0ad40bf700 msgr2=0x7f0ad40af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:30.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.105+0000 7f0ada200640 1 -- 172.21.15.42:0/2686959700 shutdown_connections 2024-07-04T17:30:30.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.106+0000 7f0ada200640 1 -- 172.21.15.42:0/2686959700 wait complete. 2024-07-04T17:30:30.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.108+0000 7f0ada200640 1 Processor -- start 2024-07-04T17:30:30.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.108+0000 7f0ada200640 1 -- start start 2024-07-04T17:30:30.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.109+0000 7f0ada200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ad406cea0 0x7f0ad41481b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:30.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.109+0000 7f0ada200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad406ef50 con 0x7f0ad406cea0 2024-07-04T17:30:30.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.109+0000 7f0ad3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ad406cea0 0x7f0ad41481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:30.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.109+0000 7f0ad3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ad406cea0 0x7f0ad41481b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40278/0 (socket says 172.21.15.42:40278) 2024-07-04T17:30:30.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.109+0000 7f0ad3400640 1 -- 172.21.15.42:0/2716771494 learned_addr learned my addr 172.21.15.42:0/2716771494 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:30.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.110+0000 7f0ad3400640 1 -- 172.21.15.42:0/2716771494 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ad4073090 con 0x7f0ad406cea0 2024-07-04T17:30:30.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.110+0000 7f0ad3400640 1 --2- 172.21.15.42:0/2716771494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ad406cea0 0x7f0ad41481b0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4007470 tx=0x7f0ac400a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:30.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.110+0000 7f0acb400640 1 -- 172.21.15.42:0/2716771494 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ac402baf0 con 0x7f0ad406cea0 2024-07-04T17:30:30.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.110+0000 7f0acb400640 1 -- 172.21.15.42:0/2716771494 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ac4007d50 con 0x7f0ad406cea0 2024-07-04T17:30:30.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.110+0000 7f0acb400640 1 -- 172.21.15.42:0/2716771494 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ac4005c90 con 0x7f0ad406cea0 2024-07-04T17:30:30.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.111+0000 7f0ada200640 1 -- 172.21.15.42:0/2716771494 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ad406ff30 con 0x7f0ad406cea0 2024-07-04T17:30:30.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.111+0000 7f0ada200640 1 -- 172.21.15.42:0/2716771494 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ad4146a00 con 0x7f0ad406cea0 2024-07-04T17:30:30.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.112+0000 7f0ada200640 1 -- 172.21.15.42:0/2716771494 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ad40b3030 con 0x7f0ad406cea0 2024-07-04T17:30:30.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.112+0000 7f0acb400640 1 -- 172.21.15.42:0/2716771494 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f0ac4003780 con 0x7f0ad406cea0 2024-07-04T17:30:30.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.113+0000 7f0acb400640 1 --2- 172.21.15.42:0/2716771494 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f0ab003d4f0 0x7f0ab003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:30.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.113+0000 7f0acb400640 1 -- 172.21.15.42:0/2716771494 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4483+0+0 (secure 0 0 0) 0x7f0ac4072160 con 0x7f0ad406cea0 2024-07-04T17:30:30.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.113+0000 7f0ad2a00640 1 --2- 172.21.15.42:0/2716771494 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f0ab003d4f0 0x7f0ab003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:30.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.114+0000 7f0ad2a00640 1 --2- 172.21.15.42:0/2716771494 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f0ab003d4f0 0x7f0ab003fa00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0abc000f60 tx=0x7f0abc007a30 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:30.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.118+0000 7f0acb400640 1 -- 172.21.15.42:0/2716771494 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f0ac403a3a0 con 0x7f0ad406cea0 2024-07-04T17:30:30.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.391+0000 7f0ada200640 1 -- 172.21.15.42:0/2716771494 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ad4069b70 con 0x7f0ad406cea0 2024-07-04T17:30:30.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.392+0000 7f0acb400640 1 -- 172.21.15.42:0/2716771494 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+6051 (secure 0 0 0) 0x7f0ac403ce00 con 0x7f0ad406cea0 2024-07-04T17:30:30.394 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:30:30.394 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":45,"btime":"2024-07-04T17:30:27:593817+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/343791977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":343791977},{"type":"v1","addr":"172.21.15.42:6827","nonce":343791977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":4722,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/3172446281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":3172446281},{"type":"v1","addr":"172.21.15.42:6829","nonce":3172446281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":4727,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/1866310217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1866310217},{"type":"v1","addr":"172.21.15.42:6831","nonce":1866310217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":4950,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2046217308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2046217308},{"type":"v1","addr":"172.21.15.42:6835","nonce":2046217308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:29:25.424728+0000","modified":"2024-07-04T17:29:28.496007+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4728":{"gid":4728,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":9,"addr":"172.21.15.42:6833/1072723036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1072723036},{"type":"v1","addr":"172.21.15.42:6833","nonce":1072723036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4728,"qdb_cluster":[4728]},"id":5}]} 2024-07-04T17:30:30.394 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 45 2024-07-04T17:30:30.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.395+0000 7f0ada200640 1 -- 172.21.15.42:0/2716771494 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f0ab003d4f0 msgr2=0x7f0ab003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:30.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.395+0000 7f0ada200640 1 --2- 172.21.15.42:0/2716771494 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f0ab003d4f0 0x7f0ab003fa00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0abc000f60 tx=0x7f0abc007a30 comp rx=0 tx=0).stop 2024-07-04T17:30:30.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.395+0000 7f0ada200640 1 -- 172.21.15.42:0/2716771494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ad406cea0 msgr2=0x7f0ad41481b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:30.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.395+0000 7f0ada200640 1 --2- 172.21.15.42:0/2716771494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ad406cea0 0x7f0ad41481b0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4007470 tx=0x7f0ac400a860 comp rx=0 tx=0).stop 2024-07-04T17:30:30.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.395+0000 7f0ada200640 1 -- 172.21.15.42:0/2716771494 shutdown_connections 2024-07-04T17:30:30.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.395+0000 7f0ada200640 1 --2- 172.21.15.42:0/2716771494 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f0ab003d4f0 0x7f0ab003fa00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:30.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.395+0000 7f0ada200640 1 --2- 172.21.15.42:0/2716771494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ad406cea0 0x7f0ad41481b0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:30.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.395+0000 7f0ada200640 1 -- 172.21.15.42:0/2716771494 >> 172.21.15.42:0/2716771494 conn(0x7f0ad40bf700 msgr2=0x7f0ad4149cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:30.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.395+0000 7f0ada200640 1 -- 172.21.15.42:0/2716771494 shutdown_connections 2024-07-04T17:30:30.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.395+0000 7f0ada200640 1 -- 172.21.15.42:0/2716771494 wait complete. 2024-07-04T17:30:30.423 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:30:30.437 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-04T17:30:30.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.702+0000 7fd6b2e00640 1 -- 172.21.15.42:0/3107014645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6ac06cea0 msgr2=0x7fd6ac0b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:30.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.702+0000 7fd6b2e00640 1 --2- 172.21.15.42:0/3107014645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6ac06cea0 0x7fd6ac0b3030 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fd69c009230 tx=0x7fd69c02a480 comp rx=0 tx=0).stop 2024-07-04T17:30:30.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.702+0000 7fd6b2e00640 1 -- 172.21.15.42:0/3107014645 shutdown_connections 2024-07-04T17:30:30.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.702+0000 7fd6b2e00640 1 --2- 172.21.15.42:0/3107014645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6ac06cea0 0x7fd6ac0b3030 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:30.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.702+0000 7fd6b2e00640 1 -- 172.21.15.42:0/3107014645 >> 172.21.15.42:0/3107014645 conn(0x7fd6ac0bf700 msgr2=0x7fd6ac0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:30.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.703+0000 7fd6b2e00640 1 -- 172.21.15.42:0/3107014645 shutdown_connections 2024-07-04T17:30:30.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.703+0000 7fd6b2e00640 1 -- 172.21.15.42:0/3107014645 wait complete. 2024-07-04T17:30:30.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.705+0000 7fd6b2e00640 1 Processor -- start 2024-07-04T17:30:30.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.705+0000 7fd6b2e00640 1 -- start start 2024-07-04T17:30:30.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.706+0000 7fd6b2e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6ac06cea0 0x7fd6ac0aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:30.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.706+0000 7fd6b2e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ac06ef50 con 0x7fd6ac06cea0 2024-07-04T17:30:30.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.706+0000 7fd6abe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6ac06cea0 0x7fd6ac0aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:30.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.706+0000 7fd6abe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6ac06cea0 0x7fd6ac0aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40284/0 (socket says 172.21.15.42:40284) 2024-07-04T17:30:30.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.706+0000 7fd6abe00640 1 -- 172.21.15.42:0/2934063646 learned_addr learned my addr 172.21.15.42:0/2934063646 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:30.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.707+0000 7fd6abe00640 1 -- 172.21.15.42:0/2934063646 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6ac073060 con 0x7fd6ac06cea0 2024-07-04T17:30:30.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.707+0000 7fd6abe00640 1 --2- 172.21.15.42:0/2934063646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6ac06cea0 0x7fd6ac0aede0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fd69c009290 tx=0x7fd69c031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:30.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.707+0000 7fd6a8c00640 1 -- 172.21.15.42:0/2934063646 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd69c041070 con 0x7fd6ac06cea0 2024-07-04T17:30:30.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.707+0000 7fd6a8c00640 1 -- 172.21.15.42:0/2934063646 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd69c003be0 con 0x7fd6ac06cea0 2024-07-04T17:30:30.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.708+0000 7fd6a8c00640 1 -- 172.21.15.42:0/2934063646 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd69c03b440 con 0x7fd6ac06cea0 2024-07-04T17:30:30.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.708+0000 7fd6b2e00640 1 -- 172.21.15.42:0/2934063646 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6ac1d32a0 con 0x7fd6ac06cea0 2024-07-04T17:30:30.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.708+0000 7fd6b2e00640 1 -- 172.21.15.42:0/2934063646 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6ac070e30 con 0x7fd6ac06cea0 2024-07-04T17:30:30.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.709+0000 7fd6b2e00640 1 -- 172.21.15.42:0/2934063646 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6ac063f70 con 0x7fd6ac06cea0 2024-07-04T17:30:30.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.709+0000 7fd6a8c00640 1 -- 172.21.15.42:0/2934063646 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7fd69c044070 con 0x7fd6ac06cea0 2024-07-04T17:30:30.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.710+0000 7fd6a8c00640 1 --2- 172.21.15.42:0/2934063646 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fd68803d4f0 0x7fd68803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:30.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.710+0000 7fd6a8c00640 1 -- 172.21.15.42:0/2934063646 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4483+0+0 (secure 0 0 0) 0x7fd69c073430 con 0x7fd6ac06cea0 2024-07-04T17:30:30.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.710+0000 7fd6a3e00640 1 --2- 172.21.15.42:0/2934063646 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fd68803d4f0 0x7fd68803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:30.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.711+0000 7fd6a3e00640 1 --2- 172.21.15.42:0/2934063646 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fd68803d4f0 0x7fd68803fa00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd694007fd0 tx=0x7fd694007920 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:30.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.715+0000 7fd6a8c00640 1 -- 172.21.15.42:0/2934063646 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fd6ac063f70 con 0x7fd6ac06cea0 2024-07-04T17:30:30.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.926+0000 7fd6b2e00640 1 -- 172.21.15.42:0/2934063646 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6ac0b3030 con 0x7fd6ac06cea0 2024-07-04T17:30:30.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.928+0000 7fd6a8c00640 1 -- 172.21.15.42:0/2934063646 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+12115 (secure 0 0 0) 0x7fd69c0730d0 con 0x7fd6ac06cea0 2024-07-04T17:30:30.929 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:30:30.929 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":60,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:30:29.776828+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:29:22.942399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.0299999713897705,"score_stable":1.0299999713897705,"optimal_score":1,"raw_score_acting":1.0299999713897705,"raw_score_stable":1.0299999713897705,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:29:24.406438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/934943970":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/4215342546":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/3491363662":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/3597724339":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/1322461005":"2024-07-05T17:30:25.586824+0000","172.21.15.42:6825/3045931584":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/461490182":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/7072365":"2024-07-04T18:30:09.692358+0000","172.21.15.42:0/176559669":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/1810008304":"2024-07-05T17:29:38.572697+0000","172.21.15.42:6834/1327681666":"2024-07-05T17:30:26.507800+0000","172.21.15.42:0/1534342117":"2024-07-05T17:29:38.572697+0000","172.21.15.42:6835/1327681666":"2024-07-05T17:30:26.507800+0000","172.21.15.42:6825/2638271195":"2024-07-05T17:29:38.572697+0000","172.21.15.42:0/2056846029":"2024-07-05T17:30:25.586824+0000","172.21.15.42:6824/2638271195":"2024-07-05T17:29:38.572697+0000","172.21.15.42:0/591942290":"2024-07-05T17:29:38.572697+0000","172.21.15.42:6824/3045931584":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/2926090870":"2024-07-05T17:29:38.572697+0000","172.21.15.42:0/136286797":"2024-07-05T17:29:38.572697+0000","172.21.15.42:0/1793407260":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/2800592161":"2024-07-05T17:29:38.572697+0000","172.21.15.42:0/1891351459":"2024-07-05T17:29:38.572697+0000","172.21.15.42:0/3114375174":"2024-07-05T17:29:38.572697+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:30:30.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.930+0000 7fd6b2e00640 1 -- 172.21.15.42:0/2934063646 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fd68803d4f0 msgr2=0x7fd68803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:30.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.930+0000 7fd6b2e00640 1 --2- 172.21.15.42:0/2934063646 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fd68803d4f0 0x7fd68803fa00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd694007fd0 tx=0x7fd694007920 comp rx=0 tx=0).stop 2024-07-04T17:30:30.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.930+0000 7fd6b2e00640 1 -- 172.21.15.42:0/2934063646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6ac06cea0 msgr2=0x7fd6ac0aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:30.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.930+0000 7fd6b2e00640 1 --2- 172.21.15.42:0/2934063646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6ac06cea0 0x7fd6ac0aede0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fd69c009290 tx=0x7fd69c031a90 comp rx=0 tx=0).stop 2024-07-04T17:30:30.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.930+0000 7fd6b2e00640 1 -- 172.21.15.42:0/2934063646 shutdown_connections 2024-07-04T17:30:30.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.930+0000 7fd6b2e00640 1 --2- 172.21.15.42:0/2934063646 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fd68803d4f0 0x7fd68803fa00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:30.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.930+0000 7fd6b2e00640 1 --2- 172.21.15.42:0/2934063646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6ac06cea0 0x7fd6ac0aede0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:30.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.930+0000 7fd6b2e00640 1 -- 172.21.15.42:0/2934063646 >> 172.21.15.42:0/2934063646 conn(0x7fd6ac0bf700 msgr2=0x7fd6ac1490c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:30.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.931+0000 7fd6b2e00640 1 -- 172.21.15.42:0/2934063646 shutdown_connections 2024-07-04T17:30:30.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:30.931+0000 7fd6b2e00640 1 -- 172.21.15.42:0/2934063646 wait complete. 2024-07-04T17:30:30.960 INFO:tasks.cephfs.filesystem:Destroying file system backup_fs and related pools 2024-07-04T17:30:30.960 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:30:31.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.240+0000 7fa6fac00640 1 -- 172.21.15.42:0/3968377380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6f4140440 msgr2=0x7fa6f406c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:31.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.240+0000 7fa6fac00640 1 --2- 172.21.15.42:0/3968377380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6f4140440 0x7fa6f406c8b0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4005fd0 tx=0x7fa6e402bc80 comp rx=0 tx=0).stop 2024-07-04T17:30:31.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.240+0000 7fa6fac00640 1 -- 172.21.15.42:0/3968377380 shutdown_connections 2024-07-04T17:30:31.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.240+0000 7fa6fac00640 1 --2- 172.21.15.42:0/3968377380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6f4140440 0x7fa6f406c8b0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:31.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.240+0000 7fa6fac00640 1 -- 172.21.15.42:0/3968377380 >> 172.21.15.42:0/3968377380 conn(0x7fa6f40bebc0 msgr2=0x7fa6f40befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:31.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.240+0000 7fa6fac00640 1 -- 172.21.15.42:0/3968377380 shutdown_connections 2024-07-04T17:30:31.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.240+0000 7fa6fac00640 1 -- 172.21.15.42:0/3968377380 wait complete. 2024-07-04T17:30:31.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.242+0000 7fa6fac00640 1 Processor -- start 2024-07-04T17:30:31.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.242+0000 7fa6fac00640 1 -- start start 2024-07-04T17:30:31.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.243+0000 7fa6fac00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6f40b20c0 0x7fa6f41d55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:31.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.243+0000 7fa6fac00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f40b5480 con 0x7fa6f40b20c0 2024-07-04T17:30:31.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.244+0000 7fa6f9800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6f40b20c0 0x7fa6f41d55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:31.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.244+0000 7fa6f9800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6f40b20c0 0x7fa6f41d55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40296/0 (socket says 172.21.15.42:40296) 2024-07-04T17:30:31.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.244+0000 7fa6f9800640 1 -- 172.21.15.42:0/1014260775 learned_addr learned my addr 172.21.15.42:0/1014260775 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:31.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.244+0000 7fa6f9800640 1 -- 172.21.15.42:0/1014260775 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f41d7230 con 0x7fa6f40b20c0 2024-07-04T17:30:31.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.245+0000 7fa6f9800640 1 --2- 172.21.15.42:0/1014260775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6f40b20c0 0x7fa6f41d55a0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800c3b0 tx=0x7fa6e800c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:31.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.245+0000 7fa6f1600640 1 -- 172.21.15.42:0/1014260775 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6e8011780 con 0x7fa6f40b20c0 2024-07-04T17:30:31.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.245+0000 7fa6f1600640 1 -- 172.21.15.42:0/1014260775 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6e8017070 con 0x7fa6f40b20c0 2024-07-04T17:30:31.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.245+0000 7fa6f1600640 1 -- 172.21.15.42:0/1014260775 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6e8011c00 con 0x7fa6f40b20c0 2024-07-04T17:30:31.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.245+0000 7fa6fac00640 1 -- 172.21.15.42:0/1014260775 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f41e1de0 con 0x7fa6f40b20c0 2024-07-04T17:30:31.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.245+0000 7fa6fac00640 1 -- 172.21.15.42:0/1014260775 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f41d7910 con 0x7fa6f40b20c0 2024-07-04T17:30:31.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.246+0000 7fa6fac00640 1 -- 172.21.15.42:0/1014260775 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6f406c780 con 0x7fa6f40b20c0 2024-07-04T17:30:31.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.246+0000 7fa6f1600640 1 -- 172.21.15.42:0/1014260775 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7fa6e801c460 con 0x7fa6f40b20c0 2024-07-04T17:30:31.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.247+0000 7fa6f1600640 1 --2- 172.21.15.42:0/1014260775 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fa6d803d4f0 0x7fa6d803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:31.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.247+0000 7fa6f1600640 1 -- 172.21.15.42:0/1014260775 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4483+0+0 (secure 0 0 0) 0x7fa6e8051a90 con 0x7fa6f40b20c0 2024-07-04T17:30:31.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.247+0000 7fa6f8e00640 1 --2- 172.21.15.42:0/1014260775 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fa6d803d4f0 0x7fa6d803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:31.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.248+0000 7fa6f8e00640 1 --2- 172.21.15.42:0/1014260775 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fa6d803d4f0 0x7fa6d803fa00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4035000 tx=0x7fa6e4004ab0 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:31.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.252+0000 7fa6f1600640 1 -- 172.21.15.42:0/1014260775 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fa6e800a140 con 0x7fa6f40b20c0 2024-07-04T17:30:31.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.507+0000 7fa6fac00640 1 -- 172.21.15.42:0/1014260775 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6f40741c0 con 0x7fa6f40b20c0 2024-07-04T17:30:31.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.508+0000 7fa6f1600640 1 -- 172.21.15.42:0/1014260775 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+6051 (secure 0 0 0) 0x7fa6e801c800 con 0x7fa6f40b20c0 2024-07-04T17:30:31.510 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:30:31.510 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":45,"btime":"2024-07-04T17:30:27:593817+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/343791977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":343791977},{"type":"v1","addr":"172.21.15.42:6827","nonce":343791977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":4722,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/3172446281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":3172446281},{"type":"v1","addr":"172.21.15.42:6829","nonce":3172446281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":4727,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/1866310217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1866310217},{"type":"v1","addr":"172.21.15.42:6831","nonce":1866310217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":4950,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2046217308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2046217308},{"type":"v1","addr":"172.21.15.42:6835","nonce":2046217308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:29:25.424728+0000","modified":"2024-07-04T17:29:28.496007+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4728":{"gid":4728,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":9,"addr":"172.21.15.42:6833/1072723036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1072723036},{"type":"v1","addr":"172.21.15.42:6833","nonce":1072723036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4728,"qdb_cluster":[4728]},"id":5}]} 2024-07-04T17:30:31.511 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 45 2024-07-04T17:30:31.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.511+0000 7fa6fac00640 1 -- 172.21.15.42:0/1014260775 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fa6d803d4f0 msgr2=0x7fa6d803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:31.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.511+0000 7fa6fac00640 1 --2- 172.21.15.42:0/1014260775 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fa6d803d4f0 0x7fa6d803fa00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4035000 tx=0x7fa6e4004ab0 comp rx=0 tx=0).stop 2024-07-04T17:30:31.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.511+0000 7fa6fac00640 1 -- 172.21.15.42:0/1014260775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6f40b20c0 msgr2=0x7fa6f41d55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:31.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.511+0000 7fa6fac00640 1 --2- 172.21.15.42:0/1014260775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6f40b20c0 0x7fa6f41d55a0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800c3b0 tx=0x7fa6e800c870 comp rx=0 tx=0).stop 2024-07-04T17:30:31.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.511+0000 7fa6fac00640 1 -- 172.21.15.42:0/1014260775 shutdown_connections 2024-07-04T17:30:31.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.511+0000 7fa6fac00640 1 --2- 172.21.15.42:0/1014260775 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fa6d803d4f0 0x7fa6d803fa00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:31.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.511+0000 7fa6fac00640 1 --2- 172.21.15.42:0/1014260775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6f40b20c0 0x7fa6f41d55a0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:31.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.511+0000 7fa6fac00640 1 -- 172.21.15.42:0/1014260775 >> 172.21.15.42:0/1014260775 conn(0x7fa6f40bebc0 msgr2=0x7fa6f413f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:31.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.511+0000 7fa6fac00640 1 -- 172.21.15.42:0/1014260775 shutdown_connections 2024-07-04T17:30:31.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.512+0000 7fa6fac00640 1 -- 172.21.15.42:0/1014260775 wait complete. 2024-07-04T17:30:31.539 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:30:31.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.818+0000 7f052d200640 1 -- 172.21.15.42:0/2857285709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05280bad70 msgr2=0x7f052806c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:31.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.818+0000 7f052d200640 1 --2- 172.21.15.42:0/2857285709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05280bad70 0x7f052806c2c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f0510005fd0 tx=0x7f051002a560 comp rx=0 tx=0).stop 2024-07-04T17:30:31.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.818+0000 7f052d200640 1 -- 172.21.15.42:0/2857285709 shutdown_connections 2024-07-04T17:30:31.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.818+0000 7f052d200640 1 --2- 172.21.15.42:0/2857285709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05280bad70 0x7f052806c2c0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:31.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.818+0000 7f052d200640 1 -- 172.21.15.42:0/2857285709 >> 172.21.15.42:0/2857285709 conn(0x7f05280bebc0 msgr2=0x7f05280befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:31.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.818+0000 7f052d200640 1 -- 172.21.15.42:0/2857285709 shutdown_connections 2024-07-04T17:30:31.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.818+0000 7f052d200640 1 -- 172.21.15.42:0/2857285709 wait complete. 2024-07-04T17:30:31.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.820+0000 7f052d200640 1 Processor -- start 2024-07-04T17:30:31.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.821+0000 7f052d200640 1 -- start start 2024-07-04T17:30:31.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.822+0000 7f052d200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f052806d580 0x7f05280bb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:31.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.822+0000 7f052d200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f05280b2810 con 0x7f052806d580 2024-07-04T17:30:31.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.822+0000 7f0527400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f052806d580 0x7f05280bb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:31.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.822+0000 7f0527400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f052806d580 0x7f05280bb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40312/0 (socket says 172.21.15.42:40312) 2024-07-04T17:30:31.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.822+0000 7f0527400640 1 -- 172.21.15.42:0/1042373502 learned_addr learned my addr 172.21.15.42:0/1042373502 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:31.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.823+0000 7f0527400640 1 -- 172.21.15.42:0/1042373502 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05280b8230 con 0x7f052806d580 2024-07-04T17:30:31.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.823+0000 7f0527400640 1 --2- 172.21.15.42:0/1042373502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f052806d580 0x7f05280bb660 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f051800c2f0 tx=0x7f051800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:31.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.823+0000 7f051fe00640 1 -- 172.21.15.42:0/1042373502 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0518017070 con 0x7f052806d580 2024-07-04T17:30:31.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.823+0000 7f051fe00640 1 -- 172.21.15.42:0/1042373502 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0518005410 con 0x7f052806d580 2024-07-04T17:30:31.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.823+0000 7f051fe00640 1 -- 172.21.15.42:0/1042373502 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0518003780 con 0x7f052806d580 2024-07-04T17:30:31.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.823+0000 7f052d200640 1 -- 172.21.15.42:0/1042373502 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05281e48c0 con 0x7f052806d580 2024-07-04T17:30:31.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.824+0000 7f052d200640 1 -- 172.21.15.42:0/1042373502 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05280b61a0 con 0x7f052806d580 2024-07-04T17:30:31.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.825+0000 7f051fe00640 1 -- 172.21.15.42:0/1042373502 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f051801a070 con 0x7f052806d580 2024-07-04T17:30:31.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.825+0000 7f052d200640 1 -- 172.21.15.42:0/1042373502 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05280b8400 con 0x7f052806d580 2024-07-04T17:30:31.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.825+0000 7f051fe00640 1 --2- 172.21.15.42:0/1042373502 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f050403d4f0 0x7f050403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:31.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.826+0000 7f051fe00640 1 -- 172.21.15.42:0/1042373502 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4483+0+0 (secure 0 0 0) 0x7f0518051090 con 0x7f052806d580 2024-07-04T17:30:31.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.826+0000 7f051ea00640 1 --2- 172.21.15.42:0/1042373502 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f050403d4f0 0x7f050403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:31.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.826+0000 7f051ea00640 1 --2- 172.21.15.42:0/1042373502 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f050403d4f0 0x7f050403fa00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0510033000 tx=0x7f0510004730 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:31.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:31.831+0000 7f051fe00640 1 -- 172.21.15.42:0/1042373502 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f0518010300 con 0x7f052806d580 2024-07-04T17:30:32.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.078+0000 7f052d200640 1 -- 172.21.15.42:0/1042373502 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05280b8020 con 0x7f052806d580 2024-07-04T17:30:32.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.080+0000 7f051fe00640 1 -- 172.21.15.42:0/1042373502 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+6051 (secure 0 0 0) 0x7f0518010e20 con 0x7f052806d580 2024-07-04T17:30:32.081 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:30:32.081 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":45,"btime":"2024-07-04T17:30:27:593817+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/343791977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":343791977},{"type":"v1","addr":"172.21.15.42:6827","nonce":343791977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":4722,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/3172446281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":3172446281},{"type":"v1","addr":"172.21.15.42:6829","nonce":3172446281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":4727,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/1866310217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1866310217},{"type":"v1","addr":"172.21.15.42:6831","nonce":1866310217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":4950,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2046217308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2046217308},{"type":"v1","addr":"172.21.15.42:6835","nonce":2046217308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:29:25.424728+0000","modified":"2024-07-04T17:29:28.496007+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4728":{"gid":4728,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":9,"addr":"172.21.15.42:6833/1072723036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1072723036},{"type":"v1","addr":"172.21.15.42:6833","nonce":1072723036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4728,"qdb_cluster":[4728]},"id":5}]} 2024-07-04T17:30:32.082 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 45 2024-07-04T17:30:32.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.082+0000 7f052d200640 1 -- 172.21.15.42:0/1042373502 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f050403d4f0 msgr2=0x7f050403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:32.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.082+0000 7f052d200640 1 --2- 172.21.15.42:0/1042373502 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f050403d4f0 0x7f050403fa00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0510033000 tx=0x7f0510004730 comp rx=0 tx=0).stop 2024-07-04T17:30:32.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.082+0000 7f052d200640 1 -- 172.21.15.42:0/1042373502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f052806d580 msgr2=0x7f05280bb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:32.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.082+0000 7f052d200640 1 --2- 172.21.15.42:0/1042373502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f052806d580 0x7f05280bb660 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f051800c2f0 tx=0x7f051800c7b0 comp rx=0 tx=0).stop 2024-07-04T17:30:32.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.082+0000 7f052d200640 1 -- 172.21.15.42:0/1042373502 shutdown_connections 2024-07-04T17:30:32.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.083+0000 7f052d200640 1 --2- 172.21.15.42:0/1042373502 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f050403d4f0 0x7f050403fa00 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:32.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.083+0000 7f052d200640 1 --2- 172.21.15.42:0/1042373502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f052806d580 0x7f05280bb660 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:32.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.083+0000 7f052d200640 1 -- 172.21.15.42:0/1042373502 >> 172.21.15.42:0/1042373502 conn(0x7f05280bebc0 msgr2=0x7f0528141700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:32.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.083+0000 7f052d200640 1 -- 172.21.15.42:0/1042373502 shutdown_connections 2024-07-04T17:30:32.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.083+0000 7f052d200640 1 -- 172.21.15.42:0/1042373502 wait complete. 2024-07-04T17:30:32.106 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:30:32.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.375+0000 7f7943e00640 1 -- 172.21.15.42:0/2322567342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79440b1760 msgr2=0x7f79440b8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:32.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.375+0000 7f7943e00640 1 --2- 172.21.15.42:0/2322567342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79440b1760 0x7f79440b8da0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f7934005fd0 tx=0x7f793402aac0 comp rx=0 tx=0).stop 2024-07-04T17:30:32.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.375+0000 7f7943e00640 1 -- 172.21.15.42:0/2322567342 shutdown_connections 2024-07-04T17:30:32.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.375+0000 7f7943e00640 1 --2- 172.21.15.42:0/2322567342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79440b1760 0x7f79440b8da0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:32.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.375+0000 7f7943e00640 1 -- 172.21.15.42:0/2322567342 >> 172.21.15.42:0/2322567342 conn(0x7f79440bebc0 msgr2=0x7f79440befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:32.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.375+0000 7f7943e00640 1 -- 172.21.15.42:0/2322567342 shutdown_connections 2024-07-04T17:30:32.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.375+0000 7f7943e00640 1 -- 172.21.15.42:0/2322567342 wait complete. 2024-07-04T17:30:32.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.377+0000 7f7943e00640 1 Processor -- start 2024-07-04T17:30:32.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.378+0000 7f7943e00640 1 -- start start 2024-07-04T17:30:32.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.379+0000 7f7943e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f794406c020 0x7f79441d6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:32.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.379+0000 7f7943e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f794414eed0 con 0x7f794406c020 2024-07-04T17:30:32.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.379+0000 7f7942a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f794406c020 0x7f79441d6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:32.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.379+0000 7f7942a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f794406c020 0x7f79441d6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40328/0 (socket says 172.21.15.42:40328) 2024-07-04T17:30:32.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.379+0000 7f7942a00640 1 -- 172.21.15.42:0/1035805833 learned_addr learned my addr 172.21.15.42:0/1035805833 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:32.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.379+0000 7f7942a00640 1 -- 172.21.15.42:0/1035805833 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79440a9620 con 0x7f794406c020 2024-07-04T17:30:32.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.380+0000 7f7942a00640 1 --2- 172.21.15.42:0/1035805833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f794406c020 0x7f79441d6390 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f7934006010 tx=0x7f7934003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:32.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.380+0000 7f793b400640 1 -- 172.21.15.42:0/1035805833 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7934044070 con 0x7f794406c020 2024-07-04T17:30:32.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.380+0000 7f793b400640 1 -- 172.21.15.42:0/1035805833 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7934004250 con 0x7f794406c020 2024-07-04T17:30:32.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.380+0000 7f793b400640 1 -- 172.21.15.42:0/1035805833 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7934004590 con 0x7f794406c020 2024-07-04T17:30:32.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.380+0000 7f7943e00640 1 -- 172.21.15.42:0/1035805833 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79441d9490 con 0x7f794406c020 2024-07-04T17:30:32.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.380+0000 7f7943e00640 1 -- 172.21.15.42:0/1035805833 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79441d9130 con 0x7f794406c020 2024-07-04T17:30:32.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.382+0000 7f7943e00640 1 -- 172.21.15.42:0/1035805833 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79440b8da0 con 0x7f794406c020 2024-07-04T17:30:32.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.382+0000 7f793b400640 1 -- 172.21.15.42:0/1035805833 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f7934047070 con 0x7f794406c020 2024-07-04T17:30:32.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.382+0000 7f793b400640 1 --2- 172.21.15.42:0/1035805833 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f792403d4f0 0x7f792403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:32.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.382+0000 7f793b400640 1 -- 172.21.15.42:0/1035805833 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4483+0+0 (secure 0 0 0) 0x7f793403c070 con 0x7f794406c020 2024-07-04T17:30:32.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.382+0000 7f7942000640 1 --2- 172.21.15.42:0/1035805833 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f792403d4f0 0x7f792403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:32.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.383+0000 7f7942000640 1 --2- 172.21.15.42:0/1035805833 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f792403d4f0 0x7f792403fa00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f79440b8760 tx=0x7f79300079f0 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:32.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.388+0000 7f793b400640 1 -- 172.21.15.42:0/1035805833 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f7934032030 con 0x7f794406c020 2024-07-04T17:30:32.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.598+0000 7f7943e00640 1 -- 172.21.15.42:0/1035805833 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7944068e30 con 0x7f794406c020 2024-07-04T17:30:32.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.600+0000 7f793b400640 1 -- 172.21.15.42:0/1035805833 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+12115 (secure 0 0 0) 0x7f7934047360 con 0x7f794406c020 2024-07-04T17:30:32.601 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:30:32.601 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":60,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:30:29.776828+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:29:22.942399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.0299999713897705,"score_stable":1.0299999713897705,"optimal_score":1,"raw_score_acting":1.0299999713897705,"raw_score_stable":1.0299999713897705,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:29:24.406438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/934943970":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/4215342546":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/3491363662":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/3597724339":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/1322461005":"2024-07-05T17:30:25.586824+0000","172.21.15.42:6825/3045931584":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/461490182":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/7072365":"2024-07-04T18:30:09.692358+0000","172.21.15.42:0/176559669":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/1810008304":"2024-07-05T17:29:38.572697+0000","172.21.15.42:6834/1327681666":"2024-07-05T17:30:26.507800+0000","172.21.15.42:0/1534342117":"2024-07-05T17:29:38.572697+0000","172.21.15.42:6835/1327681666":"2024-07-05T17:30:26.507800+0000","172.21.15.42:6825/2638271195":"2024-07-05T17:29:38.572697+0000","172.21.15.42:0/2056846029":"2024-07-05T17:30:25.586824+0000","172.21.15.42:6824/2638271195":"2024-07-05T17:29:38.572697+0000","172.21.15.42:0/591942290":"2024-07-05T17:29:38.572697+0000","172.21.15.42:6824/3045931584":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/2926090870":"2024-07-05T17:29:38.572697+0000","172.21.15.42:0/136286797":"2024-07-05T17:29:38.572697+0000","172.21.15.42:0/1793407260":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/2800592161":"2024-07-05T17:29:38.572697+0000","172.21.15.42:0/1891351459":"2024-07-05T17:29:38.572697+0000","172.21.15.42:0/3114375174":"2024-07-05T17:29:38.572697+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:30:32.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.602+0000 7f7943e00640 1 -- 172.21.15.42:0/1035805833 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f792403d4f0 msgr2=0x7f792403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:32.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.602+0000 7f7943e00640 1 --2- 172.21.15.42:0/1035805833 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f792403d4f0 0x7f792403fa00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f79440b8760 tx=0x7f79300079f0 comp rx=0 tx=0).stop 2024-07-04T17:30:32.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.603+0000 7f7943e00640 1 -- 172.21.15.42:0/1035805833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f794406c020 msgr2=0x7f79441d6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:32.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.603+0000 7f7943e00640 1 --2- 172.21.15.42:0/1035805833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f794406c020 0x7f79441d6390 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f7934006010 tx=0x7f7934003f10 comp rx=0 tx=0).stop 2024-07-04T17:30:32.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.603+0000 7f7943e00640 1 -- 172.21.15.42:0/1035805833 shutdown_connections 2024-07-04T17:30:32.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.603+0000 7f7943e00640 1 --2- 172.21.15.42:0/1035805833 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f792403d4f0 0x7f792403fa00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:32.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.603+0000 7f7943e00640 1 --2- 172.21.15.42:0/1035805833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f794406c020 0x7f79441d6390 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:32.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.603+0000 7f7943e00640 1 -- 172.21.15.42:0/1035805833 >> 172.21.15.42:0/1035805833 conn(0x7f79440bebc0 msgr2=0x7f79440be360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:32.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.603+0000 7f7943e00640 1 -- 172.21.15.42:0/1035805833 shutdown_connections 2024-07-04T17:30:32.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.603+0000 7f7943e00640 1 -- 172.21.15.42:0/1035805833 wait complete. 2024-07-04T17:30:32.628 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail backup_fs 2024-07-04T17:30:32.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.907+0000 7fa54d800640 1 -- 172.21.15.42:0/601457069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa54806cf90 msgr2=0x7fa548141dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:32.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.907+0000 7fa54d800640 1 --2- 172.21.15.42:0/601457069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa54806cf90 0x7fa548141dd0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fa5380097b0 tx=0x7fa53802a530 comp rx=0 tx=0).stop 2024-07-04T17:30:32.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.907+0000 7fa54d800640 1 -- 172.21.15.42:0/601457069 shutdown_connections 2024-07-04T17:30:32.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.907+0000 7fa54d800640 1 --2- 172.21.15.42:0/601457069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa54806cf90 0x7fa548141dd0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:32.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.907+0000 7fa54d800640 1 -- 172.21.15.42:0/601457069 >> 172.21.15.42:0/601457069 conn(0x7fa5480bebc0 msgr2=0x7fa5480befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:32.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.908+0000 7fa54d800640 1 -- 172.21.15.42:0/601457069 shutdown_connections 2024-07-04T17:30:32.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.908+0000 7fa54d800640 1 -- 172.21.15.42:0/601457069 wait complete. 2024-07-04T17:30:32.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.910+0000 7fa54d800640 1 Processor -- start 2024-07-04T17:30:32.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.910+0000 7fa54d800640 1 -- start start 2024-07-04T17:30:32.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.911+0000 7fa54d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa54806cf90 0x7fa548139f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:32.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.911+0000 7fa54d800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa5480babe0 con 0x7fa54806cf90 2024-07-04T17:30:32.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.911+0000 7fa547e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa54806cf90 0x7fa548139f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:32.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.911+0000 7fa547e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa54806cf90 0x7fa548139f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40344/0 (socket says 172.21.15.42:40344) 2024-07-04T17:30:32.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.911+0000 7fa547e00640 1 -- 172.21.15.42:0/1257042758 learned_addr learned my addr 172.21.15.42:0/1257042758 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:32.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.912+0000 7fa547e00640 1 -- 172.21.15.42:0/1257042758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa54813bc90 con 0x7fa54806cf90 2024-07-04T17:30:32.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.912+0000 7fa547e00640 1 --2- 172.21.15.42:0/1257042758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa54806cf90 0x7fa548139f60 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fa5380098d0 tx=0x7fa538033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:32.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.913+0000 7fa544c00640 1 -- 172.21.15.42:0/1257042758 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa538031bc0 con 0x7fa54806cf90 2024-07-04T17:30:32.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.913+0000 7fa544c00640 1 -- 172.21.15.42:0/1257042758 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa538031e50 con 0x7fa54806cf90 2024-07-04T17:30:32.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.913+0000 7fa544c00640 1 -- 172.21.15.42:0/1257042758 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa538033260 con 0x7fa54806cf90 2024-07-04T17:30:32.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.913+0000 7fa54d800640 1 -- 172.21.15.42:0/1257042758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa548138d10 con 0x7fa54806cf90 2024-07-04T17:30:32.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.913+0000 7fa54d800640 1 -- 172.21.15.42:0/1257042758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa548139230 con 0x7fa54806cf90 2024-07-04T17:30:32.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.914+0000 7fa54d800640 1 -- 172.21.15.42:0/1257042758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa54813bab0 con 0x7fa54806cf90 2024-07-04T17:30:32.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.914+0000 7fa544c00640 1 -- 172.21.15.42:0/1257042758 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7fa538033400 con 0x7fa54806cf90 2024-07-04T17:30:32.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.914+0000 7fa544c00640 1 --2- 172.21.15.42:0/1257042758 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fa52403d4f0 0x7fa52403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:32.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.915+0000 7fa544c00640 1 -- 172.21.15.42:0/1257042758 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4483+0+0 (secure 0 0 0) 0x7fa538073270 con 0x7fa54806cf90 2024-07-04T17:30:32.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.915+0000 7fa53fe00640 1 --2- 172.21.15.42:0/1257042758 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fa52403d4f0 0x7fa52403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:32.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.916+0000 7fa53fe00640 1 --2- 172.21.15.42:0/1257042758 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fa52403d4f0 0x7fa52403fa00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa5300015c0 tx=0x7fa530007a90 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:32.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:32.919+0000 7fa544c00640 1 -- 172.21.15.42:0/1257042758 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fa54813bab0 con 0x7fa54806cf90 2024-07-04T17:30:33.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.174+0000 7fa54d800640 1 -- 172.21.15.42:0/1257042758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "backup_fs"} v 0) -- 0x7fa5480bb1b0 con 0x7fa54806cf90 2024-07-04T17:30:33.177 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-07-04T17:30:33.176+0000 7f8616e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-04T17:30:33.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.177+0000 7fa544c00640 1 -- 172.21.15.42:0/1257042758 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "backup_fs"}]=0 backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v46) ==== 167+0+0 (secure 0 0 0) 0x7fa53803f030 con 0x7fa54806cf90 2024-07-04T17:30:33.179 INFO:teuthology.orchestra.run.smithi042.stderr:backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-04T17:30:33.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.181+0000 7fa54d800640 1 -- 172.21.15.42:0/1257042758 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fa52403d4f0 msgr2=0x7fa52403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:33.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.181+0000 7fa54d800640 1 --2- 172.21.15.42:0/1257042758 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fa52403d4f0 0x7fa52403fa00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa5300015c0 tx=0x7fa530007a90 comp rx=0 tx=0).stop 2024-07-04T17:30:33.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.181+0000 7fa54d800640 1 -- 172.21.15.42:0/1257042758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa54806cf90 msgr2=0x7fa548139f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:33.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.181+0000 7fa54d800640 1 --2- 172.21.15.42:0/1257042758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa54806cf90 0x7fa548139f60 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fa5380098d0 tx=0x7fa538033e40 comp rx=0 tx=0).stop 2024-07-04T17:30:33.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.181+0000 7fa54d800640 1 -- 172.21.15.42:0/1257042758 shutdown_connections 2024-07-04T17:30:33.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.181+0000 7fa54d800640 1 --2- 172.21.15.42:0/1257042758 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fa52403d4f0 0x7fa52403fa00 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:33.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.181+0000 7fa54d800640 1 --2- 172.21.15.42:0/1257042758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa54806cf90 0x7fa548139f60 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:33.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.181+0000 7fa54d800640 1 -- 172.21.15.42:0/1257042758 >> 172.21.15.42:0/1257042758 conn(0x7fa5480bebc0 msgr2=0x7fa5480b9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:33.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.182+0000 7fa54d800640 1 -- 172.21.15.42:0/1257042758 shutdown_connections 2024-07-04T17:30:33.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.182+0000 7fa54d800640 1 -- 172.21.15.42:0/1257042758 wait complete. 2024-07-04T17:30:33.208 INFO:tasks.ceph.mds.c.smithi042.stderr: -4111> 2024-07-04T17:29:26.400+0000 7f0c8ee00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-04T17:30:33.213 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm backup_fs --yes-i-really-mean-it 2024-07-04T17:30:33.346 INFO:tasks.ceph.mds.c.smithi042.stdout:starting mds.c at 2024-07-04T17:30:33.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.485+0000 7f0285400640 1 -- 172.21.15.42:0/3584038223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0280070cf0 msgr2=0x7f0280149ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:33.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.485+0000 7f0285400640 1 --2- 172.21.15.42:0/3584038223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0280070cf0 0x7f0280149ea0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f026800b2b0 tx=0x7f026802ac80 comp rx=0 tx=0).stop 2024-07-04T17:30:33.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.486+0000 7f0285400640 1 -- 172.21.15.42:0/3584038223 shutdown_connections 2024-07-04T17:30:33.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.486+0000 7f0285400640 1 --2- 172.21.15.42:0/3584038223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0280070cf0 0x7f0280149ea0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:33.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.486+0000 7f0285400640 1 -- 172.21.15.42:0/3584038223 >> 172.21.15.42:0/3584038223 conn(0x7f028006fe40 msgr2=0x7f028006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:33.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.486+0000 7f0285400640 1 -- 172.21.15.42:0/3584038223 shutdown_connections 2024-07-04T17:30:33.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.486+0000 7f0285400640 1 -- 172.21.15.42:0/3584038223 wait complete. 2024-07-04T17:30:33.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.488+0000 7f0285400640 1 Processor -- start 2024-07-04T17:30:33.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.488+0000 7f0285400640 1 -- start start 2024-07-04T17:30:33.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.489+0000 7f0285400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0280070cf0 0x7f028014cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:33.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.489+0000 7f0285400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0280071ef0 con 0x7f0280070cf0 2024-07-04T17:30:33.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.489+0000 7f027e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0280070cf0 0x7f028014cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:33.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.490+0000 7f027e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0280070cf0 0x7f028014cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40368/0 (socket says 172.21.15.42:40368) 2024-07-04T17:30:33.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.490+0000 7f027e800640 1 -- 172.21.15.42:0/4209050977 learned_addr learned my addr 172.21.15.42:0/4209050977 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:33.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.490+0000 7f027e800640 1 -- 172.21.15.42:0/4209050977 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0280145c70 con 0x7f0280070cf0 2024-07-04T17:30:33.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.490+0000 7f027e800640 1 --2- 172.21.15.42:0/4209050977 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0280070cf0 0x7f028014cab0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f027000c320 tx=0x7f027000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:33.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.491+0000 7f0277400640 1 -- 172.21.15.42:0/4209050977 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02700116c0 con 0x7f0280070cf0 2024-07-04T17:30:33.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.491+0000 7f0277400640 1 -- 172.21.15.42:0/4209050977 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0270017070 con 0x7f0280070cf0 2024-07-04T17:30:33.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.491+0000 7f0285400640 1 -- 172.21.15.42:0/4209050977 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02801e34d0 con 0x7f0280070cf0 2024-07-04T17:30:33.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.491+0000 7f0277400640 1 -- 172.21.15.42:0/4209050977 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0270011b00 con 0x7f0280070cf0 2024-07-04T17:30:33.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.491+0000 7f0285400640 1 -- 172.21.15.42:0/4209050977 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0280144d20 con 0x7f0280070cf0 2024-07-04T17:30:33.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.492+0000 7f0285400640 1 -- 172.21.15.42:0/4209050977 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f028014cff0 con 0x7f0280070cf0 2024-07-04T17:30:33.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.492+0000 7f0277400640 1 -- 172.21.15.42:0/4209050977 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f0270011ca0 con 0x7f0280070cf0 2024-07-04T17:30:33.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.493+0000 7f0277400640 1 --2- 172.21.15.42:0/4209050977 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f025c03d4f0 0x7f025c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:33.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.493+0000 7f0277400640 1 -- 172.21.15.42:0/4209050977 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4569+0+0 (secure 0 0 0) 0x7f0270051a60 con 0x7f0280070cf0 2024-07-04T17:30:33.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.493+0000 7f0275e00640 1 --2- 172.21.15.42:0/4209050977 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f025c03d4f0 0x7f025c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:33.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.494+0000 7f0275e00640 1 --2- 172.21.15.42:0/4209050977 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f025c03d4f0 0x7f025c03fa00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0268035000 tx=0x7f0268004730 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:33.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.497+0000 7f0277400640 1 -- 172.21.15.42:0/4209050977 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f028014cff0 con 0x7f0280070cf0 2024-07-04T17:30:33.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:33.751+0000 7f0285400640 1 -- 172.21.15.42:0/4209050977 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true} v 0) -- 0x7f0280149ea0 con 0x7f0280070cf0 2024-07-04T17:30:34.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.179+0000 7f0277400640 1 -- 172.21.15.42:0/4209050977 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true}]=0 v47) ==== 107+0+0 (secure 0 0 0) 0x7f027000a140 con 0x7f0280070cf0 2024-07-04T17:30:34.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.184+0000 7f0285400640 1 -- 172.21.15.42:0/4209050977 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f025c03d4f0 msgr2=0x7f025c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:34.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.184+0000 7f0285400640 1 --2- 172.21.15.42:0/4209050977 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f025c03d4f0 0x7f025c03fa00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0268035000 tx=0x7f0268004730 comp rx=0 tx=0).stop 2024-07-04T17:30:34.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.184+0000 7f0285400640 1 -- 172.21.15.42:0/4209050977 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0280070cf0 msgr2=0x7f028014cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:34.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.184+0000 7f0285400640 1 --2- 172.21.15.42:0/4209050977 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0280070cf0 0x7f028014cab0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f027000c320 tx=0x7f027000c7e0 comp rx=0 tx=0).stop 2024-07-04T17:30:34.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.184+0000 7f0285400640 1 -- 172.21.15.42:0/4209050977 shutdown_connections 2024-07-04T17:30:34.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.184+0000 7f0285400640 1 --2- 172.21.15.42:0/4209050977 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f025c03d4f0 0x7f025c03fa00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:34.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.184+0000 7f0285400640 1 --2- 172.21.15.42:0/4209050977 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0280070cf0 0x7f028014cab0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:34.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.184+0000 7f0285400640 1 -- 172.21.15.42:0/4209050977 >> 172.21.15.42:0/4209050977 conn(0x7f028006fe40 msgr2=0x7f0280073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:34.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.185+0000 7f0285400640 1 -- 172.21.15.42:0/4209050977 shutdown_connections 2024-07-04T17:30:34.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.185+0000 7f0285400640 1 -- 172.21.15.42:0/4209050977 wait complete. 2024-07-04T17:30:34.219 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_metadata backup_fs_metadata --yes-i-really-really-mean-it 2024-07-04T17:30:34.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.499+0000 7f7919a00640 1 -- 172.21.15.42:0/1615268877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79140b1100 msgr2=0x7f791413f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:34.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.499+0000 7f7919a00640 1 --2- 172.21.15.42:0/1615268877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79140b1100 0x7f791413f800 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f78fc009230 tx=0x7f78fc02a480 comp rx=0 tx=0).stop 2024-07-04T17:30:34.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.499+0000 7f7919a00640 1 -- 172.21.15.42:0/1615268877 shutdown_connections 2024-07-04T17:30:34.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.499+0000 7f7919a00640 1 --2- 172.21.15.42:0/1615268877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79140b1100 0x7f791413f800 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:34.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.499+0000 7f7919a00640 1 -- 172.21.15.42:0/1615268877 >> 172.21.15.42:0/1615268877 conn(0x7f791406c020 msgr2=0x7f791406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:34.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.499+0000 7f7919a00640 1 -- 172.21.15.42:0/1615268877 shutdown_connections 2024-07-04T17:30:34.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.499+0000 7f7919a00640 1 -- 172.21.15.42:0/1615268877 wait complete. 2024-07-04T17:30:34.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.501+0000 7f7919a00640 1 Processor -- start 2024-07-04T17:30:34.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.501+0000 7f7919a00640 1 -- start start 2024-07-04T17:30:34.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.502+0000 7f7919a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79140a94d0 0x7f7914134190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:34.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.502+0000 7f7919a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f79141401e0 con 0x7f79140a94d0 2024-07-04T17:30:34.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.503+0000 7f7912a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79140a94d0 0x7f7914134190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:34.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.503+0000 7f7912a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79140a94d0 0x7f7914134190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48700/0 (socket says 172.21.15.42:48700) 2024-07-04T17:30:34.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.503+0000 7f7912a00640 1 -- 172.21.15.42:0/3037240485 learned_addr learned my addr 172.21.15.42:0/3037240485 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:34.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.503+0000 7f7912a00640 1 -- 172.21.15.42:0/3037240485 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7914135770 con 0x7f79140a94d0 2024-07-04T17:30:34.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.503+0000 7f7912a00640 1 --2- 172.21.15.42:0/3037240485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79140a94d0 0x7f7914134190 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f790400ca90 tx=0x7f790400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:34.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.504+0000 7f790b400640 1 -- 172.21.15.42:0/3037240485 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7904016020 con 0x7f79140a94d0 2024-07-04T17:30:34.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.504+0000 7f790b400640 1 -- 172.21.15.42:0/3037240485 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f790400abf0 con 0x7f79140a94d0 2024-07-04T17:30:34.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.504+0000 7f790b400640 1 -- 172.21.15.42:0/3037240485 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7904012420 con 0x7f79140a94d0 2024-07-04T17:30:34.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.504+0000 7f7919a00640 1 -- 172.21.15.42:0/3037240485 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79141e23e0 con 0x7f79140a94d0 2024-07-04T17:30:34.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.504+0000 7f7919a00640 1 -- 172.21.15.42:0/3037240485 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7914135e50 con 0x7f79140a94d0 2024-07-04T17:30:34.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.505+0000 7f7919a00640 1 -- 172.21.15.42:0/3037240485 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f791413f800 con 0x7f79140a94d0 2024-07-04T17:30:34.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.505+0000 7f790b400640 1 -- 172.21.15.42:0/3037240485 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f7904019020 con 0x7f79140a94d0 2024-07-04T17:30:34.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.506+0000 7f790b400640 1 --2- 172.21.15.42:0/3037240485 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f78f003d4f0 0x7f78f003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:34.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.506+0000 7f790b400640 1 -- 172.21.15.42:0/3037240485 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4569+0+0 (secure 0 0 0) 0x7f7904052360 con 0x7f79140a94d0 2024-07-04T17:30:34.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.506+0000 7f790a000640 1 --2- 172.21.15.42:0/3037240485 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f78f003d4f0 0x7f78f003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:34.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.507+0000 7f790a000640 1 --2- 172.21.15.42:0/3037240485 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f78f003d4f0 0x7f78f003fa00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f78fc034000 tx=0x7f78fc004730 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:34.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.511+0000 7f790b400640 1 -- 172.21.15.42:0/3037240485 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f7904020020 con 0x7f79140a94d0 2024-07-04T17:30:34.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:34.722+0000 7f7919a00640 1 -- 172.21.15.42:0/3037240485 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7914069b70 con 0x7f79140a94d0 2024-07-04T17:30:35.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.187+0000 7f790b400640 1 -- 172.21.15.42:0/3037240485 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_metadata' removed v62) ==== 190+0+0 (secure 0 0 0) 0x7f79140b1510 con 0x7f79140a94d0 2024-07-04T17:30:35.190 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'backup_fs_metadata' removed 2024-07-04T17:30:35.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.191+0000 7f7919a00640 1 -- 172.21.15.42:0/3037240485 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f78f003d4f0 msgr2=0x7f78f003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:35.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.191+0000 7f7919a00640 1 --2- 172.21.15.42:0/3037240485 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f78f003d4f0 0x7f78f003fa00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f78fc034000 tx=0x7f78fc004730 comp rx=0 tx=0).stop 2024-07-04T17:30:35.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.191+0000 7f7919a00640 1 -- 172.21.15.42:0/3037240485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79140a94d0 msgr2=0x7f7914134190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:35.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.191+0000 7f7919a00640 1 --2- 172.21.15.42:0/3037240485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79140a94d0 0x7f7914134190 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f790400ca90 tx=0x7f790400cf50 comp rx=0 tx=0).stop 2024-07-04T17:30:35.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.192+0000 7f7919a00640 1 -- 172.21.15.42:0/3037240485 shutdown_connections 2024-07-04T17:30:35.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.192+0000 7f7919a00640 1 --2- 172.21.15.42:0/3037240485 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f78f003d4f0 0x7f78f003fa00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:35.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.192+0000 7f7919a00640 1 --2- 172.21.15.42:0/3037240485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79140a94d0 0x7f7914134190 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:35.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.192+0000 7f7919a00640 1 -- 172.21.15.42:0/3037240485 >> 172.21.15.42:0/3037240485 conn(0x7f791406c020 msgr2=0x7f79141d7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:35.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.192+0000 7f7919a00640 1 -- 172.21.15.42:0/3037240485 shutdown_connections 2024-07-04T17:30:35.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.192+0000 7f7919a00640 1 -- 172.21.15.42:0/3037240485 wait complete. 2024-07-04T17:30:35.231 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_data backup_fs_data --yes-i-really-really-mean-it 2024-07-04T17:30:35.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.521+0000 7f615fa00640 1 -- 172.21.15.42:0/3293604769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61580bd770 msgr2=0x7f61580bdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:35.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.521+0000 7f615fa00640 1 --2- 172.21.15.42:0/3293604769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61580bd770 0x7f61580bdb80 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f61400097b0 tx=0x7f614002a540 comp rx=0 tx=0).stop 2024-07-04T17:30:35.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.521+0000 7f615fa00640 1 -- 172.21.15.42:0/3293604769 shutdown_connections 2024-07-04T17:30:35.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.521+0000 7f615fa00640 1 --2- 172.21.15.42:0/3293604769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61580bd770 0x7f61580bdb80 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:35.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.521+0000 7f615fa00640 1 -- 172.21.15.42:0/3293604769 >> 172.21.15.42:0/3293604769 conn(0x7f615806e2f0 msgr2=0x7f615806e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:35.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.521+0000 7f615fa00640 1 -- 172.21.15.42:0/3293604769 shutdown_connections 2024-07-04T17:30:35.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.521+0000 7f615fa00640 1 -- 172.21.15.42:0/3293604769 wait complete. 2024-07-04T17:30:35.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.524+0000 7f615fa00640 1 Processor -- start 2024-07-04T17:30:35.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.524+0000 7f615fa00640 1 -- start start 2024-07-04T17:30:35.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.525+0000 7f615fa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61580bc4e0 0x7f615813cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:35.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.525+0000 7f615fa00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f61580b7810 con 0x7f61580bc4e0 2024-07-04T17:30:35.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.525+0000 7f615d000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61580bc4e0 0x7f615813cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:35.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.526+0000 7f615d000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61580bc4e0 0x7f615813cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48718/0 (socket says 172.21.15.42:48718) 2024-07-04T17:30:35.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.526+0000 7f615d000640 1 -- 172.21.15.42:0/3379404459 learned_addr learned my addr 172.21.15.42:0/3379404459 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:35.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.526+0000 7f615d000640 1 -- 172.21.15.42:0/3379404459 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f615813a030 con 0x7f61580bc4e0 2024-07-04T17:30:35.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.526+0000 7f615d000640 1 --2- 172.21.15.42:0/3379404459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61580bc4e0 0x7f615813cbc0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f614800c3a0 tx=0x7f614800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:35.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.527+0000 7f6154c00640 1 -- 172.21.15.42:0/3379404459 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61480116c0 con 0x7f61580bc4e0 2024-07-04T17:30:35.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.527+0000 7f6154c00640 1 -- 172.21.15.42:0/3379404459 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6148017070 con 0x7f61580bc4e0 2024-07-04T17:30:35.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.527+0000 7f6154c00640 1 -- 172.21.15.42:0/3379404459 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6148011ac0 con 0x7f61580bc4e0 2024-07-04T17:30:35.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.527+0000 7f615fa00640 1 -- 172.21.15.42:0/3379404459 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6158138430 con 0x7f61580bc4e0 2024-07-04T17:30:35.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.527+0000 7f615fa00640 1 -- 172.21.15.42:0/3379404459 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f615813b510 con 0x7f61580bc4e0 2024-07-04T17:30:35.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.528+0000 7f615fa00640 1 -- 172.21.15.42:0/3379404459 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61580008d0 con 0x7f61580bc4e0 2024-07-04T17:30:35.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.528+0000 7f6154c00640 1 -- 172.21.15.42:0/3379404459 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f6148011c60 con 0x7f61580bc4e0 2024-07-04T17:30:35.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.529+0000 7f6154c00640 1 --2- 172.21.15.42:0/3379404459 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f613403d4f0 0x7f613403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:35.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.529+0000 7f6154c00640 1 -- 172.21.15.42:0/3379404459 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4125+0+0 (secure 0 0 0) 0x7f6148051970 con 0x7f61580bc4e0 2024-07-04T17:30:35.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.529+0000 7f6157e00640 1 --2- 172.21.15.42:0/3379404459 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f613403d4f0 0x7f613403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:35.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.530+0000 7f6157e00640 1 --2- 172.21.15.42:0/3379404459 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f613403d4f0 0x7f613403fa00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6140034000 tx=0x7f6140004730 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:35.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.534+0000 7f6154c00640 1 -- 172.21.15.42:0/3379404459 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f6148015d50 con 0x7f61580bc4e0 2024-07-04T17:30:35.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:35.745+0000 7f615fa00640 1 -- 172.21.15.42:0/3379404459 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f615813d2e0 con 0x7f61580bc4e0 2024-07-04T17:30:36.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.194+0000 7f6154c00640 1 -- 172.21.15.42:0/3379404459 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_data' removed v63) ==== 178+0+0 (secure 0 0 0) 0x7f614801ac80 con 0x7f61580bc4e0 2024-07-04T17:30:36.196 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'backup_fs_data' removed 2024-07-04T17:30:36.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.199+0000 7f615fa00640 1 -- 172.21.15.42:0/3379404459 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f613403d4f0 msgr2=0x7f613403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:36.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.199+0000 7f615fa00640 1 --2- 172.21.15.42:0/3379404459 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f613403d4f0 0x7f613403fa00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6140034000 tx=0x7f6140004730 comp rx=0 tx=0).stop 2024-07-04T17:30:36.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.199+0000 7f615fa00640 1 -- 172.21.15.42:0/3379404459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61580bc4e0 msgr2=0x7f615813cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:36.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.199+0000 7f615fa00640 1 --2- 172.21.15.42:0/3379404459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61580bc4e0 0x7f615813cbc0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f614800c3a0 tx=0x7f614800c860 comp rx=0 tx=0).stop 2024-07-04T17:30:36.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.199+0000 7f615fa00640 1 -- 172.21.15.42:0/3379404459 shutdown_connections 2024-07-04T17:30:36.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.199+0000 7f615fa00640 1 --2- 172.21.15.42:0/3379404459 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f613403d4f0 0x7f613403fa00 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:36.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.199+0000 7f615fa00640 1 --2- 172.21.15.42:0/3379404459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61580bc4e0 0x7f615813cbc0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:36.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.199+0000 7f615fa00640 1 -- 172.21.15.42:0/3379404459 >> 172.21.15.42:0/3379404459 conn(0x7f615806e2f0 msgr2=0x7f61580b5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:36.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.201+0000 7f615fa00640 1 -- 172.21.15.42:0/3379404459 shutdown_connections 2024-07-04T17:30:36.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.202+0000 7f615fa00640 1 -- 172.21.15.42:0/3379404459 wait complete. 2024-07-04T17:30:36.243 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 22 2024-07-04T17:30:36.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.534+0000 7feebda00640 1 -- 172.21.15.42:0/1741886096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feeb806c020 msgr2=0x7feeb814a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:36.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.534+0000 7feebda00640 1 --2- 172.21.15.42:0/1741886096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feeb806c020 0x7feeb814a9e0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7feea4005fd0 tx=0x7feea402a5f0 comp rx=0 tx=0).stop 2024-07-04T17:30:36.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.534+0000 7feebda00640 1 -- 172.21.15.42:0/1741886096 shutdown_connections 2024-07-04T17:30:36.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.534+0000 7feebda00640 1 --2- 172.21.15.42:0/1741886096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feeb806c020 0x7feeb814a9e0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:36.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.534+0000 7feebda00640 1 -- 172.21.15.42:0/1741886096 >> 172.21.15.42:0/1741886096 conn(0x7feeb80bebc0 msgr2=0x7feeb80befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:36.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.534+0000 7feebda00640 1 -- 172.21.15.42:0/1741886096 shutdown_connections 2024-07-04T17:30:36.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.534+0000 7feebda00640 1 -- 172.21.15.42:0/1741886096 wait complete. 2024-07-04T17:30:36.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.537+0000 7feebda00640 1 Processor -- start 2024-07-04T17:30:36.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.537+0000 7feebda00640 1 -- start start 2024-07-04T17:30:36.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.538+0000 7feebda00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feeb806c020 0x7feeb81da210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:36.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.538+0000 7feebda00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7feeb814b590 con 0x7feeb806c020 2024-07-04T17:30:36.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.538+0000 7feeb7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feeb806c020 0x7feeb81da210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:36.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.538+0000 7feeb7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feeb806c020 0x7feeb81da210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48724/0 (socket says 172.21.15.42:48724) 2024-07-04T17:30:36.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.538+0000 7feeb7e00640 1 -- 172.21.15.42:0/2709043423 learned_addr learned my addr 172.21.15.42:0/2709043423 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:36.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.539+0000 7feeb7e00640 1 -- 172.21.15.42:0/2709043423 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb8142830 con 0x7feeb806c020 2024-07-04T17:30:36.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.539+0000 7feeb7e00640 1 --2- 172.21.15.42:0/2709043423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feeb806c020 0x7feeb81da210 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7feea800c340 tx=0x7feea800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:36.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.539+0000 7feeafe00640 1 -- 172.21.15.42:0/2709043423 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feea8005410 con 0x7feeb806c020 2024-07-04T17:30:36.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.539+0000 7feeafe00640 1 -- 172.21.15.42:0/2709043423 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feea8007030 con 0x7feeb806c020 2024-07-04T17:30:36.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.539+0000 7feeafe00640 1 -- 172.21.15.42:0/2709043423 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feea80073f0 con 0x7feeb806c020 2024-07-04T17:30:36.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.539+0000 7feebda00640 1 -- 172.21.15.42:0/2709043423 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeb8141780 con 0x7feeb806c020 2024-07-04T17:30:36.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.540+0000 7feebda00640 1 -- 172.21.15.42:0/2709043423 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeb8142090 con 0x7feeb806c020 2024-07-04T17:30:36.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.541+0000 7feebda00640 1 -- 172.21.15.42:0/2709043423 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feeb8142d10 con 0x7feeb806c020 2024-07-04T17:30:36.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.541+0000 7feeafe00640 1 -- 172.21.15.42:0/2709043423 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7feea801a020 con 0x7feeb806c020 2024-07-04T17:30:36.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.541+0000 7feeafe00640 1 --2- 172.21.15.42:0/2709043423 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fee9803d4f0 0x7fee9803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:36.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.541+0000 7feeafe00640 1 -- 172.21.15.42:0/2709043423 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 3705+0+0 (secure 0 0 0) 0x7feea8051130 con 0x7feeb806c020 2024-07-04T17:30:36.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.542+0000 7feeb7400640 1 --2- 172.21.15.42:0/2709043423 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fee9803d4f0 0x7fee9803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:36.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.543+0000 7feeb7400640 1 --2- 172.21.15.42:0/2709043423 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fee9803d4f0 0x7fee9803fa00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7feea4033000 tx=0x7feea4004730 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:36.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.547+0000 7feeafe00640 1 -- 172.21.15.42:0/2709043423 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7feea8007590 con 0x7feeb806c020 2024-07-04T17:30:36.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.759+0000 7feebda00640 1 -- 172.21.15.42:0/2709043423 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config reset", "num": 22} v 0) -- 0x7feeb81419b0 con 0x7feeb806c020 2024-07-04T17:30:36.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.762+0000 7feeafe00640 1 -- 172.21.15.42:0/2709043423 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 22}]=0 v26) ==== 71+0+0 (secure 0 0 0) 0x7feea800dce0 con 0x7feeb806c020 2024-07-04T17:30:36.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.765+0000 7feebda00640 1 -- 172.21.15.42:0/2709043423 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fee9803d4f0 msgr2=0x7fee9803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:36.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.765+0000 7feebda00640 1 --2- 172.21.15.42:0/2709043423 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fee9803d4f0 0x7fee9803fa00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7feea4033000 tx=0x7feea4004730 comp rx=0 tx=0).stop 2024-07-04T17:30:36.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.765+0000 7feebda00640 1 -- 172.21.15.42:0/2709043423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feeb806c020 msgr2=0x7feeb81da210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:36.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.765+0000 7feebda00640 1 --2- 172.21.15.42:0/2709043423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feeb806c020 0x7feeb81da210 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7feea800c340 tx=0x7feea800c800 comp rx=0 tx=0).stop 2024-07-04T17:30:36.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.765+0000 7feebda00640 1 -- 172.21.15.42:0/2709043423 shutdown_connections 2024-07-04T17:30:36.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.765+0000 7feebda00640 1 --2- 172.21.15.42:0/2709043423 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fee9803d4f0 0x7fee9803fa00 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:36.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.765+0000 7feebda00640 1 --2- 172.21.15.42:0/2709043423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feeb806c020 0x7feeb81da210 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:36.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.765+0000 7feebda00640 1 -- 172.21.15.42:0/2709043423 >> 172.21.15.42:0/2709043423 conn(0x7feeb80bebc0 msgr2=0x7feeb814c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:36.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.766+0000 7feebda00640 1 -- 172.21.15.42:0/2709043423 shutdown_connections 2024-07-04T17:30:36.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:36.766+0000 7feebda00640 1 -- 172.21.15.42:0/2709043423 wait complete. 2024-07-04T17:30:36.795 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_mirroring.TestMirroring.test_add_directory_path_normalization' 2024-07-04T17:30:37.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:37.090+0000 7f0aa4e00640 1 -- 172.21.15.42:0/3104260774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0aa006cea0 msgr2=0x7f0aa00b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:37.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:37.090+0000 7f0aa4e00640 1 --2- 172.21.15.42:0/3104260774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0aa006cea0 0x7f0aa00b3030 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f0a90005fd0 tx=0x7f0a9002ae80 comp rx=0 tx=0).stop 2024-07-04T17:30:37.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:37.090+0000 7f0aa4e00640 1 -- 172.21.15.42:0/3104260774 shutdown_connections 2024-07-04T17:30:37.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:37.090+0000 7f0aa4e00640 1 --2- 172.21.15.42:0/3104260774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0aa006cea0 0x7f0aa00b3030 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:37.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:37.090+0000 7f0aa4e00640 1 -- 172.21.15.42:0/3104260774 >> 172.21.15.42:0/3104260774 conn(0x7f0aa00bf700 msgr2=0x7f0aa00af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:37.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:37.090+0000 7f0aa4e00640 1 -- 172.21.15.42:0/3104260774 shutdown_connections 2024-07-04T17:30:37.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:37.090+0000 7f0aa4e00640 1 -- 172.21.15.42:0/3104260774 wait complete. 2024-07-04T17:30:37.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:37.092+0000 7f0aa4e00640 1 Processor -- start 2024-07-04T17:30:37.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:37.092+0000 7f0aa4e00640 1 -- start start 2024-07-04T17:30:37.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:37.093+0000 7f0aa4e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0aa006cea0 0x7f0aa01d72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:37.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:37.093+0000 7f0aa4e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa006ef50 con 0x7f0aa006cea0 2024-07-04T17:30:37.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:37.094+0000 7f0a9de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0aa006cea0 0x7f0aa01d72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:37.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:37.094+0000 7f0a9de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0aa006cea0 0x7f0aa01d72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48736/0 (socket says 172.21.15.42:48736) 2024-07-04T17:30:37.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:37.094+0000 7f0a9de00640 1 -- 172.21.15.42:0/2252403824 learned_addr learned my addr 172.21.15.42:0/2252403824 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:37.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:37.094+0000 7f0a9de00640 1 -- 172.21.15.42:0/2252403824 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa00ae3e0 con 0x7f0aa006cea0 2024-07-04T17:30:37.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:37.094+0000 7f0a9de00640 1 --2- 172.21.15.42:0/2252403824 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0aa006cea0 0x7f0aa01d72e0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f0a9000b3c0 tx=0x7f0a9002cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:37.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:37.095+0000 7f0a96000640 1 -- 172.21.15.42:0/2252403824 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a90041020 con 0x7f0aa006cea0 2024-07-04T17:30:37.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:37.095+0000 7f0a96000640 1 -- 172.21.15.42:0/2252403824 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a900095a0 con 0x7f0aa006cea0 2024-07-04T17:30:37.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:37.095+0000 7f0a96000640 1 -- 172.21.15.42:0/2252403824 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a900098c0 con 0x7f0aa006cea0 2024-07-04T17:30:37.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:37.095+0000 7f0aa4e00640 1 -- 172.21.15.42:0/2252403824 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aa0071890 con 0x7f0aa006cea0 2024-07-04T17:30:37.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:37.095+0000 7f0aa4e00640 1 -- 172.21.15.42:0/2252403824 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aa0071f10 con 0x7f0aa006cea0 2024-07-04T17:30:37.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:37.096+0000 7f0aa4e00640 1 -- 172.21.15.42:0/2252403824 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aa00b3030 con 0x7f0aa006cea0 2024-07-04T17:30:37.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:37.096+0000 7f0a96000640 1 -- 172.21.15.42:0/2252403824 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f0a90044020 con 0x7f0aa006cea0 2024-07-04T17:30:37.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:37.097+0000 7f0a96000640 1 --2- 172.21.15.42:0/2252403824 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f0a8003d4a0 0x7f0a8003f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:37.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:37.097+0000 7f0a96000640 1 -- 172.21.15.42:0/2252403824 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 3705+0+0 (secure 0 0 0) 0x7f0a90039040 con 0x7f0aa006cea0 2024-07-04T17:30:37.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:37.097+0000 7f0a9d400640 1 --2- 172.21.15.42:0/2252403824 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f0a8003d4a0 0x7f0a8003f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:37.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:37.098+0000 7f0a9d400640 1 --2- 172.21.15.42:0/2252403824 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f0a8003d4a0 0x7f0a8003f9b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c001c60 tx=0x7f0a8c007d70 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:37.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:37.102+0000 7f0a96000640 1 -- 172.21.15.42:0/2252403824 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f0a9004b030 con 0x7f0aa006cea0 2024-07-04T17:30:37.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:37.389+0000 7f0aa4e00640 1 -- 172.21.15.42:0/2252403824 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_mirroring.TestMirroring.test_add_directory_path_normalization"]} v 0) -- 0x7f0aa0071a60 con 0x7f0aa006cea0 2024-07-04T17:30:38.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.202+0000 7f0a96000640 1 -- 172.21.15.42:0/2252403824 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_mirroring.TestMirroring.test_add_directory_path_normalization"]}]=0 v233) ==== 158+0+0 (secure 0 0 0) 0x7f0aa00ae3e0 con 0x7f0aa006cea0 2024-07-04T17:30:38.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.206+0000 7f0aa4e00640 1 -- 172.21.15.42:0/2252403824 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f0a8003d4a0 msgr2=0x7f0a8003f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:38.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.206+0000 7f0aa4e00640 1 --2- 172.21.15.42:0/2252403824 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f0a8003d4a0 0x7f0a8003f9b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c001c60 tx=0x7f0a8c007d70 comp rx=0 tx=0).stop 2024-07-04T17:30:38.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.206+0000 7f0aa4e00640 1 -- 172.21.15.42:0/2252403824 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0aa006cea0 msgr2=0x7f0aa01d72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:38.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.206+0000 7f0aa4e00640 1 --2- 172.21.15.42:0/2252403824 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0aa006cea0 0x7f0aa01d72e0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f0a9000b3c0 tx=0x7f0a9002cd70 comp rx=0 tx=0).stop 2024-07-04T17:30:38.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.206+0000 7f0aa4e00640 1 -- 172.21.15.42:0/2252403824 shutdown_connections 2024-07-04T17:30:38.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.206+0000 7f0aa4e00640 1 --2- 172.21.15.42:0/2252403824 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f0a8003d4a0 0x7f0a8003f9b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:38.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.206+0000 7f0aa4e00640 1 --2- 172.21.15.42:0/2252403824 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0aa006cea0 0x7f0aa01d72e0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:38.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.206+0000 7f0aa4e00640 1 -- 172.21.15.42:0/2252403824 >> 172.21.15.42:0/2252403824 conn(0x7f0aa00bf700 msgr2=0x7f0aa00aa3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:38.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.206+0000 7f0aa4e00640 1 -- 172.21.15.42:0/2252403824 shutdown_connections 2024-07-04T17:30:38.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.207+0000 7f0aa4e00640 1 -- 172.21.15.42:0/2252403824 wait complete. 2024-07-04T17:30:38.240 INFO:tasks.cephfs_test_runner:test_add_directory_path_normalization (tasks.cephfs.test_mirroring.TestMirroring) ... ok 2024-07-04T17:30:38.240 INFO:tasks.cephfs_test_runner:Starting test: test_add_directory_with_mirroring_disabled (tasks.cephfs.test_mirroring.TestMirroring) 2024-07-04T17:30:38.241 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:30:38.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.530+0000 7ff672600640 1 -- 172.21.15.42:0/1495914493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff66c06cea0 msgr2=0x7ff66c0b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:38.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.530+0000 7ff672600640 1 --2- 172.21.15.42:0/1495914493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff66c06cea0 0x7ff66c0b3030 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7ff65c005fd0 tx=0x7ff65c02b930 comp rx=0 tx=0).stop 2024-07-04T17:30:38.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.530+0000 7ff672600640 1 -- 172.21.15.42:0/1495914493 shutdown_connections 2024-07-04T17:30:38.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.530+0000 7ff672600640 1 --2- 172.21.15.42:0/1495914493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff66c06cea0 0x7ff66c0b3030 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:38.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.530+0000 7ff672600640 1 -- 172.21.15.42:0/1495914493 >> 172.21.15.42:0/1495914493 conn(0x7ff66c0bf700 msgr2=0x7ff66c0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:38.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.530+0000 7ff672600640 1 -- 172.21.15.42:0/1495914493 shutdown_connections 2024-07-04T17:30:38.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.530+0000 7ff672600640 1 -- 172.21.15.42:0/1495914493 wait complete. 2024-07-04T17:30:38.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.532+0000 7ff672600640 1 Processor -- start 2024-07-04T17:30:38.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.533+0000 7ff672600640 1 -- start start 2024-07-04T17:30:38.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.534+0000 7ff672600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff66c06cea0 0x7ff66c0733e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:38.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.534+0000 7ff672600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c0b0370 con 0x7ff66c06cea0 2024-07-04T17:30:38.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.534+0000 7ff66b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff66c06cea0 0x7ff66c0733e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:38.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.534+0000 7ff66b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff66c06cea0 0x7ff66c0733e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48762/0 (socket says 172.21.15.42:48762) 2024-07-04T17:30:38.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.534+0000 7ff66b400640 1 -- 172.21.15.42:0/4087588141 learned_addr learned my addr 172.21.15.42:0/4087588141 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:38.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.535+0000 7ff66b400640 1 -- 172.21.15.42:0/4087588141 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66c1d8440 con 0x7ff66c06cea0 2024-07-04T17:30:38.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.535+0000 7ff66b400640 1 --2- 172.21.15.42:0/4087588141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff66c06cea0 0x7ff66c0733e0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7ff65400c320 tx=0x7ff65400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:38.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.535+0000 7ff663400640 1 -- 172.21.15.42:0/4087588141 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff654017070 con 0x7ff66c06cea0 2024-07-04T17:30:38.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.535+0000 7ff663400640 1 -- 172.21.15.42:0/4087588141 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff654005310 con 0x7ff66c06cea0 2024-07-04T17:30:38.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.535+0000 7ff672600640 1 -- 172.21.15.42:0/4087588141 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff66c1dcd50 con 0x7ff66c06cea0 2024-07-04T17:30:38.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.535+0000 7ff663400640 1 -- 172.21.15.42:0/4087588141 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6540035f0 con 0x7ff66c06cea0 2024-07-04T17:30:38.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.535+0000 7ff672600640 1 -- 172.21.15.42:0/4087588141 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff66c1d8060 con 0x7ff66c06cea0 2024-07-04T17:30:38.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.537+0000 7ff672600640 1 -- 172.21.15.42:0/4087588141 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff66c0b3030 con 0x7ff66c06cea0 2024-07-04T17:30:38.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.537+0000 7ff663400640 1 -- 172.21.15.42:0/4087588141 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7ff65401a070 con 0x7ff66c06cea0 2024-07-04T17:30:38.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.537+0000 7ff663400640 1 --2- 172.21.15.42:0/4087588141 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ff64803d4f0 0x7ff64803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:38.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.537+0000 7ff663400640 1 -- 172.21.15.42:0/4087588141 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 3705+0+0 (secure 0 0 0) 0x7ff654050c70 con 0x7ff66c06cea0 2024-07-04T17:30:38.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.538+0000 7ff66aa00640 1 --2- 172.21.15.42:0/4087588141 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ff64803d4f0 0x7ff64803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:38.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.539+0000 7ff66aa00640 1 --2- 172.21.15.42:0/4087588141 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ff64803d4f0 0x7ff64803fa00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff65c035000 tx=0x7ff65c004730 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:38.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.543+0000 7ff663400640 1 -- 172.21.15.42:0/4087588141 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7ff65401bc60 con 0x7ff66c06cea0 2024-07-04T17:30:38.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.755+0000 7ff672600640 1 -- 172.21.15.42:0/4087588141 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff66c0af050 con 0x7ff66c06cea0 2024-07-04T17:30:38.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.756+0000 7ff663400640 1 -- 172.21.15.42:0/4087588141 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+8397 (secure 0 0 0) 0x7ff65401e0d0 con 0x7ff66c06cea0 2024-07-04T17:30:38.757 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:30:38.757 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":63,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:30:36.189606+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6833/1072723036":"2024-07-05T17:30:33.176670+0000","172.21.15.42:6832/1072723036":"2024-07-05T17:30:33.176670+0000","172.21.15.42:0/934943970":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/4215342546":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/3491363662":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/3597724339":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/1322461005":"2024-07-05T17:30:25.586824+0000","172.21.15.42:6825/3045931584":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/461490182":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/7072365":"2024-07-04T18:30:09.692358+0000","172.21.15.42:0/176559669":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/1810008304":"2024-07-05T17:29:38.572697+0000","172.21.15.42:6834/1327681666":"2024-07-05T17:30:26.507800+0000","172.21.15.42:0/1534342117":"2024-07-05T17:29:38.572697+0000","172.21.15.42:6835/1327681666":"2024-07-05T17:30:26.507800+0000","172.21.15.42:6825/2638271195":"2024-07-05T17:29:38.572697+0000","172.21.15.42:0/2056846029":"2024-07-05T17:30:25.586824+0000","172.21.15.42:6824/2638271195":"2024-07-05T17:29:38.572697+0000","172.21.15.42:0/591942290":"2024-07-05T17:29:38.572697+0000","172.21.15.42:6824/3045931584":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/2926090870":"2024-07-05T17:29:38.572697+0000","172.21.15.42:0/136286797":"2024-07-05T17:29:38.572697+0000","172.21.15.42:0/1793407260":"2024-07-05T17:30:25.586824+0000","172.21.15.42:0/2800592161":"2024-07-05T17:29:38.572697+0000","172.21.15.42:0/1891351459":"2024-07-05T17:29:38.572697+0000","172.21.15.42:0/3114375174":"2024-07-05T17:29:38.572697+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:30:38.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.759+0000 7ff672600640 1 -- 172.21.15.42:0/4087588141 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ff64803d4f0 msgr2=0x7ff64803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:38.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.759+0000 7ff672600640 1 --2- 172.21.15.42:0/4087588141 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ff64803d4f0 0x7ff64803fa00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff65c035000 tx=0x7ff65c004730 comp rx=0 tx=0).stop 2024-07-04T17:30:38.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.759+0000 7ff672600640 1 -- 172.21.15.42:0/4087588141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff66c06cea0 msgr2=0x7ff66c0733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:38.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.759+0000 7ff672600640 1 --2- 172.21.15.42:0/4087588141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff66c06cea0 0x7ff66c0733e0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7ff65400c320 tx=0x7ff65400c7e0 comp rx=0 tx=0).stop 2024-07-04T17:30:38.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.759+0000 7ff672600640 1 -- 172.21.15.42:0/4087588141 shutdown_connections 2024-07-04T17:30:38.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.759+0000 7ff672600640 1 --2- 172.21.15.42:0/4087588141 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ff64803d4f0 0x7ff64803fa00 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:38.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.759+0000 7ff672600640 1 --2- 172.21.15.42:0/4087588141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff66c06cea0 0x7ff66c0733e0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:38.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.759+0000 7ff672600640 1 -- 172.21.15.42:0/4087588141 >> 172.21.15.42:0/4087588141 conn(0x7ff66c0bf700 msgr2=0x7ff66c14a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:38.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.759+0000 7ff672600640 1 -- 172.21.15.42:0/4087588141 shutdown_connections 2024-07-04T17:30:38.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:38.759+0000 7ff672600640 1 -- 172.21.15.42:0/4087588141 wait complete. 2024-07-04T17:30:38.787 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-04T17:26:03.297630+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-04T17:30:38.788 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-04T17:30:39.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.085+0000 7f02b7e00640 1 -- 172.21.15.42:0/1278529783 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02b80b1ac0 msgr2=0x7f02b80b2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:39.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.085+0000 7f02b7e00640 1 --2- 172.21.15.42:0/1278529783 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02b80b1ac0 0x7f02b80b2c80 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f02b80b34b0 tx=0x7f02a802ae80 comp rx=0 tx=0).stop 2024-07-04T17:30:39.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.085+0000 7f02b7e00640 1 -- 172.21.15.42:0/1278529783 shutdown_connections 2024-07-04T17:30:39.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.085+0000 7f02b7e00640 1 --2- 172.21.15.42:0/1278529783 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02b80b1ac0 0x7f02b80b2c80 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:39.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.085+0000 7f02b7e00640 1 -- 172.21.15.42:0/1278529783 >> 172.21.15.42:0/1278529783 conn(0x7f02b80bf260 msgr2=0x7f02b80af5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:39.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.085+0000 7f02b7e00640 1 -- 172.21.15.42:0/1278529783 shutdown_connections 2024-07-04T17:30:39.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.085+0000 7f02b7e00640 1 -- 172.21.15.42:0/1278529783 wait complete. 2024-07-04T17:30:39.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.088+0000 7f02b7e00640 1 Processor -- start 2024-07-04T17:30:39.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.088+0000 7f02b7e00640 1 -- start start 2024-07-04T17:30:39.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.090+0000 7f02b7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02b80afeb0 0x7f02b80b7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:39.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.090+0000 7f02b7e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f02b814c7a0 con 0x7f02b80afeb0 2024-07-04T17:30:39.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.090+0000 7f02b6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02b80afeb0 0x7f02b80b7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:39.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.090+0000 7f02b6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02b80afeb0 0x7f02b80b7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48780/0 (socket says 172.21.15.42:48780) 2024-07-04T17:30:39.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.090+0000 7f02b6a00640 1 -- 172.21.15.42:0/1944166257 learned_addr learned my addr 172.21.15.42:0/1944166257 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:39.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.091+0000 7f02b6a00640 1 -- 172.21.15.42:0/1944166257 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b80b79d0 con 0x7f02b80afeb0 2024-07-04T17:30:39.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.091+0000 7f02b6a00640 1 --2- 172.21.15.42:0/1944166257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02b80afeb0 0x7f02b80b7490 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f02a800b4b0 tx=0x7f02a802cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:39.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.091+0000 7f02aea00640 1 -- 172.21.15.42:0/1944166257 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02a8041020 con 0x7f02b80afeb0 2024-07-04T17:30:39.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.092+0000 7f02aea00640 1 -- 172.21.15.42:0/1944166257 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02a80095c0 con 0x7f02b80afeb0 2024-07-04T17:30:39.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.092+0000 7f02aea00640 1 -- 172.21.15.42:0/1944166257 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02a8009940 con 0x7f02b80afeb0 2024-07-04T17:30:39.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.092+0000 7f02b7e00640 1 -- 172.21.15.42:0/1944166257 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02b80b7ca0 con 0x7f02b80afeb0 2024-07-04T17:30:39.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.092+0000 7f02b7e00640 1 -- 172.21.15.42:0/1944166257 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02b8147a00 con 0x7f02b80afeb0 2024-07-04T17:30:39.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.093+0000 7f02b7e00640 1 -- 172.21.15.42:0/1944166257 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02b81493f0 con 0x7f02b80afeb0 2024-07-04T17:30:39.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.093+0000 7f02aea00640 1 -- 172.21.15.42:0/1944166257 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f02a8044020 con 0x7f02b80afeb0 2024-07-04T17:30:39.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.094+0000 7f02aea00640 1 --2- 172.21.15.42:0/1944166257 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f029403d690 0x7f029403fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:39.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.094+0000 7f02aea00640 1 -- 172.21.15.42:0/1944166257 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 3705+0+0 (secure 0 0 0) 0x7f02a8039040 con 0x7f02b80afeb0 2024-07-04T17:30:39.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.094+0000 7f02afe00640 1 --2- 172.21.15.42:0/1944166257 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f029403d690 0x7f029403fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:39.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.095+0000 7f02afe00640 1 --2- 172.21.15.42:0/1944166257 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f029403d690 0x7f029403fba0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f02b80b9490 tx=0x7f02a00079f0 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:39.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.099+0000 7f02aea00640 1 -- 172.21.15.42:0/1944166257 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f02b81493f0 con 0x7f02b80afeb0 2024-07-04T17:30:39.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.309+0000 7f02b7e00640 1 -- 172.21.15.42:0/1944166257 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f02b8147c30 con 0x7f02b80afeb0 2024-07-04T17:30:39.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.310+0000 7f02aea00640 1 -- 172.21.15.42:0/1944166257 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) ==== 93+0+10 (secure 0 0 0) 0x7f02a804b030 con 0x7f02b80afeb0 2024-07-04T17:30:39.311 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-07-04T17:30:39.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.313+0000 7f02b7e00640 1 -- 172.21.15.42:0/1944166257 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f029403d690 msgr2=0x7f029403fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:39.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.313+0000 7f02b7e00640 1 --2- 172.21.15.42:0/1944166257 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f029403d690 0x7f029403fba0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f02b80b9490 tx=0x7f02a00079f0 comp rx=0 tx=0).stop 2024-07-04T17:30:39.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.313+0000 7f02b7e00640 1 -- 172.21.15.42:0/1944166257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02b80afeb0 msgr2=0x7f02b80b7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:39.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.313+0000 7f02b7e00640 1 --2- 172.21.15.42:0/1944166257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02b80afeb0 0x7f02b80b7490 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f02a800b4b0 tx=0x7f02a802cd70 comp rx=0 tx=0).stop 2024-07-04T17:30:39.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.313+0000 7f02b7e00640 1 -- 172.21.15.42:0/1944166257 shutdown_connections 2024-07-04T17:30:39.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.313+0000 7f02b7e00640 1 --2- 172.21.15.42:0/1944166257 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f029403d690 0x7f029403fba0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:39.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.313+0000 7f02b7e00640 1 --2- 172.21.15.42:0/1944166257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02b80afeb0 0x7f02b80b7490 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:39.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.313+0000 7f02b7e00640 1 -- 172.21.15.42:0/1944166257 >> 172.21.15.42:0/1944166257 conn(0x7f02b80bf260 msgr2=0x7f02b80be010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:39.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.314+0000 7f02b7e00640 1 -- 172.21.15.42:0/1944166257 shutdown_connections 2024-07-04T17:30:39.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.314+0000 7f02b7e00640 1 -- 172.21.15.42:0/1944166257 wait complete. 2024-07-04T17:30:39.341 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_mirroring.TestMirroring.test_add_directory_with_mirroring_disabled' 2024-07-04T17:30:39.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.642+0000 7fe657e00640 1 -- 172.21.15.42:0/3483925955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe65006cea0 msgr2=0x7fe6500b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:39.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.642+0000 7fe657e00640 1 --2- 172.21.15.42:0/3483925955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe65006cea0 0x7fe6500b3030 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fe63c0097b0 tx=0x7fe63c02a540 comp rx=0 tx=0).stop 2024-07-04T17:30:39.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.642+0000 7fe657e00640 1 -- 172.21.15.42:0/3483925955 shutdown_connections 2024-07-04T17:30:39.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.642+0000 7fe657e00640 1 --2- 172.21.15.42:0/3483925955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe65006cea0 0x7fe6500b3030 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:39.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.642+0000 7fe657e00640 1 -- 172.21.15.42:0/3483925955 >> 172.21.15.42:0/3483925955 conn(0x7fe6500bf700 msgr2=0x7fe6500af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:39.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.642+0000 7fe657e00640 1 -- 172.21.15.42:0/3483925955 shutdown_connections 2024-07-04T17:30:39.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.642+0000 7fe657e00640 1 -- 172.21.15.42:0/3483925955 wait complete. 2024-07-04T17:30:39.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.645+0000 7fe657e00640 1 Processor -- start 2024-07-04T17:30:39.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.645+0000 7fe657e00640 1 -- start start 2024-07-04T17:30:39.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.646+0000 7fe657e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe65006cea0 0x7fe650070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:39.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.646+0000 7fe657e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe6500b0580 con 0x7fe65006cea0 2024-07-04T17:30:39.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.646+0000 7fe655400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe65006cea0 0x7fe650070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:39.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.646+0000 7fe655400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe65006cea0 0x7fe650070560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48792/0 (socket says 172.21.15.42:48792) 2024-07-04T17:30:39.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.646+0000 7fe655400640 1 -- 172.21.15.42:0/2430447873 learned_addr learned my addr 172.21.15.42:0/2430447873 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:39.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.647+0000 7fe655400640 1 -- 172.21.15.42:0/2430447873 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe650071ad0 con 0x7fe65006cea0 2024-07-04T17:30:39.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.647+0000 7fe655400640 1 --2- 172.21.15.42:0/2430447873 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe65006cea0 0x7fe650070560 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fe64000c3b0 tx=0x7fe64000c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:39.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.647+0000 7fe64d600640 1 -- 172.21.15.42:0/2430447873 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe640011780 con 0x7fe65006cea0 2024-07-04T17:30:39.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.647+0000 7fe64d600640 1 -- 172.21.15.42:0/2430447873 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe640017070 con 0x7fe65006cea0 2024-07-04T17:30:39.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.647+0000 7fe64d600640 1 -- 172.21.15.42:0/2430447873 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe640011c00 con 0x7fe65006cea0 2024-07-04T17:30:39.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.647+0000 7fe657e00640 1 -- 172.21.15.42:0/2430447873 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6500acd60 con 0x7fe65006cea0 2024-07-04T17:30:39.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.648+0000 7fe657e00640 1 -- 172.21.15.42:0/2430447873 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe650072170 con 0x7fe65006cea0 2024-07-04T17:30:39.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.649+0000 7fe657e00640 1 -- 172.21.15.42:0/2430447873 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe650063f70 con 0x7fe65006cea0 2024-07-04T17:30:39.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.649+0000 7fe64d600640 1 -- 172.21.15.42:0/2430447873 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7fe64001c460 con 0x7fe65006cea0 2024-07-04T17:30:39.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.650+0000 7fe64d600640 1 --2- 172.21.15.42:0/2430447873 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fe63003d4f0 0x7fe63003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:39.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.650+0000 7fe64d600640 1 -- 172.21.15.42:0/2430447873 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 3705+0+0 (secure 0 0 0) 0x7fe640051710 con 0x7fe65006cea0 2024-07-04T17:30:39.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.650+0000 7fe654a00640 1 --2- 172.21.15.42:0/2430447873 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fe63003d4f0 0x7fe63003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:39.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.651+0000 7fe654a00640 1 --2- 172.21.15.42:0/2430447873 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fe63003d4f0 0x7fe63003fa00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe63c034000 tx=0x7fe63c004730 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:39.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.655+0000 7fe64d600640 1 -- 172.21.15.42:0/2430447873 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fe650063f70 con 0x7fe65006cea0 2024-07-04T17:30:39.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:39.956+0000 7fe657e00640 1 -- 172.21.15.42:0/2430447873 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_mirroring.TestMirroring.test_add_directory_with_mirroring_disabled"]} v 0) -- 0x7fe6500b3030 con 0x7fe65006cea0 2024-07-04T17:30:40.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.210+0000 7fe64d600640 1 -- 172.21.15.42:0/2430447873 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_mirroring.TestMirroring.test_add_directory_with_mirroring_disabled"]}]=0 v235) ==== 166+0+0 (secure 0 0 0) 0x7fe640010c60 con 0x7fe65006cea0 2024-07-04T17:30:40.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.214+0000 7fe657e00640 1 -- 172.21.15.42:0/2430447873 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fe63003d4f0 msgr2=0x7fe63003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:40.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.214+0000 7fe657e00640 1 --2- 172.21.15.42:0/2430447873 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fe63003d4f0 0x7fe63003fa00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe63c034000 tx=0x7fe63c004730 comp rx=0 tx=0).stop 2024-07-04T17:30:40.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.214+0000 7fe657e00640 1 -- 172.21.15.42:0/2430447873 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe65006cea0 msgr2=0x7fe650070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:40.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.214+0000 7fe657e00640 1 --2- 172.21.15.42:0/2430447873 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe65006cea0 0x7fe650070560 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fe64000c3b0 tx=0x7fe64000c870 comp rx=0 tx=0).stop 2024-07-04T17:30:40.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.214+0000 7fe657e00640 1 -- 172.21.15.42:0/2430447873 shutdown_connections 2024-07-04T17:30:40.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.214+0000 7fe657e00640 1 --2- 172.21.15.42:0/2430447873 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fe63003d4f0 0x7fe63003fa00 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:40.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.214+0000 7fe657e00640 1 --2- 172.21.15.42:0/2430447873 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe65006cea0 0x7fe650070560 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:40.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.214+0000 7fe657e00640 1 -- 172.21.15.42:0/2430447873 >> 172.21.15.42:0/2430447873 conn(0x7fe6500bf700 msgr2=0x7fe65014b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:40.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.214+0000 7fe657e00640 1 -- 172.21.15.42:0/2430447873 shutdown_connections 2024-07-04T17:30:40.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.214+0000 7fe657e00640 1 -- 172.21.15.42:0/2430447873 wait complete. 2024-07-04T17:30:40.244 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-07-04T17:30:40.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.537+0000 7fe49ac00640 1 -- 172.21.15.42:0/311569520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4940bad70 msgr2=0x7fe49406c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:40.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.537+0000 7fe49ac00640 1 --2- 172.21.15.42:0/311569520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4940bad70 0x7fe49406c2c0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fe488009fd0 tx=0x7fe48802b0b0 comp rx=0 tx=0).stop 2024-07-04T17:30:40.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.537+0000 7fe49ac00640 1 -- 172.21.15.42:0/311569520 shutdown_connections 2024-07-04T17:30:40.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.537+0000 7fe49ac00640 1 --2- 172.21.15.42:0/311569520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4940bad70 0x7fe49406c2c0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:40.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.537+0000 7fe49ac00640 1 -- 172.21.15.42:0/311569520 >> 172.21.15.42:0/311569520 conn(0x7fe4940bebc0 msgr2=0x7fe4940befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:40.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.538+0000 7fe49ac00640 1 -- 172.21.15.42:0/311569520 shutdown_connections 2024-07-04T17:30:40.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.538+0000 7fe49ac00640 1 -- 172.21.15.42:0/311569520 wait complete. 2024-07-04T17:30:40.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.540+0000 7fe49ac00640 1 Processor -- start 2024-07-04T17:30:40.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.540+0000 7fe49ac00640 1 -- start start 2024-07-04T17:30:40.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.541+0000 7fe49ac00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4940b5650 0x7fe4941d4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:40.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.541+0000 7fe49ac00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe4940b27c0 con 0x7fe4940b5650 2024-07-04T17:30:40.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.542+0000 7fe499800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4940b5650 0x7fe4941d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:40.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.542+0000 7fe499800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4940b5650 0x7fe4941d4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48812/0 (socket says 172.21.15.42:48812) 2024-07-04T17:30:40.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.542+0000 7fe499800640 1 -- 172.21.15.42:0/4079077263 learned_addr learned my addr 172.21.15.42:0/4079077263 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:40.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.542+0000 7fe499800640 1 -- 172.21.15.42:0/4079077263 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4941d4c90 con 0x7fe4940b5650 2024-07-04T17:30:40.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.542+0000 7fe499800640 1 --2- 172.21.15.42:0/4079077263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4940b5650 0x7fe4941d4660 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fe48800b310 tx=0x7fe488009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:40.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.543+0000 7fe491600640 1 -- 172.21.15.42:0/4079077263 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe488003aa0 con 0x7fe4940b5650 2024-07-04T17:30:40.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.543+0000 7fe491600640 1 -- 172.21.15.42:0/4079077263 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe488003d30 con 0x7fe4940b5650 2024-07-04T17:30:40.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.543+0000 7fe49ac00640 1 -- 172.21.15.42:0/4079077263 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4941d39b0 con 0x7fe4940b5650 2024-07-04T17:30:40.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.543+0000 7fe491600640 1 -- 172.21.15.42:0/4079077263 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe488033970 con 0x7fe4940b5650 2024-07-04T17:30:40.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.543+0000 7fe49ac00640 1 -- 172.21.15.42:0/4079077263 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4940b5ff0 con 0x7fe4940b5650 2024-07-04T17:30:40.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.544+0000 7fe49ac00640 1 -- 172.21.15.42:0/4079077263 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe49406c2c0 con 0x7fe4940b5650 2024-07-04T17:30:40.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.544+0000 7fe491600640 1 -- 172.21.15.42:0/4079077263 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7fe488042020 con 0x7fe4940b5650 2024-07-04T17:30:40.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.545+0000 7fe491600640 1 --2- 172.21.15.42:0/4079077263 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fe47403d4f0 0x7fe47403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:40.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.545+0000 7fe491600640 1 -- 172.21.15.42:0/4079077263 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 3705+0+0 (secure 0 0 0) 0x7fe488040020 con 0x7fe4940b5650 2024-07-04T17:30:40.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.545+0000 7fe498e00640 1 --2- 172.21.15.42:0/4079077263 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fe47403d4f0 0x7fe47403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:40.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.546+0000 7fe498e00640 1 --2- 172.21.15.42:0/4079077263 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fe47403d4f0 0x7fe47403fa00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe4800015c0 tx=0x7fe480007a90 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:40.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.550+0000 7fe491600640 1 -- 172.21.15.42:0/4079077263 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fe488049020 con 0x7fe4940b5650 2024-07-04T17:30:40.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.760+0000 7fe49ac00640 1 -- 172.21.15.42:0/4079077263 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fe4941d4e60 con 0x7fe4940b5650 2024-07-04T17:30:40.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.761+0000 7fe491600640 1 -- 172.21.15.42:0/4079077263 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v26) ==== 59+0+1628 (secure 0 0 0) 0x7fe48803fe00 con 0x7fe4940b5650 2024-07-04T17:30:40.762 INFO:teuthology.orchestra.run.smithi042.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-07-04T17:30:40.762 INFO:teuthology.orchestra.run.smithi042.stdout:mon advanced mon_op_complaint_time 120 2024-07-04T17:30:40.762 INFO:teuthology.orchestra.run.smithi042.stdout:mon dev mon_warn_on_pool_no_app false 2024-07-04T17:30:40.763 INFO:teuthology.orchestra.run.smithi042.stdout:osd advanced osd_op_complaint_time 180.000000 2024-07-04T17:30:40.764 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3541.919687 2024-07-04T17:30:40.764 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3558.529833 2024-07-04T17:30:40.764 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3688.441103 2024-07-04T17:30:40.764 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced debug_mds 20/20 2024-07-04T17:30:40.764 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced debug_mds_balancer 20/20 2024-07-04T17:30:40.764 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced debug_ms 1/1 2024-07-04T17:30:40.764 INFO:teuthology.orchestra.run.smithi042.stdout:mds dev mds_debug_frag true 2024-07-04T17:30:40.764 INFO:teuthology.orchestra.run.smithi042.stdout:mds dev mds_debug_scatterstat true 2024-07-04T17:30:40.764 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced mds_op_complaint_time 180.000000 2024-07-04T17:30:40.765 INFO:teuthology.orchestra.run.smithi042.stdout:mds dev mds_verify_scatter true 2024-07-04T17:30:40.765 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced osd_op_complaint_time 180.000000 2024-07-04T17:30:40.765 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced rados_mon_op_timeout 900 2024-07-04T17:30:40.765 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced rados_osd_op_timeout 900 2024-07-04T17:30:40.765 INFO:teuthology.orchestra.run.smithi042.stdout:client advanced client_mount_timeout 600 2024-07-04T17:30:40.765 INFO:teuthology.orchestra.run.smithi042.stdout:client advanced debug_client 20/20 2024-07-04T17:30:40.765 INFO:teuthology.orchestra.run.smithi042.stdout:client advanced debug_ms 1/1 2024-07-04T17:30:40.765 INFO:teuthology.orchestra.run.smithi042.stdout:client advanced rados_mon_op_timeout 900 2024-07-04T17:30:40.765 INFO:teuthology.orchestra.run.smithi042.stdout:client advanced rados_osd_op_timeout 900 2024-07-04T17:30:40.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.764+0000 7fe49ac00640 1 -- 172.21.15.42:0/4079077263 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fe47403d4f0 msgr2=0x7fe47403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:40.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.764+0000 7fe49ac00640 1 --2- 172.21.15.42:0/4079077263 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fe47403d4f0 0x7fe47403fa00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe4800015c0 tx=0x7fe480007a90 comp rx=0 tx=0).stop 2024-07-04T17:30:40.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.764+0000 7fe49ac00640 1 -- 172.21.15.42:0/4079077263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4940b5650 msgr2=0x7fe4941d4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:40.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.764+0000 7fe49ac00640 1 --2- 172.21.15.42:0/4079077263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4940b5650 0x7fe4941d4660 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fe48800b310 tx=0x7fe488009ae0 comp rx=0 tx=0).stop 2024-07-04T17:30:40.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.764+0000 7fe49ac00640 1 -- 172.21.15.42:0/4079077263 shutdown_connections 2024-07-04T17:30:40.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.764+0000 7fe49ac00640 1 --2- 172.21.15.42:0/4079077263 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fe47403d4f0 0x7fe47403fa00 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:40.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.764+0000 7fe49ac00640 1 --2- 172.21.15.42:0/4079077263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4940b5650 0x7fe4941d4660 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:40.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.764+0000 7fe49ac00640 1 -- 172.21.15.42:0/4079077263 >> 172.21.15.42:0/4079077263 conn(0x7fe4940bebc0 msgr2=0x7fe4940bcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:40.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.764+0000 7fe49ac00640 1 -- 172.21.15.42:0/4079077263 shutdown_connections 2024-07-04T17:30:40.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:40.764+0000 7fe49ac00640 1 -- 172.21.15.42:0/4079077263 wait complete. 2024-07-04T17:30:40.789 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-07-04T17:30:41.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.080+0000 7f8edb200640 1 -- 172.21.15.42:0/2536951649 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8edc0b3730 msgr2=0x7f8edc073e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:41.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.080+0000 7f8edb200640 1 --2- 172.21.15.42:0/2536951649 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8edc0b3730 0x7f8edc073e00 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40097b0 tx=0x7f8ec402a530 comp rx=0 tx=0).stop 2024-07-04T17:30:41.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.080+0000 7f8edb200640 1 -- 172.21.15.42:0/2536951649 shutdown_connections 2024-07-04T17:30:41.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.080+0000 7f8edb200640 1 --2- 172.21.15.42:0/2536951649 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8edc0b3730 0x7f8edc073e00 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:41.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.080+0000 7f8edb200640 1 -- 172.21.15.42:0/2536951649 >> 172.21.15.42:0/2536951649 conn(0x7f8edc0719c0 msgr2=0x7f8edc06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:41.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.080+0000 7f8edb200640 1 -- 172.21.15.42:0/2536951649 shutdown_connections 2024-07-04T17:30:41.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.080+0000 7f8edb200640 1 -- 172.21.15.42:0/2536951649 wait complete. 2024-07-04T17:30:41.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.082+0000 7f8edb200640 1 Processor -- start 2024-07-04T17:30:41.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.082+0000 7f8edb200640 1 -- start start 2024-07-04T17:30:41.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.084+0000 7f8edb200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8edc0b3730 0x7f8edc0b0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:41.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.084+0000 7f8edb200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc072750 con 0x7f8edc0b3730 2024-07-04T17:30:41.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.084+0000 7f8ed9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8edc0b3730 0x7f8edc0b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:41.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.084+0000 7f8ed9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8edc0b3730 0x7f8edc0b0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48828/0 (socket says 172.21.15.42:48828) 2024-07-04T17:30:41.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.084+0000 7f8ed9e00640 1 -- 172.21.15.42:0/1818121055 learned_addr learned my addr 172.21.15.42:0/1818121055 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:41.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.085+0000 7f8ed9e00640 1 -- 172.21.15.42:0/1818121055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8edc0ae1c0 con 0x7f8edc0b3730 2024-07-04T17:30:41.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.085+0000 7f8ed9e00640 1 --2- 172.21.15.42:0/1818121055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8edc0b3730 0x7f8edc0b0160 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40098d0 tx=0x7f8ec4033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:41.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.085+0000 7f8ed2000640 1 -- 172.21.15.42:0/1818121055 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ec4031880 con 0x7f8edc0b3730 2024-07-04T17:30:41.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.085+0000 7f8ed2000640 1 -- 172.21.15.42:0/1818121055 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ec4031b10 con 0x7f8edc0b3730 2024-07-04T17:30:41.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.085+0000 7f8ed2000640 1 -- 172.21.15.42:0/1818121055 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ec4031df0 con 0x7f8edc0b3730 2024-07-04T17:30:41.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.085+0000 7f8edb200640 1 -- 172.21.15.42:0/1818121055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8edc0afc60 con 0x7f8edc0b3730 2024-07-04T17:30:41.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.085+0000 7f8edb200640 1 -- 172.21.15.42:0/1818121055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8edc0ae680 con 0x7f8edc0b3730 2024-07-04T17:30:41.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.087+0000 7f8edb200640 1 -- 172.21.15.42:0/1818121055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8edc0adf60 con 0x7f8edc0b3730 2024-07-04T17:30:41.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.087+0000 7f8ed2000640 1 -- 172.21.15.42:0/1818121055 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f8ec4033070 con 0x7f8edc0b3730 2024-07-04T17:30:41.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.087+0000 7f8ed2000640 1 --2- 172.21.15.42:0/1818121055 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f8eb803d4f0 0x7f8eb803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:41.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.087+0000 7f8ed2000640 1 -- 172.21.15.42:0/1818121055 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 3705+0+0 (secure 0 0 0) 0x7f8ec4072d40 con 0x7f8edc0b3730 2024-07-04T17:30:41.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.088+0000 7f8ed9400640 1 --2- 172.21.15.42:0/1818121055 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f8eb803d4f0 0x7f8eb803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:41.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.089+0000 7f8ed9400640 1 --2- 172.21.15.42:0/1818121055 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f8eb803d4f0 0x7f8eb803fa00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc002020 tx=0x7f8ecc001670 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:41.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.093+0000 7f8ed2000640 1 -- 172.21.15.42:0/1818121055 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f8ec403f030 con 0x7f8edc0b3730 2024-07-04T17:30:41.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.305+0000 7f8edb200640 1 -- 172.21.15.42:0/1818121055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f8edc0ae390 con 0x7f8edc0b3730 2024-07-04T17:30:41.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.309+0000 7f8ed2000640 1 -- 172.21.15.42:0/1818121055 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v27)=0 v27) ==== 122+0+0 (secure 0 0 0) 0x7f8ec403a3a0 con 0x7f8edc0b3730 2024-07-04T17:30:41.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.312+0000 7f8edb200640 1 -- 172.21.15.42:0/1818121055 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f8eb803d4f0 msgr2=0x7f8eb803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:41.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.312+0000 7f8edb200640 1 --2- 172.21.15.42:0/1818121055 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f8eb803d4f0 0x7f8eb803fa00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc002020 tx=0x7f8ecc001670 comp rx=0 tx=0).stop 2024-07-04T17:30:41.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.312+0000 7f8edb200640 1 -- 172.21.15.42:0/1818121055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8edc0b3730 msgr2=0x7f8edc0b0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:41.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.312+0000 7f8edb200640 1 --2- 172.21.15.42:0/1818121055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8edc0b3730 0x7f8edc0b0160 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40098d0 tx=0x7f8ec4033ad0 comp rx=0 tx=0).stop 2024-07-04T17:30:41.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.313+0000 7f8edb200640 1 -- 172.21.15.42:0/1818121055 shutdown_connections 2024-07-04T17:30:41.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.313+0000 7f8edb200640 1 --2- 172.21.15.42:0/1818121055 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f8eb803d4f0 0x7f8eb803fa00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:41.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.313+0000 7f8edb200640 1 --2- 172.21.15.42:0/1818121055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8edc0b3730 0x7f8edc0b0160 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:41.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.313+0000 7f8edb200640 1 -- 172.21.15.42:0/1818121055 >> 172.21.15.42:0/1818121055 conn(0x7f8edc0719c0 msgr2=0x7f8edc14a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:41.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.313+0000 7f8edb200640 1 -- 172.21.15.42:0/1818121055 shutdown_connections 2024-07-04T17:30:41.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.313+0000 7f8edb200640 1 -- 172.21.15.42:0/1818121055 wait complete. 2024-07-04T17:30:41.340 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-04T17:30:41.340 DEBUG:teuthology.orchestra.run.smithi042:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-04T17:30:41.443 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-04T17:30:41.443 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:30:41.443 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:30:41.497 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-07-04T17:30:41.497 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:30:41.497 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:30:41.555 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-07-04T17:30:41.555 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-07-04T17:30:41.555 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:30:41.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.843+0000 7ffb6a800640 1 -- 172.21.15.42:0/2695493171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb640b2d20 msgr2=0x7ffb640b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:41.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.843+0000 7ffb6a800640 1 --2- 172.21.15.42:0/2695493171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb640b2d20 0x7ffb640b01f0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0097b0 tx=0x7ffb4c02a540 comp rx=0 tx=0).stop 2024-07-04T17:30:41.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.843+0000 7ffb6a800640 1 -- 172.21.15.42:0/2695493171 shutdown_connections 2024-07-04T17:30:41.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.843+0000 7ffb6a800640 1 --2- 172.21.15.42:0/2695493171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb640b2d20 0x7ffb640b01f0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:41.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.843+0000 7ffb6a800640 1 -- 172.21.15.42:0/2695493171 >> 172.21.15.42:0/2695493171 conn(0x7ffb640bf700 msgr2=0x7ffb640af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:41.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.843+0000 7ffb6a800640 1 -- 172.21.15.42:0/2695493171 shutdown_connections 2024-07-04T17:30:41.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.843+0000 7ffb6a800640 1 -- 172.21.15.42:0/2695493171 wait complete. 2024-07-04T17:30:41.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.845+0000 7ffb6a800640 1 Processor -- start 2024-07-04T17:30:41.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.846+0000 7ffb6a800640 1 -- start start 2024-07-04T17:30:41.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.847+0000 7ffb6a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb640b2d20 0x7ffb64073950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:41.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.847+0000 7ffb6a800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6406cf50 con 0x7ffb640b2d20 2024-07-04T17:30:41.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.847+0000 7ffb63400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb640b2d20 0x7ffb64073950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:41.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.847+0000 7ffb63400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb640b2d20 0x7ffb64073950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48840/0 (socket says 172.21.15.42:48840) 2024-07-04T17:30:41.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.847+0000 7ffb63400640 1 -- 172.21.15.42:0/3513945223 learned_addr learned my addr 172.21.15.42:0/3513945223 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:41.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.848+0000 7ffb63400640 1 -- 172.21.15.42:0/3513945223 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb640b6870 con 0x7ffb640b2d20 2024-07-04T17:30:41.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.848+0000 7ffb63400640 1 --2- 172.21.15.42:0/3513945223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb640b2d20 0x7ffb64073950 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400c380 tx=0x7ffb5400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:41.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.848+0000 7ffb5b400640 1 -- 172.21.15.42:0/3513945223 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb540116c0 con 0x7ffb640b2d20 2024-07-04T17:30:41.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.848+0000 7ffb5b400640 1 -- 172.21.15.42:0/3513945223 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb54017070 con 0x7ffb640b2d20 2024-07-04T17:30:41.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.848+0000 7ffb5b400640 1 -- 172.21.15.42:0/3513945223 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb54011b20 con 0x7ffb640b2d20 2024-07-04T17:30:41.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.848+0000 7ffb6a800640 1 -- 172.21.15.42:0/3513945223 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb64137110 con 0x7ffb640b2d20 2024-07-04T17:30:41.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.849+0000 7ffb6a800640 1 -- 172.21.15.42:0/3513945223 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb640b5af0 con 0x7ffb640b2d20 2024-07-04T17:30:41.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.850+0000 7ffb6a800640 1 -- 172.21.15.42:0/3513945223 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb640b01f0 con 0x7ffb640b2d20 2024-07-04T17:30:41.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.850+0000 7ffb5b400640 1 -- 172.21.15.42:0/3513945223 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7ffb54011cc0 con 0x7ffb640b2d20 2024-07-04T17:30:41.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.850+0000 7ffb5b400640 1 --2- 172.21.15.42:0/3513945223 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ffb4003d4f0 0x7ffb4003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:41.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.851+0000 7ffb5b400640 1 -- 172.21.15.42:0/3513945223 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 3705+0+0 (secure 0 0 0) 0x7ffb54051710 con 0x7ffb640b2d20 2024-07-04T17:30:41.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.851+0000 7ffb5be00640 1 --2- 172.21.15.42:0/3513945223 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ffb4003d4f0 0x7ffb4003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:41.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.852+0000 7ffb5be00640 1 --2- 172.21.15.42:0/3513945223 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ffb4003d4f0 0x7ffb4003fa00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c034000 tx=0x7ffb4c004730 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:41.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:41.856+0000 7ffb5b400640 1 -- 172.21.15.42:0/3513945223 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7ffb5400a140 con 0x7ffb640b2d20 2024-07-04T17:30:42.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.107+0000 7ffb6a800640 1 -- 172.21.15.42:0/3513945223 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb640b6590 con 0x7ffb640b2d20 2024-07-04T17:30:42.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.108+0000 7ffb5b400640 1 -- 172.21.15.42:0/3513945223 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+4733 (secure 0 0 0) 0x7ffb540513d0 con 0x7ffb640b2d20 2024-07-04T17:30:42.109 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:30:42.109 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":47,"btime":"2024-07-04T17:30:34:178852+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4721,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/343791977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":343791977},{"type":"v1","addr":"172.21.15.42:6827","nonce":343791977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":4722,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/3172446281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":3172446281},{"type":"v1","addr":"172.21.15.42:6829","nonce":3172446281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":4727,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/1866310217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1866310217},{"type":"v1","addr":"172.21.15.42:6831","nonce":1866310217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":4950,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/2046217308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":2046217308},{"type":"v1","addr":"172.21.15.42:6835","nonce":2046217308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":4981,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6833/3491817453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":3491817453},{"type":"v1","addr":"172.21.15.42:6833","nonce":3491817453}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[]} 2024-07-04T17:30:42.110 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 47 2024-07-04T17:30:42.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.111+0000 7ffb6a800640 1 -- 172.21.15.42:0/3513945223 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ffb4003d4f0 msgr2=0x7ffb4003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:42.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.111+0000 7ffb6a800640 1 --2- 172.21.15.42:0/3513945223 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ffb4003d4f0 0x7ffb4003fa00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c034000 tx=0x7ffb4c004730 comp rx=0 tx=0).stop 2024-07-04T17:30:42.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.111+0000 7ffb6a800640 1 -- 172.21.15.42:0/3513945223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb640b2d20 msgr2=0x7ffb64073950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:42.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.111+0000 7ffb6a800640 1 --2- 172.21.15.42:0/3513945223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb640b2d20 0x7ffb64073950 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400c380 tx=0x7ffb5400c840 comp rx=0 tx=0).stop 2024-07-04T17:30:42.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.111+0000 7ffb6a800640 1 -- 172.21.15.42:0/3513945223 shutdown_connections 2024-07-04T17:30:42.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.111+0000 7ffb6a800640 1 --2- 172.21.15.42:0/3513945223 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ffb4003d4f0 0x7ffb4003fa00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:42.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.111+0000 7ffb6a800640 1 --2- 172.21.15.42:0/3513945223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb640b2d20 0x7ffb64073950 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:42.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.111+0000 7ffb6a800640 1 -- 172.21.15.42:0/3513945223 >> 172.21.15.42:0/3513945223 conn(0x7ffb640bf700 msgr2=0x7ffb640bd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:42.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.111+0000 7ffb6a800640 1 -- 172.21.15.42:0/3513945223 shutdown_connections 2024-07-04T17:30:42.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.111+0000 7ffb6a800640 1 -- 172.21.15.42:0/3513945223 wait complete. 2024-07-04T17:30:42.138 INFO:tasks.ceph.mds.a:Restarting daemon 2024-07-04T17:30:42.138 INFO:tasks.ceph.mds.a:Stopping old one... 2024-07-04T17:30:42.139 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-07-04T17:30:42.139 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:30:42.139 INFO:tasks.ceph.mds.b:Restarting daemon 2024-07-04T17:30:42.139 INFO:tasks.ceph.mds.b:Stopping old one... 2024-07-04T17:30:42.139 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-07-04T17:30:42.139 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:30:42.139 INFO:tasks.ceph.mds.c:Restarting daemon 2024-07-04T17:30:42.140 INFO:tasks.ceph.mds.c:Stopping old one... 2024-07-04T17:30:42.140 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-07-04T17:30:42.140 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:30:42.140 INFO:tasks.ceph.mds.d:Restarting daemon 2024-07-04T17:30:42.140 INFO:tasks.ceph.mds.d:Stopping old one... 2024-07-04T17:30:42.140 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-07-04T17:30:42.140 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:30:42.140 INFO:tasks.ceph.mds.e:Restarting daemon 2024-07-04T17:30:42.141 INFO:tasks.ceph.mds.e:Stopping old one... 2024-07-04T17:30:42.141 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-07-04T17:30:42.141 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:30:42.168 INFO:tasks.ceph.mds.b:Stopped 2024-07-04T17:30:42.168 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-07-04T17:30:42.171 INFO:tasks.ceph.mds.a:Stopped 2024-07-04T17:30:42.171 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-07-04T17:30:42.182 INFO:tasks.ceph.mds.e:Stopped 2024-07-04T17:30:42.182 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-07-04T17:30:42.185 INFO:tasks.ceph.mds.c:Stopped 2024-07-04T17:30:42.185 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-07-04T17:30:42.190 INFO:tasks.ceph.mds.b:Started 2024-07-04T17:30:42.190 INFO:tasks.ceph.mds.a:Started 2024-07-04T17:30:42.191 DEBUG:teuthology.parallel:result is None 2024-07-04T17:30:42.191 DEBUG:teuthology.parallel:result is None 2024-07-04T17:30:42.192 INFO:tasks.ceph.mds.d:Stopped 2024-07-04T17:30:42.192 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-07-04T17:30:42.194 INFO:tasks.ceph.mds.e:Started 2024-07-04T17:30:42.194 DEBUG:teuthology.parallel:result is None 2024-07-04T17:30:42.197 INFO:tasks.ceph.mds.c:Started 2024-07-04T17:30:42.197 DEBUG:teuthology.parallel:result is None 2024-07-04T17:30:42.200 INFO:tasks.ceph.mds.d:Started 2024-07-04T17:30:42.200 DEBUG:teuthology.parallel:result is None 2024-07-04T17:30:42.200 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-07-04T17:30:42.440 INFO:tasks.ceph.mds.c.smithi042.stdout:starting mds.c at 2024-07-04T17:30:42.454 INFO:tasks.ceph.mds.d.smithi042.stdout:starting mds.d at 2024-07-04T17:30:42.493 INFO:tasks.ceph.mds.a.smithi042.stdout:starting mds.a at 2024-07-04T17:30:42.511 INFO:tasks.ceph.mds.b.smithi042.stdout:starting mds.b at 2024-07-04T17:30:42.527 INFO:tasks.ceph.mds.e.smithi042.stdout:starting mds.e at 2024-07-04T17:30:42.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.571+0000 7fbaf6c00640 1 -- 172.21.15.42:0/1319532932 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbaf006c2c0 msgr2=0x7fbaf00b5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:42.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.571+0000 7fbaf6c00640 1 --2- 172.21.15.42:0/1319532932 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbaf006c2c0 0x7fbaf00b5650 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fbadc005fd0 tx=0x7fbadc02ae00 comp rx=0 tx=0).stop 2024-07-04T17:30:42.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.572+0000 7fbaf6c00640 1 -- 172.21.15.42:0/1319532932 shutdown_connections 2024-07-04T17:30:42.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.572+0000 7fbaf6c00640 1 --2- 172.21.15.42:0/1319532932 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbaf006c2c0 0x7fbaf00b5650 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:42.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.572+0000 7fbaf6c00640 1 -- 172.21.15.42:0/1319532932 >> 172.21.15.42:0/1319532932 conn(0x7fbaf00bebc0 msgr2=0x7fbaf00befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:42.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.572+0000 7fbaf6c00640 1 -- 172.21.15.42:0/1319532932 shutdown_connections 2024-07-04T17:30:42.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.572+0000 7fbaf6c00640 1 -- 172.21.15.42:0/1319532932 wait complete. 2024-07-04T17:30:42.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.574+0000 7fbaf6c00640 1 Processor -- start 2024-07-04T17:30:42.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.574+0000 7fbaf6c00640 1 -- start start 2024-07-04T17:30:42.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.575+0000 7fbaf6c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbaf006c2c0 0x7fbaf0072c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:42.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.575+0000 7fbaf6c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf00b2810 con 0x7fbaf006c2c0 2024-07-04T17:30:42.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.575+0000 7fbaf5800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbaf006c2c0 0x7fbaf0072c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:42.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.575+0000 7fbaf5800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbaf006c2c0 0x7fbaf0072c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48926/0 (socket says 172.21.15.42:48926) 2024-07-04T17:30:42.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.575+0000 7fbaf5800640 1 -- 172.21.15.42:0/3071693983 learned_addr learned my addr 172.21.15.42:0/3071693983 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:42.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.576+0000 7fbaf5800640 1 -- 172.21.15.42:0/3071693983 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf00b7630 con 0x7fbaf006c2c0 2024-07-04T17:30:42.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.576+0000 7fbaf5800640 1 --2- 172.21.15.42:0/3071693983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbaf006c2c0 0x7fbaf0072c70 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fbae400c320 tx=0x7fbae400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:42.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.576+0000 7fbaed600640 1 -- 172.21.15.42:0/3071693983 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbae4017070 con 0x7fbaf006c2c0 2024-07-04T17:30:42.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.576+0000 7fbaed600640 1 -- 172.21.15.42:0/3071693983 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbae4005310 con 0x7fbaf006c2c0 2024-07-04T17:30:42.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.576+0000 7fbaed600640 1 -- 172.21.15.42:0/3071693983 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbae40035f0 con 0x7fbaf006c2c0 2024-07-04T17:30:42.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.576+0000 7fbaf6c00640 1 -- 172.21.15.42:0/3071693983 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaf013b9e0 con 0x7fbaf006c2c0 2024-07-04T17:30:42.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.577+0000 7fbaf6c00640 1 -- 172.21.15.42:0/3071693983 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaf00b67a0 con 0x7fbaf006c2c0 2024-07-04T17:30:42.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.577+0000 7fbaf6c00640 1 -- 172.21.15.42:0/3071693983 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaf013bbb0 con 0x7fbaf006c2c0 2024-07-04T17:30:42.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.578+0000 7fbaed600640 1 -- 172.21.15.42:0/3071693983 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7fbae401a070 con 0x7fbaf006c2c0 2024-07-04T17:30:42.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.578+0000 7fbaed600640 1 --2- 172.21.15.42:0/3071693983 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fbad003d4f0 0x7fbad003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:42.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.578+0000 7fbaed600640 1 -- 172.21.15.42:0/3071693983 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 3705+0+0 (secure 0 0 0) 0x7fbae4050ce0 con 0x7fbaf006c2c0 2024-07-04T17:30:42.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.579+0000 7fbaf4e00640 1 --2- 172.21.15.42:0/3071693983 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fbad003d4f0 0x7fbad003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:42.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.580+0000 7fbaf4e00640 1 --2- 172.21.15.42:0/3071693983 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fbad003d4f0 0x7fbad003fa00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbadc034000 tx=0x7fbadc004730 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:42.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.583+0000 7fbaed600640 1 -- 172.21.15.42:0/3071693983 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fbaf006c6d0 con 0x7fbaf006c2c0 2024-07-04T17:30:42.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:42.801+0000 7fbaf6c00640 1 -- 172.21.15.42:0/3071693983 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fbaf00b5650 con 0x7fbaf006c2c0 2024-07-04T17:30:43.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.327+0000 7fbaed600640 1 -- 172.21.15.42:0/3071693983 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v65) ==== 97+0+0 (secure 0 0 0) 0x7fbae401ace0 con 0x7fbaf006c2c0 2024-07-04T17:30:43.329 INFO:teuthology.orchestra.run.smithi042.stderr: removed all blocklist entries 2024-07-04T17:30:43.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.331+0000 7fbaf6c00640 1 -- 172.21.15.42:0/3071693983 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fbad003d4f0 msgr2=0x7fbad003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:43.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.331+0000 7fbaf6c00640 1 --2- 172.21.15.42:0/3071693983 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fbad003d4f0 0x7fbad003fa00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbadc034000 tx=0x7fbadc004730 comp rx=0 tx=0).stop 2024-07-04T17:30:43.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.332+0000 7fbaf6c00640 1 -- 172.21.15.42:0/3071693983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbaf006c2c0 msgr2=0x7fbaf0072c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:43.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.332+0000 7fbaf6c00640 1 --2- 172.21.15.42:0/3071693983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbaf006c2c0 0x7fbaf0072c70 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fbae400c320 tx=0x7fbae400c7e0 comp rx=0 tx=0).stop 2024-07-04T17:30:43.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.332+0000 7fbaf6c00640 1 -- 172.21.15.42:0/3071693983 shutdown_connections 2024-07-04T17:30:43.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.332+0000 7fbaf6c00640 1 --2- 172.21.15.42:0/3071693983 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fbad003d4f0 0x7fbad003fa00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:43.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.332+0000 7fbaf6c00640 1 --2- 172.21.15.42:0/3071693983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbaf006c2c0 0x7fbaf0072c70 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:43.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.332+0000 7fbaf6c00640 1 -- 172.21.15.42:0/3071693983 >> 172.21.15.42:0/3071693983 conn(0x7fbaf00bebc0 msgr2=0x7fbaf013fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:43.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.332+0000 7fbaf6c00640 1 -- 172.21.15.42:0/3071693983 shutdown_connections 2024-07-04T17:30:43.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.332+0000 7fbaf6c00640 1 -- 172.21.15.42:0/3071693983 wait complete. 2024-07-04T17:30:43.366 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-07-04T17:30:43.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.637+0000 7f6039200640 1 -- 172.21.15.42:0/3717863837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60340b37d0 msgr2=0x7f60340b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:43.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.637+0000 7f6039200640 1 --2- 172.21.15.42:0/3717863837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60340b37d0 0x7f60340b01f0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f60240097b0 tx=0x7f602402a540 comp rx=0 tx=0).stop 2024-07-04T17:30:43.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.637+0000 7f6039200640 1 -- 172.21.15.42:0/3717863837 shutdown_connections 2024-07-04T17:30:43.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.637+0000 7f6039200640 1 --2- 172.21.15.42:0/3717863837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60340b37d0 0x7f60340b01f0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:43.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.637+0000 7f6039200640 1 -- 172.21.15.42:0/3717863837 >> 172.21.15.42:0/3717863837 conn(0x7f60340bf700 msgr2=0x7f60340af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:43.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.637+0000 7f6039200640 1 -- 172.21.15.42:0/3717863837 shutdown_connections 2024-07-04T17:30:43.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.637+0000 7f6039200640 1 -- 172.21.15.42:0/3717863837 wait complete. 2024-07-04T17:30:43.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.639+0000 7f6039200640 1 Processor -- start 2024-07-04T17:30:43.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.640+0000 7f6039200640 1 -- start start 2024-07-04T17:30:43.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.641+0000 7f6039200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60340b2d20 0x7f6034140ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:43.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.641+0000 7f6039200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f603406ce30 con 0x7f60340b2d20 2024-07-04T17:30:43.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.641+0000 7f6031e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60340b2d20 0x7f6034140ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:43.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.641+0000 7f6031e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60340b2d20 0x7f6034140ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48948/0 (socket says 172.21.15.42:48948) 2024-07-04T17:30:43.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.641+0000 7f6031e00640 1 -- 172.21.15.42:0/346181546 learned_addr learned my addr 172.21.15.42:0/346181546 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:43.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.642+0000 7f6031e00640 1 -- 172.21.15.42:0/346181546 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60341d7d50 con 0x7f60340b2d20 2024-07-04T17:30:43.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.642+0000 7f6031e00640 1 --2- 172.21.15.42:0/346181546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60340b2d20 0x7f6034140ee0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f601c00c2f0 tx=0x7f601c00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:43.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.642+0000 7f602a000640 1 -- 172.21.15.42:0/346181546 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f601c0116c0 con 0x7f60340b2d20 2024-07-04T17:30:43.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.642+0000 7f602a000640 1 -- 172.21.15.42:0/346181546 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f601c017070 con 0x7f60340b2d20 2024-07-04T17:30:43.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.642+0000 7f6039200640 1 -- 172.21.15.42:0/346181546 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60341e7000 con 0x7f60340b2d20 2024-07-04T17:30:43.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.642+0000 7f602a000640 1 -- 172.21.15.42:0/346181546 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f601c011ac0 con 0x7f60340b2d20 2024-07-04T17:30:43.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.642+0000 7f6039200640 1 -- 172.21.15.42:0/346181546 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6034141420 con 0x7f60340b2d20 2024-07-04T17:30:43.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.643+0000 7f6039200640 1 -- 172.21.15.42:0/346181546 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f603406d490 con 0x7f60340b2d20 2024-07-04T17:30:43.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.644+0000 7f602a000640 1 -- 172.21.15.42:0/346181546 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f601c011c60 con 0x7f60340b2d20 2024-07-04T17:30:43.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.644+0000 7f602a000640 1 --2- 172.21.15.42:0/346181546 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f601003d4f0 0x7f601003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:43.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.644+0000 7f602a000640 1 -- 172.21.15.42:0/346181546 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 2587+0+0 (secure 0 0 0) 0x7f601c0511e0 con 0x7f60340b2d20 2024-07-04T17:30:43.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.644+0000 7f6031400640 1 --2- 172.21.15.42:0/346181546 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f601003d4f0 0x7f601003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:43.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.645+0000 7f6031400640 1 --2- 172.21.15.42:0/346181546 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f601003d4f0 0x7f601003fa00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6024034000 tx=0x7f6024004730 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:43.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.649+0000 7f602a000640 1 -- 172.21.15.42:0/346181546 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f601c00a140 con 0x7f60340b2d20 2024-07-04T17:30:43.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.929+0000 7f6039200640 1 -- 172.21.15.42:0/346181546 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f60340af6e0 con 0x7f60340b2d20 2024-07-04T17:30:43.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.931+0000 7f602a000640 1 -- 172.21.15.42:0/346181546 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) ==== 80+0+4064 (secure 0 0 0) 0x7f601c010cd0 con 0x7f60340b2d20 2024-07-04T17:30:43.932 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:30:43.932 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:30:43.932 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_dump": [ 2024-07-04T17:30:43.932 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:43.932 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.a", 2024-07-04T17:30:43.933 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAM24Zm2VMEFxAAQqdcGux6Ker7jEr1dGAfzA==", 2024-07-04T17:30:43.933 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:30:43.933 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:30:43.933 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:30:43.933 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:30:43.933 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:30:43.933 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:43.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:43.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:43.933 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.b", 2024-07-04T17:30:43.933 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAM24ZmS/S7KxAAfjTZrLYR9qLxuTvd2zsCcQ==", 2024-07-04T17:30:43.933 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:30:43.933 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:30:43.934 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:30:43.934 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:30:43.934 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:30:43.934 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:43.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:43.934 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:43.934 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.c", 2024-07-04T17:30:43.934 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAN24Zm2JrRBBAAFMYIM2fLsr9EOfcyh3tZpA==", 2024-07-04T17:30:43.934 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:30:43.934 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:30:43.934 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:30:43.934 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:30:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:30:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.d", 2024-07-04T17:30:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAN24ZmPCGLFxAAhsh3XaAHCyauCGxh/3DFsw==", 2024-07-04T17:30:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:30:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:30:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:30:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:30:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:30:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:43.935 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:43.936 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.e", 2024-07-04T17:30:43.936 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAN24ZmVIaSKhAAhoBdDalwQ4eoo0rqb6p0Vw==", 2024-07-04T17:30:43.936 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:30:43.936 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:30:43.936 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:30:43.936 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:30:43.936 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:30:43.936 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:43.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:43.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:43.936 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.0", 2024-07-04T17:30:43.936 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAQ24Zm4spcCBAA4kTvvR7IOVa01K5ZCrJ35w==", 2024-07-04T17:30:43.936 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:30:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-07-04T17:30:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-07-04T17:30:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:30:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.1", 2024-07-04T17:30:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAT24Zmz1sfGRAAgfH5eIiQrufnPq6F94rHWg==", 2024-07-04T17:30:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:30:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-07-04T17:30:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-07-04T17:30:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:30:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:43.937 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:43.938 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.2", 2024-07-04T17:30:43.938 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAW24ZmJdQHORAAEjAM9OBqSIRhX148GC6pcw==", 2024-07-04T17:30:43.938 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:30:43.938 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-07-04T17:30:43.938 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-07-04T17:30:43.938 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:30:43.938 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:43.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:43.938 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:43.938 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.0", 2024-07-04T17:30:43.938 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAO24ZmUMmYABAAh4pNlkb9JsHQtK8uCJFQCA==", 2024-07-04T17:30:43.938 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:30:43.938 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:30:43.939 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow r", 2024-07-04T17:30:43.939 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-07-04T17:30:43.939 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:43.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:43.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:43.939 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.1", 2024-07-04T17:30:43.939 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAO24Zm0pAJEBAAntDeQ/oWDlJ+3xC7q5EQnQ==", 2024-07-04T17:30:43.939 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:30:43.939 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:30:43.939 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-07-04T17:30:43.939 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow rw", 2024-07-04T17:30:43.939 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rwx" 2024-07-04T17:30:43.939 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:43.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:43.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:43.940 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.admin", 2024-07-04T17:30:43.940 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAL24ZmMe3iHhAAFciH9cbUAIHLSAJvXy30EQ==", 2024-07-04T17:30:43.940 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:30:43.940 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-07-04T17:30:43.940 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:30:43.940 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:30:43.940 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:30:43.940 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:43.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:43.940 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:43.940 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.mirror", 2024-07-04T17:30:43.940 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAO24ZmBJzBHhAAR4Zg36YTtVizxm/g/Cdk0w==", 2024-07-04T17:30:43.940 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:30:43.941 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow r", 2024-07-04T17:30:43.941 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-07-04T17:30:43.941 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "profile cephfs-mirror", 2024-07-04T17:30:43.941 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rw tag cephfs metadata=*, allow r tag cephfs data=*" 2024-07-04T17:30:43.941 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:43.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:43.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:43.941 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.mirror_remote", 2024-07-04T17:30:43.941 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAO24Zm+BsqLhAA19GVuEjfWQeMe/fSc+dmoA==", 2024-07-04T17:30:43.941 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:30:43.941 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow rwps", 2024-07-04T17:30:43.941 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-07-04T17:30:43.941 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow r", 2024-07-04T17:30:43.941 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rw tag cephfs *=*" 2024-07-04T17:30:43.942 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:43.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:43.942 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:43.942 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mgr.x", 2024-07-04T17:30:43.942 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAM24Zm47tDCBAAjszudEEB4QHLlWqPXyZc2g==", 2024-07-04T17:30:43.942 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:30:43.942 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-07-04T17:30:43.942 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile mgr", 2024-07-04T17:30:43.942 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:30:43.942 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:43.942 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:43.942 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-07-04T17:30:43.942 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:30:43.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.933+0000 7f6039200640 1 -- 172.21.15.42:0/346181546 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f601003d4f0 msgr2=0x7f601003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:43.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.933+0000 7f6039200640 1 --2- 172.21.15.42:0/346181546 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f601003d4f0 0x7f601003fa00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6024034000 tx=0x7f6024004730 comp rx=0 tx=0).stop 2024-07-04T17:30:43.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.933+0000 7f6039200640 1 -- 172.21.15.42:0/346181546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60340b2d20 msgr2=0x7f6034140ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:43.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.933+0000 7f6039200640 1 --2- 172.21.15.42:0/346181546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60340b2d20 0x7f6034140ee0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f601c00c2f0 tx=0x7f601c00c7b0 comp rx=0 tx=0).stop 2024-07-04T17:30:43.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.934+0000 7f6039200640 1 -- 172.21.15.42:0/346181546 shutdown_connections 2024-07-04T17:30:43.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.934+0000 7f6039200640 1 --2- 172.21.15.42:0/346181546 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f601003d4f0 0x7f601003fa00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:43.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.934+0000 7f6039200640 1 --2- 172.21.15.42:0/346181546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60340b2d20 0x7f6034140ee0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:43.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.934+0000 7f6039200640 1 -- 172.21.15.42:0/346181546 >> 172.21.15.42:0/346181546 conn(0x7f60340bf700 msgr2=0x7f60340bc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:43.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.934+0000 7f6039200640 1 -- 172.21.15.42:0/346181546 shutdown_connections 2024-07-04T17:30:43.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:43.934+0000 7f6039200640 1 -- 172.21.15.42:0/346181546 wait complete. 2024-07-04T17:30:43.959 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-04T17:30:44.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.242+0000 7f27fc400640 1 -- 172.21.15.42:0/3197231642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27f40b2d20 msgr2=0x7f27f40b37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:44.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.242+0000 7f27fc400640 1 --2- 172.21.15.42:0/3197231642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27f40b2d20 0x7f27f40b37d0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f27e4005fd0 tx=0x7f27e402a8b0 comp rx=0 tx=0).stop 2024-07-04T17:30:44.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.242+0000 7f27fc400640 1 -- 172.21.15.42:0/3197231642 shutdown_connections 2024-07-04T17:30:44.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.242+0000 7f27fc400640 1 --2- 172.21.15.42:0/3197231642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27f40b2d20 0x7f27f40b37d0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:44.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.242+0000 7f27fc400640 1 -- 172.21.15.42:0/3197231642 >> 172.21.15.42:0/3197231642 conn(0x7f27f40bf700 msgr2=0x7f27f40af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:44.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.242+0000 7f27fc400640 1 -- 172.21.15.42:0/3197231642 shutdown_connections 2024-07-04T17:30:44.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.243+0000 7f27fc400640 1 -- 172.21.15.42:0/3197231642 wait complete. 2024-07-04T17:30:44.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.244+0000 7f27fc400640 1 Processor -- start 2024-07-04T17:30:44.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.245+0000 7f27fc400640 1 -- start start 2024-07-04T17:30:44.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.246+0000 7f27fc400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27f40b01f0 0x7f27f40b5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:44.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.246+0000 7f27fc400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f27f406cc20 con 0x7f27f40b01f0 2024-07-04T17:30:44.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.246+0000 7f27f9a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27f40b01f0 0x7f27f40b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:44.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.246+0000 7f27f9a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27f40b01f0 0x7f27f40b5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39078/0 (socket says 172.21.15.42:39078) 2024-07-04T17:30:44.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.246+0000 7f27f9a00640 1 -- 172.21.15.42:0/1543502074 learned_addr learned my addr 172.21.15.42:0/1543502074 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:44.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.247+0000 7f27f9a00640 1 -- 172.21.15.42:0/1543502074 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f40b56b0 con 0x7f27f40b01f0 2024-07-04T17:30:44.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.247+0000 7f27f9a00640 1 --2- 172.21.15.42:0/1543502074 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27f40b01f0 0x7f27f40b5170 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f27e4009c50 tx=0x7f27e4033b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:44.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.247+0000 7f27f1600640 1 -- 172.21.15.42:0/1543502074 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27e40319f0 con 0x7f27f40b01f0 2024-07-04T17:30:44.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.247+0000 7f27f1600640 1 -- 172.21.15.42:0/1543502074 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27e4031c80 con 0x7f27f40b01f0 2024-07-04T17:30:44.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.247+0000 7f27f1600640 1 -- 172.21.15.42:0/1543502074 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27e4033070 con 0x7f27f40b01f0 2024-07-04T17:30:44.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.247+0000 7f27fc400640 1 -- 172.21.15.42:0/1543502074 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27f41dcc30 con 0x7f27f40b01f0 2024-07-04T17:30:44.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.248+0000 7f27fc400640 1 -- 172.21.15.42:0/1543502074 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27f40ae3c0 con 0x7f27f40b01f0 2024-07-04T17:30:44.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.249+0000 7f27fc400640 1 -- 172.21.15.42:0/1543502074 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27f406d300 con 0x7f27f40b01f0 2024-07-04T17:30:44.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.249+0000 7f27f1600640 1 -- 172.21.15.42:0/1543502074 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f27e4033210 con 0x7f27f40b01f0 2024-07-04T17:30:44.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.249+0000 7f27f1600640 1 --2- 172.21.15.42:0/1543502074 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f27d403d4f0 0x7f27d403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:44.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.250+0000 7f27f1600640 1 -- 172.21.15.42:0/1543502074 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 2587+0+0 (secure 0 0 0) 0x7f27e4072dc0 con 0x7f27f40b01f0 2024-07-04T17:30:44.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.250+0000 7f27f9000640 1 --2- 172.21.15.42:0/1543502074 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f27d403d4f0 0x7f27d403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:44.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.251+0000 7f27f9000640 1 --2- 172.21.15.42:0/1543502074 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f27d403d4f0 0x7f27d403fa00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f27e0000f60 tx=0x7f27e0007a30 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:44.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.254+0000 7f27f1600640 1 -- 172.21.15.42:0/1543502074 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f27e403f030 con 0x7f27f40b01f0 2024-07-04T17:30:44.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.466+0000 7f27fc400640 1 -- 172.21.15.42:0/1543502074 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f27f41dce00 con 0x7f27f40b01f0 2024-07-04T17:30:44.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.467+0000 7f27f1600640 1 -- 172.21.15.42:0/1543502074 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v68) ==== 84+0+65 (secure 0 0 0) 0x7f27e4033500 con 0x7f27f40b01f0 2024-07-04T17:30:44.468 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:30:44.468 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-07-04T17:30:44.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:30:44.468 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 1, 2024-07-04T17:30:44.468 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": ".mgr" 2024-07-04T17:30:44.468 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:44.468 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-07-04T17:30:44.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.469+0000 7f27fc400640 1 -- 172.21.15.42:0/1543502074 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f27d403d4f0 msgr2=0x7f27d403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:44.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.469+0000 7f27fc400640 1 --2- 172.21.15.42:0/1543502074 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f27d403d4f0 0x7f27d403fa00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f27e0000f60 tx=0x7f27e0007a30 comp rx=0 tx=0).stop 2024-07-04T17:30:44.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.469+0000 7f27fc400640 1 -- 172.21.15.42:0/1543502074 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27f40b01f0 msgr2=0x7f27f40b5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:44.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.469+0000 7f27fc400640 1 --2- 172.21.15.42:0/1543502074 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27f40b01f0 0x7f27f40b5170 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f27e4009c50 tx=0x7f27e4033b10 comp rx=0 tx=0).stop 2024-07-04T17:30:44.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.469+0000 7f27fc400640 1 -- 172.21.15.42:0/1543502074 shutdown_connections 2024-07-04T17:30:44.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.469+0000 7f27fc400640 1 --2- 172.21.15.42:0/1543502074 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f27d403d4f0 0x7f27d403fa00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:44.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.469+0000 7f27fc400640 1 --2- 172.21.15.42:0/1543502074 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f27f40b01f0 0x7f27f40b5170 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:44.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.469+0000 7f27fc400640 1 -- 172.21.15.42:0/1543502074 >> 172.21.15.42:0/1543502074 conn(0x7f27f40bf700 msgr2=0x7f27f40b1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:44.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.470+0000 7f27fc400640 1 -- 172.21.15.42:0/1543502074 shutdown_connections 2024-07-04T17:30:44.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.470+0000 7f27fc400640 1 -- 172.21.15.42:0/1543502074 wait complete. 2024-07-04T17:30:44.498 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-04T17:30:44.498 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-04T17:30:44.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.778+0000 7f99a2000640 1 -- 172.21.15.42:0/71613640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f999c06e0c0 msgr2=0x7f999c073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.778+0000 7f99a2000640 1 --2- 172.21.15.42:0/71613640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f999c06e0c0 0x7f999c073800 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f998c005fd0 tx=0x7f998c02a8b0 comp rx=0 tx=0).stop 2024-07-04T17:30:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.778+0000 7f99a2000640 1 -- 172.21.15.42:0/71613640 shutdown_connections 2024-07-04T17:30:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.778+0000 7f99a2000640 1 --2- 172.21.15.42:0/71613640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f999c06e0c0 0x7f999c073800 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.778+0000 7f99a2000640 1 -- 172.21.15.42:0/71613640 >> 172.21.15.42:0/71613640 conn(0x7f999c071b50 msgr2=0x7f999c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.778+0000 7f99a2000640 1 -- 172.21.15.42:0/71613640 shutdown_connections 2024-07-04T17:30:44.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.778+0000 7f99a2000640 1 -- 172.21.15.42:0/71613640 wait complete. 2024-07-04T17:30:44.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.780+0000 7f99a2000640 1 Processor -- start 2024-07-04T17:30:44.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.780+0000 7f99a2000640 1 -- start start 2024-07-04T17:30:44.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.781+0000 7f99a2000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f999c06e0c0 0x7f999c1401c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:44.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.781+0000 7f99a2000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f999c1438d0 con 0x7f999c06e0c0 2024-07-04T17:30:44.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.782+0000 7f99a0c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f999c06e0c0 0x7f999c1401c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:44.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.782+0000 7f99a0c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f999c06e0c0 0x7f999c1401c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39096/0 (socket says 172.21.15.42:39096) 2024-07-04T17:30:44.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.782+0000 7f99a0c00640 1 -- 172.21.15.42:0/31242622 learned_addr learned my addr 172.21.15.42:0/31242622 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:44.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.782+0000 7f99a0c00640 1 -- 172.21.15.42:0/31242622 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f999c147710 con 0x7f999c06e0c0 2024-07-04T17:30:44.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.782+0000 7f99a0c00640 1 --2- 172.21.15.42:0/31242622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f999c06e0c0 0x7f999c1401c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f998c006010 tx=0x7f998c033b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:44.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.783+0000 7f9998c00640 1 -- 172.21.15.42:0/31242622 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f998c0319f0 con 0x7f999c06e0c0 2024-07-04T17:30:44.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.783+0000 7f9998c00640 1 -- 172.21.15.42:0/31242622 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f998c031b90 con 0x7f999c06e0c0 2024-07-04T17:30:44.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.783+0000 7f9998c00640 1 -- 172.21.15.42:0/31242622 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f998c0330d0 con 0x7f999c06e0c0 2024-07-04T17:30:44.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.783+0000 7f99a2000640 1 -- 172.21.15.42:0/31242622 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f999c13f730 con 0x7f999c06e0c0 2024-07-04T17:30:44.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.783+0000 7f99a2000640 1 -- 172.21.15.42:0/31242622 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f999c0b0ac0 con 0x7f999c06e0c0 2024-07-04T17:30:44.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.784+0000 7f99a2000640 1 -- 172.21.15.42:0/31242622 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f999c06dca0 con 0x7f999c06e0c0 2024-07-04T17:30:44.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.784+0000 7f9998c00640 1 -- 172.21.15.42:0/31242622 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f998c033270 con 0x7f999c06e0c0 2024-07-04T17:30:44.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.785+0000 7f9998c00640 1 --2- 172.21.15.42:0/31242622 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f997c03d4f0 0x7f997c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:44.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.785+0000 7f9998c00640 1 -- 172.21.15.42:0/31242622 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 2587+0+0 (secure 0 0 0) 0x7f998c072db0 con 0x7f999c06e0c0 2024-07-04T17:30:44.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.785+0000 7f999be00640 1 --2- 172.21.15.42:0/31242622 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f997c03d4f0 0x7f997c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:44.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.786+0000 7f999be00640 1 --2- 172.21.15.42:0/31242622 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f997c03d4f0 0x7f997c03fa00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f99880015c0 tx=0x7f9988007a90 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:44.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.790+0000 7f9998c00640 1 -- 172.21.15.42:0/31242622 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f999c06dca0 con 0x7f999c06e0c0 2024-07-04T17:30:44.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:44.994+0000 7f99a2000640 1 -- 172.21.15.42:0/31242622 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f999c0b0db0 con 0x7f999c06e0c0 2024-07-04T17:30:45.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.347+0000 7f9998c00640 1 -- 172.21.15.42:0/31242622 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v69) ==== 138+0+0 (secure 0 0 0) 0x7f998c03f030 con 0x7f999c06e0c0 2024-07-04T17:30:45.348 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' created 2024-07-04T17:30:45.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.350+0000 7f99a2000640 1 -- 172.21.15.42:0/31242622 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f997c03d4f0 msgr2=0x7f997c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:45.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.350+0000 7f99a2000640 1 --2- 172.21.15.42:0/31242622 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f997c03d4f0 0x7f997c03fa00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f99880015c0 tx=0x7f9988007a90 comp rx=0 tx=0).stop 2024-07-04T17:30:45.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.350+0000 7f99a2000640 1 -- 172.21.15.42:0/31242622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f999c06e0c0 msgr2=0x7f999c1401c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:45.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.350+0000 7f99a2000640 1 --2- 172.21.15.42:0/31242622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f999c06e0c0 0x7f999c1401c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f998c006010 tx=0x7f998c033b10 comp rx=0 tx=0).stop 2024-07-04T17:30:45.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.350+0000 7f99a2000640 1 -- 172.21.15.42:0/31242622 shutdown_connections 2024-07-04T17:30:45.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.350+0000 7f99a2000640 1 --2- 172.21.15.42:0/31242622 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f997c03d4f0 0x7f997c03fa00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:45.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.350+0000 7f99a2000640 1 --2- 172.21.15.42:0/31242622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f999c06e0c0 0x7f999c1401c0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:45.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.350+0000 7f99a2000640 1 -- 172.21.15.42:0/31242622 >> 172.21.15.42:0/31242622 conn(0x7f999c071b50 msgr2=0x7f999c0b58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:45.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.351+0000 7f99a2000640 1 -- 172.21.15.42:0/31242622 shutdown_connections 2024-07-04T17:30:45.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.351+0000 7f99a2000640 1 -- 172.21.15.42:0/31242622 wait complete. 2024-07-04T17:30:45.388 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-04T17:30:45.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.666+0000 7fb471000640 1 -- 172.21.15.42:0/3529706975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb46c140480 msgr2=0x7fb46c0b1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:45.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.666+0000 7fb471000640 1 --2- 172.21.15.42:0/3529706975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb46c140480 0x7fb46c0b1020 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fb45c005fd0 tx=0x7fb45c02a5f0 comp rx=0 tx=0).stop 2024-07-04T17:30:45.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.666+0000 7fb471000640 1 -- 172.21.15.42:0/3529706975 shutdown_connections 2024-07-04T17:30:45.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.666+0000 7fb471000640 1 --2- 172.21.15.42:0/3529706975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb46c140480 0x7fb46c0b1020 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:45.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.666+0000 7fb471000640 1 -- 172.21.15.42:0/3529706975 >> 172.21.15.42:0/3529706975 conn(0x7fb46c06c020 msgr2=0x7fb46c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:45.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.666+0000 7fb471000640 1 -- 172.21.15.42:0/3529706975 shutdown_connections 2024-07-04T17:30:45.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.666+0000 7fb471000640 1 -- 172.21.15.42:0/3529706975 wait complete. 2024-07-04T17:30:45.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.668+0000 7fb471000640 1 Processor -- start 2024-07-04T17:30:45.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.669+0000 7fb471000640 1 -- start start 2024-07-04T17:30:45.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.670+0000 7fb471000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb46c074200 0x7fb46c144d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:45.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.670+0000 7fb471000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c150f60 con 0x7fb46c074200 2024-07-04T17:30:45.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.670+0000 7fb46b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb46c074200 0x7fb46c144d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:45.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.670+0000 7fb46b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb46c074200 0x7fb46c144d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39102/0 (socket says 172.21.15.42:39102) 2024-07-04T17:30:45.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.670+0000 7fb46b400640 1 -- 172.21.15.42:0/3886287937 learned_addr learned my addr 172.21.15.42:0/3886287937 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:45.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.671+0000 7fb46b400640 1 -- 172.21.15.42:0/3886287937 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46c144830 con 0x7fb46c074200 2024-07-04T17:30:45.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.671+0000 7fb46b400640 1 --2- 172.21.15.42:0/3886287937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb46c074200 0x7fb46c144d30 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009460 tx=0x7fb45c0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:45.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.671+0000 7fb463e00640 1 -- 172.21.15.42:0/3886287937 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb45c03f070 con 0x7fb46c074200 2024-07-04T17:30:45.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.671+0000 7fb463e00640 1 -- 172.21.15.42:0/3886287937 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb45c004f70 con 0x7fb46c074200 2024-07-04T17:30:45.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.671+0000 7fb471000640 1 -- 172.21.15.42:0/3886287937 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb46c13b100 con 0x7fb46c074200 2024-07-04T17:30:45.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.671+0000 7fb463e00640 1 -- 172.21.15.42:0/3886287937 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb45c035030 con 0x7fb46c074200 2024-07-04T17:30:45.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.672+0000 7fb471000640 1 -- 172.21.15.42:0/3886287937 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb46c1dc8f0 con 0x7fb46c074200 2024-07-04T17:30:45.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.673+0000 7fb471000640 1 -- 172.21.15.42:0/3886287937 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb46c0b1020 con 0x7fb46c074200 2024-07-04T17:30:45.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.673+0000 7fb463e00640 1 -- 172.21.15.42:0/3886287937 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7fb45c042070 con 0x7fb46c074200 2024-07-04T17:30:45.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.674+0000 7fb463e00640 1 --2- 172.21.15.42:0/3886287937 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fb44803d520 0x7fb44803fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:45.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.674+0000 7fb463e00640 1 -- 172.21.15.42:0/3886287937 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 2957+0+0 (secure 0 0 0) 0x7fb45c072640 con 0x7fb46c074200 2024-07-04T17:30:45.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.674+0000 7fb462a00640 1 --2- 172.21.15.42:0/3886287937 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fb44803d520 0x7fb44803fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:45.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.675+0000 7fb462a00640 1 --2- 172.21.15.42:0/3886287937 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fb44803d520 0x7fb44803fa30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb454007fd0 tx=0x7fb4540078e0 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:45.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.680+0000 7fb463e00640 1 -- 172.21.15.42:0/3886287937 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fb45c003c80 con 0x7fb46c074200 2024-07-04T17:30:45.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:45.887+0000 7fb471000640 1 -- 172.21.15.42:0/3886287937 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb46c068e30 con 0x7fb46c074200 2024-07-04T17:30:46.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.359+0000 7fb463e00640 1 -- 172.21.15.42:0/3886287937 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v70) ==== 170+0+0 (secure 0 0 0) 0x7fb45c003760 con 0x7fb46c074200 2024-07-04T17:30:46.361 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' created 2024-07-04T17:30:46.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.363+0000 7fb471000640 1 -- 172.21.15.42:0/3886287937 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fb44803d520 msgr2=0x7fb44803fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:46.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.363+0000 7fb471000640 1 --2- 172.21.15.42:0/3886287937 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fb44803d520 0x7fb44803fa30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb454007fd0 tx=0x7fb4540078e0 comp rx=0 tx=0).stop 2024-07-04T17:30:46.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.363+0000 7fb471000640 1 -- 172.21.15.42:0/3886287937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb46c074200 msgr2=0x7fb46c144d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:46.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.363+0000 7fb471000640 1 --2- 172.21.15.42:0/3886287937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb46c074200 0x7fb46c144d30 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009460 tx=0x7fb45c0049d0 comp rx=0 tx=0).stop 2024-07-04T17:30:46.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.363+0000 7fb471000640 1 -- 172.21.15.42:0/3886287937 shutdown_connections 2024-07-04T17:30:46.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.363+0000 7fb471000640 1 --2- 172.21.15.42:0/3886287937 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fb44803d520 0x7fb44803fa30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:46.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.363+0000 7fb471000640 1 --2- 172.21.15.42:0/3886287937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb46c074200 0x7fb46c144d30 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:46.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.363+0000 7fb471000640 1 -- 172.21.15.42:0/3886287937 >> 172.21.15.42:0/3886287937 conn(0x7fb46c06c020 msgr2=0x7fb46c0b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:46.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.365+0000 7fb471000640 1 -- 172.21.15.42:0/3886287937 shutdown_connections 2024-07-04T17:30:46.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.365+0000 7fb471000640 1 -- 172.21.15.42:0/3886287937 wait complete. 2024-07-04T17:30:46.416 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-04T17:30:46.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.678+0000 7f7c4fe00640 1 -- 172.21.15.42:0/1986553423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c500701d0 msgr2=0x7f7c500abe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:46.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.678+0000 7f7c4fe00640 1 --2- 172.21.15.42:0/1986553423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c500701d0 0x7f7c500abe40 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f7c40005fd0 tx=0x7f7c4002a5f0 comp rx=0 tx=0).stop 2024-07-04T17:30:46.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.678+0000 7f7c4fe00640 1 -- 172.21.15.42:0/1986553423 shutdown_connections 2024-07-04T17:30:46.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.678+0000 7f7c4fe00640 1 --2- 172.21.15.42:0/1986553423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c500701d0 0x7f7c500abe40 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:46.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.678+0000 7f7c4fe00640 1 -- 172.21.15.42:0/1986553423 >> 172.21.15.42:0/1986553423 conn(0x7f7c500afaa0 msgr2=0x7f7c500afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:46.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.678+0000 7f7c4fe00640 1 -- 172.21.15.42:0/1986553423 shutdown_connections 2024-07-04T17:30:46.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.678+0000 7f7c4fe00640 1 -- 172.21.15.42:0/1986553423 wait complete. 2024-07-04T17:30:46.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.680+0000 7f7c4fe00640 1 Processor -- start 2024-07-04T17:30:46.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.680+0000 7f7c4fe00640 1 -- start start 2024-07-04T17:30:46.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.681+0000 7f7c4fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c500701d0 0x7f7c50149dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:46.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.681+0000 7f7c4fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7c500ab190 con 0x7f7c500701d0 2024-07-04T17:30:46.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.682+0000 7f7c4ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c500701d0 0x7f7c50149dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:46.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.682+0000 7f7c4ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c500701d0 0x7f7c50149dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39124/0 (socket says 172.21.15.42:39124) 2024-07-04T17:30:46.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.682+0000 7f7c4ea00640 1 -- 172.21.15.42:0/4148729228 learned_addr learned my addr 172.21.15.42:0/4148729228 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:46.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.682+0000 7f7c4ea00640 1 -- 172.21.15.42:0/4148729228 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c5014aaf0 con 0x7f7c500701d0 2024-07-04T17:30:46.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.683+0000 7f7c4ea00640 1 --2- 172.21.15.42:0/4148729228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c500701d0 0x7f7c50149dd0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009850 tx=0x7f7c400049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:46.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.683+0000 7f7c46a00640 1 -- 172.21.15.42:0/4148729228 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c4003e030 con 0x7f7c500701d0 2024-07-04T17:30:46.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.683+0000 7f7c46a00640 1 -- 172.21.15.42:0/4148729228 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c400035f0 con 0x7f7c500701d0 2024-07-04T17:30:46.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.683+0000 7f7c4fe00640 1 -- 172.21.15.42:0/4148729228 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c5014b8f0 con 0x7f7c500701d0 2024-07-04T17:30:46.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.683+0000 7f7c46a00640 1 -- 172.21.15.42:0/4148729228 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c40035040 con 0x7f7c500701d0 2024-07-04T17:30:46.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.683+0000 7f7c4fe00640 1 -- 172.21.15.42:0/4148729228 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c5014f1d0 con 0x7f7c500701d0 2024-07-04T17:30:46.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.684+0000 7f7c4fe00640 1 -- 172.21.15.42:0/4148729228 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c500abe40 con 0x7f7c500701d0 2024-07-04T17:30:46.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.685+0000 7f7c46a00640 1 -- 172.21.15.42:0/4148729228 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f7c40041030 con 0x7f7c500701d0 2024-07-04T17:30:46.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.685+0000 7f7c46a00640 1 --2- 172.21.15.42:0/4148729228 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f7c2c03d4f0 0x7f7c2c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:46.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.685+0000 7f7c46a00640 1 -- 172.21.15.42:0/4148729228 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 3339+0+0 (secure 0 0 0) 0x7f7c4003f070 con 0x7f7c500701d0 2024-07-04T17:30:46.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.686+0000 7f7c47e00640 1 --2- 172.21.15.42:0/4148729228 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f7c2c03d4f0 0x7f7c2c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:46.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.687+0000 7f7c47e00640 1 --2- 172.21.15.42:0/4148729228 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f7c2c03d4f0 0x7f7c2c03fa00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7c38000f60 tx=0x7f7c38007a30 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:46.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.691+0000 7f7c46a00640 1 -- 172.21.15.42:0/4148729228 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f7c4004bd20 con 0x7f7c500701d0 2024-07-04T17:30:46.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:46.944+0000 7f7c4fe00640 1 -- 172.21.15.42:0/4148729228 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f7c500aa6e0 con 0x7f7c500701d0 2024-07-04T17:30:47.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.361+0000 7f7c46a00640 1 -- 172.21.15.42:0/4148729228 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v50) ==== 174+0+0 (secure 0 0 0) 0x7f7c40072090 con 0x7f7c500701d0 2024-07-04T17:30:47.363 INFO:teuthology.orchestra.run.smithi042.stderr:new fs with metadata pool 12 and data pool 13 2024-07-04T17:30:47.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.366+0000 7f7c4fe00640 1 -- 172.21.15.42:0/4148729228 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f7c2c03d4f0 msgr2=0x7f7c2c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:47.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.366+0000 7f7c4fe00640 1 --2- 172.21.15.42:0/4148729228 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f7c2c03d4f0 0x7f7c2c03fa00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7c38000f60 tx=0x7f7c38007a30 comp rx=0 tx=0).stop 2024-07-04T17:30:47.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.366+0000 7f7c4fe00640 1 -- 172.21.15.42:0/4148729228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c500701d0 msgr2=0x7f7c50149dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:47.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.366+0000 7f7c4fe00640 1 --2- 172.21.15.42:0/4148729228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c500701d0 0x7f7c50149dd0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009850 tx=0x7f7c400049d0 comp rx=0 tx=0).stop 2024-07-04T17:30:47.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.367+0000 7f7c4fe00640 1 -- 172.21.15.42:0/4148729228 shutdown_connections 2024-07-04T17:30:47.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.367+0000 7f7c4fe00640 1 --2- 172.21.15.42:0/4148729228 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f7c2c03d4f0 0x7f7c2c03fa00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:47.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.367+0000 7f7c4fe00640 1 --2- 172.21.15.42:0/4148729228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c500701d0 0x7f7c50149dd0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:47.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.367+0000 7f7c4fe00640 1 -- 172.21.15.42:0/4148729228 >> 172.21.15.42:0/4148729228 conn(0x7f7c500afaa0 msgr2=0x7f7c500b36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:47.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.367+0000 7f7c4fe00640 1 -- 172.21.15.42:0/4148729228 shutdown_connections 2024-07-04T17:30:47.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.368+0000 7f7c4fe00640 1 -- 172.21.15.42:0/4148729228 wait complete. 2024-07-04T17:30:47.433 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:30:47.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.690+0000 7fb0a8200640 1 -- 172.21.15.42:0/3183509500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0a0140e70 msgr2=0x7fb0a00bb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:47.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.690+0000 7fb0a8200640 1 --2- 172.21.15.42:0/3183509500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0a0140e70 0x7fb0a00bb1f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fb0900097b0 tx=0x7fb09002a530 comp rx=0 tx=0).stop 2024-07-04T17:30:47.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.691+0000 7fb0a8200640 1 -- 172.21.15.42:0/3183509500 shutdown_connections 2024-07-04T17:30:47.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.691+0000 7fb0a8200640 1 --2- 172.21.15.42:0/3183509500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0a0140e70 0x7fb0a00bb1f0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:47.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.691+0000 7fb0a8200640 1 -- 172.21.15.42:0/3183509500 >> 172.21.15.42:0/3183509500 conn(0x7fb0a00bf700 msgr2=0x7fb0a00af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:47.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.691+0000 7fb0a8200640 1 -- 172.21.15.42:0/3183509500 shutdown_connections 2024-07-04T17:30:47.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.691+0000 7fb0a8200640 1 -- 172.21.15.42:0/3183509500 wait complete. 2024-07-04T17:30:47.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.693+0000 7fb0a8200640 1 Processor -- start 2024-07-04T17:30:47.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.693+0000 7fb0a8200640 1 -- start start 2024-07-04T17:30:47.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.694+0000 7fb0a8200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0a00a9fa0 0x7fb0a013c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:47.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.694+0000 7fb0a8200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a00aae30 con 0x7fb0a00a9fa0 2024-07-04T17:30:47.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.695+0000 7fb0a5800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0a00a9fa0 0x7fb0a013c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:47.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.695+0000 7fb0a5800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0a00a9fa0 0x7fb0a013c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39140/0 (socket says 172.21.15.42:39140) 2024-07-04T17:30:47.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.695+0000 7fb0a5800640 1 -- 172.21.15.42:0/1986029817 learned_addr learned my addr 172.21.15.42:0/1986029817 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:47.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.695+0000 7fb0a5800640 1 -- 172.21.15.42:0/1986029817 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a00b9620 con 0x7fb0a00a9fa0 2024-07-04T17:30:47.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.696+0000 7fb0a5800640 1 --2- 172.21.15.42:0/1986029817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0a00a9fa0 0x7fb0a013c6d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fb0900098d0 tx=0x7fb090033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:47.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.696+0000 7fb09d600640 1 -- 172.21.15.42:0/1986029817 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb090031880 con 0x7fb0a00a9fa0 2024-07-04T17:30:47.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.696+0000 7fb09d600640 1 -- 172.21.15.42:0/1986029817 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb090031b10 con 0x7fb0a00a9fa0 2024-07-04T17:30:47.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.696+0000 7fb09d600640 1 -- 172.21.15.42:0/1986029817 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb090031df0 con 0x7fb0a00a9fa0 2024-07-04T17:30:47.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.696+0000 7fb0a8200640 1 -- 172.21.15.42:0/1986029817 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0a01d9050 con 0x7fb0a00a9fa0 2024-07-04T17:30:47.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.696+0000 7fb0a8200640 1 -- 172.21.15.42:0/1986029817 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0a00ad0f0 con 0x7fb0a00a9fa0 2024-07-04T17:30:47.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.697+0000 7fb0a8200640 1 -- 172.21.15.42:0/1986029817 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0a00bb1f0 con 0x7fb0a00a9fa0 2024-07-04T17:30:47.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.698+0000 7fb09d600640 1 -- 172.21.15.42:0/1986029817 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7fb0900330d0 con 0x7fb0a00a9fa0 2024-07-04T17:30:47.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.698+0000 7fb09d600640 1 --2- 172.21.15.42:0/1986029817 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fb08003d4f0 0x7fb08003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:47.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.698+0000 7fb09d600640 1 -- 172.21.15.42:0/1986029817 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 3439+0+0 (secure 0 0 0) 0x7fb090072d10 con 0x7fb0a00a9fa0 2024-07-04T17:30:47.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.698+0000 7fb0a4e00640 1 --2- 172.21.15.42:0/1986029817 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fb08003d4f0 0x7fb08003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:47.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.699+0000 7fb0a4e00640 1 --2- 172.21.15.42:0/1986029817 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fb08003d4f0 0x7fb08003fa00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb08c007fd0 tx=0x7fb08c007920 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:47.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.703+0000 7fb09d600640 1 -- 172.21.15.42:0/1986029817 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fb09003f030 con 0x7fb0a00a9fa0 2024-07-04T17:30:47.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.907+0000 7fb0a8200640 1 -- 172.21.15.42:0/1986029817 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb0a0069b70 con 0x7fb0a00a9fa0 2024-07-04T17:30:47.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.910+0000 7fb09d600640 1 -- 172.21.15.42:0/1986029817 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+10619 (secure 0 0 0) 0x7fb09003a3a0 con 0x7fb0a00a9fa0 2024-07-04T17:30:47.911 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:30:47.911 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":71,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:30:46.947058+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:30:45.017030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.4099999666213989,"score_stable":1.4099999666213989,"optimal_score":1,"raw_score_acting":1.4099999666213989,"raw_score_stable":1.4099999666213989,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-07-04T17:30:45.910003+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:30:47.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.913+0000 7fb0a8200640 1 -- 172.21.15.42:0/1986029817 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fb08003d4f0 msgr2=0x7fb08003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:47.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.913+0000 7fb0a8200640 1 --2- 172.21.15.42:0/1986029817 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fb08003d4f0 0x7fb08003fa00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb08c007fd0 tx=0x7fb08c007920 comp rx=0 tx=0).stop 2024-07-04T17:30:47.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.913+0000 7fb0a8200640 1 -- 172.21.15.42:0/1986029817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0a00a9fa0 msgr2=0x7fb0a013c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:47.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.913+0000 7fb0a8200640 1 --2- 172.21.15.42:0/1986029817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0a00a9fa0 0x7fb0a013c6d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fb0900098d0 tx=0x7fb090033ad0 comp rx=0 tx=0).stop 2024-07-04T17:30:47.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.913+0000 7fb0a8200640 1 -- 172.21.15.42:0/1986029817 shutdown_connections 2024-07-04T17:30:47.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.913+0000 7fb0a8200640 1 --2- 172.21.15.42:0/1986029817 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fb08003d4f0 0x7fb08003fa00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:47.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.913+0000 7fb0a8200640 1 --2- 172.21.15.42:0/1986029817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0a00a9fa0 0x7fb0a013c6d0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:47.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.913+0000 7fb0a8200640 1 -- 172.21.15.42:0/1986029817 >> 172.21.15.42:0/1986029817 conn(0x7fb0a00bf700 msgr2=0x7fb0a00b0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:47.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.913+0000 7fb0a8200640 1 -- 172.21.15.42:0/1986029817 shutdown_connections 2024-07-04T17:30:47.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:47.913+0000 7fb0a8200640 1 -- 172.21.15.42:0/1986029817 wait complete. 2024-07-04T17:30:47.940 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:30:48.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.219+0000 7f4fb2000640 1 -- 172.21.15.42:0/2848082763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fac0b1d20 msgr2=0x7f4fac0b6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:48.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.219+0000 7f4fb2000640 1 --2- 172.21.15.42:0/2848082763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fac0b1d20 0x7f4fac0b6230 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f4f940097b0 tx=0x7f4f9402a540 comp rx=0 tx=0).stop 2024-07-04T17:30:48.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.220+0000 7f4fb2000640 1 -- 172.21.15.42:0/2848082763 shutdown_connections 2024-07-04T17:30:48.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.220+0000 7f4fb2000640 1 --2- 172.21.15.42:0/2848082763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fac0b1d20 0x7f4fac0b6230 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:48.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.220+0000 7f4fb2000640 1 -- 172.21.15.42:0/2848082763 >> 172.21.15.42:0/2848082763 conn(0x7f4fac0bebc0 msgr2=0x7f4fac0befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:48.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.220+0000 7f4fb2000640 1 -- 172.21.15.42:0/2848082763 shutdown_connections 2024-07-04T17:30:48.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.220+0000 7f4fb2000640 1 -- 172.21.15.42:0/2848082763 wait complete. 2024-07-04T17:30:48.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.222+0000 7f4fb2000640 1 Processor -- start 2024-07-04T17:30:48.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.222+0000 7f4fb2000640 1 -- start start 2024-07-04T17:30:48.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.223+0000 7f4fb2000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fac0b1d20 0x7f4fac1428f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:48.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.224+0000 7f4fb2000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4fac0b8b40 con 0x7f4fac0b1d20 2024-07-04T17:30:48.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.224+0000 7f4fb0c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fac0b1d20 0x7f4fac1428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:48.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.224+0000 7f4fb0c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fac0b1d20 0x7f4fac1428f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39170/0 (socket says 172.21.15.42:39170) 2024-07-04T17:30:48.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.224+0000 7f4fb0c00640 1 -- 172.21.15.42:0/1519283686 learned_addr learned my addr 172.21.15.42:0/1519283686 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:48.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.224+0000 7f4fb0c00640 1 -- 172.21.15.42:0/1519283686 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fac1d3990 con 0x7f4fac0b1d20 2024-07-04T17:30:48.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.225+0000 7f4fb0c00640 1 --2- 172.21.15.42:0/1519283686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fac0b1d20 0x7f4fac1428f0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c00c6b0 tx=0x7f4f9c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:48.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.225+0000 7f4fa9600640 1 -- 172.21.15.42:0/1519283686 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f9c016020 con 0x7f4fac0b1d20 2024-07-04T17:30:48.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.225+0000 7f4fa9600640 1 -- 172.21.15.42:0/1519283686 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f9c00a5c0 con 0x7f4fac0b1d20 2024-07-04T17:30:48.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.225+0000 7f4fb2000640 1 -- 172.21.15.42:0/1519283686 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fac1d5090 con 0x7f4fac0b1d20 2024-07-04T17:30:48.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.225+0000 7f4fa9600640 1 -- 172.21.15.42:0/1519283686 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f9c00a960 con 0x7f4fac0b1d20 2024-07-04T17:30:48.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.225+0000 7f4fb2000640 1 -- 172.21.15.42:0/1519283686 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fac1d8c10 con 0x7f4fac0b1d20 2024-07-04T17:30:48.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.226+0000 7f4fb2000640 1 -- 172.21.15.42:0/1519283686 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fac0b91d0 con 0x7f4fac0b1d20 2024-07-04T17:30:48.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.226+0000 7f4fa9600640 1 -- 172.21.15.42:0/1519283686 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f4f9c019020 con 0x7f4fac0b1d20 2024-07-04T17:30:48.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.227+0000 7f4fa9600640 1 --2- 172.21.15.42:0/1519283686 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f4f8803d4f0 0x7f4f8803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:48.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.227+0000 7f4fa9600640 1 -- 172.21.15.42:0/1519283686 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 3439+0+0 (secure 0 0 0) 0x7f4f9c051560 con 0x7f4fac0b1d20 2024-07-04T17:30:48.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.227+0000 7f4fa3e00640 1 --2- 172.21.15.42:0/1519283686 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f4f8803d4f0 0x7f4f8803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:48.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.228+0000 7f4fa3e00640 1 --2- 172.21.15.42:0/1519283686 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f4f8803d4f0 0x7f4f8803fa00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4f94034000 tx=0x7f4f94004730 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:48.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.232+0000 7f4fa9600640 1 -- 172.21.15.42:0/1519283686 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f4f9c020030 con 0x7f4fac0b1d20 2024-07-04T17:30:48.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.450+0000 7f4fb2000640 1 -- 172.21.15.42:0/1519283686 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4fac068e30 con 0x7f4fac0b1d20 2024-07-04T17:30:48.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.452+0000 7f4fa9600640 1 -- 172.21.15.42:0/1519283686 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+10606 (secure 0 0 0) 0x7f4fac13ec10 con 0x7f4fac0b1d20 2024-07-04T17:30:48.454 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:30:48.454 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":72,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:30:48.375893+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:30:45.017030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.4099999666213989,"score_stable":1.4099999666213989,"optimal_score":1,"raw_score_acting":1.4099999666213989,"raw_score_stable":1.4099999666213989,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-07-04T17:30:45.910003+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:30:48.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.455+0000 7f4fb2000640 1 -- 172.21.15.42:0/1519283686 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f4f8803d4f0 msgr2=0x7f4f8803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:48.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.455+0000 7f4fb2000640 1 --2- 172.21.15.42:0/1519283686 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f4f8803d4f0 0x7f4f8803fa00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4f94034000 tx=0x7f4f94004730 comp rx=0 tx=0).stop 2024-07-04T17:30:48.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.455+0000 7f4fb2000640 1 -- 172.21.15.42:0/1519283686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fac0b1d20 msgr2=0x7f4fac1428f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:48.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.455+0000 7f4fb2000640 1 --2- 172.21.15.42:0/1519283686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fac0b1d20 0x7f4fac1428f0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c00c6b0 tx=0x7f4f9c00cb70 comp rx=0 tx=0).stop 2024-07-04T17:30:48.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.455+0000 7f4fb2000640 1 -- 172.21.15.42:0/1519283686 shutdown_connections 2024-07-04T17:30:48.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.455+0000 7f4fb2000640 1 --2- 172.21.15.42:0/1519283686 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f4f8803d4f0 0x7f4f8803fa00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:48.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.455+0000 7f4fb2000640 1 --2- 172.21.15.42:0/1519283686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fac0b1d20 0x7f4fac1428f0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:48.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.455+0000 7f4fb2000640 1 -- 172.21.15.42:0/1519283686 >> 172.21.15.42:0/1519283686 conn(0x7f4fac0bebc0 msgr2=0x7f4fac0b6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:48.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.456+0000 7f4fb2000640 1 -- 172.21.15.42:0/1519283686 shutdown_connections 2024-07-04T17:30:48.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.456+0000 7f4fb2000640 1 -- 172.21.15.42:0/1519283686 wait complete. 2024-07-04T17:30:48.481 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-07-04T17:30:48.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.758+0000 7f4e8fe00640 1 -- 172.21.15.42:0/430411853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e900abe40 msgr2=0x7f4e900aa900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:48.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.758+0000 7f4e8fe00640 1 --2- 172.21.15.42:0/430411853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e900abe40 0x7f4e900aa900 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f4e80005fd0 tx=0x7f4e8002aac0 comp rx=0 tx=0).stop 2024-07-04T17:30:48.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.758+0000 7f4e8fe00640 1 -- 172.21.15.42:0/430411853 shutdown_connections 2024-07-04T17:30:48.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.758+0000 7f4e8fe00640 1 --2- 172.21.15.42:0/430411853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e900abe40 0x7f4e900aa900 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:48.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.758+0000 7f4e8fe00640 1 -- 172.21.15.42:0/430411853 >> 172.21.15.42:0/430411853 conn(0x7f4e900afaa0 msgr2=0x7f4e900afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:48.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.758+0000 7f4e8fe00640 1 -- 172.21.15.42:0/430411853 shutdown_connections 2024-07-04T17:30:48.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.758+0000 7f4e8fe00640 1 -- 172.21.15.42:0/430411853 wait complete. 2024-07-04T17:30:48.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.760+0000 7f4e8fe00640 1 Processor -- start 2024-07-04T17:30:48.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.761+0000 7f4e8fe00640 1 -- start start 2024-07-04T17:30:48.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.762+0000 7f4e8fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e900701d0 0x7f4e9014a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:48.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.762+0000 7f4e8fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4e90073ed0 con 0x7f4e900701d0 2024-07-04T17:30:48.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.762+0000 7f4e8ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e900701d0 0x7f4e9014a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:48.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.762+0000 7f4e8ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e900701d0 0x7f4e9014a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39194/0 (socket says 172.21.15.42:39194) 2024-07-04T17:30:48.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.762+0000 7f4e8ea00640 1 -- 172.21.15.42:0/706509984 learned_addr learned my addr 172.21.15.42:0/706509984 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:48.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.762+0000 7f4e8ea00640 1 -- 172.21.15.42:0/706509984 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e90148fa0 con 0x7f4e900701d0 2024-07-04T17:30:48.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.763+0000 7f4e8ea00640 1 --2- 172.21.15.42:0/706509984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e900701d0 0x7f4e9014a640 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f4e78001eb0 tx=0x7f4e7800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:48.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.763+0000 7f4e87400640 1 -- 172.21.15.42:0/706509984 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e78017070 con 0x7f4e900701d0 2024-07-04T17:30:48.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.763+0000 7f4e87400640 1 -- 172.21.15.42:0/706509984 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e780052f0 con 0x7f4e900701d0 2024-07-04T17:30:48.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.763+0000 7f4e8fe00640 1 -- 172.21.15.42:0/706509984 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e9014aec0 con 0x7f4e900701d0 2024-07-04T17:30:48.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.763+0000 7f4e87400640 1 -- 172.21.15.42:0/706509984 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e780035f0 con 0x7f4e900701d0 2024-07-04T17:30:48.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.763+0000 7f4e8fe00640 1 -- 172.21.15.42:0/706509984 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e9014ab80 con 0x7f4e900701d0 2024-07-04T17:30:48.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.764+0000 7f4e8fe00640 1 -- 172.21.15.42:0/706509984 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e9014e5e0 con 0x7f4e900701d0 2024-07-04T17:30:48.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.765+0000 7f4e87400640 1 -- 172.21.15.42:0/706509984 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f4e7801a070 con 0x7f4e900701d0 2024-07-04T17:30:48.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.765+0000 7f4e87400640 1 --2- 172.21.15.42:0/706509984 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f4e6c03d4f0 0x7f4e6c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:48.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.765+0000 7f4e87400640 1 -- 172.21.15.42:0/706509984 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 3439+0+0 (secure 0 0 0) 0x7f4e78050ad0 con 0x7f4e900701d0 2024-07-04T17:30:48.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.765+0000 7f4e8e000640 1 --2- 172.21.15.42:0/706509984 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f4e6c03d4f0 0x7f4e6c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:48.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.766+0000 7f4e8e000640 1 --2- 172.21.15.42:0/706509984 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f4e6c03d4f0 0x7f4e6c03fa00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4e80035000 tx=0x7f4e80004ab0 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:48.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:48.772+0000 7f4e87400640 1 -- 172.21.15.42:0/706509984 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f4e780102e0 con 0x7f4e900701d0 2024-07-04T17:30:49.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.017+0000 7f4e8fe00640 1 -- 172.21.15.42:0/706509984 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f4e900008d0 con 0x7f4e900701d0 2024-07-04T17:30:49.335 INFO:tasks.ceph.mds.e.smithi042.stderr:2024-07-04T17:30:49.334+0000 7f50a2200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-04T17:30:49.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.385+0000 7f4e87400640 1 -- 172.21.15.42:0/706509984 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v53) ==== 118+0+0 (secure 0 0 0) 0x7f4e7801be00 con 0x7f4e900701d0 2024-07-04T17:30:49.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.388+0000 7f4e8fe00640 1 -- 172.21.15.42:0/706509984 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f4e6c03d4f0 msgr2=0x7f4e6c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:49.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.388+0000 7f4e8fe00640 1 --2- 172.21.15.42:0/706509984 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f4e6c03d4f0 0x7f4e6c03fa00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4e80035000 tx=0x7f4e80004ab0 comp rx=0 tx=0).stop 2024-07-04T17:30:49.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.388+0000 7f4e8fe00640 1 -- 172.21.15.42:0/706509984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e900701d0 msgr2=0x7f4e9014a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:49.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.388+0000 7f4e8fe00640 1 --2- 172.21.15.42:0/706509984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e900701d0 0x7f4e9014a640 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f4e78001eb0 tx=0x7f4e7800c810 comp rx=0 tx=0).stop 2024-07-04T17:30:49.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.389+0000 7f4e8fe00640 1 -- 172.21.15.42:0/706509984 shutdown_connections 2024-07-04T17:30:49.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.389+0000 7f4e8fe00640 1 --2- 172.21.15.42:0/706509984 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f4e6c03d4f0 0x7f4e6c03fa00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:49.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.389+0000 7f4e8fe00640 1 --2- 172.21.15.42:0/706509984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e900701d0 0x7f4e9014a640 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:49.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.389+0000 7f4e8fe00640 1 -- 172.21.15.42:0/706509984 >> 172.21.15.42:0/706509984 conn(0x7f4e900afaa0 msgr2=0x7f4e90075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:49.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.389+0000 7f4e8fe00640 1 -- 172.21.15.42:0/706509984 shutdown_connections 2024-07-04T17:30:49.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.389+0000 7f4e8fe00640 1 -- 172.21.15.42:0/706509984 wait complete. 2024-07-04T17:30:49.421 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-04T17:30:49.421 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-07-04T17:30:49.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.718+0000 7fe9f2c00640 1 -- 172.21.15.42:0/1437449593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9ec0ac5b0 msgr2=0x7fe9ec0b0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:49.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.718+0000 7fe9f2c00640 1 --2- 172.21.15.42:0/1437449593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9ec0ac5b0 0x7fe9ec0b0db0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fe9d80097b0 tx=0x7fe9d802a4b0 comp rx=0 tx=0).stop 2024-07-04T17:30:49.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.718+0000 7fe9f2c00640 1 -- 172.21.15.42:0/1437449593 shutdown_connections 2024-07-04T17:30:49.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.718+0000 7fe9f2c00640 1 --2- 172.21.15.42:0/1437449593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9ec0ac5b0 0x7fe9ec0b0db0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:49.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.718+0000 7fe9f2c00640 1 -- 172.21.15.42:0/1437449593 >> 172.21.15.42:0/1437449593 conn(0x7fe9ec0afaa0 msgr2=0x7fe9ec0afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:49.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.718+0000 7fe9f2c00640 1 -- 172.21.15.42:0/1437449593 shutdown_connections 2024-07-04T17:30:49.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.718+0000 7fe9f2c00640 1 -- 172.21.15.42:0/1437449593 wait complete. 2024-07-04T17:30:49.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.721+0000 7fe9f2c00640 1 Processor -- start 2024-07-04T17:30:49.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.721+0000 7fe9f2c00640 1 -- start start 2024-07-04T17:30:49.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.723+0000 7fe9f2c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9ec06da60 0x7fe9ec0b82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:49.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.723+0000 7fe9f2c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec0ae590 con 0x7fe9ec06da60 2024-07-04T17:30:49.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.723+0000 7fe9f1800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9ec06da60 0x7fe9ec0b82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:49.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.723+0000 7fe9f1800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9ec06da60 0x7fe9ec0b82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39214/0 (socket says 172.21.15.42:39214) 2024-07-04T17:30:49.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.723+0000 7fe9f1800640 1 -- 172.21.15.42:0/2795661124 learned_addr learned my addr 172.21.15.42:0/2795661124 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:49.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.724+0000 7fe9f1800640 1 -- 172.21.15.42:0/2795661124 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ec13ad60 con 0x7fe9ec06da60 2024-07-04T17:30:49.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.724+0000 7fe9f1800640 1 --2- 172.21.15.42:0/2795661124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9ec06da60 0x7fe9ec0b82b0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fe9e000cb40 tx=0x7fe9e000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:49.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.724+0000 7fe9e9600640 1 -- 172.21.15.42:0/2795661124 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9e0016020 con 0x7fe9ec06da60 2024-07-04T17:30:49.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.724+0000 7fe9e9600640 1 -- 172.21.15.42:0/2795661124 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9e000abf0 con 0x7fe9ec06da60 2024-07-04T17:30:49.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.724+0000 7fe9f2c00640 1 -- 172.21.15.42:0/2795661124 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9ec0b8980 con 0x7fe9ec06da60 2024-07-04T17:30:49.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.724+0000 7fe9e9600640 1 -- 172.21.15.42:0/2795661124 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9e0012420 con 0x7fe9ec06da60 2024-07-04T17:30:49.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.724+0000 7fe9f2c00640 1 -- 172.21.15.42:0/2795661124 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9ec1dac60 con 0x7fe9ec06da60 2024-07-04T17:30:49.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.726+0000 7fe9f2c00640 1 -- 172.21.15.42:0/2795661124 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9ec0b0db0 con 0x7fe9ec06da60 2024-07-04T17:30:49.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.726+0000 7fe9e9600640 1 -- 172.21.15.42:0/2795661124 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7fe9e0019020 con 0x7fe9ec06da60 2024-07-04T17:30:49.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.727+0000 7fe9e9600640 1 --2- 172.21.15.42:0/2795661124 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fe9cc03d540 0x7fe9cc03fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:49.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.727+0000 7fe9e9600640 1 -- 172.21.15.42:0/2795661124 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 3439+0+0 (secure 0 0 0) 0x7fe9e0051bd0 con 0x7fe9ec06da60 2024-07-04T17:30:49.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.727+0000 7fe9f0e00640 1 --2- 172.21.15.42:0/2795661124 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fe9cc03d540 0x7fe9cc03fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:49.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.728+0000 7fe9f0e00640 1 --2- 172.21.15.42:0/2795661124 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fe9cc03d540 0x7fe9cc03fa50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8034000 tx=0x7fe9d8004730 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:49.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.732+0000 7fe9e9600640 1 -- 172.21.15.42:0/2795661124 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fe9e0020030 con 0x7fe9ec06da60 2024-07-04T17:30:49.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:49.988+0000 7fe9f2c00640 1 -- 172.21.15.42:0/2795661124 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe9ec068e30 con 0x7fe9ec06da60 2024-07-04T17:30:50.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.400+0000 7fe9e9600640 1 -- 172.21.15.42:0/2795661124 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v54) ==== 122+0+0 (secure 0 0 0) 0x7fe9e000d150 con 0x7fe9ec06da60 2024-07-04T17:30:50.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.404+0000 7fe9f2c00640 1 -- 172.21.15.42:0/2795661124 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fe9cc03d540 msgr2=0x7fe9cc03fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:50.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.404+0000 7fe9f2c00640 1 --2- 172.21.15.42:0/2795661124 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fe9cc03d540 0x7fe9cc03fa50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8034000 tx=0x7fe9d8004730 comp rx=0 tx=0).stop 2024-07-04T17:30:50.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.405+0000 7fe9f2c00640 1 -- 172.21.15.42:0/2795661124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9ec06da60 msgr2=0x7fe9ec0b82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:50.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.405+0000 7fe9f2c00640 1 --2- 172.21.15.42:0/2795661124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9ec06da60 0x7fe9ec0b82b0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fe9e000cb40 tx=0x7fe9e000cf40 comp rx=0 tx=0).stop 2024-07-04T17:30:50.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.405+0000 7fe9f2c00640 1 -- 172.21.15.42:0/2795661124 shutdown_connections 2024-07-04T17:30:50.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.405+0000 7fe9f2c00640 1 --2- 172.21.15.42:0/2795661124 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fe9cc03d540 0x7fe9cc03fa50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:50.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.405+0000 7fe9f2c00640 1 --2- 172.21.15.42:0/2795661124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9ec06da60 0x7fe9ec0b82b0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:50.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.405+0000 7fe9f2c00640 1 -- 172.21.15.42:0/2795661124 >> 172.21.15.42:0/2795661124 conn(0x7fe9ec0afaa0 msgr2=0x7fe9ec0aebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:50.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.405+0000 7fe9f2c00640 1 -- 172.21.15.42:0/2795661124 shutdown_connections 2024-07-04T17:30:50.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.405+0000 7fe9f2c00640 1 -- 172.21.15.42:0/2795661124 wait complete. 2024-07-04T17:30:50.442 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-07-04T17:30:50.443 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:30:50.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.745+0000 7f8662e00640 1 -- 172.21.15.42:0/3839170228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f865c0b3030 msgr2=0x7f865c0b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:50.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.745+0000 7f8662e00640 1 --2- 172.21.15.42:0/3839170228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f865c0b3030 0x7f865c0b01f0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f8648005fd0 tx=0x7f864802b010 comp rx=0 tx=0).stop 2024-07-04T17:30:50.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.745+0000 7f8662e00640 1 -- 172.21.15.42:0/3839170228 shutdown_connections 2024-07-04T17:30:50.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.745+0000 7f8662e00640 1 --2- 172.21.15.42:0/3839170228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f865c0b3030 0x7f865c0b01f0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:50.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.745+0000 7f8662e00640 1 -- 172.21.15.42:0/3839170228 >> 172.21.15.42:0/3839170228 conn(0x7f865c0bf700 msgr2=0x7f865c0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:50.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.745+0000 7f8662e00640 1 -- 172.21.15.42:0/3839170228 shutdown_connections 2024-07-04T17:30:50.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.745+0000 7f8662e00640 1 -- 172.21.15.42:0/3839170228 wait complete. 2024-07-04T17:30:50.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.747+0000 7f8662e00640 1 Processor -- start 2024-07-04T17:30:50.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.747+0000 7f8662e00640 1 -- start start 2024-07-04T17:30:50.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.749+0000 7f8662e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f865c06cea0 0x7f865c1d7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:50.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.749+0000 7f8662e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f865c06ef50 con 0x7f865c06cea0 2024-07-04T17:30:50.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.749+0000 7f865be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f865c06cea0 0x7f865c1d7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:50.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.749+0000 7f865be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f865c06cea0 0x7f865c1d7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39234/0 (socket says 172.21.15.42:39234) 2024-07-04T17:30:50.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.749+0000 7f865be00640 1 -- 172.21.15.42:0/3389391775 learned_addr learned my addr 172.21.15.42:0/3389391775 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:50.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.749+0000 7f865be00640 1 -- 172.21.15.42:0/3389391775 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865c0710d0 con 0x7f865c06cea0 2024-07-04T17:30:50.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.750+0000 7f865be00640 1 --2- 172.21.15.42:0/3389391775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f865c06cea0 0x7f865c1d7310 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f864c00c380 tx=0x7f864c00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:50.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.750+0000 7f8653e00640 1 -- 172.21.15.42:0/3389391775 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f864c017070 con 0x7f865c06cea0 2024-07-04T17:30:50.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.750+0000 7f8653e00640 1 -- 172.21.15.42:0/3389391775 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f864c0052f0 con 0x7f865c06cea0 2024-07-04T17:30:50.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.750+0000 7f8653e00640 1 -- 172.21.15.42:0/3389391775 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f864c0035f0 con 0x7f865c06cea0 2024-07-04T17:30:50.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.750+0000 7f8662e00640 1 -- 172.21.15.42:0/3389391775 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f865c1d95e0 con 0x7f865c06cea0 2024-07-04T17:30:50.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.750+0000 7f8662e00640 1 -- 172.21.15.42:0/3389391775 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f865c0ae1d0 con 0x7f865c06cea0 2024-07-04T17:30:50.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.751+0000 7f8662e00640 1 -- 172.21.15.42:0/3389391775 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f865c0b01f0 con 0x7f865c06cea0 2024-07-04T17:30:50.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.752+0000 7f8653e00640 1 -- 172.21.15.42:0/3389391775 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f864c01a070 con 0x7f865c06cea0 2024-07-04T17:30:50.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.752+0000 7f8653e00640 1 --2- 172.21.15.42:0/3389391775 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f863c03d4f0 0x7f863c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:50.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.752+0000 7f8653e00640 1 -- 172.21.15.42:0/3389391775 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 3439+0+0 (secure 0 0 0) 0x7f864c050c40 con 0x7f865c06cea0 2024-07-04T17:30:50.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.752+0000 7f865b400640 1 --2- 172.21.15.42:0/3389391775 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f863c03d4f0 0x7f863c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:50.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.753+0000 7f865b400640 1 --2- 172.21.15.42:0/3389391775 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f863c03d4f0 0x7f863c03fa00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8648035000 tx=0x7f8648004ab0 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:50.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:50.757+0000 7f8653e00640 1 -- 172.21.15.42:0/3389391775 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f864c016170 con 0x7f865c06cea0 2024-07-04T17:30:51.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.005+0000 7f8662e00640 1 -- 172.21.15.42:0/3389391775 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f865c0ae400 con 0x7f865c06cea0 2024-07-04T17:30:51.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.006+0000 7f8653e00640 1 -- 172.21.15.42:0/3389391775 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+6050 (secure 0 0 0) 0x7f864c0192c0 con 0x7f865c06cea0 2024-07-04T17:30:51.008 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:30:51.008 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":54,"btime":"2024-07-04T17:30:50:394659+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5017,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/2312272332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2312272332},{"type":"v1","addr":"172.21.15.42:6827","nonce":2312272332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5020,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1285161495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1285161495},{"type":"v1","addr":"172.21.15.42:6829","nonce":1285161495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5023,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/1583070649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1583070649},{"type":"v1","addr":"172.21.15.42:6831","nonce":1583070649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5026,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6833/2038326930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2038326930},{"type":"v1","addr":"172.21.15.42:6833","nonce":2038326930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:30:46.947014+0000","modified":"2024-07-04T17:30:49.991386+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5029":{"gid":5029,"name":"e","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6835/1458267161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1458267161},{"type":"v1","addr":"172.21.15.42:6835","nonce":1458267161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5029,"qdb_cluster":[5029]},"id":6}]} 2024-07-04T17:30:51.008 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 54 2024-07-04T17:30:51.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.009+0000 7f8662e00640 1 -- 172.21.15.42:0/3389391775 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f863c03d4f0 msgr2=0x7f863c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:51.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.009+0000 7f8662e00640 1 --2- 172.21.15.42:0/3389391775 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f863c03d4f0 0x7f863c03fa00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8648035000 tx=0x7f8648004ab0 comp rx=0 tx=0).stop 2024-07-04T17:30:51.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.009+0000 7f8662e00640 1 -- 172.21.15.42:0/3389391775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f865c06cea0 msgr2=0x7f865c1d7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:51.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.009+0000 7f8662e00640 1 --2- 172.21.15.42:0/3389391775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f865c06cea0 0x7f865c1d7310 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f864c00c380 tx=0x7f864c00c840 comp rx=0 tx=0).stop 2024-07-04T17:30:51.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.009+0000 7f8662e00640 1 -- 172.21.15.42:0/3389391775 shutdown_connections 2024-07-04T17:30:51.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.009+0000 7f8662e00640 1 --2- 172.21.15.42:0/3389391775 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f863c03d4f0 0x7f863c03fa00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:51.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.009+0000 7f8662e00640 1 --2- 172.21.15.42:0/3389391775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f865c06cea0 0x7f865c1d7310 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:51.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.009+0000 7f8662e00640 1 -- 172.21.15.42:0/3389391775 >> 172.21.15.42:0/3389391775 conn(0x7f865c0bf700 msgr2=0x7f865c1492d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:51.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.010+0000 7f8662e00640 1 -- 172.21.15.42:0/3389391775 shutdown_connections 2024-07-04T17:30:51.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.010+0000 7f8662e00640 1 -- 172.21.15.42:0/3389391775 wait complete. 2024-07-04T17:30:51.035 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:30:51.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.319+0000 7fd6e9c00640 1 -- 172.21.15.42:0/3859030030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6e40adac0 msgr2=0x7fd6e406e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:51.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.319+0000 7fd6e9c00640 1 --2- 172.21.15.42:0/3859030030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6e40adac0 0x7fd6e406e120 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc005fd0 tx=0x7fd6cc02a8b0 comp rx=0 tx=0).stop 2024-07-04T17:30:51.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.319+0000 7fd6e9c00640 1 -- 172.21.15.42:0/3859030030 shutdown_connections 2024-07-04T17:30:51.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.319+0000 7fd6e9c00640 1 --2- 172.21.15.42:0/3859030030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6e40adac0 0x7fd6e406e120 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:51.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.319+0000 7fd6e9c00640 1 -- 172.21.15.42:0/3859030030 >> 172.21.15.42:0/3859030030 conn(0x7fd6e4071960 msgr2=0x7fd6e406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:51.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.319+0000 7fd6e9c00640 1 -- 172.21.15.42:0/3859030030 shutdown_connections 2024-07-04T17:30:51.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.319+0000 7fd6e9c00640 1 -- 172.21.15.42:0/3859030030 wait complete. 2024-07-04T17:30:51.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.322+0000 7fd6e9c00640 1 Processor -- start 2024-07-04T17:30:51.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.322+0000 7fd6e9c00640 1 -- start start 2024-07-04T17:30:51.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.323+0000 7fd6e9c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6e40adac0 0x7fd6e41d8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:51.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.323+0000 7fd6e9c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e406e660 con 0x7fd6e40adac0 2024-07-04T17:30:51.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.323+0000 7fd6e3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6e40adac0 0x7fd6e41d8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:51.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.323+0000 7fd6e3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6e40adac0 0x7fd6e41d8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39248/0 (socket says 172.21.15.42:39248) 2024-07-04T17:30:51.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.323+0000 7fd6e3e00640 1 -- 172.21.15.42:0/3494895747 learned_addr learned my addr 172.21.15.42:0/3494895747 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:51.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.324+0000 7fd6e3e00640 1 -- 172.21.15.42:0/3494895747 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e40b6310 con 0x7fd6e40adac0 2024-07-04T17:30:51.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.324+0000 7fd6e3e00640 1 --2- 172.21.15.42:0/3494895747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6e40adac0 0x7fd6e41d8d30 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fd6d400c380 tx=0x7fd6d400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:51.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.324+0000 7fd6dbe00640 1 -- 172.21.15.42:0/3494895747 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6d40107a0 con 0x7fd6e40adac0 2024-07-04T17:30:51.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.324+0000 7fd6dbe00640 1 -- 172.21.15.42:0/3494895747 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6d4017070 con 0x7fd6e40adac0 2024-07-04T17:30:51.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.324+0000 7fd6dbe00640 1 -- 172.21.15.42:0/3494895747 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6d4010c00 con 0x7fd6e40adac0 2024-07-04T17:30:51.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.324+0000 7fd6e9c00640 1 -- 172.21.15.42:0/3494895747 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e41e3550 con 0x7fd6e40adac0 2024-07-04T17:30:51.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.325+0000 7fd6e9c00640 1 -- 172.21.15.42:0/3494895747 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e40b1160 con 0x7fd6e40adac0 2024-07-04T17:30:51.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.326+0000 7fd6dbe00640 1 -- 172.21.15.42:0/3494895747 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7fd6d401c460 con 0x7fd6e40adac0 2024-07-04T17:30:51.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.326+0000 7fd6e9c00640 1 -- 172.21.15.42:0/3494895747 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6e40b1390 con 0x7fd6e40adac0 2024-07-04T17:30:51.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.326+0000 7fd6dbe00640 1 --2- 172.21.15.42:0/3494895747 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fd6c003d4f0 0x7fd6c003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:51.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.327+0000 7fd6dbe00640 1 -- 172.21.15.42:0/3494895747 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 3439+0+0 (secure 0 0 0) 0x7fd6d4051630 con 0x7fd6e40adac0 2024-07-04T17:30:51.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.327+0000 7fd6e3400640 1 --2- 172.21.15.42:0/3494895747 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fd6c003d4f0 0x7fd6c003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:51.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.328+0000 7fd6e3400640 1 --2- 172.21.15.42:0/3494895747 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fd6c003d4f0 0x7fd6c003fa00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc034000 tx=0x7fd6cc004ab0 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:51.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.333+0000 7fd6dbe00640 1 -- 172.21.15.42:0/3494895747 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fd6e40b1390 con 0x7fd6e40adac0 2024-07-04T17:30:51.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.546+0000 7fd6e9c00640 1 -- 172.21.15.42:0/3494895747 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6e406ed20 con 0x7fd6e40adac0 2024-07-04T17:30:51.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.549+0000 7fd6dbe00640 1 -- 172.21.15.42:0/3494895747 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+10606 (secure 0 0 0) 0x7fd6d401c800 con 0x7fd6e40adac0 2024-07-04T17:30:51.550 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:30:51.550 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":72,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:30:48.375893+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:30:45.017030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.4099999666213989,"score_stable":1.4099999666213989,"optimal_score":1,"raw_score_acting":1.4099999666213989,"raw_score_stable":1.4099999666213989,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-07-04T17:30:45.910003+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:30:51.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.551+0000 7fd6e9c00640 1 -- 172.21.15.42:0/3494895747 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fd6c003d4f0 msgr2=0x7fd6c003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:51.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.551+0000 7fd6e9c00640 1 --2- 172.21.15.42:0/3494895747 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fd6c003d4f0 0x7fd6c003fa00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc034000 tx=0x7fd6cc004ab0 comp rx=0 tx=0).stop 2024-07-04T17:30:51.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.551+0000 7fd6e9c00640 1 -- 172.21.15.42:0/3494895747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6e40adac0 msgr2=0x7fd6e41d8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:51.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.551+0000 7fd6e9c00640 1 --2- 172.21.15.42:0/3494895747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6e40adac0 0x7fd6e41d8d30 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fd6d400c380 tx=0x7fd6d400c840 comp rx=0 tx=0).stop 2024-07-04T17:30:51.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.552+0000 7fd6e9c00640 1 -- 172.21.15.42:0/3494895747 shutdown_connections 2024-07-04T17:30:51.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.552+0000 7fd6e9c00640 1 --2- 172.21.15.42:0/3494895747 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fd6c003d4f0 0x7fd6c003fa00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:51.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.552+0000 7fd6e9c00640 1 --2- 172.21.15.42:0/3494895747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6e40adac0 0x7fd6e41d8d30 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:51.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.552+0000 7fd6e9c00640 1 -- 172.21.15.42:0/3494895747 >> 172.21.15.42:0/3494895747 conn(0x7fd6e4071960 msgr2=0x7fd6e4150f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:51.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.552+0000 7fd6e9c00640 1 -- 172.21.15.42:0/3494895747 shutdown_connections 2024-07-04T17:30:51.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.552+0000 7fd6e9c00640 1 -- 172.21.15.42:0/3494895747 wait complete. 2024-07-04T17:30:51.581 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-04T17:30:51.581 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:30:51.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.852+0000 7f5011c00640 1 -- 172.21.15.42:0/4083542683 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f500c06cea0 msgr2=0x7f500c0b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:51.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.852+0000 7f5011c00640 1 --2- 172.21.15.42:0/4083542683 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f500c06cea0 0x7f500c0b3030 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc005fd0 tx=0x7f4ffc02ae80 comp rx=0 tx=0).stop 2024-07-04T17:30:51.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.852+0000 7f5011c00640 1 -- 172.21.15.42:0/4083542683 shutdown_connections 2024-07-04T17:30:51.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.852+0000 7f5011c00640 1 --2- 172.21.15.42:0/4083542683 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f500c06cea0 0x7f500c0b3030 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:51.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.852+0000 7f5011c00640 1 -- 172.21.15.42:0/4083542683 >> 172.21.15.42:0/4083542683 conn(0x7f500c0bf700 msgr2=0x7f500c0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:51.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.852+0000 7f5011c00640 1 -- 172.21.15.42:0/4083542683 shutdown_connections 2024-07-04T17:30:51.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.852+0000 7f5011c00640 1 -- 172.21.15.42:0/4083542683 wait complete. 2024-07-04T17:30:51.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.854+0000 7f5011c00640 1 Processor -- start 2024-07-04T17:30:51.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.854+0000 7f5011c00640 1 -- start start 2024-07-04T17:30:51.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.856+0000 7f5011c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f500c06cea0 0x7f500c0719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:51.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.856+0000 7f5011c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f500c06ef50 con 0x7f500c06cea0 2024-07-04T17:30:51.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.856+0000 7f500aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f500c06cea0 0x7f500c0719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:51.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.856+0000 7f500aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f500c06cea0 0x7f500c0719d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39278/0 (socket says 172.21.15.42:39278) 2024-07-04T17:30:51.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.856+0000 7f500aa00640 1 -- 172.21.15.42:0/3046354745 learned_addr learned my addr 172.21.15.42:0/3046354745 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:51.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.856+0000 7f500aa00640 1 -- 172.21.15.42:0/3046354745 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f500c0714f0 con 0x7f500c06cea0 2024-07-04T17:30:51.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.857+0000 7f500aa00640 1 --2- 172.21.15.42:0/3046354745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f500c06cea0 0x7f500c0719d0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc00b3c0 tx=0x7f4ffc02cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:51.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.857+0000 7f5002a00640 1 -- 172.21.15.42:0/3046354745 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ffc041020 con 0x7f500c06cea0 2024-07-04T17:30:51.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.857+0000 7f5002a00640 1 -- 172.21.15.42:0/3046354745 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ffc0095a0 con 0x7f500c06cea0 2024-07-04T17:30:51.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.857+0000 7f5002a00640 1 -- 172.21.15.42:0/3046354745 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ffc0098c0 con 0x7f500c06cea0 2024-07-04T17:30:51.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.857+0000 7f5011c00640 1 -- 172.21.15.42:0/3046354745 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f500c0acaf0 con 0x7f500c06cea0 2024-07-04T17:30:51.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.857+0000 7f5011c00640 1 -- 172.21.15.42:0/3046354745 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f500c071270 con 0x7f500c06cea0 2024-07-04T17:30:51.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.859+0000 7f5011c00640 1 -- 172.21.15.42:0/3046354745 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f500c063f70 con 0x7f500c06cea0 2024-07-04T17:30:51.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.859+0000 7f5002a00640 1 -- 172.21.15.42:0/3046354745 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f4ffc044020 con 0x7f500c06cea0 2024-07-04T17:30:51.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.859+0000 7f5002a00640 1 --2- 172.21.15.42:0/3046354745 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f4fec03d4a0 0x7f4fec03f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:51.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.860+0000 7f5002a00640 1 -- 172.21.15.42:0/3046354745 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 3439+0+0 (secure 0 0 0) 0x7f4ffc039040 con 0x7f500c06cea0 2024-07-04T17:30:51.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.860+0000 7f500a000640 1 --2- 172.21.15.42:0/3046354745 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f4fec03d4a0 0x7f4fec03f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:51.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.861+0000 7f500a000640 1 --2- 172.21.15.42:0/3046354745 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f4fec03d4a0 0x7f4fec03f9b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8000f60 tx=0x7f4ff8007a30 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:51.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:51.865+0000 7f5002a00640 1 -- 172.21.15.42:0/3046354745 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f500c063f70 con 0x7f500c06cea0 2024-07-04T17:30:52.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.074+0000 7f5011c00640 1 -- 172.21.15.42:0/3046354745 --> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f500c070ec0 con 0x7f4fec03d4a0 2024-07-04T17:30:52.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.081+0000 7f5002a00640 1 -- 172.21.15.42:0/3046354745 <== mgr.4906 v2:172.21.15.42:6824/3473020706 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246084 (secure 0 0 0) 0x7f500c070ec0 con 0x7f4fec03d4a0 2024-07-04T17:30:52.086 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:30:52.087 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:30:52.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.086+0000 7f5011c00640 1 -- 172.21.15.42:0/3046354745 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f4fec03d4a0 msgr2=0x7f4fec03f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:52.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.086+0000 7f5011c00640 1 --2- 172.21.15.42:0/3046354745 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f4fec03d4a0 0x7f4fec03f9b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8000f60 tx=0x7f4ff8007a30 comp rx=0 tx=0).stop 2024-07-04T17:30:52.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.086+0000 7f5011c00640 1 -- 172.21.15.42:0/3046354745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f500c06cea0 msgr2=0x7f500c0719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:52.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.086+0000 7f5011c00640 1 --2- 172.21.15.42:0/3046354745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f500c06cea0 0x7f500c0719d0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc00b3c0 tx=0x7f4ffc02cd70 comp rx=0 tx=0).stop 2024-07-04T17:30:52.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.086+0000 7f5011c00640 1 -- 172.21.15.42:0/3046354745 shutdown_connections 2024-07-04T17:30:52.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.086+0000 7f5011c00640 1 --2- 172.21.15.42:0/3046354745 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f4fec03d4a0 0x7f4fec03f9b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:52.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.087+0000 7f5011c00640 1 --2- 172.21.15.42:0/3046354745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f500c06cea0 0x7f500c0719d0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:52.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.087+0000 7f5011c00640 1 -- 172.21.15.42:0/3046354745 >> 172.21.15.42:0/3046354745 conn(0x7f500c0bf700 msgr2=0x7f500c14ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:52.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.087+0000 7f5011c00640 1 -- 172.21.15.42:0/3046354745 shutdown_connections 2024-07-04T17:30:52.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.087+0000 7f5011c00640 1 -- 172.21.15.42:0/3046354745 wait complete. 2024-07-04T17:30:52.114 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-07-04T17:30:51.627627+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":296,"num_read_kb":750,"num_write":172,"num_write_kb":1120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":105216,"kb_used_data":3336,"kb_used_omap":4,"kb_used_meta":101819,"kb_avail":283010304,"statfs":{"total":289910292480,"available":289802551296,"internally_reserved":0,"allocated":3416064,"data_stored":2031391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":104262991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001997"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.447256+0000","last_change":"2024-07-04T17:30:47.430443+0000","last_active":"2024-07-04T17:30:48.447256+0000","last_peered":"2024-07-04T17:30:48.447256+0000","last_clean":"2024-07-04T17:30:48.447256+0000","last_became_active":"2024-07-04T17:30:47.425325+0000","last_became_peered":"2024-07-04T17:30:47.425325+0000","last_unstale":"2024-07-04T17:30:48.447256+0000","last_undegraded":"2024-07-04T17:30:48.447256+0000","last_fullsized":"2024-07-04T17:30:48.447256+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:44:15.710215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.448048+0000","last_change":"2024-07-04T17:30:47.429677+0000","last_active":"2024-07-04T17:30:48.448048+0000","last_peered":"2024-07-04T17:30:48.448048+0000","last_clean":"2024-07-04T17:30:48.448048+0000","last_became_active":"2024-07-04T17:30:47.428028+0000","last_became_peered":"2024-07-04T17:30:47.428028+0000","last_unstale":"2024-07-04T17:30:48.448048+0000","last_undegraded":"2024-07-04T17:30:48.448048+0000","last_fullsized":"2024-07-04T17:30:48.448048+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:41:29.873067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.389700+0000","last_change":"2024-07-04T17:30:47.423800+0000","last_active":"2024-07-04T17:30:48.389700+0000","last_peered":"2024-07-04T17:30:48.389700+0000","last_clean":"2024-07-04T17:30:48.389700+0000","last_became_active":"2024-07-04T17:30:47.423361+0000","last_became_peered":"2024-07-04T17:30:47.423361+0000","last_unstale":"2024-07-04T17:30:48.389700+0000","last_undegraded":"2024-07-04T17:30:48.389700+0000","last_fullsized":"2024-07-04T17:30:48.389700+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:25:11.379938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.389171+0000","last_change":"2024-07-04T17:30:47.422048+0000","last_active":"2024-07-04T17:30:48.389171+0000","last_peered":"2024-07-04T17:30:48.389171+0000","last_clean":"2024-07-04T17:30:48.389171+0000","last_became_active":"2024-07-04T17:30:47.421591+0000","last_became_peered":"2024-07-04T17:30:47.421591+0000","last_unstale":"2024-07-04T17:30:48.389171+0000","last_undegraded":"2024-07-04T17:30:48.389171+0000","last_fullsized":"2024-07-04T17:30:48.389171+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:58:22.418379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.389433+0000","last_change":"2024-07-04T17:30:47.414496+0000","last_active":"2024-07-04T17:30:48.389433+0000","last_peered":"2024-07-04T17:30:48.389433+0000","last_clean":"2024-07-04T17:30:48.389433+0000","last_became_active":"2024-07-04T17:30:47.414141+0000","last_became_peered":"2024-07-04T17:30:47.414141+0000","last_unstale":"2024-07-04T17:30:48.389433+0000","last_undegraded":"2024-07-04T17:30:48.389433+0000","last_fullsized":"2024-07-04T17:30:48.389433+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:09:08.612743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.388769+0000","last_change":"2024-07-04T17:30:47.419292+0000","last_active":"2024-07-04T17:30:48.388769+0000","last_peered":"2024-07-04T17:30:48.388769+0000","last_clean":"2024-07-04T17:30:48.388769+0000","last_became_active":"2024-07-04T17:30:47.418053+0000","last_became_peered":"2024-07-04T17:30:47.418053+0000","last_unstale":"2024-07-04T17:30:48.388769+0000","last_undegraded":"2024-07-04T17:30:48.388769+0000","last_fullsized":"2024-07-04T17:30:48.388769+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:02:17.626429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.388802+0000","last_change":"2024-07-04T17:30:47.436774+0000","last_active":"2024-07-04T17:30:48.388802+0000","last_peered":"2024-07-04T17:30:48.388802+0000","last_clean":"2024-07-04T17:30:48.388802+0000","last_became_active":"2024-07-04T17:30:47.436573+0000","last_became_peered":"2024-07-04T17:30:47.436573+0000","last_unstale":"2024-07-04T17:30:48.388802+0000","last_undegraded":"2024-07-04T17:30:48.388802+0000","last_fullsized":"2024-07-04T17:30:48.388802+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:27:06.393591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.674818+0000","last_change":"2024-07-04T17:30:47.389819+0000","last_active":"2024-07-04T17:30:48.674818+0000","last_peered":"2024-07-04T17:30:48.674818+0000","last_clean":"2024-07-04T17:30:48.674818+0000","last_became_active":"2024-07-04T17:30:47.387439+0000","last_became_peered":"2024-07-04T17:30:47.387439+0000","last_unstale":"2024-07-04T17:30:48.674818+0000","last_undegraded":"2024-07-04T17:30:48.674818+0000","last_fullsized":"2024-07-04T17:30:48.674818+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:56:14.415266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.447755+0000","last_change":"2024-07-04T17:30:47.395945+0000","last_active":"2024-07-04T17:30:48.447755+0000","last_peered":"2024-07-04T17:30:48.447755+0000","last_clean":"2024-07-04T17:30:48.447755+0000","last_became_active":"2024-07-04T17:30:47.395631+0000","last_became_peered":"2024-07-04T17:30:47.395631+0000","last_unstale":"2024-07-04T17:30:48.447755+0000","last_undegraded":"2024-07-04T17:30:48.447755+0000","last_fullsized":"2024-07-04T17:30:48.447755+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:27:30.220822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.389806+0000","last_change":"2024-07-04T17:30:47.436357+0000","last_active":"2024-07-04T17:30:48.389806+0000","last_peered":"2024-07-04T17:30:48.389806+0000","last_clean":"2024-07-04T17:30:48.389806+0000","last_became_active":"2024-07-04T17:30:47.435542+0000","last_became_peered":"2024-07-04T17:30:47.435542+0000","last_unstale":"2024-07-04T17:30:48.389806+0000","last_undegraded":"2024-07-04T17:30:48.389806+0000","last_fullsized":"2024-07-04T17:30:48.389806+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:08:21.911035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.447172+0000","last_change":"2024-07-04T17:30:47.430335+0000","last_active":"2024-07-04T17:30:48.447172+0000","last_peered":"2024-07-04T17:30:48.447172+0000","last_clean":"2024-07-04T17:30:48.447172+0000","last_became_active":"2024-07-04T17:30:47.427068+0000","last_became_peered":"2024-07-04T17:30:47.427068+0000","last_unstale":"2024-07-04T17:30:48.447172+0000","last_undegraded":"2024-07-04T17:30:48.447172+0000","last_fullsized":"2024-07-04T17:30:48.447172+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:09:53.897531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.675565+0000","last_change":"2024-07-04T17:30:47.398367+0000","last_active":"2024-07-04T17:30:48.675565+0000","last_peered":"2024-07-04T17:30:48.675565+0000","last_clean":"2024-07-04T17:30:48.675565+0000","last_became_active":"2024-07-04T17:30:47.394431+0000","last_became_peered":"2024-07-04T17:30:47.394431+0000","last_unstale":"2024-07-04T17:30:48.675565+0000","last_undegraded":"2024-07-04T17:30:48.675565+0000","last_fullsized":"2024-07-04T17:30:48.675565+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:14:28.773355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.676075+0000","last_change":"2024-07-04T17:30:47.426130+0000","last_active":"2024-07-04T17:30:48.676075+0000","last_peered":"2024-07-04T17:30:48.676075+0000","last_clean":"2024-07-04T17:30:48.676075+0000","last_became_active":"2024-07-04T17:30:47.416510+0000","last_became_peered":"2024-07-04T17:30:47.416510+0000","last_unstale":"2024-07-04T17:30:48.676075+0000","last_undegraded":"2024-07-04T17:30:48.676075+0000","last_fullsized":"2024-07-04T17:30:48.676075+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:53:42.201549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.389174+0000","last_change":"2024-07-04T17:30:47.417678+0000","last_active":"2024-07-04T17:30:48.389174+0000","last_peered":"2024-07-04T17:30:48.389174+0000","last_clean":"2024-07-04T17:30:48.389174+0000","last_became_active":"2024-07-04T17:30:47.414976+0000","last_became_peered":"2024-07-04T17:30:47.414976+0000","last_unstale":"2024-07-04T17:30:48.389174+0000","last_undegraded":"2024-07-04T17:30:48.389174+0000","last_fullsized":"2024-07-04T17:30:48.389174+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:40:08.588002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.444678+0000","last_change":"2024-07-04T17:30:47.396959+0000","last_active":"2024-07-04T17:30:48.444678+0000","last_peered":"2024-07-04T17:30:48.444678+0000","last_clean":"2024-07-04T17:30:48.444678+0000","last_became_active":"2024-07-04T17:30:47.396561+0000","last_became_peered":"2024-07-04T17:30:47.396561+0000","last_unstale":"2024-07-04T17:30:48.444678+0000","last_undegraded":"2024-07-04T17:30:48.444678+0000","last_fullsized":"2024-07-04T17:30:48.444678+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:03:40.493555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.388228+0000","last_change":"2024-07-04T17:30:47.420235+0000","last_active":"2024-07-04T17:30:48.388228+0000","last_peered":"2024-07-04T17:30:48.388228+0000","last_clean":"2024-07-04T17:30:48.388228+0000","last_became_active":"2024-07-04T17:30:47.419803+0000","last_became_peered":"2024-07-04T17:30:47.419803+0000","last_unstale":"2024-07-04T17:30:48.388228+0000","last_undegraded":"2024-07-04T17:30:48.388228+0000","last_fullsized":"2024-07-04T17:30:48.388228+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:27:30.220822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.677831+0000","last_change":"2024-07-04T17:30:47.426146+0000","last_active":"2024-07-04T17:30:48.677831+0000","last_peered":"2024-07-04T17:30:48.677831+0000","last_clean":"2024-07-04T17:30:48.677831+0000","last_became_active":"2024-07-04T17:30:47.421447+0000","last_became_peered":"2024-07-04T17:30:47.421447+0000","last_unstale":"2024-07-04T17:30:48.677831+0000","last_undegraded":"2024-07-04T17:30:48.677831+0000","last_fullsized":"2024-07-04T17:30:48.677831+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:14:40.880998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.390674+0000","last_change":"2024-07-04T17:30:47.436646+0000","last_active":"2024-07-04T17:30:48.390674+0000","last_peered":"2024-07-04T17:30:48.390674+0000","last_clean":"2024-07-04T17:30:48.390674+0000","last_became_active":"2024-07-04T17:30:47.435999+0000","last_became_peered":"2024-07-04T17:30:47.435999+0000","last_unstale":"2024-07-04T17:30:48.390674+0000","last_undegraded":"2024-07-04T17:30:48.390674+0000","last_fullsized":"2024-07-04T17:30:48.390674+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:27:44.098370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.676452+0000","last_change":"2024-07-04T17:30:47.398164+0000","last_active":"2024-07-04T17:30:48.676452+0000","last_peered":"2024-07-04T17:30:48.676452+0000","last_clean":"2024-07-04T17:30:48.676452+0000","last_became_active":"2024-07-04T17:30:47.394563+0000","last_became_peered":"2024-07-04T17:30:47.394563+0000","last_unstale":"2024-07-04T17:30:48.676452+0000","last_undegraded":"2024-07-04T17:30:48.676452+0000","last_fullsized":"2024-07-04T17:30:48.676452+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:51:16.498378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.389248+0000","last_change":"2024-07-04T17:30:47.438690+0000","last_active":"2024-07-04T17:30:48.389248+0000","last_peered":"2024-07-04T17:30:48.389248+0000","last_clean":"2024-07-04T17:30:48.389248+0000","last_became_active":"2024-07-04T17:30:47.438362+0000","last_became_peered":"2024-07-04T17:30:47.438362+0000","last_unstale":"2024-07-04T17:30:48.389248+0000","last_undegraded":"2024-07-04T17:30:48.389248+0000","last_fullsized":"2024-07-04T17:30:48.389248+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:28:11.855854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.389658+0000","last_change":"2024-07-04T17:30:47.435331+0000","last_active":"2024-07-04T17:30:48.389658+0000","last_peered":"2024-07-04T17:30:48.389658+0000","last_clean":"2024-07-04T17:30:48.389658+0000","last_became_active":"2024-07-04T17:30:47.434100+0000","last_became_peered":"2024-07-04T17:30:47.434100+0000","last_unstale":"2024-07-04T17:30:48.389658+0000","last_undegraded":"2024-07-04T17:30:48.389658+0000","last_fullsized":"2024-07-04T17:30:48.389658+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:47:32.884542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.445752+0000","last_change":"2024-07-04T17:30:47.399163+0000","last_active":"2024-07-04T17:30:48.445752+0000","last_peered":"2024-07-04T17:30:48.445752+0000","last_clean":"2024-07-04T17:30:48.445752+0000","last_became_active":"2024-07-04T17:30:47.398747+0000","last_became_peered":"2024-07-04T17:30:47.398747+0000","last_unstale":"2024-07-04T17:30:48.445752+0000","last_undegraded":"2024-07-04T17:30:48.445752+0000","last_fullsized":"2024-07-04T17:30:48.445752+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:24:01.574518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.674332+0000","last_change":"2024-07-04T17:30:47.421142+0000","last_active":"2024-07-04T17:30:48.674332+0000","last_peered":"2024-07-04T17:30:48.674332+0000","last_clean":"2024-07-04T17:30:48.674332+0000","last_became_active":"2024-07-04T17:30:47.416769+0000","last_became_peered":"2024-07-04T17:30:47.416769+0000","last_unstale":"2024-07-04T17:30:48.674332+0000","last_undegraded":"2024-07-04T17:30:48.674332+0000","last_fullsized":"2024-07-04T17:30:48.674332+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:19:39.765910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.388251+0000","last_change":"2024-07-04T17:30:47.421958+0000","last_active":"2024-07-04T17:30:48.388251+0000","last_peered":"2024-07-04T17:30:48.388251+0000","last_clean":"2024-07-04T17:30:48.388251+0000","last_became_active":"2024-07-04T17:30:47.421313+0000","last_became_peered":"2024-07-04T17:30:47.421313+0000","last_unstale":"2024-07-04T17:30:48.388251+0000","last_undegraded":"2024-07-04T17:30:48.388251+0000","last_fullsized":"2024-07-04T17:30:48.388251+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:44:44.984203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.447909+0000","last_change":"2024-07-04T17:30:47.398351+0000","last_active":"2024-07-04T17:30:48.447909+0000","last_peered":"2024-07-04T17:30:48.447909+0000","last_clean":"2024-07-04T17:30:48.447909+0000","last_became_active":"2024-07-04T17:30:47.395987+0000","last_became_peered":"2024-07-04T17:30:47.395987+0000","last_unstale":"2024-07-04T17:30:48.447909+0000","last_undegraded":"2024-07-04T17:30:48.447909+0000","last_fullsized":"2024-07-04T17:30:48.447909+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:44:44.984203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.448046+0000","last_change":"2024-07-04T17:30:47.398645+0000","last_active":"2024-07-04T17:30:48.448046+0000","last_peered":"2024-07-04T17:30:48.448046+0000","last_clean":"2024-07-04T17:30:48.448046+0000","last_became_active":"2024-07-04T17:30:47.394765+0000","last_became_peered":"2024-07-04T17:30:47.394765+0000","last_unstale":"2024-07-04T17:30:48.448046+0000","last_undegraded":"2024-07-04T17:30:48.448046+0000","last_fullsized":"2024-07-04T17:30:48.448046+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:59:48.588528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.390792+0000","last_change":"2024-07-04T17:30:47.439010+0000","last_active":"2024-07-04T17:30:48.390792+0000","last_peered":"2024-07-04T17:30:48.390792+0000","last_clean":"2024-07-04T17:30:48.390792+0000","last_became_active":"2024-07-04T17:30:47.436372+0000","last_became_peered":"2024-07-04T17:30:47.436372+0000","last_unstale":"2024-07-04T17:30:48.390792+0000","last_undegraded":"2024-07-04T17:30:48.390792+0000","last_fullsized":"2024-07-04T17:30:48.390792+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:08:49.032762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.446678+0000","last_change":"2024-07-04T17:30:47.430081+0000","last_active":"2024-07-04T17:30:48.446678+0000","last_peered":"2024-07-04T17:30:48.446678+0000","last_clean":"2024-07-04T17:30:48.446678+0000","last_became_active":"2024-07-04T17:30:47.425964+0000","last_became_peered":"2024-07-04T17:30:47.425964+0000","last_unstale":"2024-07-04T17:30:48.446678+0000","last_undegraded":"2024-07-04T17:30:48.446678+0000","last_fullsized":"2024-07-04T17:30:48.446678+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:44:12.909185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.445756+0000","last_change":"2024-07-04T17:30:47.389158+0000","last_active":"2024-07-04T17:30:48.445756+0000","last_peered":"2024-07-04T17:30:48.445756+0000","last_clean":"2024-07-04T17:30:48.445756+0000","last_became_active":"2024-07-04T17:30:47.388127+0000","last_became_peered":"2024-07-04T17:30:47.388127+0000","last_unstale":"2024-07-04T17:30:48.445756+0000","last_undegraded":"2024-07-04T17:30:48.445756+0000","last_fullsized":"2024-07-04T17:30:48.445756+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:49:47.842800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.446376+0000","last_change":"2024-07-04T17:30:47.427109+0000","last_active":"2024-07-04T17:30:48.446376+0000","last_peered":"2024-07-04T17:30:48.446376+0000","last_clean":"2024-07-04T17:30:48.446376+0000","last_became_active":"2024-07-04T17:30:47.426697+0000","last_became_peered":"2024-07-04T17:30:47.426697+0000","last_unstale":"2024-07-04T17:30:48.446376+0000","last_undegraded":"2024-07-04T17:30:48.446376+0000","last_fullsized":"2024-07-04T17:30:48.446376+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:20:37.646569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.391183+0000","last_change":"2024-07-04T17:30:47.435621+0000","last_active":"2024-07-04T17:30:48.391183+0000","last_peered":"2024-07-04T17:30:48.391183+0000","last_clean":"2024-07-04T17:30:48.391183+0000","last_became_active":"2024-07-04T17:30:47.435230+0000","last_became_peered":"2024-07-04T17:30:47.435230+0000","last_unstale":"2024-07-04T17:30:48.391183+0000","last_undegraded":"2024-07-04T17:30:48.391183+0000","last_fullsized":"2024-07-04T17:30:48.391183+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:29:26.886812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.388559+0000","last_change":"2024-07-04T17:30:47.414726+0000","last_active":"2024-07-04T17:30:48.388559+0000","last_peered":"2024-07-04T17:30:48.388559+0000","last_clean":"2024-07-04T17:30:48.388559+0000","last_became_active":"2024-07-04T17:30:47.414412+0000","last_became_peered":"2024-07-04T17:30:47.414412+0000","last_unstale":"2024-07-04T17:30:48.388559+0000","last_undegraded":"2024-07-04T17:30:48.388559+0000","last_fullsized":"2024-07-04T17:30:48.388559+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:59:48.588528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.677238+0000","last_change":"2024-07-04T17:30:47.426760+0000","last_active":"2024-07-04T17:30:48.677238+0000","last_peered":"2024-07-04T17:30:48.677238+0000","last_clean":"2024-07-04T17:30:48.677238+0000","last_became_active":"2024-07-04T17:30:47.426331+0000","last_became_peered":"2024-07-04T17:30:47.426331+0000","last_unstale":"2024-07-04T17:30:48.677238+0000","last_undegraded":"2024-07-04T17:30:48.677238+0000","last_fullsized":"2024-07-04T17:30:48.677238+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:47:13.050952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"71'4","reported_seq":18,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.677594+0000","last_change":"2024-07-04T17:30:46.417984+0000","last_active":"2024-07-04T17:30:48.677594+0000","last_peered":"2024-07-04T17:30:48.677594+0000","last_clean":"2024-07-04T17:30:48.677594+0000","last_became_active":"2024-07-04T17:30:46.417091+0000","last_became_peered":"2024-07-04T17:30:46.417091+0000","last_unstale":"2024-07-04T17:30:48.677594+0000","last_undegraded":"2024-07-04T17:30:48.677594+0000","last_fullsized":"2024-07-04T17:30:48.677594+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:29:34.252784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.448229+0000","last_change":"2024-07-04T17:30:47.419884+0000","last_active":"2024-07-04T17:30:48.448229+0000","last_peered":"2024-07-04T17:30:48.448229+0000","last_clean":"2024-07-04T17:30:48.448229+0000","last_became_active":"2024-07-04T17:30:47.419588+0000","last_became_peered":"2024-07-04T17:30:47.419588+0000","last_unstale":"2024-07-04T17:30:48.448229+0000","last_undegraded":"2024-07-04T17:30:48.448229+0000","last_fullsized":"2024-07-04T17:30:48.448229+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:02:17.626429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"71'1","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.390445+0000","last_change":"2024-07-04T17:30:46.394693+0000","last_active":"2024-07-04T17:30:48.390445+0000","last_peered":"2024-07-04T17:30:48.390445+0000","last_clean":"2024-07-04T17:30:48.390445+0000","last_became_active":"2024-07-04T17:30:46.394315+0000","last_became_peered":"2024-07-04T17:30:46.394315+0000","last_unstale":"2024-07-04T17:30:48.390445+0000","last_undegraded":"2024-07-04T17:30:48.390445+0000","last_fullsized":"2024-07-04T17:30:48.390445+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:56:39.574079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.675889+0000","last_change":"2024-07-04T17:30:47.398047+0000","last_active":"2024-07-04T17:30:48.675889+0000","last_peered":"2024-07-04T17:30:48.675889+0000","last_clean":"2024-07-04T17:30:48.675889+0000","last_became_active":"2024-07-04T17:30:47.393052+0000","last_became_peered":"2024-07-04T17:30:47.393052+0000","last_unstale":"2024-07-04T17:30:48.675889+0000","last_undegraded":"2024-07-04T17:30:48.675889+0000","last_fullsized":"2024-07-04T17:30:48.675889+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:13:17.681466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"71'2","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.676938+0000","last_change":"2024-07-04T17:30:46.401872+0000","last_active":"2024-07-04T17:30:48.676938+0000","last_peered":"2024-07-04T17:30:48.676938+0000","last_clean":"2024-07-04T17:30:48.676938+0000","last_became_active":"2024-07-04T17:30:46.401510+0000","last_became_peered":"2024-07-04T17:30:46.401510+0000","last_unstale":"2024-07-04T17:30:48.676938+0000","last_undegraded":"2024-07-04T17:30:48.676938+0000","last_fullsized":"2024-07-04T17:30:48.676938+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:02:03.254039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.447224+0000","last_change":"2024-07-04T17:30:47.418834+0000","last_active":"2024-07-04T17:30:48.447224+0000","last_peered":"2024-07-04T17:30:48.447224+0000","last_clean":"2024-07-04T17:30:48.447224+0000","last_became_active":"2024-07-04T17:30:47.418449+0000","last_became_peered":"2024-07-04T17:30:47.418449+0000","last_unstale":"2024-07-04T17:30:48.447224+0000","last_undegraded":"2024-07-04T17:30:48.447224+0000","last_fullsized":"2024-07-04T17:30:48.447224+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:59:40.100483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"71'2","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.676054+0000","last_change":"2024-07-04T17:30:46.417993+0000","last_active":"2024-07-04T17:30:48.676054+0000","last_peered":"2024-07-04T17:30:48.676054+0000","last_clean":"2024-07-04T17:30:48.676054+0000","last_became_active":"2024-07-04T17:30:46.417603+0000","last_became_peered":"2024-07-04T17:30:46.417603+0000","last_unstale":"2024-07-04T17:30:48.676054+0000","last_undegraded":"2024-07-04T17:30:48.676054+0000","last_fullsized":"2024-07-04T17:30:48.676054+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:18:43.863213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.446247+0000","last_change":"2024-07-04T17:30:47.427026+0000","last_active":"2024-07-04T17:30:48.446247+0000","last_peered":"2024-07-04T17:30:48.446247+0000","last_clean":"2024-07-04T17:30:48.446247+0000","last_became_active":"2024-07-04T17:30:47.426173+0000","last_became_peered":"2024-07-04T17:30:47.426173+0000","last_unstale":"2024-07-04T17:30:48.446247+0000","last_undegraded":"2024-07-04T17:30:48.446247+0000","last_fullsized":"2024-07-04T17:30:48.446247+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:00:08.547137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"71'1","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.676456+0000","last_change":"2024-07-04T17:30:46.408705+0000","last_active":"2024-07-04T17:30:48.676456+0000","last_peered":"2024-07-04T17:30:48.676456+0000","last_clean":"2024-07-04T17:30:48.676456+0000","last_became_active":"2024-07-04T17:30:46.408366+0000","last_became_peered":"2024-07-04T17:30:46.408366+0000","last_unstale":"2024-07-04T17:30:48.676456+0000","last_undegraded":"2024-07-04T17:30:48.676456+0000","last_fullsized":"2024-07-04T17:30:48.676456+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:07:19.994029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.390231+0000","last_change":"2024-07-04T17:30:47.435295+0000","last_active":"2024-07-04T17:30:48.390231+0000","last_peered":"2024-07-04T17:30:48.390231+0000","last_clean":"2024-07-04T17:30:48.390231+0000","last_became_active":"2024-07-04T17:30:47.434416+0000","last_became_peered":"2024-07-04T17:30:47.434416+0000","last_unstale":"2024-07-04T17:30:48.390231+0000","last_undegraded":"2024-07-04T17:30:48.390231+0000","last_fullsized":"2024-07-04T17:30:48.390231+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:20:27.029139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"71'1","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.389430+0000","last_change":"2024-07-04T17:30:46.409671+0000","last_active":"2024-07-04T17:30:48.389430+0000","last_peered":"2024-07-04T17:30:48.389430+0000","last_clean":"2024-07-04T17:30:48.389430+0000","last_became_active":"2024-07-04T17:30:46.409031+0000","last_became_peered":"2024-07-04T17:30:46.409031+0000","last_unstale":"2024-07-04T17:30:48.389430+0000","last_undegraded":"2024-07-04T17:30:48.389430+0000","last_fullsized":"2024-07-04T17:30:48.389430+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:06:35.004818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.445364+0000","last_change":"2024-07-04T17:30:47.394313+0000","last_active":"2024-07-04T17:30:48.445364+0000","last_peered":"2024-07-04T17:30:48.445364+0000","last_clean":"2024-07-04T17:30:48.445364+0000","last_became_active":"2024-07-04T17:30:47.393967+0000","last_became_peered":"2024-07-04T17:30:47.393967+0000","last_unstale":"2024-07-04T17:30:48.445364+0000","last_undegraded":"2024-07-04T17:30:48.445364+0000","last_fullsized":"2024-07-04T17:30:48.445364+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:13:54.972134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.674483+0000","last_change":"2024-07-04T17:30:46.414098+0000","last_active":"2024-07-04T17:30:48.674483+0000","last_peered":"2024-07-04T17:30:48.674483+0000","last_clean":"2024-07-04T17:30:48.674483+0000","last_became_active":"2024-07-04T17:30:46.413464+0000","last_became_peered":"2024-07-04T17:30:46.413464+0000","last_unstale":"2024-07-04T17:30:48.674483+0000","last_undegraded":"2024-07-04T17:30:48.674483+0000","last_fullsized":"2024-07-04T17:30:48.674483+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:43:49.510457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.391133+0000","last_change":"2024-07-04T17:30:47.424416+0000","last_active":"2024-07-04T17:30:48.391133+0000","last_peered":"2024-07-04T17:30:48.391133+0000","last_clean":"2024-07-04T17:30:48.391133+0000","last_became_active":"2024-07-04T17:30:47.423168+0000","last_became_peered":"2024-07-04T17:30:47.423168+0000","last_unstale":"2024-07-04T17:30:48.391133+0000","last_undegraded":"2024-07-04T17:30:48.391133+0000","last_fullsized":"2024-07-04T17:30:48.391133+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:02:04.258323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.445674+0000","last_change":"2024-07-04T17:30:46.414564+0000","last_active":"2024-07-04T17:30:48.445674+0000","last_peered":"2024-07-04T17:30:48.445674+0000","last_clean":"2024-07-04T17:30:48.445674+0000","last_became_active":"2024-07-04T17:30:46.414116+0000","last_became_peered":"2024-07-04T17:30:46.414116+0000","last_unstale":"2024-07-04T17:30:48.445674+0000","last_undegraded":"2024-07-04T17:30:48.445674+0000","last_fullsized":"2024-07-04T17:30:48.445674+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:49:01.886056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.677477+0000","last_change":"2024-07-04T17:30:47.397516+0000","last_active":"2024-07-04T17:30:48.677477+0000","last_peered":"2024-07-04T17:30:48.677477+0000","last_clean":"2024-07-04T17:30:48.677477+0000","last_became_active":"2024-07-04T17:30:47.397119+0000","last_became_peered":"2024-07-04T17:30:47.397119+0000","last_unstale":"2024-07-04T17:30:48.677477+0000","last_undegraded":"2024-07-04T17:30:48.677477+0000","last_fullsized":"2024-07-04T17:30:48.677477+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:06:56.459812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.448762+0000","last_change":"2024-07-04T17:30:46.415510+0000","last_active":"2024-07-04T17:30:48.448762+0000","last_peered":"2024-07-04T17:30:48.448762+0000","last_clean":"2024-07-04T17:30:48.448762+0000","last_became_active":"2024-07-04T17:30:46.415113+0000","last_became_peered":"2024-07-04T17:30:46.415113+0000","last_unstale":"2024-07-04T17:30:48.448762+0000","last_undegraded":"2024-07-04T17:30:48.448762+0000","last_fullsized":"2024-07-04T17:30:48.448762+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:13:24.938223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.448422+0000","last_change":"2024-07-04T17:30:47.430636+0000","last_active":"2024-07-04T17:30:48.448422+0000","last_peered":"2024-07-04T17:30:48.448422+0000","last_clean":"2024-07-04T17:30:48.448422+0000","last_became_active":"2024-07-04T17:30:47.425674+0000","last_became_peered":"2024-07-04T17:30:47.425674+0000","last_unstale":"2024-07-04T17:30:48.448422+0000","last_undegraded":"2024-07-04T17:30:48.448422+0000","last_fullsized":"2024-07-04T17:30:48.448422+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:27:06.393591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.390523+0000","last_change":"2024-07-04T17:30:46.407826+0000","last_active":"2024-07-04T17:30:48.390523+0000","last_peered":"2024-07-04T17:30:48.390523+0000","last_clean":"2024-07-04T17:30:48.390523+0000","last_became_active":"2024-07-04T17:30:46.407115+0000","last_became_peered":"2024-07-04T17:30:46.407115+0000","last_unstale":"2024-07-04T17:30:48.390523+0000","last_undegraded":"2024-07-04T17:30:48.390523+0000","last_fullsized":"2024-07-04T17:30:48.390523+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:47:37.889965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.675770+0000","last_change":"2024-07-04T17:30:47.400589+0000","last_active":"2024-07-04T17:30:48.675770+0000","last_peered":"2024-07-04T17:30:48.675770+0000","last_clean":"2024-07-04T17:30:48.675770+0000","last_became_active":"2024-07-04T17:30:47.395213+0000","last_became_peered":"2024-07-04T17:30:47.395213+0000","last_unstale":"2024-07-04T17:30:48.675770+0000","last_undegraded":"2024-07-04T17:30:48.675770+0000","last_fullsized":"2024-07-04T17:30:48.675770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:02:27.554854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"71'3","reported_seq":17,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.447505+0000","last_change":"2024-07-04T17:30:46.413645+0000","last_active":"2024-07-04T17:30:48.447505+0000","last_peered":"2024-07-04T17:30:48.447505+0000","last_clean":"2024-07-04T17:30:48.447505+0000","last_became_active":"2024-07-04T17:30:46.413314+0000","last_became_peered":"2024-07-04T17:30:46.413314+0000","last_unstale":"2024-07-04T17:30:48.447505+0000","last_undegraded":"2024-07-04T17:30:48.447505+0000","last_fullsized":"2024-07-04T17:30:48.447505+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:56:15.192711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.390793+0000","last_change":"2024-07-04T17:30:47.439341+0000","last_active":"2024-07-04T17:30:48.390793+0000","last_peered":"2024-07-04T17:30:48.390793+0000","last_clean":"2024-07-04T17:30:48.390793+0000","last_became_active":"2024-07-04T17:30:47.438643+0000","last_became_peered":"2024-07-04T17:30:47.438643+0000","last_unstale":"2024-07-04T17:30:48.390793+0000","last_undegraded":"2024-07-04T17:30:48.390793+0000","last_fullsized":"2024-07-04T17:30:48.390793+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:18:57.306251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.14","version":"71'6","reported_seq":20,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.676259+0000","last_change":"2024-07-04T17:30:46.418509+0000","last_active":"2024-07-04T17:30:48.676259+0000","last_peered":"2024-07-04T17:30:48.676259+0000","last_clean":"2024-07-04T17:30:48.676259+0000","last_became_active":"2024-07-04T17:30:46.418161+0000","last_became_peered":"2024-07-04T17:30:46.418161+0000","last_unstale":"2024-07-04T17:30:48.676259+0000","last_undegraded":"2024-07-04T17:30:48.676259+0000","last_fullsized":"2024-07-04T17:30:48.676259+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:26:41.897228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.674710+0000","last_change":"2024-07-04T17:30:47.388099+0000","last_active":"2024-07-04T17:30:48.674710+0000","last_peered":"2024-07-04T17:30:48.674710+0000","last_clean":"2024-07-04T17:30:48.674710+0000","last_became_active":"2024-07-04T17:30:47.387785+0000","last_became_peered":"2024-07-04T17:30:47.387785+0000","last_unstale":"2024-07-04T17:30:48.674710+0000","last_undegraded":"2024-07-04T17:30:48.674710+0000","last_fullsized":"2024-07-04T17:30:48.674710+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:53:19.566358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"71'2","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.446139+0000","last_change":"2024-07-04T17:30:46.404600+0000","last_active":"2024-07-04T17:30:48.446139+0000","last_peered":"2024-07-04T17:30:48.446139+0000","last_clean":"2024-07-04T17:30:48.446139+0000","last_became_active":"2024-07-04T17:30:46.404314+0000","last_became_peered":"2024-07-04T17:30:46.404314+0000","last_unstale":"2024-07-04T17:30:48.446139+0000","last_undegraded":"2024-07-04T17:30:48.446139+0000","last_fullsized":"2024-07-04T17:30:48.446139+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:25:07.446781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.445827+0000","last_change":"2024-07-04T17:30:47.387822+0000","last_active":"2024-07-04T17:30:48.445827+0000","last_peered":"2024-07-04T17:30:48.445827+0000","last_clean":"2024-07-04T17:30:48.445827+0000","last_became_active":"2024-07-04T17:30:47.387451+0000","last_became_peered":"2024-07-04T17:30:47.387451+0000","last_unstale":"2024-07-04T17:30:48.445827+0000","last_undegraded":"2024-07-04T17:30:48.445827+0000","last_fullsized":"2024-07-04T17:30:48.445827+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:46:22.229171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.675153+0000","last_change":"2024-07-04T17:30:46.410149+0000","last_active":"2024-07-04T17:30:48.675153+0000","last_peered":"2024-07-04T17:30:48.675153+0000","last_clean":"2024-07-04T17:30:48.675153+0000","last_became_active":"2024-07-04T17:30:46.409816+0000","last_became_peered":"2024-07-04T17:30:46.409816+0000","last_unstale":"2024-07-04T17:30:48.675153+0000","last_undegraded":"2024-07-04T17:30:48.675153+0000","last_fullsized":"2024-07-04T17:30:48.675153+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:51:54.551650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.445011+0000","last_change":"2024-07-04T17:30:47.431466+0000","last_active":"2024-07-04T17:30:48.445011+0000","last_peered":"2024-07-04T17:30:48.445011+0000","last_clean":"2024-07-04T17:30:48.445011+0000","last_became_active":"2024-07-04T17:30:47.430961+0000","last_became_peered":"2024-07-04T17:30:47.430961+0000","last_unstale":"2024-07-04T17:30:48.445011+0000","last_undegraded":"2024-07-04T17:30:48.445011+0000","last_fullsized":"2024-07-04T17:30:48.445011+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:50:08.536272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.391277+0000","last_change":"2024-07-04T17:30:46.396182+0000","last_active":"2024-07-04T17:30:48.391277+0000","last_peered":"2024-07-04T17:30:48.391277+0000","last_clean":"2024-07-04T17:30:48.391277+0000","last_became_active":"2024-07-04T17:30:46.393987+0000","last_became_peered":"2024-07-04T17:30:46.393987+0000","last_unstale":"2024-07-04T17:30:48.391277+0000","last_undegraded":"2024-07-04T17:30:48.391277+0000","last_fullsized":"2024-07-04T17:30:48.391277+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:44:44.996919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.444809+0000","last_change":"2024-07-04T17:30:47.395226+0000","last_active":"2024-07-04T17:30:48.444809+0000","last_peered":"2024-07-04T17:30:48.444809+0000","last_clean":"2024-07-04T17:30:48.444809+0000","last_became_active":"2024-07-04T17:30:47.394913+0000","last_became_peered":"2024-07-04T17:30:47.394913+0000","last_unstale":"2024-07-04T17:30:48.444809+0000","last_undegraded":"2024-07-04T17:30:48.444809+0000","last_fullsized":"2024-07-04T17:30:48.444809+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:31:15.378585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"71'8","reported_seq":22,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.674846+0000","last_change":"2024-07-04T17:30:46.398618+0000","last_active":"2024-07-04T17:30:48.674846+0000","last_peered":"2024-07-04T17:30:48.674846+0000","last_clean":"2024-07-04T17:30:48.674846+0000","last_became_active":"2024-07-04T17:30:46.398274+0000","last_became_peered":"2024-07-04T17:30:46.398274+0000","last_unstale":"2024-07-04T17:30:48.674846+0000","last_undegraded":"2024-07-04T17:30:48.674846+0000","last_fullsized":"2024-07-04T17:30:48.674846+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:13:50.002315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.677763+0000","last_change":"2024-07-04T17:30:47.397472+0000","last_active":"2024-07-04T17:30:48.677763+0000","last_peered":"2024-07-04T17:30:48.677763+0000","last_clean":"2024-07-04T17:30:48.677763+0000","last_became_active":"2024-07-04T17:30:47.396989+0000","last_became_peered":"2024-07-04T17:30:47.396989+0000","last_unstale":"2024-07-04T17:30:48.677763+0000","last_undegraded":"2024-07-04T17:30:48.677763+0000","last_fullsized":"2024-07-04T17:30:48.677763+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:40:22.762798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"71'1","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.677714+0000","last_change":"2024-07-04T17:30:46.412075+0000","last_active":"2024-07-04T17:30:48.677714+0000","last_peered":"2024-07-04T17:30:48.677714+0000","last_clean":"2024-07-04T17:30:48.677714+0000","last_became_active":"2024-07-04T17:30:46.411727+0000","last_became_peered":"2024-07-04T17:30:46.411727+0000","last_unstale":"2024-07-04T17:30:48.677714+0000","last_undegraded":"2024-07-04T17:30:48.677714+0000","last_fullsized":"2024-07-04T17:30:48.677714+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:12:43.261982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.391044+0000","last_change":"2024-07-04T17:30:47.436755+0000","last_active":"2024-07-04T17:30:48.391044+0000","last_peered":"2024-07-04T17:30:48.391044+0000","last_clean":"2024-07-04T17:30:48.391044+0000","last_became_active":"2024-07-04T17:30:47.436409+0000","last_became_peered":"2024-07-04T17:30:47.436409+0000","last_unstale":"2024-07-04T17:30:48.391044+0000","last_undegraded":"2024-07-04T17:30:48.391044+0000","last_fullsized":"2024-07-04T17:30:48.391044+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:13:51.006598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"71'2","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.678181+0000","last_change":"2024-07-04T17:30:46.409103+0000","last_active":"2024-07-04T17:30:48.678181+0000","last_peered":"2024-07-04T17:30:48.678181+0000","last_clean":"2024-07-04T17:30:48.678181+0000","last_became_active":"2024-07-04T17:30:46.406340+0000","last_became_peered":"2024-07-04T17:30:46.406340+0000","last_unstale":"2024-07-04T17:30:48.678181+0000","last_undegraded":"2024-07-04T17:30:48.678181+0000","last_fullsized":"2024-07-04T17:30:48.678181+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:43:03.614042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.676571+0000","last_change":"2024-07-04T17:30:47.394855+0000","last_active":"2024-07-04T17:30:48.676571+0000","last_peered":"2024-07-04T17:30:48.676571+0000","last_clean":"2024-07-04T17:30:48.676571+0000","last_became_active":"2024-07-04T17:30:47.393038+0000","last_became_peered":"2024-07-04T17:30:47.393038+0000","last_unstale":"2024-07-04T17:30:48.676571+0000","last_undegraded":"2024-07-04T17:30:48.676571+0000","last_fullsized":"2024-07-04T17:30:48.676571+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:25:35.950854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.676208+0000","last_change":"2024-07-04T17:30:46.409923+0000","last_active":"2024-07-04T17:30:48.676208+0000","last_peered":"2024-07-04T17:30:48.676208+0000","last_clean":"2024-07-04T17:30:48.676208+0000","last_became_active":"2024-07-04T17:30:46.409591+0000","last_became_peered":"2024-07-04T17:30:46.409591+0000","last_unstale":"2024-07-04T17:30:48.676208+0000","last_undegraded":"2024-07-04T17:30:48.676208+0000","last_fullsized":"2024-07-04T17:30:48.676208+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:12:05.853612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.389299+0000","last_change":"2024-07-04T17:30:47.419189+0000","last_active":"2024-07-04T17:30:48.389299+0000","last_peered":"2024-07-04T17:30:48.389299+0000","last_clean":"2024-07-04T17:30:48.389299+0000","last_became_active":"2024-07-04T17:30:47.417882+0000","last_became_peered":"2024-07-04T17:30:47.417882+0000","last_unstale":"2024-07-04T17:30:48.389299+0000","last_undegraded":"2024-07-04T17:30:48.389299+0000","last_fullsized":"2024-07-04T17:30:48.389299+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:28:21.769409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"71'2","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.676587+0000","last_change":"2024-07-04T17:30:46.412068+0000","last_active":"2024-07-04T17:30:48.676587+0000","last_peered":"2024-07-04T17:30:48.676587+0000","last_clean":"2024-07-04T17:30:48.676587+0000","last_became_active":"2024-07-04T17:30:46.411711+0000","last_became_peered":"2024-07-04T17:30:46.411711+0000","last_unstale":"2024-07-04T17:30:48.676587+0000","last_undegraded":"2024-07-04T17:30:48.676587+0000","last_fullsized":"2024-07-04T17:30:48.676587+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:42:52.442755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.390468+0000","last_change":"2024-07-04T17:30:47.436121+0000","last_active":"2024-07-04T17:30:48.390468+0000","last_peered":"2024-07-04T17:30:48.390468+0000","last_clean":"2024-07-04T17:30:48.390468+0000","last_became_active":"2024-07-04T17:30:47.435790+0000","last_became_peered":"2024-07-04T17:30:47.435790+0000","last_unstale":"2024-07-04T17:30:48.390468+0000","last_undegraded":"2024-07-04T17:30:48.390468+0000","last_fullsized":"2024-07-04T17:30:48.390468+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:25:40.409565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.389535+0000","last_change":"2024-07-04T17:30:46.408582+0000","last_active":"2024-07-04T17:30:48.389535+0000","last_peered":"2024-07-04T17:30:48.389535+0000","last_clean":"2024-07-04T17:30:48.389535+0000","last_became_active":"2024-07-04T17:30:46.408297+0000","last_became_peered":"2024-07-04T17:30:46.408297+0000","last_unstale":"2024-07-04T17:30:48.389535+0000","last_undegraded":"2024-07-04T17:30:48.389535+0000","last_fullsized":"2024-07-04T17:30:48.389535+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:48:21.667965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.388592+0000","last_change":"2024-07-04T17:30:47.435916+0000","last_active":"2024-07-04T17:30:48.388592+0000","last_peered":"2024-07-04T17:30:48.388592+0000","last_clean":"2024-07-04T17:30:48.388592+0000","last_became_active":"2024-07-04T17:30:47.435562+0000","last_became_peered":"2024-07-04T17:30:47.435562+0000","last_unstale":"2024-07-04T17:30:48.388592+0000","last_undegraded":"2024-07-04T17:30:48.388592+0000","last_fullsized":"2024-07-04T17:30:48.388592+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:41:29.873067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.391449+0000","last_change":"2024-07-04T17:30:46.394311+0000","last_active":"2024-07-04T17:30:48.391449+0000","last_peered":"2024-07-04T17:30:48.391449+0000","last_clean":"2024-07-04T17:30:48.391449+0000","last_became_active":"2024-07-04T17:30:46.393593+0000","last_became_peered":"2024-07-04T17:30:46.393593+0000","last_unstale":"2024-07-04T17:30:48.391449+0000","last_undegraded":"2024-07-04T17:30:48.391449+0000","last_fullsized":"2024-07-04T17:30:48.391449+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:48:17.986899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.674771+0000","last_change":"2024-07-04T17:30:47.420543+0000","last_active":"2024-07-04T17:30:48.674771+0000","last_peered":"2024-07-04T17:30:48.674771+0000","last_clean":"2024-07-04T17:30:48.674771+0000","last_became_active":"2024-07-04T17:30:47.416196+0000","last_became_peered":"2024-07-04T17:30:47.416196+0000","last_unstale":"2024-07-04T17:30:48.674771+0000","last_undegraded":"2024-07-04T17:30:48.674771+0000","last_fullsized":"2024-07-04T17:30:48.674771+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:50:20.456752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.388720+0000","last_change":"2024-07-04T17:30:46.408040+0000","last_active":"2024-07-04T17:30:48.388720+0000","last_peered":"2024-07-04T17:30:48.388720+0000","last_clean":"2024-07-04T17:30:48.388720+0000","last_became_active":"2024-07-04T17:30:46.407167+0000","last_became_peered":"2024-07-04T17:30:46.407167+0000","last_unstale":"2024-07-04T17:30:48.388720+0000","last_undegraded":"2024-07-04T17:30:48.388720+0000","last_fullsized":"2024-07-04T17:30:48.388720+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:46:18.222946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.677970+0000","last_change":"2024-07-04T17:30:47.397337+0000","last_active":"2024-07-04T17:30:48.677970+0000","last_peered":"2024-07-04T17:30:48.677970+0000","last_clean":"2024-07-04T17:30:48.677970+0000","last_became_active":"2024-07-04T17:30:47.396668+0000","last_became_peered":"2024-07-04T17:30:47.396668+0000","last_unstale":"2024-07-04T17:30:48.677970+0000","last_undegraded":"2024-07-04T17:30:48.677970+0000","last_fullsized":"2024-07-04T17:30:48.677970+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:44:37.013444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"71'2","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.449310+0000","last_change":"2024-07-04T17:30:46.414092+0000","last_active":"2024-07-04T17:30:48.449310+0000","last_peered":"2024-07-04T17:30:48.449310+0000","last_clean":"2024-07-04T17:30:48.449310+0000","last_became_active":"2024-07-04T17:30:46.413746+0000","last_became_peered":"2024-07-04T17:30:46.413746+0000","last_unstale":"2024-07-04T17:30:48.449310+0000","last_undegraded":"2024-07-04T17:30:48.449310+0000","last_fullsized":"2024-07-04T17:30:48.449310+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:56:21.477169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.448824+0000","last_change":"2024-07-04T17:30:47.388955+0000","last_active":"2024-07-04T17:30:48.448824+0000","last_peered":"2024-07-04T17:30:48.448824+0000","last_clean":"2024-07-04T17:30:48.448824+0000","last_became_active":"2024-07-04T17:30:47.387588+0000","last_became_peered":"2024-07-04T17:30:47.387588+0000","last_unstale":"2024-07-04T17:30:48.448824+0000","last_undegraded":"2024-07-04T17:30:48.448824+0000","last_fullsized":"2024-07-04T17:30:48.448824+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:58:22.418379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.6","version":"72'1","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.448619+0000","last_change":"2024-07-04T17:30:46.405443+0000","last_active":"2024-07-04T17:30:48.448619+0000","last_peered":"2024-07-04T17:30:48.448619+0000","last_clean":"2024-07-04T17:30:48.448619+0000","last_became_active":"2024-07-04T17:30:46.405066+0000","last_became_peered":"2024-07-04T17:30:46.405066+0000","last_unstale":"2024-07-04T17:30:48.448619+0000","last_undegraded":"2024-07-04T17:30:48.448619+0000","last_fullsized":"2024-07-04T17:30:48.448619+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:34:24.763771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.675947+0000","last_change":"2024-07-04T17:30:47.420498+0000","last_active":"2024-07-04T17:30:48.675947+0000","last_peered":"2024-07-04T17:30:48.675947+0000","last_clean":"2024-07-04T17:30:48.675947+0000","last_became_active":"2024-07-04T17:30:47.416342+0000","last_became_peered":"2024-07-04T17:30:47.416342+0000","last_unstale":"2024-07-04T17:30:48.675947+0000","last_undegraded":"2024-07-04T17:30:48.675947+0000","last_fullsized":"2024-07-04T17:30:48.675947+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:09:18.581049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.677022+0000","last_change":"2024-07-04T17:30:46.401530+0000","last_active":"2024-07-04T17:30:48.677022+0000","last_peered":"2024-07-04T17:30:48.677022+0000","last_clean":"2024-07-04T17:30:48.677022+0000","last_became_active":"2024-07-04T17:30:46.401058+0000","last_became_peered":"2024-07-04T17:30:46.401058+0000","last_unstale":"2024-07-04T17:30:48.677022+0000","last_undegraded":"2024-07-04T17:30:48.677022+0000","last_fullsized":"2024-07-04T17:30:48.677022+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:29:25.882528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.391106+0000","last_change":"2024-07-04T17:30:47.439494+0000","last_active":"2024-07-04T17:30:48.391106+0000","last_peered":"2024-07-04T17:30:48.391106+0000","last_clean":"2024-07-04T17:30:48.391106+0000","last_became_active":"2024-07-04T17:30:47.438655+0000","last_became_peered":"2024-07-04T17:30:47.438655+0000","last_unstale":"2024-07-04T17:30:48.391106+0000","last_undegraded":"2024-07-04T17:30:48.391106+0000","last_fullsized":"2024-07-04T17:30:48.391106+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:37:58.800114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.389300+0000","last_change":"2024-07-04T17:30:46.398565+0000","last_active":"2024-07-04T17:30:48.389300+0000","last_peered":"2024-07-04T17:30:48.389300+0000","last_clean":"2024-07-04T17:30:48.389300+0000","last_became_active":"2024-07-04T17:30:46.394788+0000","last_became_peered":"2024-07-04T17:30:46.394788+0000","last_unstale":"2024-07-04T17:30:48.389300+0000","last_undegraded":"2024-07-04T17:30:48.389300+0000","last_fullsized":"2024-07-04T17:30:48.389300+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:58:03.149737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.446705+0000","last_change":"2024-07-04T17:30:47.426754+0000","last_active":"2024-07-04T17:30:48.446705+0000","last_peered":"2024-07-04T17:30:48.446705+0000","last_clean":"2024-07-04T17:30:48.446705+0000","last_became_active":"2024-07-04T17:30:47.425712+0000","last_became_peered":"2024-07-04T17:30:47.425712+0000","last_unstale":"2024-07-04T17:30:48.446705+0000","last_undegraded":"2024-07-04T17:30:48.446705+0000","last_fullsized":"2024-07-04T17:30:48.446705+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:07:15.865385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.676717+0000","last_change":"2024-07-04T17:30:46.401607+0000","last_active":"2024-07-04T17:30:48.676717+0000","last_peered":"2024-07-04T17:30:48.676717+0000","last_clean":"2024-07-04T17:30:48.676717+0000","last_became_active":"2024-07-04T17:30:46.401314+0000","last_became_peered":"2024-07-04T17:30:46.401314+0000","last_unstale":"2024-07-04T17:30:48.676717+0000","last_undegraded":"2024-07-04T17:30:48.676717+0000","last_fullsized":"2024-07-04T17:30:48.676717+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:21:59.991349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.676664+0000","last_change":"2024-07-04T17:30:47.424977+0000","last_active":"2024-07-04T17:30:48.676664+0000","last_peered":"2024-07-04T17:30:48.676664+0000","last_clean":"2024-07-04T17:30:48.676664+0000","last_became_active":"2024-07-04T17:30:47.416042+0000","last_became_peered":"2024-07-04T17:30:47.416042+0000","last_unstale":"2024-07-04T17:30:48.676664+0000","last_undegraded":"2024-07-04T17:30:48.676664+0000","last_fullsized":"2024-07-04T17:30:48.676664+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:41:40.720108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.389567+0000","last_change":"2024-07-04T17:30:46.408561+0000","last_active":"2024-07-04T17:30:48.389567+0000","last_peered":"2024-07-04T17:30:48.389567+0000","last_clean":"2024-07-04T17:30:48.389567+0000","last_became_active":"2024-07-04T17:30:46.408199+0000","last_became_peered":"2024-07-04T17:30:46.408199+0000","last_unstale":"2024-07-04T17:30:48.389567+0000","last_undegraded":"2024-07-04T17:30:48.389567+0000","last_fullsized":"2024-07-04T17:30:48.389567+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:15:50.228374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.445481+0000","last_change":"2024-07-04T17:30:47.419127+0000","last_active":"2024-07-04T17:30:48.445481+0000","last_peered":"2024-07-04T17:30:48.445481+0000","last_clean":"2024-07-04T17:30:48.445481+0000","last_became_active":"2024-07-04T17:30:47.418557+0000","last_became_peered":"2024-07-04T17:30:47.418557+0000","last_unstale":"2024-07-04T17:30:48.445481+0000","last_undegraded":"2024-07-04T17:30:48.445481+0000","last_fullsized":"2024-07-04T17:30:48.445481+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:16:53.257166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.445422+0000","last_change":"2024-07-04T17:30:46.404540+0000","last_active":"2024-07-04T17:30:48.445422+0000","last_peered":"2024-07-04T17:30:48.445422+0000","last_clean":"2024-07-04T17:30:48.445422+0000","last_became_active":"2024-07-04T17:30:46.402800+0000","last_became_peered":"2024-07-04T17:30:46.402800+0000","last_unstale":"2024-07-04T17:30:48.445422+0000","last_undegraded":"2024-07-04T17:30:48.445422+0000","last_fullsized":"2024-07-04T17:30:48.445422+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:57:35.604132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.445179+0000","last_change":"2024-07-04T17:30:47.427679+0000","last_active":"2024-07-04T17:30:48.445179+0000","last_peered":"2024-07-04T17:30:48.445179+0000","last_clean":"2024-07-04T17:30:48.445179+0000","last_became_active":"2024-07-04T17:30:47.427451+0000","last_became_peered":"2024-07-04T17:30:47.427451+0000","last_unstale":"2024-07-04T17:30:48.445179+0000","last_undegraded":"2024-07-04T17:30:48.445179+0000","last_fullsized":"2024-07-04T17:30:48.445179+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:29:16.626050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.675127+0000","last_change":"2024-07-04T17:30:46.402877+0000","last_active":"2024-07-04T17:30:48.675127+0000","last_peered":"2024-07-04T17:30:48.675127+0000","last_clean":"2024-07-04T17:30:48.675127+0000","last_became_active":"2024-07-04T17:30:46.402551+0000","last_became_peered":"2024-07-04T17:30:46.402551+0000","last_unstale":"2024-07-04T17:30:48.675127+0000","last_undegraded":"2024-07-04T17:30:48.675127+0000","last_fullsized":"2024-07-04T17:30:48.675127+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:37:57.795830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.446820+0000","last_change":"2024-07-04T17:30:47.390337+0000","last_active":"2024-07-04T17:30:48.446820+0000","last_peered":"2024-07-04T17:30:48.446820+0000","last_clean":"2024-07-04T17:30:48.446820+0000","last_became_active":"2024-07-04T17:30:47.389504+0000","last_became_peered":"2024-07-04T17:30:47.389504+0000","last_unstale":"2024-07-04T17:30:48.446820+0000","last_undegraded":"2024-07-04T17:30:48.446820+0000","last_fullsized":"2024-07-04T17:30:48.446820+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:08:26.050071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"71'1","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.447517+0000","last_change":"2024-07-04T17:30:46.398516+0000","last_active":"2024-07-04T17:30:48.447517+0000","last_peered":"2024-07-04T17:30:48.447517+0000","last_clean":"2024-07-04T17:30:48.447517+0000","last_became_active":"2024-07-04T17:30:46.398169+0000","last_became_peered":"2024-07-04T17:30:46.398169+0000","last_unstale":"2024-07-04T17:30:48.447517+0000","last_undegraded":"2024-07-04T17:30:48.447517+0000","last_fullsized":"2024-07-04T17:30:48.447517+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:40:51.947038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"58'80","reported_seq":296,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.445743+0000","last_change":"2024-07-04T17:26:04.953559+0000","last_active":"2024-07-04T17:30:48.445743+0000","last_peered":"2024-07-04T17:30:48.445743+0000","last_clean":"2024-07-04T17:30:48.445743+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:30:48.445743+0000","last_undegraded":"2024-07-04T17:30:48.445743+0000","last_fullsized":"2024-07-04T17:30:48.445743+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_clean_scrub_stamp":"2024-07-04T17:26:03.933432+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.064181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":296,"num_read_kb":750,"num_write":127,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":296,"num_read_kb":750,"num_write":127,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80,"ondisk_log_size":80,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738427,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31916,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339924,"statfs":{"total":96636764160,"available":96604082176,"internally_reserved":0,"allocated":749568,"data_stored":284321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738428,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36644,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335196,"statfs":{"total":96636764160,"available":96599240704,"internally_reserved":0,"allocated":1327104,"data_stored":872507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738426,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36656,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335184,"statfs":{"total":96636764160,"available":96599228416,"internally_reserved":0,"allocated":1339392,"data_stored":874563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:30:52.119 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:30:52.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.444+0000 7f4fe1a00640 1 -- 172.21.15.42:0/3653601949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fdc141590 msgr2=0x7f4fdc1419a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:52.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.444+0000 7f4fe1a00640 1 --2- 172.21.15.42:0/3653601949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fdc141590 0x7f4fdc1419a0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8005fd0 tx=0x7f4fc802a5f0 comp rx=0 tx=0).stop 2024-07-04T17:30:52.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.444+0000 7f4fe1a00640 1 -- 172.21.15.42:0/3653601949 shutdown_connections 2024-07-04T17:30:52.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.444+0000 7f4fe1a00640 1 --2- 172.21.15.42:0/3653601949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fdc141590 0x7f4fdc1419a0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:52.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.444+0000 7f4fe1a00640 1 -- 172.21.15.42:0/3653601949 >> 172.21.15.42:0/3653601949 conn(0x7f4fdc06c940 msgr2=0x7f4fdc06ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:52.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.444+0000 7f4fe1a00640 1 -- 172.21.15.42:0/3653601949 shutdown_connections 2024-07-04T17:30:52.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.444+0000 7f4fe1a00640 1 -- 172.21.15.42:0/3653601949 wait complete. 2024-07-04T17:30:52.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.446+0000 7f4fe1a00640 1 Processor -- start 2024-07-04T17:30:52.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.447+0000 7f4fe1a00640 1 -- start start 2024-07-04T17:30:52.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.448+0000 7f4fe1a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fdc0bc040 0x7f4fdc0c2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:52.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.448+0000 7f4fe1a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4fdc0b7530 con 0x7f4fdc0bc040 2024-07-04T17:30:52.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.448+0000 7f4fdaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fdc0bc040 0x7f4fdc0c2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:52.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.448+0000 7f4fdaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fdc0bc040 0x7f4fdc0c2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39306/0 (socket says 172.21.15.42:39306) 2024-07-04T17:30:52.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.448+0000 7f4fdaa00640 1 -- 172.21.15.42:0/3296146494 learned_addr learned my addr 172.21.15.42:0/3296146494 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:52.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.449+0000 7f4fdaa00640 1 -- 172.21.15.42:0/3296146494 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fdc0c4720 con 0x7f4fdc0bc040 2024-07-04T17:30:52.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.449+0000 7f4fdaa00640 1 --2- 172.21.15.42:0/3296146494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fdc0bc040 0x7f4fdc0c2580 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc00c2f0 tx=0x7f4fcc00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:52.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.449+0000 7f4fd2a00640 1 -- 172.21.15.42:0/3296146494 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fcc017070 con 0x7f4fdc0bc040 2024-07-04T17:30:52.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.449+0000 7f4fd2a00640 1 -- 172.21.15.42:0/3296146494 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4fcc005410 con 0x7f4fdc0bc040 2024-07-04T17:30:52.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.449+0000 7f4fd2a00640 1 -- 172.21.15.42:0/3296146494 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fcc003780 con 0x7f4fdc0bc040 2024-07-04T17:30:52.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.449+0000 7f4fe1a00640 1 -- 172.21.15.42:0/3296146494 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fdc1e46a0 con 0x7f4fdc0bc040 2024-07-04T17:30:52.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.449+0000 7f4fe1a00640 1 -- 172.21.15.42:0/3296146494 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fdc0b2c10 con 0x7f4fdc0bc040 2024-07-04T17:30:52.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.451+0000 7f4fe1a00640 1 -- 172.21.15.42:0/3296146494 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fdc146120 con 0x7f4fdc0bc040 2024-07-04T17:30:52.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.451+0000 7f4fd2a00640 1 -- 172.21.15.42:0/3296146494 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f4fcc01a070 con 0x7f4fdc0bc040 2024-07-04T17:30:52.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.452+0000 7f4fd2a00640 1 --2- 172.21.15.42:0/3296146494 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f4fbc03d4f0 0x7f4fbc03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:52.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.452+0000 7f4fd2a00640 1 -- 172.21.15.42:0/3296146494 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 3439+0+0 (secure 0 0 0) 0x7f4fcc050ba0 con 0x7f4fdc0bc040 2024-07-04T17:30:52.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.452+0000 7f4fda000640 1 --2- 172.21.15.42:0/3296146494 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f4fbc03d4f0 0x7f4fbc03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:52.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.453+0000 7f4fda000640 1 --2- 172.21.15.42:0/3296146494 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f4fbc03d4f0 0x7f4fbc03fa00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8033000 tx=0x7f4fc8004730 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:52.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.457+0000 7f4fd2a00640 1 -- 172.21.15.42:0/3296146494 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f4fcc01ad80 con 0x7f4fdc0bc040 2024-07-04T17:30:52.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.660+0000 7f4fe1a00640 1 -- 172.21.15.42:0/3296146494 --> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4fdc0b7bf0 con 0x7f4fbc03d4f0 2024-07-04T17:30:52.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.668+0000 7f4fd2a00640 1 -- 172.21.15.42:0/3296146494 <== mgr.4906 v2:172.21.15.42:6824/3473020706 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246084 (secure 0 0 0) 0x7f4fdc0b7bf0 con 0x7f4fbc03d4f0 2024-07-04T17:30:52.673 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:30:52.674 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:30:52.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.674+0000 7f4fe1a00640 1 -- 172.21.15.42:0/3296146494 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f4fbc03d4f0 msgr2=0x7f4fbc03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:52.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.674+0000 7f4fe1a00640 1 --2- 172.21.15.42:0/3296146494 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f4fbc03d4f0 0x7f4fbc03fa00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8033000 tx=0x7f4fc8004730 comp rx=0 tx=0).stop 2024-07-04T17:30:52.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.674+0000 7f4fe1a00640 1 -- 172.21.15.42:0/3296146494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fdc0bc040 msgr2=0x7f4fdc0c2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:52.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.674+0000 7f4fe1a00640 1 --2- 172.21.15.42:0/3296146494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fdc0bc040 0x7f4fdc0c2580 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc00c2f0 tx=0x7f4fcc00c7b0 comp rx=0 tx=0).stop 2024-07-04T17:30:52.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.675+0000 7f4fe1a00640 1 -- 172.21.15.42:0/3296146494 shutdown_connections 2024-07-04T17:30:52.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.675+0000 7f4fe1a00640 1 --2- 172.21.15.42:0/3296146494 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f4fbc03d4f0 0x7f4fbc03fa00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:52.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.675+0000 7f4fe1a00640 1 --2- 172.21.15.42:0/3296146494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fdc0bc040 0x7f4fdc0c2580 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:52.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.675+0000 7f4fe1a00640 1 -- 172.21.15.42:0/3296146494 >> 172.21.15.42:0/3296146494 conn(0x7f4fdc06c940 msgr2=0x7f4fdc0740b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:52.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.675+0000 7f4fe1a00640 1 -- 172.21.15.42:0/3296146494 shutdown_connections 2024-07-04T17:30:52.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:52.675+0000 7f4fe1a00640 1 -- 172.21.15.42:0/3296146494 wait complete. 2024-07-04T17:30:52.703 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-07-04T17:30:51.627627+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":296,"num_read_kb":750,"num_write":172,"num_write_kb":1120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":105216,"kb_used_data":3336,"kb_used_omap":4,"kb_used_meta":101819,"kb_avail":283010304,"statfs":{"total":289910292480,"available":289802551296,"internally_reserved":0,"allocated":3416064,"data_stored":2031391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":104262991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001997"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.447256+0000","last_change":"2024-07-04T17:30:47.430443+0000","last_active":"2024-07-04T17:30:48.447256+0000","last_peered":"2024-07-04T17:30:48.447256+0000","last_clean":"2024-07-04T17:30:48.447256+0000","last_became_active":"2024-07-04T17:30:47.425325+0000","last_became_peered":"2024-07-04T17:30:47.425325+0000","last_unstale":"2024-07-04T17:30:48.447256+0000","last_undegraded":"2024-07-04T17:30:48.447256+0000","last_fullsized":"2024-07-04T17:30:48.447256+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:44:15.710215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.448048+0000","last_change":"2024-07-04T17:30:47.429677+0000","last_active":"2024-07-04T17:30:48.448048+0000","last_peered":"2024-07-04T17:30:48.448048+0000","last_clean":"2024-07-04T17:30:48.448048+0000","last_became_active":"2024-07-04T17:30:47.428028+0000","last_became_peered":"2024-07-04T17:30:47.428028+0000","last_unstale":"2024-07-04T17:30:48.448048+0000","last_undegraded":"2024-07-04T17:30:48.448048+0000","last_fullsized":"2024-07-04T17:30:48.448048+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:41:29.873067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.389700+0000","last_change":"2024-07-04T17:30:47.423800+0000","last_active":"2024-07-04T17:30:48.389700+0000","last_peered":"2024-07-04T17:30:48.389700+0000","last_clean":"2024-07-04T17:30:48.389700+0000","last_became_active":"2024-07-04T17:30:47.423361+0000","last_became_peered":"2024-07-04T17:30:47.423361+0000","last_unstale":"2024-07-04T17:30:48.389700+0000","last_undegraded":"2024-07-04T17:30:48.389700+0000","last_fullsized":"2024-07-04T17:30:48.389700+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:25:11.379938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.389171+0000","last_change":"2024-07-04T17:30:47.422048+0000","last_active":"2024-07-04T17:30:48.389171+0000","last_peered":"2024-07-04T17:30:48.389171+0000","last_clean":"2024-07-04T17:30:48.389171+0000","last_became_active":"2024-07-04T17:30:47.421591+0000","last_became_peered":"2024-07-04T17:30:47.421591+0000","last_unstale":"2024-07-04T17:30:48.389171+0000","last_undegraded":"2024-07-04T17:30:48.389171+0000","last_fullsized":"2024-07-04T17:30:48.389171+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:58:22.418379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.389433+0000","last_change":"2024-07-04T17:30:47.414496+0000","last_active":"2024-07-04T17:30:48.389433+0000","last_peered":"2024-07-04T17:30:48.389433+0000","last_clean":"2024-07-04T17:30:48.389433+0000","last_became_active":"2024-07-04T17:30:47.414141+0000","last_became_peered":"2024-07-04T17:30:47.414141+0000","last_unstale":"2024-07-04T17:30:48.389433+0000","last_undegraded":"2024-07-04T17:30:48.389433+0000","last_fullsized":"2024-07-04T17:30:48.389433+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:09:08.612743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.388769+0000","last_change":"2024-07-04T17:30:47.419292+0000","last_active":"2024-07-04T17:30:48.388769+0000","last_peered":"2024-07-04T17:30:48.388769+0000","last_clean":"2024-07-04T17:30:48.388769+0000","last_became_active":"2024-07-04T17:30:47.418053+0000","last_became_peered":"2024-07-04T17:30:47.418053+0000","last_unstale":"2024-07-04T17:30:48.388769+0000","last_undegraded":"2024-07-04T17:30:48.388769+0000","last_fullsized":"2024-07-04T17:30:48.388769+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:02:17.626429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.388802+0000","last_change":"2024-07-04T17:30:47.436774+0000","last_active":"2024-07-04T17:30:48.388802+0000","last_peered":"2024-07-04T17:30:48.388802+0000","last_clean":"2024-07-04T17:30:48.388802+0000","last_became_active":"2024-07-04T17:30:47.436573+0000","last_became_peered":"2024-07-04T17:30:47.436573+0000","last_unstale":"2024-07-04T17:30:48.388802+0000","last_undegraded":"2024-07-04T17:30:48.388802+0000","last_fullsized":"2024-07-04T17:30:48.388802+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:27:06.393591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.674818+0000","last_change":"2024-07-04T17:30:47.389819+0000","last_active":"2024-07-04T17:30:48.674818+0000","last_peered":"2024-07-04T17:30:48.674818+0000","last_clean":"2024-07-04T17:30:48.674818+0000","last_became_active":"2024-07-04T17:30:47.387439+0000","last_became_peered":"2024-07-04T17:30:47.387439+0000","last_unstale":"2024-07-04T17:30:48.674818+0000","last_undegraded":"2024-07-04T17:30:48.674818+0000","last_fullsized":"2024-07-04T17:30:48.674818+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:56:14.415266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.447755+0000","last_change":"2024-07-04T17:30:47.395945+0000","last_active":"2024-07-04T17:30:48.447755+0000","last_peered":"2024-07-04T17:30:48.447755+0000","last_clean":"2024-07-04T17:30:48.447755+0000","last_became_active":"2024-07-04T17:30:47.395631+0000","last_became_peered":"2024-07-04T17:30:47.395631+0000","last_unstale":"2024-07-04T17:30:48.447755+0000","last_undegraded":"2024-07-04T17:30:48.447755+0000","last_fullsized":"2024-07-04T17:30:48.447755+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:27:30.220822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.389806+0000","last_change":"2024-07-04T17:30:47.436357+0000","last_active":"2024-07-04T17:30:48.389806+0000","last_peered":"2024-07-04T17:30:48.389806+0000","last_clean":"2024-07-04T17:30:48.389806+0000","last_became_active":"2024-07-04T17:30:47.435542+0000","last_became_peered":"2024-07-04T17:30:47.435542+0000","last_unstale":"2024-07-04T17:30:48.389806+0000","last_undegraded":"2024-07-04T17:30:48.389806+0000","last_fullsized":"2024-07-04T17:30:48.389806+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:08:21.911035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.447172+0000","last_change":"2024-07-04T17:30:47.430335+0000","last_active":"2024-07-04T17:30:48.447172+0000","last_peered":"2024-07-04T17:30:48.447172+0000","last_clean":"2024-07-04T17:30:48.447172+0000","last_became_active":"2024-07-04T17:30:47.427068+0000","last_became_peered":"2024-07-04T17:30:47.427068+0000","last_unstale":"2024-07-04T17:30:48.447172+0000","last_undegraded":"2024-07-04T17:30:48.447172+0000","last_fullsized":"2024-07-04T17:30:48.447172+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:09:53.897531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.675565+0000","last_change":"2024-07-04T17:30:47.398367+0000","last_active":"2024-07-04T17:30:48.675565+0000","last_peered":"2024-07-04T17:30:48.675565+0000","last_clean":"2024-07-04T17:30:48.675565+0000","last_became_active":"2024-07-04T17:30:47.394431+0000","last_became_peered":"2024-07-04T17:30:47.394431+0000","last_unstale":"2024-07-04T17:30:48.675565+0000","last_undegraded":"2024-07-04T17:30:48.675565+0000","last_fullsized":"2024-07-04T17:30:48.675565+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:14:28.773355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.676075+0000","last_change":"2024-07-04T17:30:47.426130+0000","last_active":"2024-07-04T17:30:48.676075+0000","last_peered":"2024-07-04T17:30:48.676075+0000","last_clean":"2024-07-04T17:30:48.676075+0000","last_became_active":"2024-07-04T17:30:47.416510+0000","last_became_peered":"2024-07-04T17:30:47.416510+0000","last_unstale":"2024-07-04T17:30:48.676075+0000","last_undegraded":"2024-07-04T17:30:48.676075+0000","last_fullsized":"2024-07-04T17:30:48.676075+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:53:42.201549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.389174+0000","last_change":"2024-07-04T17:30:47.417678+0000","last_active":"2024-07-04T17:30:48.389174+0000","last_peered":"2024-07-04T17:30:48.389174+0000","last_clean":"2024-07-04T17:30:48.389174+0000","last_became_active":"2024-07-04T17:30:47.414976+0000","last_became_peered":"2024-07-04T17:30:47.414976+0000","last_unstale":"2024-07-04T17:30:48.389174+0000","last_undegraded":"2024-07-04T17:30:48.389174+0000","last_fullsized":"2024-07-04T17:30:48.389174+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:40:08.588002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.444678+0000","last_change":"2024-07-04T17:30:47.396959+0000","last_active":"2024-07-04T17:30:48.444678+0000","last_peered":"2024-07-04T17:30:48.444678+0000","last_clean":"2024-07-04T17:30:48.444678+0000","last_became_active":"2024-07-04T17:30:47.396561+0000","last_became_peered":"2024-07-04T17:30:47.396561+0000","last_unstale":"2024-07-04T17:30:48.444678+0000","last_undegraded":"2024-07-04T17:30:48.444678+0000","last_fullsized":"2024-07-04T17:30:48.444678+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:03:40.493555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.388228+0000","last_change":"2024-07-04T17:30:47.420235+0000","last_active":"2024-07-04T17:30:48.388228+0000","last_peered":"2024-07-04T17:30:48.388228+0000","last_clean":"2024-07-04T17:30:48.388228+0000","last_became_active":"2024-07-04T17:30:47.419803+0000","last_became_peered":"2024-07-04T17:30:47.419803+0000","last_unstale":"2024-07-04T17:30:48.388228+0000","last_undegraded":"2024-07-04T17:30:48.388228+0000","last_fullsized":"2024-07-04T17:30:48.388228+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:27:30.220822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.677831+0000","last_change":"2024-07-04T17:30:47.426146+0000","last_active":"2024-07-04T17:30:48.677831+0000","last_peered":"2024-07-04T17:30:48.677831+0000","last_clean":"2024-07-04T17:30:48.677831+0000","last_became_active":"2024-07-04T17:30:47.421447+0000","last_became_peered":"2024-07-04T17:30:47.421447+0000","last_unstale":"2024-07-04T17:30:48.677831+0000","last_undegraded":"2024-07-04T17:30:48.677831+0000","last_fullsized":"2024-07-04T17:30:48.677831+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:14:40.880998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.390674+0000","last_change":"2024-07-04T17:30:47.436646+0000","last_active":"2024-07-04T17:30:48.390674+0000","last_peered":"2024-07-04T17:30:48.390674+0000","last_clean":"2024-07-04T17:30:48.390674+0000","last_became_active":"2024-07-04T17:30:47.435999+0000","last_became_peered":"2024-07-04T17:30:47.435999+0000","last_unstale":"2024-07-04T17:30:48.390674+0000","last_undegraded":"2024-07-04T17:30:48.390674+0000","last_fullsized":"2024-07-04T17:30:48.390674+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:27:44.098370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.676452+0000","last_change":"2024-07-04T17:30:47.398164+0000","last_active":"2024-07-04T17:30:48.676452+0000","last_peered":"2024-07-04T17:30:48.676452+0000","last_clean":"2024-07-04T17:30:48.676452+0000","last_became_active":"2024-07-04T17:30:47.394563+0000","last_became_peered":"2024-07-04T17:30:47.394563+0000","last_unstale":"2024-07-04T17:30:48.676452+0000","last_undegraded":"2024-07-04T17:30:48.676452+0000","last_fullsized":"2024-07-04T17:30:48.676452+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:51:16.498378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.389248+0000","last_change":"2024-07-04T17:30:47.438690+0000","last_active":"2024-07-04T17:30:48.389248+0000","last_peered":"2024-07-04T17:30:48.389248+0000","last_clean":"2024-07-04T17:30:48.389248+0000","last_became_active":"2024-07-04T17:30:47.438362+0000","last_became_peered":"2024-07-04T17:30:47.438362+0000","last_unstale":"2024-07-04T17:30:48.389248+0000","last_undegraded":"2024-07-04T17:30:48.389248+0000","last_fullsized":"2024-07-04T17:30:48.389248+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:28:11.855854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.389658+0000","last_change":"2024-07-04T17:30:47.435331+0000","last_active":"2024-07-04T17:30:48.389658+0000","last_peered":"2024-07-04T17:30:48.389658+0000","last_clean":"2024-07-04T17:30:48.389658+0000","last_became_active":"2024-07-04T17:30:47.434100+0000","last_became_peered":"2024-07-04T17:30:47.434100+0000","last_unstale":"2024-07-04T17:30:48.389658+0000","last_undegraded":"2024-07-04T17:30:48.389658+0000","last_fullsized":"2024-07-04T17:30:48.389658+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:47:32.884542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.445752+0000","last_change":"2024-07-04T17:30:47.399163+0000","last_active":"2024-07-04T17:30:48.445752+0000","last_peered":"2024-07-04T17:30:48.445752+0000","last_clean":"2024-07-04T17:30:48.445752+0000","last_became_active":"2024-07-04T17:30:47.398747+0000","last_became_peered":"2024-07-04T17:30:47.398747+0000","last_unstale":"2024-07-04T17:30:48.445752+0000","last_undegraded":"2024-07-04T17:30:48.445752+0000","last_fullsized":"2024-07-04T17:30:48.445752+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:24:01.574518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.674332+0000","last_change":"2024-07-04T17:30:47.421142+0000","last_active":"2024-07-04T17:30:48.674332+0000","last_peered":"2024-07-04T17:30:48.674332+0000","last_clean":"2024-07-04T17:30:48.674332+0000","last_became_active":"2024-07-04T17:30:47.416769+0000","last_became_peered":"2024-07-04T17:30:47.416769+0000","last_unstale":"2024-07-04T17:30:48.674332+0000","last_undegraded":"2024-07-04T17:30:48.674332+0000","last_fullsized":"2024-07-04T17:30:48.674332+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:19:39.765910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.388251+0000","last_change":"2024-07-04T17:30:47.421958+0000","last_active":"2024-07-04T17:30:48.388251+0000","last_peered":"2024-07-04T17:30:48.388251+0000","last_clean":"2024-07-04T17:30:48.388251+0000","last_became_active":"2024-07-04T17:30:47.421313+0000","last_became_peered":"2024-07-04T17:30:47.421313+0000","last_unstale":"2024-07-04T17:30:48.388251+0000","last_undegraded":"2024-07-04T17:30:48.388251+0000","last_fullsized":"2024-07-04T17:30:48.388251+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:44:44.984203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.447909+0000","last_change":"2024-07-04T17:30:47.398351+0000","last_active":"2024-07-04T17:30:48.447909+0000","last_peered":"2024-07-04T17:30:48.447909+0000","last_clean":"2024-07-04T17:30:48.447909+0000","last_became_active":"2024-07-04T17:30:47.395987+0000","last_became_peered":"2024-07-04T17:30:47.395987+0000","last_unstale":"2024-07-04T17:30:48.447909+0000","last_undegraded":"2024-07-04T17:30:48.447909+0000","last_fullsized":"2024-07-04T17:30:48.447909+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:44:44.984203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.448046+0000","last_change":"2024-07-04T17:30:47.398645+0000","last_active":"2024-07-04T17:30:48.448046+0000","last_peered":"2024-07-04T17:30:48.448046+0000","last_clean":"2024-07-04T17:30:48.448046+0000","last_became_active":"2024-07-04T17:30:47.394765+0000","last_became_peered":"2024-07-04T17:30:47.394765+0000","last_unstale":"2024-07-04T17:30:48.448046+0000","last_undegraded":"2024-07-04T17:30:48.448046+0000","last_fullsized":"2024-07-04T17:30:48.448046+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:59:48.588528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.390792+0000","last_change":"2024-07-04T17:30:47.439010+0000","last_active":"2024-07-04T17:30:48.390792+0000","last_peered":"2024-07-04T17:30:48.390792+0000","last_clean":"2024-07-04T17:30:48.390792+0000","last_became_active":"2024-07-04T17:30:47.436372+0000","last_became_peered":"2024-07-04T17:30:47.436372+0000","last_unstale":"2024-07-04T17:30:48.390792+0000","last_undegraded":"2024-07-04T17:30:48.390792+0000","last_fullsized":"2024-07-04T17:30:48.390792+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:08:49.032762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.446678+0000","last_change":"2024-07-04T17:30:47.430081+0000","last_active":"2024-07-04T17:30:48.446678+0000","last_peered":"2024-07-04T17:30:48.446678+0000","last_clean":"2024-07-04T17:30:48.446678+0000","last_became_active":"2024-07-04T17:30:47.425964+0000","last_became_peered":"2024-07-04T17:30:47.425964+0000","last_unstale":"2024-07-04T17:30:48.446678+0000","last_undegraded":"2024-07-04T17:30:48.446678+0000","last_fullsized":"2024-07-04T17:30:48.446678+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:44:12.909185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.445756+0000","last_change":"2024-07-04T17:30:47.389158+0000","last_active":"2024-07-04T17:30:48.445756+0000","last_peered":"2024-07-04T17:30:48.445756+0000","last_clean":"2024-07-04T17:30:48.445756+0000","last_became_active":"2024-07-04T17:30:47.388127+0000","last_became_peered":"2024-07-04T17:30:47.388127+0000","last_unstale":"2024-07-04T17:30:48.445756+0000","last_undegraded":"2024-07-04T17:30:48.445756+0000","last_fullsized":"2024-07-04T17:30:48.445756+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:49:47.842800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.446376+0000","last_change":"2024-07-04T17:30:47.427109+0000","last_active":"2024-07-04T17:30:48.446376+0000","last_peered":"2024-07-04T17:30:48.446376+0000","last_clean":"2024-07-04T17:30:48.446376+0000","last_became_active":"2024-07-04T17:30:47.426697+0000","last_became_peered":"2024-07-04T17:30:47.426697+0000","last_unstale":"2024-07-04T17:30:48.446376+0000","last_undegraded":"2024-07-04T17:30:48.446376+0000","last_fullsized":"2024-07-04T17:30:48.446376+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:20:37.646569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.391183+0000","last_change":"2024-07-04T17:30:47.435621+0000","last_active":"2024-07-04T17:30:48.391183+0000","last_peered":"2024-07-04T17:30:48.391183+0000","last_clean":"2024-07-04T17:30:48.391183+0000","last_became_active":"2024-07-04T17:30:47.435230+0000","last_became_peered":"2024-07-04T17:30:47.435230+0000","last_unstale":"2024-07-04T17:30:48.391183+0000","last_undegraded":"2024-07-04T17:30:48.391183+0000","last_fullsized":"2024-07-04T17:30:48.391183+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:29:26.886812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.388559+0000","last_change":"2024-07-04T17:30:47.414726+0000","last_active":"2024-07-04T17:30:48.388559+0000","last_peered":"2024-07-04T17:30:48.388559+0000","last_clean":"2024-07-04T17:30:48.388559+0000","last_became_active":"2024-07-04T17:30:47.414412+0000","last_became_peered":"2024-07-04T17:30:47.414412+0000","last_unstale":"2024-07-04T17:30:48.388559+0000","last_undegraded":"2024-07-04T17:30:48.388559+0000","last_fullsized":"2024-07-04T17:30:48.388559+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:59:48.588528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.677238+0000","last_change":"2024-07-04T17:30:47.426760+0000","last_active":"2024-07-04T17:30:48.677238+0000","last_peered":"2024-07-04T17:30:48.677238+0000","last_clean":"2024-07-04T17:30:48.677238+0000","last_became_active":"2024-07-04T17:30:47.426331+0000","last_became_peered":"2024-07-04T17:30:47.426331+0000","last_unstale":"2024-07-04T17:30:48.677238+0000","last_undegraded":"2024-07-04T17:30:48.677238+0000","last_fullsized":"2024-07-04T17:30:48.677238+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:47:13.050952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"71'4","reported_seq":18,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.677594+0000","last_change":"2024-07-04T17:30:46.417984+0000","last_active":"2024-07-04T17:30:48.677594+0000","last_peered":"2024-07-04T17:30:48.677594+0000","last_clean":"2024-07-04T17:30:48.677594+0000","last_became_active":"2024-07-04T17:30:46.417091+0000","last_became_peered":"2024-07-04T17:30:46.417091+0000","last_unstale":"2024-07-04T17:30:48.677594+0000","last_undegraded":"2024-07-04T17:30:48.677594+0000","last_fullsized":"2024-07-04T17:30:48.677594+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:29:34.252784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.448229+0000","last_change":"2024-07-04T17:30:47.419884+0000","last_active":"2024-07-04T17:30:48.448229+0000","last_peered":"2024-07-04T17:30:48.448229+0000","last_clean":"2024-07-04T17:30:48.448229+0000","last_became_active":"2024-07-04T17:30:47.419588+0000","last_became_peered":"2024-07-04T17:30:47.419588+0000","last_unstale":"2024-07-04T17:30:48.448229+0000","last_undegraded":"2024-07-04T17:30:48.448229+0000","last_fullsized":"2024-07-04T17:30:48.448229+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:02:17.626429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"71'1","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.390445+0000","last_change":"2024-07-04T17:30:46.394693+0000","last_active":"2024-07-04T17:30:48.390445+0000","last_peered":"2024-07-04T17:30:48.390445+0000","last_clean":"2024-07-04T17:30:48.390445+0000","last_became_active":"2024-07-04T17:30:46.394315+0000","last_became_peered":"2024-07-04T17:30:46.394315+0000","last_unstale":"2024-07-04T17:30:48.390445+0000","last_undegraded":"2024-07-04T17:30:48.390445+0000","last_fullsized":"2024-07-04T17:30:48.390445+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:56:39.574079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.675889+0000","last_change":"2024-07-04T17:30:47.398047+0000","last_active":"2024-07-04T17:30:48.675889+0000","last_peered":"2024-07-04T17:30:48.675889+0000","last_clean":"2024-07-04T17:30:48.675889+0000","last_became_active":"2024-07-04T17:30:47.393052+0000","last_became_peered":"2024-07-04T17:30:47.393052+0000","last_unstale":"2024-07-04T17:30:48.675889+0000","last_undegraded":"2024-07-04T17:30:48.675889+0000","last_fullsized":"2024-07-04T17:30:48.675889+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:13:17.681466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"71'2","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.676938+0000","last_change":"2024-07-04T17:30:46.401872+0000","last_active":"2024-07-04T17:30:48.676938+0000","last_peered":"2024-07-04T17:30:48.676938+0000","last_clean":"2024-07-04T17:30:48.676938+0000","last_became_active":"2024-07-04T17:30:46.401510+0000","last_became_peered":"2024-07-04T17:30:46.401510+0000","last_unstale":"2024-07-04T17:30:48.676938+0000","last_undegraded":"2024-07-04T17:30:48.676938+0000","last_fullsized":"2024-07-04T17:30:48.676938+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:02:03.254039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.447224+0000","last_change":"2024-07-04T17:30:47.418834+0000","last_active":"2024-07-04T17:30:48.447224+0000","last_peered":"2024-07-04T17:30:48.447224+0000","last_clean":"2024-07-04T17:30:48.447224+0000","last_became_active":"2024-07-04T17:30:47.418449+0000","last_became_peered":"2024-07-04T17:30:47.418449+0000","last_unstale":"2024-07-04T17:30:48.447224+0000","last_undegraded":"2024-07-04T17:30:48.447224+0000","last_fullsized":"2024-07-04T17:30:48.447224+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:59:40.100483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"71'2","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.676054+0000","last_change":"2024-07-04T17:30:46.417993+0000","last_active":"2024-07-04T17:30:48.676054+0000","last_peered":"2024-07-04T17:30:48.676054+0000","last_clean":"2024-07-04T17:30:48.676054+0000","last_became_active":"2024-07-04T17:30:46.417603+0000","last_became_peered":"2024-07-04T17:30:46.417603+0000","last_unstale":"2024-07-04T17:30:48.676054+0000","last_undegraded":"2024-07-04T17:30:48.676054+0000","last_fullsized":"2024-07-04T17:30:48.676054+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:18:43.863213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.446247+0000","last_change":"2024-07-04T17:30:47.427026+0000","last_active":"2024-07-04T17:30:48.446247+0000","last_peered":"2024-07-04T17:30:48.446247+0000","last_clean":"2024-07-04T17:30:48.446247+0000","last_became_active":"2024-07-04T17:30:47.426173+0000","last_became_peered":"2024-07-04T17:30:47.426173+0000","last_unstale":"2024-07-04T17:30:48.446247+0000","last_undegraded":"2024-07-04T17:30:48.446247+0000","last_fullsized":"2024-07-04T17:30:48.446247+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:00:08.547137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"71'1","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.676456+0000","last_change":"2024-07-04T17:30:46.408705+0000","last_active":"2024-07-04T17:30:48.676456+0000","last_peered":"2024-07-04T17:30:48.676456+0000","last_clean":"2024-07-04T17:30:48.676456+0000","last_became_active":"2024-07-04T17:30:46.408366+0000","last_became_peered":"2024-07-04T17:30:46.408366+0000","last_unstale":"2024-07-04T17:30:48.676456+0000","last_undegraded":"2024-07-04T17:30:48.676456+0000","last_fullsized":"2024-07-04T17:30:48.676456+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:07:19.994029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.390231+0000","last_change":"2024-07-04T17:30:47.435295+0000","last_active":"2024-07-04T17:30:48.390231+0000","last_peered":"2024-07-04T17:30:48.390231+0000","last_clean":"2024-07-04T17:30:48.390231+0000","last_became_active":"2024-07-04T17:30:47.434416+0000","last_became_peered":"2024-07-04T17:30:47.434416+0000","last_unstale":"2024-07-04T17:30:48.390231+0000","last_undegraded":"2024-07-04T17:30:48.390231+0000","last_fullsized":"2024-07-04T17:30:48.390231+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:20:27.029139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"71'1","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.389430+0000","last_change":"2024-07-04T17:30:46.409671+0000","last_active":"2024-07-04T17:30:48.389430+0000","last_peered":"2024-07-04T17:30:48.389430+0000","last_clean":"2024-07-04T17:30:48.389430+0000","last_became_active":"2024-07-04T17:30:46.409031+0000","last_became_peered":"2024-07-04T17:30:46.409031+0000","last_unstale":"2024-07-04T17:30:48.389430+0000","last_undegraded":"2024-07-04T17:30:48.389430+0000","last_fullsized":"2024-07-04T17:30:48.389430+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:06:35.004818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.445364+0000","last_change":"2024-07-04T17:30:47.394313+0000","last_active":"2024-07-04T17:30:48.445364+0000","last_peered":"2024-07-04T17:30:48.445364+0000","last_clean":"2024-07-04T17:30:48.445364+0000","last_became_active":"2024-07-04T17:30:47.393967+0000","last_became_peered":"2024-07-04T17:30:47.393967+0000","last_unstale":"2024-07-04T17:30:48.445364+0000","last_undegraded":"2024-07-04T17:30:48.445364+0000","last_fullsized":"2024-07-04T17:30:48.445364+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:13:54.972134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.674483+0000","last_change":"2024-07-04T17:30:46.414098+0000","last_active":"2024-07-04T17:30:48.674483+0000","last_peered":"2024-07-04T17:30:48.674483+0000","last_clean":"2024-07-04T17:30:48.674483+0000","last_became_active":"2024-07-04T17:30:46.413464+0000","last_became_peered":"2024-07-04T17:30:46.413464+0000","last_unstale":"2024-07-04T17:30:48.674483+0000","last_undegraded":"2024-07-04T17:30:48.674483+0000","last_fullsized":"2024-07-04T17:30:48.674483+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:43:49.510457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.391133+0000","last_change":"2024-07-04T17:30:47.424416+0000","last_active":"2024-07-04T17:30:48.391133+0000","last_peered":"2024-07-04T17:30:48.391133+0000","last_clean":"2024-07-04T17:30:48.391133+0000","last_became_active":"2024-07-04T17:30:47.423168+0000","last_became_peered":"2024-07-04T17:30:47.423168+0000","last_unstale":"2024-07-04T17:30:48.391133+0000","last_undegraded":"2024-07-04T17:30:48.391133+0000","last_fullsized":"2024-07-04T17:30:48.391133+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:02:04.258323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.445674+0000","last_change":"2024-07-04T17:30:46.414564+0000","last_active":"2024-07-04T17:30:48.445674+0000","last_peered":"2024-07-04T17:30:48.445674+0000","last_clean":"2024-07-04T17:30:48.445674+0000","last_became_active":"2024-07-04T17:30:46.414116+0000","last_became_peered":"2024-07-04T17:30:46.414116+0000","last_unstale":"2024-07-04T17:30:48.445674+0000","last_undegraded":"2024-07-04T17:30:48.445674+0000","last_fullsized":"2024-07-04T17:30:48.445674+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:49:01.886056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.677477+0000","last_change":"2024-07-04T17:30:47.397516+0000","last_active":"2024-07-04T17:30:48.677477+0000","last_peered":"2024-07-04T17:30:48.677477+0000","last_clean":"2024-07-04T17:30:48.677477+0000","last_became_active":"2024-07-04T17:30:47.397119+0000","last_became_peered":"2024-07-04T17:30:47.397119+0000","last_unstale":"2024-07-04T17:30:48.677477+0000","last_undegraded":"2024-07-04T17:30:48.677477+0000","last_fullsized":"2024-07-04T17:30:48.677477+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:06:56.459812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.448762+0000","last_change":"2024-07-04T17:30:46.415510+0000","last_active":"2024-07-04T17:30:48.448762+0000","last_peered":"2024-07-04T17:30:48.448762+0000","last_clean":"2024-07-04T17:30:48.448762+0000","last_became_active":"2024-07-04T17:30:46.415113+0000","last_became_peered":"2024-07-04T17:30:46.415113+0000","last_unstale":"2024-07-04T17:30:48.448762+0000","last_undegraded":"2024-07-04T17:30:48.448762+0000","last_fullsized":"2024-07-04T17:30:48.448762+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:13:24.938223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.448422+0000","last_change":"2024-07-04T17:30:47.430636+0000","last_active":"2024-07-04T17:30:48.448422+0000","last_peered":"2024-07-04T17:30:48.448422+0000","last_clean":"2024-07-04T17:30:48.448422+0000","last_became_active":"2024-07-04T17:30:47.425674+0000","last_became_peered":"2024-07-04T17:30:47.425674+0000","last_unstale":"2024-07-04T17:30:48.448422+0000","last_undegraded":"2024-07-04T17:30:48.448422+0000","last_fullsized":"2024-07-04T17:30:48.448422+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:27:06.393591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.390523+0000","last_change":"2024-07-04T17:30:46.407826+0000","last_active":"2024-07-04T17:30:48.390523+0000","last_peered":"2024-07-04T17:30:48.390523+0000","last_clean":"2024-07-04T17:30:48.390523+0000","last_became_active":"2024-07-04T17:30:46.407115+0000","last_became_peered":"2024-07-04T17:30:46.407115+0000","last_unstale":"2024-07-04T17:30:48.390523+0000","last_undegraded":"2024-07-04T17:30:48.390523+0000","last_fullsized":"2024-07-04T17:30:48.390523+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:47:37.889965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.675770+0000","last_change":"2024-07-04T17:30:47.400589+0000","last_active":"2024-07-04T17:30:48.675770+0000","last_peered":"2024-07-04T17:30:48.675770+0000","last_clean":"2024-07-04T17:30:48.675770+0000","last_became_active":"2024-07-04T17:30:47.395213+0000","last_became_peered":"2024-07-04T17:30:47.395213+0000","last_unstale":"2024-07-04T17:30:48.675770+0000","last_undegraded":"2024-07-04T17:30:48.675770+0000","last_fullsized":"2024-07-04T17:30:48.675770+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:02:27.554854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"71'3","reported_seq":17,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.447505+0000","last_change":"2024-07-04T17:30:46.413645+0000","last_active":"2024-07-04T17:30:48.447505+0000","last_peered":"2024-07-04T17:30:48.447505+0000","last_clean":"2024-07-04T17:30:48.447505+0000","last_became_active":"2024-07-04T17:30:46.413314+0000","last_became_peered":"2024-07-04T17:30:46.413314+0000","last_unstale":"2024-07-04T17:30:48.447505+0000","last_undegraded":"2024-07-04T17:30:48.447505+0000","last_fullsized":"2024-07-04T17:30:48.447505+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:56:15.192711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.390793+0000","last_change":"2024-07-04T17:30:47.439341+0000","last_active":"2024-07-04T17:30:48.390793+0000","last_peered":"2024-07-04T17:30:48.390793+0000","last_clean":"2024-07-04T17:30:48.390793+0000","last_became_active":"2024-07-04T17:30:47.438643+0000","last_became_peered":"2024-07-04T17:30:47.438643+0000","last_unstale":"2024-07-04T17:30:48.390793+0000","last_undegraded":"2024-07-04T17:30:48.390793+0000","last_fullsized":"2024-07-04T17:30:48.390793+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:18:57.306251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.14","version":"71'6","reported_seq":20,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.676259+0000","last_change":"2024-07-04T17:30:46.418509+0000","last_active":"2024-07-04T17:30:48.676259+0000","last_peered":"2024-07-04T17:30:48.676259+0000","last_clean":"2024-07-04T17:30:48.676259+0000","last_became_active":"2024-07-04T17:30:46.418161+0000","last_became_peered":"2024-07-04T17:30:46.418161+0000","last_unstale":"2024-07-04T17:30:48.676259+0000","last_undegraded":"2024-07-04T17:30:48.676259+0000","last_fullsized":"2024-07-04T17:30:48.676259+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:26:41.897228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.674710+0000","last_change":"2024-07-04T17:30:47.388099+0000","last_active":"2024-07-04T17:30:48.674710+0000","last_peered":"2024-07-04T17:30:48.674710+0000","last_clean":"2024-07-04T17:30:48.674710+0000","last_became_active":"2024-07-04T17:30:47.387785+0000","last_became_peered":"2024-07-04T17:30:47.387785+0000","last_unstale":"2024-07-04T17:30:48.674710+0000","last_undegraded":"2024-07-04T17:30:48.674710+0000","last_fullsized":"2024-07-04T17:30:48.674710+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:53:19.566358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"71'2","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.446139+0000","last_change":"2024-07-04T17:30:46.404600+0000","last_active":"2024-07-04T17:30:48.446139+0000","last_peered":"2024-07-04T17:30:48.446139+0000","last_clean":"2024-07-04T17:30:48.446139+0000","last_became_active":"2024-07-04T17:30:46.404314+0000","last_became_peered":"2024-07-04T17:30:46.404314+0000","last_unstale":"2024-07-04T17:30:48.446139+0000","last_undegraded":"2024-07-04T17:30:48.446139+0000","last_fullsized":"2024-07-04T17:30:48.446139+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:25:07.446781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.445827+0000","last_change":"2024-07-04T17:30:47.387822+0000","last_active":"2024-07-04T17:30:48.445827+0000","last_peered":"2024-07-04T17:30:48.445827+0000","last_clean":"2024-07-04T17:30:48.445827+0000","last_became_active":"2024-07-04T17:30:47.387451+0000","last_became_peered":"2024-07-04T17:30:47.387451+0000","last_unstale":"2024-07-04T17:30:48.445827+0000","last_undegraded":"2024-07-04T17:30:48.445827+0000","last_fullsized":"2024-07-04T17:30:48.445827+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:46:22.229171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.675153+0000","last_change":"2024-07-04T17:30:46.410149+0000","last_active":"2024-07-04T17:30:48.675153+0000","last_peered":"2024-07-04T17:30:48.675153+0000","last_clean":"2024-07-04T17:30:48.675153+0000","last_became_active":"2024-07-04T17:30:46.409816+0000","last_became_peered":"2024-07-04T17:30:46.409816+0000","last_unstale":"2024-07-04T17:30:48.675153+0000","last_undegraded":"2024-07-04T17:30:48.675153+0000","last_fullsized":"2024-07-04T17:30:48.675153+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:51:54.551650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.445011+0000","last_change":"2024-07-04T17:30:47.431466+0000","last_active":"2024-07-04T17:30:48.445011+0000","last_peered":"2024-07-04T17:30:48.445011+0000","last_clean":"2024-07-04T17:30:48.445011+0000","last_became_active":"2024-07-04T17:30:47.430961+0000","last_became_peered":"2024-07-04T17:30:47.430961+0000","last_unstale":"2024-07-04T17:30:48.445011+0000","last_undegraded":"2024-07-04T17:30:48.445011+0000","last_fullsized":"2024-07-04T17:30:48.445011+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:50:08.536272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.391277+0000","last_change":"2024-07-04T17:30:46.396182+0000","last_active":"2024-07-04T17:30:48.391277+0000","last_peered":"2024-07-04T17:30:48.391277+0000","last_clean":"2024-07-04T17:30:48.391277+0000","last_became_active":"2024-07-04T17:30:46.393987+0000","last_became_peered":"2024-07-04T17:30:46.393987+0000","last_unstale":"2024-07-04T17:30:48.391277+0000","last_undegraded":"2024-07-04T17:30:48.391277+0000","last_fullsized":"2024-07-04T17:30:48.391277+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:44:44.996919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.444809+0000","last_change":"2024-07-04T17:30:47.395226+0000","last_active":"2024-07-04T17:30:48.444809+0000","last_peered":"2024-07-04T17:30:48.444809+0000","last_clean":"2024-07-04T17:30:48.444809+0000","last_became_active":"2024-07-04T17:30:47.394913+0000","last_became_peered":"2024-07-04T17:30:47.394913+0000","last_unstale":"2024-07-04T17:30:48.444809+0000","last_undegraded":"2024-07-04T17:30:48.444809+0000","last_fullsized":"2024-07-04T17:30:48.444809+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:31:15.378585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"71'8","reported_seq":22,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.674846+0000","last_change":"2024-07-04T17:30:46.398618+0000","last_active":"2024-07-04T17:30:48.674846+0000","last_peered":"2024-07-04T17:30:48.674846+0000","last_clean":"2024-07-04T17:30:48.674846+0000","last_became_active":"2024-07-04T17:30:46.398274+0000","last_became_peered":"2024-07-04T17:30:46.398274+0000","last_unstale":"2024-07-04T17:30:48.674846+0000","last_undegraded":"2024-07-04T17:30:48.674846+0000","last_fullsized":"2024-07-04T17:30:48.674846+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:13:50.002315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.677763+0000","last_change":"2024-07-04T17:30:47.397472+0000","last_active":"2024-07-04T17:30:48.677763+0000","last_peered":"2024-07-04T17:30:48.677763+0000","last_clean":"2024-07-04T17:30:48.677763+0000","last_became_active":"2024-07-04T17:30:47.396989+0000","last_became_peered":"2024-07-04T17:30:47.396989+0000","last_unstale":"2024-07-04T17:30:48.677763+0000","last_undegraded":"2024-07-04T17:30:48.677763+0000","last_fullsized":"2024-07-04T17:30:48.677763+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:40:22.762798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"71'1","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.677714+0000","last_change":"2024-07-04T17:30:46.412075+0000","last_active":"2024-07-04T17:30:48.677714+0000","last_peered":"2024-07-04T17:30:48.677714+0000","last_clean":"2024-07-04T17:30:48.677714+0000","last_became_active":"2024-07-04T17:30:46.411727+0000","last_became_peered":"2024-07-04T17:30:46.411727+0000","last_unstale":"2024-07-04T17:30:48.677714+0000","last_undegraded":"2024-07-04T17:30:48.677714+0000","last_fullsized":"2024-07-04T17:30:48.677714+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:12:43.261982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.391044+0000","last_change":"2024-07-04T17:30:47.436755+0000","last_active":"2024-07-04T17:30:48.391044+0000","last_peered":"2024-07-04T17:30:48.391044+0000","last_clean":"2024-07-04T17:30:48.391044+0000","last_became_active":"2024-07-04T17:30:47.436409+0000","last_became_peered":"2024-07-04T17:30:47.436409+0000","last_unstale":"2024-07-04T17:30:48.391044+0000","last_undegraded":"2024-07-04T17:30:48.391044+0000","last_fullsized":"2024-07-04T17:30:48.391044+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:13:51.006598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"71'2","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.678181+0000","last_change":"2024-07-04T17:30:46.409103+0000","last_active":"2024-07-04T17:30:48.678181+0000","last_peered":"2024-07-04T17:30:48.678181+0000","last_clean":"2024-07-04T17:30:48.678181+0000","last_became_active":"2024-07-04T17:30:46.406340+0000","last_became_peered":"2024-07-04T17:30:46.406340+0000","last_unstale":"2024-07-04T17:30:48.678181+0000","last_undegraded":"2024-07-04T17:30:48.678181+0000","last_fullsized":"2024-07-04T17:30:48.678181+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:43:03.614042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.676571+0000","last_change":"2024-07-04T17:30:47.394855+0000","last_active":"2024-07-04T17:30:48.676571+0000","last_peered":"2024-07-04T17:30:48.676571+0000","last_clean":"2024-07-04T17:30:48.676571+0000","last_became_active":"2024-07-04T17:30:47.393038+0000","last_became_peered":"2024-07-04T17:30:47.393038+0000","last_unstale":"2024-07-04T17:30:48.676571+0000","last_undegraded":"2024-07-04T17:30:48.676571+0000","last_fullsized":"2024-07-04T17:30:48.676571+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:25:35.950854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.676208+0000","last_change":"2024-07-04T17:30:46.409923+0000","last_active":"2024-07-04T17:30:48.676208+0000","last_peered":"2024-07-04T17:30:48.676208+0000","last_clean":"2024-07-04T17:30:48.676208+0000","last_became_active":"2024-07-04T17:30:46.409591+0000","last_became_peered":"2024-07-04T17:30:46.409591+0000","last_unstale":"2024-07-04T17:30:48.676208+0000","last_undegraded":"2024-07-04T17:30:48.676208+0000","last_fullsized":"2024-07-04T17:30:48.676208+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:12:05.853612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.389299+0000","last_change":"2024-07-04T17:30:47.419189+0000","last_active":"2024-07-04T17:30:48.389299+0000","last_peered":"2024-07-04T17:30:48.389299+0000","last_clean":"2024-07-04T17:30:48.389299+0000","last_became_active":"2024-07-04T17:30:47.417882+0000","last_became_peered":"2024-07-04T17:30:47.417882+0000","last_unstale":"2024-07-04T17:30:48.389299+0000","last_undegraded":"2024-07-04T17:30:48.389299+0000","last_fullsized":"2024-07-04T17:30:48.389299+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:28:21.769409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"71'2","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.676587+0000","last_change":"2024-07-04T17:30:46.412068+0000","last_active":"2024-07-04T17:30:48.676587+0000","last_peered":"2024-07-04T17:30:48.676587+0000","last_clean":"2024-07-04T17:30:48.676587+0000","last_became_active":"2024-07-04T17:30:46.411711+0000","last_became_peered":"2024-07-04T17:30:46.411711+0000","last_unstale":"2024-07-04T17:30:48.676587+0000","last_undegraded":"2024-07-04T17:30:48.676587+0000","last_fullsized":"2024-07-04T17:30:48.676587+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:42:52.442755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.390468+0000","last_change":"2024-07-04T17:30:47.436121+0000","last_active":"2024-07-04T17:30:48.390468+0000","last_peered":"2024-07-04T17:30:48.390468+0000","last_clean":"2024-07-04T17:30:48.390468+0000","last_became_active":"2024-07-04T17:30:47.435790+0000","last_became_peered":"2024-07-04T17:30:47.435790+0000","last_unstale":"2024-07-04T17:30:48.390468+0000","last_undegraded":"2024-07-04T17:30:48.390468+0000","last_fullsized":"2024-07-04T17:30:48.390468+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:25:40.409565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.389535+0000","last_change":"2024-07-04T17:30:46.408582+0000","last_active":"2024-07-04T17:30:48.389535+0000","last_peered":"2024-07-04T17:30:48.389535+0000","last_clean":"2024-07-04T17:30:48.389535+0000","last_became_active":"2024-07-04T17:30:46.408297+0000","last_became_peered":"2024-07-04T17:30:46.408297+0000","last_unstale":"2024-07-04T17:30:48.389535+0000","last_undegraded":"2024-07-04T17:30:48.389535+0000","last_fullsized":"2024-07-04T17:30:48.389535+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:48:21.667965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.388592+0000","last_change":"2024-07-04T17:30:47.435916+0000","last_active":"2024-07-04T17:30:48.388592+0000","last_peered":"2024-07-04T17:30:48.388592+0000","last_clean":"2024-07-04T17:30:48.388592+0000","last_became_active":"2024-07-04T17:30:47.435562+0000","last_became_peered":"2024-07-04T17:30:47.435562+0000","last_unstale":"2024-07-04T17:30:48.388592+0000","last_undegraded":"2024-07-04T17:30:48.388592+0000","last_fullsized":"2024-07-04T17:30:48.388592+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:41:29.873067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.391449+0000","last_change":"2024-07-04T17:30:46.394311+0000","last_active":"2024-07-04T17:30:48.391449+0000","last_peered":"2024-07-04T17:30:48.391449+0000","last_clean":"2024-07-04T17:30:48.391449+0000","last_became_active":"2024-07-04T17:30:46.393593+0000","last_became_peered":"2024-07-04T17:30:46.393593+0000","last_unstale":"2024-07-04T17:30:48.391449+0000","last_undegraded":"2024-07-04T17:30:48.391449+0000","last_fullsized":"2024-07-04T17:30:48.391449+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:48:17.986899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.674771+0000","last_change":"2024-07-04T17:30:47.420543+0000","last_active":"2024-07-04T17:30:48.674771+0000","last_peered":"2024-07-04T17:30:48.674771+0000","last_clean":"2024-07-04T17:30:48.674771+0000","last_became_active":"2024-07-04T17:30:47.416196+0000","last_became_peered":"2024-07-04T17:30:47.416196+0000","last_unstale":"2024-07-04T17:30:48.674771+0000","last_undegraded":"2024-07-04T17:30:48.674771+0000","last_fullsized":"2024-07-04T17:30:48.674771+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:50:20.456752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.388720+0000","last_change":"2024-07-04T17:30:46.408040+0000","last_active":"2024-07-04T17:30:48.388720+0000","last_peered":"2024-07-04T17:30:48.388720+0000","last_clean":"2024-07-04T17:30:48.388720+0000","last_became_active":"2024-07-04T17:30:46.407167+0000","last_became_peered":"2024-07-04T17:30:46.407167+0000","last_unstale":"2024-07-04T17:30:48.388720+0000","last_undegraded":"2024-07-04T17:30:48.388720+0000","last_fullsized":"2024-07-04T17:30:48.388720+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:46:18.222946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.677970+0000","last_change":"2024-07-04T17:30:47.397337+0000","last_active":"2024-07-04T17:30:48.677970+0000","last_peered":"2024-07-04T17:30:48.677970+0000","last_clean":"2024-07-04T17:30:48.677970+0000","last_became_active":"2024-07-04T17:30:47.396668+0000","last_became_peered":"2024-07-04T17:30:47.396668+0000","last_unstale":"2024-07-04T17:30:48.677970+0000","last_undegraded":"2024-07-04T17:30:48.677970+0000","last_fullsized":"2024-07-04T17:30:48.677970+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:44:37.013444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"71'2","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.449310+0000","last_change":"2024-07-04T17:30:46.414092+0000","last_active":"2024-07-04T17:30:48.449310+0000","last_peered":"2024-07-04T17:30:48.449310+0000","last_clean":"2024-07-04T17:30:48.449310+0000","last_became_active":"2024-07-04T17:30:46.413746+0000","last_became_peered":"2024-07-04T17:30:46.413746+0000","last_unstale":"2024-07-04T17:30:48.449310+0000","last_undegraded":"2024-07-04T17:30:48.449310+0000","last_fullsized":"2024-07-04T17:30:48.449310+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:56:21.477169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.448824+0000","last_change":"2024-07-04T17:30:47.388955+0000","last_active":"2024-07-04T17:30:48.448824+0000","last_peered":"2024-07-04T17:30:48.448824+0000","last_clean":"2024-07-04T17:30:48.448824+0000","last_became_active":"2024-07-04T17:30:47.387588+0000","last_became_peered":"2024-07-04T17:30:47.387588+0000","last_unstale":"2024-07-04T17:30:48.448824+0000","last_undegraded":"2024-07-04T17:30:48.448824+0000","last_fullsized":"2024-07-04T17:30:48.448824+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:58:22.418379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.6","version":"72'1","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.448619+0000","last_change":"2024-07-04T17:30:46.405443+0000","last_active":"2024-07-04T17:30:48.448619+0000","last_peered":"2024-07-04T17:30:48.448619+0000","last_clean":"2024-07-04T17:30:48.448619+0000","last_became_active":"2024-07-04T17:30:46.405066+0000","last_became_peered":"2024-07-04T17:30:46.405066+0000","last_unstale":"2024-07-04T17:30:48.448619+0000","last_undegraded":"2024-07-04T17:30:48.448619+0000","last_fullsized":"2024-07-04T17:30:48.448619+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:34:24.763771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.675947+0000","last_change":"2024-07-04T17:30:47.420498+0000","last_active":"2024-07-04T17:30:48.675947+0000","last_peered":"2024-07-04T17:30:48.675947+0000","last_clean":"2024-07-04T17:30:48.675947+0000","last_became_active":"2024-07-04T17:30:47.416342+0000","last_became_peered":"2024-07-04T17:30:47.416342+0000","last_unstale":"2024-07-04T17:30:48.675947+0000","last_undegraded":"2024-07-04T17:30:48.675947+0000","last_fullsized":"2024-07-04T17:30:48.675947+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:09:18.581049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.677022+0000","last_change":"2024-07-04T17:30:46.401530+0000","last_active":"2024-07-04T17:30:48.677022+0000","last_peered":"2024-07-04T17:30:48.677022+0000","last_clean":"2024-07-04T17:30:48.677022+0000","last_became_active":"2024-07-04T17:30:46.401058+0000","last_became_peered":"2024-07-04T17:30:46.401058+0000","last_unstale":"2024-07-04T17:30:48.677022+0000","last_undegraded":"2024-07-04T17:30:48.677022+0000","last_fullsized":"2024-07-04T17:30:48.677022+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:29:25.882528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.391106+0000","last_change":"2024-07-04T17:30:47.439494+0000","last_active":"2024-07-04T17:30:48.391106+0000","last_peered":"2024-07-04T17:30:48.391106+0000","last_clean":"2024-07-04T17:30:48.391106+0000","last_became_active":"2024-07-04T17:30:47.438655+0000","last_became_peered":"2024-07-04T17:30:47.438655+0000","last_unstale":"2024-07-04T17:30:48.391106+0000","last_undegraded":"2024-07-04T17:30:48.391106+0000","last_fullsized":"2024-07-04T17:30:48.391106+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:37:58.800114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.389300+0000","last_change":"2024-07-04T17:30:46.398565+0000","last_active":"2024-07-04T17:30:48.389300+0000","last_peered":"2024-07-04T17:30:48.389300+0000","last_clean":"2024-07-04T17:30:48.389300+0000","last_became_active":"2024-07-04T17:30:46.394788+0000","last_became_peered":"2024-07-04T17:30:46.394788+0000","last_unstale":"2024-07-04T17:30:48.389300+0000","last_undegraded":"2024-07-04T17:30:48.389300+0000","last_fullsized":"2024-07-04T17:30:48.389300+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:58:03.149737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.446705+0000","last_change":"2024-07-04T17:30:47.426754+0000","last_active":"2024-07-04T17:30:48.446705+0000","last_peered":"2024-07-04T17:30:48.446705+0000","last_clean":"2024-07-04T17:30:48.446705+0000","last_became_active":"2024-07-04T17:30:47.425712+0000","last_became_peered":"2024-07-04T17:30:47.425712+0000","last_unstale":"2024-07-04T17:30:48.446705+0000","last_undegraded":"2024-07-04T17:30:48.446705+0000","last_fullsized":"2024-07-04T17:30:48.446705+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:07:15.865385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.676717+0000","last_change":"2024-07-04T17:30:46.401607+0000","last_active":"2024-07-04T17:30:48.676717+0000","last_peered":"2024-07-04T17:30:48.676717+0000","last_clean":"2024-07-04T17:30:48.676717+0000","last_became_active":"2024-07-04T17:30:46.401314+0000","last_became_peered":"2024-07-04T17:30:46.401314+0000","last_unstale":"2024-07-04T17:30:48.676717+0000","last_undegraded":"2024-07-04T17:30:48.676717+0000","last_fullsized":"2024-07-04T17:30:48.676717+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:21:59.991349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.676664+0000","last_change":"2024-07-04T17:30:47.424977+0000","last_active":"2024-07-04T17:30:48.676664+0000","last_peered":"2024-07-04T17:30:48.676664+0000","last_clean":"2024-07-04T17:30:48.676664+0000","last_became_active":"2024-07-04T17:30:47.416042+0000","last_became_peered":"2024-07-04T17:30:47.416042+0000","last_unstale":"2024-07-04T17:30:48.676664+0000","last_undegraded":"2024-07-04T17:30:48.676664+0000","last_fullsized":"2024-07-04T17:30:48.676664+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:41:40.720108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.389567+0000","last_change":"2024-07-04T17:30:46.408561+0000","last_active":"2024-07-04T17:30:48.389567+0000","last_peered":"2024-07-04T17:30:48.389567+0000","last_clean":"2024-07-04T17:30:48.389567+0000","last_became_active":"2024-07-04T17:30:46.408199+0000","last_became_peered":"2024-07-04T17:30:46.408199+0000","last_unstale":"2024-07-04T17:30:48.389567+0000","last_undegraded":"2024-07-04T17:30:48.389567+0000","last_fullsized":"2024-07-04T17:30:48.389567+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:15:50.228374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.445481+0000","last_change":"2024-07-04T17:30:47.419127+0000","last_active":"2024-07-04T17:30:48.445481+0000","last_peered":"2024-07-04T17:30:48.445481+0000","last_clean":"2024-07-04T17:30:48.445481+0000","last_became_active":"2024-07-04T17:30:47.418557+0000","last_became_peered":"2024-07-04T17:30:47.418557+0000","last_unstale":"2024-07-04T17:30:48.445481+0000","last_undegraded":"2024-07-04T17:30:48.445481+0000","last_fullsized":"2024-07-04T17:30:48.445481+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:16:53.257166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.445422+0000","last_change":"2024-07-04T17:30:46.404540+0000","last_active":"2024-07-04T17:30:48.445422+0000","last_peered":"2024-07-04T17:30:48.445422+0000","last_clean":"2024-07-04T17:30:48.445422+0000","last_became_active":"2024-07-04T17:30:46.402800+0000","last_became_peered":"2024-07-04T17:30:46.402800+0000","last_unstale":"2024-07-04T17:30:48.445422+0000","last_undegraded":"2024-07-04T17:30:48.445422+0000","last_fullsized":"2024-07-04T17:30:48.445422+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:57:35.604132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.445179+0000","last_change":"2024-07-04T17:30:47.427679+0000","last_active":"2024-07-04T17:30:48.445179+0000","last_peered":"2024-07-04T17:30:48.445179+0000","last_clean":"2024-07-04T17:30:48.445179+0000","last_became_active":"2024-07-04T17:30:47.427451+0000","last_became_peered":"2024-07-04T17:30:47.427451+0000","last_unstale":"2024-07-04T17:30:48.445179+0000","last_undegraded":"2024-07-04T17:30:48.445179+0000","last_fullsized":"2024-07-04T17:30:48.445179+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:29:16.626050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.675127+0000","last_change":"2024-07-04T17:30:46.402877+0000","last_active":"2024-07-04T17:30:48.675127+0000","last_peered":"2024-07-04T17:30:48.675127+0000","last_clean":"2024-07-04T17:30:48.675127+0000","last_became_active":"2024-07-04T17:30:46.402551+0000","last_became_peered":"2024-07-04T17:30:46.402551+0000","last_unstale":"2024-07-04T17:30:48.675127+0000","last_undegraded":"2024-07-04T17:30:48.675127+0000","last_fullsized":"2024-07-04T17:30:48.675127+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:37:57.795830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.446820+0000","last_change":"2024-07-04T17:30:47.390337+0000","last_active":"2024-07-04T17:30:48.446820+0000","last_peered":"2024-07-04T17:30:48.446820+0000","last_clean":"2024-07-04T17:30:48.446820+0000","last_became_active":"2024-07-04T17:30:47.389504+0000","last_became_peered":"2024-07-04T17:30:47.389504+0000","last_unstale":"2024-07-04T17:30:48.446820+0000","last_undegraded":"2024-07-04T17:30:48.446820+0000","last_fullsized":"2024-07-04T17:30:48.446820+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:08:26.050071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"71'1","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.447517+0000","last_change":"2024-07-04T17:30:46.398516+0000","last_active":"2024-07-04T17:30:48.447517+0000","last_peered":"2024-07-04T17:30:48.447517+0000","last_clean":"2024-07-04T17:30:48.447517+0000","last_became_active":"2024-07-04T17:30:46.398169+0000","last_became_peered":"2024-07-04T17:30:46.398169+0000","last_unstale":"2024-07-04T17:30:48.447517+0000","last_undegraded":"2024-07-04T17:30:48.447517+0000","last_fullsized":"2024-07-04T17:30:48.447517+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:40:51.947038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"58'80","reported_seq":296,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-07-04T17:30:48.445743+0000","last_change":"2024-07-04T17:26:04.953559+0000","last_active":"2024-07-04T17:30:48.445743+0000","last_peered":"2024-07-04T17:30:48.445743+0000","last_clean":"2024-07-04T17:30:48.445743+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:30:48.445743+0000","last_undegraded":"2024-07-04T17:30:48.445743+0000","last_fullsized":"2024-07-04T17:30:48.445743+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_clean_scrub_stamp":"2024-07-04T17:26:03.933432+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.064181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":296,"num_read_kb":750,"num_write":127,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":296,"num_read_kb":750,"num_write":127,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80,"ondisk_log_size":80,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738427,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31916,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339924,"statfs":{"total":96636764160,"available":96604082176,"internally_reserved":0,"allocated":749568,"data_stored":284321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738428,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36644,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335196,"statfs":{"total":96636764160,"available":96599240704,"internally_reserved":0,"allocated":1327104,"data_stored":872507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738426,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36656,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335184,"statfs":{"total":96636764160,"available":96599228416,"internally_reserved":0,"allocated":1339392,"data_stored":874563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:30:52.707 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-04T17:30:52.707 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-07-04T17:30:53.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.031+0000 7f354e000640 1 -- 172.21.15.42:0/1747161151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f354806d080 msgr2=0x7f35480adee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:53.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.031+0000 7f354e000640 1 --2- 172.21.15.42:0/1747161151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f354806d080 0x7f35480adee0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f35380097b0 tx=0x7f353802a540 comp rx=0 tx=0).stop 2024-07-04T17:30:53.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.031+0000 7f354e000640 1 -- 172.21.15.42:0/1747161151 shutdown_connections 2024-07-04T17:30:53.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.031+0000 7f354e000640 1 --2- 172.21.15.42:0/1747161151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f354806d080 0x7f35480adee0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:53.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.031+0000 7f354e000640 1 -- 172.21.15.42:0/1747161151 >> 172.21.15.42:0/1747161151 conn(0x7f354806c020 msgr2=0x7f354806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:53.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.031+0000 7f354e000640 1 -- 172.21.15.42:0/1747161151 shutdown_connections 2024-07-04T17:30:53.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.031+0000 7f354e000640 1 -- 172.21.15.42:0/1747161151 wait complete. 2024-07-04T17:30:53.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.034+0000 7f354e000640 1 Processor -- start 2024-07-04T17:30:53.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.034+0000 7f354e000640 1 -- start start 2024-07-04T17:30:53.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.036+0000 7f354e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f354806d080 0x7f354813b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:53.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.036+0000 7f354e000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3548141790 con 0x7f354806d080 2024-07-04T17:30:53.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.036+0000 7f3547400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f354806d080 0x7f354813b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:53.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.036+0000 7f3547400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f354806d080 0x7f354813b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39318/0 (socket says 172.21.15.42:39318) 2024-07-04T17:30:53.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.036+0000 7f3547400640 1 -- 172.21.15.42:0/1872491672 learned_addr learned my addr 172.21.15.42:0/1872491672 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:53.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.037+0000 7f3547400640 1 -- 172.21.15.42:0/1872491672 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354813bb00 con 0x7f354806d080 2024-07-04T17:30:53.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.037+0000 7f3547400640 1 --2- 172.21.15.42:0/1872491672 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f354806d080 0x7f354813b510 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f35380098d0 tx=0x7f3538033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:53.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.037+0000 7f353fe00640 1 -- 172.21.15.42:0/1872491672 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3538031bc0 con 0x7f354806d080 2024-07-04T17:30:53.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.038+0000 7f353fe00640 1 -- 172.21.15.42:0/1872491672 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3538031e50 con 0x7f354806d080 2024-07-04T17:30:53.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.038+0000 7f354e000640 1 -- 172.21.15.42:0/1872491672 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35481d74e0 con 0x7f354806d080 2024-07-04T17:30:53.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.038+0000 7f353fe00640 1 -- 172.21.15.42:0/1872491672 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3538033200 con 0x7f354806d080 2024-07-04T17:30:53.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.038+0000 7f354e000640 1 -- 172.21.15.42:0/1872491672 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f354813c1c0 con 0x7f354806d080 2024-07-04T17:30:53.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.039+0000 7f354e000640 1 -- 172.21.15.42:0/1872491672 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3548141e40 con 0x7f354806d080 2024-07-04T17:30:53.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.039+0000 7f353fe00640 1 -- 172.21.15.42:0/1872491672 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f35380333a0 con 0x7f354806d080 2024-07-04T17:30:53.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.040+0000 7f353fe00640 1 --2- 172.21.15.42:0/1872491672 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f352403d4f0 0x7f352403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:53.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.040+0000 7f353fe00640 1 -- 172.21.15.42:0/1872491672 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 3439+0+0 (secure 0 0 0) 0x7f3538072e70 con 0x7f354806d080 2024-07-04T17:30:53.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.040+0000 7f353ea00640 1 --2- 172.21.15.42:0/1872491672 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f352403d4f0 0x7f352403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:53.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.041+0000 7f353ea00640 1 --2- 172.21.15.42:0/1872491672 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f352403d4f0 0x7f352403fa00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f35300015c0 tx=0x7f3530007a90 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:53.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.045+0000 7f353fe00640 1 -- 172.21.15.42:0/1872491672 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f353803f030 con 0x7f354806d080 2024-07-04T17:30:53.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.338+0000 7f354e000640 1 -- 172.21.15.42:0/1872491672 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f3548069b70 con 0x7f354806d080 2024-07-04T17:30:53.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.341+0000 7f353fe00640 1 -- 172.21.15.42:0/1872491672 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f3538033690 con 0x7f354806d080 2024-07-04T17:30:53.342 INFO:teuthology.orchestra.run.smithi042.stdout:[client.0] 2024-07-04T17:30:53.342 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAO24ZmUMmYABAAh4pNlkb9JsHQtK8uCJFQCA== 2024-07-04T17:30:53.342 INFO:teuthology.orchestra.run.smithi042.stdout: caps mds = "allow" 2024-07-04T17:30:53.342 INFO:teuthology.orchestra.run.smithi042.stdout: caps mon = "allow r" 2024-07-04T17:30:53.342 INFO:teuthology.orchestra.run.smithi042.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-07-04T17:30:53.343 INFO:teuthology.orchestra.run.smithi042.stderr:updated caps for client.0 2024-07-04T17:30:53.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.343+0000 7f354e000640 1 -- 172.21.15.42:0/1872491672 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f352403d4f0 msgr2=0x7f352403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:53.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.343+0000 7f354e000640 1 --2- 172.21.15.42:0/1872491672 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f352403d4f0 0x7f352403fa00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f35300015c0 tx=0x7f3530007a90 comp rx=0 tx=0).stop 2024-07-04T17:30:53.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.343+0000 7f354e000640 1 -- 172.21.15.42:0/1872491672 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f354806d080 msgr2=0x7f354813b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:53.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.343+0000 7f354e000640 1 --2- 172.21.15.42:0/1872491672 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f354806d080 0x7f354813b510 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f35380098d0 tx=0x7f3538033e40 comp rx=0 tx=0).stop 2024-07-04T17:30:53.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.343+0000 7f354e000640 1 -- 172.21.15.42:0/1872491672 shutdown_connections 2024-07-04T17:30:53.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.343+0000 7f354e000640 1 --2- 172.21.15.42:0/1872491672 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f352403d4f0 0x7f352403fa00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:53.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.343+0000 7f354e000640 1 --2- 172.21.15.42:0/1872491672 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f354806d080 0x7f354813b510 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:53.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.344+0000 7f354e000640 1 -- 172.21.15.42:0/1872491672 >> 172.21.15.42:0/1872491672 conn(0x7f354806c020 msgr2=0x7f3548140570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:53.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.344+0000 7f354e000640 1 -- 172.21.15.42:0/1872491672 shutdown_connections 2024-07-04T17:30:53.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.344+0000 7f354e000640 1 -- 172.21.15.42:0/1872491672 wait complete. 2024-07-04T17:30:53.372 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:30:53.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.659+0000 7fa5aae00640 1 -- 172.21.15.42:0/2369075654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5a40b1d20 msgr2=0x7fa5a40b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:53.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.659+0000 7fa5aae00640 1 --2- 172.21.15.42:0/2369075654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5a40b1d20 0x7fa5a40b6250 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fa590005fd0 tx=0x7fa59002a5f0 comp rx=0 tx=0).stop 2024-07-04T17:30:53.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.659+0000 7fa5aae00640 1 -- 172.21.15.42:0/2369075654 shutdown_connections 2024-07-04T17:30:53.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.659+0000 7fa5aae00640 1 --2- 172.21.15.42:0/2369075654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5a40b1d20 0x7fa5a40b6250 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:53.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.659+0000 7fa5aae00640 1 -- 172.21.15.42:0/2369075654 >> 172.21.15.42:0/2369075654 conn(0x7fa5a40bebc0 msgr2=0x7fa5a40befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:53.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.659+0000 7fa5aae00640 1 -- 172.21.15.42:0/2369075654 shutdown_connections 2024-07-04T17:30:53.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.659+0000 7fa5aae00640 1 -- 172.21.15.42:0/2369075654 wait complete. 2024-07-04T17:30:53.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.661+0000 7fa5aae00640 1 Processor -- start 2024-07-04T17:30:53.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.662+0000 7fa5aae00640 1 -- start start 2024-07-04T17:30:53.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.663+0000 7fa5aae00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5a40b1d20 0x7fa5a40b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:53.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.663+0000 7fa5aae00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a40b8b40 con 0x7fa5a40b1d20 2024-07-04T17:30:53.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.663+0000 7fa5a9a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5a40b1d20 0x7fa5a40b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:53.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.663+0000 7fa5a9a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5a40b1d20 0x7fa5a40b0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39324/0 (socket says 172.21.15.42:39324) 2024-07-04T17:30:53.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.663+0000 7fa5a9a00640 1 -- 172.21.15.42:0/3244846135 learned_addr learned my addr 172.21.15.42:0/3244846135 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:53.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.664+0000 7fa5a9a00640 1 -- 172.21.15.42:0/3244846135 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a41d7f70 con 0x7fa5a40b1d20 2024-07-04T17:30:53.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.664+0000 7fa5a9a00640 1 --2- 172.21.15.42:0/3244846135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5a40b1d20 0x7fa5a40b0200 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fa59800c320 tx=0x7fa59800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:53.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.664+0000 7fa5a1600640 1 -- 172.21.15.42:0/3244846135 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa598017070 con 0x7fa5a40b1d20 2024-07-04T17:30:53.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.664+0000 7fa5a1600640 1 -- 172.21.15.42:0/3244846135 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa598005310 con 0x7fa5a40b1d20 2024-07-04T17:30:53.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.664+0000 7fa5aae00640 1 -- 172.21.15.42:0/3244846135 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a41d7c80 con 0x7fa5a40b1d20 2024-07-04T17:30:53.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.664+0000 7fa5a1600640 1 -- 172.21.15.42:0/3244846135 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5980035f0 con 0x7fa5a40b1d20 2024-07-04T17:30:53.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.664+0000 7fa5aae00640 1 -- 172.21.15.42:0/3244846135 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a413e230 con 0x7fa5a40b1d20 2024-07-04T17:30:53.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.666+0000 7fa5aae00640 1 -- 172.21.15.42:0/3244846135 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5a40b9170 con 0x7fa5a40b1d20 2024-07-04T17:30:53.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.666+0000 7fa5a1600640 1 -- 172.21.15.42:0/3244846135 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7fa59801a070 con 0x7fa5a40b1d20 2024-07-04T17:30:53.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.666+0000 7fa5a1600640 1 --2- 172.21.15.42:0/3244846135 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fa58403d4f0 0x7fa58403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:53.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.666+0000 7fa5a1600640 1 -- 172.21.15.42:0/3244846135 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 3439+0+0 (secure 0 0 0) 0x7fa598050bd0 con 0x7fa5a40b1d20 2024-07-04T17:30:53.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.667+0000 7fa5a9000640 1 --2- 172.21.15.42:0/3244846135 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fa58403d4f0 0x7fa58403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:53.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.667+0000 7fa5a9000640 1 --2- 172.21.15.42:0/3244846135 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fa58403d4f0 0x7fa58403fa00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa590033000 tx=0x7fa590004730 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:53.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.671+0000 7fa5a1600640 1 -- 172.21.15.42:0/3244846135 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fa59801ace0 con 0x7fa5a40b1d20 2024-07-04T17:30:53.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.911+0000 7fa5aae00640 1 -- 172.21.15.42:0/3244846135 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5a4068e30 con 0x7fa5a40b1d20 2024-07-04T17:30:53.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.913+0000 7fa5a1600640 1 -- 172.21.15.42:0/3244846135 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+6050 (secure 0 0 0) 0x7fa59801e0d0 con 0x7fa5a40b1d20 2024-07-04T17:30:53.914 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:30:53.914 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":54,"btime":"2024-07-04T17:30:50:394659+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5017,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/2312272332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2312272332},{"type":"v1","addr":"172.21.15.42:6827","nonce":2312272332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5020,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1285161495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1285161495},{"type":"v1","addr":"172.21.15.42:6829","nonce":1285161495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5023,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/1583070649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1583070649},{"type":"v1","addr":"172.21.15.42:6831","nonce":1583070649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5026,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6833/2038326930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2038326930},{"type":"v1","addr":"172.21.15.42:6833","nonce":2038326930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:30:46.947014+0000","modified":"2024-07-04T17:30:49.991386+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5029":{"gid":5029,"name":"e","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6835/1458267161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1458267161},{"type":"v1","addr":"172.21.15.42:6835","nonce":1458267161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5029,"qdb_cluster":[5029]},"id":6}]} 2024-07-04T17:30:53.914 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 54 2024-07-04T17:30:53.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.915+0000 7fa5aae00640 1 -- 172.21.15.42:0/3244846135 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fa58403d4f0 msgr2=0x7fa58403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:53.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.915+0000 7fa5aae00640 1 --2- 172.21.15.42:0/3244846135 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fa58403d4f0 0x7fa58403fa00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa590033000 tx=0x7fa590004730 comp rx=0 tx=0).stop 2024-07-04T17:30:53.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.915+0000 7fa5aae00640 1 -- 172.21.15.42:0/3244846135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5a40b1d20 msgr2=0x7fa5a40b0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:53.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.915+0000 7fa5aae00640 1 --2- 172.21.15.42:0/3244846135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5a40b1d20 0x7fa5a40b0200 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fa59800c320 tx=0x7fa59800c7e0 comp rx=0 tx=0).stop 2024-07-04T17:30:53.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.915+0000 7fa5aae00640 1 -- 172.21.15.42:0/3244846135 shutdown_connections 2024-07-04T17:30:53.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.915+0000 7fa5aae00640 1 --2- 172.21.15.42:0/3244846135 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fa58403d4f0 0x7fa58403fa00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:53.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.915+0000 7fa5aae00640 1 --2- 172.21.15.42:0/3244846135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5a40b1d20 0x7fa5a40b0200 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:53.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.915+0000 7fa5aae00640 1 -- 172.21.15.42:0/3244846135 >> 172.21.15.42:0/3244846135 conn(0x7fa5a40bebc0 msgr2=0x7fa5a40befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:53.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.916+0000 7fa5aae00640 1 -- 172.21.15.42:0/3244846135 shutdown_connections 2024-07-04T17:30:53.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:53.916+0000 7fa5aae00640 1 -- 172.21.15.42:0/3244846135 wait complete. 2024-07-04T17:30:54.444 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-04T17:30:46.947014+0000', 'modified': '2024-07-04T17:30:49.991386+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5029}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5029': {'gid': 5029, 'name': 'e', 'rank': 0, 'incarnation': 50, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.42:6835/1458267161', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6834', 'nonce': 1458267161}, {'type': 'v1', 'addr': '172.21.15.42:6835', 'nonce': 1458267161}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5029, 'qdb_cluster': [5029]} 2024-07-04T17:30:54.444 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-04T17:30:54.444 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-07-04T17:30:54.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.746+0000 7f0580e00640 1 -- 172.21.15.42:0/3130313348 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f057c0bcf90 msgr2=0x7f057c0bd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:54.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.746+0000 7f0580e00640 1 --2- 172.21.15.42:0/3130313348 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f057c0bcf90 0x7f057c0bd3a0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f056c005fd0 tx=0x7f056c02aac0 comp rx=0 tx=0).stop 2024-07-04T17:30:54.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.746+0000 7f0580e00640 1 -- 172.21.15.42:0/3130313348 shutdown_connections 2024-07-04T17:30:54.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.746+0000 7f0580e00640 1 --2- 172.21.15.42:0/3130313348 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f057c0bcf90 0x7f057c0bd3a0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:54.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.746+0000 7f0580e00640 1 -- 172.21.15.42:0/3130313348 >> 172.21.15.42:0/3130313348 conn(0x7f057c0688a0 msgr2=0x7f057c06db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:54.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.746+0000 7f0580e00640 1 -- 172.21.15.42:0/3130313348 shutdown_connections 2024-07-04T17:30:54.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.746+0000 7f0580e00640 1 -- 172.21.15.42:0/3130313348 wait complete. 2024-07-04T17:30:54.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.748+0000 7f0580e00640 1 Processor -- start 2024-07-04T17:30:54.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.748+0000 7f0580e00640 1 -- start start 2024-07-04T17:30:54.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.749+0000 7f0580e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f057c0bbef0 0x7f057c0c1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:54.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.749+0000 7f0580e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f057c0b79a0 con 0x7f057c0bbef0 2024-07-04T17:30:54.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.750+0000 7f057b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f057c0bbef0 0x7f057c0c1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:54.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.750+0000 7f057b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f057c0bbef0 0x7f057c0c1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40318/0 (socket says 172.21.15.42:40318) 2024-07-04T17:30:54.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.750+0000 7f057b400640 1 -- 172.21.15.42:0/3479218155 learned_addr learned my addr 172.21.15.42:0/3479218155 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:54.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.750+0000 7f057b400640 1 -- 172.21.15.42:0/3479218155 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f057c0c2370 con 0x7f057c0bbef0 2024-07-04T17:30:54.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.751+0000 7f057b400640 1 --2- 172.21.15.42:0/3479218155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f057c0bbef0 0x7f057c0c1d00 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f056400c320 tx=0x7f056400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:54.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.751+0000 7f0573400640 1 -- 172.21.15.42:0/3479218155 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0564017070 con 0x7f057c0bbef0 2024-07-04T17:30:54.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.751+0000 7f0573400640 1 -- 172.21.15.42:0/3479218155 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05640052f0 con 0x7f057c0bbef0 2024-07-04T17:30:54.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.751+0000 7f0573400640 1 -- 172.21.15.42:0/3479218155 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05640035f0 con 0x7f057c0bbef0 2024-07-04T17:30:54.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.751+0000 7f0580e00640 1 -- 172.21.15.42:0/3479218155 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f057c1e4500 con 0x7f057c0bbef0 2024-07-04T17:30:54.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.751+0000 7f0580e00640 1 -- 172.21.15.42:0/3479218155 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f057c1389f0 con 0x7f057c0bbef0 2024-07-04T17:30:54.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.752+0000 7f0573400640 1 -- 172.21.15.42:0/3479218155 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f056401a070 con 0x7f057c0bbef0 2024-07-04T17:30:54.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.753+0000 7f0573400640 1 --2- 172.21.15.42:0/3479218155 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f055803d4f0 0x7f055803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:54.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.753+0000 7f0573400640 1 -- 172.21.15.42:0/3479218155 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 3439+0+0 (secure 0 0 0) 0x7f0564050bd0 con 0x7f057c0bbef0 2024-07-04T17:30:54.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.754+0000 7f057aa00640 1 --2- 172.21.15.42:0/3479218155 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f055803d4f0 0x7f055803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:54.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.754+0000 7f057aa00640 1 --2- 172.21.15.42:0/3479218155 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f055803d4f0 0x7f055803fa00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f056c035000 tx=0x7f056c004ab0 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:54.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.755+0000 7f0589122740 1 Processor -- start 2024-07-04T17:30:54.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.755+0000 7f0589122740 1 -- start start 2024-07-04T17:30:54.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.757+0000 7f0589122740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235dee3270 0x56235e289d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:54.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.757+0000 7f0589122740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56235e1fa5b0 con 0x56235dee3270 2024-07-04T17:30:54.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.757+0000 7f057b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235dee3270 0x56235e289d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:54.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.757+0000 7f057b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235dee3270 0x56235e289d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40320/0 (socket says 172.21.15.42:40320) 2024-07-04T17:30:54.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.757+0000 7f057b400640 1 -- 172.21.15.42:0/2523418427 learned_addr learned my addr 172.21.15.42:0/2523418427 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:54.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.757+0000 7f057b400640 1 -- 172.21.15.42:0/2523418427 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56235df4e850 con 0x56235dee3270 2024-07-04T17:30:54.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.758+0000 7f057b400640 1 --2- 172.21.15.42:0/2523418427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235dee3270 0x56235e289d30 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f0564019f40 tx=0x7f05640162e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dfae29b4c7bd653 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:54.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.758+0000 7f056be00640 1 -- 172.21.15.42:0/2523418427 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f056404c070 con 0x56235dee3270 2024-07-04T17:30:54.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.758+0000 7f056be00640 1 -- 172.21.15.42:0/2523418427 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0564048a50 con 0x56235dee3270 2024-07-04T17:30:54.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.758+0000 7f056be00640 1 -- 172.21.15.42:0/2523418427 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0564041cf0 con 0x56235dee3270 2024-07-04T17:30:54.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.759+0000 7f0589122740 1 -- 172.21.15.42:0/2523418427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235dee3270 msgr2=0x56235e289d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:54.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.759+0000 7f0589122740 1 --2- 172.21.15.42:0/2523418427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235dee3270 0x56235e289d30 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f0564019f40 tx=0x7f05640162e0 comp rx=0 tx=0).stop 2024-07-04T17:30:54.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.759+0000 7f0589122740 1 -- 172.21.15.42:0/2523418427 shutdown_connections 2024-07-04T17:30:54.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.759+0000 7f0589122740 1 --2- 172.21.15.42:0/2523418427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235dee3270 0x56235e289d30 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:54.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.759+0000 7f0589122740 1 -- 172.21.15.42:0/2523418427 >> 172.21.15.42:0/2523418427 conn(0x56235ded73c0 msgr2=0x56235e2884d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:54.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.759+0000 7f0589122740 1 -- 172.21.15.42:0/2523418427 shutdown_connections 2024-07-04T17:30:54.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.759+0000 7f0589122740 1 -- 172.21.15.42:0/2523418427 wait complete. 2024-07-04T17:30:54.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.760+0000 7f0589122740 1 -- start start 2024-07-04T17:30:54.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.760+0000 7f0589122740 1 Processor -- start 2024-07-04T17:30:54.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.762+0000 7f0589122740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235dee3270 0x56235e2cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:54.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.762+0000 7f0589122740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56235e1edb90 con 0x56235dee3270 2024-07-04T17:30:54.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.762+0000 7f0589122740 10 client.0 _ll_register_callbacks cb 0x56235e1928a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T17:30:54.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.762+0000 7f057b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235dee3270 0x56235e2cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:54.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.762+0000 7f057b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235dee3270 0x56235e2cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40322/0 (socket says 172.21.15.42:40322) 2024-07-04T17:30:54.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.762+0000 7f057b400640 1 -- 172.21.15.42:0/2790446637 learned_addr learned my addr 172.21.15.42:0/2790446637 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:54.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.763+0000 7f057b400640 1 -- 172.21.15.42:0/2790446637 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56235dfc56b0 con 0x56235dee3270 2024-07-04T17:30:54.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.763+0000 7f057b400640 1 --2- 172.21.15.42:0/2790446637 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235dee3270 0x56235e2cc8c0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f0564005040 tx=0x7f0564048c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c3782846fd7e32c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:54.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.763+0000 7f0568c00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-07-04T17:30:54.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.763+0000 7f0568c00640 1 -- 172.21.15.42:0/2790446637 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f056404c070 con 0x56235dee3270 2024-07-04T17:30:54.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.763+0000 7f0568c00640 1 -- 172.21.15.42:0/2790446637 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 3439+0+0 (secure 0 0 0) 0x7f0564003790 con 0x56235dee3270 2024-07-04T17:30:54.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.763+0000 7f0589122740 1 -- 172.21.15.42:0/2790446637 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56235e18a6c0 con 0x56235dee3270 2024-07-04T17:30:54.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.763+0000 7f0568c00640 1 -- 172.21.15.42:0/2790446637 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f0564003990 con 0x56235dee3270 2024-07-04T17:30:54.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.763+0000 7f0589122740 10 client.5082 fetch_fsmap learned FSMap version 54 2024-07-04T17:30:54.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.763+0000 7f0589122740 1 -- 172.21.15.42:0/2790446637 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x56235e2a5930 con 0x56235dee3270 2024-07-04T17:30:54.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.764+0000 7f0568c00640 1 -- 172.21.15.42:0/2790446637 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56235e18a6c0 con 0x56235dee3270 2024-07-04T17:30:54.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.764+0000 7f0568c00640 1 -- 172.21.15.42:0/2790446637 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 54) ==== 4107+0+0 (secure 0 0 0) 0x7f0564019590 con 0x56235dee3270 2024-07-04T17:30:54.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.764+0000 7f0589122740 10 client.5082 fetch_fsmap finished waiting for FSMap version 54 2024-07-04T17:30:54.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.764+0000 7f0589122740 10 client.5082 resolve_mds: resolved name 'e' to daemon mds.e 2024-07-04T17:30:54.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.764+0000 7f0589122740 20 client.5082 populate_metadata read hostname 'smithi042' 2024-07-04T17:30:54.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.764+0000 7f0589122740 1 --2- 172.21.15.42:0/2790446637 >> [v2:172.21.15.42:6834/1458267161,v1:172.21.15.42:6835/1458267161] conn(0x56235deba3b0 0x56235e2b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:54.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.764+0000 7f0589122740 4 client.5082 mds_command: new command op to 5029 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T17:30:54.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.764+0000 7f0589122740 1 -- 172.21.15.42:0/2790446637 --> [v2:172.21.15.42:6834/1458267161,v1:172.21.15.42:6835/1458267161] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56235e1fc500 con 0x56235deba3b0 2024-07-04T17:30:54.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.764+0000 7f057aa00640 1 --2- 172.21.15.42:0/2790446637 >> [v2:172.21.15.42:6834/1458267161,v1:172.21.15.42:6835/1458267161] conn(0x56235deba3b0 0x56235e2b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:54.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.765+0000 7f057aa00640 1 --2- 172.21.15.42:0/2790446637 >> [v2:172.21.15.42:6834/1458267161,v1:172.21.15.42:6835/1458267161] conn(0x56235deba3b0 0x56235e2b19f0 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc9ec41944419122 server_cookie=d4e2c3de93cae4ac in_seq=0 out_seq=0 2024-07-04T17:30:54.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.765+0000 7f0568c00640 10 client.5082 ms_handle_connect on v2:172.21.15.42:6834/1458267161 2024-07-04T17:30:54.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.766+0000 7f0568c00640 1 -- 172.21.15.42:0/2790446637 <== mds.0 v2:172.21.15.42:6834/1458267161 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56235e1fc500 con 0x56235deba3b0 2024-07-04T17:30:54.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.766+0000 7f0568c00640 10 client.5082 handle_command_reply: tid=0 2024-07-04T17:30:54.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.766+0000 7f0589122740 1 client.5082 shutdown 2024-07-04T17:30:54.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.766+0000 7f0589122740 1 -- 172.21.15.42:0/2790446637 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235dee3270 msgr2=0x56235e2cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:54.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.766+0000 7f0589122740 1 --2- 172.21.15.42:0/2790446637 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235dee3270 0x56235e2cc8c0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f0564005040 tx=0x7f0564048c80 comp rx=0 tx=0).stop 2024-07-04T17:30:54.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.766+0000 7f0589122740 1 -- 172.21.15.42:0/2790446637 shutdown_connections 2024-07-04T17:30:54.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.766+0000 7f0589122740 1 --2- 172.21.15.42:0/2790446637 >> [v2:172.21.15.42:6834/1458267161,v1:172.21.15.42:6835/1458267161] conn(0x56235deba3b0 0x56235e2b19f0 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:54.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.766+0000 7f0589122740 1 --2- 172.21.15.42:0/2790446637 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235dee3270 0x56235e2cc8c0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:54.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.766+0000 7f0589122740 1 -- 172.21.15.42:0/2790446637 >> 172.21.15.42:0/2790446637 conn(0x56235ded73c0 msgr2=0x56235e2b5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:54.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.766+0000 7f0568c00640 0 client.5082 ms_handle_reset on v2:172.21.15.42:6834/1458267161 2024-07-04T17:30:54.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.767+0000 7f0589122740 1 -- 172.21.15.42:0/2790446637 shutdown_connections 2024-07-04T17:30:54.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.767+0000 7f0589122740 1 -- 172.21.15.42:0/2790446637 wait complete. 2024-07-04T17:30:54.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.767+0000 7f0589122740 20 client.5082 trim_cache size 0 max 16384 2024-07-04T17:30:54.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.791+0000 7f0589122740 1 Processor -- start 2024-07-04T17:30:54.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.792+0000 7f0589122740 1 -- start start 2024-07-04T17:30:54.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.793+0000 7f0589122740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235dee3270 0x56235e286eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:54.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.793+0000 7f0589122740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f057c1da510 con 0x56235dee3270 2024-07-04T17:30:54.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.793+0000 7f057b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235dee3270 0x56235e286eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:54.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.793+0000 7f057b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235dee3270 0x56235e286eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40332/0 (socket says 172.21.15.42:40332) 2024-07-04T17:30:54.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.793+0000 7f057b400640 1 -- 172.21.15.42:0/2038090031 learned_addr learned my addr 172.21.15.42:0/2038090031 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:54.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.794+0000 7f057b400640 1 -- 172.21.15.42:0/2038090031 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56235e28a0a0 con 0x56235dee3270 2024-07-04T17:30:54.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.794+0000 7f057b400640 1 --2- 172.21.15.42:0/2038090031 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235dee3270 0x56235e286eb0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f0564019fd0 tx=0x7f0564003b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e697110bf56c2c3 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:54.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.794+0000 7f056b400640 1 -- 172.21.15.42:0/2038090031 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f056404c040 con 0x56235dee3270 2024-07-04T17:30:54.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.794+0000 7f056b400640 1 -- 172.21.15.42:0/2038090031 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0564034020 con 0x56235dee3270 2024-07-04T17:30:54.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.794+0000 7f056b400640 1 -- 172.21.15.42:0/2038090031 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0564003790 con 0x56235dee3270 2024-07-04T17:30:54.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.795+0000 7f0589122740 1 -- 172.21.15.42:0/2038090031 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235dee3270 msgr2=0x56235e286eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:54.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.795+0000 7f0589122740 1 --2- 172.21.15.42:0/2038090031 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235dee3270 0x56235e286eb0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f0564019fd0 tx=0x7f0564003b50 comp rx=0 tx=0).stop 2024-07-04T17:30:54.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.795+0000 7f0589122740 1 -- 172.21.15.42:0/2038090031 shutdown_connections 2024-07-04T17:30:54.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.795+0000 7f0589122740 1 --2- 172.21.15.42:0/2038090031 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235dee3270 0x56235e286eb0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:54.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.795+0000 7f0589122740 1 -- 172.21.15.42:0/2038090031 >> 172.21.15.42:0/2038090031 conn(0x56235ded73c0 msgr2=0x56235e2b5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:54.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.795+0000 7f0589122740 1 -- 172.21.15.42:0/2038090031 shutdown_connections 2024-07-04T17:30:54.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.795+0000 7f0589122740 1 -- 172.21.15.42:0/2038090031 wait complete. 2024-07-04T17:30:54.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.796+0000 7f0589122740 1 -- start start 2024-07-04T17:30:54.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.796+0000 7f0589122740 1 Processor -- start 2024-07-04T17:30:54.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.798+0000 7f0589122740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235dee3270 0x56235e2d7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:54.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.798+0000 7f0589122740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56235e1e9b50 con 0x56235dee3270 2024-07-04T17:30:54.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.798+0000 7f0589122740 10 client.0 _ll_register_callbacks cb 0x7f056404e310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T17:30:54.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.798+0000 7f057b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235dee3270 0x56235e2d7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:54.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.798+0000 7f057b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235dee3270 0x56235e2d7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40340/0 (socket says 172.21.15.42:40340) 2024-07-04T17:30:54.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.798+0000 7f057b400640 1 -- 172.21.15.42:0/4252419286 learned_addr learned my addr 172.21.15.42:0/4252419286 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:54.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.799+0000 7f057b400640 1 -- 172.21.15.42:0/4252419286 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56235e2d8aa0 con 0x56235dee3270 2024-07-04T17:30:54.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.799+0000 7f057b400640 1 --2- 172.21.15.42:0/4252419286 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235dee3270 0x56235e2d7930 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f056401aff0 tx=0x7f0564001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3baee4cfbfe04fd server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:54.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.799+0000 7f0568c00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-07-04T17:30:54.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.799+0000 7f0568c00640 1 -- 172.21.15.42:0/4252419286 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f056404c040 con 0x56235dee3270 2024-07-04T17:30:54.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.799+0000 7f0568c00640 1 -- 172.21.15.42:0/4252419286 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 3439+0+0 (secure 0 0 0) 0x7f0564035070 con 0x56235dee3270 2024-07-04T17:30:54.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.799+0000 7f0589122740 1 -- 172.21.15.42:0/4252419286 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0564034020 con 0x56235dee3270 2024-07-04T17:30:54.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.800+0000 7f0568c00640 1 -- 172.21.15.42:0/4252419286 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f0564003790 con 0x56235dee3270 2024-07-04T17:30:54.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.800+0000 7f0589122740 10 client.5084 fetch_fsmap learned FSMap version 54 2024-07-04T17:30:54.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.800+0000 7f0589122740 1 -- 172.21.15.42:0/4252419286 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x56235e2d8aa0 con 0x56235dee3270 2024-07-04T17:30:54.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.800+0000 7f0568c00640 1 -- 172.21.15.42:0/4252419286 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0564034020 con 0x56235dee3270 2024-07-04T17:30:54.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.800+0000 7f0568c00640 1 -- 172.21.15.42:0/4252419286 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 54) ==== 4107+0+0 (secure 0 0 0) 0x7f056401e0d0 con 0x56235dee3270 2024-07-04T17:30:54.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.800+0000 7f0589122740 10 client.5084 fetch_fsmap finished waiting for FSMap version 54 2024-07-04T17:30:54.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.800+0000 7f0589122740 10 client.5084 resolve_mds: resolved name 'e' to daemon mds.e 2024-07-04T17:30:54.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.800+0000 7f0589122740 20 client.5084 populate_metadata read hostname 'smithi042' 2024-07-04T17:30:54.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.800+0000 7f0589122740 1 --2- 172.21.15.42:0/4252419286 >> [v2:172.21.15.42:6834/1458267161,v1:172.21.15.42:6835/1458267161] conn(0x56235dea1e40 0x56235e332e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:54.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.800+0000 7f0589122740 4 client.5084 mds_command: new command op to 5029 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-04T17:30:54.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.800+0000 7f0589122740 1 -- 172.21.15.42:0/4252419286 --> [v2:172.21.15.42:6834/1458267161,v1:172.21.15.42:6835/1458267161] -- command(tid 0: {"prefix": "status"}) -- 0x56235e2ae340 con 0x56235dea1e40 2024-07-04T17:30:54.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.801+0000 7f057aa00640 1 --2- 172.21.15.42:0/4252419286 >> [v2:172.21.15.42:6834/1458267161,v1:172.21.15.42:6835/1458267161] conn(0x56235dea1e40 0x56235e332e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:54.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.801+0000 7f057aa00640 1 --2- 172.21.15.42:0/4252419286 >> [v2:172.21.15.42:6834/1458267161,v1:172.21.15.42:6835/1458267161] conn(0x56235dea1e40 0x56235e332e20 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f63a57a907d2f00 server_cookie=8850f46cd47c843b in_seq=0 out_seq=0 2024-07-04T17:30:54.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.801+0000 7f0568c00640 10 client.5084 ms_handle_connect on v2:172.21.15.42:6834/1458267161 2024-07-04T17:30:54.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.802+0000 7f0568c00640 1 -- 172.21.15.42:0/4252419286 <== mds.0 v2:172.21.15.42:6834/1458267161 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x56235e2ae340 con 0x56235dea1e40 2024-07-04T17:30:54.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.802+0000 7f0568c00640 10 client.5084 handle_command_reply: tid=0 2024-07-04T17:30:54.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.802+0000 7f0589122740 1 client.5084 shutdown 2024-07-04T17:30:54.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.802+0000 7f0589122740 1 -- 172.21.15.42:0/4252419286 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235dee3270 msgr2=0x56235e2d7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:54.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.802+0000 7f0589122740 1 --2- 172.21.15.42:0/4252419286 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235dee3270 0x56235e2d7930 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f056401aff0 tx=0x7f0564001630 comp rx=0 tx=0).stop 2024-07-04T17:30:54.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.802+0000 7f0589122740 1 -- 172.21.15.42:0/4252419286 shutdown_connections 2024-07-04T17:30:54.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.802+0000 7f0589122740 1 --2- 172.21.15.42:0/4252419286 >> [v2:172.21.15.42:6834/1458267161,v1:172.21.15.42:6835/1458267161] conn(0x56235dea1e40 0x56235e332e20 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:54.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.802+0000 7f0589122740 1 --2- 172.21.15.42:0/4252419286 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56235dee3270 0x56235e2d7930 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:54.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.802+0000 7f0589122740 1 -- 172.21.15.42:0/4252419286 >> 172.21.15.42:0/4252419286 conn(0x56235ded73c0 msgr2=0x56235e2a4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:54.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.802+0000 7f0568c00640 0 client.5084 ms_handle_reset on v2:172.21.15.42:6834/1458267161 2024-07-04T17:30:54.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.802+0000 7f0589122740 1 -- 172.21.15.42:0/4252419286 shutdown_connections 2024-07-04T17:30:54.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.802+0000 7f0589122740 1 -- 172.21.15.42:0/4252419286 wait complete. 2024-07-04T17:30:54.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.802+0000 7f0589122740 20 client.5084 trim_cache size 0 max 16384 2024-07-04T17:30:54.805 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:30:54.805 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "be4f300a-93ef-4489-b540-37ece39b0aac", 2024-07-04T17:30:54.805 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-07-04T17:30:54.805 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 5029, 2024-07-04T17:30:54.805 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-07-04T17:30:54.805 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-07-04T17:30:54.805 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-07-04T17:30:54.806 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 7.4383678040000003, 2024-07-04T17:30:54.806 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 54, 2024-07-04T17:30:54.806 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 72, 2024-07-04T17:30:54.806 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 72, 2024-07-04T17:30:54.806 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 12.271069036 2024-07-04T17:30:54.806 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:30:54.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.803+0000 7f0580e00640 1 -- 172.21.15.42:0/3479218155 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f055803d4f0 msgr2=0x7f055803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:54.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.803+0000 7f0580e00640 1 --2- 172.21.15.42:0/3479218155 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f055803d4f0 0x7f055803fa00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f056c035000 tx=0x7f056c004ab0 comp rx=0 tx=0).stop 2024-07-04T17:30:54.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.803+0000 7f0580e00640 1 -- 172.21.15.42:0/3479218155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f057c0bbef0 msgr2=0x7f057c0c1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:54.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.803+0000 7f0580e00640 1 --2- 172.21.15.42:0/3479218155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f057c0bbef0 0x7f057c0c1d00 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f056400c320 tx=0x7f056400c7e0 comp rx=0 tx=0).stop 2024-07-04T17:30:54.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.803+0000 7f0580e00640 1 -- 172.21.15.42:0/3479218155 shutdown_connections 2024-07-04T17:30:54.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.803+0000 7f0580e00640 1 --2- 172.21.15.42:0/3479218155 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f055803d4f0 0x7f055803fa00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:54.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.803+0000 7f0580e00640 1 --2- 172.21.15.42:0/3479218155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f057c0bbef0 0x7f057c0c1d00 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:54.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.803+0000 7f0580e00640 1 -- 172.21.15.42:0/3479218155 >> 172.21.15.42:0/3479218155 conn(0x7f057c0688a0 msgr2=0x7f057c075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:54.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.803+0000 7f0580e00640 1 -- 172.21.15.42:0/3479218155 shutdown_connections 2024-07-04T17:30:54.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:54.803+0000 7f0580e00640 1 -- 172.21.15.42:0/3479218155 wait complete. 2024-07-04T17:30:54.830 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-04T17:30:55.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.115+0000 7f8915000640 1 -- 172.21.15.42:0/1606644230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89100b1c40 msgr2=0x7f89100b2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:55.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.115+0000 7f8915000640 1 --2- 172.21.15.42:0/1606644230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89100b1c40 0x7f89100b2be0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f8910074560 tx=0x7f890002a5f0 comp rx=0 tx=0).stop 2024-07-04T17:30:55.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.115+0000 7f8915000640 1 -- 172.21.15.42:0/1606644230 shutdown_connections 2024-07-04T17:30:55.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.115+0000 7f8915000640 1 --2- 172.21.15.42:0/1606644230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89100b1c40 0x7f89100b2be0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:55.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.115+0000 7f8915000640 1 -- 172.21.15.42:0/1606644230 >> 172.21.15.42:0/1606644230 conn(0x7f8910071960 msgr2=0x7f891006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:55.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.115+0000 7f8915000640 1 -- 172.21.15.42:0/1606644230 shutdown_connections 2024-07-04T17:30:55.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.115+0000 7f8915000640 1 -- 172.21.15.42:0/1606644230 wait complete. 2024-07-04T17:30:55.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.117+0000 7f8915000640 1 Processor -- start 2024-07-04T17:30:55.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.117+0000 7f8915000640 1 -- start start 2024-07-04T17:30:55.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.119+0000 7f8915000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8910073870 0x7f89100bc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:55.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.119+0000 7f8915000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f891014f360 con 0x7f8910073870 2024-07-04T17:30:55.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.119+0000 7f890f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8910073870 0x7f89100bc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:55.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.119+0000 7f890f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8910073870 0x7f89100bc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40356/0 (socket says 172.21.15.42:40356) 2024-07-04T17:30:55.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.119+0000 7f890f400640 1 -- 172.21.15.42:0/1659368819 learned_addr learned my addr 172.21.15.42:0/1659368819 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:55.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.119+0000 7f890f400640 1 -- 172.21.15.42:0/1659368819 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89101e45c0 con 0x7f8910073870 2024-07-04T17:30:55.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.120+0000 7f890f400640 1 --2- 172.21.15.42:0/1659368819 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8910073870 0x7f89100bc680 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f8900009850 tx=0x7f8900004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:55.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.120+0000 7f8907400640 1 -- 172.21.15.42:0/1659368819 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f890003e030 con 0x7f8910073870 2024-07-04T17:30:55.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.120+0000 7f8907400640 1 -- 172.21.15.42:0/1659368819 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89000035f0 con 0x7f8910073870 2024-07-04T17:30:55.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.120+0000 7f8915000640 1 -- 172.21.15.42:0/1659368819 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89100bad20 con 0x7f8910073870 2024-07-04T17:30:55.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.120+0000 7f8907400640 1 -- 172.21.15.42:0/1659368819 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8900035040 con 0x7f8910073870 2024-07-04T17:30:55.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.120+0000 7f8915000640 1 -- 172.21.15.42:0/1659368819 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89100bb1f0 con 0x7f8910073870 2024-07-04T17:30:55.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.122+0000 7f8907400640 1 -- 172.21.15.42:0/1659368819 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f8900041030 con 0x7f8910073870 2024-07-04T17:30:55.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.122+0000 7f8915000640 1 -- 172.21.15.42:0/1659368819 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89100b2be0 con 0x7f8910073870 2024-07-04T17:30:55.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.122+0000 7f8907400640 1 --2- 172.21.15.42:0/1659368819 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f88f003d4f0 0x7f88f003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:55.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.122+0000 7f8907400640 1 -- 172.21.15.42:0/1659368819 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 3439+0+0 (secure 0 0 0) 0x7f890003f070 con 0x7f8910073870 2024-07-04T17:30:55.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.123+0000 7f890ea00640 1 --2- 172.21.15.42:0/1659368819 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f88f003d4f0 0x7f88f003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:55.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.123+0000 7f890ea00640 1 --2- 172.21.15.42:0/1659368819 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f88f003d4f0 0x7f88f003fa00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f88fc001c60 tx=0x7f88fc007d70 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:55.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.129+0000 7f8907400640 1 -- 172.21.15.42:0/1659368819 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f8900005240 con 0x7f8910073870 2024-07-04T17:30:55.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.385+0000 7f8915000640 1 -- 172.21.15.42:0/1659368819 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f89100b3a90 con 0x7f8910073870 2024-07-04T17:30:55.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.386+0000 7f8907400640 1 -- 172.21.15.42:0/1659368819 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v54) ==== 53+0+73 (secure 0 0 0) 0x7f89000720c0 con 0x7f8910073870 2024-07-04T17:30:55.387 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-04T17:30:55.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.389+0000 7f8915000640 1 -- 172.21.15.42:0/1659368819 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f88f003d4f0 msgr2=0x7f88f003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:55.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.389+0000 7f8915000640 1 --2- 172.21.15.42:0/1659368819 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f88f003d4f0 0x7f88f003fa00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f88fc001c60 tx=0x7f88fc007d70 comp rx=0 tx=0).stop 2024-07-04T17:30:55.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.389+0000 7f8915000640 1 -- 172.21.15.42:0/1659368819 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8910073870 msgr2=0x7f89100bc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:55.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.389+0000 7f8915000640 1 --2- 172.21.15.42:0/1659368819 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8910073870 0x7f89100bc680 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f8900009850 tx=0x7f8900004a40 comp rx=0 tx=0).stop 2024-07-04T17:30:55.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.389+0000 7f8915000640 1 -- 172.21.15.42:0/1659368819 shutdown_connections 2024-07-04T17:30:55.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.389+0000 7f8915000640 1 --2- 172.21.15.42:0/1659368819 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f88f003d4f0 0x7f88f003fa00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:55.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.389+0000 7f8915000640 1 --2- 172.21.15.42:0/1659368819 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8910073870 0x7f89100bc680 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:55.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.389+0000 7f8915000640 1 -- 172.21.15.42:0/1659368819 >> 172.21.15.42:0/1659368819 conn(0x7f8910071960 msgr2=0x7f89100c0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:55.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.389+0000 7f8915000640 1 -- 172.21.15.42:0/1659368819 shutdown_connections 2024-07-04T17:30:55.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55.389+0000 7f8915000640 1 -- 172.21.15.42:0/1659368819 wait complete. 2024-07-04T17:30:55.416 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-07-04T17:30:55.417 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-04T17:30:55.417 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2024-07-04T17:30:55.417 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-04T17:30:55.417 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-04T17:30:55.417 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-07-04T17:30:55.417 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-07-04T17:30:55.417 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-04T17:30:55.417 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-04T17:30:55.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:30:55.417 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2024-07-04T17:30:55.465 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-04T17:30:55.465 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-04T17:30:55.465 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-07-04T17:30:55.465 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:30:55.466 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-07-04T17:30:55.466 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:30:55.466 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:30:55.466 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2024-07-04T17:30:55.466 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f0 2024-07-04T17:30:55.466 INFO:teuthology.orchestra.run.smithi042.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:30:55.466 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2024-07-04T17:30:55.466 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f1 2024-07-04T17:30:55.466 INFO:teuthology.orchestra.run.smithi042.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-04T17:30:55.466 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:30:55.466 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f0 2024-07-04T17:30:55.466 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-07-04T17:30:55.466 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 42315sec preferred_lft 42315sec 2024-07-04T17:30:55.467 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::ec4:7aff:febd:1440/64 scope link noprefixroute 2024-07-04T17:30:55.467 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:30:55.467 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:30:55.467 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:30:55.467 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f1 2024-07-04T17:30:55.467 INFO:teuthology.orchestra.run.smithi042.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-04T17:30:55.467 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 32:75:d6:f1:31:e1 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:30:55.467 INFO:teuthology.orchestra.run.smithi042.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-04T17:30:55.467 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:30:55.467 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::8839:a9ff:fe74:17f5/64 scope link 2024-07-04T17:30:55.467 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:30:55.467 INFO:teuthology.orchestra.run.smithi042.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-04T17:30:55.467 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether e6:ab:57:fb:ca:9e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:30:55.467 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::e4ab:57ff:fefb:ca9e/64 scope link 2024-07-04T17:30:55.468 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:30:55.468 INFO:teuthology.orchestra.run.smithi042.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-04T17:30:55.468 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 32:75:d6:f1:31:e1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:30:55.468 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::3075:d6ff:fef1:31e1/64 scope link 2024-07-04T17:30:55.468 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:30:55.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:30:55.469 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-04T17:30:55.545 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-07-04T17:30:55.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:30:55.549 DEBUG:teuthology.orchestra.run.smithi042:> route 2024-07-04T17:30:55.607 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2024-07-04T17:30:55.607 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-04T17:30:55.607 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-04T17:30:55.608 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-04T17:30:55.608 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-04T17:30:55.609 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-04T17:30:55.609 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-07-04T17:30:55.609 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-04T17:30:55.609 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-04T17:30:55.609 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-04T17:30:55.609 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-07-04T17:30:55.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-04T17:30:55.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:30:55.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:30:55.833 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-07-04T17:30:55.882 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-04T17:30:55.882 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2024-07-04T17:30:55.882 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-04T17:30:55.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:30:55.882 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.0 up 2024-07-04T17:30:55.966 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-07-04T17:30:55.966 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-04T17:30:55.967 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-04T17:30:56.022 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-04T17:30:56.022 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-04T17:30:56.022 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-04T17:30:56.076 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:30:56.164 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:30:56.222 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:30:56.222 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:30:56.222 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:30:56.222 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:30:56.222 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:30:56.222 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:30:56.222 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:30:56.222 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:30:56.222 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:30:56.223 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:30:56.223 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:30:56.223 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:30:56.223 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:30:56.223 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:30:56.223 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:30:56.223 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:30:56.223 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:30:56.223 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:30:56.223 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:30:56.223 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:30:56.223 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:30:56.224 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:30:56.224 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:30:56.224 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:30:56.224 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:30:56.224 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:30:56.224 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:30:56.224 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:30:56.224 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:30:56.224 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:30:56.224 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:30:56.224 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:30:56.225 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:30:56.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:30:56.226 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:30:56.278 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-07-04T17:30:56.278 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-07-04T17:30:56.283 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:30:56.352 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:30:56.395 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:30:56.395 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:30:56.395 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:30:56.395 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:30:56.395 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:30:56.396 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:30:56.396 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:30:56.396 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:30:56.396 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:30:56.396 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:30:56.396 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:30:56.396 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:30:56.396 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:30:56.396 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:30:56.396 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:30:56.396 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:30:56.396 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:30:56.396 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:30:56.396 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:30:56.397 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:30:56.397 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:30:56.397 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:30:56.397 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:30:56.397 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:30:56.397 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:30:56.397 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:30:56.397 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:30:56.397 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:30:56.397 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:30:56.397 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:30:56.397 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:30:56.398 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:30:56.398 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:30:56.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:30:56.399 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:30:56.536 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:2024-07-04T17:30:56.535+0000 7f2292561480 -1 init, newargv = 0x7f227c004640 newargc=15 2024-07-04T17:30:56.537 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[56478]: starting ceph client 2024-07-04T17:30:56.544 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[56478]: starting fuse 2024-07-04T17:30:57.587 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:30:57.686 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:30:57.745 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:30:57.745 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:30:57.745 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:30:57.745 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:30:57.745 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:30:57.745 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:30:57.746 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:30:57.746 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:30:57.746 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:30:57.746 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:30:57.746 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:30:57.746 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:30:57.746 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:30:57.746 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:30:57.746 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:30:57.746 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:30:57.746 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:30:57.746 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:30:57.746 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:30:57.747 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:30:57.747 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:30:57.747 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:30:57.747 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:30:57.747 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:30:57.747 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:30:57.747 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:30:57.747 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:30:57.747 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:30:57.747 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:30:57.747 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:30:57.747 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:30:57.748 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:30:57.748 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:30:57.748 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-07-04T17:30:57.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:30:57.749 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:30:57.802 INFO:teuthology.orchestra.run.smithi042.stdout:42 2024-07-04T17:30:57.803 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-07-04T17:30:57.803 DEBUG:teuthology.orchestra.run.smithi042:> sudo stdin-killer -- python3 -c ' 2024-07-04T17:30:57.803 DEBUG:teuthology.orchestra.run.smithi042:> import glob 2024-07-04T17:30:57.803 DEBUG:teuthology.orchestra.run.smithi042:> import re 2024-07-04T17:30:57.803 DEBUG:teuthology.orchestra.run.smithi042:> import os 2024-07-04T17:30:57.803 DEBUG:teuthology.orchestra.run.smithi042:> import subprocess 2024-07-04T17:30:57.803 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:30:57.803 DEBUG:teuthology.orchestra.run.smithi042:> def _find_admin_socket(client_name): 2024-07-04T17:30:57.804 DEBUG:teuthology.orchestra.run.smithi042:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-07-04T17:30:57.804 DEBUG:teuthology.orchestra.run.smithi042:> files = glob.glob(asok_path) 2024-07-04T17:30:57.804 DEBUG:teuthology.orchestra.run.smithi042:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-07-04T17:30:57.804 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:30:57.804 DEBUG:teuthology.orchestra.run.smithi042:> # Given a non-glob path, it better be there 2024-07-04T17:30:57.804 DEBUG:teuthology.orchestra.run.smithi042:> if "*" not in asok_path: 2024-07-04T17:30:57.804 DEBUG:teuthology.orchestra.run.smithi042:> assert(len(files) == 1) 2024-07-04T17:30:57.804 DEBUG:teuthology.orchestra.run.smithi042:> return files[0] 2024-07-04T17:30:57.804 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:30:57.804 DEBUG:teuthology.orchestra.run.smithi042:> for f in files: 2024-07-04T17:30:57.804 DEBUG:teuthology.orchestra.run.smithi042:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-07-04T17:30:57.804 DEBUG:teuthology.orchestra.run.smithi042:> if os.path.exists("/proc/{0}".format(pid)): 2024-07-04T17:30:57.804 DEBUG:teuthology.orchestra.run.smithi042:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-07-04T17:30:57.804 DEBUG:teuthology.orchestra.run.smithi042:> contents = proc_f.read() 2024-07-04T17:30:57.804 DEBUG:teuthology.orchestra.run.smithi042:> if mountpoint in contents: 2024-07-04T17:30:57.804 DEBUG:teuthology.orchestra.run.smithi042:> return f 2024-07-04T17:30:57.805 DEBUG:teuthology.orchestra.run.smithi042:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-07-04T17:30:57.805 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:30:57.805 DEBUG:teuthology.orchestra.run.smithi042:> print(_find_admin_socket("client.0")) 2024-07-04T17:30:57.805 DEBUG:teuthology.orchestra.run.smithi042:> ' 2024-07-04T17:30:57.983 INFO:teuthology.orchestra.run.smithi042.stdout:/var/run/ceph/ceph-client.0.56478.asok 2024-07-04T17:30:57.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:30:58.002 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.56478.asok 2024-07-04T17:30:58.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:30:58.002 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.56478.asok status 2024-07-04T17:30:58.315 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:30:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": { 2024-07-04T17:30:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_sha1": "9dba1e836b4756bf3a32be2a4936f371dff55008", 2024-07-04T17:30:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.0.0-4667-g9dba1e83 (9dba1e836b4756bf3a32be2a4936f371dff55008) squid (dev)", 2024-07-04T17:30:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: "entity_id": "0", 2024-07-04T17:30:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": "smithi042", 2024-07-04T17:30:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-07-04T17:30:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: "pid": "56478", 2024-07-04T17:30:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: "root": "/" 2024-07-04T17:30:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_count": 0, 2024-07-04T17:30:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_pinned_count": 0, 2024-07-04T17:30:58.316 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 5090, 2024-07-04T17:30:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: "inst": { 2024-07-04T17:30:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: "name": { 2024-07-04T17:30:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "client", 2024-07-04T17:30:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: "num": 5090 2024-07-04T17:30:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-07-04T17:30:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-07-04T17:30:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-07-04T17:30:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 438276672 2024-07-04T17:30:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:30:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-07-04T17:30:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-07-04T17:30:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-07-04T17:30:58.317 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 438276672 2024-07-04T17:30:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:30:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: "inst_str": "client.5090 192.168.0.1:0/438276672", 2024-07-04T17:30:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: "addr_str": "192.168.0.1:0/438276672", 2024-07-04T17:30:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: "inode_count": 1, 2024-07-04T17:30:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_epoch": 54, 2024-07-04T17:30:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch": 72, 2024-07-04T17:30:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch_barrier": 0, 2024-07-04T17:30:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklisted": false, 2024-07-04T17:30:58.318 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs" 2024-07-04T17:30:58.318 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:30:58.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:30:58.338 DEBUG:teuthology.orchestra.run.smithi042:> stat --file-system '--printf=%T 2024-07-04T17:30:58.338 DEBUG:teuthology.orchestra.run.smithi042:> ' -- /home/ubuntu/cephtest/mnt.0 2024-07-04T17:30:58.393 INFO:teuthology.orchestra.run.smithi042.stdout:fuseblk 2024-07-04T17:30:58.393 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-07-04T17:30:58.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:30:58.394 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-04T17:30:58.484 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-04T17:30:58.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:58.784+0000 7f4f87e00640 1 -- 172.21.15.42:0/3258612302 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f8806c2c0 msgr2=0x7f4f880b5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:58.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:58.784+0000 7f4f87e00640 1 --2- 172.21.15.42:0/3258612302 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f8806c2c0 0x7f4f880b5650 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f4f70005fd0 tx=0x7f4f7002b890 comp rx=0 tx=0).stop 2024-07-04T17:30:58.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:58.784+0000 7f4f87e00640 1 -- 172.21.15.42:0/3258612302 shutdown_connections 2024-07-04T17:30:58.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:58.784+0000 7f4f87e00640 1 --2- 172.21.15.42:0/3258612302 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f8806c2c0 0x7f4f880b5650 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:58.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:58.784+0000 7f4f87e00640 1 -- 172.21.15.42:0/3258612302 >> 172.21.15.42:0/3258612302 conn(0x7f4f880bebc0 msgr2=0x7f4f880befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:58.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:58.784+0000 7f4f87e00640 1 -- 172.21.15.42:0/3258612302 shutdown_connections 2024-07-04T17:30:58.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:58.784+0000 7f4f87e00640 1 -- 172.21.15.42:0/3258612302 wait complete. 2024-07-04T17:30:58.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:58.786+0000 7f4f87e00640 1 Processor -- start 2024-07-04T17:30:58.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:58.786+0000 7f4f87e00640 1 -- start start 2024-07-04T17:30:58.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:58.788+0000 7f4f87e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f8806c2c0 0x7f4f8813af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:58.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:58.788+0000 7f4f87e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4f880b2810 con 0x7f4f8806c2c0 2024-07-04T17:30:58.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:58.788+0000 7f4f86a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f8806c2c0 0x7f4f8813af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:58.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:58.788+0000 7f4f86a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f8806c2c0 0x7f4f8813af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40382/0 (socket says 172.21.15.42:40382) 2024-07-04T17:30:58.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:58.788+0000 7f4f86a00640 1 -- 172.21.15.42:0/106838122 learned_addr learned my addr 172.21.15.42:0/106838122 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:30:58.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:58.788+0000 7f4f86a00640 1 -- 172.21.15.42:0/106838122 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f8813e000 con 0x7f4f8806c2c0 2024-07-04T17:30:58.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:58.789+0000 7f4f86a00640 1 --2- 172.21.15.42:0/106838122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f8806c2c0 0x7f4f8813af80 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800c6b0 tx=0x7f4f7800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:58.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:58.789+0000 7f4f7ea00640 1 -- 172.21.15.42:0/106838122 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f78003aa0 con 0x7f4f8806c2c0 2024-07-04T17:30:58.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:58.789+0000 7f4f7ea00640 1 -- 172.21.15.42:0/106838122 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f78003c60 con 0x7f4f8806c2c0 2024-07-04T17:30:58.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:58.789+0000 7f4f7ea00640 1 -- 172.21.15.42:0/106838122 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f780050d0 con 0x7f4f8806c2c0 2024-07-04T17:30:58.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:58.789+0000 7f4f87e00640 1 -- 172.21.15.42:0/106838122 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f8813b550 con 0x7f4f8806c2c0 2024-07-04T17:30:58.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:58.789+0000 7f4f87e00640 1 -- 172.21.15.42:0/106838122 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f88139500 con 0x7f4f8806c2c0 2024-07-04T17:30:58.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:58.790+0000 7f4f87e00640 1 -- 172.21.15.42:0/106838122 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f880b5650 con 0x7f4f8806c2c0 2024-07-04T17:30:58.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:58.791+0000 7f4f7ea00640 1 -- 172.21.15.42:0/106838122 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f4f78005290 con 0x7f4f8806c2c0 2024-07-04T17:30:58.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:58.791+0000 7f4f7ea00640 1 --2- 172.21.15.42:0/106838122 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f4f6403d4f0 0x7f4f6403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:30:58.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:58.791+0000 7f4f7ea00640 1 -- 172.21.15.42:0/106838122 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 3439+0+0 (secure 0 0 0) 0x7f4f780516e0 con 0x7f4f8806c2c0 2024-07-04T17:30:58.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:58.791+0000 7f4f86000640 1 --2- 172.21.15.42:0/106838122 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f4f6403d4f0 0x7f4f6403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:30:58.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:58.792+0000 7f4f86000640 1 --2- 172.21.15.42:0/106838122 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f4f6403d4f0 0x7f4f6403fa00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4f70035000 tx=0x7f4f70004730 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:30:58.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:58.797+0000 7f4f7ea00640 1 -- 172.21.15.42:0/106838122 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f4f8806c6d0 con 0x7f4f8806c2c0 2024-07-04T17:30:59.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:59.050+0000 7f4f87e00640 1 -- 172.21.15.42:0/106838122 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4f8813e220 con 0x7f4f8806c2c0 2024-07-04T17:30:59.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:59.051+0000 7f4f7ea00640 1 -- 172.21.15.42:0/106838122 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v54) ==== 53+0+73 (secure 0 0 0) 0x7f4f7800d150 con 0x7f4f8806c2c0 2024-07-04T17:30:59.053 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-04T17:30:59.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:59.054+0000 7f4f87e00640 1 -- 172.21.15.42:0/106838122 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f4f6403d4f0 msgr2=0x7f4f6403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:59.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:59.054+0000 7f4f87e00640 1 --2- 172.21.15.42:0/106838122 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f4f6403d4f0 0x7f4f6403fa00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4f70035000 tx=0x7f4f70004730 comp rx=0 tx=0).stop 2024-07-04T17:30:59.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:59.054+0000 7f4f87e00640 1 -- 172.21.15.42:0/106838122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f8806c2c0 msgr2=0x7f4f8813af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:30:59.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:59.054+0000 7f4f87e00640 1 --2- 172.21.15.42:0/106838122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f8806c2c0 0x7f4f8813af80 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800c6b0 tx=0x7f4f7800cb70 comp rx=0 tx=0).stop 2024-07-04T17:30:59.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:59.054+0000 7f4f87e00640 1 -- 172.21.15.42:0/106838122 shutdown_connections 2024-07-04T17:30:59.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:59.054+0000 7f4f87e00640 1 --2- 172.21.15.42:0/106838122 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f4f6403d4f0 0x7f4f6403fa00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:59.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:59.054+0000 7f4f87e00640 1 --2- 172.21.15.42:0/106838122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f8806c2c0 0x7f4f8813af80 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:30:59.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:59.054+0000 7f4f87e00640 1 -- 172.21.15.42:0/106838122 >> 172.21.15.42:0/106838122 conn(0x7f4f880bebc0 msgr2=0x7f4f880b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:30:59.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:59.055+0000 7f4f87e00640 1 -- 172.21.15.42:0/106838122 shutdown_connections 2024-07-04T17:30:59.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:59.055+0000 7f4f87e00640 1 -- 172.21.15.42:0/106838122 wait complete. 2024-07-04T17:30:59.079 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-07-04T17:30:59.080 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-04T17:30:59.080 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2024-07-04T17:30:59.080 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-07-04T17:30:59.080 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-07-04T17:30:59.080 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-07-04T17:30:59.080 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-07-04T17:30:59.081 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-04T17:30:59.081 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-07-04T17:30:59.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:30:59.081 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2024-07-04T17:30:59.133 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-04T17:30:59.133 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-04T17:30:59.133 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-07-04T17:30:59.133 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:30:59.133 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-07-04T17:30:59.133 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:30:59.133 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:30:59.134 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2024-07-04T17:30:59.134 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f0 2024-07-04T17:30:59.134 INFO:teuthology.orchestra.run.smithi042.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:30:59.134 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2024-07-04T17:30:59.134 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f1 2024-07-04T17:30:59.134 INFO:teuthology.orchestra.run.smithi042.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-04T17:30:59.134 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:30:59.134 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f0 2024-07-04T17:30:59.134 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-07-04T17:30:59.134 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 42311sec preferred_lft 42311sec 2024-07-04T17:30:59.134 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::ec4:7aff:febd:1440/64 scope link noprefixroute 2024-07-04T17:30:59.134 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:30:59.135 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:30:59.135 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:30:59.135 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f1 2024-07-04T17:30:59.135 INFO:teuthology.orchestra.run.smithi042.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-04T17:30:59.135 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 32:75:d6:f1:31:e1 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:30:59.135 INFO:teuthology.orchestra.run.smithi042.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-04T17:30:59.135 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:30:59.135 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::8839:a9ff:fe74:17f5/64 scope link 2024-07-04T17:30:59.135 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:30:59.135 INFO:teuthology.orchestra.run.smithi042.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-04T17:30:59.135 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether e6:ab:57:fb:ca:9e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:30:59.135 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::e4ab:57ff:fefb:ca9e/64 scope link 2024-07-04T17:30:59.136 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:30:59.136 INFO:teuthology.orchestra.run.smithi042.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-04T17:30:59.136 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 32:75:d6:f1:31:e1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:30:59.136 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::3075:d6ff:fef1:31e1/64 scope link 2024-07-04T17:30:59.136 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:30:59.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:30:59.137 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-04T17:30:59.217 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-07-04T17:30:59.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:30:59.221 DEBUG:teuthology.orchestra.run.smithi042:> route 2024-07-04T17:30:59.275 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2024-07-04T17:30:59.275 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-04T17:30:59.275 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-04T17:30:59.275 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-04T17:30:59.275 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-04T17:30:59.276 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-04T17:30:59.276 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-07-04T17:30:59.276 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-04T17:30:59.276 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-04T17:30:59.276 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-04T17:30:59.276 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-07-04T17:30:59.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-04T17:30:59.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:30:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:30:59.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:30:59.494 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-07-04T17:30:59.547 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-04T17:30:59.547 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2024-07-04T17:30:59.548 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-07-04T17:30:59.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:30:59.548 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.1 up 2024-07-04T17:30:59.637 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-07-04T17:30:59.638 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-04T17:30:59.638 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-07-04T17:30:59.693 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-07-04T17:30:59.694 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-04T17:30:59.694 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-07-04T17:30:59.747 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:30:59.827 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:30:59.882 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:30:59.882 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:30:59.882 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:30:59.882 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:30:59.882 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:30:59.883 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:30:59.883 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:30:59.883 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:30:59.883 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:30:59.883 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:30:59.883 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:30:59.883 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:30:59.883 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:30:59.883 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:30:59.883 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:30:59.883 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:30:59.884 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:30:59.884 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:30:59.884 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:30:59.884 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:30:59.884 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:30:59.884 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:30:59.884 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:30:59.884 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:30:59.884 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:30:59.884 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:30:59.884 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:30:59.884 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:30:59.884 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:30:59.885 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:30:59.885 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:30:59.885 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:30:59.885 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:30:59.885 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-07-04T17:30:59.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:30:59.886 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:30:59.941 INFO:teuthology.orchestra.run.smithi042.stdout:42 2024-07-04T17:30:59.942 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-07-04T17:30:59.942 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-07-04T17:30:59.947 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:31:00.019 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:31:00.065 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:31:00.065 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:31:00.065 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:31:00.065 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:31:00.065 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:31:00.066 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:31:00.066 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:31:00.066 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:31:00.066 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:31:00.066 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:31:00.066 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:31:00.066 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:31:00.066 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:31:00.066 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:31:00.066 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:31:00.066 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:31:00.066 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:31:00.066 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:31:00.067 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:31:00.067 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:31:00.067 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:31:00.067 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:31:00.067 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:31:00.067 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:31:00.067 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:31:00.067 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:31:00.067 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:31:00.067 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:31:00.067 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:31:00.067 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:31:00.068 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:31:00.068 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:31:00.068 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:31:00.068 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-07-04T17:31:00.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:31:00.069 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:31:00.155 INFO:teuthology.orchestra.run.smithi042.stdout:42 2024-07-04T17:31:00.202 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi042.stderr:2024-07-04T17:31:00.201+0000 7f9291e78480 -1 init, newargv = 0x7f9278001080 newargc=15 2024-07-04T17:31:00.202 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi042.stderr:ceph-fuse[56947]: starting ceph client 2024-07-04T17:31:00.210 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi042.stderr:ceph-fuse[56947]: starting fuse 2024-07-04T17:31:00.564 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-04T17:31:01.257 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:31:01.353 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:31:01.411 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:31:01.411 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:31:01.411 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:31:01.412 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:31:01.412 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:31:01.412 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:31:01.412 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:31:01.412 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:31:01.412 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:31:01.412 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:31:01.412 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:31:01.412 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:31:01.412 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:31:01.412 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:31:01.412 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:31:01.412 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:31:01.412 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:31:01.413 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:31:01.413 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:31:01.413 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:31:01.413 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:31:01.413 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:31:01.413 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:31:01.413 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:31:01.413 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:31:01.413 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:31:01.413 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:31:01.413 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:31:01.413 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:31:01.413 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:31:01.413 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:31:01.414 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:31:01.414 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:31:01.414 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-07-04T17:31:01.414 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.1 2024-07-04T17:31:01.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:31:01.415 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:31:01.478 INFO:teuthology.orchestra.run.smithi042.stdout:42 2024-07-04T17:31:01.478 INFO:teuthology.orchestra.run.smithi042.stdout:43 2024-07-04T17:31:01.479 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 43] 2024-07-04T17:31:01.479 DEBUG:teuthology.orchestra.run.smithi042:> sudo stdin-killer -- python3 -c ' 2024-07-04T17:31:01.479 DEBUG:teuthology.orchestra.run.smithi042:> import glob 2024-07-04T17:31:01.479 DEBUG:teuthology.orchestra.run.smithi042:> import re 2024-07-04T17:31:01.479 DEBUG:teuthology.orchestra.run.smithi042:> import os 2024-07-04T17:31:01.479 DEBUG:teuthology.orchestra.run.smithi042:> import subprocess 2024-07-04T17:31:01.479 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:31:01.479 DEBUG:teuthology.orchestra.run.smithi042:> def _find_admin_socket(client_name): 2024-07-04T17:31:01.479 DEBUG:teuthology.orchestra.run.smithi042:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-07-04T17:31:01.479 DEBUG:teuthology.orchestra.run.smithi042:> files = glob.glob(asok_path) 2024-07-04T17:31:01.479 DEBUG:teuthology.orchestra.run.smithi042:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-07-04T17:31:01.479 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:31:01.479 DEBUG:teuthology.orchestra.run.smithi042:> # Given a non-glob path, it better be there 2024-07-04T17:31:01.480 DEBUG:teuthology.orchestra.run.smithi042:> if "*" not in asok_path: 2024-07-04T17:31:01.480 DEBUG:teuthology.orchestra.run.smithi042:> assert(len(files) == 1) 2024-07-04T17:31:01.480 DEBUG:teuthology.orchestra.run.smithi042:> return files[0] 2024-07-04T17:31:01.480 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:31:01.480 DEBUG:teuthology.orchestra.run.smithi042:> for f in files: 2024-07-04T17:31:01.480 DEBUG:teuthology.orchestra.run.smithi042:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-07-04T17:31:01.480 DEBUG:teuthology.orchestra.run.smithi042:> if os.path.exists("/proc/{0}".format(pid)): 2024-07-04T17:31:01.480 DEBUG:teuthology.orchestra.run.smithi042:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-07-04T17:31:01.480 DEBUG:teuthology.orchestra.run.smithi042:> contents = proc_f.read() 2024-07-04T17:31:01.480 DEBUG:teuthology.orchestra.run.smithi042:> if mountpoint in contents: 2024-07-04T17:31:01.480 DEBUG:teuthology.orchestra.run.smithi042:> return f 2024-07-04T17:31:01.480 DEBUG:teuthology.orchestra.run.smithi042:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-07-04T17:31:01.480 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:31:01.480 DEBUG:teuthology.orchestra.run.smithi042:> print(_find_admin_socket("client.1")) 2024-07-04T17:31:01.481 DEBUG:teuthology.orchestra.run.smithi042:> ' 2024-07-04T17:31:01.625 INFO:teuthology.orchestra.run.smithi042.stdout:/var/run/ceph/ceph-client.1.56947.asok 2024-07-04T17:31:01.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:31:01.642 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.56947.asok 2024-07-04T17:31:01.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:31:01.642 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.56947.asok status 2024-07-04T17:31:01.945 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:31:01.946 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": { 2024-07-04T17:31:01.946 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_sha1": "9dba1e836b4756bf3a32be2a4936f371dff55008", 2024-07-04T17:31:01.946 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.0.0-4667-g9dba1e83 (9dba1e836b4756bf3a32be2a4936f371dff55008) squid (dev)", 2024-07-04T17:31:01.946 INFO:teuthology.orchestra.run.smithi042.stdout: "entity_id": "1", 2024-07-04T17:31:01.946 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": "smithi042", 2024-07-04T17:31:01.946 INFO:teuthology.orchestra.run.smithi042.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-07-04T17:31:01.946 INFO:teuthology.orchestra.run.smithi042.stdout: "pid": "56947", 2024-07-04T17:31:01.946 INFO:teuthology.orchestra.run.smithi042.stdout: "root": "/" 2024-07-04T17:31:01.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:31:01.946 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_count": 0, 2024-07-04T17:31:01.946 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_pinned_count": 0, 2024-07-04T17:31:01.947 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 5096, 2024-07-04T17:31:01.947 INFO:teuthology.orchestra.run.smithi042.stdout: "inst": { 2024-07-04T17:31:01.947 INFO:teuthology.orchestra.run.smithi042.stdout: "name": { 2024-07-04T17:31:01.947 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "client", 2024-07-04T17:31:01.947 INFO:teuthology.orchestra.run.smithi042.stdout: "num": 5096 2024-07-04T17:31:01.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:31:01.947 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-07-04T17:31:01.947 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-07-04T17:31:01.947 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.2:0", 2024-07-04T17:31:01.947 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4003707710 2024-07-04T17:31:01.947 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:31:01.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:31:01.947 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-07-04T17:31:01.947 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-07-04T17:31:01.948 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.2:0", 2024-07-04T17:31:01.948 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4003707710 2024-07-04T17:31:01.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:31:01.948 INFO:teuthology.orchestra.run.smithi042.stdout: "inst_str": "client.5096 192.168.0.2:0/4003707710", 2024-07-04T17:31:01.948 INFO:teuthology.orchestra.run.smithi042.stdout: "addr_str": "192.168.0.2:0/4003707710", 2024-07-04T17:31:01.948 INFO:teuthology.orchestra.run.smithi042.stdout: "inode_count": 1, 2024-07-04T17:31:01.948 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_epoch": 54, 2024-07-04T17:31:01.948 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch": 72, 2024-07-04T17:31:01.948 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch_barrier": 0, 2024-07-04T17:31:01.948 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklisted": false, 2024-07-04T17:31:01.948 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs" 2024-07-04T17:31:01.948 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:31:01.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:31:01.968 DEBUG:teuthology.orchestra.run.smithi042:> stat --file-system '--printf=%T 2024-07-04T17:31:01.968 DEBUG:teuthology.orchestra.run.smithi042:> ' -- /home/ubuntu/cephtest/mnt.1 2024-07-04T17:31:02.020 INFO:teuthology.orchestra.run.smithi042.stdout:fuseblk 2024-07-04T17:31:02.020 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-07-04T17:31:02.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:31:02.020 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-07-04T17:31:02.106 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-07-04T17:31:02.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.408+0000 7f62fbe00640 1 -- 172.21.15.42:0/487467247 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62f4073470 msgr2=0x7f62f40a9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:02.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.408+0000 7f62fbe00640 1 --2- 172.21.15.42:0/487467247 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62f4073470 0x7f62f40a9990 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f62e00097b0 tx=0x7f62e002a540 comp rx=0 tx=0).stop 2024-07-04T17:31:02.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.408+0000 7f62fbe00640 1 -- 172.21.15.42:0/487467247 shutdown_connections 2024-07-04T17:31:02.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.408+0000 7f62fbe00640 1 --2- 172.21.15.42:0/487467247 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62f4073470 0x7f62f40a9990 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:02.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.408+0000 7f62fbe00640 1 -- 172.21.15.42:0/487467247 >> 172.21.15.42:0/487467247 conn(0x7f62f406c020 msgr2=0x7f62f406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:02.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.408+0000 7f62fbe00640 1 -- 172.21.15.42:0/487467247 shutdown_connections 2024-07-04T17:31:02.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.408+0000 7f62fbe00640 1 -- 172.21.15.42:0/487467247 wait complete. 2024-07-04T17:31:02.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.410+0000 7f62fbe00640 1 Processor -- start 2024-07-04T17:31:02.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.411+0000 7f62fbe00640 1 -- start start 2024-07-04T17:31:02.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.412+0000 7f62fbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62f4073470 0x7f62f41d7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:02.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.412+0000 7f62fbe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f62f413f430 con 0x7f62f4073470 2024-07-04T17:31:02.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.412+0000 7f62f9400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62f4073470 0x7f62f41d7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:02.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.412+0000 7f62f9400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62f4073470 0x7f62f41d7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40398/0 (socket says 172.21.15.42:40398) 2024-07-04T17:31:02.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.412+0000 7f62f9400640 1 -- 172.21.15.42:0/3640691128 learned_addr learned my addr 172.21.15.42:0/3640691128 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:02.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.413+0000 7f62f9400640 1 -- 172.21.15.42:0/3640691128 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f413b4f0 con 0x7f62f4073470 2024-07-04T17:31:02.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.413+0000 7f62f9400640 1 --2- 172.21.15.42:0/3640691128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62f4073470 0x7f62f41d7180 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f62e800c3b0 tx=0x7f62e800c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:02.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.414+0000 7f62f1600640 1 -- 172.21.15.42:0/3640691128 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62e8011780 con 0x7f62f4073470 2024-07-04T17:31:02.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.414+0000 7f62f1600640 1 -- 172.21.15.42:0/3640691128 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f62e8017070 con 0x7f62f4073470 2024-07-04T17:31:02.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.414+0000 7f62f1600640 1 -- 172.21.15.42:0/3640691128 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62e8011c00 con 0x7f62f4073470 2024-07-04T17:31:02.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.414+0000 7f62fbe00640 1 -- 172.21.15.42:0/3640691128 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62f41e1360 con 0x7f62f4073470 2024-07-04T17:31:02.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.414+0000 7f62fbe00640 1 -- 172.21.15.42:0/3640691128 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62f41d76c0 con 0x7f62f4073470 2024-07-04T17:31:02.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.415+0000 7f62f1600640 1 -- 172.21.15.42:0/3640691128 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f62e801c460 con 0x7f62f4073470 2024-07-04T17:31:02.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.415+0000 7f62fbe00640 1 -- 172.21.15.42:0/3640691128 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62f413fb00 con 0x7f62f4073470 2024-07-04T17:31:02.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.416+0000 7f62f1600640 1 --2- 172.21.15.42:0/3640691128 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f62d403d4a0 0x7f62d403f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:02.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.416+0000 7f62f1600640 1 -- 172.21.15.42:0/3640691128 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 3439+0+0 (secure 0 0 0) 0x7f62e8051680 con 0x7f62f4073470 2024-07-04T17:31:02.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.416+0000 7f62f8a00640 1 --2- 172.21.15.42:0/3640691128 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f62d403d4a0 0x7f62d403f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:02.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.417+0000 7f62f8a00640 1 --2- 172.21.15.42:0/3640691128 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f62d403d4a0 0x7f62d403f9b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f62e0034000 tx=0x7f62e0004730 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:02.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.422+0000 7f62f1600640 1 -- 172.21.15.42:0/3640691128 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f62e8011e20 con 0x7f62f4073470 2024-07-04T17:31:02.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.675+0000 7f62fbe00640 1 -- 172.21.15.42:0/3640691128 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f62f4069b70 con 0x7f62f4073470 2024-07-04T17:31:02.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.691+0000 7f62f1600640 1 -- 172.21.15.42:0/3640691128 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v55) ==== 136+0+0 (secure 0 0 0) 0x7f62e80513a0 con 0x7f62f4073470 2024-07-04T17:31:02.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.694+0000 7f62fbe00640 1 -- 172.21.15.42:0/3640691128 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f62d403d4a0 msgr2=0x7f62d403f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:02.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.694+0000 7f62fbe00640 1 --2- 172.21.15.42:0/3640691128 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f62d403d4a0 0x7f62d403f9b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f62e0034000 tx=0x7f62e0004730 comp rx=0 tx=0).stop 2024-07-04T17:31:02.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.694+0000 7f62fbe00640 1 -- 172.21.15.42:0/3640691128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62f4073470 msgr2=0x7f62f41d7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:02.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.694+0000 7f62fbe00640 1 --2- 172.21.15.42:0/3640691128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62f4073470 0x7f62f41d7180 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f62e800c3b0 tx=0x7f62e800c870 comp rx=0 tx=0).stop 2024-07-04T17:31:02.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.694+0000 7f62fbe00640 1 -- 172.21.15.42:0/3640691128 shutdown_connections 2024-07-04T17:31:02.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.694+0000 7f62fbe00640 1 --2- 172.21.15.42:0/3640691128 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f62d403d4a0 0x7f62d403f9b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:02.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.694+0000 7f62fbe00640 1 --2- 172.21.15.42:0/3640691128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62f4073470 0x7f62f41d7180 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:02.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.694+0000 7f62fbe00640 1 -- 172.21.15.42:0/3640691128 >> 172.21.15.42:0/3640691128 conn(0x7f62f406c020 msgr2=0x7f62f40b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:02.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.695+0000 7f62fbe00640 1 -- 172.21.15.42:0/3640691128 shutdown_connections 2024-07-04T17:31:02.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:02.695+0000 7f62fbe00640 1 -- 172.21.15.42:0/3640691128 wait complete. 2024-07-04T17:31:02.726 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-04T17:31:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.010+0000 7f8f64a00640 1 -- 172.21.15.42:0/11169356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f600bd100 msgr2=0x7f8f6006ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.010+0000 7f8f64a00640 1 --2- 172.21.15.42:0/11169356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f600bd100 0x7f8f6006ceb0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f8f480097b0 tx=0x7f8f4802a540 comp rx=0 tx=0).stop 2024-07-04T17:31:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.010+0000 7f8f64a00640 1 -- 172.21.15.42:0/11169356 shutdown_connections 2024-07-04T17:31:03.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.010+0000 7f8f64a00640 1 --2- 172.21.15.42:0/11169356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f600bd100 0x7f8f6006ceb0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:03.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.010+0000 7f8f64a00640 1 -- 172.21.15.42:0/11169356 >> 172.21.15.42:0/11169356 conn(0x7f8f600c03e0 msgr2=0x7f8f600c07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:03.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.010+0000 7f8f64a00640 1 -- 172.21.15.42:0/11169356 shutdown_connections 2024-07-04T17:31:03.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.010+0000 7f8f64a00640 1 -- 172.21.15.42:0/11169356 wait complete. 2024-07-04T17:31:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.012+0000 7f8f64a00640 1 Processor -- start 2024-07-04T17:31:03.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.012+0000 7f8f64a00640 1 -- start start 2024-07-04T17:31:03.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.014+0000 7f8f64a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f600bd100 0x7f8f60144cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:03.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.014+0000 7f8f64a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6014df30 con 0x7f8f600bd100 2024-07-04T17:31:03.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.014+0000 7f8f5de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f600bd100 0x7f8f60144cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:03.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.014+0000 7f8f5de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f600bd100 0x7f8f60144cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40424/0 (socket says 172.21.15.42:40424) 2024-07-04T17:31:03.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.014+0000 7f8f5de00640 1 -- 172.21.15.42:0/34836147 learned_addr learned my addr 172.21.15.42:0/34836147 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:03.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.014+0000 7f8f5de00640 1 -- 172.21.15.42:0/34836147 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f60150de0 con 0x7f8f600bd100 2024-07-04T17:31:03.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.015+0000 7f8f5de00640 1 --2- 172.21.15.42:0/34836147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f600bd100 0x7f8f60144cb0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f8f5000c380 tx=0x7f8f5000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:03.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.015+0000 7f8f56000640 1 -- 172.21.15.42:0/34836147 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f500116c0 con 0x7f8f600bd100 2024-07-04T17:31:03.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.015+0000 7f8f56000640 1 -- 172.21.15.42:0/34836147 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f50017070 con 0x7f8f600bd100 2024-07-04T17:31:03.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.015+0000 7f8f56000640 1 -- 172.21.15.42:0/34836147 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f50011b20 con 0x7f8f600bd100 2024-07-04T17:31:03.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.015+0000 7f8f64a00640 1 -- 172.21.15.42:0/34836147 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f60146550 con 0x7f8f600bd100 2024-07-04T17:31:03.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.015+0000 7f8f64a00640 1 -- 172.21.15.42:0/34836147 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f601512e0 con 0x7f8f600bd100 2024-07-04T17:31:03.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.017+0000 7f8f64a00640 1 -- 172.21.15.42:0/34836147 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f6006ceb0 con 0x7f8f600bd100 2024-07-04T17:31:03.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.017+0000 7f8f56000640 1 -- 172.21.15.42:0/34836147 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f8f5001c460 con 0x7f8f600bd100 2024-07-04T17:31:03.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.017+0000 7f8f56000640 1 --2- 172.21.15.42:0/34836147 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f8f3c03d4a0 0x7f8f3c03f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:03.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.017+0000 7f8f56000640 1 -- 172.21.15.42:0/34836147 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 3439+0+0 (secure 0 0 0) 0x7f8f50051710 con 0x7f8f600bd100 2024-07-04T17:31:03.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.018+0000 7f8f57e00640 1 --2- 172.21.15.42:0/34836147 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f8f3c03d4a0 0x7f8f3c03f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:03.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.018+0000 7f8f57e00640 1 --2- 172.21.15.42:0/34836147 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f8f3c03d4a0 0x7f8f3c03f9b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8f48034000 tx=0x7f8f48004730 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:03.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.023+0000 7f8f56000640 1 -- 172.21.15.42:0/34836147 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f8f5000a140 con 0x7f8f600bd100 2024-07-04T17:31:03.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.240+0000 7f8f64a00640 1 -- 172.21.15.42:0/34836147 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f8f6014c300 con 0x7f8f600bd100 2024-07-04T17:31:03.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.240+0000 7f8f56000640 1 -- 172.21.15.42:0/34836147 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) ==== 84+0+209 (secure 0 0 0) 0x7f8f50011cc0 con 0x7f8f600bd100 2024-07-04T17:31:03.242 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:31:03.242 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-07-04T17:31:03.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:31:03.242 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 1, 2024-07-04T17:31:03.242 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": ".mgr" 2024-07-04T17:31:03.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:31:03.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:31:03.242 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 12, 2024-07-04T17:31:03.242 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": "cephfs_metadata" 2024-07-04T17:31:03.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:31:03.242 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:31:03.243 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 13, 2024-07-04T17:31:03.243 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": "cephfs_data" 2024-07-04T17:31:03.243 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:31:03.243 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-07-04T17:31:03.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.243+0000 7f8f64a00640 1 -- 172.21.15.42:0/34836147 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f8f3c03d4a0 msgr2=0x7f8f3c03f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:03.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.243+0000 7f8f64a00640 1 --2- 172.21.15.42:0/34836147 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f8f3c03d4a0 0x7f8f3c03f9b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8f48034000 tx=0x7f8f48004730 comp rx=0 tx=0).stop 2024-07-04T17:31:03.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.243+0000 7f8f64a00640 1 -- 172.21.15.42:0/34836147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f600bd100 msgr2=0x7f8f60144cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:03.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.243+0000 7f8f64a00640 1 --2- 172.21.15.42:0/34836147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f600bd100 0x7f8f60144cb0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f8f5000c380 tx=0x7f8f5000c840 comp rx=0 tx=0).stop 2024-07-04T17:31:03.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.243+0000 7f8f64a00640 1 -- 172.21.15.42:0/34836147 shutdown_connections 2024-07-04T17:31:03.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.243+0000 7f8f64a00640 1 --2- 172.21.15.42:0/34836147 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f8f3c03d4a0 0x7f8f3c03f9b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:03.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.243+0000 7f8f64a00640 1 --2- 172.21.15.42:0/34836147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f600bd100 0x7f8f60144cb0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:03.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.243+0000 7f8f64a00640 1 -- 172.21.15.42:0/34836147 >> 172.21.15.42:0/34836147 conn(0x7f8f600c03e0 msgr2=0x7f8f60075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:03.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.243+0000 7f8f64a00640 1 -- 172.21.15.42:0/34836147 shutdown_connections 2024-07-04T17:31:03.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.243+0000 7f8f64a00640 1 -- 172.21.15.42:0/34836147 wait complete. 2024-07-04T17:31:03.270 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'backup_fs' 2024-07-04T17:31:03.270 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_metadata --pg_num_min 64 2024-07-04T17:31:03.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.563+0000 7f5779600640 1 -- 172.21.15.42:0/2591202823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f577406d130 msgr2=0x7f5774071d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:03.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.563+0000 7f5779600640 1 --2- 172.21.15.42:0/2591202823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f577406d130 0x7f5774071d90 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f575c0097b0 tx=0x7f575c02a540 comp rx=0 tx=0).stop 2024-07-04T17:31:03.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.563+0000 7f5779600640 1 -- 172.21.15.42:0/2591202823 shutdown_connections 2024-07-04T17:31:03.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.563+0000 7f5779600640 1 --2- 172.21.15.42:0/2591202823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f577406d130 0x7f5774071d90 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:03.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.563+0000 7f5779600640 1 -- 172.21.15.42:0/2591202823 >> 172.21.15.42:0/2591202823 conn(0x7f577406c020 msgr2=0x7f577406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:03.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.563+0000 7f5779600640 1 -- 172.21.15.42:0/2591202823 shutdown_connections 2024-07-04T17:31:03.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.563+0000 7f5779600640 1 -- 172.21.15.42:0/2591202823 wait complete. 2024-07-04T17:31:03.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.565+0000 7f5779600640 1 Processor -- start 2024-07-04T17:31:03.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.565+0000 7f5779600640 1 -- start start 2024-07-04T17:31:03.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.566+0000 7f5779600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f577406d130 0x7f5774072ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:03.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.566+0000 7f5779600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5774141900 con 0x7f577406d130 2024-07-04T17:31:03.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.567+0000 7f5772a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f577406d130 0x7f5774072ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:03.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.567+0000 7f5772a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f577406d130 0x7f5774072ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40448/0 (socket says 172.21.15.42:40448) 2024-07-04T17:31:03.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.567+0000 7f5772a00640 1 -- 172.21.15.42:0/3605149796 learned_addr learned my addr 172.21.15.42:0/3605149796 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:03.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.567+0000 7f5772a00640 1 -- 172.21.15.42:0/3605149796 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57740738c0 con 0x7f577406d130 2024-07-04T17:31:03.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.568+0000 7f5772a00640 1 --2- 172.21.15.42:0/3605149796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f577406d130 0x7f5774072ed0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f5764001eb0 tx=0x7f576400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:03.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.568+0000 7f576aa00640 1 -- 172.21.15.42:0/3605149796 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57640116c0 con 0x7f577406d130 2024-07-04T17:31:03.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.568+0000 7f576aa00640 1 -- 172.21.15.42:0/3605149796 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5764017070 con 0x7f577406d130 2024-07-04T17:31:03.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.568+0000 7f576aa00640 1 -- 172.21.15.42:0/3605149796 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5764011ae0 con 0x7f577406d130 2024-07-04T17:31:03.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.568+0000 7f5779600640 1 -- 172.21.15.42:0/3605149796 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5774072bf0 con 0x7f577406d130 2024-07-04T17:31:03.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.568+0000 7f5779600640 1 -- 172.21.15.42:0/3605149796 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5774073410 con 0x7f577406d130 2024-07-04T17:31:03.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.569+0000 7f5779600640 1 -- 172.21.15.42:0/3605149796 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5774071d90 con 0x7f577406d130 2024-07-04T17:31:03.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.569+0000 7f576aa00640 1 -- 172.21.15.42:0/3605149796 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f5764011c80 con 0x7f577406d130 2024-07-04T17:31:03.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.570+0000 7f576aa00640 1 --2- 172.21.15.42:0/3605149796 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f575003d4f0 0x7f575003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:03.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.570+0000 7f576aa00640 1 -- 172.21.15.42:0/3605149796 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 3439+0+0 (secure 0 0 0) 0x7f5764051670 con 0x7f577406d130 2024-07-04T17:31:03.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.570+0000 7f576be00640 1 --2- 172.21.15.42:0/3605149796 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f575003d4f0 0x7f575003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:03.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.571+0000 7f576be00640 1 --2- 172.21.15.42:0/3605149796 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f575003d4f0 0x7f575003fa00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f575c034000 tx=0x7f575c004730 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:03.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.576+0000 7f576aa00640 1 -- 172.21.15.42:0/3605149796 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f57740afc70 con 0x7f577406d130 2024-07-04T17:31:03.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:03.790+0000 7f5779600640 1 -- 172.21.15.42:0/3605149796 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64} v 0) -- 0x7f57740759a0 con 0x7f577406d130 2024-07-04T17:31:04.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:04.718+0000 7f576aa00640 1 -- 172.21.15.42:0/3605149796 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64}]=0 pool 'backup_fs_metadata' created v73) ==== 144+0+0 (secure 0 0 0) 0x7f576400a140 con 0x7f577406d130 2024-07-04T17:31:04.719 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'backup_fs_metadata' created 2024-07-04T17:31:04.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:04.721+0000 7f5779600640 1 -- 172.21.15.42:0/3605149796 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f575003d4f0 msgr2=0x7f575003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:04.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:04.721+0000 7f5779600640 1 --2- 172.21.15.42:0/3605149796 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f575003d4f0 0x7f575003fa00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f575c034000 tx=0x7f575c004730 comp rx=0 tx=0).stop 2024-07-04T17:31:04.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:04.722+0000 7f5779600640 1 -- 172.21.15.42:0/3605149796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f577406d130 msgr2=0x7f5774072ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:04.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:04.722+0000 7f5779600640 1 --2- 172.21.15.42:0/3605149796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f577406d130 0x7f5774072ed0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f5764001eb0 tx=0x7f576400c810 comp rx=0 tx=0).stop 2024-07-04T17:31:04.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:04.722+0000 7f5779600640 1 -- 172.21.15.42:0/3605149796 shutdown_connections 2024-07-04T17:31:04.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:04.722+0000 7f5779600640 1 --2- 172.21.15.42:0/3605149796 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f575003d4f0 0x7f575003fa00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:04.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:04.722+0000 7f5779600640 1 --2- 172.21.15.42:0/3605149796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f577406d130 0x7f5774072ed0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:04.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:04.722+0000 7f5779600640 1 -- 172.21.15.42:0/3605149796 >> 172.21.15.42:0/3605149796 conn(0x7f577406c020 msgr2=0x7f57741d8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:04.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:04.722+0000 7f5779600640 1 -- 172.21.15.42:0/3605149796 shutdown_connections 2024-07-04T17:31:04.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:04.722+0000 7f5779600640 1 -- 172.21.15.42:0/3605149796 wait complete. 2024-07-04T17:31:04.763 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-04T17:31:05.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.037+0000 7f9da5600640 1 -- 172.21.15.42:0/3710407980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9da00a9be0 msgr2=0x7f9da013efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:05.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.037+0000 7f9da5600640 1 --2- 172.21.15.42:0/3710407980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9da00a9be0 0x7f9da013efb0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0097b0 tx=0x7f9d8c02a530 comp rx=0 tx=0).stop 2024-07-04T17:31:05.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.038+0000 7f9da5600640 1 -- 172.21.15.42:0/3710407980 shutdown_connections 2024-07-04T17:31:05.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.038+0000 7f9da5600640 1 --2- 172.21.15.42:0/3710407980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9da00a9be0 0x7f9da013efb0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:05.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.038+0000 7f9da5600640 1 -- 172.21.15.42:0/3710407980 >> 172.21.15.42:0/3710407980 conn(0x7f9da006c020 msgr2=0x7f9da006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:05.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.038+0000 7f9da5600640 1 -- 172.21.15.42:0/3710407980 shutdown_connections 2024-07-04T17:31:05.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.038+0000 7f9da5600640 1 -- 172.21.15.42:0/3710407980 wait complete. 2024-07-04T17:31:05.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.040+0000 7f9da5600640 1 Processor -- start 2024-07-04T17:31:05.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.040+0000 7f9da5600640 1 -- start start 2024-07-04T17:31:05.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.041+0000 7f9da5600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9da00a9be0 0x7f9da01d91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:05.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.041+0000 7f9da5600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9da01402d0 con 0x7f9da00a9be0 2024-07-04T17:31:05.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.042+0000 7f9d9fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9da00a9be0 0x7f9da01d91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:05.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.042+0000 7f9d9fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9da00a9be0 0x7f9da01d91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47060/0 (socket says 172.21.15.42:47060) 2024-07-04T17:31:05.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.042+0000 7f9d9fe00640 1 -- 172.21.15.42:0/2438808142 learned_addr learned my addr 172.21.15.42:0/2438808142 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:05.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.042+0000 7f9d9fe00640 1 -- 172.21.15.42:0/2438808142 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da00bff20 con 0x7f9da00a9be0 2024-07-04T17:31:05.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.043+0000 7f9d9fe00640 1 --2- 172.21.15.42:0/2438808142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9da00a9be0 0x7f9da01d91d0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000c3b0 tx=0x7f9d9000c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:05.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.043+0000 7f9d97e00640 1 -- 172.21.15.42:0/2438808142 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d90011780 con 0x7f9da00a9be0 2024-07-04T17:31:05.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.043+0000 7f9d97e00640 1 -- 172.21.15.42:0/2438808142 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d90017070 con 0x7f9da00a9be0 2024-07-04T17:31:05.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.043+0000 7f9d97e00640 1 -- 172.21.15.42:0/2438808142 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d90011c00 con 0x7f9da00a9be0 2024-07-04T17:31:05.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.043+0000 7f9da5600640 1 -- 172.21.15.42:0/2438808142 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da01e32e0 con 0x7f9da00a9be0 2024-07-04T17:31:05.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.043+0000 7f9da5600640 1 -- 172.21.15.42:0/2438808142 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da01d70a0 con 0x7f9da00a9be0 2024-07-04T17:31:05.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.044+0000 7f9da5600640 1 -- 172.21.15.42:0/2438808142 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9da0072e50 con 0x7f9da00a9be0 2024-07-04T17:31:05.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.045+0000 7f9d97e00640 1 -- 172.21.15.42:0/2438808142 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f9d9001c460 con 0x7f9da00a9be0 2024-07-04T17:31:05.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.045+0000 7f9d97e00640 1 --2- 172.21.15.42:0/2438808142 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f9d8003d4f0 0x7f9d8003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:05.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.045+0000 7f9d97e00640 1 -- 172.21.15.42:0/2438808142 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 3812+0+0 (secure 0 0 0) 0x7f9d900517c0 con 0x7f9da00a9be0 2024-07-04T17:31:05.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.046+0000 7f9d9f400640 1 --2- 172.21.15.42:0/2438808142 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f9d8003d4f0 0x7f9d8003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:05.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.046+0000 7f9d9f400640 1 --2- 172.21.15.42:0/2438808142 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f9d8003d4f0 0x7f9d8003fa00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c034000 tx=0x7f9d8c004730 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:05.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.051+0000 7f9d97e00640 1 -- 172.21.15.42:0/2438808142 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f9d90011da0 con 0x7f9da00a9be0 2024-07-04T17:31:05.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.262+0000 7f9da5600640 1 -- 172.21.15.42:0/2438808142 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f9da01409c0 con 0x7f9da00a9be0 2024-07-04T17:31:05.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.731+0000 7f9d97e00640 1 -- 172.21.15.42:0/2438808142 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'backup_fs_data' created v74) ==== 176+0+0 (secure 0 0 0) 0x7f9d9001c800 con 0x7f9da00a9be0 2024-07-04T17:31:05.733 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'backup_fs_data' created 2024-07-04T17:31:05.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.735+0000 7f9da5600640 1 -- 172.21.15.42:0/2438808142 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f9d8003d4f0 msgr2=0x7f9d8003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:05.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.735+0000 7f9da5600640 1 --2- 172.21.15.42:0/2438808142 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f9d8003d4f0 0x7f9d8003fa00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c034000 tx=0x7f9d8c004730 comp rx=0 tx=0).stop 2024-07-04T17:31:05.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.735+0000 7f9da5600640 1 -- 172.21.15.42:0/2438808142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9da00a9be0 msgr2=0x7f9da01d91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:05.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.735+0000 7f9da5600640 1 --2- 172.21.15.42:0/2438808142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9da00a9be0 0x7f9da01d91d0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000c3b0 tx=0x7f9d9000c870 comp rx=0 tx=0).stop 2024-07-04T17:31:05.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.736+0000 7f9da5600640 1 -- 172.21.15.42:0/2438808142 shutdown_connections 2024-07-04T17:31:05.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.736+0000 7f9da5600640 1 --2- 172.21.15.42:0/2438808142 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f9d8003d4f0 0x7f9d8003fa00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:05.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.736+0000 7f9da5600640 1 --2- 172.21.15.42:0/2438808142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9da00a9be0 0x7f9da01d91d0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:05.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.736+0000 7f9da5600640 1 -- 172.21.15.42:0/2438808142 >> 172.21.15.42:0/2438808142 conn(0x7f9da006c020 msgr2=0x7f9da00ae910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:05.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.736+0000 7f9da5600640 1 -- 172.21.15.42:0/2438808142 shutdown_connections 2024-07-04T17:31:05.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:05.736+0000 7f9da5600640 1 -- 172.21.15.42:0/2438808142 wait complete. 2024-07-04T17:31:05.790 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new backup_fs backup_fs_metadata backup_fs_data 2024-07-04T17:31:06.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.053+0000 7fe56ee00640 1 -- 172.21.15.42:0/1246964998 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5680b9630 msgr2=0x7fe56806c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:06.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.053+0000 7fe56ee00640 1 --2- 172.21.15.42:0/1246964998 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5680b9630 0x7fe56806c970 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fe5580097b0 tx=0x7fe55802a540 comp rx=0 tx=0).stop 2024-07-04T17:31:06.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.053+0000 7fe56ee00640 1 -- 172.21.15.42:0/1246964998 shutdown_connections 2024-07-04T17:31:06.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.053+0000 7fe56ee00640 1 --2- 172.21.15.42:0/1246964998 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5680b9630 0x7fe56806c970 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:06.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.053+0000 7fe56ee00640 1 -- 172.21.15.42:0/1246964998 >> 172.21.15.42:0/1246964998 conn(0x7fe5680bf260 msgr2=0x7fe5680af5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:06.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.053+0000 7fe56ee00640 1 -- 172.21.15.42:0/1246964998 shutdown_connections 2024-07-04T17:31:06.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.053+0000 7fe56ee00640 1 -- 172.21.15.42:0/1246964998 wait complete. 2024-07-04T17:31:06.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.056+0000 7fe56ee00640 1 Processor -- start 2024-07-04T17:31:06.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.056+0000 7fe56ee00640 1 -- start start 2024-07-04T17:31:06.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.057+0000 7fe56ee00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5680b38a0 0x7fe5680b81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:06.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.057+0000 7fe56ee00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe5680babb0 con 0x7fe5680b38a0 2024-07-04T17:31:06.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.057+0000 7fe56da00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5680b38a0 0x7fe5680b81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:06.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.057+0000 7fe56da00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5680b38a0 0x7fe5680b81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47080/0 (socket says 172.21.15.42:47080) 2024-07-04T17:31:06.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.057+0000 7fe56da00640 1 -- 172.21.15.42:0/2843512406 learned_addr learned my addr 172.21.15.42:0/2843512406 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:06.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.058+0000 7fe56da00640 1 -- 172.21.15.42:0/2843512406 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5681e3630 con 0x7fe5680b38a0 2024-07-04T17:31:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.058+0000 7fe56da00640 1 --2- 172.21.15.42:0/2843512406 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5680b38a0 0x7fe5680b81f0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fe55c00c3b0 tx=0x7fe55c00c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.058+0000 7fe565600640 1 -- 172.21.15.42:0/2843512406 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe55c011780 con 0x7fe5680b38a0 2024-07-04T17:31:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.059+0000 7fe565600640 1 -- 172.21.15.42:0/2843512406 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe55c017070 con 0x7fe5680b38a0 2024-07-04T17:31:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.059+0000 7fe565600640 1 -- 172.21.15.42:0/2843512406 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe55c011c00 con 0x7fe5680b38a0 2024-07-04T17:31:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.059+0000 7fe56ee00640 1 -- 172.21.15.42:0/2843512406 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5680b7a40 con 0x7fe5680b38a0 2024-07-04T17:31:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.059+0000 7fe56ee00640 1 -- 172.21.15.42:0/2843512406 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe568141610 con 0x7fe5680b38a0 2024-07-04T17:31:06.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.060+0000 7fe56ee00640 1 -- 172.21.15.42:0/2843512406 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5681e3450 con 0x7fe5680b38a0 2024-07-04T17:31:06.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.060+0000 7fe565600640 1 -- 172.21.15.42:0/2843512406 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7fe55c01c460 con 0x7fe5680b38a0 2024-07-04T17:31:06.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.061+0000 7fe565600640 1 --2- 172.21.15.42:0/2843512406 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fe54c03d4f0 0x7fe54c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:06.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.061+0000 7fe565600640 1 -- 172.21.15.42:0/2843512406 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 4197+0+0 (secure 0 0 0) 0x7fe55c051860 con 0x7fe5680b38a0 2024-07-04T17:31:06.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.061+0000 7fe56d000640 1 --2- 172.21.15.42:0/2843512406 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fe54c03d4f0 0x7fe54c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:06.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.062+0000 7fe56d000640 1 --2- 172.21.15.42:0/2843512406 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fe54c03d4f0 0x7fe54c03fa00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe558034000 tx=0x7fe558004730 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:06.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.067+0000 7fe565600640 1 -- 172.21.15.42:0/2843512406 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fe5681e3450 con 0x7fe5680b38a0 2024-07-04T17:31:06.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.321+0000 7fe56ee00640 1 -- 172.21.15.42:0/2843512406 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"} v 0) -- 0x7fe5680b8730 con 0x7fe5680b38a0 2024-07-04T17:31:06.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.733+0000 7fe565600640 1 -- 172.21.15.42:0/2843512406 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"}]=0 new fs with metadata pool 14 and data pool 15 v56) ==== 183+0+0 (secure 0 0 0) 0x7fe55c011da0 con 0x7fe5680b38a0 2024-07-04T17:31:06.735 INFO:teuthology.orchestra.run.smithi042.stderr:new fs with metadata pool 14 and data pool 15 2024-07-04T17:31:06.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.737+0000 7fe56ee00640 1 -- 172.21.15.42:0/2843512406 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fe54c03d4f0 msgr2=0x7fe54c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:06.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.737+0000 7fe56ee00640 1 --2- 172.21.15.42:0/2843512406 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fe54c03d4f0 0x7fe54c03fa00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe558034000 tx=0x7fe558004730 comp rx=0 tx=0).stop 2024-07-04T17:31:06.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.737+0000 7fe56ee00640 1 -- 172.21.15.42:0/2843512406 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5680b38a0 msgr2=0x7fe5680b81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:06.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.737+0000 7fe56ee00640 1 --2- 172.21.15.42:0/2843512406 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5680b38a0 0x7fe5680b81f0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fe55c00c3b0 tx=0x7fe55c00c870 comp rx=0 tx=0).stop 2024-07-04T17:31:06.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.738+0000 7fe56ee00640 1 -- 172.21.15.42:0/2843512406 shutdown_connections 2024-07-04T17:31:06.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.738+0000 7fe56ee00640 1 --2- 172.21.15.42:0/2843512406 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fe54c03d4f0 0x7fe54c03fa00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:06.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.738+0000 7fe56ee00640 1 --2- 172.21.15.42:0/2843512406 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5680b38a0 0x7fe5680b81f0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:06.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.738+0000 7fe56ee00640 1 -- 172.21.15.42:0/2843512406 >> 172.21.15.42:0/2843512406 conn(0x7fe5680bf260 msgr2=0x7fe568142560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:06.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.738+0000 7fe56ee00640 1 -- 172.21.15.42:0/2843512406 shutdown_connections 2024-07-04T17:31:06.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:06.738+0000 7fe56ee00640 1 -- 172.21.15.42:0/2843512406 wait complete. 2024-07-04T17:31:06.791 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:31:07.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.049+0000 7f1b82a00640 1 -- 172.21.15.42:0/4033812209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b7c0aabc0 msgr2=0x7f1b7c0a9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:07.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.049+0000 7f1b82a00640 1 --2- 172.21.15.42:0/4033812209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b7c0aabc0 0x7f1b7c0a9fa0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c0097b0 tx=0x7f1b6c02a530 comp rx=0 tx=0).stop 2024-07-04T17:31:07.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.050+0000 7f1b82a00640 1 -- 172.21.15.42:0/4033812209 shutdown_connections 2024-07-04T17:31:07.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.050+0000 7f1b82a00640 1 --2- 172.21.15.42:0/4033812209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b7c0aabc0 0x7f1b7c0a9fa0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:07.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.050+0000 7f1b82a00640 1 -- 172.21.15.42:0/4033812209 >> 172.21.15.42:0/4033812209 conn(0x7f1b7c0bf700 msgr2=0x7f1b7c0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:07.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.050+0000 7f1b82a00640 1 -- 172.21.15.42:0/4033812209 shutdown_connections 2024-07-04T17:31:07.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.050+0000 7f1b82a00640 1 -- 172.21.15.42:0/4033812209 wait complete. 2024-07-04T17:31:07.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.052+0000 7f1b82a00640 1 Processor -- start 2024-07-04T17:31:07.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.052+0000 7f1b82a00640 1 -- start start 2024-07-04T17:31:07.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.053+0000 7f1b82a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b7c0b38f0 0x7f1b7c071160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:07.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.053+0000 7f1b82a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c0ab180 con 0x7f1b7c0b38f0 2024-07-04T17:31:07.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.053+0000 7f1b7be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b7c0b38f0 0x7f1b7c071160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:07.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.053+0000 7f1b7be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b7c0b38f0 0x7f1b7c071160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47086/0 (socket says 172.21.15.42:47086) 2024-07-04T17:31:07.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.053+0000 7f1b7be00640 1 -- 172.21.15.42:0/2360168510 learned_addr learned my addr 172.21.15.42:0/2360168510 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:07.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.054+0000 7f1b7be00640 1 -- 172.21.15.42:0/2360168510 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7c073820 con 0x7f1b7c0b38f0 2024-07-04T17:31:07.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.054+0000 7f1b7be00640 1 --2- 172.21.15.42:0/2360168510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b7c0b38f0 0x7f1b7c071160 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c0098d0 tx=0x7f1b6c033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:07.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.054+0000 7f1b73e00640 1 -- 172.21.15.42:0/2360168510 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b6c031a90 con 0x7f1b7c0b38f0 2024-07-04T17:31:07.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.054+0000 7f1b73e00640 1 -- 172.21.15.42:0/2360168510 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b6c031d20 con 0x7f1b7c0b38f0 2024-07-04T17:31:07.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.054+0000 7f1b73e00640 1 -- 172.21.15.42:0/2360168510 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b6c033260 con 0x7f1b7c0b38f0 2024-07-04T17:31:07.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.054+0000 7f1b82a00640 1 -- 172.21.15.42:0/2360168510 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b7c072cb0 con 0x7f1b7c0b38f0 2024-07-04T17:31:07.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.054+0000 7f1b82a00640 1 -- 172.21.15.42:0/2360168510 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b7c1dc590 con 0x7f1b7c0b38f0 2024-07-04T17:31:07.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.055+0000 7f1b82a00640 1 -- 172.21.15.42:0/2360168510 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b7c0aabc0 con 0x7f1b7c0b38f0 2024-07-04T17:31:07.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.056+0000 7f1b73e00640 1 -- 172.21.15.42:0/2360168510 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f1b6c033400 con 0x7f1b7c0b38f0 2024-07-04T17:31:07.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.056+0000 7f1b73e00640 1 --2- 172.21.15.42:0/2360168510 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f1b5803d4f0 0x7f1b5803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:07.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.056+0000 7f1b73e00640 1 -- 172.21.15.42:0/2360168510 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 4303+0+0 (secure 0 0 0) 0x7f1b6c0730b0 con 0x7f1b7c0b38f0 2024-07-04T17:31:07.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.057+0000 7f1b7b400640 1 --2- 172.21.15.42:0/2360168510 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f1b5803d4f0 0x7f1b5803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:07.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.057+0000 7f1b7b400640 1 --2- 172.21.15.42:0/2360168510 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f1b5803d4f0 0x7f1b5803fa00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1b64007fd0 tx=0x7f1b64007920 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:07.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.062+0000 7f1b73e00640 1 -- 172.21.15.42:0/2360168510 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f1b6c03f030 con 0x7f1b7c0b38f0 2024-07-04T17:31:07.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.265+0000 7f1b82a00640 1 -- 172.21.15.42:0/2360168510 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b7c069b70 con 0x7f1b7c0b38f0 2024-07-04T17:31:07.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.267+0000 7f1b73e00640 1 -- 172.21.15.42:0/2360168510 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+14463 (secure 0 0 0) 0x7f1b6c04b3a0 con 0x7f1b7c0b38f0 2024-07-04T17:31:07.268 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:31:07.269 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":75,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:31:06.323361+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:30:45.017030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.4099999666213989,"score_stable":1.4099999666213989,"optimal_score":1,"raw_score_acting":1.4099999666213989,"raw_score_stable":1.4099999666213989,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-07-04T17:30:45.910003+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:31:03.814034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:31:05.281219+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1699999570846558,"score_stable":1.1699999570846558,"optimal_score":1,"raw_score_acting":1.1699999570846558,"raw_score_stable":1.1699999570846558,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:31:07.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.269+0000 7f1b82a00640 1 -- 172.21.15.42:0/2360168510 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f1b5803d4f0 msgr2=0x7f1b5803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:07.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.269+0000 7f1b82a00640 1 --2- 172.21.15.42:0/2360168510 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f1b5803d4f0 0x7f1b5803fa00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1b64007fd0 tx=0x7f1b64007920 comp rx=0 tx=0).stop 2024-07-04T17:31:07.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.270+0000 7f1b82a00640 1 -- 172.21.15.42:0/2360168510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b7c0b38f0 msgr2=0x7f1b7c071160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:07.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.270+0000 7f1b82a00640 1 --2- 172.21.15.42:0/2360168510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b7c0b38f0 0x7f1b7c071160 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c0098d0 tx=0x7f1b6c033c10 comp rx=0 tx=0).stop 2024-07-04T17:31:07.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.270+0000 7f1b82a00640 1 -- 172.21.15.42:0/2360168510 shutdown_connections 2024-07-04T17:31:07.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.270+0000 7f1b82a00640 1 --2- 172.21.15.42:0/2360168510 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f1b5803d4f0 0x7f1b5803fa00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:07.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.270+0000 7f1b82a00640 1 --2- 172.21.15.42:0/2360168510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b7c0b38f0 0x7f1b7c071160 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:07.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.270+0000 7f1b82a00640 1 -- 172.21.15.42:0/2360168510 >> 172.21.15.42:0/2360168510 conn(0x7f1b7c0bf700 msgr2=0x7f1b7c0b2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:07.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.270+0000 7f1b82a00640 1 -- 172.21.15.42:0/2360168510 shutdown_connections 2024-07-04T17:31:07.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.270+0000 7f1b82a00640 1 -- 172.21.15.42:0/2360168510 wait complete. 2024-07-04T17:31:07.295 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:31:07.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.576+0000 7f822a000640 1 -- 172.21.15.42:0/3748900021 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8224144fe0 msgr2=0x7f822406c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:07.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.576+0000 7f822a000640 1 --2- 172.21.15.42:0/3748900021 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8224144fe0 0x7f822406c020 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f82100097b0 tx=0x7f821002a530 comp rx=0 tx=0).stop 2024-07-04T17:31:07.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.576+0000 7f822a000640 1 -- 172.21.15.42:0/3748900021 shutdown_connections 2024-07-04T17:31:07.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.576+0000 7f822a000640 1 --2- 172.21.15.42:0/3748900021 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8224144fe0 0x7f822406c020 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:07.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.576+0000 7f822a000640 1 -- 172.21.15.42:0/3748900021 >> 172.21.15.42:0/3748900021 conn(0x7f82240bebc0 msgr2=0x7f82240befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:07.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.576+0000 7f822a000640 1 -- 172.21.15.42:0/3748900021 shutdown_connections 2024-07-04T17:31:07.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.576+0000 7f822a000640 1 -- 172.21.15.42:0/3748900021 wait complete. 2024-07-04T17:31:07.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.578+0000 7f822a000640 1 Processor -- start 2024-07-04T17:31:07.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.579+0000 7f822a000640 1 -- start start 2024-07-04T17:31:07.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.580+0000 7f822a000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82240ba830 0x7f82241d85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:07.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.580+0000 7f822a000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f82240b1e30 con 0x7f82240ba830 2024-07-04T17:31:07.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.580+0000 7f8228c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82240ba830 0x7f82241d85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:07.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.580+0000 7f8228c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82240ba830 0x7f82241d85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47096/0 (socket says 172.21.15.42:47096) 2024-07-04T17:31:07.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.580+0000 7f8228c00640 1 -- 172.21.15.42:0/2099929494 learned_addr learned my addr 172.21.15.42:0/2099929494 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:07.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.581+0000 7f8228c00640 1 -- 172.21.15.42:0/2099929494 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82240a9610 con 0x7f82240ba830 2024-07-04T17:31:07.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.581+0000 7f8228c00640 1 --2- 172.21.15.42:0/2099929494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82240ba830 0x7f82241d85b0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f821400cb40 tx=0x7f82140021e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:07.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.581+0000 7f8220c00640 1 -- 172.21.15.42:0/2099929494 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8214016020 con 0x7f82240ba830 2024-07-04T17:31:07.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.581+0000 7f8220c00640 1 -- 172.21.15.42:0/2099929494 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f821400abd0 con 0x7f82240ba830 2024-07-04T17:31:07.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.581+0000 7f8220c00640 1 -- 172.21.15.42:0/2099929494 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8214012420 con 0x7f82240ba830 2024-07-04T17:31:07.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.582+0000 7f822a000640 1 -- 172.21.15.42:0/2099929494 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82240aad10 con 0x7f82240ba830 2024-07-04T17:31:07.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.582+0000 7f822a000640 1 -- 172.21.15.42:0/2099929494 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f822413ffb0 con 0x7f82240ba830 2024-07-04T17:31:07.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.583+0000 7f822a000640 1 -- 172.21.15.42:0/2099929494 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f822406c020 con 0x7f82240ba830 2024-07-04T17:31:07.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.583+0000 7f8220c00640 1 -- 172.21.15.42:0/2099929494 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f8214019020 con 0x7f82240ba830 2024-07-04T17:31:07.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.584+0000 7f8220c00640 1 --2- 172.21.15.42:0/2099929494 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f820403d4f0 0x7f820403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:07.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.584+0000 7f8220c00640 1 -- 172.21.15.42:0/2099929494 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 4303+0+0 (secure 0 0 0) 0x7f8214052340 con 0x7f82240ba830 2024-07-04T17:31:07.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.584+0000 7f8223e00640 1 --2- 172.21.15.42:0/2099929494 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f820403d4f0 0x7f820403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:07.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.585+0000 7f8223e00640 1 --2- 172.21.15.42:0/2099929494 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f820403d4f0 0x7f820403fa00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8210034000 tx=0x7f8210004730 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:07.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.589+0000 7f8220c00640 1 -- 172.21.15.42:0/2099929494 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f8214020020 con 0x7f82240ba830 2024-07-04T17:31:07.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.812+0000 7f822a000640 1 -- 172.21.15.42:0/2099929494 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82241e22e0 con 0x7f82240ba830 2024-07-04T17:31:07.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.814+0000 7f8220c00640 1 -- 172.21.15.42:0/2099929494 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+14450 (secure 0 0 0) 0x7f82240a9610 con 0x7f82240ba830 2024-07-04T17:31:07.815 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:31:07.815 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":76,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:31:07.746950+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:30:45.017030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.4099999666213989,"score_stable":1.4099999666213989,"optimal_score":1,"raw_score_acting":1.4099999666213989,"raw_score_stable":1.4099999666213989,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-07-04T17:30:45.910003+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:31:03.814034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:31:05.281219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1699999570846558,"score_stable":1.1699999570846558,"optimal_score":1,"raw_score_acting":1.1699999570846558,"raw_score_stable":1.1699999570846558,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:31:07.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.816+0000 7f822a000640 1 -- 172.21.15.42:0/2099929494 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f820403d4f0 msgr2=0x7f820403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:07.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.816+0000 7f822a000640 1 --2- 172.21.15.42:0/2099929494 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f820403d4f0 0x7f820403fa00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8210034000 tx=0x7f8210004730 comp rx=0 tx=0).stop 2024-07-04T17:31:07.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.816+0000 7f822a000640 1 -- 172.21.15.42:0/2099929494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82240ba830 msgr2=0x7f82241d85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:07.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.816+0000 7f822a000640 1 --2- 172.21.15.42:0/2099929494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82240ba830 0x7f82241d85b0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f821400cb40 tx=0x7f82140021e0 comp rx=0 tx=0).stop 2024-07-04T17:31:07.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.817+0000 7f822a000640 1 -- 172.21.15.42:0/2099929494 shutdown_connections 2024-07-04T17:31:07.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.817+0000 7f822a000640 1 --2- 172.21.15.42:0/2099929494 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f820403d4f0 0x7f820403fa00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:07.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.817+0000 7f822a000640 1 --2- 172.21.15.42:0/2099929494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82240ba830 0x7f82241d85b0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:07.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.817+0000 7f822a000640 1 -- 172.21.15.42:0/2099929494 >> 172.21.15.42:0/2099929494 conn(0x7f82240bebc0 msgr2=0x7f822406fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:07.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.817+0000 7f822a000640 1 -- 172.21.15.42:0/2099929494 shutdown_connections 2024-07-04T17:31:07.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:07.817+0000 7f822a000640 1 -- 172.21.15.42:0/2099929494 wait complete. 2024-07-04T17:31:07.842 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs standby_count_wanted 0 2024-07-04T17:31:08.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.105+0000 7fb64ca00640 1 -- 172.21.15.42:0/1787877533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6480b0db0 msgr2=0x7fb6480abc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:08.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.105+0000 7fb64ca00640 1 --2- 172.21.15.42:0/1787877533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6480b0db0 0x7fb6480abc80 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fb634009230 tx=0x7fb63402b620 comp rx=0 tx=0).stop 2024-07-04T17:31:08.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.105+0000 7fb64ca00640 1 -- 172.21.15.42:0/1787877533 shutdown_connections 2024-07-04T17:31:08.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.105+0000 7fb64ca00640 1 --2- 172.21.15.42:0/1787877533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6480b0db0 0x7fb6480abc80 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:08.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.105+0000 7fb64ca00640 1 -- 172.21.15.42:0/1787877533 >> 172.21.15.42:0/1787877533 conn(0x7fb6480afaa0 msgr2=0x7fb6480afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:08.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.105+0000 7fb64ca00640 1 -- 172.21.15.42:0/1787877533 shutdown_connections 2024-07-04T17:31:08.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.105+0000 7fb64ca00640 1 -- 172.21.15.42:0/1787877533 wait complete. 2024-07-04T17:31:08.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.107+0000 7fb64ca00640 1 Processor -- start 2024-07-04T17:31:08.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.107+0000 7fb64ca00640 1 -- start start 2024-07-04T17:31:08.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.108+0000 7fb64ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6480ac5b0 0x7fb648073b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:08.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.108+0000 7fb64ca00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb6480ae590 con 0x7fb6480ac5b0 2024-07-04T17:31:08.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.109+0000 7fb647400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6480ac5b0 0x7fb648073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:08.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.109+0000 7fb647400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6480ac5b0 0x7fb648073b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47104/0 (socket says 172.21.15.42:47104) 2024-07-04T17:31:08.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.109+0000 7fb647400640 1 -- 172.21.15.42:0/3842388641 learned_addr learned my addr 172.21.15.42:0/3842388641 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:08.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.109+0000 7fb647400640 1 -- 172.21.15.42:0/3842388641 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb648072520 con 0x7fb6480ac5b0 2024-07-04T17:31:08.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.109+0000 7fb647400640 1 --2- 172.21.15.42:0/3842388641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6480ac5b0 0x7fb648073b00 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fb63800c380 tx=0x7fb63800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:08.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.110+0000 7fb63f400640 1 -- 172.21.15.42:0/3842388641 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6380116c0 con 0x7fb6480ac5b0 2024-07-04T17:31:08.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.110+0000 7fb63f400640 1 -- 172.21.15.42:0/3842388641 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb638017070 con 0x7fb6480ac5b0 2024-07-04T17:31:08.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.110+0000 7fb63f400640 1 -- 172.21.15.42:0/3842388641 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb638011b20 con 0x7fb6480ac5b0 2024-07-04T17:31:08.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.110+0000 7fb64ca00640 1 -- 172.21.15.42:0/3842388641 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb648142ab0 con 0x7fb6480ac5b0 2024-07-04T17:31:08.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.110+0000 7fb64ca00640 1 -- 172.21.15.42:0/3842388641 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb64813f9e0 con 0x7fb6480ac5b0 2024-07-04T17:31:08.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.111+0000 7fb64ca00640 1 -- 172.21.15.42:0/3842388641 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6480abc80 con 0x7fb6480ac5b0 2024-07-04T17:31:08.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.111+0000 7fb63f400640 1 -- 172.21.15.42:0/3842388641 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7fb638011cc0 con 0x7fb6480ac5b0 2024-07-04T17:31:08.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.112+0000 7fb63f400640 1 --2- 172.21.15.42:0/3842388641 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fb62803d4f0 0x7fb62803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:08.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.112+0000 7fb63f400640 1 -- 172.21.15.42:0/3842388641 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 4303+0+0 (secure 0 0 0) 0x7fb6380519f0 con 0x7fb6480ac5b0 2024-07-04T17:31:08.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.112+0000 7fb646a00640 1 --2- 172.21.15.42:0/3842388641 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fb62803d4f0 0x7fb62803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:08.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.113+0000 7fb646a00640 1 --2- 172.21.15.42:0/3842388641 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fb62803d4f0 0x7fb62803fa00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb634035000 tx=0x7fb634004730 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:08.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.117+0000 7fb63f400640 1 -- 172.21.15.42:0/3842388641 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fb638010cf0 con 0x7fb6480ac5b0 2024-07-04T17:31:08.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.356+0000 7fb64ca00640 1 -- 172.21.15.42:0/3842388641 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb648074050 con 0x7fb6480ac5b0 2024-07-04T17:31:08.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.756+0000 7fb63f400640 1 -- 172.21.15.42:0/3842388641 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"}]=0 v59) ==== 121+0+0 (secure 0 0 0) 0x7fb638019310 con 0x7fb6480ac5b0 2024-07-04T17:31:08.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.760+0000 7fb64ca00640 1 -- 172.21.15.42:0/3842388641 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fb62803d4f0 msgr2=0x7fb62803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:08.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.760+0000 7fb64ca00640 1 --2- 172.21.15.42:0/3842388641 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fb62803d4f0 0x7fb62803fa00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb634035000 tx=0x7fb634004730 comp rx=0 tx=0).stop 2024-07-04T17:31:08.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.760+0000 7fb64ca00640 1 -- 172.21.15.42:0/3842388641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6480ac5b0 msgr2=0x7fb648073b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:08.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.760+0000 7fb64ca00640 1 --2- 172.21.15.42:0/3842388641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6480ac5b0 0x7fb648073b00 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fb63800c380 tx=0x7fb63800c840 comp rx=0 tx=0).stop 2024-07-04T17:31:08.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.760+0000 7fb64ca00640 1 -- 172.21.15.42:0/3842388641 shutdown_connections 2024-07-04T17:31:08.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.760+0000 7fb64ca00640 1 --2- 172.21.15.42:0/3842388641 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fb62803d4f0 0x7fb62803fa00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:08.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.760+0000 7fb64ca00640 1 --2- 172.21.15.42:0/3842388641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6480ac5b0 0x7fb648073b00 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:08.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.760+0000 7fb64ca00640 1 -- 172.21.15.42:0/3842388641 >> 172.21.15.42:0/3842388641 conn(0x7fb6480afaa0 msgr2=0x7fb648075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:08.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.761+0000 7fb64ca00640 1 -- 172.21.15.42:0/3842388641 shutdown_connections 2024-07-04T17:31:08.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:08.761+0000 7fb64ca00640 1 -- 172.21.15.42:0/3842388641 wait complete. 2024-07-04T17:31:08.798 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-04T17:31:08.799 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs allow_standby_replay false 2024-07-04T17:31:09.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.094+0000 7f4440c00640 1 -- 172.21.15.42:0/4158276866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f443c0a94d0 msgr2=0x7f443c0b1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:09.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.094+0000 7f4440c00640 1 --2- 172.21.15.42:0/4158276866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f443c0a94d0 0x7f443c0b1100 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f442c00b2b0 tx=0x7f442c02ad10 comp rx=0 tx=0).stop 2024-07-04T17:31:09.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.095+0000 7f4440c00640 1 -- 172.21.15.42:0/4158276866 shutdown_connections 2024-07-04T17:31:09.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.095+0000 7f4440c00640 1 --2- 172.21.15.42:0/4158276866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f443c0a94d0 0x7f443c0b1100 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:09.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.095+0000 7f4440c00640 1 -- 172.21.15.42:0/4158276866 >> 172.21.15.42:0/4158276866 conn(0x7f443c06c020 msgr2=0x7f443c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:09.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.095+0000 7f4440c00640 1 -- 172.21.15.42:0/4158276866 shutdown_connections 2024-07-04T17:31:09.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.095+0000 7f4440c00640 1 -- 172.21.15.42:0/4158276866 wait complete. 2024-07-04T17:31:09.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.097+0000 7f4440c00640 1 Processor -- start 2024-07-04T17:31:09.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.098+0000 7f4440c00640 1 -- start start 2024-07-04T17:31:09.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.099+0000 7f4440c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f443c073b50 0x7f443c0af440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:09.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.100+0000 7f4440c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f443c072740 con 0x7f443c073b50 2024-07-04T17:31:09.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.100+0000 7f4439e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f443c073b50 0x7f443c0af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:09.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.100+0000 7f4439e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f443c073b50 0x7f443c0af440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47122/0 (socket says 172.21.15.42:47122) 2024-07-04T17:31:09.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.100+0000 7f4439e00640 1 -- 172.21.15.42:0/1094019590 learned_addr learned my addr 172.21.15.42:0/1094019590 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:09.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.100+0000 7f4439e00640 1 -- 172.21.15.42:0/1094019590 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f443c0acdc0 con 0x7f443c073b50 2024-07-04T17:31:09.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.101+0000 7f4439e00640 1 --2- 172.21.15.42:0/1094019590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f443c073b50 0x7f443c0af440 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f442c000c00 tx=0x7f442c0099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:09.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.101+0000 7f4432a00640 1 -- 172.21.15.42:0/1094019590 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f442c038030 con 0x7f443c073b50 2024-07-04T17:31:09.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.101+0000 7f4432a00640 1 -- 172.21.15.42:0/1094019590 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f442c042070 con 0x7f443c073b50 2024-07-04T17:31:09.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.101+0000 7f4432a00640 1 -- 172.21.15.42:0/1094019590 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f442c0035f0 con 0x7f443c073b50 2024-07-04T17:31:09.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.101+0000 7f4440c00640 1 -- 172.21.15.42:0/1094019590 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f443c0ae810 con 0x7f443c073b50 2024-07-04T17:31:09.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.101+0000 7f4440c00640 1 -- 172.21.15.42:0/1094019590 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f443c13b7f0 con 0x7f443c073b50 2024-07-04T17:31:09.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.103+0000 7f4440c00640 1 -- 172.21.15.42:0/1094019590 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f443c0b1100 con 0x7f443c073b50 2024-07-04T17:31:09.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.103+0000 7f4432a00640 1 -- 172.21.15.42:0/1094019590 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f442c003790 con 0x7f443c073b50 2024-07-04T17:31:09.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.103+0000 7f4432a00640 1 --2- 172.21.15.42:0/1094019590 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f441803d4a0 0x7f441803f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:09.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.103+0000 7f4432a00640 1 -- 172.21.15.42:0/1094019590 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 4303+0+0 (secure 0 0 0) 0x7f442c074b20 con 0x7f443c073b50 2024-07-04T17:31:09.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.104+0000 7f4431400640 1 --2- 172.21.15.42:0/1094019590 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f441803d4a0 0x7f441803f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:09.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.105+0000 7f4431400640 1 --2- 172.21.15.42:0/1094019590 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f441803d4a0 0x7f441803f9b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f443c073660 tx=0x7f4424007a90 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:09.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.111+0000 7f4432a00640 1 -- 172.21.15.42:0/1094019590 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f442c045360 con 0x7f443c073b50 2024-07-04T17:31:09.335 INFO:tasks.ceph.mds.b.smithi042.stderr:2024-07-04T17:31:09.334+0000 7f5784400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-04T17:31:09.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.364+0000 7f4440c00640 1 -- 172.21.15.42:0/1094019590 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f443c0acf90 con 0x7f443c073b50 2024-07-04T17:31:09.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.772+0000 7f4432a00640 1 -- 172.21.15.42:0/1094019590 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"}]=0 v60) ==== 125+0+0 (secure 0 0 0) 0x7f442c031b70 con 0x7f443c073b50 2024-07-04T17:31:09.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.775+0000 7f4440c00640 1 -- 172.21.15.42:0/1094019590 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f441803d4a0 msgr2=0x7f441803f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:09.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.775+0000 7f4440c00640 1 --2- 172.21.15.42:0/1094019590 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f441803d4a0 0x7f441803f9b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f443c073660 tx=0x7f4424007a90 comp rx=0 tx=0).stop 2024-07-04T17:31:09.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.775+0000 7f4440c00640 1 -- 172.21.15.42:0/1094019590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f443c073b50 msgr2=0x7f443c0af440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:09.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.775+0000 7f4440c00640 1 --2- 172.21.15.42:0/1094019590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f443c073b50 0x7f443c0af440 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f442c000c00 tx=0x7f442c0099c0 comp rx=0 tx=0).stop 2024-07-04T17:31:09.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.776+0000 7f4440c00640 1 -- 172.21.15.42:0/1094019590 shutdown_connections 2024-07-04T17:31:09.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.776+0000 7f4440c00640 1 --2- 172.21.15.42:0/1094019590 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f441803d4a0 0x7f441803f9b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:09.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.776+0000 7f4440c00640 1 --2- 172.21.15.42:0/1094019590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f443c073b50 0x7f443c0af440 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:09.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.776+0000 7f4440c00640 1 -- 172.21.15.42:0/1094019590 >> 172.21.15.42:0/1094019590 conn(0x7f443c06c020 msgr2=0x7f443c075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:09.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.776+0000 7f4440c00640 1 -- 172.21.15.42:0/1094019590 shutdown_connections 2024-07-04T17:31:09.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:09.776+0000 7f4440c00640 1 -- 172.21.15.42:0/1094019590 wait complete. 2024-07-04T17:31:09.813 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'backup_fs' 2024-07-04T17:31:09.813 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:31:10.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.104+0000 7f0b73e00640 1 -- 172.21.15.42:0/2641449388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b7406c2c0 msgr2=0x7f0b740b5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:10.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.104+0000 7f0b73e00640 1 --2- 172.21.15.42:0/2641449388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b7406c2c0 0x7f0b740b5650 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f0b6400b2b0 tx=0x7f0b6402ac80 comp rx=0 tx=0).stop 2024-07-04T17:31:10.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.104+0000 7f0b73e00640 1 -- 172.21.15.42:0/2641449388 shutdown_connections 2024-07-04T17:31:10.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.104+0000 7f0b73e00640 1 --2- 172.21.15.42:0/2641449388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b7406c2c0 0x7f0b740b5650 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:10.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.105+0000 7f0b73e00640 1 -- 172.21.15.42:0/2641449388 >> 172.21.15.42:0/2641449388 conn(0x7f0b740bebc0 msgr2=0x7f0b740befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:10.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.105+0000 7f0b73e00640 1 -- 172.21.15.42:0/2641449388 shutdown_connections 2024-07-04T17:31:10.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.105+0000 7f0b73e00640 1 -- 172.21.15.42:0/2641449388 wait complete. 2024-07-04T17:31:10.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.107+0000 7f0b73e00640 1 Processor -- start 2024-07-04T17:31:10.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.107+0000 7f0b73e00640 1 -- start start 2024-07-04T17:31:10.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.108+0000 7f0b73e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b7406c2c0 0x7f0b740b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:10.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.108+0000 7f0b73e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0b740b2810 con 0x7f0b7406c2c0 2024-07-04T17:31:10.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.108+0000 7f0b72a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b7406c2c0 0x7f0b740b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:10.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.108+0000 7f0b72a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b7406c2c0 0x7f0b740b5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47130/0 (socket says 172.21.15.42:47130) 2024-07-04T17:31:10.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.108+0000 7f0b72a00640 1 -- 172.21.15.42:0/1272117945 learned_addr learned my addr 172.21.15.42:0/1272117945 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:10.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.109+0000 7f0b72a00640 1 -- 172.21.15.42:0/1272117945 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b740b6ff0 con 0x7f0b7406c2c0 2024-07-04T17:31:10.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.109+0000 7f0b72a00640 1 --2- 172.21.15.42:0/1272117945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b7406c2c0 0x7f0b740b5e10 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c00c320 tx=0x7f0b5c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:10.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.109+0000 7f0b6aa00640 1 -- 172.21.15.42:0/1272117945 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b5c0116c0 con 0x7f0b7406c2c0 2024-07-04T17:31:10.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.109+0000 7f0b6aa00640 1 -- 172.21.15.42:0/1272117945 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b5c017070 con 0x7f0b7406c2c0 2024-07-04T17:31:10.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.109+0000 7f0b6aa00640 1 -- 172.21.15.42:0/1272117945 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b5c011b40 con 0x7f0b7406c2c0 2024-07-04T17:31:10.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.110+0000 7f0b73e00640 1 -- 172.21.15.42:0/1272117945 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b740b6c30 con 0x7f0b7406c2c0 2024-07-04T17:31:10.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.110+0000 7f0b73e00640 1 -- 172.21.15.42:0/1272117945 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b741d3e50 con 0x7f0b7406c2c0 2024-07-04T17:31:10.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.111+0000 7f0b73e00640 1 -- 172.21.15.42:0/1272117945 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b740b6e00 con 0x7f0b7406c2c0 2024-07-04T17:31:10.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.111+0000 7f0b6aa00640 1 -- 172.21.15.42:0/1272117945 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f0b5c011ce0 con 0x7f0b7406c2c0 2024-07-04T17:31:10.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.111+0000 7f0b6aa00640 1 --2- 172.21.15.42:0/1272117945 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f0b5003d4f0 0x7f0b5003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:10.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.111+0000 7f0b6aa00640 1 -- 172.21.15.42:0/1272117945 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 4303+0+0 (secure 0 0 0) 0x7f0b5c0519f0 con 0x7f0b7406c2c0 2024-07-04T17:31:10.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.112+0000 7f0b72000640 1 --2- 172.21.15.42:0/1272117945 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f0b5003d4f0 0x7f0b5003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:10.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.112+0000 7f0b72000640 1 --2- 172.21.15.42:0/1272117945 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f0b5003d4f0 0x7f0b5003fa00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0b64035000 tx=0x7f0b64004730 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:10.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.116+0000 7f0b6aa00640 1 -- 172.21.15.42:0/1272117945 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f0b7406c6d0 con 0x7f0b7406c2c0 2024-07-04T17:31:10.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.363+0000 7f0b73e00640 1 -- 172.21.15.42:0/1272117945 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b740b5650 con 0x7f0b7406c2c0 2024-07-04T17:31:10.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.364+0000 7f0b6aa00640 1 -- 172.21.15.42:0/1272117945 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+7369 (secure 0 0 0) 0x7f0b5c00a140 con 0x7f0b7406c2c0 2024-07-04T17:31:10.366 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:31:10.366 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":60,"btime":"2024-07-04T17:31:09:763642+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5017,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/2312272332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2312272332},{"type":"v1","addr":"172.21.15.42:6827","nonce":2312272332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5020,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1285161495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1285161495},{"type":"v1","addr":"172.21.15.42:6829","nonce":1285161495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5023,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/1583070649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1583070649},{"type":"v1","addr":"172.21.15.42:6831","nonce":1583070649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:30:46.947014+0000","modified":"2024-07-04T17:30:49.991386+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5029":{"gid":5029,"name":"e","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6835/1458267161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1458267161},{"type":"v1","addr":"172.21.15.42:6835","nonce":1458267161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5029,"qdb_cluster":[5029]},"id":6},{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:31:06.323313+0000","modified":"2024-07-04T17:31:09.367074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5026":{"gid":5026,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6833/2038326930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2038326930},{"type":"v1","addr":"172.21.15.42:6833","nonce":2038326930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5026,"qdb_cluster":[5026]},"id":7}]} 2024-07-04T17:31:10.366 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 60 2024-07-04T17:31:10.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.367+0000 7f0b73e00640 1 -- 172.21.15.42:0/1272117945 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f0b5003d4f0 msgr2=0x7f0b5003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:10.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.367+0000 7f0b73e00640 1 --2- 172.21.15.42:0/1272117945 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f0b5003d4f0 0x7f0b5003fa00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0b64035000 tx=0x7f0b64004730 comp rx=0 tx=0).stop 2024-07-04T17:31:10.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.367+0000 7f0b73e00640 1 -- 172.21.15.42:0/1272117945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b7406c2c0 msgr2=0x7f0b740b5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:10.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.367+0000 7f0b73e00640 1 --2- 172.21.15.42:0/1272117945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b7406c2c0 0x7f0b740b5e10 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c00c320 tx=0x7f0b5c00c7e0 comp rx=0 tx=0).stop 2024-07-04T17:31:10.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.367+0000 7f0b73e00640 1 -- 172.21.15.42:0/1272117945 shutdown_connections 2024-07-04T17:31:10.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.367+0000 7f0b73e00640 1 --2- 172.21.15.42:0/1272117945 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f0b5003d4f0 0x7f0b5003fa00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:10.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.367+0000 7f0b73e00640 1 --2- 172.21.15.42:0/1272117945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b7406c2c0 0x7f0b740b5e10 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:10.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.367+0000 7f0b73e00640 1 -- 172.21.15.42:0/1272117945 >> 172.21.15.42:0/1272117945 conn(0x7f0b740bebc0 msgr2=0x7f0b740befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:10.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.367+0000 7f0b73e00640 1 -- 172.21.15.42:0/1272117945 shutdown_connections 2024-07-04T17:31:10.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.367+0000 7f0b73e00640 1 -- 172.21.15.42:0/1272117945 wait complete. 2024-07-04T17:31:10.392 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:31:10.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.673+0000 7f2e84e00640 1 -- 172.21.15.42:0/2924204767 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e800b1d20 msgr2=0x7f2e800b6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:10.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.673+0000 7f2e84e00640 1 --2- 172.21.15.42:0/2924204767 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e800b1d20 0x7f2e800b6230 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c0097b0 tx=0x7f2e6c02a540 comp rx=0 tx=0).stop 2024-07-04T17:31:10.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.673+0000 7f2e84e00640 1 -- 172.21.15.42:0/2924204767 shutdown_connections 2024-07-04T17:31:10.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.673+0000 7f2e84e00640 1 --2- 172.21.15.42:0/2924204767 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e800b1d20 0x7f2e800b6230 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:10.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.673+0000 7f2e84e00640 1 -- 172.21.15.42:0/2924204767 >> 172.21.15.42:0/2924204767 conn(0x7f2e800bebc0 msgr2=0x7f2e800befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:10.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.673+0000 7f2e84e00640 1 -- 172.21.15.42:0/2924204767 shutdown_connections 2024-07-04T17:31:10.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.673+0000 7f2e84e00640 1 -- 172.21.15.42:0/2924204767 wait complete. 2024-07-04T17:31:10.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.675+0000 7f2e84e00640 1 Processor -- start 2024-07-04T17:31:10.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.676+0000 7f2e84e00640 1 -- start start 2024-07-04T17:31:10.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.677+0000 7f2e84e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e800b1d20 0x7f2e8013a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:10.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.677+0000 7f2e84e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2e800b8b40 con 0x7f2e800b1d20 2024-07-04T17:31:10.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.677+0000 7f2e7f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e800b1d20 0x7f2e8013a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:10.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.677+0000 7f2e7f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e800b1d20 0x7f2e8013a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47150/0 (socket says 172.21.15.42:47150) 2024-07-04T17:31:10.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.677+0000 7f2e7f400640 1 -- 172.21.15.42:0/3960066829 learned_addr learned my addr 172.21.15.42:0/3960066829 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:10.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.678+0000 7f2e7f400640 1 -- 172.21.15.42:0/3960066829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e800734f0 con 0x7f2e800b1d20 2024-07-04T17:31:10.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.678+0000 7f2e7f400640 1 --2- 172.21.15.42:0/3960066829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e800b1d20 0x7f2e8013a8e0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f2e7000c320 tx=0x7f2e7000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:10.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.678+0000 7f2e77400640 1 -- 172.21.15.42:0/3960066829 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e700116c0 con 0x7f2e800b1d20 2024-07-04T17:31:10.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.678+0000 7f2e77400640 1 -- 172.21.15.42:0/3960066829 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e70017070 con 0x7f2e800b1d20 2024-07-04T17:31:10.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.678+0000 7f2e77400640 1 -- 172.21.15.42:0/3960066829 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e70011b40 con 0x7f2e800b1d20 2024-07-04T17:31:10.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.678+0000 7f2e84e00640 1 -- 172.21.15.42:0/3960066829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e800739d0 con 0x7f2e800b1d20 2024-07-04T17:31:10.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.679+0000 7f2e84e00640 1 -- 172.21.15.42:0/3960066829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e801d7650 con 0x7f2e800b1d20 2024-07-04T17:31:10.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.679+0000 7f2e84e00640 1 -- 172.21.15.42:0/3960066829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e800b91d0 con 0x7f2e800b1d20 2024-07-04T17:31:10.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.680+0000 7f2e77400640 1 -- 172.21.15.42:0/3960066829 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f2e70011ce0 con 0x7f2e800b1d20 2024-07-04T17:31:10.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.680+0000 7f2e77400640 1 --2- 172.21.15.42:0/3960066829 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f2e6003d4f0 0x7f2e6003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:10.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.680+0000 7f2e77400640 1 -- 172.21.15.42:0/3960066829 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 4303+0+0 (secure 0 0 0) 0x7f2e700519b0 con 0x7f2e800b1d20 2024-07-04T17:31:10.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.681+0000 7f2e7ea00640 1 --2- 172.21.15.42:0/3960066829 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f2e6003d4f0 0x7f2e6003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:10.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.681+0000 7f2e7ea00640 1 --2- 172.21.15.42:0/3960066829 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f2e6003d4f0 0x7f2e6003fa00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c034000 tx=0x7f2e6c004730 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:10.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.685+0000 7f2e77400640 1 -- 172.21.15.42:0/3960066829 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f2e8013ec10 con 0x7f2e800b1d20 2024-07-04T17:31:10.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.888+0000 7f2e84e00640 1 -- 172.21.15.42:0/3960066829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2e80068e30 con 0x7f2e800b1d20 2024-07-04T17:31:10.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.891+0000 7f2e77400640 1 -- 172.21.15.42:0/3960066829 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+14450 (secure 0 0 0) 0x7f2e7000a140 con 0x7f2e800b1d20 2024-07-04T17:31:10.892 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:31:10.893 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":76,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:31:07.746950+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:30:45.017030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.4099999666213989,"score_stable":1.4099999666213989,"optimal_score":1,"raw_score_acting":1.4099999666213989,"raw_score_stable":1.4099999666213989,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-07-04T17:30:45.910003+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:31:03.814034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:31:05.281219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1699999570846558,"score_stable":1.1699999570846558,"optimal_score":1,"raw_score_acting":1.1699999570846558,"raw_score_stable":1.1699999570846558,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:31:10.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.894+0000 7f2e84e00640 1 -- 172.21.15.42:0/3960066829 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f2e6003d4f0 msgr2=0x7f2e6003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:10.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.894+0000 7f2e84e00640 1 --2- 172.21.15.42:0/3960066829 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f2e6003d4f0 0x7f2e6003fa00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c034000 tx=0x7f2e6c004730 comp rx=0 tx=0).stop 2024-07-04T17:31:10.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.894+0000 7f2e84e00640 1 -- 172.21.15.42:0/3960066829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e800b1d20 msgr2=0x7f2e8013a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:10.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.894+0000 7f2e84e00640 1 --2- 172.21.15.42:0/3960066829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e800b1d20 0x7f2e8013a8e0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f2e7000c320 tx=0x7f2e7000c7e0 comp rx=0 tx=0).stop 2024-07-04T17:31:10.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.894+0000 7f2e84e00640 1 -- 172.21.15.42:0/3960066829 shutdown_connections 2024-07-04T17:31:10.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.894+0000 7f2e84e00640 1 --2- 172.21.15.42:0/3960066829 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f2e6003d4f0 0x7f2e6003fa00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:10.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.894+0000 7f2e84e00640 1 --2- 172.21.15.42:0/3960066829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e800b1d20 0x7f2e8013a8e0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:10.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.894+0000 7f2e84e00640 1 -- 172.21.15.42:0/3960066829 >> 172.21.15.42:0/3960066829 conn(0x7f2e800bebc0 msgr2=0x7f2e8006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:10.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.894+0000 7f2e84e00640 1 -- 172.21.15.42:0/3960066829 shutdown_connections 2024-07-04T17:31:10.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:10.894+0000 7f2e84e00640 1 -- 172.21.15.42:0/3960066829 wait complete. 2024-07-04T17:31:10.922 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-04T17:31:10.923 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:31:11.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.205+0000 7f054ae00640 1 -- 172.21.15.42:0/164255069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f054406edd0 msgr2=0x7f05440b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:11.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.205+0000 7f054ae00640 1 --2- 172.21.15.42:0/164255069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f054406edd0 0x7f05440b3030 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f0534005fd0 tx=0x7f053402bc00 comp rx=0 tx=0).stop 2024-07-04T17:31:11.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.205+0000 7f054ae00640 1 -- 172.21.15.42:0/164255069 shutdown_connections 2024-07-04T17:31:11.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.205+0000 7f054ae00640 1 --2- 172.21.15.42:0/164255069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f054406edd0 0x7f05440b3030 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:11.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.205+0000 7f054ae00640 1 -- 172.21.15.42:0/164255069 >> 172.21.15.42:0/164255069 conn(0x7f05440bf700 msgr2=0x7f05440af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:11.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.206+0000 7f054ae00640 1 -- 172.21.15.42:0/164255069 shutdown_connections 2024-07-04T17:31:11.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.206+0000 7f054ae00640 1 -- 172.21.15.42:0/164255069 wait complete. 2024-07-04T17:31:11.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.208+0000 7f054ae00640 1 Processor -- start 2024-07-04T17:31:11.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.208+0000 7f054ae00640 1 -- start start 2024-07-04T17:31:11.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.209+0000 7f054ae00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f054406cea0 0x7f05441d87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:11.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.209+0000 7f054ae00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f05440b0370 con 0x7f054406cea0 2024-07-04T17:31:11.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.210+0000 7f0543e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f054406cea0 0x7f05441d87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:11.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.210+0000 7f0543e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f054406cea0 0x7f05441d87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47164/0 (socket says 172.21.15.42:47164) 2024-07-04T17:31:11.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.210+0000 7f0543e00640 1 -- 172.21.15.42:0/3041054070 learned_addr learned my addr 172.21.15.42:0/3041054070 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:11.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.210+0000 7f0543e00640 1 -- 172.21.15.42:0/3041054070 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05440724b0 con 0x7f054406cea0 2024-07-04T17:31:11.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.211+0000 7f0543e00640 1 --2- 172.21.15.42:0/3041054070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f054406cea0 0x7f05441d87f0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f0534036040 tx=0x7f0534004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:11.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.211+0000 7f0540c00640 1 -- 172.21.15.42:0/3041054070 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0534041020 con 0x7f054406cea0 2024-07-04T17:31:11.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.211+0000 7f0540c00640 1 -- 172.21.15.42:0/3041054070 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f053403d440 con 0x7f054406cea0 2024-07-04T17:31:11.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.211+0000 7f0540c00640 1 -- 172.21.15.42:0/3041054070 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f053403d7c0 con 0x7f054406cea0 2024-07-04T17:31:11.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.211+0000 7f054ae00640 1 -- 172.21.15.42:0/3041054070 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05441e4a90 con 0x7f054406cea0 2024-07-04T17:31:11.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.211+0000 7f054ae00640 1 -- 172.21.15.42:0/3041054070 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0544146330 con 0x7f054406cea0 2024-07-04T17:31:11.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.212+0000 7f054ae00640 1 -- 172.21.15.42:0/3041054070 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05440b3030 con 0x7f054406cea0 2024-07-04T17:31:11.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.212+0000 7f0540c00640 1 -- 172.21.15.42:0/3041054070 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f0534044020 con 0x7f054406cea0 2024-07-04T17:31:11.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.213+0000 7f0540c00640 1 --2- 172.21.15.42:0/3041054070 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f052003d4f0 0x7f052003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:11.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.213+0000 7f0540c00640 1 -- 172.21.15.42:0/3041054070 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 4303+0+0 (secure 0 0 0) 0x7f0534073860 con 0x7f054406cea0 2024-07-04T17:31:11.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.213+0000 7f053b400640 1 --2- 172.21.15.42:0/3041054070 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f052003d4f0 0x7f052003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:11.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.214+0000 7f053b400640 1 --2- 172.21.15.42:0/3041054070 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f052003d4f0 0x7f052003fa00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f052c007fd0 tx=0x7f052c007920 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:11.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.219+0000 7f0540c00640 1 -- 172.21.15.42:0/3041054070 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f053404b030 con 0x7f054406cea0 2024-07-04T17:31:11.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.425+0000 7f054ae00640 1 -- 172.21.15.42:0/3041054070 --> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0544069b70 con 0x7f052003d4f0 2024-07-04T17:31:11.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.444+0000 7f0540c00640 1 -- 172.21.15.42:0/3041054070 <== mgr.4906 v2:172.21.15.42:6824/3473020706 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+477154 (secure 0 0 0) 0x7f0544069b70 con 0x7f052003d4f0 2024-07-04T17:31:11.450 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:31:11.456 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:31:11.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.454+0000 7f054ae00640 1 -- 172.21.15.42:0/3041054070 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f052003d4f0 msgr2=0x7f052003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:11.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.454+0000 7f054ae00640 1 --2- 172.21.15.42:0/3041054070 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f052003d4f0 0x7f052003fa00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f052c007fd0 tx=0x7f052c007920 comp rx=0 tx=0).stop 2024-07-04T17:31:11.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.454+0000 7f054ae00640 1 -- 172.21.15.42:0/3041054070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f054406cea0 msgr2=0x7f05441d87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:11.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.454+0000 7f054ae00640 1 --2- 172.21.15.42:0/3041054070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f054406cea0 0x7f05441d87f0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f0534036040 tx=0x7f0534004270 comp rx=0 tx=0).stop 2024-07-04T17:31:11.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.455+0000 7f054ae00640 1 -- 172.21.15.42:0/3041054070 shutdown_connections 2024-07-04T17:31:11.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.455+0000 7f054ae00640 1 --2- 172.21.15.42:0/3041054070 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f052003d4f0 0x7f052003fa00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:11.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.455+0000 7f054ae00640 1 --2- 172.21.15.42:0/3041054070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f054406cea0 0x7f05441d87f0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:11.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.455+0000 7f054ae00640 1 -- 172.21.15.42:0/3041054070 >> 172.21.15.42:0/3041054070 conn(0x7f05440bf700 msgr2=0x7f05440bb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:11.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.455+0000 7f054ae00640 1 -- 172.21.15.42:0/3041054070 shutdown_connections 2024-07-04T17:31:11.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.455+0000 7f054ae00640 1 -- 172.21.15.42:0/3041054070 wait complete. 2024-07-04T17:31:11.482 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":44,"stamp":"2024-07-04T17:31:09.633977+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594475,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":296,"num_read_kb":750,"num_write":174,"num_write_kb":1123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121,"ondisk_log_size":121,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":281,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":105264,"kb_used_data":3396,"kb_used_omap":4,"kb_used_meta":101819,"kb_avail":283010256,"statfs":{"total":289910292480,"available":289802502144,"internally_reserved":0,"allocated":3477504,"data_stored":2062478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":104262991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.913483"},"pg_stats":[{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.764236+0000","last_change":"2024-07-04T17:31:05.764236+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.764236+0000","last_undegraded":"2024-07-04T17:31:05.764236+0000","last_fullsized":"2024-07-04T17:31:05.764236+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3f","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.743989+0000","last_change":"2024-07-04T17:30:47.430443+0000","last_active":"2024-07-04T17:31:05.743989+0000","last_peered":"2024-07-04T17:31:05.743989+0000","last_clean":"2024-07-04T17:31:05.743989+0000","last_became_active":"2024-07-04T17:30:47.425325+0000","last_became_peered":"2024-07-04T17:30:47.425325+0000","last_unstale":"2024-07-04T17:31:05.743989+0000","last_undegraded":"2024-07-04T17:31:05.743989+0000","last_fullsized":"2024-07-04T17:31:05.743989+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:44:15.710215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.766572+0000","last_change":"2024-07-04T17:31:05.766572+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.766572+0000","last_undegraded":"2024-07-04T17:31:05.766572+0000","last_fullsized":"2024-07-04T17:31:05.766572+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.742125+0000","last_change":"2024-07-04T17:30:47.429677+0000","last_active":"2024-07-04T17:31:05.742125+0000","last_peered":"2024-07-04T17:31:05.742125+0000","last_clean":"2024-07-04T17:31:05.742125+0000","last_became_active":"2024-07-04T17:30:47.428028+0000","last_became_peered":"2024-07-04T17:30:47.428028+0000","last_unstale":"2024-07-04T17:31:05.742125+0000","last_undegraded":"2024-07-04T17:31:05.742125+0000","last_fullsized":"2024-07-04T17:31:05.742125+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:41:29.873067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.764392+0000","last_change":"2024-07-04T17:31:05.764392+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.764392+0000","last_undegraded":"2024-07-04T17:31:05.764392+0000","last_fullsized":"2024-07-04T17:31:05.764392+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.782151+0000","last_change":"2024-07-04T17:30:47.423800+0000","last_active":"2024-07-04T17:31:05.782151+0000","last_peered":"2024-07-04T17:31:05.782151+0000","last_clean":"2024-07-04T17:31:05.782151+0000","last_became_active":"2024-07-04T17:30:47.423361+0000","last_became_peered":"2024-07-04T17:30:47.423361+0000","last_unstale":"2024-07-04T17:31:05.782151+0000","last_undegraded":"2024-07-04T17:31:05.782151+0000","last_fullsized":"2024-07-04T17:31:05.782151+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:25:11.379938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.759161+0000","last_change":"2024-07-04T17:31:05.759161+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.759161+0000","last_undegraded":"2024-07-04T17:31:05.759161+0000","last_fullsized":"2024-07-04T17:31:05.759161+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.775406+0000","last_change":"2024-07-04T17:30:47.422048+0000","last_active":"2024-07-04T17:31:05.775406+0000","last_peered":"2024-07-04T17:31:05.775406+0000","last_clean":"2024-07-04T17:31:05.775406+0000","last_became_active":"2024-07-04T17:30:47.421591+0000","last_became_peered":"2024-07-04T17:30:47.421591+0000","last_unstale":"2024-07-04T17:31:05.775406+0000","last_undegraded":"2024-07-04T17:31:05.775406+0000","last_fullsized":"2024-07-04T17:31:05.775406+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:58:22.418379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.747551+0000","last_change":"2024-07-04T17:31:05.747551+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.747551+0000","last_undegraded":"2024-07-04T17:31:05.747551+0000","last_fullsized":"2024-07-04T17:31:05.747551+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.774843+0000","last_change":"2024-07-04T17:30:47.414496+0000","last_active":"2024-07-04T17:31:05.774843+0000","last_peered":"2024-07-04T17:31:05.774843+0000","last_clean":"2024-07-04T17:31:05.774843+0000","last_became_active":"2024-07-04T17:30:47.414141+0000","last_became_peered":"2024-07-04T17:30:47.414141+0000","last_unstale":"2024-07-04T17:31:05.774843+0000","last_undegraded":"2024-07-04T17:31:05.774843+0000","last_fullsized":"2024-07-04T17:31:05.774843+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:09:08.612743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.757554+0000","last_change":"2024-07-04T17:31:05.757554+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.757554+0000","last_undegraded":"2024-07-04T17:31:05.757554+0000","last_fullsized":"2024-07-04T17:31:05.757554+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.772368+0000","last_change":"2024-07-04T17:30:47.419292+0000","last_active":"2024-07-04T17:31:05.772368+0000","last_peered":"2024-07-04T17:31:05.772368+0000","last_clean":"2024-07-04T17:31:05.772368+0000","last_became_active":"2024-07-04T17:30:47.418053+0000","last_became_peered":"2024-07-04T17:30:47.418053+0000","last_unstale":"2024-07-04T17:31:05.772368+0000","last_undegraded":"2024-07-04T17:31:05.772368+0000","last_fullsized":"2024-07-04T17:31:05.772368+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:02:17.626429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.748250+0000","last_change":"2024-07-04T17:31:05.748250+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.748250+0000","last_undegraded":"2024-07-04T17:31:05.748250+0000","last_fullsized":"2024-07-04T17:31:05.748250+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.773328+0000","last_change":"2024-07-04T17:30:47.436774+0000","last_active":"2024-07-04T17:31:05.773328+0000","last_peered":"2024-07-04T17:31:05.773328+0000","last_clean":"2024-07-04T17:31:05.773328+0000","last_became_active":"2024-07-04T17:30:47.436573+0000","last_became_peered":"2024-07-04T17:30:47.436573+0000","last_unstale":"2024-07-04T17:31:05.773328+0000","last_undegraded":"2024-07-04T17:31:05.773328+0000","last_fullsized":"2024-07-04T17:31:05.773328+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:27:06.393591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.761742+0000","last_change":"2024-07-04T17:31:05.761742+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.761742+0000","last_undegraded":"2024-07-04T17:31:05.761742+0000","last_fullsized":"2024-07-04T17:31:05.761742+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.761222+0000","last_change":"2024-07-04T17:30:47.389819+0000","last_active":"2024-07-04T17:31:05.761222+0000","last_peered":"2024-07-04T17:31:05.761222+0000","last_clean":"2024-07-04T17:31:05.761222+0000","last_became_active":"2024-07-04T17:30:47.387439+0000","last_became_peered":"2024-07-04T17:30:47.387439+0000","last_unstale":"2024-07-04T17:31:05.761222+0000","last_undegraded":"2024-07-04T17:31:05.761222+0000","last_fullsized":"2024-07-04T17:31:05.761222+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:56:14.415266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.765997+0000","last_change":"2024-07-04T17:31:05.765997+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.765997+0000","last_undegraded":"2024-07-04T17:31:05.765997+0000","last_fullsized":"2024-07-04T17:31:05.765997+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.743967+0000","last_change":"2024-07-04T17:30:47.395945+0000","last_active":"2024-07-04T17:31:05.743967+0000","last_peered":"2024-07-04T17:31:05.743967+0000","last_clean":"2024-07-04T17:31:05.743967+0000","last_became_active":"2024-07-04T17:30:47.395631+0000","last_became_peered":"2024-07-04T17:30:47.395631+0000","last_unstale":"2024-07-04T17:31:05.743967+0000","last_undegraded":"2024-07-04T17:31:05.743967+0000","last_fullsized":"2024-07-04T17:31:05.743967+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:27:30.220822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.762901+0000","last_change":"2024-07-04T17:31:05.762901+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.762901+0000","last_undegraded":"2024-07-04T17:31:05.762901+0000","last_fullsized":"2024-07-04T17:31:05.762901+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.782838+0000","last_change":"2024-07-04T17:30:47.436357+0000","last_active":"2024-07-04T17:31:05.782838+0000","last_peered":"2024-07-04T17:31:05.782838+0000","last_clean":"2024-07-04T17:31:05.782838+0000","last_became_active":"2024-07-04T17:30:47.435542+0000","last_became_peered":"2024-07-04T17:30:47.435542+0000","last_unstale":"2024-07-04T17:31:05.782838+0000","last_undegraded":"2024-07-04T17:31:05.782838+0000","last_fullsized":"2024-07-04T17:31:05.782838+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:08:21.911035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.752656+0000","last_change":"2024-07-04T17:31:05.752656+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.752656+0000","last_undegraded":"2024-07-04T17:31:05.752656+0000","last_fullsized":"2024-07-04T17:31:05.752656+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.740529+0000","last_change":"2024-07-04T17:30:47.430335+0000","last_active":"2024-07-04T17:31:05.740529+0000","last_peered":"2024-07-04T17:31:05.740529+0000","last_clean":"2024-07-04T17:31:05.740529+0000","last_became_active":"2024-07-04T17:30:47.427068+0000","last_became_peered":"2024-07-04T17:30:47.427068+0000","last_unstale":"2024-07-04T17:31:05.740529+0000","last_undegraded":"2024-07-04T17:31:05.740529+0000","last_fullsized":"2024-07-04T17:31:05.740529+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:09:53.897531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.758667+0000","last_change":"2024-07-04T17:31:05.758667+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.758667+0000","last_undegraded":"2024-07-04T17:31:05.758667+0000","last_fullsized":"2024-07-04T17:31:05.758667+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.763397+0000","last_change":"2024-07-04T17:30:47.398367+0000","last_active":"2024-07-04T17:31:05.763397+0000","last_peered":"2024-07-04T17:31:05.763397+0000","last_clean":"2024-07-04T17:31:05.763397+0000","last_became_active":"2024-07-04T17:30:47.394431+0000","last_became_peered":"2024-07-04T17:30:47.394431+0000","last_unstale":"2024-07-04T17:31:05.763397+0000","last_undegraded":"2024-07-04T17:31:05.763397+0000","last_fullsized":"2024-07-04T17:31:05.763397+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:14:28.773355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.761188+0000","last_change":"2024-07-04T17:31:05.761188+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.761188+0000","last_undegraded":"2024-07-04T17:31:05.761188+0000","last_fullsized":"2024-07-04T17:31:05.761188+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.761674+0000","last_change":"2024-07-04T17:30:47.426130+0000","last_active":"2024-07-04T17:31:05.761674+0000","last_peered":"2024-07-04T17:31:05.761674+0000","last_clean":"2024-07-04T17:31:05.761674+0000","last_became_active":"2024-07-04T17:30:47.416510+0000","last_became_peered":"2024-07-04T17:30:47.416510+0000","last_unstale":"2024-07-04T17:31:05.761674+0000","last_undegraded":"2024-07-04T17:31:05.761674+0000","last_fullsized":"2024-07-04T17:31:05.761674+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:53:42.201549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.747425+0000","last_change":"2024-07-04T17:31:05.747425+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.747425+0000","last_undegraded":"2024-07-04T17:31:05.747425+0000","last_fullsized":"2024-07-04T17:31:05.747425+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.772506+0000","last_change":"2024-07-04T17:30:47.417678+0000","last_active":"2024-07-04T17:31:05.772506+0000","last_peered":"2024-07-04T17:31:05.772506+0000","last_clean":"2024-07-04T17:31:05.772506+0000","last_became_active":"2024-07-04T17:30:47.414976+0000","last_became_peered":"2024-07-04T17:30:47.414976+0000","last_unstale":"2024-07-04T17:31:05.772506+0000","last_undegraded":"2024-07-04T17:31:05.772506+0000","last_fullsized":"2024-07-04T17:31:05.772506+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:40:08.588002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.748835+0000","last_change":"2024-07-04T17:31:05.748835+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.748835+0000","last_undegraded":"2024-07-04T17:31:05.748835+0000","last_fullsized":"2024-07-04T17:31:05.748835+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.740058+0000","last_change":"2024-07-04T17:30:47.396959+0000","last_active":"2024-07-04T17:31:05.740058+0000","last_peered":"2024-07-04T17:31:05.740058+0000","last_clean":"2024-07-04T17:31:05.740058+0000","last_became_active":"2024-07-04T17:30:47.396561+0000","last_became_peered":"2024-07-04T17:30:47.396561+0000","last_unstale":"2024-07-04T17:31:05.740058+0000","last_undegraded":"2024-07-04T17:31:05.740058+0000","last_fullsized":"2024-07-04T17:31:05.740058+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:03:40.493555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.756452+0000","last_change":"2024-07-04T17:31:05.756452+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.756452+0000","last_undegraded":"2024-07-04T17:31:05.756452+0000","last_fullsized":"2024-07-04T17:31:05.756452+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.768205+0000","last_change":"2024-07-04T17:30:47.420235+0000","last_active":"2024-07-04T17:31:05.768205+0000","last_peered":"2024-07-04T17:31:05.768205+0000","last_clean":"2024-07-04T17:31:05.768205+0000","last_became_active":"2024-07-04T17:30:47.419803+0000","last_became_peered":"2024-07-04T17:30:47.419803+0000","last_unstale":"2024-07-04T17:31:05.768205+0000","last_undegraded":"2024-07-04T17:31:05.768205+0000","last_fullsized":"2024-07-04T17:31:05.768205+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:27:30.220822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.764855+0000","last_change":"2024-07-04T17:31:05.764855+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.764855+0000","last_undegraded":"2024-07-04T17:31:05.764855+0000","last_fullsized":"2024-07-04T17:31:05.764855+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.766372+0000","last_change":"2024-07-04T17:30:47.426146+0000","last_active":"2024-07-04T17:31:05.766372+0000","last_peered":"2024-07-04T17:31:05.766372+0000","last_clean":"2024-07-04T17:31:05.766372+0000","last_became_active":"2024-07-04T17:30:47.421447+0000","last_became_peered":"2024-07-04T17:30:47.421447+0000","last_unstale":"2024-07-04T17:31:05.766372+0000","last_undegraded":"2024-07-04T17:31:05.766372+0000","last_fullsized":"2024-07-04T17:31:05.766372+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:14:40.880998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.760741+0000","last_change":"2024-07-04T17:31:05.760741+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.760741+0000","last_undegraded":"2024-07-04T17:31:05.760741+0000","last_fullsized":"2024-07-04T17:31:05.760741+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.782979+0000","last_change":"2024-07-04T17:30:47.436646+0000","last_active":"2024-07-04T17:31:05.782979+0000","last_peered":"2024-07-04T17:31:05.782979+0000","last_clean":"2024-07-04T17:31:05.782979+0000","last_became_active":"2024-07-04T17:30:47.435999+0000","last_became_peered":"2024-07-04T17:30:47.435999+0000","last_unstale":"2024-07-04T17:31:05.782979+0000","last_undegraded":"2024-07-04T17:31:05.782979+0000","last_fullsized":"2024-07-04T17:31:05.782979+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:27:44.098370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.756035+0000","last_change":"2024-07-04T17:31:05.756035+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.756035+0000","last_undegraded":"2024-07-04T17:31:05.756035+0000","last_fullsized":"2024-07-04T17:31:05.756035+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.764795+0000","last_change":"2024-07-04T17:30:47.398164+0000","last_active":"2024-07-04T17:31:05.764795+0000","last_peered":"2024-07-04T17:31:05.764795+0000","last_clean":"2024-07-04T17:31:05.764795+0000","last_became_active":"2024-07-04T17:30:47.394563+0000","last_became_peered":"2024-07-04T17:30:47.394563+0000","last_unstale":"2024-07-04T17:31:05.764795+0000","last_undegraded":"2024-07-04T17:31:05.764795+0000","last_fullsized":"2024-07-04T17:31:05.764795+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:51:16.498378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.758203+0000","last_change":"2024-07-04T17:31:05.758203+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.758203+0000","last_undegraded":"2024-07-04T17:31:05.758203+0000","last_fullsized":"2024-07-04T17:31:05.758203+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.781294+0000","last_change":"2024-07-04T17:30:47.438690+0000","last_active":"2024-07-04T17:31:05.781294+0000","last_peered":"2024-07-04T17:31:05.781294+0000","last_clean":"2024-07-04T17:31:05.781294+0000","last_became_active":"2024-07-04T17:30:47.438362+0000","last_became_peered":"2024-07-04T17:30:47.438362+0000","last_unstale":"2024-07-04T17:31:05.781294+0000","last_undegraded":"2024-07-04T17:31:05.781294+0000","last_fullsized":"2024-07-04T17:31:05.781294+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:28:11.855854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.746955+0000","last_change":"2024-07-04T17:31:05.746955+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.746955+0000","last_undegraded":"2024-07-04T17:31:05.746955+0000","last_fullsized":"2024-07-04T17:31:05.746955+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.775093+0000","last_change":"2024-07-04T17:30:47.435331+0000","last_active":"2024-07-04T17:31:05.775093+0000","last_peered":"2024-07-04T17:31:05.775093+0000","last_clean":"2024-07-04T17:31:05.775093+0000","last_became_active":"2024-07-04T17:30:47.434100+0000","last_became_peered":"2024-07-04T17:30:47.434100+0000","last_unstale":"2024-07-04T17:31:05.775093+0000","last_undegraded":"2024-07-04T17:31:05.775093+0000","last_fullsized":"2024-07-04T17:31:05.775093+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:47:32.884542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.753046+0000","last_change":"2024-07-04T17:31:05.753046+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.753046+0000","last_undegraded":"2024-07-04T17:31:05.753046+0000","last_fullsized":"2024-07-04T17:31:05.753046+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.739766+0000","last_change":"2024-07-04T17:30:47.399163+0000","last_active":"2024-07-04T17:31:05.739766+0000","last_peered":"2024-07-04T17:31:05.739766+0000","last_clean":"2024-07-04T17:31:05.739766+0000","last_became_active":"2024-07-04T17:30:47.398747+0000","last_became_peered":"2024-07-04T17:30:47.398747+0000","last_unstale":"2024-07-04T17:31:05.739766+0000","last_undegraded":"2024-07-04T17:31:05.739766+0000","last_fullsized":"2024-07-04T17:31:05.739766+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:24:01.574518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.749467+0000","last_change":"2024-07-04T17:31:05.749467+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.749467+0000","last_undegraded":"2024-07-04T17:31:05.749467+0000","last_fullsized":"2024-07-04T17:31:05.749467+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.762599+0000","last_change":"2024-07-04T17:30:47.421142+0000","last_active":"2024-07-04T17:31:05.762599+0000","last_peered":"2024-07-04T17:31:05.762599+0000","last_clean":"2024-07-04T17:31:05.762599+0000","last_became_active":"2024-07-04T17:30:47.416769+0000","last_became_peered":"2024-07-04T17:30:47.416769+0000","last_unstale":"2024-07-04T17:31:05.762599+0000","last_undegraded":"2024-07-04T17:31:05.762599+0000","last_fullsized":"2024-07-04T17:31:05.762599+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:19:39.765910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.755813+0000","last_change":"2024-07-04T17:31:05.755813+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.755813+0000","last_undegraded":"2024-07-04T17:31:05.755813+0000","last_fullsized":"2024-07-04T17:31:05.755813+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.768344+0000","last_change":"2024-07-04T17:30:47.421958+0000","last_active":"2024-07-04T17:31:05.768344+0000","last_peered":"2024-07-04T17:31:05.768344+0000","last_clean":"2024-07-04T17:31:05.768344+0000","last_became_active":"2024-07-04T17:30:47.421313+0000","last_became_peered":"2024-07-04T17:30:47.421313+0000","last_unstale":"2024-07-04T17:31:05.768344+0000","last_undegraded":"2024-07-04T17:31:05.768344+0000","last_fullsized":"2024-07-04T17:31:05.768344+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:44:44.984203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.760534+0000","last_change":"2024-07-04T17:31:05.760534+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.760534+0000","last_undegraded":"2024-07-04T17:31:05.760534+0000","last_fullsized":"2024-07-04T17:31:05.760534+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.744057+0000","last_change":"2024-07-04T17:30:47.398351+0000","last_active":"2024-07-04T17:31:05.744057+0000","last_peered":"2024-07-04T17:31:05.744057+0000","last_clean":"2024-07-04T17:31:05.744057+0000","last_became_active":"2024-07-04T17:30:47.395987+0000","last_became_peered":"2024-07-04T17:30:47.395987+0000","last_unstale":"2024-07-04T17:31:05.744057+0000","last_undegraded":"2024-07-04T17:31:05.744057+0000","last_fullsized":"2024-07-04T17:31:05.744057+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:44:44.984203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.765611+0000","last_change":"2024-07-04T17:31:05.765611+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.765611+0000","last_undegraded":"2024-07-04T17:31:05.765611+0000","last_fullsized":"2024-07-04T17:31:05.765611+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.742173+0000","last_change":"2024-07-04T17:30:47.398645+0000","last_active":"2024-07-04T17:31:05.742173+0000","last_peered":"2024-07-04T17:31:05.742173+0000","last_clean":"2024-07-04T17:31:05.742173+0000","last_became_active":"2024-07-04T17:30:47.394765+0000","last_became_peered":"2024-07-04T17:30:47.394765+0000","last_unstale":"2024-07-04T17:31:05.742173+0000","last_undegraded":"2024-07-04T17:31:05.742173+0000","last_fullsized":"2024-07-04T17:31:05.742173+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:59:48.588528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.757969+0000","last_change":"2024-07-04T17:31:05.757969+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.757969+0000","last_undegraded":"2024-07-04T17:31:05.757969+0000","last_fullsized":"2024-07-04T17:31:05.757969+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.782531+0000","last_change":"2024-07-04T17:30:47.439010+0000","last_active":"2024-07-04T17:31:05.782531+0000","last_peered":"2024-07-04T17:31:05.782531+0000","last_clean":"2024-07-04T17:31:05.782531+0000","last_became_active":"2024-07-04T17:30:47.436372+0000","last_became_peered":"2024-07-04T17:30:47.436372+0000","last_unstale":"2024-07-04T17:31:05.782531+0000","last_undegraded":"2024-07-04T17:31:05.782531+0000","last_fullsized":"2024-07-04T17:31:05.782531+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:08:49.032762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.755848+0000","last_change":"2024-07-04T17:31:05.755848+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.755848+0000","last_undegraded":"2024-07-04T17:31:05.755848+0000","last_fullsized":"2024-07-04T17:31:05.755848+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.740624+0000","last_change":"2024-07-04T17:30:47.430081+0000","last_active":"2024-07-04T17:31:05.740624+0000","last_peered":"2024-07-04T17:31:05.740624+0000","last_clean":"2024-07-04T17:31:05.740624+0000","last_became_active":"2024-07-04T17:30:47.425964+0000","last_became_peered":"2024-07-04T17:30:47.425964+0000","last_unstale":"2024-07-04T17:31:05.740624+0000","last_undegraded":"2024-07-04T17:31:05.740624+0000","last_fullsized":"2024-07-04T17:31:05.740624+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:44:12.909185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.760447+0000","last_change":"2024-07-04T17:31:05.760447+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.760447+0000","last_undegraded":"2024-07-04T17:31:05.760447+0000","last_fullsized":"2024-07-04T17:31:05.760447+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.741742+0000","last_change":"2024-07-04T17:30:47.389158+0000","last_active":"2024-07-04T17:31:05.741742+0000","last_peered":"2024-07-04T17:31:05.741742+0000","last_clean":"2024-07-04T17:31:05.741742+0000","last_became_active":"2024-07-04T17:30:47.388127+0000","last_became_peered":"2024-07-04T17:30:47.388127+0000","last_unstale":"2024-07-04T17:31:05.741742+0000","last_undegraded":"2024-07-04T17:31:05.741742+0000","last_fullsized":"2024-07-04T17:31:05.741742+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:49:47.842800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.758922+0000","last_change":"2024-07-04T17:31:05.758922+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.758922+0000","last_undegraded":"2024-07-04T17:31:05.758922+0000","last_fullsized":"2024-07-04T17:31:05.758922+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.740131+0000","last_change":"2024-07-04T17:30:47.427109+0000","last_active":"2024-07-04T17:31:05.740131+0000","last_peered":"2024-07-04T17:31:05.740131+0000","last_clean":"2024-07-04T17:31:05.740131+0000","last_became_active":"2024-07-04T17:30:47.426697+0000","last_became_peered":"2024-07-04T17:30:47.426697+0000","last_unstale":"2024-07-04T17:31:05.740131+0000","last_undegraded":"2024-07-04T17:31:05.740131+0000","last_fullsized":"2024-07-04T17:31:05.740131+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:20:37.646569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.746733+0000","last_change":"2024-07-04T17:31:05.746733+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.746733+0000","last_undegraded":"2024-07-04T17:31:05.746733+0000","last_fullsized":"2024-07-04T17:31:05.746733+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.773518+0000","last_change":"2024-07-04T17:30:47.435621+0000","last_active":"2024-07-04T17:31:05.773518+0000","last_peered":"2024-07-04T17:31:05.773518+0000","last_clean":"2024-07-04T17:31:05.773518+0000","last_became_active":"2024-07-04T17:30:47.435230+0000","last_became_peered":"2024-07-04T17:30:47.435230+0000","last_unstale":"2024-07-04T17:31:05.773518+0000","last_undegraded":"2024-07-04T17:31:05.773518+0000","last_fullsized":"2024-07-04T17:31:05.773518+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:29:26.886812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.755906+0000","last_change":"2024-07-04T17:31:05.755906+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.755906+0000","last_undegraded":"2024-07-04T17:31:05.755906+0000","last_fullsized":"2024-07-04T17:31:05.755906+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.768456+0000","last_change":"2024-07-04T17:30:47.414726+0000","last_active":"2024-07-04T17:31:05.768456+0000","last_peered":"2024-07-04T17:31:05.768456+0000","last_clean":"2024-07-04T17:31:05.768456+0000","last_became_active":"2024-07-04T17:30:47.414412+0000","last_became_peered":"2024-07-04T17:30:47.414412+0000","last_unstale":"2024-07-04T17:31:05.768456+0000","last_undegraded":"2024-07-04T17:31:05.768456+0000","last_fullsized":"2024-07-04T17:31:05.768456+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:59:48.588528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.758114+0000","last_change":"2024-07-04T17:31:05.758114+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.758114+0000","last_undegraded":"2024-07-04T17:31:05.758114+0000","last_fullsized":"2024-07-04T17:31:05.758114+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.793814+0000","last_change":"2024-07-04T17:31:05.791225+0000","last_active":"2024-07-04T17:31:05.793814+0000","last_peered":"2024-07-04T17:31:05.793814+0000","last_clean":"2024-07-04T17:31:05.793814+0000","last_became_active":"2024-07-04T17:31:05.789499+0000","last_became_peered":"2024-07-04T17:31:05.789499+0000","last_unstale":"2024-07-04T17:31:05.793814+0000","last_undegraded":"2024-07-04T17:31:05.793814+0000","last_fullsized":"2024-07-04T17:31:05.793814+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:13:57.135398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.764977+0000","last_change":"2024-07-04T17:30:47.426760+0000","last_active":"2024-07-04T17:31:05.764977+0000","last_peered":"2024-07-04T17:31:05.764977+0000","last_clean":"2024-07-04T17:31:05.764977+0000","last_became_active":"2024-07-04T17:30:47.426331+0000","last_became_peered":"2024-07-04T17:30:47.426331+0000","last_unstale":"2024-07-04T17:31:05.764977+0000","last_undegraded":"2024-07-04T17:31:05.764977+0000","last_fullsized":"2024-07-04T17:31:05.764977+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:47:13.050952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"71'4","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.766413+0000","last_change":"2024-07-04T17:30:46.417984+0000","last_active":"2024-07-04T17:31:05.766413+0000","last_peered":"2024-07-04T17:31:05.766413+0000","last_clean":"2024-07-04T17:31:05.766413+0000","last_became_active":"2024-07-04T17:30:46.417091+0000","last_became_peered":"2024-07-04T17:30:46.417091+0000","last_unstale":"2024-07-04T17:31:05.766413+0000","last_undegraded":"2024-07-04T17:31:05.766413+0000","last_fullsized":"2024-07-04T17:31:05.766413+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:29:34.252784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.755921+0000","last_change":"2024-07-04T17:31:05.755921+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.755921+0000","last_undegraded":"2024-07-04T17:31:05.755921+0000","last_fullsized":"2024-07-04T17:31:05.755921+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.773518+0000","last_change":"2024-07-04T17:31:05.770412+0000","last_active":"2024-07-04T17:31:05.773518+0000","last_peered":"2024-07-04T17:31:05.773518+0000","last_clean":"2024-07-04T17:31:05.773518+0000","last_became_active":"2024-07-04T17:31:05.770076+0000","last_became_peered":"2024-07-04T17:31:05.770076+0000","last_unstale":"2024-07-04T17:31:05.773518+0000","last_undegraded":"2024-07-04T17:31:05.773518+0000","last_fullsized":"2024-07-04T17:31:05.773518+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:02:23.953242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.744183+0000","last_change":"2024-07-04T17:30:47.419884+0000","last_active":"2024-07-04T17:31:05.744183+0000","last_peered":"2024-07-04T17:31:05.744183+0000","last_clean":"2024-07-04T17:31:05.744183+0000","last_became_active":"2024-07-04T17:30:47.419588+0000","last_became_peered":"2024-07-04T17:30:47.419588+0000","last_unstale":"2024-07-04T17:31:05.744183+0000","last_undegraded":"2024-07-04T17:31:05.744183+0000","last_fullsized":"2024-07-04T17:31:05.744183+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:02:17.626429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"71'1","reported_seq":19,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.783122+0000","last_change":"2024-07-04T17:30:46.394693+0000","last_active":"2024-07-04T17:31:05.783122+0000","last_peered":"2024-07-04T17:31:05.783122+0000","last_clean":"2024-07-04T17:31:05.783122+0000","last_became_active":"2024-07-04T17:30:46.394315+0000","last_became_peered":"2024-07-04T17:30:46.394315+0000","last_unstale":"2024-07-04T17:31:05.783122+0000","last_undegraded":"2024-07-04T17:31:05.783122+0000","last_fullsized":"2024-07-04T17:31:05.783122+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:56:39.574079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.757930+0000","last_change":"2024-07-04T17:31:05.757930+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.757930+0000","last_undegraded":"2024-07-04T17:31:05.757930+0000","last_fullsized":"2024-07-04T17:31:05.757930+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.773850+0000","last_change":"2024-07-04T17:31:05.771337+0000","last_active":"2024-07-04T17:31:05.773850+0000","last_peered":"2024-07-04T17:31:05.773850+0000","last_clean":"2024-07-04T17:31:05.773850+0000","last_became_active":"2024-07-04T17:31:05.770842+0000","last_became_peered":"2024-07-04T17:31:05.770842+0000","last_unstale":"2024-07-04T17:31:05.773850+0000","last_undegraded":"2024-07-04T17:31:05.773850+0000","last_fullsized":"2024-07-04T17:31:05.773850+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:10:58.466495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.763509+0000","last_change":"2024-07-04T17:30:47.398047+0000","last_active":"2024-07-04T17:31:05.763509+0000","last_peered":"2024-07-04T17:31:05.763509+0000","last_clean":"2024-07-04T17:31:05.763509+0000","last_became_active":"2024-07-04T17:30:47.393052+0000","last_became_peered":"2024-07-04T17:30:47.393052+0000","last_unstale":"2024-07-04T17:31:05.763509+0000","last_undegraded":"2024-07-04T17:31:05.763509+0000","last_fullsized":"2024-07-04T17:31:05.763509+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:13:17.681466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"71'2","reported_seq":20,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.765046+0000","last_change":"2024-07-04T17:30:46.401872+0000","last_active":"2024-07-04T17:31:05.765046+0000","last_peered":"2024-07-04T17:31:05.765046+0000","last_clean":"2024-07-04T17:31:05.765046+0000","last_became_active":"2024-07-04T17:30:46.401510+0000","last_became_peered":"2024-07-04T17:30:46.401510+0000","last_unstale":"2024-07-04T17:31:05.765046+0000","last_undegraded":"2024-07-04T17:31:05.765046+0000","last_fullsized":"2024-07-04T17:31:05.765046+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:02:03.254039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.757638+0000","last_change":"2024-07-04T17:31:05.757638+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.757638+0000","last_undegraded":"2024-07-04T17:31:05.757638+0000","last_fullsized":"2024-07-04T17:31:05.757638+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.772782+0000","last_change":"2024-07-04T17:31:05.768631+0000","last_active":"2024-07-04T17:31:05.772782+0000","last_peered":"2024-07-04T17:31:05.772782+0000","last_clean":"2024-07-04T17:31:05.772782+0000","last_became_active":"2024-07-04T17:31:05.768099+0000","last_became_peered":"2024-07-04T17:31:05.768099+0000","last_unstale":"2024-07-04T17:31:05.772782+0000","last_undegraded":"2024-07-04T17:31:05.772782+0000","last_fullsized":"2024-07-04T17:31:05.772782+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:23:59.339429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.740704+0000","last_change":"2024-07-04T17:30:47.418834+0000","last_active":"2024-07-04T17:31:05.740704+0000","last_peered":"2024-07-04T17:31:05.740704+0000","last_clean":"2024-07-04T17:31:05.740704+0000","last_became_active":"2024-07-04T17:30:47.418449+0000","last_became_peered":"2024-07-04T17:30:47.418449+0000","last_unstale":"2024-07-04T17:31:05.740704+0000","last_undegraded":"2024-07-04T17:31:05.740704+0000","last_fullsized":"2024-07-04T17:31:05.740704+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:59:40.100483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"71'2","reported_seq":20,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.763710+0000","last_change":"2024-07-04T17:30:46.417993+0000","last_active":"2024-07-04T17:31:05.763710+0000","last_peered":"2024-07-04T17:31:05.763710+0000","last_clean":"2024-07-04T17:31:05.763710+0000","last_became_active":"2024-07-04T17:30:46.417603+0000","last_became_peered":"2024-07-04T17:30:46.417603+0000","last_unstale":"2024-07-04T17:31:05.763710+0000","last_undegraded":"2024-07-04T17:31:05.763710+0000","last_fullsized":"2024-07-04T17:31:05.763710+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:18:43.863213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.757068+0000","last_change":"2024-07-04T17:31:05.757068+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.757068+0000","last_undegraded":"2024-07-04T17:31:05.757068+0000","last_fullsized":"2024-07-04T17:31:05.757068+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.771506+0000","last_change":"2024-07-04T17:31:05.768330+0000","last_active":"2024-07-04T17:31:05.771506+0000","last_peered":"2024-07-04T17:31:05.771506+0000","last_clean":"2024-07-04T17:31:05.771506+0000","last_became_active":"2024-07-04T17:31:05.767975+0000","last_became_peered":"2024-07-04T17:31:05.767975+0000","last_unstale":"2024-07-04T17:31:05.771506+0000","last_undegraded":"2024-07-04T17:31:05.771506+0000","last_fullsized":"2024-07-04T17:31:05.771506+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:49:11.329011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.741987+0000","last_change":"2024-07-04T17:30:47.427026+0000","last_active":"2024-07-04T17:31:05.741987+0000","last_peered":"2024-07-04T17:31:05.741987+0000","last_clean":"2024-07-04T17:31:05.741987+0000","last_became_active":"2024-07-04T17:30:47.426173+0000","last_became_peered":"2024-07-04T17:30:47.426173+0000","last_unstale":"2024-07-04T17:31:05.741987+0000","last_undegraded":"2024-07-04T17:31:05.741987+0000","last_fullsized":"2024-07-04T17:31:05.741987+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:00:08.547137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"71'1","reported_seq":19,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.761925+0000","last_change":"2024-07-04T17:30:46.408705+0000","last_active":"2024-07-04T17:31:05.761925+0000","last_peered":"2024-07-04T17:31:05.761925+0000","last_clean":"2024-07-04T17:31:05.761925+0000","last_became_active":"2024-07-04T17:30:46.408366+0000","last_became_peered":"2024-07-04T17:30:46.408366+0000","last_unstale":"2024-07-04T17:31:05.761925+0000","last_undegraded":"2024-07-04T17:31:05.761925+0000","last_fullsized":"2024-07-04T17:31:05.761925+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:07:19.994029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.754794+0000","last_change":"2024-07-04T17:31:05.754794+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.754794+0000","last_undegraded":"2024-07-04T17:31:05.754794+0000","last_fullsized":"2024-07-04T17:31:05.754794+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.768399+0000","last_change":"2024-07-04T17:31:05.763717+0000","last_active":"2024-07-04T17:31:05.768399+0000","last_peered":"2024-07-04T17:31:05.768399+0000","last_clean":"2024-07-04T17:31:05.768399+0000","last_became_active":"2024-07-04T17:31:05.763408+0000","last_became_peered":"2024-07-04T17:31:05.763408+0000","last_unstale":"2024-07-04T17:31:05.768399+0000","last_undegraded":"2024-07-04T17:31:05.768399+0000","last_fullsized":"2024-07-04T17:31:05.768399+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:47:51.242357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.775207+0000","last_change":"2024-07-04T17:30:47.435295+0000","last_active":"2024-07-04T17:31:05.775207+0000","last_peered":"2024-07-04T17:31:05.775207+0000","last_clean":"2024-07-04T17:31:05.775207+0000","last_became_active":"2024-07-04T17:30:47.434416+0000","last_became_peered":"2024-07-04T17:30:47.434416+0000","last_unstale":"2024-07-04T17:31:05.775207+0000","last_undegraded":"2024-07-04T17:31:05.775207+0000","last_fullsized":"2024-07-04T17:31:05.775207+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:20:27.029139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"71'1","reported_seq":19,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.774341+0000","last_change":"2024-07-04T17:30:46.409671+0000","last_active":"2024-07-04T17:31:05.774341+0000","last_peered":"2024-07-04T17:31:05.774341+0000","last_clean":"2024-07-04T17:31:05.774341+0000","last_became_active":"2024-07-04T17:30:46.409031+0000","last_became_peered":"2024-07-04T17:30:46.409031+0000","last_unstale":"2024-07-04T17:31:05.774341+0000","last_undegraded":"2024-07-04T17:31:05.774341+0000","last_fullsized":"2024-07-04T17:31:05.774341+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:06:35.004818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.755179+0000","last_change":"2024-07-04T17:31:05.755179+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.755179+0000","last_undegraded":"2024-07-04T17:31:05.755179+0000","last_fullsized":"2024-07-04T17:31:05.755179+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.778920+0000","last_change":"2024-07-04T17:31:05.776134+0000","last_active":"2024-07-04T17:31:05.778920+0000","last_peered":"2024-07-04T17:31:05.778920+0000","last_clean":"2024-07-04T17:31:05.778920+0000","last_became_active":"2024-07-04T17:31:05.775776+0000","last_became_peered":"2024-07-04T17:31:05.775776+0000","last_unstale":"2024-07-04T17:31:05.778920+0000","last_undegraded":"2024-07-04T17:31:05.778920+0000","last_fullsized":"2024-07-04T17:31:05.778920+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:20:45.386954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.739696+0000","last_change":"2024-07-04T17:30:47.394313+0000","last_active":"2024-07-04T17:31:05.739696+0000","last_peered":"2024-07-04T17:31:05.739696+0000","last_clean":"2024-07-04T17:31:05.739696+0000","last_became_active":"2024-07-04T17:30:47.393967+0000","last_became_peered":"2024-07-04T17:30:47.393967+0000","last_unstale":"2024-07-04T17:31:05.739696+0000","last_undegraded":"2024-07-04T17:31:05.739696+0000","last_fullsized":"2024-07-04T17:31:05.739696+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:13:54.972134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.762836+0000","last_change":"2024-07-04T17:30:46.414098+0000","last_active":"2024-07-04T17:31:05.762836+0000","last_peered":"2024-07-04T17:31:05.762836+0000","last_clean":"2024-07-04T17:31:05.762836+0000","last_became_active":"2024-07-04T17:30:46.413464+0000","last_became_peered":"2024-07-04T17:30:46.413464+0000","last_unstale":"2024-07-04T17:31:05.762836+0000","last_undegraded":"2024-07-04T17:31:05.762836+0000","last_fullsized":"2024-07-04T17:31:05.762836+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:43:49.510457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.752754+0000","last_change":"2024-07-04T17:31:05.752754+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.752754+0000","last_undegraded":"2024-07-04T17:31:05.752754+0000","last_fullsized":"2024-07-04T17:31:05.752754+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.773948+0000","last_change":"2024-07-04T17:31:05.770550+0000","last_active":"2024-07-04T17:31:05.773948+0000","last_peered":"2024-07-04T17:31:05.773948+0000","last_clean":"2024-07-04T17:31:05.773948+0000","last_became_active":"2024-07-04T17:31:05.770248+0000","last_became_peered":"2024-07-04T17:31:05.770248+0000","last_unstale":"2024-07-04T17:31:05.773948+0000","last_undegraded":"2024-07-04T17:31:05.773948+0000","last_fullsized":"2024-07-04T17:31:05.773948+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:01:48.251728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.773620+0000","last_change":"2024-07-04T17:30:47.424416+0000","last_active":"2024-07-04T17:31:05.773620+0000","last_peered":"2024-07-04T17:31:05.773620+0000","last_clean":"2024-07-04T17:31:05.773620+0000","last_became_active":"2024-07-04T17:30:47.423168+0000","last_became_peered":"2024-07-04T17:30:47.423168+0000","last_unstale":"2024-07-04T17:31:05.773620+0000","last_undegraded":"2024-07-04T17:31:05.773620+0000","last_fullsized":"2024-07-04T17:31:05.773620+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:02:04.258323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.740257+0000","last_change":"2024-07-04T17:30:46.414564+0000","last_active":"2024-07-04T17:31:05.740257+0000","last_peered":"2024-07-04T17:31:05.740257+0000","last_clean":"2024-07-04T17:31:05.740257+0000","last_became_active":"2024-07-04T17:30:46.414116+0000","last_became_peered":"2024-07-04T17:30:46.414116+0000","last_unstale":"2024-07-04T17:31:05.740257+0000","last_undegraded":"2024-07-04T17:31:05.740257+0000","last_fullsized":"2024-07-04T17:31:05.740257+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:49:01.886056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.764480+0000","last_change":"2024-07-04T17:31:05.764480+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.764480+0000","last_undegraded":"2024-07-04T17:31:05.764480+0000","last_fullsized":"2024-07-04T17:31:05.764480+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.773776+0000","last_change":"2024-07-04T17:31:05.771852+0000","last_active":"2024-07-04T17:31:05.773776+0000","last_peered":"2024-07-04T17:31:05.773776+0000","last_clean":"2024-07-04T17:31:05.773776+0000","last_became_active":"2024-07-04T17:31:05.771389+0000","last_became_peered":"2024-07-04T17:31:05.771389+0000","last_unstale":"2024-07-04T17:31:05.773776+0000","last_undegraded":"2024-07-04T17:31:05.773776+0000","last_fullsized":"2024-07-04T17:31:05.773776+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:08:40.268850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.765111+0000","last_change":"2024-07-04T17:30:47.397516+0000","last_active":"2024-07-04T17:31:05.765111+0000","last_peered":"2024-07-04T17:31:05.765111+0000","last_clean":"2024-07-04T17:31:05.765111+0000","last_became_active":"2024-07-04T17:30:47.397119+0000","last_became_peered":"2024-07-04T17:30:47.397119+0000","last_unstale":"2024-07-04T17:31:05.765111+0000","last_undegraded":"2024-07-04T17:31:05.765111+0000","last_fullsized":"2024-07-04T17:31:05.765111+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:06:56.459812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.749267+0000","last_change":"2024-07-04T17:30:46.415510+0000","last_active":"2024-07-04T17:31:05.749267+0000","last_peered":"2024-07-04T17:31:05.749267+0000","last_clean":"2024-07-04T17:31:05.749267+0000","last_became_active":"2024-07-04T17:30:46.415113+0000","last_became_peered":"2024-07-04T17:30:46.415113+0000","last_unstale":"2024-07-04T17:31:05.749267+0000","last_undegraded":"2024-07-04T17:31:05.749267+0000","last_fullsized":"2024-07-04T17:31:05.749267+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:13:24.938223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.759980+0000","last_change":"2024-07-04T17:31:05.759980+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.759980+0000","last_undegraded":"2024-07-04T17:31:05.759980+0000","last_fullsized":"2024-07-04T17:31:05.759980+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.782967+0000","last_change":"2024-07-04T17:31:05.780150+0000","last_active":"2024-07-04T17:31:05.782967+0000","last_peered":"2024-07-04T17:31:05.782967+0000","last_clean":"2024-07-04T17:31:05.782967+0000","last_became_active":"2024-07-04T17:31:05.779680+0000","last_became_peered":"2024-07-04T17:31:05.779680+0000","last_unstale":"2024-07-04T17:31:05.782967+0000","last_undegraded":"2024-07-04T17:31:05.782967+0000","last_fullsized":"2024-07-04T17:31:05.782967+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:02:22.616138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.744353+0000","last_change":"2024-07-04T17:30:47.430636+0000","last_active":"2024-07-04T17:31:05.744353+0000","last_peered":"2024-07-04T17:31:05.744353+0000","last_clean":"2024-07-04T17:31:05.744353+0000","last_became_active":"2024-07-04T17:30:47.425674+0000","last_became_peered":"2024-07-04T17:30:47.425674+0000","last_unstale":"2024-07-04T17:31:05.744353+0000","last_undegraded":"2024-07-04T17:31:05.744353+0000","last_fullsized":"2024-07-04T17:31:05.744353+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:27:06.393591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.783250+0000","last_change":"2024-07-04T17:30:46.407826+0000","last_active":"2024-07-04T17:31:05.783250+0000","last_peered":"2024-07-04T17:31:05.783250+0000","last_clean":"2024-07-04T17:31:05.783250+0000","last_became_active":"2024-07-04T17:30:46.407115+0000","last_became_peered":"2024-07-04T17:30:46.407115+0000","last_unstale":"2024-07-04T17:31:05.783250+0000","last_undegraded":"2024-07-04T17:31:05.783250+0000","last_fullsized":"2024-07-04T17:31:05.783250+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:47:37.889965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.753884+0000","last_change":"2024-07-04T17:31:05.753884+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.753884+0000","last_undegraded":"2024-07-04T17:31:05.753884+0000","last_fullsized":"2024-07-04T17:31:05.753884+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.779788+0000","last_change":"2024-07-04T17:31:05.777612+0000","last_active":"2024-07-04T17:31:05.779788+0000","last_peered":"2024-07-04T17:31:05.779788+0000","last_clean":"2024-07-04T17:31:05.779788+0000","last_became_active":"2024-07-04T17:31:05.777042+0000","last_became_peered":"2024-07-04T17:31:05.777042+0000","last_unstale":"2024-07-04T17:31:05.779788+0000","last_undegraded":"2024-07-04T17:31:05.779788+0000","last_fullsized":"2024-07-04T17:31:05.779788+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:09:07.390577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.763585+0000","last_change":"2024-07-04T17:30:47.400589+0000","last_active":"2024-07-04T17:31:05.763585+0000","last_peered":"2024-07-04T17:31:05.763585+0000","last_clean":"2024-07-04T17:31:05.763585+0000","last_became_active":"2024-07-04T17:30:47.395213+0000","last_became_peered":"2024-07-04T17:30:47.395213+0000","last_unstale":"2024-07-04T17:31:05.763585+0000","last_undegraded":"2024-07-04T17:31:05.763585+0000","last_fullsized":"2024-07-04T17:31:05.763585+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:02:27.554854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"71'3","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.741870+0000","last_change":"2024-07-04T17:30:46.413645+0000","last_active":"2024-07-04T17:31:05.741870+0000","last_peered":"2024-07-04T17:31:05.741870+0000","last_clean":"2024-07-04T17:31:05.741870+0000","last_became_active":"2024-07-04T17:30:46.413314+0000","last_became_peered":"2024-07-04T17:30:46.413314+0000","last_unstale":"2024-07-04T17:31:05.741870+0000","last_undegraded":"2024-07-04T17:31:05.741870+0000","last_fullsized":"2024-07-04T17:31:05.741870+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:56:15.192711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.758556+0000","last_change":"2024-07-04T17:31:05.758556+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.758556+0000","last_undegraded":"2024-07-04T17:31:05.758556+0000","last_fullsized":"2024-07-04T17:31:05.758556+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.768577+0000","last_change":"2024-07-04T17:31:05.764774+0000","last_active":"2024-07-04T17:31:05.768577+0000","last_peered":"2024-07-04T17:31:05.768577+0000","last_clean":"2024-07-04T17:31:05.768577+0000","last_became_active":"2024-07-04T17:31:05.764469+0000","last_became_peered":"2024-07-04T17:31:05.764469+0000","last_unstale":"2024-07-04T17:31:05.768577+0000","last_undegraded":"2024-07-04T17:31:05.768577+0000","last_fullsized":"2024-07-04T17:31:05.768577+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:25:29.737754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.782573+0000","last_change":"2024-07-04T17:30:47.439341+0000","last_active":"2024-07-04T17:31:05.782573+0000","last_peered":"2024-07-04T17:31:05.782573+0000","last_clean":"2024-07-04T17:31:05.782573+0000","last_became_active":"2024-07-04T17:30:47.438643+0000","last_became_peered":"2024-07-04T17:30:47.438643+0000","last_unstale":"2024-07-04T17:31:05.782573+0000","last_undegraded":"2024-07-04T17:31:05.782573+0000","last_fullsized":"2024-07-04T17:31:05.782573+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:18:57.306251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.14","version":"72'8","reported_seq":26,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.763806+0000","last_change":"2024-07-04T17:30:46.418509+0000","last_active":"2024-07-04T17:31:05.763806+0000","last_peered":"2024-07-04T17:31:05.763806+0000","last_clean":"2024-07-04T17:31:05.763806+0000","last_became_active":"2024-07-04T17:30:46.418161+0000","last_became_peered":"2024-07-04T17:30:46.418161+0000","last_unstale":"2024-07-04T17:31:05.763806+0000","last_undegraded":"2024-07-04T17:31:05.763806+0000","last_fullsized":"2024-07-04T17:31:05.763806+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:26:41.897228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3309,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.755703+0000","last_change":"2024-07-04T17:31:05.755703+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.755703+0000","last_undegraded":"2024-07-04T17:31:05.755703+0000","last_fullsized":"2024-07-04T17:31:05.755703+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.778975+0000","last_change":"2024-07-04T17:31:05.776905+0000","last_active":"2024-07-04T17:31:05.778975+0000","last_peered":"2024-07-04T17:31:05.778975+0000","last_clean":"2024-07-04T17:31:05.778975+0000","last_became_active":"2024-07-04T17:31:05.776569+0000","last_became_peered":"2024-07-04T17:31:05.776569+0000","last_unstale":"2024-07-04T17:31:05.778975+0000","last_undegraded":"2024-07-04T17:31:05.778975+0000","last_fullsized":"2024-07-04T17:31:05.778975+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:28:02.456185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.763200+0000","last_change":"2024-07-04T17:30:47.388099+0000","last_active":"2024-07-04T17:31:05.763200+0000","last_peered":"2024-07-04T17:31:05.763200+0000","last_clean":"2024-07-04T17:31:05.763200+0000","last_became_active":"2024-07-04T17:30:47.387785+0000","last_became_peered":"2024-07-04T17:30:47.387785+0000","last_unstale":"2024-07-04T17:31:05.763200+0000","last_undegraded":"2024-07-04T17:31:05.763200+0000","last_fullsized":"2024-07-04T17:31:05.763200+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:53:19.566358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"71'2","reported_seq":20,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.740029+0000","last_change":"2024-07-04T17:30:46.404600+0000","last_active":"2024-07-04T17:31:05.740029+0000","last_peered":"2024-07-04T17:31:05.740029+0000","last_clean":"2024-07-04T17:31:05.740029+0000","last_became_active":"2024-07-04T17:30:46.404314+0000","last_became_peered":"2024-07-04T17:30:46.404314+0000","last_unstale":"2024-07-04T17:31:05.740029+0000","last_undegraded":"2024-07-04T17:31:05.740029+0000","last_fullsized":"2024-07-04T17:31:05.740029+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:25:07.446781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.751607+0000","last_change":"2024-07-04T17:31:05.751607+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.751607+0000","last_undegraded":"2024-07-04T17:31:05.751607+0000","last_fullsized":"2024-07-04T17:31:05.751607+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.766431+0000","last_change":"2024-07-04T17:31:05.762848+0000","last_active":"2024-07-04T17:31:05.766431+0000","last_peered":"2024-07-04T17:31:05.766431+0000","last_clean":"2024-07-04T17:31:05.766431+0000","last_became_active":"2024-07-04T17:31:05.762475+0000","last_became_peered":"2024-07-04T17:31:05.762475+0000","last_unstale":"2024-07-04T17:31:05.766431+0000","last_undegraded":"2024-07-04T17:31:05.766431+0000","last_fullsized":"2024-07-04T17:31:05.766431+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:09:26.970559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.739877+0000","last_change":"2024-07-04T17:30:47.387822+0000","last_active":"2024-07-04T17:31:05.739877+0000","last_peered":"2024-07-04T17:31:05.739877+0000","last_clean":"2024-07-04T17:31:05.739877+0000","last_became_active":"2024-07-04T17:30:47.387451+0000","last_became_peered":"2024-07-04T17:30:47.387451+0000","last_unstale":"2024-07-04T17:31:05.739877+0000","last_undegraded":"2024-07-04T17:31:05.739877+0000","last_fullsized":"2024-07-04T17:31:05.739877+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:46:22.229171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.763529+0000","last_change":"2024-07-04T17:30:46.410149+0000","last_active":"2024-07-04T17:31:05.763529+0000","last_peered":"2024-07-04T17:31:05.763529+0000","last_clean":"2024-07-04T17:31:05.763529+0000","last_became_active":"2024-07-04T17:30:46.409816+0000","last_became_peered":"2024-07-04T17:30:46.409816+0000","last_unstale":"2024-07-04T17:31:05.763529+0000","last_undegraded":"2024-07-04T17:31:05.763529+0000","last_fullsized":"2024-07-04T17:31:05.763529+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:51:54.551650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.753673+0000","last_change":"2024-07-04T17:31:05.753673+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.753673+0000","last_undegraded":"2024-07-04T17:31:05.753673+0000","last_fullsized":"2024-07-04T17:31:05.753673+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.792572+0000","last_change":"2024-07-04T17:31:05.786206+0000","last_active":"2024-07-04T17:31:05.792572+0000","last_peered":"2024-07-04T17:31:05.792572+0000","last_clean":"2024-07-04T17:31:05.792572+0000","last_became_active":"2024-07-04T17:31:05.785869+0000","last_became_peered":"2024-07-04T17:31:05.785869+0000","last_unstale":"2024-07-04T17:31:05.792572+0000","last_undegraded":"2024-07-04T17:31:05.792572+0000","last_fullsized":"2024-07-04T17:31:05.792572+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:13:02.624081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.739901+0000","last_change":"2024-07-04T17:30:47.431466+0000","last_active":"2024-07-04T17:31:05.739901+0000","last_peered":"2024-07-04T17:31:05.739901+0000","last_clean":"2024-07-04T17:31:05.739901+0000","last_became_active":"2024-07-04T17:30:47.430961+0000","last_became_peered":"2024-07-04T17:30:47.430961+0000","last_unstale":"2024-07-04T17:31:05.739901+0000","last_undegraded":"2024-07-04T17:31:05.739901+0000","last_fullsized":"2024-07-04T17:31:05.739901+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:50:08.536272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.773719+0000","last_change":"2024-07-04T17:30:46.396182+0000","last_active":"2024-07-04T17:31:05.773719+0000","last_peered":"2024-07-04T17:31:05.773719+0000","last_clean":"2024-07-04T17:31:05.773719+0000","last_became_active":"2024-07-04T17:30:46.393987+0000","last_became_peered":"2024-07-04T17:30:46.393987+0000","last_unstale":"2024-07-04T17:31:05.773719+0000","last_undegraded":"2024-07-04T17:31:05.773719+0000","last_fullsized":"2024-07-04T17:31:05.773719+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:44:44.996919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.754772+0000","last_change":"2024-07-04T17:31:05.754772+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.754772+0000","last_undegraded":"2024-07-04T17:31:05.754772+0000","last_fullsized":"2024-07-04T17:31:05.754772+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.793064+0000","last_change":"2024-07-04T17:31:05.786657+0000","last_active":"2024-07-04T17:31:05.793064+0000","last_peered":"2024-07-04T17:31:05.793064+0000","last_clean":"2024-07-04T17:31:05.793064+0000","last_became_active":"2024-07-04T17:31:05.785951+0000","last_became_peered":"2024-07-04T17:31:05.785951+0000","last_unstale":"2024-07-04T17:31:05.793064+0000","last_undegraded":"2024-07-04T17:31:05.793064+0000","last_fullsized":"2024-07-04T17:31:05.793064+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:43:22.976141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.740187+0000","last_change":"2024-07-04T17:30:47.395226+0000","last_active":"2024-07-04T17:31:05.740187+0000","last_peered":"2024-07-04T17:31:05.740187+0000","last_clean":"2024-07-04T17:31:05.740187+0000","last_became_active":"2024-07-04T17:30:47.394913+0000","last_became_peered":"2024-07-04T17:30:47.394913+0000","last_unstale":"2024-07-04T17:31:05.740187+0000","last_undegraded":"2024-07-04T17:31:05.740187+0000","last_fullsized":"2024-07-04T17:31:05.740187+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:31:15.378585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"71'8","reported_seq":26,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.761458+0000","last_change":"2024-07-04T17:30:46.398618+0000","last_active":"2024-07-04T17:31:05.761458+0000","last_peered":"2024-07-04T17:31:05.761458+0000","last_clean":"2024-07-04T17:31:05.761458+0000","last_became_active":"2024-07-04T17:30:46.398274+0000","last_became_peered":"2024-07-04T17:30:46.398274+0000","last_unstale":"2024-07-04T17:31:05.761458+0000","last_undegraded":"2024-07-04T17:31:05.761458+0000","last_fullsized":"2024-07-04T17:31:05.761458+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:13:50.002315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.757557+0000","last_change":"2024-07-04T17:31:05.757557+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.757557+0000","last_undegraded":"2024-07-04T17:31:05.757557+0000","last_fullsized":"2024-07-04T17:31:05.757557+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.793793+0000","last_change":"2024-07-04T17:31:05.789959+0000","last_active":"2024-07-04T17:31:05.793793+0000","last_peered":"2024-07-04T17:31:05.793793+0000","last_clean":"2024-07-04T17:31:05.793793+0000","last_became_active":"2024-07-04T17:31:05.789081+0000","last_became_peered":"2024-07-04T17:31:05.789081+0000","last_unstale":"2024-07-04T17:31:05.793793+0000","last_undegraded":"2024-07-04T17:31:05.793793+0000","last_fullsized":"2024-07-04T17:31:05.793793+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:59:54.903117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.765248+0000","last_change":"2024-07-04T17:30:47.397472+0000","last_active":"2024-07-04T17:31:05.765248+0000","last_peered":"2024-07-04T17:31:05.765248+0000","last_clean":"2024-07-04T17:31:05.765248+0000","last_became_active":"2024-07-04T17:30:47.396989+0000","last_became_peered":"2024-07-04T17:30:47.396989+0000","last_unstale":"2024-07-04T17:31:05.765248+0000","last_undegraded":"2024-07-04T17:31:05.765248+0000","last_fullsized":"2024-07-04T17:31:05.765248+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:40:22.762798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"71'1","reported_seq":19,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.766521+0000","last_change":"2024-07-04T17:30:46.412075+0000","last_active":"2024-07-04T17:31:05.766521+0000","last_peered":"2024-07-04T17:31:05.766521+0000","last_clean":"2024-07-04T17:31:05.766521+0000","last_became_active":"2024-07-04T17:30:46.411727+0000","last_became_peered":"2024-07-04T17:30:46.411727+0000","last_unstale":"2024-07-04T17:31:05.766521+0000","last_undegraded":"2024-07-04T17:31:05.766521+0000","last_fullsized":"2024-07-04T17:31:05.766521+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:12:43.261982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.754636+0000","last_change":"2024-07-04T17:31:05.754636+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.754636+0000","last_undegraded":"2024-07-04T17:31:05.754636+0000","last_fullsized":"2024-07-04T17:31:05.754636+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.793186+0000","last_change":"2024-07-04T17:31:05.787590+0000","last_active":"2024-07-04T17:31:05.793186+0000","last_peered":"2024-07-04T17:31:05.793186+0000","last_clean":"2024-07-04T17:31:05.793186+0000","last_became_active":"2024-07-04T17:31:05.787211+0000","last_became_peered":"2024-07-04T17:31:05.787211+0000","last_unstale":"2024-07-04T17:31:05.793186+0000","last_undegraded":"2024-07-04T17:31:05.793186+0000","last_fullsized":"2024-07-04T17:31:05.793186+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:22:41.821667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.783861+0000","last_change":"2024-07-04T17:30:47.436755+0000","last_active":"2024-07-04T17:31:05.783861+0000","last_peered":"2024-07-04T17:31:05.783861+0000","last_clean":"2024-07-04T17:31:05.783861+0000","last_became_active":"2024-07-04T17:30:47.436409+0000","last_became_peered":"2024-07-04T17:30:47.436409+0000","last_unstale":"2024-07-04T17:31:05.783861+0000","last_undegraded":"2024-07-04T17:31:05.783861+0000","last_fullsized":"2024-07-04T17:31:05.783861+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:13:51.006598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"71'2","reported_seq":20,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.765886+0000","last_change":"2024-07-04T17:30:46.409103+0000","last_active":"2024-07-04T17:31:05.765886+0000","last_peered":"2024-07-04T17:31:05.765886+0000","last_clean":"2024-07-04T17:31:05.765886+0000","last_became_active":"2024-07-04T17:30:46.406340+0000","last_became_peered":"2024-07-04T17:30:46.406340+0000","last_unstale":"2024-07-04T17:31:05.765886+0000","last_undegraded":"2024-07-04T17:31:05.765886+0000","last_fullsized":"2024-07-04T17:31:05.765886+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:43:03.614042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.758019+0000","last_change":"2024-07-04T17:31:05.758019+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.758019+0000","last_undegraded":"2024-07-04T17:31:05.758019+0000","last_fullsized":"2024-07-04T17:31:05.758019+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.780607+0000","last_change":"2024-07-04T17:31:05.779260+0000","last_active":"2024-07-04T17:31:05.780607+0000","last_peered":"2024-07-04T17:31:05.780607+0000","last_clean":"2024-07-04T17:31:05.780607+0000","last_became_active":"2024-07-04T17:31:05.778992+0000","last_became_peered":"2024-07-04T17:31:05.778992+0000","last_unstale":"2024-07-04T17:31:05.780607+0000","last_undegraded":"2024-07-04T17:31:05.780607+0000","last_fullsized":"2024-07-04T17:31:05.780607+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:38:17.157929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.764922+0000","last_change":"2024-07-04T17:30:47.394855+0000","last_active":"2024-07-04T17:31:05.764922+0000","last_peered":"2024-07-04T17:31:05.764922+0000","last_clean":"2024-07-04T17:31:05.764922+0000","last_became_active":"2024-07-04T17:30:47.393038+0000","last_became_peered":"2024-07-04T17:30:47.393038+0000","last_unstale":"2024-07-04T17:31:05.764922+0000","last_undegraded":"2024-07-04T17:31:05.764922+0000","last_fullsized":"2024-07-04T17:31:05.764922+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:25:35.950854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.763848+0000","last_change":"2024-07-04T17:30:46.409923+0000","last_active":"2024-07-04T17:31:05.763848+0000","last_peered":"2024-07-04T17:31:05.763848+0000","last_clean":"2024-07-04T17:31:05.763848+0000","last_became_active":"2024-07-04T17:30:46.409591+0000","last_became_peered":"2024-07-04T17:30:46.409591+0000","last_unstale":"2024-07-04T17:31:05.763848+0000","last_undegraded":"2024-07-04T17:31:05.763848+0000","last_fullsized":"2024-07-04T17:31:05.763848+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:12:05.853612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.749493+0000","last_change":"2024-07-04T17:31:05.749493+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.749493+0000","last_undegraded":"2024-07-04T17:31:05.749493+0000","last_fullsized":"2024-07-04T17:31:05.749493+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.780975+0000","last_change":"2024-07-04T17:31:05.776628+0000","last_active":"2024-07-04T17:31:05.780975+0000","last_peered":"2024-07-04T17:31:05.780975+0000","last_clean":"2024-07-04T17:31:05.780975+0000","last_became_active":"2024-07-04T17:31:05.776232+0000","last_became_peered":"2024-07-04T17:31:05.776232+0000","last_unstale":"2024-07-04T17:31:05.780975+0000","last_undegraded":"2024-07-04T17:31:05.780975+0000","last_fullsized":"2024-07-04T17:31:05.780975+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:53:39.261278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.774200+0000","last_change":"2024-07-04T17:30:47.419189+0000","last_active":"2024-07-04T17:31:05.774200+0000","last_peered":"2024-07-04T17:31:05.774200+0000","last_clean":"2024-07-04T17:31:05.774200+0000","last_became_active":"2024-07-04T17:30:47.417882+0000","last_became_peered":"2024-07-04T17:30:47.417882+0000","last_unstale":"2024-07-04T17:31:05.774200+0000","last_undegraded":"2024-07-04T17:31:05.774200+0000","last_fullsized":"2024-07-04T17:31:05.774200+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:28:21.769409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"71'2","reported_seq":20,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.762041+0000","last_change":"2024-07-04T17:30:46.412068+0000","last_active":"2024-07-04T17:31:05.762041+0000","last_peered":"2024-07-04T17:31:05.762041+0000","last_clean":"2024-07-04T17:31:05.762041+0000","last_became_active":"2024-07-04T17:30:46.411711+0000","last_became_peered":"2024-07-04T17:30:46.411711+0000","last_unstale":"2024-07-04T17:31:05.762041+0000","last_undegraded":"2024-07-04T17:31:05.762041+0000","last_fullsized":"2024-07-04T17:31:05.762041+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:42:52.442755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.751094+0000","last_change":"2024-07-04T17:31:05.751094+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.751094+0000","last_undegraded":"2024-07-04T17:31:05.751094+0000","last_fullsized":"2024-07-04T17:31:05.751094+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.771937+0000","last_change":"2024-07-04T17:31:05.769021+0000","last_active":"2024-07-04T17:31:05.771937+0000","last_peered":"2024-07-04T17:31:05.771937+0000","last_clean":"2024-07-04T17:31:05.771937+0000","last_became_active":"2024-07-04T17:31:05.768410+0000","last_became_peered":"2024-07-04T17:31:05.768410+0000","last_unstale":"2024-07-04T17:31:05.771937+0000","last_undegraded":"2024-07-04T17:31:05.771937+0000","last_fullsized":"2024-07-04T17:31:05.771937+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:29:17.500699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.775399+0000","last_change":"2024-07-04T17:30:47.436121+0000","last_active":"2024-07-04T17:31:05.775399+0000","last_peered":"2024-07-04T17:31:05.775399+0000","last_clean":"2024-07-04T17:31:05.775399+0000","last_became_active":"2024-07-04T17:30:47.435790+0000","last_became_peered":"2024-07-04T17:30:47.435790+0000","last_unstale":"2024-07-04T17:31:05.775399+0000","last_undegraded":"2024-07-04T17:31:05.775399+0000","last_fullsized":"2024-07-04T17:31:05.775399+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:25:40.409565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.774469+0000","last_change":"2024-07-04T17:30:46.408582+0000","last_active":"2024-07-04T17:31:05.774469+0000","last_peered":"2024-07-04T17:31:05.774469+0000","last_clean":"2024-07-04T17:31:05.774469+0000","last_became_active":"2024-07-04T17:30:46.408297+0000","last_became_peered":"2024-07-04T17:30:46.408297+0000","last_unstale":"2024-07-04T17:31:05.774469+0000","last_undegraded":"2024-07-04T17:31:05.774469+0000","last_fullsized":"2024-07-04T17:31:05.774469+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:48:21.667965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.749834+0000","last_change":"2024-07-04T17:31:05.749834+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.749834+0000","last_undegraded":"2024-07-04T17:31:05.749834+0000","last_fullsized":"2024-07-04T17:31:05.749834+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.793399+0000","last_change":"2024-07-04T17:31:05.787604+0000","last_active":"2024-07-04T17:31:05.793399+0000","last_peered":"2024-07-04T17:31:05.793399+0000","last_clean":"2024-07-04T17:31:05.793399+0000","last_became_active":"2024-07-04T17:31:05.787243+0000","last_became_peered":"2024-07-04T17:31:05.787243+0000","last_unstale":"2024-07-04T17:31:05.793399+0000","last_undegraded":"2024-07-04T17:31:05.793399+0000","last_fullsized":"2024-07-04T17:31:05.793399+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:16:12.478699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.768623+0000","last_change":"2024-07-04T17:30:47.435916+0000","last_active":"2024-07-04T17:31:05.768623+0000","last_peered":"2024-07-04T17:31:05.768623+0000","last_clean":"2024-07-04T17:31:05.768623+0000","last_became_active":"2024-07-04T17:30:47.435562+0000","last_became_peered":"2024-07-04T17:30:47.435562+0000","last_unstale":"2024-07-04T17:31:05.768623+0000","last_undegraded":"2024-07-04T17:31:05.768623+0000","last_fullsized":"2024-07-04T17:31:05.768623+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:41:29.873067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.773817+0000","last_change":"2024-07-04T17:30:46.394311+0000","last_active":"2024-07-04T17:31:05.773817+0000","last_peered":"2024-07-04T17:31:05.773817+0000","last_clean":"2024-07-04T17:31:05.773817+0000","last_became_active":"2024-07-04T17:30:46.393593+0000","last_became_peered":"2024-07-04T17:30:46.393593+0000","last_unstale":"2024-07-04T17:31:05.773817+0000","last_undegraded":"2024-07-04T17:31:05.773817+0000","last_fullsized":"2024-07-04T17:31:05.773817+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:48:17.986899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.750065+0000","last_change":"2024-07-04T17:31:05.750065+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.750065+0000","last_undegraded":"2024-07-04T17:31:05.750065+0000","last_fullsized":"2024-07-04T17:31:05.750065+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.794159+0000","last_change":"2024-07-04T17:31:05.792346+0000","last_active":"2024-07-04T17:31:05.794159+0000","last_peered":"2024-07-04T17:31:05.794159+0000","last_clean":"2024-07-04T17:31:05.794159+0000","last_became_active":"2024-07-04T17:31:05.790572+0000","last_became_peered":"2024-07-04T17:31:05.790572+0000","last_unstale":"2024-07-04T17:31:05.794159+0000","last_undegraded":"2024-07-04T17:31:05.794159+0000","last_fullsized":"2024-07-04T17:31:05.794159+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:29:46.749462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.762719+0000","last_change":"2024-07-04T17:30:47.420543+0000","last_active":"2024-07-04T17:31:05.762719+0000","last_peered":"2024-07-04T17:31:05.762719+0000","last_clean":"2024-07-04T17:31:05.762719+0000","last_became_active":"2024-07-04T17:30:47.416196+0000","last_became_peered":"2024-07-04T17:30:47.416196+0000","last_unstale":"2024-07-04T17:31:05.762719+0000","last_undegraded":"2024-07-04T17:31:05.762719+0000","last_fullsized":"2024-07-04T17:31:05.762719+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:50:20.456752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.768726+0000","last_change":"2024-07-04T17:30:46.408040+0000","last_active":"2024-07-04T17:31:05.768726+0000","last_peered":"2024-07-04T17:31:05.768726+0000","last_clean":"2024-07-04T17:31:05.768726+0000","last_became_active":"2024-07-04T17:30:46.407167+0000","last_became_peered":"2024-07-04T17:30:46.407167+0000","last_unstale":"2024-07-04T17:31:05.768726+0000","last_undegraded":"2024-07-04T17:31:05.768726+0000","last_fullsized":"2024-07-04T17:31:05.768726+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:46:18.222946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.753576+0000","last_change":"2024-07-04T17:31:05.753576+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.753576+0000","last_undegraded":"2024-07-04T17:31:05.753576+0000","last_fullsized":"2024-07-04T17:31:05.753576+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.793394+0000","last_change":"2024-07-04T17:31:05.789688+0000","last_active":"2024-07-04T17:31:05.793394+0000","last_peered":"2024-07-04T17:31:05.793394+0000","last_clean":"2024-07-04T17:31:05.793394+0000","last_became_active":"2024-07-04T17:31:05.788654+0000","last_became_peered":"2024-07-04T17:31:05.788654+0000","last_unstale":"2024-07-04T17:31:05.793394+0000","last_undegraded":"2024-07-04T17:31:05.793394+0000","last_fullsized":"2024-07-04T17:31:05.793394+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:25:19.949756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.765368+0000","last_change":"2024-07-04T17:30:47.397337+0000","last_active":"2024-07-04T17:31:05.765368+0000","last_peered":"2024-07-04T17:31:05.765368+0000","last_clean":"2024-07-04T17:31:05.765368+0000","last_became_active":"2024-07-04T17:30:47.396668+0000","last_became_peered":"2024-07-04T17:30:47.396668+0000","last_unstale":"2024-07-04T17:31:05.765368+0000","last_undegraded":"2024-07-04T17:31:05.765368+0000","last_fullsized":"2024-07-04T17:31:05.765368+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:44:37.013444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"71'2","reported_seq":20,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.749426+0000","last_change":"2024-07-04T17:30:46.414092+0000","last_active":"2024-07-04T17:31:05.749426+0000","last_peered":"2024-07-04T17:31:05.749426+0000","last_clean":"2024-07-04T17:31:05.749426+0000","last_became_active":"2024-07-04T17:30:46.413746+0000","last_became_peered":"2024-07-04T17:30:46.413746+0000","last_unstale":"2024-07-04T17:31:05.749426+0000","last_undegraded":"2024-07-04T17:31:05.749426+0000","last_fullsized":"2024-07-04T17:31:05.749426+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:56:21.477169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.756250+0000","last_change":"2024-07-04T17:31:05.756250+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.756250+0000","last_undegraded":"2024-07-04T17:31:05.756250+0000","last_fullsized":"2024-07-04T17:31:05.756250+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.792791+0000","last_change":"2024-07-04T17:31:05.788028+0000","last_active":"2024-07-04T17:31:05.792791+0000","last_peered":"2024-07-04T17:31:05.792791+0000","last_clean":"2024-07-04T17:31:05.792791+0000","last_became_active":"2024-07-04T17:31:05.785915+0000","last_became_peered":"2024-07-04T17:31:05.785915+0000","last_unstale":"2024-07-04T17:31:05.792791+0000","last_undegraded":"2024-07-04T17:31:05.792791+0000","last_fullsized":"2024-07-04T17:31:05.792791+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:19:03.225312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.744340+0000","last_change":"2024-07-04T17:30:47.388955+0000","last_active":"2024-07-04T17:31:05.744340+0000","last_peered":"2024-07-04T17:31:05.744340+0000","last_clean":"2024-07-04T17:31:05.744340+0000","last_became_active":"2024-07-04T17:30:47.387588+0000","last_became_peered":"2024-07-04T17:30:47.387588+0000","last_unstale":"2024-07-04T17:31:05.744340+0000","last_undegraded":"2024-07-04T17:31:05.744340+0000","last_fullsized":"2024-07-04T17:31:05.744340+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:58:22.418379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.6","version":"72'1","reported_seq":19,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.742402+0000","last_change":"2024-07-04T17:30:46.405443+0000","last_active":"2024-07-04T17:31:05.742402+0000","last_peered":"2024-07-04T17:31:05.742402+0000","last_clean":"2024-07-04T17:31:05.742402+0000","last_became_active":"2024-07-04T17:30:46.405066+0000","last_became_peered":"2024-07-04T17:30:46.405066+0000","last_unstale":"2024-07-04T17:31:05.742402+0000","last_undegraded":"2024-07-04T17:31:05.742402+0000","last_fullsized":"2024-07-04T17:31:05.742402+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:34:24.763771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.752523+0000","last_change":"2024-07-04T17:31:05.752523+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.752523+0000","last_undegraded":"2024-07-04T17:31:05.752523+0000","last_fullsized":"2024-07-04T17:31:05.752523+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.779763+0000","last_change":"2024-07-04T17:31:05.777491+0000","last_active":"2024-07-04T17:31:05.779763+0000","last_peered":"2024-07-04T17:31:05.779763+0000","last_clean":"2024-07-04T17:31:05.779763+0000","last_became_active":"2024-07-04T17:31:05.776618+0000","last_became_peered":"2024-07-04T17:31:05.776618+0000","last_unstale":"2024-07-04T17:31:05.779763+0000","last_undegraded":"2024-07-04T17:31:05.779763+0000","last_fullsized":"2024-07-04T17:31:05.779763+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:19:15.664067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.763677+0000","last_change":"2024-07-04T17:30:47.420498+0000","last_active":"2024-07-04T17:31:05.763677+0000","last_peered":"2024-07-04T17:31:05.763677+0000","last_clean":"2024-07-04T17:31:05.763677+0000","last_became_active":"2024-07-04T17:30:47.416342+0000","last_became_peered":"2024-07-04T17:30:47.416342+0000","last_unstale":"2024-07-04T17:31:05.763677+0000","last_undegraded":"2024-07-04T17:31:05.763677+0000","last_fullsized":"2024-07-04T17:31:05.763677+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:09:18.581049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.765105+0000","last_change":"2024-07-04T17:30:46.401530+0000","last_active":"2024-07-04T17:31:05.765105+0000","last_peered":"2024-07-04T17:31:05.765105+0000","last_clean":"2024-07-04T17:31:05.765105+0000","last_became_active":"2024-07-04T17:30:46.401058+0000","last_became_peered":"2024-07-04T17:30:46.401058+0000","last_unstale":"2024-07-04T17:31:05.765105+0000","last_undegraded":"2024-07-04T17:31:05.765105+0000","last_fullsized":"2024-07-04T17:31:05.765105+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:29:25.882528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.754370+0000","last_change":"2024-07-04T17:31:05.754370+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.754370+0000","last_undegraded":"2024-07-04T17:31:05.754370+0000","last_fullsized":"2024-07-04T17:31:05.754370+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.793244+0000","last_change":"2024-07-04T17:31:05.788840+0000","last_active":"2024-07-04T17:31:05.793244+0000","last_peered":"2024-07-04T17:31:05.793244+0000","last_clean":"2024-07-04T17:31:05.793244+0000","last_became_active":"2024-07-04T17:31:05.788410+0000","last_became_peered":"2024-07-04T17:31:05.788410+0000","last_unstale":"2024-07-04T17:31:05.793244+0000","last_undegraded":"2024-07-04T17:31:05.793244+0000","last_fullsized":"2024-07-04T17:31:05.793244+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:42:15.929482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.782796+0000","last_change":"2024-07-04T17:30:47.439494+0000","last_active":"2024-07-04T17:31:05.782796+0000","last_peered":"2024-07-04T17:31:05.782796+0000","last_clean":"2024-07-04T17:31:05.782796+0000","last_became_active":"2024-07-04T17:30:47.438655+0000","last_became_peered":"2024-07-04T17:30:47.438655+0000","last_unstale":"2024-07-04T17:31:05.782796+0000","last_undegraded":"2024-07-04T17:31:05.782796+0000","last_fullsized":"2024-07-04T17:31:05.782796+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:37:58.800114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.781446+0000","last_change":"2024-07-04T17:30:46.398565+0000","last_active":"2024-07-04T17:31:05.781446+0000","last_peered":"2024-07-04T17:31:05.781446+0000","last_clean":"2024-07-04T17:31:05.781446+0000","last_became_active":"2024-07-04T17:30:46.394788+0000","last_became_peered":"2024-07-04T17:30:46.394788+0000","last_unstale":"2024-07-04T17:31:05.781446+0000","last_undegraded":"2024-07-04T17:31:05.781446+0000","last_fullsized":"2024-07-04T17:31:05.781446+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:58:03.149737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.748726+0000","last_change":"2024-07-04T17:31:05.748726+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.748726+0000","last_undegraded":"2024-07-04T17:31:05.748726+0000","last_fullsized":"2024-07-04T17:31:05.748726+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.778947+0000","last_change":"2024-07-04T17:31:05.776324+0000","last_active":"2024-07-04T17:31:05.778947+0000","last_peered":"2024-07-04T17:31:05.778947+0000","last_clean":"2024-07-04T17:31:05.778947+0000","last_became_active":"2024-07-04T17:31:05.775829+0000","last_became_peered":"2024-07-04T17:31:05.775829+0000","last_unstale":"2024-07-04T17:31:05.778947+0000","last_undegraded":"2024-07-04T17:31:05.778947+0000","last_fullsized":"2024-07-04T17:31:05.778947+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:25:58.767381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.742576+0000","last_change":"2024-07-04T17:30:47.426754+0000","last_active":"2024-07-04T17:31:05.742576+0000","last_peered":"2024-07-04T17:31:05.742576+0000","last_clean":"2024-07-04T17:31:05.742576+0000","last_became_active":"2024-07-04T17:30:47.425712+0000","last_became_peered":"2024-07-04T17:30:47.425712+0000","last_unstale":"2024-07-04T17:31:05.742576+0000","last_undegraded":"2024-07-04T17:31:05.742576+0000","last_fullsized":"2024-07-04T17:31:05.742576+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:07:15.865385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.762184+0000","last_change":"2024-07-04T17:30:46.401607+0000","last_active":"2024-07-04T17:31:05.762184+0000","last_peered":"2024-07-04T17:31:05.762184+0000","last_clean":"2024-07-04T17:31:05.762184+0000","last_became_active":"2024-07-04T17:30:46.401314+0000","last_became_peered":"2024-07-04T17:30:46.401314+0000","last_unstale":"2024-07-04T17:31:05.762184+0000","last_undegraded":"2024-07-04T17:31:05.762184+0000","last_fullsized":"2024-07-04T17:31:05.762184+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:21:59.991349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.752208+0000","last_change":"2024-07-04T17:31:05.752208+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.752208+0000","last_undegraded":"2024-07-04T17:31:05.752208+0000","last_fullsized":"2024-07-04T17:31:05.752208+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.793396+0000","last_change":"2024-07-04T17:31:05.788550+0000","last_active":"2024-07-04T17:31:05.793396+0000","last_peered":"2024-07-04T17:31:05.793396+0000","last_clean":"2024-07-04T17:31:05.793396+0000","last_became_active":"2024-07-04T17:31:05.788144+0000","last_became_peered":"2024-07-04T17:31:05.788144+0000","last_unstale":"2024-07-04T17:31:05.793396+0000","last_undegraded":"2024-07-04T17:31:05.793396+0000","last_fullsized":"2024-07-04T17:31:05.793396+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:27:01.259327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.761797+0000","last_change":"2024-07-04T17:30:47.424977+0000","last_active":"2024-07-04T17:31:05.761797+0000","last_peered":"2024-07-04T17:31:05.761797+0000","last_clean":"2024-07-04T17:31:05.761797+0000","last_became_active":"2024-07-04T17:30:47.416042+0000","last_became_peered":"2024-07-04T17:30:47.416042+0000","last_unstale":"2024-07-04T17:31:05.761797+0000","last_undegraded":"2024-07-04T17:31:05.761797+0000","last_fullsized":"2024-07-04T17:31:05.761797+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:41:40.720108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.774580+0000","last_change":"2024-07-04T17:30:46.408561+0000","last_active":"2024-07-04T17:31:05.774580+0000","last_peered":"2024-07-04T17:31:05.774580+0000","last_clean":"2024-07-04T17:31:05.774580+0000","last_became_active":"2024-07-04T17:30:46.408199+0000","last_became_peered":"2024-07-04T17:30:46.408199+0000","last_unstale":"2024-07-04T17:31:05.774580+0000","last_undegraded":"2024-07-04T17:31:05.774580+0000","last_fullsized":"2024-07-04T17:31:05.774580+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:15:50.228374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.753568+0000","last_change":"2024-07-04T17:31:05.753568+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.753568+0000","last_undegraded":"2024-07-04T17:31:05.753568+0000","last_fullsized":"2024-07-04T17:31:05.753568+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.793189+0000","last_change":"2024-07-04T17:31:05.787619+0000","last_active":"2024-07-04T17:31:05.793189+0000","last_peered":"2024-07-04T17:31:05.793189+0000","last_clean":"2024-07-04T17:31:05.793189+0000","last_became_active":"2024-07-04T17:31:05.787257+0000","last_became_peered":"2024-07-04T17:31:05.787257+0000","last_unstale":"2024-07-04T17:31:05.793189+0000","last_undegraded":"2024-07-04T17:31:05.793189+0000","last_fullsized":"2024-07-04T17:31:05.793189+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:59:51.883448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.740087+0000","last_change":"2024-07-04T17:30:47.419127+0000","last_active":"2024-07-04T17:31:05.740087+0000","last_peered":"2024-07-04T17:31:05.740087+0000","last_clean":"2024-07-04T17:31:05.740087+0000","last_became_active":"2024-07-04T17:30:47.418557+0000","last_became_peered":"2024-07-04T17:30:47.418557+0000","last_unstale":"2024-07-04T17:31:05.740087+0000","last_undegraded":"2024-07-04T17:31:05.740087+0000","last_fullsized":"2024-07-04T17:31:05.740087+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:16:53.257166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.743016+0000","last_change":"2024-07-04T17:30:46.404540+0000","last_active":"2024-07-04T17:31:05.743016+0000","last_peered":"2024-07-04T17:31:05.743016+0000","last_clean":"2024-07-04T17:31:05.743016+0000","last_became_active":"2024-07-04T17:30:46.402800+0000","last_became_peered":"2024-07-04T17:30:46.402800+0000","last_unstale":"2024-07-04T17:31:05.743016+0000","last_undegraded":"2024-07-04T17:31:05.743016+0000","last_fullsized":"2024-07-04T17:31:05.743016+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:57:35.604132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.751503+0000","last_change":"2024-07-04T17:31:05.751503+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.751503+0000","last_undegraded":"2024-07-04T17:31:05.751503+0000","last_fullsized":"2024-07-04T17:31:05.751503+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.793293+0000","last_change":"2024-07-04T17:31:05.789134+0000","last_active":"2024-07-04T17:31:05.793293+0000","last_peered":"2024-07-04T17:31:05.793293+0000","last_clean":"2024-07-04T17:31:05.793293+0000","last_became_active":"2024-07-04T17:31:05.786364+0000","last_became_peered":"2024-07-04T17:31:05.786364+0000","last_unstale":"2024-07-04T17:31:05.793293+0000","last_undegraded":"2024-07-04T17:31:05.793293+0000","last_fullsized":"2024-07-04T17:31:05.793293+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:12:25.215711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.740347+0000","last_change":"2024-07-04T17:30:47.427679+0000","last_active":"2024-07-04T17:31:05.740347+0000","last_peered":"2024-07-04T17:31:05.740347+0000","last_clean":"2024-07-04T17:31:05.740347+0000","last_became_active":"2024-07-04T17:30:47.427451+0000","last_became_peered":"2024-07-04T17:30:47.427451+0000","last_unstale":"2024-07-04T17:31:05.740347+0000","last_undegraded":"2024-07-04T17:31:05.740347+0000","last_fullsized":"2024-07-04T17:31:05.740347+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:29:16.626050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.761593+0000","last_change":"2024-07-04T17:30:46.402877+0000","last_active":"2024-07-04T17:31:05.761593+0000","last_peered":"2024-07-04T17:31:05.761593+0000","last_clean":"2024-07-04T17:31:05.761593+0000","last_became_active":"2024-07-04T17:30:46.402551+0000","last_became_peered":"2024-07-04T17:30:46.402551+0000","last_unstale":"2024-07-04T17:31:05.761593+0000","last_undegraded":"2024-07-04T17:31:05.761593+0000","last_fullsized":"2024-07-04T17:31:05.761593+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:37:57.795830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":74,"state":"creating+peering","last_fresh":"2024-07-04T17:31:05.755924+0000","last_change":"2024-07-04T17:31:05.755924+0000","last_active":"2024-07-04T17:31:05.720604+0000","last_peered":"2024-07-04T17:31:05.720604+0000","last_clean":"2024-07-04T17:31:05.720604+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-07-04T17:31:05.755924+0000","last_undegraded":"2024-07-04T17:31:05.755924+0000","last_fullsized":"2024-07-04T17:31:05.755924+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.779809+0000","last_change":"2024-07-04T17:31:05.777764+0000","last_active":"2024-07-04T17:31:05.779809+0000","last_peered":"2024-07-04T17:31:05.779809+0000","last_clean":"2024-07-04T17:31:05.779809+0000","last_became_active":"2024-07-04T17:31:05.777059+0000","last_became_peered":"2024-07-04T17:31:05.777059+0000","last_unstale":"2024-07-04T17:31:05.779809+0000","last_undegraded":"2024-07-04T17:31:05.779809+0000","last_fullsized":"2024-07-04T17:31:05.779809+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:14:09.364414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.743684+0000","last_change":"2024-07-04T17:30:47.390337+0000","last_active":"2024-07-04T17:31:05.743684+0000","last_peered":"2024-07-04T17:31:05.743684+0000","last_clean":"2024-07-04T17:31:05.743684+0000","last_became_active":"2024-07-04T17:30:47.389504+0000","last_became_peered":"2024-07-04T17:30:47.389504+0000","last_unstale":"2024-07-04T17:31:05.743684+0000","last_undegraded":"2024-07-04T17:31:05.743684+0000","last_fullsized":"2024-07-04T17:31:05.743684+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:08:26.050071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"71'1","reported_seq":19,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.742002+0000","last_change":"2024-07-04T17:30:46.398516+0000","last_active":"2024-07-04T17:31:05.742002+0000","last_peered":"2024-07-04T17:31:05.742002+0000","last_clean":"2024-07-04T17:31:05.742002+0000","last_became_active":"2024-07-04T17:30:46.398169+0000","last_became_peered":"2024-07-04T17:30:46.398169+0000","last_unstale":"2024-07-04T17:31:05.742002+0000","last_undegraded":"2024-07-04T17:31:05.742002+0000","last_fullsized":"2024-07-04T17:31:05.742002+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:40:51.947038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"58'80","reported_seq":300,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-07-04T17:31:05.742501+0000","last_change":"2024-07-04T17:26:04.953559+0000","last_active":"2024-07-04T17:31:05.742501+0000","last_peered":"2024-07-04T17:31:05.742501+0000","last_clean":"2024-07-04T17:31:05.742501+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:31:05.742501+0000","last_undegraded":"2024-07-04T17:31:05.742501+0000","last_fullsized":"2024-07-04T17:31:05.742501+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_clean_scrub_stamp":"2024-07-04T17:26:03.933432+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.064181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":296,"num_read_kb":750,"num_write":127,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":4107,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":3},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":296,"num_read_kb":750,"num_write":127,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80,"ondisk_log_size":80,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738430,"num_pgs":73,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31932,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339908,"statfs":{"total":96636764160,"available":96604065792,"internally_reserved":0,"allocated":770048,"data_stored":295249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738431,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36660,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335180,"statfs":{"total":96636764160,"available":96599224320,"internally_reserved":0,"allocated":1347584,"data_stored":881738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738429,"num_pgs":109,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36672,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335168,"statfs":{"total":96636764160,"available":96599212032,"internally_reserved":0,"allocated":1359872,"data_stored":885491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:31:11.491 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:31:11.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.812+0000 7fbac6c00640 1 -- 172.21.15.42:0/2093524743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbac00b17d0 msgr2=0x7fbac00b1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:11.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.812+0000 7fbac6c00640 1 --2- 172.21.15.42:0/2093524743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbac00b17d0 0x7fbac00b1be0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fbaac0097b0 tx=0x7fbaac02a4a0 comp rx=0 tx=0).stop 2024-07-04T17:31:11.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.812+0000 7fbac6c00640 1 -- 172.21.15.42:0/2093524743 shutdown_connections 2024-07-04T17:31:11.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.812+0000 7fbac6c00640 1 --2- 172.21.15.42:0/2093524743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbac00b17d0 0x7fbac00b1be0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:11.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.812+0000 7fbac6c00640 1 -- 172.21.15.42:0/2093524743 >> 172.21.15.42:0/2093524743 conn(0x7fbac00bebc0 msgr2=0x7fbac00befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:11.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.812+0000 7fbac6c00640 1 -- 172.21.15.42:0/2093524743 shutdown_connections 2024-07-04T17:31:11.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.812+0000 7fbac6c00640 1 -- 172.21.15.42:0/2093524743 wait complete. 2024-07-04T17:31:11.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.814+0000 7fbac6c00640 1 Processor -- start 2024-07-04T17:31:11.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.814+0000 7fbac6c00640 1 -- start start 2024-07-04T17:31:11.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.815+0000 7fbac6c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbac006c200 0x7fbac0144400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:11.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.815+0000 7fbac6c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbac014ae80 con 0x7fbac006c200 2024-07-04T17:31:11.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.816+0000 7fbac5800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbac006c200 0x7fbac0144400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:11.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.816+0000 7fbac5800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbac006c200 0x7fbac0144400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47184/0 (socket says 172.21.15.42:47184) 2024-07-04T17:31:11.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.816+0000 7fbac5800640 1 -- 172.21.15.42:0/1483853588 learned_addr learned my addr 172.21.15.42:0/1483853588 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:11.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.816+0000 7fbac5800640 1 -- 172.21.15.42:0/1483853588 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac01d6fd0 con 0x7fbac006c200 2024-07-04T17:31:11.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.816+0000 7fbac5800640 1 --2- 172.21.15.42:0/1483853588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbac006c200 0x7fbac0144400 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fbab400c3b0 tx=0x7fbab400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:11.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.817+0000 7fbabd600640 1 -- 172.21.15.42:0/1483853588 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbab4011780 con 0x7fbac006c200 2024-07-04T17:31:11.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.817+0000 7fbabd600640 1 -- 172.21.15.42:0/1483853588 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbab4017070 con 0x7fbac006c200 2024-07-04T17:31:11.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.817+0000 7fbabd600640 1 -- 172.21.15.42:0/1483853588 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbab4011c00 con 0x7fbac006c200 2024-07-04T17:31:11.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.817+0000 7fbac6c00640 1 -- 172.21.15.42:0/1483853588 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbac01d60f0 con 0x7fbac006c200 2024-07-04T17:31:11.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.817+0000 7fbac6c00640 1 -- 172.21.15.42:0/1483853588 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbac01d76b0 con 0x7fbac006c200 2024-07-04T17:31:11.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.818+0000 7fbac6c00640 1 -- 172.21.15.42:0/1483853588 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbac00b44b0 con 0x7fbac006c200 2024-07-04T17:31:11.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.818+0000 7fbabd600640 1 -- 172.21.15.42:0/1483853588 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7fbab401c460 con 0x7fbac006c200 2024-07-04T17:31:11.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.819+0000 7fbabd600640 1 --2- 172.21.15.42:0/1483853588 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fbaa003d4f0 0x7fbaa003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:11.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.819+0000 7fbabd600640 1 -- 172.21.15.42:0/1483853588 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 4303+0+0 (secure 0 0 0) 0x7fbab40518c0 con 0x7fbac006c200 2024-07-04T17:31:11.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.819+0000 7fbac4e00640 1 --2- 172.21.15.42:0/1483853588 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fbaa003d4f0 0x7fbaa003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:11.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.820+0000 7fbac4e00640 1 --2- 172.21.15.42:0/1483853588 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fbaa003d4f0 0x7fbaa003fa00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbaac034000 tx=0x7fbaac004730 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:11.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:11.824+0000 7fbabd600640 1 -- 172.21.15.42:0/1483853588 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fbac00b44b0 con 0x7fbac006c200 2024-07-04T17:31:12.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.021+0000 7fbac6c00640 1 -- 172.21.15.42:0/1483853588 --> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbac014b4c0 con 0x7fbaa003d4f0 2024-07-04T17:31:12.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.032+0000 7fbabd600640 1 -- 172.21.15.42:0/1483853588 <== mgr.4906 v2:172.21.15.42:6824/3473020706 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+482453 (secure 0 0 0) 0x7fbac014b4c0 con 0x7fbaa003d4f0 2024-07-04T17:31:12.036 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:31:12.043 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:31:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.043+0000 7fbac6c00640 1 -- 172.21.15.42:0/1483853588 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fbaa003d4f0 msgr2=0x7fbaa003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.043+0000 7fbac6c00640 1 --2- 172.21.15.42:0/1483853588 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fbaa003d4f0 0x7fbaa003fa00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbaac034000 tx=0x7fbaac004730 comp rx=0 tx=0).stop 2024-07-04T17:31:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.043+0000 7fbac6c00640 1 -- 172.21.15.42:0/1483853588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbac006c200 msgr2=0x7fbac0144400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.043+0000 7fbac6c00640 1 --2- 172.21.15.42:0/1483853588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbac006c200 0x7fbac0144400 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fbab400c3b0 tx=0x7fbab400c870 comp rx=0 tx=0).stop 2024-07-04T17:31:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.043+0000 7fbac6c00640 1 -- 172.21.15.42:0/1483853588 shutdown_connections 2024-07-04T17:31:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.043+0000 7fbac6c00640 1 --2- 172.21.15.42:0/1483853588 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fbaa003d4f0 0x7fbaa003fa00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.043+0000 7fbac6c00640 1 --2- 172.21.15.42:0/1483853588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbac006c200 0x7fbac0144400 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.043+0000 7fbac6c00640 1 -- 172.21.15.42:0/1483853588 >> 172.21.15.42:0/1483853588 conn(0x7fbac00bebc0 msgr2=0x7fbac00b95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.043+0000 7fbac6c00640 1 -- 172.21.15.42:0/1483853588 shutdown_connections 2024-07-04T17:31:12.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.043+0000 7fbac6c00640 1 -- 172.21.15.42:0/1483853588 wait complete. 2024-07-04T17:31:12.071 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":45,"stamp":"2024-07-04T17:31:11.634666+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596885,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":296,"num_read_kb":750,"num_write":219,"num_write_kb":1137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":372,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":105400,"kb_used_data":3532,"kb_used_omap":4,"kb_used_meta":101819,"kb_avail":283010120,"statfs":{"total":289910292480,"available":289802362880,"internally_reserved":0,"allocated":3616768,"data_stored":2097994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":104262991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001893"},"pg_stats":[{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.762445+0000","last_change":"2024-07-04T17:31:06.793604+0000","last_active":"2024-07-04T17:31:07.762445+0000","last_peered":"2024-07-04T17:31:07.762445+0000","last_clean":"2024-07-04T17:31:07.762445+0000","last_became_active":"2024-07-04T17:31:06.793237+0000","last_became_peered":"2024-07-04T17:31:06.793237+0000","last_unstale":"2024-07-04T17:31:07.762445+0000","last_undegraded":"2024-07-04T17:31:07.762445+0000","last_fullsized":"2024-07-04T17:31:07.762445+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:54:01.573215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3f","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.766392+0000","last_change":"2024-07-04T17:30:47.430443+0000","last_active":"2024-07-04T17:31:07.766392+0000","last_peered":"2024-07-04T17:31:07.766392+0000","last_clean":"2024-07-04T17:31:07.766392+0000","last_became_active":"2024-07-04T17:30:47.425325+0000","last_became_peered":"2024-07-04T17:30:47.425325+0000","last_unstale":"2024-07-04T17:31:07.766392+0000","last_undegraded":"2024-07-04T17:31:07.766392+0000","last_fullsized":"2024-07-04T17:31:07.766392+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:44:15.710215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.789561+0000","last_change":"2024-07-04T17:31:06.794627+0000","last_active":"2024-07-04T17:31:07.789561+0000","last_peered":"2024-07-04T17:31:07.789561+0000","last_clean":"2024-07-04T17:31:07.789561+0000","last_became_active":"2024-07-04T17:31:06.788743+0000","last_became_peered":"2024-07-04T17:31:06.788743+0000","last_unstale":"2024-07-04T17:31:07.789561+0000","last_undegraded":"2024-07-04T17:31:07.789561+0000","last_fullsized":"2024-07-04T17:31:07.789561+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:02:24.967093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.766933+0000","last_change":"2024-07-04T17:30:47.429677+0000","last_active":"2024-07-04T17:31:07.766933+0000","last_peered":"2024-07-04T17:31:07.766933+0000","last_clean":"2024-07-04T17:31:07.766933+0000","last_became_active":"2024-07-04T17:30:47.428028+0000","last_became_peered":"2024-07-04T17:30:47.428028+0000","last_unstale":"2024-07-04T17:31:07.766933+0000","last_undegraded":"2024-07-04T17:31:07.766933+0000","last_fullsized":"2024-07-04T17:31:07.766933+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:41:29.873067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.762666+0000","last_change":"2024-07-04T17:31:06.805220+0000","last_active":"2024-07-04T17:31:07.762666+0000","last_peered":"2024-07-04T17:31:07.762666+0000","last_clean":"2024-07-04T17:31:07.762666+0000","last_became_active":"2024-07-04T17:31:06.804830+0000","last_became_peered":"2024-07-04T17:31:06.804830+0000","last_unstale":"2024-07-04T17:31:07.762666+0000","last_undegraded":"2024-07-04T17:31:07.762666+0000","last_fullsized":"2024-07-04T17:31:07.762666+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:25:55.322520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.794742+0000","last_change":"2024-07-04T17:30:47.423800+0000","last_active":"2024-07-04T17:31:07.794742+0000","last_peered":"2024-07-04T17:31:07.794742+0000","last_clean":"2024-07-04T17:31:07.794742+0000","last_became_active":"2024-07-04T17:30:47.423361+0000","last_became_peered":"2024-07-04T17:30:47.423361+0000","last_unstale":"2024-07-04T17:31:07.794742+0000","last_undegraded":"2024-07-04T17:31:07.794742+0000","last_fullsized":"2024-07-04T17:31:07.794742+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:25:11.379938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.777956+0000","last_change":"2024-07-04T17:31:06.808856+0000","last_active":"2024-07-04T17:31:07.777956+0000","last_peered":"2024-07-04T17:31:07.777956+0000","last_clean":"2024-07-04T17:31:07.777956+0000","last_became_active":"2024-07-04T17:31:06.808081+0000","last_became_peered":"2024-07-04T17:31:06.808081+0000","last_unstale":"2024-07-04T17:31:07.777956+0000","last_undegraded":"2024-07-04T17:31:07.777956+0000","last_fullsized":"2024-07-04T17:31:07.777956+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:06:01.257567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.790474+0000","last_change":"2024-07-04T17:30:47.422048+0000","last_active":"2024-07-04T17:31:07.790474+0000","last_peered":"2024-07-04T17:31:07.790474+0000","last_clean":"2024-07-04T17:31:07.790474+0000","last_became_active":"2024-07-04T17:30:47.421591+0000","last_became_peered":"2024-07-04T17:30:47.421591+0000","last_unstale":"2024-07-04T17:31:07.790474+0000","last_undegraded":"2024-07-04T17:31:07.790474+0000","last_fullsized":"2024-07-04T17:31:07.790474+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:58:22.418379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.772392+0000","last_change":"2024-07-04T17:31:06.785949+0000","last_active":"2024-07-04T17:31:07.772392+0000","last_peered":"2024-07-04T17:31:07.772392+0000","last_clean":"2024-07-04T17:31:07.772392+0000","last_became_active":"2024-07-04T17:31:06.785619+0000","last_became_peered":"2024-07-04T17:31:06.785619+0000","last_unstale":"2024-07-04T17:31:07.772392+0000","last_undegraded":"2024-07-04T17:31:07.772392+0000","last_fullsized":"2024-07-04T17:31:07.772392+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:58:59.664388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.788213+0000","last_change":"2024-07-04T17:30:47.414496+0000","last_active":"2024-07-04T17:31:07.788213+0000","last_peered":"2024-07-04T17:31:07.788213+0000","last_clean":"2024-07-04T17:31:07.788213+0000","last_became_active":"2024-07-04T17:30:47.414141+0000","last_became_peered":"2024-07-04T17:30:47.414141+0000","last_unstale":"2024-07-04T17:31:07.788213+0000","last_undegraded":"2024-07-04T17:31:07.788213+0000","last_fullsized":"2024-07-04T17:31:07.788213+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:09:08.612743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.772190+0000","last_change":"2024-07-04T17:31:06.783816+0000","last_active":"2024-07-04T17:31:07.772190+0000","last_peered":"2024-07-04T17:31:07.772190+0000","last_clean":"2024-07-04T17:31:07.772190+0000","last_became_active":"2024-07-04T17:31:06.782805+0000","last_became_peered":"2024-07-04T17:31:06.782805+0000","last_unstale":"2024-07-04T17:31:07.772190+0000","last_undegraded":"2024-07-04T17:31:07.772190+0000","last_fullsized":"2024-07-04T17:31:07.772190+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:09:52.268828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.794475+0000","last_change":"2024-07-04T17:30:47.419292+0000","last_active":"2024-07-04T17:31:07.794475+0000","last_peered":"2024-07-04T17:31:07.794475+0000","last_clean":"2024-07-04T17:31:07.794475+0000","last_became_active":"2024-07-04T17:30:47.418053+0000","last_became_peered":"2024-07-04T17:30:47.418053+0000","last_unstale":"2024-07-04T17:31:07.794475+0000","last_undegraded":"2024-07-04T17:31:07.794475+0000","last_fullsized":"2024-07-04T17:31:07.794475+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:02:17.626429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.774916+0000","last_change":"2024-07-04T17:31:06.788782+0000","last_active":"2024-07-04T17:31:07.774916+0000","last_peered":"2024-07-04T17:31:07.774916+0000","last_clean":"2024-07-04T17:31:07.774916+0000","last_became_active":"2024-07-04T17:31:06.788443+0000","last_became_peered":"2024-07-04T17:31:06.788443+0000","last_unstale":"2024-07-04T17:31:07.774916+0000","last_undegraded":"2024-07-04T17:31:07.774916+0000","last_fullsized":"2024-07-04T17:31:07.774916+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:08:08.378367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.787268+0000","last_change":"2024-07-04T17:30:47.436774+0000","last_active":"2024-07-04T17:31:07.787268+0000","last_peered":"2024-07-04T17:31:07.787268+0000","last_clean":"2024-07-04T17:31:07.787268+0000","last_became_active":"2024-07-04T17:30:47.436573+0000","last_became_peered":"2024-07-04T17:30:47.436573+0000","last_unstale":"2024-07-04T17:31:07.787268+0000","last_undegraded":"2024-07-04T17:31:07.787268+0000","last_fullsized":"2024-07-04T17:31:07.787268+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:27:06.393591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.761382+0000","last_change":"2024-07-04T17:31:06.805499+0000","last_active":"2024-07-04T17:31:07.761382+0000","last_peered":"2024-07-04T17:31:07.761382+0000","last_clean":"2024-07-04T17:31:07.761382+0000","last_became_active":"2024-07-04T17:31:06.805150+0000","last_became_peered":"2024-07-04T17:31:06.805150+0000","last_unstale":"2024-07-04T17:31:07.761382+0000","last_undegraded":"2024-07-04T17:31:07.761382+0000","last_fullsized":"2024-07-04T17:31:07.761382+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:09:37.952715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.773087+0000","last_change":"2024-07-04T17:30:47.389819+0000","last_active":"2024-07-04T17:31:07.773087+0000","last_peered":"2024-07-04T17:31:07.773087+0000","last_clean":"2024-07-04T17:31:07.773087+0000","last_became_active":"2024-07-04T17:30:47.387439+0000","last_became_peered":"2024-07-04T17:30:47.387439+0000","last_unstale":"2024-07-04T17:31:07.773087+0000","last_undegraded":"2024-07-04T17:31:07.773087+0000","last_fullsized":"2024-07-04T17:31:07.773087+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:56:14.415266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.790424+0000","last_change":"2024-07-04T17:31:06.794974+0000","last_active":"2024-07-04T17:31:07.790424+0000","last_peered":"2024-07-04T17:31:07.790424+0000","last_clean":"2024-07-04T17:31:07.790424+0000","last_became_active":"2024-07-04T17:31:06.794604+0000","last_became_peered":"2024-07-04T17:31:06.794604+0000","last_unstale":"2024-07-04T17:31:07.790424+0000","last_undegraded":"2024-07-04T17:31:07.790424+0000","last_fullsized":"2024-07-04T17:31:07.790424+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:42:16.943333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.766645+0000","last_change":"2024-07-04T17:30:47.395945+0000","last_active":"2024-07-04T17:31:07.766645+0000","last_peered":"2024-07-04T17:31:07.766645+0000","last_clean":"2024-07-04T17:31:07.766645+0000","last_became_active":"2024-07-04T17:30:47.395631+0000","last_became_peered":"2024-07-04T17:30:47.395631+0000","last_unstale":"2024-07-04T17:31:07.766645+0000","last_undegraded":"2024-07-04T17:31:07.766645+0000","last_fullsized":"2024-07-04T17:31:07.766645+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:27:30.220822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.771178+0000","last_change":"2024-07-04T17:31:06.792174+0000","last_active":"2024-07-04T17:31:07.771178+0000","last_peered":"2024-07-04T17:31:07.771178+0000","last_clean":"2024-07-04T17:31:07.771178+0000","last_became_active":"2024-07-04T17:31:06.790584+0000","last_became_peered":"2024-07-04T17:31:06.790584+0000","last_unstale":"2024-07-04T17:31:07.771178+0000","last_undegraded":"2024-07-04T17:31:07.771178+0000","last_fullsized":"2024-07-04T17:31:07.771178+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:39:53.589626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.794035+0000","last_change":"2024-07-04T17:30:47.436357+0000","last_active":"2024-07-04T17:31:07.794035+0000","last_peered":"2024-07-04T17:31:07.794035+0000","last_clean":"2024-07-04T17:31:07.794035+0000","last_became_active":"2024-07-04T17:30:47.435542+0000","last_became_peered":"2024-07-04T17:30:47.435542+0000","last_unstale":"2024-07-04T17:31:07.794035+0000","last_undegraded":"2024-07-04T17:31:07.794035+0000","last_fullsized":"2024-07-04T17:31:07.794035+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:08:21.911035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.794391+0000","last_change":"2024-07-04T17:31:06.812147+0000","last_active":"2024-07-04T17:31:07.794391+0000","last_peered":"2024-07-04T17:31:07.794391+0000","last_clean":"2024-07-04T17:31:07.794391+0000","last_became_active":"2024-07-04T17:31:06.811831+0000","last_became_peered":"2024-07-04T17:31:06.811831+0000","last_unstale":"2024-07-04T17:31:07.794391+0000","last_undegraded":"2024-07-04T17:31:07.794391+0000","last_fullsized":"2024-07-04T17:31:07.794391+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:36:54.001860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.763185+0000","last_change":"2024-07-04T17:30:47.430335+0000","last_active":"2024-07-04T17:31:07.763185+0000","last_peered":"2024-07-04T17:31:07.763185+0000","last_clean":"2024-07-04T17:31:07.763185+0000","last_became_active":"2024-07-04T17:30:47.427068+0000","last_became_peered":"2024-07-04T17:30:47.427068+0000","last_unstale":"2024-07-04T17:31:07.763185+0000","last_undegraded":"2024-07-04T17:31:07.763185+0000","last_fullsized":"2024-07-04T17:31:07.763185+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:09:53.897531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.777867+0000","last_change":"2024-07-04T17:31:06.814602+0000","last_active":"2024-07-04T17:31:07.777867+0000","last_peered":"2024-07-04T17:31:07.777867+0000","last_clean":"2024-07-04T17:31:07.777867+0000","last_became_active":"2024-07-04T17:31:06.808787+0000","last_became_peered":"2024-07-04T17:31:06.808787+0000","last_unstale":"2024-07-04T17:31:07.777867+0000","last_undegraded":"2024-07-04T17:31:07.777867+0000","last_fullsized":"2024-07-04T17:31:07.777867+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:54:06.025315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.777839+0000","last_change":"2024-07-04T17:30:47.398367+0000","last_active":"2024-07-04T17:31:07.777839+0000","last_peered":"2024-07-04T17:31:07.777839+0000","last_clean":"2024-07-04T17:31:07.777839+0000","last_became_active":"2024-07-04T17:30:47.394431+0000","last_became_peered":"2024-07-04T17:30:47.394431+0000","last_unstale":"2024-07-04T17:31:07.777839+0000","last_undegraded":"2024-07-04T17:31:07.777839+0000","last_fullsized":"2024-07-04T17:31:07.777839+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:14:28.773355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.760665+0000","last_change":"2024-07-04T17:31:06.804476+0000","last_active":"2024-07-04T17:31:07.760665+0000","last_peered":"2024-07-04T17:31:07.760665+0000","last_clean":"2024-07-04T17:31:07.760665+0000","last_became_active":"2024-07-04T17:31:06.802039+0000","last_became_peered":"2024-07-04T17:31:06.802039+0000","last_unstale":"2024-07-04T17:31:07.760665+0000","last_undegraded":"2024-07-04T17:31:07.760665+0000","last_fullsized":"2024-07-04T17:31:07.760665+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:50:39.828418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.774680+0000","last_change":"2024-07-04T17:30:47.426130+0000","last_active":"2024-07-04T17:31:07.774680+0000","last_peered":"2024-07-04T17:31:07.774680+0000","last_clean":"2024-07-04T17:31:07.774680+0000","last_became_active":"2024-07-04T17:30:47.416510+0000","last_became_peered":"2024-07-04T17:30:47.416510+0000","last_unstale":"2024-07-04T17:31:07.774680+0000","last_undegraded":"2024-07-04T17:31:07.774680+0000","last_fullsized":"2024-07-04T17:31:07.774680+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:53:42.201549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.772954+0000","last_change":"2024-07-04T17:31:06.783134+0000","last_active":"2024-07-04T17:31:07.772954+0000","last_peered":"2024-07-04T17:31:07.772954+0000","last_clean":"2024-07-04T17:31:07.772954+0000","last_became_active":"2024-07-04T17:31:06.782038+0000","last_became_peered":"2024-07-04T17:31:06.782038+0000","last_unstale":"2024-07-04T17:31:07.772954+0000","last_undegraded":"2024-07-04T17:31:07.772954+0000","last_fullsized":"2024-07-04T17:31:07.772954+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:45:58.454220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.794786+0000","last_change":"2024-07-04T17:30:47.417678+0000","last_active":"2024-07-04T17:31:07.794786+0000","last_peered":"2024-07-04T17:31:07.794786+0000","last_clean":"2024-07-04T17:31:07.794786+0000","last_became_active":"2024-07-04T17:30:47.414976+0000","last_became_peered":"2024-07-04T17:30:47.414976+0000","last_unstale":"2024-07-04T17:31:07.794786+0000","last_undegraded":"2024-07-04T17:31:07.794786+0000","last_fullsized":"2024-07-04T17:31:07.794786+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:40:08.588002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.774472+0000","last_change":"2024-07-04T17:31:06.797987+0000","last_active":"2024-07-04T17:31:07.774472+0000","last_peered":"2024-07-04T17:31:07.774472+0000","last_clean":"2024-07-04T17:31:07.774472+0000","last_became_active":"2024-07-04T17:31:06.797714+0000","last_became_peered":"2024-07-04T17:31:06.797714+0000","last_unstale":"2024-07-04T17:31:07.774472+0000","last_undegraded":"2024-07-04T17:31:07.774472+0000","last_fullsized":"2024-07-04T17:31:07.774472+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:54:05.082610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.761332+0000","last_change":"2024-07-04T17:30:47.396959+0000","last_active":"2024-07-04T17:31:07.761332+0000","last_peered":"2024-07-04T17:31:07.761332+0000","last_clean":"2024-07-04T17:31:07.761332+0000","last_became_active":"2024-07-04T17:30:47.396561+0000","last_became_peered":"2024-07-04T17:30:47.396561+0000","last_unstale":"2024-07-04T17:31:07.761332+0000","last_undegraded":"2024-07-04T17:31:07.761332+0000","last_fullsized":"2024-07-04T17:31:07.761332+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:03:40.493555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.787016+0000","last_change":"2024-07-04T17:31:06.781726+0000","last_active":"2024-07-04T17:31:07.787016+0000","last_peered":"2024-07-04T17:31:07.787016+0000","last_clean":"2024-07-04T17:31:07.787016+0000","last_became_active":"2024-07-04T17:31:06.780028+0000","last_became_peered":"2024-07-04T17:31:06.780028+0000","last_unstale":"2024-07-04T17:31:07.787016+0000","last_undegraded":"2024-07-04T17:31:07.787016+0000","last_fullsized":"2024-07-04T17:31:07.787016+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:30:53.779258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.788428+0000","last_change":"2024-07-04T17:30:47.420235+0000","last_active":"2024-07-04T17:31:07.788428+0000","last_peered":"2024-07-04T17:31:07.788428+0000","last_clean":"2024-07-04T17:31:07.788428+0000","last_became_active":"2024-07-04T17:30:47.419803+0000","last_became_peered":"2024-07-04T17:30:47.419803+0000","last_unstale":"2024-07-04T17:31:07.788428+0000","last_undegraded":"2024-07-04T17:31:07.788428+0000","last_fullsized":"2024-07-04T17:31:07.788428+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:27:30.220822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.790944+0000","last_change":"2024-07-04T17:31:06.810415+0000","last_active":"2024-07-04T17:31:07.790944+0000","last_peered":"2024-07-04T17:31:07.790944+0000","last_clean":"2024-07-04T17:31:07.790944+0000","last_became_active":"2024-07-04T17:31:06.809741+0000","last_became_peered":"2024-07-04T17:31:06.809741+0000","last_unstale":"2024-07-04T17:31:07.790944+0000","last_undegraded":"2024-07-04T17:31:07.790944+0000","last_fullsized":"2024-07-04T17:31:07.790944+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:59:55.916968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.778793+0000","last_change":"2024-07-04T17:30:47.426146+0000","last_active":"2024-07-04T17:31:07.778793+0000","last_peered":"2024-07-04T17:31:07.778793+0000","last_clean":"2024-07-04T17:31:07.778793+0000","last_became_active":"2024-07-04T17:30:47.421447+0000","last_became_peered":"2024-07-04T17:30:47.421447+0000","last_unstale":"2024-07-04T17:31:07.778793+0000","last_undegraded":"2024-07-04T17:31:07.778793+0000","last_fullsized":"2024-07-04T17:31:07.778793+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:14:40.880998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.763468+0000","last_change":"2024-07-04T17:31:06.775809+0000","last_active":"2024-07-04T17:31:07.763468+0000","last_peered":"2024-07-04T17:31:07.763468+0000","last_clean":"2024-07-04T17:31:07.763468+0000","last_became_active":"2024-07-04T17:31:06.775145+0000","last_became_peered":"2024-07-04T17:31:06.775145+0000","last_unstale":"2024-07-04T17:31:07.763468+0000","last_undegraded":"2024-07-04T17:31:07.763468+0000","last_fullsized":"2024-07-04T17:31:07.763468+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:07:15.831478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.794173+0000","last_change":"2024-07-04T17:30:47.436646+0000","last_active":"2024-07-04T17:31:07.794173+0000","last_peered":"2024-07-04T17:31:07.794173+0000","last_clean":"2024-07-04T17:31:07.794173+0000","last_became_active":"2024-07-04T17:30:47.435999+0000","last_became_peered":"2024-07-04T17:30:47.435999+0000","last_unstale":"2024-07-04T17:31:07.794173+0000","last_undegraded":"2024-07-04T17:31:07.794173+0000","last_fullsized":"2024-07-04T17:31:07.794173+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:27:44.098370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.781898+0000","last_change":"2024-07-04T17:31:06.808886+0000","last_active":"2024-07-04T17:31:07.781898+0000","last_peered":"2024-07-04T17:31:07.781898+0000","last_clean":"2024-07-04T17:31:07.781898+0000","last_became_active":"2024-07-04T17:31:06.808212+0000","last_became_peered":"2024-07-04T17:31:06.808212+0000","last_unstale":"2024-07-04T17:31:07.781898+0000","last_undegraded":"2024-07-04T17:31:07.781898+0000","last_fullsized":"2024-07-04T17:31:07.781898+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:16:43.460255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.778965+0000","last_change":"2024-07-04T17:30:47.398164+0000","last_active":"2024-07-04T17:31:07.778965+0000","last_peered":"2024-07-04T17:31:07.778965+0000","last_clean":"2024-07-04T17:31:07.778965+0000","last_became_active":"2024-07-04T17:30:47.394563+0000","last_became_peered":"2024-07-04T17:30:47.394563+0000","last_unstale":"2024-07-04T17:31:07.778965+0000","last_undegraded":"2024-07-04T17:31:07.778965+0000","last_fullsized":"2024-07-04T17:31:07.778965+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:51:16.498378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.778080+0000","last_change":"2024-07-04T17:31:06.797091+0000","last_active":"2024-07-04T17:31:07.778080+0000","last_peered":"2024-07-04T17:31:07.778080+0000","last_clean":"2024-07-04T17:31:07.778080+0000","last_became_active":"2024-07-04T17:31:06.796760+0000","last_became_peered":"2024-07-04T17:31:06.796760+0000","last_unstale":"2024-07-04T17:31:07.778080+0000","last_undegraded":"2024-07-04T17:31:07.778080+0000","last_fullsized":"2024-07-04T17:31:07.778080+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:34:00.727741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.790607+0000","last_change":"2024-07-04T17:30:47.438690+0000","last_active":"2024-07-04T17:31:07.790607+0000","last_peered":"2024-07-04T17:31:07.790607+0000","last_clean":"2024-07-04T17:31:07.790607+0000","last_became_active":"2024-07-04T17:30:47.438362+0000","last_became_peered":"2024-07-04T17:30:47.438362+0000","last_unstale":"2024-07-04T17:31:07.790607+0000","last_undegraded":"2024-07-04T17:31:07.790607+0000","last_fullsized":"2024-07-04T17:31:07.790607+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:28:11.855854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.772492+0000","last_change":"2024-07-04T17:31:06.762688+0000","last_active":"2024-07-04T17:31:07.772492+0000","last_peered":"2024-07-04T17:31:07.772492+0000","last_clean":"2024-07-04T17:31:07.772492+0000","last_became_active":"2024-07-04T17:31:06.762317+0000","last_became_peered":"2024-07-04T17:31:06.762317+0000","last_unstale":"2024-07-04T17:31:07.772492+0000","last_undegraded":"2024-07-04T17:31:07.772492+0000","last_fullsized":"2024-07-04T17:31:07.772492+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:03:31.127398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.788440+0000","last_change":"2024-07-04T17:30:47.435331+0000","last_active":"2024-07-04T17:31:07.788440+0000","last_peered":"2024-07-04T17:31:07.788440+0000","last_clean":"2024-07-04T17:31:07.788440+0000","last_became_active":"2024-07-04T17:30:47.434100+0000","last_became_peered":"2024-07-04T17:30:47.434100+0000","last_unstale":"2024-07-04T17:31:07.788440+0000","last_undegraded":"2024-07-04T17:31:07.788440+0000","last_fullsized":"2024-07-04T17:31:07.788440+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:47:32.884542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.786707+0000","last_change":"2024-07-04T17:31:06.791355+0000","last_active":"2024-07-04T17:31:07.786707+0000","last_peered":"2024-07-04T17:31:07.786707+0000","last_clean":"2024-07-04T17:31:07.786707+0000","last_became_active":"2024-07-04T17:31:06.788869+0000","last_became_peered":"2024-07-04T17:31:06.788869+0000","last_unstale":"2024-07-04T17:31:07.786707+0000","last_undegraded":"2024-07-04T17:31:07.786707+0000","last_fullsized":"2024-07-04T17:31:07.786707+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:44:56.385110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.767426+0000","last_change":"2024-07-04T17:30:47.399163+0000","last_active":"2024-07-04T17:31:07.767426+0000","last_peered":"2024-07-04T17:31:07.767426+0000","last_clean":"2024-07-04T17:31:07.767426+0000","last_became_active":"2024-07-04T17:30:47.398747+0000","last_became_peered":"2024-07-04T17:30:47.398747+0000","last_unstale":"2024-07-04T17:31:07.767426+0000","last_undegraded":"2024-07-04T17:31:07.767426+0000","last_fullsized":"2024-07-04T17:31:07.767426+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:24:01.574518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.788395+0000","last_change":"2024-07-04T17:31:06.778979+0000","last_active":"2024-07-04T17:31:07.788395+0000","last_peered":"2024-07-04T17:31:07.788395+0000","last_clean":"2024-07-04T17:31:07.788395+0000","last_became_active":"2024-07-04T17:31:06.776516+0000","last_became_peered":"2024-07-04T17:31:06.776516+0000","last_unstale":"2024-07-04T17:31:07.788395+0000","last_undegraded":"2024-07-04T17:31:07.788395+0000","last_fullsized":"2024-07-04T17:31:07.788395+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:29:18.514550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.772881+0000","last_change":"2024-07-04T17:30:47.421142+0000","last_active":"2024-07-04T17:31:07.772881+0000","last_peered":"2024-07-04T17:31:07.772881+0000","last_clean":"2024-07-04T17:31:07.772881+0000","last_became_active":"2024-07-04T17:30:47.416769+0000","last_became_peered":"2024-07-04T17:30:47.416769+0000","last_unstale":"2024-07-04T17:31:07.772881+0000","last_undegraded":"2024-07-04T17:31:07.772881+0000","last_fullsized":"2024-07-04T17:31:07.772881+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:19:39.765910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.773830+0000","last_change":"2024-07-04T17:31:06.775164+0000","last_active":"2024-07-04T17:31:07.773830+0000","last_peered":"2024-07-04T17:31:07.773830+0000","last_clean":"2024-07-04T17:31:07.773830+0000","last_became_active":"2024-07-04T17:31:06.774616+0000","last_became_peered":"2024-07-04T17:31:06.774616+0000","last_unstale":"2024-07-04T17:31:07.773830+0000","last_undegraded":"2024-07-04T17:31:07.773830+0000","last_fullsized":"2024-07-04T17:31:07.773830+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:52:16.933852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.788565+0000","last_change":"2024-07-04T17:30:47.421958+0000","last_active":"2024-07-04T17:31:07.788565+0000","last_peered":"2024-07-04T17:31:07.788565+0000","last_clean":"2024-07-04T17:31:07.788565+0000","last_became_active":"2024-07-04T17:30:47.421313+0000","last_became_peered":"2024-07-04T17:30:47.421313+0000","last_unstale":"2024-07-04T17:31:07.788565+0000","last_undegraded":"2024-07-04T17:31:07.788565+0000","last_fullsized":"2024-07-04T17:31:07.788565+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:44:44.984203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.790973+0000","last_change":"2024-07-04T17:31:06.800887+0000","last_active":"2024-07-04T17:31:07.790973+0000","last_peered":"2024-07-04T17:31:07.790973+0000","last_clean":"2024-07-04T17:31:07.790973+0000","last_became_active":"2024-07-04T17:31:06.799695+0000","last_became_peered":"2024-07-04T17:31:06.799695+0000","last_unstale":"2024-07-04T17:31:07.790973+0000","last_undegraded":"2024-07-04T17:31:07.790973+0000","last_fullsized":"2024-07-04T17:31:07.790973+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:25:20.963607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.766769+0000","last_change":"2024-07-04T17:30:47.398351+0000","last_active":"2024-07-04T17:31:07.766769+0000","last_peered":"2024-07-04T17:31:07.766769+0000","last_clean":"2024-07-04T17:31:07.766769+0000","last_became_active":"2024-07-04T17:30:47.395987+0000","last_became_peered":"2024-07-04T17:30:47.395987+0000","last_unstale":"2024-07-04T17:31:07.766769+0000","last_undegraded":"2024-07-04T17:31:07.766769+0000","last_fullsized":"2024-07-04T17:31:07.766769+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:44:44.984203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.789875+0000","last_change":"2024-07-04T17:31:06.806936+0000","last_active":"2024-07-04T17:31:07.789875+0000","last_peered":"2024-07-04T17:31:07.789875+0000","last_clean":"2024-07-04T17:31:07.789875+0000","last_became_active":"2024-07-04T17:31:06.806243+0000","last_became_peered":"2024-07-04T17:31:06.806243+0000","last_unstale":"2024-07-04T17:31:07.789875+0000","last_undegraded":"2024-07-04T17:31:07.789875+0000","last_fullsized":"2024-07-04T17:31:07.789875+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:53:40.275129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.767209+0000","last_change":"2024-07-04T17:30:47.398645+0000","last_active":"2024-07-04T17:31:07.767209+0000","last_peered":"2024-07-04T17:31:07.767209+0000","last_clean":"2024-07-04T17:31:07.767209+0000","last_became_active":"2024-07-04T17:30:47.394765+0000","last_became_peered":"2024-07-04T17:30:47.394765+0000","last_unstale":"2024-07-04T17:31:07.767209+0000","last_undegraded":"2024-07-04T17:31:07.767209+0000","last_fullsized":"2024-07-04T17:31:07.767209+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:59:48.588528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.778348+0000","last_change":"2024-07-04T17:31:06.808534+0000","last_active":"2024-07-04T17:31:07.778348+0000","last_peered":"2024-07-04T17:31:07.778348+0000","last_clean":"2024-07-04T17:31:07.778348+0000","last_became_active":"2024-07-04T17:31:06.807784+0000","last_became_peered":"2024-07-04T17:31:06.807784+0000","last_unstale":"2024-07-04T17:31:07.778348+0000","last_undegraded":"2024-07-04T17:31:07.778348+0000","last_fullsized":"2024-07-04T17:31:07.778348+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:35:46.747556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.795054+0000","last_change":"2024-07-04T17:30:47.439010+0000","last_active":"2024-07-04T17:31:07.795054+0000","last_peered":"2024-07-04T17:31:07.795054+0000","last_clean":"2024-07-04T17:31:07.795054+0000","last_became_active":"2024-07-04T17:30:47.436372+0000","last_became_peered":"2024-07-04T17:30:47.436372+0000","last_unstale":"2024-07-04T17:31:07.795054+0000","last_undegraded":"2024-07-04T17:31:07.795054+0000","last_fullsized":"2024-07-04T17:31:07.795054+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:08:49.032762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.778345+0000","last_change":"2024-07-04T17:31:06.809104+0000","last_active":"2024-07-04T17:31:07.778345+0000","last_peered":"2024-07-04T17:31:07.778345+0000","last_clean":"2024-07-04T17:31:07.778345+0000","last_became_active":"2024-07-04T17:31:06.808518+0000","last_became_peered":"2024-07-04T17:31:06.808518+0000","last_unstale":"2024-07-04T17:31:07.778345+0000","last_undegraded":"2024-07-04T17:31:07.778345+0000","last_fullsized":"2024-07-04T17:31:07.778345+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:13:37.972258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.770854+0000","last_change":"2024-07-04T17:30:47.430081+0000","last_active":"2024-07-04T17:31:07.770854+0000","last_peered":"2024-07-04T17:31:07.770854+0000","last_clean":"2024-07-04T17:31:07.770854+0000","last_became_active":"2024-07-04T17:30:47.425964+0000","last_became_peered":"2024-07-04T17:30:47.425964+0000","last_unstale":"2024-07-04T17:31:07.770854+0000","last_undegraded":"2024-07-04T17:31:07.770854+0000","last_fullsized":"2024-07-04T17:31:07.770854+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:44:12.909185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.760782+0000","last_change":"2024-07-04T17:31:06.806160+0000","last_active":"2024-07-04T17:31:07.760782+0000","last_peered":"2024-07-04T17:31:07.760782+0000","last_clean":"2024-07-04T17:31:07.760782+0000","last_became_active":"2024-07-04T17:31:06.805744+0000","last_became_peered":"2024-07-04T17:31:06.805744+0000","last_unstale":"2024-07-04T17:31:07.760782+0000","last_undegraded":"2024-07-04T17:31:07.760782+0000","last_fullsized":"2024-07-04T17:31:07.760782+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:56:33.786932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.767125+0000","last_change":"2024-07-04T17:30:47.389158+0000","last_active":"2024-07-04T17:31:07.767125+0000","last_peered":"2024-07-04T17:31:07.767125+0000","last_clean":"2024-07-04T17:31:07.767125+0000","last_became_active":"2024-07-04T17:30:47.388127+0000","last_became_peered":"2024-07-04T17:30:47.388127+0000","last_unstale":"2024-07-04T17:31:07.767125+0000","last_undegraded":"2024-07-04T17:31:07.767125+0000","last_fullsized":"2024-07-04T17:31:07.767125+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:49:47.842800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.762222+0000","last_change":"2024-07-04T17:31:06.804746+0000","last_active":"2024-07-04T17:31:07.762222+0000","last_peered":"2024-07-04T17:31:07.762222+0000","last_clean":"2024-07-04T17:31:07.762222+0000","last_became_active":"2024-07-04T17:31:06.804201+0000","last_became_peered":"2024-07-04T17:31:06.804201+0000","last_unstale":"2024-07-04T17:31:07.762222+0000","last_undegraded":"2024-07-04T17:31:07.762222+0000","last_fullsized":"2024-07-04T17:31:07.762222+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:13:37.053132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.767623+0000","last_change":"2024-07-04T17:30:47.427109+0000","last_active":"2024-07-04T17:31:07.767623+0000","last_peered":"2024-07-04T17:31:07.767623+0000","last_clean":"2024-07-04T17:31:07.767623+0000","last_became_active":"2024-07-04T17:30:47.426697+0000","last_became_peered":"2024-07-04T17:30:47.426697+0000","last_unstale":"2024-07-04T17:31:07.767623+0000","last_undegraded":"2024-07-04T17:31:07.767623+0000","last_fullsized":"2024-07-04T17:31:07.767623+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:20:37.646569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.775192+0000","last_change":"2024-07-04T17:31:06.786111+0000","last_active":"2024-07-04T17:31:07.775192+0000","last_peered":"2024-07-04T17:31:07.775192+0000","last_clean":"2024-07-04T17:31:07.775192+0000","last_became_active":"2024-07-04T17:31:06.782570+0000","last_became_peered":"2024-07-04T17:31:06.782570+0000","last_unstale":"2024-07-04T17:31:07.775192+0000","last_undegraded":"2024-07-04T17:31:07.775192+0000","last_fullsized":"2024-07-04T17:31:07.775192+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:27:52.882512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.787406+0000","last_change":"2024-07-04T17:30:47.435621+0000","last_active":"2024-07-04T17:31:07.787406+0000","last_peered":"2024-07-04T17:31:07.787406+0000","last_clean":"2024-07-04T17:31:07.787406+0000","last_became_active":"2024-07-04T17:30:47.435230+0000","last_became_peered":"2024-07-04T17:30:47.435230+0000","last_unstale":"2024-07-04T17:31:07.787406+0000","last_undegraded":"2024-07-04T17:31:07.787406+0000","last_fullsized":"2024-07-04T17:31:07.787406+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:29:26.886812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.787446+0000","last_change":"2024-07-04T17:31:06.795115+0000","last_active":"2024-07-04T17:31:07.787446+0000","last_peered":"2024-07-04T17:31:07.787446+0000","last_clean":"2024-07-04T17:31:07.787446+0000","last_became_active":"2024-07-04T17:31:06.794762+0000","last_became_peered":"2024-07-04T17:31:06.794762+0000","last_unstale":"2024-07-04T17:31:07.787446+0000","last_undegraded":"2024-07-04T17:31:07.787446+0000","last_fullsized":"2024-07-04T17:31:07.787446+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:35:48.506068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.790274+0000","last_change":"2024-07-04T17:30:47.414726+0000","last_active":"2024-07-04T17:31:07.790274+0000","last_peered":"2024-07-04T17:31:07.790274+0000","last_clean":"2024-07-04T17:31:07.790274+0000","last_became_active":"2024-07-04T17:30:47.414412+0000","last_became_peered":"2024-07-04T17:30:47.414412+0000","last_unstale":"2024-07-04T17:31:07.790274+0000","last_undegraded":"2024-07-04T17:31:07.790274+0000","last_fullsized":"2024-07-04T17:31:07.790274+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:59:48.588528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.766739+0000","last_change":"2024-07-04T17:31:06.773912+0000","last_active":"2024-07-04T17:31:07.766739+0000","last_peered":"2024-07-04T17:31:07.766739+0000","last_clean":"2024-07-04T17:31:07.766739+0000","last_became_active":"2024-07-04T17:31:06.771396+0000","last_became_peered":"2024-07-04T17:31:06.771396+0000","last_unstale":"2024-07-04T17:31:07.766739+0000","last_undegraded":"2024-07-04T17:31:07.766739+0000","last_fullsized":"2024-07-04T17:31:07.766739+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:15:00.252664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.778124+0000","last_change":"2024-07-04T17:31:05.791225+0000","last_active":"2024-07-04T17:31:07.778124+0000","last_peered":"2024-07-04T17:31:07.778124+0000","last_clean":"2024-07-04T17:31:07.778124+0000","last_became_active":"2024-07-04T17:31:05.789499+0000","last_became_peered":"2024-07-04T17:31:05.789499+0000","last_unstale":"2024-07-04T17:31:07.778124+0000","last_undegraded":"2024-07-04T17:31:07.778124+0000","last_fullsized":"2024-07-04T17:31:07.778124+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:40:49.033282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.781737+0000","last_change":"2024-07-04T17:30:47.426760+0000","last_active":"2024-07-04T17:31:07.781737+0000","last_peered":"2024-07-04T17:31:07.781737+0000","last_clean":"2024-07-04T17:31:07.781737+0000","last_became_active":"2024-07-04T17:30:47.426331+0000","last_became_peered":"2024-07-04T17:30:47.426331+0000","last_unstale":"2024-07-04T17:31:07.781737+0000","last_undegraded":"2024-07-04T17:31:07.781737+0000","last_fullsized":"2024-07-04T17:31:07.781737+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:47:13.050952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"71'4","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.779026+0000","last_change":"2024-07-04T17:30:46.417984+0000","last_active":"2024-07-04T17:31:07.779026+0000","last_peered":"2024-07-04T17:31:07.779026+0000","last_clean":"2024-07-04T17:31:07.779026+0000","last_became_active":"2024-07-04T17:30:46.417091+0000","last_became_peered":"2024-07-04T17:30:46.417091+0000","last_unstale":"2024-07-04T17:31:07.779026+0000","last_undegraded":"2024-07-04T17:31:07.779026+0000","last_fullsized":"2024-07-04T17:31:07.779026+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:29:34.252784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.777986+0000","last_change":"2024-07-04T17:31:06.779063+0000","last_active":"2024-07-04T17:31:07.777986+0000","last_peered":"2024-07-04T17:31:07.777986+0000","last_clean":"2024-07-04T17:31:07.777986+0000","last_became_active":"2024-07-04T17:31:06.778773+0000","last_became_peered":"2024-07-04T17:31:06.778773+0000","last_unstale":"2024-07-04T17:31:07.777986+0000","last_undegraded":"2024-07-04T17:31:07.777986+0000","last_fullsized":"2024-07-04T17:31:07.777986+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:59:35.898050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.775377+0000","last_change":"2024-07-04T17:31:05.770412+0000","last_active":"2024-07-04T17:31:07.775377+0000","last_peered":"2024-07-04T17:31:07.775377+0000","last_clean":"2024-07-04T17:31:07.775377+0000","last_became_active":"2024-07-04T17:31:05.770076+0000","last_became_peered":"2024-07-04T17:31:05.770076+0000","last_unstale":"2024-07-04T17:31:07.775377+0000","last_undegraded":"2024-07-04T17:31:07.775377+0000","last_fullsized":"2024-07-04T17:31:07.775377+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:33:18.299053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.766884+0000","last_change":"2024-07-04T17:30:47.419884+0000","last_active":"2024-07-04T17:31:07.766884+0000","last_peered":"2024-07-04T17:31:07.766884+0000","last_clean":"2024-07-04T17:31:07.766884+0000","last_became_active":"2024-07-04T17:30:47.419588+0000","last_became_peered":"2024-07-04T17:30:47.419588+0000","last_unstale":"2024-07-04T17:31:07.766884+0000","last_undegraded":"2024-07-04T17:31:07.766884+0000","last_fullsized":"2024-07-04T17:31:07.766884+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:02:17.626429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"71'1","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.794713+0000","last_change":"2024-07-04T17:30:46.394693+0000","last_active":"2024-07-04T17:31:07.794713+0000","last_peered":"2024-07-04T17:31:07.794713+0000","last_clean":"2024-07-04T17:31:07.794713+0000","last_became_active":"2024-07-04T17:30:46.394315+0000","last_became_peered":"2024-07-04T17:30:46.394315+0000","last_unstale":"2024-07-04T17:31:07.794713+0000","last_undegraded":"2024-07-04T17:31:07.794713+0000","last_fullsized":"2024-07-04T17:31:07.794713+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:56:39.574079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.790159+0000","last_change":"2024-07-04T17:31:06.807941+0000","last_active":"2024-07-04T17:31:07.790159+0000","last_peered":"2024-07-04T17:31:07.790159+0000","last_clean":"2024-07-04T17:31:07.790159+0000","last_became_active":"2024-07-04T17:31:06.807236+0000","last_became_peered":"2024-07-04T17:31:06.807236+0000","last_unstale":"2024-07-04T17:31:07.790159+0000","last_undegraded":"2024-07-04T17:31:07.790159+0000","last_fullsized":"2024-07-04T17:31:07.790159+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:59:52.897299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"75'4","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.778277+0000","last_change":"2024-07-04T17:31:05.771337+0000","last_active":"2024-07-04T17:31:07.778277+0000","last_peered":"2024-07-04T17:31:07.778277+0000","last_clean":"2024-07-04T17:31:07.778277+0000","last_became_active":"2024-07-04T17:31:05.770842+0000","last_became_peered":"2024-07-04T17:31:05.770842+0000","last_unstale":"2024-07-04T17:31:07.778277+0000","last_undegraded":"2024-07-04T17:31:07.778277+0000","last_fullsized":"2024-07-04T17:31:07.778277+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:32:52.855531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.779015+0000","last_change":"2024-07-04T17:30:47.398047+0000","last_active":"2024-07-04T17:31:07.779015+0000","last_peered":"2024-07-04T17:31:07.779015+0000","last_clean":"2024-07-04T17:31:07.779015+0000","last_became_active":"2024-07-04T17:30:47.393052+0000","last_became_peered":"2024-07-04T17:30:47.393052+0000","last_unstale":"2024-07-04T17:31:07.779015+0000","last_undegraded":"2024-07-04T17:31:07.779015+0000","last_fullsized":"2024-07-04T17:31:07.779015+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:13:17.681466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"71'2","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.781521+0000","last_change":"2024-07-04T17:30:46.401872+0000","last_active":"2024-07-04T17:31:07.781521+0000","last_peered":"2024-07-04T17:31:07.781521+0000","last_clean":"2024-07-04T17:31:07.781521+0000","last_became_active":"2024-07-04T17:30:46.401510+0000","last_became_peered":"2024-07-04T17:30:46.401510+0000","last_unstale":"2024-07-04T17:31:07.781521+0000","last_undegraded":"2024-07-04T17:31:07.781521+0000","last_fullsized":"2024-07-04T17:31:07.781521+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:02:03.254039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.762957+0000","last_change":"2024-07-04T17:31:06.805374+0000","last_active":"2024-07-04T17:31:07.762957+0000","last_peered":"2024-07-04T17:31:07.762957+0000","last_clean":"2024-07-04T17:31:07.762957+0000","last_became_active":"2024-07-04T17:31:06.804909+0000","last_became_peered":"2024-07-04T17:31:06.804909+0000","last_unstale":"2024-07-04T17:31:07.762957+0000","last_undegraded":"2024-07-04T17:31:07.762957+0000","last_fullsized":"2024-07-04T17:31:07.762957+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:42:00.091775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1e","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.778782+0000","last_change":"2024-07-04T17:31:05.768631+0000","last_active":"2024-07-04T17:31:07.778782+0000","last_peered":"2024-07-04T17:31:07.778782+0000","last_clean":"2024-07-04T17:31:07.778782+0000","last_became_active":"2024-07-04T17:31:05.768099+0000","last_became_peered":"2024-07-04T17:31:05.768099+0000","last_unstale":"2024-07-04T17:31:07.778782+0000","last_undegraded":"2024-07-04T17:31:07.778782+0000","last_fullsized":"2024-07-04T17:31:07.778782+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:33:31.434365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.763256+0000","last_change":"2024-07-04T17:30:47.418834+0000","last_active":"2024-07-04T17:31:07.763256+0000","last_peered":"2024-07-04T17:31:07.763256+0000","last_clean":"2024-07-04T17:31:07.763256+0000","last_became_active":"2024-07-04T17:30:47.418449+0000","last_became_peered":"2024-07-04T17:30:47.418449+0000","last_unstale":"2024-07-04T17:31:07.763256+0000","last_undegraded":"2024-07-04T17:31:07.763256+0000","last_fullsized":"2024-07-04T17:31:07.763256+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:59:40.100483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"71'2","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.778880+0000","last_change":"2024-07-04T17:30:46.417993+0000","last_active":"2024-07-04T17:31:07.778880+0000","last_peered":"2024-07-04T17:31:07.778880+0000","last_clean":"2024-07-04T17:31:07.778880+0000","last_became_active":"2024-07-04T17:30:46.417603+0000","last_became_peered":"2024-07-04T17:30:46.417603+0000","last_unstale":"2024-07-04T17:31:07.778880+0000","last_undegraded":"2024-07-04T17:31:07.778880+0000","last_fullsized":"2024-07-04T17:31:07.778880+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:18:43.863213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.772434+0000","last_change":"2024-07-04T17:31:06.786387+0000","last_active":"2024-07-04T17:31:07.772434+0000","last_peered":"2024-07-04T17:31:07.772434+0000","last_clean":"2024-07-04T17:31:07.772434+0000","last_became_active":"2024-07-04T17:31:06.782444+0000","last_became_peered":"2024-07-04T17:31:06.782444+0000","last_unstale":"2024-07-04T17:31:07.772434+0000","last_undegraded":"2024-07-04T17:31:07.772434+0000","last_fullsized":"2024-07-04T17:31:07.772434+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:55:11.570539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.772647+0000","last_change":"2024-07-04T17:31:05.768330+0000","last_active":"2024-07-04T17:31:07.772647+0000","last_peered":"2024-07-04T17:31:07.772647+0000","last_clean":"2024-07-04T17:31:07.772647+0000","last_became_active":"2024-07-04T17:31:05.767975+0000","last_became_peered":"2024-07-04T17:31:05.767975+0000","last_unstale":"2024-07-04T17:31:07.772647+0000","last_undegraded":"2024-07-04T17:31:07.772647+0000","last_fullsized":"2024-07-04T17:31:07.772647+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:36:52.988009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.767689+0000","last_change":"2024-07-04T17:30:47.427026+0000","last_active":"2024-07-04T17:31:07.767689+0000","last_peered":"2024-07-04T17:31:07.767689+0000","last_clean":"2024-07-04T17:31:07.767689+0000","last_became_active":"2024-07-04T17:30:47.426173+0000","last_became_peered":"2024-07-04T17:30:47.426173+0000","last_unstale":"2024-07-04T17:31:07.767689+0000","last_undegraded":"2024-07-04T17:31:07.767689+0000","last_fullsized":"2024-07-04T17:31:07.767689+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:00:08.547137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"71'1","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.774919+0000","last_change":"2024-07-04T17:30:46.408705+0000","last_active":"2024-07-04T17:31:07.774919+0000","last_peered":"2024-07-04T17:31:07.774919+0000","last_clean":"2024-07-04T17:31:07.774919+0000","last_became_active":"2024-07-04T17:30:46.408366+0000","last_became_peered":"2024-07-04T17:30:46.408366+0000","last_unstale":"2024-07-04T17:31:07.774919+0000","last_undegraded":"2024-07-04T17:31:07.774919+0000","last_fullsized":"2024-07-04T17:31:07.774919+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:07:19.994029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.760994+0000","last_change":"2024-07-04T17:31:06.771014+0000","last_active":"2024-07-04T17:31:07.760994+0000","last_peered":"2024-07-04T17:31:07.760994+0000","last_clean":"2024-07-04T17:31:07.760994+0000","last_became_active":"2024-07-04T17:31:06.770712+0000","last_became_peered":"2024-07-04T17:31:06.770712+0000","last_unstale":"2024-07-04T17:31:07.760994+0000","last_undegraded":"2024-07-04T17:31:07.760994+0000","last_fullsized":"2024-07-04T17:31:07.760994+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:14:48.145021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.766402+0000","last_change":"2024-07-04T17:31:05.763717+0000","last_active":"2024-07-04T17:31:07.766402+0000","last_peered":"2024-07-04T17:31:07.766402+0000","last_clean":"2024-07-04T17:31:07.766402+0000","last_became_active":"2024-07-04T17:31:05.763408+0000","last_became_peered":"2024-07-04T17:31:05.763408+0000","last_unstale":"2024-07-04T17:31:07.766402+0000","last_undegraded":"2024-07-04T17:31:07.766402+0000","last_fullsized":"2024-07-04T17:31:07.766402+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:44:08.872556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.788541+0000","last_change":"2024-07-04T17:30:47.435295+0000","last_active":"2024-07-04T17:31:07.788541+0000","last_peered":"2024-07-04T17:31:07.788541+0000","last_clean":"2024-07-04T17:31:07.788541+0000","last_became_active":"2024-07-04T17:30:47.434416+0000","last_became_peered":"2024-07-04T17:30:47.434416+0000","last_unstale":"2024-07-04T17:31:07.788541+0000","last_undegraded":"2024-07-04T17:31:07.788541+0000","last_fullsized":"2024-07-04T17:31:07.788541+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:20:27.029139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"71'1","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.795060+0000","last_change":"2024-07-04T17:30:46.409671+0000","last_active":"2024-07-04T17:31:07.795060+0000","last_peered":"2024-07-04T17:31:07.795060+0000","last_clean":"2024-07-04T17:31:07.795060+0000","last_became_active":"2024-07-04T17:30:46.409031+0000","last_became_peered":"2024-07-04T17:30:46.409031+0000","last_unstale":"2024-07-04T17:31:07.795060+0000","last_undegraded":"2024-07-04T17:31:07.795060+0000","last_fullsized":"2024-07-04T17:31:07.795060+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:06:35.004818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.767305+0000","last_change":"2024-07-04T17:31:06.772831+0000","last_active":"2024-07-04T17:31:07.767305+0000","last_peered":"2024-07-04T17:31:07.767305+0000","last_clean":"2024-07-04T17:31:07.767305+0000","last_became_active":"2024-07-04T17:31:06.771682+0000","last_became_peered":"2024-07-04T17:31:06.771682+0000","last_unstale":"2024-07-04T17:31:07.767305+0000","last_undegraded":"2024-07-04T17:31:07.767305+0000","last_fullsized":"2024-07-04T17:31:07.767305+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:44:56.385110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.766841+0000","last_change":"2024-07-04T17:31:05.776134+0000","last_active":"2024-07-04T17:31:07.766841+0000","last_peered":"2024-07-04T17:31:07.766841+0000","last_clean":"2024-07-04T17:31:07.766841+0000","last_became_active":"2024-07-04T17:31:05.775776+0000","last_became_peered":"2024-07-04T17:31:05.775776+0000","last_unstale":"2024-07-04T17:31:07.766841+0000","last_undegraded":"2024-07-04T17:31:07.766841+0000","last_fullsized":"2024-07-04T17:31:07.766841+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:08:45.260384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.762715+0000","last_change":"2024-07-04T17:30:47.394313+0000","last_active":"2024-07-04T17:31:07.762715+0000","last_peered":"2024-07-04T17:31:07.762715+0000","last_clean":"2024-07-04T17:31:07.762715+0000","last_became_active":"2024-07-04T17:30:47.393967+0000","last_became_peered":"2024-07-04T17:30:47.393967+0000","last_unstale":"2024-07-04T17:31:07.762715+0000","last_undegraded":"2024-07-04T17:31:07.762715+0000","last_fullsized":"2024-07-04T17:31:07.762715+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:13:54.972134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.773006+0000","last_change":"2024-07-04T17:30:46.414098+0000","last_active":"2024-07-04T17:31:07.773006+0000","last_peered":"2024-07-04T17:31:07.773006+0000","last_clean":"2024-07-04T17:31:07.773006+0000","last_became_active":"2024-07-04T17:30:46.413464+0000","last_became_peered":"2024-07-04T17:30:46.413464+0000","last_unstale":"2024-07-04T17:31:07.773006+0000","last_undegraded":"2024-07-04T17:31:07.773006+0000","last_fullsized":"2024-07-04T17:31:07.773006+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:43:49.510457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.773838+0000","last_change":"2024-07-04T17:31:06.762778+0000","last_active":"2024-07-04T17:31:07.773838+0000","last_peered":"2024-07-04T17:31:07.773838+0000","last_clean":"2024-07-04T17:31:07.773838+0000","last_became_active":"2024-07-04T17:31:06.762433+0000","last_became_peered":"2024-07-04T17:31:06.762433+0000","last_unstale":"2024-07-04T17:31:07.773838+0000","last_undegraded":"2024-07-04T17:31:07.773838+0000","last_fullsized":"2024-07-04T17:31:07.773838+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:05:57.855010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1a","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.773971+0000","last_change":"2024-07-04T17:31:05.770550+0000","last_active":"2024-07-04T17:31:07.773971+0000","last_peered":"2024-07-04T17:31:07.773971+0000","last_clean":"2024-07-04T17:31:07.773971+0000","last_became_active":"2024-07-04T17:31:05.770248+0000","last_became_peered":"2024-07-04T17:31:05.770248+0000","last_unstale":"2024-07-04T17:31:07.773971+0000","last_undegraded":"2024-07-04T17:31:07.773971+0000","last_fullsized":"2024-07-04T17:31:07.773971+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:24:22.331288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.787462+0000","last_change":"2024-07-04T17:30:47.424416+0000","last_active":"2024-07-04T17:31:07.787462+0000","last_peered":"2024-07-04T17:31:07.787462+0000","last_clean":"2024-07-04T17:31:07.787462+0000","last_became_active":"2024-07-04T17:30:47.423168+0000","last_became_peered":"2024-07-04T17:30:47.423168+0000","last_unstale":"2024-07-04T17:31:07.787462+0000","last_undegraded":"2024-07-04T17:31:07.787462+0000","last_fullsized":"2024-07-04T17:31:07.787462+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:02:04.258323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.763097+0000","last_change":"2024-07-04T17:30:46.414564+0000","last_active":"2024-07-04T17:31:07.763097+0000","last_peered":"2024-07-04T17:31:07.763097+0000","last_clean":"2024-07-04T17:31:07.763097+0000","last_became_active":"2024-07-04T17:30:46.414116+0000","last_became_peered":"2024-07-04T17:30:46.414116+0000","last_unstale":"2024-07-04T17:31:07.763097+0000","last_undegraded":"2024-07-04T17:31:07.763097+0000","last_fullsized":"2024-07-04T17:31:07.763097+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:49:01.886056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.789776+0000","last_change":"2024-07-04T17:31:06.791599+0000","last_active":"2024-07-04T17:31:07.789776+0000","last_peered":"2024-07-04T17:31:07.789776+0000","last_clean":"2024-07-04T17:31:07.789776+0000","last_became_active":"2024-07-04T17:31:06.789061+0000","last_became_peered":"2024-07-04T17:31:06.789061+0000","last_unstale":"2024-07-04T17:31:07.789776+0000","last_undegraded":"2024-07-04T17:31:07.789776+0000","last_fullsized":"2024-07-04T17:31:07.789776+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:01:49.265579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.15","version":"75'3","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.763098+0000","last_change":"2024-07-04T17:31:05.771852+0000","last_active":"2024-07-04T17:31:07.763098+0000","last_peered":"2024-07-04T17:31:07.763098+0000","last_clean":"2024-07-04T17:31:07.763098+0000","last_became_active":"2024-07-04T17:31:05.771389+0000","last_became_peered":"2024-07-04T17:31:05.771389+0000","last_unstale":"2024-07-04T17:31:07.763098+0000","last_undegraded":"2024-07-04T17:31:07.763098+0000","last_fullsized":"2024-07-04T17:31:07.763098+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:58:32.536818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.781881+0000","last_change":"2024-07-04T17:30:47.397516+0000","last_active":"2024-07-04T17:31:07.781881+0000","last_peered":"2024-07-04T17:31:07.781881+0000","last_clean":"2024-07-04T17:31:07.781881+0000","last_became_active":"2024-07-04T17:30:47.397119+0000","last_became_peered":"2024-07-04T17:30:47.397119+0000","last_unstale":"2024-07-04T17:31:07.781881+0000","last_undegraded":"2024-07-04T17:31:07.781881+0000","last_fullsized":"2024-07-04T17:31:07.781881+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:06:56.459812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.767338+0000","last_change":"2024-07-04T17:30:46.415510+0000","last_active":"2024-07-04T17:31:07.767338+0000","last_peered":"2024-07-04T17:31:07.767338+0000","last_clean":"2024-07-04T17:31:07.767338+0000","last_became_active":"2024-07-04T17:30:46.415113+0000","last_became_peered":"2024-07-04T17:30:46.415113+0000","last_unstale":"2024-07-04T17:31:07.767338+0000","last_undegraded":"2024-07-04T17:31:07.767338+0000","last_fullsized":"2024-07-04T17:31:07.767338+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:13:24.938223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.794126+0000","last_change":"2024-07-04T17:31:06.801119+0000","last_active":"2024-07-04T17:31:07.794126+0000","last_peered":"2024-07-04T17:31:07.794126+0000","last_clean":"2024-07-04T17:31:07.794126+0000","last_became_active":"2024-07-04T17:31:06.800582+0000","last_became_peered":"2024-07-04T17:31:06.800582+0000","last_unstale":"2024-07-04T17:31:07.794126+0000","last_undegraded":"2024-07-04T17:31:07.794126+0000","last_fullsized":"2024-07-04T17:31:07.794126+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:03:53.195699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"75'6","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.767546+0000","last_change":"2024-07-04T17:31:05.780150+0000","last_active":"2024-07-04T17:31:07.767546+0000","last_peered":"2024-07-04T17:31:07.767546+0000","last_clean":"2024-07-04T17:31:07.767546+0000","last_became_active":"2024-07-04T17:31:05.779680+0000","last_became_peered":"2024-07-04T17:31:05.779680+0000","last_unstale":"2024-07-04T17:31:07.767546+0000","last_undegraded":"2024-07-04T17:31:07.767546+0000","last_fullsized":"2024-07-04T17:31:07.767546+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:29:46.749462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.766996+0000","last_change":"2024-07-04T17:30:47.430636+0000","last_active":"2024-07-04T17:31:07.766996+0000","last_peered":"2024-07-04T17:31:07.766996+0000","last_clean":"2024-07-04T17:31:07.766996+0000","last_became_active":"2024-07-04T17:30:47.425674+0000","last_became_peered":"2024-07-04T17:30:47.425674+0000","last_unstale":"2024-07-04T17:31:07.766996+0000","last_undegraded":"2024-07-04T17:31:07.766996+0000","last_fullsized":"2024-07-04T17:31:07.766996+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:27:06.393591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.794699+0000","last_change":"2024-07-04T17:30:46.407826+0000","last_active":"2024-07-04T17:31:07.794699+0000","last_peered":"2024-07-04T17:31:07.794699+0000","last_clean":"2024-07-04T17:31:07.794699+0000","last_became_active":"2024-07-04T17:30:46.407115+0000","last_became_peered":"2024-07-04T17:30:46.407115+0000","last_unstale":"2024-07-04T17:31:07.794699+0000","last_undegraded":"2024-07-04T17:31:07.794699+0000","last_fullsized":"2024-07-04T17:31:07.794699+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:47:37.889965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.778545+0000","last_change":"2024-07-04T17:31:06.782407+0000","last_active":"2024-07-04T17:31:07.778545+0000","last_peered":"2024-07-04T17:31:07.778545+0000","last_clean":"2024-07-04T17:31:07.778545+0000","last_became_active":"2024-07-04T17:31:06.782021+0000","last_became_peered":"2024-07-04T17:31:06.782021+0000","last_unstale":"2024-07-04T17:31:07.778545+0000","last_undegraded":"2024-07-04T17:31:07.778545+0000","last_fullsized":"2024-07-04T17:31:07.778545+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:12:47.818392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.763278+0000","last_change":"2024-07-04T17:31:05.777612+0000","last_active":"2024-07-04T17:31:07.763278+0000","last_peered":"2024-07-04T17:31:07.763278+0000","last_clean":"2024-07-04T17:31:07.763278+0000","last_became_active":"2024-07-04T17:31:05.777042+0000","last_became_peered":"2024-07-04T17:31:05.777042+0000","last_unstale":"2024-07-04T17:31:07.763278+0000","last_undegraded":"2024-07-04T17:31:07.763278+0000","last_fullsized":"2024-07-04T17:31:07.763278+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:55:58.043715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.778182+0000","last_change":"2024-07-04T17:30:47.400589+0000","last_active":"2024-07-04T17:31:07.778182+0000","last_peered":"2024-07-04T17:31:07.778182+0000","last_clean":"2024-07-04T17:31:07.778182+0000","last_became_active":"2024-07-04T17:30:47.395213+0000","last_became_peered":"2024-07-04T17:30:47.395213+0000","last_unstale":"2024-07-04T17:31:07.778182+0000","last_undegraded":"2024-07-04T17:31:07.778182+0000","last_fullsized":"2024-07-04T17:31:07.778182+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:02:27.554854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"71'3","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.763321+0000","last_change":"2024-07-04T17:30:46.413645+0000","last_active":"2024-07-04T17:31:07.763321+0000","last_peered":"2024-07-04T17:31:07.763321+0000","last_clean":"2024-07-04T17:31:07.763321+0000","last_became_active":"2024-07-04T17:30:46.413314+0000","last_became_peered":"2024-07-04T17:30:46.413314+0000","last_unstale":"2024-07-04T17:31:07.763321+0000","last_undegraded":"2024-07-04T17:31:07.763321+0000","last_fullsized":"2024-07-04T17:31:07.763321+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:56:15.192711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.790044+0000","last_change":"2024-07-04T17:31:06.800046+0000","last_active":"2024-07-04T17:31:07.790044+0000","last_peered":"2024-07-04T17:31:07.790044+0000","last_clean":"2024-07-04T17:31:07.790044+0000","last_became_active":"2024-07-04T17:31:06.799696+0000","last_became_peered":"2024-07-04T17:31:06.799696+0000","last_unstale":"2024-07-04T17:31:07.790044+0000","last_undegraded":"2024-07-04T17:31:07.790044+0000","last_fullsized":"2024-07-04T17:31:07.790044+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:22:42.835518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.763050+0000","last_change":"2024-07-04T17:31:05.764774+0000","last_active":"2024-07-04T17:31:07.763050+0000","last_peered":"2024-07-04T17:31:07.763050+0000","last_clean":"2024-07-04T17:31:07.763050+0000","last_became_active":"2024-07-04T17:31:05.764469+0000","last_became_peered":"2024-07-04T17:31:05.764469+0000","last_unstale":"2024-07-04T17:31:07.763050+0000","last_undegraded":"2024-07-04T17:31:07.763050+0000","last_fullsized":"2024-07-04T17:31:07.763050+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:27:01.259327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.795107+0000","last_change":"2024-07-04T17:30:47.439341+0000","last_active":"2024-07-04T17:31:07.795107+0000","last_peered":"2024-07-04T17:31:07.795107+0000","last_clean":"2024-07-04T17:31:07.795107+0000","last_became_active":"2024-07-04T17:30:47.438643+0000","last_became_peered":"2024-07-04T17:30:47.438643+0000","last_unstale":"2024-07-04T17:31:07.795107+0000","last_undegraded":"2024-07-04T17:31:07.795107+0000","last_fullsized":"2024-07-04T17:31:07.795107+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:18:57.306251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.14","version":"72'8","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.781973+0000","last_change":"2024-07-04T17:30:46.418509+0000","last_active":"2024-07-04T17:31:07.781973+0000","last_peered":"2024-07-04T17:31:07.781973+0000","last_clean":"2024-07-04T17:31:07.781973+0000","last_became_active":"2024-07-04T17:30:46.418161+0000","last_became_peered":"2024-07-04T17:30:46.418161+0000","last_unstale":"2024-07-04T17:31:07.781973+0000","last_undegraded":"2024-07-04T17:31:07.781973+0000","last_fullsized":"2024-07-04T17:31:07.781973+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:26:41.897228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3309,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.772410+0000","last_change":"2024-07-04T17:31:06.788358+0000","last_active":"2024-07-04T17:31:07.772410+0000","last_peered":"2024-07-04T17:31:07.772410+0000","last_clean":"2024-07-04T17:31:07.772410+0000","last_became_active":"2024-07-04T17:31:06.783447+0000","last_became_peered":"2024-07-04T17:31:06.783447+0000","last_unstale":"2024-07-04T17:31:07.772410+0000","last_undegraded":"2024-07-04T17:31:07.772410+0000","last_fullsized":"2024-07-04T17:31:07.772410+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:43:44.861900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.761276+0000","last_change":"2024-07-04T17:31:05.776905+0000","last_active":"2024-07-04T17:31:07.761276+0000","last_peered":"2024-07-04T17:31:07.761276+0000","last_clean":"2024-07-04T17:31:07.761276+0000","last_became_active":"2024-07-04T17:31:05.776569+0000","last_became_peered":"2024-07-04T17:31:05.776569+0000","last_unstale":"2024-07-04T17:31:07.761276+0000","last_undegraded":"2024-07-04T17:31:07.761276+0000","last_fullsized":"2024-07-04T17:31:07.761276+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:43:22.976141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.774301+0000","last_change":"2024-07-04T17:30:47.388099+0000","last_active":"2024-07-04T17:31:07.774301+0000","last_peered":"2024-07-04T17:31:07.774301+0000","last_clean":"2024-07-04T17:31:07.774301+0000","last_became_active":"2024-07-04T17:30:47.387785+0000","last_became_peered":"2024-07-04T17:30:47.387785+0000","last_unstale":"2024-07-04T17:31:07.774301+0000","last_undegraded":"2024-07-04T17:31:07.774301+0000","last_fullsized":"2024-07-04T17:31:07.774301+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:53:19.566358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"71'2","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.767658+0000","last_change":"2024-07-04T17:30:46.404600+0000","last_active":"2024-07-04T17:31:07.767658+0000","last_peered":"2024-07-04T17:31:07.767658+0000","last_clean":"2024-07-04T17:31:07.767658+0000","last_became_active":"2024-07-04T17:30:46.404314+0000","last_became_peered":"2024-07-04T17:30:46.404314+0000","last_unstale":"2024-07-04T17:31:07.767658+0000","last_undegraded":"2024-07-04T17:31:07.767658+0000","last_fullsized":"2024-07-04T17:31:07.767658+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:25:07.446781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.786819+0000","last_change":"2024-07-04T17:31:06.787869+0000","last_active":"2024-07-04T17:31:07.786819+0000","last_peered":"2024-07-04T17:31:07.786819+0000","last_clean":"2024-07-04T17:31:07.786819+0000","last_became_active":"2024-07-04T17:31:06.782681+0000","last_became_peered":"2024-07-04T17:31:06.782681+0000","last_unstale":"2024-07-04T17:31:07.786819+0000","last_undegraded":"2024-07-04T17:31:07.786819+0000","last_fullsized":"2024-07-04T17:31:07.786819+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:39:53.589626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"75'8","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.766499+0000","last_change":"2024-07-04T17:31:05.762848+0000","last_active":"2024-07-04T17:31:07.766499+0000","last_peered":"2024-07-04T17:31:07.766499+0000","last_clean":"2024-07-04T17:31:07.766499+0000","last_became_active":"2024-07-04T17:31:05.762475+0000","last_became_peered":"2024-07-04T17:31:05.762475+0000","last_unstale":"2024-07-04T17:31:07.766499+0000","last_undegraded":"2024-07-04T17:31:07.766499+0000","last_fullsized":"2024-07-04T17:31:07.766499+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:22:19.353448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.767875+0000","last_change":"2024-07-04T17:30:47.387822+0000","last_active":"2024-07-04T17:31:07.767875+0000","last_peered":"2024-07-04T17:31:07.767875+0000","last_clean":"2024-07-04T17:31:07.767875+0000","last_became_active":"2024-07-04T17:30:47.387451+0000","last_became_peered":"2024-07-04T17:30:47.387451+0000","last_unstale":"2024-07-04T17:31:07.767875+0000","last_undegraded":"2024-07-04T17:31:07.767875+0000","last_fullsized":"2024-07-04T17:31:07.767875+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:46:22.229171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.774635+0000","last_change":"2024-07-04T17:30:46.410149+0000","last_active":"2024-07-04T17:31:07.774635+0000","last_peered":"2024-07-04T17:31:07.774635+0000","last_clean":"2024-07-04T17:31:07.774635+0000","last_became_active":"2024-07-04T17:30:46.409816+0000","last_became_peered":"2024-07-04T17:30:46.409816+0000","last_unstale":"2024-07-04T17:31:07.774635+0000","last_undegraded":"2024-07-04T17:31:07.774635+0000","last_fullsized":"2024-07-04T17:31:07.774635+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:51:54.551650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.774150+0000","last_change":"2024-07-04T17:31:06.808506+0000","last_active":"2024-07-04T17:31:07.774150+0000","last_peered":"2024-07-04T17:31:07.774150+0000","last_clean":"2024-07-04T17:31:07.774150+0000","last_became_active":"2024-07-04T17:31:06.808168+0000","last_became_peered":"2024-07-04T17:31:06.808168+0000","last_unstale":"2024-07-04T17:31:07.774150+0000","last_undegraded":"2024-07-04T17:31:07.774150+0000","last_fullsized":"2024-07-04T17:31:07.774150+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:20:28.052433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.13","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.787958+0000","last_change":"2024-07-04T17:31:05.786206+0000","last_active":"2024-07-04T17:31:07.787958+0000","last_peered":"2024-07-04T17:31:07.787958+0000","last_clean":"2024-07-04T17:31:07.787958+0000","last_became_active":"2024-07-04T17:31:05.785869+0000","last_became_peered":"2024-07-04T17:31:05.785869+0000","last_unstale":"2024-07-04T17:31:07.787958+0000","last_undegraded":"2024-07-04T17:31:07.787958+0000","last_fullsized":"2024-07-04T17:31:07.787958+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:57:22.818904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.762844+0000","last_change":"2024-07-04T17:30:47.431466+0000","last_active":"2024-07-04T17:31:07.762844+0000","last_peered":"2024-07-04T17:31:07.762844+0000","last_clean":"2024-07-04T17:31:07.762844+0000","last_became_active":"2024-07-04T17:30:47.430961+0000","last_became_peered":"2024-07-04T17:30:47.430961+0000","last_unstale":"2024-07-04T17:31:07.762844+0000","last_undegraded":"2024-07-04T17:31:07.762844+0000","last_fullsized":"2024-07-04T17:31:07.762844+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:50:08.536272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.787553+0000","last_change":"2024-07-04T17:30:46.396182+0000","last_active":"2024-07-04T17:31:07.787553+0000","last_peered":"2024-07-04T17:31:07.787553+0000","last_clean":"2024-07-04T17:31:07.787553+0000","last_became_active":"2024-07-04T17:30:46.393987+0000","last_became_peered":"2024-07-04T17:30:46.393987+0000","last_unstale":"2024-07-04T17:31:07.787553+0000","last_undegraded":"2024-07-04T17:31:07.787553+0000","last_fullsized":"2024-07-04T17:31:07.787553+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:44:44.996919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.766725+0000","last_change":"2024-07-04T17:31:06.802414+0000","last_active":"2024-07-04T17:31:07.766725+0000","last_peered":"2024-07-04T17:31:07.766725+0000","last_clean":"2024-07-04T17:31:07.766725+0000","last_became_active":"2024-07-04T17:31:06.802058+0000","last_became_peered":"2024-07-04T17:31:06.802058+0000","last_unstale":"2024-07-04T17:31:07.766725+0000","last_undegraded":"2024-07-04T17:31:07.766725+0000","last_fullsized":"2024-07-04T17:31:07.766725+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:40:42.134464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.790571+0000","last_change":"2024-07-04T17:31:05.786657+0000","last_active":"2024-07-04T17:31:07.790571+0000","last_peered":"2024-07-04T17:31:07.790571+0000","last_clean":"2024-07-04T17:31:07.790571+0000","last_became_active":"2024-07-04T17:31:05.785951+0000","last_became_peered":"2024-07-04T17:31:05.785951+0000","last_unstale":"2024-07-04T17:31:07.790571+0000","last_undegraded":"2024-07-04T17:31:07.790571+0000","last_fullsized":"2024-07-04T17:31:07.790571+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:55:58.043715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.761415+0000","last_change":"2024-07-04T17:30:47.395226+0000","last_active":"2024-07-04T17:31:07.761415+0000","last_peered":"2024-07-04T17:31:07.761415+0000","last_clean":"2024-07-04T17:31:07.761415+0000","last_became_active":"2024-07-04T17:30:47.394913+0000","last_became_peered":"2024-07-04T17:30:47.394913+0000","last_unstale":"2024-07-04T17:31:07.761415+0000","last_undegraded":"2024-07-04T17:31:07.761415+0000","last_fullsized":"2024-07-04T17:31:07.761415+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:31:15.378585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"71'8","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.773350+0000","last_change":"2024-07-04T17:30:46.398618+0000","last_active":"2024-07-04T17:31:07.773350+0000","last_peered":"2024-07-04T17:31:07.773350+0000","last_clean":"2024-07-04T17:31:07.773350+0000","last_became_active":"2024-07-04T17:30:46.398274+0000","last_became_peered":"2024-07-04T17:30:46.398274+0000","last_unstale":"2024-07-04T17:31:07.773350+0000","last_undegraded":"2024-07-04T17:31:07.773350+0000","last_fullsized":"2024-07-04T17:31:07.773350+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:13:50.002315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.789924+0000","last_change":"2024-07-04T17:31:06.807190+0000","last_active":"2024-07-04T17:31:07.789924+0000","last_peered":"2024-07-04T17:31:07.789924+0000","last_clean":"2024-07-04T17:31:07.789924+0000","last_became_active":"2024-07-04T17:31:06.806646+0000","last_became_peered":"2024-07-04T17:31:06.806646+0000","last_unstale":"2024-07-04T17:31:07.789924+0000","last_undegraded":"2024-07-04T17:31:07.789924+0000","last_fullsized":"2024-07-04T17:31:07.789924+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:10:59.480346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.778811+0000","last_change":"2024-07-04T17:31:05.789959+0000","last_active":"2024-07-04T17:31:07.778811+0000","last_peered":"2024-07-04T17:31:07.778811+0000","last_clean":"2024-07-04T17:31:07.778811+0000","last_became_active":"2024-07-04T17:31:05.789081+0000","last_became_peered":"2024-07-04T17:31:05.789081+0000","last_unstale":"2024-07-04T17:31:07.778811+0000","last_undegraded":"2024-07-04T17:31:07.778811+0000","last_fullsized":"2024-07-04T17:31:07.778811+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:57:23.773041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.781996+0000","last_change":"2024-07-04T17:30:47.397472+0000","last_active":"2024-07-04T17:31:07.781996+0000","last_peered":"2024-07-04T17:31:07.781996+0000","last_clean":"2024-07-04T17:31:07.781996+0000","last_became_active":"2024-07-04T17:30:47.396989+0000","last_became_peered":"2024-07-04T17:30:47.396989+0000","last_unstale":"2024-07-04T17:31:07.781996+0000","last_undegraded":"2024-07-04T17:31:07.781996+0000","last_fullsized":"2024-07-04T17:31:07.781996+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:40:22.762798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"71'1","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.781675+0000","last_change":"2024-07-04T17:30:46.412075+0000","last_active":"2024-07-04T17:31:07.781675+0000","last_peered":"2024-07-04T17:31:07.781675+0000","last_clean":"2024-07-04T17:31:07.781675+0000","last_became_active":"2024-07-04T17:30:46.411727+0000","last_became_peered":"2024-07-04T17:30:46.411727+0000","last_unstale":"2024-07-04T17:31:07.781675+0000","last_undegraded":"2024-07-04T17:31:07.781675+0000","last_fullsized":"2024-07-04T17:31:07.781675+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:12:43.261982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.762638+0000","last_change":"2024-07-04T17:31:06.804419+0000","last_active":"2024-07-04T17:31:07.762638+0000","last_peered":"2024-07-04T17:31:07.762638+0000","last_clean":"2024-07-04T17:31:07.762638+0000","last_became_active":"2024-07-04T17:31:06.804096+0000","last_became_peered":"2024-07-04T17:31:06.804096+0000","last_unstale":"2024-07-04T17:31:07.762638+0000","last_undegraded":"2024-07-04T17:31:07.762638+0000","last_fullsized":"2024-07-04T17:31:07.762638+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:51:35.870044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.775419+0000","last_change":"2024-07-04T17:31:05.787590+0000","last_active":"2024-07-04T17:31:07.775419+0000","last_peered":"2024-07-04T17:31:07.775419+0000","last_clean":"2024-07-04T17:31:07.775419+0000","last_became_active":"2024-07-04T17:31:05.787211+0000","last_became_peered":"2024-07-04T17:31:05.787211+0000","last_unstale":"2024-07-04T17:31:07.775419+0000","last_undegraded":"2024-07-04T17:31:07.775419+0000","last_fullsized":"2024-07-04T17:31:07.775419+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:47:47.359180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.794494+0000","last_change":"2024-07-04T17:30:47.436755+0000","last_active":"2024-07-04T17:31:07.794494+0000","last_peered":"2024-07-04T17:31:07.794494+0000","last_clean":"2024-07-04T17:31:07.794494+0000","last_became_active":"2024-07-04T17:30:47.436409+0000","last_became_peered":"2024-07-04T17:30:47.436409+0000","last_unstale":"2024-07-04T17:31:07.794494+0000","last_undegraded":"2024-07-04T17:31:07.794494+0000","last_fullsized":"2024-07-04T17:31:07.794494+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:13:51.006598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"71'2","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.782115+0000","last_change":"2024-07-04T17:30:46.409103+0000","last_active":"2024-07-04T17:31:07.782115+0000","last_peered":"2024-07-04T17:31:07.782115+0000","last_clean":"2024-07-04T17:31:07.782115+0000","last_became_active":"2024-07-04T17:30:46.406340+0000","last_became_peered":"2024-07-04T17:30:46.406340+0000","last_unstale":"2024-07-04T17:31:07.782115+0000","last_undegraded":"2024-07-04T17:31:07.782115+0000","last_fullsized":"2024-07-04T17:31:07.782115+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:43:03.614042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.790403+0000","last_change":"2024-07-04T17:31:06.810247+0000","last_active":"2024-07-04T17:31:07.790403+0000","last_peered":"2024-07-04T17:31:07.790403+0000","last_clean":"2024-07-04T17:31:07.790403+0000","last_became_active":"2024-07-04T17:31:06.809469+0000","last_became_peered":"2024-07-04T17:31:06.809469+0000","last_unstale":"2024-07-04T17:31:07.790403+0000","last_undegraded":"2024-07-04T17:31:07.790403+0000","last_fullsized":"2024-07-04T17:31:07.790403+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:16:13.492550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.763402+0000","last_change":"2024-07-04T17:31:05.779260+0000","last_active":"2024-07-04T17:31:07.763402+0000","last_peered":"2024-07-04T17:31:07.763402+0000","last_clean":"2024-07-04T17:31:07.763402+0000","last_became_active":"2024-07-04T17:31:05.778992+0000","last_became_peered":"2024-07-04T17:31:05.778992+0000","last_unstale":"2024-07-04T17:31:07.763402+0000","last_undegraded":"2024-07-04T17:31:07.763402+0000","last_fullsized":"2024-07-04T17:31:07.763402+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:12:25.215711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.778944+0000","last_change":"2024-07-04T17:30:47.394855+0000","last_active":"2024-07-04T17:31:07.778944+0000","last_peered":"2024-07-04T17:31:07.778944+0000","last_clean":"2024-07-04T17:31:07.778944+0000","last_became_active":"2024-07-04T17:30:47.393038+0000","last_became_peered":"2024-07-04T17:30:47.393038+0000","last_unstale":"2024-07-04T17:31:07.778944+0000","last_undegraded":"2024-07-04T17:31:07.778944+0000","last_fullsized":"2024-07-04T17:31:07.778944+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:25:35.950854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.782352+0000","last_change":"2024-07-04T17:30:46.409923+0000","last_active":"2024-07-04T17:31:07.782352+0000","last_peered":"2024-07-04T17:31:07.782352+0000","last_clean":"2024-07-04T17:31:07.782352+0000","last_became_active":"2024-07-04T17:30:46.409591+0000","last_became_peered":"2024-07-04T17:30:46.409591+0000","last_unstale":"2024-07-04T17:31:07.782352+0000","last_undegraded":"2024-07-04T17:31:07.782352+0000","last_fullsized":"2024-07-04T17:31:07.782352+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:12:05.853612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.786740+0000","last_change":"2024-07-04T17:31:06.776374+0000","last_active":"2024-07-04T17:31:07.786740+0000","last_peered":"2024-07-04T17:31:07.786740+0000","last_clean":"2024-07-04T17:31:07.786740+0000","last_became_active":"2024-07-04T17:31:06.776006+0000","last_became_peered":"2024-07-04T17:31:06.776006+0000","last_unstale":"2024-07-04T17:31:07.786740+0000","last_undegraded":"2024-07-04T17:31:07.786740+0000","last_fullsized":"2024-07-04T17:31:07.786740+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:07:00.246635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.772851+0000","last_change":"2024-07-04T17:31:05.776628+0000","last_active":"2024-07-04T17:31:07.772851+0000","last_peered":"2024-07-04T17:31:07.772851+0000","last_clean":"2024-07-04T17:31:07.772851+0000","last_became_active":"2024-07-04T17:31:05.776232+0000","last_became_peered":"2024-07-04T17:31:05.776232+0000","last_unstale":"2024-07-04T17:31:07.772851+0000","last_undegraded":"2024-07-04T17:31:07.772851+0000","last_fullsized":"2024-07-04T17:31:07.772851+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:37:15.837542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.794890+0000","last_change":"2024-07-04T17:30:47.419189+0000","last_active":"2024-07-04T17:31:07.794890+0000","last_peered":"2024-07-04T17:31:07.794890+0000","last_clean":"2024-07-04T17:31:07.794890+0000","last_became_active":"2024-07-04T17:30:47.417882+0000","last_became_peered":"2024-07-04T17:30:47.417882+0000","last_unstale":"2024-07-04T17:31:07.794890+0000","last_undegraded":"2024-07-04T17:31:07.794890+0000","last_fullsized":"2024-07-04T17:31:07.794890+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:28:21.769409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"71'2","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.775042+0000","last_change":"2024-07-04T17:30:46.412068+0000","last_active":"2024-07-04T17:31:07.775042+0000","last_peered":"2024-07-04T17:31:07.775042+0000","last_clean":"2024-07-04T17:31:07.775042+0000","last_became_active":"2024-07-04T17:30:46.411711+0000","last_became_peered":"2024-07-04T17:30:46.411711+0000","last_unstale":"2024-07-04T17:31:07.775042+0000","last_undegraded":"2024-07-04T17:31:07.775042+0000","last_fullsized":"2024-07-04T17:31:07.775042+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:42:52.442755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.786818+0000","last_change":"2024-07-04T17:31:06.791069+0000","last_active":"2024-07-04T17:31:07.786818+0000","last_peered":"2024-07-04T17:31:07.786818+0000","last_clean":"2024-07-04T17:31:07.786818+0000","last_became_active":"2024-07-04T17:31:06.790763+0000","last_became_peered":"2024-07-04T17:31:06.790763+0000","last_unstale":"2024-07-04T17:31:07.786818+0000","last_undegraded":"2024-07-04T17:31:07.786818+0000","last_fullsized":"2024-07-04T17:31:07.786818+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:31:59.177767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.772636+0000","last_change":"2024-07-04T17:31:05.769021+0000","last_active":"2024-07-04T17:31:07.772636+0000","last_peered":"2024-07-04T17:31:07.772636+0000","last_clean":"2024-07-04T17:31:07.772636+0000","last_became_active":"2024-07-04T17:31:05.768410+0000","last_became_peered":"2024-07-04T17:31:05.768410+0000","last_unstale":"2024-07-04T17:31:07.772636+0000","last_undegraded":"2024-07-04T17:31:07.772636+0000","last_fullsized":"2024-07-04T17:31:07.772636+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:42:26.235625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.789653+0000","last_change":"2024-07-04T17:30:47.436121+0000","last_active":"2024-07-04T17:31:07.789653+0000","last_peered":"2024-07-04T17:31:07.789653+0000","last_clean":"2024-07-04T17:31:07.789653+0000","last_became_active":"2024-07-04T17:30:47.435790+0000","last_became_peered":"2024-07-04T17:30:47.435790+0000","last_unstale":"2024-07-04T17:31:07.789653+0000","last_undegraded":"2024-07-04T17:31:07.789653+0000","last_fullsized":"2024-07-04T17:31:07.789653+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:25:40.409565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.795071+0000","last_change":"2024-07-04T17:30:46.408582+0000","last_active":"2024-07-04T17:31:07.795071+0000","last_peered":"2024-07-04T17:31:07.795071+0000","last_clean":"2024-07-04T17:31:07.795071+0000","last_became_active":"2024-07-04T17:30:46.408297+0000","last_became_peered":"2024-07-04T17:30:46.408297+0000","last_unstale":"2024-07-04T17:31:07.795071+0000","last_undegraded":"2024-07-04T17:31:07.795071+0000","last_fullsized":"2024-07-04T17:31:07.795071+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:48:21.667965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.794727+0000","last_change":"2024-07-04T17:31:06.782258+0000","last_active":"2024-07-04T17:31:07.794727+0000","last_peered":"2024-07-04T17:31:07.794727+0000","last_clean":"2024-07-04T17:31:07.794727+0000","last_became_active":"2024-07-04T17:31:06.780838+0000","last_became_peered":"2024-07-04T17:31:06.780838+0000","last_unstale":"2024-07-04T17:31:07.794727+0000","last_undegraded":"2024-07-04T17:31:07.794727+0000","last_fullsized":"2024-07-04T17:31:07.794727+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:24:23.345139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.773971+0000","last_change":"2024-07-04T17:31:05.787604+0000","last_active":"2024-07-04T17:31:07.773971+0000","last_peered":"2024-07-04T17:31:07.773971+0000","last_clean":"2024-07-04T17:31:07.773971+0000","last_became_active":"2024-07-04T17:31:05.787243+0000","last_became_peered":"2024-07-04T17:31:05.787243+0000","last_unstale":"2024-07-04T17:31:07.773971+0000","last_undegraded":"2024-07-04T17:31:07.773971+0000","last_fullsized":"2024-07-04T17:31:07.773971+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:24:43.565343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.790522+0000","last_change":"2024-07-04T17:30:47.435916+0000","last_active":"2024-07-04T17:31:07.790522+0000","last_peered":"2024-07-04T17:31:07.790522+0000","last_clean":"2024-07-04T17:31:07.790522+0000","last_became_active":"2024-07-04T17:30:47.435562+0000","last_became_peered":"2024-07-04T17:30:47.435562+0000","last_unstale":"2024-07-04T17:31:07.790522+0000","last_undegraded":"2024-07-04T17:31:07.790522+0000","last_fullsized":"2024-07-04T17:31:07.790522+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:41:29.873067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.787644+0000","last_change":"2024-07-04T17:30:46.394311+0000","last_active":"2024-07-04T17:31:07.787644+0000","last_peered":"2024-07-04T17:31:07.787644+0000","last_clean":"2024-07-04T17:31:07.787644+0000","last_became_active":"2024-07-04T17:30:46.393593+0000","last_became_peered":"2024-07-04T17:30:46.393593+0000","last_unstale":"2024-07-04T17:31:07.787644+0000","last_undegraded":"2024-07-04T17:31:07.787644+0000","last_fullsized":"2024-07-04T17:31:07.787644+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:48:17.986899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.788085+0000","last_change":"2024-07-04T17:31:06.776805+0000","last_active":"2024-07-04T17:31:07.788085+0000","last_peered":"2024-07-04T17:31:07.788085+0000","last_clean":"2024-07-04T17:31:07.788085+0000","last_became_active":"2024-07-04T17:31:06.776110+0000","last_became_peered":"2024-07-04T17:31:06.776110+0000","last_unstale":"2024-07-04T17:31:07.788085+0000","last_undegraded":"2024-07-04T17:31:07.788085+0000","last_fullsized":"2024-07-04T17:31:07.788085+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:24:00.353280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.790576+0000","last_change":"2024-07-04T17:31:05.792346+0000","last_active":"2024-07-04T17:31:07.790576+0000","last_peered":"2024-07-04T17:31:07.790576+0000","last_clean":"2024-07-04T17:31:07.790576+0000","last_became_active":"2024-07-04T17:31:05.790572+0000","last_became_peered":"2024-07-04T17:31:05.790572+0000","last_unstale":"2024-07-04T17:31:07.790576+0000","last_undegraded":"2024-07-04T17:31:07.790576+0000","last_fullsized":"2024-07-04T17:31:07.790576+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:18:07.499429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.772979+0000","last_change":"2024-07-04T17:30:47.420543+0000","last_active":"2024-07-04T17:31:07.772979+0000","last_peered":"2024-07-04T17:31:07.772979+0000","last_clean":"2024-07-04T17:31:07.772979+0000","last_became_active":"2024-07-04T17:30:47.416196+0000","last_became_peered":"2024-07-04T17:30:47.416196+0000","last_unstale":"2024-07-04T17:31:07.772979+0000","last_undegraded":"2024-07-04T17:31:07.772979+0000","last_fullsized":"2024-07-04T17:31:07.772979+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:50:20.456752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.790659+0000","last_change":"2024-07-04T17:30:46.408040+0000","last_active":"2024-07-04T17:31:07.790659+0000","last_peered":"2024-07-04T17:31:07.790659+0000","last_clean":"2024-07-04T17:31:07.790659+0000","last_became_active":"2024-07-04T17:30:46.407167+0000","last_became_peered":"2024-07-04T17:30:46.407167+0000","last_unstale":"2024-07-04T17:31:07.790659+0000","last_undegraded":"2024-07-04T17:31:07.790659+0000","last_fullsized":"2024-07-04T17:31:07.790659+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:46:18.222946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.767255+0000","last_change":"2024-07-04T17:31:06.776936+0000","last_active":"2024-07-04T17:31:07.767255+0000","last_peered":"2024-07-04T17:31:07.767255+0000","last_clean":"2024-07-04T17:31:07.767255+0000","last_became_active":"2024-07-04T17:31:06.776635+0000","last_became_peered":"2024-07-04T17:31:06.776635+0000","last_unstale":"2024-07-04T17:31:07.767255+0000","last_undegraded":"2024-07-04T17:31:07.767255+0000","last_fullsized":"2024-07-04T17:31:07.767255+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:07:00.246635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.778647+0000","last_change":"2024-07-04T17:31:05.789688+0000","last_active":"2024-07-04T17:31:07.778647+0000","last_peered":"2024-07-04T17:31:07.778647+0000","last_clean":"2024-07-04T17:31:07.778647+0000","last_became_active":"2024-07-04T17:31:05.788654+0000","last_became_peered":"2024-07-04T17:31:05.788654+0000","last_unstale":"2024-07-04T17:31:07.778647+0000","last_undegraded":"2024-07-04T17:31:07.778647+0000","last_fullsized":"2024-07-04T17:31:07.778647+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:13:25.948396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.782081+0000","last_change":"2024-07-04T17:30:47.397337+0000","last_active":"2024-07-04T17:31:07.782081+0000","last_peered":"2024-07-04T17:31:07.782081+0000","last_clean":"2024-07-04T17:31:07.782081+0000","last_became_active":"2024-07-04T17:30:47.396668+0000","last_became_peered":"2024-07-04T17:30:47.396668+0000","last_unstale":"2024-07-04T17:31:07.782081+0000","last_undegraded":"2024-07-04T17:31:07.782081+0000","last_fullsized":"2024-07-04T17:31:07.782081+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:44:37.013444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"71'2","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.767545+0000","last_change":"2024-07-04T17:30:46.414092+0000","last_active":"2024-07-04T17:31:07.767545+0000","last_peered":"2024-07-04T17:31:07.767545+0000","last_clean":"2024-07-04T17:31:07.767545+0000","last_became_active":"2024-07-04T17:30:46.413746+0000","last_became_peered":"2024-07-04T17:30:46.413746+0000","last_unstale":"2024-07-04T17:31:07.767545+0000","last_undegraded":"2024-07-04T17:31:07.767545+0000","last_fullsized":"2024-07-04T17:31:07.767545+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:56:21.477169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.794252+0000","last_change":"2024-07-04T17:31:06.801233+0000","last_active":"2024-07-04T17:31:07.794252+0000","last_peered":"2024-07-04T17:31:07.794252+0000","last_clean":"2024-07-04T17:31:07.794252+0000","last_became_active":"2024-07-04T17:31:06.800829+0000","last_became_peered":"2024-07-04T17:31:06.800829+0000","last_unstale":"2024-07-04T17:31:07.794252+0000","last_undegraded":"2024-07-04T17:31:07.794252+0000","last_fullsized":"2024-07-04T17:31:07.794252+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:42:27.249476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.790076+0000","last_change":"2024-07-04T17:31:05.788028+0000","last_active":"2024-07-04T17:31:07.790076+0000","last_peered":"2024-07-04T17:31:07.790076+0000","last_clean":"2024-07-04T17:31:07.790076+0000","last_became_active":"2024-07-04T17:31:05.785915+0000","last_became_peered":"2024-07-04T17:31:05.785915+0000","last_unstale":"2024-07-04T17:31:07.790076+0000","last_undegraded":"2024-07-04T17:31:07.790076+0000","last_fullsized":"2024-07-04T17:31:07.790076+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:34:20.307541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.767113+0000","last_change":"2024-07-04T17:30:47.388955+0000","last_active":"2024-07-04T17:31:07.767113+0000","last_peered":"2024-07-04T17:31:07.767113+0000","last_clean":"2024-07-04T17:31:07.767113+0000","last_became_active":"2024-07-04T17:30:47.387588+0000","last_became_peered":"2024-07-04T17:30:47.387588+0000","last_unstale":"2024-07-04T17:31:07.767113+0000","last_undegraded":"2024-07-04T17:31:07.767113+0000","last_fullsized":"2024-07-04T17:31:07.767113+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:58:22.418379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.6","version":"72'1","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.767486+0000","last_change":"2024-07-04T17:30:46.405443+0000","last_active":"2024-07-04T17:31:07.767486+0000","last_peered":"2024-07-04T17:31:07.767486+0000","last_clean":"2024-07-04T17:31:07.767486+0000","last_became_active":"2024-07-04T17:30:46.405066+0000","last_became_peered":"2024-07-04T17:30:46.405066+0000","last_unstale":"2024-07-04T17:31:07.767486+0000","last_undegraded":"2024-07-04T17:31:07.767486+0000","last_fullsized":"2024-07-04T17:31:07.767486+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:34:24.763771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.778573+0000","last_change":"2024-07-04T17:31:06.784538+0000","last_active":"2024-07-04T17:31:07.778573+0000","last_peered":"2024-07-04T17:31:07.778573+0000","last_clean":"2024-07-04T17:31:07.778573+0000","last_became_active":"2024-07-04T17:31:06.784209+0000","last_became_peered":"2024-07-04T17:31:06.784209+0000","last_unstale":"2024-07-04T17:31:07.778573+0000","last_undegraded":"2024-07-04T17:31:07.778573+0000","last_fullsized":"2024-07-04T17:31:07.778573+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:42:42.606843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"75'2","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.763547+0000","last_change":"2024-07-04T17:31:05.777491+0000","last_active":"2024-07-04T17:31:07.763547+0000","last_peered":"2024-07-04T17:31:07.763547+0000","last_clean":"2024-07-04T17:31:07.763547+0000","last_became_active":"2024-07-04T17:31:05.776618+0000","last_became_peered":"2024-07-04T17:31:05.776618+0000","last_unstale":"2024-07-04T17:31:07.763547+0000","last_undegraded":"2024-07-04T17:31:07.763547+0000","last_fullsized":"2024-07-04T17:31:07.763547+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:34:20.307541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.778532+0000","last_change":"2024-07-04T17:30:47.420498+0000","last_active":"2024-07-04T17:31:07.778532+0000","last_peered":"2024-07-04T17:31:07.778532+0000","last_clean":"2024-07-04T17:31:07.778532+0000","last_became_active":"2024-07-04T17:30:47.416342+0000","last_became_peered":"2024-07-04T17:30:47.416342+0000","last_unstale":"2024-07-04T17:31:07.778532+0000","last_undegraded":"2024-07-04T17:31:07.778532+0000","last_fullsized":"2024-07-04T17:31:07.778532+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:09:18.581049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.781678+0000","last_change":"2024-07-04T17:30:46.401530+0000","last_active":"2024-07-04T17:31:07.781678+0000","last_peered":"2024-07-04T17:31:07.781678+0000","last_clean":"2024-07-04T17:31:07.781678+0000","last_became_active":"2024-07-04T17:30:46.401058+0000","last_became_peered":"2024-07-04T17:30:46.401058+0000","last_unstale":"2024-07-04T17:31:07.781678+0000","last_undegraded":"2024-07-04T17:31:07.781678+0000","last_fullsized":"2024-07-04T17:31:07.781678+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:29:25.882528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.763159+0000","last_change":"2024-07-04T17:31:06.802590+0000","last_active":"2024-07-04T17:31:07.763159+0000","last_peered":"2024-07-04T17:31:07.763159+0000","last_clean":"2024-07-04T17:31:07.763159+0000","last_became_active":"2024-07-04T17:31:06.801862+0000","last_became_peered":"2024-07-04T17:31:06.801862+0000","last_unstale":"2024-07-04T17:31:07.763159+0000","last_undegraded":"2024-07-04T17:31:07.763159+0000","last_fullsized":"2024-07-04T17:31:07.763159+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:47:32.422618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"76'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.781881+0000","last_change":"2024-07-04T17:31:05.788840+0000","last_active":"2024-07-04T17:31:07.781881+0000","last_peered":"2024-07-04T17:31:07.781881+0000","last_clean":"2024-07-04T17:31:07.781881+0000","last_became_active":"2024-07-04T17:31:05.788410+0000","last_became_peered":"2024-07-04T17:31:05.788410+0000","last_unstale":"2024-07-04T17:31:07.781881+0000","last_undegraded":"2024-07-04T17:31:07.781881+0000","last_fullsized":"2024-07-04T17:31:07.781881+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:28:10.070066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.795463+0000","last_change":"2024-07-04T17:30:47.439494+0000","last_active":"2024-07-04T17:31:07.795463+0000","last_peered":"2024-07-04T17:31:07.795463+0000","last_clean":"2024-07-04T17:31:07.795463+0000","last_became_active":"2024-07-04T17:30:47.438655+0000","last_became_peered":"2024-07-04T17:30:47.438655+0000","last_unstale":"2024-07-04T17:31:07.795463+0000","last_undegraded":"2024-07-04T17:31:07.795463+0000","last_fullsized":"2024-07-04T17:31:07.795463+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:37:58.800114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.790740+0000","last_change":"2024-07-04T17:30:46.398565+0000","last_active":"2024-07-04T17:31:07.790740+0000","last_peered":"2024-07-04T17:31:07.790740+0000","last_clean":"2024-07-04T17:31:07.790740+0000","last_became_active":"2024-07-04T17:30:46.394788+0000","last_became_peered":"2024-07-04T17:30:46.394788+0000","last_unstale":"2024-07-04T17:31:07.790740+0000","last_undegraded":"2024-07-04T17:31:07.790740+0000","last_fullsized":"2024-07-04T17:31:07.790740+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:58:03.149737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.787387+0000","last_change":"2024-07-04T17:31:06.795245+0000","last_active":"2024-07-04T17:31:07.787387+0000","last_peered":"2024-07-04T17:31:07.787387+0000","last_clean":"2024-07-04T17:31:07.787387+0000","last_became_active":"2024-07-04T17:31:06.794908+0000","last_became_peered":"2024-07-04T17:31:06.794908+0000","last_unstale":"2024-07-04T17:31:07.787387+0000","last_undegraded":"2024-07-04T17:31:07.787387+0000","last_fullsized":"2024-07-04T17:31:07.787387+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:43:09.190414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.761111+0000","last_change":"2024-07-04T17:31:05.776324+0000","last_active":"2024-07-04T17:31:07.761111+0000","last_peered":"2024-07-04T17:31:07.761111+0000","last_clean":"2024-07-04T17:31:07.761111+0000","last_became_active":"2024-07-04T17:31:05.775829+0000","last_became_peered":"2024-07-04T17:31:05.775829+0000","last_unstale":"2024-07-04T17:31:07.761111+0000","last_undegraded":"2024-07-04T17:31:07.761111+0000","last_fullsized":"2024-07-04T17:31:07.761111+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:13:02.624081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.767809+0000","last_change":"2024-07-04T17:30:47.426754+0000","last_active":"2024-07-04T17:31:07.767809+0000","last_peered":"2024-07-04T17:31:07.767809+0000","last_clean":"2024-07-04T17:31:07.767809+0000","last_became_active":"2024-07-04T17:30:47.425712+0000","last_became_peered":"2024-07-04T17:30:47.425712+0000","last_unstale":"2024-07-04T17:31:07.767809+0000","last_undegraded":"2024-07-04T17:31:07.767809+0000","last_fullsized":"2024-07-04T17:31:07.767809+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:07:15.865385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.775160+0000","last_change":"2024-07-04T17:30:46.401607+0000","last_active":"2024-07-04T17:31:07.775160+0000","last_peered":"2024-07-04T17:31:07.775160+0000","last_clean":"2024-07-04T17:31:07.775160+0000","last_became_active":"2024-07-04T17:30:46.401314+0000","last_became_peered":"2024-07-04T17:30:46.401314+0000","last_unstale":"2024-07-04T17:31:07.775160+0000","last_undegraded":"2024-07-04T17:31:07.775160+0000","last_fullsized":"2024-07-04T17:31:07.775160+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:21:59.991349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.761136+0000","last_change":"2024-07-04T17:31:06.804835+0000","last_active":"2024-07-04T17:31:07.761136+0000","last_peered":"2024-07-04T17:31:07.761136+0000","last_clean":"2024-07-04T17:31:07.761136+0000","last_became_active":"2024-07-04T17:31:06.804252+0000","last_became_peered":"2024-07-04T17:31:06.804252+0000","last_unstale":"2024-07-04T17:31:07.761136+0000","last_undegraded":"2024-07-04T17:31:07.761136+0000","last_fullsized":"2024-07-04T17:31:07.761136+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:02:46.926520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.787718+0000","last_change":"2024-07-04T17:31:05.788550+0000","last_active":"2024-07-04T17:31:07.787718+0000","last_peered":"2024-07-04T17:31:07.787718+0000","last_clean":"2024-07-04T17:31:07.787718+0000","last_became_active":"2024-07-04T17:31:05.788144+0000","last_became_peered":"2024-07-04T17:31:05.788144+0000","last_unstale":"2024-07-04T17:31:07.787718+0000","last_undegraded":"2024-07-04T17:31:07.787718+0000","last_fullsized":"2024-07-04T17:31:07.787718+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:08:45.260384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.774799+0000","last_change":"2024-07-04T17:30:47.424977+0000","last_active":"2024-07-04T17:31:07.774799+0000","last_peered":"2024-07-04T17:31:07.774799+0000","last_clean":"2024-07-04T17:31:07.774799+0000","last_became_active":"2024-07-04T17:30:47.416042+0000","last_became_peered":"2024-07-04T17:30:47.416042+0000","last_unstale":"2024-07-04T17:31:07.774799+0000","last_undegraded":"2024-07-04T17:31:07.774799+0000","last_fullsized":"2024-07-04T17:31:07.774799+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:41:40.720108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.795213+0000","last_change":"2024-07-04T17:30:46.408561+0000","last_active":"2024-07-04T17:31:07.795213+0000","last_peered":"2024-07-04T17:31:07.795213+0000","last_clean":"2024-07-04T17:31:07.795213+0000","last_became_active":"2024-07-04T17:30:46.408199+0000","last_became_peered":"2024-07-04T17:30:46.408199+0000","last_unstale":"2024-07-04T17:31:07.795213+0000","last_undegraded":"2024-07-04T17:31:07.795213+0000","last_fullsized":"2024-07-04T17:31:07.795213+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:15:50.228374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.787968+0000","last_change":"2024-07-04T17:31:06.782008+0000","last_active":"2024-07-04T17:31:07.787968+0000","last_peered":"2024-07-04T17:31:07.787968+0000","last_clean":"2024-07-04T17:31:07.787968+0000","last_became_active":"2024-07-04T17:31:06.780461+0000","last_became_peered":"2024-07-04T17:31:06.780461+0000","last_unstale":"2024-07-04T17:31:07.787968+0000","last_undegraded":"2024-07-04T17:31:07.787968+0000","last_fullsized":"2024-07-04T17:31:07.787968+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:49:12.342862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.774136+0000","last_change":"2024-07-04T17:31:05.787619+0000","last_active":"2024-07-04T17:31:07.774136+0000","last_peered":"2024-07-04T17:31:07.774136+0000","last_clean":"2024-07-04T17:31:07.774136+0000","last_became_active":"2024-07-04T17:31:05.787257+0000","last_became_peered":"2024-07-04T17:31:05.787257+0000","last_unstale":"2024-07-04T17:31:07.774136+0000","last_undegraded":"2024-07-04T17:31:07.774136+0000","last_fullsized":"2024-07-04T17:31:07.774136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:27:16.612883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.762964+0000","last_change":"2024-07-04T17:30:47.419127+0000","last_active":"2024-07-04T17:31:07.762964+0000","last_peered":"2024-07-04T17:31:07.762964+0000","last_clean":"2024-07-04T17:31:07.762964+0000","last_became_active":"2024-07-04T17:30:47.418557+0000","last_became_peered":"2024-07-04T17:30:47.418557+0000","last_unstale":"2024-07-04T17:31:07.762964+0000","last_undegraded":"2024-07-04T17:31:07.762964+0000","last_fullsized":"2024-07-04T17:31:07.762964+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:16:53.257166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.761625+0000","last_change":"2024-07-04T17:30:46.404540+0000","last_active":"2024-07-04T17:31:07.761625+0000","last_peered":"2024-07-04T17:31:07.761625+0000","last_clean":"2024-07-04T17:31:07.761625+0000","last_became_active":"2024-07-04T17:30:46.402800+0000","last_became_peered":"2024-07-04T17:30:46.402800+0000","last_unstale":"2024-07-04T17:31:07.761625+0000","last_undegraded":"2024-07-04T17:31:07.761625+0000","last_fullsized":"2024-07-04T17:31:07.761625+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:57:35.604132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.774017+0000","last_change":"2024-07-04T17:31:06.772406+0000","last_active":"2024-07-04T17:31:07.774017+0000","last_peered":"2024-07-04T17:31:07.774017+0000","last_clean":"2024-07-04T17:31:07.774017+0000","last_became_active":"2024-07-04T17:31:06.771978+0000","last_became_peered":"2024-07-04T17:31:06.771978+0000","last_unstale":"2024-07-04T17:31:07.774017+0000","last_undegraded":"2024-07-04T17:31:07.774017+0000","last_fullsized":"2024-07-04T17:31:07.774017+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:48:00.105154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.790920+0000","last_change":"2024-07-04T17:31:05.789134+0000","last_active":"2024-07-04T17:31:07.790920+0000","last_peered":"2024-07-04T17:31:07.790920+0000","last_clean":"2024-07-04T17:31:07.790920+0000","last_became_active":"2024-07-04T17:31:05.786364+0000","last_became_peered":"2024-07-04T17:31:05.786364+0000","last_unstale":"2024-07-04T17:31:07.790920+0000","last_undegraded":"2024-07-04T17:31:07.790920+0000","last_fullsized":"2024-07-04T17:31:07.790920+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:54:58.239395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.761456+0000","last_change":"2024-07-04T17:30:47.427679+0000","last_active":"2024-07-04T17:31:07.761456+0000","last_peered":"2024-07-04T17:31:07.761456+0000","last_clean":"2024-07-04T17:31:07.761456+0000","last_became_active":"2024-07-04T17:30:47.427451+0000","last_became_peered":"2024-07-04T17:30:47.427451+0000","last_unstale":"2024-07-04T17:31:07.761456+0000","last_undegraded":"2024-07-04T17:31:07.761456+0000","last_fullsized":"2024-07-04T17:31:07.761456+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:29:16.626050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.773348+0000","last_change":"2024-07-04T17:30:46.402877+0000","last_active":"2024-07-04T17:31:07.773348+0000","last_peered":"2024-07-04T17:31:07.773348+0000","last_clean":"2024-07-04T17:31:07.773348+0000","last_became_active":"2024-07-04T17:30:46.402551+0000","last_became_peered":"2024-07-04T17:30:46.402551+0000","last_unstale":"2024-07-04T17:31:07.773348+0000","last_undegraded":"2024-07-04T17:31:07.773348+0000","last_fullsized":"2024-07-04T17:31:07.773348+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:37:57.795830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.793884+0000","last_change":"2024-07-04T17:31:06.808200+0000","last_active":"2024-07-04T17:31:07.793884+0000","last_peered":"2024-07-04T17:31:07.793884+0000","last_clean":"2024-07-04T17:31:07.793884+0000","last_became_active":"2024-07-04T17:31:06.807640+0000","last_became_peered":"2024-07-04T17:31:06.807640+0000","last_unstale":"2024-07-04T17:31:07.793884+0000","last_undegraded":"2024-07-04T17:31:07.793884+0000","last_fullsized":"2024-07-04T17:31:07.793884+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:05.720604+0000","last_clean_scrub_stamp":"2024-07-04T17:31:05.720604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:13:58.149249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"75'1","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.763477+0000","last_change":"2024-07-04T17:31:05.777764+0000","last_active":"2024-07-04T17:31:07.763477+0000","last_peered":"2024-07-04T17:31:07.763477+0000","last_clean":"2024-07-04T17:31:07.763477+0000","last_became_active":"2024-07-04T17:31:05.777059+0000","last_became_peered":"2024-07-04T17:31:05.777059+0000","last_unstale":"2024-07-04T17:31:07.763477+0000","last_undegraded":"2024-07-04T17:31:07.763477+0000","last_fullsized":"2024-07-04T17:31:07.763477+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:04.706753+0000","last_clean_scrub_stamp":"2024-07-04T17:31:04.706753+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:18:07.499429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.771004+0000","last_change":"2024-07-04T17:30:47.390337+0000","last_active":"2024-07-04T17:31:07.771004+0000","last_peered":"2024-07-04T17:31:07.771004+0000","last_clean":"2024-07-04T17:31:07.771004+0000","last_became_active":"2024-07-04T17:30:47.389504+0000","last_became_peered":"2024-07-04T17:30:47.389504+0000","last_unstale":"2024-07-04T17:31:07.771004+0000","last_undegraded":"2024-07-04T17:31:07.771004+0000","last_fullsized":"2024-07-04T17:31:07.771004+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:46.348937+0000","last_clean_scrub_stamp":"2024-07-04T17:30:46.348937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:08:26.050071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"71'1","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.763395+0000","last_change":"2024-07-04T17:30:46.398516+0000","last_active":"2024-07-04T17:31:07.763395+0000","last_peered":"2024-07-04T17:31:07.763395+0000","last_clean":"2024-07-04T17:31:07.763395+0000","last_became_active":"2024-07-04T17:30:46.398169+0000","last_became_peered":"2024-07-04T17:30:46.398169+0000","last_unstale":"2024-07-04T17:31:07.763395+0000","last_undegraded":"2024-07-04T17:31:07.763395+0000","last_fullsized":"2024-07-04T17:31:07.763395+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:30:45.344654+0000","last_clean_scrub_stamp":"2024-07-04T17:30:45.344654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:40:51.947038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"58'80","reported_seq":304,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-07-04T17:31:07.766567+0000","last_change":"2024-07-04T17:26:04.953559+0000","last_active":"2024-07-04T17:31:07.766567+0000","last_peered":"2024-07-04T17:31:07.766567+0000","last_clean":"2024-07-04T17:31:07.766567+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:31:07.766567+0000","last_undegraded":"2024-07-04T17:31:07.766567+0000","last_fullsized":"2024-07-04T17:31:07.766567+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_clean_scrub_stamp":"2024-07-04T17:26:03.933432+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.064181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":296,"num_read_kb":750,"num_write":127,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":4107,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":3},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":296,"num_read_kb":750,"num_write":127,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80,"ondisk_log_size":80,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738431,"num_pgs":114,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":819200,"data_stored":307219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738432,"num_pgs":126,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36708,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335132,"statfs":{"total":96636764160,"available":96599175168,"internally_reserved":0,"allocated":1400832,"data_stored":894310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738430,"num_pgs":132,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36704,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335136,"statfs":{"total":96636764160,"available":96599179264,"internally_reserved":0,"allocated":1396736,"data_stored":896465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:31:12.078 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-04T17:31:12.078 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:31:12.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.407+0000 7f79d6200640 1 -- 172.21.15.42:0/4040946033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79d006cea0 msgr2=0x7f79d00b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:12.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.407+0000 7f79d6200640 1 --2- 172.21.15.42:0/4040946033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79d006cea0 0x7f79d00b3030 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f79c0009fd0 tx=0x7f79c002a8b0 comp rx=0 tx=0).stop 2024-07-04T17:31:12.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.408+0000 7f79d6200640 1 -- 172.21.15.42:0/4040946033 shutdown_connections 2024-07-04T17:31:12.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.408+0000 7f79d6200640 1 --2- 172.21.15.42:0/4040946033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79d006cea0 0x7f79d00b3030 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:12.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.408+0000 7f79d6200640 1 -- 172.21.15.42:0/4040946033 >> 172.21.15.42:0/4040946033 conn(0x7f79d00bf700 msgr2=0x7f79d00af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:12.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.408+0000 7f79d6200640 1 -- 172.21.15.42:0/4040946033 shutdown_connections 2024-07-04T17:31:12.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.408+0000 7f79d6200640 1 -- 172.21.15.42:0/4040946033 wait complete. 2024-07-04T17:31:12.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.410+0000 7f79d6200640 1 Processor -- start 2024-07-04T17:31:12.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.410+0000 7f79d6200640 1 -- start start 2024-07-04T17:31:12.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.412+0000 7f79d6200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79d006cea0 0x7f79d01d72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:12.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.412+0000 7f79d6200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f79d006ef50 con 0x7f79d006cea0 2024-07-04T17:31:12.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.412+0000 7f79cf400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79d006cea0 0x7f79d01d72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:12.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.412+0000 7f79cf400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79d006cea0 0x7f79d01d72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47202/0 (socket says 172.21.15.42:47202) 2024-07-04T17:31:12.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.412+0000 7f79cf400640 1 -- 172.21.15.42:0/4136156229 learned_addr learned my addr 172.21.15.42:0/4136156229 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:12.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.412+0000 7f79cf400640 1 -- 172.21.15.42:0/4136156229 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d00ac410 con 0x7f79d006cea0 2024-07-04T17:31:12.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.413+0000 7f79cf400640 1 --2- 172.21.15.42:0/4136156229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79d006cea0 0x7f79d01d72e0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f79c0000f80 tx=0x7f79c002f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:12.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.413+0000 7f79c7400640 1 -- 172.21.15.42:0/4136156229 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f79c0041030 con 0x7f79d006cea0 2024-07-04T17:31:12.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.413+0000 7f79c7400640 1 -- 172.21.15.42:0/4136156229 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f79c0037040 con 0x7f79d006cea0 2024-07-04T17:31:12.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.413+0000 7f79c7400640 1 -- 172.21.15.42:0/4136156229 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f79c0036070 con 0x7f79d006cea0 2024-07-04T17:31:12.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.413+0000 7f79d6200640 1 -- 172.21.15.42:0/4136156229 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79d00ade20 con 0x7f79d006cea0 2024-07-04T17:31:12.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.413+0000 7f79d6200640 1 -- 172.21.15.42:0/4136156229 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79d0148de0 con 0x7f79d006cea0 2024-07-04T17:31:12.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.415+0000 7f79d6200640 1 -- 172.21.15.42:0/4136156229 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79d00ac5e0 con 0x7f79d006cea0 2024-07-04T17:31:12.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.415+0000 7f79c7400640 1 -- 172.21.15.42:0/4136156229 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f79c0044070 con 0x7f79d006cea0 2024-07-04T17:31:12.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.415+0000 7f79c7400640 1 --2- 172.21.15.42:0/4136156229 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f79ac03d4f0 0x7f79ac03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:12.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.415+0000 7f79c7400640 1 -- 172.21.15.42:0/4136156229 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 4303+0+0 (secure 0 0 0) 0x7f79c0042070 con 0x7f79d006cea0 2024-07-04T17:31:12.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.415+0000 7f79cea00640 1 --2- 172.21.15.42:0/4136156229 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f79ac03d4f0 0x7f79ac03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:12.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.416+0000 7f79cea00640 1 --2- 172.21.15.42:0/4136156229 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f79ac03d4f0 0x7f79ac03fa00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f79b8007fd0 tx=0x7f79b8007920 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:12.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.420+0000 7f79c7400640 1 -- 172.21.15.42:0/4136156229 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f79c0077070 con 0x7f79d006cea0 2024-07-04T17:31:12.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.670+0000 7f79d6200640 1 -- 172.21.15.42:0/4136156229 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79d00b3030 con 0x7f79d006cea0 2024-07-04T17:31:12.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.671+0000 7f79c7400640 1 -- 172.21.15.42:0/4136156229 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+7369 (secure 0 0 0) 0x7f79c000a250 con 0x7f79d006cea0 2024-07-04T17:31:12.673 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:31:12.673 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":60,"btime":"2024-07-04T17:31:09:763642+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5017,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/2312272332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2312272332},{"type":"v1","addr":"172.21.15.42:6827","nonce":2312272332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5020,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1285161495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1285161495},{"type":"v1","addr":"172.21.15.42:6829","nonce":1285161495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5023,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/1583070649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1583070649},{"type":"v1","addr":"172.21.15.42:6831","nonce":1583070649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:30:46.947014+0000","modified":"2024-07-04T17:30:49.991386+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5029":{"gid":5029,"name":"e","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6835/1458267161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1458267161},{"type":"v1","addr":"172.21.15.42:6835","nonce":1458267161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5029,"qdb_cluster":[5029]},"id":6},{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:31:06.323313+0000","modified":"2024-07-04T17:31:09.367074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5026":{"gid":5026,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6833/2038326930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2038326930},{"type":"v1","addr":"172.21.15.42:6833","nonce":2038326930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5026,"qdb_cluster":[5026]},"id":7}]} 2024-07-04T17:31:12.673 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 60 2024-07-04T17:31:12.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.674+0000 7f79d6200640 1 -- 172.21.15.42:0/4136156229 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f79ac03d4f0 msgr2=0x7f79ac03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:12.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.674+0000 7f79d6200640 1 --2- 172.21.15.42:0/4136156229 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f79ac03d4f0 0x7f79ac03fa00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f79b8007fd0 tx=0x7f79b8007920 comp rx=0 tx=0).stop 2024-07-04T17:31:12.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.674+0000 7f79d6200640 1 -- 172.21.15.42:0/4136156229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79d006cea0 msgr2=0x7f79d01d72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:12.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.674+0000 7f79d6200640 1 --2- 172.21.15.42:0/4136156229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79d006cea0 0x7f79d01d72e0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f79c0000f80 tx=0x7f79c002f610 comp rx=0 tx=0).stop 2024-07-04T17:31:12.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.674+0000 7f79d6200640 1 -- 172.21.15.42:0/4136156229 shutdown_connections 2024-07-04T17:31:12.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.674+0000 7f79d6200640 1 --2- 172.21.15.42:0/4136156229 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f79ac03d4f0 0x7f79ac03fa00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:12.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.674+0000 7f79d6200640 1 --2- 172.21.15.42:0/4136156229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79d006cea0 0x7f79d01d72e0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:12.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.674+0000 7f79d6200640 1 -- 172.21.15.42:0/4136156229 >> 172.21.15.42:0/4136156229 conn(0x7f79d00bf700 msgr2=0x7f79d014ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:12.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.674+0000 7f79d6200640 1 -- 172.21.15.42:0/4136156229 shutdown_connections 2024-07-04T17:31:12.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:12.674+0000 7f79d6200640 1 -- 172.21.15.42:0/4136156229 wait complete. 2024-07-04T17:31:13.203 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 60, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-04T17:31:06.323313+0000', 'modified': '2024-07-04T17:31:09.367074+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5026}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5026': {'gid': 5026, 'name': 'b', 'rank': 0, 'incarnation': 56, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.42:6833/2038326930', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6832', 'nonce': 2038326930}, {'type': 'v1', 'addr': '172.21.15.42:6833', 'nonce': 2038326930}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'backup_fs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5026, 'qdb_cluster': [5026]} 2024-07-04T17:31:13.203 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-04T17:31:13.203 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-07-04T17:31:13.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.495+0000 7f2491600640 1 -- 172.21.15.42:0/245082508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f248c0ba8b0 msgr2=0x7f248c0b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:13.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.495+0000 7f2491600640 1 --2- 172.21.15.42:0/245082508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f248c0ba8b0 0x7f248c0b3550 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f247c005fd0 tx=0x7f247c02a8b0 comp rx=0 tx=0).stop 2024-07-04T17:31:13.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.495+0000 7f2491600640 1 -- 172.21.15.42:0/245082508 shutdown_connections 2024-07-04T17:31:13.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.495+0000 7f2491600640 1 --2- 172.21.15.42:0/245082508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f248c0ba8b0 0x7f248c0b3550 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:13.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.495+0000 7f2491600640 1 -- 172.21.15.42:0/245082508 >> 172.21.15.42:0/245082508 conn(0x7f248c0bebc0 msgr2=0x7f248c0befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:13.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.495+0000 7f2491600640 1 -- 172.21.15.42:0/245082508 shutdown_connections 2024-07-04T17:31:13.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.495+0000 7f2491600640 1 -- 172.21.15.42:0/245082508 wait complete. 2024-07-04T17:31:13.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.497+0000 7f2491600640 1 Processor -- start 2024-07-04T17:31:13.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.497+0000 7f2491600640 1 -- start start 2024-07-04T17:31:13.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.499+0000 7f2491600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f248c0b1d20 0x7f248c138d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:13.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.499+0000 7f2491600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f248c0b8b40 con 0x7f248c0b1d20 2024-07-04T17:31:13.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.499+0000 7f248be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f248c0b1d20 0x7f248c138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:13.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.499+0000 7f248be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f248c0b1d20 0x7f248c138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47212/0 (socket says 172.21.15.42:47212) 2024-07-04T17:31:13.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.499+0000 7f248be00640 1 -- 172.21.15.42:0/2560742459 learned_addr learned my addr 172.21.15.42:0/2560742459 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:13.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.499+0000 7f248be00640 1 -- 172.21.15.42:0/2560742459 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248c073590 con 0x7f248c0b1d20 2024-07-04T17:31:13.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.500+0000 7f248be00640 1 --2- 172.21.15.42:0/2560742459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f248c0b1d20 0x7f248c138d10 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f247c000f80 tx=0x7f247c033b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:13.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.500+0000 7f2483e00640 1 -- 172.21.15.42:0/2560742459 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f247c0319f0 con 0x7f248c0b1d20 2024-07-04T17:31:13.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.500+0000 7f2483e00640 1 -- 172.21.15.42:0/2560742459 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f247c031c80 con 0x7f248c0b1d20 2024-07-04T17:31:13.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.500+0000 7f2483e00640 1 -- 172.21.15.42:0/2560742459 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f247c0330d0 con 0x7f248c0b1d20 2024-07-04T17:31:13.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.500+0000 7f2491600640 1 -- 172.21.15.42:0/2560742459 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f248c139250 con 0x7f248c0b1d20 2024-07-04T17:31:13.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.500+0000 7f2491600640 1 -- 172.21.15.42:0/2560742459 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f248c1354b0 con 0x7f248c0b1d20 2024-07-04T17:31:13.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.501+0000 7f2483e00640 1 -- 172.21.15.42:0/2560742459 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7f247c033270 con 0x7f248c0b1d20 2024-07-04T17:31:13.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.502+0000 7f2483e00640 1 --2- 172.21.15.42:0/2560742459 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f246803d4f0 0x7f246803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:13.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.502+0000 7f2483e00640 1 -- 172.21.15.42:0/2560742459 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 4303+0+0 (secure 0 0 0) 0x7f247c073490 con 0x7f248c0b1d20 2024-07-04T17:31:13.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.502+0000 7f248b400640 1 --2- 172.21.15.42:0/2560742459 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f246803d4f0 0x7f246803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:13.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.503+0000 7f248b400640 1 --2- 172.21.15.42:0/2560742459 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f246803d4f0 0x7f246803fa00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2474000f60 tx=0x7f2474007a30 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:13.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.504+0000 7f2499c15740 1 Processor -- start 2024-07-04T17:31:13.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.504+0000 7f2499c15740 1 -- start start 2024-07-04T17:31:13.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.505+0000 7f2499c15740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f52a0dd270 0x55f52a483780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:13.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.505+0000 7f2499c15740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f52a3f45b0 con 0x55f52a0dd270 2024-07-04T17:31:13.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.506+0000 7f248be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f52a0dd270 0x55f52a483780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:13.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.506+0000 7f248be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f52a0dd270 0x55f52a483780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47220/0 (socket says 172.21.15.42:47220) 2024-07-04T17:31:13.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.506+0000 7f248be00640 1 -- 172.21.15.42:0/2944380362 learned_addr learned my addr 172.21.15.42:0/2944380362 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:13.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.506+0000 7f248be00640 1 -- 172.21.15.42:0/2944380362 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f52a148850 con 0x55f52a0dd270 2024-07-04T17:31:13.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.506+0000 7f248be00640 1 --2- 172.21.15.42:0/2944380362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f52a0dd270 0x55f52a483780 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f247c03be00 tx=0x7f247c039560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd153e9a8730fde5 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:13.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.506+0000 7f247be00640 1 -- 172.21.15.42:0/2944380362 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f247c033690 con 0x55f52a0dd270 2024-07-04T17:31:13.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.507+0000 7f247be00640 1 -- 172.21.15.42:0/2944380362 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f247c004510 con 0x55f52a0dd270 2024-07-04T17:31:13.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.507+0000 7f247be00640 1 -- 172.21.15.42:0/2944380362 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f247c004040 con 0x55f52a0dd270 2024-07-04T17:31:13.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.507+0000 7f2499c15740 1 -- 172.21.15.42:0/2944380362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f52a0dd270 msgr2=0x55f52a483780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:13.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.507+0000 7f2499c15740 1 --2- 172.21.15.42:0/2944380362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f52a0dd270 0x55f52a483780 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f247c03be00 tx=0x7f247c039560 comp rx=0 tx=0).stop 2024-07-04T17:31:13.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.507+0000 7f2499c15740 1 -- 172.21.15.42:0/2944380362 shutdown_connections 2024-07-04T17:31:13.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.507+0000 7f2499c15740 1 --2- 172.21.15.42:0/2944380362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f52a0dd270 0x55f52a483780 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:13.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.507+0000 7f2499c15740 1 -- 172.21.15.42:0/2944380362 >> 172.21.15.42:0/2944380362 conn(0x55f52a0d13c0 msgr2=0x55f52a481fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:13.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.508+0000 7f2499c15740 1 -- 172.21.15.42:0/2944380362 shutdown_connections 2024-07-04T17:31:13.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.508+0000 7f2499c15740 1 -- 172.21.15.42:0/2944380362 wait complete. 2024-07-04T17:31:13.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.509+0000 7f2499c15740 1 -- start start 2024-07-04T17:31:13.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.509+0000 7f2499c15740 1 Processor -- start 2024-07-04T17:31:13.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.510+0000 7f2499c15740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f52a0dd270 0x55f52a4c9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:13.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.510+0000 7f2499c15740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f52a3e7b90 con 0x55f52a0dd270 2024-07-04T17:31:13.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.510+0000 7f2499c15740 10 client.0 _ll_register_callbacks cb 0x55f52a38c8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T17:31:13.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.510+0000 7f248be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f52a0dd270 0x55f52a4c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:13.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.510+0000 7f248be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f52a0dd270 0x55f52a4c9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47226/0 (socket says 172.21.15.42:47226) 2024-07-04T17:31:13.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.510+0000 7f248be00640 1 -- 172.21.15.42:0/1106521152 learned_addr learned my addr 172.21.15.42:0/1106521152 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:13.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.511+0000 7f248be00640 1 -- 172.21.15.42:0/1106521152 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f52a1bf6b0 con 0x55f52a0dd270 2024-07-04T17:31:13.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.511+0000 7f248be00640 1 --2- 172.21.15.42:0/1106521152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f52a0dd270 0x55f52a4c9610 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f247c04b510 tx=0x7f247c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b36e495d4d8291c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:13.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.511+0000 7f2478c00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-07-04T17:31:13.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.511+0000 7f2478c00640 1 -- 172.21.15.42:0/1106521152 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f247c049e20 con 0x55f52a0dd270 2024-07-04T17:31:13.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.511+0000 7f2478c00640 1 -- 172.21.15.42:0/1106521152 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 4303+0+0 (secure 0 0 0) 0x7f247c071df0 con 0x55f52a0dd270 2024-07-04T17:31:13.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.511+0000 7f2499c15740 1 -- 172.21.15.42:0/1106521152 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f52a3846c0 con 0x55f52a0dd270 2024-07-04T17:31:13.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.511+0000 7f2478c00640 1 -- 172.21.15.42:0/1106521152 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f247c071ff0 con 0x55f52a0dd270 2024-07-04T17:31:13.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.511+0000 7f2499c15740 10 client.5142 fetch_fsmap learned FSMap version 60 2024-07-04T17:31:13.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.511+0000 7f2499c15740 1 -- 172.21.15.42:0/1106521152 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) -- 0x55f52a4a8490 con 0x55f52a0dd270 2024-07-04T17:31:13.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.511+0000 7f2478c00640 1 -- 172.21.15.42:0/1106521152 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55f52a3846c0 con 0x55f52a0dd270 2024-07-04T17:31:13.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.512+0000 7f2478c00640 1 -- 172.21.15.42:0/1106521152 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 60) ==== 4754+0+0 (secure 0 0 0) 0x7f247c06a070 con 0x55f52a0dd270 2024-07-04T17:31:13.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.512+0000 7f2499c15740 10 client.5142 fetch_fsmap finished waiting for FSMap version 60 2024-07-04T17:31:13.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.512+0000 7f2499c15740 10 client.5142 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-04T17:31:13.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.512+0000 7f2499c15740 20 client.5142 populate_metadata read hostname 'smithi042' 2024-07-04T17:31:13.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.512+0000 7f2499c15740 1 --2- 172.21.15.42:0/1106521152 >> [v2:172.21.15.42:6832/2038326930,v1:172.21.15.42:6833/2038326930] conn(0x55f52a0b43b0 0x55f52a4c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:13.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.512+0000 7f2499c15740 4 client.5142 mds_command: new command op to 5026 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T17:31:13.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.512+0000 7f2499c15740 1 -- 172.21.15.42:0/1106521152 --> [v2:172.21.15.42:6832/2038326930,v1:172.21.15.42:6833/2038326930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f52a3f6500 con 0x55f52a0b43b0 2024-07-04T17:31:13.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.512+0000 7f248b400640 1 --2- 172.21.15.42:0/1106521152 >> [v2:172.21.15.42:6832/2038326930,v1:172.21.15.42:6833/2038326930] conn(0x55f52a0b43b0 0x55f52a4c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:13.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.513+0000 7f248b400640 1 --2- 172.21.15.42:0/1106521152 >> [v2:172.21.15.42:6832/2038326930,v1:172.21.15.42:6833/2038326930] conn(0x55f52a0b43b0 0x55f52a4c60a0 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b503e5ed51cbc605 server_cookie=a108fcad25e84361 in_seq=0 out_seq=0 2024-07-04T17:31:13.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.513+0000 7f2478c00640 10 client.5142 ms_handle_connect on v2:172.21.15.42:6832/2038326930 2024-07-04T17:31:13.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.513+0000 7f2478c00640 1 -- 172.21.15.42:0/1106521152 <== mds.0 v2:172.21.15.42:6832/2038326930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f52a3f6500 con 0x55f52a0b43b0 2024-07-04T17:31:13.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.513+0000 7f2478c00640 10 client.5142 handle_command_reply: tid=0 2024-07-04T17:31:13.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.513+0000 7f2499c15740 1 client.5142 shutdown 2024-07-04T17:31:13.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.514+0000 7f2499c15740 1 -- 172.21.15.42:0/1106521152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f52a0dd270 msgr2=0x55f52a4c9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:13.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.514+0000 7f2499c15740 1 --2- 172.21.15.42:0/1106521152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f52a0dd270 0x55f52a4c9610 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f247c04b510 tx=0x7f247c004270 comp rx=0 tx=0).stop 2024-07-04T17:31:13.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.514+0000 7f2499c15740 1 -- 172.21.15.42:0/1106521152 shutdown_connections 2024-07-04T17:31:13.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.514+0000 7f2499c15740 1 --2- 172.21.15.42:0/1106521152 >> [v2:172.21.15.42:6832/2038326930,v1:172.21.15.42:6833/2038326930] conn(0x55f52a0b43b0 0x55f52a4c60a0 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:13.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.514+0000 7f2499c15740 1 --2- 172.21.15.42:0/1106521152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f52a0dd270 0x55f52a4c9610 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:13.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.514+0000 7f2499c15740 1 -- 172.21.15.42:0/1106521152 >> 172.21.15.42:0/1106521152 conn(0x55f52a0d13c0 msgr2=0x55f52a483f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:13.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.514+0000 7f2478c00640 0 client.5142 ms_handle_reset on v2:172.21.15.42:6832/2038326930 2024-07-04T17:31:13.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.514+0000 7f2499c15740 1 -- 172.21.15.42:0/1106521152 shutdown_connections 2024-07-04T17:31:13.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.514+0000 7f2499c15740 1 -- 172.21.15.42:0/1106521152 wait complete. 2024-07-04T17:31:13.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.514+0000 7f2499c15740 20 client.5142 trim_cache size 0 max 16384 2024-07-04T17:31:13.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.535+0000 7f2499c15740 1 Processor -- start 2024-07-04T17:31:13.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.535+0000 7f2499c15740 1 -- start start 2024-07-04T17:31:13.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.536+0000 7f2499c15740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f52a0dd270 0x55f52a49e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:13.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.536+0000 7f2499c15740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f248c135170 con 0x55f52a0dd270 2024-07-04T17:31:13.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.536+0000 7f248be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f52a0dd270 0x55f52a49e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:13.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.536+0000 7f248be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f52a0dd270 0x55f52a49e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47234/0 (socket says 172.21.15.42:47234) 2024-07-04T17:31:13.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.536+0000 7f248be00640 1 -- 172.21.15.42:0/2659657430 learned_addr learned my addr 172.21.15.42:0/2659657430 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:13.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.537+0000 7f248be00640 1 -- 172.21.15.42:0/2659657430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f52a4a4830 con 0x55f52a0dd270 2024-07-04T17:31:13.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.537+0000 7f248be00640 1 --2- 172.21.15.42:0/2659657430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f52a0dd270 0x55f52a49e1b0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f247c03af40 tx=0x7f247c070390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac28b18c96a5d54 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:13.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.537+0000 7f247b400640 1 -- 172.21.15.42:0/2659657430 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f247c070480 con 0x55f52a0dd270 2024-07-04T17:31:13.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.537+0000 7f247b400640 1 -- 172.21.15.42:0/2659657430 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f247c0045c0 con 0x55f52a0dd270 2024-07-04T17:31:13.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.537+0000 7f247b400640 1 -- 172.21.15.42:0/2659657430 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f247c039910 con 0x55f52a0dd270 2024-07-04T17:31:13.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.538+0000 7f2499c15740 1 -- 172.21.15.42:0/2659657430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f52a0dd270 msgr2=0x55f52a49e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:13.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.538+0000 7f2499c15740 1 --2- 172.21.15.42:0/2659657430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f52a0dd270 0x55f52a49e1b0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f247c03af40 tx=0x7f247c070390 comp rx=0 tx=0).stop 2024-07-04T17:31:13.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.538+0000 7f2499c15740 1 -- 172.21.15.42:0/2659657430 shutdown_connections 2024-07-04T17:31:13.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.538+0000 7f2499c15740 1 --2- 172.21.15.42:0/2659657430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f52a0dd270 0x55f52a49e1b0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:13.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.538+0000 7f2499c15740 1 -- 172.21.15.42:0/2659657430 >> 172.21.15.42:0/2659657430 conn(0x55f52a0d13c0 msgr2=0x55f52a4b0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:13.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.538+0000 7f2499c15740 1 -- 172.21.15.42:0/2659657430 shutdown_connections 2024-07-04T17:31:13.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.538+0000 7f2499c15740 1 -- 172.21.15.42:0/2659657430 wait complete. 2024-07-04T17:31:13.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.539+0000 7f2499c15740 1 -- start start 2024-07-04T17:31:13.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.540+0000 7f2499c15740 1 Processor -- start 2024-07-04T17:31:13.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.541+0000 7f2499c15740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f52a0dd270 0x55f52a481c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:13.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.541+0000 7f2499c15740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f52a3e3b50 con 0x55f52a0dd270 2024-07-04T17:31:13.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.541+0000 7f2499c15740 10 client.0 _ll_register_callbacks cb 0x7f247c004510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T17:31:13.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.541+0000 7f248be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f52a0dd270 0x55f52a481c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:13.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.541+0000 7f248be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f52a0dd270 0x55f52a481c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47238/0 (socket says 172.21.15.42:47238) 2024-07-04T17:31:13.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.541+0000 7f248be00640 1 -- 172.21.15.42:0/3891999265 learned_addr learned my addr 172.21.15.42:0/3891999265 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:13.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.542+0000 7f248be00640 1 -- 172.21.15.42:0/3891999265 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f52a4a32e0 con 0x55f52a0dd270 2024-07-04T17:31:13.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.542+0000 7f248be00640 1 --2- 172.21.15.42:0/3891999265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f52a0dd270 0x55f52a481c70 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f247c03be00 tx=0x7f247c039f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e560300a891435 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:13.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.542+0000 7f2478c00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-07-04T17:31:13.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.542+0000 7f2478c00640 1 -- 172.21.15.42:0/3891999265 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f247c03a3a0 con 0x55f52a0dd270 2024-07-04T17:31:13.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.542+0000 7f2478c00640 1 -- 172.21.15.42:0/3891999265 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 4303+0+0 (secure 0 0 0) 0x7f247c03a540 con 0x55f52a0dd270 2024-07-04T17:31:13.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.542+0000 7f2499c15740 1 -- 172.21.15.42:0/3891999265 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f247c0045c0 con 0x55f52a0dd270 2024-07-04T17:31:13.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.543+0000 7f2478c00640 1 -- 172.21.15.42:0/3891999265 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f247c03a740 con 0x55f52a0dd270 2024-07-04T17:31:13.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.543+0000 7f2499c15740 10 client.5144 fetch_fsmap learned FSMap version 60 2024-07-04T17:31:13.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.543+0000 7f2499c15740 1 -- 172.21.15.42:0/3891999265 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) -- 0x55f52a4a32e0 con 0x55f52a0dd270 2024-07-04T17:31:13.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.543+0000 7f2478c00640 1 -- 172.21.15.42:0/3891999265 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f247c0045c0 con 0x55f52a0dd270 2024-07-04T17:31:13.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.543+0000 7f2478c00640 1 -- 172.21.15.42:0/3891999265 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 60) ==== 4754+0+0 (secure 0 0 0) 0x7f247c05b020 con 0x55f52a0dd270 2024-07-04T17:31:13.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.543+0000 7f2499c15740 10 client.5144 fetch_fsmap finished waiting for FSMap version 60 2024-07-04T17:31:13.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.543+0000 7f2499c15740 10 client.5144 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-04T17:31:13.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.543+0000 7f2499c15740 20 client.5144 populate_metadata read hostname 'smithi042' 2024-07-04T17:31:13.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.543+0000 7f2499c15740 1 --2- 172.21.15.42:0/3891999265 >> [v2:172.21.15.42:6832/2038326930,v1:172.21.15.42:6833/2038326930] conn(0x55f52a09be40 0x55f52a49fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:13.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.543+0000 7f2499c15740 4 client.5144 mds_command: new command op to 5026 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-04T17:31:13.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.543+0000 7f2499c15740 1 -- 172.21.15.42:0/3891999265 --> [v2:172.21.15.42:6832/2038326930,v1:172.21.15.42:6833/2038326930] -- command(tid 0: {"prefix": "status"}) -- 0x55f52a3f1fa0 con 0x55f52a09be40 2024-07-04T17:31:13.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.543+0000 7f248b400640 1 --2- 172.21.15.42:0/3891999265 >> [v2:172.21.15.42:6832/2038326930,v1:172.21.15.42:6833/2038326930] conn(0x55f52a09be40 0x55f52a49fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:13.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.544+0000 7f248b400640 1 --2- 172.21.15.42:0/3891999265 >> [v2:172.21.15.42:6832/2038326930,v1:172.21.15.42:6833/2038326930] conn(0x55f52a09be40 0x55f52a49fa70 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7355828618b5063 server_cookie=699efdcae3d73a9 in_seq=0 out_seq=0 2024-07-04T17:31:13.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.544+0000 7f2478c00640 10 client.5144 ms_handle_connect on v2:172.21.15.42:6832/2038326930 2024-07-04T17:31:13.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.544+0000 7f2478c00640 1 -- 172.21.15.42:0/3891999265 <== mds.0 v2:172.21.15.42:6832/2038326930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55f52a3f1fa0 con 0x55f52a09be40 2024-07-04T17:31:13.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.544+0000 7f2478c00640 10 client.5144 handle_command_reply: tid=0 2024-07-04T17:31:13.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.544+0000 7f2499c15740 1 client.5144 shutdown 2024-07-04T17:31:13.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.544+0000 7f2499c15740 1 -- 172.21.15.42:0/3891999265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f52a0dd270 msgr2=0x55f52a481c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:13.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.544+0000 7f2499c15740 1 --2- 172.21.15.42:0/3891999265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f52a0dd270 0x55f52a481c70 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f247c03be00 tx=0x7f247c039f10 comp rx=0 tx=0).stop 2024-07-04T17:31:13.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.545+0000 7f2499c15740 1 -- 172.21.15.42:0/3891999265 shutdown_connections 2024-07-04T17:31:13.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.545+0000 7f2499c15740 1 --2- 172.21.15.42:0/3891999265 >> [v2:172.21.15.42:6832/2038326930,v1:172.21.15.42:6833/2038326930] conn(0x55f52a09be40 0x55f52a49fa70 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:13.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.545+0000 7f2499c15740 1 --2- 172.21.15.42:0/3891999265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f52a0dd270 0x55f52a481c70 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:13.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.545+0000 7f2499c15740 1 -- 172.21.15.42:0/3891999265 >> 172.21.15.42:0/3891999265 conn(0x55f52a0d13c0 msgr2=0x55f52a49e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:13.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.545+0000 7f2478c00640 0 client.5144 ms_handle_reset on v2:172.21.15.42:6832/2038326930 2024-07-04T17:31:13.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.545+0000 7f2499c15740 1 -- 172.21.15.42:0/3891999265 shutdown_connections 2024-07-04T17:31:13.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.545+0000 7f2499c15740 1 -- 172.21.15.42:0/3891999265 wait complete. 2024-07-04T17:31:13.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.545+0000 7f2499c15740 20 client.5144 trim_cache size 0 max 16384 2024-07-04T17:31:13.549 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:31:13.549 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "be4f300a-93ef-4489-b540-37ece39b0aac", 2024-07-04T17:31:13.549 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-07-04T17:31:13.549 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 5026, 2024-07-04T17:31:13.549 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-07-04T17:31:13.550 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-07-04T17:31:13.550 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "backup_fs", 2024-07-04T17:31:13.550 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 6.8079257770000003, 2024-07-04T17:31:13.550 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 60, 2024-07-04T17:31:13.550 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 76, 2024-07-04T17:31:13.550 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 76, 2024-07-04T17:31:13.550 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 31.031040484999998 2024-07-04T17:31:13.550 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:31:13.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.546+0000 7f2491600640 1 -- 172.21.15.42:0/2560742459 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f246803d4f0 msgr2=0x7f246803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:13.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.546+0000 7f2491600640 1 --2- 172.21.15.42:0/2560742459 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f246803d4f0 0x7f246803fa00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2474000f60 tx=0x7f2474007a30 comp rx=0 tx=0).stop 2024-07-04T17:31:13.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.546+0000 7f2491600640 1 -- 172.21.15.42:0/2560742459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f248c0b1d20 msgr2=0x7f248c138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:13.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.546+0000 7f2491600640 1 --2- 172.21.15.42:0/2560742459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f248c0b1d20 0x7f248c138d10 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f247c000f80 tx=0x7f247c033b10 comp rx=0 tx=0).stop 2024-07-04T17:31:13.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.546+0000 7f2491600640 1 -- 172.21.15.42:0/2560742459 shutdown_connections 2024-07-04T17:31:13.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.546+0000 7f2491600640 1 --2- 172.21.15.42:0/2560742459 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f246803d4f0 0x7f246803fa00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:13.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.546+0000 7f2491600640 1 --2- 172.21.15.42:0/2560742459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f248c0b1d20 0x7f248c138d10 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:13.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.546+0000 7f2491600640 1 -- 172.21.15.42:0/2560742459 >> 172.21.15.42:0/2560742459 conn(0x7f248c0bebc0 msgr2=0x7f248c075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:13.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.546+0000 7f2491600640 1 -- 172.21.15.42:0/2560742459 shutdown_connections 2024-07-04T17:31:13.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.546+0000 7f2491600640 1 -- 172.21.15.42:0/2560742459 wait complete. 2024-07-04T17:31:13.571 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable mirroring 2024-07-04T17:31:13.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.850+0000 7fed0cc00640 1 -- 172.21.15.42:0/1089043532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed080ac5b0 msgr2=0x7fed080b0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:13.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.850+0000 7fed0cc00640 1 --2- 172.21.15.42:0/1089043532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed080ac5b0 0x7fed080b0db0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fecf000a8c0 tx=0x7fecf002b860 comp rx=0 tx=0).stop 2024-07-04T17:31:13.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.851+0000 7fed0cc00640 1 -- 172.21.15.42:0/1089043532 shutdown_connections 2024-07-04T17:31:13.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.851+0000 7fed0cc00640 1 --2- 172.21.15.42:0/1089043532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed080ac5b0 0x7fed080b0db0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:13.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.851+0000 7fed0cc00640 1 -- 172.21.15.42:0/1089043532 >> 172.21.15.42:0/1089043532 conn(0x7fed080afaa0 msgr2=0x7fed080afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:13.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.851+0000 7fed0cc00640 1 -- 172.21.15.42:0/1089043532 shutdown_connections 2024-07-04T17:31:13.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.851+0000 7fed0cc00640 1 -- 172.21.15.42:0/1089043532 wait complete. 2024-07-04T17:31:13.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.853+0000 7fed0cc00640 1 Processor -- start 2024-07-04T17:31:13.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.853+0000 7fed0cc00640 1 -- start start 2024-07-04T17:31:13.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.855+0000 7fed0cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed0806da60 0x7fed081d7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:13.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.855+0000 7fed0cc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fed080ae640 con 0x7fed0806da60 2024-07-04T17:31:13.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.855+0000 7fed07400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed0806da60 0x7fed081d7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:13.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.855+0000 7fed07400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed0806da60 0x7fed081d7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42458/0 (socket says 172.21.15.42:42458) 2024-07-04T17:31:13.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.855+0000 7fed07400640 1 -- 172.21.15.42:0/650263355 learned_addr learned my addr 172.21.15.42:0/650263355 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:13.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.856+0000 7fed07400640 1 -- 172.21.15.42:0/650263355 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed081d6c50 con 0x7fed0806da60 2024-07-04T17:31:13.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.856+0000 7fed07400640 1 --2- 172.21.15.42:0/650263355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed0806da60 0x7fed081d7a10 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fecf800ca60 tx=0x7fecf800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:13.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.856+0000 7fecff400640 1 -- 172.21.15.42:0/650263355 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fecf8016020 con 0x7fed0806da60 2024-07-04T17:31:13.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.856+0000 7fecff400640 1 -- 172.21.15.42:0/650263355 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fecf800ab90 con 0x7fed0806da60 2024-07-04T17:31:13.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.856+0000 7fecff400640 1 -- 172.21.15.42:0/650263355 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fecf8012420 con 0x7fed0806da60 2024-07-04T17:31:13.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.856+0000 7fed0cc00640 1 -- 172.21.15.42:0/650263355 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed081e1fc0 con 0x7fed0806da60 2024-07-04T17:31:13.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.857+0000 7fed0cc00640 1 -- 172.21.15.42:0/650263355 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed0813d160 con 0x7fed0806da60 2024-07-04T17:31:13.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.858+0000 7fed0cc00640 1 -- 172.21.15.42:0/650263355 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed080b0db0 con 0x7fed0806da60 2024-07-04T17:31:13.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.858+0000 7fecff400640 1 -- 172.21.15.42:0/650263355 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) ==== 49615+0+0 (secure 0 0 0) 0x7fecf8019020 con 0x7fed0806da60 2024-07-04T17:31:13.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.858+0000 7fecff400640 1 --2- 172.21.15.42:0/650263355 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fece403d4f0 0x7fece403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:13.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.858+0000 7fecff400640 1 -- 172.21.15.42:0/650263355 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 4303+0+0 (secure 0 0 0) 0x7fecf8052340 con 0x7fed0806da60 2024-07-04T17:31:13.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.859+0000 7fed06a00640 1 --2- 172.21.15.42:0/650263355 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fece403d4f0 0x7fece403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:13.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.859+0000 7fed06a00640 1 --2- 172.21.15.42:0/650263355 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fece403d4f0 0x7fece403fa00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fecf0036000 tx=0x7fecf0004730 comp rx=0 tx=0).ready entity=mgr.4906 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:13.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:13.864+0000 7fecff400640 1 -- 172.21.15.42:0/650263355 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fecf8020020 con 0x7fed0806da60 2024-07-04T17:31:14.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:14.125+0000 7fed0cc00640 1 -- 172.21.15.42:0/650263355 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "mirroring"} v 0) -- 0x7fed08068e30 con 0x7fed0806da60 2024-07-04T17:31:14.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:14.792+0000 7fecff400640 1 -- 172.21.15.42:0/650263355 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "mirroring"}]=0 v26) ==== 88+0+0 (secure 0 0 0) 0x7fecf80520d0 con 0x7fed0806da60 2024-07-04T17:31:14.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:14.797+0000 7fed0cc00640 1 -- 172.21.15.42:0/650263355 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fece403d4f0 msgr2=0x7fece403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:14.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:14.797+0000 7fed0cc00640 1 --2- 172.21.15.42:0/650263355 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fece403d4f0 0x7fece403fa00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fecf0036000 tx=0x7fecf0004730 comp rx=0 tx=0).stop 2024-07-04T17:31:14.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:14.797+0000 7fed0cc00640 1 -- 172.21.15.42:0/650263355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed0806da60 msgr2=0x7fed081d7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:14.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:14.797+0000 7fed0cc00640 1 --2- 172.21.15.42:0/650263355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed0806da60 0x7fed081d7a10 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fecf800ca60 tx=0x7fecf800cf20 comp rx=0 tx=0).stop 2024-07-04T17:31:14.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:14.798+0000 7fed0cc00640 1 -- 172.21.15.42:0/650263355 shutdown_connections 2024-07-04T17:31:14.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:14.798+0000 7fed0cc00640 1 --2- 172.21.15.42:0/650263355 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7fece403d4f0 0x7fece403fa00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:14.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:14.798+0000 7fed0cc00640 1 --2- 172.21.15.42:0/650263355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed0806da60 0x7fed081d7a10 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:14.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:14.798+0000 7fed0cc00640 1 -- 172.21.15.42:0/650263355 >> 172.21.15.42:0/650263355 conn(0x7fed080afaa0 msgr2=0x7fed080a9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:14.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:14.800+0000 7fed0cc00640 1 -- 172.21.15.42:0/650263355 shutdown_connections 2024-07-04T17:31:14.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:14.801+0000 7fed0cc00640 1 -- 172.21.15.42:0/650263355 wait complete. 2024-07-04T17:31:14.836 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:31:14.837 DEBUG:teuthology.orchestra.run.smithi042:mirror status for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@6 2024-07-04T17:31:15.056 INFO:teuthology.orchestra.run.smithi042.stderr:no valid command found; 10 closest matches: 2024-07-04T17:31:15.056 INFO:teuthology.orchestra.run.smithi042.stderr:0 2024-07-04T17:31:15.056 INFO:teuthology.orchestra.run.smithi042.stderr:1 2024-07-04T17:31:15.056 INFO:teuthology.orchestra.run.smithi042.stderr:2 2024-07-04T17:31:15.056 INFO:teuthology.orchestra.run.smithi042.stderr:abort 2024-07-04T17:31:15.056 INFO:teuthology.orchestra.run.smithi042.stderr:assert 2024-07-04T17:31:15.056 INFO:teuthology.orchestra.run.smithi042.stderr:config diff 2024-07-04T17:31:15.056 INFO:teuthology.orchestra.run.smithi042.stderr:config diff get 2024-07-04T17:31:15.056 INFO:teuthology.orchestra.run.smithi042.stderr:config get 2024-07-04T17:31:15.057 INFO:teuthology.orchestra.run.smithi042.stderr:config help [] 2024-07-04T17:31:15.057 INFO:teuthology.orchestra.run.smithi042.stderr:config set ... 2024-07-04T17:31:15.057 INFO:teuthology.orchestra.run.smithi042.stderr:admin_socket: invalid command 2024-07-04T17:31:15.067 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-07-04T17:31:15.067 WARNING:tasks.cephfs.test_mirroring:mirror daemon command with label "mirror status for fs: cephfs" failed: Command failed (mirror status for fs: cephfs) on smithi042 with status 22: 'ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@6' 2024-07-04T17:31:15.067 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable mirroring 2024-07-04T17:31:15.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.332+0000 7f7b8c200640 1 -- 172.21.15.42:0/3343460271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b840b02b0 msgr2=0x7f7b84073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:15.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.332+0000 7f7b8c200640 1 --2- 172.21.15.42:0/3343460271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b840b02b0 0x7f7b84073c90 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f7b780097b0 tx=0x7f7b7802a540 comp rx=0 tx=0).stop 2024-07-04T17:31:15.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.333+0000 7f7b8c200640 1 -- 172.21.15.42:0/3343460271 shutdown_connections 2024-07-04T17:31:15.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.333+0000 7f7b8c200640 1 --2- 172.21.15.42:0/3343460271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b840b02b0 0x7f7b84073c90 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:15.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.333+0000 7f7b8c200640 1 -- 172.21.15.42:0/3343460271 >> 172.21.15.42:0/3343460271 conn(0x7f7b8406fe40 msgr2=0x7f7b8406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:15.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.333+0000 7f7b8c200640 1 -- 172.21.15.42:0/3343460271 shutdown_connections 2024-07-04T17:31:15.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.333+0000 7f7b8c200640 1 -- 172.21.15.42:0/3343460271 wait complete. 2024-07-04T17:31:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.335+0000 7f7b8c200640 1 Processor -- start 2024-07-04T17:31:15.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.335+0000 7f7b8c200640 1 -- start start 2024-07-04T17:31:15.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.336+0000 7f7b8c200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b840741d0 0x7f7b840710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:15.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.336+0000 7f7b8c200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7b84141790 con 0x7f7b840741d0 2024-07-04T17:31:15.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.337+0000 7f7b89800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b840741d0 0x7f7b840710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:15.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.337+0000 7f7b89800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b840741d0 0x7f7b840710c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42496/0 (socket says 172.21.15.42:42496) 2024-07-04T17:31:15.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.337+0000 7f7b89800640 1 -- 172.21.15.42:0/55857892 learned_addr learned my addr 172.21.15.42:0/55857892 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:15.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.337+0000 7f7b89800640 1 -- 172.21.15.42:0/55857892 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b840bfce0 con 0x7f7b840741d0 2024-07-04T17:31:15.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.337+0000 7f7b89800640 1 --2- 172.21.15.42:0/55857892 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b840741d0 0x7f7b840710c0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f7b780098d0 tx=0x7f7b78033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:15.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.338+0000 7f7b81600640 1 -- 172.21.15.42:0/55857892 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b78031bc0 con 0x7f7b840741d0 2024-07-04T17:31:15.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.338+0000 7f7b81600640 1 -- 172.21.15.42:0/55857892 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b78031e50 con 0x7f7b840741d0 2024-07-04T17:31:15.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.338+0000 7f7b8c200640 1 -- 172.21.15.42:0/55857892 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b840b4fa0 con 0x7f7b840741d0 2024-07-04T17:31:15.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.338+0000 7f7b81600640 1 -- 172.21.15.42:0/55857892 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b78033200 con 0x7f7b840741d0 2024-07-04T17:31:15.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.338+0000 7f7b8c200640 1 -- 172.21.15.42:0/55857892 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b840b4c20 con 0x7f7b840741d0 2024-07-04T17:31:15.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.339+0000 7f7b8c200640 1 -- 172.21.15.42:0/55857892 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b84073c90 con 0x7f7b840741d0 2024-07-04T17:31:15.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.340+0000 7f7b81600640 1 -- 172.21.15.42:0/55857892 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) ==== 49628+0+0 (secure 0 0 0) 0x7f7b780333a0 con 0x7f7b840741d0 2024-07-04T17:31:15.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.340+0000 7f7b81600640 1 --2- 172.21.15.42:0/55857892 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f7b6403d540 0x7f7b6403fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:15.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.340+0000 7f7b81600640 1 -- 172.21.15.42:0/55857892 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 4303+0+0 (secure 0 0 0) 0x7f7b78073220 con 0x7f7b840741d0 2024-07-04T17:31:15.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.340+0000 7f7b88e00640 1 -- 172.21.15.42:0/55857892 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f7b6403d540 msgr2=0x7f7b6403fa50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/3473020706 2024-07-04T17:31:15.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.340+0000 7f7b88e00640 1 --2- 172.21.15.42:0/55857892 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f7b6403d540 0x7f7b6403fa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:31:15.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.346+0000 7f7b81600640 1 -- 172.21.15.42:0/55857892 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f7b7803f030 con 0x7f7b840741d0 2024-07-04T17:31:15.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.541+0000 7f7b88e00640 1 -- 172.21.15.42:0/55857892 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f7b6403d540 msgr2=0x7f7b6403fa50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/3473020706 2024-07-04T17:31:15.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.541+0000 7f7b88e00640 1 --2- 172.21.15.42:0/55857892 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f7b6403d540 0x7f7b6403fa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:31:15.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.604+0000 7f7b8c200640 1 -- 172.21.15.42:0/55857892 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "mirroring"} v 0) -- 0x7f7b84069b70 con 0x7f7b840741d0 2024-07-04T17:31:15.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.809+0000 7f7b81600640 1 -- 172.21.15.42:0/55857892 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "mirroring"}]=0 v27) ==== 89+0+0 (secure 0 0 0) 0x7f7b78003b10 con 0x7f7b840741d0 2024-07-04T17:31:15.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.813+0000 7f7b8c200640 1 -- 172.21.15.42:0/55857892 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f7b6403d540 msgr2=0x7f7b6403fa50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:31:15.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.813+0000 7f7b8c200640 1 --2- 172.21.15.42:0/55857892 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f7b6403d540 0x7f7b6403fa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:15.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.813+0000 7f7b8c200640 1 -- 172.21.15.42:0/55857892 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b840741d0 msgr2=0x7f7b840710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:15.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.813+0000 7f7b8c200640 1 --2- 172.21.15.42:0/55857892 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b840741d0 0x7f7b840710c0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f7b780098d0 tx=0x7f7b78033e40 comp rx=0 tx=0).stop 2024-07-04T17:31:15.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.814+0000 7f7b8c200640 1 -- 172.21.15.42:0/55857892 shutdown_connections 2024-07-04T17:31:15.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.814+0000 7f7b8c200640 1 --2- 172.21.15.42:0/55857892 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f7b6403d540 0x7f7b6403fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:15.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.814+0000 7f7b8c200640 1 --2- 172.21.15.42:0/55857892 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b840741d0 0x7f7b840710c0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:15.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.814+0000 7f7b8c200640 1 -- 172.21.15.42:0/55857892 >> 172.21.15.42:0/55857892 conn(0x7f7b8406fe40 msgr2=0x7f7b84140940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:15.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.815+0000 7f7b8c200640 1 -- 172.21.15.42:0/55857892 shutdown_connections 2024-07-04T17:31:15.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:15.815+0000 7f7b8c200640 1 -- 172.21.15.42:0/55857892 wait complete. 2024-07-04T17:31:15.843 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-04T17:31:15.843 DEBUG:teuthology.orchestra.run.smithi042:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-04T17:31:15.941 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-04T17:31:15.942 INFO:tasks.cephfs.mount:Terminating background process 2024-07-04T17:31:15.942 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:31:15.942 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:31:15.990 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:31:15.990 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:31:16.077 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-04T17:31:16.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.399+0000 7ffaa7200640 1 -- 172.21.15.42:0/257394893 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaa80b8f80 msgr2=0x7ffaa806c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:16.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.399+0000 7ffaa7200640 1 --2- 172.21.15.42:0/257394893 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaa80b8f80 0x7ffaa806c020 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7ffa900097b0 tx=0x7ffa9002a540 comp rx=0 tx=0).stop 2024-07-04T17:31:16.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.399+0000 7ffaa7200640 1 -- 172.21.15.42:0/257394893 shutdown_connections 2024-07-04T17:31:16.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.399+0000 7ffaa7200640 1 --2- 172.21.15.42:0/257394893 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaa80b8f80 0x7ffaa806c020 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:16.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.399+0000 7ffaa7200640 1 -- 172.21.15.42:0/257394893 >> 172.21.15.42:0/257394893 conn(0x7ffaa80bebc0 msgr2=0x7ffaa80befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:16.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.399+0000 7ffaa7200640 1 -- 172.21.15.42:0/257394893 shutdown_connections 2024-07-04T17:31:16.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.399+0000 7ffaa7200640 1 -- 172.21.15.42:0/257394893 wait complete. 2024-07-04T17:31:16.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.402+0000 7ffaa7200640 1 Processor -- start 2024-07-04T17:31:16.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.402+0000 7ffaa7200640 1 -- start start 2024-07-04T17:31:16.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.403+0000 7ffaa7200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaa80b2110 0x7ffaa80aa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:16.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.403+0000 7ffaa7200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa80b28b0 con 0x7ffaa80b2110 2024-07-04T17:31:16.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.403+0000 7ffaa5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaa80b2110 0x7ffaa80aa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:16.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.403+0000 7ffaa5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaa80b2110 0x7ffaa80aa2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42520/0 (socket says 172.21.15.42:42520) 2024-07-04T17:31:16.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.403+0000 7ffaa5e00640 1 -- 172.21.15.42:0/1074961340 learned_addr learned my addr 172.21.15.42:0/1074961340 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:16.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.404+0000 7ffaa5e00640 1 -- 172.21.15.42:0/1074961340 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa80abd30 con 0x7ffaa80b2110 2024-07-04T17:31:16.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.404+0000 7ffaa5e00640 1 --2- 172.21.15.42:0/1074961340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaa80b2110 0x7ffaa80aa2f0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7ffa900098d0 tx=0x7ffa90033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:16.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.405+0000 7ffa9e000640 1 -- 172.21.15.42:0/1074961340 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa90031bc0 con 0x7ffaa80b2110 2024-07-04T17:31:16.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.405+0000 7ffa9e000640 1 -- 172.21.15.42:0/1074961340 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa90031e50 con 0x7ffaa80b2110 2024-07-04T17:31:16.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.405+0000 7ffa9e000640 1 -- 172.21.15.42:0/1074961340 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa90033200 con 0x7ffaa80b2110 2024-07-04T17:31:16.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.405+0000 7ffaa7200640 1 -- 172.21.15.42:0/1074961340 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaa80a9d60 con 0x7ffaa80b2110 2024-07-04T17:31:16.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.405+0000 7ffaa7200640 1 -- 172.21.15.42:0/1074961340 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaa80a9a70 con 0x7ffaa80b2110 2024-07-04T17:31:16.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.406+0000 7ffaa7200640 1 -- 172.21.15.42:0/1074961340 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaa80a94d0 con 0x7ffaa80b2110 2024-07-04T17:31:16.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.407+0000 7ffa9e000640 1 -- 172.21.15.42:0/1074961340 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 49615+0+0 (secure 0 0 0) 0x7ffa900333a0 con 0x7ffaa80b2110 2024-07-04T17:31:16.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.407+0000 7ffa9e000640 1 --2- 172.21.15.42:0/1074961340 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ffa8403d4f0 0x7ffa8403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:16.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.407+0000 7ffa9e000640 1 -- 172.21.15.42:0/1074961340 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 4303+0+0 (secure 0 0 0) 0x7ffa90073210 con 0x7ffaa80b2110 2024-07-04T17:31:16.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.407+0000 7ffaa5400640 1 -- 172.21.15.42:0/1074961340 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ffa8403d4f0 msgr2=0x7ffa8403fa00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/3473020706 2024-07-04T17:31:16.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.407+0000 7ffaa5400640 1 --2- 172.21.15.42:0/1074961340 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ffa8403d4f0 0x7ffa8403fa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:31:16.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.413+0000 7ffa9e000640 1 -- 172.21.15.42:0/1074961340 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7ffa9003f030 con 0x7ffaa80b2110 2024-07-04T17:31:16.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.608+0000 7ffaa5400640 1 -- 172.21.15.42:0/1074961340 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ffa8403d4f0 msgr2=0x7ffa8403fa00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/3473020706 2024-07-04T17:31:16.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.608+0000 7ffaa5400640 1 --2- 172.21.15.42:0/1074961340 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ffa8403d4f0 0x7ffa8403fa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:31:16.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.633+0000 7ffaa7200640 1 -- 172.21.15.42:0/1074961340 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ffaa80ba920 con 0x7ffaa80b2110 2024-07-04T17:31:16.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.634+0000 7ffa9e000640 1 -- 172.21.15.42:0/1074961340 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v76) ==== 80+0+0 (secure 0 0 0) 0x7ffa90033690 con 0x7ffaa80b2110 2024-07-04T17:31:16.635 INFO:teuthology.orchestra.run.smithi042.stderr:listed 0 entries 2024-07-04T17:31:16.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.637+0000 7ffaa7200640 1 -- 172.21.15.42:0/1074961340 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ffa8403d4f0 msgr2=0x7ffa8403fa00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:31:16.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.637+0000 7ffaa7200640 1 --2- 172.21.15.42:0/1074961340 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ffa8403d4f0 0x7ffa8403fa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:16.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.637+0000 7ffaa7200640 1 -- 172.21.15.42:0/1074961340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaa80b2110 msgr2=0x7ffaa80aa2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:16.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.637+0000 7ffaa7200640 1 --2- 172.21.15.42:0/1074961340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaa80b2110 0x7ffaa80aa2f0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7ffa900098d0 tx=0x7ffa90033e40 comp rx=0 tx=0).stop 2024-07-04T17:31:16.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.637+0000 7ffaa7200640 1 -- 172.21.15.42:0/1074961340 shutdown_connections 2024-07-04T17:31:16.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.637+0000 7ffaa7200640 1 --2- 172.21.15.42:0/1074961340 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ffa8403d4f0 0x7ffa8403fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:16.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.637+0000 7ffaa7200640 1 --2- 172.21.15.42:0/1074961340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaa80b2110 0x7ffaa80aa2f0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:16.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.637+0000 7ffaa7200640 1 -- 172.21.15.42:0/1074961340 >> 172.21.15.42:0/1074961340 conn(0x7ffaa80bebc0 msgr2=0x7ffaa80b9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:16.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.637+0000 7ffaa7200640 1 -- 172.21.15.42:0/1074961340 shutdown_connections 2024-07-04T17:31:16.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:16.637+0000 7ffaa7200640 1 -- 172.21.15.42:0/1074961340 wait complete. 2024-07-04T17:31:16.663 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi042.front.sepia.ceph.com... 2024-07-04T17:31:16.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:31:16.663 DEBUG:teuthology.orchestra.run.smithi042:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-07-04T17:31:16.726 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[56478]: fuse finished with error 0 and tester_r 0 2024-07-04T17:31:16.731 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-07-04T17:31:16.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:31:16.731 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-04T17:31:16.873 INFO:tasks.cephfs.mount:Terminating background process 2024-07-04T17:31:16.873 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:31:16.873 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:31:16.921 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:31:16.921 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:31:17.017 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-04T17:31:17.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.299+0000 7efcdc400640 1 -- 172.21.15.42:0/94904703 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efcd40b02b0 msgr2=0x7efcd40b3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:17.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.299+0000 7efcdc400640 1 --2- 172.21.15.42:0/94904703 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efcd40b02b0 0x7efcd40b3650 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7efcc8005fd0 tx=0x7efcc802aa30 comp rx=0 tx=0).stop 2024-07-04T17:31:17.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.300+0000 7efcdc400640 1 -- 172.21.15.42:0/94904703 shutdown_connections 2024-07-04T17:31:17.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.300+0000 7efcdc400640 1 --2- 172.21.15.42:0/94904703 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efcd40b02b0 0x7efcd40b3650 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:17.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.300+0000 7efcdc400640 1 -- 172.21.15.42:0/94904703 >> 172.21.15.42:0/94904703 conn(0x7efcd40bf700 msgr2=0x7efcd40af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:17.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.300+0000 7efcdc400640 1 -- 172.21.15.42:0/94904703 shutdown_connections 2024-07-04T17:31:17.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.300+0000 7efcdc400640 1 -- 172.21.15.42:0/94904703 wait complete. 2024-07-04T17:31:17.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.302+0000 7efcdc400640 1 Processor -- start 2024-07-04T17:31:17.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.302+0000 7efcdc400640 1 -- start start 2024-07-04T17:31:17.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.303+0000 7efcdc400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efcd40b02b0 0x7efcd4070930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:17.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.303+0000 7efcdc400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efcd406c710 con 0x7efcd40b02b0 2024-07-04T17:31:17.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.303+0000 7efcd9a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efcd40b02b0 0x7efcd4070930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:17.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.303+0000 7efcd9a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efcd40b02b0 0x7efcd4070930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42548/0 (socket says 172.21.15.42:42548) 2024-07-04T17:31:17.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.303+0000 7efcd9a00640 1 -- 172.21.15.42:0/2527434264 learned_addr learned my addr 172.21.15.42:0/2527434264 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:17.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.304+0000 7efcd9a00640 1 -- 172.21.15.42:0/2527434264 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd4072380 con 0x7efcd40b02b0 2024-07-04T17:31:17.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.304+0000 7efcd9a00640 1 --2- 172.21.15.42:0/2527434264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efcd40b02b0 0x7efcd4070930 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7efcc8009bf0 tx=0x7efcc8003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:17.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.304+0000 7efcd1600640 1 -- 172.21.15.42:0/2527434264 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efcc8046070 con 0x7efcd40b02b0 2024-07-04T17:31:17.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.304+0000 7efcd1600640 1 -- 172.21.15.42:0/2527434264 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efcc80044f0 con 0x7efcd40b02b0 2024-07-04T17:31:17.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.304+0000 7efcd1600640 1 -- 172.21.15.42:0/2527434264 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efcc8033070 con 0x7efcd40b02b0 2024-07-04T17:31:17.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.304+0000 7efcdc400640 1 -- 172.21.15.42:0/2527434264 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcd40adb80 con 0x7efcd40b02b0 2024-07-04T17:31:17.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.304+0000 7efcdc400640 1 -- 172.21.15.42:0/2527434264 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcd41e16a0 con 0x7efcd40b02b0 2024-07-04T17:31:17.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.306+0000 7efcd1600640 1 -- 172.21.15.42:0/2527434264 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 49615+0+0 (secure 0 0 0) 0x7efcc8049070 con 0x7efcd40b02b0 2024-07-04T17:31:17.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.306+0000 7efcdc400640 1 -- 172.21.15.42:0/2527434264 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcd406cd50 con 0x7efcd40b02b0 2024-07-04T17:31:17.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.306+0000 7efcd1600640 1 --2- 172.21.15.42:0/2527434264 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7efcb403d4f0 0x7efcb403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:17.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.306+0000 7efcd1600640 1 -- 172.21.15.42:0/2527434264 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 4303+0+0 (secure 0 0 0) 0x7efcc8073150 con 0x7efcd40b02b0 2024-07-04T17:31:17.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.306+0000 7efcd9000640 1 -- 172.21.15.42:0/2527434264 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7efcb403d4f0 msgr2=0x7efcb403fa00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/3473020706 2024-07-04T17:31:17.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.306+0000 7efcd9000640 1 --2- 172.21.15.42:0/2527434264 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7efcb403d4f0 0x7efcb403fa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:31:17.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.312+0000 7efcd1600640 1 -- 172.21.15.42:0/2527434264 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7efcc803e070 con 0x7efcd40b02b0 2024-07-04T17:31:17.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.507+0000 7efcd9000640 1 -- 172.21.15.42:0/2527434264 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7efcb403d4f0 msgr2=0x7efcb403fa00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/3473020706 2024-07-04T17:31:17.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.507+0000 7efcd9000640 1 --2- 172.21.15.42:0/2527434264 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7efcb403d4f0 0x7efcb403fa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:31:17.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.534+0000 7efcdc400640 1 -- 172.21.15.42:0/2527434264 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efcd40beb10 con 0x7efcd40b02b0 2024-07-04T17:31:17.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.535+0000 7efcd1600640 1 -- 172.21.15.42:0/2527434264 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v76) ==== 80+0+0 (secure 0 0 0) 0x7efcc8004810 con 0x7efcd40b02b0 2024-07-04T17:31:17.536 INFO:teuthology.orchestra.run.smithi042.stderr:listed 0 entries 2024-07-04T17:31:17.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.538+0000 7efcdc400640 1 -- 172.21.15.42:0/2527434264 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7efcb403d4f0 msgr2=0x7efcb403fa00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:31:17.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.538+0000 7efcdc400640 1 --2- 172.21.15.42:0/2527434264 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7efcb403d4f0 0x7efcb403fa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:17.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.538+0000 7efcdc400640 1 -- 172.21.15.42:0/2527434264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efcd40b02b0 msgr2=0x7efcd4070930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:17.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.538+0000 7efcdc400640 1 --2- 172.21.15.42:0/2527434264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efcd40b02b0 0x7efcd4070930 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7efcc8009bf0 tx=0x7efcc8003f30 comp rx=0 tx=0).stop 2024-07-04T17:31:17.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.538+0000 7efcdc400640 1 -- 172.21.15.42:0/2527434264 shutdown_connections 2024-07-04T17:31:17.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.538+0000 7efcdc400640 1 --2- 172.21.15.42:0/2527434264 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7efcb403d4f0 0x7efcb403fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:17.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.538+0000 7efcdc400640 1 --2- 172.21.15.42:0/2527434264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efcd40b02b0 0x7efcd4070930 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:17.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.538+0000 7efcdc400640 1 -- 172.21.15.42:0/2527434264 >> 172.21.15.42:0/2527434264 conn(0x7efcd40bf700 msgr2=0x7efcd40aaca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:17.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.539+0000 7efcdc400640 1 -- 172.21.15.42:0/2527434264 shutdown_connections 2024-07-04T17:31:17.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:17.539+0000 7efcdc400640 1 -- 172.21.15.42:0/2527434264 wait complete. 2024-07-04T17:31:17.586 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi042.front.sepia.ceph.com... 2024-07-04T17:31:17.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:31:17.586 DEBUG:teuthology.orchestra.run.smithi042:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-07-04T17:31:17.658 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi042.stderr:ceph-fuse[56947]: fuse finished with error 0 and tester_r 0 2024-07-04T17:31:17.662 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-07-04T17:31:17.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:31:17.662 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-07-04T17:31:17.729 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:31:18.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.045+0000 7f9801000640 1 -- 172.21.15.42:0/308198364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97fc0bad70 msgr2=0x7f97fc06c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:18.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.045+0000 7f9801000640 1 --2- 172.21.15.42:0/308198364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97fc0bad70 0x7f97fc06c2c0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f97ec009fd0 tx=0x7f97ec02b0b0 comp rx=0 tx=0).stop 2024-07-04T17:31:18.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.046+0000 7f9801000640 1 -- 172.21.15.42:0/308198364 shutdown_connections 2024-07-04T17:31:18.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.046+0000 7f9801000640 1 --2- 172.21.15.42:0/308198364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97fc0bad70 0x7f97fc06c2c0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:18.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.046+0000 7f9801000640 1 -- 172.21.15.42:0/308198364 >> 172.21.15.42:0/308198364 conn(0x7f97fc0bebc0 msgr2=0x7f97fc0befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:18.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.046+0000 7f9801000640 1 -- 172.21.15.42:0/308198364 shutdown_connections 2024-07-04T17:31:18.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.046+0000 7f9801000640 1 -- 172.21.15.42:0/308198364 wait complete. 2024-07-04T17:31:18.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.048+0000 7f9801000640 1 Processor -- start 2024-07-04T17:31:18.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.048+0000 7f9801000640 1 -- start start 2024-07-04T17:31:18.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.049+0000 7f9801000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97fc0b5650 0x7f97fc150020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:18.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.049+0000 7f9801000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc0b27c0 con 0x7f97fc0b5650 2024-07-04T17:31:18.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.049+0000 7f97fb400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97fc0b5650 0x7f97fc150020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:18.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.049+0000 7f97fb400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97fc0b5650 0x7f97fc150020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42574/0 (socket says 172.21.15.42:42574) 2024-07-04T17:31:18.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.049+0000 7f97fb400640 1 -- 172.21.15.42:0/3332309702 learned_addr learned my addr 172.21.15.42:0/3332309702 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:18.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.050+0000 7f97fb400640 1 -- 172.21.15.42:0/3332309702 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97fc0b77a0 con 0x7f97fc0b5650 2024-07-04T17:31:18.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.050+0000 7f97fb400640 1 --2- 172.21.15.42:0/3332309702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97fc0b5650 0x7f97fc150020 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f97ec00b310 tx=0x7f97ec009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:18.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.050+0000 7f97f3400640 1 -- 172.21.15.42:0/3332309702 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97ec003aa0 con 0x7f97fc0b5650 2024-07-04T17:31:18.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.050+0000 7f97f3400640 1 -- 172.21.15.42:0/3332309702 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f97ec003d30 con 0x7f97fc0b5650 2024-07-04T17:31:18.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.050+0000 7f97f3400640 1 -- 172.21.15.42:0/3332309702 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97ec0338b0 con 0x7f97fc0b5650 2024-07-04T17:31:18.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.050+0000 7f9801000640 1 -- 172.21.15.42:0/3332309702 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97fc0aeae0 con 0x7f97fc0b5650 2024-07-04T17:31:18.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.051+0000 7f9801000640 1 -- 172.21.15.42:0/3332309702 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97fc13c450 con 0x7f97fc0b5650 2024-07-04T17:31:18.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.052+0000 7f9801000640 1 -- 172.21.15.42:0/3332309702 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97fc06c2c0 con 0x7f97fc0b5650 2024-07-04T17:31:18.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.052+0000 7f97f3400640 1 -- 172.21.15.42:0/3332309702 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 49615+0+0 (secure 0 0 0) 0x7f97ec042020 con 0x7f97fc0b5650 2024-07-04T17:31:18.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.052+0000 7f97f3400640 1 --2- 172.21.15.42:0/3332309702 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f97dc03d4f0 0x7f97dc03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:18.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.052+0000 7f97f3400640 1 -- 172.21.15.42:0/3332309702 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 4303+0+0 (secure 0 0 0) 0x7f97ec040020 con 0x7f97fc0b5650 2024-07-04T17:31:18.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.052+0000 7f97faa00640 1 -- 172.21.15.42:0/3332309702 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f97dc03d4f0 msgr2=0x7f97dc03fa00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/3473020706 2024-07-04T17:31:18.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.052+0000 7f97faa00640 1 --2- 172.21.15.42:0/3332309702 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f97dc03d4f0 0x7f97dc03fa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:31:18.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.057+0000 7f97f3400640 1 -- 172.21.15.42:0/3332309702 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f97ec049020 con 0x7f97fc0b5650 2024-07-04T17:31:18.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.253+0000 7f97faa00640 1 -- 172.21.15.42:0/3332309702 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f97dc03d4f0 msgr2=0x7f97dc03fa00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/3473020706 2024-07-04T17:31:18.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.253+0000 7f97faa00640 1 --2- 172.21.15.42:0/3332309702 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f97dc03d4f0 0x7f97dc03fa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:31:18.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.306+0000 7f9801000640 1 -- 172.21.15.42:0/3332309702 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97fc0bf2f0 con 0x7f97fc0b5650 2024-07-04T17:31:18.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.307+0000 7f97f3400640 1 -- 172.21.15.42:0/3332309702 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+7369 (secure 0 0 0) 0x7f97ec037e00 con 0x7f97fc0b5650 2024-07-04T17:31:18.309 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:31:18.309 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":60,"btime":"2024-07-04T17:31:09:763642+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5017,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/2312272332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2312272332},{"type":"v1","addr":"172.21.15.42:6827","nonce":2312272332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5020,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1285161495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1285161495},{"type":"v1","addr":"172.21.15.42:6829","nonce":1285161495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5023,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/1583070649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1583070649},{"type":"v1","addr":"172.21.15.42:6831","nonce":1583070649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:30:46.947014+0000","modified":"2024-07-04T17:30:49.991386+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5029":{"gid":5029,"name":"e","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6835/1458267161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1458267161},{"type":"v1","addr":"172.21.15.42:6835","nonce":1458267161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5029,"qdb_cluster":[5029]},"id":6},{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:31:06.323313+0000","modified":"2024-07-04T17:31:09.367074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5026":{"gid":5026,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6833/2038326930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2038326930},{"type":"v1","addr":"172.21.15.42:6833","nonce":2038326930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5026,"qdb_cluster":[5026]},"id":7}]} 2024-07-04T17:31:18.309 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 60 2024-07-04T17:31:18.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.310+0000 7f9801000640 1 -- 172.21.15.42:0/3332309702 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f97dc03d4f0 msgr2=0x7f97dc03fa00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:31:18.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.310+0000 7f9801000640 1 --2- 172.21.15.42:0/3332309702 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f97dc03d4f0 0x7f97dc03fa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:18.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.310+0000 7f9801000640 1 -- 172.21.15.42:0/3332309702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97fc0b5650 msgr2=0x7f97fc150020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:18.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.310+0000 7f9801000640 1 --2- 172.21.15.42:0/3332309702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97fc0b5650 0x7f97fc150020 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f97ec00b310 tx=0x7f97ec009ae0 comp rx=0 tx=0).stop 2024-07-04T17:31:18.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.310+0000 7f9801000640 1 -- 172.21.15.42:0/3332309702 shutdown_connections 2024-07-04T17:31:18.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.310+0000 7f9801000640 1 --2- 172.21.15.42:0/3332309702 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f97dc03d4f0 0x7f97dc03fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:18.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.310+0000 7f9801000640 1 --2- 172.21.15.42:0/3332309702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97fc0b5650 0x7f97fc150020 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:18.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.310+0000 7f9801000640 1 -- 172.21.15.42:0/3332309702 >> 172.21.15.42:0/3332309702 conn(0x7f97fc0bebc0 msgr2=0x7f97fc13dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:18.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.311+0000 7f9801000640 1 -- 172.21.15.42:0/3332309702 shutdown_connections 2024-07-04T17:31:18.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.311+0000 7f9801000640 1 -- 172.21.15.42:0/3332309702 wait complete. 2024-07-04T17:31:18.339 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:31:18.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.610+0000 7f5cb8800640 1 -- 172.21.15.42:0/1496313724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cb00a9fa0 msgr2=0x7f5cb00b0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:18.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.610+0000 7f5cb8800640 1 --2- 172.21.15.42:0/1496313724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cb00a9fa0 0x7f5cb00b0010 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f5ca00097b0 tx=0x7f5ca002a4a0 comp rx=0 tx=0).stop 2024-07-04T17:31:18.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.610+0000 7f5cb8800640 1 -- 172.21.15.42:0/1496313724 shutdown_connections 2024-07-04T17:31:18.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.610+0000 7f5cb8800640 1 --2- 172.21.15.42:0/1496313724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cb00a9fa0 0x7f5cb00b0010 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:18.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.610+0000 7f5cb8800640 1 -- 172.21.15.42:0/1496313724 >> 172.21.15.42:0/1496313724 conn(0x7f5cb00bf700 msgr2=0x7f5cb00af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:18.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.611+0000 7f5cb8800640 1 -- 172.21.15.42:0/1496313724 shutdown_connections 2024-07-04T17:31:18.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.611+0000 7f5cb8800640 1 -- 172.21.15.42:0/1496313724 wait complete. 2024-07-04T17:31:18.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.613+0000 7f5cb8800640 1 Processor -- start 2024-07-04T17:31:18.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.613+0000 7f5cb8800640 1 -- start start 2024-07-04T17:31:18.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.614+0000 7f5cb8800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cb00a9fa0 0x7f5cb00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:18.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.614+0000 7f5cb8800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb00aae30 con 0x7f5cb00a9fa0 2024-07-04T17:31:18.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.614+0000 7f5cb5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cb00a9fa0 0x7f5cb00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:18.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.614+0000 7f5cb5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cb00a9fa0 0x7f5cb00b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42594/0 (socket says 172.21.15.42:42594) 2024-07-04T17:31:18.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.614+0000 7f5cb5e00640 1 -- 172.21.15.42:0/2038811339 learned_addr learned my addr 172.21.15.42:0/2038811339 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:18.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.615+0000 7f5cb5e00640 1 -- 172.21.15.42:0/2038811339 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb00b5a20 con 0x7f5cb00a9fa0 2024-07-04T17:31:18.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.615+0000 7f5cb5e00640 1 --2- 172.21.15.42:0/2038811339 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cb00a9fa0 0x7f5cb00b5f40 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400c2f0 tx=0x7f5ca400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:18.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.615+0000 7f5caa000640 1 -- 172.21.15.42:0/2038811339 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ca4017070 con 0x7f5cb00a9fa0 2024-07-04T17:31:18.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.615+0000 7f5caa000640 1 -- 172.21.15.42:0/2038811339 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ca4005410 con 0x7f5cb00a9fa0 2024-07-04T17:31:18.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.615+0000 7f5caa000640 1 -- 172.21.15.42:0/2038811339 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ca4003780 con 0x7f5cb00a9fa0 2024-07-04T17:31:18.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.615+0000 7f5cb8800640 1 -- 172.21.15.42:0/2038811339 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cb013f7c0 con 0x7f5cb00a9fa0 2024-07-04T17:31:18.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.616+0000 7f5cb8800640 1 -- 172.21.15.42:0/2038811339 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cb00b5470 con 0x7f5cb00a9fa0 2024-07-04T17:31:18.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.617+0000 7f5cb8800640 1 -- 172.21.15.42:0/2038811339 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cb00ab560 con 0x7f5cb00a9fa0 2024-07-04T17:31:18.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.617+0000 7f5caa000640 1 -- 172.21.15.42:0/2038811339 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 49615+0+0 (secure 0 0 0) 0x7f5ca401a070 con 0x7f5cb00a9fa0 2024-07-04T17:31:18.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.617+0000 7f5caa000640 1 --2- 172.21.15.42:0/2038811339 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f5c9403d4f0 0x7f5c9403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:18.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.617+0000 7f5caa000640 1 -- 172.21.15.42:0/2038811339 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 4303+0+0 (secure 0 0 0) 0x7f5ca4050f00 con 0x7f5cb00a9fa0 2024-07-04T17:31:18.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.618+0000 7f5cb5400640 1 -- 172.21.15.42:0/2038811339 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f5c9403d4f0 msgr2=0x7f5c9403fa00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/3473020706 2024-07-04T17:31:18.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.618+0000 7f5cb5400640 1 --2- 172.21.15.42:0/2038811339 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f5c9403d4f0 0x7f5c9403fa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:31:18.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.622+0000 7f5caa000640 1 -- 172.21.15.42:0/2038811339 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f5ca401ad80 con 0x7f5cb00a9fa0 2024-07-04T17:31:18.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.819+0000 7f5cb5400640 1 -- 172.21.15.42:0/2038811339 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f5c9403d4f0 msgr2=0x7f5c9403fa00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/3473020706 2024-07-04T17:31:18.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.819+0000 7f5cb5400640 1 --2- 172.21.15.42:0/2038811339 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f5c9403d4f0 0x7f5c9403fa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:31:18.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.875+0000 7f5cb8800640 1 -- 172.21.15.42:0/2038811339 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5cb0069b70 con 0x7f5cb00a9fa0 2024-07-04T17:31:18.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.876+0000 7f5caa000640 1 -- 172.21.15.42:0/2038811339 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+7369 (secure 0 0 0) 0x7f5ca401b3b0 con 0x7f5cb00a9fa0 2024-07-04T17:31:18.878 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:31:18.878 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":60,"btime":"2024-07-04T17:31:09:763642+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5017,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/2312272332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2312272332},{"type":"v1","addr":"172.21.15.42:6827","nonce":2312272332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5020,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1285161495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1285161495},{"type":"v1","addr":"172.21.15.42:6829","nonce":1285161495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5023,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/1583070649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1583070649},{"type":"v1","addr":"172.21.15.42:6831","nonce":1583070649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:30:46.947014+0000","modified":"2024-07-04T17:30:49.991386+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5029":{"gid":5029,"name":"e","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6835/1458267161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1458267161},{"type":"v1","addr":"172.21.15.42:6835","nonce":1458267161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5029,"qdb_cluster":[5029]},"id":6},{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:31:06.323313+0000","modified":"2024-07-04T17:31:09.367074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5026":{"gid":5026,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6833/2038326930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2038326930},{"type":"v1","addr":"172.21.15.42:6833","nonce":2038326930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5026,"qdb_cluster":[5026]},"id":7}]} 2024-07-04T17:31:18.878 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 60 2024-07-04T17:31:18.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.879+0000 7f5cb8800640 1 -- 172.21.15.42:0/2038811339 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f5c9403d4f0 msgr2=0x7f5c9403fa00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:31:18.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.879+0000 7f5cb8800640 1 --2- 172.21.15.42:0/2038811339 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f5c9403d4f0 0x7f5c9403fa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:18.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.879+0000 7f5cb8800640 1 -- 172.21.15.42:0/2038811339 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cb00a9fa0 msgr2=0x7f5cb00b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:18.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.879+0000 7f5cb8800640 1 --2- 172.21.15.42:0/2038811339 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cb00a9fa0 0x7f5cb00b5f40 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400c2f0 tx=0x7f5ca400c7b0 comp rx=0 tx=0).stop 2024-07-04T17:31:18.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.879+0000 7f5cb8800640 1 -- 172.21.15.42:0/2038811339 shutdown_connections 2024-07-04T17:31:18.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.879+0000 7f5cb8800640 1 --2- 172.21.15.42:0/2038811339 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7f5c9403d4f0 0x7f5c9403fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:18.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.879+0000 7f5cb8800640 1 --2- 172.21.15.42:0/2038811339 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cb00a9fa0 0x7f5cb00b5f40 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:18.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.879+0000 7f5cb8800640 1 -- 172.21.15.42:0/2038811339 >> 172.21.15.42:0/2038811339 conn(0x7f5cb00bf700 msgr2=0x7f5cb0141280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:18.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.879+0000 7f5cb8800640 1 -- 172.21.15.42:0/2038811339 shutdown_connections 2024-07-04T17:31:18.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:18.879+0000 7f5cb8800640 1 -- 172.21.15.42:0/2038811339 wait complete. 2024-07-04T17:31:18.905 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:31:19.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.176+0000 7ff4cb200640 1 -- 172.21.15.42:0/3035623546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4cc06c200 msgr2=0x7ff4cc14ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:19.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.176+0000 7ff4cb200640 1 --2- 172.21.15.42:0/3035623546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4cc06c200 0x7ff4cc14ef10 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc0097b0 tx=0x7ff4bc02a4b0 comp rx=0 tx=0).stop 2024-07-04T17:31:19.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.176+0000 7ff4cb200640 1 -- 172.21.15.42:0/3035623546 shutdown_connections 2024-07-04T17:31:19.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.176+0000 7ff4cb200640 1 --2- 172.21.15.42:0/3035623546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4cc06c200 0x7ff4cc14ef10 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:19.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.176+0000 7ff4cb200640 1 -- 172.21.15.42:0/3035623546 >> 172.21.15.42:0/3035623546 conn(0x7ff4cc0bebc0 msgr2=0x7ff4cc0befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:19.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.176+0000 7ff4cb200640 1 -- 172.21.15.42:0/3035623546 shutdown_connections 2024-07-04T17:31:19.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.176+0000 7ff4cb200640 1 -- 172.21.15.42:0/3035623546 wait complete. 2024-07-04T17:31:19.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.178+0000 7ff4cb200640 1 Processor -- start 2024-07-04T17:31:19.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.178+0000 7ff4cb200640 1 -- start start 2024-07-04T17:31:19.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.179+0000 7ff4cb200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4cc06c200 0x7ff4cc147f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:19.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.179+0000 7ff4cb200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc14ae50 con 0x7ff4cc06c200 2024-07-04T17:31:19.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.179+0000 7ff4c9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4cc06c200 0x7ff4cc147f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:19.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.179+0000 7ff4c9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4cc06c200 0x7ff4cc147f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42612/0 (socket says 172.21.15.42:42612) 2024-07-04T17:31:19.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.179+0000 7ff4c9e00640 1 -- 172.21.15.42:0/3924421745 learned_addr learned my addr 172.21.15.42:0/3924421745 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:19.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.180+0000 7ff4c9e00640 1 -- 172.21.15.42:0/3924421745 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4cc1d5bf0 con 0x7ff4cc06c200 2024-07-04T17:31:19.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.180+0000 7ff4c9e00640 1 --2- 172.21.15.42:0/3924421745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4cc06c200 0x7ff4cc147f20 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc0098d0 tx=0x7ff4bc033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:19.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.180+0000 7ff4c2000640 1 -- 172.21.15.42:0/3924421745 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4bc031bc0 con 0x7ff4cc06c200 2024-07-04T17:31:19.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.180+0000 7ff4c2000640 1 -- 172.21.15.42:0/3924421745 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4bc031e50 con 0x7ff4cc06c200 2024-07-04T17:31:19.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.180+0000 7ff4cb200640 1 -- 172.21.15.42:0/3924421745 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4cc0b6530 con 0x7ff4cc06c200 2024-07-04T17:31:19.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.180+0000 7ff4c2000640 1 -- 172.21.15.42:0/3924421745 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4bc033260 con 0x7ff4cc06c200 2024-07-04T17:31:19.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.181+0000 7ff4cb200640 1 -- 172.21.15.42:0/3924421745 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4cc0b7130 con 0x7ff4cc06c200 2024-07-04T17:31:19.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.182+0000 7ff4cb200640 1 -- 172.21.15.42:0/3924421745 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4cc14b490 con 0x7ff4cc06c200 2024-07-04T17:31:19.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.182+0000 7ff4c2000640 1 -- 172.21.15.42:0/3924421745 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 27) ==== 49615+0+0 (secure 0 0 0) 0x7ff4bc033400 con 0x7ff4cc06c200 2024-07-04T17:31:19.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.182+0000 7ff4c2000640 1 --2- 172.21.15.42:0/3924421745 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ff4a803d4a0 0x7ff4a803f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:19.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.182+0000 7ff4c2000640 1 -- 172.21.15.42:0/3924421745 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 4303+0+0 (secure 0 0 0) 0x7ff4bc073250 con 0x7ff4cc06c200 2024-07-04T17:31:19.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.182+0000 7ff4c3e00640 1 -- 172.21.15.42:0/3924421745 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ff4a803d4a0 msgr2=0x7ff4a803f9b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/3473020706 2024-07-04T17:31:19.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.183+0000 7ff4c3e00640 1 --2- 172.21.15.42:0/3924421745 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ff4a803d4a0 0x7ff4a803f9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:31:19.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.192+0000 7ff4c2000640 1 -- 172.21.15.42:0/3924421745 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7ff4bc03f030 con 0x7ff4cc06c200 2024-07-04T17:31:19.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.384+0000 7ff4c3e00640 1 -- 172.21.15.42:0/3924421745 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ff4a803d4a0 msgr2=0x7ff4a803f9b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6824/3473020706 2024-07-04T17:31:19.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.384+0000 7ff4c3e00640 1 --2- 172.21.15.42:0/3924421745 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ff4a803d4a0 0x7ff4a803f9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:31:19.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.408+0000 7ff4cb200640 1 -- 172.21.15.42:0/3924421745 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4cc0b68a0 con 0x7ff4cc06c200 2024-07-04T17:31:19.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.411+0000 7ff4c2000640 1 -- 172.21.15.42:0/3924421745 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+14450 (secure 0 0 0) 0x7ff4bc003b10 con 0x7ff4cc06c200 2024-07-04T17:31:19.412 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:31:19.412 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":76,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:31:07.746950+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:30:45.017030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.4099999666213989,"score_stable":1.4099999666213989,"optimal_score":1,"raw_score_acting":1.4099999666213989,"raw_score_stable":1.4099999666213989,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-07-04T17:30:45.910003+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:31:03.814034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:31:05.281219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1699999570846558,"score_stable":1.1699999570846558,"optimal_score":1,"raw_score_acting":1.1699999570846558,"raw_score_stable":1.1699999570846558,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:31:19.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.413+0000 7ff4cb200640 1 -- 172.21.15.42:0/3924421745 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ff4a803d4a0 msgr2=0x7ff4a803f9b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:31:19.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.413+0000 7ff4cb200640 1 --2- 172.21.15.42:0/3924421745 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ff4a803d4a0 0x7ff4a803f9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:19.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.413+0000 7ff4cb200640 1 -- 172.21.15.42:0/3924421745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4cc06c200 msgr2=0x7ff4cc147f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:19.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.413+0000 7ff4cb200640 1 --2- 172.21.15.42:0/3924421745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4cc06c200 0x7ff4cc147f20 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc0098d0 tx=0x7ff4bc033e40 comp rx=0 tx=0).stop 2024-07-04T17:31:19.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.414+0000 7ff4cb200640 1 -- 172.21.15.42:0/3924421745 shutdown_connections 2024-07-04T17:31:19.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.414+0000 7ff4cb200640 1 --2- 172.21.15.42:0/3924421745 >> [v2:172.21.15.42:6824/3473020706,v1:172.21.15.42:6825/3473020706] conn(0x7ff4a803d4a0 0x7ff4a803f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:19.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.414+0000 7ff4cb200640 1 --2- 172.21.15.42:0/3924421745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4cc06c200 0x7ff4cc147f20 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:19.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.414+0000 7ff4cb200640 1 -- 172.21.15.42:0/3924421745 >> 172.21.15.42:0/3924421745 conn(0x7ff4cc0bebc0 msgr2=0x7ff4cc0bbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:19.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.414+0000 7ff4cb200640 1 -- 172.21.15.42:0/3924421745 shutdown_connections 2024-07-04T17:31:19.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.414+0000 7ff4cb200640 1 -- 172.21.15.42:0/3924421745 wait complete. 2024-07-04T17:31:19.444 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-04T17:31:19.444 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:31:19.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.714+0000 7fb8cf000640 1 -- 172.21.15.42:0/1875915039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8c80a9fa0 msgr2=0x7fb8c80b0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:19.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.714+0000 7fb8cf000640 1 --2- 172.21.15.42:0/1875915039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8c80a9fa0 0x7fb8c80b0010 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0005fd0 tx=0x7fb8b002a560 comp rx=0 tx=0).stop 2024-07-04T17:31:19.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.714+0000 7fb8cf000640 1 -- 172.21.15.42:0/1875915039 shutdown_connections 2024-07-04T17:31:19.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.714+0000 7fb8cf000640 1 --2- 172.21.15.42:0/1875915039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8c80a9fa0 0x7fb8c80b0010 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:19.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.714+0000 7fb8cf000640 1 -- 172.21.15.42:0/1875915039 >> 172.21.15.42:0/1875915039 conn(0x7fb8c80bf700 msgr2=0x7fb8c80af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:19.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.714+0000 7fb8cf000640 1 -- 172.21.15.42:0/1875915039 shutdown_connections 2024-07-04T17:31:19.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.714+0000 7fb8cf000640 1 -- 172.21.15.42:0/1875915039 wait complete. 2024-07-04T17:31:19.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.716+0000 7fb8cf000640 1 Processor -- start 2024-07-04T17:31:19.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.716+0000 7fb8cf000640 1 -- start start 2024-07-04T17:31:19.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.717+0000 7fb8cf000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8c80a9fa0 0x7fb8c81d9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:19.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.718+0000 7fb8cf000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c80aae30 con 0x7fb8c80a9fa0 2024-07-04T17:31:19.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.718+0000 7fb8c7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8c80a9fa0 0x7fb8c81d9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:19.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.718+0000 7fb8c7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8c80a9fa0 0x7fb8c81d9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42636/0 (socket says 172.21.15.42:42636) 2024-07-04T17:31:19.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.718+0000 7fb8c7e00640 1 -- 172.21.15.42:0/1928547918 learned_addr learned my addr 172.21.15.42:0/1928547918 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:19.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.719+0000 7fb8c7e00640 1 -- 172.21.15.42:0/1928547918 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c81d8270 con 0x7fb8c80a9fa0 2024-07-04T17:31:19.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.719+0000 7fb8c7e00640 1 --2- 172.21.15.42:0/1928547918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8c80a9fa0 0x7fb8c81d9c00 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800c320 tx=0x7fb8b800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:19.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.719+0000 7fb8c4c00640 1 -- 172.21.15.42:0/1928547918 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8b8017070 con 0x7fb8c80a9fa0 2024-07-04T17:31:19.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.719+0000 7fb8c4c00640 1 -- 172.21.15.42:0/1928547918 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8b8005310 con 0x7fb8c80a9fa0 2024-07-04T17:31:19.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.719+0000 7fb8cf000640 1 -- 172.21.15.42:0/1928547918 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8c81e3670 con 0x7fb8c80a9fa0 2024-07-04T17:31:19.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.719+0000 7fb8c4c00640 1 -- 172.21.15.42:0/1928547918 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8b80035f0 con 0x7fb8c80a9fa0 2024-07-04T17:31:19.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.719+0000 7fb8cf000640 1 -- 172.21.15.42:0/1928547918 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8c81d3ed0 con 0x7fb8c80a9fa0 2024-07-04T17:31:19.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.721+0000 7fb8cf000640 1 -- 172.21.15.42:0/1928547918 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8c80ab520 con 0x7fb8c80a9fa0 2024-07-04T17:31:19.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.721+0000 7fb8c4c00640 1 -- 172.21.15.42:0/1928547918 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 28) ==== 49122+0+0 (secure 0 0 0) 0x7fb8b801a070 con 0x7fb8c80a9fa0 2024-07-04T17:31:19.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.721+0000 7fb8c4c00640 1 -- 172.21.15.42:0/1928547918 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4733+0+0 (secure 0 0 0) 0x7fb8b8050810 con 0x7fb8c80a9fa0 2024-07-04T17:31:19.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.726+0000 7fb8c4c00640 1 -- 172.21.15.42:0/1928547918 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fb8b801bc30 con 0x7fb8c80a9fa0 2024-07-04T17:31:19.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.981+0000 7fb8cf000640 1 -- 172.21.15.42:0/1928547918 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8c8069b70 con 0x7fb8c80a9fa0 2024-07-04T17:31:19.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.982+0000 7fb8c4c00640 1 -- 172.21.15.42:0/1928547918 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+7369 (secure 0 0 0) 0x7fb8b8010be0 con 0x7fb8c80a9fa0 2024-07-04T17:31:19.984 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:31:19.984 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":60,"btime":"2024-07-04T17:31:09:763642+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5017,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/2312272332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2312272332},{"type":"v1","addr":"172.21.15.42:6827","nonce":2312272332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5020,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1285161495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1285161495},{"type":"v1","addr":"172.21.15.42:6829","nonce":1285161495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5023,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/1583070649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1583070649},{"type":"v1","addr":"172.21.15.42:6831","nonce":1583070649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:30:46.947014+0000","modified":"2024-07-04T17:30:49.991386+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5029":{"gid":5029,"name":"e","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6835/1458267161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1458267161},{"type":"v1","addr":"172.21.15.42:6835","nonce":1458267161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5029,"qdb_cluster":[5029]},"id":6},{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:31:06.323313+0000","modified":"2024-07-04T17:31:09.367074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5026":{"gid":5026,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6833/2038326930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2038326930},{"type":"v1","addr":"172.21.15.42:6833","nonce":2038326930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5026,"qdb_cluster":[5026]},"id":7}]} 2024-07-04T17:31:19.984 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 60 2024-07-04T17:31:19.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.985+0000 7fb8cf000640 1 -- 172.21.15.42:0/1928547918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8c80a9fa0 msgr2=0x7fb8c81d9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:19.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.985+0000 7fb8cf000640 1 --2- 172.21.15.42:0/1928547918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8c80a9fa0 0x7fb8c81d9c00 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800c320 tx=0x7fb8b800c7e0 comp rx=0 tx=0).stop 2024-07-04T17:31:19.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.985+0000 7fb8cf000640 1 -- 172.21.15.42:0/1928547918 shutdown_connections 2024-07-04T17:31:19.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.985+0000 7fb8cf000640 1 --2- 172.21.15.42:0/1928547918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8c80a9fa0 0x7fb8c81d9c00 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:19.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.985+0000 7fb8cf000640 1 -- 172.21.15.42:0/1928547918 >> 172.21.15.42:0/1928547918 conn(0x7fb8c80bf700 msgr2=0x7fb8c80b1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:19.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.986+0000 7fb8cf000640 1 -- 172.21.15.42:0/1928547918 shutdown_connections 2024-07-04T17:31:19.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:19.986+0000 7fb8cf000640 1 -- 172.21.15.42:0/1928547918 wait complete. 2024-07-04T17:31:20.012 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:31:20.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.281+0000 7f0e13c00640 1 -- 172.21.15.42:0/3627306676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e0c0b02b0 msgr2=0x7f0e0c0b3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:20.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.281+0000 7f0e13c00640 1 --2- 172.21.15.42:0/3627306676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e0c0b02b0 0x7f0e0c0b3650 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc005fd0 tx=0x7f0dfc02a820 comp rx=0 tx=0).stop 2024-07-04T17:31:20.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.281+0000 7f0e13c00640 1 -- 172.21.15.42:0/3627306676 shutdown_connections 2024-07-04T17:31:20.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.281+0000 7f0e13c00640 1 --2- 172.21.15.42:0/3627306676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e0c0b02b0 0x7f0e0c0b3650 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:20.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.281+0000 7f0e13c00640 1 -- 172.21.15.42:0/3627306676 >> 172.21.15.42:0/3627306676 conn(0x7f0e0c0bf700 msgr2=0x7f0e0c0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:20.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.281+0000 7f0e13c00640 1 -- 172.21.15.42:0/3627306676 shutdown_connections 2024-07-04T17:31:20.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.281+0000 7f0e13c00640 1 -- 172.21.15.42:0/3627306676 wait complete. 2024-07-04T17:31:20.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.283+0000 7f0e13c00640 1 Processor -- start 2024-07-04T17:31:20.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.283+0000 7f0e13c00640 1 -- start start 2024-07-04T17:31:20.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.284+0000 7f0e13c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e0c0b02b0 0x7f0e0c0b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:20.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.284+0000 7f0e13c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c06c710 con 0x7f0e0c0b02b0 2024-07-04T17:31:20.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.285+0000 7f0e11200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e0c0b02b0 0x7f0e0c0b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:20.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.285+0000 7f0e11200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e0c0b02b0 0x7f0e0c0b7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42670/0 (socket says 172.21.15.42:42670) 2024-07-04T17:31:20.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.285+0000 7f0e11200640 1 -- 172.21.15.42:0/59549329 learned_addr learned my addr 172.21.15.42:0/59549329 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:20.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.285+0000 7f0e11200640 1 -- 172.21.15.42:0/59549329 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e0c0b9420 con 0x7f0e0c0b02b0 2024-07-04T17:31:20.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.285+0000 7f0e11200640 1 --2- 172.21.15.42:0/59549329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e0c0b02b0 0x7f0e0c0b7140 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc009c50 tx=0x7f0dfc032dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:20.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.286+0000 7f0e08c00640 1 -- 172.21.15.42:0/59549329 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0dfc037050 con 0x7f0e0c0b02b0 2024-07-04T17:31:20.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.286+0000 7f0e08c00640 1 -- 172.21.15.42:0/59549329 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0dfc043070 con 0x7f0e0c0b02b0 2024-07-04T17:31:20.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.286+0000 7f0e08c00640 1 -- 172.21.15.42:0/59549329 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0dfc0043e0 con 0x7f0e0c0b02b0 2024-07-04T17:31:20.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.286+0000 7f0e13c00640 1 -- 172.21.15.42:0/59549329 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e0c1e4570 con 0x7f0e0c0b02b0 2024-07-04T17:31:20.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.286+0000 7f0e13c00640 1 -- 172.21.15.42:0/59549329 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e0c0ba900 con 0x7f0e0c0b02b0 2024-07-04T17:31:20.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.287+0000 7f0e08c00640 1 -- 172.21.15.42:0/59549329 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 28) ==== 49122+0+0 (secure 0 0 0) 0x7f0dfc004580 con 0x7f0e0c0b02b0 2024-07-04T17:31:20.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.287+0000 7f0e13c00640 1 -- 172.21.15.42:0/59549329 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e0c06cd50 con 0x7f0e0c0b02b0 2024-07-04T17:31:20.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.288+0000 7f0e08c00640 1 -- 172.21.15.42:0/59549329 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4733+0+0 (secure 0 0 0) 0x7f0dfc02f070 con 0x7f0e0c0b02b0 2024-07-04T17:31:20.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.293+0000 7f0e08c00640 1 -- 172.21.15.42:0/59549329 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f0dfc048460 con 0x7f0e0c0b02b0 2024-07-04T17:31:20.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.542+0000 7f0e13c00640 1 -- 172.21.15.42:0/59549329 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e0c0be150 con 0x7f0e0c0b02b0 2024-07-04T17:31:20.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.543+0000 7f0e08c00640 1 -- 172.21.15.42:0/59549329 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+7369 (secure 0 0 0) 0x7f0dfc048790 con 0x7f0e0c0b02b0 2024-07-04T17:31:20.545 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:31:20.545 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":60,"btime":"2024-07-04T17:31:09:763642+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5017,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/2312272332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2312272332},{"type":"v1","addr":"172.21.15.42:6827","nonce":2312272332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5020,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1285161495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1285161495},{"type":"v1","addr":"172.21.15.42:6829","nonce":1285161495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5023,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/1583070649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1583070649},{"type":"v1","addr":"172.21.15.42:6831","nonce":1583070649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:30:46.947014+0000","modified":"2024-07-04T17:30:49.991386+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5029},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5029":{"gid":5029,"name":"e","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6835/1458267161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1458267161},{"type":"v1","addr":"172.21.15.42:6835","nonce":1458267161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5029,"qdb_cluster":[5029]},"id":6},{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:31:06.323313+0000","modified":"2024-07-04T17:31:09.367074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5026":{"gid":5026,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6833/2038326930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2038326930},{"type":"v1","addr":"172.21.15.42:6833","nonce":2038326930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5026,"qdb_cluster":[5026]},"id":7}]} 2024-07-04T17:31:20.545 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 60 2024-07-04T17:31:20.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.546+0000 7f0e13c00640 1 -- 172.21.15.42:0/59549329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e0c0b02b0 msgr2=0x7f0e0c0b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:20.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.546+0000 7f0e13c00640 1 --2- 172.21.15.42:0/59549329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e0c0b02b0 0x7f0e0c0b7140 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc009c50 tx=0x7f0dfc032dc0 comp rx=0 tx=0).stop 2024-07-04T17:31:20.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.546+0000 7f0e13c00640 1 -- 172.21.15.42:0/59549329 shutdown_connections 2024-07-04T17:31:20.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.546+0000 7f0e13c00640 1 --2- 172.21.15.42:0/59549329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e0c0b02b0 0x7f0e0c0b7140 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:20.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.546+0000 7f0e13c00640 1 -- 172.21.15.42:0/59549329 >> 172.21.15.42:0/59549329 conn(0x7f0e0c0bf700 msgr2=0x7f0e0c145320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:20.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.546+0000 7f0e13c00640 1 -- 172.21.15.42:0/59549329 shutdown_connections 2024-07-04T17:31:20.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.546+0000 7f0e13c00640 1 -- 172.21.15.42:0/59549329 wait complete. 2024-07-04T17:31:20.571 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:31:20.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.828+0000 7f19f6600640 1 -- 172.21.15.42:0/2967356541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19f00b3030 msgr2=0x7f19f00b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:20.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.828+0000 7f19f6600640 1 --2- 172.21.15.42:0/2967356541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19f00b3030 0x7f19f00b01f0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f19d800b2b0 tx=0x7f19d802ad10 comp rx=0 tx=0).stop 2024-07-04T17:31:20.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.828+0000 7f19f6600640 1 -- 172.21.15.42:0/2967356541 shutdown_connections 2024-07-04T17:31:20.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.828+0000 7f19f6600640 1 --2- 172.21.15.42:0/2967356541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19f00b3030 0x7f19f00b01f0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:20.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.828+0000 7f19f6600640 1 -- 172.21.15.42:0/2967356541 >> 172.21.15.42:0/2967356541 conn(0x7f19f00bf700 msgr2=0x7f19f00af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:20.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.828+0000 7f19f6600640 1 -- 172.21.15.42:0/2967356541 shutdown_connections 2024-07-04T17:31:20.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.828+0000 7f19f6600640 1 -- 172.21.15.42:0/2967356541 wait complete. 2024-07-04T17:31:20.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.830+0000 7f19f6600640 1 Processor -- start 2024-07-04T17:31:20.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.830+0000 7f19f6600640 1 -- start start 2024-07-04T17:31:20.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.831+0000 7f19f6600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19f006cea0 0x7f19f01d31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:20.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.831+0000 7f19f6600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f19f006ef50 con 0x7f19f006cea0 2024-07-04T17:31:20.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.832+0000 7f19ef400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19f006cea0 0x7f19f01d31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:20.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.832+0000 7f19ef400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19f006cea0 0x7f19f01d31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42684/0 (socket says 172.21.15.42:42684) 2024-07-04T17:31:20.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.832+0000 7f19ef400640 1 -- 172.21.15.42:0/1543254087 learned_addr learned my addr 172.21.15.42:0/1543254087 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:20.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.832+0000 7f19ef400640 1 -- 172.21.15.42:0/1543254087 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f0070260 con 0x7f19f006cea0 2024-07-04T17:31:20.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.833+0000 7f19ef400640 1 --2- 172.21.15.42:0/1543254087 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19f006cea0 0x7f19f01d31a0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f19e000c380 tx=0x7f19e000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:20.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.833+0000 7f19e7400640 1 -- 172.21.15.42:0/1543254087 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19e00116c0 con 0x7f19f006cea0 2024-07-04T17:31:20.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.833+0000 7f19e7400640 1 -- 172.21.15.42:0/1543254087 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19e0017070 con 0x7f19f006cea0 2024-07-04T17:31:20.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.833+0000 7f19e7400640 1 -- 172.21.15.42:0/1543254087 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19e0011b20 con 0x7f19f006cea0 2024-07-04T17:31:20.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.833+0000 7f19f6600640 1 -- 172.21.15.42:0/1543254087 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f0146b40 con 0x7f19f006cea0 2024-07-04T17:31:20.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.833+0000 7f19f6600640 1 -- 172.21.15.42:0/1543254087 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f01463d0 con 0x7f19f006cea0 2024-07-04T17:31:20.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.834+0000 7f19e7400640 1 -- 172.21.15.42:0/1543254087 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 28) ==== 49122+0+0 (secure 0 0 0) 0x7f19e0011cc0 con 0x7f19f006cea0 2024-07-04T17:31:20.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.834+0000 7f19f6600640 1 -- 172.21.15.42:0/1543254087 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19f00b01f0 con 0x7f19f006cea0 2024-07-04T17:31:20.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.835+0000 7f19e7400640 1 -- 172.21.15.42:0/1543254087 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4733+0+0 (secure 0 0 0) 0x7f19e0051350 con 0x7f19f006cea0 2024-07-04T17:31:20.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:20.840+0000 7f19e7400640 1 -- 172.21.15.42:0/1543254087 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f19e000a140 con 0x7f19f006cea0 2024-07-04T17:31:21.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.072+0000 7f19f6600640 1 -- 172.21.15.42:0/1543254087 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f19f0146600 con 0x7f19f006cea0 2024-07-04T17:31:21.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.074+0000 7f19e7400640 1 -- 172.21.15.42:0/1543254087 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+15073 (secure 0 0 0) 0x7f19e0051080 con 0x7f19f006cea0 2024-07-04T17:31:21.075 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:31:21.075 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":77,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:31:19.572227+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:30:45.017030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.4099999666213989,"score_stable":1.4099999666213989,"optimal_score":1,"raw_score_acting":1.4099999666213989,"raw_score_stable":1.4099999666213989,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-07-04T17:30:45.910003+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:31:03.814034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:31:05.281219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1699999570846558,"score_stable":1.1699999570846558,"optimal_score":1,"raw_score_acting":1.1699999570846558,"raw_score_stable":1.1699999570846558,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1725822177":"2024-07-05T17:31:19.572051+0000","172.21.15.42:0/1378827460":"2024-07-05T17:31:19.572051+0000","172.21.15.42:0/2771347343":"2024-07-05T17:31:19.572051+0000","172.21.15.42:0/2082652266":"2024-07-05T17:31:19.572051+0000","172.21.15.42:6825/3473020706":"2024-07-05T17:31:19.572051+0000","172.21.15.42:6824/3473020706":"2024-07-05T17:31:19.572051+0000","172.21.15.42:0/963042308":"2024-07-05T17:31:19.572051+0000","172.21.15.42:0/404958516":"2024-07-05T17:31:19.572051+0000","172.21.15.42:0/1340275465":"2024-07-05T17:31:19.572051+0000","172.21.15.42:0/1154056961":"2024-07-05T17:31:19.572051+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:31:21.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.077+0000 7f19f6600640 1 -- 172.21.15.42:0/1543254087 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19f006cea0 msgr2=0x7f19f01d31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:21.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.077+0000 7f19f6600640 1 --2- 172.21.15.42:0/1543254087 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19f006cea0 0x7f19f01d31a0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f19e000c380 tx=0x7f19e000c840 comp rx=0 tx=0).stop 2024-07-04T17:31:21.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.077+0000 7f19f6600640 1 -- 172.21.15.42:0/1543254087 shutdown_connections 2024-07-04T17:31:21.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.077+0000 7f19f6600640 1 --2- 172.21.15.42:0/1543254087 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19f006cea0 0x7f19f01d31a0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:21.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.077+0000 7f19f6600640 1 -- 172.21.15.42:0/1543254087 >> 172.21.15.42:0/1543254087 conn(0x7f19f00bf700 msgr2=0x7f19f01492d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:21.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.077+0000 7f19f6600640 1 -- 172.21.15.42:0/1543254087 shutdown_connections 2024-07-04T17:31:21.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.077+0000 7f19f6600640 1 -- 172.21.15.42:0/1543254087 wait complete. 2024-07-04T17:31:21.105 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-07-04T17:31:21.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.385+0000 7fdc49800640 1 -- 172.21.15.42:0/514097209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc440b3030 msgr2=0x7fdc440b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:21.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.385+0000 7fdc49800640 1 --2- 172.21.15.42:0/514097209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc440b3030 0x7fdc440b01f0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fdc3000b2b0 tx=0x7fdc3002ad10 comp rx=0 tx=0).stop 2024-07-04T17:31:21.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.385+0000 7fdc49800640 1 -- 172.21.15.42:0/514097209 shutdown_connections 2024-07-04T17:31:21.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.385+0000 7fdc49800640 1 --2- 172.21.15.42:0/514097209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc440b3030 0x7fdc440b01f0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:21.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.385+0000 7fdc49800640 1 -- 172.21.15.42:0/514097209 >> 172.21.15.42:0/514097209 conn(0x7fdc440bf700 msgr2=0x7fdc440af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:21.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.385+0000 7fdc49800640 1 -- 172.21.15.42:0/514097209 shutdown_connections 2024-07-04T17:31:21.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.385+0000 7fdc49800640 1 -- 172.21.15.42:0/514097209 wait complete. 2024-07-04T17:31:21.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.387+0000 7fdc49800640 1 Processor -- start 2024-07-04T17:31:21.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.387+0000 7fdc49800640 1 -- start start 2024-07-04T17:31:21.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.389+0000 7fdc49800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc4406cea0 0x7fdc44071250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:21.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.389+0000 7fdc49800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4406ef50 con 0x7fdc4406cea0 2024-07-04T17:31:21.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.389+0000 7fdc42a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc4406cea0 0x7fdc44071250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:21.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.389+0000 7fdc42a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc4406cea0 0x7fdc44071250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42710/0 (socket says 172.21.15.42:42710) 2024-07-04T17:31:21.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.389+0000 7fdc42a00640 1 -- 172.21.15.42:0/1481675917 learned_addr learned my addr 172.21.15.42:0/1481675917 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:21.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.389+0000 7fdc42a00640 1 -- 172.21.15.42:0/1481675917 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc44073e80 con 0x7fdc4406cea0 2024-07-04T17:31:21.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.390+0000 7fdc42a00640 1 --2- 172.21.15.42:0/1481675917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc4406cea0 0x7fdc44071250 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fdc3400c380 tx=0x7fdc3400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:21.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.390+0000 7fdc3aa00640 1 -- 172.21.15.42:0/1481675917 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc340116c0 con 0x7fdc4406cea0 2024-07-04T17:31:21.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.390+0000 7fdc3aa00640 1 -- 172.21.15.42:0/1481675917 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc34017070 con 0x7fdc4406cea0 2024-07-04T17:31:21.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.390+0000 7fdc3aa00640 1 -- 172.21.15.42:0/1481675917 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc34011b20 con 0x7fdc4406cea0 2024-07-04T17:31:21.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.390+0000 7fdc49800640 1 -- 172.21.15.42:0/1481675917 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc44146b60 con 0x7fdc4406cea0 2024-07-04T17:31:21.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.390+0000 7fdc49800640 1 -- 172.21.15.42:0/1481675917 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc44071d20 con 0x7fdc4406cea0 2024-07-04T17:31:21.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.391+0000 7fdc3aa00640 1 -- 172.21.15.42:0/1481675917 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 28) ==== 49122+0+0 (secure 0 0 0) 0x7fdc34011cc0 con 0x7fdc4406cea0 2024-07-04T17:31:21.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.391+0000 7fdc49800640 1 -- 172.21.15.42:0/1481675917 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc440b01f0 con 0x7fdc4406cea0 2024-07-04T17:31:21.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.392+0000 7fdc3aa00640 1 -- 172.21.15.42:0/1481675917 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 4733+0+0 (secure 0 0 0) 0x7fdc340513b0 con 0x7fdc4406cea0 2024-07-04T17:31:21.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.397+0000 7fdc3aa00640 1 -- 172.21.15.42:0/1481675917 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fdc34010cf0 con 0x7fdc4406cea0 2024-07-04T17:31:21.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.647+0000 7fdc49800640 1 -- 172.21.15.42:0/1481675917 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fdc44069b70 con 0x7fdc4406cea0 2024-07-04T17:31:21.649 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-07-04T17:31:21.648+0000 7f8616e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-04T17:31:21.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.649+0000 7fdc3aa00640 1 -- 172.21.15.42:0/1481675917 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v61) ==== 161+0+0 (secure 0 0 0) 0x7fdc34051080 con 0x7fdc4406cea0 2024-07-04T17:31:21.651 INFO:teuthology.orchestra.run.smithi042.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-04T17:31:21.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.652+0000 7fdc49800640 1 -- 172.21.15.42:0/1481675917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc4406cea0 msgr2=0x7fdc44071250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:21.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.652+0000 7fdc49800640 1 --2- 172.21.15.42:0/1481675917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc4406cea0 0x7fdc44071250 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fdc3400c380 tx=0x7fdc3400c840 comp rx=0 tx=0).stop 2024-07-04T17:31:21.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.653+0000 7fdc49800640 1 -- 172.21.15.42:0/1481675917 shutdown_connections 2024-07-04T17:31:21.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.653+0000 7fdc49800640 1 --2- 172.21.15.42:0/1481675917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc4406cea0 0x7fdc44071250 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:21.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.653+0000 7fdc49800640 1 -- 172.21.15.42:0/1481675917 >> 172.21.15.42:0/1481675917 conn(0x7fdc440bf700 msgr2=0x7fdc440aa3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:21.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.653+0000 7fdc49800640 1 -- 172.21.15.42:0/1481675917 shutdown_connections 2024-07-04T17:31:21.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.653+0000 7fdc49800640 1 -- 172.21.15.42:0/1481675917 wait complete. 2024-07-04T17:31:21.665 INFO:tasks.ceph.mds.e.smithi042.stderr: -2029> 2024-07-04T17:30:49.334+0000 7f50a2200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-04T17:31:21.679 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-07-04T17:31:21.811 INFO:tasks.ceph.mds.e.smithi042.stdout:starting mds.e at 2024-07-04T17:31:21.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.937+0000 7fdbb0800640 1 -- 172.21.15.42:0/875603830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdba80b1c10 msgr2=0x7fdba80b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:21.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.937+0000 7fdbb0800640 1 --2- 172.21.15.42:0/875603830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdba80b1c10 0x7fdba80b30d0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c0097b0 tx=0x7fdb9c02a4a0 comp rx=0 tx=0).stop 2024-07-04T17:31:21.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.937+0000 7fdbb0800640 1 -- 172.21.15.42:0/875603830 shutdown_connections 2024-07-04T17:31:21.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.937+0000 7fdbb0800640 1 --2- 172.21.15.42:0/875603830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdba80b1c10 0x7fdba80b30d0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:21.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.937+0000 7fdbb0800640 1 -- 172.21.15.42:0/875603830 >> 172.21.15.42:0/875603830 conn(0x7fdba806fe40 msgr2=0x7fdba806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:21.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.937+0000 7fdbb0800640 1 -- 172.21.15.42:0/875603830 shutdown_connections 2024-07-04T17:31:21.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.937+0000 7fdbb0800640 1 -- 172.21.15.42:0/875603830 wait complete. 2024-07-04T17:31:21.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.939+0000 7fdbb0800640 1 Processor -- start 2024-07-04T17:31:21.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.939+0000 7fdbb0800640 1 -- start start 2024-07-04T17:31:21.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.940+0000 7fdbb0800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdba8070d60 0x7fdba80b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:21.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.940+0000 7fdbb0800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdba80b2a40 con 0x7fdba8070d60 2024-07-04T17:31:21.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.941+0000 7fdbade00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdba8070d60 0x7fdba80b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:21.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.941+0000 7fdbade00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdba8070d60 0x7fdba80b4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42744/0 (socket says 172.21.15.42:42744) 2024-07-04T17:31:21.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.941+0000 7fdbade00640 1 -- 172.21.15.42:0/2849173468 learned_addr learned my addr 172.21.15.42:0/2849173468 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:21.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.941+0000 7fdbade00640 1 -- 172.21.15.42:0/2849173468 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba80a9df0 con 0x7fdba8070d60 2024-07-04T17:31:21.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.942+0000 7fdbade00640 1 --2- 172.21.15.42:0/2849173468 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdba8070d60 0x7fdba80b4e00 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c0098d0 tx=0x7fdb9c033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:21.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.942+0000 7fdba2000640 1 -- 172.21.15.42:0/2849173468 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb9c031a90 con 0x7fdba8070d60 2024-07-04T17:31:21.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.942+0000 7fdba2000640 1 -- 172.21.15.42:0/2849173468 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb9c031d20 con 0x7fdba8070d60 2024-07-04T17:31:21.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.942+0000 7fdbb0800640 1 -- 172.21.15.42:0/2849173468 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdba81da790 con 0x7fdba8070d60 2024-07-04T17:31:21.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.942+0000 7fdba2000640 1 -- 172.21.15.42:0/2849173468 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb9c033200 con 0x7fdba8070d60 2024-07-04T17:31:21.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.942+0000 7fdbb0800640 1 -- 172.21.15.42:0/2849173468 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdba80af560 con 0x7fdba8070d60 2024-07-04T17:31:21.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.943+0000 7fdbb0800640 1 -- 172.21.15.42:0/2849173468 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdba80a9c10 con 0x7fdba8070d60 2024-07-04T17:31:21.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.943+0000 7fdba2000640 1 -- 172.21.15.42:0/2849173468 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 28) ==== 49122+0+0 (secure 0 0 0) 0x7fdb9c0333a0 con 0x7fdba8070d60 2024-07-04T17:31:21.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.944+0000 7fdba2000640 1 -- 172.21.15.42:0/2849173468 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 4819+0+0 (secure 0 0 0) 0x7fdb9c0728b0 con 0x7fdba8070d60 2024-07-04T17:31:21.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:21.949+0000 7fdba2000640 1 -- 172.21.15.42:0/2849173468 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fdba80a9c10 con 0x7fdba8070d60 2024-07-04T17:31:22.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.195+0000 7fdbb0800640 1 -- 172.21.15.42:0/2849173468 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fdba80ad9f0 con 0x7fdba8070d60 2024-07-04T17:31:22.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.652+0000 7fdba2000640 1 -- 172.21.15.42:0/2849173468 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v62) ==== 104+0+0 (secure 0 0 0) 0x7fdba80ad9f0 con 0x7fdba8070d60 2024-07-04T17:31:22.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.655+0000 7fdbb0800640 1 -- 172.21.15.42:0/2849173468 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdba8070d60 msgr2=0x7fdba80b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:22.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.655+0000 7fdbb0800640 1 --2- 172.21.15.42:0/2849173468 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdba8070d60 0x7fdba80b4e00 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c0098d0 tx=0x7fdb9c033c10 comp rx=0 tx=0).stop 2024-07-04T17:31:22.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.655+0000 7fdbb0800640 1 -- 172.21.15.42:0/2849173468 shutdown_connections 2024-07-04T17:31:22.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.655+0000 7fdbb0800640 1 --2- 172.21.15.42:0/2849173468 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdba8070d60 0x7fdba80b4e00 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:22.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.655+0000 7fdbb0800640 1 -- 172.21.15.42:0/2849173468 >> 172.21.15.42:0/2849173468 conn(0x7fdba806fe40 msgr2=0x7fdba806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:22.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.655+0000 7fdbb0800640 1 -- 172.21.15.42:0/2849173468 shutdown_connections 2024-07-04T17:31:22.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.655+0000 7fdbb0800640 1 -- 172.21.15.42:0/2849173468 wait complete. 2024-07-04T17:31:22.680 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-04T17:31:22.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.960+0000 7f0b72c00640 1 -- 172.21.15.42:0/3258338292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b6c06d7b0 msgr2=0x7f0b6c0bc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:22.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.960+0000 7f0b72c00640 1 --2- 172.21.15.42:0/3258338292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b6c06d7b0 0x7f0b6c0bc100 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c005fd0 tx=0x7f0b5c02aac0 comp rx=0 tx=0).stop 2024-07-04T17:31:22.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.961+0000 7f0b72c00640 1 -- 172.21.15.42:0/3258338292 shutdown_connections 2024-07-04T17:31:22.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.961+0000 7f0b72c00640 1 --2- 172.21.15.42:0/3258338292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b6c06d7b0 0x7f0b6c0bc100 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:22.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.961+0000 7f0b72c00640 1 -- 172.21.15.42:0/3258338292 >> 172.21.15.42:0/3258338292 conn(0x7f0b6c0c0110 msgr2=0x7f0b6c0c0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:22.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.961+0000 7f0b72c00640 1 -- 172.21.15.42:0/3258338292 shutdown_connections 2024-07-04T17:31:22.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.961+0000 7f0b72c00640 1 -- 172.21.15.42:0/3258338292 wait complete. 2024-07-04T17:31:22.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.964+0000 7f0b72c00640 1 Processor -- start 2024-07-04T17:31:22.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.964+0000 7f0b72c00640 1 -- start start 2024-07-04T17:31:22.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.966+0000 7f0b72c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b6c06d7b0 0x7f0b6c073070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:22.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.966+0000 7f0b72c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c0bca90 con 0x7f0b6c06d7b0 2024-07-04T17:31:22.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.967+0000 7f0b71800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b6c06d7b0 0x7f0b6c073070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:22.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.967+0000 7f0b71800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b6c06d7b0 0x7f0b6c073070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42752/0 (socket says 172.21.15.42:42752) 2024-07-04T17:31:22.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.967+0000 7f0b71800640 1 -- 172.21.15.42:0/1841229076 learned_addr learned my addr 172.21.15.42:0/1841229076 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:22.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.967+0000 7f0b71800640 1 -- 172.21.15.42:0/1841229076 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6c1d98c0 con 0x7f0b6c06d7b0 2024-07-04T17:31:22.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.967+0000 7f0b71800640 1 --2- 172.21.15.42:0/1841229076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b6c06d7b0 0x7f0b6c073070 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c006010 tx=0x7f0b5c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:22.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.968+0000 7f0b69600640 1 -- 172.21.15.42:0/1841229076 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b5c044070 con 0x7f0b6c06d7b0 2024-07-04T17:31:22.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.968+0000 7f0b69600640 1 -- 172.21.15.42:0/1841229076 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b5c004380 con 0x7f0b6c06d7b0 2024-07-04T17:31:22.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.968+0000 7f0b72c00640 1 -- 172.21.15.42:0/1841229076 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b6c1d9400 con 0x7f0b6c06d7b0 2024-07-04T17:31:22.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.968+0000 7f0b69600640 1 -- 172.21.15.42:0/1841229076 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b5c0046c0 con 0x7f0b6c06d7b0 2024-07-04T17:31:22.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.968+0000 7f0b72c00640 1 -- 172.21.15.42:0/1841229076 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b6c1d7eb0 con 0x7f0b6c06d7b0 2024-07-04T17:31:22.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.969+0000 7f0b69600640 1 -- 172.21.15.42:0/1841229076 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 28) ==== 49122+0+0 (secure 0 0 0) 0x7f0b5c047070 con 0x7f0b6c06d7b0 2024-07-04T17:31:22.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.969+0000 7f0b72c00640 1 -- 172.21.15.42:0/1841229076 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b6c0aa0f0 con 0x7f0b6c06d7b0 2024-07-04T17:31:22.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.970+0000 7f0b69600640 1 -- 172.21.15.42:0/1841229076 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 4819+0+0 (secure 0 0 0) 0x7f0b5c03c070 con 0x7f0b6c06d7b0 2024-07-04T17:31:22.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:22.975+0000 7f0b69600640 1 -- 172.21.15.42:0/1841229076 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f0b5c032030 con 0x7f0b6c06d7b0 2024-07-04T17:31:23.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:23.191+0000 7f0b72c00640 1 -- 172.21.15.42:0/1841229076 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0b6c0bd1c0 con 0x7f0b6c06d7b0 2024-07-04T17:31:23.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:23.834+0000 7f0b69600640 1 -- 172.21.15.42:0/1841229076 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v79) ==== 181+0+0 (secure 0 0 0) 0x7f0b5c047360 con 0x7f0b6c06d7b0 2024-07-04T17:31:23.836 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' removed 2024-07-04T17:31:23.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:23.838+0000 7f0b72c00640 1 -- 172.21.15.42:0/1841229076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b6c06d7b0 msgr2=0x7f0b6c073070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:23.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:23.838+0000 7f0b72c00640 1 --2- 172.21.15.42:0/1841229076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b6c06d7b0 0x7f0b6c073070 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c006010 tx=0x7f0b5c003f10 comp rx=0 tx=0).stop 2024-07-04T17:31:23.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:23.839+0000 7f0b72c00640 1 -- 172.21.15.42:0/1841229076 shutdown_connections 2024-07-04T17:31:23.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:23.839+0000 7f0b72c00640 1 --2- 172.21.15.42:0/1841229076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b6c06d7b0 0x7f0b6c073070 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:23.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:23.839+0000 7f0b72c00640 1 -- 172.21.15.42:0/1841229076 >> 172.21.15.42:0/1841229076 conn(0x7f0b6c0c0110 msgr2=0x7f0b6c0b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:23.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:23.839+0000 7f0b72c00640 1 -- 172.21.15.42:0/1841229076 shutdown_connections 2024-07-04T17:31:23.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:23.840+0000 7f0b72c00640 1 -- 172.21.15.42:0/1841229076 wait complete. 2024-07-04T17:31:23.876 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-04T17:31:24.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.141+0000 7f1745c00640 1 -- 172.21.15.42:0/2779425395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1740073470 msgr2=0x7f17400a9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:24.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.141+0000 7f1745c00640 1 --2- 172.21.15.42:0/2779425395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1740073470 0x7f17400a9990 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f17280097b0 tx=0x7f172802a540 comp rx=0 tx=0).stop 2024-07-04T17:31:24.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.141+0000 7f1745c00640 1 -- 172.21.15.42:0/2779425395 shutdown_connections 2024-07-04T17:31:24.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.141+0000 7f1745c00640 1 --2- 172.21.15.42:0/2779425395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1740073470 0x7f17400a9990 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:24.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.141+0000 7f1745c00640 1 -- 172.21.15.42:0/2779425395 >> 172.21.15.42:0/2779425395 conn(0x7f174006c020 msgr2=0x7f174006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:24.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.141+0000 7f1745c00640 1 -- 172.21.15.42:0/2779425395 shutdown_connections 2024-07-04T17:31:24.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.142+0000 7f1745c00640 1 -- 172.21.15.42:0/2779425395 wait complete. 2024-07-04T17:31:24.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.143+0000 7f1745c00640 1 Processor -- start 2024-07-04T17:31:24.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.144+0000 7f1745c00640 1 -- start start 2024-07-04T17:31:24.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.145+0000 7f1745c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1740073470 0x7f17401d67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:24.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.145+0000 7f1745c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f174013f430 con 0x7f1740073470 2024-07-04T17:31:24.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.145+0000 7f173ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1740073470 0x7f17401d67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:24.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.145+0000 7f173ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1740073470 0x7f17401d67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55946/0 (socket says 172.21.15.42:55946) 2024-07-04T17:31:24.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.145+0000 7f173ea00640 1 -- 172.21.15.42:0/2182589297 learned_addr learned my addr 172.21.15.42:0/2182589297 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:24.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.146+0000 7f173ea00640 1 -- 172.21.15.42:0/2182589297 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17400ac4b0 con 0x7f1740073470 2024-07-04T17:31:24.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.146+0000 7f173ea00640 1 --2- 172.21.15.42:0/2182589297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1740073470 0x7f17401d67c0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f173000c320 tx=0x7f173000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:24.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.146+0000 7f1736a00640 1 -- 172.21.15.42:0/2182589297 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17300116c0 con 0x7f1740073470 2024-07-04T17:31:24.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.146+0000 7f1736a00640 1 -- 172.21.15.42:0/2182589297 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1730017070 con 0x7f1740073470 2024-07-04T17:31:24.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.146+0000 7f1736a00640 1 -- 172.21.15.42:0/2182589297 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1730011b40 con 0x7f1740073470 2024-07-04T17:31:24.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.146+0000 7f1745c00640 1 -- 172.21.15.42:0/2182589297 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17401e43b0 con 0x7f1740073470 2024-07-04T17:31:24.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.146+0000 7f1745c00640 1 -- 172.21.15.42:0/2182589297 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17400ab9c0 con 0x7f1740073470 2024-07-04T17:31:24.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.148+0000 7f1736a00640 1 -- 172.21.15.42:0/2182589297 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 28) ==== 49122+0+0 (secure 0 0 0) 0x7f1730011ce0 con 0x7f1740073470 2024-07-04T17:31:24.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.148+0000 7f1745c00640 1 -- 172.21.15.42:0/2182589297 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17401e3310 con 0x7f1740073470 2024-07-04T17:31:24.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.148+0000 7f1736a00640 1 -- 172.21.15.42:0/2182589297 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4381+0+0 (secure 0 0 0) 0x7f1730051180 con 0x7f1740073470 2024-07-04T17:31:24.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.153+0000 7f1736a00640 1 -- 172.21.15.42:0/2182589297 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f173001c6f0 con 0x7f1740073470 2024-07-04T17:31:24.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.371+0000 7f1745c00640 1 -- 172.21.15.42:0/2182589297 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f174013fb00 con 0x7f1740073470 2024-07-04T17:31:24.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.404+0000 7f1736a00640 1 -- 172.21.15.42:0/2182589297 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v80) ==== 169+0+0 (secure 0 0 0) 0x7f173001c970 con 0x7f1740073470 2024-07-04T17:31:24.406 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' removed 2024-07-04T17:31:24.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.411+0000 7f1745c00640 1 -- 172.21.15.42:0/2182589297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1740073470 msgr2=0x7f17401d67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:24.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.411+0000 7f1745c00640 1 --2- 172.21.15.42:0/2182589297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1740073470 0x7f17401d67c0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f173000c320 tx=0x7f173000c7e0 comp rx=0 tx=0).stop 2024-07-04T17:31:24.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.413+0000 7f1745c00640 1 -- 172.21.15.42:0/2182589297 shutdown_connections 2024-07-04T17:31:24.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.413+0000 7f1745c00640 1 --2- 172.21.15.42:0/2182589297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1740073470 0x7f17401d67c0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:24.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.413+0000 7f1745c00640 1 -- 172.21.15.42:0/2182589297 >> 172.21.15.42:0/2182589297 conn(0x7f174006c020 msgr2=0x7f174006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:24.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.414+0000 7f1745c00640 1 -- 172.21.15.42:0/2182589297 shutdown_connections 2024-07-04T17:31:24.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.415+0000 7f1745c00640 1 -- 172.21.15.42:0/2182589297 wait complete. 2024-07-04T17:31:24.447 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:31:24.484 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:31:24.483+0000 7f9b68c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:31:24.484 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:31:24.483+0000 7f9b68c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:31:24.484 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:31:24.483+0000 7f9b68c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:31:24.484 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:31:24.483+0000 7f9b68c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:31:24.484 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:31:24.483+0000 7f9b68c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:31:24.485 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:31:24.483+0000 7f9b6aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:31:24.485 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:31:24.483+0000 7f9b6aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:31:24.485 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:31:24.483+0000 7f9b6aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:31:24.485 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:31:24.483+0000 7f9b6aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:31:24.485 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:31:24.483+0000 7f9b6aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:31:24.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.718+0000 7fa310e00640 1 -- 172.21.15.42:0/4072676975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa30c0b38f0 msgr2=0x7fa30c0b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:24.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.718+0000 7fa310e00640 1 --2- 172.21.15.42:0/4072676975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa30c0b38f0 0x7fa30c0b2d20 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fa2f40097b0 tx=0x7fa2f402a540 comp rx=0 tx=0).stop 2024-07-04T17:31:24.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.718+0000 7fa310e00640 1 -- 172.21.15.42:0/4072676975 shutdown_connections 2024-07-04T17:31:24.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.718+0000 7fa310e00640 1 --2- 172.21.15.42:0/4072676975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa30c0b38f0 0x7fa30c0b2d20 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:24.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.718+0000 7fa310e00640 1 -- 172.21.15.42:0/4072676975 >> 172.21.15.42:0/4072676975 conn(0x7fa30c0bf700 msgr2=0x7fa30c0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:24.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.718+0000 7fa310e00640 1 -- 172.21.15.42:0/4072676975 shutdown_connections 2024-07-04T17:31:24.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.718+0000 7fa310e00640 1 -- 172.21.15.42:0/4072676975 wait complete. 2024-07-04T17:31:24.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.720+0000 7fa310e00640 1 Processor -- start 2024-07-04T17:31:24.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.720+0000 7fa310e00640 1 -- start start 2024-07-04T17:31:24.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.721+0000 7fa310e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa30c0b01f0 0x7fa30c070b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:24.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.721+0000 7fa310e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c06cd40 con 0x7fa30c0b01f0 2024-07-04T17:31:24.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.721+0000 7fa309e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa30c0b01f0 0x7fa30c070b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:24.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.721+0000 7fa309e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa30c0b01f0 0x7fa30c070b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56066/0 (socket says 172.21.15.42:56066) 2024-07-04T17:31:24.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.721+0000 7fa309e00640 1 -- 172.21.15.42:0/1721171914 learned_addr learned my addr 172.21.15.42:0/1721171914 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:24.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.722+0000 7fa309e00640 1 -- 172.21.15.42:0/1721171914 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa30c072110 con 0x7fa30c0b01f0 2024-07-04T17:31:24.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.722+0000 7fa309e00640 1 --2- 172.21.15.42:0/1721171914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa30c0b01f0 0x7fa30c070b60 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fa2f40098d0 tx=0x7fa2f4033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:24.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.722+0000 7fa302000640 1 -- 172.21.15.42:0/1721171914 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2f4031880 con 0x7fa30c0b01f0 2024-07-04T17:31:24.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.722+0000 7fa302000640 1 -- 172.21.15.42:0/1721171914 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2f4031b10 con 0x7fa30c0b01f0 2024-07-04T17:31:24.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.722+0000 7fa302000640 1 -- 172.21.15.42:0/1721171914 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2f4031df0 con 0x7fa30c0b01f0 2024-07-04T17:31:24.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.723+0000 7fa310e00640 1 -- 172.21.15.42:0/1721171914 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa30c06ffe0 con 0x7fa30c0b01f0 2024-07-04T17:31:24.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.723+0000 7fa310e00640 1 -- 172.21.15.42:0/1721171914 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa30c1d78a0 con 0x7fa30c0b01f0 2024-07-04T17:31:24.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.724+0000 7fa310e00640 1 -- 172.21.15.42:0/1721171914 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa30c06d400 con 0x7fa30c0b01f0 2024-07-04T17:31:24.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.724+0000 7fa302000640 1 -- 172.21.15.42:0/1721171914 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 29) ==== 49122+0+0 (secure 0 0 0) 0x7fa2f40330d0 con 0x7fa30c0b01f0 2024-07-04T17:31:24.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.725+0000 7fa302000640 1 -- 172.21.15.42:0/1721171914 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 3967+0+0 (secure 0 0 0) 0x7fa2f40726b0 con 0x7fa30c0b01f0 2024-07-04T17:31:24.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.730+0000 7fa302000640 1 -- 172.21.15.42:0/1721171914 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fa2f403f030 con 0x7fa30c0b01f0 2024-07-04T17:31:24.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.973+0000 7fa310e00640 1 -- 172.21.15.42:0/1721171914 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa30c074050 con 0x7fa30c0b01f0 2024-07-04T17:31:24.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.974+0000 7fa302000640 1 -- 172.21.15.42:0/1721171914 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+6054 (secure 0 0 0) 0x7fa2f403a3a0 con 0x7fa30c0b01f0 2024-07-04T17:31:24.976 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:31:24.976 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":62,"btime":"2024-07-04T17:31:22:651088+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5017,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/2312272332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2312272332},{"type":"v1","addr":"172.21.15.42:6827","nonce":2312272332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5020,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1285161495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1285161495},{"type":"v1","addr":"172.21.15.42:6829","nonce":1285161495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5023,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/1583070649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1583070649},{"type":"v1","addr":"172.21.15.42:6831","nonce":1583070649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5184,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/1315018856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":1315018856},{"type":"v1","addr":"172.21.15.42:6825","nonce":1315018856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:31:06.323313+0000","modified":"2024-07-04T17:31:09.367074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5026":{"gid":5026,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6833/2038326930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2038326930},{"type":"v1","addr":"172.21.15.42:6833","nonce":2038326930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5026,"qdb_cluster":[5026]},"id":7}]} 2024-07-04T17:31:24.976 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 62 2024-07-04T17:31:24.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.977+0000 7fa310e00640 1 -- 172.21.15.42:0/1721171914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa30c0b01f0 msgr2=0x7fa30c070b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:24.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.977+0000 7fa310e00640 1 --2- 172.21.15.42:0/1721171914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa30c0b01f0 0x7fa30c070b60 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fa2f40098d0 tx=0x7fa2f4033ad0 comp rx=0 tx=0).stop 2024-07-04T17:31:24.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.977+0000 7fa310e00640 1 -- 172.21.15.42:0/1721171914 shutdown_connections 2024-07-04T17:31:24.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.977+0000 7fa310e00640 1 --2- 172.21.15.42:0/1721171914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa30c0b01f0 0x7fa30c070b60 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:24.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.977+0000 7fa310e00640 1 -- 172.21.15.42:0/1721171914 >> 172.21.15.42:0/1721171914 conn(0x7fa30c0bf700 msgr2=0x7fa30c0abae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:24.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.977+0000 7fa310e00640 1 -- 172.21.15.42:0/1721171914 shutdown_connections 2024-07-04T17:31:24.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:24.978+0000 7fa310e00640 1 -- 172.21.15.42:0/1721171914 wait complete. 2024-07-04T17:31:25.003 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:31:25.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.331+0000 7f2e85400640 1 -- 172.21.15.42:0/2269376115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e8006cea0 msgr2=0x7f2e800b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:25.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.331+0000 7f2e85400640 1 --2- 172.21.15.42:0/2269376115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e8006cea0 0x7f2e800b3030 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f2e700097b0 tx=0x7f2e7002a540 comp rx=0 tx=0).stop 2024-07-04T17:31:25.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.331+0000 7f2e85400640 1 -- 172.21.15.42:0/2269376115 shutdown_connections 2024-07-04T17:31:25.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.331+0000 7f2e85400640 1 --2- 172.21.15.42:0/2269376115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e8006cea0 0x7f2e800b3030 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:25.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.331+0000 7f2e85400640 1 -- 172.21.15.42:0/2269376115 >> 172.21.15.42:0/2269376115 conn(0x7f2e800bf700 msgr2=0x7f2e800af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:25.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.331+0000 7f2e85400640 1 -- 172.21.15.42:0/2269376115 shutdown_connections 2024-07-04T17:31:25.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.331+0000 7f2e85400640 1 -- 172.21.15.42:0/2269376115 wait complete. 2024-07-04T17:31:25.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.333+0000 7f2e85400640 1 Processor -- start 2024-07-04T17:31:25.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.333+0000 7f2e85400640 1 -- start start 2024-07-04T17:31:25.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.334+0000 7f2e85400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e8006cea0 0x7f2e800be040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:25.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.334+0000 7f2e85400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8006ef50 con 0x7f2e8006cea0 2024-07-04T17:31:25.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.335+0000 7f2e7e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e8006cea0 0x7f2e800be040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:25.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.335+0000 7f2e7e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e8006cea0 0x7f2e800be040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56094/0 (socket says 172.21.15.42:56094) 2024-07-04T17:31:25.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.335+0000 7f2e7e800640 1 -- 172.21.15.42:0/2624671585 learned_addr learned my addr 172.21.15.42:0/2624671585 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:25.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.335+0000 7f2e7e800640 1 -- 172.21.15.42:0/2624671585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e80141340 con 0x7f2e8006cea0 2024-07-04T17:31:25.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.335+0000 7f2e7e800640 1 --2- 172.21.15.42:0/2624671585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e8006cea0 0x7f2e800be040 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f2e700098d0 tx=0x7f2e70033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:25.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.336+0000 7f2e76a00640 1 -- 172.21.15.42:0/2624671585 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e70031bc0 con 0x7f2e8006cea0 2024-07-04T17:31:25.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.336+0000 7f2e76a00640 1 -- 172.21.15.42:0/2624671585 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e70031e50 con 0x7f2e8006cea0 2024-07-04T17:31:25.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.336+0000 7f2e76a00640 1 -- 172.21.15.42:0/2624671585 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e70033260 con 0x7f2e8006cea0 2024-07-04T17:31:25.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.336+0000 7f2e85400640 1 -- 172.21.15.42:0/2624671585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e801e3aa0 con 0x7f2e8006cea0 2024-07-04T17:31:25.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.336+0000 7f2e85400640 1 -- 172.21.15.42:0/2624671585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e801e35a0 con 0x7f2e8006cea0 2024-07-04T17:31:25.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.337+0000 7f2e76a00640 1 -- 172.21.15.42:0/2624671585 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 29) ==== 49122+0+0 (secure 0 0 0) 0x7f2e70033400 con 0x7f2e8006cea0 2024-07-04T17:31:25.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.337+0000 7f2e85400640 1 -- 172.21.15.42:0/2624671585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e800b3030 con 0x7f2e8006cea0 2024-07-04T17:31:25.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.338+0000 7f2e76a00640 1 -- 172.21.15.42:0/2624671585 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 3967+0+0 (secure 0 0 0) 0x7f2e70072770 con 0x7f2e8006cea0 2024-07-04T17:31:25.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.343+0000 7f2e76a00640 1 -- 172.21.15.42:0/2624671585 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f2e7003f030 con 0x7f2e8006cea0 2024-07-04T17:31:25.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.423+0000 7f2e76a00640 1 -- 172.21.15.42:0/2624671585 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 30) ==== 49249+0+0 (secure 0 0 0) 0x7f2e700035f0 con 0x7f2e8006cea0 2024-07-04T17:31:25.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.423+0000 7f2e76a00640 1 --2- 172.21.15.42:0/2624671585 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f2e600417f0 0x7f2e60043c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:25.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.425+0000 7f2e7de00640 1 --2- 172.21.15.42:0/2624671585 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f2e600417f0 0x7f2e60043c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:25.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.429+0000 7f2e7de00640 1 --2- 172.21.15.42:0/2624671585 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f2e600417f0 0x7f2e60043c10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c0015c0 tx=0x7f2e6c007a90 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:25.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.567+0000 7f2e85400640 1 -- 172.21.15.42:0/2624671585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2e800a9990 con 0x7f2e8006cea0 2024-07-04T17:31:25.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.570+0000 7f2e76a00640 1 -- 172.21.15.42:0/2624671585 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+11367 (secure 0 0 0) 0x7f2e7003a3a0 con 0x7f2e8006cea0 2024-07-04T17:31:25.571 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:31:25.571 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":80,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:31:24.403607+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:31:03.814034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:31:05.281219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1699999570846558,"score_stable":1.1699999570846558,"optimal_score":1,"raw_score_acting":1.1699999570846558,"raw_score_stable":1.1699999570846558,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6835/1458267161":"2024-07-05T17:31:21.649384+0000","172.21.15.42:6834/1458267161":"2024-07-05T17:31:21.649384+0000","172.21.15.42:0/1725822177":"2024-07-05T17:31:19.572051+0000","172.21.15.42:0/1378827460":"2024-07-05T17:31:19.572051+0000","172.21.15.42:0/2771347343":"2024-07-05T17:31:19.572051+0000","172.21.15.42:0/2082652266":"2024-07-05T17:31:19.572051+0000","172.21.15.42:6825/3473020706":"2024-07-05T17:31:19.572051+0000","172.21.15.42:6824/3473020706":"2024-07-05T17:31:19.572051+0000","172.21.15.42:0/963042308":"2024-07-05T17:31:19.572051+0000","172.21.15.42:0/404958516":"2024-07-05T17:31:19.572051+0000","172.21.15.42:0/1340275465":"2024-07-05T17:31:19.572051+0000","172.21.15.42:0/1154056961":"2024-07-05T17:31:19.572051+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:31:25.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.573+0000 7f2e85400640 1 -- 172.21.15.42:0/2624671585 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f2e600417f0 msgr2=0x7f2e60043c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:25.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.573+0000 7f2e85400640 1 --2- 172.21.15.42:0/2624671585 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f2e600417f0 0x7f2e60043c10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c0015c0 tx=0x7f2e6c007a90 comp rx=0 tx=0).stop 2024-07-04T17:31:25.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.573+0000 7f2e85400640 1 -- 172.21.15.42:0/2624671585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e8006cea0 msgr2=0x7f2e800be040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:25.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.573+0000 7f2e85400640 1 --2- 172.21.15.42:0/2624671585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e8006cea0 0x7f2e800be040 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f2e700098d0 tx=0x7f2e70033e40 comp rx=0 tx=0).stop 2024-07-04T17:31:25.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.573+0000 7f2e85400640 1 -- 172.21.15.42:0/2624671585 shutdown_connections 2024-07-04T17:31:25.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.573+0000 7f2e85400640 1 --2- 172.21.15.42:0/2624671585 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f2e600417f0 0x7f2e60043c10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:25.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.573+0000 7f2e85400640 1 --2- 172.21.15.42:0/2624671585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e8006cea0 0x7f2e800be040 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:25.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.573+0000 7f2e85400640 1 -- 172.21.15.42:0/2624671585 >> 172.21.15.42:0/2624671585 conn(0x7f2e800bf700 msgr2=0x7f2e801493e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:25.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.573+0000 7f2e85400640 1 -- 172.21.15.42:0/2624671585 shutdown_connections 2024-07-04T17:31:25.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.574+0000 7f2e85400640 1 -- 172.21.15.42:0/2624671585 wait complete. 2024-07-04T17:31:25.600 INFO:tasks.cephfs.filesystem:Destroying file system backup_fs and related pools 2024-07-04T17:31:25.601 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:31:25.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.894+0000 7fe763e00640 1 -- 172.21.15.42:0/371731712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7640ba8b0 msgr2=0x7fe7640b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:25.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.894+0000 7fe763e00640 1 --2- 172.21.15.42:0/371731712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7640ba8b0 0x7fe7640b3550 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fe7540097b0 tx=0x7fe75402a540 comp rx=0 tx=0).stop 2024-07-04T17:31:25.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.894+0000 7fe763e00640 1 -- 172.21.15.42:0/371731712 shutdown_connections 2024-07-04T17:31:25.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.894+0000 7fe763e00640 1 --2- 172.21.15.42:0/371731712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7640ba8b0 0x7fe7640b3550 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:25.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.894+0000 7fe763e00640 1 -- 172.21.15.42:0/371731712 >> 172.21.15.42:0/371731712 conn(0x7fe7640bebc0 msgr2=0x7fe7640befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:25.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.894+0000 7fe763e00640 1 -- 172.21.15.42:0/371731712 shutdown_connections 2024-07-04T17:31:25.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.894+0000 7fe763e00640 1 -- 172.21.15.42:0/371731712 wait complete. 2024-07-04T17:31:25.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.897+0000 7fe763e00640 1 Processor -- start 2024-07-04T17:31:25.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.897+0000 7fe763e00640 1 -- start start 2024-07-04T17:31:25.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.898+0000 7fe763e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7640b1d20 0x7fe764141b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:25.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.898+0000 7fe763e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe7640b8b40 con 0x7fe7640b1d20 2024-07-04T17:31:25.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.899+0000 7fe762a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7640b1d20 0x7fe764141b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:25.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.899+0000 7fe762a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7640b1d20 0x7fe764141b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56112/0 (socket says 172.21.15.42:56112) 2024-07-04T17:31:25.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.899+0000 7fe762a00640 1 -- 172.21.15.42:0/1083750518 learned_addr learned my addr 172.21.15.42:0/1083750518 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:25.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.899+0000 7fe762a00640 1 -- 172.21.15.42:0/1083750518 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7640be520 con 0x7fe7640b1d20 2024-07-04T17:31:25.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.899+0000 7fe762a00640 1 --2- 172.21.15.42:0/1083750518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7640b1d20 0x7fe764141b20 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fe7540098d0 tx=0x7fe754033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:25.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.900+0000 7fe75b400640 1 -- 172.21.15.42:0/1083750518 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe754031bc0 con 0x7fe7640b1d20 2024-07-04T17:31:25.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.900+0000 7fe75b400640 1 -- 172.21.15.42:0/1083750518 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe754031e50 con 0x7fe7640b1d20 2024-07-04T17:31:25.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.900+0000 7fe75b400640 1 -- 172.21.15.42:0/1083750518 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe754033200 con 0x7fe7640b1d20 2024-07-04T17:31:25.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.900+0000 7fe763e00640 1 -- 172.21.15.42:0/1083750518 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7640be0f0 con 0x7fe7640b1d20 2024-07-04T17:31:25.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.900+0000 7fe763e00640 1 -- 172.21.15.42:0/1083750518 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7641d5a20 con 0x7fe7640b1d20 2024-07-04T17:31:25.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.901+0000 7fe763e00640 1 -- 172.21.15.42:0/1083750518 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7640be320 con 0x7fe7640b1d20 2024-07-04T17:31:25.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.901+0000 7fe75b400640 1 -- 172.21.15.42:0/1083750518 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 30) ==== 49249+0+0 (secure 0 0 0) 0x7fe7540333a0 con 0x7fe7640b1d20 2024-07-04T17:31:25.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.902+0000 7fe75b400640 1 --2- 172.21.15.42:0/1083750518 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fe74003d0b0 0x7fe74003f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:25.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.902+0000 7fe75b400640 1 -- 172.21.15.42:0/1083750518 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 3967+0+0 (secure 0 0 0) 0x7fe7540729d0 con 0x7fe7640b1d20 2024-07-04T17:31:25.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.902+0000 7fe762000640 1 --2- 172.21.15.42:0/1083750518 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fe74003d0b0 0x7fe74003f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:25.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.903+0000 7fe762000640 1 --2- 172.21.15.42:0/1083750518 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fe74003d0b0 0x7fe74003f5c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe7641442c0 tx=0x7fe74c007a90 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:25.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:25.907+0000 7fe75b400640 1 -- 172.21.15.42:0/1083750518 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fe75403f030 con 0x7fe7640b1d20 2024-07-04T17:31:26.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.155+0000 7fe763e00640 1 -- 172.21.15.42:0/1083750518 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7640b9240 con 0x7fe7640b1d20 2024-07-04T17:31:26.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.156+0000 7fe75b400640 1 -- 172.21.15.42:0/1083750518 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+6054 (secure 0 0 0) 0x7fe7640b6940 con 0x7fe7640b1d20 2024-07-04T17:31:26.158 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:31:26.158 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":62,"btime":"2024-07-04T17:31:22:651088+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5017,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/2312272332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2312272332},{"type":"v1","addr":"172.21.15.42:6827","nonce":2312272332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5020,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1285161495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1285161495},{"type":"v1","addr":"172.21.15.42:6829","nonce":1285161495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5023,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/1583070649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1583070649},{"type":"v1","addr":"172.21.15.42:6831","nonce":1583070649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5184,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/1315018856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":1315018856},{"type":"v1","addr":"172.21.15.42:6825","nonce":1315018856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:31:06.323313+0000","modified":"2024-07-04T17:31:09.367074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5026":{"gid":5026,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6833/2038326930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2038326930},{"type":"v1","addr":"172.21.15.42:6833","nonce":2038326930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5026,"qdb_cluster":[5026]},"id":7}]} 2024-07-04T17:31:26.158 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 62 2024-07-04T17:31:26.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.159+0000 7fe763e00640 1 -- 172.21.15.42:0/1083750518 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fe74003d0b0 msgr2=0x7fe74003f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:26.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.159+0000 7fe763e00640 1 --2- 172.21.15.42:0/1083750518 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fe74003d0b0 0x7fe74003f5c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe7641442c0 tx=0x7fe74c007a90 comp rx=0 tx=0).stop 2024-07-04T17:31:26.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.159+0000 7fe763e00640 1 -- 172.21.15.42:0/1083750518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7640b1d20 msgr2=0x7fe764141b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:26.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.159+0000 7fe763e00640 1 --2- 172.21.15.42:0/1083750518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7640b1d20 0x7fe764141b20 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fe7540098d0 tx=0x7fe754033e40 comp rx=0 tx=0).stop 2024-07-04T17:31:26.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.159+0000 7fe763e00640 1 -- 172.21.15.42:0/1083750518 shutdown_connections 2024-07-04T17:31:26.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.159+0000 7fe763e00640 1 --2- 172.21.15.42:0/1083750518 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fe74003d0b0 0x7fe74003f5c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:26.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.159+0000 7fe763e00640 1 --2- 172.21.15.42:0/1083750518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7640b1d20 0x7fe764141b20 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:26.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.159+0000 7fe763e00640 1 -- 172.21.15.42:0/1083750518 >> 172.21.15.42:0/1083750518 conn(0x7fe7640bebc0 msgr2=0x7fe764142f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:26.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.159+0000 7fe763e00640 1 -- 172.21.15.42:0/1083750518 shutdown_connections 2024-07-04T17:31:26.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.159+0000 7fe763e00640 1 -- 172.21.15.42:0/1083750518 wait complete. 2024-07-04T17:31:26.185 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:31:26.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.478+0000 7f7668e00640 1 -- 172.21.15.42:0/3127434083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76640b2980 msgr2=0x7f76640bd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:26.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.478+0000 7f7668e00640 1 --2- 172.21.15.42:0/3127434083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76640b2980 0x7f76640bd250 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f7654005fd0 tx=0x7f765402aac0 comp rx=0 tx=0).stop 2024-07-04T17:31:26.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.479+0000 7f7668e00640 1 -- 172.21.15.42:0/3127434083 shutdown_connections 2024-07-04T17:31:26.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.479+0000 7f7668e00640 1 --2- 172.21.15.42:0/3127434083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76640b2980 0x7f76640bd250 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:26.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.479+0000 7f7668e00640 1 -- 172.21.15.42:0/3127434083 >> 172.21.15.42:0/3127434083 conn(0x7f76640c03e0 msgr2=0x7f76640c07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:26.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.479+0000 7f7668e00640 1 -- 172.21.15.42:0/3127434083 shutdown_connections 2024-07-04T17:31:26.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.479+0000 7f7668e00640 1 -- 172.21.15.42:0/3127434083 wait complete. 2024-07-04T17:31:26.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.481+0000 7f7668e00640 1 Processor -- start 2024-07-04T17:31:26.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.481+0000 7f7668e00640 1 -- start start 2024-07-04T17:31:26.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.483+0000 7f7668e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7664071dc0 0x7f76640b5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:26.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.483+0000 7f7668e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f76640b0fe0 con 0x7f7664071dc0 2024-07-04T17:31:26.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.483+0000 7f7661e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7664071dc0 0x7f76640b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:26.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.483+0000 7f7661e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7664071dc0 0x7f76640b5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56158/0 (socket says 172.21.15.42:56158) 2024-07-04T17:31:26.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.483+0000 7f7661e00640 1 -- 172.21.15.42:0/1421907605 learned_addr learned my addr 172.21.15.42:0/1421907605 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:26.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.483+0000 7f7661e00640 1 -- 172.21.15.42:0/1421907605 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76641e36c0 con 0x7f7664071dc0 2024-07-04T17:31:26.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.484+0000 7f7661e00640 1 --2- 172.21.15.42:0/1421907605 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7664071dc0 0x7f76640b5350 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f7654006010 tx=0x7f7654003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:26.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.484+0000 7f765a000640 1 -- 172.21.15.42:0/1421907605 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7654044070 con 0x7f7664071dc0 2024-07-04T17:31:26.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.484+0000 7f765a000640 1 -- 172.21.15.42:0/1421907605 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7654004250 con 0x7f7664071dc0 2024-07-04T17:31:26.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.484+0000 7f765a000640 1 -- 172.21.15.42:0/1421907605 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7654004590 con 0x7f7664071dc0 2024-07-04T17:31:26.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.484+0000 7f7668e00640 1 -- 172.21.15.42:0/1421907605 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76640aea60 con 0x7f7664071dc0 2024-07-04T17:31:26.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.484+0000 7f7668e00640 1 -- 172.21.15.42:0/1421907605 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76640b6820 con 0x7f7664071dc0 2024-07-04T17:31:26.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.486+0000 7f765a000640 1 -- 172.21.15.42:0/1421907605 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 30) ==== 49249+0+0 (secure 0 0 0) 0x7f7654047070 con 0x7f7664071dc0 2024-07-04T17:31:26.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.486+0000 7f7668e00640 1 -- 172.21.15.42:0/1421907605 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76640bd250 con 0x7f7664071dc0 2024-07-04T17:31:26.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.486+0000 7f765a000640 1 --2- 172.21.15.42:0/1421907605 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f764403d100 0x7f764403f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:26.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.487+0000 7f765a000640 1 -- 172.21.15.42:0/1421907605 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 3967+0+0 (secure 0 0 0) 0x7f765403c070 con 0x7f7664071dc0 2024-07-04T17:31:26.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.487+0000 7f7661400640 1 --2- 172.21.15.42:0/1421907605 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f764403d100 0x7f764403f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:26.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.488+0000 7f7661400640 1 --2- 172.21.15.42:0/1421907605 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f764403d100 0x7f764403f610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f766406e260 tx=0x7f76500079f0 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:26.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.493+0000 7f765a000640 1 -- 172.21.15.42:0/1421907605 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f7654032030 con 0x7f7664071dc0 2024-07-04T17:31:26.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.749+0000 7f7668e00640 1 -- 172.21.15.42:0/1421907605 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7664069b70 con 0x7f7664071dc0 2024-07-04T17:31:26.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.750+0000 7f765a000640 1 -- 172.21.15.42:0/1421907605 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+6054 (secure 0 0 0) 0x7f7654047360 con 0x7f7664071dc0 2024-07-04T17:31:26.751 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:31:26.751 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":62,"btime":"2024-07-04T17:31:22:651088+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5017,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/2312272332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2312272332},{"type":"v1","addr":"172.21.15.42:6827","nonce":2312272332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5020,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1285161495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1285161495},{"type":"v1","addr":"172.21.15.42:6829","nonce":1285161495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5023,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/1583070649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1583070649},{"type":"v1","addr":"172.21.15.42:6831","nonce":1583070649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5184,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/1315018856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":1315018856},{"type":"v1","addr":"172.21.15.42:6825","nonce":1315018856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:31:06.323313+0000","modified":"2024-07-04T17:31:09.367074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5026":{"gid":5026,"name":"b","rank":0,"incarnation":56,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6833/2038326930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2038326930},{"type":"v1","addr":"172.21.15.42:6833","nonce":2038326930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5026,"qdb_cluster":[5026]},"id":7}]} 2024-07-04T17:31:26.751 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 62 2024-07-04T17:31:26.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.752+0000 7f7668e00640 1 -- 172.21.15.42:0/1421907605 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f764403d100 msgr2=0x7f764403f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:26.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.752+0000 7f7668e00640 1 --2- 172.21.15.42:0/1421907605 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f764403d100 0x7f764403f610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f766406e260 tx=0x7f76500079f0 comp rx=0 tx=0).stop 2024-07-04T17:31:26.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.752+0000 7f7668e00640 1 -- 172.21.15.42:0/1421907605 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7664071dc0 msgr2=0x7f76640b5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:26.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.752+0000 7f7668e00640 1 --2- 172.21.15.42:0/1421907605 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7664071dc0 0x7f76640b5350 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f7654006010 tx=0x7f7654003f10 comp rx=0 tx=0).stop 2024-07-04T17:31:26.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.753+0000 7f7668e00640 1 -- 172.21.15.42:0/1421907605 shutdown_connections 2024-07-04T17:31:26.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.753+0000 7f7668e00640 1 --2- 172.21.15.42:0/1421907605 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f764403d100 0x7f764403f610 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:26.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.753+0000 7f7668e00640 1 --2- 172.21.15.42:0/1421907605 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7664071dc0 0x7f76640b5350 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:26.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.753+0000 7f7668e00640 1 -- 172.21.15.42:0/1421907605 >> 172.21.15.42:0/1421907605 conn(0x7f76640c03e0 msgr2=0x7f76641d6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:26.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.753+0000 7f7668e00640 1 -- 172.21.15.42:0/1421907605 shutdown_connections 2024-07-04T17:31:26.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:26.753+0000 7f7668e00640 1 -- 172.21.15.42:0/1421907605 wait complete. 2024-07-04T17:31:26.780 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:31:27.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.065+0000 7f32e7200640 1 -- 172.21.15.42:0/4209763843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32e806c020 msgr2=0x7f32e814a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:27.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.065+0000 7f32e7200640 1 --2- 172.21.15.42:0/4209763843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32e806c020 0x7f32e814a9e0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f32d40097b0 tx=0x7f32d402a540 comp rx=0 tx=0).stop 2024-07-04T17:31:27.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.065+0000 7f32e7200640 1 -- 172.21.15.42:0/4209763843 shutdown_connections 2024-07-04T17:31:27.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.065+0000 7f32e7200640 1 --2- 172.21.15.42:0/4209763843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32e806c020 0x7f32e814a9e0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:27.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.065+0000 7f32e7200640 1 -- 172.21.15.42:0/4209763843 >> 172.21.15.42:0/4209763843 conn(0x7f32e80bebc0 msgr2=0x7f32e80befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:27.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.065+0000 7f32e7200640 1 -- 172.21.15.42:0/4209763843 shutdown_connections 2024-07-04T17:31:27.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.065+0000 7f32e7200640 1 -- 172.21.15.42:0/4209763843 wait complete. 2024-07-04T17:31:27.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.067+0000 7f32e7200640 1 Processor -- start 2024-07-04T17:31:27.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.068+0000 7f32e7200640 1 -- start start 2024-07-04T17:31:27.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.069+0000 7f32e7200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32e806c020 0x7f32e81d5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:27.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.069+0000 7f32e7200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f32e814b550 con 0x7f32e806c020 2024-07-04T17:31:27.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.069+0000 7f32e5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32e806c020 0x7f32e81d5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:27.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.069+0000 7f32e5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32e806c020 0x7f32e81d5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56174/0 (socket says 172.21.15.42:56174) 2024-07-04T17:31:27.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.069+0000 7f32e5e00640 1 -- 172.21.15.42:0/858005670 learned_addr learned my addr 172.21.15.42:0/858005670 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:27.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.070+0000 7f32e5e00640 1 -- 172.21.15.42:0/858005670 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e81d8fc0 con 0x7f32e806c020 2024-07-04T17:31:27.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.070+0000 7f32e5e00640 1 --2- 172.21.15.42:0/858005670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32e806c020 0x7f32e81d5a60 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f32d8001eb0 tx=0x7f32d800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:27.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.070+0000 7f32de000640 1 -- 172.21.15.42:0/858005670 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32d80116c0 con 0x7f32e806c020 2024-07-04T17:31:27.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.070+0000 7f32de000640 1 -- 172.21.15.42:0/858005670 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32d8017070 con 0x7f32e806c020 2024-07-04T17:31:27.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.070+0000 7f32de000640 1 -- 172.21.15.42:0/858005670 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32d8011ae0 con 0x7f32e806c020 2024-07-04T17:31:27.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.070+0000 7f32e7200640 1 -- 172.21.15.42:0/858005670 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32e80b1070 con 0x7f32e806c020 2024-07-04T17:31:27.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.071+0000 7f32e7200640 1 -- 172.21.15.42:0/858005670 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32e81d7c80 con 0x7f32e806c020 2024-07-04T17:31:27.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.072+0000 7f32e7200640 1 -- 172.21.15.42:0/858005670 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32e814bc80 con 0x7f32e806c020 2024-07-04T17:31:27.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.072+0000 7f32de000640 1 -- 172.21.15.42:0/858005670 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 49457+0+0 (secure 0 0 0) 0x7f32d801c460 con 0x7f32e806c020 2024-07-04T17:31:27.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.073+0000 7f32de000640 1 --2- 172.21.15.42:0/858005670 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f32c803d2f0 0x7f32c803f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:27.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.073+0000 7f32de000640 1 -- 172.21.15.42:0/858005670 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 3967+0+0 (secure 0 0 0) 0x7f32d80515d0 con 0x7f32e806c020 2024-07-04T17:31:27.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.073+0000 7f32e5400640 1 --2- 172.21.15.42:0/858005670 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f32c803d2f0 0x7f32c803f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:27.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.074+0000 7f32e5400640 1 --2- 172.21.15.42:0/858005670 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f32c803d2f0 0x7f32c803f800 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f32d4034000 tx=0x7f32d4004730 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:27.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.078+0000 7f32de000640 1 -- 172.21.15.42:0/858005670 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f32d800a140 con 0x7f32e806c020 2024-07-04T17:31:27.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.286+0000 7f32e7200640 1 -- 172.21.15.42:0/858005670 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f32e814f540 con 0x7f32e806c020 2024-07-04T17:31:27.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.289+0000 7f32de000640 1 -- 172.21.15.42:0/858005670 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+11367 (secure 0 0 0) 0x7f32d801c800 con 0x7f32e806c020 2024-07-04T17:31:27.290 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:31:27.290 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":80,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:31:24.403607+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:31:03.814034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:31:05.281219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1699999570846558,"score_stable":1.1699999570846558,"optimal_score":1,"raw_score_acting":1.1699999570846558,"raw_score_stable":1.1699999570846558,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6835/1458267161":"2024-07-05T17:31:21.649384+0000","172.21.15.42:6834/1458267161":"2024-07-05T17:31:21.649384+0000","172.21.15.42:0/1725822177":"2024-07-05T17:31:19.572051+0000","172.21.15.42:0/1378827460":"2024-07-05T17:31:19.572051+0000","172.21.15.42:0/2771347343":"2024-07-05T17:31:19.572051+0000","172.21.15.42:0/2082652266":"2024-07-05T17:31:19.572051+0000","172.21.15.42:6825/3473020706":"2024-07-05T17:31:19.572051+0000","172.21.15.42:6824/3473020706":"2024-07-05T17:31:19.572051+0000","172.21.15.42:0/963042308":"2024-07-05T17:31:19.572051+0000","172.21.15.42:0/404958516":"2024-07-05T17:31:19.572051+0000","172.21.15.42:0/1340275465":"2024-07-05T17:31:19.572051+0000","172.21.15.42:0/1154056961":"2024-07-05T17:31:19.572051+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:31:27.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.291+0000 7f32e7200640 1 -- 172.21.15.42:0/858005670 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f32c803d2f0 msgr2=0x7f32c803f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:27.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.291+0000 7f32e7200640 1 --2- 172.21.15.42:0/858005670 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f32c803d2f0 0x7f32c803f800 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f32d4034000 tx=0x7f32d4004730 comp rx=0 tx=0).stop 2024-07-04T17:31:27.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.291+0000 7f32e7200640 1 -- 172.21.15.42:0/858005670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32e806c020 msgr2=0x7f32e81d5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:27.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.291+0000 7f32e7200640 1 --2- 172.21.15.42:0/858005670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32e806c020 0x7f32e81d5a60 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f32d8001eb0 tx=0x7f32d800c810 comp rx=0 tx=0).stop 2024-07-04T17:31:27.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.292+0000 7f32e7200640 1 -- 172.21.15.42:0/858005670 shutdown_connections 2024-07-04T17:31:27.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.292+0000 7f32e7200640 1 --2- 172.21.15.42:0/858005670 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f32c803d2f0 0x7f32c803f800 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:27.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.292+0000 7f32e7200640 1 --2- 172.21.15.42:0/858005670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32e806c020 0x7f32e81d5a60 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:27.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.292+0000 7f32e7200640 1 -- 172.21.15.42:0/858005670 >> 172.21.15.42:0/858005670 conn(0x7f32e80bebc0 msgr2=0x7f32e80b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:27.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.292+0000 7f32e7200640 1 -- 172.21.15.42:0/858005670 shutdown_connections 2024-07-04T17:31:27.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.292+0000 7f32e7200640 1 -- 172.21.15.42:0/858005670 wait complete. 2024-07-04T17:31:27.319 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail backup_fs 2024-07-04T17:31:27.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.611+0000 7fd18da00640 1 -- 172.21.15.42:0/2447420438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1880bcf90 msgr2=0x7fd1880bd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:27.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.611+0000 7fd18da00640 1 --2- 172.21.15.42:0/2447420438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1880bcf90 0x7fd1880bd3a0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fd1740097b0 tx=0x7fd17402a540 comp rx=0 tx=0).stop 2024-07-04T17:31:27.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.611+0000 7fd18da00640 1 -- 172.21.15.42:0/2447420438 shutdown_connections 2024-07-04T17:31:27.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.611+0000 7fd18da00640 1 --2- 172.21.15.42:0/2447420438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1880bcf90 0x7fd1880bd3a0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:27.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.611+0000 7fd18da00640 1 -- 172.21.15.42:0/2447420438 >> 172.21.15.42:0/2447420438 conn(0x7fd1880688a0 msgr2=0x7fd18806db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:27.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.611+0000 7fd18da00640 1 -- 172.21.15.42:0/2447420438 shutdown_connections 2024-07-04T17:31:27.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.611+0000 7fd18da00640 1 -- 172.21.15.42:0/2447420438 wait complete. 2024-07-04T17:31:27.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.613+0000 7fd18da00640 1 Processor -- start 2024-07-04T17:31:27.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.613+0000 7fd18da00640 1 -- start start 2024-07-04T17:31:27.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.615+0000 7fd18da00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1880bbef0 0x7fd1880c04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:27.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.615+0000 7fd18da00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd1880b79a0 con 0x7fd1880bbef0 2024-07-04T17:31:27.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.615+0000 7fd187e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1880bbef0 0x7fd1880c04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:27.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.615+0000 7fd187e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1880bbef0 0x7fd1880c04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56186/0 (socket says 172.21.15.42:56186) 2024-07-04T17:31:27.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.615+0000 7fd187e00640 1 -- 172.21.15.42:0/736023732 learned_addr learned my addr 172.21.15.42:0/736023732 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:27.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.615+0000 7fd187e00640 1 -- 172.21.15.42:0/736023732 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd188137590 con 0x7fd1880bbef0 2024-07-04T17:31:27.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.616+0000 7fd187e00640 1 --2- 172.21.15.42:0/736023732 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1880bbef0 0x7fd1880c04f0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fd17800c320 tx=0x7fd17800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:27.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.616+0000 7fd17fe00640 1 -- 172.21.15.42:0/736023732 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1780116c0 con 0x7fd1880bbef0 2024-07-04T17:31:27.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.616+0000 7fd17fe00640 1 -- 172.21.15.42:0/736023732 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd178017070 con 0x7fd1880bbef0 2024-07-04T17:31:27.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.616+0000 7fd17fe00640 1 -- 172.21.15.42:0/736023732 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd178011b00 con 0x7fd1880bbef0 2024-07-04T17:31:27.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.616+0000 7fd18da00640 1 -- 172.21.15.42:0/736023732 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1881e41d0 con 0x7fd1880bbef0 2024-07-04T17:31:27.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.616+0000 7fd18da00640 1 -- 172.21.15.42:0/736023732 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1881e46a0 con 0x7fd1880bbef0 2024-07-04T17:31:27.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.617+0000 7fd18da00640 1 -- 172.21.15.42:0/736023732 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1880b8020 con 0x7fd1880bbef0 2024-07-04T17:31:27.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.618+0000 7fd17fe00640 1 -- 172.21.15.42:0/736023732 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 49457+0+0 (secure 0 0 0) 0x7fd178011ca0 con 0x7fd1880bbef0 2024-07-04T17:31:27.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.618+0000 7fd17fe00640 1 --2- 172.21.15.42:0/736023732 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fd16803d340 0x7fd16803f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:27.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.618+0000 7fd17fe00640 1 -- 172.21.15.42:0/736023732 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 3967+0+0 (secure 0 0 0) 0x7fd178051530 con 0x7fd1880bbef0 2024-07-04T17:31:27.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.618+0000 7fd187400640 1 --2- 172.21.15.42:0/736023732 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fd16803d340 0x7fd16803f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:27.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.619+0000 7fd187400640 1 --2- 172.21.15.42:0/736023732 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fd16803d340 0x7fd16803f850 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd174034000 tx=0x7fd174004730 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:27.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.624+0000 7fd17fe00640 1 -- 172.21.15.42:0/736023732 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fd1880bc300 con 0x7fd1880bbef0 2024-07-04T17:31:27.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.876+0000 7fd18da00640 1 -- 172.21.15.42:0/736023732 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "backup_fs"} v 0) -- 0x7fd188068e30 con 0x7fd1880bbef0 2024-07-04T17:31:27.878 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-07-04T17:31:27.877+0000 7f8616e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-04T17:31:27.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.879+0000 7fd17fe00640 1 -- 172.21.15.42:0/736023732 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "backup_fs"}]=0 backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v63) ==== 167+0+0 (secure 0 0 0) 0x7fd17800a140 con 0x7fd1880bbef0 2024-07-04T17:31:27.880 INFO:teuthology.orchestra.run.smithi042.stderr:backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-04T17:31:27.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.883+0000 7fd18da00640 1 -- 172.21.15.42:0/736023732 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fd16803d340 msgr2=0x7fd16803f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:27.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.883+0000 7fd18da00640 1 --2- 172.21.15.42:0/736023732 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fd16803d340 0x7fd16803f850 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd174034000 tx=0x7fd174004730 comp rx=0 tx=0).stop 2024-07-04T17:31:27.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.883+0000 7fd18da00640 1 -- 172.21.15.42:0/736023732 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1880bbef0 msgr2=0x7fd1880c04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:27.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.883+0000 7fd18da00640 1 --2- 172.21.15.42:0/736023732 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1880bbef0 0x7fd1880c04f0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fd17800c320 tx=0x7fd17800c7e0 comp rx=0 tx=0).stop 2024-07-04T17:31:27.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.883+0000 7fd18da00640 1 -- 172.21.15.42:0/736023732 shutdown_connections 2024-07-04T17:31:27.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.883+0000 7fd18da00640 1 --2- 172.21.15.42:0/736023732 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fd16803d340 0x7fd16803f850 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:27.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.883+0000 7fd18da00640 1 --2- 172.21.15.42:0/736023732 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1880bbef0 0x7fd1880c04f0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:27.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.883+0000 7fd18da00640 1 -- 172.21.15.42:0/736023732 >> 172.21.15.42:0/736023732 conn(0x7fd1880688a0 msgr2=0x7fd188075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:27.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.883+0000 7fd18da00640 1 -- 172.21.15.42:0/736023732 shutdown_connections 2024-07-04T17:31:27.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:27.883+0000 7fd18da00640 1 -- 172.21.15.42:0/736023732 wait complete. 2024-07-04T17:31:27.890 INFO:tasks.ceph.mds.b.smithi042.stderr: -1047> 2024-07-04T17:31:09.334+0000 7f5784400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-04T17:31:27.914 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm backup_fs --yes-i-really-mean-it 2024-07-04T17:31:28.021 INFO:tasks.ceph.mds.b.smithi042.stdout:starting mds.b at 2024-07-04T17:31:28.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.188+0000 7fe74ba00640 1 -- 172.21.15.42:0/3661863673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7440b24a0 msgr2=0x7fe744070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:28.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.188+0000 7fe74ba00640 1 --2- 172.21.15.42:0/3661863673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7440b24a0 0x7fe744070cf0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fe738009230 tx=0x7fe73802a3f0 comp rx=0 tx=0).stop 2024-07-04T17:31:28.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.188+0000 7fe74ba00640 1 -- 172.21.15.42:0/3661863673 shutdown_connections 2024-07-04T17:31:28.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.188+0000 7fe74ba00640 1 --2- 172.21.15.42:0/3661863673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7440b24a0 0x7fe744070cf0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:28.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.188+0000 7fe74ba00640 1 -- 172.21.15.42:0/3661863673 >> 172.21.15.42:0/3661863673 conn(0x7fe74406fe40 msgr2=0x7fe74406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:28.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.188+0000 7fe74ba00640 1 -- 172.21.15.42:0/3661863673 shutdown_connections 2024-07-04T17:31:28.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.188+0000 7fe74ba00640 1 -- 172.21.15.42:0/3661863673 wait complete. 2024-07-04T17:31:28.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.190+0000 7fe74ba00640 1 Processor -- start 2024-07-04T17:31:28.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.191+0000 7fe74ba00640 1 -- start start 2024-07-04T17:31:28.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.192+0000 7fe74ba00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7440b1500 0x7fe74414d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:28.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.192+0000 7fe74ba00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe744071d70 con 0x7fe7440b1500 2024-07-04T17:31:28.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.193+0000 7fe749000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7440b1500 0x7fe74414d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:28.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.193+0000 7fe749000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7440b1500 0x7fe74414d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56228/0 (socket says 172.21.15.42:56228) 2024-07-04T17:31:28.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.193+0000 7fe749000640 1 -- 172.21.15.42:0/3678897294 learned_addr learned my addr 172.21.15.42:0/3678897294 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:28.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.193+0000 7fe749000640 1 -- 172.21.15.42:0/3678897294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe74414cce0 con 0x7fe7440b1500 2024-07-04T17:31:28.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.194+0000 7fe749000640 1 --2- 172.21.15.42:0/3678897294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7440b1500 0x7fe74414d4a0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fe738009230 tx=0x7fe738032c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:28.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.194+0000 7fe740c00640 1 -- 172.21.15.42:0/3678897294 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe73803f030 con 0x7fe7440b1500 2024-07-04T17:31:28.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.194+0000 7fe740c00640 1 -- 172.21.15.42:0/3678897294 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe738003a40 con 0x7fe7440b1500 2024-07-04T17:31:28.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.194+0000 7fe740c00640 1 -- 172.21.15.42:0/3678897294 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe738003dc0 con 0x7fe7440b1500 2024-07-04T17:31:28.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.194+0000 7fe74ba00640 1 -- 172.21.15.42:0/3678897294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7441d50a0 con 0x7fe7440b1500 2024-07-04T17:31:28.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.194+0000 7fe74ba00640 1 -- 172.21.15.42:0/3678897294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe744141620 con 0x7fe7440b1500 2024-07-04T17:31:28.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.195+0000 7fe740c00640 1 -- 172.21.15.42:0/3678897294 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) ==== 49457+0+0 (secure 0 0 0) 0x7fe738042030 con 0x7fe7440b1500 2024-07-04T17:31:28.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.196+0000 7fe74ba00640 1 -- 172.21.15.42:0/3678897294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7441418a0 con 0x7fe7440b1500 2024-07-04T17:31:28.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.196+0000 7fe740c00640 1 --2- 172.21.15.42:0/3678897294 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fe72403d340 0x7fe72403f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:28.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.196+0000 7fe740c00640 1 -- 172.21.15.42:0/3678897294 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4053+0+0 (secure 0 0 0) 0x7fe738040070 con 0x7fe7440b1500 2024-07-04T17:31:28.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.196+0000 7fe743e00640 1 --2- 172.21.15.42:0/3678897294 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fe72403d340 0x7fe72403f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:28.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.197+0000 7fe743e00640 1 --2- 172.21.15.42:0/3678897294 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fe72403d340 0x7fe72403f850 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe730007fd0 tx=0x7fe730007920 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:28.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.203+0000 7fe740c00640 1 -- 172.21.15.42:0/3678897294 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fe738049e30 con 0x7fe7440b1500 2024-07-04T17:31:28.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.451+0000 7fe74ba00640 1 -- 172.21.15.42:0/3678897294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true} v 0) -- 0x7fe744070cf0 con 0x7fe7440b1500 2024-07-04T17:31:28.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.881+0000 7fe740c00640 1 -- 172.21.15.42:0/3678897294 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true}]=0 v64) ==== 107+0+0 (secure 0 0 0) 0x7fe73803bd90 con 0x7fe7440b1500 2024-07-04T17:31:28.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.884+0000 7fe74ba00640 1 -- 172.21.15.42:0/3678897294 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fe72403d340 msgr2=0x7fe72403f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:28.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.884+0000 7fe74ba00640 1 --2- 172.21.15.42:0/3678897294 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fe72403d340 0x7fe72403f850 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe730007fd0 tx=0x7fe730007920 comp rx=0 tx=0).stop 2024-07-04T17:31:28.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.884+0000 7fe74ba00640 1 -- 172.21.15.42:0/3678897294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7440b1500 msgr2=0x7fe74414d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:28.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.884+0000 7fe74ba00640 1 --2- 172.21.15.42:0/3678897294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7440b1500 0x7fe74414d4a0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fe738009230 tx=0x7fe738032c00 comp rx=0 tx=0).stop 2024-07-04T17:31:28.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.884+0000 7fe74ba00640 1 -- 172.21.15.42:0/3678897294 shutdown_connections 2024-07-04T17:31:28.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.884+0000 7fe74ba00640 1 --2- 172.21.15.42:0/3678897294 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fe72403d340 0x7fe72403f850 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:28.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.884+0000 7fe74ba00640 1 --2- 172.21.15.42:0/3678897294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7440b1500 0x7fe74414d4a0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:28.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.884+0000 7fe74ba00640 1 -- 172.21.15.42:0/3678897294 >> 172.21.15.42:0/3678897294 conn(0x7fe74406fe40 msgr2=0x7fe744075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:28.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.885+0000 7fe74ba00640 1 -- 172.21.15.42:0/3678897294 shutdown_connections 2024-07-04T17:31:28.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:28.885+0000 7fe74ba00640 1 -- 172.21.15.42:0/3678897294 wait complete. 2024-07-04T17:31:28.921 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_metadata backup_fs_metadata --yes-i-really-really-mean-it 2024-07-04T17:31:29.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.205+0000 7ff576a00640 1 -- 172.21.15.42:0/2495895588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5700abe40 msgr2=0x7ff5700aa900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:29.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.205+0000 7ff576a00640 1 --2- 172.21.15.42:0/2495895588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5700abe40 0x7ff5700aa900 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7ff560005fd0 tx=0x7ff56002a5f0 comp rx=0 tx=0).stop 2024-07-04T17:31:29.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.205+0000 7ff576a00640 1 -- 172.21.15.42:0/2495895588 shutdown_connections 2024-07-04T17:31:29.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.205+0000 7ff576a00640 1 --2- 172.21.15.42:0/2495895588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5700abe40 0x7ff5700aa900 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:29.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.205+0000 7ff576a00640 1 -- 172.21.15.42:0/2495895588 >> 172.21.15.42:0/2495895588 conn(0x7ff5700afaa0 msgr2=0x7ff5700afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:29.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.205+0000 7ff576a00640 1 -- 172.21.15.42:0/2495895588 shutdown_connections 2024-07-04T17:31:29.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.205+0000 7ff576a00640 1 -- 172.21.15.42:0/2495895588 wait complete. 2024-07-04T17:31:29.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.207+0000 7ff576a00640 1 Processor -- start 2024-07-04T17:31:29.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.207+0000 7ff576a00640 1 -- start start 2024-07-04T17:31:29.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.208+0000 7ff576a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5700701d0 0x7ff5701d7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:29.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.208+0000 7ff576a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff570073ed0 con 0x7ff5700701d0 2024-07-04T17:31:29.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.208+0000 7ff575600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5700701d0 0x7ff5701d7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:29.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.209+0000 7ff575600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5700701d0 0x7ff5701d7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56248/0 (socket says 172.21.15.42:56248) 2024-07-04T17:31:29.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.209+0000 7ff575600640 1 -- 172.21.15.42:0/4068998000 learned_addr learned my addr 172.21.15.42:0/4068998000 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:29.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.209+0000 7ff575600640 1 -- 172.21.15.42:0/4068998000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5701d6ec0 con 0x7ff5700701d0 2024-07-04T17:31:29.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.209+0000 7ff575600640 1 --2- 172.21.15.42:0/4068998000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5700701d0 0x7ff5701d7690 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7ff564001eb0 tx=0x7ff56400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:29.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.209+0000 7ff56d600640 1 -- 172.21.15.42:0/4068998000 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff564017070 con 0x7ff5700701d0 2024-07-04T17:31:29.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.209+0000 7ff56d600640 1 -- 172.21.15.42:0/4068998000 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5640052f0 con 0x7ff5700701d0 2024-07-04T17:31:29.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.209+0000 7ff576a00640 1 -- 172.21.15.42:0/4068998000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5701d8c70 con 0x7ff5700701d0 2024-07-04T17:31:29.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.209+0000 7ff56d600640 1 -- 172.21.15.42:0/4068998000 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5640035f0 con 0x7ff5700701d0 2024-07-04T17:31:29.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.210+0000 7ff576a00640 1 -- 172.21.15.42:0/4068998000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff570144640 con 0x7ff5700701d0 2024-07-04T17:31:29.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.211+0000 7ff576a00640 1 -- 172.21.15.42:0/4068998000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5700008d0 con 0x7ff5700701d0 2024-07-04T17:31:29.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.211+0000 7ff56d600640 1 -- 172.21.15.42:0/4068998000 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7ff56401a070 con 0x7ff5700701d0 2024-07-04T17:31:29.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.211+0000 7ff56d600640 1 --2- 172.21.15.42:0/4068998000 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7ff55403d3d0 0x7ff55403f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:29.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.211+0000 7ff56d600640 1 -- 172.21.15.42:0/4068998000 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4053+0+0 (secure 0 0 0) 0x7ff564050cd0 con 0x7ff5700701d0 2024-07-04T17:31:29.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.212+0000 7ff574c00640 1 --2- 172.21.15.42:0/4068998000 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7ff55403d3d0 0x7ff55403f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:29.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.212+0000 7ff574c00640 1 --2- 172.21.15.42:0/4068998000 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7ff55403d3d0 0x7ff55403f8e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff560033000 tx=0x7ff560004730 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:29.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.216+0000 7ff56d600640 1 -- 172.21.15.42:0/4068998000 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7ff56401be20 con 0x7ff5700701d0 2024-07-04T17:31:29.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.420+0000 7ff576a00640 1 -- 172.21.15.42:0/4068998000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff5701d73b0 con 0x7ff5700701d0 2024-07-04T17:31:29.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.891+0000 7ff56d600640 1 -- 172.21.15.42:0/4068998000 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_metadata' removed v82) ==== 190+0+0 (secure 0 0 0) 0x7ff5640102e0 con 0x7ff5700701d0 2024-07-04T17:31:29.893 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'backup_fs_metadata' removed 2024-07-04T17:31:29.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.895+0000 7ff576a00640 1 -- 172.21.15.42:0/4068998000 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7ff55403d3d0 msgr2=0x7ff55403f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:29.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.895+0000 7ff576a00640 1 --2- 172.21.15.42:0/4068998000 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7ff55403d3d0 0x7ff55403f8e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff560033000 tx=0x7ff560004730 comp rx=0 tx=0).stop 2024-07-04T17:31:29.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.895+0000 7ff576a00640 1 -- 172.21.15.42:0/4068998000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5700701d0 msgr2=0x7ff5701d7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:29.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.895+0000 7ff576a00640 1 --2- 172.21.15.42:0/4068998000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5700701d0 0x7ff5701d7690 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7ff564001eb0 tx=0x7ff56400c810 comp rx=0 tx=0).stop 2024-07-04T17:31:29.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.896+0000 7ff576a00640 1 -- 172.21.15.42:0/4068998000 shutdown_connections 2024-07-04T17:31:29.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.896+0000 7ff576a00640 1 --2- 172.21.15.42:0/4068998000 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7ff55403d3d0 0x7ff55403f8e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:29.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.896+0000 7ff576a00640 1 --2- 172.21.15.42:0/4068998000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5700701d0 0x7ff5701d7690 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:29.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.896+0000 7ff576a00640 1 -- 172.21.15.42:0/4068998000 >> 172.21.15.42:0/4068998000 conn(0x7ff5700afaa0 msgr2=0x7ff570075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:29.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.896+0000 7ff576a00640 1 -- 172.21.15.42:0/4068998000 shutdown_connections 2024-07-04T17:31:29.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:29.896+0000 7ff576a00640 1 -- 172.21.15.42:0/4068998000 wait complete. 2024-07-04T17:31:29.936 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_data backup_fs_data --yes-i-really-really-mean-it 2024-07-04T17:31:30.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.229+0000 7f51fa600640 1 -- 172.21.15.42:0/2373914688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51f4073470 msgr2=0x7f51f40a9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:30.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.229+0000 7f51fa600640 1 --2- 172.21.15.42:0/2373914688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51f4073470 0x7f51f40a9990 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f51e4005fd0 tx=0x7f51e402a5f0 comp rx=0 tx=0).stop 2024-07-04T17:31:30.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.229+0000 7f51fa600640 1 -- 172.21.15.42:0/2373914688 shutdown_connections 2024-07-04T17:31:30.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.229+0000 7f51fa600640 1 --2- 172.21.15.42:0/2373914688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51f4073470 0x7f51f40a9990 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:30.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.229+0000 7f51fa600640 1 -- 172.21.15.42:0/2373914688 >> 172.21.15.42:0/2373914688 conn(0x7f51f406c020 msgr2=0x7f51f406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:30.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.229+0000 7f51fa600640 1 -- 172.21.15.42:0/2373914688 shutdown_connections 2024-07-04T17:31:30.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.229+0000 7f51fa600640 1 -- 172.21.15.42:0/2373914688 wait complete. 2024-07-04T17:31:30.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.231+0000 7f51fa600640 1 Processor -- start 2024-07-04T17:31:30.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.232+0000 7f51fa600640 1 -- start start 2024-07-04T17:31:30.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.233+0000 7f51fa600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51f4073470 0x7f51f40bb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:30.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.233+0000 7f51fa600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f51f413f430 con 0x7f51f4073470 2024-07-04T17:31:30.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.233+0000 7f51f3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51f4073470 0x7f51f40bb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:30.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.233+0000 7f51f3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51f4073470 0x7f51f40bb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56266/0 (socket says 172.21.15.42:56266) 2024-07-04T17:31:30.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.233+0000 7f51f3400640 1 -- 172.21.15.42:0/238987601 learned_addr learned my addr 172.21.15.42:0/238987601 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:30.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.234+0000 7f51f3400640 1 -- 172.21.15.42:0/238987601 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f40af120 con 0x7f51f4073470 2024-07-04T17:31:30.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.234+0000 7f51f3400640 1 --2- 172.21.15.42:0/238987601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51f4073470 0x7f51f40bb4f0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00c380 tx=0x7f51dc00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:30.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.234+0000 7f51ebe00640 1 -- 172.21.15.42:0/238987601 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51dc017070 con 0x7f51f4073470 2024-07-04T17:31:30.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.234+0000 7f51ebe00640 1 -- 172.21.15.42:0/238987601 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51dc0052f0 con 0x7f51f4073470 2024-07-04T17:31:30.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.234+0000 7f51ebe00640 1 -- 172.21.15.42:0/238987601 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51dc0035f0 con 0x7f51f4073470 2024-07-04T17:31:30.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.234+0000 7f51fa600640 1 -- 172.21.15.42:0/238987601 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51f40bcd70 con 0x7f51f4073470 2024-07-04T17:31:30.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.234+0000 7f51fa600640 1 -- 172.21.15.42:0/238987601 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51f40bef50 con 0x7f51f4073470 2024-07-04T17:31:30.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.236+0000 7f51fa600640 1 -- 172.21.15.42:0/238987601 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51f40aef10 con 0x7f51f4073470 2024-07-04T17:31:30.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.236+0000 7f51ebe00640 1 -- 172.21.15.42:0/238987601 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7f51dc01a070 con 0x7f51f4073470 2024-07-04T17:31:30.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.236+0000 7f51ebe00640 1 --2- 172.21.15.42:0/238987601 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f51d003d3d0 0x7f51d003f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:30.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.236+0000 7f51ebe00640 1 -- 172.21.15.42:0/238987601 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 3609+0+0 (secure 0 0 0) 0x7f51dc050b10 con 0x7f51f4073470 2024-07-04T17:31:30.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.237+0000 7f51eaa00640 1 --2- 172.21.15.42:0/238987601 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f51d003d3d0 0x7f51d003f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:30.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.237+0000 7f51eaa00640 1 --2- 172.21.15.42:0/238987601 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f51d003d3d0 0x7f51d003f8e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f51e4033000 tx=0x7f51e4004730 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:30.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.242+0000 7f51ebe00640 1 -- 172.21.15.42:0/238987601 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f51dc016170 con 0x7f51f4073470 2024-07-04T17:31:30.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.451+0000 7f51fa600640 1 -- 172.21.15.42:0/238987601 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f51f4140c80 con 0x7f51f4073470 2024-07-04T17:31:30.688 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-04T17:31:30.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.893+0000 7f51ebe00640 1 -- 172.21.15.42:0/238987601 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_data' removed v83) ==== 178+0+0 (secure 0 0 0) 0x7f51dc0192c0 con 0x7f51f4073470 2024-07-04T17:31:30.895 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'backup_fs_data' removed 2024-07-04T17:31:30.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.897+0000 7f51fa600640 1 -- 172.21.15.42:0/238987601 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f51d003d3d0 msgr2=0x7f51d003f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:30.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.897+0000 7f51fa600640 1 --2- 172.21.15.42:0/238987601 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f51d003d3d0 0x7f51d003f8e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f51e4033000 tx=0x7f51e4004730 comp rx=0 tx=0).stop 2024-07-04T17:31:30.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.897+0000 7f51fa600640 1 -- 172.21.15.42:0/238987601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51f4073470 msgr2=0x7f51f40bb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:30.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.897+0000 7f51fa600640 1 --2- 172.21.15.42:0/238987601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51f4073470 0x7f51f40bb4f0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00c380 tx=0x7f51dc00c840 comp rx=0 tx=0).stop 2024-07-04T17:31:30.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.897+0000 7f51fa600640 1 -- 172.21.15.42:0/238987601 shutdown_connections 2024-07-04T17:31:30.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.897+0000 7f51fa600640 1 --2- 172.21.15.42:0/238987601 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f51d003d3d0 0x7f51d003f8e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:30.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.897+0000 7f51fa600640 1 --2- 172.21.15.42:0/238987601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51f4073470 0x7f51f40bb4f0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:30.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.897+0000 7f51fa600640 1 -- 172.21.15.42:0/238987601 >> 172.21.15.42:0/238987601 conn(0x7f51f406c020 msgr2=0x7f51f40b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:30.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.897+0000 7f51fa600640 1 -- 172.21.15.42:0/238987601 shutdown_connections 2024-07-04T17:31:30.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:30.897+0000 7f51fa600640 1 -- 172.21.15.42:0/238987601 wait complete. 2024-07-04T17:31:30.946 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 22 2024-07-04T17:31:31.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.227+0000 7efce9c00640 1 -- 172.21.15.42:0/2976729814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efce40adaa0 msgr2=0x7efce406d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:31.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.227+0000 7efce9c00640 1 --2- 172.21.15.42:0/2976729814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efce40adaa0 0x7efce406d110 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7efcd40097b0 tx=0x7efcd402a540 comp rx=0 tx=0).stop 2024-07-04T17:31:31.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.227+0000 7efce9c00640 1 -- 172.21.15.42:0/2976729814 shutdown_connections 2024-07-04T17:31:31.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.227+0000 7efce9c00640 1 --2- 172.21.15.42:0/2976729814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efce40adaa0 0x7efce406d110 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:31.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.227+0000 7efce9c00640 1 -- 172.21.15.42:0/2976729814 >> 172.21.15.42:0/2976729814 conn(0x7efce406c020 msgr2=0x7efce406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:31.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.227+0000 7efce9c00640 1 -- 172.21.15.42:0/2976729814 shutdown_connections 2024-07-04T17:31:31.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.227+0000 7efce9c00640 1 -- 172.21.15.42:0/2976729814 wait complete. 2024-07-04T17:31:31.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.229+0000 7efce9c00640 1 Processor -- start 2024-07-04T17:31:31.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.230+0000 7efce9c00640 1 -- start start 2024-07-04T17:31:31.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.231+0000 7efce9c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efce40adaa0 0x7efce414afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:31.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.231+0000 7efce9c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efce41413d0 con 0x7efce40adaa0 2024-07-04T17:31:31.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.231+0000 7efce3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efce40adaa0 0x7efce414afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:31.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.231+0000 7efce3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efce40adaa0 0x7efce414afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56276/0 (socket says 172.21.15.42:56276) 2024-07-04T17:31:31.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.231+0000 7efce3e00640 1 -- 172.21.15.42:0/3519678700 learned_addr learned my addr 172.21.15.42:0/3519678700 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:31.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.232+0000 7efce3e00640 1 -- 172.21.15.42:0/3519678700 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efce4149430 con 0x7efce40adaa0 2024-07-04T17:31:31.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.232+0000 7efce3e00640 1 --2- 172.21.15.42:0/3519678700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efce40adaa0 0x7efce414afb0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7efcd4009780 tx=0x7efcd4033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:31.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.232+0000 7efcdbe00640 1 -- 172.21.15.42:0/3519678700 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efcd4031bc0 con 0x7efce40adaa0 2024-07-04T17:31:31.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.232+0000 7efcdbe00640 1 -- 172.21.15.42:0/3519678700 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efcd4031e50 con 0x7efce40adaa0 2024-07-04T17:31:31.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.232+0000 7efcdbe00640 1 -- 172.21.15.42:0/3519678700 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efcd4033200 con 0x7efce40adaa0 2024-07-04T17:31:31.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.232+0000 7efce9c00640 1 -- 172.21.15.42:0/3519678700 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efce414b6c0 con 0x7efce40adaa0 2024-07-04T17:31:31.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.232+0000 7efce9c00640 1 -- 172.21.15.42:0/3519678700 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efce4140db0 con 0x7efce40adaa0 2024-07-04T17:31:31.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.234+0000 7efce9c00640 1 -- 172.21.15.42:0/3519678700 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efce414b4f0 con 0x7efce40adaa0 2024-07-04T17:31:31.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.234+0000 7efcdbe00640 1 -- 172.21.15.42:0/3519678700 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7efcd40333a0 con 0x7efce40adaa0 2024-07-04T17:31:31.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.234+0000 7efcdbe00640 1 --2- 172.21.15.42:0/3519678700 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7efcc003d3d0 0x7efcc003f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:31.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.234+0000 7efcdbe00640 1 -- 172.21.15.42:0/3519678700 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 3189+0+0 (secure 0 0 0) 0x7efcd4072aa0 con 0x7efce40adaa0 2024-07-04T17:31:31.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.235+0000 7efce3400640 1 --2- 172.21.15.42:0/3519678700 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7efcc003d3d0 0x7efcc003f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:31.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.236+0000 7efce3400640 1 --2- 172.21.15.42:0/3519678700 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7efcc003d3d0 0x7efcc003f8e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efce4147530 tx=0x7efccc001650 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:31.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.239+0000 7efcdbe00640 1 -- 172.21.15.42:0/3519678700 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7efcd403f030 con 0x7efce40adaa0 2024-07-04T17:31:31.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.450+0000 7efce9c00640 1 -- 172.21.15.42:0/3519678700 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config reset", "num": 22} v 0) -- 0x7efce4141ae0 con 0x7efce40adaa0 2024-07-04T17:31:31.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.454+0000 7efcdbe00640 1 -- 172.21.15.42:0/3519678700 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 22}]=0 v28) ==== 71+0+0 (secure 0 0 0) 0x7efcd4031660 con 0x7efce40adaa0 2024-07-04T17:31:31.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.458+0000 7efce9c00640 1 -- 172.21.15.42:0/3519678700 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7efcc003d3d0 msgr2=0x7efcc003f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:31.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.458+0000 7efce9c00640 1 --2- 172.21.15.42:0/3519678700 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7efcc003d3d0 0x7efcc003f8e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efce4147530 tx=0x7efccc001650 comp rx=0 tx=0).stop 2024-07-04T17:31:31.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.458+0000 7efce9c00640 1 -- 172.21.15.42:0/3519678700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efce40adaa0 msgr2=0x7efce414afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:31.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.458+0000 7efce9c00640 1 --2- 172.21.15.42:0/3519678700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efce40adaa0 0x7efce414afb0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7efcd4009780 tx=0x7efcd4033e40 comp rx=0 tx=0).stop 2024-07-04T17:31:31.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.458+0000 7efce9c00640 1 -- 172.21.15.42:0/3519678700 shutdown_connections 2024-07-04T17:31:31.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.458+0000 7efce9c00640 1 --2- 172.21.15.42:0/3519678700 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7efcc003d3d0 0x7efcc003f8e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:31.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.458+0000 7efce9c00640 1 --2- 172.21.15.42:0/3519678700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efce40adaa0 0x7efce414afb0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:31.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.458+0000 7efce9c00640 1 -- 172.21.15.42:0/3519678700 >> 172.21.15.42:0/3519678700 conn(0x7efce406c020 msgr2=0x7efce4145ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:31.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.458+0000 7efce9c00640 1 -- 172.21.15.42:0/3519678700 shutdown_connections 2024-07-04T17:31:31.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.458+0000 7efce9c00640 1 -- 172.21.15.42:0/3519678700 wait complete. 2024-07-04T17:31:31.486 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_mirroring.TestMirroring.test_add_directory_with_mirroring_disabled' 2024-07-04T17:31:31.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.780+0000 7fe924800640 1 -- 172.21.15.42:0/47822917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe91c06cea0 msgr2=0x7fe91c0b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:31.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.780+0000 7fe924800640 1 --2- 172.21.15.42:0/47822917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe91c06cea0 0x7fe91c0b3030 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fe910009230 tx=0x7fe91002a480 comp rx=0 tx=0).stop 2024-07-04T17:31:31.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.780+0000 7fe924800640 1 -- 172.21.15.42:0/47822917 shutdown_connections 2024-07-04T17:31:31.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.780+0000 7fe924800640 1 --2- 172.21.15.42:0/47822917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe91c06cea0 0x7fe91c0b3030 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:31.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.780+0000 7fe924800640 1 -- 172.21.15.42:0/47822917 >> 172.21.15.42:0/47822917 conn(0x7fe91c0bf700 msgr2=0x7fe91c0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:31.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.780+0000 7fe924800640 1 -- 172.21.15.42:0/47822917 shutdown_connections 2024-07-04T17:31:31.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.780+0000 7fe924800640 1 -- 172.21.15.42:0/47822917 wait complete. 2024-07-04T17:31:31.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.782+0000 7fe924800640 1 Processor -- start 2024-07-04T17:31:31.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.782+0000 7fe924800640 1 -- start start 2024-07-04T17:31:31.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.784+0000 7fe924800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe91c06cea0 0x7fe91c0aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:31.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.784+0000 7fe924800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe91c06ef50 con 0x7fe91c06cea0 2024-07-04T17:31:31.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.784+0000 7fe921e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe91c06cea0 0x7fe91c0aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:31.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.784+0000 7fe921e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe91c06cea0 0x7fe91c0aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56296/0 (socket says 172.21.15.42:56296) 2024-07-04T17:31:31.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.784+0000 7fe921e00640 1 -- 172.21.15.42:0/1967825476 learned_addr learned my addr 172.21.15.42:0/1967825476 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:31.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.784+0000 7fe921e00640 1 -- 172.21.15.42:0/1967825476 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe91c073060 con 0x7fe91c06cea0 2024-07-04T17:31:31.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.785+0000 7fe921e00640 1 --2- 172.21.15.42:0/1967825476 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe91c06cea0 0x7fe91c0aede0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fe910009290 tx=0x7fe910031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:31.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.785+0000 7fe916000640 1 -- 172.21.15.42:0/1967825476 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe910041070 con 0x7fe91c06cea0 2024-07-04T17:31:31.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.785+0000 7fe916000640 1 -- 172.21.15.42:0/1967825476 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe910003be0 con 0x7fe91c06cea0 2024-07-04T17:31:31.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.785+0000 7fe924800640 1 -- 172.21.15.42:0/1967825476 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe91c1d32a0 con 0x7fe91c06cea0 2024-07-04T17:31:31.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.785+0000 7fe916000640 1 -- 172.21.15.42:0/1967825476 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe91003b440 con 0x7fe91c06cea0 2024-07-04T17:31:31.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.785+0000 7fe924800640 1 -- 172.21.15.42:0/1967825476 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe91c070e30 con 0x7fe91c06cea0 2024-07-04T17:31:31.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.786+0000 7fe924800640 1 -- 172.21.15.42:0/1967825476 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe91c063f70 con 0x7fe91c06cea0 2024-07-04T17:31:31.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.787+0000 7fe916000640 1 -- 172.21.15.42:0/1967825476 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7fe910044070 con 0x7fe91c06cea0 2024-07-04T17:31:31.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.787+0000 7fe916000640 1 --2- 172.21.15.42:0/1967825476 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fe90003d380 0x7fe90003f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:31.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.787+0000 7fe916000640 1 -- 172.21.15.42:0/1967825476 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 3189+0+0 (secure 0 0 0) 0x7fe910072db0 con 0x7fe91c06cea0 2024-07-04T17:31:31.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.788+0000 7fe921400640 1 --2- 172.21.15.42:0/1967825476 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fe90003d380 0x7fe90003f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:31.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.788+0000 7fe921400640 1 --2- 172.21.15.42:0/1967825476 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fe90003d380 0x7fe90003f890 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe90c007fd0 tx=0x7fe90c007920 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:31.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:31.794+0000 7fe916000640 1 -- 172.21.15.42:0/1967825476 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fe91c063f70 con 0x7fe91c06cea0 2024-07-04T17:31:32.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:32.083+0000 7fe924800640 1 -- 172.21.15.42:0/1967825476 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_mirroring.TestMirroring.test_add_directory_with_mirroring_disabled"]} v 0) -- 0x7fe91c0b3030 con 0x7fe91c06cea0 2024-07-04T17:31:32.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:32.916+0000 7fe916000640 1 -- 172.21.15.42:0/1967825476 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_mirroring.TestMirroring.test_add_directory_with_mirroring_disabled"]}]=0 v285) ==== 163+0+0 (secure 0 0 0) 0x7fe910041e40 con 0x7fe91c06cea0 2024-07-04T17:31:32.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:32.919+0000 7fe924800640 1 -- 172.21.15.42:0/1967825476 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fe90003d380 msgr2=0x7fe90003f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:32.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:32.919+0000 7fe924800640 1 --2- 172.21.15.42:0/1967825476 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fe90003d380 0x7fe90003f890 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe90c007fd0 tx=0x7fe90c007920 comp rx=0 tx=0).stop 2024-07-04T17:31:32.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:32.919+0000 7fe924800640 1 -- 172.21.15.42:0/1967825476 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe91c06cea0 msgr2=0x7fe91c0aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:32.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:32.919+0000 7fe924800640 1 --2- 172.21.15.42:0/1967825476 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe91c06cea0 0x7fe91c0aede0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fe910009290 tx=0x7fe910031a90 comp rx=0 tx=0).stop 2024-07-04T17:31:32.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:32.920+0000 7fe924800640 1 -- 172.21.15.42:0/1967825476 shutdown_connections 2024-07-04T17:31:32.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:32.920+0000 7fe924800640 1 --2- 172.21.15.42:0/1967825476 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fe90003d380 0x7fe90003f890 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:32.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:32.920+0000 7fe924800640 1 --2- 172.21.15.42:0/1967825476 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe91c06cea0 0x7fe91c0aede0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:32.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:32.920+0000 7fe924800640 1 -- 172.21.15.42:0/1967825476 >> 172.21.15.42:0/1967825476 conn(0x7fe91c0bf700 msgr2=0x7fe91c1490c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:32.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:32.920+0000 7fe924800640 1 -- 172.21.15.42:0/1967825476 shutdown_connections 2024-07-04T17:31:32.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:32.920+0000 7fe924800640 1 -- 172.21.15.42:0/1967825476 wait complete. 2024-07-04T17:31:32.955 INFO:tasks.cephfs_test_runner:test_add_directory_with_mirroring_disabled (tasks.cephfs.test_mirroring.TestMirroring) ... ok 2024-07-04T17:31:32.955 INFO:tasks.cephfs_test_runner:Starting test: test_add_relative_directory_path (tasks.cephfs.test_mirroring.TestMirroring) 2024-07-04T17:31:32.956 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:31:33.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.251+0000 7fdf87200640 1 -- 172.21.15.42:0/1184839486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf8006c200 msgr2=0x7fdf8014ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:33.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.251+0000 7fdf87200640 1 --2- 172.21.15.42:0/1184839486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf8006c200 0x7fdf8014ef10 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fdf74005fd0 tx=0x7fdf7402a560 comp rx=0 tx=0).stop 2024-07-04T17:31:33.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.251+0000 7fdf87200640 1 -- 172.21.15.42:0/1184839486 shutdown_connections 2024-07-04T17:31:33.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.251+0000 7fdf87200640 1 --2- 172.21.15.42:0/1184839486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf8006c200 0x7fdf8014ef10 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:33.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.251+0000 7fdf87200640 1 -- 172.21.15.42:0/1184839486 >> 172.21.15.42:0/1184839486 conn(0x7fdf800bebc0 msgr2=0x7fdf800befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:33.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.251+0000 7fdf87200640 1 -- 172.21.15.42:0/1184839486 shutdown_connections 2024-07-04T17:31:33.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.251+0000 7fdf87200640 1 -- 172.21.15.42:0/1184839486 wait complete. 2024-07-04T17:31:33.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.253+0000 7fdf87200640 1 Processor -- start 2024-07-04T17:31:33.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.254+0000 7fdf87200640 1 -- start start 2024-07-04T17:31:33.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.255+0000 7fdf87200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf8006c200 0x7fdf800b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:33.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.255+0000 7fdf87200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8014ae50 con 0x7fdf8006c200 2024-07-04T17:31:33.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.255+0000 7fdf85e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf8006c200 0x7fdf800b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.255+0000 7fdf85e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf8006c200 0x7fdf800b29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56326/0 (socket says 172.21.15.42:56326) 2024-07-04T17:31:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.255+0000 7fdf85e00640 1 -- 172.21.15.42:0/26573703 learned_addr learned my addr 172.21.15.42:0/26573703 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.256+0000 7fdf85e00640 1 -- 172.21.15.42:0/26573703 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf801e3450 con 0x7fdf8006c200 2024-07-04T17:31:33.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.256+0000 7fdf85e00640 1 --2- 172.21.15.42:0/26573703 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf8006c200 0x7fdf800b29c0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fdf74009850 tx=0x7fdf740049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:33.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.256+0000 7fdf7a000640 1 -- 172.21.15.42:0/26573703 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf7403e030 con 0x7fdf8006c200 2024-07-04T17:31:33.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.256+0000 7fdf7a000640 1 -- 172.21.15.42:0/26573703 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf740035f0 con 0x7fdf8006c200 2024-07-04T17:31:33.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.256+0000 7fdf87200640 1 -- 172.21.15.42:0/26573703 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf80148560 con 0x7fdf8006c200 2024-07-04T17:31:33.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.256+0000 7fdf7a000640 1 -- 172.21.15.42:0/26573703 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf74035040 con 0x7fdf8006c200 2024-07-04T17:31:33.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.257+0000 7fdf87200640 1 -- 172.21.15.42:0/26573703 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf80070830 con 0x7fdf8006c200 2024-07-04T17:31:33.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.258+0000 7fdf87200640 1 -- 172.21.15.42:0/26573703 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf801e6970 con 0x7fdf8006c200 2024-07-04T17:31:33.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.258+0000 7fdf7a000640 1 -- 172.21.15.42:0/26573703 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7fdf74041030 con 0x7fdf8006c200 2024-07-04T17:31:33.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.259+0000 7fdf7a000640 1 --2- 172.21.15.42:0/26573703 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fdf6403d380 0x7fdf6403f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:33.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.259+0000 7fdf7a000640 1 -- 172.21.15.42:0/26573703 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 3189+0+0 (secure 0 0 0) 0x7fdf7403f070 con 0x7fdf8006c200 2024-07-04T17:31:33.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.259+0000 7fdf85400640 1 --2- 172.21.15.42:0/26573703 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fdf6403d380 0x7fdf6403f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:33.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.260+0000 7fdf85400640 1 --2- 172.21.15.42:0/26573703 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fdf6403d380 0x7fdf6403f890 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdf800bac80 tx=0x7fdf700079f0 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:33.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.264+0000 7fdf7a000640 1 -- 172.21.15.42:0/26573703 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fdf740039e0 con 0x7fdf8006c200 2024-07-04T17:31:33.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.468+0000 7fdf87200640 1 -- 172.21.15.42:0/26573703 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf8014b490 con 0x7fdf8006c200 2024-07-04T17:31:33.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.469+0000 7fdf7a000640 1 -- 172.21.15.42:0/26573703 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+7653 (secure 0 0 0) 0x7fdf74042cc0 con 0x7fdf8006c200 2024-07-04T17:31:33.471 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:31:33.471 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":83,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:31:30.892145+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6833/2038326930":"2024-07-05T17:31:27.878258+0000","172.21.15.42:6832/2038326930":"2024-07-05T17:31:27.878258+0000","172.21.15.42:6835/1458267161":"2024-07-05T17:31:21.649384+0000","172.21.15.42:6834/1458267161":"2024-07-05T17:31:21.649384+0000","172.21.15.42:0/1725822177":"2024-07-05T17:31:19.572051+0000","172.21.15.42:0/1378827460":"2024-07-05T17:31:19.572051+0000","172.21.15.42:0/2771347343":"2024-07-05T17:31:19.572051+0000","172.21.15.42:0/2082652266":"2024-07-05T17:31:19.572051+0000","172.21.15.42:6825/3473020706":"2024-07-05T17:31:19.572051+0000","172.21.15.42:6824/3473020706":"2024-07-05T17:31:19.572051+0000","172.21.15.42:0/963042308":"2024-07-05T17:31:19.572051+0000","172.21.15.42:0/404958516":"2024-07-05T17:31:19.572051+0000","172.21.15.42:0/1340275465":"2024-07-05T17:31:19.572051+0000","172.21.15.42:0/1154056961":"2024-07-05T17:31:19.572051+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:31:33.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.472+0000 7fdf87200640 1 -- 172.21.15.42:0/26573703 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fdf6403d380 msgr2=0x7fdf6403f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:33.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.472+0000 7fdf87200640 1 --2- 172.21.15.42:0/26573703 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fdf6403d380 0x7fdf6403f890 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdf800bac80 tx=0x7fdf700079f0 comp rx=0 tx=0).stop 2024-07-04T17:31:33.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.472+0000 7fdf87200640 1 -- 172.21.15.42:0/26573703 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf8006c200 msgr2=0x7fdf800b29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:33.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.472+0000 7fdf87200640 1 --2- 172.21.15.42:0/26573703 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf8006c200 0x7fdf800b29c0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fdf74009850 tx=0x7fdf740049d0 comp rx=0 tx=0).stop 2024-07-04T17:31:33.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.472+0000 7fdf87200640 1 -- 172.21.15.42:0/26573703 shutdown_connections 2024-07-04T17:31:33.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.472+0000 7fdf87200640 1 --2- 172.21.15.42:0/26573703 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fdf6403d380 0x7fdf6403f890 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:33.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.472+0000 7fdf87200640 1 --2- 172.21.15.42:0/26573703 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf8006c200 0x7fdf800b29c0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:33.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.472+0000 7fdf87200640 1 -- 172.21.15.42:0/26573703 >> 172.21.15.42:0/26573703 conn(0x7fdf800bebc0 msgr2=0x7fdf800bbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:33.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.472+0000 7fdf87200640 1 -- 172.21.15.42:0/26573703 shutdown_connections 2024-07-04T17:31:33.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.472+0000 7fdf87200640 1 -- 172.21.15.42:0/26573703 wait complete. 2024-07-04T17:31:33.499 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-04T17:26:03.297630+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-04T17:31:33.499 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-04T17:31:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.797+0000 7ff20b200640 1 -- 172.21.15.42:0/3682529110 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2040b7760 msgr2=0x7ff2040b7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.797+0000 7ff20b200640 1 --2- 172.21.15.42:0/3682529110 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2040b7760 0x7ff2040b7b70 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7ff1f40097b0 tx=0x7ff1f402a530 comp rx=0 tx=0).stop 2024-07-04T17:31:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.798+0000 7ff20b200640 1 -- 172.21.15.42:0/3682529110 shutdown_connections 2024-07-04T17:31:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.798+0000 7ff20b200640 1 --2- 172.21.15.42:0/3682529110 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2040b7760 0x7ff2040b7b70 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:33.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.798+0000 7ff20b200640 1 -- 172.21.15.42:0/3682529110 >> 172.21.15.42:0/3682529110 conn(0x7ff2040bf260 msgr2=0x7ff2040af5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:33.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.798+0000 7ff20b200640 1 -- 172.21.15.42:0/3682529110 shutdown_connections 2024-07-04T17:31:33.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.798+0000 7ff20b200640 1 -- 172.21.15.42:0/3682529110 wait complete. 2024-07-04T17:31:33.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.800+0000 7ff20b200640 1 Processor -- start 2024-07-04T17:31:33.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.800+0000 7ff20b200640 1 -- start start 2024-07-04T17:31:33.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.801+0000 7ff20b200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2040aff70 0x7ff2041d8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:33.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.801+0000 7ff20b200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff2041484b0 con 0x7ff2040aff70 2024-07-04T17:31:33.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.801+0000 7ff209e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2040aff70 0x7ff2041d8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:33.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.801+0000 7ff209e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2040aff70 0x7ff2041d8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56332/0 (socket says 172.21.15.42:56332) 2024-07-04T17:31:33.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.801+0000 7ff209e00640 1 -- 172.21.15.42:0/1308454021 learned_addr learned my addr 172.21.15.42:0/1308454021 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:33.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.802+0000 7ff209e00640 1 -- 172.21.15.42:0/1308454021 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2040af040 con 0x7ff2040aff70 2024-07-04T17:31:33.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.802+0000 7ff209e00640 1 --2- 172.21.15.42:0/1308454021 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2040aff70 0x7ff2041d8ce0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7ff1f800c6b0 tx=0x7ff1f800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:33.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.802+0000 7ff1fe000640 1 -- 172.21.15.42:0/1308454021 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1f8006890 con 0x7ff2040aff70 2024-07-04T17:31:33.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.802+0000 7ff1fe000640 1 -- 172.21.15.42:0/1308454021 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1f8017070 con 0x7ff2040aff70 2024-07-04T17:31:33.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.802+0000 7ff1fe000640 1 -- 172.21.15.42:0/1308454021 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1f8006d10 con 0x7ff2040aff70 2024-07-04T17:31:33.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.802+0000 7ff20b200640 1 -- 172.21.15.42:0/1308454021 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2040be840 con 0x7ff2040aff70 2024-07-04T17:31:33.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.803+0000 7ff20b200640 1 -- 172.21.15.42:0/1308454021 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2040adef0 con 0x7ff2040aff70 2024-07-04T17:31:33.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.804+0000 7ff20b200640 1 -- 172.21.15.42:0/1308454021 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff204148af0 con 0x7ff2040aff70 2024-07-04T17:31:33.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.804+0000 7ff1fe000640 1 -- 172.21.15.42:0/1308454021 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7ff1f8007050 con 0x7ff2040aff70 2024-07-04T17:31:33.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.804+0000 7ff1fe000640 1 --2- 172.21.15.42:0/1308454021 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7ff1e803d3d0 0x7ff1e803f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:33.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.805+0000 7ff1fe000640 1 -- 172.21.15.42:0/1308454021 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 3189+0+0 (secure 0 0 0) 0x7ff1f8051320 con 0x7ff2040aff70 2024-07-04T17:31:33.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.805+0000 7ff209400640 1 --2- 172.21.15.42:0/1308454021 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7ff1e803d3d0 0x7ff1e803f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:33.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.806+0000 7ff209400640 1 --2- 172.21.15.42:0/1308454021 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7ff1e803d3d0 0x7ff1e803f8e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4034000 tx=0x7ff1f4004730 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:33.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:33.810+0000 7ff1fe000640 1 -- 172.21.15.42:0/1308454021 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7ff2040adef0 con 0x7ff2040aff70 2024-07-04T17:31:34.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.018+0000 7ff20b200640 1 -- 172.21.15.42:0/1308454021 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff204068e30 con 0x7ff2040aff70 2024-07-04T17:31:34.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.019+0000 7ff1fe000640 1 -- 172.21.15.42:0/1308454021 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7ff2040b0380 con 0x7ff2040aff70 2024-07-04T17:31:34.021 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-07-04T17:31:34.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.022+0000 7ff20b200640 1 -- 172.21.15.42:0/1308454021 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7ff1e803d3d0 msgr2=0x7ff1e803f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:34.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.022+0000 7ff20b200640 1 --2- 172.21.15.42:0/1308454021 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7ff1e803d3d0 0x7ff1e803f8e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4034000 tx=0x7ff1f4004730 comp rx=0 tx=0).stop 2024-07-04T17:31:34.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.022+0000 7ff20b200640 1 -- 172.21.15.42:0/1308454021 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2040aff70 msgr2=0x7ff2041d8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:34.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.022+0000 7ff20b200640 1 --2- 172.21.15.42:0/1308454021 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2040aff70 0x7ff2041d8ce0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7ff1f800c6b0 tx=0x7ff1f800cb70 comp rx=0 tx=0).stop 2024-07-04T17:31:34.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.022+0000 7ff20b200640 1 -- 172.21.15.42:0/1308454021 shutdown_connections 2024-07-04T17:31:34.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.022+0000 7ff20b200640 1 --2- 172.21.15.42:0/1308454021 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7ff1e803d3d0 0x7ff1e803f8e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:34.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.022+0000 7ff20b200640 1 --2- 172.21.15.42:0/1308454021 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2040aff70 0x7ff2041d8ce0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:34.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.022+0000 7ff20b200640 1 -- 172.21.15.42:0/1308454021 >> 172.21.15.42:0/1308454021 conn(0x7ff2040bf260 msgr2=0x7ff2041468a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:34.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.023+0000 7ff20b200640 1 -- 172.21.15.42:0/1308454021 shutdown_connections 2024-07-04T17:31:34.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.023+0000 7ff20b200640 1 -- 172.21.15.42:0/1308454021 wait complete. 2024-07-04T17:31:34.049 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_mirroring.TestMirroring.test_add_relative_directory_path' 2024-07-04T17:31:34.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.337+0000 7fd8f5200640 1 -- 172.21.15.42:0/2957254816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8f00b37d0 msgr2=0x7fd8f00b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:34.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.337+0000 7fd8f5200640 1 --2- 172.21.15.42:0/2957254816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8f00b37d0 0x7fd8f00b01f0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8005fd0 tx=0x7fd8d802a5f0 comp rx=0 tx=0).stop 2024-07-04T17:31:34.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.337+0000 7fd8f5200640 1 -- 172.21.15.42:0/2957254816 shutdown_connections 2024-07-04T17:31:34.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.337+0000 7fd8f5200640 1 --2- 172.21.15.42:0/2957254816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8f00b37d0 0x7fd8f00b01f0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:34.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.337+0000 7fd8f5200640 1 -- 172.21.15.42:0/2957254816 >> 172.21.15.42:0/2957254816 conn(0x7fd8f00bf700 msgr2=0x7fd8f00af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:34.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.337+0000 7fd8f5200640 1 -- 172.21.15.42:0/2957254816 shutdown_connections 2024-07-04T17:31:34.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.337+0000 7fd8f5200640 1 -- 172.21.15.42:0/2957254816 wait complete. 2024-07-04T17:31:34.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.339+0000 7fd8f5200640 1 Processor -- start 2024-07-04T17:31:34.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.340+0000 7fd8f5200640 1 -- start start 2024-07-04T17:31:34.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.341+0000 7fd8f5200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8f00b2d20 0x7fd8f00aee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:34.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.341+0000 7fd8f5200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f006ce30 con 0x7fd8f00b2d20 2024-07-04T17:31:34.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.342+0000 7fd8ede00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8f00b2d20 0x7fd8f00aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:34.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.342+0000 7fd8ede00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8f00b2d20 0x7fd8f00aee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44396/0 (socket says 172.21.15.42:44396) 2024-07-04T17:31:34.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.342+0000 7fd8ede00640 1 -- 172.21.15.42:0/2220653612 learned_addr learned my addr 172.21.15.42:0/2220653612 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:34.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.342+0000 7fd8ede00640 1 -- 172.21.15.42:0/2220653612 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f0074380 con 0x7fd8f00b2d20 2024-07-04T17:31:34.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.342+0000 7fd8ede00640 1 --2- 172.21.15.42:0/2220653612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8f00b2d20 0x7fd8f00aee80 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0001eb0 tx=0x7fd8e000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:34.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.343+0000 7fd8e6000640 1 -- 172.21.15.42:0/2220653612 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8e0017070 con 0x7fd8f00b2d20 2024-07-04T17:31:34.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.343+0000 7fd8e6000640 1 -- 172.21.15.42:0/2220653612 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8e00052f0 con 0x7fd8f00b2d20 2024-07-04T17:31:34.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.343+0000 7fd8e6000640 1 -- 172.21.15.42:0/2220653612 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8e00035f0 con 0x7fd8f00b2d20 2024-07-04T17:31:34.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.343+0000 7fd8f5200640 1 -- 172.21.15.42:0/2220653612 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8f00af3c0 con 0x7fd8f00b2d20 2024-07-04T17:31:34.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.343+0000 7fd8f5200640 1 -- 172.21.15.42:0/2220653612 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8f00ae0c0 con 0x7fd8f00b2d20 2024-07-04T17:31:34.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.344+0000 7fd8f5200640 1 -- 172.21.15.42:0/2220653612 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8f006d490 con 0x7fd8f00b2d20 2024-07-04T17:31:34.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.344+0000 7fd8e6000640 1 -- 172.21.15.42:0/2220653612 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7fd8e001a070 con 0x7fd8f00b2d20 2024-07-04T17:31:34.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.345+0000 7fd8e6000640 1 --2- 172.21.15.42:0/2220653612 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fd8cc03d3d0 0x7fd8cc03f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:34.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.345+0000 7fd8e6000640 1 -- 172.21.15.42:0/2220653612 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 3189+0+0 (secure 0 0 0) 0x7fd8e00508f0 con 0x7fd8f00b2d20 2024-07-04T17:31:34.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.345+0000 7fd8e7e00640 1 --2- 172.21.15.42:0/2220653612 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fd8cc03d3d0 0x7fd8cc03f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:34.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.346+0000 7fd8e7e00640 1 --2- 172.21.15.42:0/2220653612 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fd8cc03d3d0 0x7fd8cc03f8e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8033000 tx=0x7fd8d8004730 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:34.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.350+0000 7fd8e6000640 1 -- 172.21.15.42:0/2220653612 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fd8e00102e0 con 0x7fd8f00b2d20 2024-07-04T17:31:34.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.648+0000 7fd8f5200640 1 -- 172.21.15.42:0/2220653612 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_mirroring.TestMirroring.test_add_relative_directory_path"]} v 0) -- 0x7fd8f00af6e0 con 0x7fd8f00b2d20 2024-07-04T17:31:34.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.924+0000 7fd8e6000640 1 -- 172.21.15.42:0/2220653612 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_mirroring.TestMirroring.test_add_relative_directory_path"]}]=0 v287) ==== 156+0+0 (secure 0 0 0) 0x7fd8e0050be0 con 0x7fd8f00b2d20 2024-07-04T17:31:34.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.927+0000 7fd8f5200640 1 -- 172.21.15.42:0/2220653612 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fd8cc03d3d0 msgr2=0x7fd8cc03f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:34.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.927+0000 7fd8f5200640 1 --2- 172.21.15.42:0/2220653612 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fd8cc03d3d0 0x7fd8cc03f8e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8033000 tx=0x7fd8d8004730 comp rx=0 tx=0).stop 2024-07-04T17:31:34.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.927+0000 7fd8f5200640 1 -- 172.21.15.42:0/2220653612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8f00b2d20 msgr2=0x7fd8f00aee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:34.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.927+0000 7fd8f5200640 1 --2- 172.21.15.42:0/2220653612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8f00b2d20 0x7fd8f00aee80 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0001eb0 tx=0x7fd8e000c810 comp rx=0 tx=0).stop 2024-07-04T17:31:34.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.928+0000 7fd8f5200640 1 -- 172.21.15.42:0/2220653612 shutdown_connections 2024-07-04T17:31:34.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.928+0000 7fd8f5200640 1 --2- 172.21.15.42:0/2220653612 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fd8cc03d3d0 0x7fd8cc03f8e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:34.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.928+0000 7fd8f5200640 1 --2- 172.21.15.42:0/2220653612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8f00b2d20 0x7fd8f00aee80 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:34.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.928+0000 7fd8f5200640 1 -- 172.21.15.42:0/2220653612 >> 172.21.15.42:0/2220653612 conn(0x7fd8f00bf700 msgr2=0x7fd8f00bc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:34.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.928+0000 7fd8f5200640 1 -- 172.21.15.42:0/2220653612 shutdown_connections 2024-07-04T17:31:34.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:34.928+0000 7fd8f5200640 1 -- 172.21.15.42:0/2220653612 wait complete. 2024-07-04T17:31:34.959 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-07-04T17:31:35.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.246+0000 7fca06600640 1 -- 172.21.15.42:0/2838769680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca0006c020 msgr2=0x7fca0014a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:35.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.246+0000 7fca06600640 1 --2- 172.21.15.42:0/2838769680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca0006c020 0x7fca0014a9e0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec005fd0 tx=0x7fc9ec02a8b0 comp rx=0 tx=0).stop 2024-07-04T17:31:35.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.246+0000 7fca06600640 1 -- 172.21.15.42:0/2838769680 shutdown_connections 2024-07-04T17:31:35.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.246+0000 7fca06600640 1 --2- 172.21.15.42:0/2838769680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca0006c020 0x7fca0014a9e0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:35.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.247+0000 7fca06600640 1 -- 172.21.15.42:0/2838769680 >> 172.21.15.42:0/2838769680 conn(0x7fca000bebc0 msgr2=0x7fca000befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:35.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.247+0000 7fca06600640 1 -- 172.21.15.42:0/2838769680 shutdown_connections 2024-07-04T17:31:35.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.247+0000 7fca06600640 1 -- 172.21.15.42:0/2838769680 wait complete. 2024-07-04T17:31:35.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.249+0000 7fca06600640 1 Processor -- start 2024-07-04T17:31:35.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.249+0000 7fca06600640 1 -- start start 2024-07-04T17:31:35.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.250+0000 7fca06600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca0006c020 0x7fca001d5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:35.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.250+0000 7fca06600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fca0014b550 con 0x7fca0006c020 2024-07-04T17:31:35.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.250+0000 7fca05200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca0006c020 0x7fca001d5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:35.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.250+0000 7fca05200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca0006c020 0x7fca001d5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44412/0 (socket says 172.21.15.42:44412) 2024-07-04T17:31:35.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.250+0000 7fca05200640 1 -- 172.21.15.42:0/3541305331 learned_addr learned my addr 172.21.15.42:0/3541305331 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:35.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.251+0000 7fca05200640 1 -- 172.21.15.42:0/3541305331 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca001d8fc0 con 0x7fca0006c020 2024-07-04T17:31:35.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.251+0000 7fca05200640 1 --2- 172.21.15.42:0/3541305331 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca0006c020 0x7fca001d5a60 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0001eb0 tx=0x7fc9f000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:35.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.251+0000 7fc9fcc00640 1 -- 172.21.15.42:0/3541305331 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9f00116c0 con 0x7fca0006c020 2024-07-04T17:31:35.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.252+0000 7fc9fcc00640 1 -- 172.21.15.42:0/3541305331 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9f0017070 con 0x7fca0006c020 2024-07-04T17:31:35.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.252+0000 7fca06600640 1 -- 172.21.15.42:0/3541305331 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca000b1070 con 0x7fca0006c020 2024-07-04T17:31:35.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.252+0000 7fc9fcc00640 1 -- 172.21.15.42:0/3541305331 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9f0011ae0 con 0x7fca0006c020 2024-07-04T17:31:35.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.252+0000 7fca06600640 1 -- 172.21.15.42:0/3541305331 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca001d7c80 con 0x7fca0006c020 2024-07-04T17:31:35.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.253+0000 7fc9fcc00640 1 -- 172.21.15.42:0/3541305331 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7fc9f0011c80 con 0x7fca0006c020 2024-07-04T17:31:35.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.253+0000 7fca06600640 1 -- 172.21.15.42:0/3541305331 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca0014bc80 con 0x7fca0006c020 2024-07-04T17:31:35.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.254+0000 7fc9fcc00640 1 --2- 172.21.15.42:0/3541305331 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fc9e003d3d0 0x7fc9e003f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:35.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.254+0000 7fc9fcc00640 1 -- 172.21.15.42:0/3541305331 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 3189+0+0 (secure 0 0 0) 0x7fc9f00512e0 con 0x7fca0006c020 2024-07-04T17:31:35.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.254+0000 7fc9ffe00640 1 --2- 172.21.15.42:0/3541305331 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fc9e003d3d0 0x7fc9e003f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:35.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.255+0000 7fc9ffe00640 1 --2- 172.21.15.42:0/3541305331 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fc9e003d3d0 0x7fc9e003f8e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec034000 tx=0x7fc9ec004ab0 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:35.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.258+0000 7fc9fcc00640 1 -- 172.21.15.42:0/3541305331 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fc9f000a140 con 0x7fca0006c020 2024-07-04T17:31:35.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.467+0000 7fca06600640 1 -- 172.21.15.42:0/3541305331 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fca0014f540 con 0x7fca0006c020 2024-07-04T17:31:35.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.468+0000 7fc9fcc00640 1 -- 172.21.15.42:0/3541305331 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v28) ==== 59+0+1628 (secure 0 0 0) 0x7fc9f0010cf0 con 0x7fca0006c020 2024-07-04T17:31:35.469 INFO:teuthology.orchestra.run.smithi042.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-07-04T17:31:35.469 INFO:teuthology.orchestra.run.smithi042.stdout:mon advanced mon_op_complaint_time 120 2024-07-04T17:31:35.469 INFO:teuthology.orchestra.run.smithi042.stdout:mon dev mon_warn_on_pool_no_app false 2024-07-04T17:31:35.469 INFO:teuthology.orchestra.run.smithi042.stdout:osd advanced osd_op_complaint_time 180.000000 2024-07-04T17:31:35.469 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3541.919687 2024-07-04T17:31:35.469 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3558.529833 2024-07-04T17:31:35.469 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3688.441103 2024-07-04T17:31:35.469 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced debug_mds 20/20 2024-07-04T17:31:35.469 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced debug_mds_balancer 20/20 2024-07-04T17:31:35.470 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced debug_ms 1/1 2024-07-04T17:31:35.470 INFO:teuthology.orchestra.run.smithi042.stdout:mds dev mds_debug_frag true 2024-07-04T17:31:35.470 INFO:teuthology.orchestra.run.smithi042.stdout:mds dev mds_debug_scatterstat true 2024-07-04T17:31:35.470 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced mds_op_complaint_time 180.000000 2024-07-04T17:31:35.470 INFO:teuthology.orchestra.run.smithi042.stdout:mds dev mds_verify_scatter true 2024-07-04T17:31:35.470 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced osd_op_complaint_time 180.000000 2024-07-04T17:31:35.470 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced rados_mon_op_timeout 900 2024-07-04T17:31:35.470 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced rados_osd_op_timeout 900 2024-07-04T17:31:35.470 INFO:teuthology.orchestra.run.smithi042.stdout:client advanced client_mount_timeout 600 2024-07-04T17:31:35.470 INFO:teuthology.orchestra.run.smithi042.stdout:client advanced debug_client 20/20 2024-07-04T17:31:35.470 INFO:teuthology.orchestra.run.smithi042.stdout:client advanced debug_ms 1/1 2024-07-04T17:31:35.470 INFO:teuthology.orchestra.run.smithi042.stdout:client advanced rados_mon_op_timeout 900 2024-07-04T17:31:35.470 INFO:teuthology.orchestra.run.smithi042.stdout:client advanced rados_osd_op_timeout 900 2024-07-04T17:31:35.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.470+0000 7fca06600640 1 -- 172.21.15.42:0/3541305331 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fc9e003d3d0 msgr2=0x7fc9e003f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:35.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.470+0000 7fca06600640 1 --2- 172.21.15.42:0/3541305331 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fc9e003d3d0 0x7fc9e003f8e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec034000 tx=0x7fc9ec004ab0 comp rx=0 tx=0).stop 2024-07-04T17:31:35.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.470+0000 7fca06600640 1 -- 172.21.15.42:0/3541305331 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca0006c020 msgr2=0x7fca001d5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:35.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.470+0000 7fca06600640 1 --2- 172.21.15.42:0/3541305331 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca0006c020 0x7fca001d5a60 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0001eb0 tx=0x7fc9f000c810 comp rx=0 tx=0).stop 2024-07-04T17:31:35.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.471+0000 7fca06600640 1 -- 172.21.15.42:0/3541305331 shutdown_connections 2024-07-04T17:31:35.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.471+0000 7fca06600640 1 --2- 172.21.15.42:0/3541305331 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fc9e003d3d0 0x7fc9e003f8e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:35.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.471+0000 7fca06600640 1 --2- 172.21.15.42:0/3541305331 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca0006c020 0x7fca001d5a60 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:35.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.471+0000 7fca06600640 1 -- 172.21.15.42:0/3541305331 >> 172.21.15.42:0/3541305331 conn(0x7fca000bebc0 msgr2=0x7fca000b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:35.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.471+0000 7fca06600640 1 -- 172.21.15.42:0/3541305331 shutdown_connections 2024-07-04T17:31:35.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.471+0000 7fca06600640 1 -- 172.21.15.42:0/3541305331 wait complete. 2024-07-04T17:31:35.497 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-07-04T17:31:35.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.782+0000 7f51e6e00640 1 -- 172.21.15.42:0/2784967411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51e0149cb0 msgr2=0x7f51e00b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:35.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.782+0000 7f51e6e00640 1 --2- 172.21.15.42:0/2784967411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51e0149cb0 0x7f51e00b25d0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f51c80097b0 tx=0x7f51c802a540 comp rx=0 tx=0).stop 2024-07-04T17:31:35.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.782+0000 7f51e6e00640 1 -- 172.21.15.42:0/2784967411 shutdown_connections 2024-07-04T17:31:35.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.782+0000 7f51e6e00640 1 --2- 172.21.15.42:0/2784967411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51e0149cb0 0x7f51e00b25d0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:35.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.782+0000 7f51e6e00640 1 -- 172.21.15.42:0/2784967411 >> 172.21.15.42:0/2784967411 conn(0x7f51e00bf4d0 msgr2=0x7f51e00af870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:35.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.782+0000 7f51e6e00640 1 -- 172.21.15.42:0/2784967411 shutdown_connections 2024-07-04T17:31:35.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.783+0000 7f51e6e00640 1 -- 172.21.15.42:0/2784967411 wait complete. 2024-07-04T17:31:35.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.784+0000 7f51e6e00640 1 Processor -- start 2024-07-04T17:31:35.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.785+0000 7f51e6e00640 1 -- start start 2024-07-04T17:31:35.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.786+0000 7f51e6e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51e006ee40 0x7f51e01535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:35.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.786+0000 7f51e6e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f51e00b0310 con 0x7f51e006ee40 2024-07-04T17:31:35.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.786+0000 7f51dfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51e006ee40 0x7f51e01535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:35.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.786+0000 7f51dfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51e006ee40 0x7f51e01535a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44432/0 (socket says 172.21.15.42:44432) 2024-07-04T17:31:35.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.786+0000 7f51dfe00640 1 -- 172.21.15.42:0/1341117633 learned_addr learned my addr 172.21.15.42:0/1341117633 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:35.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.787+0000 7f51dfe00640 1 -- 172.21.15.42:0/1341117633 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e0153b80 con 0x7f51e006ee40 2024-07-04T17:31:35.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.787+0000 7f51dfe00640 1 --2- 172.21.15.42:0/1341117633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51e006ee40 0x7f51e01535a0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f51d000c380 tx=0x7f51d000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:35.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.787+0000 7f51dcc00640 1 -- 172.21.15.42:0/1341117633 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51d00116c0 con 0x7f51e006ee40 2024-07-04T17:31:35.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.787+0000 7f51dcc00640 1 -- 172.21.15.42:0/1341117633 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51d0017070 con 0x7f51e006ee40 2024-07-04T17:31:35.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.787+0000 7f51dcc00640 1 -- 172.21.15.42:0/1341117633 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51d0011b20 con 0x7f51e006ee40 2024-07-04T17:31:35.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.787+0000 7f51e6e00640 1 -- 172.21.15.42:0/1341117633 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51e01e4190 con 0x7f51e006ee40 2024-07-04T17:31:35.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.787+0000 7f51e6e00640 1 -- 172.21.15.42:0/1341117633 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51e0151940 con 0x7f51e006ee40 2024-07-04T17:31:35.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.788+0000 7f51e6e00640 1 -- 172.21.15.42:0/1341117633 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51e00b25d0 con 0x7f51e006ee40 2024-07-04T17:31:35.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.788+0000 7f51dcc00640 1 -- 172.21.15.42:0/1341117633 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7f51d001c460 con 0x7f51e006ee40 2024-07-04T17:31:35.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.789+0000 7f51dcc00640 1 --2- 172.21.15.42:0/1341117633 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f51bc03d380 0x7f51bc03f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:35.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.789+0000 7f51dcc00640 1 -- 172.21.15.42:0/1341117633 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 3189+0+0 (secure 0 0 0) 0x7f51d0051430 con 0x7f51e006ee40 2024-07-04T17:31:35.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.789+0000 7f51d7400640 1 --2- 172.21.15.42:0/1341117633 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f51bc03d380 0x7f51bc03f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:35.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.790+0000 7f51d7400640 1 --2- 172.21.15.42:0/1341117633 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f51bc03d380 0x7f51bc03f890 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f51c8034000 tx=0x7f51c8004730 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:35.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:35.794+0000 7f51dcc00640 1 -- 172.21.15.42:0/1341117633 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f51d0011cc0 con 0x7f51e006ee40 2024-07-04T17:31:36.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.007+0000 7f51e6e00640 1 -- 172.21.15.42:0/1341117633 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f51e0148a80 con 0x7f51e006ee40 2024-07-04T17:31:36.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.010+0000 7f51dcc00640 1 -- 172.21.15.42:0/1341117633 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v29)=0 v29) ==== 122+0+0 (secure 0 0 0) 0x7f51d001c800 con 0x7f51e006ee40 2024-07-04T17:31:36.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.013+0000 7f51e6e00640 1 -- 172.21.15.42:0/1341117633 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f51bc03d380 msgr2=0x7f51bc03f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:36.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.013+0000 7f51e6e00640 1 --2- 172.21.15.42:0/1341117633 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f51bc03d380 0x7f51bc03f890 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f51c8034000 tx=0x7f51c8004730 comp rx=0 tx=0).stop 2024-07-04T17:31:36.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.013+0000 7f51e6e00640 1 -- 172.21.15.42:0/1341117633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51e006ee40 msgr2=0x7f51e01535a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:36.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.013+0000 7f51e6e00640 1 --2- 172.21.15.42:0/1341117633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51e006ee40 0x7f51e01535a0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f51d000c380 tx=0x7f51d000c840 comp rx=0 tx=0).stop 2024-07-04T17:31:36.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.013+0000 7f51e6e00640 1 -- 172.21.15.42:0/1341117633 shutdown_connections 2024-07-04T17:31:36.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.013+0000 7f51e6e00640 1 --2- 172.21.15.42:0/1341117633 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f51bc03d380 0x7f51bc03f890 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:36.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.013+0000 7f51e6e00640 1 --2- 172.21.15.42:0/1341117633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51e006ee40 0x7f51e01535a0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:36.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.013+0000 7f51e6e00640 1 -- 172.21.15.42:0/1341117633 >> 172.21.15.42:0/1341117633 conn(0x7f51e00bf4d0 msgr2=0x7f51e0149040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:36.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.014+0000 7f51e6e00640 1 -- 172.21.15.42:0/1341117633 shutdown_connections 2024-07-04T17:31:36.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.014+0000 7f51e6e00640 1 -- 172.21.15.42:0/1341117633 wait complete. 2024-07-04T17:31:36.041 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-04T17:31:36.041 DEBUG:teuthology.orchestra.run.smithi042:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-04T17:31:36.148 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-04T17:31:36.149 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:31:36.149 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:31:36.200 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-07-04T17:31:36.200 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:31:36.200 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:31:36.248 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-07-04T17:31:36.248 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-07-04T17:31:36.248 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:31:36.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.587+0000 7f4544c00640 1 -- 172.21.15.42:0/1609671476 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45400b3030 msgr2=0x7f45400b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:36.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.587+0000 7f4544c00640 1 --2- 172.21.15.42:0/1609671476 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45400b3030 0x7f45400b01f0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f452800b2b0 tx=0x7f452802ad10 comp rx=0 tx=0).stop 2024-07-04T17:31:36.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.587+0000 7f4544c00640 1 -- 172.21.15.42:0/1609671476 shutdown_connections 2024-07-04T17:31:36.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.587+0000 7f4544c00640 1 --2- 172.21.15.42:0/1609671476 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45400b3030 0x7f45400b01f0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:36.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.587+0000 7f4544c00640 1 -- 172.21.15.42:0/1609671476 >> 172.21.15.42:0/1609671476 conn(0x7f45400bf700 msgr2=0x7f45400af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:36.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.587+0000 7f4544c00640 1 -- 172.21.15.42:0/1609671476 shutdown_connections 2024-07-04T17:31:36.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.587+0000 7f4544c00640 1 -- 172.21.15.42:0/1609671476 wait complete. 2024-07-04T17:31:36.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.589+0000 7f4544c00640 1 Processor -- start 2024-07-04T17:31:36.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.589+0000 7f4544c00640 1 -- start start 2024-07-04T17:31:36.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.591+0000 7f4544c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f454006cea0 0x7f45400ad180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:36.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.591+0000 7f4544c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f454006ef50 con 0x7f454006cea0 2024-07-04T17:31:36.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.591+0000 7f453de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f454006cea0 0x7f45400ad180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:36.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.591+0000 7f453de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f454006cea0 0x7f45400ad180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44442/0 (socket says 172.21.15.42:44442) 2024-07-04T17:31:36.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.591+0000 7f453de00640 1 -- 172.21.15.42:0/1540825461 learned_addr learned my addr 172.21.15.42:0/1540825461 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:36.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.591+0000 7f453de00640 1 -- 172.21.15.42:0/1540825461 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4540073970 con 0x7f454006cea0 2024-07-04T17:31:36.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.592+0000 7f453de00640 1 --2- 172.21.15.42:0/1540825461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f454006cea0 0x7f45400ad180 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f453000c320 tx=0x7f453000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:36.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.592+0000 7f4536000640 1 -- 172.21.15.42:0/1540825461 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45300116c0 con 0x7f454006cea0 2024-07-04T17:31:36.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.592+0000 7f4536000640 1 -- 172.21.15.42:0/1540825461 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4530017070 con 0x7f454006cea0 2024-07-04T17:31:36.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.592+0000 7f4536000640 1 -- 172.21.15.42:0/1540825461 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4530011b40 con 0x7f454006cea0 2024-07-04T17:31:36.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.592+0000 7f4544c00640 1 -- 172.21.15.42:0/1540825461 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45400adbe0 con 0x7f454006cea0 2024-07-04T17:31:36.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.592+0000 7f4544c00640 1 -- 172.21.15.42:0/1540825461 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4540073e10 con 0x7f454006cea0 2024-07-04T17:31:36.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.593+0000 7f4544c00640 1 -- 172.21.15.42:0/1540825461 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45400b01f0 con 0x7f454006cea0 2024-07-04T17:31:36.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.593+0000 7f4536000640 1 -- 172.21.15.42:0/1540825461 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7f4530011ce0 con 0x7f454006cea0 2024-07-04T17:31:36.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.594+0000 7f4536000640 1 --2- 172.21.15.42:0/1540825461 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f451c03d3d0 0x7f451c03f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:36.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.594+0000 7f4536000640 1 -- 172.21.15.42:0/1540825461 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 3189+0+0 (secure 0 0 0) 0x7f4530051360 con 0x7f454006cea0 2024-07-04T17:31:36.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.594+0000 7f453d400640 1 --2- 172.21.15.42:0/1540825461 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f451c03d3d0 0x7f451c03f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:36.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.595+0000 7f453d400640 1 --2- 172.21.15.42:0/1540825461 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f451c03d3d0 0x7f451c03f8e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4528035000 tx=0x7f4528004730 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:36.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.599+0000 7f4536000640 1 -- 172.21.15.42:0/1540825461 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f453000a140 con 0x7f454006cea0 2024-07-04T17:31:36.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.851+0000 7f4544c00640 1 -- 172.21.15.42:0/1540825461 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4540073b40 con 0x7f454006cea0 2024-07-04T17:31:36.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.853+0000 7f4536000640 1 -- 172.21.15.42:0/1540825461 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+4736 (secure 0 0 0) 0x7f453001ac60 con 0x7f454006cea0 2024-07-04T17:31:36.854 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:31:36.854 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":64,"btime":"2024-07-04T17:31:28:880119+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5017,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/2312272332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2312272332},{"type":"v1","addr":"172.21.15.42:6827","nonce":2312272332}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":5020,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1285161495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1285161495},{"type":"v1","addr":"172.21.15.42:6829","nonce":1285161495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5023,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/1583070649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1583070649},{"type":"v1","addr":"172.21.15.42:6831","nonce":1583070649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5184,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/1315018856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":1315018856},{"type":"v1","addr":"172.21.15.42:6825","nonce":1315018856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":5226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6833/3474987521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":3474987521},{"type":"v1","addr":"172.21.15.42:6833","nonce":3474987521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[]} 2024-07-04T17:31:36.855 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 64 2024-07-04T17:31:36.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.856+0000 7f4544c00640 1 -- 172.21.15.42:0/1540825461 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f451c03d3d0 msgr2=0x7f451c03f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:36.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.856+0000 7f4544c00640 1 --2- 172.21.15.42:0/1540825461 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f451c03d3d0 0x7f451c03f8e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4528035000 tx=0x7f4528004730 comp rx=0 tx=0).stop 2024-07-04T17:31:36.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.856+0000 7f4544c00640 1 -- 172.21.15.42:0/1540825461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f454006cea0 msgr2=0x7f45400ad180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:36.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.856+0000 7f4544c00640 1 --2- 172.21.15.42:0/1540825461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f454006cea0 0x7f45400ad180 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f453000c320 tx=0x7f453000c7e0 comp rx=0 tx=0).stop 2024-07-04T17:31:36.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.856+0000 7f4544c00640 1 -- 172.21.15.42:0/1540825461 shutdown_connections 2024-07-04T17:31:36.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.856+0000 7f4544c00640 1 --2- 172.21.15.42:0/1540825461 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f451c03d3d0 0x7f451c03f8e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:36.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.856+0000 7f4544c00640 1 --2- 172.21.15.42:0/1540825461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f454006cea0 0x7f45400ad180 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:36.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.856+0000 7f4544c00640 1 -- 172.21.15.42:0/1540825461 >> 172.21.15.42:0/1540825461 conn(0x7f45400bf700 msgr2=0x7f45400aa3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:36.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.856+0000 7f4544c00640 1 -- 172.21.15.42:0/1540825461 shutdown_connections 2024-07-04T17:31:36.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:36.856+0000 7f4544c00640 1 -- 172.21.15.42:0/1540825461 wait complete. 2024-07-04T17:31:36.883 INFO:tasks.ceph.mds.a:Restarting daemon 2024-07-04T17:31:36.883 INFO:tasks.ceph.mds.a:Stopping old one... 2024-07-04T17:31:36.883 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-07-04T17:31:36.883 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:31:36.883 INFO:tasks.ceph.mds.b:Restarting daemon 2024-07-04T17:31:36.883 INFO:tasks.ceph.mds.b:Stopping old one... 2024-07-04T17:31:36.884 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-07-04T17:31:36.884 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:31:36.884 INFO:tasks.ceph.mds.c:Restarting daemon 2024-07-04T17:31:36.884 INFO:tasks.ceph.mds.c:Stopping old one... 2024-07-04T17:31:36.884 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-07-04T17:31:36.884 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:31:36.884 INFO:tasks.ceph.mds.d:Restarting daemon 2024-07-04T17:31:36.885 INFO:tasks.ceph.mds.d:Stopping old one... 2024-07-04T17:31:36.885 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-07-04T17:31:36.885 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:31:36.885 INFO:tasks.ceph.mds.e:Restarting daemon 2024-07-04T17:31:36.885 INFO:tasks.ceph.mds.e:Stopping old one... 2024-07-04T17:31:36.885 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-07-04T17:31:36.885 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:31:36.926 INFO:tasks.ceph.mds.b:Stopped 2024-07-04T17:31:36.927 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-07-04T17:31:36.933 INFO:tasks.ceph.mds.b:Started 2024-07-04T17:31:36.934 DEBUG:teuthology.parallel:result is None 2024-07-04T17:31:36.934 INFO:tasks.ceph.mds.d:Stopped 2024-07-04T17:31:36.935 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-07-04T17:31:36.935 INFO:tasks.ceph.mds.a:Stopped 2024-07-04T17:31:36.935 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-07-04T17:31:36.942 INFO:tasks.ceph.mds.c:Stopped 2024-07-04T17:31:36.942 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-07-04T17:31:36.945 INFO:tasks.ceph.mds.e:Stopped 2024-07-04T17:31:36.945 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-07-04T17:31:36.949 INFO:tasks.ceph.mds.d:Started 2024-07-04T17:31:36.950 INFO:tasks.ceph.mds.a:Started 2024-07-04T17:31:36.950 DEBUG:teuthology.parallel:result is None 2024-07-04T17:31:36.950 DEBUG:teuthology.parallel:result is None 2024-07-04T17:31:36.953 INFO:tasks.ceph.mds.c:Started 2024-07-04T17:31:36.953 DEBUG:teuthology.parallel:result is None 2024-07-04T17:31:36.957 INFO:tasks.ceph.mds.e:Started 2024-07-04T17:31:36.957 DEBUG:teuthology.parallel:result is None 2024-07-04T17:31:36.957 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-07-04T17:31:37.226 INFO:tasks.ceph.mds.e.smithi042.stdout:starting mds.e at 2024-07-04T17:31:37.243 INFO:tasks.ceph.mds.b.smithi042.stdout:starting mds.b at 2024-07-04T17:31:37.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:37.259+0000 7ff40aa00640 1 -- 172.21.15.42:0/1787601029 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3f80055e0 con 0x7ff40c0b37d0 2024-07-04T17:31:37.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:37.260+0000 7ff412c00640 1 -- 172.21.15.42:0/1787601029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff40c0b37d0 msgr2=0x7ff40c0b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:37.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:37.260+0000 7ff412c00640 1 --2- 172.21.15.42:0/1787601029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff40c0b37d0 0x7ff40c0b01f0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8005fd0 tx=0x7ff3f802a5f0 comp rx=0 tx=0).stop 2024-07-04T17:31:37.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:37.260+0000 7ff412c00640 1 -- 172.21.15.42:0/1787601029 shutdown_connections 2024-07-04T17:31:37.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:37.260+0000 7ff412c00640 1 --2- 172.21.15.42:0/1787601029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff40c0b37d0 0x7ff40c0b01f0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:37.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:37.260+0000 7ff412c00640 1 -- 172.21.15.42:0/1787601029 >> 172.21.15.42:0/1787601029 conn(0x7ff40c0bf700 msgr2=0x7ff40c0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:37.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:37.260+0000 7ff412c00640 1 -- 172.21.15.42:0/1787601029 shutdown_connections 2024-07-04T17:31:37.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:37.260+0000 7ff412c00640 1 -- 172.21.15.42:0/1787601029 wait complete. 2024-07-04T17:31:37.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:37.262+0000 7ff412c00640 1 Processor -- start 2024-07-04T17:31:37.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:37.262+0000 7ff412c00640 1 -- start start 2024-07-04T17:31:37.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:37.264+0000 7ff412c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff40c0b2d20 0x7ff40c072090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:37.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:37.264+0000 7ff412c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff40c06ce30 con 0x7ff40c0b2d20 2024-07-04T17:31:37.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:37.264+0000 7ff40be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff40c0b2d20 0x7ff40c072090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:37.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:37.264+0000 7ff40be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff40c0b2d20 0x7ff40c072090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44520/0 (socket says 172.21.15.42:44520) 2024-07-04T17:31:37.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:37.264+0000 7ff40be00640 1 -- 172.21.15.42:0/4166302980 learned_addr learned my addr 172.21.15.42:0/4166302980 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:37.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:37.265+0000 7ff40be00640 1 -- 172.21.15.42:0/4166302980 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff40c071bd0 con 0x7ff40c0b2d20 2024-07-04T17:31:37.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:37.265+0000 7ff40be00640 1 --2- 172.21.15.42:0/4166302980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff40c0b2d20 0x7ff40c072090 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc00c2f0 tx=0x7ff3fc00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:37.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:37.266+0000 7ff403e00640 1 -- 172.21.15.42:0/4166302980 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3fc017070 con 0x7ff40c0b2d20 2024-07-04T17:31:37.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:37.266+0000 7ff403e00640 1 -- 172.21.15.42:0/4166302980 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3fc0052f0 con 0x7ff40c0b2d20 2024-07-04T17:31:37.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:37.266+0000 7ff403e00640 1 -- 172.21.15.42:0/4166302980 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3fc0035f0 con 0x7ff40c0b2d20 2024-07-04T17:31:37.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:37.266+0000 7ff412c00640 1 -- 172.21.15.42:0/4166302980 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff40c14e6b0 con 0x7ff40c0b2d20 2024-07-04T17:31:37.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:37.266+0000 7ff412c00640 1 -- 172.21.15.42:0/4166302980 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff40c1e1650 con 0x7ff40c0b2d20 2024-07-04T17:31:37.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:37.267+0000 7ff412c00640 1 -- 172.21.15.42:0/4166302980 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff40c06d490 con 0x7ff40c0b2d20 2024-07-04T17:31:37.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:37.268+0000 7ff403e00640 1 -- 172.21.15.42:0/4166302980 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7ff3fc01a070 con 0x7ff40c0b2d20 2024-07-04T17:31:37.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:37.268+0000 7ff403e00640 1 --2- 172.21.15.42:0/4166302980 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7ff3ec03d420 0x7ff3ec03f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:37.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:37.268+0000 7ff403e00640 1 -- 172.21.15.42:0/4166302980 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 3189+0+0 (secure 0 0 0) 0x7ff3fc050900 con 0x7ff40c0b2d20 2024-07-04T17:31:37.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:37.269+0000 7ff40b400640 1 --2- 172.21.15.42:0/4166302980 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7ff3ec03d420 0x7ff3ec03f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:37.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:37.269+0000 7ff40b400640 1 --2- 172.21.15.42:0/4166302980 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7ff3ec03d420 0x7ff3ec03f930 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8033000 tx=0x7ff3f8004730 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:37.271 INFO:tasks.ceph.mds.c.smithi042.stdout:starting mds.c at 2024-07-04T17:31:37.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:37.274+0000 7ff403e00640 1 -- 172.21.15.42:0/4166302980 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7ff3fc01bdf0 con 0x7ff40c0b2d20 2024-07-04T17:31:37.278 INFO:tasks.ceph.mds.d.smithi042.stdout:starting mds.d at 2024-07-04T17:31:37.283 INFO:tasks.ceph.mds.a.smithi042.stdout:starting mds.a at 2024-07-04T17:31:37.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:37.495+0000 7ff412c00640 1 -- 172.21.15.42:0/4166302980 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ff40c071da0 con 0x7ff40c0b2d20 2024-07-04T17:31:38.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.031+0000 7ff403e00640 1 -- 172.21.15.42:0/4166302980 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v85) ==== 97+0+0 (secure 0 0 0) 0x7ff3fc011910 con 0x7ff40c0b2d20 2024-07-04T17:31:38.033 INFO:teuthology.orchestra.run.smithi042.stderr: removed all blocklist entries 2024-07-04T17:31:38.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.035+0000 7ff412c00640 1 -- 172.21.15.42:0/4166302980 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7ff3ec03d420 msgr2=0x7ff3ec03f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:38.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.035+0000 7ff412c00640 1 --2- 172.21.15.42:0/4166302980 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7ff3ec03d420 0x7ff3ec03f930 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8033000 tx=0x7ff3f8004730 comp rx=0 tx=0).stop 2024-07-04T17:31:38.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.035+0000 7ff412c00640 1 -- 172.21.15.42:0/4166302980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff40c0b2d20 msgr2=0x7ff40c072090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:38.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.035+0000 7ff412c00640 1 --2- 172.21.15.42:0/4166302980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff40c0b2d20 0x7ff40c072090 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc00c2f0 tx=0x7ff3fc00c7b0 comp rx=0 tx=0).stop 2024-07-04T17:31:38.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.036+0000 7ff412c00640 1 -- 172.21.15.42:0/4166302980 shutdown_connections 2024-07-04T17:31:38.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.036+0000 7ff412c00640 1 --2- 172.21.15.42:0/4166302980 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7ff3ec03d420 0x7ff3ec03f930 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:38.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.036+0000 7ff412c00640 1 --2- 172.21.15.42:0/4166302980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff40c0b2d20 0x7ff40c072090 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:38.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.036+0000 7ff412c00640 1 -- 172.21.15.42:0/4166302980 >> 172.21.15.42:0/4166302980 conn(0x7ff40c0bf700 msgr2=0x7ff40c141760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:38.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.036+0000 7ff412c00640 1 -- 172.21.15.42:0/4166302980 shutdown_connections 2024-07-04T17:31:38.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.036+0000 7ff412c00640 1 -- 172.21.15.42:0/4166302980 wait complete. 2024-07-04T17:31:38.077 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-07-04T17:31:38.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.355+0000 7f7be7200640 1 -- 172.21.15.42:0/2726657562 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7be00bc040 msgr2=0x7f7be00bd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:38.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.355+0000 7f7be7200640 1 --2- 172.21.15.42:0/2726657562 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7be00bc040 0x7f7be00bd2d0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0005fd0 tx=0x7f7bd002a5f0 comp rx=0 tx=0).stop 2024-07-04T17:31:38.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.355+0000 7f7be7200640 1 -- 172.21.15.42:0/2726657562 shutdown_connections 2024-07-04T17:31:38.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.355+0000 7f7be7200640 1 --2- 172.21.15.42:0/2726657562 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7be00bc040 0x7f7be00bd2d0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:38.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.355+0000 7f7be7200640 1 -- 172.21.15.42:0/2726657562 >> 172.21.15.42:0/2726657562 conn(0x7f7be006c940 msgr2=0x7f7be006ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:38.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.355+0000 7f7be7200640 1 -- 172.21.15.42:0/2726657562 shutdown_connections 2024-07-04T17:31:38.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.355+0000 7f7be7200640 1 -- 172.21.15.42:0/2726657562 wait complete. 2024-07-04T17:31:38.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.357+0000 7f7be7200640 1 Processor -- start 2024-07-04T17:31:38.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.357+0000 7f7be7200640 1 -- start start 2024-07-04T17:31:38.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.358+0000 7f7be7200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7be00bc040 0x7f7be01d92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:38.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.358+0000 7f7be7200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7be00b7530 con 0x7f7be00bc040 2024-07-04T17:31:38.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.358+0000 7f7bdfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7be00bc040 0x7f7be01d92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:38.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.358+0000 7f7bdfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7be00bc040 0x7f7be01d92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44592/0 (socket says 172.21.15.42:44592) 2024-07-04T17:31:38.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.359+0000 7f7bdfe00640 1 -- 172.21.15.42:0/3035688514 learned_addr learned my addr 172.21.15.42:0/3035688514 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:38.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.359+0000 7f7bdfe00640 1 -- 172.21.15.42:0/3035688514 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be01e3720 con 0x7f7be00bc040 2024-07-04T17:31:38.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.359+0000 7f7bdfe00640 1 --2- 172.21.15.42:0/3035688514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7be00bc040 0x7f7be01d92b0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0009850 tx=0x7f7bd00049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:38.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.359+0000 7f7bd7e00640 1 -- 172.21.15.42:0/3035688514 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bd003e030 con 0x7f7be00bc040 2024-07-04T17:31:38.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.359+0000 7f7bd7e00640 1 -- 172.21.15.42:0/3035688514 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7bd00035f0 con 0x7f7be00bc040 2024-07-04T17:31:38.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.359+0000 7f7be7200640 1 -- 172.21.15.42:0/3035688514 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7be01e39a0 con 0x7f7be00bc040 2024-07-04T17:31:38.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.359+0000 7f7bd7e00640 1 -- 172.21.15.42:0/3035688514 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bd0035040 con 0x7f7be00bc040 2024-07-04T17:31:38.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.360+0000 7f7be7200640 1 -- 172.21.15.42:0/3035688514 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7be01d9a60 con 0x7f7be00bc040 2024-07-04T17:31:38.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.361+0000 7f7be7200640 1 -- 172.21.15.42:0/3035688514 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7be00b7c00 con 0x7f7be00bc040 2024-07-04T17:31:38.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.361+0000 7f7bd7e00640 1 -- 172.21.15.42:0/3035688514 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7f7bd0041030 con 0x7f7be00bc040 2024-07-04T17:31:38.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.361+0000 7f7bd7e00640 1 --2- 172.21.15.42:0/3035688514 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f7bc003d3d0 0x7f7bc003f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:38.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.361+0000 7f7bd7e00640 1 -- 172.21.15.42:0/3035688514 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 2587+0+0 (secure 0 0 0) 0x7f7bd003f070 con 0x7f7be00bc040 2024-07-04T17:31:38.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.362+0000 7f7bdf400640 1 --2- 172.21.15.42:0/3035688514 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f7bc003d3d0 0x7f7bc003f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:38.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.362+0000 7f7bdf400640 1 --2- 172.21.15.42:0/3035688514 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f7bc003d3d0 0x7f7bc003f8e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc001c60 tx=0x7f7bcc007d70 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:38.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.366+0000 7f7bd7e00640 1 -- 172.21.15.42:0/3035688514 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f7bd004bd20 con 0x7f7be00bc040 2024-07-04T17:31:38.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.642+0000 7f7be7200640 1 -- 172.21.15.42:0/3035688514 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f7be01d97f0 con 0x7f7be00bc040 2024-07-04T17:31:38.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.643+0000 7f7bd7e00640 1 -- 172.21.15.42:0/3035688514 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+4064 (secure 0 0 0) 0x7f7bd0030c90 con 0x7f7be00bc040 2024-07-04T17:31:38.645 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:31:38.645 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:31:38.645 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_dump": [ 2024-07-04T17:31:38.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:31:38.645 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.a", 2024-07-04T17:31:38.645 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAM24Zm2VMEFxAAQqdcGux6Ker7jEr1dGAfzA==", 2024-07-04T17:31:38.645 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:31:38.645 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:31:38.645 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:31:38.645 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:31:38.646 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:31:38.646 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:31:38.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:31:38.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:31:38.646 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.b", 2024-07-04T17:31:38.646 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAM24ZmS/S7KxAAfjTZrLYR9qLxuTvd2zsCcQ==", 2024-07-04T17:31:38.646 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:31:38.646 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:31:38.646 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:31:38.646 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:31:38.646 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:31:38.646 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:31:38.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:31:38.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:31:38.647 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.c", 2024-07-04T17:31:38.647 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAN24Zm2JrRBBAAFMYIM2fLsr9EOfcyh3tZpA==", 2024-07-04T17:31:38.647 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:31:38.647 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:31:38.647 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:31:38.647 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:31:38.647 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:31:38.647 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:31:38.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:31:38.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:31:38.647 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.d", 2024-07-04T17:31:38.647 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAN24ZmPCGLFxAAhsh3XaAHCyauCGxh/3DFsw==", 2024-07-04T17:31:38.648 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:31:38.648 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:31:38.648 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:31:38.648 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:31:38.648 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:31:38.648 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:31:38.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:31:38.648 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:31:38.648 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.e", 2024-07-04T17:31:38.648 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAN24ZmVIaSKhAAhoBdDalwQ4eoo0rqb6p0Vw==", 2024-07-04T17:31:38.648 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:31:38.648 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:31:38.648 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:31:38.648 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:31:38.649 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:31:38.649 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:31:38.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:31:38.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:31:38.649 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.0", 2024-07-04T17:31:38.649 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAQ24Zm4spcCBAA4kTvvR7IOVa01K5ZCrJ35w==", 2024-07-04T17:31:38.649 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:31:38.649 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-07-04T17:31:38.649 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-07-04T17:31:38.649 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:31:38.649 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:31:38.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:31:38.649 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:31:38.649 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.1", 2024-07-04T17:31:38.650 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAT24Zmz1sfGRAAgfH5eIiQrufnPq6F94rHWg==", 2024-07-04T17:31:38.650 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:31:38.650 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-07-04T17:31:38.650 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-07-04T17:31:38.650 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:31:38.650 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:31:38.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:31:38.650 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:31:38.650 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.2", 2024-07-04T17:31:38.650 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAW24ZmJdQHORAAEjAM9OBqSIRhX148GC6pcw==", 2024-07-04T17:31:38.650 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:31:38.650 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-07-04T17:31:38.650 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-07-04T17:31:38.650 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:31:38.651 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:31:38.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:31:38.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:31:38.651 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.0", 2024-07-04T17:31:38.651 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAO24ZmUMmYABAAh4pNlkb9JsHQtK8uCJFQCA==", 2024-07-04T17:31:38.651 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:31:38.651 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:31:38.651 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow r", 2024-07-04T17:31:38.651 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-07-04T17:31:38.651 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:31:38.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:31:38.651 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:31:38.651 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.1", 2024-07-04T17:31:38.651 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAO24Zm0pAJEBAAntDeQ/oWDlJ+3xC7q5EQnQ==", 2024-07-04T17:31:38.652 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:31:38.652 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:31:38.652 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-07-04T17:31:38.652 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow rw", 2024-07-04T17:31:38.652 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rwx" 2024-07-04T17:31:38.652 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:31:38.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:31:38.652 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:31:38.652 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.admin", 2024-07-04T17:31:38.652 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAL24ZmMe3iHhAAFciH9cbUAIHLSAJvXy30EQ==", 2024-07-04T17:31:38.652 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:31:38.652 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-07-04T17:31:38.652 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:31:38.652 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:31:38.653 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:31:38.653 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:31:38.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:31:38.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:31:38.653 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.mirror", 2024-07-04T17:31:38.653 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAO24ZmBJzBHhAAR4Zg36YTtVizxm/g/Cdk0w==", 2024-07-04T17:31:38.653 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:31:38.653 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow r", 2024-07-04T17:31:38.653 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-07-04T17:31:38.653 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "profile cephfs-mirror", 2024-07-04T17:31:38.653 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rw tag cephfs metadata=*, allow r tag cephfs data=*" 2024-07-04T17:31:38.653 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:31:38.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:31:38.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:31:38.653 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.mirror_remote", 2024-07-04T17:31:38.654 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAO24Zm+BsqLhAA19GVuEjfWQeMe/fSc+dmoA==", 2024-07-04T17:31:38.654 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:31:38.654 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow rwps", 2024-07-04T17:31:38.654 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-07-04T17:31:38.654 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow r", 2024-07-04T17:31:38.654 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rw tag cephfs *=*" 2024-07-04T17:31:38.654 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:31:38.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:31:38.654 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:31:38.654 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mgr.x", 2024-07-04T17:31:38.654 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAM24Zm47tDCBAAjszudEEB4QHLlWqPXyZc2g==", 2024-07-04T17:31:38.655 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:31:38.655 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-07-04T17:31:38.655 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile mgr", 2024-07-04T17:31:38.655 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:31:38.655 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:31:38.655 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:31:38.655 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-07-04T17:31:38.655 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:31:38.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.646+0000 7f7be7200640 1 -- 172.21.15.42:0/3035688514 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f7bc003d3d0 msgr2=0x7f7bc003f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:38.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.646+0000 7f7be7200640 1 --2- 172.21.15.42:0/3035688514 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f7bc003d3d0 0x7f7bc003f8e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc001c60 tx=0x7f7bcc007d70 comp rx=0 tx=0).stop 2024-07-04T17:31:38.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.646+0000 7f7be7200640 1 -- 172.21.15.42:0/3035688514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7be00bc040 msgr2=0x7f7be01d92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:38.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.646+0000 7f7be7200640 1 --2- 172.21.15.42:0/3035688514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7be00bc040 0x7f7be01d92b0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0009850 tx=0x7f7bd00049d0 comp rx=0 tx=0).stop 2024-07-04T17:31:38.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.646+0000 7f7be7200640 1 -- 172.21.15.42:0/3035688514 shutdown_connections 2024-07-04T17:31:38.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.646+0000 7f7be7200640 1 --2- 172.21.15.42:0/3035688514 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f7bc003d3d0 0x7f7bc003f8e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:38.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.646+0000 7f7be7200640 1 --2- 172.21.15.42:0/3035688514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7be00bc040 0x7f7be01d92b0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:38.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.646+0000 7f7be7200640 1 -- 172.21.15.42:0/3035688514 >> 172.21.15.42:0/3035688514 conn(0x7f7be006c940 msgr2=0x7f7be00b65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:38.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.646+0000 7f7be7200640 1 -- 172.21.15.42:0/3035688514 shutdown_connections 2024-07-04T17:31:38.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.646+0000 7f7be7200640 1 -- 172.21.15.42:0/3035688514 wait complete. 2024-07-04T17:31:38.671 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-04T17:31:38.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.949+0000 7f507d000640 1 -- 172.21.15.42:0/1386341686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f507806cea0 msgr2=0x7f50780b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:38.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.949+0000 7f507d000640 1 --2- 172.21.15.42:0/1386341686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f507806cea0 0x7f50780b3030 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f5064005fd0 tx=0x7f506402a5f0 comp rx=0 tx=0).stop 2024-07-04T17:31:38.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.949+0000 7f507d000640 1 -- 172.21.15.42:0/1386341686 shutdown_connections 2024-07-04T17:31:38.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.949+0000 7f507d000640 1 --2- 172.21.15.42:0/1386341686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f507806cea0 0x7f50780b3030 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:38.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.949+0000 7f507d000640 1 -- 172.21.15.42:0/1386341686 >> 172.21.15.42:0/1386341686 conn(0x7f50780bf700 msgr2=0x7f50780af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:38.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.949+0000 7f507d000640 1 -- 172.21.15.42:0/1386341686 shutdown_connections 2024-07-04T17:31:38.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.949+0000 7f507d000640 1 -- 172.21.15.42:0/1386341686 wait complete. 2024-07-04T17:31:38.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.952+0000 7f507d000640 1 Processor -- start 2024-07-04T17:31:38.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.952+0000 7f507d000640 1 -- start start 2024-07-04T17:31:38.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.954+0000 7f507d000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f507806edd0 0x7f50780acc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:38.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.954+0000 7f507d000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f50780b0580 con 0x7f507806edd0 2024-07-04T17:31:38.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.954+0000 7f5075e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f507806edd0 0x7f50780acc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:38.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.954+0000 7f5075e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f507806edd0 0x7f50780acc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44612/0 (socket says 172.21.15.42:44612) 2024-07-04T17:31:38.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.954+0000 7f5075e00640 1 -- 172.21.15.42:0/1693942952 learned_addr learned my addr 172.21.15.42:0/1693942952 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:38.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.954+0000 7f5075e00640 1 -- 172.21.15.42:0/1693942952 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50781e3750 con 0x7f507806edd0 2024-07-04T17:31:38.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.955+0000 7f5075e00640 1 --2- 172.21.15.42:0/1693942952 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f507806edd0 0x7f50780acc00 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f506800c380 tx=0x7f506800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:38.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.955+0000 7f506e000640 1 -- 172.21.15.42:0/1693942952 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5068017070 con 0x7f507806edd0 2024-07-04T17:31:38.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.955+0000 7f506e000640 1 -- 172.21.15.42:0/1693942952 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50680052f0 con 0x7f507806edd0 2024-07-04T17:31:38.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.955+0000 7f506e000640 1 -- 172.21.15.42:0/1693942952 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50680035f0 con 0x7f507806edd0 2024-07-04T17:31:38.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.955+0000 7f507d000640 1 -- 172.21.15.42:0/1693942952 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50780ae090 con 0x7f507806edd0 2024-07-04T17:31:38.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.955+0000 7f507d000640 1 -- 172.21.15.42:0/1693942952 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50780add80 con 0x7f507806edd0 2024-07-04T17:31:38.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.956+0000 7f507d000640 1 -- 172.21.15.42:0/1693942952 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50780b3030 con 0x7f507806edd0 2024-07-04T17:31:38.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.956+0000 7f506e000640 1 -- 172.21.15.42:0/1693942952 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7f506801a070 con 0x7f507806edd0 2024-07-04T17:31:38.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.957+0000 7f506e000640 1 --2- 172.21.15.42:0/1693942952 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f505803d3d0 0x7f505803f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:38.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.957+0000 7f506e000640 1 -- 172.21.15.42:0/1693942952 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 2587+0+0 (secure 0 0 0) 0x7f5068050680 con 0x7f507806edd0 2024-07-04T17:31:38.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.957+0000 7f5075400640 1 --2- 172.21.15.42:0/1693942952 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f505803d3d0 0x7f505803f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:38.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.958+0000 7f5075400640 1 --2- 172.21.15.42:0/1693942952 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f505803d3d0 0x7f505803f8e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5064033000 tx=0x7f5064004730 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:38.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:38.962+0000 7f506e000640 1 -- 172.21.15.42:0/1693942952 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f50781e3750 con 0x7f507806edd0 2024-07-04T17:31:39.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.178+0000 7f507d000640 1 -- 172.21.15.42:0/1693942952 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f50780af0c0 con 0x7f507806edd0 2024-07-04T17:31:39.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.179+0000 7f506e000640 1 -- 172.21.15.42:0/1693942952 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v88) ==== 84+0+65 (secure 0 0 0) 0x7f50680192c0 con 0x7f507806edd0 2024-07-04T17:31:39.180 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:31:39.180 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-07-04T17:31:39.180 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:31:39.181 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 1, 2024-07-04T17:31:39.181 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": ".mgr" 2024-07-04T17:31:39.181 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:31:39.181 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-07-04T17:31:39.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.181+0000 7f507d000640 1 -- 172.21.15.42:0/1693942952 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f505803d3d0 msgr2=0x7f505803f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:39.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.181+0000 7f507d000640 1 --2- 172.21.15.42:0/1693942952 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f505803d3d0 0x7f505803f8e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5064033000 tx=0x7f5064004730 comp rx=0 tx=0).stop 2024-07-04T17:31:39.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.181+0000 7f507d000640 1 -- 172.21.15.42:0/1693942952 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f507806edd0 msgr2=0x7f50780acc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:39.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.181+0000 7f507d000640 1 --2- 172.21.15.42:0/1693942952 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f507806edd0 0x7f50780acc00 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f506800c380 tx=0x7f506800c840 comp rx=0 tx=0).stop 2024-07-04T17:31:39.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.181+0000 7f507d000640 1 -- 172.21.15.42:0/1693942952 shutdown_connections 2024-07-04T17:31:39.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.181+0000 7f507d000640 1 --2- 172.21.15.42:0/1693942952 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f505803d3d0 0x7f505803f8e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:39.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.181+0000 7f507d000640 1 --2- 172.21.15.42:0/1693942952 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f507806edd0 0x7f50780acc00 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:39.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.181+0000 7f507d000640 1 -- 172.21.15.42:0/1693942952 >> 172.21.15.42:0/1693942952 conn(0x7f50780bf700 msgr2=0x7f5078075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:39.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.182+0000 7f507d000640 1 -- 172.21.15.42:0/1693942952 shutdown_connections 2024-07-04T17:31:39.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.182+0000 7f507d000640 1 -- 172.21.15.42:0/1693942952 wait complete. 2024-07-04T17:31:39.207 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-04T17:31:39.207 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-04T17:31:39.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.491+0000 7fd17d000640 1 -- 172.21.15.42:0/406920677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1780ac9d0 msgr2=0x7fd1780aef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:39.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.491+0000 7fd17d000640 1 --2- 172.21.15.42:0/406920677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1780ac9d0 0x7fd1780aef80 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fd1640097b0 tx=0x7fd16402a4a0 comp rx=0 tx=0).stop 2024-07-04T17:31:39.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.492+0000 7fd17d000640 1 -- 172.21.15.42:0/406920677 shutdown_connections 2024-07-04T17:31:39.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.492+0000 7fd17d000640 1 --2- 172.21.15.42:0/406920677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1780ac9d0 0x7fd1780aef80 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:39.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.492+0000 7fd17d000640 1 -- 172.21.15.42:0/406920677 >> 172.21.15.42:0/406920677 conn(0x7fd1780afaa0 msgr2=0x7fd1780afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:39.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.492+0000 7fd17d000640 1 -- 172.21.15.42:0/406920677 shutdown_connections 2024-07-04T17:31:39.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.492+0000 7fd17d000640 1 -- 172.21.15.42:0/406920677 wait complete. 2024-07-04T17:31:39.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.494+0000 7fd17d000640 1 Processor -- start 2024-07-04T17:31:39.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.494+0000 7fd17d000640 1 -- start start 2024-07-04T17:31:39.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.495+0000 7fd17d000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1780ac9d0 0x7fd178143100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:39.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.495+0000 7fd17d000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd1781415a0 con 0x7fd1780ac9d0 2024-07-04T17:31:39.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.495+0000 7fd177400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1780ac9d0 0x7fd178143100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:39.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.495+0000 7fd177400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1780ac9d0 0x7fd178143100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44626/0 (socket says 172.21.15.42:44626) 2024-07-04T17:31:39.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.495+0000 7fd177400640 1 -- 172.21.15.42:0/1543555661 learned_addr learned my addr 172.21.15.42:0/1543555661 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:39.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.496+0000 7fd177400640 1 -- 172.21.15.42:0/1543555661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1780ab420 con 0x7fd1780ac9d0 2024-07-04T17:31:39.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.496+0000 7fd177400640 1 --2- 172.21.15.42:0/1543555661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1780ac9d0 0x7fd178143100 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fd168001eb0 tx=0x7fd16800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:39.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.496+0000 7fd16f400640 1 -- 172.21.15.42:0/1543555661 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1680116c0 con 0x7fd1780ac9d0 2024-07-04T17:31:39.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.497+0000 7fd17d000640 1 -- 172.21.15.42:0/1543555661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1780b5bc0 con 0x7fd1780ac9d0 2024-07-04T17:31:39.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.497+0000 7fd16f400640 1 -- 172.21.15.42:0/1543555661 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd168017070 con 0x7fd1780ac9d0 2024-07-04T17:31:39.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.497+0000 7fd16f400640 1 -- 172.21.15.42:0/1543555661 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd168011ae0 con 0x7fd1780ac9d0 2024-07-04T17:31:39.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.497+0000 7fd17d000640 1 -- 172.21.15.42:0/1543555661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd178144ca0 con 0x7fd1780ac9d0 2024-07-04T17:31:39.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.498+0000 7fd17d000640 1 -- 172.21.15.42:0/1543555661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd17813e960 con 0x7fd1780ac9d0 2024-07-04T17:31:39.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.498+0000 7fd16f400640 1 -- 172.21.15.42:0/1543555661 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7fd16801c460 con 0x7fd1780ac9d0 2024-07-04T17:31:39.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.498+0000 7fd16f400640 1 --2- 172.21.15.42:0/1543555661 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fd15803d380 0x7fd15803f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:39.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.499+0000 7fd16f400640 1 -- 172.21.15.42:0/1543555661 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 2587+0+0 (secure 0 0 0) 0x7fd168051160 con 0x7fd1780ac9d0 2024-07-04T17:31:39.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.499+0000 7fd176a00640 1 --2- 172.21.15.42:0/1543555661 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fd15803d380 0x7fd15803f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:39.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.499+0000 7fd176a00640 1 --2- 172.21.15.42:0/1543555661 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fd15803d380 0x7fd15803f890 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd164034000 tx=0x7fd164004730 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:39.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.504+0000 7fd16f400640 1 -- 172.21.15.42:0/1543555661 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fd178144ca0 con 0x7fd1780ac9d0 2024-07-04T17:31:39.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:39.705+0000 7fd17d000640 1 -- 172.21.15.42:0/1543555661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd178141c50 con 0x7fd1780ac9d0 2024-07-04T17:31:40.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.061+0000 7fd16f400640 1 -- 172.21.15.42:0/1543555661 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v89) ==== 138+0+0 (secure 0 0 0) 0x7fd1780b1280 con 0x7fd1780ac9d0 2024-07-04T17:31:40.063 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' created 2024-07-04T17:31:40.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.064+0000 7fd17d000640 1 -- 172.21.15.42:0/1543555661 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fd15803d380 msgr2=0x7fd15803f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:40.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.064+0000 7fd17d000640 1 --2- 172.21.15.42:0/1543555661 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fd15803d380 0x7fd15803f890 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd164034000 tx=0x7fd164004730 comp rx=0 tx=0).stop 2024-07-04T17:31:40.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.064+0000 7fd17d000640 1 -- 172.21.15.42:0/1543555661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1780ac9d0 msgr2=0x7fd178143100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:40.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.064+0000 7fd17d000640 1 --2- 172.21.15.42:0/1543555661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1780ac9d0 0x7fd178143100 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fd168001eb0 tx=0x7fd16800c810 comp rx=0 tx=0).stop 2024-07-04T17:31:40.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.065+0000 7fd17d000640 1 -- 172.21.15.42:0/1543555661 shutdown_connections 2024-07-04T17:31:40.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.065+0000 7fd17d000640 1 --2- 172.21.15.42:0/1543555661 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fd15803d380 0x7fd15803f890 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:40.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.065+0000 7fd17d000640 1 --2- 172.21.15.42:0/1543555661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1780ac9d0 0x7fd178143100 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:40.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.065+0000 7fd17d000640 1 -- 172.21.15.42:0/1543555661 >> 172.21.15.42:0/1543555661 conn(0x7fd1780afaa0 msgr2=0x7fd17813f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:40.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.065+0000 7fd17d000640 1 -- 172.21.15.42:0/1543555661 shutdown_connections 2024-07-04T17:31:40.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.065+0000 7fd17d000640 1 -- 172.21.15.42:0/1543555661 wait complete. 2024-07-04T17:31:40.105 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-04T17:31:40.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.373+0000 7f878fe00640 1 -- 172.21.15.42:0/2234051965 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87900a9be0 msgr2=0x7f879013efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:40.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.373+0000 7f878fe00640 1 --2- 172.21.15.42:0/2234051965 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87900a9be0 0x7f879013efb0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f87780097b0 tx=0x7f877802a530 comp rx=0 tx=0).stop 2024-07-04T17:31:40.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.374+0000 7f878fe00640 1 -- 172.21.15.42:0/2234051965 shutdown_connections 2024-07-04T17:31:40.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.374+0000 7f878fe00640 1 --2- 172.21.15.42:0/2234051965 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87900a9be0 0x7f879013efb0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:40.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.374+0000 7f878fe00640 1 -- 172.21.15.42:0/2234051965 >> 172.21.15.42:0/2234051965 conn(0x7f879006c020 msgr2=0x7f879006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:40.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.374+0000 7f878fe00640 1 -- 172.21.15.42:0/2234051965 shutdown_connections 2024-07-04T17:31:40.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.374+0000 7f878fe00640 1 -- 172.21.15.42:0/2234051965 wait complete. 2024-07-04T17:31:40.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.376+0000 7f878fe00640 1 Processor -- start 2024-07-04T17:31:40.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.376+0000 7f878fe00640 1 -- start start 2024-07-04T17:31:40.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.377+0000 7f878fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87900a9be0 0x7f87901d7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:40.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.377+0000 7f878fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f87901402d0 con 0x7f87900a9be0 2024-07-04T17:31:40.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.377+0000 7f878ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87900a9be0 0x7f87901d7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:40.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.377+0000 7f878ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87900a9be0 0x7f87901d7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44632/0 (socket says 172.21.15.42:44632) 2024-07-04T17:31:40.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.377+0000 7f878ea00640 1 -- 172.21.15.42:0/602670166 learned_addr learned my addr 172.21.15.42:0/602670166 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:40.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.378+0000 7f878ea00640 1 -- 172.21.15.42:0/602670166 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87901d7710 con 0x7f87900a9be0 2024-07-04T17:31:40.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.378+0000 7f878ea00640 1 --2- 172.21.15.42:0/602670166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87900a9be0 0x7f87901d7bf0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f878000c380 tx=0x7f878000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:40.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.378+0000 7f8786a00640 1 -- 172.21.15.42:0/602670166 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87800116c0 con 0x7f87900a9be0 2024-07-04T17:31:40.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.378+0000 7f8786a00640 1 -- 172.21.15.42:0/602670166 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8780017070 con 0x7f87900a9be0 2024-07-04T17:31:40.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.378+0000 7f8786a00640 1 -- 172.21.15.42:0/602670166 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8780011b20 con 0x7f87900a9be0 2024-07-04T17:31:40.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.378+0000 7f878fe00640 1 -- 172.21.15.42:0/602670166 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87901d7410 con 0x7f87900a9be0 2024-07-04T17:31:40.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.378+0000 7f878fe00640 1 -- 172.21.15.42:0/602670166 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87901d7000 con 0x7f87900a9be0 2024-07-04T17:31:40.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.379+0000 7f878fe00640 1 -- 172.21.15.42:0/602670166 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87901409c0 con 0x7f87900a9be0 2024-07-04T17:31:40.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.380+0000 7f8786a00640 1 -- 172.21.15.42:0/602670166 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7f8780011cc0 con 0x7f87900a9be0 2024-07-04T17:31:40.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.381+0000 7f8786a00640 1 --2- 172.21.15.42:0/602670166 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f876c03d420 0x7f876c03f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:40.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.381+0000 7f8786a00640 1 -- 172.21.15.42:0/602670166 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 2957+0+0 (secure 0 0 0) 0x7f8780051280 con 0x7f87900a9be0 2024-07-04T17:31:40.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.381+0000 7f8787e00640 1 --2- 172.21.15.42:0/602670166 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f876c03d420 0x7f876c03f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:40.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.382+0000 7f8787e00640 1 --2- 172.21.15.42:0/602670166 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f876c03d420 0x7f876c03f930 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8778034000 tx=0x7f8778004730 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:40.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.386+0000 7f8786a00640 1 -- 172.21.15.42:0/602670166 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f8780010cf0 con 0x7f87900a9be0 2024-07-04T17:31:40.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:40.593+0000 7f878fe00640 1 -- 172.21.15.42:0/602670166 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f8790068e30 con 0x7f87900a9be0 2024-07-04T17:31:41.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.076+0000 7f8786a00640 1 -- 172.21.15.42:0/602670166 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v90) ==== 170+0+0 (secure 0 0 0) 0x7f8780019310 con 0x7f87900a9be0 2024-07-04T17:31:41.078 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' created 2024-07-04T17:31:41.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.081+0000 7f878fe00640 1 -- 172.21.15.42:0/602670166 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f876c03d420 msgr2=0x7f876c03f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:41.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.081+0000 7f878fe00640 1 --2- 172.21.15.42:0/602670166 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f876c03d420 0x7f876c03f930 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8778034000 tx=0x7f8778004730 comp rx=0 tx=0).stop 2024-07-04T17:31:41.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.081+0000 7f878fe00640 1 -- 172.21.15.42:0/602670166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87900a9be0 msgr2=0x7f87901d7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:41.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.081+0000 7f878fe00640 1 --2- 172.21.15.42:0/602670166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87900a9be0 0x7f87901d7bf0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f878000c380 tx=0x7f878000c840 comp rx=0 tx=0).stop 2024-07-04T17:31:41.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.081+0000 7f878fe00640 1 -- 172.21.15.42:0/602670166 shutdown_connections 2024-07-04T17:31:41.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.081+0000 7f878fe00640 1 --2- 172.21.15.42:0/602670166 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f876c03d420 0x7f876c03f930 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:41.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.081+0000 7f878fe00640 1 --2- 172.21.15.42:0/602670166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87900a9be0 0x7f87901d7bf0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:41.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.081+0000 7f878fe00640 1 -- 172.21.15.42:0/602670166 >> 172.21.15.42:0/602670166 conn(0x7f879006c020 msgr2=0x7f87900ae910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:41.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.082+0000 7f878fe00640 1 -- 172.21.15.42:0/602670166 shutdown_connections 2024-07-04T17:31:41.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.082+0000 7f878fe00640 1 -- 172.21.15.42:0/602670166 wait complete. 2024-07-04T17:31:41.136 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-04T17:31:41.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.397+0000 7f4fe1e00640 1 -- 172.21.15.42:0/2351988788 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fdc073470 msgr2=0x7f4fdc0a9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:41.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.397+0000 7f4fe1e00640 1 --2- 172.21.15.42:0/2351988788 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fdc073470 0x7f4fdc0a9990 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f4fc40097b0 tx=0x7f4fc402a540 comp rx=0 tx=0).stop 2024-07-04T17:31:41.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.397+0000 7f4fe1e00640 1 -- 172.21.15.42:0/2351988788 shutdown_connections 2024-07-04T17:31:41.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.397+0000 7f4fe1e00640 1 --2- 172.21.15.42:0/2351988788 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fdc073470 0x7f4fdc0a9990 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:41.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.397+0000 7f4fe1e00640 1 -- 172.21.15.42:0/2351988788 >> 172.21.15.42:0/2351988788 conn(0x7f4fdc06c020 msgr2=0x7f4fdc06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:41.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.397+0000 7f4fe1e00640 1 -- 172.21.15.42:0/2351988788 shutdown_connections 2024-07-04T17:31:41.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.397+0000 7f4fe1e00640 1 -- 172.21.15.42:0/2351988788 wait complete. 2024-07-04T17:31:41.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.400+0000 7f4fe1e00640 1 Processor -- start 2024-07-04T17:31:41.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.400+0000 7f4fe1e00640 1 -- start start 2024-07-04T17:31:41.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.402+0000 7f4fe1e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fdc073470 0x7f4fdc1d79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:41.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.402+0000 7f4fe1e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4fdc13f430 con 0x7f4fdc073470 2024-07-04T17:31:41.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.402+0000 7f4fdaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fdc073470 0x7f4fdc1d79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:41.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.402+0000 7f4fdaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fdc073470 0x7f4fdc1d79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44638/0 (socket says 172.21.15.42:44638) 2024-07-04T17:31:41.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.402+0000 7f4fdaa00640 1 -- 172.21.15.42:0/1431530747 learned_addr learned my addr 172.21.15.42:0/1431530747 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:41.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.403+0000 7f4fdaa00640 1 -- 172.21.15.42:0/1431530747 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fdc0ac800 con 0x7f4fdc073470 2024-07-04T17:31:41.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.403+0000 7f4fdaa00640 1 --2- 172.21.15.42:0/1431530747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fdc073470 0x7f4fdc1d79f0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc00c2f0 tx=0x7f4fcc00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:41.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.403+0000 7f4fd3e00640 1 -- 172.21.15.42:0/1431530747 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fcc017070 con 0x7f4fdc073470 2024-07-04T17:31:41.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.403+0000 7f4fd3e00640 1 -- 172.21.15.42:0/1431530747 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4fcc005410 con 0x7f4fdc073470 2024-07-04T17:31:41.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.403+0000 7f4fd3e00640 1 -- 172.21.15.42:0/1431530747 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fcc003780 con 0x7f4fdc073470 2024-07-04T17:31:41.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.403+0000 7f4fe1e00640 1 -- 172.21.15.42:0/1431530747 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fdc0ac2e0 con 0x7f4fdc073470 2024-07-04T17:31:41.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.404+0000 7f4fe1e00640 1 -- 172.21.15.42:0/1431530747 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fdc13c050 con 0x7f4fdc073470 2024-07-04T17:31:41.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.405+0000 7f4fe1e00640 1 -- 172.21.15.42:0/1431530747 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fdc13fb40 con 0x7f4fdc073470 2024-07-04T17:31:41.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.405+0000 7f4fd3e00640 1 -- 172.21.15.42:0/1431530747 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7f4fcc01a070 con 0x7f4fdc073470 2024-07-04T17:31:41.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.406+0000 7f4fd3e00640 1 --2- 172.21.15.42:0/1431530747 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f4fb803d420 0x7f4fb803f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:41.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.406+0000 7f4fd3e00640 1 -- 172.21.15.42:0/1431530747 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 3339+0+0 (secure 0 0 0) 0x7f4fcc050950 con 0x7f4fdc073470 2024-07-04T17:31:41.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.406+0000 7f4fd2000640 1 --2- 172.21.15.42:0/1431530747 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f4fb803d420 0x7f4fb803f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:41.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.407+0000 7f4fd2000640 1 --2- 172.21.15.42:0/1431530747 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f4fb803d420 0x7f4fb803f930 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4034000 tx=0x7f4fc4004730 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:41.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.412+0000 7f4fd3e00640 1 -- 172.21.15.42:0/1431530747 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f4fcc01ad80 con 0x7f4fdc073470 2024-07-04T17:31:41.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:41.662+0000 7f4fe1e00640 1 -- 172.21.15.42:0/1431530747 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4fdc069b70 con 0x7f4fdc073470 2024-07-04T17:31:42.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.078+0000 7f4fd3e00640 1 -- 172.21.15.42:0/1431530747 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v67) ==== 174+0+0 (secure 0 0 0) 0x7f4fcc01b3b0 con 0x7f4fdc073470 2024-07-04T17:31:42.080 INFO:teuthology.orchestra.run.smithi042.stderr:new fs with metadata pool 16 and data pool 17 2024-07-04T17:31:42.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.083+0000 7f4fe1e00640 1 -- 172.21.15.42:0/1431530747 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f4fb803d420 msgr2=0x7f4fb803f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:42.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.083+0000 7f4fe1e00640 1 --2- 172.21.15.42:0/1431530747 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f4fb803d420 0x7f4fb803f930 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4034000 tx=0x7f4fc4004730 comp rx=0 tx=0).stop 2024-07-04T17:31:42.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.083+0000 7f4fe1e00640 1 -- 172.21.15.42:0/1431530747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fdc073470 msgr2=0x7f4fdc1d79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:42.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.083+0000 7f4fe1e00640 1 --2- 172.21.15.42:0/1431530747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fdc073470 0x7f4fdc1d79f0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc00c2f0 tx=0x7f4fcc00c7b0 comp rx=0 tx=0).stop 2024-07-04T17:31:42.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.083+0000 7f4fe1e00640 1 -- 172.21.15.42:0/1431530747 shutdown_connections 2024-07-04T17:31:42.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.083+0000 7f4fe1e00640 1 --2- 172.21.15.42:0/1431530747 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f4fb803d420 0x7f4fb803f930 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:42.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.083+0000 7f4fe1e00640 1 --2- 172.21.15.42:0/1431530747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4fdc073470 0x7f4fdc1d79f0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:42.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.083+0000 7f4fe1e00640 1 -- 172.21.15.42:0/1431530747 >> 172.21.15.42:0/1431530747 conn(0x7f4fdc06c020 msgr2=0x7f4fdc144500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:42.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.084+0000 7f4fe1e00640 1 -- 172.21.15.42:0/1431530747 shutdown_connections 2024-07-04T17:31:42.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.084+0000 7f4fe1e00640 1 -- 172.21.15.42:0/1431530747 wait complete. 2024-07-04T17:31:42.132 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:31:42.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.389+0000 7f52b9a00640 1 -- 172.21.15.42:0/2230188366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52b4144fe0 msgr2=0x7f52b406c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:42.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.389+0000 7f52b9a00640 1 --2- 172.21.15.42:0/2230188366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52b4144fe0 0x7f52b406c020 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f529c005fd0 tx=0x7f529c02a5f0 comp rx=0 tx=0).stop 2024-07-04T17:31:42.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.389+0000 7f52b9a00640 1 -- 172.21.15.42:0/2230188366 shutdown_connections 2024-07-04T17:31:42.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.389+0000 7f52b9a00640 1 --2- 172.21.15.42:0/2230188366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52b4144fe0 0x7f52b406c020 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:42.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.389+0000 7f52b9a00640 1 -- 172.21.15.42:0/2230188366 >> 172.21.15.42:0/2230188366 conn(0x7f52b40bebc0 msgr2=0x7f52b40befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:42.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.389+0000 7f52b9a00640 1 -- 172.21.15.42:0/2230188366 shutdown_connections 2024-07-04T17:31:42.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.389+0000 7f52b9a00640 1 -- 172.21.15.42:0/2230188366 wait complete. 2024-07-04T17:31:42.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.391+0000 7f52b9a00640 1 Processor -- start 2024-07-04T17:31:42.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.391+0000 7f52b9a00640 1 -- start start 2024-07-04T17:31:42.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.392+0000 7f52b9a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52b40ba830 0x7f52b40ad250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:42.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.392+0000 7f52b9a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f52b40b1e30 con 0x7f52b40ba830 2024-07-04T17:31:42.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.393+0000 7f52b3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52b40ba830 0x7f52b40ad250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:42.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.393+0000 7f52b3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52b40ba830 0x7f52b40ad250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44664/0 (socket says 172.21.15.42:44664) 2024-07-04T17:31:42.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.393+0000 7f52b3e00640 1 -- 172.21.15.42:0/3297216640 learned_addr learned my addr 172.21.15.42:0/3297216640 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:42.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.393+0000 7f52b3e00640 1 -- 172.21.15.42:0/3297216640 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b40acd70 con 0x7f52b40ba830 2024-07-04T17:31:42.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.394+0000 7f52b3e00640 1 --2- 172.21.15.42:0/3297216640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52b40ba830 0x7f52b40ad250 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f52a400c6b0 tx=0x7f52a400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:42.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.394+0000 7f52abe00640 1 -- 172.21.15.42:0/3297216640 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52a4003aa0 con 0x7f52b40ba830 2024-07-04T17:31:42.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.394+0000 7f52abe00640 1 -- 172.21.15.42:0/3297216640 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52a4003c60 con 0x7f52b40ba830 2024-07-04T17:31:42.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.394+0000 7f52b9a00640 1 -- 172.21.15.42:0/3297216640 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b413fd40 con 0x7f52b40ba830 2024-07-04T17:31:42.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.394+0000 7f52abe00640 1 -- 172.21.15.42:0/3297216640 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52a4005070 con 0x7f52b40ba830 2024-07-04T17:31:42.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.394+0000 7f52b9a00640 1 -- 172.21.15.42:0/3297216640 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b40ac9d0 con 0x7f52b40ba830 2024-07-04T17:31:42.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.395+0000 7f52b9a00640 1 -- 172.21.15.42:0/3297216640 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52b40bce60 con 0x7f52b40ba830 2024-07-04T17:31:42.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.396+0000 7f52abe00640 1 -- 172.21.15.42:0/3297216640 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7f52a4005230 con 0x7f52b40ba830 2024-07-04T17:31:42.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.396+0000 7f52abe00640 1 --2- 172.21.15.42:0/3297216640 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f529003d3d0 0x7f529003f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:42.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.396+0000 7f52abe00640 1 -- 172.21.15.42:0/3297216640 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 3439+0+0 (secure 0 0 0) 0x7f52a40514e0 con 0x7f52b40ba830 2024-07-04T17:31:42.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.396+0000 7f52b3400640 1 --2- 172.21.15.42:0/3297216640 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f529003d3d0 0x7f529003f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:42.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.397+0000 7f52b3400640 1 --2- 172.21.15.42:0/3297216640 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f529003d3d0 0x7f529003f8e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f529c033000 tx=0x7f529c004730 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:42.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.401+0000 7f52abe00640 1 -- 172.21.15.42:0/3297216640 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f52b40bce60 con 0x7f52b40ba830 2024-07-04T17:31:42.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.602+0000 7f52b9a00640 1 -- 172.21.15.42:0/3297216640 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52b406c020 con 0x7f52b40ba830 2024-07-04T17:31:42.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.604+0000 7f52abe00640 1 -- 172.21.15.42:0/3297216640 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+10619 (secure 0 0 0) 0x7f52a400a660 con 0x7f52b40ba830 2024-07-04T17:31:42.605 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:31:42.605 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":91,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:31:41.664504+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:31:39.727656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-07-04T17:31:40.615610+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:31:42.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.607+0000 7f52b9a00640 1 -- 172.21.15.42:0/3297216640 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f529003d3d0 msgr2=0x7f529003f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:42.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.607+0000 7f52b9a00640 1 --2- 172.21.15.42:0/3297216640 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f529003d3d0 0x7f529003f8e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f529c033000 tx=0x7f529c004730 comp rx=0 tx=0).stop 2024-07-04T17:31:42.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.607+0000 7f52b9a00640 1 -- 172.21.15.42:0/3297216640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52b40ba830 msgr2=0x7f52b40ad250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:42.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.607+0000 7f52b9a00640 1 --2- 172.21.15.42:0/3297216640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52b40ba830 0x7f52b40ad250 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f52a400c6b0 tx=0x7f52a400cb70 comp rx=0 tx=0).stop 2024-07-04T17:31:42.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.607+0000 7f52b9a00640 1 -- 172.21.15.42:0/3297216640 shutdown_connections 2024-07-04T17:31:42.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.607+0000 7f52b9a00640 1 --2- 172.21.15.42:0/3297216640 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f529003d3d0 0x7f529003f8e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:42.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.607+0000 7f52b9a00640 1 --2- 172.21.15.42:0/3297216640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52b40ba830 0x7f52b40ad250 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:42.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.607+0000 7f52b9a00640 1 -- 172.21.15.42:0/3297216640 >> 172.21.15.42:0/3297216640 conn(0x7f52b40bebc0 msgr2=0x7f52b4142d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:42.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.607+0000 7f52b9a00640 1 -- 172.21.15.42:0/3297216640 shutdown_connections 2024-07-04T17:31:42.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.607+0000 7f52b9a00640 1 -- 172.21.15.42:0/3297216640 wait complete. 2024-07-04T17:31:42.635 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:31:42.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.903+0000 7f0689600640 1 -- 172.21.15.42:0/3855906015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06840b1c40 msgr2=0x7f06840b2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:42.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.903+0000 7f0689600640 1 --2- 172.21.15.42:0/3855906015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06840b1c40 0x7f06840b2be0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f0684074560 tx=0x7f067402a520 comp rx=0 tx=0).stop 2024-07-04T17:31:42.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.903+0000 7f0689600640 1 -- 172.21.15.42:0/3855906015 shutdown_connections 2024-07-04T17:31:42.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.903+0000 7f0689600640 1 --2- 172.21.15.42:0/3855906015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06840b1c40 0x7f06840b2be0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:42.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.903+0000 7f0689600640 1 -- 172.21.15.42:0/3855906015 >> 172.21.15.42:0/3855906015 conn(0x7f0684071960 msgr2=0x7f068406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:42.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.903+0000 7f0689600640 1 -- 172.21.15.42:0/3855906015 shutdown_connections 2024-07-04T17:31:42.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.903+0000 7f0689600640 1 -- 172.21.15.42:0/3855906015 wait complete. 2024-07-04T17:31:42.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.905+0000 7f0689600640 1 Processor -- start 2024-07-04T17:31:42.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.905+0000 7f0689600640 1 -- start start 2024-07-04T17:31:42.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.906+0000 7f0689600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0684073870 0x7f06840b6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:42.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.906+0000 7f0689600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f068414f360 con 0x7f0684073870 2024-07-04T17:31:42.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.907+0000 7f0683e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0684073870 0x7f06840b6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:42.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.907+0000 7f0683e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0684073870 0x7f06840b6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44688/0 (socket says 172.21.15.42:44688) 2024-07-04T17:31:42.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.907+0000 7f0683e00640 1 -- 172.21.15.42:0/3807486466 learned_addr learned my addr 172.21.15.42:0/3807486466 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:42.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.907+0000 7f0683e00640 1 -- 172.21.15.42:0/3807486466 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06840b6d90 con 0x7f0684073870 2024-07-04T17:31:42.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.908+0000 7f0683e00640 1 --2- 172.21.15.42:0/3807486466 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0684073870 0x7f06840b6740 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f067402ab10 tx=0x7f0674033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:42.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.908+0000 7f0680c00640 1 -- 172.21.15.42:0/3807486466 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0674031bc0 con 0x7f0684073870 2024-07-04T17:31:42.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.908+0000 7f0680c00640 1 -- 172.21.15.42:0/3807486466 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0674031e50 con 0x7f0684073870 2024-07-04T17:31:42.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.908+0000 7f0680c00640 1 -- 172.21.15.42:0/3807486466 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0674033260 con 0x7f0684073870 2024-07-04T17:31:42.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.908+0000 7f0689600640 1 -- 172.21.15.42:0/3807486466 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06841e1f30 con 0x7f0684073870 2024-07-04T17:31:42.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.908+0000 7f0689600640 1 -- 172.21.15.42:0/3807486466 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f068414b490 con 0x7f0684073870 2024-07-04T17:31:42.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.909+0000 7f0689600640 1 -- 172.21.15.42:0/3807486466 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f068414b780 con 0x7f0684073870 2024-07-04T17:31:42.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.909+0000 7f0680c00640 1 -- 172.21.15.42:0/3807486466 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7f0674033400 con 0x7f0684073870 2024-07-04T17:31:42.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.910+0000 7f0680c00640 1 --2- 172.21.15.42:0/3807486466 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f066003d420 0x7f066003f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:42.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.910+0000 7f0680c00640 1 -- 172.21.15.42:0/3807486466 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 3439+0+0 (secure 0 0 0) 0x7f0674072d20 con 0x7f0684073870 2024-07-04T17:31:42.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.910+0000 7f067b400640 1 --2- 172.21.15.42:0/3807486466 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f066003d420 0x7f066003f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:42.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.911+0000 7f067b400640 1 --2- 172.21.15.42:0/3807486466 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f066003d420 0x7f066003f930 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f066c0015c0 tx=0x7f066c007a90 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:42.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:42.915+0000 7f0680c00640 1 -- 172.21.15.42:0/3807486466 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f067403f030 con 0x7f0684073870 2024-07-04T17:31:43.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.125+0000 7f0689600640 1 -- 172.21.15.42:0/3807486466 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f06840b2be0 con 0x7f0684073870 2024-07-04T17:31:43.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.126+0000 7f0680c00640 1 -- 172.21.15.42:0/3807486466 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+10606 (secure 0 0 0) 0x7f0674048e20 con 0x7f0684073870 2024-07-04T17:31:43.129 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:31:43.129 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":92,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:31:43.091088+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:31:39.727656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-07-04T17:31:40.615610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:31:43.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.129+0000 7f0689600640 1 -- 172.21.15.42:0/3807486466 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f066003d420 msgr2=0x7f066003f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:43.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.129+0000 7f0689600640 1 --2- 172.21.15.42:0/3807486466 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f066003d420 0x7f066003f930 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f066c0015c0 tx=0x7f066c007a90 comp rx=0 tx=0).stop 2024-07-04T17:31:43.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.129+0000 7f0689600640 1 -- 172.21.15.42:0/3807486466 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0684073870 msgr2=0x7f06840b6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:43.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.129+0000 7f0689600640 1 --2- 172.21.15.42:0/3807486466 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0684073870 0x7f06840b6740 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f067402ab10 tx=0x7f0674033e40 comp rx=0 tx=0).stop 2024-07-04T17:31:43.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.129+0000 7f0689600640 1 -- 172.21.15.42:0/3807486466 shutdown_connections 2024-07-04T17:31:43.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.129+0000 7f0689600640 1 --2- 172.21.15.42:0/3807486466 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f066003d420 0x7f066003f930 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:43.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.129+0000 7f0689600640 1 --2- 172.21.15.42:0/3807486466 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0684073870 0x7f06840b6740 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:43.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.129+0000 7f0689600640 1 -- 172.21.15.42:0/3807486466 >> 172.21.15.42:0/3807486466 conn(0x7f0684071960 msgr2=0x7f06840b0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:43.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.129+0000 7f0689600640 1 -- 172.21.15.42:0/3807486466 shutdown_connections 2024-07-04T17:31:43.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.129+0000 7f0689600640 1 -- 172.21.15.42:0/3807486466 wait complete. 2024-07-04T17:31:43.155 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-07-04T17:31:43.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.435+0000 7f386fe00640 1 -- 172.21.15.42:0/2418515870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38700abe40 msgr2=0x7f38700aa900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:43.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.435+0000 7f386fe00640 1 --2- 172.21.15.42:0/2418515870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38700abe40 0x7f38700aa900 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f3858005fd0 tx=0x7f385802a8b0 comp rx=0 tx=0).stop 2024-07-04T17:31:43.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.435+0000 7f386fe00640 1 -- 172.21.15.42:0/2418515870 shutdown_connections 2024-07-04T17:31:43.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.435+0000 7f386fe00640 1 --2- 172.21.15.42:0/2418515870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38700abe40 0x7f38700aa900 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:43.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.435+0000 7f386fe00640 1 -- 172.21.15.42:0/2418515870 >> 172.21.15.42:0/2418515870 conn(0x7f38700afaa0 msgr2=0x7f38700afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:43.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.435+0000 7f386fe00640 1 -- 172.21.15.42:0/2418515870 shutdown_connections 2024-07-04T17:31:43.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.435+0000 7f386fe00640 1 -- 172.21.15.42:0/2418515870 wait complete. 2024-07-04T17:31:43.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.437+0000 7f386fe00640 1 Processor -- start 2024-07-04T17:31:43.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.437+0000 7f386fe00640 1 -- start start 2024-07-04T17:31:43.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.439+0000 7f386fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38700701d0 0x7f387014db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:43.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.439+0000 7f386fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3870073ed0 con 0x7f38700701d0 2024-07-04T17:31:43.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.439+0000 7f386ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38700701d0 0x7f387014db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:43.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.439+0000 7f386ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38700701d0 0x7f387014db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44706/0 (socket says 172.21.15.42:44706) 2024-07-04T17:31:43.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.439+0000 7f386ea00640 1 -- 172.21.15.42:0/4071443975 learned_addr learned my addr 172.21.15.42:0/4071443975 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:43.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.439+0000 7f386ea00640 1 -- 172.21.15.42:0/4071443975 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f387014eaa0 con 0x7f38700701d0 2024-07-04T17:31:43.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.440+0000 7f386ea00640 1 --2- 172.21.15.42:0/4071443975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38700701d0 0x7f387014db50 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f386000c380 tx=0x7f386000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:43.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.440+0000 7f3866a00640 1 -- 172.21.15.42:0/4071443975 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38600107a0 con 0x7f38700701d0 2024-07-04T17:31:43.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.440+0000 7f3866a00640 1 -- 172.21.15.42:0/4071443975 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3860017070 con 0x7f38700701d0 2024-07-04T17:31:43.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.440+0000 7f3866a00640 1 -- 172.21.15.42:0/4071443975 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3860010c00 con 0x7f38700701d0 2024-07-04T17:31:43.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.440+0000 7f386fe00640 1 -- 172.21.15.42:0/4071443975 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f387014e0b0 con 0x7f38700701d0 2024-07-04T17:31:43.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.440+0000 7f386fe00640 1 -- 172.21.15.42:0/4071443975 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f387014cd80 con 0x7f38700701d0 2024-07-04T17:31:43.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.441+0000 7f386fe00640 1 -- 172.21.15.42:0/4071443975 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38700a9c70 con 0x7f38700701d0 2024-07-04T17:31:43.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.441+0000 7f3866a00640 1 -- 172.21.15.42:0/4071443975 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7f386001c460 con 0x7f38700701d0 2024-07-04T17:31:43.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.442+0000 7f3866a00640 1 --2- 172.21.15.42:0/4071443975 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f384c03d3d0 0x7f384c03f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:43.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.442+0000 7f3866a00640 1 -- 172.21.15.42:0/4071443975 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 3439+0+0 (secure 0 0 0) 0x7f3860051500 con 0x7f38700701d0 2024-07-04T17:31:43.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.442+0000 7f386e000640 1 --2- 172.21.15.42:0/4071443975 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f384c03d3d0 0x7f384c03f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:43.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.443+0000 7f386e000640 1 --2- 172.21.15.42:0/4071443975 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f384c03d3d0 0x7f384c03f8e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3858034000 tx=0x7f3858004ab0 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:43.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.447+0000 7f3866a00640 1 -- 172.21.15.42:0/4071443975 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f38700a9c70 con 0x7f38700701d0 2024-07-04T17:31:43.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:43.701+0000 7f386fe00640 1 -- 172.21.15.42:0/4071443975 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3870068e30 con 0x7f38700701d0 2024-07-04T17:31:44.038 INFO:tasks.ceph.mds.a.smithi042.stderr:2024-07-04T17:31:44.037+0000 7fdfe2a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-04T17:31:44.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.107+0000 7f3866a00640 1 -- 172.21.15.42:0/4071443975 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v70) ==== 118+0+0 (secure 0 0 0) 0x7f386001c800 con 0x7f38700701d0 2024-07-04T17:31:44.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.111+0000 7f386fe00640 1 -- 172.21.15.42:0/4071443975 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f384c03d3d0 msgr2=0x7f384c03f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:44.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.111+0000 7f386fe00640 1 --2- 172.21.15.42:0/4071443975 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f384c03d3d0 0x7f384c03f8e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3858034000 tx=0x7f3858004ab0 comp rx=0 tx=0).stop 2024-07-04T17:31:44.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.111+0000 7f386fe00640 1 -- 172.21.15.42:0/4071443975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38700701d0 msgr2=0x7f387014db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:44.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.111+0000 7f386fe00640 1 --2- 172.21.15.42:0/4071443975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38700701d0 0x7f387014db50 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f386000c380 tx=0x7f386000c840 comp rx=0 tx=0).stop 2024-07-04T17:31:44.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.111+0000 7f386fe00640 1 -- 172.21.15.42:0/4071443975 shutdown_connections 2024-07-04T17:31:44.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.111+0000 7f386fe00640 1 --2- 172.21.15.42:0/4071443975 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f384c03d3d0 0x7f384c03f8e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:44.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.112+0000 7f386fe00640 1 --2- 172.21.15.42:0/4071443975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38700701d0 0x7f387014db50 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:44.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.112+0000 7f386fe00640 1 -- 172.21.15.42:0/4071443975 >> 172.21.15.42:0/4071443975 conn(0x7f38700afaa0 msgr2=0x7f38701d7370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:44.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.112+0000 7f386fe00640 1 -- 172.21.15.42:0/4071443975 shutdown_connections 2024-07-04T17:31:44.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.112+0000 7f386fe00640 1 -- 172.21.15.42:0/4071443975 wait complete. 2024-07-04T17:31:44.142 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-04T17:31:44.142 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-07-04T17:31:44.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.443+0000 7f6fef200640 1 -- 172.21.15.42:0/456341870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ff00b0fe0 msgr2=0x7f6ff00adef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:44.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.443+0000 7f6fef200640 1 --2- 172.21.15.42:0/456341870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ff00b0fe0 0x7f6ff00adef0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0097b0 tx=0x7f6fdc02a530 comp rx=0 tx=0).stop 2024-07-04T17:31:44.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.443+0000 7f6fef200640 1 -- 172.21.15.42:0/456341870 shutdown_connections 2024-07-04T17:31:44.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.443+0000 7f6fef200640 1 --2- 172.21.15.42:0/456341870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ff00b0fe0 0x7f6ff00adef0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:44.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.443+0000 7f6fef200640 1 -- 172.21.15.42:0/456341870 >> 172.21.15.42:0/456341870 conn(0x7f6ff00afaa0 msgr2=0x7f6ff00afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:44.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.444+0000 7f6fef200640 1 -- 172.21.15.42:0/456341870 shutdown_connections 2024-07-04T17:31:44.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.444+0000 7f6fef200640 1 -- 172.21.15.42:0/456341870 wait complete. 2024-07-04T17:31:44.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.446+0000 7f6fef200640 1 Processor -- start 2024-07-04T17:31:44.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.446+0000 7f6fef200640 1 -- start start 2024-07-04T17:31:44.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.447+0000 7f6fef200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ff00b0bd0 0x7f6ff0148d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:44.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.447+0000 7f6fef200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff00accd0 con 0x7f6ff00b0bd0 2024-07-04T17:31:44.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.447+0000 7f6fede00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ff00b0bd0 0x7f6ff0148d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:44.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.448+0000 7f6fede00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ff00b0bd0 0x7f6ff0148d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54232/0 (socket says 172.21.15.42:54232) 2024-07-04T17:31:44.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.448+0000 7f6fede00640 1 -- 172.21.15.42:0/4105958829 learned_addr learned my addr 172.21.15.42:0/4105958829 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:44.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.448+0000 7f6fede00640 1 -- 172.21.15.42:0/4105958829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff0071450 con 0x7f6ff00b0bd0 2024-07-04T17:31:44.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.448+0000 7f6fede00640 1 --2- 172.21.15.42:0/4105958829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ff00b0bd0 0x7f6ff0148d70 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000c3b0 tx=0x7f6fe000c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:44.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.449+0000 7f6fe6000640 1 -- 172.21.15.42:0/4105958829 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fe0011780 con 0x7f6ff00b0bd0 2024-07-04T17:31:44.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.449+0000 7f6fe6000640 1 -- 172.21.15.42:0/4105958829 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fe0017070 con 0x7f6ff00b0bd0 2024-07-04T17:31:44.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.449+0000 7f6fe6000640 1 -- 172.21.15.42:0/4105958829 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fe0011c00 con 0x7f6ff00b0bd0 2024-07-04T17:31:44.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.449+0000 7f6fef200640 1 -- 172.21.15.42:0/4105958829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ff0144b40 con 0x7f6ff00b0bd0 2024-07-04T17:31:44.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.449+0000 7f6fef200640 1 -- 172.21.15.42:0/4105958829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ff0144640 con 0x7f6ff00b0bd0 2024-07-04T17:31:44.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.450+0000 7f6fef200640 1 -- 172.21.15.42:0/4105958829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ff00ad330 con 0x7f6ff00b0bd0 2024-07-04T17:31:44.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.450+0000 7f6fe6000640 1 -- 172.21.15.42:0/4105958829 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7f6fe001c460 con 0x7f6ff00b0bd0 2024-07-04T17:31:44.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.451+0000 7f6fe6000640 1 --2- 172.21.15.42:0/4105958829 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f6fd003d3d0 0x7f6fd003f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:44.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.451+0000 7f6fe6000640 1 -- 172.21.15.42:0/4105958829 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 3439+0+0 (secure 0 0 0) 0x7f6fe0051490 con 0x7f6ff00b0bd0 2024-07-04T17:31:44.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.451+0000 7f6fed400640 1 --2- 172.21.15.42:0/4105958829 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f6fd003d3d0 0x7f6fd003f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:44.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.452+0000 7f6fed400640 1 --2- 172.21.15.42:0/4105958829 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f6fd003d3d0 0x7f6fd003f8e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc034000 tx=0x7f6fdc004730 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:44.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.456+0000 7f6fe6000640 1 -- 172.21.15.42:0/4105958829 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f6fe000a140 con 0x7f6ff00b0bd0 2024-07-04T17:31:44.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:44.706+0000 7f6fef200640 1 -- 172.21.15.42:0/4105958829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f6ff0068e30 con 0x7f6ff00b0bd0 2024-07-04T17:31:45.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.115+0000 7f6fe6000640 1 -- 172.21.15.42:0/4105958829 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v71) ==== 122+0+0 (secure 0 0 0) 0x7f6fe001c800 con 0x7f6ff00b0bd0 2024-07-04T17:31:45.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.119+0000 7f6fef200640 1 -- 172.21.15.42:0/4105958829 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f6fd003d3d0 msgr2=0x7f6fd003f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:45.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.119+0000 7f6fef200640 1 --2- 172.21.15.42:0/4105958829 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f6fd003d3d0 0x7f6fd003f8e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc034000 tx=0x7f6fdc004730 comp rx=0 tx=0).stop 2024-07-04T17:31:45.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.119+0000 7f6fef200640 1 -- 172.21.15.42:0/4105958829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ff00b0bd0 msgr2=0x7f6ff0148d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:45.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.119+0000 7f6fef200640 1 --2- 172.21.15.42:0/4105958829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ff00b0bd0 0x7f6ff0148d70 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000c3b0 tx=0x7f6fe000c870 comp rx=0 tx=0).stop 2024-07-04T17:31:45.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.120+0000 7f6fef200640 1 -- 172.21.15.42:0/4105958829 shutdown_connections 2024-07-04T17:31:45.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.120+0000 7f6fef200640 1 --2- 172.21.15.42:0/4105958829 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f6fd003d3d0 0x7f6fd003f8e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:45.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.120+0000 7f6fef200640 1 --2- 172.21.15.42:0/4105958829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ff00b0bd0 0x7f6ff0148d70 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:45.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.120+0000 7f6fef200640 1 -- 172.21.15.42:0/4105958829 >> 172.21.15.42:0/4105958829 conn(0x7f6ff00afaa0 msgr2=0x7f6ff014cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:45.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.120+0000 7f6fef200640 1 -- 172.21.15.42:0/4105958829 shutdown_connections 2024-07-04T17:31:45.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.120+0000 7f6fef200640 1 -- 172.21.15.42:0/4105958829 wait complete. 2024-07-04T17:31:45.152 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-07-04T17:31:45.152 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:31:45.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.452+0000 7fadb5400640 1 -- 172.21.15.42:0/1421290025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadb00bad70 msgr2=0x7fadb006c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:45.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.452+0000 7fadb5400640 1 --2- 172.21.15.42:0/1421290025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadb00bad70 0x7fadb006c2c0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fada0005fd0 tx=0x7fada002af90 comp rx=0 tx=0).stop 2024-07-04T17:31:45.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.452+0000 7fadb5400640 1 -- 172.21.15.42:0/1421290025 shutdown_connections 2024-07-04T17:31:45.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.452+0000 7fadb5400640 1 --2- 172.21.15.42:0/1421290025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadb00bad70 0x7fadb006c2c0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:45.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.452+0000 7fadb5400640 1 -- 172.21.15.42:0/1421290025 >> 172.21.15.42:0/1421290025 conn(0x7fadb00bebc0 msgr2=0x7fadb00befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:45.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.452+0000 7fadb5400640 1 -- 172.21.15.42:0/1421290025 shutdown_connections 2024-07-04T17:31:45.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.452+0000 7fadb5400640 1 -- 172.21.15.42:0/1421290025 wait complete. 2024-07-04T17:31:45.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.454+0000 7fadb5400640 1 Processor -- start 2024-07-04T17:31:45.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.454+0000 7fadb5400640 1 -- start start 2024-07-04T17:31:45.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.456+0000 7fadb5400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadb00b5650 0x7fadb0073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:45.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.456+0000 7fadb5400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fadb00b27c0 con 0x7fadb00b5650 2024-07-04T17:31:45.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.456+0000 7fadafe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadb00b5650 0x7fadb0073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:45.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.456+0000 7fadafe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadb00b5650 0x7fadb0073660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54254/0 (socket says 172.21.15.42:54254) 2024-07-04T17:31:45.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.456+0000 7fadafe00640 1 -- 172.21.15.42:0/4259479230 learned_addr learned my addr 172.21.15.42:0/4259479230 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:45.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.456+0000 7fadafe00640 1 -- 172.21.15.42:0/4259479230 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb01d4820 con 0x7fadb00b5650 2024-07-04T17:31:45.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.457+0000 7fadafe00640 1 --2- 172.21.15.42:0/4259479230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadb00b5650 0x7fadb0073660 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fada0006010 tx=0x7fada0009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:45.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.457+0000 7fada7e00640 1 -- 172.21.15.42:0/4259479230 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fada0040020 con 0x7fadb00b5650 2024-07-04T17:31:45.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.457+0000 7fada7e00640 1 -- 172.21.15.42:0/4259479230 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fada0004040 con 0x7fadb00b5650 2024-07-04T17:31:45.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.457+0000 7fada7e00640 1 -- 172.21.15.42:0/4259479230 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fada0038040 con 0x7fadb00b5650 2024-07-04T17:31:45.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.457+0000 7fadb5400640 1 -- 172.21.15.42:0/4259479230 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadb01dcf60 con 0x7fadb00b5650 2024-07-04T17:31:45.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.457+0000 7fadb5400640 1 -- 172.21.15.42:0/4259479230 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadb01d3b80 con 0x7fadb00b5650 2024-07-04T17:31:45.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.458+0000 7fadb5400640 1 -- 172.21.15.42:0/4259479230 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadb006c2c0 con 0x7fadb00b5650 2024-07-04T17:31:45.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.459+0000 7fada7e00640 1 -- 172.21.15.42:0/4259479230 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7fada0043020 con 0x7fadb00b5650 2024-07-04T17:31:45.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.459+0000 7fada7e00640 1 --2- 172.21.15.42:0/4259479230 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fad8c03d3d0 0x7fad8c03f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:45.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.459+0000 7fada7e00640 1 -- 172.21.15.42:0/4259479230 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 3439+0+0 (secure 0 0 0) 0x7fada0071e50 con 0x7fadb00b5650 2024-07-04T17:31:45.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.459+0000 7fadaf400640 1 --2- 172.21.15.42:0/4259479230 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fad8c03d3d0 0x7fad8c03f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:45.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.460+0000 7fadaf400640 1 --2- 172.21.15.42:0/4259479230 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fad8c03d3d0 0x7fad8c03f8e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fad98001560 tx=0x7fad980079f0 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:45.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.464+0000 7fada7e00640 1 -- 172.21.15.42:0/4259479230 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fada004a030 con 0x7fadb00b5650 2024-07-04T17:31:45.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.717+0000 7fadb5400640 1 -- 172.21.15.42:0/4259479230 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fadb0068e30 con 0x7fadb00b5650 2024-07-04T17:31:45.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.718+0000 7fada7e00640 1 -- 172.21.15.42:0/4259479230 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+6038 (secure 0 0 0) 0x7fada0041e50 con 0x7fadb00b5650 2024-07-04T17:31:45.719 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:31:45.720 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":71,"btime":"2024-07-04T17:31:45:109183+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/244742173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":244742173},{"type":"v1","addr":"172.21.15.42:6825","nonce":244742173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5265,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/973681058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":973681058},{"type":"v1","addr":"172.21.15.42:6827","nonce":973681058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/480649366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":480649366},{"type":"v1","addr":"172.21.15.42:6829","nonce":480649366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5276,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/327636003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":327636003},{"type":"v1","addr":"172.21.15.42:6831","nonce":327636003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:31:41.664414+0000","modified":"2024-07-04T17:31:44.709115+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5277":{"gid":5277,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6833/3777807763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":3777807763},{"type":"v1","addr":"172.21.15.42:6833","nonce":3777807763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5277,"qdb_cluster":[5277]},"id":8}]} 2024-07-04T17:31:45.720 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 71 2024-07-04T17:31:45.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.721+0000 7fadb5400640 1 -- 172.21.15.42:0/4259479230 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fad8c03d3d0 msgr2=0x7fad8c03f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:45.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.721+0000 7fadb5400640 1 --2- 172.21.15.42:0/4259479230 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fad8c03d3d0 0x7fad8c03f8e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fad98001560 tx=0x7fad980079f0 comp rx=0 tx=0).stop 2024-07-04T17:31:45.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.721+0000 7fadb5400640 1 -- 172.21.15.42:0/4259479230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadb00b5650 msgr2=0x7fadb0073660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:45.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.721+0000 7fadb5400640 1 --2- 172.21.15.42:0/4259479230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadb00b5650 0x7fadb0073660 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fada0006010 tx=0x7fada0009900 comp rx=0 tx=0).stop 2024-07-04T17:31:45.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.721+0000 7fadb5400640 1 -- 172.21.15.42:0/4259479230 shutdown_connections 2024-07-04T17:31:45.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.721+0000 7fadb5400640 1 --2- 172.21.15.42:0/4259479230 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fad8c03d3d0 0x7fad8c03f8e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:45.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.721+0000 7fadb5400640 1 --2- 172.21.15.42:0/4259479230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadb00b5650 0x7fadb0073660 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:45.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.721+0000 7fadb5400640 1 -- 172.21.15.42:0/4259479230 >> 172.21.15.42:0/4259479230 conn(0x7fadb00bebc0 msgr2=0x7fadb013fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:45.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.722+0000 7fadb5400640 1 -- 172.21.15.42:0/4259479230 shutdown_connections 2024-07-04T17:31:45.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:45.722+0000 7fadb5400640 1 -- 172.21.15.42:0/4259479230 wait complete. 2024-07-04T17:31:45.749 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:31:46.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.034+0000 7fe415800640 1 -- 172.21.15.42:0/1720051546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4100b38b0 msgr2=0x7fe4100b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:46.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.034+0000 7fe415800640 1 --2- 172.21.15.42:0/1720051546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4100b38b0 0x7fe4100b2d20 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fe400005fd0 tx=0x7fe40002a5f0 comp rx=0 tx=0).stop 2024-07-04T17:31:46.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.034+0000 7fe415800640 1 -- 172.21.15.42:0/1720051546 shutdown_connections 2024-07-04T17:31:46.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.034+0000 7fe415800640 1 --2- 172.21.15.42:0/1720051546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4100b38b0 0x7fe4100b2d20 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:46.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.034+0000 7fe415800640 1 -- 172.21.15.42:0/1720051546 >> 172.21.15.42:0/1720051546 conn(0x7fe4100bf700 msgr2=0x7fe4100af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:46.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.034+0000 7fe415800640 1 -- 172.21.15.42:0/1720051546 shutdown_connections 2024-07-04T17:31:46.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.034+0000 7fe415800640 1 -- 172.21.15.42:0/1720051546 wait complete. 2024-07-04T17:31:46.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.037+0000 7fe415800640 1 Processor -- start 2024-07-04T17:31:46.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.037+0000 7fe415800640 1 -- start start 2024-07-04T17:31:46.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.038+0000 7fe415800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4100b1e00 0x7fe4100ad7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:46.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.038+0000 7fe415800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe4100b06e0 con 0x7fe4100b1e00 2024-07-04T17:31:46.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.038+0000 7fe40ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4100b1e00 0x7fe4100ad7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:46.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.038+0000 7fe40ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4100b1e00 0x7fe4100ad7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54282/0 (socket says 172.21.15.42:54282) 2024-07-04T17:31:46.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.038+0000 7fe40ea00640 1 -- 172.21.15.42:0/2878871594 learned_addr learned my addr 172.21.15.42:0/2878871594 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:46.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.039+0000 7fe40ea00640 1 -- 172.21.15.42:0/2878871594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4100add70 con 0x7fe4100b1e00 2024-07-04T17:31:46.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.039+0000 7fe40ea00640 1 --2- 172.21.15.42:0/2878871594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4100b1e00 0x7fe4100ad7e0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fe400009850 tx=0x7fe4000049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:46.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.039+0000 7fe406a00640 1 -- 172.21.15.42:0/2878871594 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe40003e030 con 0x7fe4100b1e00 2024-07-04T17:31:46.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.040+0000 7fe406a00640 1 -- 172.21.15.42:0/2878871594 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4000035f0 con 0x7fe4100b1e00 2024-07-04T17:31:46.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.040+0000 7fe406a00640 1 -- 172.21.15.42:0/2878871594 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe400035040 con 0x7fe4100b1e00 2024-07-04T17:31:46.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.040+0000 7fe415800640 1 -- 172.21.15.42:0/2878871594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4100aecd0 con 0x7fe4100b1e00 2024-07-04T17:31:46.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.040+0000 7fe415800640 1 -- 172.21.15.42:0/2878871594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe41013f2f0 con 0x7fe4100b1e00 2024-07-04T17:31:46.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.041+0000 7fe415800640 1 -- 172.21.15.42:0/2878871594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe41006c590 con 0x7fe4100b1e00 2024-07-04T17:31:46.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.041+0000 7fe406a00640 1 -- 172.21.15.42:0/2878871594 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7fe400041030 con 0x7fe4100b1e00 2024-07-04T17:31:46.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.042+0000 7fe406a00640 1 --2- 172.21.15.42:0/2878871594 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fe3ec03d3d0 0x7fe3ec03f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:46.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.042+0000 7fe406a00640 1 -- 172.21.15.42:0/2878871594 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 3439+0+0 (secure 0 0 0) 0x7fe40003f070 con 0x7fe4100b1e00 2024-07-04T17:31:46.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.042+0000 7fe407e00640 1 --2- 172.21.15.42:0/2878871594 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fe3ec03d3d0 0x7fe3ec03f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:46.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.043+0000 7fe407e00640 1 --2- 172.21.15.42:0/2878871594 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fe3ec03d3d0 0x7fe3ec03f8e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe41006ed80 tx=0x7fe3f80079f0 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:46.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.047+0000 7fe406a00640 1 -- 172.21.15.42:0/2878871594 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fe40004bd60 con 0x7fe4100b1e00 2024-07-04T17:31:46.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.264+0000 7fe415800640 1 -- 172.21.15.42:0/2878871594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe410069b70 con 0x7fe4100b1e00 2024-07-04T17:31:46.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.265+0000 7fe406a00640 1 -- 172.21.15.42:0/2878871594 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+10606 (secure 0 0 0) 0x7fe400030cc0 con 0x7fe4100b1e00 2024-07-04T17:31:46.266 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:31:46.266 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":92,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:31:43.091088+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:31:39.727656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-07-04T17:31:40.615610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:31:46.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.267+0000 7fe415800640 1 -- 172.21.15.42:0/2878871594 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fe3ec03d3d0 msgr2=0x7fe3ec03f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:46.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.268+0000 7fe415800640 1 --2- 172.21.15.42:0/2878871594 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fe3ec03d3d0 0x7fe3ec03f8e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe41006ed80 tx=0x7fe3f80079f0 comp rx=0 tx=0).stop 2024-07-04T17:31:46.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.268+0000 7fe415800640 1 -- 172.21.15.42:0/2878871594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4100b1e00 msgr2=0x7fe4100ad7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:46.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.268+0000 7fe415800640 1 --2- 172.21.15.42:0/2878871594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4100b1e00 0x7fe4100ad7e0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fe400009850 tx=0x7fe4000049d0 comp rx=0 tx=0).stop 2024-07-04T17:31:46.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.268+0000 7fe415800640 1 -- 172.21.15.42:0/2878871594 shutdown_connections 2024-07-04T17:31:46.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.268+0000 7fe415800640 1 --2- 172.21.15.42:0/2878871594 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fe3ec03d3d0 0x7fe3ec03f8e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:46.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.268+0000 7fe415800640 1 --2- 172.21.15.42:0/2878871594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4100b1e00 0x7fe4100ad7e0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:46.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.268+0000 7fe415800640 1 -- 172.21.15.42:0/2878871594 >> 172.21.15.42:0/2878871594 conn(0x7fe4100bf700 msgr2=0x7fe4100bd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:46.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.268+0000 7fe415800640 1 -- 172.21.15.42:0/2878871594 shutdown_connections 2024-07-04T17:31:46.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.268+0000 7fe415800640 1 -- 172.21.15.42:0/2878871594 wait complete. 2024-07-04T17:31:46.295 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-04T17:31:46.295 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:31:46.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.574+0000 7fadfe800640 1 -- 172.21.15.42:0/1906568852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadf80b17d0 msgr2=0x7fadf80b1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:46.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.574+0000 7fadfe800640 1 --2- 172.21.15.42:0/1906568852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadf80b17d0 0x7fadf80b1be0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fade4005fd0 tx=0x7fade402a560 comp rx=0 tx=0).stop 2024-07-04T17:31:46.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.574+0000 7fadfe800640 1 -- 172.21.15.42:0/1906568852 shutdown_connections 2024-07-04T17:31:46.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.574+0000 7fadfe800640 1 --2- 172.21.15.42:0/1906568852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadf80b17d0 0x7fadf80b1be0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:46.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.574+0000 7fadfe800640 1 -- 172.21.15.42:0/1906568852 >> 172.21.15.42:0/1906568852 conn(0x7fadf80bebc0 msgr2=0x7fadf80befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:46.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.574+0000 7fadfe800640 1 -- 172.21.15.42:0/1906568852 shutdown_connections 2024-07-04T17:31:46.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.574+0000 7fadfe800640 1 -- 172.21.15.42:0/1906568852 wait complete. 2024-07-04T17:31:46.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.576+0000 7fadfe800640 1 Processor -- start 2024-07-04T17:31:46.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.577+0000 7fadfe800640 1 -- start start 2024-07-04T17:31:46.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.578+0000 7fadfe800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadf806c200 0x7fadf8146420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:46.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.578+0000 7fadfe800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fadf814ae80 con 0x7fadf806c200 2024-07-04T17:31:46.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.578+0000 7fadfd400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadf806c200 0x7fadf8146420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:46.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.578+0000 7fadfd400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadf806c200 0x7fadf8146420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54308/0 (socket says 172.21.15.42:54308) 2024-07-04T17:31:46.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.578+0000 7fadfd400640 1 -- 172.21.15.42:0/410736274 learned_addr learned my addr 172.21.15.42:0/410736274 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:46.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.578+0000 7fadfd400640 1 -- 172.21.15.42:0/410736274 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf80aefa0 con 0x7fadf806c200 2024-07-04T17:31:46.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.579+0000 7fadfd400640 1 --2- 172.21.15.42:0/410736274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadf806c200 0x7fadf8146420 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fadec00c380 tx=0x7fadec00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:46.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.579+0000 7fadf5600640 1 -- 172.21.15.42:0/410736274 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadec017070 con 0x7fadf806c200 2024-07-04T17:31:46.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.579+0000 7fadf5600640 1 -- 172.21.15.42:0/410736274 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fadec0052f0 con 0x7fadf806c200 2024-07-04T17:31:46.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.579+0000 7fadfe800640 1 -- 172.21.15.42:0/410736274 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadf80aeae0 con 0x7fadf806c200 2024-07-04T17:31:46.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.579+0000 7fadf5600640 1 -- 172.21.15.42:0/410736274 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadec0035f0 con 0x7fadf806c200 2024-07-04T17:31:46.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.579+0000 7fadfe800640 1 -- 172.21.15.42:0/410736274 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadf81e3ff0 con 0x7fadf806c200 2024-07-04T17:31:46.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.580+0000 7fadfe800640 1 -- 172.21.15.42:0/410736274 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadf81d78e0 con 0x7fadf806c200 2024-07-04T17:31:46.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.581+0000 7fadf5600640 1 -- 172.21.15.42:0/410736274 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7fadec01a070 con 0x7fadf806c200 2024-07-04T17:31:46.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.581+0000 7fadf5600640 1 --2- 172.21.15.42:0/410736274 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fadd803d380 0x7fadd803f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:46.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.581+0000 7fadf5600640 1 -- 172.21.15.42:0/410736274 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 3439+0+0 (secure 0 0 0) 0x7fadec050a10 con 0x7fadf806c200 2024-07-04T17:31:46.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.581+0000 7fadfca00640 1 --2- 172.21.15.42:0/410736274 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fadd803d380 0x7fadd803f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:46.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.582+0000 7fadfca00640 1 --2- 172.21.15.42:0/410736274 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fadd803d380 0x7fadd803f890 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fade4033000 tx=0x7fade4004730 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:46.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.586+0000 7fadf5600640 1 -- 172.21.15.42:0/410736274 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fadec01e0d0 con 0x7fadf806c200 2024-07-04T17:31:46.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.789+0000 7fadfe800640 1 -- 172.21.15.42:0/410736274 --> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fadf81e4390 con 0x7fadd803d380 2024-07-04T17:31:46.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.805+0000 7fadf5600640 1 -- 172.21.15.42:0/410736274 <== mgr.5173 v2:172.21.15.42:6834/1139872311 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246080 (secure 0 0 0) 0x7fadf81e4390 con 0x7fadd803d380 2024-07-04T17:31:46.810 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:31:46.812 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:31:46.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.812+0000 7fadfe800640 1 -- 172.21.15.42:0/410736274 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fadd803d380 msgr2=0x7fadd803f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:46.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.812+0000 7fadfe800640 1 --2- 172.21.15.42:0/410736274 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fadd803d380 0x7fadd803f890 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fade4033000 tx=0x7fade4004730 comp rx=0 tx=0).stop 2024-07-04T17:31:46.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.812+0000 7fadfe800640 1 -- 172.21.15.42:0/410736274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadf806c200 msgr2=0x7fadf8146420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:46.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.812+0000 7fadfe800640 1 --2- 172.21.15.42:0/410736274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadf806c200 0x7fadf8146420 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fadec00c380 tx=0x7fadec00c840 comp rx=0 tx=0).stop 2024-07-04T17:31:46.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.812+0000 7fadfe800640 1 -- 172.21.15.42:0/410736274 shutdown_connections 2024-07-04T17:31:46.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.812+0000 7fadfe800640 1 --2- 172.21.15.42:0/410736274 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fadd803d380 0x7fadd803f890 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:46.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.812+0000 7fadfe800640 1 --2- 172.21.15.42:0/410736274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadf806c200 0x7fadf8146420 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:46.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.812+0000 7fadfe800640 1 -- 172.21.15.42:0/410736274 >> 172.21.15.42:0/410736274 conn(0x7fadf80bebc0 msgr2=0x7fadf8149570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:46.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.812+0000 7fadfe800640 1 -- 172.21.15.42:0/410736274 shutdown_connections 2024-07-04T17:31:46.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:46.812+0000 7fadfe800640 1 -- 172.21.15.42:0/410736274 wait complete. 2024-07-04T17:31:46.837 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-07-04T17:31:46.437949+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":356,"num_read_kb":926,"num_write":184,"num_write_kb":1120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126,"ondisk_log_size":126,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":109964,"kb_used_data":3936,"kb_used_omap":4,"kb_used_meta":105979,"kb_avail":283005556,"statfs":{"total":289910292480,"available":289797689344,"internally_reserved":0,"allocated":4030464,"data_stored":2280385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":108522831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001977"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.485394+0000","last_change":"2024-07-04T17:31:42.147550+0000","last_active":"2024-07-04T17:31:43.485394+0000","last_peered":"2024-07-04T17:31:43.485394+0000","last_clean":"2024-07-04T17:31:43.485394+0000","last_became_active":"2024-07-04T17:31:42.146638+0000","last_became_peered":"2024-07-04T17:31:42.146638+0000","last_unstale":"2024-07-04T17:31:43.485394+0000","last_undegraded":"2024-07-04T17:31:43.485394+0000","last_fullsized":"2024-07-04T17:31:43.485394+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:17:39.808949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.103805+0000","last_change":"2024-07-04T17:31:42.132420+0000","last_active":"2024-07-04T17:31:43.103805+0000","last_peered":"2024-07-04T17:31:43.103805+0000","last_clean":"2024-07-04T17:31:43.103805+0000","last_became_active":"2024-07-04T17:31:42.132031+0000","last_became_peered":"2024-07-04T17:31:42.132031+0000","last_unstale":"2024-07-04T17:31:43.103805+0000","last_undegraded":"2024-07-04T17:31:43.103805+0000","last_fullsized":"2024-07-04T17:31:43.103805+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:41:23.204256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.487015+0000","last_change":"2024-07-04T17:31:42.146020+0000","last_active":"2024-07-04T17:31:43.487015+0000","last_peered":"2024-07-04T17:31:43.487015+0000","last_clean":"2024-07-04T17:31:43.487015+0000","last_became_active":"2024-07-04T17:31:42.143682+0000","last_became_peered":"2024-07-04T17:31:42.143682+0000","last_unstale":"2024-07-04T17:31:43.487015+0000","last_undegraded":"2024-07-04T17:31:43.487015+0000","last_fullsized":"2024-07-04T17:31:43.487015+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:06:33.198375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.482306+0000","last_change":"2024-07-04T17:31:42.148476+0000","last_active":"2024-07-04T17:31:43.482306+0000","last_peered":"2024-07-04T17:31:43.482306+0000","last_clean":"2024-07-04T17:31:43.482306+0000","last_became_active":"2024-07-04T17:31:42.148083+0000","last_became_peered":"2024-07-04T17:31:42.148083+0000","last_unstale":"2024-07-04T17:31:43.482306+0000","last_undegraded":"2024-07-04T17:31:43.482306+0000","last_fullsized":"2024-07-04T17:31:43.482306+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:19:47.089588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.103647+0000","last_change":"2024-07-04T17:31:42.118035+0000","last_active":"2024-07-04T17:31:43.103647+0000","last_peered":"2024-07-04T17:31:43.103647+0000","last_clean":"2024-07-04T17:31:43.103647+0000","last_became_active":"2024-07-04T17:31:42.117722+0000","last_became_peered":"2024-07-04T17:31:42.117722+0000","last_unstale":"2024-07-04T17:31:43.103647+0000","last_undegraded":"2024-07-04T17:31:43.103647+0000","last_fullsized":"2024-07-04T17:31:43.103647+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:17:18.803619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.484697+0000","last_change":"2024-07-04T17:31:42.146322+0000","last_active":"2024-07-04T17:31:43.484697+0000","last_peered":"2024-07-04T17:31:43.484697+0000","last_clean":"2024-07-04T17:31:43.484697+0000","last_became_active":"2024-07-04T17:31:42.146004+0000","last_became_peered":"2024-07-04T17:31:42.146004+0000","last_unstale":"2024-07-04T17:31:43.484697+0000","last_undegraded":"2024-07-04T17:31:43.484697+0000","last_fullsized":"2024-07-04T17:31:43.484697+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:10:27.612193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.102518+0000","last_change":"2024-07-04T17:31:42.131777+0000","last_active":"2024-07-04T17:31:43.102518+0000","last_peered":"2024-07-04T17:31:43.102518+0000","last_clean":"2024-07-04T17:31:43.102518+0000","last_became_active":"2024-07-04T17:31:42.127915+0000","last_became_peered":"2024-07-04T17:31:42.127915+0000","last_unstale":"2024-07-04T17:31:43.102518+0000","last_undegraded":"2024-07-04T17:31:43.102518+0000","last_fullsized":"2024-07-04T17:31:43.102518+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:14:13.315623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.103763+0000","last_change":"2024-07-04T17:31:42.127048+0000","last_active":"2024-07-04T17:31:43.103763+0000","last_peered":"2024-07-04T17:31:43.103763+0000","last_clean":"2024-07-04T17:31:43.103763+0000","last_became_active":"2024-07-04T17:31:42.126695+0000","last_became_peered":"2024-07-04T17:31:42.126695+0000","last_unstale":"2024-07-04T17:31:43.103763+0000","last_undegraded":"2024-07-04T17:31:43.103763+0000","last_fullsized":"2024-07-04T17:31:43.103763+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:46:15.072312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.485609+0000","last_change":"2024-07-04T17:31:42.134470+0000","last_active":"2024-07-04T17:31:43.485609+0000","last_peered":"2024-07-04T17:31:43.485609+0000","last_clean":"2024-07-04T17:31:43.485609+0000","last_became_active":"2024-07-04T17:31:42.134096+0000","last_became_peered":"2024-07-04T17:31:42.134096+0000","last_unstale":"2024-07-04T17:31:43.485609+0000","last_undegraded":"2024-07-04T17:31:43.485609+0000","last_fullsized":"2024-07-04T17:31:43.485609+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:07:42.785561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.483766+0000","last_change":"2024-07-04T17:31:42.109492+0000","last_active":"2024-07-04T17:31:43.483766+0000","last_peered":"2024-07-04T17:31:43.483766+0000","last_clean":"2024-07-04T17:31:43.483766+0000","last_became_active":"2024-07-04T17:31:42.108502+0000","last_became_peered":"2024-07-04T17:31:42.108502+0000","last_unstale":"2024-07-04T17:31:43.483766+0000","last_undegraded":"2024-07-04T17:31:43.483766+0000","last_fullsized":"2024-07-04T17:31:43.483766+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:39:41.707779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.482991+0000","last_change":"2024-07-04T17:31:42.138904+0000","last_active":"2024-07-04T17:31:43.482991+0000","last_peered":"2024-07-04T17:31:43.482991+0000","last_clean":"2024-07-04T17:31:43.482991+0000","last_became_active":"2024-07-04T17:31:42.138265+0000","last_became_peered":"2024-07-04T17:31:42.138265+0000","last_unstale":"2024-07-04T17:31:43.482991+0000","last_undegraded":"2024-07-04T17:31:43.482991+0000","last_fullsized":"2024-07-04T17:31:43.482991+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:26.894424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.482403+0000","last_change":"2024-07-04T17:31:42.110720+0000","last_active":"2024-07-04T17:31:43.482403+0000","last_peered":"2024-07-04T17:31:43.482403+0000","last_clean":"2024-07-04T17:31:43.482403+0000","last_became_active":"2024-07-04T17:31:42.110428+0000","last_became_peered":"2024-07-04T17:31:42.110428+0000","last_unstale":"2024-07-04T17:31:43.482403+0000","last_undegraded":"2024-07-04T17:31:43.482403+0000","last_fullsized":"2024-07-04T17:31:43.482403+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:03:49.795002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.103910+0000","last_change":"2024-07-04T17:31:42.132759+0000","last_active":"2024-07-04T17:31:43.103910+0000","last_peered":"2024-07-04T17:31:43.103910+0000","last_clean":"2024-07-04T17:31:43.103910+0000","last_became_active":"2024-07-04T17:31:42.132493+0000","last_became_peered":"2024-07-04T17:31:42.132493+0000","last_unstale":"2024-07-04T17:31:43.103910+0000","last_undegraded":"2024-07-04T17:31:43.103910+0000","last_fullsized":"2024-07-04T17:31:43.103910+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:41:07.944963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.484272+0000","last_change":"2024-07-04T17:31:42.137702+0000","last_active":"2024-07-04T17:31:43.484272+0000","last_peered":"2024-07-04T17:31:43.484272+0000","last_clean":"2024-07-04T17:31:43.484272+0000","last_became_active":"2024-07-04T17:31:42.134081+0000","last_became_peered":"2024-07-04T17:31:42.134081+0000","last_unstale":"2024-07-04T17:31:43.484272+0000","last_undegraded":"2024-07-04T17:31:43.484272+0000","last_fullsized":"2024-07-04T17:31:43.484272+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:26:29.769982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.481505+0000","last_change":"2024-07-04T17:31:42.126095+0000","last_active":"2024-07-04T17:31:43.481505+0000","last_peered":"2024-07-04T17:31:43.481505+0000","last_clean":"2024-07-04T17:31:43.481505+0000","last_became_active":"2024-07-04T17:31:42.125783+0000","last_became_peered":"2024-07-04T17:31:42.125783+0000","last_unstale":"2024-07-04T17:31:43.481505+0000","last_undegraded":"2024-07-04T17:31:43.481505+0000","last_fullsized":"2024-07-04T17:31:43.481505+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:01:18.486569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.481014+0000","last_change":"2024-07-04T17:31:42.124307+0000","last_active":"2024-07-04T17:31:43.481014+0000","last_peered":"2024-07-04T17:31:43.481014+0000","last_clean":"2024-07-04T17:31:43.481014+0000","last_became_active":"2024-07-04T17:31:42.121958+0000","last_became_peered":"2024-07-04T17:31:42.121958+0000","last_unstale":"2024-07-04T17:31:43.481014+0000","last_undegraded":"2024-07-04T17:31:43.481014+0000","last_fullsized":"2024-07-04T17:31:43.481014+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:14:03.072947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.487920+0000","last_change":"2024-07-04T17:31:42.127722+0000","last_active":"2024-07-04T17:31:43.487920+0000","last_peered":"2024-07-04T17:31:43.487920+0000","last_clean":"2024-07-04T17:31:43.487920+0000","last_became_active":"2024-07-04T17:31:42.127401+0000","last_became_peered":"2024-07-04T17:31:42.127401+0000","last_unstale":"2024-07-04T17:31:43.487920+0000","last_undegraded":"2024-07-04T17:31:43.487920+0000","last_fullsized":"2024-07-04T17:31:43.487920+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:08:43.721732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.487246+0000","last_change":"2024-07-04T17:31:42.131382+0000","last_active":"2024-07-04T17:31:43.487246+0000","last_peered":"2024-07-04T17:31:43.487246+0000","last_clean":"2024-07-04T17:31:43.487246+0000","last_became_active":"2024-07-04T17:31:42.131053+0000","last_became_peered":"2024-07-04T17:31:42.131053+0000","last_unstale":"2024-07-04T17:31:43.487246+0000","last_undegraded":"2024-07-04T17:31:43.487246+0000","last_fullsized":"2024-07-04T17:31:43.487246+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:52:52.277217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.487533+0000","last_change":"2024-07-04T17:31:42.143453+0000","last_active":"2024-07-04T17:31:43.487533+0000","last_peered":"2024-07-04T17:31:43.487533+0000","last_clean":"2024-07-04T17:31:43.487533+0000","last_became_active":"2024-07-04T17:31:42.142952+0000","last_became_peered":"2024-07-04T17:31:42.142952+0000","last_unstale":"2024-07-04T17:31:43.487533+0000","last_undegraded":"2024-07-04T17:31:43.487533+0000","last_fullsized":"2024-07-04T17:31:43.487533+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:21:03.395798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.486539+0000","last_change":"2024-07-04T17:31:42.141851+0000","last_active":"2024-07-04T17:31:43.486539+0000","last_peered":"2024-07-04T17:31:43.486539+0000","last_clean":"2024-07-04T17:31:43.486539+0000","last_became_active":"2024-07-04T17:31:42.139731+0000","last_became_peered":"2024-07-04T17:31:42.139731+0000","last_unstale":"2024-07-04T17:31:43.486539+0000","last_undegraded":"2024-07-04T17:31:43.486539+0000","last_fullsized":"2024-07-04T17:31:43.486539+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:46:33.797585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.104046+0000","last_change":"2024-07-04T17:31:42.127926+0000","last_active":"2024-07-04T17:31:43.104046+0000","last_peered":"2024-07-04T17:31:43.104046+0000","last_clean":"2024-07-04T17:31:43.104046+0000","last_became_active":"2024-07-04T17:31:42.127596+0000","last_became_peered":"2024-07-04T17:31:42.127596+0000","last_unstale":"2024-07-04T17:31:43.104046+0000","last_undegraded":"2024-07-04T17:31:43.104046+0000","last_fullsized":"2024-07-04T17:31:43.104046+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:20:44.775608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.102444+0000","last_change":"2024-07-04T17:31:42.116203+0000","last_active":"2024-07-04T17:31:43.102444+0000","last_peered":"2024-07-04T17:31:43.102444+0000","last_clean":"2024-07-04T17:31:43.102444+0000","last_became_active":"2024-07-04T17:31:42.115668+0000","last_became_peered":"2024-07-04T17:31:42.115668+0000","last_unstale":"2024-07-04T17:31:43.102444+0000","last_undegraded":"2024-07-04T17:31:43.102444+0000","last_fullsized":"2024-07-04T17:31:43.102444+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:06:36.600932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.482035+0000","last_change":"2024-07-04T17:31:42.153157+0000","last_active":"2024-07-04T17:31:43.482035+0000","last_peered":"2024-07-04T17:31:43.482035+0000","last_clean":"2024-07-04T17:31:43.482035+0000","last_became_active":"2024-07-04T17:31:42.150615+0000","last_became_peered":"2024-07-04T17:31:42.150615+0000","last_unstale":"2024-07-04T17:31:43.482035+0000","last_undegraded":"2024-07-04T17:31:43.482035+0000","last_fullsized":"2024-07-04T17:31:43.482035+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:50:49.706615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.102453+0000","last_change":"2024-07-04T17:31:42.126765+0000","last_active":"2024-07-04T17:31:43.102453+0000","last_peered":"2024-07-04T17:31:43.102453+0000","last_clean":"2024-07-04T17:31:43.102453+0000","last_became_active":"2024-07-04T17:31:42.126443+0000","last_became_peered":"2024-07-04T17:31:42.126443+0000","last_unstale":"2024-07-04T17:31:43.102453+0000","last_undegraded":"2024-07-04T17:31:43.102453+0000","last_fullsized":"2024-07-04T17:31:43.102453+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:34:36.071106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.104147+0000","last_change":"2024-07-04T17:31:42.129239+0000","last_active":"2024-07-04T17:31:43.104147+0000","last_peered":"2024-07-04T17:31:43.104147+0000","last_clean":"2024-07-04T17:31:43.104147+0000","last_became_active":"2024-07-04T17:31:42.127715+0000","last_became_peered":"2024-07-04T17:31:42.127715+0000","last_unstale":"2024-07-04T17:31:43.104147+0000","last_undegraded":"2024-07-04T17:31:43.104147+0000","last_fullsized":"2024-07-04T17:31:43.104147+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:51:01.891207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.484265+0000","last_change":"2024-07-04T17:31:42.142475+0000","last_active":"2024-07-04T17:31:43.484265+0000","last_peered":"2024-07-04T17:31:43.484265+0000","last_clean":"2024-07-04T17:31:43.484265+0000","last_became_active":"2024-07-04T17:31:42.142140+0000","last_became_peered":"2024-07-04T17:31:42.142140+0000","last_unstale":"2024-07-04T17:31:43.484265+0000","last_undegraded":"2024-07-04T17:31:43.484265+0000","last_fullsized":"2024-07-04T17:31:43.484265+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:12:40.862972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.487611+0000","last_change":"2024-07-04T17:31:42.130865+0000","last_active":"2024-07-04T17:31:43.487611+0000","last_peered":"2024-07-04T17:31:43.487611+0000","last_clean":"2024-07-04T17:31:43.487611+0000","last_became_active":"2024-07-04T17:31:42.130064+0000","last_became_peered":"2024-07-04T17:31:42.130064+0000","last_unstale":"2024-07-04T17:31:43.487611+0000","last_undegraded":"2024-07-04T17:31:43.487611+0000","last_fullsized":"2024-07-04T17:31:43.487611+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:54:40.425975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.482511+0000","last_change":"2024-07-04T17:31:42.139174+0000","last_active":"2024-07-04T17:31:43.482511+0000","last_peered":"2024-07-04T17:31:43.482511+0000","last_clean":"2024-07-04T17:31:43.482511+0000","last_became_active":"2024-07-04T17:31:42.138397+0000","last_became_peered":"2024-07-04T17:31:42.138397+0000","last_unstale":"2024-07-04T17:31:43.482511+0000","last_undegraded":"2024-07-04T17:31:43.482511+0000","last_fullsized":"2024-07-04T17:31:43.482511+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:21:29.625345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.482894+0000","last_change":"2024-07-04T17:31:42.111650+0000","last_active":"2024-07-04T17:31:43.482894+0000","last_peered":"2024-07-04T17:31:43.482894+0000","last_clean":"2024-07-04T17:31:43.482894+0000","last_became_active":"2024-07-04T17:31:42.111045+0000","last_became_peered":"2024-07-04T17:31:42.111045+0000","last_unstale":"2024-07-04T17:31:43.482894+0000","last_undegraded":"2024-07-04T17:31:43.482894+0000","last_fullsized":"2024-07-04T17:31:43.482894+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:11:39.466653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.102717+0000","last_change":"2024-07-04T17:31:42.115990+0000","last_active":"2024-07-04T17:31:43.102717+0000","last_peered":"2024-07-04T17:31:43.102717+0000","last_clean":"2024-07-04T17:31:43.102717+0000","last_became_active":"2024-07-04T17:31:42.115413+0000","last_became_peered":"2024-07-04T17:31:42.115413+0000","last_unstale":"2024-07-04T17:31:43.102717+0000","last_undegraded":"2024-07-04T17:31:43.102717+0000","last_fullsized":"2024-07-04T17:31:43.102717+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:36:22.090921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.482053+0000","last_change":"2024-07-04T17:31:42.150544+0000","last_active":"2024-07-04T17:31:43.482053+0000","last_peered":"2024-07-04T17:31:43.482053+0000","last_clean":"2024-07-04T17:31:43.482053+0000","last_became_active":"2024-07-04T17:31:42.150217+0000","last_became_peered":"2024-07-04T17:31:42.150217+0000","last_unstale":"2024-07-04T17:31:43.482053+0000","last_undegraded":"2024-07-04T17:31:43.482053+0000","last_fullsized":"2024-07-04T17:31:43.482053+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:50:45.348242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.480600+0000","last_change":"2024-07-04T17:31:42.137976+0000","last_active":"2024-07-04T17:31:43.480600+0000","last_peered":"2024-07-04T17:31:43.480600+0000","last_clean":"2024-07-04T17:31:43.480600+0000","last_became_active":"2024-07-04T17:31:42.137669+0000","last_became_peered":"2024-07-04T17:31:42.137669+0000","last_unstale":"2024-07-04T17:31:43.480600+0000","last_undegraded":"2024-07-04T17:31:43.480600+0000","last_fullsized":"2024-07-04T17:31:43.480600+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:07:25.661148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.487496+0000","last_change":"2024-07-04T17:31:42.139568+0000","last_active":"2024-07-04T17:31:43.487496+0000","last_peered":"2024-07-04T17:31:43.487496+0000","last_clean":"2024-07-04T17:31:43.487496+0000","last_became_active":"2024-07-04T17:31:42.139247+0000","last_became_peered":"2024-07-04T17:31:42.139247+0000","last_unstale":"2024-07-04T17:31:43.487496+0000","last_undegraded":"2024-07-04T17:31:43.487496+0000","last_fullsized":"2024-07-04T17:31:43.487496+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:48:35.448519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"91'4","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.486350+0000","last_change":"2024-07-04T17:31:41.125254+0000","last_active":"2024-07-04T17:31:43.486350+0000","last_peered":"2024-07-04T17:31:43.486350+0000","last_clean":"2024-07-04T17:31:43.486350+0000","last_became_active":"2024-07-04T17:31:41.119432+0000","last_became_peered":"2024-07-04T17:31:41.119432+0000","last_unstale":"2024-07-04T17:31:43.486350+0000","last_undegraded":"2024-07-04T17:31:43.486350+0000","last_fullsized":"2024-07-04T17:31:43.486350+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:40:27.856616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.485731+0000","last_change":"2024-07-04T17:31:42.143241+0000","last_active":"2024-07-04T17:31:43.485731+0000","last_peered":"2024-07-04T17:31:43.485731+0000","last_clean":"2024-07-04T17:31:43.485731+0000","last_became_active":"2024-07-04T17:31:42.142919+0000","last_became_peered":"2024-07-04T17:31:42.142919+0000","last_unstale":"2024-07-04T17:31:43.485731+0000","last_undegraded":"2024-07-04T17:31:43.485731+0000","last_fullsized":"2024-07-04T17:31:43.485731+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:42:50.042411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.104077+0000","last_change":"2024-07-04T17:31:41.131289+0000","last_active":"2024-07-04T17:31:43.104077+0000","last_peered":"2024-07-04T17:31:43.104077+0000","last_clean":"2024-07-04T17:31:43.104077+0000","last_became_active":"2024-07-04T17:31:41.130956+0000","last_became_peered":"2024-07-04T17:31:41.130956+0000","last_unstale":"2024-07-04T17:31:43.104077+0000","last_undegraded":"2024-07-04T17:31:43.104077+0000","last_fullsized":"2024-07-04T17:31:43.104077+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:11:01.631006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.486259+0000","last_change":"2024-07-04T17:31:42.126808+0000","last_active":"2024-07-04T17:31:43.486259+0000","last_peered":"2024-07-04T17:31:43.486259+0000","last_clean":"2024-07-04T17:31:43.486259+0000","last_became_active":"2024-07-04T17:31:42.126497+0000","last_became_peered":"2024-07-04T17:31:42.126497+0000","last_unstale":"2024-07-04T17:31:43.486259+0000","last_undegraded":"2024-07-04T17:31:43.486259+0000","last_fullsized":"2024-07-04T17:31:43.486259+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:04:06.470763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.103323+0000","last_change":"2024-07-04T17:31:41.129438+0000","last_active":"2024-07-04T17:31:43.103323+0000","last_peered":"2024-07-04T17:31:43.103323+0000","last_clean":"2024-07-04T17:31:43.103323+0000","last_became_active":"2024-07-04T17:31:41.127900+0000","last_became_peered":"2024-07-04T17:31:41.127900+0000","last_unstale":"2024-07-04T17:31:43.103323+0000","last_undegraded":"2024-07-04T17:31:43.103323+0000","last_fullsized":"2024-07-04T17:31:43.103323+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:12:36.772353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.488028+0000","last_change":"2024-07-04T17:31:42.138160+0000","last_active":"2024-07-04T17:31:43.488028+0000","last_peered":"2024-07-04T17:31:43.488028+0000","last_clean":"2024-07-04T17:31:43.488028+0000","last_became_active":"2024-07-04T17:31:42.136593+0000","last_became_peered":"2024-07-04T17:31:42.136593+0000","last_unstale":"2024-07-04T17:31:43.488028+0000","last_undegraded":"2024-07-04T17:31:43.488028+0000","last_fullsized":"2024-07-04T17:31:43.488028+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:28:28.225877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.103585+0000","last_change":"2024-07-04T17:31:41.116679+0000","last_active":"2024-07-04T17:31:43.103585+0000","last_peered":"2024-07-04T17:31:43.103585+0000","last_clean":"2024-07-04T17:31:43.103585+0000","last_became_active":"2024-07-04T17:31:41.116240+0000","last_became_peered":"2024-07-04T17:31:41.116240+0000","last_unstale":"2024-07-04T17:31:43.103585+0000","last_undegraded":"2024-07-04T17:31:43.103585+0000","last_fullsized":"2024-07-04T17:31:43.103585+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:40:21.938736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.484746+0000","last_change":"2024-07-04T17:31:42.132447+0000","last_active":"2024-07-04T17:31:43.484746+0000","last_peered":"2024-07-04T17:31:43.484746+0000","last_clean":"2024-07-04T17:31:43.484746+0000","last_became_active":"2024-07-04T17:31:42.132121+0000","last_became_peered":"2024-07-04T17:31:42.132121+0000","last_unstale":"2024-07-04T17:31:43.484746+0000","last_undegraded":"2024-07-04T17:31:43.484746+0000","last_fullsized":"2024-07-04T17:31:43.484746+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:55:55.603144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.484977+0000","last_change":"2024-07-04T17:31:41.129634+0000","last_active":"2024-07-04T17:31:43.484977+0000","last_peered":"2024-07-04T17:31:43.484977+0000","last_clean":"2024-07-04T17:31:43.484977+0000","last_became_active":"2024-07-04T17:31:41.129326+0000","last_became_peered":"2024-07-04T17:31:41.129326+0000","last_unstale":"2024-07-04T17:31:43.484977+0000","last_undegraded":"2024-07-04T17:31:43.484977+0000","last_fullsized":"2024-07-04T17:31:43.484977+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:37:28.340891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.483445+0000","last_change":"2024-07-04T17:31:42.111322+0000","last_active":"2024-07-04T17:31:43.483445+0000","last_peered":"2024-07-04T17:31:43.483445+0000","last_clean":"2024-07-04T17:31:43.483445+0000","last_became_active":"2024-07-04T17:31:42.110186+0000","last_became_peered":"2024-07-04T17:31:42.110186+0000","last_unstale":"2024-07-04T17:31:43.483445+0000","last_undegraded":"2024-07-04T17:31:43.483445+0000","last_fullsized":"2024-07-04T17:31:43.483445+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:43:36.161285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.103053+0000","last_change":"2024-07-04T17:31:41.118623+0000","last_active":"2024-07-04T17:31:43.103053+0000","last_peered":"2024-07-04T17:31:43.103053+0000","last_clean":"2024-07-04T17:31:43.103053+0000","last_became_active":"2024-07-04T17:31:41.116368+0000","last_became_peered":"2024-07-04T17:31:41.116368+0000","last_unstale":"2024-07-04T17:31:43.103053+0000","last_undegraded":"2024-07-04T17:31:43.103053+0000","last_fullsized":"2024-07-04T17:31:43.103053+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:34:37.181739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.481060+0000","last_change":"2024-07-04T17:31:42.124019+0000","last_active":"2024-07-04T17:31:43.481060+0000","last_peered":"2024-07-04T17:31:43.481060+0000","last_clean":"2024-07-04T17:31:43.481060+0000","last_became_active":"2024-07-04T17:31:42.123684+0000","last_became_peered":"2024-07-04T17:31:42.123684+0000","last_unstale":"2024-07-04T17:31:43.481060+0000","last_undegraded":"2024-07-04T17:31:43.481060+0000","last_fullsized":"2024-07-04T17:31:43.481060+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:12:26.032030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.483071+0000","last_change":"2024-07-04T17:31:41.122775+0000","last_active":"2024-07-04T17:31:43.483071+0000","last_peered":"2024-07-04T17:31:43.483071+0000","last_clean":"2024-07-04T17:31:43.483071+0000","last_became_active":"2024-07-04T17:31:41.121256+0000","last_became_peered":"2024-07-04T17:31:41.121256+0000","last_unstale":"2024-07-04T17:31:43.483071+0000","last_undegraded":"2024-07-04T17:31:43.483071+0000","last_fullsized":"2024-07-04T17:31:43.483071+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:41:24.386164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.481961+0000","last_change":"2024-07-04T17:31:42.138950+0000","last_active":"2024-07-04T17:31:43.481961+0000","last_peered":"2024-07-04T17:31:43.481961+0000","last_clean":"2024-07-04T17:31:43.481961+0000","last_became_active":"2024-07-04T17:31:42.138614+0000","last_became_peered":"2024-07-04T17:31:42.138614+0000","last_unstale":"2024-07-04T17:31:43.481961+0000","last_undegraded":"2024-07-04T17:31:43.481961+0000","last_fullsized":"2024-07-04T17:31:43.481961+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:00:43.285683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.481356+0000","last_change":"2024-07-04T17:31:41.114447+0000","last_active":"2024-07-04T17:31:43.481356+0000","last_peered":"2024-07-04T17:31:43.481356+0000","last_clean":"2024-07-04T17:31:43.481356+0000","last_became_active":"2024-07-04T17:31:41.114127+0000","last_became_peered":"2024-07-04T17:31:41.114127+0000","last_unstale":"2024-07-04T17:31:43.481356+0000","last_undegraded":"2024-07-04T17:31:43.481356+0000","last_fullsized":"2024-07-04T17:31:43.481356+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:21:00.022601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.104006+0000","last_change":"2024-07-04T17:31:42.133055+0000","last_active":"2024-07-04T17:31:43.104006+0000","last_peered":"2024-07-04T17:31:43.104006+0000","last_clean":"2024-07-04T17:31:43.104006+0000","last_became_active":"2024-07-04T17:31:42.131725+0000","last_became_peered":"2024-07-04T17:31:42.131725+0000","last_unstale":"2024-07-04T17:31:43.104006+0000","last_undegraded":"2024-07-04T17:31:43.104006+0000","last_fullsized":"2024-07-04T17:31:43.104006+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:21:59.669492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.104366+0000","last_change":"2024-07-04T17:31:41.131784+0000","last_active":"2024-07-04T17:31:43.104366+0000","last_peered":"2024-07-04T17:31:43.104366+0000","last_clean":"2024-07-04T17:31:43.104366+0000","last_became_active":"2024-07-04T17:31:41.130750+0000","last_became_peered":"2024-07-04T17:31:41.130750+0000","last_unstale":"2024-07-04T17:31:43.104366+0000","last_undegraded":"2024-07-04T17:31:43.104366+0000","last_fullsized":"2024-07-04T17:31:43.104366+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:55:29.200978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.104189+0000","last_change":"2024-07-04T17:31:42.128979+0000","last_active":"2024-07-04T17:31:43.104189+0000","last_peered":"2024-07-04T17:31:43.104189+0000","last_clean":"2024-07-04T17:31:43.104189+0000","last_became_active":"2024-07-04T17:31:42.126866+0000","last_became_peered":"2024-07-04T17:31:42.126866+0000","last_unstale":"2024-07-04T17:31:43.104189+0000","last_undegraded":"2024-07-04T17:31:43.104189+0000","last_fullsized":"2024-07-04T17:31:43.104189+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:06:17.477749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.104184+0000","last_change":"2024-07-04T17:31:41.132406+0000","last_active":"2024-07-04T17:31:43.104184+0000","last_peered":"2024-07-04T17:31:43.104184+0000","last_clean":"2024-07-04T17:31:43.104184+0000","last_became_active":"2024-07-04T17:31:41.130594+0000","last_became_peered":"2024-07-04T17:31:41.130594+0000","last_unstale":"2024-07-04T17:31:43.104184+0000","last_undegraded":"2024-07-04T17:31:43.104184+0000","last_fullsized":"2024-07-04T17:31:43.104184+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:01:17.482235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.486470+0000","last_change":"2024-07-04T17:31:42.136357+0000","last_active":"2024-07-04T17:31:43.486470+0000","last_peered":"2024-07-04T17:31:43.486470+0000","last_clean":"2024-07-04T17:31:43.486470+0000","last_became_active":"2024-07-04T17:31:42.136025+0000","last_became_peered":"2024-07-04T17:31:42.136025+0000","last_unstale":"2024-07-04T17:31:43.486470+0000","last_undegraded":"2024-07-04T17:31:43.486470+0000","last_fullsized":"2024-07-04T17:31:43.486470+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:55:46.913903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.15","version":"91'3","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.483131+0000","last_change":"2024-07-04T17:31:41.124114+0000","last_active":"2024-07-04T17:31:43.483131+0000","last_peered":"2024-07-04T17:31:43.483131+0000","last_clean":"2024-07-04T17:31:43.483131+0000","last_became_active":"2024-07-04T17:31:41.121166+0000","last_became_peered":"2024-07-04T17:31:41.121166+0000","last_unstale":"2024-07-04T17:31:43.483131+0000","last_undegraded":"2024-07-04T17:31:43.483131+0000","last_fullsized":"2024-07-04T17:31:43.483131+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:44:40.148021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.103227+0000","last_change":"2024-07-04T17:31:42.134181+0000","last_active":"2024-07-04T17:31:43.103227+0000","last_peered":"2024-07-04T17:31:43.103227+0000","last_clean":"2024-07-04T17:31:43.103227+0000","last_became_active":"2024-07-04T17:31:42.133671+0000","last_became_peered":"2024-07-04T17:31:42.133671+0000","last_unstale":"2024-07-04T17:31:43.103227+0000","last_undegraded":"2024-07-04T17:31:43.103227+0000","last_fullsized":"2024-07-04T17:31:43.103227+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:13:23.161757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"91'6","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.482777+0000","last_change":"2024-07-04T17:31:41.123522+0000","last_active":"2024-07-04T17:31:43.482777+0000","last_peered":"2024-07-04T17:31:43.482777+0000","last_clean":"2024-07-04T17:31:43.482777+0000","last_became_active":"2024-07-04T17:31:41.121523+0000","last_became_peered":"2024-07-04T17:31:41.121523+0000","last_unstale":"2024-07-04T17:31:43.482777+0000","last_undegraded":"2024-07-04T17:31:43.482777+0000","last_fullsized":"2024-07-04T17:31:43.482777+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:36:59.958154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.102966+0000","last_change":"2024-07-04T17:31:42.116278+0000","last_active":"2024-07-04T17:31:43.102966+0000","last_peered":"2024-07-04T17:31:43.102966+0000","last_clean":"2024-07-04T17:31:43.102966+0000","last_became_active":"2024-07-04T17:31:42.115861+0000","last_became_peered":"2024-07-04T17:31:42.115861+0000","last_unstale":"2024-07-04T17:31:43.102966+0000","last_undegraded":"2024-07-04T17:31:43.102966+0000","last_fullsized":"2024-07-04T17:31:43.102966+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:43:17.950207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.13","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.485539+0000","last_change":"2024-07-04T17:31:41.128711+0000","last_active":"2024-07-04T17:31:43.485539+0000","last_peered":"2024-07-04T17:31:43.485539+0000","last_clean":"2024-07-04T17:31:43.485539+0000","last_became_active":"2024-07-04T17:31:41.127816+0000","last_became_peered":"2024-07-04T17:31:41.127816+0000","last_unstale":"2024-07-04T17:31:43.485539+0000","last_undegraded":"2024-07-04T17:31:43.485539+0000","last_fullsized":"2024-07-04T17:31:43.485539+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:37:51.190424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.483431+0000","last_change":"2024-07-04T17:31:42.111369+0000","last_active":"2024-07-04T17:31:43.483431+0000","last_peered":"2024-07-04T17:31:43.483431+0000","last_clean":"2024-07-04T17:31:43.483431+0000","last_became_active":"2024-07-04T17:31:42.110632+0000","last_became_peered":"2024-07-04T17:31:42.110632+0000","last_unstale":"2024-07-04T17:31:43.483431+0000","last_undegraded":"2024-07-04T17:31:43.483431+0000","last_fullsized":"2024-07-04T17:31:43.483431+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:31:20.968644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.485135+0000","last_change":"2024-07-04T17:31:41.126486+0000","last_active":"2024-07-04T17:31:43.485135+0000","last_peered":"2024-07-04T17:31:43.485135+0000","last_clean":"2024-07-04T17:31:43.485135+0000","last_became_active":"2024-07-04T17:31:41.126142+0000","last_became_peered":"2024-07-04T17:31:41.126142+0000","last_unstale":"2024-07-04T17:31:43.485135+0000","last_undegraded":"2024-07-04T17:31:43.485135+0000","last_fullsized":"2024-07-04T17:31:43.485135+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:25:18.918225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.482432+0000","last_change":"2024-07-04T17:31:42.134513+0000","last_active":"2024-07-04T17:31:43.482432+0000","last_peered":"2024-07-04T17:31:43.482432+0000","last_clean":"2024-07-04T17:31:43.482432+0000","last_became_active":"2024-07-04T17:31:42.134166+0000","last_became_peered":"2024-07-04T17:31:42.134166+0000","last_unstale":"2024-07-04T17:31:43.482432+0000","last_undegraded":"2024-07-04T17:31:43.482432+0000","last_fullsized":"2024-07-04T17:31:43.482432+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:17:13.766129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.10","version":"91'8","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.102648+0000","last_change":"2024-07-04T17:31:41.123889+0000","last_active":"2024-07-04T17:31:43.102648+0000","last_peered":"2024-07-04T17:31:43.102648+0000","last_clean":"2024-07-04T17:31:43.102648+0000","last_became_active":"2024-07-04T17:31:41.123555+0000","last_became_peered":"2024-07-04T17:31:41.123555+0000","last_unstale":"2024-07-04T17:31:43.102648+0000","last_undegraded":"2024-07-04T17:31:43.102648+0000","last_fullsized":"2024-07-04T17:31:43.102648+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:07:24.656814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.104056+0000","last_change":"2024-07-04T17:31:42.117660+0000","last_active":"2024-07-04T17:31:43.104056+0000","last_peered":"2024-07-04T17:31:43.104056+0000","last_clean":"2024-07-04T17:31:43.104056+0000","last_became_active":"2024-07-04T17:31:42.115485+0000","last_became_peered":"2024-07-04T17:31:42.115485+0000","last_unstale":"2024-07-04T17:31:43.104056+0000","last_undegraded":"2024-07-04T17:31:43.104056+0000","last_fullsized":"2024-07-04T17:31:43.104056+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:10:45.367563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.486616+0000","last_change":"2024-07-04T17:31:41.110852+0000","last_active":"2024-07-04T17:31:43.486616+0000","last_peered":"2024-07-04T17:31:43.486616+0000","last_clean":"2024-07-04T17:31:43.486616+0000","last_became_active":"2024-07-04T17:31:41.109092+0000","last_became_peered":"2024-07-04T17:31:41.109092+0000","last_unstale":"2024-07-04T17:31:43.486616+0000","last_undegraded":"2024-07-04T17:31:43.486616+0000","last_fullsized":"2024-07-04T17:31:43.486616+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:47:10.505606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.488119+0000","last_change":"2024-07-04T17:31:42.127586+0000","last_active":"2024-07-04T17:31:43.488119+0000","last_peered":"2024-07-04T17:31:43.488119+0000","last_clean":"2024-07-04T17:31:43.488119+0000","last_became_active":"2024-07-04T17:31:42.127263+0000","last_became_peered":"2024-07-04T17:31:42.127263+0000","last_unstale":"2024-07-04T17:31:43.488119+0000","last_undegraded":"2024-07-04T17:31:43.488119+0000","last_fullsized":"2024-07-04T17:31:43.488119+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:54:41.368680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.104222+0000","last_change":"2024-07-04T17:31:41.127224+0000","last_active":"2024-07-04T17:31:43.104222+0000","last_peered":"2024-07-04T17:31:43.104222+0000","last_clean":"2024-07-04T17:31:43.104222+0000","last_became_active":"2024-07-04T17:31:41.126878+0000","last_became_peered":"2024-07-04T17:31:41.126878+0000","last_unstale":"2024-07-04T17:31:43.104222+0000","last_undegraded":"2024-07-04T17:31:43.104222+0000","last_fullsized":"2024-07-04T17:31:43.104222+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:17:12.761795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.482831+0000","last_change":"2024-07-04T17:31:42.138910+0000","last_active":"2024-07-04T17:31:43.482831+0000","last_peered":"2024-07-04T17:31:43.482831+0000","last_clean":"2024-07-04T17:31:43.482831+0000","last_became_active":"2024-07-04T17:31:42.137936+0000","last_became_peered":"2024-07-04T17:31:42.137936+0000","last_unstale":"2024-07-04T17:31:43.482831+0000","last_undegraded":"2024-07-04T17:31:43.482831+0000","last_fullsized":"2024-07-04T17:31:43.482831+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:30:17.422306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.d","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.483428+0000","last_change":"2024-07-04T17:31:41.125962+0000","last_active":"2024-07-04T17:31:43.483428+0000","last_peered":"2024-07-04T17:31:43.483428+0000","last_clean":"2024-07-04T17:31:43.483428+0000","last_became_active":"2024-07-04T17:31:41.121577+0000","last_became_peered":"2024-07-04T17:31:41.121577+0000","last_unstale":"2024-07-04T17:31:43.483428+0000","last_undegraded":"2024-07-04T17:31:43.483428+0000","last_fullsized":"2024-07-04T17:31:43.483428+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:02:24.336737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.483058+0000","last_change":"2024-07-04T17:31:42.138658+0000","last_active":"2024-07-04T17:31:43.483058+0000","last_peered":"2024-07-04T17:31:43.483058+0000","last_clean":"2024-07-04T17:31:43.483058+0000","last_became_active":"2024-07-04T17:31:42.137836+0000","last_became_peered":"2024-07-04T17:31:42.137836+0000","last_unstale":"2024-07-04T17:31:43.483058+0000","last_undegraded":"2024-07-04T17:31:43.483058+0000","last_fullsized":"2024-07-04T17:31:43.483058+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:32:01.472783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.482935+0000","last_change":"2024-07-04T17:31:41.111490+0000","last_active":"2024-07-04T17:31:43.482935+0000","last_peered":"2024-07-04T17:31:43.482935+0000","last_clean":"2024-07-04T17:31:43.482935+0000","last_became_active":"2024-07-04T17:31:41.110881+0000","last_became_peered":"2024-07-04T17:31:41.110881+0000","last_unstale":"2024-07-04T17:31:43.482935+0000","last_undegraded":"2024-07-04T17:31:43.482935+0000","last_fullsized":"2024-07-04T17:31:43.482935+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:44:12.481565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.485214+0000","last_change":"2024-07-04T17:31:42.144839+0000","last_active":"2024-07-04T17:31:43.485214+0000","last_peered":"2024-07-04T17:31:43.485214+0000","last_clean":"2024-07-04T17:31:43.485214+0000","last_became_active":"2024-07-04T17:31:42.142261+0000","last_became_peered":"2024-07-04T17:31:42.142261+0000","last_unstale":"2024-07-04T17:31:43.485214+0000","last_undegraded":"2024-07-04T17:31:43.485214+0000","last_fullsized":"2024-07-04T17:31:43.485214+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:44:20.205264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.485739+0000","last_change":"2024-07-04T17:31:41.128172+0000","last_active":"2024-07-04T17:31:43.485739+0000","last_peered":"2024-07-04T17:31:43.485739+0000","last_clean":"2024-07-04T17:31:43.485739+0000","last_became_active":"2024-07-04T17:31:41.127350+0000","last_became_peered":"2024-07-04T17:31:41.127350+0000","last_unstale":"2024-07-04T17:31:43.485739+0000","last_undegraded":"2024-07-04T17:31:43.485739+0000","last_fullsized":"2024-07-04T17:31:43.485739+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:24:57.684170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.483774+0000","last_change":"2024-07-04T17:31:42.148767+0000","last_active":"2024-07-04T17:31:43.483774+0000","last_peered":"2024-07-04T17:31:43.483774+0000","last_clean":"2024-07-04T17:31:43.483774+0000","last_became_active":"2024-07-04T17:31:42.148091+0000","last_became_peered":"2024-07-04T17:31:42.148091+0000","last_unstale":"2024-07-04T17:31:43.483774+0000","last_undegraded":"2024-07-04T17:31:43.483774+0000","last_fullsized":"2024-07-04T17:31:43.483774+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:42:04.145597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.103220+0000","last_change":"2024-07-04T17:31:41.131853+0000","last_active":"2024-07-04T17:31:43.103220+0000","last_peered":"2024-07-04T17:31:43.103220+0000","last_clean":"2024-07-04T17:31:43.103220+0000","last_became_active":"2024-07-04T17:31:41.129778+0000","last_became_peered":"2024-07-04T17:31:41.129778+0000","last_unstale":"2024-07-04T17:31:43.103220+0000","last_undegraded":"2024-07-04T17:31:43.103220+0000","last_fullsized":"2024-07-04T17:31:43.103220+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:59:40.102134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.481262+0000","last_change":"2024-07-04T17:31:42.111799+0000","last_active":"2024-07-04T17:31:43.481262+0000","last_peered":"2024-07-04T17:31:43.481262+0000","last_clean":"2024-07-04T17:31:43.481262+0000","last_became_active":"2024-07-04T17:31:42.110114+0000","last_became_peered":"2024-07-04T17:31:42.110114+0000","last_unstale":"2024-07-04T17:31:43.481262+0000","last_undegraded":"2024-07-04T17:31:43.481262+0000","last_fullsized":"2024-07-04T17:31:43.481262+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:45:25.449722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.483393+0000","last_change":"2024-07-04T17:31:41.120856+0000","last_active":"2024-07-04T17:31:43.483393+0000","last_peered":"2024-07-04T17:31:43.483393+0000","last_clean":"2024-07-04T17:31:43.483393+0000","last_became_active":"2024-07-04T17:31:41.120338+0000","last_became_peered":"2024-07-04T17:31:41.120338+0000","last_unstale":"2024-07-04T17:31:43.483393+0000","last_undegraded":"2024-07-04T17:31:43.483393+0000","last_fullsized":"2024-07-04T17:31:43.483393+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:14:01.301278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.482906+0000","last_change":"2024-07-04T17:31:42.124705+0000","last_active":"2024-07-04T17:31:43.482906+0000","last_peered":"2024-07-04T17:31:43.482906+0000","last_clean":"2024-07-04T17:31:43.482906+0000","last_became_active":"2024-07-04T17:31:42.124266+0000","last_became_peered":"2024-07-04T17:31:42.124266+0000","last_unstale":"2024-07-04T17:31:43.482906+0000","last_undegraded":"2024-07-04T17:31:43.482906+0000","last_fullsized":"2024-07-04T17:31:43.482906+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:18:01.138996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.481382+0000","last_change":"2024-07-04T17:31:41.114344+0000","last_active":"2024-07-04T17:31:43.481382+0000","last_peered":"2024-07-04T17:31:43.481382+0000","last_clean":"2024-07-04T17:31:43.481382+0000","last_became_active":"2024-07-04T17:31:41.114006+0000","last_became_peered":"2024-07-04T17:31:41.114006+0000","last_unstale":"2024-07-04T17:31:43.481382+0000","last_undegraded":"2024-07-04T17:31:43.481382+0000","last_fullsized":"2024-07-04T17:31:43.481382+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:57:55.654579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.484777+0000","last_change":"2024-07-04T17:31:42.142194+0000","last_active":"2024-07-04T17:31:43.484777+0000","last_peered":"2024-07-04T17:31:43.484777+0000","last_clean":"2024-07-04T17:31:43.484777+0000","last_became_active":"2024-07-04T17:31:42.139517+0000","last_became_peered":"2024-07-04T17:31:42.139517+0000","last_unstale":"2024-07-04T17:31:43.484777+0000","last_undegraded":"2024-07-04T17:31:43.484777+0000","last_fullsized":"2024-07-04T17:31:43.484777+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:12:09.965552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.488088+0000","last_change":"2024-07-04T17:31:41.126950+0000","last_active":"2024-07-04T17:31:43.488088+0000","last_peered":"2024-07-04T17:31:43.488088+0000","last_clean":"2024-07-04T17:31:43.488088+0000","last_became_active":"2024-07-04T17:31:41.126611+0000","last_became_peered":"2024-07-04T17:31:41.126611+0000","last_unstale":"2024-07-04T17:31:43.488088+0000","last_undegraded":"2024-07-04T17:31:43.488088+0000","last_fullsized":"2024-07-04T17:31:43.488088+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:27:51.965765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.486898+0000","last_change":"2024-07-04T17:31:42.107634+0000","last_active":"2024-07-04T17:31:43.486898+0000","last_peered":"2024-07-04T17:31:43.486898+0000","last_clean":"2024-07-04T17:31:43.486898+0000","last_became_active":"2024-07-04T17:31:42.107327+0000","last_became_peered":"2024-07-04T17:31:42.107327+0000","last_unstale":"2024-07-04T17:31:43.486898+0000","last_undegraded":"2024-07-04T17:31:43.486898+0000","last_fullsized":"2024-07-04T17:31:43.486898+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:57:08.828739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"92'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.487379+0000","last_change":"2024-07-04T17:31:41.126383+0000","last_active":"2024-07-04T17:31:43.487379+0000","last_peered":"2024-07-04T17:31:43.487379+0000","last_clean":"2024-07-04T17:31:43.487379+0000","last_became_active":"2024-07-04T17:31:41.126045+0000","last_became_peered":"2024-07-04T17:31:41.126045+0000","last_unstale":"2024-07-04T17:31:43.487379+0000","last_undegraded":"2024-07-04T17:31:43.487379+0000","last_fullsized":"2024-07-04T17:31:43.487379+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:57:59.125923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.482709+0000","last_change":"2024-07-04T17:31:42.143202+0000","last_active":"2024-07-04T17:31:43.482709+0000","last_peered":"2024-07-04T17:31:43.482709+0000","last_clean":"2024-07-04T17:31:43.482709+0000","last_became_active":"2024-07-04T17:31:42.141656+0000","last_became_peered":"2024-07-04T17:31:42.141656+0000","last_unstale":"2024-07-04T17:31:43.482709+0000","last_undegraded":"2024-07-04T17:31:43.482709+0000","last_fullsized":"2024-07-04T17:31:43.482709+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:19:20.240973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.483356+0000","last_change":"2024-07-04T17:31:41.123150+0000","last_active":"2024-07-04T17:31:43.483356+0000","last_peered":"2024-07-04T17:31:43.483356+0000","last_clean":"2024-07-04T17:31:43.483356+0000","last_became_active":"2024-07-04T17:31:41.120767+0000","last_became_peered":"2024-07-04T17:31:41.120767+0000","last_unstale":"2024-07-04T17:31:43.483356+0000","last_undegraded":"2024-07-04T17:31:43.483356+0000","last_fullsized":"2024-07-04T17:31:43.483356+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:42:19.857048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.483214+0000","last_change":"2024-07-04T17:31:42.109920+0000","last_active":"2024-07-04T17:31:43.483214+0000","last_peered":"2024-07-04T17:31:43.483214+0000","last_clean":"2024-07-04T17:31:43.483214+0000","last_became_active":"2024-07-04T17:31:42.109357+0000","last_became_peered":"2024-07-04T17:31:42.109357+0000","last_unstale":"2024-07-04T17:31:43.483214+0000","last_undegraded":"2024-07-04T17:31:43.483214+0000","last_fullsized":"2024-07-04T17:31:43.483214+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:58:16.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.103726+0000","last_change":"2024-07-04T17:31:41.119885+0000","last_active":"2024-07-04T17:31:43.103726+0000","last_peered":"2024-07-04T17:31:43.103726+0000","last_clean":"2024-07-04T17:31:43.103726+0000","last_became_active":"2024-07-04T17:31:41.116720+0000","last_became_peered":"2024-07-04T17:31:41.116720+0000","last_unstale":"2024-07-04T17:31:43.103726+0000","last_undegraded":"2024-07-04T17:31:43.103726+0000","last_fullsized":"2024-07-04T17:31:43.103726+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:30:49.905769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.484989+0000","last_change":"2024-07-04T17:31:42.125396+0000","last_active":"2024-07-04T17:31:43.484989+0000","last_peered":"2024-07-04T17:31:43.484989+0000","last_clean":"2024-07-04T17:31:43.484989+0000","last_became_active":"2024-07-04T17:31:42.123980+0000","last_became_peered":"2024-07-04T17:31:42.123980+0000","last_unstale":"2024-07-04T17:31:43.484989+0000","last_undegraded":"2024-07-04T17:31:43.484989+0000","last_fullsized":"2024-07-04T17:31:43.484989+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:59:35.007753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.485892+0000","last_change":"2024-07-04T17:31:41.128454+0000","last_active":"2024-07-04T17:31:43.485892+0000","last_peered":"2024-07-04T17:31:43.485892+0000","last_clean":"2024-07-04T17:31:43.485892+0000","last_became_active":"2024-07-04T17:31:41.127785+0000","last_became_peered":"2024-07-04T17:31:41.127785+0000","last_unstale":"2024-07-04T17:31:43.485892+0000","last_undegraded":"2024-07-04T17:31:43.485892+0000","last_fullsized":"2024-07-04T17:31:43.485892+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:48:22.712062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.483773+0000","last_change":"2024-07-04T17:31:42.111629+0000","last_active":"2024-07-04T17:31:43.483773+0000","last_peered":"2024-07-04T17:31:43.483773+0000","last_clean":"2024-07-04T17:31:43.483773+0000","last_became_active":"2024-07-04T17:31:42.110978+0000","last_became_peered":"2024-07-04T17:31:42.110978+0000","last_unstale":"2024-07-04T17:31:43.483773+0000","last_undegraded":"2024-07-04T17:31:43.483773+0000","last_fullsized":"2024-07-04T17:31:43.483773+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:12:01.825579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.484487+0000","last_change":"2024-07-04T17:31:41.124595+0000","last_active":"2024-07-04T17:31:43.484487+0000","last_peered":"2024-07-04T17:31:43.484487+0000","last_clean":"2024-07-04T17:31:43.484487+0000","last_became_active":"2024-07-04T17:31:41.124315+0000","last_became_peered":"2024-07-04T17:31:41.124315+0000","last_unstale":"2024-07-04T17:31:43.484487+0000","last_undegraded":"2024-07-04T17:31:43.484487+0000","last_fullsized":"2024-07-04T17:31:43.484487+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:10:03.734241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.481308+0000","last_change":"2024-07-04T17:31:42.112865+0000","last_active":"2024-07-04T17:31:43.481308+0000","last_peered":"2024-07-04T17:31:43.481308+0000","last_clean":"2024-07-04T17:31:43.481308+0000","last_became_active":"2024-07-04T17:31:42.109293+0000","last_became_peered":"2024-07-04T17:31:42.109293+0000","last_unstale":"2024-07-04T17:31:43.481308+0000","last_undegraded":"2024-07-04T17:31:43.481308+0000","last_fullsized":"2024-07-04T17:31:43.481308+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:07:14.667638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.102591+0000","last_change":"2024-07-04T17:31:41.116814+0000","last_active":"2024-07-04T17:31:43.102591+0000","last_peered":"2024-07-04T17:31:43.102591+0000","last_clean":"2024-07-04T17:31:43.102591+0000","last_became_active":"2024-07-04T17:31:41.116425+0000","last_became_peered":"2024-07-04T17:31:41.116425+0000","last_unstale":"2024-07-04T17:31:43.102591+0000","last_undegraded":"2024-07-04T17:31:43.102591+0000","last_fullsized":"2024-07-04T17:31:43.102591+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:18:00.134663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.482693+0000","last_change":"2024-07-04T17:31:42.145454+0000","last_active":"2024-07-04T17:31:43.482693+0000","last_peered":"2024-07-04T17:31:43.482693+0000","last_clean":"2024-07-04T17:31:43.482693+0000","last_became_active":"2024-07-04T17:31:42.144919+0000","last_became_peered":"2024-07-04T17:31:42.144919+0000","last_unstale":"2024-07-04T17:31:43.482693+0000","last_undegraded":"2024-07-04T17:31:43.482693+0000","last_fullsized":"2024-07-04T17:31:43.482693+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:30:26.887002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.481559+0000","last_change":"2024-07-04T17:31:41.111066+0000","last_active":"2024-07-04T17:31:43.481559+0000","last_peered":"2024-07-04T17:31:43.481559+0000","last_clean":"2024-07-04T17:31:43.481559+0000","last_became_active":"2024-07-04T17:31:41.110689+0000","last_became_peered":"2024-07-04T17:31:41.110689+0000","last_unstale":"2024-07-04T17:31:43.481559+0000","last_undegraded":"2024-07-04T17:31:43.481559+0000","last_fullsized":"2024-07-04T17:31:43.481559+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:24:51.654346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.480874+0000","last_change":"2024-07-04T17:31:42.124940+0000","last_active":"2024-07-04T17:31:43.480874+0000","last_peered":"2024-07-04T17:31:43.480874+0000","last_clean":"2024-07-04T17:31:43.480874+0000","last_became_active":"2024-07-04T17:31:42.124608+0000","last_became_peered":"2024-07-04T17:31:42.124608+0000","last_unstale":"2024-07-04T17:31:43.480874+0000","last_undegraded":"2024-07-04T17:31:43.480874+0000","last_fullsized":"2024-07-04T17:31:43.480874+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:11:02.635340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.103692+0000","last_change":"2024-07-04T17:31:41.125061+0000","last_active":"2024-07-04T17:31:43.103692+0000","last_peered":"2024-07-04T17:31:43.103692+0000","last_clean":"2024-07-04T17:31:43.103692+0000","last_became_active":"2024-07-04T17:31:41.122766+0000","last_became_peered":"2024-07-04T17:31:41.122766+0000","last_unstale":"2024-07-04T17:31:43.103692+0000","last_undegraded":"2024-07-04T17:31:43.103692+0000","last_fullsized":"2024-07-04T17:31:43.103692+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:00:42.281350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"80'87","reported_seq":356,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.480967+0000","last_change":"2024-07-04T17:26:04.953559+0000","last_active":"2024-07-04T17:31:43.480967+0000","last_peered":"2024-07-04T17:31:43.480967+0000","last_clean":"2024-07-04T17:31:43.480967+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:31:43.480967+0000","last_undegraded":"2024-07-04T17:31:43.480967+0000","last_fullsized":"2024-07-04T17:31:43.480967+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_clean_scrub_stamp":"2024-07-04T17:26:03.933432+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.064181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":356,"num_read_kb":926,"num_write":139,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":356,"num_read_kb":926,"num_write":139,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738436,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36264,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335576,"statfs":{"total":96636764160,"available":96599629824,"internally_reserved":0,"allocated":937984,"data_stored":365581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738437,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36848,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334992,"statfs":{"total":96636764160,"available":96599031808,"internally_reserved":0,"allocated":1544192,"data_stored":957117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36852,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334988,"statfs":{"total":96636764160,"available":96599027712,"internally_reserved":0,"allocated":1548288,"data_stored":957687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:31:46.841 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:31:47.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.170+0000 7fb936400640 1 -- 172.21.15.42:0/923855447 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb93006d580 msgr2=0x7fb93006c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:47.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.170+0000 7fb936400640 1 --2- 172.21.15.42:0/923855447 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb93006d580 0x7fb93006c2c0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fb9240097b0 tx=0x7fb92402a4a0 comp rx=0 tx=0).stop 2024-07-04T17:31:47.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.171+0000 7fb936400640 1 -- 172.21.15.42:0/923855447 shutdown_connections 2024-07-04T17:31:47.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.171+0000 7fb936400640 1 --2- 172.21.15.42:0/923855447 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb93006d580 0x7fb93006c2c0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:47.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.171+0000 7fb936400640 1 -- 172.21.15.42:0/923855447 >> 172.21.15.42:0/923855447 conn(0x7fb9300bebc0 msgr2=0x7fb9300befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:47.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.171+0000 7fb936400640 1 -- 172.21.15.42:0/923855447 shutdown_connections 2024-07-04T17:31:47.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.171+0000 7fb936400640 1 -- 172.21.15.42:0/923855447 wait complete. 2024-07-04T17:31:47.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.173+0000 7fb936400640 1 Processor -- start 2024-07-04T17:31:47.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.173+0000 7fb936400640 1 -- start start 2024-07-04T17:31:47.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.174+0000 7fb936400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb93006d580 0x7fb9300b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:47.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.174+0000 7fb936400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb9300b2810 con 0x7fb93006d580 2024-07-04T17:31:47.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.175+0000 7fb935000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb93006d580 0x7fb9300b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:47.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.175+0000 7fb935000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb93006d580 0x7fb9300b0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54320/0 (socket says 172.21.15.42:54320) 2024-07-04T17:31:47.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.175+0000 7fb935000640 1 -- 172.21.15.42:0/1962896340 learned_addr learned my addr 172.21.15.42:0/1962896340 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:47.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.175+0000 7fb935000640 1 -- 172.21.15.42:0/1962896340 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93013cea0 con 0x7fb93006d580 2024-07-04T17:31:47.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.176+0000 7fb935000640 1 --2- 172.21.15.42:0/1962896340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb93006d580 0x7fb9300b0bd0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fb9240098d0 tx=0x7fb924033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:47.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.176+0000 7fb92cc00640 1 -- 172.21.15.42:0/1962896340 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb924031a90 con 0x7fb93006d580 2024-07-04T17:31:47.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.176+0000 7fb92cc00640 1 -- 172.21.15.42:0/1962896340 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb924031d20 con 0x7fb93006d580 2024-07-04T17:31:47.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.176+0000 7fb936400640 1 -- 172.21.15.42:0/1962896340 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9300af0c0 con 0x7fb93006d580 2024-07-04T17:31:47.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.176+0000 7fb92cc00640 1 -- 172.21.15.42:0/1962896340 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb924033070 con 0x7fb93006d580 2024-07-04T17:31:47.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.176+0000 7fb936400640 1 -- 172.21.15.42:0/1962896340 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9300b1110 con 0x7fb93006d580 2024-07-04T17:31:47.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.177+0000 7fb936400640 1 -- 172.21.15.42:0/1962896340 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb93006c2c0 con 0x7fb93006d580 2024-07-04T17:31:47.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.178+0000 7fb92cc00640 1 -- 172.21.15.42:0/1962896340 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7fb924033210 con 0x7fb93006d580 2024-07-04T17:31:47.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.178+0000 7fb92cc00640 1 --2- 172.21.15.42:0/1962896340 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fb91003d3d0 0x7fb91003f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:47.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.178+0000 7fb92cc00640 1 -- 172.21.15.42:0/1962896340 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 3439+0+0 (secure 0 0 0) 0x7fb924072d00 con 0x7fb93006d580 2024-07-04T17:31:47.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.178+0000 7fb92fe00640 1 --2- 172.21.15.42:0/1962896340 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fb91003d3d0 0x7fb91003f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:47.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.179+0000 7fb92fe00640 1 --2- 172.21.15.42:0/1962896340 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fb91003d3d0 0x7fb91003f8e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb91c000f60 tx=0x7fb91c007a30 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:47.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.183+0000 7fb92cc00640 1 -- 172.21.15.42:0/1962896340 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fb92403f030 con 0x7fb93006d580 2024-07-04T17:31:47.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.391+0000 7fb936400640 1 -- 172.21.15.42:0/1962896340 --> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb9300ae4e0 con 0x7fb91003d3d0 2024-07-04T17:31:47.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.406+0000 7fb92cc00640 1 -- 172.21.15.42:0/1962896340 <== mgr.5173 v2:172.21.15.42:6834/1139872311 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246080 (secure 0 0 0) 0x7fb9300ae4e0 con 0x7fb91003d3d0 2024-07-04T17:31:47.409 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:31:47.412 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:31:47.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.413+0000 7fb936400640 1 -- 172.21.15.42:0/1962896340 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fb91003d3d0 msgr2=0x7fb91003f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:47.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.413+0000 7fb936400640 1 --2- 172.21.15.42:0/1962896340 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fb91003d3d0 0x7fb91003f8e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb91c000f60 tx=0x7fb91c007a30 comp rx=0 tx=0).stop 2024-07-04T17:31:47.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.413+0000 7fb936400640 1 -- 172.21.15.42:0/1962896340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb93006d580 msgr2=0x7fb9300b0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:47.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.413+0000 7fb936400640 1 --2- 172.21.15.42:0/1962896340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb93006d580 0x7fb9300b0bd0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fb9240098d0 tx=0x7fb924033c10 comp rx=0 tx=0).stop 2024-07-04T17:31:47.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.413+0000 7fb936400640 1 -- 172.21.15.42:0/1962896340 shutdown_connections 2024-07-04T17:31:47.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.413+0000 7fb936400640 1 --2- 172.21.15.42:0/1962896340 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fb91003d3d0 0x7fb91003f8e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:47.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.413+0000 7fb936400640 1 --2- 172.21.15.42:0/1962896340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb93006d580 0x7fb9300b0bd0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:47.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.413+0000 7fb936400640 1 -- 172.21.15.42:0/1962896340 >> 172.21.15.42:0/1962896340 conn(0x7fb9300bebc0 msgr2=0x7fb9300bcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:47.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.413+0000 7fb936400640 1 -- 172.21.15.42:0/1962896340 shutdown_connections 2024-07-04T17:31:47.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.413+0000 7fb936400640 1 -- 172.21.15.42:0/1962896340 wait complete. 2024-07-04T17:31:47.440 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-07-04T17:31:46.437949+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":356,"num_read_kb":926,"num_write":184,"num_write_kb":1120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126,"ondisk_log_size":126,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":109964,"kb_used_data":3936,"kb_used_omap":4,"kb_used_meta":105979,"kb_avail":283005556,"statfs":{"total":289910292480,"available":289797689344,"internally_reserved":0,"allocated":4030464,"data_stored":2280385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":108522831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001977"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.485394+0000","last_change":"2024-07-04T17:31:42.147550+0000","last_active":"2024-07-04T17:31:43.485394+0000","last_peered":"2024-07-04T17:31:43.485394+0000","last_clean":"2024-07-04T17:31:43.485394+0000","last_became_active":"2024-07-04T17:31:42.146638+0000","last_became_peered":"2024-07-04T17:31:42.146638+0000","last_unstale":"2024-07-04T17:31:43.485394+0000","last_undegraded":"2024-07-04T17:31:43.485394+0000","last_fullsized":"2024-07-04T17:31:43.485394+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:17:39.808949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.103805+0000","last_change":"2024-07-04T17:31:42.132420+0000","last_active":"2024-07-04T17:31:43.103805+0000","last_peered":"2024-07-04T17:31:43.103805+0000","last_clean":"2024-07-04T17:31:43.103805+0000","last_became_active":"2024-07-04T17:31:42.132031+0000","last_became_peered":"2024-07-04T17:31:42.132031+0000","last_unstale":"2024-07-04T17:31:43.103805+0000","last_undegraded":"2024-07-04T17:31:43.103805+0000","last_fullsized":"2024-07-04T17:31:43.103805+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:41:23.204256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.487015+0000","last_change":"2024-07-04T17:31:42.146020+0000","last_active":"2024-07-04T17:31:43.487015+0000","last_peered":"2024-07-04T17:31:43.487015+0000","last_clean":"2024-07-04T17:31:43.487015+0000","last_became_active":"2024-07-04T17:31:42.143682+0000","last_became_peered":"2024-07-04T17:31:42.143682+0000","last_unstale":"2024-07-04T17:31:43.487015+0000","last_undegraded":"2024-07-04T17:31:43.487015+0000","last_fullsized":"2024-07-04T17:31:43.487015+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:06:33.198375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.482306+0000","last_change":"2024-07-04T17:31:42.148476+0000","last_active":"2024-07-04T17:31:43.482306+0000","last_peered":"2024-07-04T17:31:43.482306+0000","last_clean":"2024-07-04T17:31:43.482306+0000","last_became_active":"2024-07-04T17:31:42.148083+0000","last_became_peered":"2024-07-04T17:31:42.148083+0000","last_unstale":"2024-07-04T17:31:43.482306+0000","last_undegraded":"2024-07-04T17:31:43.482306+0000","last_fullsized":"2024-07-04T17:31:43.482306+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:19:47.089588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.103647+0000","last_change":"2024-07-04T17:31:42.118035+0000","last_active":"2024-07-04T17:31:43.103647+0000","last_peered":"2024-07-04T17:31:43.103647+0000","last_clean":"2024-07-04T17:31:43.103647+0000","last_became_active":"2024-07-04T17:31:42.117722+0000","last_became_peered":"2024-07-04T17:31:42.117722+0000","last_unstale":"2024-07-04T17:31:43.103647+0000","last_undegraded":"2024-07-04T17:31:43.103647+0000","last_fullsized":"2024-07-04T17:31:43.103647+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:17:18.803619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.484697+0000","last_change":"2024-07-04T17:31:42.146322+0000","last_active":"2024-07-04T17:31:43.484697+0000","last_peered":"2024-07-04T17:31:43.484697+0000","last_clean":"2024-07-04T17:31:43.484697+0000","last_became_active":"2024-07-04T17:31:42.146004+0000","last_became_peered":"2024-07-04T17:31:42.146004+0000","last_unstale":"2024-07-04T17:31:43.484697+0000","last_undegraded":"2024-07-04T17:31:43.484697+0000","last_fullsized":"2024-07-04T17:31:43.484697+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:10:27.612193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.102518+0000","last_change":"2024-07-04T17:31:42.131777+0000","last_active":"2024-07-04T17:31:43.102518+0000","last_peered":"2024-07-04T17:31:43.102518+0000","last_clean":"2024-07-04T17:31:43.102518+0000","last_became_active":"2024-07-04T17:31:42.127915+0000","last_became_peered":"2024-07-04T17:31:42.127915+0000","last_unstale":"2024-07-04T17:31:43.102518+0000","last_undegraded":"2024-07-04T17:31:43.102518+0000","last_fullsized":"2024-07-04T17:31:43.102518+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:14:13.315623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.103763+0000","last_change":"2024-07-04T17:31:42.127048+0000","last_active":"2024-07-04T17:31:43.103763+0000","last_peered":"2024-07-04T17:31:43.103763+0000","last_clean":"2024-07-04T17:31:43.103763+0000","last_became_active":"2024-07-04T17:31:42.126695+0000","last_became_peered":"2024-07-04T17:31:42.126695+0000","last_unstale":"2024-07-04T17:31:43.103763+0000","last_undegraded":"2024-07-04T17:31:43.103763+0000","last_fullsized":"2024-07-04T17:31:43.103763+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:46:15.072312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.485609+0000","last_change":"2024-07-04T17:31:42.134470+0000","last_active":"2024-07-04T17:31:43.485609+0000","last_peered":"2024-07-04T17:31:43.485609+0000","last_clean":"2024-07-04T17:31:43.485609+0000","last_became_active":"2024-07-04T17:31:42.134096+0000","last_became_peered":"2024-07-04T17:31:42.134096+0000","last_unstale":"2024-07-04T17:31:43.485609+0000","last_undegraded":"2024-07-04T17:31:43.485609+0000","last_fullsized":"2024-07-04T17:31:43.485609+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:07:42.785561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.483766+0000","last_change":"2024-07-04T17:31:42.109492+0000","last_active":"2024-07-04T17:31:43.483766+0000","last_peered":"2024-07-04T17:31:43.483766+0000","last_clean":"2024-07-04T17:31:43.483766+0000","last_became_active":"2024-07-04T17:31:42.108502+0000","last_became_peered":"2024-07-04T17:31:42.108502+0000","last_unstale":"2024-07-04T17:31:43.483766+0000","last_undegraded":"2024-07-04T17:31:43.483766+0000","last_fullsized":"2024-07-04T17:31:43.483766+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:39:41.707779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.482991+0000","last_change":"2024-07-04T17:31:42.138904+0000","last_active":"2024-07-04T17:31:43.482991+0000","last_peered":"2024-07-04T17:31:43.482991+0000","last_clean":"2024-07-04T17:31:43.482991+0000","last_became_active":"2024-07-04T17:31:42.138265+0000","last_became_peered":"2024-07-04T17:31:42.138265+0000","last_unstale":"2024-07-04T17:31:43.482991+0000","last_undegraded":"2024-07-04T17:31:43.482991+0000","last_fullsized":"2024-07-04T17:31:43.482991+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:26.894424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.482403+0000","last_change":"2024-07-04T17:31:42.110720+0000","last_active":"2024-07-04T17:31:43.482403+0000","last_peered":"2024-07-04T17:31:43.482403+0000","last_clean":"2024-07-04T17:31:43.482403+0000","last_became_active":"2024-07-04T17:31:42.110428+0000","last_became_peered":"2024-07-04T17:31:42.110428+0000","last_unstale":"2024-07-04T17:31:43.482403+0000","last_undegraded":"2024-07-04T17:31:43.482403+0000","last_fullsized":"2024-07-04T17:31:43.482403+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:03:49.795002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.103910+0000","last_change":"2024-07-04T17:31:42.132759+0000","last_active":"2024-07-04T17:31:43.103910+0000","last_peered":"2024-07-04T17:31:43.103910+0000","last_clean":"2024-07-04T17:31:43.103910+0000","last_became_active":"2024-07-04T17:31:42.132493+0000","last_became_peered":"2024-07-04T17:31:42.132493+0000","last_unstale":"2024-07-04T17:31:43.103910+0000","last_undegraded":"2024-07-04T17:31:43.103910+0000","last_fullsized":"2024-07-04T17:31:43.103910+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:41:07.944963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.484272+0000","last_change":"2024-07-04T17:31:42.137702+0000","last_active":"2024-07-04T17:31:43.484272+0000","last_peered":"2024-07-04T17:31:43.484272+0000","last_clean":"2024-07-04T17:31:43.484272+0000","last_became_active":"2024-07-04T17:31:42.134081+0000","last_became_peered":"2024-07-04T17:31:42.134081+0000","last_unstale":"2024-07-04T17:31:43.484272+0000","last_undegraded":"2024-07-04T17:31:43.484272+0000","last_fullsized":"2024-07-04T17:31:43.484272+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:26:29.769982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.481505+0000","last_change":"2024-07-04T17:31:42.126095+0000","last_active":"2024-07-04T17:31:43.481505+0000","last_peered":"2024-07-04T17:31:43.481505+0000","last_clean":"2024-07-04T17:31:43.481505+0000","last_became_active":"2024-07-04T17:31:42.125783+0000","last_became_peered":"2024-07-04T17:31:42.125783+0000","last_unstale":"2024-07-04T17:31:43.481505+0000","last_undegraded":"2024-07-04T17:31:43.481505+0000","last_fullsized":"2024-07-04T17:31:43.481505+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:01:18.486569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.481014+0000","last_change":"2024-07-04T17:31:42.124307+0000","last_active":"2024-07-04T17:31:43.481014+0000","last_peered":"2024-07-04T17:31:43.481014+0000","last_clean":"2024-07-04T17:31:43.481014+0000","last_became_active":"2024-07-04T17:31:42.121958+0000","last_became_peered":"2024-07-04T17:31:42.121958+0000","last_unstale":"2024-07-04T17:31:43.481014+0000","last_undegraded":"2024-07-04T17:31:43.481014+0000","last_fullsized":"2024-07-04T17:31:43.481014+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:14:03.072947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.487920+0000","last_change":"2024-07-04T17:31:42.127722+0000","last_active":"2024-07-04T17:31:43.487920+0000","last_peered":"2024-07-04T17:31:43.487920+0000","last_clean":"2024-07-04T17:31:43.487920+0000","last_became_active":"2024-07-04T17:31:42.127401+0000","last_became_peered":"2024-07-04T17:31:42.127401+0000","last_unstale":"2024-07-04T17:31:43.487920+0000","last_undegraded":"2024-07-04T17:31:43.487920+0000","last_fullsized":"2024-07-04T17:31:43.487920+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:08:43.721732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.487246+0000","last_change":"2024-07-04T17:31:42.131382+0000","last_active":"2024-07-04T17:31:43.487246+0000","last_peered":"2024-07-04T17:31:43.487246+0000","last_clean":"2024-07-04T17:31:43.487246+0000","last_became_active":"2024-07-04T17:31:42.131053+0000","last_became_peered":"2024-07-04T17:31:42.131053+0000","last_unstale":"2024-07-04T17:31:43.487246+0000","last_undegraded":"2024-07-04T17:31:43.487246+0000","last_fullsized":"2024-07-04T17:31:43.487246+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:52:52.277217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.487533+0000","last_change":"2024-07-04T17:31:42.143453+0000","last_active":"2024-07-04T17:31:43.487533+0000","last_peered":"2024-07-04T17:31:43.487533+0000","last_clean":"2024-07-04T17:31:43.487533+0000","last_became_active":"2024-07-04T17:31:42.142952+0000","last_became_peered":"2024-07-04T17:31:42.142952+0000","last_unstale":"2024-07-04T17:31:43.487533+0000","last_undegraded":"2024-07-04T17:31:43.487533+0000","last_fullsized":"2024-07-04T17:31:43.487533+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:21:03.395798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.486539+0000","last_change":"2024-07-04T17:31:42.141851+0000","last_active":"2024-07-04T17:31:43.486539+0000","last_peered":"2024-07-04T17:31:43.486539+0000","last_clean":"2024-07-04T17:31:43.486539+0000","last_became_active":"2024-07-04T17:31:42.139731+0000","last_became_peered":"2024-07-04T17:31:42.139731+0000","last_unstale":"2024-07-04T17:31:43.486539+0000","last_undegraded":"2024-07-04T17:31:43.486539+0000","last_fullsized":"2024-07-04T17:31:43.486539+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:46:33.797585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.104046+0000","last_change":"2024-07-04T17:31:42.127926+0000","last_active":"2024-07-04T17:31:43.104046+0000","last_peered":"2024-07-04T17:31:43.104046+0000","last_clean":"2024-07-04T17:31:43.104046+0000","last_became_active":"2024-07-04T17:31:42.127596+0000","last_became_peered":"2024-07-04T17:31:42.127596+0000","last_unstale":"2024-07-04T17:31:43.104046+0000","last_undegraded":"2024-07-04T17:31:43.104046+0000","last_fullsized":"2024-07-04T17:31:43.104046+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:20:44.775608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.102444+0000","last_change":"2024-07-04T17:31:42.116203+0000","last_active":"2024-07-04T17:31:43.102444+0000","last_peered":"2024-07-04T17:31:43.102444+0000","last_clean":"2024-07-04T17:31:43.102444+0000","last_became_active":"2024-07-04T17:31:42.115668+0000","last_became_peered":"2024-07-04T17:31:42.115668+0000","last_unstale":"2024-07-04T17:31:43.102444+0000","last_undegraded":"2024-07-04T17:31:43.102444+0000","last_fullsized":"2024-07-04T17:31:43.102444+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:06:36.600932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.482035+0000","last_change":"2024-07-04T17:31:42.153157+0000","last_active":"2024-07-04T17:31:43.482035+0000","last_peered":"2024-07-04T17:31:43.482035+0000","last_clean":"2024-07-04T17:31:43.482035+0000","last_became_active":"2024-07-04T17:31:42.150615+0000","last_became_peered":"2024-07-04T17:31:42.150615+0000","last_unstale":"2024-07-04T17:31:43.482035+0000","last_undegraded":"2024-07-04T17:31:43.482035+0000","last_fullsized":"2024-07-04T17:31:43.482035+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:50:49.706615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.102453+0000","last_change":"2024-07-04T17:31:42.126765+0000","last_active":"2024-07-04T17:31:43.102453+0000","last_peered":"2024-07-04T17:31:43.102453+0000","last_clean":"2024-07-04T17:31:43.102453+0000","last_became_active":"2024-07-04T17:31:42.126443+0000","last_became_peered":"2024-07-04T17:31:42.126443+0000","last_unstale":"2024-07-04T17:31:43.102453+0000","last_undegraded":"2024-07-04T17:31:43.102453+0000","last_fullsized":"2024-07-04T17:31:43.102453+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:34:36.071106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.104147+0000","last_change":"2024-07-04T17:31:42.129239+0000","last_active":"2024-07-04T17:31:43.104147+0000","last_peered":"2024-07-04T17:31:43.104147+0000","last_clean":"2024-07-04T17:31:43.104147+0000","last_became_active":"2024-07-04T17:31:42.127715+0000","last_became_peered":"2024-07-04T17:31:42.127715+0000","last_unstale":"2024-07-04T17:31:43.104147+0000","last_undegraded":"2024-07-04T17:31:43.104147+0000","last_fullsized":"2024-07-04T17:31:43.104147+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:51:01.891207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.484265+0000","last_change":"2024-07-04T17:31:42.142475+0000","last_active":"2024-07-04T17:31:43.484265+0000","last_peered":"2024-07-04T17:31:43.484265+0000","last_clean":"2024-07-04T17:31:43.484265+0000","last_became_active":"2024-07-04T17:31:42.142140+0000","last_became_peered":"2024-07-04T17:31:42.142140+0000","last_unstale":"2024-07-04T17:31:43.484265+0000","last_undegraded":"2024-07-04T17:31:43.484265+0000","last_fullsized":"2024-07-04T17:31:43.484265+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:12:40.862972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.487611+0000","last_change":"2024-07-04T17:31:42.130865+0000","last_active":"2024-07-04T17:31:43.487611+0000","last_peered":"2024-07-04T17:31:43.487611+0000","last_clean":"2024-07-04T17:31:43.487611+0000","last_became_active":"2024-07-04T17:31:42.130064+0000","last_became_peered":"2024-07-04T17:31:42.130064+0000","last_unstale":"2024-07-04T17:31:43.487611+0000","last_undegraded":"2024-07-04T17:31:43.487611+0000","last_fullsized":"2024-07-04T17:31:43.487611+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:54:40.425975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.482511+0000","last_change":"2024-07-04T17:31:42.139174+0000","last_active":"2024-07-04T17:31:43.482511+0000","last_peered":"2024-07-04T17:31:43.482511+0000","last_clean":"2024-07-04T17:31:43.482511+0000","last_became_active":"2024-07-04T17:31:42.138397+0000","last_became_peered":"2024-07-04T17:31:42.138397+0000","last_unstale":"2024-07-04T17:31:43.482511+0000","last_undegraded":"2024-07-04T17:31:43.482511+0000","last_fullsized":"2024-07-04T17:31:43.482511+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:21:29.625345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.482894+0000","last_change":"2024-07-04T17:31:42.111650+0000","last_active":"2024-07-04T17:31:43.482894+0000","last_peered":"2024-07-04T17:31:43.482894+0000","last_clean":"2024-07-04T17:31:43.482894+0000","last_became_active":"2024-07-04T17:31:42.111045+0000","last_became_peered":"2024-07-04T17:31:42.111045+0000","last_unstale":"2024-07-04T17:31:43.482894+0000","last_undegraded":"2024-07-04T17:31:43.482894+0000","last_fullsized":"2024-07-04T17:31:43.482894+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:11:39.466653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.102717+0000","last_change":"2024-07-04T17:31:42.115990+0000","last_active":"2024-07-04T17:31:43.102717+0000","last_peered":"2024-07-04T17:31:43.102717+0000","last_clean":"2024-07-04T17:31:43.102717+0000","last_became_active":"2024-07-04T17:31:42.115413+0000","last_became_peered":"2024-07-04T17:31:42.115413+0000","last_unstale":"2024-07-04T17:31:43.102717+0000","last_undegraded":"2024-07-04T17:31:43.102717+0000","last_fullsized":"2024-07-04T17:31:43.102717+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:36:22.090921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.482053+0000","last_change":"2024-07-04T17:31:42.150544+0000","last_active":"2024-07-04T17:31:43.482053+0000","last_peered":"2024-07-04T17:31:43.482053+0000","last_clean":"2024-07-04T17:31:43.482053+0000","last_became_active":"2024-07-04T17:31:42.150217+0000","last_became_peered":"2024-07-04T17:31:42.150217+0000","last_unstale":"2024-07-04T17:31:43.482053+0000","last_undegraded":"2024-07-04T17:31:43.482053+0000","last_fullsized":"2024-07-04T17:31:43.482053+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:50:45.348242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.480600+0000","last_change":"2024-07-04T17:31:42.137976+0000","last_active":"2024-07-04T17:31:43.480600+0000","last_peered":"2024-07-04T17:31:43.480600+0000","last_clean":"2024-07-04T17:31:43.480600+0000","last_became_active":"2024-07-04T17:31:42.137669+0000","last_became_peered":"2024-07-04T17:31:42.137669+0000","last_unstale":"2024-07-04T17:31:43.480600+0000","last_undegraded":"2024-07-04T17:31:43.480600+0000","last_fullsized":"2024-07-04T17:31:43.480600+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:07:25.661148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.487496+0000","last_change":"2024-07-04T17:31:42.139568+0000","last_active":"2024-07-04T17:31:43.487496+0000","last_peered":"2024-07-04T17:31:43.487496+0000","last_clean":"2024-07-04T17:31:43.487496+0000","last_became_active":"2024-07-04T17:31:42.139247+0000","last_became_peered":"2024-07-04T17:31:42.139247+0000","last_unstale":"2024-07-04T17:31:43.487496+0000","last_undegraded":"2024-07-04T17:31:43.487496+0000","last_fullsized":"2024-07-04T17:31:43.487496+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:48:35.448519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"91'4","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.486350+0000","last_change":"2024-07-04T17:31:41.125254+0000","last_active":"2024-07-04T17:31:43.486350+0000","last_peered":"2024-07-04T17:31:43.486350+0000","last_clean":"2024-07-04T17:31:43.486350+0000","last_became_active":"2024-07-04T17:31:41.119432+0000","last_became_peered":"2024-07-04T17:31:41.119432+0000","last_unstale":"2024-07-04T17:31:43.486350+0000","last_undegraded":"2024-07-04T17:31:43.486350+0000","last_fullsized":"2024-07-04T17:31:43.486350+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:40:27.856616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.485731+0000","last_change":"2024-07-04T17:31:42.143241+0000","last_active":"2024-07-04T17:31:43.485731+0000","last_peered":"2024-07-04T17:31:43.485731+0000","last_clean":"2024-07-04T17:31:43.485731+0000","last_became_active":"2024-07-04T17:31:42.142919+0000","last_became_peered":"2024-07-04T17:31:42.142919+0000","last_unstale":"2024-07-04T17:31:43.485731+0000","last_undegraded":"2024-07-04T17:31:43.485731+0000","last_fullsized":"2024-07-04T17:31:43.485731+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:42:50.042411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.104077+0000","last_change":"2024-07-04T17:31:41.131289+0000","last_active":"2024-07-04T17:31:43.104077+0000","last_peered":"2024-07-04T17:31:43.104077+0000","last_clean":"2024-07-04T17:31:43.104077+0000","last_became_active":"2024-07-04T17:31:41.130956+0000","last_became_peered":"2024-07-04T17:31:41.130956+0000","last_unstale":"2024-07-04T17:31:43.104077+0000","last_undegraded":"2024-07-04T17:31:43.104077+0000","last_fullsized":"2024-07-04T17:31:43.104077+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:11:01.631006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.486259+0000","last_change":"2024-07-04T17:31:42.126808+0000","last_active":"2024-07-04T17:31:43.486259+0000","last_peered":"2024-07-04T17:31:43.486259+0000","last_clean":"2024-07-04T17:31:43.486259+0000","last_became_active":"2024-07-04T17:31:42.126497+0000","last_became_peered":"2024-07-04T17:31:42.126497+0000","last_unstale":"2024-07-04T17:31:43.486259+0000","last_undegraded":"2024-07-04T17:31:43.486259+0000","last_fullsized":"2024-07-04T17:31:43.486259+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:04:06.470763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.103323+0000","last_change":"2024-07-04T17:31:41.129438+0000","last_active":"2024-07-04T17:31:43.103323+0000","last_peered":"2024-07-04T17:31:43.103323+0000","last_clean":"2024-07-04T17:31:43.103323+0000","last_became_active":"2024-07-04T17:31:41.127900+0000","last_became_peered":"2024-07-04T17:31:41.127900+0000","last_unstale":"2024-07-04T17:31:43.103323+0000","last_undegraded":"2024-07-04T17:31:43.103323+0000","last_fullsized":"2024-07-04T17:31:43.103323+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:12:36.772353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.488028+0000","last_change":"2024-07-04T17:31:42.138160+0000","last_active":"2024-07-04T17:31:43.488028+0000","last_peered":"2024-07-04T17:31:43.488028+0000","last_clean":"2024-07-04T17:31:43.488028+0000","last_became_active":"2024-07-04T17:31:42.136593+0000","last_became_peered":"2024-07-04T17:31:42.136593+0000","last_unstale":"2024-07-04T17:31:43.488028+0000","last_undegraded":"2024-07-04T17:31:43.488028+0000","last_fullsized":"2024-07-04T17:31:43.488028+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:28:28.225877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.103585+0000","last_change":"2024-07-04T17:31:41.116679+0000","last_active":"2024-07-04T17:31:43.103585+0000","last_peered":"2024-07-04T17:31:43.103585+0000","last_clean":"2024-07-04T17:31:43.103585+0000","last_became_active":"2024-07-04T17:31:41.116240+0000","last_became_peered":"2024-07-04T17:31:41.116240+0000","last_unstale":"2024-07-04T17:31:43.103585+0000","last_undegraded":"2024-07-04T17:31:43.103585+0000","last_fullsized":"2024-07-04T17:31:43.103585+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:40:21.938736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.484746+0000","last_change":"2024-07-04T17:31:42.132447+0000","last_active":"2024-07-04T17:31:43.484746+0000","last_peered":"2024-07-04T17:31:43.484746+0000","last_clean":"2024-07-04T17:31:43.484746+0000","last_became_active":"2024-07-04T17:31:42.132121+0000","last_became_peered":"2024-07-04T17:31:42.132121+0000","last_unstale":"2024-07-04T17:31:43.484746+0000","last_undegraded":"2024-07-04T17:31:43.484746+0000","last_fullsized":"2024-07-04T17:31:43.484746+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:55:55.603144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.484977+0000","last_change":"2024-07-04T17:31:41.129634+0000","last_active":"2024-07-04T17:31:43.484977+0000","last_peered":"2024-07-04T17:31:43.484977+0000","last_clean":"2024-07-04T17:31:43.484977+0000","last_became_active":"2024-07-04T17:31:41.129326+0000","last_became_peered":"2024-07-04T17:31:41.129326+0000","last_unstale":"2024-07-04T17:31:43.484977+0000","last_undegraded":"2024-07-04T17:31:43.484977+0000","last_fullsized":"2024-07-04T17:31:43.484977+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:37:28.340891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.483445+0000","last_change":"2024-07-04T17:31:42.111322+0000","last_active":"2024-07-04T17:31:43.483445+0000","last_peered":"2024-07-04T17:31:43.483445+0000","last_clean":"2024-07-04T17:31:43.483445+0000","last_became_active":"2024-07-04T17:31:42.110186+0000","last_became_peered":"2024-07-04T17:31:42.110186+0000","last_unstale":"2024-07-04T17:31:43.483445+0000","last_undegraded":"2024-07-04T17:31:43.483445+0000","last_fullsized":"2024-07-04T17:31:43.483445+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:43:36.161285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.103053+0000","last_change":"2024-07-04T17:31:41.118623+0000","last_active":"2024-07-04T17:31:43.103053+0000","last_peered":"2024-07-04T17:31:43.103053+0000","last_clean":"2024-07-04T17:31:43.103053+0000","last_became_active":"2024-07-04T17:31:41.116368+0000","last_became_peered":"2024-07-04T17:31:41.116368+0000","last_unstale":"2024-07-04T17:31:43.103053+0000","last_undegraded":"2024-07-04T17:31:43.103053+0000","last_fullsized":"2024-07-04T17:31:43.103053+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:34:37.181739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.481060+0000","last_change":"2024-07-04T17:31:42.124019+0000","last_active":"2024-07-04T17:31:43.481060+0000","last_peered":"2024-07-04T17:31:43.481060+0000","last_clean":"2024-07-04T17:31:43.481060+0000","last_became_active":"2024-07-04T17:31:42.123684+0000","last_became_peered":"2024-07-04T17:31:42.123684+0000","last_unstale":"2024-07-04T17:31:43.481060+0000","last_undegraded":"2024-07-04T17:31:43.481060+0000","last_fullsized":"2024-07-04T17:31:43.481060+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:12:26.032030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.483071+0000","last_change":"2024-07-04T17:31:41.122775+0000","last_active":"2024-07-04T17:31:43.483071+0000","last_peered":"2024-07-04T17:31:43.483071+0000","last_clean":"2024-07-04T17:31:43.483071+0000","last_became_active":"2024-07-04T17:31:41.121256+0000","last_became_peered":"2024-07-04T17:31:41.121256+0000","last_unstale":"2024-07-04T17:31:43.483071+0000","last_undegraded":"2024-07-04T17:31:43.483071+0000","last_fullsized":"2024-07-04T17:31:43.483071+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:41:24.386164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.481961+0000","last_change":"2024-07-04T17:31:42.138950+0000","last_active":"2024-07-04T17:31:43.481961+0000","last_peered":"2024-07-04T17:31:43.481961+0000","last_clean":"2024-07-04T17:31:43.481961+0000","last_became_active":"2024-07-04T17:31:42.138614+0000","last_became_peered":"2024-07-04T17:31:42.138614+0000","last_unstale":"2024-07-04T17:31:43.481961+0000","last_undegraded":"2024-07-04T17:31:43.481961+0000","last_fullsized":"2024-07-04T17:31:43.481961+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:00:43.285683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.481356+0000","last_change":"2024-07-04T17:31:41.114447+0000","last_active":"2024-07-04T17:31:43.481356+0000","last_peered":"2024-07-04T17:31:43.481356+0000","last_clean":"2024-07-04T17:31:43.481356+0000","last_became_active":"2024-07-04T17:31:41.114127+0000","last_became_peered":"2024-07-04T17:31:41.114127+0000","last_unstale":"2024-07-04T17:31:43.481356+0000","last_undegraded":"2024-07-04T17:31:43.481356+0000","last_fullsized":"2024-07-04T17:31:43.481356+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:21:00.022601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.104006+0000","last_change":"2024-07-04T17:31:42.133055+0000","last_active":"2024-07-04T17:31:43.104006+0000","last_peered":"2024-07-04T17:31:43.104006+0000","last_clean":"2024-07-04T17:31:43.104006+0000","last_became_active":"2024-07-04T17:31:42.131725+0000","last_became_peered":"2024-07-04T17:31:42.131725+0000","last_unstale":"2024-07-04T17:31:43.104006+0000","last_undegraded":"2024-07-04T17:31:43.104006+0000","last_fullsized":"2024-07-04T17:31:43.104006+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:21:59.669492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.104366+0000","last_change":"2024-07-04T17:31:41.131784+0000","last_active":"2024-07-04T17:31:43.104366+0000","last_peered":"2024-07-04T17:31:43.104366+0000","last_clean":"2024-07-04T17:31:43.104366+0000","last_became_active":"2024-07-04T17:31:41.130750+0000","last_became_peered":"2024-07-04T17:31:41.130750+0000","last_unstale":"2024-07-04T17:31:43.104366+0000","last_undegraded":"2024-07-04T17:31:43.104366+0000","last_fullsized":"2024-07-04T17:31:43.104366+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:55:29.200978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.104189+0000","last_change":"2024-07-04T17:31:42.128979+0000","last_active":"2024-07-04T17:31:43.104189+0000","last_peered":"2024-07-04T17:31:43.104189+0000","last_clean":"2024-07-04T17:31:43.104189+0000","last_became_active":"2024-07-04T17:31:42.126866+0000","last_became_peered":"2024-07-04T17:31:42.126866+0000","last_unstale":"2024-07-04T17:31:43.104189+0000","last_undegraded":"2024-07-04T17:31:43.104189+0000","last_fullsized":"2024-07-04T17:31:43.104189+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:06:17.477749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.104184+0000","last_change":"2024-07-04T17:31:41.132406+0000","last_active":"2024-07-04T17:31:43.104184+0000","last_peered":"2024-07-04T17:31:43.104184+0000","last_clean":"2024-07-04T17:31:43.104184+0000","last_became_active":"2024-07-04T17:31:41.130594+0000","last_became_peered":"2024-07-04T17:31:41.130594+0000","last_unstale":"2024-07-04T17:31:43.104184+0000","last_undegraded":"2024-07-04T17:31:43.104184+0000","last_fullsized":"2024-07-04T17:31:43.104184+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:01:17.482235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.486470+0000","last_change":"2024-07-04T17:31:42.136357+0000","last_active":"2024-07-04T17:31:43.486470+0000","last_peered":"2024-07-04T17:31:43.486470+0000","last_clean":"2024-07-04T17:31:43.486470+0000","last_became_active":"2024-07-04T17:31:42.136025+0000","last_became_peered":"2024-07-04T17:31:42.136025+0000","last_unstale":"2024-07-04T17:31:43.486470+0000","last_undegraded":"2024-07-04T17:31:43.486470+0000","last_fullsized":"2024-07-04T17:31:43.486470+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:55:46.913903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.15","version":"91'3","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.483131+0000","last_change":"2024-07-04T17:31:41.124114+0000","last_active":"2024-07-04T17:31:43.483131+0000","last_peered":"2024-07-04T17:31:43.483131+0000","last_clean":"2024-07-04T17:31:43.483131+0000","last_became_active":"2024-07-04T17:31:41.121166+0000","last_became_peered":"2024-07-04T17:31:41.121166+0000","last_unstale":"2024-07-04T17:31:43.483131+0000","last_undegraded":"2024-07-04T17:31:43.483131+0000","last_fullsized":"2024-07-04T17:31:43.483131+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:44:40.148021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.103227+0000","last_change":"2024-07-04T17:31:42.134181+0000","last_active":"2024-07-04T17:31:43.103227+0000","last_peered":"2024-07-04T17:31:43.103227+0000","last_clean":"2024-07-04T17:31:43.103227+0000","last_became_active":"2024-07-04T17:31:42.133671+0000","last_became_peered":"2024-07-04T17:31:42.133671+0000","last_unstale":"2024-07-04T17:31:43.103227+0000","last_undegraded":"2024-07-04T17:31:43.103227+0000","last_fullsized":"2024-07-04T17:31:43.103227+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:13:23.161757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"91'6","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.482777+0000","last_change":"2024-07-04T17:31:41.123522+0000","last_active":"2024-07-04T17:31:43.482777+0000","last_peered":"2024-07-04T17:31:43.482777+0000","last_clean":"2024-07-04T17:31:43.482777+0000","last_became_active":"2024-07-04T17:31:41.121523+0000","last_became_peered":"2024-07-04T17:31:41.121523+0000","last_unstale":"2024-07-04T17:31:43.482777+0000","last_undegraded":"2024-07-04T17:31:43.482777+0000","last_fullsized":"2024-07-04T17:31:43.482777+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:36:59.958154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.102966+0000","last_change":"2024-07-04T17:31:42.116278+0000","last_active":"2024-07-04T17:31:43.102966+0000","last_peered":"2024-07-04T17:31:43.102966+0000","last_clean":"2024-07-04T17:31:43.102966+0000","last_became_active":"2024-07-04T17:31:42.115861+0000","last_became_peered":"2024-07-04T17:31:42.115861+0000","last_unstale":"2024-07-04T17:31:43.102966+0000","last_undegraded":"2024-07-04T17:31:43.102966+0000","last_fullsized":"2024-07-04T17:31:43.102966+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:43:17.950207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.13","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.485539+0000","last_change":"2024-07-04T17:31:41.128711+0000","last_active":"2024-07-04T17:31:43.485539+0000","last_peered":"2024-07-04T17:31:43.485539+0000","last_clean":"2024-07-04T17:31:43.485539+0000","last_became_active":"2024-07-04T17:31:41.127816+0000","last_became_peered":"2024-07-04T17:31:41.127816+0000","last_unstale":"2024-07-04T17:31:43.485539+0000","last_undegraded":"2024-07-04T17:31:43.485539+0000","last_fullsized":"2024-07-04T17:31:43.485539+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:37:51.190424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.483431+0000","last_change":"2024-07-04T17:31:42.111369+0000","last_active":"2024-07-04T17:31:43.483431+0000","last_peered":"2024-07-04T17:31:43.483431+0000","last_clean":"2024-07-04T17:31:43.483431+0000","last_became_active":"2024-07-04T17:31:42.110632+0000","last_became_peered":"2024-07-04T17:31:42.110632+0000","last_unstale":"2024-07-04T17:31:43.483431+0000","last_undegraded":"2024-07-04T17:31:43.483431+0000","last_fullsized":"2024-07-04T17:31:43.483431+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:31:20.968644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.485135+0000","last_change":"2024-07-04T17:31:41.126486+0000","last_active":"2024-07-04T17:31:43.485135+0000","last_peered":"2024-07-04T17:31:43.485135+0000","last_clean":"2024-07-04T17:31:43.485135+0000","last_became_active":"2024-07-04T17:31:41.126142+0000","last_became_peered":"2024-07-04T17:31:41.126142+0000","last_unstale":"2024-07-04T17:31:43.485135+0000","last_undegraded":"2024-07-04T17:31:43.485135+0000","last_fullsized":"2024-07-04T17:31:43.485135+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:25:18.918225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.482432+0000","last_change":"2024-07-04T17:31:42.134513+0000","last_active":"2024-07-04T17:31:43.482432+0000","last_peered":"2024-07-04T17:31:43.482432+0000","last_clean":"2024-07-04T17:31:43.482432+0000","last_became_active":"2024-07-04T17:31:42.134166+0000","last_became_peered":"2024-07-04T17:31:42.134166+0000","last_unstale":"2024-07-04T17:31:43.482432+0000","last_undegraded":"2024-07-04T17:31:43.482432+0000","last_fullsized":"2024-07-04T17:31:43.482432+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:17:13.766129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.10","version":"91'8","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.102648+0000","last_change":"2024-07-04T17:31:41.123889+0000","last_active":"2024-07-04T17:31:43.102648+0000","last_peered":"2024-07-04T17:31:43.102648+0000","last_clean":"2024-07-04T17:31:43.102648+0000","last_became_active":"2024-07-04T17:31:41.123555+0000","last_became_peered":"2024-07-04T17:31:41.123555+0000","last_unstale":"2024-07-04T17:31:43.102648+0000","last_undegraded":"2024-07-04T17:31:43.102648+0000","last_fullsized":"2024-07-04T17:31:43.102648+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:07:24.656814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.104056+0000","last_change":"2024-07-04T17:31:42.117660+0000","last_active":"2024-07-04T17:31:43.104056+0000","last_peered":"2024-07-04T17:31:43.104056+0000","last_clean":"2024-07-04T17:31:43.104056+0000","last_became_active":"2024-07-04T17:31:42.115485+0000","last_became_peered":"2024-07-04T17:31:42.115485+0000","last_unstale":"2024-07-04T17:31:43.104056+0000","last_undegraded":"2024-07-04T17:31:43.104056+0000","last_fullsized":"2024-07-04T17:31:43.104056+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:10:45.367563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.486616+0000","last_change":"2024-07-04T17:31:41.110852+0000","last_active":"2024-07-04T17:31:43.486616+0000","last_peered":"2024-07-04T17:31:43.486616+0000","last_clean":"2024-07-04T17:31:43.486616+0000","last_became_active":"2024-07-04T17:31:41.109092+0000","last_became_peered":"2024-07-04T17:31:41.109092+0000","last_unstale":"2024-07-04T17:31:43.486616+0000","last_undegraded":"2024-07-04T17:31:43.486616+0000","last_fullsized":"2024-07-04T17:31:43.486616+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:47:10.505606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.488119+0000","last_change":"2024-07-04T17:31:42.127586+0000","last_active":"2024-07-04T17:31:43.488119+0000","last_peered":"2024-07-04T17:31:43.488119+0000","last_clean":"2024-07-04T17:31:43.488119+0000","last_became_active":"2024-07-04T17:31:42.127263+0000","last_became_peered":"2024-07-04T17:31:42.127263+0000","last_unstale":"2024-07-04T17:31:43.488119+0000","last_undegraded":"2024-07-04T17:31:43.488119+0000","last_fullsized":"2024-07-04T17:31:43.488119+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:54:41.368680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.104222+0000","last_change":"2024-07-04T17:31:41.127224+0000","last_active":"2024-07-04T17:31:43.104222+0000","last_peered":"2024-07-04T17:31:43.104222+0000","last_clean":"2024-07-04T17:31:43.104222+0000","last_became_active":"2024-07-04T17:31:41.126878+0000","last_became_peered":"2024-07-04T17:31:41.126878+0000","last_unstale":"2024-07-04T17:31:43.104222+0000","last_undegraded":"2024-07-04T17:31:43.104222+0000","last_fullsized":"2024-07-04T17:31:43.104222+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:17:12.761795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.482831+0000","last_change":"2024-07-04T17:31:42.138910+0000","last_active":"2024-07-04T17:31:43.482831+0000","last_peered":"2024-07-04T17:31:43.482831+0000","last_clean":"2024-07-04T17:31:43.482831+0000","last_became_active":"2024-07-04T17:31:42.137936+0000","last_became_peered":"2024-07-04T17:31:42.137936+0000","last_unstale":"2024-07-04T17:31:43.482831+0000","last_undegraded":"2024-07-04T17:31:43.482831+0000","last_fullsized":"2024-07-04T17:31:43.482831+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:30:17.422306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.d","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.483428+0000","last_change":"2024-07-04T17:31:41.125962+0000","last_active":"2024-07-04T17:31:43.483428+0000","last_peered":"2024-07-04T17:31:43.483428+0000","last_clean":"2024-07-04T17:31:43.483428+0000","last_became_active":"2024-07-04T17:31:41.121577+0000","last_became_peered":"2024-07-04T17:31:41.121577+0000","last_unstale":"2024-07-04T17:31:43.483428+0000","last_undegraded":"2024-07-04T17:31:43.483428+0000","last_fullsized":"2024-07-04T17:31:43.483428+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:02:24.336737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.483058+0000","last_change":"2024-07-04T17:31:42.138658+0000","last_active":"2024-07-04T17:31:43.483058+0000","last_peered":"2024-07-04T17:31:43.483058+0000","last_clean":"2024-07-04T17:31:43.483058+0000","last_became_active":"2024-07-04T17:31:42.137836+0000","last_became_peered":"2024-07-04T17:31:42.137836+0000","last_unstale":"2024-07-04T17:31:43.483058+0000","last_undegraded":"2024-07-04T17:31:43.483058+0000","last_fullsized":"2024-07-04T17:31:43.483058+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:32:01.472783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.482935+0000","last_change":"2024-07-04T17:31:41.111490+0000","last_active":"2024-07-04T17:31:43.482935+0000","last_peered":"2024-07-04T17:31:43.482935+0000","last_clean":"2024-07-04T17:31:43.482935+0000","last_became_active":"2024-07-04T17:31:41.110881+0000","last_became_peered":"2024-07-04T17:31:41.110881+0000","last_unstale":"2024-07-04T17:31:43.482935+0000","last_undegraded":"2024-07-04T17:31:43.482935+0000","last_fullsized":"2024-07-04T17:31:43.482935+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:44:12.481565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.485214+0000","last_change":"2024-07-04T17:31:42.144839+0000","last_active":"2024-07-04T17:31:43.485214+0000","last_peered":"2024-07-04T17:31:43.485214+0000","last_clean":"2024-07-04T17:31:43.485214+0000","last_became_active":"2024-07-04T17:31:42.142261+0000","last_became_peered":"2024-07-04T17:31:42.142261+0000","last_unstale":"2024-07-04T17:31:43.485214+0000","last_undegraded":"2024-07-04T17:31:43.485214+0000","last_fullsized":"2024-07-04T17:31:43.485214+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:44:20.205264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.485739+0000","last_change":"2024-07-04T17:31:41.128172+0000","last_active":"2024-07-04T17:31:43.485739+0000","last_peered":"2024-07-04T17:31:43.485739+0000","last_clean":"2024-07-04T17:31:43.485739+0000","last_became_active":"2024-07-04T17:31:41.127350+0000","last_became_peered":"2024-07-04T17:31:41.127350+0000","last_unstale":"2024-07-04T17:31:43.485739+0000","last_undegraded":"2024-07-04T17:31:43.485739+0000","last_fullsized":"2024-07-04T17:31:43.485739+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:24:57.684170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.483774+0000","last_change":"2024-07-04T17:31:42.148767+0000","last_active":"2024-07-04T17:31:43.483774+0000","last_peered":"2024-07-04T17:31:43.483774+0000","last_clean":"2024-07-04T17:31:43.483774+0000","last_became_active":"2024-07-04T17:31:42.148091+0000","last_became_peered":"2024-07-04T17:31:42.148091+0000","last_unstale":"2024-07-04T17:31:43.483774+0000","last_undegraded":"2024-07-04T17:31:43.483774+0000","last_fullsized":"2024-07-04T17:31:43.483774+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:42:04.145597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.103220+0000","last_change":"2024-07-04T17:31:41.131853+0000","last_active":"2024-07-04T17:31:43.103220+0000","last_peered":"2024-07-04T17:31:43.103220+0000","last_clean":"2024-07-04T17:31:43.103220+0000","last_became_active":"2024-07-04T17:31:41.129778+0000","last_became_peered":"2024-07-04T17:31:41.129778+0000","last_unstale":"2024-07-04T17:31:43.103220+0000","last_undegraded":"2024-07-04T17:31:43.103220+0000","last_fullsized":"2024-07-04T17:31:43.103220+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:59:40.102134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.481262+0000","last_change":"2024-07-04T17:31:42.111799+0000","last_active":"2024-07-04T17:31:43.481262+0000","last_peered":"2024-07-04T17:31:43.481262+0000","last_clean":"2024-07-04T17:31:43.481262+0000","last_became_active":"2024-07-04T17:31:42.110114+0000","last_became_peered":"2024-07-04T17:31:42.110114+0000","last_unstale":"2024-07-04T17:31:43.481262+0000","last_undegraded":"2024-07-04T17:31:43.481262+0000","last_fullsized":"2024-07-04T17:31:43.481262+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:45:25.449722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.483393+0000","last_change":"2024-07-04T17:31:41.120856+0000","last_active":"2024-07-04T17:31:43.483393+0000","last_peered":"2024-07-04T17:31:43.483393+0000","last_clean":"2024-07-04T17:31:43.483393+0000","last_became_active":"2024-07-04T17:31:41.120338+0000","last_became_peered":"2024-07-04T17:31:41.120338+0000","last_unstale":"2024-07-04T17:31:43.483393+0000","last_undegraded":"2024-07-04T17:31:43.483393+0000","last_fullsized":"2024-07-04T17:31:43.483393+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:14:01.301278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.482906+0000","last_change":"2024-07-04T17:31:42.124705+0000","last_active":"2024-07-04T17:31:43.482906+0000","last_peered":"2024-07-04T17:31:43.482906+0000","last_clean":"2024-07-04T17:31:43.482906+0000","last_became_active":"2024-07-04T17:31:42.124266+0000","last_became_peered":"2024-07-04T17:31:42.124266+0000","last_unstale":"2024-07-04T17:31:43.482906+0000","last_undegraded":"2024-07-04T17:31:43.482906+0000","last_fullsized":"2024-07-04T17:31:43.482906+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:18:01.138996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.481382+0000","last_change":"2024-07-04T17:31:41.114344+0000","last_active":"2024-07-04T17:31:43.481382+0000","last_peered":"2024-07-04T17:31:43.481382+0000","last_clean":"2024-07-04T17:31:43.481382+0000","last_became_active":"2024-07-04T17:31:41.114006+0000","last_became_peered":"2024-07-04T17:31:41.114006+0000","last_unstale":"2024-07-04T17:31:43.481382+0000","last_undegraded":"2024-07-04T17:31:43.481382+0000","last_fullsized":"2024-07-04T17:31:43.481382+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:57:55.654579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.484777+0000","last_change":"2024-07-04T17:31:42.142194+0000","last_active":"2024-07-04T17:31:43.484777+0000","last_peered":"2024-07-04T17:31:43.484777+0000","last_clean":"2024-07-04T17:31:43.484777+0000","last_became_active":"2024-07-04T17:31:42.139517+0000","last_became_peered":"2024-07-04T17:31:42.139517+0000","last_unstale":"2024-07-04T17:31:43.484777+0000","last_undegraded":"2024-07-04T17:31:43.484777+0000","last_fullsized":"2024-07-04T17:31:43.484777+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:12:09.965552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.488088+0000","last_change":"2024-07-04T17:31:41.126950+0000","last_active":"2024-07-04T17:31:43.488088+0000","last_peered":"2024-07-04T17:31:43.488088+0000","last_clean":"2024-07-04T17:31:43.488088+0000","last_became_active":"2024-07-04T17:31:41.126611+0000","last_became_peered":"2024-07-04T17:31:41.126611+0000","last_unstale":"2024-07-04T17:31:43.488088+0000","last_undegraded":"2024-07-04T17:31:43.488088+0000","last_fullsized":"2024-07-04T17:31:43.488088+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:27:51.965765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.486898+0000","last_change":"2024-07-04T17:31:42.107634+0000","last_active":"2024-07-04T17:31:43.486898+0000","last_peered":"2024-07-04T17:31:43.486898+0000","last_clean":"2024-07-04T17:31:43.486898+0000","last_became_active":"2024-07-04T17:31:42.107327+0000","last_became_peered":"2024-07-04T17:31:42.107327+0000","last_unstale":"2024-07-04T17:31:43.486898+0000","last_undegraded":"2024-07-04T17:31:43.486898+0000","last_fullsized":"2024-07-04T17:31:43.486898+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:57:08.828739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"92'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.487379+0000","last_change":"2024-07-04T17:31:41.126383+0000","last_active":"2024-07-04T17:31:43.487379+0000","last_peered":"2024-07-04T17:31:43.487379+0000","last_clean":"2024-07-04T17:31:43.487379+0000","last_became_active":"2024-07-04T17:31:41.126045+0000","last_became_peered":"2024-07-04T17:31:41.126045+0000","last_unstale":"2024-07-04T17:31:43.487379+0000","last_undegraded":"2024-07-04T17:31:43.487379+0000","last_fullsized":"2024-07-04T17:31:43.487379+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:57:59.125923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.482709+0000","last_change":"2024-07-04T17:31:42.143202+0000","last_active":"2024-07-04T17:31:43.482709+0000","last_peered":"2024-07-04T17:31:43.482709+0000","last_clean":"2024-07-04T17:31:43.482709+0000","last_became_active":"2024-07-04T17:31:42.141656+0000","last_became_peered":"2024-07-04T17:31:42.141656+0000","last_unstale":"2024-07-04T17:31:43.482709+0000","last_undegraded":"2024-07-04T17:31:43.482709+0000","last_fullsized":"2024-07-04T17:31:43.482709+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:19:20.240973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.483356+0000","last_change":"2024-07-04T17:31:41.123150+0000","last_active":"2024-07-04T17:31:43.483356+0000","last_peered":"2024-07-04T17:31:43.483356+0000","last_clean":"2024-07-04T17:31:43.483356+0000","last_became_active":"2024-07-04T17:31:41.120767+0000","last_became_peered":"2024-07-04T17:31:41.120767+0000","last_unstale":"2024-07-04T17:31:43.483356+0000","last_undegraded":"2024-07-04T17:31:43.483356+0000","last_fullsized":"2024-07-04T17:31:43.483356+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:42:19.857048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.483214+0000","last_change":"2024-07-04T17:31:42.109920+0000","last_active":"2024-07-04T17:31:43.483214+0000","last_peered":"2024-07-04T17:31:43.483214+0000","last_clean":"2024-07-04T17:31:43.483214+0000","last_became_active":"2024-07-04T17:31:42.109357+0000","last_became_peered":"2024-07-04T17:31:42.109357+0000","last_unstale":"2024-07-04T17:31:43.483214+0000","last_undegraded":"2024-07-04T17:31:43.483214+0000","last_fullsized":"2024-07-04T17:31:43.483214+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:58:16.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.103726+0000","last_change":"2024-07-04T17:31:41.119885+0000","last_active":"2024-07-04T17:31:43.103726+0000","last_peered":"2024-07-04T17:31:43.103726+0000","last_clean":"2024-07-04T17:31:43.103726+0000","last_became_active":"2024-07-04T17:31:41.116720+0000","last_became_peered":"2024-07-04T17:31:41.116720+0000","last_unstale":"2024-07-04T17:31:43.103726+0000","last_undegraded":"2024-07-04T17:31:43.103726+0000","last_fullsized":"2024-07-04T17:31:43.103726+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:30:49.905769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.484989+0000","last_change":"2024-07-04T17:31:42.125396+0000","last_active":"2024-07-04T17:31:43.484989+0000","last_peered":"2024-07-04T17:31:43.484989+0000","last_clean":"2024-07-04T17:31:43.484989+0000","last_became_active":"2024-07-04T17:31:42.123980+0000","last_became_peered":"2024-07-04T17:31:42.123980+0000","last_unstale":"2024-07-04T17:31:43.484989+0000","last_undegraded":"2024-07-04T17:31:43.484989+0000","last_fullsized":"2024-07-04T17:31:43.484989+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:59:35.007753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.485892+0000","last_change":"2024-07-04T17:31:41.128454+0000","last_active":"2024-07-04T17:31:43.485892+0000","last_peered":"2024-07-04T17:31:43.485892+0000","last_clean":"2024-07-04T17:31:43.485892+0000","last_became_active":"2024-07-04T17:31:41.127785+0000","last_became_peered":"2024-07-04T17:31:41.127785+0000","last_unstale":"2024-07-04T17:31:43.485892+0000","last_undegraded":"2024-07-04T17:31:43.485892+0000","last_fullsized":"2024-07-04T17:31:43.485892+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:48:22.712062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.483773+0000","last_change":"2024-07-04T17:31:42.111629+0000","last_active":"2024-07-04T17:31:43.483773+0000","last_peered":"2024-07-04T17:31:43.483773+0000","last_clean":"2024-07-04T17:31:43.483773+0000","last_became_active":"2024-07-04T17:31:42.110978+0000","last_became_peered":"2024-07-04T17:31:42.110978+0000","last_unstale":"2024-07-04T17:31:43.483773+0000","last_undegraded":"2024-07-04T17:31:43.483773+0000","last_fullsized":"2024-07-04T17:31:43.483773+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:12:01.825579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.484487+0000","last_change":"2024-07-04T17:31:41.124595+0000","last_active":"2024-07-04T17:31:43.484487+0000","last_peered":"2024-07-04T17:31:43.484487+0000","last_clean":"2024-07-04T17:31:43.484487+0000","last_became_active":"2024-07-04T17:31:41.124315+0000","last_became_peered":"2024-07-04T17:31:41.124315+0000","last_unstale":"2024-07-04T17:31:43.484487+0000","last_undegraded":"2024-07-04T17:31:43.484487+0000","last_fullsized":"2024-07-04T17:31:43.484487+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:10:03.734241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.481308+0000","last_change":"2024-07-04T17:31:42.112865+0000","last_active":"2024-07-04T17:31:43.481308+0000","last_peered":"2024-07-04T17:31:43.481308+0000","last_clean":"2024-07-04T17:31:43.481308+0000","last_became_active":"2024-07-04T17:31:42.109293+0000","last_became_peered":"2024-07-04T17:31:42.109293+0000","last_unstale":"2024-07-04T17:31:43.481308+0000","last_undegraded":"2024-07-04T17:31:43.481308+0000","last_fullsized":"2024-07-04T17:31:43.481308+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:07:14.667638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.102591+0000","last_change":"2024-07-04T17:31:41.116814+0000","last_active":"2024-07-04T17:31:43.102591+0000","last_peered":"2024-07-04T17:31:43.102591+0000","last_clean":"2024-07-04T17:31:43.102591+0000","last_became_active":"2024-07-04T17:31:41.116425+0000","last_became_peered":"2024-07-04T17:31:41.116425+0000","last_unstale":"2024-07-04T17:31:43.102591+0000","last_undegraded":"2024-07-04T17:31:43.102591+0000","last_fullsized":"2024-07-04T17:31:43.102591+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:18:00.134663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.482693+0000","last_change":"2024-07-04T17:31:42.145454+0000","last_active":"2024-07-04T17:31:43.482693+0000","last_peered":"2024-07-04T17:31:43.482693+0000","last_clean":"2024-07-04T17:31:43.482693+0000","last_became_active":"2024-07-04T17:31:42.144919+0000","last_became_peered":"2024-07-04T17:31:42.144919+0000","last_unstale":"2024-07-04T17:31:43.482693+0000","last_undegraded":"2024-07-04T17:31:43.482693+0000","last_fullsized":"2024-07-04T17:31:43.482693+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:30:26.887002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.481559+0000","last_change":"2024-07-04T17:31:41.111066+0000","last_active":"2024-07-04T17:31:43.481559+0000","last_peered":"2024-07-04T17:31:43.481559+0000","last_clean":"2024-07-04T17:31:43.481559+0000","last_became_active":"2024-07-04T17:31:41.110689+0000","last_became_peered":"2024-07-04T17:31:41.110689+0000","last_unstale":"2024-07-04T17:31:43.481559+0000","last_undegraded":"2024-07-04T17:31:43.481559+0000","last_fullsized":"2024-07-04T17:31:43.481559+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:24:51.654346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.480874+0000","last_change":"2024-07-04T17:31:42.124940+0000","last_active":"2024-07-04T17:31:43.480874+0000","last_peered":"2024-07-04T17:31:43.480874+0000","last_clean":"2024-07-04T17:31:43.480874+0000","last_became_active":"2024-07-04T17:31:42.124608+0000","last_became_peered":"2024-07-04T17:31:42.124608+0000","last_unstale":"2024-07-04T17:31:43.480874+0000","last_undegraded":"2024-07-04T17:31:43.480874+0000","last_fullsized":"2024-07-04T17:31:43.480874+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:11:02.635340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.103692+0000","last_change":"2024-07-04T17:31:41.125061+0000","last_active":"2024-07-04T17:31:43.103692+0000","last_peered":"2024-07-04T17:31:43.103692+0000","last_clean":"2024-07-04T17:31:43.103692+0000","last_became_active":"2024-07-04T17:31:41.122766+0000","last_became_peered":"2024-07-04T17:31:41.122766+0000","last_unstale":"2024-07-04T17:31:43.103692+0000","last_undegraded":"2024-07-04T17:31:43.103692+0000","last_fullsized":"2024-07-04T17:31:43.103692+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:00:42.281350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"80'87","reported_seq":356,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T17:31:43.480967+0000","last_change":"2024-07-04T17:26:04.953559+0000","last_active":"2024-07-04T17:31:43.480967+0000","last_peered":"2024-07-04T17:31:43.480967+0000","last_clean":"2024-07-04T17:31:43.480967+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:31:43.480967+0000","last_undegraded":"2024-07-04T17:31:43.480967+0000","last_fullsized":"2024-07-04T17:31:43.480967+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_clean_scrub_stamp":"2024-07-04T17:26:03.933432+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.064181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":356,"num_read_kb":926,"num_write":139,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":356,"num_read_kb":926,"num_write":139,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738436,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36264,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335576,"statfs":{"total":96636764160,"available":96599629824,"internally_reserved":0,"allocated":937984,"data_stored":365581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738437,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36848,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334992,"statfs":{"total":96636764160,"available":96599031808,"internally_reserved":0,"allocated":1544192,"data_stored":957117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36852,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334988,"statfs":{"total":96636764160,"available":96599027712,"internally_reserved":0,"allocated":1548288,"data_stored":957687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:31:47.444 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-04T17:31:47.444 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-07-04T17:31:47.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.786+0000 7f542c200640 1 -- 172.21.15.42:0/2565775107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54240a94d0 msgr2=0x7f54240b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:47.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.786+0000 7f542c200640 1 --2- 172.21.15.42:0/2565775107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54240a94d0 0x7f54240b1090 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f541800b2b0 tx=0x7f541802ad10 comp rx=0 tx=0).stop 2024-07-04T17:31:47.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.787+0000 7f542c200640 1 -- 172.21.15.42:0/2565775107 shutdown_connections 2024-07-04T17:31:47.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.787+0000 7f542c200640 1 --2- 172.21.15.42:0/2565775107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54240a94d0 0x7f54240b1090 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:47.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.787+0000 7f542c200640 1 -- 172.21.15.42:0/2565775107 >> 172.21.15.42:0/2565775107 conn(0x7f542406c020 msgr2=0x7f542406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:47.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.787+0000 7f542c200640 1 -- 172.21.15.42:0/2565775107 shutdown_connections 2024-07-04T17:31:47.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.787+0000 7f542c200640 1 -- 172.21.15.42:0/2565775107 wait complete. 2024-07-04T17:31:47.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.789+0000 7f542c200640 1 Processor -- start 2024-07-04T17:31:47.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.789+0000 7f542c200640 1 -- start start 2024-07-04T17:31:47.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.790+0000 7f542c200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5424072570 0x7f54241e6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:47.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.790+0000 7f542c200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f54240721a0 con 0x7f5424072570 2024-07-04T17:31:47.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.791+0000 7f5429800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5424072570 0x7f54241e6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:47.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.791+0000 7f5429800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5424072570 0x7f54241e6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54340/0 (socket says 172.21.15.42:54340) 2024-07-04T17:31:47.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.791+0000 7f5429800640 1 -- 172.21.15.42:0/2299205573 learned_addr learned my addr 172.21.15.42:0/2299205573 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:47.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.791+0000 7f5429800640 1 -- 172.21.15.42:0/2299205573 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5424130f10 con 0x7f5424072570 2024-07-04T17:31:47.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.791+0000 7f5429800640 1 --2- 172.21.15.42:0/2299205573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5424072570 0x7f54241e6c60 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f541802b750 tx=0x7f54180097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:47.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.792+0000 7f5421600640 1 -- 172.21.15.42:0/2299205573 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5418038040 con 0x7f5424072570 2024-07-04T17:31:47.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.792+0000 7f5421600640 1 -- 172.21.15.42:0/2299205573 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5418004180 con 0x7f5424072570 2024-07-04T17:31:47.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.792+0000 7f5421600640 1 -- 172.21.15.42:0/2299205573 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54180044a0 con 0x7f5424072570 2024-07-04T17:31:47.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.792+0000 7f542c200640 1 -- 172.21.15.42:0/2299205573 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54241d6a70 con 0x7f5424072570 2024-07-04T17:31:47.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.792+0000 7f542c200640 1 -- 172.21.15.42:0/2299205573 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f542413e860 con 0x7f5424072570 2024-07-04T17:31:47.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.793+0000 7f542c200640 1 -- 172.21.15.42:0/2299205573 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54240b1090 con 0x7f5424072570 2024-07-04T17:31:47.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.793+0000 7f5421600640 1 -- 172.21.15.42:0/2299205573 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7f54180035f0 con 0x7f5424072570 2024-07-04T17:31:47.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.794+0000 7f5421600640 1 --2- 172.21.15.42:0/2299205573 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f540403d3d0 0x7f540403f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:47.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.794+0000 7f5421600640 1 -- 172.21.15.42:0/2299205573 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 3439+0+0 (secure 0 0 0) 0x7f5418072330 con 0x7f5424072570 2024-07-04T17:31:47.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.794+0000 7f5428e00640 1 --2- 172.21.15.42:0/2299205573 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f540403d3d0 0x7f540403f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:47.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.795+0000 7f5428e00640 1 --2- 172.21.15.42:0/2299205573 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f540403d3d0 0x7f540403f8e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f54100015c0 tx=0x7f5410007a90 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:47.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:47.799+0000 7f5421600640 1 -- 172.21.15.42:0/2299205573 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f5418046360 con 0x7f5424072570 2024-07-04T17:31:48.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.081+0000 7f542c200640 1 -- 172.21.15.42:0/2299205573 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f54241e73b0 con 0x7f5424072570 2024-07-04T17:31:48.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.083+0000 7f5421600640 1 -- 172.21.15.42:0/2299205573 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7f54180741c0 con 0x7f5424072570 2024-07-04T17:31:48.084 INFO:teuthology.orchestra.run.smithi042.stdout:[client.0] 2024-07-04T17:31:48.084 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAO24ZmUMmYABAAh4pNlkb9JsHQtK8uCJFQCA== 2024-07-04T17:31:48.085 INFO:teuthology.orchestra.run.smithi042.stdout: caps mds = "allow" 2024-07-04T17:31:48.085 INFO:teuthology.orchestra.run.smithi042.stdout: caps mon = "allow r" 2024-07-04T17:31:48.085 INFO:teuthology.orchestra.run.smithi042.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-07-04T17:31:48.085 INFO:teuthology.orchestra.run.smithi042.stderr:updated caps for client.0 2024-07-04T17:31:48.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.086+0000 7f542c200640 1 -- 172.21.15.42:0/2299205573 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f540403d3d0 msgr2=0x7f540403f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:48.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.086+0000 7f542c200640 1 --2- 172.21.15.42:0/2299205573 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f540403d3d0 0x7f540403f8e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f54100015c0 tx=0x7f5410007a90 comp rx=0 tx=0).stop 2024-07-04T17:31:48.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.086+0000 7f542c200640 1 -- 172.21.15.42:0/2299205573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5424072570 msgr2=0x7f54241e6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:48.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.086+0000 7f542c200640 1 --2- 172.21.15.42:0/2299205573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5424072570 0x7f54241e6c60 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f541802b750 tx=0x7f54180097d0 comp rx=0 tx=0).stop 2024-07-04T17:31:48.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.086+0000 7f542c200640 1 -- 172.21.15.42:0/2299205573 shutdown_connections 2024-07-04T17:31:48.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.086+0000 7f542c200640 1 --2- 172.21.15.42:0/2299205573 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f540403d3d0 0x7f540403f8e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:48.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.086+0000 7f542c200640 1 --2- 172.21.15.42:0/2299205573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5424072570 0x7f54241e6c60 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:48.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.086+0000 7f542c200640 1 -- 172.21.15.42:0/2299205573 >> 172.21.15.42:0/2299205573 conn(0x7f542406c020 msgr2=0x7f542413a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:48.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.086+0000 7f542c200640 1 -- 172.21.15.42:0/2299205573 shutdown_connections 2024-07-04T17:31:48.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.086+0000 7f542c200640 1 -- 172.21.15.42:0/2299205573 wait complete. 2024-07-04T17:31:48.114 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:31:48.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.407+0000 7f53d8c00640 1 -- 172.21.15.42:0/133002186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53d40b38b0 msgr2=0x7f53d40b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:48.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.407+0000 7f53d8c00640 1 --2- 172.21.15.42:0/133002186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53d40b38b0 0x7f53d40b2d20 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f53c4005fd0 tx=0x7f53c402a5f0 comp rx=0 tx=0).stop 2024-07-04T17:31:48.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.407+0000 7f53d8c00640 1 -- 172.21.15.42:0/133002186 shutdown_connections 2024-07-04T17:31:48.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.407+0000 7f53d8c00640 1 --2- 172.21.15.42:0/133002186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53d40b38b0 0x7f53d40b2d20 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:48.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.407+0000 7f53d8c00640 1 -- 172.21.15.42:0/133002186 >> 172.21.15.42:0/133002186 conn(0x7f53d40bf700 msgr2=0x7f53d40af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:48.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.407+0000 7f53d8c00640 1 -- 172.21.15.42:0/133002186 shutdown_connections 2024-07-04T17:31:48.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.407+0000 7f53d8c00640 1 -- 172.21.15.42:0/133002186 wait complete. 2024-07-04T17:31:48.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.410+0000 7f53d8c00640 1 Processor -- start 2024-07-04T17:31:48.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.410+0000 7f53d8c00640 1 -- start start 2024-07-04T17:31:48.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.412+0000 7f53d8c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53d40b1e00 0x7f53d41d4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:48.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.412+0000 7f53d8c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f53d40b06e0 con 0x7f53d40b1e00 2024-07-04T17:31:48.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.412+0000 7f53d1e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53d40b1e00 0x7f53d41d4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:48.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.412+0000 7f53d1e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53d40b1e00 0x7f53d41d4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54360/0 (socket says 172.21.15.42:54360) 2024-07-04T17:31:48.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.412+0000 7f53d1e00640 1 -- 172.21.15.42:0/1728135584 learned_addr learned my addr 172.21.15.42:0/1728135584 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:48.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.413+0000 7f53d1e00640 1 -- 172.21.15.42:0/1728135584 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d41d6740 con 0x7f53d40b1e00 2024-07-04T17:31:48.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.413+0000 7f53d1e00640 1 --2- 172.21.15.42:0/1728135584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53d40b1e00 0x7f53d41d4df0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f53c4009850 tx=0x7f53c40049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:48.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.414+0000 7f53caa00640 1 -- 172.21.15.42:0/1728135584 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53c403e030 con 0x7f53d40b1e00 2024-07-04T17:31:48.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.414+0000 7f53caa00640 1 -- 172.21.15.42:0/1728135584 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53c40035f0 con 0x7f53d40b1e00 2024-07-04T17:31:48.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.414+0000 7f53caa00640 1 -- 172.21.15.42:0/1728135584 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53c4035040 con 0x7f53d40b1e00 2024-07-04T17:31:48.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.414+0000 7f53d8c00640 1 -- 172.21.15.42:0/1728135584 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d40aa540 con 0x7f53d40b1e00 2024-07-04T17:31:48.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.414+0000 7f53d8c00640 1 -- 172.21.15.42:0/1728135584 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d41d5330 con 0x7f53d40b1e00 2024-07-04T17:31:48.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.415+0000 7f53d8c00640 1 -- 172.21.15.42:0/1728135584 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53d406c590 con 0x7f53d40b1e00 2024-07-04T17:31:48.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.416+0000 7f53caa00640 1 -- 172.21.15.42:0/1728135584 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7f53c4041030 con 0x7f53d40b1e00 2024-07-04T17:31:48.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.416+0000 7f53caa00640 1 --2- 172.21.15.42:0/1728135584 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f53b003d3d0 0x7f53b003f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:48.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.416+0000 7f53caa00640 1 -- 172.21.15.42:0/1728135584 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 3439+0+0 (secure 0 0 0) 0x7f53c403f070 con 0x7f53d40b1e00 2024-07-04T17:31:48.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.417+0000 7f53c9400640 1 --2- 172.21.15.42:0/1728135584 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f53b003d3d0 0x7f53b003f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:48.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.418+0000 7f53c9400640 1 --2- 172.21.15.42:0/1728135584 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f53b003d3d0 0x7f53b003f8e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f53d406ed80 tx=0x7f53bc0079f0 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:48.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.423+0000 7f53caa00640 1 -- 172.21.15.42:0/1728135584 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f53c404bd20 con 0x7f53d40b1e00 2024-07-04T17:31:48.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.671+0000 7f53d8c00640 1 -- 172.21.15.42:0/1728135584 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53d4069b70 con 0x7f53d40b1e00 2024-07-04T17:31:48.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.673+0000 7f53caa00640 1 -- 172.21.15.42:0/1728135584 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+6038 (secure 0 0 0) 0x7f53c4030c90 con 0x7f53d40b1e00 2024-07-04T17:31:48.674 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:31:48.674 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":71,"btime":"2024-07-04T17:31:45:109183+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/244742173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":244742173},{"type":"v1","addr":"172.21.15.42:6825","nonce":244742173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5265,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/973681058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":973681058},{"type":"v1","addr":"172.21.15.42:6827","nonce":973681058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/480649366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":480649366},{"type":"v1","addr":"172.21.15.42:6829","nonce":480649366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5276,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/327636003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":327636003},{"type":"v1","addr":"172.21.15.42:6831","nonce":327636003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:31:41.664414+0000","modified":"2024-07-04T17:31:44.709115+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5277":{"gid":5277,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6833/3777807763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":3777807763},{"type":"v1","addr":"172.21.15.42:6833","nonce":3777807763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5277,"qdb_cluster":[5277]},"id":8}]} 2024-07-04T17:31:48.675 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 71 2024-07-04T17:31:48.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.675+0000 7f53d8c00640 1 -- 172.21.15.42:0/1728135584 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f53b003d3d0 msgr2=0x7f53b003f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:48.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.675+0000 7f53d8c00640 1 --2- 172.21.15.42:0/1728135584 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f53b003d3d0 0x7f53b003f8e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f53d406ed80 tx=0x7f53bc0079f0 comp rx=0 tx=0).stop 2024-07-04T17:31:48.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.675+0000 7f53d8c00640 1 -- 172.21.15.42:0/1728135584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53d40b1e00 msgr2=0x7f53d41d4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:48.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.675+0000 7f53d8c00640 1 --2- 172.21.15.42:0/1728135584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53d40b1e00 0x7f53d41d4df0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f53c4009850 tx=0x7f53c40049d0 comp rx=0 tx=0).stop 2024-07-04T17:31:48.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.675+0000 7f53d8c00640 1 -- 172.21.15.42:0/1728135584 shutdown_connections 2024-07-04T17:31:48.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.675+0000 7f53d8c00640 1 --2- 172.21.15.42:0/1728135584 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f53b003d3d0 0x7f53b003f8e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:48.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.675+0000 7f53d8c00640 1 --2- 172.21.15.42:0/1728135584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53d40b1e00 0x7f53d41d4df0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:48.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.675+0000 7f53d8c00640 1 -- 172.21.15.42:0/1728135584 >> 172.21.15.42:0/1728135584 conn(0x7f53d40bf700 msgr2=0x7f53d41412f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:48.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.676+0000 7f53d8c00640 1 -- 172.21.15.42:0/1728135584 shutdown_connections 2024-07-04T17:31:48.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:48.676+0000 7f53d8c00640 1 -- 172.21.15.42:0/1728135584 wait complete. 2024-07-04T17:31:49.203 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-04T17:31:41.664414+0000', 'modified': '2024-07-04T17:31:44.709115+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5277}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5277': {'gid': 5277, 'name': 'a', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.42:6833/3777807763', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6832', 'nonce': 3777807763}, {'type': 'v1', 'addr': '172.21.15.42:6833', 'nonce': 3777807763}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5277, 'qdb_cluster': [5277]} 2024-07-04T17:31:49.204 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-04T17:31:49.204 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-07-04T17:31:49.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.508+0000 7f71b7000640 1 -- 172.21.15.42:0/725133229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71b00bc040 msgr2=0x7f71b00bd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:49.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.508+0000 7f71b7000640 1 --2- 172.21.15.42:0/725133229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71b00bc040 0x7f71b00bd2d0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f71a0005fd0 tx=0x7f71a002aac0 comp rx=0 tx=0).stop 2024-07-04T17:31:49.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.508+0000 7f71b7000640 1 -- 172.21.15.42:0/725133229 shutdown_connections 2024-07-04T17:31:49.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.508+0000 7f71b7000640 1 --2- 172.21.15.42:0/725133229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71b00bc040 0x7f71b00bd2d0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:49.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.508+0000 7f71b7000640 1 -- 172.21.15.42:0/725133229 >> 172.21.15.42:0/725133229 conn(0x7f71b006c940 msgr2=0x7f71b006ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:49.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.508+0000 7f71b7000640 1 -- 172.21.15.42:0/725133229 shutdown_connections 2024-07-04T17:31:49.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.508+0000 7f71b7000640 1 -- 172.21.15.42:0/725133229 wait complete. 2024-07-04T17:31:49.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.510+0000 7f71b7000640 1 Processor -- start 2024-07-04T17:31:49.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.510+0000 7f71b7000640 1 -- start start 2024-07-04T17:31:49.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.512+0000 7f71b7000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71b00bc040 0x7f71b01d9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:49.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.512+0000 7f71b7000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f71b00b7530 con 0x7f71b00bc040 2024-07-04T17:31:49.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.512+0000 7f71afe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71b00bc040 0x7f71b01d9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:49.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.512+0000 7f71afe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71b00bc040 0x7f71b01d9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54378/0 (socket says 172.21.15.42:54378) 2024-07-04T17:31:49.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.512+0000 7f71afe00640 1 -- 172.21.15.42:0/3482486555 learned_addr learned my addr 172.21.15.42:0/3482486555 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:49.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.513+0000 7f71afe00640 1 -- 172.21.15.42:0/3482486555 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b01e3740 con 0x7f71b00bc040 2024-07-04T17:31:49.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.513+0000 7f71afe00640 1 --2- 172.21.15.42:0/3482486555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71b00bc040 0x7f71b01d9610 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f71a0006010 tx=0x7f71a0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:49.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.513+0000 7f71acc00640 1 -- 172.21.15.42:0/3482486555 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71a0044070 con 0x7f71b00bc040 2024-07-04T17:31:49.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.513+0000 7f71acc00640 1 -- 172.21.15.42:0/3482486555 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71a0004250 con 0x7f71b00bc040 2024-07-04T17:31:49.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.513+0000 7f71acc00640 1 -- 172.21.15.42:0/3482486555 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71a00045d0 con 0x7f71b00bc040 2024-07-04T17:31:49.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.513+0000 7f71b7000640 1 -- 172.21.15.42:0/3482486555 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71b01e3940 con 0x7f71b00bc040 2024-07-04T17:31:49.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.513+0000 7f71b7000640 1 -- 172.21.15.42:0/3482486555 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71b00bfa90 con 0x7f71b00bc040 2024-07-04T17:31:49.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.515+0000 7f71acc00640 1 -- 172.21.15.42:0/3482486555 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7f71a0047070 con 0x7f71b00bc040 2024-07-04T17:31:49.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.516+0000 7f71acc00640 1 --2- 172.21.15.42:0/3482486555 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f718c03d3d0 0x7f718c03f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:49.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.516+0000 7f71acc00640 1 -- 172.21.15.42:0/3482486555 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 3439+0+0 (secure 0 0 0) 0x7f71a003c070 con 0x7f71b00bc040 2024-07-04T17:31:49.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.516+0000 7f71a7400640 1 --2- 172.21.15.42:0/3482486555 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f718c03d3d0 0x7f718c03f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:49.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.517+0000 7f71a7400640 1 --2- 172.21.15.42:0/3482486555 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f718c03d3d0 0x7f718c03f8e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7198001560 tx=0x7f71980079f0 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:49.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.518+0000 7f71bdfbe740 1 Processor -- start 2024-07-04T17:31:49.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.518+0000 7f71bdfbe740 1 -- start start 2024-07-04T17:31:49.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.519+0000 7f71bdfbe740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555654d95270 0x55565513bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:49.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.519+0000 7f71bdfbe740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5556550ac5b0 con 0x555654d95270 2024-07-04T17:31:49.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.520+0000 7f71afe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555654d95270 0x55565513bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:49.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.520+0000 7f71afe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555654d95270 0x55565513bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54382/0 (socket says 172.21.15.42:54382) 2024-07-04T17:31:49.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.520+0000 7f71afe00640 1 -- 172.21.15.42:0/4126657630 learned_addr learned my addr 172.21.15.42:0/4126657630 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:49.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.520+0000 7f71afe00640 1 -- 172.21.15.42:0/4126657630 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555654e00850 con 0x555654d95270 2024-07-04T17:31:49.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.520+0000 7f71afe00640 1 --2- 172.21.15.42:0/4126657630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555654d95270 0x55565513bb70 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f71a003e000 tx=0x7f71a003da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3ae5cf58a151b1 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:49.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.520+0000 7f71a4c00640 1 -- 172.21.15.42:0/4126657630 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71a0071070 con 0x555654d95270 2024-07-04T17:31:49.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.521+0000 7f71a4c00640 1 -- 172.21.15.42:0/4126657630 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71a0036400 con 0x555654d95270 2024-07-04T17:31:49.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.521+0000 7f71a4c00640 1 -- 172.21.15.42:0/4126657630 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71a003d310 con 0x555654d95270 2024-07-04T17:31:49.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.521+0000 7f71bdfbe740 1 -- 172.21.15.42:0/4126657630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555654d95270 msgr2=0x55565513bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:49.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.521+0000 7f71bdfbe740 1 --2- 172.21.15.42:0/4126657630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555654d95270 0x55565513bb70 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f71a003e000 tx=0x7f71a003da60 comp rx=0 tx=0).stop 2024-07-04T17:31:49.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.521+0000 7f71bdfbe740 1 -- 172.21.15.42:0/4126657630 shutdown_connections 2024-07-04T17:31:49.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.521+0000 7f71bdfbe740 1 --2- 172.21.15.42:0/4126657630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555654d95270 0x55565513bb70 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:49.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.521+0000 7f71bdfbe740 1 -- 172.21.15.42:0/4126657630 >> 172.21.15.42:0/4126657630 conn(0x555654d893c0 msgr2=0x55565513a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:49.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.522+0000 7f71bdfbe740 1 -- 172.21.15.42:0/4126657630 shutdown_connections 2024-07-04T17:31:49.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.522+0000 7f71bdfbe740 1 -- 172.21.15.42:0/4126657630 wait complete. 2024-07-04T17:31:49.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.522+0000 7f71bdfbe740 1 -- start start 2024-07-04T17:31:49.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.523+0000 7f71bdfbe740 1 Processor -- start 2024-07-04T17:31:49.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.524+0000 7f71bdfbe740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555654d95270 0x55565517e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:49.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.524+0000 7f71bdfbe740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55565509fb90 con 0x555654d95270 2024-07-04T17:31:49.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.524+0000 7f71bdfbe740 10 client.0 _ll_register_callbacks cb 0x5556550448a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T17:31:49.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.524+0000 7f71afe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555654d95270 0x55565517e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:49.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.524+0000 7f71afe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555654d95270 0x55565517e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54390/0 (socket says 172.21.15.42:54390) 2024-07-04T17:31:49.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.524+0000 7f71afe00640 1 -- 172.21.15.42:0/32334791 learned_addr learned my addr 172.21.15.42:0/32334791 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:49.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.525+0000 7f71afe00640 1 -- 172.21.15.42:0/32334791 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555654e776b0 con 0x555654d95270 2024-07-04T17:31:49.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.525+0000 7f71afe00640 1 --2- 172.21.15.42:0/32334791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555654d95270 0x55565517e7f0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f71a003dd10 tx=0x7f71a0036630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e6f2aa96efbef4 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:49.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.525+0000 7f719d600640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-07-04T17:31:49.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.525+0000 7f719d600640 1 -- 172.21.15.42:0/32334791 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71a0047360 con 0x555654d95270 2024-07-04T17:31:49.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.525+0000 7f719d600640 1 -- 172.21.15.42:0/32334791 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 3439+0+0 (secure 0 0 0) 0x7f71a0071070 con 0x555654d95270 2024-07-04T17:31:49.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.525+0000 7f71bdfbe740 1 -- 172.21.15.42:0/32334791 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55565503c6c0 con 0x555654d95270 2024-07-04T17:31:49.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.525+0000 7f719d600640 1 -- 172.21.15.42:0/32334791 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f71a0071270 con 0x555654d95270 2024-07-04T17:31:49.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.525+0000 7f71bdfbe740 10 client.5327 fetch_fsmap learned FSMap version 71 2024-07-04T17:31:49.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.525+0000 7f71bdfbe740 1 -- 172.21.15.42:0/32334791 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x55565517ee20 con 0x555654d95270 2024-07-04T17:31:49.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.526+0000 7f719d600640 1 -- 172.21.15.42:0/32334791 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55565503c6c0 con 0x555654d95270 2024-07-04T17:31:49.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.526+0000 7f719d600640 1 -- 172.21.15.42:0/32334791 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 71) ==== 4107+0+0 (secure 0 0 0) 0x7f71a005d080 con 0x555654d95270 2024-07-04T17:31:49.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.526+0000 7f71bdfbe740 10 client.5327 fetch_fsmap finished waiting for FSMap version 71 2024-07-04T17:31:49.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.526+0000 7f71bdfbe740 10 client.5327 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-04T17:31:49.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.526+0000 7f71bdfbe740 20 client.5327 populate_metadata read hostname 'smithi042' 2024-07-04T17:31:49.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.526+0000 7f71bdfbe740 1 --2- 172.21.15.42:0/32334791 >> [v2:172.21.15.42:6832/3777807763,v1:172.21.15.42:6833/3777807763] conn(0x555654d6c3b0 0x55565515eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:49.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.526+0000 7f71bdfbe740 4 client.5327 mds_command: new command op to 5277 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T17:31:49.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.526+0000 7f71bdfbe740 1 -- 172.21.15.42:0/32334791 --> [v2:172.21.15.42:6832/3777807763,v1:172.21.15.42:6833/3777807763] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556550ae500 con 0x555654d6c3b0 2024-07-04T17:31:49.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.526+0000 7f71a7400640 1 --2- 172.21.15.42:0/32334791 >> [v2:172.21.15.42:6832/3777807763,v1:172.21.15.42:6833/3777807763] conn(0x555654d6c3b0 0x55565515eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:49.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.527+0000 7f71a7400640 1 --2- 172.21.15.42:0/32334791 >> [v2:172.21.15.42:6832/3777807763,v1:172.21.15.42:6833/3777807763] conn(0x555654d6c3b0 0x55565515eed0 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9342e448cc29bab3 server_cookie=754592387ff0ee5e in_seq=0 out_seq=0 2024-07-04T17:31:49.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.527+0000 7f719d600640 10 client.5327 ms_handle_connect on v2:172.21.15.42:6832/3777807763 2024-07-04T17:31:49.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.528+0000 7f719d600640 1 -- 172.21.15.42:0/32334791 <== mds.0 v2:172.21.15.42:6832/3777807763 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5556550ae500 con 0x555654d6c3b0 2024-07-04T17:31:49.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.528+0000 7f719d600640 10 client.5327 handle_command_reply: tid=0 2024-07-04T17:31:49.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.528+0000 7f71bdfbe740 1 client.5327 shutdown 2024-07-04T17:31:49.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.528+0000 7f71bdfbe740 1 -- 172.21.15.42:0/32334791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555654d95270 msgr2=0x55565517e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:49.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.528+0000 7f71bdfbe740 1 --2- 172.21.15.42:0/32334791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555654d95270 0x55565517e7f0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f71a003dd10 tx=0x7f71a0036630 comp rx=0 tx=0).stop 2024-07-04T17:31:49.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.528+0000 7f71bdfbe740 1 -- 172.21.15.42:0/32334791 shutdown_connections 2024-07-04T17:31:49.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.528+0000 7f71bdfbe740 1 --2- 172.21.15.42:0/32334791 >> [v2:172.21.15.42:6832/3777807763,v1:172.21.15.42:6833/3777807763] conn(0x555654d6c3b0 0x55565515eed0 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:49.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.528+0000 7f71bdfbe740 1 --2- 172.21.15.42:0/32334791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555654d95270 0x55565517e7f0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:49.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.528+0000 7f71bdfbe740 1 -- 172.21.15.42:0/32334791 >> 172.21.15.42:0/32334791 conn(0x555654d893c0 msgr2=0x555655135080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:49.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.528+0000 7f719d600640 0 client.5327 ms_handle_reset on v2:172.21.15.42:6832/3777807763 2024-07-04T17:31:49.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.529+0000 7f71bdfbe740 1 -- 172.21.15.42:0/32334791 shutdown_connections 2024-07-04T17:31:49.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.529+0000 7f71bdfbe740 1 -- 172.21.15.42:0/32334791 wait complete. 2024-07-04T17:31:49.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.529+0000 7f71bdfbe740 20 client.5327 trim_cache size 0 max 16384 2024-07-04T17:31:49.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.550+0000 7f71bdfbe740 1 Processor -- start 2024-07-04T17:31:49.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.550+0000 7f71bdfbe740 1 -- start start 2024-07-04T17:31:49.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.551+0000 7f71bdfbe740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555654d95270 0x555655163d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:49.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.551+0000 7f71bdfbe740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f71b01e3b10 con 0x555654d95270 2024-07-04T17:31:49.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.552+0000 7f71afe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555654d95270 0x555655163d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:49.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.552+0000 7f71afe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555654d95270 0x555655163d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54402/0 (socket says 172.21.15.42:54402) 2024-07-04T17:31:49.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.552+0000 7f71afe00640 1 -- 172.21.15.42:0/3040962995 learned_addr learned my addr 172.21.15.42:0/3040962995 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:49.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.552+0000 7f71afe00640 1 -- 172.21.15.42:0/3040962995 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55565513bee0 con 0x555654d95270 2024-07-04T17:31:49.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.552+0000 7f71afe00640 1 --2- 172.21.15.42:0/3040962995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555654d95270 0x555655163d40 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f71a0003fa0 tx=0x7f71a003d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805100570a98ed50 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:49.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.553+0000 7f719fe00640 1 -- 172.21.15.42:0/3040962995 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71a0062040 con 0x555654d95270 2024-07-04T17:31:49.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.553+0000 7f719fe00640 1 -- 172.21.15.42:0/3040962995 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71a005d080 con 0x555654d95270 2024-07-04T17:31:49.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.553+0000 7f719fe00640 1 -- 172.21.15.42:0/3040962995 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71a003b4f0 con 0x555654d95270 2024-07-04T17:31:49.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.554+0000 7f71bdfbe740 1 -- 172.21.15.42:0/3040962995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555654d95270 msgr2=0x555655163d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:49.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.554+0000 7f71bdfbe740 1 --2- 172.21.15.42:0/3040962995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555654d95270 0x555655163d40 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f71a0003fa0 tx=0x7f71a003d520 comp rx=0 tx=0).stop 2024-07-04T17:31:49.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.554+0000 7f71bdfbe740 1 -- 172.21.15.42:0/3040962995 shutdown_connections 2024-07-04T17:31:49.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.554+0000 7f71bdfbe740 1 --2- 172.21.15.42:0/3040962995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555654d95270 0x555655163d40 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:49.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.554+0000 7f71bdfbe740 1 -- 172.21.15.42:0/3040962995 >> 172.21.15.42:0/3040962995 conn(0x555654d893c0 msgr2=0x5556551318b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:49.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.554+0000 7f71bdfbe740 1 -- 172.21.15.42:0/3040962995 shutdown_connections 2024-07-04T17:31:49.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.554+0000 7f71bdfbe740 1 -- 172.21.15.42:0/3040962995 wait complete. 2024-07-04T17:31:49.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.555+0000 7f71bdfbe740 1 -- start start 2024-07-04T17:31:49.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.555+0000 7f71bdfbe740 1 Processor -- start 2024-07-04T17:31:49.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.556+0000 7f71bdfbe740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555654d95270 0x55565518c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:49.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.556+0000 7f71bdfbe740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55565509bb50 con 0x555654d95270 2024-07-04T17:31:49.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.556+0000 7f71bdfbe740 10 client.0 _ll_register_callbacks cb 0x7f71a0036d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T17:31:49.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.557+0000 7f71afe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555654d95270 0x55565518c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:49.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.557+0000 7f71afe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555654d95270 0x55565518c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54414/0 (socket says 172.21.15.42:54414) 2024-07-04T17:31:49.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.557+0000 7f71afe00640 1 -- 172.21.15.42:0/2809960185 learned_addr learned my addr 172.21.15.42:0/2809960185 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:49.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.557+0000 7f71afe00640 1 -- 172.21.15.42:0/2809960185 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556551e5a20 con 0x555654d95270 2024-07-04T17:31:49.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.557+0000 7f71afe00640 1 --2- 172.21.15.42:0/2809960185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555654d95270 0x55565518c910 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f71a003dd10 tx=0x7f71a003b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdcceedebb7cffef server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:49.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.557+0000 7f719d600640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-07-04T17:31:49.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.558+0000 7f719d600640 1 -- 172.21.15.42:0/2809960185 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71a0062050 con 0x555654d95270 2024-07-04T17:31:49.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.558+0000 7f71bdfbe740 1 -- 172.21.15.42:0/2809960185 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71a005d080 con 0x555654d95270 2024-07-04T17:31:49.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.558+0000 7f719d600640 1 -- 172.21.15.42:0/2809960185 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 3439+0+0 (secure 0 0 0) 0x7f71a003baa0 con 0x555654d95270 2024-07-04T17:31:49.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.558+0000 7f719d600640 1 -- 172.21.15.42:0/2809960185 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f71a003bcf0 con 0x555654d95270 2024-07-04T17:31:49.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.558+0000 7f71bdfbe740 10 client.5329 fetch_fsmap learned FSMap version 71 2024-07-04T17:31:49.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.558+0000 7f71bdfbe740 1 -- 172.21.15.42:0/2809960185 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x5556551e5a20 con 0x555654d95270 2024-07-04T17:31:49.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.558+0000 7f719d600640 1 -- 172.21.15.42:0/2809960185 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71a005d080 con 0x555654d95270 2024-07-04T17:31:49.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.558+0000 7f719d600640 1 -- 172.21.15.42:0/2809960185 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 71) ==== 4107+0+0 (secure 0 0 0) 0x7f71a002b890 con 0x555654d95270 2024-07-04T17:31:49.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.558+0000 7f71bdfbe740 10 client.5329 fetch_fsmap finished waiting for FSMap version 71 2024-07-04T17:31:49.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.558+0000 7f71bdfbe740 10 client.5329 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-04T17:31:49.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.558+0000 7f71bdfbe740 20 client.5329 populate_metadata read hostname 'smithi042' 2024-07-04T17:31:49.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.558+0000 7f71bdfbe740 1 --2- 172.21.15.42:0/2809960185 >> [v2:172.21.15.42:6832/3777807763,v1:172.21.15.42:6833/3777807763] conn(0x555654d53e40 0x5556551e0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:49.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.558+0000 7f71bdfbe740 4 client.5329 mds_command: new command op to 5277 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-04T17:31:49.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.559+0000 7f71bdfbe740 1 -- 172.21.15.42:0/2809960185 --> [v2:172.21.15.42:6832/3777807763,v1:172.21.15.42:6833/3777807763] -- command(tid 0: {"prefix": "status"}) -- 0x5556550a9fa0 con 0x555654d53e40 2024-07-04T17:31:49.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.559+0000 7f71a7400640 1 --2- 172.21.15.42:0/2809960185 >> [v2:172.21.15.42:6832/3777807763,v1:172.21.15.42:6833/3777807763] conn(0x555654d53e40 0x5556551e0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:49.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.559+0000 7f71a7400640 1 --2- 172.21.15.42:0/2809960185 >> [v2:172.21.15.42:6832/3777807763,v1:172.21.15.42:6833/3777807763] conn(0x555654d53e40 0x5556551e0470 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c43da59845aa1ab server_cookie=a7020d7e538218eb in_seq=0 out_seq=0 2024-07-04T17:31:49.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.559+0000 7f719d600640 10 client.5329 ms_handle_connect on v2:172.21.15.42:6832/3777807763 2024-07-04T17:31:49.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.560+0000 7f719d600640 1 -- 172.21.15.42:0/2809960185 <== mds.0 v2:172.21.15.42:6832/3777807763 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5556550a9fa0 con 0x555654d53e40 2024-07-04T17:31:49.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.560+0000 7f719d600640 10 client.5329 handle_command_reply: tid=0 2024-07-04T17:31:49.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.560+0000 7f71bdfbe740 1 client.5329 shutdown 2024-07-04T17:31:49.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.560+0000 7f71bdfbe740 1 -- 172.21.15.42:0/2809960185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555654d95270 msgr2=0x55565518c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:49.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.560+0000 7f71bdfbe740 1 --2- 172.21.15.42:0/2809960185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555654d95270 0x55565518c910 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f71a003dd10 tx=0x7f71a003b570 comp rx=0 tx=0).stop 2024-07-04T17:31:49.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.560+0000 7f71bdfbe740 1 -- 172.21.15.42:0/2809960185 shutdown_connections 2024-07-04T17:31:49.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.560+0000 7f71bdfbe740 1 --2- 172.21.15.42:0/2809960185 >> [v2:172.21.15.42:6832/3777807763,v1:172.21.15.42:6833/3777807763] conn(0x555654d53e40 0x5556551e0470 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:49.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.560+0000 7f71bdfbe740 1 --2- 172.21.15.42:0/2809960185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555654d95270 0x55565518c910 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:49.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.560+0000 7f71bdfbe740 1 -- 172.21.15.42:0/2809960185 >> 172.21.15.42:0/2809960185 conn(0x555654d893c0 msgr2=0x555655135440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:49.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.560+0000 7f719d600640 0 client.5329 ms_handle_reset on v2:172.21.15.42:6832/3777807763 2024-07-04T17:31:49.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.560+0000 7f71bdfbe740 1 -- 172.21.15.42:0/2809960185 shutdown_connections 2024-07-04T17:31:49.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.560+0000 7f71bdfbe740 1 -- 172.21.15.42:0/2809960185 wait complete. 2024-07-04T17:31:49.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.560+0000 7f71bdfbe740 20 client.5329 trim_cache size 0 max 16384 2024-07-04T17:31:49.564 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:31:49.564 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "be4f300a-93ef-4489-b540-37ece39b0aac", 2024-07-04T17:31:49.564 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-07-04T17:31:49.564 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 5277, 2024-07-04T17:31:49.564 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-07-04T17:31:49.564 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-07-04T17:31:49.564 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-07-04T17:31:49.564 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 7.4794523240000004, 2024-07-04T17:31:49.564 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 71, 2024-07-04T17:31:49.564 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 92, 2024-07-04T17:31:49.564 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 92, 2024-07-04T17:31:49.564 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 12.271624428999999 2024-07-04T17:31:49.564 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:31:49.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.561+0000 7f71b7000640 1 -- 172.21.15.42:0/3482486555 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f718c03d3d0 msgr2=0x7f718c03f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:49.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.561+0000 7f71b7000640 1 --2- 172.21.15.42:0/3482486555 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f718c03d3d0 0x7f718c03f8e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7198001560 tx=0x7f71980079f0 comp rx=0 tx=0).stop 2024-07-04T17:31:49.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.561+0000 7f71b7000640 1 -- 172.21.15.42:0/3482486555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71b00bc040 msgr2=0x7f71b01d9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:49.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.561+0000 7f71b7000640 1 --2- 172.21.15.42:0/3482486555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71b00bc040 0x7f71b01d9610 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f71a0006010 tx=0x7f71a0003f10 comp rx=0 tx=0).stop 2024-07-04T17:31:49.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.561+0000 7f71b7000640 1 -- 172.21.15.42:0/3482486555 shutdown_connections 2024-07-04T17:31:49.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.561+0000 7f71b7000640 1 --2- 172.21.15.42:0/3482486555 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f718c03d3d0 0x7f718c03f8e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:49.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.561+0000 7f71b7000640 1 --2- 172.21.15.42:0/3482486555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71b00bc040 0x7f71b01d9610 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:49.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.561+0000 7f71b7000640 1 -- 172.21.15.42:0/3482486555 >> 172.21.15.42:0/3482486555 conn(0x7f71b006c940 msgr2=0x7f71b00740b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:49.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.561+0000 7f71b7000640 1 -- 172.21.15.42:0/3482486555 shutdown_connections 2024-07-04T17:31:49.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.561+0000 7f71b7000640 1 -- 172.21.15.42:0/3482486555 wait complete. 2024-07-04T17:31:49.587 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-04T17:31:49.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.878+0000 7f57bdc00640 1 -- 172.21.15.42:0/943141676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57b806cea0 msgr2=0x7f57b80b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:49.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.878+0000 7f57bdc00640 1 --2- 172.21.15.42:0/943141676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57b806cea0 0x7f57b80b3030 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f57a00097b0 tx=0x7f57a002a540 comp rx=0 tx=0).stop 2024-07-04T17:31:49.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.878+0000 7f57bdc00640 1 -- 172.21.15.42:0/943141676 shutdown_connections 2024-07-04T17:31:49.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.878+0000 7f57bdc00640 1 --2- 172.21.15.42:0/943141676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57b806cea0 0x7f57b80b3030 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:49.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.878+0000 7f57bdc00640 1 -- 172.21.15.42:0/943141676 >> 172.21.15.42:0/943141676 conn(0x7f57b80bf700 msgr2=0x7f57b80af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:49.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.878+0000 7f57bdc00640 1 -- 172.21.15.42:0/943141676 shutdown_connections 2024-07-04T17:31:49.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.878+0000 7f57bdc00640 1 -- 172.21.15.42:0/943141676 wait complete. 2024-07-04T17:31:49.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.881+0000 7f57bdc00640 1 Processor -- start 2024-07-04T17:31:49.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.881+0000 7f57bdc00640 1 -- start start 2024-07-04T17:31:49.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.882+0000 7f57bdc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57b806cea0 0x7f57b80bdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:49.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.882+0000 7f57bdc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f57b80b0580 con 0x7f57b806cea0 2024-07-04T17:31:49.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.882+0000 7f57b6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57b806cea0 0x7f57b80bdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:49.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.882+0000 7f57b6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57b806cea0 0x7f57b80bdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54430/0 (socket says 172.21.15.42:54430) 2024-07-04T17:31:49.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.882+0000 7f57b6a00640 1 -- 172.21.15.42:0/2177977396 learned_addr learned my addr 172.21.15.42:0/2177977396 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:49.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.883+0000 7f57b6a00640 1 -- 172.21.15.42:0/2177977396 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b8147e30 con 0x7f57b806cea0 2024-07-04T17:31:49.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.883+0000 7f57b6a00640 1 --2- 172.21.15.42:0/2177977396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57b806cea0 0x7f57b80bdac0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f57a800c320 tx=0x7f57a800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:49.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.883+0000 7f57aea00640 1 -- 172.21.15.42:0/2177977396 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57a80116c0 con 0x7f57b806cea0 2024-07-04T17:31:49.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.883+0000 7f57aea00640 1 -- 172.21.15.42:0/2177977396 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f57a8017070 con 0x7f57b806cea0 2024-07-04T17:31:49.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.883+0000 7f57bdc00640 1 -- 172.21.15.42:0/2177977396 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57b80ac8c0 con 0x7f57b806cea0 2024-07-04T17:31:49.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.883+0000 7f57aea00640 1 -- 172.21.15.42:0/2177977396 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57a8011b40 con 0x7f57b806cea0 2024-07-04T17:31:49.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.883+0000 7f57bdc00640 1 -- 172.21.15.42:0/2177977396 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57b80af100 con 0x7f57b806cea0 2024-07-04T17:31:49.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.885+0000 7f57bdc00640 1 -- 172.21.15.42:0/2177977396 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57b80b3030 con 0x7f57b806cea0 2024-07-04T17:31:49.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.885+0000 7f57aea00640 1 -- 172.21.15.42:0/2177977396 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7f57a8011ce0 con 0x7f57b806cea0 2024-07-04T17:31:49.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.885+0000 7f57aea00640 1 --2- 172.21.15.42:0/2177977396 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f579403d3d0 0x7f579403f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:49.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.886+0000 7f57aea00640 1 -- 172.21.15.42:0/2177977396 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 3439+0+0 (secure 0 0 0) 0x7f57a80513f0 con 0x7f57b806cea0 2024-07-04T17:31:49.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.886+0000 7f57afe00640 1 --2- 172.21.15.42:0/2177977396 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f579403d3d0 0x7f579403f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:49.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.887+0000 7f57afe00640 1 --2- 172.21.15.42:0/2177977396 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f579403d3d0 0x7f579403f8e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f57a0034000 tx=0x7f57a0004730 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:49.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:49.891+0000 7f57aea00640 1 -- 172.21.15.42:0/2177977396 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f57a800a140 con 0x7f57b806cea0 2024-07-04T17:31:50.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:50.151+0000 7f57bdc00640 1 -- 172.21.15.42:0/2177977396 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f57b8069b70 con 0x7f57b806cea0 2024-07-04T17:31:50.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:50.152+0000 7f57aea00640 1 -- 172.21.15.42:0/2177977396 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v71) ==== 53+0+73 (secure 0 0 0) 0x7f57a801ac60 con 0x7f57b806cea0 2024-07-04T17:31:50.153 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-04T17:31:50.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:50.155+0000 7f57bdc00640 1 -- 172.21.15.42:0/2177977396 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f579403d3d0 msgr2=0x7f579403f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:50.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:50.155+0000 7f57bdc00640 1 --2- 172.21.15.42:0/2177977396 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f579403d3d0 0x7f579403f8e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f57a0034000 tx=0x7f57a0004730 comp rx=0 tx=0).stop 2024-07-04T17:31:50.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:50.155+0000 7f57bdc00640 1 -- 172.21.15.42:0/2177977396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57b806cea0 msgr2=0x7f57b80bdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:50.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:50.155+0000 7f57bdc00640 1 --2- 172.21.15.42:0/2177977396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57b806cea0 0x7f57b80bdac0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f57a800c320 tx=0x7f57a800c7e0 comp rx=0 tx=0).stop 2024-07-04T17:31:50.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:50.155+0000 7f57bdc00640 1 -- 172.21.15.42:0/2177977396 shutdown_connections 2024-07-04T17:31:50.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:50.155+0000 7f57bdc00640 1 --2- 172.21.15.42:0/2177977396 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f579403d3d0 0x7f579403f8e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:50.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:50.155+0000 7f57bdc00640 1 --2- 172.21.15.42:0/2177977396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57b806cea0 0x7f57b80bdac0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:50.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:50.155+0000 7f57bdc00640 1 -- 172.21.15.42:0/2177977396 >> 172.21.15.42:0/2177977396 conn(0x7f57b80bf700 msgr2=0x7f57b8075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:50.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:50.155+0000 7f57bdc00640 1 -- 172.21.15.42:0/2177977396 shutdown_connections 2024-07-04T17:31:50.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:50.155+0000 7f57bdc00640 1 -- 172.21.15.42:0/2177977396 wait complete. 2024-07-04T17:31:50.181 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-07-04T17:31:50.181 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-04T17:31:50.181 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2024-07-04T17:31:50.181 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-04T17:31:50.181 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-04T17:31:50.181 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-07-04T17:31:50.181 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-07-04T17:31:50.182 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-04T17:31:50.182 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-04T17:31:50.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:31:50.182 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2024-07-04T17:31:50.228 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-04T17:31:50.228 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-04T17:31:50.228 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-07-04T17:31:50.228 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:31:50.228 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-07-04T17:31:50.228 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:31:50.228 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:31:50.229 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2024-07-04T17:31:50.229 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f0 2024-07-04T17:31:50.229 INFO:teuthology.orchestra.run.smithi042.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:31:50.229 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2024-07-04T17:31:50.229 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f1 2024-07-04T17:31:50.229 INFO:teuthology.orchestra.run.smithi042.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-04T17:31:50.229 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:31:50.229 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f0 2024-07-04T17:31:50.229 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-07-04T17:31:50.229 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 42260sec preferred_lft 42260sec 2024-07-04T17:31:50.229 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::ec4:7aff:febd:1440/64 scope link noprefixroute 2024-07-04T17:31:50.229 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:31:50.229 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:31:50.230 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:31:50.230 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f1 2024-07-04T17:31:50.230 INFO:teuthology.orchestra.run.smithi042.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-04T17:31:50.230 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 32:75:d6:f1:31:e1 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:31:50.230 INFO:teuthology.orchestra.run.smithi042.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-04T17:31:50.230 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:31:50.230 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::8839:a9ff:fe74:17f5/64 scope link 2024-07-04T17:31:50.230 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:31:50.230 INFO:teuthology.orchestra.run.smithi042.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-04T17:31:50.230 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether e6:ab:57:fb:ca:9e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:31:50.230 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::e4ab:57ff:fefb:ca9e/64 scope link 2024-07-04T17:31:50.230 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:31:50.230 INFO:teuthology.orchestra.run.smithi042.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-04T17:31:50.230 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 32:75:d6:f1:31:e1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:31:50.231 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::3075:d6ff:fef1:31e1/64 scope link 2024-07-04T17:31:50.231 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:31:50.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:31:50.232 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-04T17:31:50.302 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-07-04T17:31:50.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:31:50.307 DEBUG:teuthology.orchestra.run.smithi042:> route 2024-07-04T17:31:50.360 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2024-07-04T17:31:50.361 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-04T17:31:50.361 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-04T17:31:50.361 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-04T17:31:50.361 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-04T17:31:50.361 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-04T17:31:50.361 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-07-04T17:31:50.361 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-04T17:31:50.361 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-04T17:31:50.362 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-04T17:31:50.362 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-07-04T17:31:50.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-04T17:31:50.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:31:50.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:31:50.582 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-07-04T17:31:50.637 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-04T17:31:50.637 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2024-07-04T17:31:50.638 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-04T17:31:50.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:31:50.638 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.0 up 2024-07-04T17:31:50.726 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-07-04T17:31:50.726 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-04T17:31:50.726 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-04T17:31:50.785 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-04T17:31:50.786 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-04T17:31:50.786 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-04T17:31:50.838 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:31:50.923 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:31:50.981 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:31:50.981 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:31:50.981 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:31:50.981 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:31:50.981 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:31:50.982 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:31:50.982 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:31:50.982 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:31:50.982 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:31:50.982 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:31:50.982 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:31:50.982 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:31:50.982 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:31:50.982 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:31:50.982 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:31:50.982 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:31:50.982 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:31:50.982 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:31:50.983 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:31:50.983 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:31:50.983 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:31:50.983 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:31:50.983 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:31:50.983 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:31:50.983 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:31:50.983 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:31:50.983 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:31:50.983 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:31:50.983 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:31:50.983 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:31:50.983 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:31:50.983 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:31:50.984 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:31:50.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:31:50.984 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:31:51.041 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-07-04T17:31:51.041 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-07-04T17:31:51.050 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:31:51.123 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:31:51.168 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:31:51.168 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:31:51.168 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:31:51.168 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:31:51.168 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:31:51.169 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:31:51.169 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:31:51.169 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:31:51.169 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:31:51.169 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:31:51.169 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:31:51.169 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:31:51.169 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:31:51.169 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:31:51.169 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:31:51.169 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:31:51.169 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:31:51.170 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:31:51.170 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:31:51.170 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:31:51.170 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:31:51.170 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:31:51.170 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:31:51.170 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:31:51.170 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:31:51.170 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:31:51.170 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:31:51.170 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:31:51.170 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:31:51.170 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:31:51.171 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:31:51.171 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:31:51.171 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:31:51.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:31:51.172 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:31:51.311 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:2024-07-04T17:31:51.310+0000 7f21eebb8480 -1 init, newargv = 0x7f21d0004640 newargc=15 2024-07-04T17:31:51.311 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[60830]: starting ceph client 2024-07-04T17:31:51.318 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[60830]: starting fuse 2024-07-04T17:31:52.359 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:31:52.457 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:31:52.514 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:31:52.514 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:31:52.514 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:31:52.515 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:31:52.515 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:31:52.515 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:31:52.515 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:31:52.515 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:31:52.515 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:31:52.515 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:31:52.515 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:31:52.515 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:31:52.515 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:31:52.515 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:31:52.515 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:31:52.515 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:31:52.516 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:31:52.516 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:31:52.516 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:31:52.516 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:31:52.516 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:31:52.516 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:31:52.516 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:31:52.516 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:31:52.516 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:31:52.516 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:31:52.516 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:31:52.516 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:31:52.516 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:31:52.516 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:31:52.517 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:31:52.517 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:31:52.517 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:31:52.517 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-07-04T17:31:52.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:31:52.518 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:31:52.572 INFO:teuthology.orchestra.run.smithi042.stdout:42 2024-07-04T17:31:52.573 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-07-04T17:31:52.573 DEBUG:teuthology.orchestra.run.smithi042:> sudo stdin-killer -- python3 -c ' 2024-07-04T17:31:52.573 DEBUG:teuthology.orchestra.run.smithi042:> import glob 2024-07-04T17:31:52.573 DEBUG:teuthology.orchestra.run.smithi042:> import re 2024-07-04T17:31:52.573 DEBUG:teuthology.orchestra.run.smithi042:> import os 2024-07-04T17:31:52.573 DEBUG:teuthology.orchestra.run.smithi042:> import subprocess 2024-07-04T17:31:52.573 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:31:52.573 DEBUG:teuthology.orchestra.run.smithi042:> def _find_admin_socket(client_name): 2024-07-04T17:31:52.573 DEBUG:teuthology.orchestra.run.smithi042:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-07-04T17:31:52.573 DEBUG:teuthology.orchestra.run.smithi042:> files = glob.glob(asok_path) 2024-07-04T17:31:52.574 DEBUG:teuthology.orchestra.run.smithi042:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-07-04T17:31:52.574 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:31:52.574 DEBUG:teuthology.orchestra.run.smithi042:> # Given a non-glob path, it better be there 2024-07-04T17:31:52.574 DEBUG:teuthology.orchestra.run.smithi042:> if "*" not in asok_path: 2024-07-04T17:31:52.574 DEBUG:teuthology.orchestra.run.smithi042:> assert(len(files) == 1) 2024-07-04T17:31:52.574 DEBUG:teuthology.orchestra.run.smithi042:> return files[0] 2024-07-04T17:31:52.574 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:31:52.574 DEBUG:teuthology.orchestra.run.smithi042:> for f in files: 2024-07-04T17:31:52.574 DEBUG:teuthology.orchestra.run.smithi042:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-07-04T17:31:52.574 DEBUG:teuthology.orchestra.run.smithi042:> if os.path.exists("/proc/{0}".format(pid)): 2024-07-04T17:31:52.574 DEBUG:teuthology.orchestra.run.smithi042:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-07-04T17:31:52.574 DEBUG:teuthology.orchestra.run.smithi042:> contents = proc_f.read() 2024-07-04T17:31:52.574 DEBUG:teuthology.orchestra.run.smithi042:> if mountpoint in contents: 2024-07-04T17:31:52.575 DEBUG:teuthology.orchestra.run.smithi042:> return f 2024-07-04T17:31:52.575 DEBUG:teuthology.orchestra.run.smithi042:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-07-04T17:31:52.575 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:31:52.575 DEBUG:teuthology.orchestra.run.smithi042:> print(_find_admin_socket("client.0")) 2024-07-04T17:31:52.575 DEBUG:teuthology.orchestra.run.smithi042:> ' 2024-07-04T17:31:52.754 INFO:teuthology.orchestra.run.smithi042.stdout:/var/run/ceph/ceph-client.0.60830.asok 2024-07-04T17:31:52.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:31:52.772 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.60830.asok 2024-07-04T17:31:52.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:31:52.772 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.60830.asok status 2024-07-04T17:31:53.083 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:31:53.083 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": { 2024-07-04T17:31:53.083 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_sha1": "9dba1e836b4756bf3a32be2a4936f371dff55008", 2024-07-04T17:31:53.083 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.0.0-4667-g9dba1e83 (9dba1e836b4756bf3a32be2a4936f371dff55008) squid (dev)", 2024-07-04T17:31:53.083 INFO:teuthology.orchestra.run.smithi042.stdout: "entity_id": "0", 2024-07-04T17:31:53.084 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": "smithi042", 2024-07-04T17:31:53.084 INFO:teuthology.orchestra.run.smithi042.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-07-04T17:31:53.084 INFO:teuthology.orchestra.run.smithi042.stdout: "pid": "60830", 2024-07-04T17:31:53.084 INFO:teuthology.orchestra.run.smithi042.stdout: "root": "/" 2024-07-04T17:31:53.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:31:53.084 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_count": 0, 2024-07-04T17:31:53.084 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_pinned_count": 0, 2024-07-04T17:31:53.084 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 5335, 2024-07-04T17:31:53.084 INFO:teuthology.orchestra.run.smithi042.stdout: "inst": { 2024-07-04T17:31:53.084 INFO:teuthology.orchestra.run.smithi042.stdout: "name": { 2024-07-04T17:31:53.084 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "client", 2024-07-04T17:31:53.084 INFO:teuthology.orchestra.run.smithi042.stdout: "num": 5335 2024-07-04T17:31:53.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:31:53.085 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-07-04T17:31:53.085 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-07-04T17:31:53.085 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-07-04T17:31:53.085 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2211780041 2024-07-04T17:31:53.085 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:31:53.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:31:53.085 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-07-04T17:31:53.085 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-07-04T17:31:53.085 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-07-04T17:31:53.085 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2211780041 2024-07-04T17:31:53.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:31:53.085 INFO:teuthology.orchestra.run.smithi042.stdout: "inst_str": "client.5335 192.168.0.1:0/2211780041", 2024-07-04T17:31:53.086 INFO:teuthology.orchestra.run.smithi042.stdout: "addr_str": "192.168.0.1:0/2211780041", 2024-07-04T17:31:53.086 INFO:teuthology.orchestra.run.smithi042.stdout: "inode_count": 1, 2024-07-04T17:31:53.086 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_epoch": 71, 2024-07-04T17:31:53.086 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch": 92, 2024-07-04T17:31:53.086 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch_barrier": 0, 2024-07-04T17:31:53.086 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklisted": false, 2024-07-04T17:31:53.086 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs" 2024-07-04T17:31:53.086 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:31:53.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:31:53.105 DEBUG:teuthology.orchestra.run.smithi042:> stat --file-system '--printf=%T 2024-07-04T17:31:53.105 DEBUG:teuthology.orchestra.run.smithi042:> ' -- /home/ubuntu/cephtest/mnt.0 2024-07-04T17:31:53.159 INFO:teuthology.orchestra.run.smithi042.stdout:fuseblk 2024-07-04T17:31:53.160 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-07-04T17:31:53.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:31:53.160 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-04T17:31:53.252 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-04T17:31:53.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.539+0000 7ff8b8a00640 1 -- 172.21.15.42:0/936725329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8b40b2d20 msgr2=0x7ff8b40b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:53.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.539+0000 7ff8b8a00640 1 --2- 172.21.15.42:0/936725329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8b40b2d20 0x7ff8b40b01f0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ff8a00097b0 tx=0x7ff8a002a540 comp rx=0 tx=0).stop 2024-07-04T17:31:53.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.540+0000 7ff8b8a00640 1 -- 172.21.15.42:0/936725329 shutdown_connections 2024-07-04T17:31:53.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.540+0000 7ff8b8a00640 1 --2- 172.21.15.42:0/936725329 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8b40b2d20 0x7ff8b40b01f0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:53.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.540+0000 7ff8b8a00640 1 -- 172.21.15.42:0/936725329 >> 172.21.15.42:0/936725329 conn(0x7ff8b40bf700 msgr2=0x7ff8b40af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:53.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.540+0000 7ff8b8a00640 1 -- 172.21.15.42:0/936725329 shutdown_connections 2024-07-04T17:31:53.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.540+0000 7ff8b8a00640 1 -- 172.21.15.42:0/936725329 wait complete. 2024-07-04T17:31:53.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.542+0000 7ff8b8a00640 1 Processor -- start 2024-07-04T17:31:53.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.542+0000 7ff8b8a00640 1 -- start start 2024-07-04T17:31:53.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.543+0000 7ff8b8a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8b40b2d20 0x7ff8b413e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:53.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.543+0000 7ff8b8a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b406cf50 con 0x7ff8b40b2d20 2024-07-04T17:31:53.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.543+0000 7ff8b1e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8b40b2d20 0x7ff8b413e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:53.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.544+0000 7ff8b1e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8b40b2d20 0x7ff8b413e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54452/0 (socket says 172.21.15.42:54452) 2024-07-04T17:31:53.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.544+0000 7ff8b1e00640 1 -- 172.21.15.42:0/3044408007 learned_addr learned my addr 172.21.15.42:0/3044408007 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:53.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.544+0000 7ff8b1e00640 1 -- 172.21.15.42:0/3044408007 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b41dcab0 con 0x7ff8b40b2d20 2024-07-04T17:31:53.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.544+0000 7ff8b1e00640 1 --2- 172.21.15.42:0/3044408007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8b40b2d20 0x7ff8b413e280 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7ff8a400c320 tx=0x7ff8a400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:53.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.544+0000 7ff8aa000640 1 -- 172.21.15.42:0/3044408007 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8a40116c0 con 0x7ff8b40b2d20 2024-07-04T17:31:53.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.545+0000 7ff8aa000640 1 -- 172.21.15.42:0/3044408007 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8a4017070 con 0x7ff8b40b2d20 2024-07-04T17:31:53.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.545+0000 7ff8aa000640 1 -- 172.21.15.42:0/3044408007 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8a4011b40 con 0x7ff8b40b2d20 2024-07-04T17:31:53.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.545+0000 7ff8b8a00640 1 -- 172.21.15.42:0/3044408007 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b40ad730 con 0x7ff8b40b2d20 2024-07-04T17:31:53.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.545+0000 7ff8b8a00640 1 -- 172.21.15.42:0/3044408007 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b413e9a0 con 0x7ff8b40b2d20 2024-07-04T17:31:53.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.546+0000 7ff8b8a00640 1 -- 172.21.15.42:0/3044408007 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8b413ec90 con 0x7ff8b40b2d20 2024-07-04T17:31:53.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.546+0000 7ff8aa000640 1 -- 172.21.15.42:0/3044408007 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7ff8a4011ce0 con 0x7ff8b40b2d20 2024-07-04T17:31:53.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.547+0000 7ff8aa000640 1 --2- 172.21.15.42:0/3044408007 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7ff89403d3d0 0x7ff89403f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:53.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.547+0000 7ff8aa000640 1 -- 172.21.15.42:0/3044408007 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 3439+0+0 (secure 0 0 0) 0x7ff8a4051410 con 0x7ff8b40b2d20 2024-07-04T17:31:53.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.547+0000 7ff8b1400640 1 --2- 172.21.15.42:0/3044408007 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7ff89403d3d0 0x7ff89403f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:53.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.548+0000 7ff8b1400640 1 --2- 172.21.15.42:0/3044408007 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7ff89403d3d0 0x7ff89403f8e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0034000 tx=0x7ff8a0004730 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:53.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.552+0000 7ff8aa000640 1 -- 172.21.15.42:0/3044408007 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7ff8a401ac80 con 0x7ff8b40b2d20 2024-07-04T17:31:53.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.802+0000 7ff8b8a00640 1 -- 172.21.15.42:0/3044408007 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff8b40b01f0 con 0x7ff8b40b2d20 2024-07-04T17:31:53.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.803+0000 7ff8aa000640 1 -- 172.21.15.42:0/3044408007 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v71) ==== 53+0+73 (secure 0 0 0) 0x7ff8a40102e0 con 0x7ff8b40b2d20 2024-07-04T17:31:53.804 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-04T17:31:53.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.805+0000 7ff8b8a00640 1 -- 172.21.15.42:0/3044408007 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7ff89403d3d0 msgr2=0x7ff89403f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:53.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.805+0000 7ff8b8a00640 1 --2- 172.21.15.42:0/3044408007 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7ff89403d3d0 0x7ff89403f8e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0034000 tx=0x7ff8a0004730 comp rx=0 tx=0).stop 2024-07-04T17:31:53.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.805+0000 7ff8b8a00640 1 -- 172.21.15.42:0/3044408007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8b40b2d20 msgr2=0x7ff8b413e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:53.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.805+0000 7ff8b8a00640 1 --2- 172.21.15.42:0/3044408007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8b40b2d20 0x7ff8b413e280 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7ff8a400c320 tx=0x7ff8a400c7e0 comp rx=0 tx=0).stop 2024-07-04T17:31:53.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.805+0000 7ff8b8a00640 1 -- 172.21.15.42:0/3044408007 shutdown_connections 2024-07-04T17:31:53.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.805+0000 7ff8b8a00640 1 --2- 172.21.15.42:0/3044408007 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7ff89403d3d0 0x7ff89403f8e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:53.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.805+0000 7ff8b8a00640 1 --2- 172.21.15.42:0/3044408007 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8b40b2d20 0x7ff8b413e280 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:53.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.805+0000 7ff8b8a00640 1 -- 172.21.15.42:0/3044408007 >> 172.21.15.42:0/3044408007 conn(0x7ff8b40bf700 msgr2=0x7ff8b40bd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:53.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.806+0000 7ff8b8a00640 1 -- 172.21.15.42:0/3044408007 shutdown_connections 2024-07-04T17:31:53.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:53.806+0000 7ff8b8a00640 1 -- 172.21.15.42:0/3044408007 wait complete. 2024-07-04T17:31:53.836 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-07-04T17:31:53.836 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-04T17:31:53.836 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2024-07-04T17:31:53.837 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-07-04T17:31:53.837 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-07-04T17:31:53.837 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-07-04T17:31:53.837 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-07-04T17:31:53.837 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-04T17:31:53.837 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-07-04T17:31:53.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:31:53.837 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2024-07-04T17:31:53.890 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-04T17:31:53.890 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-04T17:31:53.890 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-07-04T17:31:53.890 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:31:53.890 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-07-04T17:31:53.890 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:31:53.890 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:31:53.890 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2024-07-04T17:31:53.891 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f0 2024-07-04T17:31:53.891 INFO:teuthology.orchestra.run.smithi042.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:31:53.891 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2024-07-04T17:31:53.891 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f1 2024-07-04T17:31:53.891 INFO:teuthology.orchestra.run.smithi042.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-04T17:31:53.891 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:31:53.891 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f0 2024-07-04T17:31:53.891 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-07-04T17:31:53.891 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 42256sec preferred_lft 42256sec 2024-07-04T17:31:53.891 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::ec4:7aff:febd:1440/64 scope link noprefixroute 2024-07-04T17:31:53.891 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:31:53.891 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:31:53.891 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:31:53.892 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f1 2024-07-04T17:31:53.892 INFO:teuthology.orchestra.run.smithi042.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-04T17:31:53.892 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 32:75:d6:f1:31:e1 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:31:53.892 INFO:teuthology.orchestra.run.smithi042.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-04T17:31:53.892 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:31:53.892 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::8839:a9ff:fe74:17f5/64 scope link 2024-07-04T17:31:53.892 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:31:53.892 INFO:teuthology.orchestra.run.smithi042.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-04T17:31:53.892 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether e6:ab:57:fb:ca:9e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:31:53.892 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::e4ab:57ff:fefb:ca9e/64 scope link 2024-07-04T17:31:53.892 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:31:53.892 INFO:teuthology.orchestra.run.smithi042.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-04T17:31:53.892 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 32:75:d6:f1:31:e1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:31:53.893 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::3075:d6ff:fef1:31e1/64 scope link 2024-07-04T17:31:53.893 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:31:53.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:31:53.894 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-04T17:31:53.976 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-07-04T17:31:53.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:31:53.980 DEBUG:teuthology.orchestra.run.smithi042:> route 2024-07-04T17:31:54.036 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2024-07-04T17:31:54.036 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-04T17:31:54.036 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-04T17:31:54.037 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-04T17:31:54.037 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-04T17:31:54.038 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-04T17:31:54.038 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-07-04T17:31:54.038 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-04T17:31:54.038 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-04T17:31:54.038 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-04T17:31:54.038 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-07-04T17:31:54.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-04T17:31:54.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:31:54.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:31:54.266 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-07-04T17:31:54.315 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-04T17:31:54.315 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2024-07-04T17:31:54.316 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-07-04T17:31:54.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:31:54.317 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.1 up 2024-07-04T17:31:54.408 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-07-04T17:31:54.409 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-04T17:31:54.409 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-07-04T17:31:54.467 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-07-04T17:31:54.468 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-04T17:31:54.468 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-07-04T17:31:54.523 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:31:54.609 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:31:54.667 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:31:54.667 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:31:54.667 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:31:54.668 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:31:54.668 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:31:54.668 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:31:54.668 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:31:54.668 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:31:54.668 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:31:54.668 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:31:54.668 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:31:54.668 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:31:54.668 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:31:54.668 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:31:54.668 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:31:54.668 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:31:54.668 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:31:54.669 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:31:54.669 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:31:54.669 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:31:54.669 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:31:54.669 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:31:54.669 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:31:54.669 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:31:54.669 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:31:54.669 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:31:54.669 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:31:54.669 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:31:54.669 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:31:54.669 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:31:54.669 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:31:54.670 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:31:54.670 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:31:54.670 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-07-04T17:31:54.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:31:54.671 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:31:54.723 INFO:teuthology.orchestra.run.smithi042.stdout:42 2024-07-04T17:31:54.723 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-07-04T17:31:54.723 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-07-04T17:31:54.730 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:31:54.802 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:31:54.844 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:31:54.844 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:31:54.844 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:31:54.844 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:31:54.844 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:31:54.844 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:31:54.844 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:31:54.844 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:31:54.844 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:31:54.844 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:31:54.844 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:31:54.845 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:31:54.845 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:31:54.845 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:31:54.845 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:31:54.845 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:31:54.845 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:31:54.845 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:31:54.845 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:31:54.845 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:31:54.845 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:31:54.845 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:31:54.845 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:31:54.845 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:31:54.845 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:31:54.845 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:31:54.846 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:31:54.846 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:31:54.846 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:31:54.846 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:31:54.846 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:31:54.846 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:31:54.846 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:31:54.846 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-07-04T17:31:54.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:31:54.847 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:31:54.937 INFO:teuthology.orchestra.run.smithi042.stdout:42 2024-07-04T17:31:54.982 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi042.stderr:2024-07-04T17:31:54.981+0000 7f5fa3529480 -1 init, newargv = 0x7f5f84004580 newargc=15 2024-07-04T17:31:54.982 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi042.stderr:ceph-fuse[61299]: starting ceph client 2024-07-04T17:31:54.990 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi042.stderr:ceph-fuse[61299]: starting fuse 2024-07-04T17:31:56.038 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:31:56.135 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:31:56.195 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:31:56.195 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:31:56.195 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:31:56.195 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:31:56.195 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:31:56.195 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:31:56.195 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:31:56.195 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:31:56.196 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:31:56.196 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:31:56.196 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:31:56.196 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:31:56.196 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:31:56.196 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:31:56.196 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:31:56.196 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:31:56.196 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:31:56.196 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:31:56.196 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:31:56.196 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:31:56.196 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:31:56.197 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:31:56.197 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:31:56.197 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:31:56.197 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:31:56.197 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:31:56.197 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:31:56.197 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:31:56.197 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:31:56.197 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:31:56.197 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:31:56.197 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:31:56.197 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:31:56.197 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-07-04T17:31:56.197 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.1 2024-07-04T17:31:56.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:31:56.199 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:31:56.256 INFO:teuthology.orchestra.run.smithi042.stdout:42 2024-07-04T17:31:56.257 INFO:teuthology.orchestra.run.smithi042.stdout:43 2024-07-04T17:31:56.257 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 43] 2024-07-04T17:31:56.257 DEBUG:teuthology.orchestra.run.smithi042:> sudo stdin-killer -- python3 -c ' 2024-07-04T17:31:56.258 DEBUG:teuthology.orchestra.run.smithi042:> import glob 2024-07-04T17:31:56.258 DEBUG:teuthology.orchestra.run.smithi042:> import re 2024-07-04T17:31:56.258 DEBUG:teuthology.orchestra.run.smithi042:> import os 2024-07-04T17:31:56.258 DEBUG:teuthology.orchestra.run.smithi042:> import subprocess 2024-07-04T17:31:56.258 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:31:56.258 DEBUG:teuthology.orchestra.run.smithi042:> def _find_admin_socket(client_name): 2024-07-04T17:31:56.258 DEBUG:teuthology.orchestra.run.smithi042:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-07-04T17:31:56.258 DEBUG:teuthology.orchestra.run.smithi042:> files = glob.glob(asok_path) 2024-07-04T17:31:56.258 DEBUG:teuthology.orchestra.run.smithi042:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-07-04T17:31:56.259 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:31:56.259 DEBUG:teuthology.orchestra.run.smithi042:> # Given a non-glob path, it better be there 2024-07-04T17:31:56.259 DEBUG:teuthology.orchestra.run.smithi042:> if "*" not in asok_path: 2024-07-04T17:31:56.259 DEBUG:teuthology.orchestra.run.smithi042:> assert(len(files) == 1) 2024-07-04T17:31:56.259 DEBUG:teuthology.orchestra.run.smithi042:> return files[0] 2024-07-04T17:31:56.259 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:31:56.259 DEBUG:teuthology.orchestra.run.smithi042:> for f in files: 2024-07-04T17:31:56.259 DEBUG:teuthology.orchestra.run.smithi042:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-07-04T17:31:56.259 DEBUG:teuthology.orchestra.run.smithi042:> if os.path.exists("/proc/{0}".format(pid)): 2024-07-04T17:31:56.259 DEBUG:teuthology.orchestra.run.smithi042:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-07-04T17:31:56.259 DEBUG:teuthology.orchestra.run.smithi042:> contents = proc_f.read() 2024-07-04T17:31:56.259 DEBUG:teuthology.orchestra.run.smithi042:> if mountpoint in contents: 2024-07-04T17:31:56.259 DEBUG:teuthology.orchestra.run.smithi042:> return f 2024-07-04T17:31:56.259 DEBUG:teuthology.orchestra.run.smithi042:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-07-04T17:31:56.259 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:31:56.259 DEBUG:teuthology.orchestra.run.smithi042:> print(_find_admin_socket("client.1")) 2024-07-04T17:31:56.259 DEBUG:teuthology.orchestra.run.smithi042:> ' 2024-07-04T17:31:56.441 INFO:teuthology.orchestra.run.smithi042.stdout:/var/run/ceph/ceph-client.1.61299.asok 2024-07-04T17:31:56.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:31:56.460 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.61299.asok 2024-07-04T17:31:56.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:31:56.460 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.61299.asok status 2024-07-04T17:31:56.772 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:31:56.772 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": { 2024-07-04T17:31:56.772 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_sha1": "9dba1e836b4756bf3a32be2a4936f371dff55008", 2024-07-04T17:31:56.773 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.0.0-4667-g9dba1e83 (9dba1e836b4756bf3a32be2a4936f371dff55008) squid (dev)", 2024-07-04T17:31:56.773 INFO:teuthology.orchestra.run.smithi042.stdout: "entity_id": "1", 2024-07-04T17:31:56.773 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": "smithi042", 2024-07-04T17:31:56.773 INFO:teuthology.orchestra.run.smithi042.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-07-04T17:31:56.773 INFO:teuthology.orchestra.run.smithi042.stdout: "pid": "61299", 2024-07-04T17:31:56.773 INFO:teuthology.orchestra.run.smithi042.stdout: "root": "/" 2024-07-04T17:31:56.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:31:56.774 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_count": 0, 2024-07-04T17:31:56.774 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_pinned_count": 0, 2024-07-04T17:31:56.774 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 5341, 2024-07-04T17:31:56.774 INFO:teuthology.orchestra.run.smithi042.stdout: "inst": { 2024-07-04T17:31:56.775 INFO:teuthology.orchestra.run.smithi042.stdout: "name": { 2024-07-04T17:31:56.775 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "client", 2024-07-04T17:31:56.775 INFO:teuthology.orchestra.run.smithi042.stdout: "num": 5341 2024-07-04T17:31:56.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:31:56.775 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-07-04T17:31:56.775 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-07-04T17:31:56.776 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.2:0", 2024-07-04T17:31:56.776 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3061801109 2024-07-04T17:31:56.776 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:31:56.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:31:56.776 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-07-04T17:31:56.776 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-07-04T17:31:56.777 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.2:0", 2024-07-04T17:31:56.777 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3061801109 2024-07-04T17:31:56.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:31:56.777 INFO:teuthology.orchestra.run.smithi042.stdout: "inst_str": "client.5341 192.168.0.2:0/3061801109", 2024-07-04T17:31:56.777 INFO:teuthology.orchestra.run.smithi042.stdout: "addr_str": "192.168.0.2:0/3061801109", 2024-07-04T17:31:56.777 INFO:teuthology.orchestra.run.smithi042.stdout: "inode_count": 1, 2024-07-04T17:31:56.778 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_epoch": 71, 2024-07-04T17:31:56.778 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch": 92, 2024-07-04T17:31:56.778 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch_barrier": 0, 2024-07-04T17:31:56.778 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklisted": false, 2024-07-04T17:31:56.778 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs" 2024-07-04T17:31:56.779 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:31:56.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:31:56.794 DEBUG:teuthology.orchestra.run.smithi042:> stat --file-system '--printf=%T 2024-07-04T17:31:56.794 DEBUG:teuthology.orchestra.run.smithi042:> ' -- /home/ubuntu/cephtest/mnt.1 2024-07-04T17:31:56.851 INFO:teuthology.orchestra.run.smithi042.stdout:fuseblk 2024-07-04T17:31:56.851 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-07-04T17:31:56.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:31:56.851 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-07-04T17:31:56.941 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-07-04T17:31:57.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:57.242+0000 7f552a600640 1 -- 172.21.15.42:0/2111834692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55240ac5b0 msgr2=0x7f55240b0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:57.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:57.242+0000 7f552a600640 1 --2- 172.21.15.42:0/2111834692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55240ac5b0 0x7f55240b0db0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f5510005fd0 tx=0x7f551002b890 comp rx=0 tx=0).stop 2024-07-04T17:31:57.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:57.242+0000 7f552a600640 1 -- 172.21.15.42:0/2111834692 shutdown_connections 2024-07-04T17:31:57.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:57.242+0000 7f552a600640 1 --2- 172.21.15.42:0/2111834692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55240ac5b0 0x7f55240b0db0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:57.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:57.242+0000 7f552a600640 1 -- 172.21.15.42:0/2111834692 >> 172.21.15.42:0/2111834692 conn(0x7f55240afaa0 msgr2=0x7f55240afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:57.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:57.242+0000 7f552a600640 1 -- 172.21.15.42:0/2111834692 shutdown_connections 2024-07-04T17:31:57.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:57.242+0000 7f552a600640 1 -- 172.21.15.42:0/2111834692 wait complete. 2024-07-04T17:31:57.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:57.245+0000 7f552a600640 1 Processor -- start 2024-07-04T17:31:57.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:57.245+0000 7f552a600640 1 -- start start 2024-07-04T17:31:57.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:57.245+0000 7f552a600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f552406da60 0x7f5524073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:57.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:57.245+0000 7f552a600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f55240ae590 con 0x7f552406da60 2024-07-04T17:31:57.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:57.246+0000 7f5529200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f552406da60 0x7f5524073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:57.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:57.246+0000 7f5529200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f552406da60 0x7f5524073e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34924/0 (socket says 172.21.15.42:34924) 2024-07-04T17:31:57.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:57.246+0000 7f5529200640 1 -- 172.21.15.42:0/677006112 learned_addr learned my addr 172.21.15.42:0/677006112 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:57.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:57.246+0000 7f5529200640 1 -- 172.21.15.42:0/677006112 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5524144040 con 0x7f552406da60 2024-07-04T17:31:57.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:57.247+0000 7f5529200640 1 --2- 172.21.15.42:0/677006112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f552406da60 0x7f5524073e10 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f551800c380 tx=0x7f551800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:57.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:57.247+0000 7f5520c00640 1 -- 172.21.15.42:0/677006112 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5518017070 con 0x7f552406da60 2024-07-04T17:31:57.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:57.247+0000 7f5520c00640 1 -- 172.21.15.42:0/677006112 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55180052f0 con 0x7f552406da60 2024-07-04T17:31:57.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:57.247+0000 7f552a600640 1 -- 172.21.15.42:0/677006112 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55241d7be0 con 0x7f552406da60 2024-07-04T17:31:57.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:57.247+0000 7f5520c00640 1 -- 172.21.15.42:0/677006112 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55180035f0 con 0x7f552406da60 2024-07-04T17:31:57.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:57.247+0000 7f552a600640 1 -- 172.21.15.42:0/677006112 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f552413d810 con 0x7f552406da60 2024-07-04T17:31:57.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:57.248+0000 7f5520c00640 1 -- 172.21.15.42:0/677006112 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7f551801a070 con 0x7f552406da60 2024-07-04T17:31:57.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:57.249+0000 7f552a600640 1 -- 172.21.15.42:0/677006112 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55240b0db0 con 0x7f552406da60 2024-07-04T17:31:57.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:57.249+0000 7f5520c00640 1 --2- 172.21.15.42:0/677006112 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f550403d3d0 0x7f550403f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:57.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:57.249+0000 7f5520c00640 1 -- 172.21.15.42:0/677006112 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 3439+0+0 (secure 0 0 0) 0x7f5518050a00 con 0x7f552406da60 2024-07-04T17:31:57.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:57.249+0000 7f5523e00640 1 --2- 172.21.15.42:0/677006112 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f550403d3d0 0x7f550403f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:57.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:57.250+0000 7f5523e00640 1 --2- 172.21.15.42:0/677006112 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f550403d3d0 0x7f550403f8e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5510035000 tx=0x7f5510004730 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:57.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:57.255+0000 7f5520c00640 1 -- 172.21.15.42:0/677006112 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f5518011630 con 0x7f552406da60 2024-07-04T17:31:57.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:57.513+0000 7f552a600640 1 -- 172.21.15.42:0/677006112 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f5524068e30 con 0x7f552406da60 2024-07-04T17:31:58.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.512+0000 7f5520c00640 1 -- 172.21.15.42:0/677006112 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v72) ==== 136+0+0 (secure 0 0 0) 0x7f55180192c0 con 0x7f552406da60 2024-07-04T17:31:58.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.516+0000 7f552a600640 1 -- 172.21.15.42:0/677006112 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f550403d3d0 msgr2=0x7f550403f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:58.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.516+0000 7f552a600640 1 --2- 172.21.15.42:0/677006112 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f550403d3d0 0x7f550403f8e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5510035000 tx=0x7f5510004730 comp rx=0 tx=0).stop 2024-07-04T17:31:58.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.516+0000 7f552a600640 1 -- 172.21.15.42:0/677006112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f552406da60 msgr2=0x7f5524073e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:58.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.516+0000 7f552a600640 1 --2- 172.21.15.42:0/677006112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f552406da60 0x7f5524073e10 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f551800c380 tx=0x7f551800c840 comp rx=0 tx=0).stop 2024-07-04T17:31:58.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.516+0000 7f552a600640 1 -- 172.21.15.42:0/677006112 shutdown_connections 2024-07-04T17:31:58.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.516+0000 7f552a600640 1 --2- 172.21.15.42:0/677006112 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f550403d3d0 0x7f550403f8e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:58.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.516+0000 7f552a600640 1 --2- 172.21.15.42:0/677006112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f552406da60 0x7f5524073e10 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:58.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.516+0000 7f552a600640 1 -- 172.21.15.42:0/677006112 >> 172.21.15.42:0/677006112 conn(0x7f55240afaa0 msgr2=0x7f55240a9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:58.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.517+0000 7f552a600640 1 -- 172.21.15.42:0/677006112 shutdown_connections 2024-07-04T17:31:58.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.517+0000 7f552a600640 1 -- 172.21.15.42:0/677006112 wait complete. 2024-07-04T17:31:58.546 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-04T17:31:58.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.838+0000 7faa0c200640 1 -- 172.21.15.42:0/2926276787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa040b2930 msgr2=0x7faa040b02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:58.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.838+0000 7faa0c200640 1 --2- 172.21.15.42:0/2926276787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa040b2930 0x7faa040b02b0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0005fd0 tx=0x7fa9f002a5f0 comp rx=0 tx=0).stop 2024-07-04T17:31:58.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.838+0000 7faa0c200640 1 -- 172.21.15.42:0/2926276787 shutdown_connections 2024-07-04T17:31:58.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.838+0000 7faa0c200640 1 --2- 172.21.15.42:0/2926276787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa040b2930 0x7faa040b02b0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:58.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.838+0000 7faa0c200640 1 -- 172.21.15.42:0/2926276787 >> 172.21.15.42:0/2926276787 conn(0x7faa040bf700 msgr2=0x7faa040af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:58.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.838+0000 7faa0c200640 1 -- 172.21.15.42:0/2926276787 shutdown_connections 2024-07-04T17:31:58.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.838+0000 7faa0c200640 1 -- 172.21.15.42:0/2926276787 wait complete. 2024-07-04T17:31:58.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.840+0000 7faa0c200640 1 Processor -- start 2024-07-04T17:31:58.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.841+0000 7faa0c200640 1 -- start start 2024-07-04T17:31:58.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.842+0000 7faa0c200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa0406c8b0 0x7faa040ad130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:58.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.842+0000 7faa0c200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faa04146900 con 0x7faa0406c8b0 2024-07-04T17:31:58.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.842+0000 7faa09800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa0406c8b0 0x7faa040ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:58.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.842+0000 7faa09800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa0406c8b0 0x7faa040ad130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34940/0 (socket says 172.21.15.42:34940) 2024-07-04T17:31:58.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.842+0000 7faa09800640 1 -- 172.21.15.42:0/4068423051 learned_addr learned my addr 172.21.15.42:0/4068423051 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:58.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.844+0000 7faa09800640 1 -- 172.21.15.42:0/4068423051 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa04140560 con 0x7faa0406c8b0 2024-07-04T17:31:58.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.844+0000 7faa09800640 1 --2- 172.21.15.42:0/4068423051 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa0406c8b0 0x7faa040ad130 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800c380 tx=0x7fa9f800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:58.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.844+0000 7faa01600640 1 -- 172.21.15.42:0/4068423051 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9f8017070 con 0x7faa0406c8b0 2024-07-04T17:31:58.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.844+0000 7faa01600640 1 -- 172.21.15.42:0/4068423051 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9f80052f0 con 0x7faa0406c8b0 2024-07-04T17:31:58.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.844+0000 7faa0c200640 1 -- 172.21.15.42:0/4068423051 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa04140a80 con 0x7faa0406c8b0 2024-07-04T17:31:58.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.844+0000 7faa01600640 1 -- 172.21.15.42:0/4068423051 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9f80035f0 con 0x7faa0406c8b0 2024-07-04T17:31:58.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.844+0000 7faa0c200640 1 -- 172.21.15.42:0/4068423051 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa041401e0 con 0x7faa0406c8b0 2024-07-04T17:31:58.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.846+0000 7faa0c200640 1 -- 172.21.15.42:0/4068423051 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa040b02b0 con 0x7faa0406c8b0 2024-07-04T17:31:58.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.846+0000 7faa01600640 1 -- 172.21.15.42:0/4068423051 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7fa9f801a070 con 0x7faa0406c8b0 2024-07-04T17:31:58.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.847+0000 7faa01600640 1 --2- 172.21.15.42:0/4068423051 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fa9e403d3d0 0x7fa9e403f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:58.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.847+0000 7faa01600640 1 -- 172.21.15.42:0/4068423051 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 3439+0+0 (secure 0 0 0) 0x7fa9f8050a60 con 0x7faa0406c8b0 2024-07-04T17:31:58.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.847+0000 7faa08e00640 1 --2- 172.21.15.42:0/4068423051 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fa9e403d3d0 0x7fa9e403f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:58.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.847+0000 7faa08e00640 1 --2- 172.21.15.42:0/4068423051 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fa9e403d3d0 0x7fa9e403f8e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0033000 tx=0x7fa9f0004730 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:58.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:58.852+0000 7faa01600640 1 -- 172.21.15.42:0/4068423051 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7faa040b3650 con 0x7faa0406c8b0 2024-07-04T17:31:59.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.064+0000 7faa0c200640 1 -- 172.21.15.42:0/4068423051 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7faa040aef60 con 0x7faa0406c8b0 2024-07-04T17:31:59.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.066+0000 7faa01600640 1 -- 172.21.15.42:0/4068423051 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v92) ==== 84+0+209 (secure 0 0 0) 0x7fa9f8016170 con 0x7faa0406c8b0 2024-07-04T17:31:59.066 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:31:59.066 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-07-04T17:31:59.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:31:59.067 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 1, 2024-07-04T17:31:59.067 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": ".mgr" 2024-07-04T17:31:59.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:31:59.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:31:59.067 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 16, 2024-07-04T17:31:59.067 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": "cephfs_metadata" 2024-07-04T17:31:59.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:31:59.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:31:59.067 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 17, 2024-07-04T17:31:59.067 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": "cephfs_data" 2024-07-04T17:31:59.067 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:31:59.067 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-07-04T17:31:59.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.069+0000 7faa0c200640 1 -- 172.21.15.42:0/4068423051 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fa9e403d3d0 msgr2=0x7fa9e403f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:59.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.069+0000 7faa0c200640 1 --2- 172.21.15.42:0/4068423051 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fa9e403d3d0 0x7fa9e403f8e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0033000 tx=0x7fa9f0004730 comp rx=0 tx=0).stop 2024-07-04T17:31:59.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.069+0000 7faa0c200640 1 -- 172.21.15.42:0/4068423051 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa0406c8b0 msgr2=0x7faa040ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:59.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.069+0000 7faa0c200640 1 --2- 172.21.15.42:0/4068423051 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa0406c8b0 0x7faa040ad130 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800c380 tx=0x7fa9f800c840 comp rx=0 tx=0).stop 2024-07-04T17:31:59.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.069+0000 7faa0c200640 1 -- 172.21.15.42:0/4068423051 shutdown_connections 2024-07-04T17:31:59.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.069+0000 7faa0c200640 1 --2- 172.21.15.42:0/4068423051 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fa9e403d3d0 0x7fa9e403f8e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:59.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.069+0000 7faa0c200640 1 --2- 172.21.15.42:0/4068423051 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa0406c8b0 0x7faa040ad130 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:59.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.069+0000 7faa0c200640 1 -- 172.21.15.42:0/4068423051 >> 172.21.15.42:0/4068423051 conn(0x7faa040bf700 msgr2=0x7faa040be350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:59.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.070+0000 7faa0c200640 1 -- 172.21.15.42:0/4068423051 shutdown_connections 2024-07-04T17:31:59.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.070+0000 7faa0c200640 1 -- 172.21.15.42:0/4068423051 wait complete. 2024-07-04T17:31:59.097 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'backup_fs' 2024-07-04T17:31:59.097 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_metadata --pg_num_min 64 2024-07-04T17:31:59.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.391+0000 7f4492a00640 1 -- 172.21.15.42:0/209491858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f448c0712d0 msgr2=0x7f448c06d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:31:59.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.391+0000 7f4492a00640 1 --2- 172.21.15.42:0/209491858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f448c0712d0 0x7f448c06d130 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f447c005fd0 tx=0x7f447c02a5f0 comp rx=0 tx=0).stop 2024-07-04T17:31:59.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.391+0000 7f4492a00640 1 -- 172.21.15.42:0/209491858 shutdown_connections 2024-07-04T17:31:59.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.391+0000 7f4492a00640 1 --2- 172.21.15.42:0/209491858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f448c0712d0 0x7f448c06d130 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:31:59.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.391+0000 7f4492a00640 1 -- 172.21.15.42:0/209491858 >> 172.21.15.42:0/209491858 conn(0x7f448c06c020 msgr2=0x7f448c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:31:59.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.391+0000 7f4492a00640 1 -- 172.21.15.42:0/209491858 shutdown_connections 2024-07-04T17:31:59.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.391+0000 7f4492a00640 1 -- 172.21.15.42:0/209491858 wait complete. 2024-07-04T17:31:59.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.394+0000 7f4492a00640 1 Processor -- start 2024-07-04T17:31:59.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.394+0000 7f4492a00640 1 -- start start 2024-07-04T17:31:59.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.396+0000 7f4492a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f448c0712d0 0x7f448c1d7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:59.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.396+0000 7f4492a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f448c141900 con 0x7f448c0712d0 2024-07-04T17:31:59.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.396+0000 7f448be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f448c0712d0 0x7f448c1d7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:59.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.396+0000 7f448be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f448c0712d0 0x7f448c1d7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34952/0 (socket says 172.21.15.42:34952) 2024-07-04T17:31:59.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.396+0000 7f448be00640 1 -- 172.21.15.42:0/82585695 learned_addr learned my addr 172.21.15.42:0/82585695 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:31:59.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.396+0000 7f448be00640 1 -- 172.21.15.42:0/82585695 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f448c0742e0 con 0x7f448c0712d0 2024-07-04T17:31:59.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.397+0000 7f448be00640 1 --2- 172.21.15.42:0/82585695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f448c0712d0 0x7f448c1d7610 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f447c034040 tx=0x7f447c0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:59.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.397+0000 7f4483e00640 1 -- 172.21.15.42:0/82585695 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f447c03e030 con 0x7f448c0712d0 2024-07-04T17:31:59.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.397+0000 7f4483e00640 1 -- 172.21.15.42:0/82585695 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f447c0035f0 con 0x7f448c0712d0 2024-07-04T17:31:59.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.397+0000 7f4483e00640 1 -- 172.21.15.42:0/82585695 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f447c035040 con 0x7f448c0712d0 2024-07-04T17:31:59.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.397+0000 7f4492a00640 1 -- 172.21.15.42:0/82585695 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f448c1d7cb0 con 0x7f448c0712d0 2024-07-04T17:31:59.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.397+0000 7f4492a00640 1 -- 172.21.15.42:0/82585695 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f448c13c590 con 0x7f448c0712d0 2024-07-04T17:31:59.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.399+0000 7f4492a00640 1 -- 172.21.15.42:0/82585695 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f448c06d130 con 0x7f448c0712d0 2024-07-04T17:31:59.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.399+0000 7f4483e00640 1 -- 172.21.15.42:0/82585695 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7f447c041030 con 0x7f448c0712d0 2024-07-04T17:31:59.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.399+0000 7f4483e00640 1 --2- 172.21.15.42:0/82585695 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f446c03d3d0 0x7f446c03f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:31:59.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.399+0000 7f4483e00640 1 -- 172.21.15.42:0/82585695 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 3439+0+0 (secure 0 0 0) 0x7f447c03f070 con 0x7f448c0712d0 2024-07-04T17:31:59.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.400+0000 7f448b400640 1 --2- 172.21.15.42:0/82585695 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f446c03d3d0 0x7f446c03f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:31:59.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.401+0000 7f448b400640 1 --2- 172.21.15.42:0/82585695 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f446c03d3d0 0x7f446c03f8e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4478001500 tx=0x7f4478007a30 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:31:59.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.405+0000 7f4483e00640 1 -- 172.21.15.42:0/82585695 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f447c04bd60 con 0x7f448c0712d0 2024-07-04T17:31:59.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:31:59.614+0000 7f4492a00640 1 -- 172.21.15.42:0/82585695 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64} v 0) -- 0x7f448c13c7c0 con 0x7f448c0712d0 2024-07-04T17:32:00.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.526+0000 7f4483e00640 1 -- 172.21.15.42:0/82585695 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64}]=0 pool 'backup_fs_metadata' created v93) ==== 144+0+0 (secure 0 0 0) 0x7f447c031400 con 0x7f448c0712d0 2024-07-04T17:32:00.527 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'backup_fs_metadata' created 2024-07-04T17:32:00.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.530+0000 7f4492a00640 1 -- 172.21.15.42:0/82585695 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f446c03d3d0 msgr2=0x7f446c03f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:00.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.530+0000 7f4492a00640 1 --2- 172.21.15.42:0/82585695 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f446c03d3d0 0x7f446c03f8e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4478001500 tx=0x7f4478007a30 comp rx=0 tx=0).stop 2024-07-04T17:32:00.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.530+0000 7f4492a00640 1 -- 172.21.15.42:0/82585695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f448c0712d0 msgr2=0x7f448c1d7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:00.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.530+0000 7f4492a00640 1 --2- 172.21.15.42:0/82585695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f448c0712d0 0x7f448c1d7610 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f447c034040 tx=0x7f447c0049d0 comp rx=0 tx=0).stop 2024-07-04T17:32:00.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.530+0000 7f4492a00640 1 -- 172.21.15.42:0/82585695 shutdown_connections 2024-07-04T17:32:00.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.530+0000 7f4492a00640 1 --2- 172.21.15.42:0/82585695 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f446c03d3d0 0x7f446c03f8e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:00.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.530+0000 7f4492a00640 1 --2- 172.21.15.42:0/82585695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f448c0712d0 0x7f448c1d7610 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:00.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.530+0000 7f4492a00640 1 -- 172.21.15.42:0/82585695 >> 172.21.15.42:0/82585695 conn(0x7f448c06c020 msgr2=0x7f448c0b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:00.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.530+0000 7f4492a00640 1 -- 172.21.15.42:0/82585695 shutdown_connections 2024-07-04T17:32:00.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.531+0000 7f4492a00640 1 -- 172.21.15.42:0/82585695 wait complete. 2024-07-04T17:32:00.568 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-04T17:32:00.817 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-04T17:32:00.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.843+0000 7f12b0400640 1 -- 172.21.15.42:0/2397386147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12a80a99b0 msgr2=0x7f12a80ab1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:00.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.843+0000 7f12b0400640 1 --2- 172.21.15.42:0/2397386147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12a80a99b0 0x7f12a80ab1c0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f129c0097b0 tx=0x7f129c02a530 comp rx=0 tx=0).stop 2024-07-04T17:32:00.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.843+0000 7f12b0400640 1 -- 172.21.15.42:0/2397386147 shutdown_connections 2024-07-04T17:32:00.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.843+0000 7f12b0400640 1 --2- 172.21.15.42:0/2397386147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12a80a99b0 0x7f12a80ab1c0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:00.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.843+0000 7f12b0400640 1 -- 172.21.15.42:0/2397386147 >> 172.21.15.42:0/2397386147 conn(0x7f12a806c020 msgr2=0x7f12a806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:00.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.843+0000 7f12b0400640 1 -- 172.21.15.42:0/2397386147 shutdown_connections 2024-07-04T17:32:00.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.843+0000 7f12b0400640 1 -- 172.21.15.42:0/2397386147 wait complete. 2024-07-04T17:32:00.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.845+0000 7f12b0400640 1 Processor -- start 2024-07-04T17:32:00.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.846+0000 7f12b0400640 1 -- start start 2024-07-04T17:32:00.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.847+0000 7f12b0400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12a80a99b0 0x7f12a80b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:00.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.847+0000 7f12b0400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f12a813f450 con 0x7f12a80a99b0 2024-07-04T17:32:00.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.847+0000 7f12ada00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12a80a99b0 0x7f12a80b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:00.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.847+0000 7f12ada00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12a80a99b0 0x7f12a80b75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34964/0 (socket says 172.21.15.42:34964) 2024-07-04T17:32:00.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.847+0000 7f12ada00640 1 -- 172.21.15.42:0/3689775800 learned_addr learned my addr 172.21.15.42:0/3689775800 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:00.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.848+0000 7f12ada00640 1 -- 172.21.15.42:0/3689775800 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a80b7cc0 con 0x7f12a80a99b0 2024-07-04T17:32:00.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.848+0000 7f12ada00640 1 --2- 172.21.15.42:0/3689775800 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12a80a99b0 0x7f12a80b75c0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f129c0098d0 tx=0x7f129c033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:00.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.848+0000 7f12a5600640 1 -- 172.21.15.42:0/3689775800 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f129c031880 con 0x7f12a80a99b0 2024-07-04T17:32:00.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.849+0000 7f12b0400640 1 -- 172.21.15.42:0/3689775800 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12a81e4440 con 0x7f12a80a99b0 2024-07-04T17:32:00.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.849+0000 7f12a5600640 1 -- 172.21.15.42:0/3689775800 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f129c031b10 con 0x7f12a80a99b0 2024-07-04T17:32:00.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.849+0000 7f12a5600640 1 -- 172.21.15.42:0/3689775800 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f129c031df0 con 0x7f12a80a99b0 2024-07-04T17:32:00.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.849+0000 7f12b0400640 1 -- 172.21.15.42:0/3689775800 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12a813e920 con 0x7f12a80a99b0 2024-07-04T17:32:00.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.850+0000 7f12b0400640 1 -- 172.21.15.42:0/3689775800 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12a813fb60 con 0x7f12a80a99b0 2024-07-04T17:32:00.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.850+0000 7f12a5600640 1 -- 172.21.15.42:0/3689775800 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7f129c033070 con 0x7f12a80a99b0 2024-07-04T17:32:00.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.851+0000 7f12a5600640 1 --2- 172.21.15.42:0/3689775800 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f128803d380 0x7f128803f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:00.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.851+0000 7f12a5600640 1 -- 172.21.15.42:0/3689775800 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 3812+0+0 (secure 0 0 0) 0x7f129c072cf0 con 0x7f12a80a99b0 2024-07-04T17:32:00.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.851+0000 7f12ad000640 1 --2- 172.21.15.42:0/3689775800 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f128803d380 0x7f128803f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:00.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.852+0000 7f12ad000640 1 --2- 172.21.15.42:0/3689775800 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f128803d380 0x7f128803f890 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1294007fd0 tx=0x7f1294007920 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:00.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:00.857+0000 7f12a5600640 1 -- 172.21.15.42:0/3689775800 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f129c03f030 con 0x7f12a80a99b0 2024-07-04T17:32:01.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.068+0000 7f12b0400640 1 -- 172.21.15.42:0/3689775800 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f12a8069b70 con 0x7f12a80a99b0 2024-07-04T17:32:01.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.541+0000 7f12a5600640 1 -- 172.21.15.42:0/3689775800 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'backup_fs_data' created v94) ==== 176+0+0 (secure 0 0 0) 0x7f129c03a3a0 con 0x7f12a80a99b0 2024-07-04T17:32:01.542 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'backup_fs_data' created 2024-07-04T17:32:01.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.545+0000 7f12b0400640 1 -- 172.21.15.42:0/3689775800 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f128803d380 msgr2=0x7f128803f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:01.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.545+0000 7f12b0400640 1 --2- 172.21.15.42:0/3689775800 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f128803d380 0x7f128803f890 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1294007fd0 tx=0x7f1294007920 comp rx=0 tx=0).stop 2024-07-04T17:32:01.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.545+0000 7f12b0400640 1 -- 172.21.15.42:0/3689775800 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12a80a99b0 msgr2=0x7f12a80b75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:01.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.545+0000 7f12b0400640 1 --2- 172.21.15.42:0/3689775800 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12a80a99b0 0x7f12a80b75c0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f129c0098d0 tx=0x7f129c033ad0 comp rx=0 tx=0).stop 2024-07-04T17:32:01.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.546+0000 7f12b0400640 1 -- 172.21.15.42:0/3689775800 shutdown_connections 2024-07-04T17:32:01.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.546+0000 7f12b0400640 1 --2- 172.21.15.42:0/3689775800 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f128803d380 0x7f128803f890 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:01.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.546+0000 7f12b0400640 1 --2- 172.21.15.42:0/3689775800 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12a80a99b0 0x7f12a80b75c0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:01.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.546+0000 7f12b0400640 1 -- 172.21.15.42:0/3689775800 >> 172.21.15.42:0/3689775800 conn(0x7f12a806c020 msgr2=0x7f12a8072020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:01.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.546+0000 7f12b0400640 1 -- 172.21.15.42:0/3689775800 shutdown_connections 2024-07-04T17:32:01.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.546+0000 7f12b0400640 1 -- 172.21.15.42:0/3689775800 wait complete. 2024-07-04T17:32:01.597 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new backup_fs backup_fs_metadata backup_fs_data 2024-07-04T17:32:01.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.850+0000 7ff0c0a00640 1 -- 172.21.15.42:0/222302017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0bc0aff70 msgr2=0x7ff0bc0b9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:01.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.850+0000 7ff0c0a00640 1 --2- 172.21.15.42:0/222302017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0bc0aff70 0x7ff0bc0b9b80 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac0097b0 tx=0x7ff0ac02a530 comp rx=0 tx=0).stop 2024-07-04T17:32:01.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.850+0000 7ff0c0a00640 1 -- 172.21.15.42:0/222302017 shutdown_connections 2024-07-04T17:32:01.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.850+0000 7ff0c0a00640 1 --2- 172.21.15.42:0/222302017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0bc0aff70 0x7ff0bc0b9b80 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:01.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.850+0000 7ff0c0a00640 1 -- 172.21.15.42:0/222302017 >> 172.21.15.42:0/222302017 conn(0x7ff0bc0bf260 msgr2=0x7ff0bc0af5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:01.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.850+0000 7ff0c0a00640 1 -- 172.21.15.42:0/222302017 shutdown_connections 2024-07-04T17:32:01.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.850+0000 7ff0c0a00640 1 -- 172.21.15.42:0/222302017 wait complete. 2024-07-04T17:32:01.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.852+0000 7ff0c0a00640 1 Processor -- start 2024-07-04T17:32:01.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.852+0000 7ff0c0a00640 1 -- start start 2024-07-04T17:32:01.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.853+0000 7ff0c0a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0bc0aff70 0x7ff0bc0bdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:01.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.853+0000 7ff0c0a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff0bc1484b0 con 0x7ff0bc0aff70 2024-07-04T17:32:01.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.853+0000 7ff0ba800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0bc0aff70 0x7ff0bc0bdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:01.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.853+0000 7ff0ba800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0bc0aff70 0x7ff0bc0bdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34976/0 (socket says 172.21.15.42:34976) 2024-07-04T17:32:01.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.853+0000 7ff0ba800640 1 -- 172.21.15.42:0/1291531552 learned_addr learned my addr 172.21.15.42:0/1291531552 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:01.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.854+0000 7ff0ba800640 1 -- 172.21.15.42:0/1291531552 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0bc14b4d0 con 0x7ff0bc0aff70 2024-07-04T17:32:01.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.854+0000 7ff0ba800640 1 --2- 172.21.15.42:0/1291531552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0bc0aff70 0x7ff0bc0bdd60 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac0098d0 tx=0x7ff0ac033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:01.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.854+0000 7ff0b2a00640 1 -- 172.21.15.42:0/1291531552 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0ac031880 con 0x7ff0bc0aff70 2024-07-04T17:32:01.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.854+0000 7ff0b2a00640 1 -- 172.21.15.42:0/1291531552 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0ac031b10 con 0x7ff0bc0aff70 2024-07-04T17:32:01.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.854+0000 7ff0b2a00640 1 -- 172.21.15.42:0/1291531552 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0ac031df0 con 0x7ff0bc0aff70 2024-07-04T17:32:01.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.854+0000 7ff0c0a00640 1 -- 172.21.15.42:0/1291531552 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0bc148cd0 con 0x7ff0bc0aff70 2024-07-04T17:32:01.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.855+0000 7ff0c0a00640 1 -- 172.21.15.42:0/1291531552 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0bc0b30f0 con 0x7ff0bc0aff70 2024-07-04T17:32:01.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.856+0000 7ff0c0a00640 1 -- 172.21.15.42:0/1291531552 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0bc14b6a0 con 0x7ff0bc0aff70 2024-07-04T17:32:01.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.856+0000 7ff0b2a00640 1 -- 172.21.15.42:0/1291531552 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7ff0ac0330d0 con 0x7ff0bc0aff70 2024-07-04T17:32:01.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.857+0000 7ff0b2a00640 1 --2- 172.21.15.42:0/1291531552 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7ff09c03d3d0 0x7ff09c03f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:01.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.857+0000 7ff0b2a00640 1 -- 172.21.15.42:0/1291531552 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 4197+0+0 (secure 0 0 0) 0x7ff0ac072dd0 con 0x7ff0bc0aff70 2024-07-04T17:32:01.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.857+0000 7ff0b9e00640 1 --2- 172.21.15.42:0/1291531552 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7ff09c03d3d0 0x7ff09c03f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:01.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.858+0000 7ff0b9e00640 1 --2- 172.21.15.42:0/1291531552 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7ff09c03d3d0 0x7ff09c03f8e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff0a80015e0 tx=0x7ff0a8001290 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:01.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:01.862+0000 7ff0b2a00640 1 -- 172.21.15.42:0/1291531552 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7ff0ac03f030 con 0x7ff0bc0aff70 2024-07-04T17:32:02.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.102+0000 7ff0c0a00640 1 -- 172.21.15.42:0/1291531552 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"} v 0) -- 0x7ff0bc148ac0 con 0x7ff0bc0aff70 2024-07-04T17:32:02.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.543+0000 7ff0b2a00640 1 -- 172.21.15.42:0/1291531552 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"}]=0 new fs with metadata pool 18 and data pool 19 v73) ==== 183+0+0 (secure 0 0 0) 0x7ff0ac03a3a0 con 0x7ff0bc0aff70 2024-07-04T17:32:02.544 INFO:teuthology.orchestra.run.smithi042.stderr:new fs with metadata pool 18 and data pool 19 2024-07-04T17:32:02.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.548+0000 7ff0c0a00640 1 -- 172.21.15.42:0/1291531552 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7ff09c03d3d0 msgr2=0x7ff09c03f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:02.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.548+0000 7ff0c0a00640 1 --2- 172.21.15.42:0/1291531552 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7ff09c03d3d0 0x7ff09c03f8e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff0a80015e0 tx=0x7ff0a8001290 comp rx=0 tx=0).stop 2024-07-04T17:32:02.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.548+0000 7ff0c0a00640 1 -- 172.21.15.42:0/1291531552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0bc0aff70 msgr2=0x7ff0bc0bdd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:02.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.548+0000 7ff0c0a00640 1 --2- 172.21.15.42:0/1291531552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0bc0aff70 0x7ff0bc0bdd60 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac0098d0 tx=0x7ff0ac033ad0 comp rx=0 tx=0).stop 2024-07-04T17:32:02.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.549+0000 7ff0c0a00640 1 -- 172.21.15.42:0/1291531552 shutdown_connections 2024-07-04T17:32:02.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.549+0000 7ff0c0a00640 1 --2- 172.21.15.42:0/1291531552 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7ff09c03d3d0 0x7ff09c03f8e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:02.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.549+0000 7ff0c0a00640 1 --2- 172.21.15.42:0/1291531552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0bc0aff70 0x7ff0bc0bdd60 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:02.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.549+0000 7ff0c0a00640 1 -- 172.21.15.42:0/1291531552 >> 172.21.15.42:0/1291531552 conn(0x7ff0bc0bf260 msgr2=0x7ff0bc144f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:02.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.549+0000 7ff0c0a00640 1 -- 172.21.15.42:0/1291531552 shutdown_connections 2024-07-04T17:32:02.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.549+0000 7ff0c0a00640 1 -- 172.21.15.42:0/1291531552 wait complete. 2024-07-04T17:32:02.607 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:32:02.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.861+0000 7f00f8c00640 1 -- 172.21.15.42:0/1860346821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00f40b3030 msgr2=0x7f00f40b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:02.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.861+0000 7f00f8c00640 1 --2- 172.21.15.42:0/1860346821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00f40b3030 0x7f00f40b01f0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00b2b0 tx=0x7f00dc02ad10 comp rx=0 tx=0).stop 2024-07-04T17:32:02.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.861+0000 7f00f8c00640 1 -- 172.21.15.42:0/1860346821 shutdown_connections 2024-07-04T17:32:02.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.861+0000 7f00f8c00640 1 --2- 172.21.15.42:0/1860346821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00f40b3030 0x7f00f40b01f0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:02.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.861+0000 7f00f8c00640 1 -- 172.21.15.42:0/1860346821 >> 172.21.15.42:0/1860346821 conn(0x7f00f40bf700 msgr2=0x7f00f40af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:02.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.861+0000 7f00f8c00640 1 -- 172.21.15.42:0/1860346821 shutdown_connections 2024-07-04T17:32:02.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.861+0000 7f00f8c00640 1 -- 172.21.15.42:0/1860346821 wait complete. 2024-07-04T17:32:02.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.863+0000 7f00f8c00640 1 Processor -- start 2024-07-04T17:32:02.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.863+0000 7f00f8c00640 1 -- start start 2024-07-04T17:32:02.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.864+0000 7f00f8c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00f406cea0 0x7f00f40ac5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:02.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.864+0000 7f00f8c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f00f406ef50 con 0x7f00f406cea0 2024-07-04T17:32:02.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.865+0000 7f00f1e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00f406cea0 0x7f00f40ac5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:02.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.865+0000 7f00f1e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00f406cea0 0x7f00f40ac5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35000/0 (socket says 172.21.15.42:35000) 2024-07-04T17:32:02.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.865+0000 7f00f1e00640 1 -- 172.21.15.42:0/1690012838 learned_addr learned my addr 172.21.15.42:0/1690012838 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:02.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.865+0000 7f00f1e00640 1 -- 172.21.15.42:0/1690012838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f41d5a10 con 0x7f00f406cea0 2024-07-04T17:32:02.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.865+0000 7f00f1e00640 1 --2- 172.21.15.42:0/1690012838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00f406cea0 0x7f00f40ac5f0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f00e400c380 tx=0x7f00e400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:02.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.866+0000 7f00eaa00640 1 -- 172.21.15.42:0/1690012838 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00e40116c0 con 0x7f00f406cea0 2024-07-04T17:32:02.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.866+0000 7f00eaa00640 1 -- 172.21.15.42:0/1690012838 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00e4017070 con 0x7f00f406cea0 2024-07-04T17:32:02.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.866+0000 7f00f8c00640 1 -- 172.21.15.42:0/1690012838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00f41d4860 con 0x7f00f406cea0 2024-07-04T17:32:02.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.866+0000 7f00eaa00640 1 -- 172.21.15.42:0/1690012838 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00e4011b20 con 0x7f00f406cea0 2024-07-04T17:32:02.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.866+0000 7f00f8c00640 1 -- 172.21.15.42:0/1690012838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00f41469d0 con 0x7f00f406cea0 2024-07-04T17:32:02.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.867+0000 7f00f8c00640 1 -- 172.21.15.42:0/1690012838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00f40b01f0 con 0x7f00f406cea0 2024-07-04T17:32:02.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.867+0000 7f00eaa00640 1 -- 172.21.15.42:0/1690012838 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7f00e4011cc0 con 0x7f00f406cea0 2024-07-04T17:32:02.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.868+0000 7f00eaa00640 1 --2- 172.21.15.42:0/1690012838 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f00d003d3d0 0x7f00d003f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:02.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.868+0000 7f00eaa00640 1 -- 172.21.15.42:0/1690012838 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4303+0+0 (secure 0 0 0) 0x7f00e4051750 con 0x7f00f406cea0 2024-07-04T17:32:02.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.868+0000 7f00e9400640 1 --2- 172.21.15.42:0/1690012838 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f00d003d3d0 0x7f00d003f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:02.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.869+0000 7f00e9400640 1 --2- 172.21.15.42:0/1690012838 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f00d003d3d0 0x7f00d003f8e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f00dc035000 tx=0x7f00dc004730 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:02.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:02.873+0000 7f00eaa00640 1 -- 172.21.15.42:0/1690012838 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f00e4010cf0 con 0x7f00f406cea0 2024-07-04T17:32:03.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.075+0000 7f00f8c00640 1 -- 172.21.15.42:0/1690012838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00f41d57e0 con 0x7f00f406cea0 2024-07-04T17:32:03.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.077+0000 7f00eaa00640 1 -- 172.21.15.42:0/1690012838 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+14467 (secure 0 0 0) 0x7f00e4019310 con 0x7f00f406cea0 2024-07-04T17:32:03.078 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:32:03.078 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":95,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:32:02.104007+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:31:39.727656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-07-04T17:31:40.615610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:31:59.635881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.4099999666213989,"score_stable":1.4099999666213989,"optimal_score":1,"raw_score_acting":1.4099999666213989,"raw_score_stable":1.4099999666213989,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:32:01.086110+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.0800000429153442,"score_stable":1.0800000429153442,"optimal_score":1,"raw_score_acting":1.0800000429153442,"raw_score_stable":1.0800000429153442,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:32:03.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.080+0000 7f00f8c00640 1 -- 172.21.15.42:0/1690012838 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f00d003d3d0 msgr2=0x7f00d003f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:03.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.080+0000 7f00f8c00640 1 --2- 172.21.15.42:0/1690012838 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f00d003d3d0 0x7f00d003f8e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f00dc035000 tx=0x7f00dc004730 comp rx=0 tx=0).stop 2024-07-04T17:32:03.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.080+0000 7f00f8c00640 1 -- 172.21.15.42:0/1690012838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00f406cea0 msgr2=0x7f00f40ac5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:03.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.080+0000 7f00f8c00640 1 --2- 172.21.15.42:0/1690012838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00f406cea0 0x7f00f40ac5f0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f00e400c380 tx=0x7f00e400c840 comp rx=0 tx=0).stop 2024-07-04T17:32:03.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.080+0000 7f00f8c00640 1 -- 172.21.15.42:0/1690012838 shutdown_connections 2024-07-04T17:32:03.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.080+0000 7f00f8c00640 1 --2- 172.21.15.42:0/1690012838 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f00d003d3d0 0x7f00d003f8e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:03.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.080+0000 7f00f8c00640 1 --2- 172.21.15.42:0/1690012838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00f406cea0 0x7f00f40ac5f0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:03.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.080+0000 7f00f8c00640 1 -- 172.21.15.42:0/1690012838 >> 172.21.15.42:0/1690012838 conn(0x7f00f40bf700 msgr2=0x7f00f40aa3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:03.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.081+0000 7f00f8c00640 1 -- 172.21.15.42:0/1690012838 shutdown_connections 2024-07-04T17:32:03.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.081+0000 7f00f8c00640 1 -- 172.21.15.42:0/1690012838 wait complete. 2024-07-04T17:32:03.105 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:32:03.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.387+0000 7f7e46000640 1 -- 172.21.15.42:0/583899549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e400ba830 msgr2=0x7f7e400b7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:03.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.387+0000 7f7e46000640 1 --2- 172.21.15.42:0/583899549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e400ba830 0x7f7e400b7cc0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f7e300097b0 tx=0x7f7e3002a530 comp rx=0 tx=0).stop 2024-07-04T17:32:03.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.387+0000 7f7e46000640 1 -- 172.21.15.42:0/583899549 shutdown_connections 2024-07-04T17:32:03.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.387+0000 7f7e46000640 1 --2- 172.21.15.42:0/583899549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e400ba830 0x7f7e400b7cc0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:03.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.387+0000 7f7e46000640 1 -- 172.21.15.42:0/583899549 >> 172.21.15.42:0/583899549 conn(0x7f7e400bebc0 msgr2=0x7f7e400befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:03.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.387+0000 7f7e46000640 1 -- 172.21.15.42:0/583899549 shutdown_connections 2024-07-04T17:32:03.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.387+0000 7f7e46000640 1 -- 172.21.15.42:0/583899549 wait complete. 2024-07-04T17:32:03.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.389+0000 7f7e46000640 1 Processor -- start 2024-07-04T17:32:03.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.389+0000 7f7e46000640 1 -- start start 2024-07-04T17:32:03.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.390+0000 7f7e46000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e400ba830 0x7f7e40141ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:03.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.390+0000 7f7e46000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7e400b1e30 con 0x7f7e400ba830 2024-07-04T17:32:03.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.390+0000 7f7e44c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e400ba830 0x7f7e40141ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:03.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.390+0000 7f7e44c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e400ba830 0x7f7e40141ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35022/0 (socket says 172.21.15.42:35022) 2024-07-04T17:32:03.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.391+0000 7f7e44c00640 1 -- 172.21.15.42:0/2878635155 learned_addr learned my addr 172.21.15.42:0/2878635155 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:03.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.391+0000 7f7e44c00640 1 -- 172.21.15.42:0/2878635155 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e401e3f10 con 0x7f7e400ba830 2024-07-04T17:32:03.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.391+0000 7f7e44c00640 1 --2- 172.21.15.42:0/2878635155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e400ba830 0x7f7e40141ca0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f7e300098d0 tx=0x7f7e30033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:03.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.392+0000 7f7e3cc00640 1 -- 172.21.15.42:0/2878635155 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e30031bc0 con 0x7f7e400ba830 2024-07-04T17:32:03.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.392+0000 7f7e3cc00640 1 -- 172.21.15.42:0/2878635155 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e30031e50 con 0x7f7e400ba830 2024-07-04T17:32:03.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.392+0000 7f7e46000640 1 -- 172.21.15.42:0/2878635155 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e40143740 con 0x7f7e400ba830 2024-07-04T17:32:03.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.392+0000 7f7e3cc00640 1 -- 172.21.15.42:0/2878635155 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e30033260 con 0x7f7e400ba830 2024-07-04T17:32:03.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.392+0000 7f7e46000640 1 -- 172.21.15.42:0/2878635155 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e40141760 con 0x7f7e400ba830 2024-07-04T17:32:03.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.393+0000 7f7e46000640 1 -- 172.21.15.42:0/2878635155 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e40141a50 con 0x7f7e400ba830 2024-07-04T17:32:03.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.393+0000 7f7e3cc00640 1 -- 172.21.15.42:0/2878635155 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7f7e30033400 con 0x7f7e400ba830 2024-07-04T17:32:03.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.394+0000 7f7e3cc00640 1 --2- 172.21.15.42:0/2878635155 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f7e1c03d3d0 0x7f7e1c03f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:03.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.394+0000 7f7e3cc00640 1 -- 172.21.15.42:0/2878635155 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4303+0+0 (secure 0 0 0) 0x7f7e300730d0 con 0x7f7e400ba830 2024-07-04T17:32:03.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.394+0000 7f7e3fe00640 1 --2- 172.21.15.42:0/2878635155 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f7e1c03d3d0 0x7f7e1c03f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:03.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.395+0000 7f7e3fe00640 1 --2- 172.21.15.42:0/2878635155 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f7e1c03d3d0 0x7f7e1c03f8e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7e280015c0 tx=0x7f7e28007a90 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:03.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.399+0000 7f7e3cc00640 1 -- 172.21.15.42:0/2878635155 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f7e40141a50 con 0x7f7e400ba830 2024-07-04T17:32:03.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.616+0000 7f7e46000640 1 -- 172.21.15.42:0/2878635155 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7e400bf4f0 con 0x7f7e400ba830 2024-07-04T17:32:03.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.618+0000 7f7e3cc00640 1 -- 172.21.15.42:0/2878635155 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+14454 (secure 0 0 0) 0x7f7e3003f030 con 0x7f7e400ba830 2024-07-04T17:32:03.619 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:32:03.619 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":96,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:32:03.557225+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:31:39.727656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-07-04T17:31:40.615610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:31:59.635881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.4099999666213989,"score_stable":1.4099999666213989,"optimal_score":1,"raw_score_acting":1.4099999666213989,"raw_score_stable":1.4099999666213989,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:32:01.086110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.0800000429153442,"score_stable":1.0800000429153442,"optimal_score":1,"raw_score_acting":1.0800000429153442,"raw_score_stable":1.0800000429153442,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:32:03.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.621+0000 7f7e46000640 1 -- 172.21.15.42:0/2878635155 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f7e1c03d3d0 msgr2=0x7f7e1c03f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:03.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.621+0000 7f7e46000640 1 --2- 172.21.15.42:0/2878635155 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f7e1c03d3d0 0x7f7e1c03f8e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7e280015c0 tx=0x7f7e28007a90 comp rx=0 tx=0).stop 2024-07-04T17:32:03.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.621+0000 7f7e46000640 1 -- 172.21.15.42:0/2878635155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e400ba830 msgr2=0x7f7e40141ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:03.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.621+0000 7f7e46000640 1 --2- 172.21.15.42:0/2878635155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e400ba830 0x7f7e40141ca0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f7e300098d0 tx=0x7f7e30033e40 comp rx=0 tx=0).stop 2024-07-04T17:32:03.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.622+0000 7f7e46000640 1 -- 172.21.15.42:0/2878635155 shutdown_connections 2024-07-04T17:32:03.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.622+0000 7f7e46000640 1 --2- 172.21.15.42:0/2878635155 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f7e1c03d3d0 0x7f7e1c03f8e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:03.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.622+0000 7f7e46000640 1 --2- 172.21.15.42:0/2878635155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e400ba830 0x7f7e40141ca0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:03.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.622+0000 7f7e46000640 1 -- 172.21.15.42:0/2878635155 >> 172.21.15.42:0/2878635155 conn(0x7f7e400bebc0 msgr2=0x7f7e401445a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:03.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.622+0000 7f7e46000640 1 -- 172.21.15.42:0/2878635155 shutdown_connections 2024-07-04T17:32:03.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.622+0000 7f7e46000640 1 -- 172.21.15.42:0/2878635155 wait complete. 2024-07-04T17:32:03.648 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs standby_count_wanted 0 2024-07-04T17:32:03.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.928+0000 7f9e72c00640 1 -- 172.21.15.42:0/1800318166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e6c071ca0 msgr2=0x7f9e6c06d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:03.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.928+0000 7f9e72c00640 1 --2- 172.21.15.42:0/1800318166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e6c071ca0 0x7f9e6c06d050 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f9e540097b0 tx=0x7f9e5402a540 comp rx=0 tx=0).stop 2024-07-04T17:32:03.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.928+0000 7f9e72c00640 1 -- 172.21.15.42:0/1800318166 shutdown_connections 2024-07-04T17:32:03.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.928+0000 7f9e72c00640 1 --2- 172.21.15.42:0/1800318166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e6c071ca0 0x7f9e6c06d050 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:03.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.928+0000 7f9e72c00640 1 -- 172.21.15.42:0/1800318166 >> 172.21.15.42:0/1800318166 conn(0x7f9e6c06c020 msgr2=0x7f9e6c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:03.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.928+0000 7f9e72c00640 1 -- 172.21.15.42:0/1800318166 shutdown_connections 2024-07-04T17:32:03.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.928+0000 7f9e72c00640 1 -- 172.21.15.42:0/1800318166 wait complete. 2024-07-04T17:32:03.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.931+0000 7f9e72c00640 1 Processor -- start 2024-07-04T17:32:03.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.931+0000 7f9e72c00640 1 -- start start 2024-07-04T17:32:03.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.933+0000 7f9e72c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e6c0712d0 0x7f9e6c0afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:03.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.933+0000 7f9e72c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c141760 con 0x7f9e6c0712d0 2024-07-04T17:32:03.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.933+0000 7f9e6be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e6c0712d0 0x7f9e6c0afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:03.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.933+0000 7f9e6be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e6c0712d0 0x7f9e6c0afe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35326/0 (socket says 172.21.15.42:35326) 2024-07-04T17:32:03.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.933+0000 7f9e6be00640 1 -- 172.21.15.42:0/3403254903 learned_addr learned my addr 172.21.15.42:0/3403254903 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:03.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.933+0000 7f9e6be00640 1 -- 172.21.15.42:0/3403254903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e6c0b0560 con 0x7f9e6c0712d0 2024-07-04T17:32:03.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.934+0000 7f9e6be00640 1 --2- 172.21.15.42:0/3403254903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e6c0712d0 0x7f9e6c0afe10 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c001eb0 tx=0x7f9e5c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:03.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.934+0000 7f9e68c00640 1 -- 172.21.15.42:0/3403254903 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e5c0116c0 con 0x7f9e6c0712d0 2024-07-04T17:32:03.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.934+0000 7f9e68c00640 1 -- 172.21.15.42:0/3403254903 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e5c017070 con 0x7f9e6c0712d0 2024-07-04T17:32:03.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.934+0000 7f9e68c00640 1 -- 172.21.15.42:0/3403254903 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e5c011ae0 con 0x7f9e6c0712d0 2024-07-04T17:32:03.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.934+0000 7f9e72c00640 1 -- 172.21.15.42:0/3403254903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e6c13a160 con 0x7f9e6c0712d0 2024-07-04T17:32:03.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.934+0000 7f9e72c00640 1 -- 172.21.15.42:0/3403254903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e6c1d76b0 con 0x7f9e6c0712d0 2024-07-04T17:32:03.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.935+0000 7f9e72c00640 1 -- 172.21.15.42:0/3403254903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e6c141e50 con 0x7f9e6c0712d0 2024-07-04T17:32:03.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.936+0000 7f9e68c00640 1 -- 172.21.15.42:0/3403254903 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7f9e5c011c80 con 0x7f9e6c0712d0 2024-07-04T17:32:03.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.936+0000 7f9e68c00640 1 --2- 172.21.15.42:0/3403254903 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f9e4803d3d0 0x7f9e4803f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:03.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.936+0000 7f9e68c00640 1 -- 172.21.15.42:0/3403254903 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4303+0+0 (secure 0 0 0) 0x7f9e5c051800 con 0x7f9e6c0712d0 2024-07-04T17:32:03.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.936+0000 7f9e63e00640 1 --2- 172.21.15.42:0/3403254903 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f9e4803d3d0 0x7f9e4803f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:03.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.937+0000 7f9e63e00640 1 --2- 172.21.15.42:0/3403254903 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f9e4803d3d0 0x7f9e4803f8e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9e54034000 tx=0x7f9e54004730 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:03.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:03.941+0000 7f9e68c00640 1 -- 172.21.15.42:0/3403254903 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f9e6c1d76b0 con 0x7f9e6c0712d0 2024-07-04T17:32:04.039 INFO:tasks.ceph.mds.d.smithi042.stderr:2024-07-04T17:32:04.039+0000 7f50a6600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-04T17:32:04.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.194+0000 7f9e72c00640 1 -- 172.21.15.42:0/3403254903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f9e6c075960 con 0x7f9e6c0712d0 2024-07-04T17:32:04.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.577+0000 7f9e68c00640 1 -- 172.21.15.42:0/3403254903 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"}]=0 v76) ==== 121+0+0 (secure 0 0 0) 0x7f9e5c010cd0 con 0x7f9e6c0712d0 2024-07-04T17:32:04.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.580+0000 7f9e72c00640 1 -- 172.21.15.42:0/3403254903 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f9e4803d3d0 msgr2=0x7f9e4803f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:04.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.580+0000 7f9e72c00640 1 --2- 172.21.15.42:0/3403254903 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f9e4803d3d0 0x7f9e4803f8e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9e54034000 tx=0x7f9e54004730 comp rx=0 tx=0).stop 2024-07-04T17:32:04.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.580+0000 7f9e72c00640 1 -- 172.21.15.42:0/3403254903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e6c0712d0 msgr2=0x7f9e6c0afe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:04.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.580+0000 7f9e72c00640 1 --2- 172.21.15.42:0/3403254903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e6c0712d0 0x7f9e6c0afe10 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c001eb0 tx=0x7f9e5c00c810 comp rx=0 tx=0).stop 2024-07-04T17:32:04.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.581+0000 7f9e72c00640 1 -- 172.21.15.42:0/3403254903 shutdown_connections 2024-07-04T17:32:04.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.581+0000 7f9e72c00640 1 --2- 172.21.15.42:0/3403254903 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f9e4803d3d0 0x7f9e4803f8e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:04.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.581+0000 7f9e72c00640 1 --2- 172.21.15.42:0/3403254903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e6c0712d0 0x7f9e6c0afe10 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:04.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.581+0000 7f9e72c00640 1 -- 172.21.15.42:0/3403254903 >> 172.21.15.42:0/3403254903 conn(0x7f9e6c06c020 msgr2=0x7f9e6c0b3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:04.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.581+0000 7f9e72c00640 1 -- 172.21.15.42:0/3403254903 shutdown_connections 2024-07-04T17:32:04.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.581+0000 7f9e72c00640 1 -- 172.21.15.42:0/3403254903 wait complete. 2024-07-04T17:32:04.616 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-04T17:32:04.616 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs allow_standby_replay false 2024-07-04T17:32:04.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.911+0000 7f5f7d600640 1 -- 172.21.15.42:0/1625562217 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f780abe40 msgr2=0x7f5f780aa900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:04.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.911+0000 7f5f7d600640 1 --2- 172.21.15.42:0/1625562217 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f780abe40 0x7f5f780aa900 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f5f640097b0 tx=0x7f5f6402a540 comp rx=0 tx=0).stop 2024-07-04T17:32:04.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.911+0000 7f5f7d600640 1 -- 172.21.15.42:0/1625562217 shutdown_connections 2024-07-04T17:32:04.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.911+0000 7f5f7d600640 1 --2- 172.21.15.42:0/1625562217 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f780abe40 0x7f5f780aa900 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:04.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.911+0000 7f5f7d600640 1 -- 172.21.15.42:0/1625562217 >> 172.21.15.42:0/1625562217 conn(0x7f5f780afaa0 msgr2=0x7f5f780afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:04.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.911+0000 7f5f7d600640 1 -- 172.21.15.42:0/1625562217 shutdown_connections 2024-07-04T17:32:04.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.911+0000 7f5f7d600640 1 -- 172.21.15.42:0/1625562217 wait complete. 2024-07-04T17:32:04.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.914+0000 7f5f7d600640 1 Processor -- start 2024-07-04T17:32:04.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.914+0000 7f5f7d600640 1 -- start start 2024-07-04T17:32:04.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.916+0000 7f5f7d600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f780701d0 0x7f5f7814d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:04.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.916+0000 7f5f7d600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5f78073ed0 con 0x7f5f780701d0 2024-07-04T17:32:04.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.917+0000 7f5f77e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f780701d0 0x7f5f7814d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:04.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.917+0000 7f5f77e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f780701d0 0x7f5f7814d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35352/0 (socket says 172.21.15.42:35352) 2024-07-04T17:32:04.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.917+0000 7f5f77e00640 1 -- 172.21.15.42:0/2605213085 learned_addr learned my addr 172.21.15.42:0/2605213085 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:04.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.917+0000 7f5f77e00640 1 -- 172.21.15.42:0/2605213085 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f78071120 con 0x7f5f780701d0 2024-07-04T17:32:04.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.917+0000 7f5f77e00640 1 --2- 172.21.15.42:0/2605213085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f780701d0 0x7f5f7814d380 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800c380 tx=0x7f5f6800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:04.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.918+0000 7f5f6fe00640 1 -- 172.21.15.42:0/2605213085 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f680107a0 con 0x7f5f780701d0 2024-07-04T17:32:04.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.918+0000 7f5f6fe00640 1 -- 172.21.15.42:0/2605213085 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f68017070 con 0x7f5f780701d0 2024-07-04T17:32:04.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.918+0000 7f5f7d600640 1 -- 172.21.15.42:0/2605213085 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f7814dbf0 con 0x7f5f780701d0 2024-07-04T17:32:04.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.918+0000 7f5f6fe00640 1 -- 172.21.15.42:0/2605213085 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f68010c00 con 0x7f5f780701d0 2024-07-04T17:32:04.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.918+0000 7f5f7d600640 1 -- 172.21.15.42:0/2605213085 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f7814cf50 con 0x7f5f780701d0 2024-07-04T17:32:04.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.919+0000 7f5f7d600640 1 -- 172.21.15.42:0/2605213085 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f7814df70 con 0x7f5f780701d0 2024-07-04T17:32:04.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.919+0000 7f5f6fe00640 1 -- 172.21.15.42:0/2605213085 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7f5f6801c460 con 0x7f5f780701d0 2024-07-04T17:32:04.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.920+0000 7f5f6fe00640 1 --2- 172.21.15.42:0/2605213085 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f5f5803d3d0 0x7f5f5803f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:04.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.920+0000 7f5f6fe00640 1 -- 172.21.15.42:0/2605213085 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4303+0+0 (secure 0 0 0) 0x7f5f68051770 con 0x7f5f780701d0 2024-07-04T17:32:04.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.920+0000 7f5f77400640 1 --2- 172.21.15.42:0/2605213085 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f5f5803d3d0 0x7f5f5803f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:04.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.921+0000 7f5f77400640 1 --2- 172.21.15.42:0/2605213085 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f5f5803d3d0 0x7f5f5803f8e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5f64034000 tx=0x7f5f64004730 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:04.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:04.925+0000 7f5f6fe00640 1 -- 172.21.15.42:0/2605213085 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f5f6801c800 con 0x7f5f780701d0 2024-07-04T17:32:05.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.183+0000 7f5f7d600640 1 -- 172.21.15.42:0/2605213085 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f5f78144d40 con 0x7f5f780701d0 2024-07-04T17:32:05.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.578+0000 7f5f6fe00640 1 -- 172.21.15.42:0/2605213085 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"}]=0 v77) ==== 125+0+0 (secure 0 0 0) 0x7f5f78144d40 con 0x7f5f780701d0 2024-07-04T17:32:05.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.582+0000 7f5f7d600640 1 -- 172.21.15.42:0/2605213085 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f5f5803d3d0 msgr2=0x7f5f5803f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:05.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.582+0000 7f5f7d600640 1 --2- 172.21.15.42:0/2605213085 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f5f5803d3d0 0x7f5f5803f8e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5f64034000 tx=0x7f5f64004730 comp rx=0 tx=0).stop 2024-07-04T17:32:05.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.582+0000 7f5f7d600640 1 -- 172.21.15.42:0/2605213085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f780701d0 msgr2=0x7f5f7814d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:05.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.582+0000 7f5f7d600640 1 --2- 172.21.15.42:0/2605213085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f780701d0 0x7f5f7814d380 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800c380 tx=0x7f5f6800c840 comp rx=0 tx=0).stop 2024-07-04T17:32:05.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.583+0000 7f5f7d600640 1 -- 172.21.15.42:0/2605213085 shutdown_connections 2024-07-04T17:32:05.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.583+0000 7f5f7d600640 1 --2- 172.21.15.42:0/2605213085 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f5f5803d3d0 0x7f5f5803f8e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:05.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.583+0000 7f5f7d600640 1 --2- 172.21.15.42:0/2605213085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f780701d0 0x7f5f7814d380 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:05.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.583+0000 7f5f7d600640 1 -- 172.21.15.42:0/2605213085 >> 172.21.15.42:0/2605213085 conn(0x7f5f780afaa0 msgr2=0x7f5f78144f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:05.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.583+0000 7f5f7d600640 1 -- 172.21.15.42:0/2605213085 shutdown_connections 2024-07-04T17:32:05.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.583+0000 7f5f7d600640 1 -- 172.21.15.42:0/2605213085 wait complete. 2024-07-04T17:32:05.619 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'backup_fs' 2024-07-04T17:32:05.620 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:32:05.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.913+0000 7f7382400640 1 -- 172.21.15.42:0/2014159807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f737c0b38b0 msgr2=0x7f737c0b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:05.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.913+0000 7f7382400640 1 --2- 172.21.15.42:0/2014159807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f737c0b38b0 0x7f737c0b2d20 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f736c005fd0 tx=0x7f736c02a5f0 comp rx=0 tx=0).stop 2024-07-04T17:32:05.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.913+0000 7f7382400640 1 -- 172.21.15.42:0/2014159807 shutdown_connections 2024-07-04T17:32:05.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.913+0000 7f7382400640 1 --2- 172.21.15.42:0/2014159807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f737c0b38b0 0x7f737c0b2d20 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:05.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.913+0000 7f7382400640 1 -- 172.21.15.42:0/2014159807 >> 172.21.15.42:0/2014159807 conn(0x7f737c0bf700 msgr2=0x7f737c0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:05.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.913+0000 7f7382400640 1 -- 172.21.15.42:0/2014159807 shutdown_connections 2024-07-04T17:32:05.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.913+0000 7f7382400640 1 -- 172.21.15.42:0/2014159807 wait complete. 2024-07-04T17:32:05.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.915+0000 7f7382400640 1 Processor -- start 2024-07-04T17:32:05.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.915+0000 7f7382400640 1 -- start start 2024-07-04T17:32:05.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.917+0000 7f7382400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f737c0b1e00 0x7f737c0ad7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:05.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.917+0000 7f7382400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f737c0b06e0 con 0x7f737c0b1e00 2024-07-04T17:32:05.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.917+0000 7f737b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f737c0b1e00 0x7f737c0ad7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:05.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.917+0000 7f737b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f737c0b1e00 0x7f737c0ad7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35362/0 (socket says 172.21.15.42:35362) 2024-07-04T17:32:05.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.917+0000 7f737b400640 1 -- 172.21.15.42:0/3551304393 learned_addr learned my addr 172.21.15.42:0/3551304393 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:05.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.917+0000 7f737b400640 1 -- 172.21.15.42:0/3551304393 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f737c0add70 con 0x7f737c0b1e00 2024-07-04T17:32:05.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.918+0000 7f737b400640 1 --2- 172.21.15.42:0/3551304393 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f737c0b1e00 0x7f737c0ad7e0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f736c009850 tx=0x7f736c0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:05.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.918+0000 7f7373e00640 1 -- 172.21.15.42:0/3551304393 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f736c03e030 con 0x7f737c0b1e00 2024-07-04T17:32:05.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.918+0000 7f7373e00640 1 -- 172.21.15.42:0/3551304393 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f736c0035f0 con 0x7f737c0b1e00 2024-07-04T17:32:05.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.918+0000 7f7382400640 1 -- 172.21.15.42:0/3551304393 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f737c0aecd0 con 0x7f737c0b1e00 2024-07-04T17:32:05.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.918+0000 7f7373e00640 1 -- 172.21.15.42:0/3551304393 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f736c035040 con 0x7f737c0b1e00 2024-07-04T17:32:05.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.918+0000 7f7382400640 1 -- 172.21.15.42:0/3551304393 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f737c13f2f0 con 0x7f737c0b1e00 2024-07-04T17:32:05.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.919+0000 7f7382400640 1 -- 172.21.15.42:0/3551304393 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f737c06c590 con 0x7f737c0b1e00 2024-07-04T17:32:05.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.920+0000 7f7373e00640 1 -- 172.21.15.42:0/3551304393 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7f736c041030 con 0x7f737c0b1e00 2024-07-04T17:32:05.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.920+0000 7f7373e00640 1 --2- 172.21.15.42:0/3551304393 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f735803d3d0 0x7f735803f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:05.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.921+0000 7f7373e00640 1 -- 172.21.15.42:0/3551304393 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4303+0+0 (secure 0 0 0) 0x7f736c03f070 con 0x7f737c0b1e00 2024-07-04T17:32:05.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.921+0000 7f7372a00640 1 --2- 172.21.15.42:0/3551304393 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f735803d3d0 0x7f735803f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:05.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.922+0000 7f7372a00640 1 --2- 172.21.15.42:0/3551304393 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f735803d3d0 0x7f735803f8e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f737c06ed80 tx=0x7f7364007d70 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:05.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:05.926+0000 7f7373e00640 1 -- 172.21.15.42:0/3551304393 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f736c04bd60 con 0x7f737c0b1e00 2024-07-04T17:32:06.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.192+0000 7f7382400640 1 -- 172.21.15.42:0/3551304393 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f737c069b70 con 0x7f737c0b1e00 2024-07-04T17:32:06.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.193+0000 7f7373e00640 1 -- 172.21.15.42:0/3551304393 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+7357 (secure 0 0 0) 0x7f736c030cc0 con 0x7f737c0b1e00 2024-07-04T17:32:06.193 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:32:06.193 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":77,"btime":"2024-07-04T17:32:05:576854+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/244742173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":244742173},{"type":"v1","addr":"172.21.15.42:6825","nonce":244742173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5265,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/973681058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":973681058},{"type":"v1","addr":"172.21.15.42:6827","nonce":973681058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/480649366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":480649366},{"type":"v1","addr":"172.21.15.42:6829","nonce":480649366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:31:41.664414+0000","modified":"2024-07-04T17:31:44.709115+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5277":{"gid":5277,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6833/3777807763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":3777807763},{"type":"v1","addr":"172.21.15.42:6833","nonce":3777807763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5277,"qdb_cluster":[5277]},"id":8},{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:32:02.103946+0000","modified":"2024-07-04T17:32:05.184521+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5276":{"gid":5276,"name":"d","rank":0,"incarnation":73,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6831/327636003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":327636003},{"type":"v1","addr":"172.21.15.42:6831","nonce":327636003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5276,"qdb_cluster":[5276]},"id":9}]} 2024-07-04T17:32:06.193 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 77 2024-07-04T17:32:06.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.196+0000 7f7382400640 1 -- 172.21.15.42:0/3551304393 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f735803d3d0 msgr2=0x7f735803f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:06.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.196+0000 7f7382400640 1 --2- 172.21.15.42:0/3551304393 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f735803d3d0 0x7f735803f8e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f737c06ed80 tx=0x7f7364007d70 comp rx=0 tx=0).stop 2024-07-04T17:32:06.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.196+0000 7f7382400640 1 -- 172.21.15.42:0/3551304393 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f737c0b1e00 msgr2=0x7f737c0ad7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:06.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.196+0000 7f7382400640 1 --2- 172.21.15.42:0/3551304393 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f737c0b1e00 0x7f737c0ad7e0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f736c009850 tx=0x7f736c0049d0 comp rx=0 tx=0).stop 2024-07-04T17:32:06.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.196+0000 7f7382400640 1 -- 172.21.15.42:0/3551304393 shutdown_connections 2024-07-04T17:32:06.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.196+0000 7f7382400640 1 --2- 172.21.15.42:0/3551304393 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f735803d3d0 0x7f735803f8e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:06.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.196+0000 7f7382400640 1 --2- 172.21.15.42:0/3551304393 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f737c0b1e00 0x7f737c0ad7e0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:06.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.196+0000 7f7382400640 1 -- 172.21.15.42:0/3551304393 >> 172.21.15.42:0/3551304393 conn(0x7f737c0bf700 msgr2=0x7f737c0bd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:06.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.196+0000 7f7382400640 1 -- 172.21.15.42:0/3551304393 shutdown_connections 2024-07-04T17:32:06.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.197+0000 7f7382400640 1 -- 172.21.15.42:0/3551304393 wait complete. 2024-07-04T17:32:06.224 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:32:06.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.505+0000 7fbbea200640 1 -- 172.21.15.42:0/118907153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbbe40bc040 msgr2=0x7fbbe40bd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:06.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.505+0000 7fbbea200640 1 --2- 172.21.15.42:0/118907153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbbe40bc040 0x7fbbe40bd2d0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc005fd0 tx=0x7fbbcc02a5f0 comp rx=0 tx=0).stop 2024-07-04T17:32:06.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.505+0000 7fbbea200640 1 -- 172.21.15.42:0/118907153 shutdown_connections 2024-07-04T17:32:06.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.505+0000 7fbbea200640 1 --2- 172.21.15.42:0/118907153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbbe40bc040 0x7fbbe40bd2d0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:06.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.505+0000 7fbbea200640 1 -- 172.21.15.42:0/118907153 >> 172.21.15.42:0/118907153 conn(0x7fbbe406c940 msgr2=0x7fbbe406ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:06.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.505+0000 7fbbea200640 1 -- 172.21.15.42:0/118907153 shutdown_connections 2024-07-04T17:32:06.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.505+0000 7fbbea200640 1 -- 172.21.15.42:0/118907153 wait complete. 2024-07-04T17:32:06.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.507+0000 7fbbea200640 1 Processor -- start 2024-07-04T17:32:06.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.508+0000 7fbbea200640 1 -- start start 2024-07-04T17:32:06.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.509+0000 7fbbea200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbbe40bc040 0x7fbbe41d92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:06.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.509+0000 7fbbea200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe40b7530 con 0x7fbbe40bc040 2024-07-04T17:32:06.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.509+0000 7fbbe3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbbe40bc040 0x7fbbe41d92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:06.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.509+0000 7fbbe3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbbe40bc040 0x7fbbe41d92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35376/0 (socket says 172.21.15.42:35376) 2024-07-04T17:32:06.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.509+0000 7fbbe3400640 1 -- 172.21.15.42:0/258185906 learned_addr learned my addr 172.21.15.42:0/258185906 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:06.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.510+0000 7fbbe3400640 1 -- 172.21.15.42:0/258185906 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe41e3720 con 0x7fbbe40bc040 2024-07-04T17:32:06.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.510+0000 7fbbe3400640 1 --2- 172.21.15.42:0/258185906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbbe40bc040 0x7fbbe41d92b0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc009850 tx=0x7fbbcc0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:06.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.510+0000 7fbbdb400640 1 -- 172.21.15.42:0/258185906 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbbcc03e030 con 0x7fbbe40bc040 2024-07-04T17:32:06.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.510+0000 7fbbdb400640 1 -- 172.21.15.42:0/258185906 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbcc0035f0 con 0x7fbbe40bc040 2024-07-04T17:32:06.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.510+0000 7fbbdb400640 1 -- 172.21.15.42:0/258185906 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbbcc035040 con 0x7fbbe40bc040 2024-07-04T17:32:06.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.510+0000 7fbbea200640 1 -- 172.21.15.42:0/258185906 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe41e39a0 con 0x7fbbe40bc040 2024-07-04T17:32:06.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.510+0000 7fbbea200640 1 -- 172.21.15.42:0/258185906 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe41d9a60 con 0x7fbbe40bc040 2024-07-04T17:32:06.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.511+0000 7fbbea200640 1 -- 172.21.15.42:0/258185906 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbe40b7c00 con 0x7fbbe40bc040 2024-07-04T17:32:06.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.512+0000 7fbbdb400640 1 -- 172.21.15.42:0/258185906 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7fbbcc041030 con 0x7fbbe40bc040 2024-07-04T17:32:06.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.512+0000 7fbbdb400640 1 --2- 172.21.15.42:0/258185906 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fbbc003d3d0 0x7fbbc003f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:06.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.512+0000 7fbbdb400640 1 -- 172.21.15.42:0/258185906 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4303+0+0 (secure 0 0 0) 0x7fbbcc03f070 con 0x7fbbe40bc040 2024-07-04T17:32:06.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.513+0000 7fbbe2a00640 1 --2- 172.21.15.42:0/258185906 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fbbc003d3d0 0x7fbbc003f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:06.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.513+0000 7fbbe2a00640 1 --2- 172.21.15.42:0/258185906 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fbbc003d3d0 0x7fbbc003f8e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4001560 tx=0x7fbbd40079f0 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:06.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.518+0000 7fbbdb400640 1 -- 172.21.15.42:0/258185906 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fbbcc04bd20 con 0x7fbbe40bc040 2024-07-04T17:32:06.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.730+0000 7fbbea200640 1 -- 172.21.15.42:0/258185906 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbbe41d97f0 con 0x7fbbe40bc040 2024-07-04T17:32:06.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.733+0000 7fbbdb400640 1 -- 172.21.15.42:0/258185906 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+14454 (secure 0 0 0) 0x7fbbcc030c90 con 0x7fbbe40bc040 2024-07-04T17:32:06.734 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:32:06.734 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":96,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:32:03.557225+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:31:39.727656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-07-04T17:31:40.615610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:31:59.635881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.4099999666213989,"score_stable":1.4099999666213989,"optimal_score":1,"raw_score_acting":1.4099999666213989,"raw_score_stable":1.4099999666213989,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:32:01.086110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.0800000429153442,"score_stable":1.0800000429153442,"optimal_score":1,"raw_score_acting":1.0800000429153442,"raw_score_stable":1.0800000429153442,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:32:06.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.736+0000 7fbbea200640 1 -- 172.21.15.42:0/258185906 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fbbc003d3d0 msgr2=0x7fbbc003f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:06.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.736+0000 7fbbea200640 1 --2- 172.21.15.42:0/258185906 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fbbc003d3d0 0x7fbbc003f8e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4001560 tx=0x7fbbd40079f0 comp rx=0 tx=0).stop 2024-07-04T17:32:06.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.736+0000 7fbbea200640 1 -- 172.21.15.42:0/258185906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbbe40bc040 msgr2=0x7fbbe41d92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:06.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.736+0000 7fbbea200640 1 --2- 172.21.15.42:0/258185906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbbe40bc040 0x7fbbe41d92b0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc009850 tx=0x7fbbcc0049d0 comp rx=0 tx=0).stop 2024-07-04T17:32:06.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.736+0000 7fbbea200640 1 -- 172.21.15.42:0/258185906 shutdown_connections 2024-07-04T17:32:06.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.736+0000 7fbbea200640 1 --2- 172.21.15.42:0/258185906 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fbbc003d3d0 0x7fbbc003f8e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:06.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.736+0000 7fbbea200640 1 --2- 172.21.15.42:0/258185906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbbe40bc040 0x7fbbe41d92b0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:06.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.736+0000 7fbbea200640 1 -- 172.21.15.42:0/258185906 >> 172.21.15.42:0/258185906 conn(0x7fbbe406c940 msgr2=0x7fbbe40b65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:06.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.736+0000 7fbbea200640 1 -- 172.21.15.42:0/258185906 shutdown_connections 2024-07-04T17:32:06.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:06.736+0000 7fbbea200640 1 -- 172.21.15.42:0/258185906 wait complete. 2024-07-04T17:32:06.763 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-04T17:32:06.763 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:32:07.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.054+0000 7f37a1000640 1 -- 172.21.15.42:0/2340204833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f379c0b2d20 msgr2=0x7f379c0b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:07.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.054+0000 7f37a1000640 1 --2- 172.21.15.42:0/2340204833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f379c0b2d20 0x7f379c0b01f0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f37880097b0 tx=0x7f378802a540 comp rx=0 tx=0).stop 2024-07-04T17:32:07.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.054+0000 7f37a1000640 1 -- 172.21.15.42:0/2340204833 shutdown_connections 2024-07-04T17:32:07.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.054+0000 7f37a1000640 1 --2- 172.21.15.42:0/2340204833 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f379c0b2d20 0x7f379c0b01f0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:07.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.054+0000 7f37a1000640 1 -- 172.21.15.42:0/2340204833 >> 172.21.15.42:0/2340204833 conn(0x7f379c0bf700 msgr2=0x7f379c0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:07.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.055+0000 7f37a1000640 1 -- 172.21.15.42:0/2340204833 shutdown_connections 2024-07-04T17:32:07.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.055+0000 7f37a1000640 1 -- 172.21.15.42:0/2340204833 wait complete. 2024-07-04T17:32:07.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.057+0000 7f37a1000640 1 Processor -- start 2024-07-04T17:32:07.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.057+0000 7f37a1000640 1 -- start start 2024-07-04T17:32:07.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.058+0000 7f37a1000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f379c0b2d20 0x7f379c143690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:07.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.058+0000 7f37a1000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f379c06cd40 con 0x7f379c0b2d20 2024-07-04T17:32:07.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.058+0000 7f3799e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f379c0b2d20 0x7f379c143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:07.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.059+0000 7f3799e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f379c0b2d20 0x7f379c143690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35398/0 (socket says 172.21.15.42:35398) 2024-07-04T17:32:07.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.059+0000 7f3799e00640 1 -- 172.21.15.42:0/2856428750 learned_addr learned my addr 172.21.15.42:0/2856428750 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:07.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.059+0000 7f3799e00640 1 -- 172.21.15.42:0/2856428750 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379c139a50 con 0x7f379c0b2d20 2024-07-04T17:32:07.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.059+0000 7f3799e00640 1 --2- 172.21.15.42:0/2856428750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f379c0b2d20 0x7f379c143690 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f378c001eb0 tx=0x7f378c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:07.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.060+0000 7f3792000640 1 -- 172.21.15.42:0/2856428750 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f378c0116c0 con 0x7f379c0b2d20 2024-07-04T17:32:07.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.060+0000 7f3792000640 1 -- 172.21.15.42:0/2856428750 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f378c017070 con 0x7f379c0b2d20 2024-07-04T17:32:07.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.060+0000 7f3792000640 1 -- 172.21.15.42:0/2856428750 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f378c011ae0 con 0x7f379c0b2d20 2024-07-04T17:32:07.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.060+0000 7f37a1000640 1 -- 172.21.15.42:0/2856428750 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f379c139ef0 con 0x7f379c0b2d20 2024-07-04T17:32:07.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.060+0000 7f37a1000640 1 -- 172.21.15.42:0/2856428750 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f379c073870 con 0x7f379c0b2d20 2024-07-04T17:32:07.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.061+0000 7f37a1000640 1 -- 172.21.15.42:0/2856428750 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f379c139840 con 0x7f379c0b2d20 2024-07-04T17:32:07.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.061+0000 7f3792000640 1 -- 172.21.15.42:0/2856428750 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7f378c011c80 con 0x7f379c0b2d20 2024-07-04T17:32:07.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.062+0000 7f3792000640 1 --2- 172.21.15.42:0/2856428750 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f377c03d3d0 0x7f377c03f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:07.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.062+0000 7f3792000640 1 -- 172.21.15.42:0/2856428750 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4303+0+0 (secure 0 0 0) 0x7f378c051810 con 0x7f379c0b2d20 2024-07-04T17:32:07.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.062+0000 7f3799400640 1 --2- 172.21.15.42:0/2856428750 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f377c03d3d0 0x7f377c03f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:07.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.063+0000 7f3799400640 1 --2- 172.21.15.42:0/2856428750 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f377c03d3d0 0x7f377c03f8e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3788034000 tx=0x7f3788004730 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:07.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.067+0000 7f3792000640 1 -- 172.21.15.42:0/2856428750 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f378c010cd0 con 0x7f379c0b2d20 2024-07-04T17:32:07.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.273+0000 7f37a1000640 1 -- 172.21.15.42:0/2856428750 --> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f379c06d450 con 0x7f377c03d3d0 2024-07-04T17:32:07.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.286+0000 7f3792000640 1 -- 172.21.15.42:0/2856428750 <== mgr.5173 v2:172.21.15.42:6834/1139872311 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+482444 (secure 0 0 0) 0x7f379c06d450 con 0x7f377c03d3d0 2024-07-04T17:32:07.293 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:32:07.297 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:32:07.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.293+0000 7f37a1000640 1 -- 172.21.15.42:0/2856428750 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f377c03d3d0 msgr2=0x7f377c03f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:07.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.293+0000 7f37a1000640 1 --2- 172.21.15.42:0/2856428750 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f377c03d3d0 0x7f377c03f8e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3788034000 tx=0x7f3788004730 comp rx=0 tx=0).stop 2024-07-04T17:32:07.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.293+0000 7f37a1000640 1 -- 172.21.15.42:0/2856428750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f379c0b2d20 msgr2=0x7f379c143690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:07.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.293+0000 7f37a1000640 1 --2- 172.21.15.42:0/2856428750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f379c0b2d20 0x7f379c143690 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f378c001eb0 tx=0x7f378c00c810 comp rx=0 tx=0).stop 2024-07-04T17:32:07.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.293+0000 7f37a1000640 1 -- 172.21.15.42:0/2856428750 shutdown_connections 2024-07-04T17:32:07.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.293+0000 7f37a1000640 1 --2- 172.21.15.42:0/2856428750 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f377c03d3d0 0x7f377c03f8e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:07.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.293+0000 7f37a1000640 1 --2- 172.21.15.42:0/2856428750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f379c0b2d20 0x7f379c143690 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:07.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.293+0000 7f37a1000640 1 -- 172.21.15.42:0/2856428750 >> 172.21.15.42:0/2856428750 conn(0x7f379c0bf700 msgr2=0x7f379c075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:07.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.293+0000 7f37a1000640 1 -- 172.21.15.42:0/2856428750 shutdown_connections 2024-07-04T17:32:07.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.293+0000 7f37a1000640 1 -- 172.21.15.42:0/2856428750 wait complete. 2024-07-04T17:32:07.319 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-07-04T17:32:06.446250+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596885,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":356,"num_read_kb":926,"num_write":231,"num_write_kb":1137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":299,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":110172,"kb_used_data":4132,"kb_used_omap":4,"kb_used_meta":105979,"kb_avail":283005348,"statfs":{"total":289910292480,"available":289797476352,"internally_reserved":0,"allocated":4231168,"data_stored":2346988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":108522831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.918327"},"pg_stats":[{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.992791+0000","last_change":"2024-07-04T17:32:02.593945+0000","last_active":"2024-07-04T17:32:03.992791+0000","last_peered":"2024-07-04T17:32:03.992791+0000","last_clean":"2024-07-04T17:32:03.992791+0000","last_became_active":"2024-07-04T17:32:02.592848+0000","last_became_peered":"2024-07-04T17:32:02.592848+0000","last_unstale":"2024-07-04T17:32:03.992791+0000","last_undegraded":"2024-07-04T17:32:03.992791+0000","last_fullsized":"2024-07-04T17:32:03.992791+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:55:16.109523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"80'87","reported_seq":364,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.984767+0000","last_change":"2024-07-04T17:26:04.953559+0000","last_active":"2024-07-04T17:32:03.984767+0000","last_peered":"2024-07-04T17:32:03.984767+0000","last_clean":"2024-07-04T17:32:03.984767+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:32:03.984767+0000","last_undegraded":"2024-07-04T17:32:03.984767+0000","last_fullsized":"2024-07-04T17:32:03.984767+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_clean_scrub_stamp":"2024-07-04T17:26:03.933432+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.064181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":356,"num_read_kb":926,"num_write":139,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.571765+0000","last_change":"2024-07-04T17:31:41.125061+0000","last_active":"2024-07-04T17:32:03.571765+0000","last_peered":"2024-07-04T17:32:03.571765+0000","last_clean":"2024-07-04T17:32:03.571765+0000","last_became_active":"2024-07-04T17:31:41.122766+0000","last_became_peered":"2024-07-04T17:31:41.122766+0000","last_unstale":"2024-07-04T17:32:03.571765+0000","last_undegraded":"2024-07-04T17:32:03.571765+0000","last_fullsized":"2024-07-04T17:32:03.571765+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:00:42.281350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.987553+0000","last_change":"2024-07-04T17:31:42.124940+0000","last_active":"2024-07-04T17:32:03.987553+0000","last_peered":"2024-07-04T17:32:03.987553+0000","last_clean":"2024-07-04T17:32:03.987553+0000","last_became_active":"2024-07-04T17:31:42.124608+0000","last_became_peered":"2024-07-04T17:31:42.124608+0000","last_unstale":"2024-07-04T17:32:03.987553+0000","last_undegraded":"2024-07-04T17:32:03.987553+0000","last_fullsized":"2024-07-04T17:32:03.987553+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:11:02.635340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.13","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.990871+0000","last_change":"2024-07-04T17:32:01.584681+0000","last_active":"2024-07-04T17:32:03.990871+0000","last_peered":"2024-07-04T17:32:03.990871+0000","last_clean":"2024-07-04T17:32:03.990871+0000","last_became_active":"2024-07-04T17:32:01.584237+0000","last_became_peered":"2024-07-04T17:32:01.584237+0000","last_unstale":"2024-07-04T17:32:03.990871+0000","last_undegraded":"2024-07-04T17:32:03.990871+0000","last_fullsized":"2024-07-04T17:32:03.990871+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:17:45.481113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.990690+0000","last_change":"2024-07-04T17:32:02.580087+0000","last_active":"2024-07-04T17:32:03.990690+0000","last_peered":"2024-07-04T17:32:03.990690+0000","last_clean":"2024-07-04T17:32:03.990690+0000","last_became_active":"2024-07-04T17:32:02.579751+0000","last_became_peered":"2024-07-04T17:32:02.579751+0000","last_unstale":"2024-07-04T17:32:03.990690+0000","last_undegraded":"2024-07-04T17:32:03.990690+0000","last_fullsized":"2024-07-04T17:32:03.990690+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:05:58.936715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.988239+0000","last_change":"2024-07-04T17:31:41.111066+0000","last_active":"2024-07-04T17:32:03.988239+0000","last_peered":"2024-07-04T17:32:03.988239+0000","last_clean":"2024-07-04T17:32:03.988239+0000","last_became_active":"2024-07-04T17:31:41.110689+0000","last_became_peered":"2024-07-04T17:31:41.110689+0000","last_unstale":"2024-07-04T17:32:03.988239+0000","last_undegraded":"2024-07-04T17:32:03.988239+0000","last_fullsized":"2024-07-04T17:32:03.988239+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:24:51.654346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.991127+0000","last_change":"2024-07-04T17:31:42.145454+0000","last_active":"2024-07-04T17:32:03.991127+0000","last_peered":"2024-07-04T17:32:03.991127+0000","last_clean":"2024-07-04T17:32:03.991127+0000","last_became_active":"2024-07-04T17:31:42.144919+0000","last_became_peered":"2024-07-04T17:31:42.144919+0000","last_unstale":"2024-07-04T17:32:03.991127+0000","last_undegraded":"2024-07-04T17:32:03.991127+0000","last_fullsized":"2024-07-04T17:32:03.991127+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:30:26.887002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.991864+0000","last_change":"2024-07-04T17:32:01.585972+0000","last_active":"2024-07-04T17:32:03.991864+0000","last_peered":"2024-07-04T17:32:03.991864+0000","last_clean":"2024-07-04T17:32:03.991864+0000","last_became_active":"2024-07-04T17:32:01.583249+0000","last_became_peered":"2024-07-04T17:32:01.583249+0000","last_unstale":"2024-07-04T17:32:03.991864+0000","last_undegraded":"2024-07-04T17:32:03.991864+0000","last_fullsized":"2024-07-04T17:32:03.991864+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:42:13.449049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.992861+0000","last_change":"2024-07-04T17:32:02.604765+0000","last_active":"2024-07-04T17:32:03.992861+0000","last_peered":"2024-07-04T17:32:03.992861+0000","last_clean":"2024-07-04T17:32:03.992861+0000","last_became_active":"2024-07-04T17:32:02.603246+0000","last_became_peered":"2024-07-04T17:32:02.603246+0000","last_unstale":"2024-07-04T17:32:03.992861+0000","last_undegraded":"2024-07-04T17:32:03.992861+0000","last_fullsized":"2024-07-04T17:32:03.992861+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:34:45.295638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.571936+0000","last_change":"2024-07-04T17:31:41.116814+0000","last_active":"2024-07-04T17:32:03.571936+0000","last_peered":"2024-07-04T17:32:03.571936+0000","last_clean":"2024-07-04T17:32:03.571936+0000","last_became_active":"2024-07-04T17:31:41.116425+0000","last_became_peered":"2024-07-04T17:31:41.116425+0000","last_unstale":"2024-07-04T17:32:03.571936+0000","last_undegraded":"2024-07-04T17:32:03.571936+0000","last_fullsized":"2024-07-04T17:32:03.571936+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:18:00.134663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.987772+0000","last_change":"2024-07-04T17:31:42.112865+0000","last_active":"2024-07-04T17:32:03.987772+0000","last_peered":"2024-07-04T17:32:03.987772+0000","last_clean":"2024-07-04T17:32:03.987772+0000","last_became_active":"2024-07-04T17:31:42.109293+0000","last_became_peered":"2024-07-04T17:31:42.109293+0000","last_unstale":"2024-07-04T17:32:03.987772+0000","last_undegraded":"2024-07-04T17:32:03.987772+0000","last_fullsized":"2024-07-04T17:32:03.987772+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:07:14.667638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.985688+0000","last_change":"2024-07-04T17:32:01.596072+0000","last_active":"2024-07-04T17:32:03.985688+0000","last_peered":"2024-07-04T17:32:03.985688+0000","last_clean":"2024-07-04T17:32:03.985688+0000","last_became_active":"2024-07-04T17:32:01.593045+0000","last_became_peered":"2024-07-04T17:32:01.593045+0000","last_unstale":"2024-07-04T17:32:03.985688+0000","last_undegraded":"2024-07-04T17:32:03.985688+0000","last_fullsized":"2024-07-04T17:32:03.985688+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:24:56.768417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.984166+0000","last_change":"2024-07-04T17:32:02.619573+0000","last_active":"2024-07-04T17:32:03.984166+0000","last_peered":"2024-07-04T17:32:03.984166+0000","last_clean":"2024-07-04T17:32:03.984166+0000","last_became_active":"2024-07-04T17:32:02.614841+0000","last_became_peered":"2024-07-04T17:32:02.614841+0000","last_unstale":"2024-07-04T17:32:03.984166+0000","last_undegraded":"2024-07-04T17:32:03.984166+0000","last_fullsized":"2024-07-04T17:32:03.984166+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:47:31.973324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.993143+0000","last_change":"2024-07-04T17:31:41.124595+0000","last_active":"2024-07-04T17:32:03.993143+0000","last_peered":"2024-07-04T17:32:03.993143+0000","last_clean":"2024-07-04T17:32:03.993143+0000","last_became_active":"2024-07-04T17:31:41.124315+0000","last_became_peered":"2024-07-04T17:31:41.124315+0000","last_unstale":"2024-07-04T17:32:03.993143+0000","last_undegraded":"2024-07-04T17:32:03.993143+0000","last_fullsized":"2024-07-04T17:32:03.993143+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:10:03.734241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.992466+0000","last_change":"2024-07-04T17:31:42.111629+0000","last_active":"2024-07-04T17:32:03.992466+0000","last_peered":"2024-07-04T17:32:03.992466+0000","last_clean":"2024-07-04T17:32:03.992466+0000","last_became_active":"2024-07-04T17:31:42.110978+0000","last_became_peered":"2024-07-04T17:31:42.110978+0000","last_unstale":"2024-07-04T17:32:03.992466+0000","last_undegraded":"2024-07-04T17:32:03.992466+0000","last_fullsized":"2024-07-04T17:32:03.992466+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:12:01.825579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.986994+0000","last_change":"2024-07-04T17:32:01.586152+0000","last_active":"2024-07-04T17:32:03.986994+0000","last_peered":"2024-07-04T17:32:03.986994+0000","last_clean":"2024-07-04T17:32:03.986994+0000","last_became_active":"2024-07-04T17:32:01.583465+0000","last_became_peered":"2024-07-04T17:32:01.583465+0000","last_unstale":"2024-07-04T17:32:03.986994+0000","last_undegraded":"2024-07-04T17:32:03.986994+0000","last_fullsized":"2024-07-04T17:32:03.986994+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:38:19.387018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.571065+0000","last_change":"2024-07-04T17:32:02.583838+0000","last_active":"2024-07-04T17:32:03.571065+0000","last_peered":"2024-07-04T17:32:03.571065+0000","last_clean":"2024-07-04T17:32:03.571065+0000","last_became_active":"2024-07-04T17:32:02.582748+0000","last_became_peered":"2024-07-04T17:32:02.582748+0000","last_unstale":"2024-07-04T17:32:03.571065+0000","last_undegraded":"2024-07-04T17:32:03.571065+0000","last_fullsized":"2024-07-04T17:32:03.571065+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:31:30.629913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.987969+0000","last_change":"2024-07-04T17:31:41.128454+0000","last_active":"2024-07-04T17:32:03.987969+0000","last_peered":"2024-07-04T17:32:03.987969+0000","last_clean":"2024-07-04T17:32:03.987969+0000","last_became_active":"2024-07-04T17:31:41.127785+0000","last_became_peered":"2024-07-04T17:31:41.127785+0000","last_unstale":"2024-07-04T17:32:03.987969+0000","last_undegraded":"2024-07-04T17:32:03.987969+0000","last_fullsized":"2024-07-04T17:32:03.987969+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:48:22.712062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.985890+0000","last_change":"2024-07-04T17:31:42.125396+0000","last_active":"2024-07-04T17:32:03.985890+0000","last_peered":"2024-07-04T17:32:03.985890+0000","last_clean":"2024-07-04T17:32:03.985890+0000","last_became_active":"2024-07-04T17:31:42.123980+0000","last_became_peered":"2024-07-04T17:31:42.123980+0000","last_unstale":"2024-07-04T17:32:03.985890+0000","last_undegraded":"2024-07-04T17:32:03.985890+0000","last_fullsized":"2024-07-04T17:32:03.985890+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:59:35.007753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.990654+0000","last_change":"2024-07-04T17:32:01.581492+0000","last_active":"2024-07-04T17:32:03.990654+0000","last_peered":"2024-07-04T17:32:03.990654+0000","last_clean":"2024-07-04T17:32:03.990654+0000","last_became_active":"2024-07-04T17:32:01.580962+0000","last_became_peered":"2024-07-04T17:32:01.580962+0000","last_unstale":"2024-07-04T17:32:03.990654+0000","last_undegraded":"2024-07-04T17:32:03.990654+0000","last_fullsized":"2024-07-04T17:32:03.990654+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:49:28.365441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.575945+0000","last_change":"2024-07-04T17:32:02.601444+0000","last_active":"2024-07-04T17:32:03.575945+0000","last_peered":"2024-07-04T17:32:03.575945+0000","last_clean":"2024-07-04T17:32:03.575945+0000","last_became_active":"2024-07-04T17:32:02.601078+0000","last_became_peered":"2024-07-04T17:32:02.601078+0000","last_unstale":"2024-07-04T17:32:03.575945+0000","last_undegraded":"2024-07-04T17:32:03.575945+0000","last_fullsized":"2024-07-04T17:32:03.575945+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:19:40.704357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.574906+0000","last_change":"2024-07-04T17:31:41.119885+0000","last_active":"2024-07-04T17:32:03.574906+0000","last_peered":"2024-07-04T17:32:03.574906+0000","last_clean":"2024-07-04T17:32:03.574906+0000","last_became_active":"2024-07-04T17:31:41.116720+0000","last_became_peered":"2024-07-04T17:31:41.116720+0000","last_unstale":"2024-07-04T17:32:03.574906+0000","last_undegraded":"2024-07-04T17:32:03.574906+0000","last_fullsized":"2024-07-04T17:32:03.574906+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:30:49.905769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.994142+0000","last_change":"2024-07-04T17:31:42.109920+0000","last_active":"2024-07-04T17:32:03.994142+0000","last_peered":"2024-07-04T17:32:03.994142+0000","last_clean":"2024-07-04T17:32:03.994142+0000","last_became_active":"2024-07-04T17:31:42.109357+0000","last_became_peered":"2024-07-04T17:31:42.109357+0000","last_unstale":"2024-07-04T17:32:03.994142+0000","last_undegraded":"2024-07-04T17:32:03.994142+0000","last_fullsized":"2024-07-04T17:32:03.994142+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:58:16.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.6","version":"96'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:04.558806+0000","last_change":"2024-07-04T17:32:01.595020+0000","last_active":"2024-07-04T17:32:04.558806+0000","last_peered":"2024-07-04T17:32:04.558806+0000","last_clean":"2024-07-04T17:32:04.558806+0000","last_became_active":"2024-07-04T17:32:01.594717+0000","last_became_peered":"2024-07-04T17:32:01.594717+0000","last_unstale":"2024-07-04T17:32:04.558806+0000","last_undegraded":"2024-07-04T17:32:04.558806+0000","last_fullsized":"2024-07-04T17:32:04.558806+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:58:30.462288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.995599+0000","last_change":"2024-07-04T17:32:02.617612+0000","last_active":"2024-07-04T17:32:03.995599+0000","last_peered":"2024-07-04T17:32:03.995599+0000","last_clean":"2024-07-04T17:32:03.995599+0000","last_became_active":"2024-07-04T17:32:02.615735+0000","last_became_peered":"2024-07-04T17:32:02.615735+0000","last_unstale":"2024-07-04T17:32:03.995599+0000","last_undegraded":"2024-07-04T17:32:03.995599+0000","last_fullsized":"2024-07-04T17:32:03.995599+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:31:30.629913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.995253+0000","last_change":"2024-07-04T17:31:41.123150+0000","last_active":"2024-07-04T17:32:03.995253+0000","last_peered":"2024-07-04T17:32:03.995253+0000","last_clean":"2024-07-04T17:32:03.995253+0000","last_became_active":"2024-07-04T17:31:41.120767+0000","last_became_peered":"2024-07-04T17:31:41.120767+0000","last_unstale":"2024-07-04T17:32:03.995253+0000","last_undegraded":"2024-07-04T17:32:03.995253+0000","last_fullsized":"2024-07-04T17:32:03.995253+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:42:19.857048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.992454+0000","last_change":"2024-07-04T17:31:42.143202+0000","last_active":"2024-07-04T17:32:03.992454+0000","last_peered":"2024-07-04T17:32:03.992454+0000","last_clean":"2024-07-04T17:32:03.992454+0000","last_became_active":"2024-07-04T17:31:42.141656+0000","last_became_peered":"2024-07-04T17:31:42.141656+0000","last_unstale":"2024-07-04T17:32:03.992454+0000","last_undegraded":"2024-07-04T17:32:03.992454+0000","last_fullsized":"2024-07-04T17:32:03.992454+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:19:20.240973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.7","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.576045+0000","last_change":"2024-07-04T17:32:01.598772+0000","last_active":"2024-07-04T17:32:03.576045+0000","last_peered":"2024-07-04T17:32:03.576045+0000","last_clean":"2024-07-04T17:32:03.576045+0000","last_became_active":"2024-07-04T17:32:01.597904+0000","last_became_peered":"2024-07-04T17:32:01.597904+0000","last_unstale":"2024-07-04T17:32:03.576045+0000","last_undegraded":"2024-07-04T17:32:03.576045+0000","last_fullsized":"2024-07-04T17:32:03.576045+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:20:19.495765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.993364+0000","last_change":"2024-07-04T17:32:02.602505+0000","last_active":"2024-07-04T17:32:03.993364+0000","last_peered":"2024-07-04T17:32:03.993364+0000","last_clean":"2024-07-04T17:32:03.993364+0000","last_became_active":"2024-07-04T17:32:02.601949+0000","last_became_peered":"2024-07-04T17:32:02.601949+0000","last_unstale":"2024-07-04T17:32:03.993364+0000","last_undegraded":"2024-07-04T17:32:03.993364+0000","last_fullsized":"2024-07-04T17:32:03.993364+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:36:16.683047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"92'1","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.996395+0000","last_change":"2024-07-04T17:31:41.126383+0000","last_active":"2024-07-04T17:32:03.996395+0000","last_peered":"2024-07-04T17:32:03.996395+0000","last_clean":"2024-07-04T17:32:03.996395+0000","last_became_active":"2024-07-04T17:31:41.126045+0000","last_became_peered":"2024-07-04T17:31:41.126045+0000","last_unstale":"2024-07-04T17:32:03.996395+0000","last_undegraded":"2024-07-04T17:32:03.996395+0000","last_fullsized":"2024-07-04T17:32:03.996395+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:57:59.125923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.996117+0000","last_change":"2024-07-04T17:31:42.107634+0000","last_active":"2024-07-04T17:32:03.996117+0000","last_peered":"2024-07-04T17:32:03.996117+0000","last_clean":"2024-07-04T17:32:03.996117+0000","last_became_active":"2024-07-04T17:31:42.107327+0000","last_became_peered":"2024-07-04T17:31:42.107327+0000","last_unstale":"2024-07-04T17:32:03.996117+0000","last_undegraded":"2024-07-04T17:32:03.996117+0000","last_fullsized":"2024-07-04T17:32:03.996117+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:57:08.828739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.992050+0000","last_change":"2024-07-04T17:32:01.575457+0000","last_active":"2024-07-04T17:32:03.992050+0000","last_peered":"2024-07-04T17:32:03.992050+0000","last_clean":"2024-07-04T17:32:03.992050+0000","last_became_active":"2024-07-04T17:32:01.575101+0000","last_became_peered":"2024-07-04T17:32:01.575101+0000","last_unstale":"2024-07-04T17:32:03.992050+0000","last_undegraded":"2024-07-04T17:32:03.992050+0000","last_fullsized":"2024-07-04T17:32:03.992050+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:48:06.134693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.988118+0000","last_change":"2024-07-04T17:32:02.595302+0000","last_active":"2024-07-04T17:32:03.988118+0000","last_peered":"2024-07-04T17:32:03.988118+0000","last_clean":"2024-07-04T17:32:03.988118+0000","last_became_active":"2024-07-04T17:32:02.592327+0000","last_became_peered":"2024-07-04T17:32:02.592327+0000","last_unstale":"2024-07-04T17:32:03.988118+0000","last_undegraded":"2024-07-04T17:32:03.988118+0000","last_fullsized":"2024-07-04T17:32:03.988118+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:18:39.023689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"91'2","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.996018+0000","last_change":"2024-07-04T17:31:41.126950+0000","last_active":"2024-07-04T17:32:03.996018+0000","last_peered":"2024-07-04T17:32:03.996018+0000","last_clean":"2024-07-04T17:32:03.996018+0000","last_became_active":"2024-07-04T17:31:41.126611+0000","last_became_peered":"2024-07-04T17:31:41.126611+0000","last_unstale":"2024-07-04T17:32:03.996018+0000","last_undegraded":"2024-07-04T17:32:03.996018+0000","last_fullsized":"2024-07-04T17:32:03.996018+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:27:51.965765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.996277+0000","last_change":"2024-07-04T17:31:42.142194+0000","last_active":"2024-07-04T17:32:03.996277+0000","last_peered":"2024-07-04T17:32:03.996277+0000","last_clean":"2024-07-04T17:32:03.996277+0000","last_became_active":"2024-07-04T17:31:42.139517+0000","last_became_peered":"2024-07-04T17:31:42.139517+0000","last_unstale":"2024-07-04T17:32:03.996277+0000","last_undegraded":"2024-07-04T17:32:03.996277+0000","last_fullsized":"2024-07-04T17:32:03.996277+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:12:09.965552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.994959+0000","last_change":"2024-07-04T17:32:01.588031+0000","last_active":"2024-07-04T17:32:03.994959+0000","last_peered":"2024-07-04T17:32:03.994959+0000","last_clean":"2024-07-04T17:32:03.994959+0000","last_became_active":"2024-07-04T17:32:01.586551+0000","last_became_peered":"2024-07-04T17:32:01.586551+0000","last_unstale":"2024-07-04T17:32:03.994959+0000","last_undegraded":"2024-07-04T17:32:03.994959+0000","last_fullsized":"2024-07-04T17:32:03.994959+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:10:04.034800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.985418+0000","last_change":"2024-07-04T17:32:02.615560+0000","last_active":"2024-07-04T17:32:03.985418+0000","last_peered":"2024-07-04T17:32:03.985418+0000","last_clean":"2024-07-04T17:32:03.985418+0000","last_became_active":"2024-07-04T17:32:02.615207+0000","last_became_peered":"2024-07-04T17:32:02.615207+0000","last_unstale":"2024-07-04T17:32:03.985418+0000","last_undegraded":"2024-07-04T17:32:03.985418+0000","last_fullsized":"2024-07-04T17:32:03.985418+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:14:08.180824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.988086+0000","last_change":"2024-07-04T17:31:41.114344+0000","last_active":"2024-07-04T17:32:03.988086+0000","last_peered":"2024-07-04T17:32:03.988086+0000","last_clean":"2024-07-04T17:32:03.988086+0000","last_became_active":"2024-07-04T17:31:41.114006+0000","last_became_peered":"2024-07-04T17:31:41.114006+0000","last_unstale":"2024-07-04T17:32:03.988086+0000","last_undegraded":"2024-07-04T17:32:03.988086+0000","last_fullsized":"2024-07-04T17:32:03.988086+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:57:55.654579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.985032+0000","last_change":"2024-07-04T17:31:42.124705+0000","last_active":"2024-07-04T17:32:03.985032+0000","last_peered":"2024-07-04T17:32:03.985032+0000","last_clean":"2024-07-04T17:32:03.985032+0000","last_became_active":"2024-07-04T17:31:42.124266+0000","last_became_peered":"2024-07-04T17:31:42.124266+0000","last_unstale":"2024-07-04T17:32:03.985032+0000","last_undegraded":"2024-07-04T17:32:03.985032+0000","last_fullsized":"2024-07-04T17:32:03.985032+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:18:01.138996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.572374+0000","last_change":"2024-07-04T17:32:01.591822+0000","last_active":"2024-07-04T17:32:03.572374+0000","last_peered":"2024-07-04T17:32:03.572374+0000","last_clean":"2024-07-04T17:32:03.572374+0000","last_became_active":"2024-07-04T17:32:01.591408+0000","last_became_peered":"2024-07-04T17:32:01.591408+0000","last_unstale":"2024-07-04T17:32:03.572374+0000","last_undegraded":"2024-07-04T17:32:03.572374+0000","last_fullsized":"2024-07-04T17:32:03.572374+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:03:11.489221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.571952+0000","last_change":"2024-07-04T17:32:02.606095+0000","last_active":"2024-07-04T17:32:03.571952+0000","last_peered":"2024-07-04T17:32:03.571952+0000","last_clean":"2024-07-04T17:32:03.571952+0000","last_became_active":"2024-07-04T17:32:02.604978+0000","last_became_peered":"2024-07-04T17:32:02.604978+0000","last_unstale":"2024-07-04T17:32:03.571952+0000","last_undegraded":"2024-07-04T17:32:03.571952+0000","last_fullsized":"2024-07-04T17:32:03.571952+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:11:30.547164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.985196+0000","last_change":"2024-07-04T17:31:41.120856+0000","last_active":"2024-07-04T17:32:03.985196+0000","last_peered":"2024-07-04T17:32:03.985196+0000","last_clean":"2024-07-04T17:32:03.985196+0000","last_became_active":"2024-07-04T17:31:41.120338+0000","last_became_peered":"2024-07-04T17:31:41.120338+0000","last_unstale":"2024-07-04T17:32:03.985196+0000","last_undegraded":"2024-07-04T17:32:03.985196+0000","last_fullsized":"2024-07-04T17:32:03.985196+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:14:01.301278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.987538+0000","last_change":"2024-07-04T17:31:42.111799+0000","last_active":"2024-07-04T17:32:03.987538+0000","last_peered":"2024-07-04T17:32:03.987538+0000","last_clean":"2024-07-04T17:32:03.987538+0000","last_became_active":"2024-07-04T17:31:42.110114+0000","last_became_peered":"2024-07-04T17:31:42.110114+0000","last_unstale":"2024-07-04T17:32:03.987538+0000","last_undegraded":"2024-07-04T17:32:03.987538+0000","last_fullsized":"2024-07-04T17:32:03.987538+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:45:25.449722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.b","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.985605+0000","last_change":"2024-07-04T17:32:01.596098+0000","last_active":"2024-07-04T17:32:03.985605+0000","last_peered":"2024-07-04T17:32:03.985605+0000","last_clean":"2024-07-04T17:32:03.985605+0000","last_became_active":"2024-07-04T17:32:01.593474+0000","last_became_peered":"2024-07-04T17:32:01.593474+0000","last_unstale":"2024-07-04T17:32:03.985605+0000","last_undegraded":"2024-07-04T17:32:03.985605+0000","last_fullsized":"2024-07-04T17:32:03.985605+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:58:36.722442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.988182+0000","last_change":"2024-07-04T17:32:02.580255+0000","last_active":"2024-07-04T17:32:03.988182+0000","last_peered":"2024-07-04T17:32:03.988182+0000","last_clean":"2024-07-04T17:32:03.988182+0000","last_became_active":"2024-07-04T17:32:02.579908+0000","last_became_peered":"2024-07-04T17:32:02.579908+0000","last_unstale":"2024-07-04T17:32:03.988182+0000","last_undegraded":"2024-07-04T17:32:03.988182+0000","last_fullsized":"2024-07-04T17:32:03.988182+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:04:12.247110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.573095+0000","last_change":"2024-07-04T17:31:41.131853+0000","last_active":"2024-07-04T17:32:03.573095+0000","last_peered":"2024-07-04T17:32:03.573095+0000","last_clean":"2024-07-04T17:32:03.573095+0000","last_became_active":"2024-07-04T17:31:41.129778+0000","last_became_peered":"2024-07-04T17:31:41.129778+0000","last_unstale":"2024-07-04T17:32:03.573095+0000","last_undegraded":"2024-07-04T17:32:03.573095+0000","last_fullsized":"2024-07-04T17:32:03.573095+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:59:40.102134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.992145+0000","last_change":"2024-07-04T17:31:42.148767+0000","last_active":"2024-07-04T17:32:03.992145+0000","last_peered":"2024-07-04T17:32:03.992145+0000","last_clean":"2024-07-04T17:32:03.992145+0000","last_became_active":"2024-07-04T17:31:42.148091+0000","last_became_peered":"2024-07-04T17:31:42.148091+0000","last_unstale":"2024-07-04T17:32:03.992145+0000","last_undegraded":"2024-07-04T17:32:03.992145+0000","last_fullsized":"2024-07-04T17:32:03.992145+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:42:04.145597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.986868+0000","last_change":"2024-07-04T17:32:01.584737+0000","last_active":"2024-07-04T17:32:03.986868+0000","last_peered":"2024-07-04T17:32:03.986868+0000","last_clean":"2024-07-04T17:32:03.986868+0000","last_became_active":"2024-07-04T17:32:01.583883+0000","last_became_peered":"2024-07-04T17:32:01.583883+0000","last_unstale":"2024-07-04T17:32:03.986868+0000","last_undegraded":"2024-07-04T17:32:03.986868+0000","last_fullsized":"2024-07-04T17:32:03.986868+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:09:42.708030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.983928+0000","last_change":"2024-07-04T17:32:02.610353+0000","last_active":"2024-07-04T17:32:03.983928+0000","last_peered":"2024-07-04T17:32:03.983928+0000","last_clean":"2024-07-04T17:32:03.983928+0000","last_became_active":"2024-07-04T17:32:02.609997+0000","last_became_peered":"2024-07-04T17:32:02.609997+0000","last_unstale":"2024-07-04T17:32:03.983928+0000","last_undegraded":"2024-07-04T17:32:03.983928+0000","last_fullsized":"2024-07-04T17:32:03.983928+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:45:01.615739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"91'2","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.987617+0000","last_change":"2024-07-04T17:31:41.128172+0000","last_active":"2024-07-04T17:32:03.987617+0000","last_peered":"2024-07-04T17:32:03.987617+0000","last_clean":"2024-07-04T17:32:03.987617+0000","last_became_active":"2024-07-04T17:31:41.127350+0000","last_became_peered":"2024-07-04T17:31:41.127350+0000","last_unstale":"2024-07-04T17:32:03.987617+0000","last_undegraded":"2024-07-04T17:32:03.987617+0000","last_fullsized":"2024-07-04T17:32:03.987617+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:24:57.684170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.985670+0000","last_change":"2024-07-04T17:31:42.144839+0000","last_active":"2024-07-04T17:32:03.985670+0000","last_peered":"2024-07-04T17:32:03.985670+0000","last_clean":"2024-07-04T17:32:03.985670+0000","last_became_active":"2024-07-04T17:31:42.142261+0000","last_became_peered":"2024-07-04T17:31:42.142261+0000","last_unstale":"2024-07-04T17:32:03.985670+0000","last_undegraded":"2024-07-04T17:32:03.985670+0000","last_fullsized":"2024-07-04T17:32:03.985670+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:44:20.205264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.571259+0000","last_change":"2024-07-04T17:32:01.594923+0000","last_active":"2024-07-04T17:32:03.571259+0000","last_peered":"2024-07-04T17:32:03.571259+0000","last_clean":"2024-07-04T17:32:03.571259+0000","last_became_active":"2024-07-04T17:32:01.594612+0000","last_became_peered":"2024-07-04T17:32:01.594612+0000","last_unstale":"2024-07-04T17:32:03.571259+0000","last_undegraded":"2024-07-04T17:32:03.571259+0000","last_fullsized":"2024-07-04T17:32:03.571259+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:05:47.602368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.575782+0000","last_change":"2024-07-04T17:32:02.609748+0000","last_active":"2024-07-04T17:32:03.575782+0000","last_peered":"2024-07-04T17:32:03.575782+0000","last_clean":"2024-07-04T17:32:03.575782+0000","last_became_active":"2024-07-04T17:32:02.603472+0000","last_became_peered":"2024-07-04T17:32:02.603472+0000","last_unstale":"2024-07-04T17:32:03.575782+0000","last_undegraded":"2024-07-04T17:32:03.575782+0000","last_fullsized":"2024-07-04T17:32:03.575782+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:30:47.350386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.992609+0000","last_change":"2024-07-04T17:31:41.111490+0000","last_active":"2024-07-04T17:32:03.992609+0000","last_peered":"2024-07-04T17:32:03.992609+0000","last_clean":"2024-07-04T17:32:03.992609+0000","last_became_active":"2024-07-04T17:31:41.110881+0000","last_became_peered":"2024-07-04T17:31:41.110881+0000","last_unstale":"2024-07-04T17:32:03.992609+0000","last_undegraded":"2024-07-04T17:32:03.992609+0000","last_fullsized":"2024-07-04T17:32:03.992609+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:44:12.481565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.993989+0000","last_change":"2024-07-04T17:31:42.138658+0000","last_active":"2024-07-04T17:32:03.993989+0000","last_peered":"2024-07-04T17:32:03.993989+0000","last_clean":"2024-07-04T17:32:03.993989+0000","last_became_active":"2024-07-04T17:31:42.137836+0000","last_became_peered":"2024-07-04T17:31:42.137836+0000","last_unstale":"2024-07-04T17:32:03.993989+0000","last_undegraded":"2024-07-04T17:32:03.993989+0000","last_fullsized":"2024-07-04T17:32:03.993989+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:32:01.472783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.e","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.995348+0000","last_change":"2024-07-04T17:32:01.580300+0000","last_active":"2024-07-04T17:32:03.995348+0000","last_peered":"2024-07-04T17:32:03.995348+0000","last_clean":"2024-07-04T17:32:03.995348+0000","last_became_active":"2024-07-04T17:32:01.579643+0000","last_became_peered":"2024-07-04T17:32:01.579643+0000","last_unstale":"2024-07-04T17:32:03.995348+0000","last_undegraded":"2024-07-04T17:32:03.995348+0000","last_fullsized":"2024-07-04T17:32:03.995348+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:26:23.623930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.574871+0000","last_change":"2024-07-04T17:32:02.606822+0000","last_active":"2024-07-04T17:32:03.574871+0000","last_peered":"2024-07-04T17:32:03.574871+0000","last_clean":"2024-07-04T17:32:03.574871+0000","last_became_active":"2024-07-04T17:32:02.606506+0000","last_became_peered":"2024-07-04T17:32:02.606506+0000","last_unstale":"2024-07-04T17:32:03.574871+0000","last_undegraded":"2024-07-04T17:32:03.574871+0000","last_fullsized":"2024-07-04T17:32:03.574871+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:20:07.552973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.d","version":"91'1","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.994931+0000","last_change":"2024-07-04T17:31:41.125962+0000","last_active":"2024-07-04T17:32:03.994931+0000","last_peered":"2024-07-04T17:32:03.994931+0000","last_clean":"2024-07-04T17:32:03.994931+0000","last_became_active":"2024-07-04T17:31:41.121577+0000","last_became_peered":"2024-07-04T17:31:41.121577+0000","last_unstale":"2024-07-04T17:32:03.994931+0000","last_undegraded":"2024-07-04T17:32:03.994931+0000","last_fullsized":"2024-07-04T17:32:03.994931+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:02:24.336737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.992335+0000","last_change":"2024-07-04T17:31:42.138910+0000","last_active":"2024-07-04T17:32:03.992335+0000","last_peered":"2024-07-04T17:32:03.992335+0000","last_clean":"2024-07-04T17:32:03.992335+0000","last_became_active":"2024-07-04T17:31:42.137936+0000","last_became_peered":"2024-07-04T17:31:42.137936+0000","last_unstale":"2024-07-04T17:32:03.992335+0000","last_undegraded":"2024-07-04T17:32:03.992335+0000","last_fullsized":"2024-07-04T17:32:03.992335+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:30:17.422306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.f","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.993953+0000","last_change":"2024-07-04T17:32:01.573537+0000","last_active":"2024-07-04T17:32:03.993953+0000","last_peered":"2024-07-04T17:32:03.993953+0000","last_clean":"2024-07-04T17:32:03.993953+0000","last_became_active":"2024-07-04T17:32:01.573178+0000","last_became_peered":"2024-07-04T17:32:01.573178+0000","last_unstale":"2024-07-04T17:32:03.993953+0000","last_undegraded":"2024-07-04T17:32:03.993953+0000","last_fullsized":"2024-07-04T17:32:03.993953+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:57:53.026950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.990328+0000","last_change":"2024-07-04T17:32:02.582966+0000","last_active":"2024-07-04T17:32:03.990328+0000","last_peered":"2024-07-04T17:32:03.990328+0000","last_clean":"2024-07-04T17:32:03.990328+0000","last_became_active":"2024-07-04T17:32:02.582488+0000","last_became_peered":"2024-07-04T17:32:02.582488+0000","last_unstale":"2024-07-04T17:32:03.990328+0000","last_undegraded":"2024-07-04T17:32:03.990328+0000","last_fullsized":"2024-07-04T17:32:03.990328+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:13:26.272163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"91'2","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.575949+0000","last_change":"2024-07-04T17:31:41.127224+0000","last_active":"2024-07-04T17:32:03.575949+0000","last_peered":"2024-07-04T17:32:03.575949+0000","last_clean":"2024-07-04T17:32:03.575949+0000","last_became_active":"2024-07-04T17:31:41.126878+0000","last_became_peered":"2024-07-04T17:31:41.126878+0000","last_unstale":"2024-07-04T17:32:03.575949+0000","last_undegraded":"2024-07-04T17:32:03.575949+0000","last_fullsized":"2024-07-04T17:32:03.575949+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:17:12.761795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.995715+0000","last_change":"2024-07-04T17:31:42.127586+0000","last_active":"2024-07-04T17:32:03.995715+0000","last_peered":"2024-07-04T17:32:03.995715+0000","last_clean":"2024-07-04T17:32:03.995715+0000","last_became_active":"2024-07-04T17:31:42.127263+0000","last_became_peered":"2024-07-04T17:31:42.127263+0000","last_unstale":"2024-07-04T17:32:03.995715+0000","last_undegraded":"2024-07-04T17:32:03.995715+0000","last_fullsized":"2024-07-04T17:32:03.995715+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:54:41.368680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.574182+0000","last_change":"2024-07-04T17:32:01.595196+0000","last_active":"2024-07-04T17:32:03.574182+0000","last_peered":"2024-07-04T17:32:03.574182+0000","last_clean":"2024-07-04T17:32:03.574182+0000","last_became_active":"2024-07-04T17:32:01.594870+0000","last_became_peered":"2024-07-04T17:32:01.594870+0000","last_unstale":"2024-07-04T17:32:03.574182+0000","last_undegraded":"2024-07-04T17:32:03.574182+0000","last_fullsized":"2024-07-04T17:32:03.574182+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:23:34.925305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.991947+0000","last_change":"2024-07-04T17:32:02.576103+0000","last_active":"2024-07-04T17:32:03.991947+0000","last_peered":"2024-07-04T17:32:03.991947+0000","last_clean":"2024-07-04T17:32:03.991947+0000","last_became_active":"2024-07-04T17:32:02.575604+0000","last_became_peered":"2024-07-04T17:32:02.575604+0000","last_unstale":"2024-07-04T17:32:03.991947+0000","last_undegraded":"2024-07-04T17:32:03.991947+0000","last_fullsized":"2024-07-04T17:32:03.991947+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:36:57.937525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.f","version":"91'1","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.997132+0000","last_change":"2024-07-04T17:31:41.110852+0000","last_active":"2024-07-04T17:32:03.997132+0000","last_peered":"2024-07-04T17:32:03.997132+0000","last_clean":"2024-07-04T17:32:03.997132+0000","last_became_active":"2024-07-04T17:31:41.109092+0000","last_became_peered":"2024-07-04T17:31:41.109092+0000","last_unstale":"2024-07-04T17:32:03.997132+0000","last_undegraded":"2024-07-04T17:32:03.997132+0000","last_fullsized":"2024-07-04T17:32:03.997132+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:47:10.505606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.575578+0000","last_change":"2024-07-04T17:31:42.117660+0000","last_active":"2024-07-04T17:32:03.575578+0000","last_peered":"2024-07-04T17:32:03.575578+0000","last_clean":"2024-07-04T17:32:03.575578+0000","last_became_active":"2024-07-04T17:31:42.115485+0000","last_became_peered":"2024-07-04T17:31:42.115485+0000","last_unstale":"2024-07-04T17:32:03.575578+0000","last_undegraded":"2024-07-04T17:32:03.575578+0000","last_fullsized":"2024-07-04T17:32:03.575578+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:10:45.367563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.d","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.993656+0000","last_change":"2024-07-04T17:32:01.570202+0000","last_active":"2024-07-04T17:32:03.993656+0000","last_peered":"2024-07-04T17:32:03.993656+0000","last_clean":"2024-07-04T17:32:03.993656+0000","last_became_active":"2024-07-04T17:32:01.569874+0000","last_became_peered":"2024-07-04T17:32:01.569874+0000","last_unstale":"2024-07-04T17:32:03.993656+0000","last_undegraded":"2024-07-04T17:32:03.993656+0000","last_fullsized":"2024-07-04T17:32:03.993656+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:50:25.143757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.573842+0000","last_change":"2024-07-04T17:32:02.610260+0000","last_active":"2024-07-04T17:32:03.573842+0000","last_peered":"2024-07-04T17:32:03.573842+0000","last_clean":"2024-07-04T17:32:03.573842+0000","last_became_active":"2024-07-04T17:32:02.609527+0000","last_became_peered":"2024-07-04T17:32:02.609527+0000","last_unstale":"2024-07-04T17:32:03.573842+0000","last_undegraded":"2024-07-04T17:32:03.573842+0000","last_fullsized":"2024-07-04T17:32:03.573842+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:12:46.495414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.10","version":"91'8","reported_seq":30,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.572232+0000","last_change":"2024-07-04T17:31:41.123889+0000","last_active":"2024-07-04T17:32:03.572232+0000","last_peered":"2024-07-04T17:32:03.572232+0000","last_clean":"2024-07-04T17:32:03.572232+0000","last_became_active":"2024-07-04T17:31:41.123555+0000","last_became_peered":"2024-07-04T17:31:41.123555+0000","last_unstale":"2024-07-04T17:32:03.572232+0000","last_undegraded":"2024-07-04T17:32:03.572232+0000","last_fullsized":"2024-07-04T17:32:03.572232+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:07:24.656814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.984838+0000","last_change":"2024-07-04T17:31:42.134513+0000","last_active":"2024-07-04T17:32:03.984838+0000","last_peered":"2024-07-04T17:32:03.984838+0000","last_clean":"2024-07-04T17:32:03.984838+0000","last_became_active":"2024-07-04T17:31:42.134166+0000","last_became_peered":"2024-07-04T17:31:42.134166+0000","last_unstale":"2024-07-04T17:32:03.984838+0000","last_undegraded":"2024-07-04T17:32:03.984838+0000","last_fullsized":"2024-07-04T17:32:03.984838+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:17:13.766129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.572180+0000","last_change":"2024-07-04T17:32:01.583168+0000","last_active":"2024-07-04T17:32:03.572180+0000","last_peered":"2024-07-04T17:32:03.572180+0000","last_clean":"2024-07-04T17:32:03.572180+0000","last_became_active":"2024-07-04T17:32:01.582859+0000","last_became_peered":"2024-07-04T17:32:01.582859+0000","last_unstale":"2024-07-04T17:32:03.572180+0000","last_undegraded":"2024-07-04T17:32:03.572180+0000","last_fullsized":"2024-07-04T17:32:03.572180+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:47:23.988157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.991460+0000","last_change":"2024-07-04T17:32:02.624956+0000","last_active":"2024-07-04T17:32:03.991460+0000","last_peered":"2024-07-04T17:32:03.991460+0000","last_clean":"2024-07-04T17:32:03.991460+0000","last_became_active":"2024-07-04T17:32:02.624404+0000","last_became_peered":"2024-07-04T17:32:02.624404+0000","last_unstale":"2024-07-04T17:32:03.991460+0000","last_undegraded":"2024-07-04T17:32:03.991460+0000","last_fullsized":"2024-07-04T17:32:03.991460+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:51:43.902906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.987165+0000","last_change":"2024-07-04T17:31:41.126486+0000","last_active":"2024-07-04T17:32:03.987165+0000","last_peered":"2024-07-04T17:32:03.987165+0000","last_clean":"2024-07-04T17:32:03.987165+0000","last_became_active":"2024-07-04T17:31:41.126142+0000","last_became_peered":"2024-07-04T17:31:41.126142+0000","last_unstale":"2024-07-04T17:32:03.987165+0000","last_undegraded":"2024-07-04T17:32:03.987165+0000","last_fullsized":"2024-07-04T17:32:03.987165+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:25:18.918225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.991833+0000","last_change":"2024-07-04T17:31:42.111369+0000","last_active":"2024-07-04T17:32:03.991833+0000","last_peered":"2024-07-04T17:32:03.991833+0000","last_clean":"2024-07-04T17:32:03.991833+0000","last_became_active":"2024-07-04T17:31:42.110632+0000","last_became_peered":"2024-07-04T17:31:42.110632+0000","last_unstale":"2024-07-04T17:32:03.991833+0000","last_undegraded":"2024-07-04T17:32:03.991833+0000","last_fullsized":"2024-07-04T17:32:03.991833+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:31:20.968644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"95'8","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.986817+0000","last_change":"2024-07-04T17:32:01.585065+0000","last_active":"2024-07-04T17:32:03.986817+0000","last_peered":"2024-07-04T17:32:03.986817+0000","last_clean":"2024-07-04T17:32:03.986817+0000","last_became_active":"2024-07-04T17:32:01.584410+0000","last_became_peered":"2024-07-04T17:32:01.584410+0000","last_unstale":"2024-07-04T17:32:03.986817+0000","last_undegraded":"2024-07-04T17:32:03.986817+0000","last_fullsized":"2024-07-04T17:32:03.986817+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:38:45.000810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.983677+0000","last_change":"2024-07-04T17:32:02.609945+0000","last_active":"2024-07-04T17:32:03.983677+0000","last_peered":"2024-07-04T17:32:03.983677+0000","last_clean":"2024-07-04T17:32:03.983677+0000","last_became_active":"2024-07-04T17:32:02.609574+0000","last_became_peered":"2024-07-04T17:32:02.609574+0000","last_unstale":"2024-07-04T17:32:03.983677+0000","last_undegraded":"2024-07-04T17:32:03.983677+0000","last_fullsized":"2024-07-04T17:32:03.983677+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:37:21.425873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"91'2","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.987905+0000","last_change":"2024-07-04T17:31:41.128711+0000","last_active":"2024-07-04T17:32:03.987905+0000","last_peered":"2024-07-04T17:32:03.987905+0000","last_clean":"2024-07-04T17:32:03.987905+0000","last_became_active":"2024-07-04T17:31:41.127816+0000","last_became_peered":"2024-07-04T17:31:41.127816+0000","last_unstale":"2024-07-04T17:32:03.987905+0000","last_undegraded":"2024-07-04T17:32:03.987905+0000","last_fullsized":"2024-07-04T17:32:03.987905+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:37:51.190424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.572804+0000","last_change":"2024-07-04T17:31:42.116278+0000","last_active":"2024-07-04T17:32:03.572804+0000","last_peered":"2024-07-04T17:32:03.572804+0000","last_clean":"2024-07-04T17:32:03.572804+0000","last_became_active":"2024-07-04T17:31:42.115861+0000","last_became_peered":"2024-07-04T17:31:42.115861+0000","last_unstale":"2024-07-04T17:32:03.572804+0000","last_undegraded":"2024-07-04T17:32:03.572804+0000","last_fullsized":"2024-07-04T17:32:03.572804+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:43:17.950207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.984363+0000","last_change":"2024-07-04T17:32:01.582267+0000","last_active":"2024-07-04T17:32:03.984363+0000","last_peered":"2024-07-04T17:32:03.984363+0000","last_clean":"2024-07-04T17:32:03.984363+0000","last_became_active":"2024-07-04T17:32:01.581942+0000","last_became_peered":"2024-07-04T17:32:01.581942+0000","last_unstale":"2024-07-04T17:32:03.984363+0000","last_undegraded":"2024-07-04T17:32:03.984363+0000","last_fullsized":"2024-07-04T17:32:03.984363+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:18:53.810731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.575306+0000","last_change":"2024-07-04T17:32:02.600700+0000","last_active":"2024-07-04T17:32:03.575306+0000","last_peered":"2024-07-04T17:32:03.575306+0000","last_clean":"2024-07-04T17:32:03.575306+0000","last_became_active":"2024-07-04T17:32:02.600306+0000","last_became_peered":"2024-07-04T17:32:02.600306+0000","last_unstale":"2024-07-04T17:32:03.575306+0000","last_undegraded":"2024-07-04T17:32:03.575306+0000","last_fullsized":"2024-07-04T17:32:03.575306+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:04:10.258386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"92'8","reported_seq":30,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.992604+0000","last_change":"2024-07-04T17:31:41.123522+0000","last_active":"2024-07-04T17:32:03.992604+0000","last_peered":"2024-07-04T17:32:03.992604+0000","last_clean":"2024-07-04T17:32:03.992604+0000","last_became_active":"2024-07-04T17:31:41.121523+0000","last_became_peered":"2024-07-04T17:31:41.121523+0000","last_unstale":"2024-07-04T17:32:03.992604+0000","last_undegraded":"2024-07-04T17:32:03.992604+0000","last_fullsized":"2024-07-04T17:32:03.992604+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:36:59.958154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3309,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.575023+0000","last_change":"2024-07-04T17:31:42.134181+0000","last_active":"2024-07-04T17:32:03.575023+0000","last_peered":"2024-07-04T17:32:03.575023+0000","last_clean":"2024-07-04T17:32:03.575023+0000","last_became_active":"2024-07-04T17:31:42.133671+0000","last_became_peered":"2024-07-04T17:31:42.133671+0000","last_unstale":"2024-07-04T17:32:03.575023+0000","last_undegraded":"2024-07-04T17:32:03.575023+0000","last_fullsized":"2024-07-04T17:32:03.575023+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:13:23.161757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.995661+0000","last_change":"2024-07-04T17:32:01.579982+0000","last_active":"2024-07-04T17:32:03.995661+0000","last_peered":"2024-07-04T17:32:03.995661+0000","last_clean":"2024-07-04T17:32:03.995661+0000","last_became_active":"2024-07-04T17:32:01.579645+0000","last_became_peered":"2024-07-04T17:32:01.579645+0000","last_unstale":"2024-07-04T17:32:03.995661+0000","last_undegraded":"2024-07-04T17:32:03.995661+0000","last_fullsized":"2024-07-04T17:32:03.995661+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:00:28.289678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.574613+0000","last_change":"2024-07-04T17:32:02.606795+0000","last_active":"2024-07-04T17:32:03.574613+0000","last_peered":"2024-07-04T17:32:03.574613+0000","last_clean":"2024-07-04T17:32:03.574613+0000","last_became_active":"2024-07-04T17:32:02.606068+0000","last_became_peered":"2024-07-04T17:32:02.606068+0000","last_unstale":"2024-07-04T17:32:03.574613+0000","last_undegraded":"2024-07-04T17:32:03.574613+0000","last_fullsized":"2024-07-04T17:32:03.574613+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:51:05.811627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"91'3","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.994369+0000","last_change":"2024-07-04T17:31:41.124114+0000","last_active":"2024-07-04T17:32:03.994369+0000","last_peered":"2024-07-04T17:32:03.994369+0000","last_clean":"2024-07-04T17:32:03.994369+0000","last_became_active":"2024-07-04T17:31:41.121166+0000","last_became_peered":"2024-07-04T17:31:41.121166+0000","last_unstale":"2024-07-04T17:32:03.994369+0000","last_undegraded":"2024-07-04T17:32:03.994369+0000","last_fullsized":"2024-07-04T17:32:03.994369+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:44:40.148021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.989062+0000","last_change":"2024-07-04T17:31:42.136357+0000","last_active":"2024-07-04T17:32:03.989062+0000","last_peered":"2024-07-04T17:32:03.989062+0000","last_clean":"2024-07-04T17:32:03.989062+0000","last_became_active":"2024-07-04T17:31:42.136025+0000","last_became_peered":"2024-07-04T17:31:42.136025+0000","last_unstale":"2024-07-04T17:32:03.989062+0000","last_undegraded":"2024-07-04T17:32:03.989062+0000","last_fullsized":"2024-07-04T17:32:03.989062+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:55:46.913903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.993774+0000","last_change":"2024-07-04T17:32:01.585343+0000","last_active":"2024-07-04T17:32:03.993774+0000","last_peered":"2024-07-04T17:32:03.993774+0000","last_clean":"2024-07-04T17:32:03.993774+0000","last_became_active":"2024-07-04T17:32:01.584969+0000","last_became_peered":"2024-07-04T17:32:01.584969+0000","last_unstale":"2024-07-04T17:32:03.993774+0000","last_undegraded":"2024-07-04T17:32:03.993774+0000","last_fullsized":"2024-07-04T17:32:03.993774+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:31:54.728917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.988726+0000","last_change":"2024-07-04T17:32:02.615398+0000","last_active":"2024-07-04T17:32:03.988726+0000","last_peered":"2024-07-04T17:32:03.988726+0000","last_clean":"2024-07-04T17:32:03.988726+0000","last_became_active":"2024-07-04T17:32:02.608548+0000","last_became_peered":"2024-07-04T17:32:02.608548+0000","last_unstale":"2024-07-04T17:32:03.988726+0000","last_undegraded":"2024-07-04T17:32:03.988726+0000","last_fullsized":"2024-07-04T17:32:03.988726+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:01:55.601573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.575855+0000","last_change":"2024-07-04T17:31:41.132406+0000","last_active":"2024-07-04T17:32:03.575855+0000","last_peered":"2024-07-04T17:32:03.575855+0000","last_clean":"2024-07-04T17:32:03.575855+0000","last_became_active":"2024-07-04T17:31:41.130594+0000","last_became_peered":"2024-07-04T17:31:41.130594+0000","last_unstale":"2024-07-04T17:32:03.575855+0000","last_undegraded":"2024-07-04T17:32:03.575855+0000","last_fullsized":"2024-07-04T17:32:03.575855+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:01:17.482235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.576358+0000","last_change":"2024-07-04T17:31:42.128979+0000","last_active":"2024-07-04T17:32:03.576358+0000","last_peered":"2024-07-04T17:32:03.576358+0000","last_clean":"2024-07-04T17:32:03.576358+0000","last_became_active":"2024-07-04T17:31:42.126866+0000","last_became_peered":"2024-07-04T17:31:42.126866+0000","last_unstale":"2024-07-04T17:32:03.576358+0000","last_undegraded":"2024-07-04T17:32:03.576358+0000","last_fullsized":"2024-07-04T17:32:03.576358+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:06:17.477749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.14","version":"95'6","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.988341+0000","last_change":"2024-07-04T17:32:01.596913+0000","last_active":"2024-07-04T17:32:03.988341+0000","last_peered":"2024-07-04T17:32:03.988341+0000","last_clean":"2024-07-04T17:32:03.988341+0000","last_became_active":"2024-07-04T17:32:01.596502+0000","last_became_peered":"2024-07-04T17:32:01.596502+0000","last_unstale":"2024-07-04T17:32:03.988341+0000","last_undegraded":"2024-07-04T17:32:03.988341+0000","last_fullsized":"2024-07-04T17:32:03.988341+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:57:24.893524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.991746+0000","last_change":"2024-07-04T17:32:02.612731+0000","last_active":"2024-07-04T17:32:03.991746+0000","last_peered":"2024-07-04T17:32:03.991746+0000","last_clean":"2024-07-04T17:32:03.991746+0000","last_became_active":"2024-07-04T17:32:02.608550+0000","last_became_peered":"2024-07-04T17:32:02.608550+0000","last_unstale":"2024-07-04T17:32:03.991746+0000","last_undegraded":"2024-07-04T17:32:03.991746+0000","last_fullsized":"2024-07-04T17:32:03.991746+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:38:37.557596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.576672+0000","last_change":"2024-07-04T17:31:41.131784+0000","last_active":"2024-07-04T17:32:03.576672+0000","last_peered":"2024-07-04T17:32:03.576672+0000","last_clean":"2024-07-04T17:32:03.576672+0000","last_became_active":"2024-07-04T17:31:41.130750+0000","last_became_peered":"2024-07-04T17:31:41.130750+0000","last_unstale":"2024-07-04T17:32:03.576672+0000","last_undegraded":"2024-07-04T17:32:03.576672+0000","last_fullsized":"2024-07-04T17:32:03.576672+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:55:29.200978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.575554+0000","last_change":"2024-07-04T17:31:42.133055+0000","last_active":"2024-07-04T17:32:03.575554+0000","last_peered":"2024-07-04T17:32:03.575554+0000","last_clean":"2024-07-04T17:32:03.575554+0000","last_became_active":"2024-07-04T17:31:42.131725+0000","last_became_peered":"2024-07-04T17:31:42.131725+0000","last_unstale":"2024-07-04T17:32:03.575554+0000","last_undegraded":"2024-07-04T17:32:03.575554+0000","last_fullsized":"2024-07-04T17:32:03.575554+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:21:59.669492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.15","version":"95'3","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.574494+0000","last_change":"2024-07-04T17:32:01.592999+0000","last_active":"2024-07-04T17:32:03.574494+0000","last_peered":"2024-07-04T17:32:03.574494+0000","last_clean":"2024-07-04T17:32:03.574494+0000","last_became_active":"2024-07-04T17:32:01.591679+0000","last_became_peered":"2024-07-04T17:32:01.591679+0000","last_unstale":"2024-07-04T17:32:03.574494+0000","last_undegraded":"2024-07-04T17:32:03.574494+0000","last_fullsized":"2024-07-04T17:32:03.574494+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:08:42.609506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.990452+0000","last_change":"2024-07-04T17:32:02.598164+0000","last_active":"2024-07-04T17:32:03.990452+0000","last_peered":"2024-07-04T17:32:03.990452+0000","last_clean":"2024-07-04T17:32:03.990452+0000","last_became_active":"2024-07-04T17:32:02.597819+0000","last_became_peered":"2024-07-04T17:32:02.597819+0000","last_unstale":"2024-07-04T17:32:03.990452+0000","last_undegraded":"2024-07-04T17:32:03.990452+0000","last_fullsized":"2024-07-04T17:32:03.990452+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:18:21.936559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.987936+0000","last_change":"2024-07-04T17:31:41.114447+0000","last_active":"2024-07-04T17:32:03.987936+0000","last_peered":"2024-07-04T17:32:03.987936+0000","last_clean":"2024-07-04T17:32:03.987936+0000","last_became_active":"2024-07-04T17:31:41.114127+0000","last_became_peered":"2024-07-04T17:31:41.114127+0000","last_unstale":"2024-07-04T17:32:03.987936+0000","last_undegraded":"2024-07-04T17:32:03.987936+0000","last_fullsized":"2024-07-04T17:32:03.987936+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:21:00.022601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.984620+0000","last_change":"2024-07-04T17:31:42.138950+0000","last_active":"2024-07-04T17:32:03.984620+0000","last_peered":"2024-07-04T17:32:03.984620+0000","last_clean":"2024-07-04T17:32:03.984620+0000","last_became_active":"2024-07-04T17:31:42.138614+0000","last_became_peered":"2024-07-04T17:31:42.138614+0000","last_unstale":"2024-07-04T17:32:03.984620+0000","last_undegraded":"2024-07-04T17:32:03.984620+0000","last_fullsized":"2024-07-04T17:32:03.984620+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:00:43.285683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1a","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.991588+0000","last_change":"2024-07-04T17:32:01.575001+0000","last_active":"2024-07-04T17:32:03.991588+0000","last_peered":"2024-07-04T17:32:03.991588+0000","last_clean":"2024-07-04T17:32:03.991588+0000","last_became_active":"2024-07-04T17:32:01.574551+0000","last_became_peered":"2024-07-04T17:32:01.574551+0000","last_unstale":"2024-07-04T17:32:03.991588+0000","last_undegraded":"2024-07-04T17:32:03.991588+0000","last_fullsized":"2024-07-04T17:32:03.991588+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:35:10.472781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.571667+0000","last_change":"2024-07-04T17:32:02.594405+0000","last_active":"2024-07-04T17:32:03.571667+0000","last_peered":"2024-07-04T17:32:03.571667+0000","last_clean":"2024-07-04T17:32:03.571667+0000","last_became_active":"2024-07-04T17:32:02.594043+0000","last_became_peered":"2024-07-04T17:32:02.594043+0000","last_unstale":"2024-07-04T17:32:03.571667+0000","last_undegraded":"2024-07-04T17:32:03.571667+0000","last_fullsized":"2024-07-04T17:32:03.571667+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:02:41.604232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.985136+0000","last_change":"2024-07-04T17:31:41.122775+0000","last_active":"2024-07-04T17:32:03.985136+0000","last_peered":"2024-07-04T17:32:03.985136+0000","last_clean":"2024-07-04T17:32:03.985136+0000","last_became_active":"2024-07-04T17:31:41.121256+0000","last_became_peered":"2024-07-04T17:31:41.121256+0000","last_unstale":"2024-07-04T17:32:03.985136+0000","last_undegraded":"2024-07-04T17:32:03.985136+0000","last_fullsized":"2024-07-04T17:32:03.985136+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:41:24.386164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.987226+0000","last_change":"2024-07-04T17:31:42.124019+0000","last_active":"2024-07-04T17:32:03.987226+0000","last_peered":"2024-07-04T17:32:03.987226+0000","last_clean":"2024-07-04T17:32:03.987226+0000","last_became_active":"2024-07-04T17:31:42.123684+0000","last_became_peered":"2024-07-04T17:31:42.123684+0000","last_unstale":"2024-07-04T17:32:03.987226+0000","last_undegraded":"2024-07-04T17:32:03.987226+0000","last_fullsized":"2024-07-04T17:32:03.987226+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:12:26.032030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1b","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.985519+0000","last_change":"2024-07-04T17:32:01.593554+0000","last_active":"2024-07-04T17:32:03.985519+0000","last_peered":"2024-07-04T17:32:03.985519+0000","last_clean":"2024-07-04T17:32:03.985519+0000","last_became_active":"2024-07-04T17:32:01.592637+0000","last_became_peered":"2024-07-04T17:32:01.592637+0000","last_unstale":"2024-07-04T17:32:03.985519+0000","last_undegraded":"2024-07-04T17:32:03.985519+0000","last_fullsized":"2024-07-04T17:32:03.985519+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:02:33.297199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.984078+0000","last_change":"2024-07-04T17:32:02.621078+0000","last_active":"2024-07-04T17:32:03.984078+0000","last_peered":"2024-07-04T17:32:03.984078+0000","last_clean":"2024-07-04T17:32:03.984078+0000","last_became_active":"2024-07-04T17:32:02.619930+0000","last_became_peered":"2024-07-04T17:32:02.619930+0000","last_unstale":"2024-07-04T17:32:03.984078+0000","last_undegraded":"2024-07-04T17:32:03.984078+0000","last_fullsized":"2024-07-04T17:32:03.984078+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:42:41.324766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1a","version":"91'1","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.572936+0000","last_change":"2024-07-04T17:31:41.118623+0000","last_active":"2024-07-04T17:32:03.572936+0000","last_peered":"2024-07-04T17:32:03.572936+0000","last_clean":"2024-07-04T17:32:03.572936+0000","last_became_active":"2024-07-04T17:31:41.116368+0000","last_became_peered":"2024-07-04T17:31:41.116368+0000","last_unstale":"2024-07-04T17:32:03.572936+0000","last_undegraded":"2024-07-04T17:32:03.572936+0000","last_fullsized":"2024-07-04T17:32:03.572936+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:34:37.181739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.991480+0000","last_change":"2024-07-04T17:31:42.111322+0000","last_active":"2024-07-04T17:32:03.991480+0000","last_peered":"2024-07-04T17:32:03.991480+0000","last_clean":"2024-07-04T17:32:03.991480+0000","last_became_active":"2024-07-04T17:31:42.110186+0000","last_became_peered":"2024-07-04T17:31:42.110186+0000","last_unstale":"2024-07-04T17:32:03.991480+0000","last_undegraded":"2024-07-04T17:32:03.991480+0000","last_fullsized":"2024-07-04T17:32:03.991480+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:43:36.161285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.984314+0000","last_change":"2024-07-04T17:32:01.583499+0000","last_active":"2024-07-04T17:32:03.984314+0000","last_peered":"2024-07-04T17:32:03.984314+0000","last_clean":"2024-07-04T17:32:03.984314+0000","last_became_active":"2024-07-04T17:32:01.583102+0000","last_became_peered":"2024-07-04T17:32:01.583102+0000","last_unstale":"2024-07-04T17:32:03.984314+0000","last_undegraded":"2024-07-04T17:32:03.984314+0000","last_fullsized":"2024-07-04T17:32:03.984314+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:51:21.342160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.986703+0000","last_change":"2024-07-04T17:32:02.609423+0000","last_active":"2024-07-04T17:32:03.986703+0000","last_peered":"2024-07-04T17:32:03.986703+0000","last_clean":"2024-07-04T17:32:03.986703+0000","last_became_active":"2024-07-04T17:32:02.608428+0000","last_became_peered":"2024-07-04T17:32:02.608428+0000","last_unstale":"2024-07-04T17:32:03.986703+0000","last_undegraded":"2024-07-04T17:32:03.986703+0000","last_fullsized":"2024-07-04T17:32:03.986703+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:32:11.435949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"91'1","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.985815+0000","last_change":"2024-07-04T17:31:41.129634+0000","last_active":"2024-07-04T17:32:03.985815+0000","last_peered":"2024-07-04T17:32:03.985815+0000","last_clean":"2024-07-04T17:32:03.985815+0000","last_became_active":"2024-07-04T17:31:41.129326+0000","last_became_peered":"2024-07-04T17:31:41.129326+0000","last_unstale":"2024-07-04T17:32:03.985815+0000","last_undegraded":"2024-07-04T17:32:03.985815+0000","last_fullsized":"2024-07-04T17:32:03.985815+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:37:28.340891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.993076+0000","last_change":"2024-07-04T17:31:42.132447+0000","last_active":"2024-07-04T17:32:03.993076+0000","last_peered":"2024-07-04T17:32:03.993076+0000","last_clean":"2024-07-04T17:32:03.993076+0000","last_became_active":"2024-07-04T17:31:42.132121+0000","last_became_peered":"2024-07-04T17:31:42.132121+0000","last_unstale":"2024-07-04T17:32:03.993076+0000","last_undegraded":"2024-07-04T17:32:03.993076+0000","last_fullsized":"2024-07-04T17:32:03.993076+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:55:55.603144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.989225+0000","last_change":"2024-07-04T17:32:01.580926+0000","last_active":"2024-07-04T17:32:03.989225+0000","last_peered":"2024-07-04T17:32:03.989225+0000","last_clean":"2024-07-04T17:32:03.989225+0000","last_became_active":"2024-07-04T17:32:01.580542+0000","last_became_peered":"2024-07-04T17:32:01.580542+0000","last_unstale":"2024-07-04T17:32:03.989225+0000","last_undegraded":"2024-07-04T17:32:03.989225+0000","last_fullsized":"2024-07-04T17:32:03.989225+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:14:46.722382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.575391+0000","last_change":"2024-07-04T17:32:02.610001+0000","last_active":"2024-07-04T17:32:03.575391+0000","last_peered":"2024-07-04T17:32:03.575391+0000","last_clean":"2024-07-04T17:32:03.575391+0000","last_became_active":"2024-07-04T17:32:02.603860+0000","last_became_peered":"2024-07-04T17:32:02.603860+0000","last_unstale":"2024-07-04T17:32:03.575391+0000","last_undegraded":"2024-07-04T17:32:03.575391+0000","last_fullsized":"2024-07-04T17:32:03.575391+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:21:50.088729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.1e","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.996198+0000","last_change":"2024-07-04T17:32:01.597250+0000","last_active":"2024-07-04T17:32:03.996198+0000","last_peered":"2024-07-04T17:32:03.996198+0000","last_clean":"2024-07-04T17:32:03.996198+0000","last_became_active":"2024-07-04T17:32:01.593415+0000","last_became_peered":"2024-07-04T17:32:01.593415+0000","last_unstale":"2024-07-04T17:32:03.996198+0000","last_undegraded":"2024-07-04T17:32:03.996198+0000","last_fullsized":"2024-07-04T17:32:03.996198+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:37:25.431586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"91'2","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.574715+0000","last_change":"2024-07-04T17:31:41.116679+0000","last_active":"2024-07-04T17:32:03.574715+0000","last_peered":"2024-07-04T17:32:03.574715+0000","last_clean":"2024-07-04T17:32:03.574715+0000","last_became_active":"2024-07-04T17:31:41.116240+0000","last_became_peered":"2024-07-04T17:31:41.116240+0000","last_unstale":"2024-07-04T17:32:03.574715+0000","last_undegraded":"2024-07-04T17:32:03.574715+0000","last_fullsized":"2024-07-04T17:32:03.574715+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:40:21.938736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.994977+0000","last_change":"2024-07-04T17:31:42.138160+0000","last_active":"2024-07-04T17:32:03.994977+0000","last_peered":"2024-07-04T17:32:03.994977+0000","last_clean":"2024-07-04T17:32:03.994977+0000","last_became_active":"2024-07-04T17:31:42.136593+0000","last_became_peered":"2024-07-04T17:31:42.136593+0000","last_unstale":"2024-07-04T17:32:03.994977+0000","last_undegraded":"2024-07-04T17:32:03.994977+0000","last_fullsized":"2024-07-04T17:32:03.994977+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:28:28.225877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.995040+0000","last_change":"2024-07-04T17:32:02.604178+0000","last_active":"2024-07-04T17:32:03.995040+0000","last_peered":"2024-07-04T17:32:03.995040+0000","last_clean":"2024-07-04T17:32:03.995040+0000","last_became_active":"2024-07-04T17:32:02.603832+0000","last_became_peered":"2024-07-04T17:32:02.603832+0000","last_unstale":"2024-07-04T17:32:03.995040+0000","last_undegraded":"2024-07-04T17:32:03.995040+0000","last_fullsized":"2024-07-04T17:32:03.995040+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:41:06.662377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1f","version":"95'4","reported_seq":18,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.575987+0000","last_change":"2024-07-04T17:32:01.598175+0000","last_active":"2024-07-04T17:32:03.575987+0000","last_peered":"2024-07-04T17:32:03.575987+0000","last_clean":"2024-07-04T17:32:03.575987+0000","last_became_active":"2024-07-04T17:32:01.597834+0000","last_became_peered":"2024-07-04T17:32:01.597834+0000","last_unstale":"2024-07-04T17:32:03.575987+0000","last_undegraded":"2024-07-04T17:32:03.575987+0000","last_fullsized":"2024-07-04T17:32:03.575987+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:29:56.239811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1d","version":"91'2","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.575215+0000","last_change":"2024-07-04T17:31:41.129438+0000","last_active":"2024-07-04T17:32:03.575215+0000","last_peered":"2024-07-04T17:32:03.575215+0000","last_clean":"2024-07-04T17:32:03.575215+0000","last_became_active":"2024-07-04T17:31:41.127900+0000","last_became_peered":"2024-07-04T17:31:41.127900+0000","last_unstale":"2024-07-04T17:32:03.575215+0000","last_undegraded":"2024-07-04T17:32:03.575215+0000","last_fullsized":"2024-07-04T17:32:03.575215+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:12:36.772353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.988889+0000","last_change":"2024-07-04T17:31:42.126808+0000","last_active":"2024-07-04T17:32:03.988889+0000","last_peered":"2024-07-04T17:32:03.988889+0000","last_clean":"2024-07-04T17:32:03.988889+0000","last_became_active":"2024-07-04T17:31:42.126497+0000","last_became_peered":"2024-07-04T17:31:42.126497+0000","last_unstale":"2024-07-04T17:32:03.988889+0000","last_undegraded":"2024-07-04T17:32:03.988889+0000","last_fullsized":"2024-07-04T17:32:03.988889+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:04:06.470763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.993228+0000","last_change":"2024-07-04T17:32:02.603761+0000","last_active":"2024-07-04T17:32:03.993228+0000","last_peered":"2024-07-04T17:32:03.993228+0000","last_clean":"2024-07-04T17:32:03.993228+0000","last_became_active":"2024-07-04T17:32:02.602352+0000","last_became_peered":"2024-07-04T17:32:02.602352+0000","last_unstale":"2024-07-04T17:32:03.993228+0000","last_undegraded":"2024-07-04T17:32:03.993228+0000","last_fullsized":"2024-07-04T17:32:03.993228+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:09:20.217518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"91'1","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.575733+0000","last_change":"2024-07-04T17:31:41.131289+0000","last_active":"2024-07-04T17:32:03.575733+0000","last_peered":"2024-07-04T17:32:03.575733+0000","last_clean":"2024-07-04T17:32:03.575733+0000","last_became_active":"2024-07-04T17:31:41.130956+0000","last_became_peered":"2024-07-04T17:31:41.130956+0000","last_unstale":"2024-07-04T17:32:03.575733+0000","last_undegraded":"2024-07-04T17:32:03.575733+0000","last_fullsized":"2024-07-04T17:32:03.575733+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:11:01.631006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.995864+0000","last_change":"2024-07-04T17:31:42.143241+0000","last_active":"2024-07-04T17:32:03.995864+0000","last_peered":"2024-07-04T17:32:03.995864+0000","last_clean":"2024-07-04T17:32:03.995864+0000","last_became_active":"2024-07-04T17:31:42.142919+0000","last_became_peered":"2024-07-04T17:31:42.142919+0000","last_unstale":"2024-07-04T17:32:03.995864+0000","last_undegraded":"2024-07-04T17:32:03.995864+0000","last_fullsized":"2024-07-04T17:32:03.995864+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:42:50.042411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.988503+0000","last_change":"2024-07-04T17:32:01.587782+0000","last_active":"2024-07-04T17:32:03.988503+0000","last_peered":"2024-07-04T17:32:03.988503+0000","last_clean":"2024-07-04T17:32:03.988503+0000","last_became_active":"2024-07-04T17:32:01.587353+0000","last_became_peered":"2024-07-04T17:32:01.587353+0000","last_unstale":"2024-07-04T17:32:03.988503+0000","last_undegraded":"2024-07-04T17:32:03.988503+0000","last_fullsized":"2024-07-04T17:32:03.988503+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:06:36.928702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.574009+0000","last_change":"2024-07-04T17:32:02.593632+0000","last_active":"2024-07-04T17:32:03.574009+0000","last_peered":"2024-07-04T17:32:03.574009+0000","last_clean":"2024-07-04T17:32:03.574009+0000","last_became_active":"2024-07-04T17:32:02.593298+0000","last_became_peered":"2024-07-04T17:32:02.593298+0000","last_unstale":"2024-07-04T17:32:03.574009+0000","last_undegraded":"2024-07-04T17:32:03.574009+0000","last_fullsized":"2024-07-04T17:32:03.574009+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:07:35.131023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1f","version":"91'4","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.996714+0000","last_change":"2024-07-04T17:31:41.125254+0000","last_active":"2024-07-04T17:32:03.996714+0000","last_peered":"2024-07-04T17:32:03.996714+0000","last_clean":"2024-07-04T17:32:03.996714+0000","last_became_active":"2024-07-04T17:31:41.119432+0000","last_became_peered":"2024-07-04T17:31:41.119432+0000","last_unstale":"2024-07-04T17:32:03.996714+0000","last_undegraded":"2024-07-04T17:32:03.996714+0000","last_fullsized":"2024-07-04T17:32:03.996714+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:40:27.856616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.996318+0000","last_change":"2024-07-04T17:31:42.139568+0000","last_active":"2024-07-04T17:32:03.996318+0000","last_peered":"2024-07-04T17:32:03.996318+0000","last_clean":"2024-07-04T17:32:03.996318+0000","last_became_active":"2024-07-04T17:31:42.139247+0000","last_became_peered":"2024-07-04T17:31:42.139247+0000","last_unstale":"2024-07-04T17:32:03.996318+0000","last_undegraded":"2024-07-04T17:32:03.996318+0000","last_fullsized":"2024-07-04T17:32:03.996318+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:48:35.448519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1d","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.993556+0000","last_change":"2024-07-04T17:32:01.582827+0000","last_active":"2024-07-04T17:32:03.993556+0000","last_peered":"2024-07-04T17:32:03.993556+0000","last_clean":"2024-07-04T17:32:03.993556+0000","last_became_active":"2024-07-04T17:32:01.582511+0000","last_became_peered":"2024-07-04T17:32:01.582511+0000","last_unstale":"2024-07-04T17:32:03.993556+0000","last_undegraded":"2024-07-04T17:32:03.993556+0000","last_fullsized":"2024-07-04T17:32:03.993556+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:43:00.246531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.571404+0000","last_change":"2024-07-04T17:32:02.606409+0000","last_active":"2024-07-04T17:32:03.571404+0000","last_peered":"2024-07-04T17:32:03.571404+0000","last_clean":"2024-07-04T17:32:03.571404+0000","last_became_active":"2024-07-04T17:32:02.605412+0000","last_became_peered":"2024-07-04T17:32:02.605412+0000","last_unstale":"2024-07-04T17:32:03.571404+0000","last_undegraded":"2024-07-04T17:32:03.571404+0000","last_fullsized":"2024-07-04T17:32:03.571404+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:43:48.709789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.984245+0000","last_change":"2024-07-04T17:31:42.137976+0000","last_active":"2024-07-04T17:32:03.984245+0000","last_peered":"2024-07-04T17:32:03.984245+0000","last_clean":"2024-07-04T17:32:03.984245+0000","last_became_active":"2024-07-04T17:31:42.137669+0000","last_became_peered":"2024-07-04T17:31:42.137669+0000","last_unstale":"2024-07-04T17:32:03.984245+0000","last_undegraded":"2024-07-04T17:32:03.984245+0000","last_fullsized":"2024-07-04T17:32:03.984245+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:07:25.661148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.573500+0000","last_change":"2024-07-04T17:32:02.610008+0000","last_active":"2024-07-04T17:32:03.573500+0000","last_peered":"2024-07-04T17:32:03.573500+0000","last_clean":"2024-07-04T17:32:03.573500+0000","last_became_active":"2024-07-04T17:32:02.609513+0000","last_became_peered":"2024-07-04T17:32:02.609513+0000","last_unstale":"2024-07-04T17:32:03.573500+0000","last_undegraded":"2024-07-04T17:32:03.573500+0000","last_fullsized":"2024-07-04T17:32:03.573500+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:14:23.536331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.990819+0000","last_change":"2024-07-04T17:31:42.150544+0000","last_active":"2024-07-04T17:32:03.990819+0000","last_peered":"2024-07-04T17:32:03.990819+0000","last_clean":"2024-07-04T17:32:03.990819+0000","last_became_active":"2024-07-04T17:31:42.150217+0000","last_became_peered":"2024-07-04T17:31:42.150217+0000","last_unstale":"2024-07-04T17:32:03.990819+0000","last_undegraded":"2024-07-04T17:32:03.990819+0000","last_fullsized":"2024-07-04T17:32:03.990819+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:50:45.348242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.983824+0000","last_change":"2024-07-04T17:32:02.610213+0000","last_active":"2024-07-04T17:32:03.983824+0000","last_peered":"2024-07-04T17:32:03.983824+0000","last_clean":"2024-07-04T17:32:03.983824+0000","last_became_active":"2024-07-04T17:32:02.609644+0000","last_became_peered":"2024-07-04T17:32:02.609644+0000","last_unstale":"2024-07-04T17:32:03.983824+0000","last_undegraded":"2024-07-04T17:32:03.983824+0000","last_fullsized":"2024-07-04T17:32:03.983824+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:44:33.949283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.572770+0000","last_change":"2024-07-04T17:31:42.115990+0000","last_active":"2024-07-04T17:32:03.572770+0000","last_peered":"2024-07-04T17:32:03.572770+0000","last_clean":"2024-07-04T17:32:03.572770+0000","last_became_active":"2024-07-04T17:31:42.115413+0000","last_became_peered":"2024-07-04T17:31:42.115413+0000","last_unstale":"2024-07-04T17:32:03.572770+0000","last_undegraded":"2024-07-04T17:32:03.572770+0000","last_fullsized":"2024-07-04T17:32:03.572770+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:36:22.090921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.983972+0000","last_change":"2024-07-04T17:32:02.621707+0000","last_active":"2024-07-04T17:32:03.983972+0000","last_peered":"2024-07-04T17:32:03.983972+0000","last_clean":"2024-07-04T17:32:03.983972+0000","last_became_active":"2024-07-04T17:32:02.620726+0000","last_became_peered":"2024-07-04T17:32:02.620726+0000","last_unstale":"2024-07-04T17:32:03.983972+0000","last_undegraded":"2024-07-04T17:32:03.983972+0000","last_fullsized":"2024-07-04T17:32:03.983972+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:02:45.804456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.991317+0000","last_change":"2024-07-04T17:31:42.111650+0000","last_active":"2024-07-04T17:32:03.991317+0000","last_peered":"2024-07-04T17:32:03.991317+0000","last_clean":"2024-07-04T17:32:03.991317+0000","last_became_active":"2024-07-04T17:31:42.111045+0000","last_became_peered":"2024-07-04T17:31:42.111045+0000","last_unstale":"2024-07-04T17:32:03.991317+0000","last_undegraded":"2024-07-04T17:32:03.991317+0000","last_fullsized":"2024-07-04T17:32:03.991317+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:11:39.466653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.995441+0000","last_change":"2024-07-04T17:32:02.600963+0000","last_active":"2024-07-04T17:32:03.995441+0000","last_peered":"2024-07-04T17:32:03.995441+0000","last_clean":"2024-07-04T17:32:03.995441+0000","last_became_active":"2024-07-04T17:32:02.598879+0000","last_became_peered":"2024-07-04T17:32:02.598879+0000","last_unstale":"2024-07-04T17:32:03.995441+0000","last_undegraded":"2024-07-04T17:32:03.995441+0000","last_fullsized":"2024-07-04T17:32:03.995441+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:45:52.984880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.992208+0000","last_change":"2024-07-04T17:31:42.139174+0000","last_active":"2024-07-04T17:32:03.992208+0000","last_peered":"2024-07-04T17:32:03.992208+0000","last_clean":"2024-07-04T17:32:03.992208+0000","last_became_active":"2024-07-04T17:31:42.138397+0000","last_became_peered":"2024-07-04T17:31:42.138397+0000","last_unstale":"2024-07-04T17:32:03.992208+0000","last_undegraded":"2024-07-04T17:32:03.992208+0000","last_fullsized":"2024-07-04T17:32:03.992208+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:21:29.625345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.995947+0000","last_change":"2024-07-04T17:32:02.608116+0000","last_active":"2024-07-04T17:32:03.995947+0000","last_peered":"2024-07-04T17:32:03.995947+0000","last_clean":"2024-07-04T17:32:03.995947+0000","last_became_active":"2024-07-04T17:32:02.606418+0000","last_became_peered":"2024-07-04T17:32:02.606418+0000","last_unstale":"2024-07-04T17:32:03.995947+0000","last_undegraded":"2024-07-04T17:32:03.995947+0000","last_fullsized":"2024-07-04T17:32:03.995947+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:23:11.566340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.994796+0000","last_change":"2024-07-04T17:31:42.130865+0000","last_active":"2024-07-04T17:32:03.994796+0000","last_peered":"2024-07-04T17:32:03.994796+0000","last_clean":"2024-07-04T17:32:03.994796+0000","last_became_active":"2024-07-04T17:31:42.130064+0000","last_became_peered":"2024-07-04T17:31:42.130064+0000","last_unstale":"2024-07-04T17:32:03.994796+0000","last_undegraded":"2024-07-04T17:32:03.994796+0000","last_fullsized":"2024-07-04T17:32:03.994796+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:54:40.425975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.987927+0000","last_change":"2024-07-04T17:32:02.610171+0000","last_active":"2024-07-04T17:32:03.987927+0000","last_peered":"2024-07-04T17:32:03.987927+0000","last_clean":"2024-07-04T17:32:03.987927+0000","last_became_active":"2024-07-04T17:32:02.609746+0000","last_became_peered":"2024-07-04T17:32:02.609746+0000","last_unstale":"2024-07-04T17:32:03.987927+0000","last_undegraded":"2024-07-04T17:32:03.987927+0000","last_fullsized":"2024-07-04T17:32:03.987927+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:33:45.577830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.996183+0000","last_change":"2024-07-04T17:31:42.142475+0000","last_active":"2024-07-04T17:32:03.996183+0000","last_peered":"2024-07-04T17:32:03.996183+0000","last_clean":"2024-07-04T17:32:03.996183+0000","last_became_active":"2024-07-04T17:31:42.142140+0000","last_became_peered":"2024-07-04T17:31:42.142140+0000","last_unstale":"2024-07-04T17:32:03.996183+0000","last_undegraded":"2024-07-04T17:32:03.996183+0000","last_fullsized":"2024-07-04T17:32:03.996183+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:12:40.862972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.574392+0000","last_change":"2024-07-04T17:32:02.605912+0000","last_active":"2024-07-04T17:32:03.574392+0000","last_peered":"2024-07-04T17:32:03.574392+0000","last_clean":"2024-07-04T17:32:03.574392+0000","last_became_active":"2024-07-04T17:32:02.605106+0000","last_became_peered":"2024-07-04T17:32:02.605106+0000","last_unstale":"2024-07-04T17:32:03.574392+0000","last_undegraded":"2024-07-04T17:32:03.574392+0000","last_fullsized":"2024-07-04T17:32:03.574392+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:51:10.170000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.576255+0000","last_change":"2024-07-04T17:31:42.129239+0000","last_active":"2024-07-04T17:32:03.576255+0000","last_peered":"2024-07-04T17:32:03.576255+0000","last_clean":"2024-07-04T17:32:03.576255+0000","last_became_active":"2024-07-04T17:31:42.127715+0000","last_became_peered":"2024-07-04T17:31:42.127715+0000","last_unstale":"2024-07-04T17:32:03.576255+0000","last_undegraded":"2024-07-04T17:32:03.576255+0000","last_fullsized":"2024-07-04T17:32:03.576255+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:51:01.891207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.989840+0000","last_change":"2024-07-04T17:32:02.607876+0000","last_active":"2024-07-04T17:32:03.989840+0000","last_peered":"2024-07-04T17:32:03.989840+0000","last_clean":"2024-07-04T17:32:03.989840+0000","last_became_active":"2024-07-04T17:32:02.607602+0000","last_became_peered":"2024-07-04T17:32:02.607602+0000","last_unstale":"2024-07-04T17:32:03.989840+0000","last_undegraded":"2024-07-04T17:32:03.989840+0000","last_fullsized":"2024-07-04T17:32:03.989840+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:07:22.204970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.571956+0000","last_change":"2024-07-04T17:31:42.126765+0000","last_active":"2024-07-04T17:32:03.571956+0000","last_peered":"2024-07-04T17:32:03.571956+0000","last_clean":"2024-07-04T17:32:03.571956+0000","last_became_active":"2024-07-04T17:31:42.126443+0000","last_became_peered":"2024-07-04T17:31:42.126443+0000","last_unstale":"2024-07-04T17:32:03.571956+0000","last_undegraded":"2024-07-04T17:32:03.571956+0000","last_fullsized":"2024-07-04T17:32:03.571956+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:34:36.071106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.990193+0000","last_change":"2024-07-04T17:32:02.608154+0000","last_active":"2024-07-04T17:32:03.990193+0000","last_peered":"2024-07-04T17:32:03.990193+0000","last_clean":"2024-07-04T17:32:03.990193+0000","last_became_active":"2024-07-04T17:32:02.606091+0000","last_became_peered":"2024-07-04T17:32:02.606091+0000","last_unstale":"2024-07-04T17:32:03.990193+0000","last_undegraded":"2024-07-04T17:32:03.990193+0000","last_fullsized":"2024-07-04T17:32:03.990193+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:47:34.307879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.990195+0000","last_change":"2024-07-04T17:31:42.153157+0000","last_active":"2024-07-04T17:32:03.990195+0000","last_peered":"2024-07-04T17:32:03.990195+0000","last_clean":"2024-07-04T17:32:03.990195+0000","last_became_active":"2024-07-04T17:31:42.150615+0000","last_became_peered":"2024-07-04T17:31:42.150615+0000","last_unstale":"2024-07-04T17:32:03.990195+0000","last_undegraded":"2024-07-04T17:32:03.990195+0000","last_fullsized":"2024-07-04T17:32:03.990195+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:50:49.706615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.988575+0000","last_change":"2024-07-04T17:32:02.609690+0000","last_active":"2024-07-04T17:32:03.988575+0000","last_peered":"2024-07-04T17:32:03.988575+0000","last_clean":"2024-07-04T17:32:03.988575+0000","last_became_active":"2024-07-04T17:32:02.608877+0000","last_became_peered":"2024-07-04T17:32:02.608877+0000","last_unstale":"2024-07-04T17:32:03.988575+0000","last_undegraded":"2024-07-04T17:32:03.988575+0000","last_fullsized":"2024-07-04T17:32:03.988575+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:37:21.373771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.572548+0000","last_change":"2024-07-04T17:31:42.116203+0000","last_active":"2024-07-04T17:32:03.572548+0000","last_peered":"2024-07-04T17:32:03.572548+0000","last_clean":"2024-07-04T17:32:03.572548+0000","last_became_active":"2024-07-04T17:31:42.115668+0000","last_became_peered":"2024-07-04T17:31:42.115668+0000","last_unstale":"2024-07-04T17:32:03.572548+0000","last_undegraded":"2024-07-04T17:32:03.572548+0000","last_fullsized":"2024-07-04T17:32:03.572548+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:06:36.600932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.984941+0000","last_change":"2024-07-04T17:32:02.620244+0000","last_active":"2024-07-04T17:32:03.984941+0000","last_peered":"2024-07-04T17:32:03.984941+0000","last_clean":"2024-07-04T17:32:03.984941+0000","last_became_active":"2024-07-04T17:32:02.619072+0000","last_became_peered":"2024-07-04T17:32:02.619072+0000","last_unstale":"2024-07-04T17:32:03.984941+0000","last_undegraded":"2024-07-04T17:32:03.984941+0000","last_fullsized":"2024-07-04T17:32:03.984941+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:13:14.107024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.573772+0000","last_change":"2024-07-04T17:31:42.127926+0000","last_active":"2024-07-04T17:32:03.573772+0000","last_peered":"2024-07-04T17:32:03.573772+0000","last_clean":"2024-07-04T17:32:03.573772+0000","last_became_active":"2024-07-04T17:31:42.127596+0000","last_became_peered":"2024-07-04T17:31:42.127596+0000","last_unstale":"2024-07-04T17:32:03.573772+0000","last_undegraded":"2024-07-04T17:32:03.573772+0000","last_fullsized":"2024-07-04T17:32:03.573772+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:20:44.775608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.995249+0000","last_change":"2024-07-04T17:32:02.601231+0000","last_active":"2024-07-04T17:32:03.995249+0000","last_peered":"2024-07-04T17:32:03.995249+0000","last_clean":"2024-07-04T17:32:03.995249+0000","last_became_active":"2024-07-04T17:32:02.599308+0000","last_became_peered":"2024-07-04T17:32:02.599308+0000","last_unstale":"2024-07-04T17:32:03.995249+0000","last_undegraded":"2024-07-04T17:32:03.995249+0000","last_fullsized":"2024-07-04T17:32:03.995249+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:46:29.471735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.988717+0000","last_change":"2024-07-04T17:31:42.141851+0000","last_active":"2024-07-04T17:32:03.988717+0000","last_peered":"2024-07-04T17:32:03.988717+0000","last_clean":"2024-07-04T17:32:03.988717+0000","last_became_active":"2024-07-04T17:31:42.139731+0000","last_became_peered":"2024-07-04T17:31:42.139731+0000","last_unstale":"2024-07-04T17:32:03.988717+0000","last_undegraded":"2024-07-04T17:32:03.988717+0000","last_fullsized":"2024-07-04T17:32:03.988717+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:46:33.797585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.990452+0000","last_change":"2024-07-04T17:32:02.611518+0000","last_active":"2024-07-04T17:32:03.990452+0000","last_peered":"2024-07-04T17:32:03.990452+0000","last_clean":"2024-07-04T17:32:03.990452+0000","last_became_active":"2024-07-04T17:32:02.610937+0000","last_became_peered":"2024-07-04T17:32:02.610937+0000","last_unstale":"2024-07-04T17:32:03.990452+0000","last_undegraded":"2024-07-04T17:32:03.990452+0000","last_fullsized":"2024-07-04T17:32:03.990452+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:12:42.384308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.994611+0000","last_change":"2024-07-04T17:31:42.143453+0000","last_active":"2024-07-04T17:32:03.994611+0000","last_peered":"2024-07-04T17:32:03.994611+0000","last_clean":"2024-07-04T17:32:03.994611+0000","last_became_active":"2024-07-04T17:31:42.142952+0000","last_became_peered":"2024-07-04T17:31:42.142952+0000","last_unstale":"2024-07-04T17:32:03.994611+0000","last_undegraded":"2024-07-04T17:32:03.994611+0000","last_fullsized":"2024-07-04T17:32:03.994611+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:21:03.395798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.988994+0000","last_change":"2024-07-04T17:32:02.591934+0000","last_active":"2024-07-04T17:32:03.988994+0000","last_peered":"2024-07-04T17:32:03.988994+0000","last_clean":"2024-07-04T17:32:03.988994+0000","last_became_active":"2024-07-04T17:32:02.577168+0000","last_became_peered":"2024-07-04T17:32:02.577168+0000","last_unstale":"2024-07-04T17:32:03.988994+0000","last_undegraded":"2024-07-04T17:32:03.988994+0000","last_fullsized":"2024-07-04T17:32:03.988994+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:43:36.215875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.996264+0000","last_change":"2024-07-04T17:31:42.131382+0000","last_active":"2024-07-04T17:32:03.996264+0000","last_peered":"2024-07-04T17:32:03.996264+0000","last_clean":"2024-07-04T17:32:03.996264+0000","last_became_active":"2024-07-04T17:31:42.131053+0000","last_became_peered":"2024-07-04T17:31:42.131053+0000","last_unstale":"2024-07-04T17:32:03.996264+0000","last_undegraded":"2024-07-04T17:32:03.996264+0000","last_fullsized":"2024-07-04T17:32:03.996264+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:52:52.277217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.991537+0000","last_change":"2024-07-04T17:32:02.603900+0000","last_active":"2024-07-04T17:32:03.991537+0000","last_peered":"2024-07-04T17:32:03.991537+0000","last_clean":"2024-07-04T17:32:03.991537+0000","last_became_active":"2024-07-04T17:32:02.602708+0000","last_became_peered":"2024-07-04T17:32:02.602708+0000","last_unstale":"2024-07-04T17:32:03.991537+0000","last_undegraded":"2024-07-04T17:32:03.991537+0000","last_fullsized":"2024-07-04T17:32:03.991537+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:34:06.894952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.990825+0000","last_change":"2024-07-04T17:31:42.127722+0000","last_active":"2024-07-04T17:32:03.990825+0000","last_peered":"2024-07-04T17:32:03.990825+0000","last_clean":"2024-07-04T17:32:03.990825+0000","last_became_active":"2024-07-04T17:31:42.127401+0000","last_became_peered":"2024-07-04T17:31:42.127401+0000","last_unstale":"2024-07-04T17:32:03.990825+0000","last_undegraded":"2024-07-04T17:32:03.990825+0000","last_fullsized":"2024-07-04T17:32:03.990825+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:08:43.721732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.984630+0000","last_change":"2024-07-04T17:32:02.578104+0000","last_active":"2024-07-04T17:32:03.984630+0000","last_peered":"2024-07-04T17:32:03.984630+0000","last_clean":"2024-07-04T17:32:03.984630+0000","last_became_active":"2024-07-04T17:32:02.577752+0000","last_became_peered":"2024-07-04T17:32:02.577752+0000","last_unstale":"2024-07-04T17:32:03.984630+0000","last_undegraded":"2024-07-04T17:32:03.984630+0000","last_fullsized":"2024-07-04T17:32:03.984630+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:40:49.324334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.987225+0000","last_change":"2024-07-04T17:31:42.124307+0000","last_active":"2024-07-04T17:32:03.987225+0000","last_peered":"2024-07-04T17:32:03.987225+0000","last_clean":"2024-07-04T17:32:03.987225+0000","last_became_active":"2024-07-04T17:31:42.121958+0000","last_became_peered":"2024-07-04T17:31:42.121958+0000","last_unstale":"2024-07-04T17:32:03.987225+0000","last_undegraded":"2024-07-04T17:32:03.987225+0000","last_fullsized":"2024-07-04T17:32:03.987225+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:14:03.072947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.573073+0000","last_change":"2024-07-04T17:32:02.596561+0000","last_active":"2024-07-04T17:32:03.573073+0000","last_peered":"2024-07-04T17:32:03.573073+0000","last_clean":"2024-07-04T17:32:03.573073+0000","last_became_active":"2024-07-04T17:32:02.596059+0000","last_became_peered":"2024-07-04T17:32:02.596059+0000","last_unstale":"2024-07-04T17:32:03.573073+0000","last_undegraded":"2024-07-04T17:32:03.573073+0000","last_fullsized":"2024-07-04T17:32:03.573073+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:19:30.318182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.984451+0000","last_change":"2024-07-04T17:31:42.126095+0000","last_active":"2024-07-04T17:32:03.984451+0000","last_peered":"2024-07-04T17:32:03.984451+0000","last_clean":"2024-07-04T17:32:03.984451+0000","last_became_active":"2024-07-04T17:31:42.125783+0000","last_became_peered":"2024-07-04T17:31:42.125783+0000","last_unstale":"2024-07-04T17:32:03.984451+0000","last_undegraded":"2024-07-04T17:32:03.984451+0000","last_fullsized":"2024-07-04T17:32:03.984451+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:01:18.486569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.571314+0000","last_change":"2024-07-04T17:32:02.583841+0000","last_active":"2024-07-04T17:32:03.571314+0000","last_peered":"2024-07-04T17:32:03.571314+0000","last_clean":"2024-07-04T17:32:03.571314+0000","last_became_active":"2024-07-04T17:32:02.583156+0000","last_became_peered":"2024-07-04T17:32:02.583156+0000","last_unstale":"2024-07-04T17:32:03.571314+0000","last_undegraded":"2024-07-04T17:32:03.571314+0000","last_fullsized":"2024-07-04T17:32:03.571314+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:23:11.566340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.993014+0000","last_change":"2024-07-04T17:31:42.137702+0000","last_active":"2024-07-04T17:32:03.993014+0000","last_peered":"2024-07-04T17:32:03.993014+0000","last_clean":"2024-07-04T17:32:03.993014+0000","last_became_active":"2024-07-04T17:31:42.134081+0000","last_became_peered":"2024-07-04T17:31:42.134081+0000","last_unstale":"2024-07-04T17:32:03.993014+0000","last_undegraded":"2024-07-04T17:32:03.993014+0000","last_fullsized":"2024-07-04T17:32:03.993014+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:26:29.769982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.571075+0000","last_change":"2024-07-04T17:32:02.605034+0000","last_active":"2024-07-04T17:32:03.571075+0000","last_peered":"2024-07-04T17:32:03.571075+0000","last_clean":"2024-07-04T17:32:03.571075+0000","last_became_active":"2024-07-04T17:32:02.603145+0000","last_became_peered":"2024-07-04T17:32:02.603145+0000","last_unstale":"2024-07-04T17:32:03.571075+0000","last_undegraded":"2024-07-04T17:32:03.571075+0000","last_fullsized":"2024-07-04T17:32:03.571075+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:45:52.984880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.573585+0000","last_change":"2024-07-04T17:31:42.132759+0000","last_active":"2024-07-04T17:32:03.573585+0000","last_peered":"2024-07-04T17:32:03.573585+0000","last_clean":"2024-07-04T17:32:03.573585+0000","last_became_active":"2024-07-04T17:31:42.132493+0000","last_became_peered":"2024-07-04T17:31:42.132493+0000","last_unstale":"2024-07-04T17:32:03.573585+0000","last_undegraded":"2024-07-04T17:32:03.573585+0000","last_fullsized":"2024-07-04T17:32:03.573585+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:41:07.944963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.995061+0000","last_change":"2024-07-04T17:32:02.618897+0000","last_active":"2024-07-04T17:32:03.995061+0000","last_peered":"2024-07-04T17:32:03.995061+0000","last_clean":"2024-07-04T17:32:03.995061+0000","last_became_active":"2024-07-04T17:32:02.618566+0000","last_became_peered":"2024-07-04T17:32:02.618566+0000","last_unstale":"2024-07-04T17:32:03.995061+0000","last_undegraded":"2024-07-04T17:32:03.995061+0000","last_fullsized":"2024-07-04T17:32:03.995061+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:05:00.682546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.992216+0000","last_change":"2024-07-04T17:31:42.110720+0000","last_active":"2024-07-04T17:32:03.992216+0000","last_peered":"2024-07-04T17:32:03.992216+0000","last_clean":"2024-07-04T17:32:03.992216+0000","last_became_active":"2024-07-04T17:31:42.110428+0000","last_became_peered":"2024-07-04T17:31:42.110428+0000","last_unstale":"2024-07-04T17:32:03.992216+0000","last_undegraded":"2024-07-04T17:32:03.992216+0000","last_fullsized":"2024-07-04T17:32:03.992216+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:03:49.795002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.576725+0000","last_change":"2024-07-04T17:32:02.610969+0000","last_active":"2024-07-04T17:32:03.576725+0000","last_peered":"2024-07-04T17:32:03.576725+0000","last_clean":"2024-07-04T17:32:03.576725+0000","last_became_active":"2024-07-04T17:32:02.604273+0000","last_became_peered":"2024-07-04T17:32:02.604273+0000","last_unstale":"2024-07-04T17:32:03.576725+0000","last_undegraded":"2024-07-04T17:32:03.576725+0000","last_fullsized":"2024-07-04T17:32:03.576725+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:11:59.930038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.993871+0000","last_change":"2024-07-04T17:31:42.138904+0000","last_active":"2024-07-04T17:32:03.993871+0000","last_peered":"2024-07-04T17:32:03.993871+0000","last_clean":"2024-07-04T17:32:03.993871+0000","last_became_active":"2024-07-04T17:31:42.138265+0000","last_became_peered":"2024-07-04T17:31:42.138265+0000","last_unstale":"2024-07-04T17:32:03.993871+0000","last_undegraded":"2024-07-04T17:32:03.993871+0000","last_fullsized":"2024-07-04T17:32:03.993871+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:26.894424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.991430+0000","last_change":"2024-07-04T17:32:02.622111+0000","last_active":"2024-07-04T17:32:03.991430+0000","last_peered":"2024-07-04T17:32:03.991430+0000","last_clean":"2024-07-04T17:32:03.991430+0000","last_became_active":"2024-07-04T17:32:02.621810+0000","last_became_peered":"2024-07-04T17:32:02.621810+0000","last_unstale":"2024-07-04T17:32:03.991430+0000","last_undegraded":"2024-07-04T17:32:03.991430+0000","last_fullsized":"2024-07-04T17:32:03.991430+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:12:45.045568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.996025+0000","last_change":"2024-07-04T17:31:42.109492+0000","last_active":"2024-07-04T17:32:03.996025+0000","last_peered":"2024-07-04T17:32:03.996025+0000","last_clean":"2024-07-04T17:32:03.996025+0000","last_became_active":"2024-07-04T17:31:42.108502+0000","last_became_peered":"2024-07-04T17:31:42.108502+0000","last_unstale":"2024-07-04T17:32:03.996025+0000","last_undegraded":"2024-07-04T17:32:03.996025+0000","last_fullsized":"2024-07-04T17:32:03.996025+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:39:41.707779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.990944+0000","last_change":"2024-07-04T17:32:02.604036+0000","last_active":"2024-07-04T17:32:03.990944+0000","last_peered":"2024-07-04T17:32:03.990944+0000","last_clean":"2024-07-04T17:32:03.990944+0000","last_became_active":"2024-07-04T17:32:02.603062+0000","last_became_peered":"2024-07-04T17:32:02.603062+0000","last_unstale":"2024-07-04T17:32:03.990944+0000","last_undegraded":"2024-07-04T17:32:03.990944+0000","last_fullsized":"2024-07-04T17:32:03.990944+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:34:12.680957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.994685+0000","last_change":"2024-07-04T17:31:42.134470+0000","last_active":"2024-07-04T17:32:03.994685+0000","last_peered":"2024-07-04T17:32:03.994685+0000","last_clean":"2024-07-04T17:32:03.994685+0000","last_became_active":"2024-07-04T17:31:42.134096+0000","last_became_peered":"2024-07-04T17:31:42.134096+0000","last_unstale":"2024-07-04T17:32:03.994685+0000","last_undegraded":"2024-07-04T17:32:03.994685+0000","last_fullsized":"2024-07-04T17:32:03.994685+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:07:42.785561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.989640+0000","last_change":"2024-07-04T17:32:02.603154+0000","last_active":"2024-07-04T17:32:03.989640+0000","last_peered":"2024-07-04T17:32:03.989640+0000","last_clean":"2024-07-04T17:32:03.989640+0000","last_became_active":"2024-07-04T17:32:02.602818+0000","last_became_peered":"2024-07-04T17:32:02.602818+0000","last_unstale":"2024-07-04T17:32:03.989640+0000","last_undegraded":"2024-07-04T17:32:03.989640+0000","last_fullsized":"2024-07-04T17:32:03.989640+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:49:39.286469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.571746+0000","last_change":"2024-07-04T17:31:42.127048+0000","last_active":"2024-07-04T17:32:03.571746+0000","last_peered":"2024-07-04T17:32:03.571746+0000","last_clean":"2024-07-04T17:32:03.571746+0000","last_became_active":"2024-07-04T17:31:42.126695+0000","last_became_peered":"2024-07-04T17:31:42.126695+0000","last_unstale":"2024-07-04T17:32:03.571746+0000","last_undegraded":"2024-07-04T17:32:03.571746+0000","last_fullsized":"2024-07-04T17:32:03.571746+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:46:15.072312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.572934+0000","last_change":"2024-07-04T17:32:02.610521+0000","last_active":"2024-07-04T17:32:03.572934+0000","last_peered":"2024-07-04T17:32:03.572934+0000","last_clean":"2024-07-04T17:32:03.572934+0000","last_became_active":"2024-07-04T17:32:02.609949+0000","last_became_peered":"2024-07-04T17:32:02.609949+0000","last_unstale":"2024-07-04T17:32:03.572934+0000","last_undegraded":"2024-07-04T17:32:03.572934+0000","last_fullsized":"2024-07-04T17:32:03.572934+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:23:26.532448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.571807+0000","last_change":"2024-07-04T17:31:42.131777+0000","last_active":"2024-07-04T17:32:03.571807+0000","last_peered":"2024-07-04T17:32:03.571807+0000","last_clean":"2024-07-04T17:32:03.571807+0000","last_became_active":"2024-07-04T17:31:42.127915+0000","last_became_peered":"2024-07-04T17:31:42.127915+0000","last_unstale":"2024-07-04T17:32:03.571807+0000","last_undegraded":"2024-07-04T17:32:03.571807+0000","last_fullsized":"2024-07-04T17:32:03.571807+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:14:13.315623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.570740+0000","last_change":"2024-07-04T17:32:02.584120+0000","last_active":"2024-07-04T17:32:03.570740+0000","last_peered":"2024-07-04T17:32:03.570740+0000","last_clean":"2024-07-04T17:32:03.570740+0000","last_became_active":"2024-07-04T17:32:02.583539+0000","last_became_peered":"2024-07-04T17:32:02.583539+0000","last_unstale":"2024-07-04T17:32:03.570740+0000","last_undegraded":"2024-07-04T17:32:03.570740+0000","last_fullsized":"2024-07-04T17:32:03.570740+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:05:00.682546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.985575+0000","last_change":"2024-07-04T17:31:42.146322+0000","last_active":"2024-07-04T17:32:03.985575+0000","last_peered":"2024-07-04T17:32:03.985575+0000","last_clean":"2024-07-04T17:32:03.985575+0000","last_became_active":"2024-07-04T17:31:42.146004+0000","last_became_peered":"2024-07-04T17:31:42.146004+0000","last_unstale":"2024-07-04T17:32:03.985575+0000","last_undegraded":"2024-07-04T17:32:03.985575+0000","last_fullsized":"2024-07-04T17:32:03.985575+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:10:27.612193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.571048+0000","last_change":"2024-07-04T17:32:02.614219+0000","last_active":"2024-07-04T17:32:03.571048+0000","last_peered":"2024-07-04T17:32:03.571048+0000","last_clean":"2024-07-04T17:32:03.571048+0000","last_became_active":"2024-07-04T17:32:02.613331+0000","last_became_peered":"2024-07-04T17:32:02.613331+0000","last_unstale":"2024-07-04T17:32:03.571048+0000","last_undegraded":"2024-07-04T17:32:03.571048+0000","last_fullsized":"2024-07-04T17:32:03.571048+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:46:29.471735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.573443+0000","last_change":"2024-07-04T17:31:42.118035+0000","last_active":"2024-07-04T17:32:03.573443+0000","last_peered":"2024-07-04T17:32:03.573443+0000","last_clean":"2024-07-04T17:32:03.573443+0000","last_became_active":"2024-07-04T17:31:42.117722+0000","last_became_peered":"2024-07-04T17:31:42.117722+0000","last_unstale":"2024-07-04T17:32:03.573443+0000","last_undegraded":"2024-07-04T17:32:03.573443+0000","last_fullsized":"2024-07-04T17:32:03.573443+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:17:18.803619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.994869+0000","last_change":"2024-07-04T17:32:02.605817+0000","last_active":"2024-07-04T17:32:03.994869+0000","last_peered":"2024-07-04T17:32:03.994869+0000","last_clean":"2024-07-04T17:32:03.994869+0000","last_became_active":"2024-07-04T17:32:02.603201+0000","last_became_peered":"2024-07-04T17:32:02.603201+0000","last_unstale":"2024-07-04T17:32:03.994869+0000","last_undegraded":"2024-07-04T17:32:03.994869+0000","last_fullsized":"2024-07-04T17:32:03.994869+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:30:29.511665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.992079+0000","last_change":"2024-07-04T17:31:42.148476+0000","last_active":"2024-07-04T17:32:03.992079+0000","last_peered":"2024-07-04T17:32:03.992079+0000","last_clean":"2024-07-04T17:32:03.992079+0000","last_became_active":"2024-07-04T17:31:42.148083+0000","last_became_peered":"2024-07-04T17:31:42.148083+0000","last_unstale":"2024-07-04T17:32:03.992079+0000","last_undegraded":"2024-07-04T17:32:03.992079+0000","last_fullsized":"2024-07-04T17:32:03.992079+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:19:47.089588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.3f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.576559+0000","last_change":"2024-07-04T17:32:02.609478+0000","last_active":"2024-07-04T17:32:03.576559+0000","last_peered":"2024-07-04T17:32:03.576559+0000","last_clean":"2024-07-04T17:32:03.576559+0000","last_became_active":"2024-07-04T17:32:02.603062+0000","last_became_peered":"2024-07-04T17:32:02.603062+0000","last_unstale":"2024-07-04T17:32:03.576559+0000","last_undegraded":"2024-07-04T17:32:03.576559+0000","last_fullsized":"2024-07-04T17:32:03.576559+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:30:37.885691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.994489+0000","last_change":"2024-07-04T17:31:42.146020+0000","last_active":"2024-07-04T17:32:03.994489+0000","last_peered":"2024-07-04T17:32:03.994489+0000","last_clean":"2024-07-04T17:32:03.994489+0000","last_became_active":"2024-07-04T17:31:42.143682+0000","last_became_peered":"2024-07-04T17:31:42.143682+0000","last_unstale":"2024-07-04T17:32:03.994489+0000","last_undegraded":"2024-07-04T17:32:03.994489+0000","last_fullsized":"2024-07-04T17:32:03.994489+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:06:33.198375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.574022+0000","last_change":"2024-07-04T17:32:02.581917+0000","last_active":"2024-07-04T17:32:03.574022+0000","last_peered":"2024-07-04T17:32:03.574022+0000","last_clean":"2024-07-04T17:32:03.574022+0000","last_became_active":"2024-07-04T17:32:02.581630+0000","last_became_peered":"2024-07-04T17:32:02.581630+0000","last_unstale":"2024-07-04T17:32:03.574022+0000","last_undegraded":"2024-07-04T17:32:03.574022+0000","last_fullsized":"2024-07-04T17:32:03.574022+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:45:45.913107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.575361+0000","last_change":"2024-07-04T17:31:42.132420+0000","last_active":"2024-07-04T17:32:03.575361+0000","last_peered":"2024-07-04T17:32:03.575361+0000","last_clean":"2024-07-04T17:32:03.575361+0000","last_became_active":"2024-07-04T17:31:42.132031+0000","last_became_peered":"2024-07-04T17:31:42.132031+0000","last_unstale":"2024-07-04T17:32:03.575361+0000","last_undegraded":"2024-07-04T17:32:03.575361+0000","last_fullsized":"2024-07-04T17:32:03.575361+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:41:23.204256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.990322+0000","last_change":"2024-07-04T17:32:02.604131+0000","last_active":"2024-07-04T17:32:03.990322+0000","last_peered":"2024-07-04T17:32:03.990322+0000","last_clean":"2024-07-04T17:32:03.990322+0000","last_became_active":"2024-07-04T17:32:02.603425+0000","last_became_peered":"2024-07-04T17:32:02.603425+0000","last_unstale":"2024-07-04T17:32:03.990322+0000","last_undegraded":"2024-07-04T17:32:03.990322+0000","last_fullsized":"2024-07-04T17:32:03.990322+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:05:44.906220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3f","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.994680+0000","last_change":"2024-07-04T17:31:42.147550+0000","last_active":"2024-07-04T17:32:03.994680+0000","last_peered":"2024-07-04T17:32:03.994680+0000","last_clean":"2024-07-04T17:32:03.994680+0000","last_became_active":"2024-07-04T17:31:42.146638+0000","last_became_peered":"2024-07-04T17:31:42.146638+0000","last_unstale":"2024-07-04T17:32:03.994680+0000","last_undegraded":"2024-07-04T17:32:03.994680+0000","last_fullsized":"2024-07-04T17:32:03.994680+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:17:39.808949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":4107,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":356,"num_read_kb":926,"num_write":139,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738440,"num_pgs":128,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36316,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335524,"statfs":{"total":96636764160,"available":96599576576,"internally_reserved":0,"allocated":995328,"data_stored":385750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738441,"num_pgs":103,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36928,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334912,"statfs":{"total":96636764160,"available":96598949888,"internally_reserved":0,"allocated":1617920,"data_stored":980653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36928,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334912,"statfs":{"total":96636764160,"available":96598949888,"internally_reserved":0,"allocated":1617920,"data_stored":980585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:32:07.327 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:32:07.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.654+0000 7f4532c00640 1 -- 172.21.15.42:0/4122058461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f452c13f3b0 msgr2=0x7f452c13f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:07.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.654+0000 7f4532c00640 1 --2- 172.21.15.42:0/4122058461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f452c13f3b0 0x7f452c13f7c0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f451c005fd0 tx=0x7f451c02aac0 comp rx=0 tx=0).stop 2024-07-04T17:32:07.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.654+0000 7f4532c00640 1 -- 172.21.15.42:0/4122058461 shutdown_connections 2024-07-04T17:32:07.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.654+0000 7f4532c00640 1 --2- 172.21.15.42:0/4122058461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f452c13f3b0 0x7f452c13f7c0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:07.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.654+0000 7f4532c00640 1 -- 172.21.15.42:0/4122058461 >> 172.21.15.42:0/4122058461 conn(0x7f452c06c020 msgr2=0x7f452c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:07.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.654+0000 7f4532c00640 1 -- 172.21.15.42:0/4122058461 shutdown_connections 2024-07-04T17:32:07.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.654+0000 7f4532c00640 1 -- 172.21.15.42:0/4122058461 wait complete. 2024-07-04T17:32:07.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.656+0000 7f4532c00640 1 Processor -- start 2024-07-04T17:32:07.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.656+0000 7f4532c00640 1 -- start start 2024-07-04T17:32:07.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.658+0000 7f4532c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f452c072800 0x7f452c0b6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:07.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.658+0000 7f4532c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f452c0aa5f0 con 0x7f452c072800 2024-07-04T17:32:07.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.658+0000 7f4531800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f452c072800 0x7f452c0b6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:07.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.658+0000 7f4531800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f452c072800 0x7f452c0b6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35420/0 (socket says 172.21.15.42:35420) 2024-07-04T17:32:07.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.658+0000 7f4531800640 1 -- 172.21.15.42:0/422169463 learned_addr learned my addr 172.21.15.42:0/422169463 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:07.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.659+0000 7f4531800640 1 -- 172.21.15.42:0/422169463 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452c1e5060 con 0x7f452c072800 2024-07-04T17:32:07.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.659+0000 7f4531800640 1 --2- 172.21.15.42:0/422169463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f452c072800 0x7f452c0b6670 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f452000c380 tx=0x7f452000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:07.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.659+0000 7f4529600640 1 -- 172.21.15.42:0/422169463 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4520017070 con 0x7f452c072800 2024-07-04T17:32:07.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.659+0000 7f4529600640 1 -- 172.21.15.42:0/422169463 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45200052f0 con 0x7f452c072800 2024-07-04T17:32:07.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.659+0000 7f4529600640 1 -- 172.21.15.42:0/422169463 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45200035f0 con 0x7f452c072800 2024-07-04T17:32:07.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.659+0000 7f4532c00640 1 -- 172.21.15.42:0/422169463 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f452c0aeef0 con 0x7f452c072800 2024-07-04T17:32:07.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.659+0000 7f4532c00640 1 -- 172.21.15.42:0/422169463 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f452c0adb30 con 0x7f452c072800 2024-07-04T17:32:07.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.661+0000 7f4532c00640 1 -- 172.21.15.42:0/422169463 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f452c0aac70 con 0x7f452c072800 2024-07-04T17:32:07.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.661+0000 7f4529600640 1 -- 172.21.15.42:0/422169463 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7f452001a070 con 0x7f452c072800 2024-07-04T17:32:07.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.661+0000 7f4529600640 1 --2- 172.21.15.42:0/422169463 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f451003d3d0 0x7f451003f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:07.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.661+0000 7f4529600640 1 -- 172.21.15.42:0/422169463 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4303+0+0 (secure 0 0 0) 0x7f4520050d00 con 0x7f452c072800 2024-07-04T17:32:07.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.662+0000 7f4530e00640 1 --2- 172.21.15.42:0/422169463 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f451003d3d0 0x7f451003f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:07.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.663+0000 7f4530e00640 1 --2- 172.21.15.42:0/422169463 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f451003d3d0 0x7f451003f8e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f451c035000 tx=0x7f451c004ab0 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:07.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.666+0000 7f4529600640 1 -- 172.21.15.42:0/422169463 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f4520016170 con 0x7f452c072800 2024-07-04T17:32:07.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.872+0000 7f4532c00640 1 -- 172.21.15.42:0/422169463 --> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f452c0b8010 con 0x7f451003d3d0 2024-07-04T17:32:07.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.883+0000 7f4529600640 1 -- 172.21.15.42:0/422169463 <== mgr.5173 v2:172.21.15.42:6834/1139872311 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+482444 (secure 0 0 0) 0x7f452c0b8010 con 0x7f451003d3d0 2024-07-04T17:32:07.891 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:32:07.894 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:32:07.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.893+0000 7f4532c00640 1 -- 172.21.15.42:0/422169463 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f451003d3d0 msgr2=0x7f451003f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:07.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.893+0000 7f4532c00640 1 --2- 172.21.15.42:0/422169463 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f451003d3d0 0x7f451003f8e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f451c035000 tx=0x7f451c004ab0 comp rx=0 tx=0).stop 2024-07-04T17:32:07.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.893+0000 7f4532c00640 1 -- 172.21.15.42:0/422169463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f452c072800 msgr2=0x7f452c0b6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:07.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.893+0000 7f4532c00640 1 --2- 172.21.15.42:0/422169463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f452c072800 0x7f452c0b6670 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f452000c380 tx=0x7f452000c840 comp rx=0 tx=0).stop 2024-07-04T17:32:07.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.893+0000 7f4532c00640 1 -- 172.21.15.42:0/422169463 shutdown_connections 2024-07-04T17:32:07.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.893+0000 7f4532c00640 1 --2- 172.21.15.42:0/422169463 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f451003d3d0 0x7f451003f8e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:07.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.893+0000 7f4532c00640 1 --2- 172.21.15.42:0/422169463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f452c072800 0x7f452c0b6670 secure :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f452000c380 tx=0x7f452000c840 comp rx=0 tx=0).stop 2024-07-04T17:32:07.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.893+0000 7f4532c00640 1 -- 172.21.15.42:0/422169463 >> 172.21.15.42:0/422169463 conn(0x7f452c06c020 msgr2=0x7f452c1441a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:07.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.894+0000 7f4532c00640 1 -- 172.21.15.42:0/422169463 shutdown_connections 2024-07-04T17:32:07.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:07.895+0000 7f4532c00640 1 -- 172.21.15.42:0/422169463 wait complete. 2024-07-04T17:32:07.922 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-07-04T17:32:06.446250+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596885,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":356,"num_read_kb":926,"num_write":231,"num_write_kb":1137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":299,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":110172,"kb_used_data":4132,"kb_used_omap":4,"kb_used_meta":105979,"kb_avail":283005348,"statfs":{"total":289910292480,"available":289797476352,"internally_reserved":0,"allocated":4231168,"data_stored":2346988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":108522831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.918327"},"pg_stats":[{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.992791+0000","last_change":"2024-07-04T17:32:02.593945+0000","last_active":"2024-07-04T17:32:03.992791+0000","last_peered":"2024-07-04T17:32:03.992791+0000","last_clean":"2024-07-04T17:32:03.992791+0000","last_became_active":"2024-07-04T17:32:02.592848+0000","last_became_peered":"2024-07-04T17:32:02.592848+0000","last_unstale":"2024-07-04T17:32:03.992791+0000","last_undegraded":"2024-07-04T17:32:03.992791+0000","last_fullsized":"2024-07-04T17:32:03.992791+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:55:16.109523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"80'87","reported_seq":364,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.984767+0000","last_change":"2024-07-04T17:26:04.953559+0000","last_active":"2024-07-04T17:32:03.984767+0000","last_peered":"2024-07-04T17:32:03.984767+0000","last_clean":"2024-07-04T17:32:03.984767+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:32:03.984767+0000","last_undegraded":"2024-07-04T17:32:03.984767+0000","last_fullsized":"2024-07-04T17:32:03.984767+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_clean_scrub_stamp":"2024-07-04T17:26:03.933432+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.064181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":356,"num_read_kb":926,"num_write":139,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.571765+0000","last_change":"2024-07-04T17:31:41.125061+0000","last_active":"2024-07-04T17:32:03.571765+0000","last_peered":"2024-07-04T17:32:03.571765+0000","last_clean":"2024-07-04T17:32:03.571765+0000","last_became_active":"2024-07-04T17:31:41.122766+0000","last_became_peered":"2024-07-04T17:31:41.122766+0000","last_unstale":"2024-07-04T17:32:03.571765+0000","last_undegraded":"2024-07-04T17:32:03.571765+0000","last_fullsized":"2024-07-04T17:32:03.571765+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:00:42.281350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.987553+0000","last_change":"2024-07-04T17:31:42.124940+0000","last_active":"2024-07-04T17:32:03.987553+0000","last_peered":"2024-07-04T17:32:03.987553+0000","last_clean":"2024-07-04T17:32:03.987553+0000","last_became_active":"2024-07-04T17:31:42.124608+0000","last_became_peered":"2024-07-04T17:31:42.124608+0000","last_unstale":"2024-07-04T17:32:03.987553+0000","last_undegraded":"2024-07-04T17:32:03.987553+0000","last_fullsized":"2024-07-04T17:32:03.987553+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:11:02.635340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.13","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.990871+0000","last_change":"2024-07-04T17:32:01.584681+0000","last_active":"2024-07-04T17:32:03.990871+0000","last_peered":"2024-07-04T17:32:03.990871+0000","last_clean":"2024-07-04T17:32:03.990871+0000","last_became_active":"2024-07-04T17:32:01.584237+0000","last_became_peered":"2024-07-04T17:32:01.584237+0000","last_unstale":"2024-07-04T17:32:03.990871+0000","last_undegraded":"2024-07-04T17:32:03.990871+0000","last_fullsized":"2024-07-04T17:32:03.990871+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:17:45.481113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.990690+0000","last_change":"2024-07-04T17:32:02.580087+0000","last_active":"2024-07-04T17:32:03.990690+0000","last_peered":"2024-07-04T17:32:03.990690+0000","last_clean":"2024-07-04T17:32:03.990690+0000","last_became_active":"2024-07-04T17:32:02.579751+0000","last_became_peered":"2024-07-04T17:32:02.579751+0000","last_unstale":"2024-07-04T17:32:03.990690+0000","last_undegraded":"2024-07-04T17:32:03.990690+0000","last_fullsized":"2024-07-04T17:32:03.990690+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:05:58.936715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.988239+0000","last_change":"2024-07-04T17:31:41.111066+0000","last_active":"2024-07-04T17:32:03.988239+0000","last_peered":"2024-07-04T17:32:03.988239+0000","last_clean":"2024-07-04T17:32:03.988239+0000","last_became_active":"2024-07-04T17:31:41.110689+0000","last_became_peered":"2024-07-04T17:31:41.110689+0000","last_unstale":"2024-07-04T17:32:03.988239+0000","last_undegraded":"2024-07-04T17:32:03.988239+0000","last_fullsized":"2024-07-04T17:32:03.988239+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:24:51.654346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.991127+0000","last_change":"2024-07-04T17:31:42.145454+0000","last_active":"2024-07-04T17:32:03.991127+0000","last_peered":"2024-07-04T17:32:03.991127+0000","last_clean":"2024-07-04T17:32:03.991127+0000","last_became_active":"2024-07-04T17:31:42.144919+0000","last_became_peered":"2024-07-04T17:31:42.144919+0000","last_unstale":"2024-07-04T17:32:03.991127+0000","last_undegraded":"2024-07-04T17:32:03.991127+0000","last_fullsized":"2024-07-04T17:32:03.991127+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:30:26.887002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.991864+0000","last_change":"2024-07-04T17:32:01.585972+0000","last_active":"2024-07-04T17:32:03.991864+0000","last_peered":"2024-07-04T17:32:03.991864+0000","last_clean":"2024-07-04T17:32:03.991864+0000","last_became_active":"2024-07-04T17:32:01.583249+0000","last_became_peered":"2024-07-04T17:32:01.583249+0000","last_unstale":"2024-07-04T17:32:03.991864+0000","last_undegraded":"2024-07-04T17:32:03.991864+0000","last_fullsized":"2024-07-04T17:32:03.991864+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:42:13.449049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.992861+0000","last_change":"2024-07-04T17:32:02.604765+0000","last_active":"2024-07-04T17:32:03.992861+0000","last_peered":"2024-07-04T17:32:03.992861+0000","last_clean":"2024-07-04T17:32:03.992861+0000","last_became_active":"2024-07-04T17:32:02.603246+0000","last_became_peered":"2024-07-04T17:32:02.603246+0000","last_unstale":"2024-07-04T17:32:03.992861+0000","last_undegraded":"2024-07-04T17:32:03.992861+0000","last_fullsized":"2024-07-04T17:32:03.992861+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:34:45.295638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.571936+0000","last_change":"2024-07-04T17:31:41.116814+0000","last_active":"2024-07-04T17:32:03.571936+0000","last_peered":"2024-07-04T17:32:03.571936+0000","last_clean":"2024-07-04T17:32:03.571936+0000","last_became_active":"2024-07-04T17:31:41.116425+0000","last_became_peered":"2024-07-04T17:31:41.116425+0000","last_unstale":"2024-07-04T17:32:03.571936+0000","last_undegraded":"2024-07-04T17:32:03.571936+0000","last_fullsized":"2024-07-04T17:32:03.571936+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:18:00.134663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.987772+0000","last_change":"2024-07-04T17:31:42.112865+0000","last_active":"2024-07-04T17:32:03.987772+0000","last_peered":"2024-07-04T17:32:03.987772+0000","last_clean":"2024-07-04T17:32:03.987772+0000","last_became_active":"2024-07-04T17:31:42.109293+0000","last_became_peered":"2024-07-04T17:31:42.109293+0000","last_unstale":"2024-07-04T17:32:03.987772+0000","last_undegraded":"2024-07-04T17:32:03.987772+0000","last_fullsized":"2024-07-04T17:32:03.987772+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:07:14.667638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.985688+0000","last_change":"2024-07-04T17:32:01.596072+0000","last_active":"2024-07-04T17:32:03.985688+0000","last_peered":"2024-07-04T17:32:03.985688+0000","last_clean":"2024-07-04T17:32:03.985688+0000","last_became_active":"2024-07-04T17:32:01.593045+0000","last_became_peered":"2024-07-04T17:32:01.593045+0000","last_unstale":"2024-07-04T17:32:03.985688+0000","last_undegraded":"2024-07-04T17:32:03.985688+0000","last_fullsized":"2024-07-04T17:32:03.985688+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:24:56.768417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.984166+0000","last_change":"2024-07-04T17:32:02.619573+0000","last_active":"2024-07-04T17:32:03.984166+0000","last_peered":"2024-07-04T17:32:03.984166+0000","last_clean":"2024-07-04T17:32:03.984166+0000","last_became_active":"2024-07-04T17:32:02.614841+0000","last_became_peered":"2024-07-04T17:32:02.614841+0000","last_unstale":"2024-07-04T17:32:03.984166+0000","last_undegraded":"2024-07-04T17:32:03.984166+0000","last_fullsized":"2024-07-04T17:32:03.984166+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:47:31.973324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.993143+0000","last_change":"2024-07-04T17:31:41.124595+0000","last_active":"2024-07-04T17:32:03.993143+0000","last_peered":"2024-07-04T17:32:03.993143+0000","last_clean":"2024-07-04T17:32:03.993143+0000","last_became_active":"2024-07-04T17:31:41.124315+0000","last_became_peered":"2024-07-04T17:31:41.124315+0000","last_unstale":"2024-07-04T17:32:03.993143+0000","last_undegraded":"2024-07-04T17:32:03.993143+0000","last_fullsized":"2024-07-04T17:32:03.993143+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:10:03.734241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.992466+0000","last_change":"2024-07-04T17:31:42.111629+0000","last_active":"2024-07-04T17:32:03.992466+0000","last_peered":"2024-07-04T17:32:03.992466+0000","last_clean":"2024-07-04T17:32:03.992466+0000","last_became_active":"2024-07-04T17:31:42.110978+0000","last_became_peered":"2024-07-04T17:31:42.110978+0000","last_unstale":"2024-07-04T17:32:03.992466+0000","last_undegraded":"2024-07-04T17:32:03.992466+0000","last_fullsized":"2024-07-04T17:32:03.992466+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:12:01.825579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.986994+0000","last_change":"2024-07-04T17:32:01.586152+0000","last_active":"2024-07-04T17:32:03.986994+0000","last_peered":"2024-07-04T17:32:03.986994+0000","last_clean":"2024-07-04T17:32:03.986994+0000","last_became_active":"2024-07-04T17:32:01.583465+0000","last_became_peered":"2024-07-04T17:32:01.583465+0000","last_unstale":"2024-07-04T17:32:03.986994+0000","last_undegraded":"2024-07-04T17:32:03.986994+0000","last_fullsized":"2024-07-04T17:32:03.986994+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:38:19.387018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.571065+0000","last_change":"2024-07-04T17:32:02.583838+0000","last_active":"2024-07-04T17:32:03.571065+0000","last_peered":"2024-07-04T17:32:03.571065+0000","last_clean":"2024-07-04T17:32:03.571065+0000","last_became_active":"2024-07-04T17:32:02.582748+0000","last_became_peered":"2024-07-04T17:32:02.582748+0000","last_unstale":"2024-07-04T17:32:03.571065+0000","last_undegraded":"2024-07-04T17:32:03.571065+0000","last_fullsized":"2024-07-04T17:32:03.571065+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:31:30.629913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.987969+0000","last_change":"2024-07-04T17:31:41.128454+0000","last_active":"2024-07-04T17:32:03.987969+0000","last_peered":"2024-07-04T17:32:03.987969+0000","last_clean":"2024-07-04T17:32:03.987969+0000","last_became_active":"2024-07-04T17:31:41.127785+0000","last_became_peered":"2024-07-04T17:31:41.127785+0000","last_unstale":"2024-07-04T17:32:03.987969+0000","last_undegraded":"2024-07-04T17:32:03.987969+0000","last_fullsized":"2024-07-04T17:32:03.987969+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:48:22.712062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.985890+0000","last_change":"2024-07-04T17:31:42.125396+0000","last_active":"2024-07-04T17:32:03.985890+0000","last_peered":"2024-07-04T17:32:03.985890+0000","last_clean":"2024-07-04T17:32:03.985890+0000","last_became_active":"2024-07-04T17:31:42.123980+0000","last_became_peered":"2024-07-04T17:31:42.123980+0000","last_unstale":"2024-07-04T17:32:03.985890+0000","last_undegraded":"2024-07-04T17:32:03.985890+0000","last_fullsized":"2024-07-04T17:32:03.985890+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:59:35.007753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.990654+0000","last_change":"2024-07-04T17:32:01.581492+0000","last_active":"2024-07-04T17:32:03.990654+0000","last_peered":"2024-07-04T17:32:03.990654+0000","last_clean":"2024-07-04T17:32:03.990654+0000","last_became_active":"2024-07-04T17:32:01.580962+0000","last_became_peered":"2024-07-04T17:32:01.580962+0000","last_unstale":"2024-07-04T17:32:03.990654+0000","last_undegraded":"2024-07-04T17:32:03.990654+0000","last_fullsized":"2024-07-04T17:32:03.990654+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:49:28.365441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.575945+0000","last_change":"2024-07-04T17:32:02.601444+0000","last_active":"2024-07-04T17:32:03.575945+0000","last_peered":"2024-07-04T17:32:03.575945+0000","last_clean":"2024-07-04T17:32:03.575945+0000","last_became_active":"2024-07-04T17:32:02.601078+0000","last_became_peered":"2024-07-04T17:32:02.601078+0000","last_unstale":"2024-07-04T17:32:03.575945+0000","last_undegraded":"2024-07-04T17:32:03.575945+0000","last_fullsized":"2024-07-04T17:32:03.575945+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:19:40.704357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.574906+0000","last_change":"2024-07-04T17:31:41.119885+0000","last_active":"2024-07-04T17:32:03.574906+0000","last_peered":"2024-07-04T17:32:03.574906+0000","last_clean":"2024-07-04T17:32:03.574906+0000","last_became_active":"2024-07-04T17:31:41.116720+0000","last_became_peered":"2024-07-04T17:31:41.116720+0000","last_unstale":"2024-07-04T17:32:03.574906+0000","last_undegraded":"2024-07-04T17:32:03.574906+0000","last_fullsized":"2024-07-04T17:32:03.574906+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:30:49.905769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.994142+0000","last_change":"2024-07-04T17:31:42.109920+0000","last_active":"2024-07-04T17:32:03.994142+0000","last_peered":"2024-07-04T17:32:03.994142+0000","last_clean":"2024-07-04T17:32:03.994142+0000","last_became_active":"2024-07-04T17:31:42.109357+0000","last_became_peered":"2024-07-04T17:31:42.109357+0000","last_unstale":"2024-07-04T17:32:03.994142+0000","last_undegraded":"2024-07-04T17:32:03.994142+0000","last_fullsized":"2024-07-04T17:32:03.994142+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:58:16.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.6","version":"96'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:04.558806+0000","last_change":"2024-07-04T17:32:01.595020+0000","last_active":"2024-07-04T17:32:04.558806+0000","last_peered":"2024-07-04T17:32:04.558806+0000","last_clean":"2024-07-04T17:32:04.558806+0000","last_became_active":"2024-07-04T17:32:01.594717+0000","last_became_peered":"2024-07-04T17:32:01.594717+0000","last_unstale":"2024-07-04T17:32:04.558806+0000","last_undegraded":"2024-07-04T17:32:04.558806+0000","last_fullsized":"2024-07-04T17:32:04.558806+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:58:30.462288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.995599+0000","last_change":"2024-07-04T17:32:02.617612+0000","last_active":"2024-07-04T17:32:03.995599+0000","last_peered":"2024-07-04T17:32:03.995599+0000","last_clean":"2024-07-04T17:32:03.995599+0000","last_became_active":"2024-07-04T17:32:02.615735+0000","last_became_peered":"2024-07-04T17:32:02.615735+0000","last_unstale":"2024-07-04T17:32:03.995599+0000","last_undegraded":"2024-07-04T17:32:03.995599+0000","last_fullsized":"2024-07-04T17:32:03.995599+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:31:30.629913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.995253+0000","last_change":"2024-07-04T17:31:41.123150+0000","last_active":"2024-07-04T17:32:03.995253+0000","last_peered":"2024-07-04T17:32:03.995253+0000","last_clean":"2024-07-04T17:32:03.995253+0000","last_became_active":"2024-07-04T17:31:41.120767+0000","last_became_peered":"2024-07-04T17:31:41.120767+0000","last_unstale":"2024-07-04T17:32:03.995253+0000","last_undegraded":"2024-07-04T17:32:03.995253+0000","last_fullsized":"2024-07-04T17:32:03.995253+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:42:19.857048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.992454+0000","last_change":"2024-07-04T17:31:42.143202+0000","last_active":"2024-07-04T17:32:03.992454+0000","last_peered":"2024-07-04T17:32:03.992454+0000","last_clean":"2024-07-04T17:32:03.992454+0000","last_became_active":"2024-07-04T17:31:42.141656+0000","last_became_peered":"2024-07-04T17:31:42.141656+0000","last_unstale":"2024-07-04T17:32:03.992454+0000","last_undegraded":"2024-07-04T17:32:03.992454+0000","last_fullsized":"2024-07-04T17:32:03.992454+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:19:20.240973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.7","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.576045+0000","last_change":"2024-07-04T17:32:01.598772+0000","last_active":"2024-07-04T17:32:03.576045+0000","last_peered":"2024-07-04T17:32:03.576045+0000","last_clean":"2024-07-04T17:32:03.576045+0000","last_became_active":"2024-07-04T17:32:01.597904+0000","last_became_peered":"2024-07-04T17:32:01.597904+0000","last_unstale":"2024-07-04T17:32:03.576045+0000","last_undegraded":"2024-07-04T17:32:03.576045+0000","last_fullsized":"2024-07-04T17:32:03.576045+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:20:19.495765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.993364+0000","last_change":"2024-07-04T17:32:02.602505+0000","last_active":"2024-07-04T17:32:03.993364+0000","last_peered":"2024-07-04T17:32:03.993364+0000","last_clean":"2024-07-04T17:32:03.993364+0000","last_became_active":"2024-07-04T17:32:02.601949+0000","last_became_peered":"2024-07-04T17:32:02.601949+0000","last_unstale":"2024-07-04T17:32:03.993364+0000","last_undegraded":"2024-07-04T17:32:03.993364+0000","last_fullsized":"2024-07-04T17:32:03.993364+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:36:16.683047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"92'1","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.996395+0000","last_change":"2024-07-04T17:31:41.126383+0000","last_active":"2024-07-04T17:32:03.996395+0000","last_peered":"2024-07-04T17:32:03.996395+0000","last_clean":"2024-07-04T17:32:03.996395+0000","last_became_active":"2024-07-04T17:31:41.126045+0000","last_became_peered":"2024-07-04T17:31:41.126045+0000","last_unstale":"2024-07-04T17:32:03.996395+0000","last_undegraded":"2024-07-04T17:32:03.996395+0000","last_fullsized":"2024-07-04T17:32:03.996395+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:57:59.125923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.996117+0000","last_change":"2024-07-04T17:31:42.107634+0000","last_active":"2024-07-04T17:32:03.996117+0000","last_peered":"2024-07-04T17:32:03.996117+0000","last_clean":"2024-07-04T17:32:03.996117+0000","last_became_active":"2024-07-04T17:31:42.107327+0000","last_became_peered":"2024-07-04T17:31:42.107327+0000","last_unstale":"2024-07-04T17:32:03.996117+0000","last_undegraded":"2024-07-04T17:32:03.996117+0000","last_fullsized":"2024-07-04T17:32:03.996117+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:57:08.828739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.992050+0000","last_change":"2024-07-04T17:32:01.575457+0000","last_active":"2024-07-04T17:32:03.992050+0000","last_peered":"2024-07-04T17:32:03.992050+0000","last_clean":"2024-07-04T17:32:03.992050+0000","last_became_active":"2024-07-04T17:32:01.575101+0000","last_became_peered":"2024-07-04T17:32:01.575101+0000","last_unstale":"2024-07-04T17:32:03.992050+0000","last_undegraded":"2024-07-04T17:32:03.992050+0000","last_fullsized":"2024-07-04T17:32:03.992050+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:48:06.134693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.988118+0000","last_change":"2024-07-04T17:32:02.595302+0000","last_active":"2024-07-04T17:32:03.988118+0000","last_peered":"2024-07-04T17:32:03.988118+0000","last_clean":"2024-07-04T17:32:03.988118+0000","last_became_active":"2024-07-04T17:32:02.592327+0000","last_became_peered":"2024-07-04T17:32:02.592327+0000","last_unstale":"2024-07-04T17:32:03.988118+0000","last_undegraded":"2024-07-04T17:32:03.988118+0000","last_fullsized":"2024-07-04T17:32:03.988118+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:18:39.023689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"91'2","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.996018+0000","last_change":"2024-07-04T17:31:41.126950+0000","last_active":"2024-07-04T17:32:03.996018+0000","last_peered":"2024-07-04T17:32:03.996018+0000","last_clean":"2024-07-04T17:32:03.996018+0000","last_became_active":"2024-07-04T17:31:41.126611+0000","last_became_peered":"2024-07-04T17:31:41.126611+0000","last_unstale":"2024-07-04T17:32:03.996018+0000","last_undegraded":"2024-07-04T17:32:03.996018+0000","last_fullsized":"2024-07-04T17:32:03.996018+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:27:51.965765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.996277+0000","last_change":"2024-07-04T17:31:42.142194+0000","last_active":"2024-07-04T17:32:03.996277+0000","last_peered":"2024-07-04T17:32:03.996277+0000","last_clean":"2024-07-04T17:32:03.996277+0000","last_became_active":"2024-07-04T17:31:42.139517+0000","last_became_peered":"2024-07-04T17:31:42.139517+0000","last_unstale":"2024-07-04T17:32:03.996277+0000","last_undegraded":"2024-07-04T17:32:03.996277+0000","last_fullsized":"2024-07-04T17:32:03.996277+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:12:09.965552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.994959+0000","last_change":"2024-07-04T17:32:01.588031+0000","last_active":"2024-07-04T17:32:03.994959+0000","last_peered":"2024-07-04T17:32:03.994959+0000","last_clean":"2024-07-04T17:32:03.994959+0000","last_became_active":"2024-07-04T17:32:01.586551+0000","last_became_peered":"2024-07-04T17:32:01.586551+0000","last_unstale":"2024-07-04T17:32:03.994959+0000","last_undegraded":"2024-07-04T17:32:03.994959+0000","last_fullsized":"2024-07-04T17:32:03.994959+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:10:04.034800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.985418+0000","last_change":"2024-07-04T17:32:02.615560+0000","last_active":"2024-07-04T17:32:03.985418+0000","last_peered":"2024-07-04T17:32:03.985418+0000","last_clean":"2024-07-04T17:32:03.985418+0000","last_became_active":"2024-07-04T17:32:02.615207+0000","last_became_peered":"2024-07-04T17:32:02.615207+0000","last_unstale":"2024-07-04T17:32:03.985418+0000","last_undegraded":"2024-07-04T17:32:03.985418+0000","last_fullsized":"2024-07-04T17:32:03.985418+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:14:08.180824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.988086+0000","last_change":"2024-07-04T17:31:41.114344+0000","last_active":"2024-07-04T17:32:03.988086+0000","last_peered":"2024-07-04T17:32:03.988086+0000","last_clean":"2024-07-04T17:32:03.988086+0000","last_became_active":"2024-07-04T17:31:41.114006+0000","last_became_peered":"2024-07-04T17:31:41.114006+0000","last_unstale":"2024-07-04T17:32:03.988086+0000","last_undegraded":"2024-07-04T17:32:03.988086+0000","last_fullsized":"2024-07-04T17:32:03.988086+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:57:55.654579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.985032+0000","last_change":"2024-07-04T17:31:42.124705+0000","last_active":"2024-07-04T17:32:03.985032+0000","last_peered":"2024-07-04T17:32:03.985032+0000","last_clean":"2024-07-04T17:32:03.985032+0000","last_became_active":"2024-07-04T17:31:42.124266+0000","last_became_peered":"2024-07-04T17:31:42.124266+0000","last_unstale":"2024-07-04T17:32:03.985032+0000","last_undegraded":"2024-07-04T17:32:03.985032+0000","last_fullsized":"2024-07-04T17:32:03.985032+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:18:01.138996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.572374+0000","last_change":"2024-07-04T17:32:01.591822+0000","last_active":"2024-07-04T17:32:03.572374+0000","last_peered":"2024-07-04T17:32:03.572374+0000","last_clean":"2024-07-04T17:32:03.572374+0000","last_became_active":"2024-07-04T17:32:01.591408+0000","last_became_peered":"2024-07-04T17:32:01.591408+0000","last_unstale":"2024-07-04T17:32:03.572374+0000","last_undegraded":"2024-07-04T17:32:03.572374+0000","last_fullsized":"2024-07-04T17:32:03.572374+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:03:11.489221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.571952+0000","last_change":"2024-07-04T17:32:02.606095+0000","last_active":"2024-07-04T17:32:03.571952+0000","last_peered":"2024-07-04T17:32:03.571952+0000","last_clean":"2024-07-04T17:32:03.571952+0000","last_became_active":"2024-07-04T17:32:02.604978+0000","last_became_peered":"2024-07-04T17:32:02.604978+0000","last_unstale":"2024-07-04T17:32:03.571952+0000","last_undegraded":"2024-07-04T17:32:03.571952+0000","last_fullsized":"2024-07-04T17:32:03.571952+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:11:30.547164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.985196+0000","last_change":"2024-07-04T17:31:41.120856+0000","last_active":"2024-07-04T17:32:03.985196+0000","last_peered":"2024-07-04T17:32:03.985196+0000","last_clean":"2024-07-04T17:32:03.985196+0000","last_became_active":"2024-07-04T17:31:41.120338+0000","last_became_peered":"2024-07-04T17:31:41.120338+0000","last_unstale":"2024-07-04T17:32:03.985196+0000","last_undegraded":"2024-07-04T17:32:03.985196+0000","last_fullsized":"2024-07-04T17:32:03.985196+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:14:01.301278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.987538+0000","last_change":"2024-07-04T17:31:42.111799+0000","last_active":"2024-07-04T17:32:03.987538+0000","last_peered":"2024-07-04T17:32:03.987538+0000","last_clean":"2024-07-04T17:32:03.987538+0000","last_became_active":"2024-07-04T17:31:42.110114+0000","last_became_peered":"2024-07-04T17:31:42.110114+0000","last_unstale":"2024-07-04T17:32:03.987538+0000","last_undegraded":"2024-07-04T17:32:03.987538+0000","last_fullsized":"2024-07-04T17:32:03.987538+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:45:25.449722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.b","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.985605+0000","last_change":"2024-07-04T17:32:01.596098+0000","last_active":"2024-07-04T17:32:03.985605+0000","last_peered":"2024-07-04T17:32:03.985605+0000","last_clean":"2024-07-04T17:32:03.985605+0000","last_became_active":"2024-07-04T17:32:01.593474+0000","last_became_peered":"2024-07-04T17:32:01.593474+0000","last_unstale":"2024-07-04T17:32:03.985605+0000","last_undegraded":"2024-07-04T17:32:03.985605+0000","last_fullsized":"2024-07-04T17:32:03.985605+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:58:36.722442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.988182+0000","last_change":"2024-07-04T17:32:02.580255+0000","last_active":"2024-07-04T17:32:03.988182+0000","last_peered":"2024-07-04T17:32:03.988182+0000","last_clean":"2024-07-04T17:32:03.988182+0000","last_became_active":"2024-07-04T17:32:02.579908+0000","last_became_peered":"2024-07-04T17:32:02.579908+0000","last_unstale":"2024-07-04T17:32:03.988182+0000","last_undegraded":"2024-07-04T17:32:03.988182+0000","last_fullsized":"2024-07-04T17:32:03.988182+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:04:12.247110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.573095+0000","last_change":"2024-07-04T17:31:41.131853+0000","last_active":"2024-07-04T17:32:03.573095+0000","last_peered":"2024-07-04T17:32:03.573095+0000","last_clean":"2024-07-04T17:32:03.573095+0000","last_became_active":"2024-07-04T17:31:41.129778+0000","last_became_peered":"2024-07-04T17:31:41.129778+0000","last_unstale":"2024-07-04T17:32:03.573095+0000","last_undegraded":"2024-07-04T17:32:03.573095+0000","last_fullsized":"2024-07-04T17:32:03.573095+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:59:40.102134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.992145+0000","last_change":"2024-07-04T17:31:42.148767+0000","last_active":"2024-07-04T17:32:03.992145+0000","last_peered":"2024-07-04T17:32:03.992145+0000","last_clean":"2024-07-04T17:32:03.992145+0000","last_became_active":"2024-07-04T17:31:42.148091+0000","last_became_peered":"2024-07-04T17:31:42.148091+0000","last_unstale":"2024-07-04T17:32:03.992145+0000","last_undegraded":"2024-07-04T17:32:03.992145+0000","last_fullsized":"2024-07-04T17:32:03.992145+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:42:04.145597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.986868+0000","last_change":"2024-07-04T17:32:01.584737+0000","last_active":"2024-07-04T17:32:03.986868+0000","last_peered":"2024-07-04T17:32:03.986868+0000","last_clean":"2024-07-04T17:32:03.986868+0000","last_became_active":"2024-07-04T17:32:01.583883+0000","last_became_peered":"2024-07-04T17:32:01.583883+0000","last_unstale":"2024-07-04T17:32:03.986868+0000","last_undegraded":"2024-07-04T17:32:03.986868+0000","last_fullsized":"2024-07-04T17:32:03.986868+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:09:42.708030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.983928+0000","last_change":"2024-07-04T17:32:02.610353+0000","last_active":"2024-07-04T17:32:03.983928+0000","last_peered":"2024-07-04T17:32:03.983928+0000","last_clean":"2024-07-04T17:32:03.983928+0000","last_became_active":"2024-07-04T17:32:02.609997+0000","last_became_peered":"2024-07-04T17:32:02.609997+0000","last_unstale":"2024-07-04T17:32:03.983928+0000","last_undegraded":"2024-07-04T17:32:03.983928+0000","last_fullsized":"2024-07-04T17:32:03.983928+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:45:01.615739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"91'2","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.987617+0000","last_change":"2024-07-04T17:31:41.128172+0000","last_active":"2024-07-04T17:32:03.987617+0000","last_peered":"2024-07-04T17:32:03.987617+0000","last_clean":"2024-07-04T17:32:03.987617+0000","last_became_active":"2024-07-04T17:31:41.127350+0000","last_became_peered":"2024-07-04T17:31:41.127350+0000","last_unstale":"2024-07-04T17:32:03.987617+0000","last_undegraded":"2024-07-04T17:32:03.987617+0000","last_fullsized":"2024-07-04T17:32:03.987617+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:24:57.684170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.985670+0000","last_change":"2024-07-04T17:31:42.144839+0000","last_active":"2024-07-04T17:32:03.985670+0000","last_peered":"2024-07-04T17:32:03.985670+0000","last_clean":"2024-07-04T17:32:03.985670+0000","last_became_active":"2024-07-04T17:31:42.142261+0000","last_became_peered":"2024-07-04T17:31:42.142261+0000","last_unstale":"2024-07-04T17:32:03.985670+0000","last_undegraded":"2024-07-04T17:32:03.985670+0000","last_fullsized":"2024-07-04T17:32:03.985670+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:44:20.205264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.571259+0000","last_change":"2024-07-04T17:32:01.594923+0000","last_active":"2024-07-04T17:32:03.571259+0000","last_peered":"2024-07-04T17:32:03.571259+0000","last_clean":"2024-07-04T17:32:03.571259+0000","last_became_active":"2024-07-04T17:32:01.594612+0000","last_became_peered":"2024-07-04T17:32:01.594612+0000","last_unstale":"2024-07-04T17:32:03.571259+0000","last_undegraded":"2024-07-04T17:32:03.571259+0000","last_fullsized":"2024-07-04T17:32:03.571259+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:05:47.602368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.575782+0000","last_change":"2024-07-04T17:32:02.609748+0000","last_active":"2024-07-04T17:32:03.575782+0000","last_peered":"2024-07-04T17:32:03.575782+0000","last_clean":"2024-07-04T17:32:03.575782+0000","last_became_active":"2024-07-04T17:32:02.603472+0000","last_became_peered":"2024-07-04T17:32:02.603472+0000","last_unstale":"2024-07-04T17:32:03.575782+0000","last_undegraded":"2024-07-04T17:32:03.575782+0000","last_fullsized":"2024-07-04T17:32:03.575782+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:30:47.350386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.992609+0000","last_change":"2024-07-04T17:31:41.111490+0000","last_active":"2024-07-04T17:32:03.992609+0000","last_peered":"2024-07-04T17:32:03.992609+0000","last_clean":"2024-07-04T17:32:03.992609+0000","last_became_active":"2024-07-04T17:31:41.110881+0000","last_became_peered":"2024-07-04T17:31:41.110881+0000","last_unstale":"2024-07-04T17:32:03.992609+0000","last_undegraded":"2024-07-04T17:32:03.992609+0000","last_fullsized":"2024-07-04T17:32:03.992609+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:44:12.481565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.993989+0000","last_change":"2024-07-04T17:31:42.138658+0000","last_active":"2024-07-04T17:32:03.993989+0000","last_peered":"2024-07-04T17:32:03.993989+0000","last_clean":"2024-07-04T17:32:03.993989+0000","last_became_active":"2024-07-04T17:31:42.137836+0000","last_became_peered":"2024-07-04T17:31:42.137836+0000","last_unstale":"2024-07-04T17:32:03.993989+0000","last_undegraded":"2024-07-04T17:32:03.993989+0000","last_fullsized":"2024-07-04T17:32:03.993989+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:32:01.472783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.e","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.995348+0000","last_change":"2024-07-04T17:32:01.580300+0000","last_active":"2024-07-04T17:32:03.995348+0000","last_peered":"2024-07-04T17:32:03.995348+0000","last_clean":"2024-07-04T17:32:03.995348+0000","last_became_active":"2024-07-04T17:32:01.579643+0000","last_became_peered":"2024-07-04T17:32:01.579643+0000","last_unstale":"2024-07-04T17:32:03.995348+0000","last_undegraded":"2024-07-04T17:32:03.995348+0000","last_fullsized":"2024-07-04T17:32:03.995348+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:26:23.623930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.574871+0000","last_change":"2024-07-04T17:32:02.606822+0000","last_active":"2024-07-04T17:32:03.574871+0000","last_peered":"2024-07-04T17:32:03.574871+0000","last_clean":"2024-07-04T17:32:03.574871+0000","last_became_active":"2024-07-04T17:32:02.606506+0000","last_became_peered":"2024-07-04T17:32:02.606506+0000","last_unstale":"2024-07-04T17:32:03.574871+0000","last_undegraded":"2024-07-04T17:32:03.574871+0000","last_fullsized":"2024-07-04T17:32:03.574871+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:20:07.552973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.d","version":"91'1","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.994931+0000","last_change":"2024-07-04T17:31:41.125962+0000","last_active":"2024-07-04T17:32:03.994931+0000","last_peered":"2024-07-04T17:32:03.994931+0000","last_clean":"2024-07-04T17:32:03.994931+0000","last_became_active":"2024-07-04T17:31:41.121577+0000","last_became_peered":"2024-07-04T17:31:41.121577+0000","last_unstale":"2024-07-04T17:32:03.994931+0000","last_undegraded":"2024-07-04T17:32:03.994931+0000","last_fullsized":"2024-07-04T17:32:03.994931+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:02:24.336737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.992335+0000","last_change":"2024-07-04T17:31:42.138910+0000","last_active":"2024-07-04T17:32:03.992335+0000","last_peered":"2024-07-04T17:32:03.992335+0000","last_clean":"2024-07-04T17:32:03.992335+0000","last_became_active":"2024-07-04T17:31:42.137936+0000","last_became_peered":"2024-07-04T17:31:42.137936+0000","last_unstale":"2024-07-04T17:32:03.992335+0000","last_undegraded":"2024-07-04T17:32:03.992335+0000","last_fullsized":"2024-07-04T17:32:03.992335+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:30:17.422306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.f","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.993953+0000","last_change":"2024-07-04T17:32:01.573537+0000","last_active":"2024-07-04T17:32:03.993953+0000","last_peered":"2024-07-04T17:32:03.993953+0000","last_clean":"2024-07-04T17:32:03.993953+0000","last_became_active":"2024-07-04T17:32:01.573178+0000","last_became_peered":"2024-07-04T17:32:01.573178+0000","last_unstale":"2024-07-04T17:32:03.993953+0000","last_undegraded":"2024-07-04T17:32:03.993953+0000","last_fullsized":"2024-07-04T17:32:03.993953+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:57:53.026950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.990328+0000","last_change":"2024-07-04T17:32:02.582966+0000","last_active":"2024-07-04T17:32:03.990328+0000","last_peered":"2024-07-04T17:32:03.990328+0000","last_clean":"2024-07-04T17:32:03.990328+0000","last_became_active":"2024-07-04T17:32:02.582488+0000","last_became_peered":"2024-07-04T17:32:02.582488+0000","last_unstale":"2024-07-04T17:32:03.990328+0000","last_undegraded":"2024-07-04T17:32:03.990328+0000","last_fullsized":"2024-07-04T17:32:03.990328+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:13:26.272163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"91'2","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.575949+0000","last_change":"2024-07-04T17:31:41.127224+0000","last_active":"2024-07-04T17:32:03.575949+0000","last_peered":"2024-07-04T17:32:03.575949+0000","last_clean":"2024-07-04T17:32:03.575949+0000","last_became_active":"2024-07-04T17:31:41.126878+0000","last_became_peered":"2024-07-04T17:31:41.126878+0000","last_unstale":"2024-07-04T17:32:03.575949+0000","last_undegraded":"2024-07-04T17:32:03.575949+0000","last_fullsized":"2024-07-04T17:32:03.575949+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:17:12.761795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.995715+0000","last_change":"2024-07-04T17:31:42.127586+0000","last_active":"2024-07-04T17:32:03.995715+0000","last_peered":"2024-07-04T17:32:03.995715+0000","last_clean":"2024-07-04T17:32:03.995715+0000","last_became_active":"2024-07-04T17:31:42.127263+0000","last_became_peered":"2024-07-04T17:31:42.127263+0000","last_unstale":"2024-07-04T17:32:03.995715+0000","last_undegraded":"2024-07-04T17:32:03.995715+0000","last_fullsized":"2024-07-04T17:32:03.995715+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:54:41.368680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.574182+0000","last_change":"2024-07-04T17:32:01.595196+0000","last_active":"2024-07-04T17:32:03.574182+0000","last_peered":"2024-07-04T17:32:03.574182+0000","last_clean":"2024-07-04T17:32:03.574182+0000","last_became_active":"2024-07-04T17:32:01.594870+0000","last_became_peered":"2024-07-04T17:32:01.594870+0000","last_unstale":"2024-07-04T17:32:03.574182+0000","last_undegraded":"2024-07-04T17:32:03.574182+0000","last_fullsized":"2024-07-04T17:32:03.574182+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:23:34.925305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.991947+0000","last_change":"2024-07-04T17:32:02.576103+0000","last_active":"2024-07-04T17:32:03.991947+0000","last_peered":"2024-07-04T17:32:03.991947+0000","last_clean":"2024-07-04T17:32:03.991947+0000","last_became_active":"2024-07-04T17:32:02.575604+0000","last_became_peered":"2024-07-04T17:32:02.575604+0000","last_unstale":"2024-07-04T17:32:03.991947+0000","last_undegraded":"2024-07-04T17:32:03.991947+0000","last_fullsized":"2024-07-04T17:32:03.991947+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:36:57.937525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.f","version":"91'1","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.997132+0000","last_change":"2024-07-04T17:31:41.110852+0000","last_active":"2024-07-04T17:32:03.997132+0000","last_peered":"2024-07-04T17:32:03.997132+0000","last_clean":"2024-07-04T17:32:03.997132+0000","last_became_active":"2024-07-04T17:31:41.109092+0000","last_became_peered":"2024-07-04T17:31:41.109092+0000","last_unstale":"2024-07-04T17:32:03.997132+0000","last_undegraded":"2024-07-04T17:32:03.997132+0000","last_fullsized":"2024-07-04T17:32:03.997132+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:47:10.505606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.575578+0000","last_change":"2024-07-04T17:31:42.117660+0000","last_active":"2024-07-04T17:32:03.575578+0000","last_peered":"2024-07-04T17:32:03.575578+0000","last_clean":"2024-07-04T17:32:03.575578+0000","last_became_active":"2024-07-04T17:31:42.115485+0000","last_became_peered":"2024-07-04T17:31:42.115485+0000","last_unstale":"2024-07-04T17:32:03.575578+0000","last_undegraded":"2024-07-04T17:32:03.575578+0000","last_fullsized":"2024-07-04T17:32:03.575578+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:10:45.367563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.d","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.993656+0000","last_change":"2024-07-04T17:32:01.570202+0000","last_active":"2024-07-04T17:32:03.993656+0000","last_peered":"2024-07-04T17:32:03.993656+0000","last_clean":"2024-07-04T17:32:03.993656+0000","last_became_active":"2024-07-04T17:32:01.569874+0000","last_became_peered":"2024-07-04T17:32:01.569874+0000","last_unstale":"2024-07-04T17:32:03.993656+0000","last_undegraded":"2024-07-04T17:32:03.993656+0000","last_fullsized":"2024-07-04T17:32:03.993656+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:50:25.143757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.573842+0000","last_change":"2024-07-04T17:32:02.610260+0000","last_active":"2024-07-04T17:32:03.573842+0000","last_peered":"2024-07-04T17:32:03.573842+0000","last_clean":"2024-07-04T17:32:03.573842+0000","last_became_active":"2024-07-04T17:32:02.609527+0000","last_became_peered":"2024-07-04T17:32:02.609527+0000","last_unstale":"2024-07-04T17:32:03.573842+0000","last_undegraded":"2024-07-04T17:32:03.573842+0000","last_fullsized":"2024-07-04T17:32:03.573842+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:12:46.495414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.10","version":"91'8","reported_seq":30,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.572232+0000","last_change":"2024-07-04T17:31:41.123889+0000","last_active":"2024-07-04T17:32:03.572232+0000","last_peered":"2024-07-04T17:32:03.572232+0000","last_clean":"2024-07-04T17:32:03.572232+0000","last_became_active":"2024-07-04T17:31:41.123555+0000","last_became_peered":"2024-07-04T17:31:41.123555+0000","last_unstale":"2024-07-04T17:32:03.572232+0000","last_undegraded":"2024-07-04T17:32:03.572232+0000","last_fullsized":"2024-07-04T17:32:03.572232+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:07:24.656814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.984838+0000","last_change":"2024-07-04T17:31:42.134513+0000","last_active":"2024-07-04T17:32:03.984838+0000","last_peered":"2024-07-04T17:32:03.984838+0000","last_clean":"2024-07-04T17:32:03.984838+0000","last_became_active":"2024-07-04T17:31:42.134166+0000","last_became_peered":"2024-07-04T17:31:42.134166+0000","last_unstale":"2024-07-04T17:32:03.984838+0000","last_undegraded":"2024-07-04T17:32:03.984838+0000","last_fullsized":"2024-07-04T17:32:03.984838+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:17:13.766129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.572180+0000","last_change":"2024-07-04T17:32:01.583168+0000","last_active":"2024-07-04T17:32:03.572180+0000","last_peered":"2024-07-04T17:32:03.572180+0000","last_clean":"2024-07-04T17:32:03.572180+0000","last_became_active":"2024-07-04T17:32:01.582859+0000","last_became_peered":"2024-07-04T17:32:01.582859+0000","last_unstale":"2024-07-04T17:32:03.572180+0000","last_undegraded":"2024-07-04T17:32:03.572180+0000","last_fullsized":"2024-07-04T17:32:03.572180+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:47:23.988157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.991460+0000","last_change":"2024-07-04T17:32:02.624956+0000","last_active":"2024-07-04T17:32:03.991460+0000","last_peered":"2024-07-04T17:32:03.991460+0000","last_clean":"2024-07-04T17:32:03.991460+0000","last_became_active":"2024-07-04T17:32:02.624404+0000","last_became_peered":"2024-07-04T17:32:02.624404+0000","last_unstale":"2024-07-04T17:32:03.991460+0000","last_undegraded":"2024-07-04T17:32:03.991460+0000","last_fullsized":"2024-07-04T17:32:03.991460+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:51:43.902906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.987165+0000","last_change":"2024-07-04T17:31:41.126486+0000","last_active":"2024-07-04T17:32:03.987165+0000","last_peered":"2024-07-04T17:32:03.987165+0000","last_clean":"2024-07-04T17:32:03.987165+0000","last_became_active":"2024-07-04T17:31:41.126142+0000","last_became_peered":"2024-07-04T17:31:41.126142+0000","last_unstale":"2024-07-04T17:32:03.987165+0000","last_undegraded":"2024-07-04T17:32:03.987165+0000","last_fullsized":"2024-07-04T17:32:03.987165+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:25:18.918225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.991833+0000","last_change":"2024-07-04T17:31:42.111369+0000","last_active":"2024-07-04T17:32:03.991833+0000","last_peered":"2024-07-04T17:32:03.991833+0000","last_clean":"2024-07-04T17:32:03.991833+0000","last_became_active":"2024-07-04T17:31:42.110632+0000","last_became_peered":"2024-07-04T17:31:42.110632+0000","last_unstale":"2024-07-04T17:32:03.991833+0000","last_undegraded":"2024-07-04T17:32:03.991833+0000","last_fullsized":"2024-07-04T17:32:03.991833+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:31:20.968644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"95'8","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.986817+0000","last_change":"2024-07-04T17:32:01.585065+0000","last_active":"2024-07-04T17:32:03.986817+0000","last_peered":"2024-07-04T17:32:03.986817+0000","last_clean":"2024-07-04T17:32:03.986817+0000","last_became_active":"2024-07-04T17:32:01.584410+0000","last_became_peered":"2024-07-04T17:32:01.584410+0000","last_unstale":"2024-07-04T17:32:03.986817+0000","last_undegraded":"2024-07-04T17:32:03.986817+0000","last_fullsized":"2024-07-04T17:32:03.986817+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:38:45.000810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.983677+0000","last_change":"2024-07-04T17:32:02.609945+0000","last_active":"2024-07-04T17:32:03.983677+0000","last_peered":"2024-07-04T17:32:03.983677+0000","last_clean":"2024-07-04T17:32:03.983677+0000","last_became_active":"2024-07-04T17:32:02.609574+0000","last_became_peered":"2024-07-04T17:32:02.609574+0000","last_unstale":"2024-07-04T17:32:03.983677+0000","last_undegraded":"2024-07-04T17:32:03.983677+0000","last_fullsized":"2024-07-04T17:32:03.983677+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:37:21.425873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"91'2","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.987905+0000","last_change":"2024-07-04T17:31:41.128711+0000","last_active":"2024-07-04T17:32:03.987905+0000","last_peered":"2024-07-04T17:32:03.987905+0000","last_clean":"2024-07-04T17:32:03.987905+0000","last_became_active":"2024-07-04T17:31:41.127816+0000","last_became_peered":"2024-07-04T17:31:41.127816+0000","last_unstale":"2024-07-04T17:32:03.987905+0000","last_undegraded":"2024-07-04T17:32:03.987905+0000","last_fullsized":"2024-07-04T17:32:03.987905+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:37:51.190424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.572804+0000","last_change":"2024-07-04T17:31:42.116278+0000","last_active":"2024-07-04T17:32:03.572804+0000","last_peered":"2024-07-04T17:32:03.572804+0000","last_clean":"2024-07-04T17:32:03.572804+0000","last_became_active":"2024-07-04T17:31:42.115861+0000","last_became_peered":"2024-07-04T17:31:42.115861+0000","last_unstale":"2024-07-04T17:32:03.572804+0000","last_undegraded":"2024-07-04T17:32:03.572804+0000","last_fullsized":"2024-07-04T17:32:03.572804+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:43:17.950207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.984363+0000","last_change":"2024-07-04T17:32:01.582267+0000","last_active":"2024-07-04T17:32:03.984363+0000","last_peered":"2024-07-04T17:32:03.984363+0000","last_clean":"2024-07-04T17:32:03.984363+0000","last_became_active":"2024-07-04T17:32:01.581942+0000","last_became_peered":"2024-07-04T17:32:01.581942+0000","last_unstale":"2024-07-04T17:32:03.984363+0000","last_undegraded":"2024-07-04T17:32:03.984363+0000","last_fullsized":"2024-07-04T17:32:03.984363+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:18:53.810731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.575306+0000","last_change":"2024-07-04T17:32:02.600700+0000","last_active":"2024-07-04T17:32:03.575306+0000","last_peered":"2024-07-04T17:32:03.575306+0000","last_clean":"2024-07-04T17:32:03.575306+0000","last_became_active":"2024-07-04T17:32:02.600306+0000","last_became_peered":"2024-07-04T17:32:02.600306+0000","last_unstale":"2024-07-04T17:32:03.575306+0000","last_undegraded":"2024-07-04T17:32:03.575306+0000","last_fullsized":"2024-07-04T17:32:03.575306+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:04:10.258386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"92'8","reported_seq":30,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.992604+0000","last_change":"2024-07-04T17:31:41.123522+0000","last_active":"2024-07-04T17:32:03.992604+0000","last_peered":"2024-07-04T17:32:03.992604+0000","last_clean":"2024-07-04T17:32:03.992604+0000","last_became_active":"2024-07-04T17:31:41.121523+0000","last_became_peered":"2024-07-04T17:31:41.121523+0000","last_unstale":"2024-07-04T17:32:03.992604+0000","last_undegraded":"2024-07-04T17:32:03.992604+0000","last_fullsized":"2024-07-04T17:32:03.992604+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:36:59.958154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3309,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.575023+0000","last_change":"2024-07-04T17:31:42.134181+0000","last_active":"2024-07-04T17:32:03.575023+0000","last_peered":"2024-07-04T17:32:03.575023+0000","last_clean":"2024-07-04T17:32:03.575023+0000","last_became_active":"2024-07-04T17:31:42.133671+0000","last_became_peered":"2024-07-04T17:31:42.133671+0000","last_unstale":"2024-07-04T17:32:03.575023+0000","last_undegraded":"2024-07-04T17:32:03.575023+0000","last_fullsized":"2024-07-04T17:32:03.575023+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:13:23.161757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.995661+0000","last_change":"2024-07-04T17:32:01.579982+0000","last_active":"2024-07-04T17:32:03.995661+0000","last_peered":"2024-07-04T17:32:03.995661+0000","last_clean":"2024-07-04T17:32:03.995661+0000","last_became_active":"2024-07-04T17:32:01.579645+0000","last_became_peered":"2024-07-04T17:32:01.579645+0000","last_unstale":"2024-07-04T17:32:03.995661+0000","last_undegraded":"2024-07-04T17:32:03.995661+0000","last_fullsized":"2024-07-04T17:32:03.995661+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:00:28.289678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.574613+0000","last_change":"2024-07-04T17:32:02.606795+0000","last_active":"2024-07-04T17:32:03.574613+0000","last_peered":"2024-07-04T17:32:03.574613+0000","last_clean":"2024-07-04T17:32:03.574613+0000","last_became_active":"2024-07-04T17:32:02.606068+0000","last_became_peered":"2024-07-04T17:32:02.606068+0000","last_unstale":"2024-07-04T17:32:03.574613+0000","last_undegraded":"2024-07-04T17:32:03.574613+0000","last_fullsized":"2024-07-04T17:32:03.574613+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:51:05.811627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"91'3","reported_seq":25,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.994369+0000","last_change":"2024-07-04T17:31:41.124114+0000","last_active":"2024-07-04T17:32:03.994369+0000","last_peered":"2024-07-04T17:32:03.994369+0000","last_clean":"2024-07-04T17:32:03.994369+0000","last_became_active":"2024-07-04T17:31:41.121166+0000","last_became_peered":"2024-07-04T17:31:41.121166+0000","last_unstale":"2024-07-04T17:32:03.994369+0000","last_undegraded":"2024-07-04T17:32:03.994369+0000","last_fullsized":"2024-07-04T17:32:03.994369+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:44:40.148021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.989062+0000","last_change":"2024-07-04T17:31:42.136357+0000","last_active":"2024-07-04T17:32:03.989062+0000","last_peered":"2024-07-04T17:32:03.989062+0000","last_clean":"2024-07-04T17:32:03.989062+0000","last_became_active":"2024-07-04T17:31:42.136025+0000","last_became_peered":"2024-07-04T17:31:42.136025+0000","last_unstale":"2024-07-04T17:32:03.989062+0000","last_undegraded":"2024-07-04T17:32:03.989062+0000","last_fullsized":"2024-07-04T17:32:03.989062+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:55:46.913903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.993774+0000","last_change":"2024-07-04T17:32:01.585343+0000","last_active":"2024-07-04T17:32:03.993774+0000","last_peered":"2024-07-04T17:32:03.993774+0000","last_clean":"2024-07-04T17:32:03.993774+0000","last_became_active":"2024-07-04T17:32:01.584969+0000","last_became_peered":"2024-07-04T17:32:01.584969+0000","last_unstale":"2024-07-04T17:32:03.993774+0000","last_undegraded":"2024-07-04T17:32:03.993774+0000","last_fullsized":"2024-07-04T17:32:03.993774+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:31:54.728917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.988726+0000","last_change":"2024-07-04T17:32:02.615398+0000","last_active":"2024-07-04T17:32:03.988726+0000","last_peered":"2024-07-04T17:32:03.988726+0000","last_clean":"2024-07-04T17:32:03.988726+0000","last_became_active":"2024-07-04T17:32:02.608548+0000","last_became_peered":"2024-07-04T17:32:02.608548+0000","last_unstale":"2024-07-04T17:32:03.988726+0000","last_undegraded":"2024-07-04T17:32:03.988726+0000","last_fullsized":"2024-07-04T17:32:03.988726+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:01:55.601573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.575855+0000","last_change":"2024-07-04T17:31:41.132406+0000","last_active":"2024-07-04T17:32:03.575855+0000","last_peered":"2024-07-04T17:32:03.575855+0000","last_clean":"2024-07-04T17:32:03.575855+0000","last_became_active":"2024-07-04T17:31:41.130594+0000","last_became_peered":"2024-07-04T17:31:41.130594+0000","last_unstale":"2024-07-04T17:32:03.575855+0000","last_undegraded":"2024-07-04T17:32:03.575855+0000","last_fullsized":"2024-07-04T17:32:03.575855+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:01:17.482235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.576358+0000","last_change":"2024-07-04T17:31:42.128979+0000","last_active":"2024-07-04T17:32:03.576358+0000","last_peered":"2024-07-04T17:32:03.576358+0000","last_clean":"2024-07-04T17:32:03.576358+0000","last_became_active":"2024-07-04T17:31:42.126866+0000","last_became_peered":"2024-07-04T17:31:42.126866+0000","last_unstale":"2024-07-04T17:32:03.576358+0000","last_undegraded":"2024-07-04T17:32:03.576358+0000","last_fullsized":"2024-07-04T17:32:03.576358+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:06:17.477749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.14","version":"95'6","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.988341+0000","last_change":"2024-07-04T17:32:01.596913+0000","last_active":"2024-07-04T17:32:03.988341+0000","last_peered":"2024-07-04T17:32:03.988341+0000","last_clean":"2024-07-04T17:32:03.988341+0000","last_became_active":"2024-07-04T17:32:01.596502+0000","last_became_peered":"2024-07-04T17:32:01.596502+0000","last_unstale":"2024-07-04T17:32:03.988341+0000","last_undegraded":"2024-07-04T17:32:03.988341+0000","last_fullsized":"2024-07-04T17:32:03.988341+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:57:24.893524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.991746+0000","last_change":"2024-07-04T17:32:02.612731+0000","last_active":"2024-07-04T17:32:03.991746+0000","last_peered":"2024-07-04T17:32:03.991746+0000","last_clean":"2024-07-04T17:32:03.991746+0000","last_became_active":"2024-07-04T17:32:02.608550+0000","last_became_peered":"2024-07-04T17:32:02.608550+0000","last_unstale":"2024-07-04T17:32:03.991746+0000","last_undegraded":"2024-07-04T17:32:03.991746+0000","last_fullsized":"2024-07-04T17:32:03.991746+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:38:37.557596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.576672+0000","last_change":"2024-07-04T17:31:41.131784+0000","last_active":"2024-07-04T17:32:03.576672+0000","last_peered":"2024-07-04T17:32:03.576672+0000","last_clean":"2024-07-04T17:32:03.576672+0000","last_became_active":"2024-07-04T17:31:41.130750+0000","last_became_peered":"2024-07-04T17:31:41.130750+0000","last_unstale":"2024-07-04T17:32:03.576672+0000","last_undegraded":"2024-07-04T17:32:03.576672+0000","last_fullsized":"2024-07-04T17:32:03.576672+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:55:29.200978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.575554+0000","last_change":"2024-07-04T17:31:42.133055+0000","last_active":"2024-07-04T17:32:03.575554+0000","last_peered":"2024-07-04T17:32:03.575554+0000","last_clean":"2024-07-04T17:32:03.575554+0000","last_became_active":"2024-07-04T17:31:42.131725+0000","last_became_peered":"2024-07-04T17:31:42.131725+0000","last_unstale":"2024-07-04T17:32:03.575554+0000","last_undegraded":"2024-07-04T17:32:03.575554+0000","last_fullsized":"2024-07-04T17:32:03.575554+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:21:59.669492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.15","version":"95'3","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.574494+0000","last_change":"2024-07-04T17:32:01.592999+0000","last_active":"2024-07-04T17:32:03.574494+0000","last_peered":"2024-07-04T17:32:03.574494+0000","last_clean":"2024-07-04T17:32:03.574494+0000","last_became_active":"2024-07-04T17:32:01.591679+0000","last_became_peered":"2024-07-04T17:32:01.591679+0000","last_unstale":"2024-07-04T17:32:03.574494+0000","last_undegraded":"2024-07-04T17:32:03.574494+0000","last_fullsized":"2024-07-04T17:32:03.574494+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:08:42.609506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.990452+0000","last_change":"2024-07-04T17:32:02.598164+0000","last_active":"2024-07-04T17:32:03.990452+0000","last_peered":"2024-07-04T17:32:03.990452+0000","last_clean":"2024-07-04T17:32:03.990452+0000","last_became_active":"2024-07-04T17:32:02.597819+0000","last_became_peered":"2024-07-04T17:32:02.597819+0000","last_unstale":"2024-07-04T17:32:03.990452+0000","last_undegraded":"2024-07-04T17:32:03.990452+0000","last_fullsized":"2024-07-04T17:32:03.990452+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:18:21.936559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.987936+0000","last_change":"2024-07-04T17:31:41.114447+0000","last_active":"2024-07-04T17:32:03.987936+0000","last_peered":"2024-07-04T17:32:03.987936+0000","last_clean":"2024-07-04T17:32:03.987936+0000","last_became_active":"2024-07-04T17:31:41.114127+0000","last_became_peered":"2024-07-04T17:31:41.114127+0000","last_unstale":"2024-07-04T17:32:03.987936+0000","last_undegraded":"2024-07-04T17:32:03.987936+0000","last_fullsized":"2024-07-04T17:32:03.987936+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:21:00.022601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.984620+0000","last_change":"2024-07-04T17:31:42.138950+0000","last_active":"2024-07-04T17:32:03.984620+0000","last_peered":"2024-07-04T17:32:03.984620+0000","last_clean":"2024-07-04T17:32:03.984620+0000","last_became_active":"2024-07-04T17:31:42.138614+0000","last_became_peered":"2024-07-04T17:31:42.138614+0000","last_unstale":"2024-07-04T17:32:03.984620+0000","last_undegraded":"2024-07-04T17:32:03.984620+0000","last_fullsized":"2024-07-04T17:32:03.984620+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:00:43.285683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1a","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.991588+0000","last_change":"2024-07-04T17:32:01.575001+0000","last_active":"2024-07-04T17:32:03.991588+0000","last_peered":"2024-07-04T17:32:03.991588+0000","last_clean":"2024-07-04T17:32:03.991588+0000","last_became_active":"2024-07-04T17:32:01.574551+0000","last_became_peered":"2024-07-04T17:32:01.574551+0000","last_unstale":"2024-07-04T17:32:03.991588+0000","last_undegraded":"2024-07-04T17:32:03.991588+0000","last_fullsized":"2024-07-04T17:32:03.991588+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:35:10.472781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.571667+0000","last_change":"2024-07-04T17:32:02.594405+0000","last_active":"2024-07-04T17:32:03.571667+0000","last_peered":"2024-07-04T17:32:03.571667+0000","last_clean":"2024-07-04T17:32:03.571667+0000","last_became_active":"2024-07-04T17:32:02.594043+0000","last_became_peered":"2024-07-04T17:32:02.594043+0000","last_unstale":"2024-07-04T17:32:03.571667+0000","last_undegraded":"2024-07-04T17:32:03.571667+0000","last_fullsized":"2024-07-04T17:32:03.571667+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:02:41.604232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.985136+0000","last_change":"2024-07-04T17:31:41.122775+0000","last_active":"2024-07-04T17:32:03.985136+0000","last_peered":"2024-07-04T17:32:03.985136+0000","last_clean":"2024-07-04T17:32:03.985136+0000","last_became_active":"2024-07-04T17:31:41.121256+0000","last_became_peered":"2024-07-04T17:31:41.121256+0000","last_unstale":"2024-07-04T17:32:03.985136+0000","last_undegraded":"2024-07-04T17:32:03.985136+0000","last_fullsized":"2024-07-04T17:32:03.985136+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:41:24.386164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.987226+0000","last_change":"2024-07-04T17:31:42.124019+0000","last_active":"2024-07-04T17:32:03.987226+0000","last_peered":"2024-07-04T17:32:03.987226+0000","last_clean":"2024-07-04T17:32:03.987226+0000","last_became_active":"2024-07-04T17:31:42.123684+0000","last_became_peered":"2024-07-04T17:31:42.123684+0000","last_unstale":"2024-07-04T17:32:03.987226+0000","last_undegraded":"2024-07-04T17:32:03.987226+0000","last_fullsized":"2024-07-04T17:32:03.987226+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:12:26.032030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1b","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.985519+0000","last_change":"2024-07-04T17:32:01.593554+0000","last_active":"2024-07-04T17:32:03.985519+0000","last_peered":"2024-07-04T17:32:03.985519+0000","last_clean":"2024-07-04T17:32:03.985519+0000","last_became_active":"2024-07-04T17:32:01.592637+0000","last_became_peered":"2024-07-04T17:32:01.592637+0000","last_unstale":"2024-07-04T17:32:03.985519+0000","last_undegraded":"2024-07-04T17:32:03.985519+0000","last_fullsized":"2024-07-04T17:32:03.985519+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:02:33.297199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.984078+0000","last_change":"2024-07-04T17:32:02.621078+0000","last_active":"2024-07-04T17:32:03.984078+0000","last_peered":"2024-07-04T17:32:03.984078+0000","last_clean":"2024-07-04T17:32:03.984078+0000","last_became_active":"2024-07-04T17:32:02.619930+0000","last_became_peered":"2024-07-04T17:32:02.619930+0000","last_unstale":"2024-07-04T17:32:03.984078+0000","last_undegraded":"2024-07-04T17:32:03.984078+0000","last_fullsized":"2024-07-04T17:32:03.984078+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:42:41.324766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1a","version":"91'1","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.572936+0000","last_change":"2024-07-04T17:31:41.118623+0000","last_active":"2024-07-04T17:32:03.572936+0000","last_peered":"2024-07-04T17:32:03.572936+0000","last_clean":"2024-07-04T17:32:03.572936+0000","last_became_active":"2024-07-04T17:31:41.116368+0000","last_became_peered":"2024-07-04T17:31:41.116368+0000","last_unstale":"2024-07-04T17:32:03.572936+0000","last_undegraded":"2024-07-04T17:32:03.572936+0000","last_fullsized":"2024-07-04T17:32:03.572936+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:34:37.181739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.991480+0000","last_change":"2024-07-04T17:31:42.111322+0000","last_active":"2024-07-04T17:32:03.991480+0000","last_peered":"2024-07-04T17:32:03.991480+0000","last_clean":"2024-07-04T17:32:03.991480+0000","last_became_active":"2024-07-04T17:31:42.110186+0000","last_became_peered":"2024-07-04T17:31:42.110186+0000","last_unstale":"2024-07-04T17:32:03.991480+0000","last_undegraded":"2024-07-04T17:32:03.991480+0000","last_fullsized":"2024-07-04T17:32:03.991480+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:43:36.161285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.984314+0000","last_change":"2024-07-04T17:32:01.583499+0000","last_active":"2024-07-04T17:32:03.984314+0000","last_peered":"2024-07-04T17:32:03.984314+0000","last_clean":"2024-07-04T17:32:03.984314+0000","last_became_active":"2024-07-04T17:32:01.583102+0000","last_became_peered":"2024-07-04T17:32:01.583102+0000","last_unstale":"2024-07-04T17:32:03.984314+0000","last_undegraded":"2024-07-04T17:32:03.984314+0000","last_fullsized":"2024-07-04T17:32:03.984314+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:51:21.342160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.986703+0000","last_change":"2024-07-04T17:32:02.609423+0000","last_active":"2024-07-04T17:32:03.986703+0000","last_peered":"2024-07-04T17:32:03.986703+0000","last_clean":"2024-07-04T17:32:03.986703+0000","last_became_active":"2024-07-04T17:32:02.608428+0000","last_became_peered":"2024-07-04T17:32:02.608428+0000","last_unstale":"2024-07-04T17:32:03.986703+0000","last_undegraded":"2024-07-04T17:32:03.986703+0000","last_fullsized":"2024-07-04T17:32:03.986703+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:32:11.435949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"91'1","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.985815+0000","last_change":"2024-07-04T17:31:41.129634+0000","last_active":"2024-07-04T17:32:03.985815+0000","last_peered":"2024-07-04T17:32:03.985815+0000","last_clean":"2024-07-04T17:32:03.985815+0000","last_became_active":"2024-07-04T17:31:41.129326+0000","last_became_peered":"2024-07-04T17:31:41.129326+0000","last_unstale":"2024-07-04T17:32:03.985815+0000","last_undegraded":"2024-07-04T17:32:03.985815+0000","last_fullsized":"2024-07-04T17:32:03.985815+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:37:28.340891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.993076+0000","last_change":"2024-07-04T17:31:42.132447+0000","last_active":"2024-07-04T17:32:03.993076+0000","last_peered":"2024-07-04T17:32:03.993076+0000","last_clean":"2024-07-04T17:32:03.993076+0000","last_became_active":"2024-07-04T17:31:42.132121+0000","last_became_peered":"2024-07-04T17:31:42.132121+0000","last_unstale":"2024-07-04T17:32:03.993076+0000","last_undegraded":"2024-07-04T17:32:03.993076+0000","last_fullsized":"2024-07-04T17:32:03.993076+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:55:55.603144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.989225+0000","last_change":"2024-07-04T17:32:01.580926+0000","last_active":"2024-07-04T17:32:03.989225+0000","last_peered":"2024-07-04T17:32:03.989225+0000","last_clean":"2024-07-04T17:32:03.989225+0000","last_became_active":"2024-07-04T17:32:01.580542+0000","last_became_peered":"2024-07-04T17:32:01.580542+0000","last_unstale":"2024-07-04T17:32:03.989225+0000","last_undegraded":"2024-07-04T17:32:03.989225+0000","last_fullsized":"2024-07-04T17:32:03.989225+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:14:46.722382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.575391+0000","last_change":"2024-07-04T17:32:02.610001+0000","last_active":"2024-07-04T17:32:03.575391+0000","last_peered":"2024-07-04T17:32:03.575391+0000","last_clean":"2024-07-04T17:32:03.575391+0000","last_became_active":"2024-07-04T17:32:02.603860+0000","last_became_peered":"2024-07-04T17:32:02.603860+0000","last_unstale":"2024-07-04T17:32:03.575391+0000","last_undegraded":"2024-07-04T17:32:03.575391+0000","last_fullsized":"2024-07-04T17:32:03.575391+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:21:50.088729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.1e","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.996198+0000","last_change":"2024-07-04T17:32:01.597250+0000","last_active":"2024-07-04T17:32:03.996198+0000","last_peered":"2024-07-04T17:32:03.996198+0000","last_clean":"2024-07-04T17:32:03.996198+0000","last_became_active":"2024-07-04T17:32:01.593415+0000","last_became_peered":"2024-07-04T17:32:01.593415+0000","last_unstale":"2024-07-04T17:32:03.996198+0000","last_undegraded":"2024-07-04T17:32:03.996198+0000","last_fullsized":"2024-07-04T17:32:03.996198+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:37:25.431586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"91'2","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.574715+0000","last_change":"2024-07-04T17:31:41.116679+0000","last_active":"2024-07-04T17:32:03.574715+0000","last_peered":"2024-07-04T17:32:03.574715+0000","last_clean":"2024-07-04T17:32:03.574715+0000","last_became_active":"2024-07-04T17:31:41.116240+0000","last_became_peered":"2024-07-04T17:31:41.116240+0000","last_unstale":"2024-07-04T17:32:03.574715+0000","last_undegraded":"2024-07-04T17:32:03.574715+0000","last_fullsized":"2024-07-04T17:32:03.574715+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:40:21.938736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.994977+0000","last_change":"2024-07-04T17:31:42.138160+0000","last_active":"2024-07-04T17:32:03.994977+0000","last_peered":"2024-07-04T17:32:03.994977+0000","last_clean":"2024-07-04T17:32:03.994977+0000","last_became_active":"2024-07-04T17:31:42.136593+0000","last_became_peered":"2024-07-04T17:31:42.136593+0000","last_unstale":"2024-07-04T17:32:03.994977+0000","last_undegraded":"2024-07-04T17:32:03.994977+0000","last_fullsized":"2024-07-04T17:32:03.994977+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:28:28.225877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.995040+0000","last_change":"2024-07-04T17:32:02.604178+0000","last_active":"2024-07-04T17:32:03.995040+0000","last_peered":"2024-07-04T17:32:03.995040+0000","last_clean":"2024-07-04T17:32:03.995040+0000","last_became_active":"2024-07-04T17:32:02.603832+0000","last_became_peered":"2024-07-04T17:32:02.603832+0000","last_unstale":"2024-07-04T17:32:03.995040+0000","last_undegraded":"2024-07-04T17:32:03.995040+0000","last_fullsized":"2024-07-04T17:32:03.995040+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:41:06.662377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1f","version":"95'4","reported_seq":18,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.575987+0000","last_change":"2024-07-04T17:32:01.598175+0000","last_active":"2024-07-04T17:32:03.575987+0000","last_peered":"2024-07-04T17:32:03.575987+0000","last_clean":"2024-07-04T17:32:03.575987+0000","last_became_active":"2024-07-04T17:32:01.597834+0000","last_became_peered":"2024-07-04T17:32:01.597834+0000","last_unstale":"2024-07-04T17:32:03.575987+0000","last_undegraded":"2024-07-04T17:32:03.575987+0000","last_fullsized":"2024-07-04T17:32:03.575987+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:29:56.239811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1d","version":"91'2","reported_seq":24,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.575215+0000","last_change":"2024-07-04T17:31:41.129438+0000","last_active":"2024-07-04T17:32:03.575215+0000","last_peered":"2024-07-04T17:32:03.575215+0000","last_clean":"2024-07-04T17:32:03.575215+0000","last_became_active":"2024-07-04T17:31:41.127900+0000","last_became_peered":"2024-07-04T17:31:41.127900+0000","last_unstale":"2024-07-04T17:32:03.575215+0000","last_undegraded":"2024-07-04T17:32:03.575215+0000","last_fullsized":"2024-07-04T17:32:03.575215+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:12:36.772353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.988889+0000","last_change":"2024-07-04T17:31:42.126808+0000","last_active":"2024-07-04T17:32:03.988889+0000","last_peered":"2024-07-04T17:32:03.988889+0000","last_clean":"2024-07-04T17:32:03.988889+0000","last_became_active":"2024-07-04T17:31:42.126497+0000","last_became_peered":"2024-07-04T17:31:42.126497+0000","last_unstale":"2024-07-04T17:32:03.988889+0000","last_undegraded":"2024-07-04T17:32:03.988889+0000","last_fullsized":"2024-07-04T17:32:03.988889+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:04:06.470763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.993228+0000","last_change":"2024-07-04T17:32:02.603761+0000","last_active":"2024-07-04T17:32:03.993228+0000","last_peered":"2024-07-04T17:32:03.993228+0000","last_clean":"2024-07-04T17:32:03.993228+0000","last_became_active":"2024-07-04T17:32:02.602352+0000","last_became_peered":"2024-07-04T17:32:02.602352+0000","last_unstale":"2024-07-04T17:32:03.993228+0000","last_undegraded":"2024-07-04T17:32:03.993228+0000","last_fullsized":"2024-07-04T17:32:03.993228+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:09:20.217518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"91'1","reported_seq":23,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.575733+0000","last_change":"2024-07-04T17:31:41.131289+0000","last_active":"2024-07-04T17:32:03.575733+0000","last_peered":"2024-07-04T17:32:03.575733+0000","last_clean":"2024-07-04T17:32:03.575733+0000","last_became_active":"2024-07-04T17:31:41.130956+0000","last_became_peered":"2024-07-04T17:31:41.130956+0000","last_unstale":"2024-07-04T17:32:03.575733+0000","last_undegraded":"2024-07-04T17:32:03.575733+0000","last_fullsized":"2024-07-04T17:32:03.575733+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:11:01.631006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.995864+0000","last_change":"2024-07-04T17:31:42.143241+0000","last_active":"2024-07-04T17:32:03.995864+0000","last_peered":"2024-07-04T17:32:03.995864+0000","last_clean":"2024-07-04T17:32:03.995864+0000","last_became_active":"2024-07-04T17:31:42.142919+0000","last_became_peered":"2024-07-04T17:31:42.142919+0000","last_unstale":"2024-07-04T17:32:03.995864+0000","last_undegraded":"2024-07-04T17:32:03.995864+0000","last_fullsized":"2024-07-04T17:32:03.995864+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:42:50.042411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.988503+0000","last_change":"2024-07-04T17:32:01.587782+0000","last_active":"2024-07-04T17:32:03.988503+0000","last_peered":"2024-07-04T17:32:03.988503+0000","last_clean":"2024-07-04T17:32:03.988503+0000","last_became_active":"2024-07-04T17:32:01.587353+0000","last_became_peered":"2024-07-04T17:32:01.587353+0000","last_unstale":"2024-07-04T17:32:03.988503+0000","last_undegraded":"2024-07-04T17:32:03.988503+0000","last_fullsized":"2024-07-04T17:32:03.988503+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:06:36.928702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.574009+0000","last_change":"2024-07-04T17:32:02.593632+0000","last_active":"2024-07-04T17:32:03.574009+0000","last_peered":"2024-07-04T17:32:03.574009+0000","last_clean":"2024-07-04T17:32:03.574009+0000","last_became_active":"2024-07-04T17:32:02.593298+0000","last_became_peered":"2024-07-04T17:32:02.593298+0000","last_unstale":"2024-07-04T17:32:03.574009+0000","last_undegraded":"2024-07-04T17:32:03.574009+0000","last_fullsized":"2024-07-04T17:32:03.574009+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:07:35.131023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1f","version":"91'4","reported_seq":26,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.996714+0000","last_change":"2024-07-04T17:31:41.125254+0000","last_active":"2024-07-04T17:32:03.996714+0000","last_peered":"2024-07-04T17:32:03.996714+0000","last_clean":"2024-07-04T17:32:03.996714+0000","last_became_active":"2024-07-04T17:31:41.119432+0000","last_became_peered":"2024-07-04T17:31:41.119432+0000","last_unstale":"2024-07-04T17:32:03.996714+0000","last_undegraded":"2024-07-04T17:32:03.996714+0000","last_fullsized":"2024-07-04T17:32:03.996714+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:40.059635+0000","last_clean_scrub_stamp":"2024-07-04T17:31:40.059635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:40:27.856616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.996318+0000","last_change":"2024-07-04T17:31:42.139568+0000","last_active":"2024-07-04T17:32:03.996318+0000","last_peered":"2024-07-04T17:32:03.996318+0000","last_clean":"2024-07-04T17:32:03.996318+0000","last_became_active":"2024-07-04T17:31:42.139247+0000","last_became_peered":"2024-07-04T17:31:42.139247+0000","last_unstale":"2024-07-04T17:32:03.996318+0000","last_undegraded":"2024-07-04T17:32:03.996318+0000","last_fullsized":"2024-07-04T17:32:03.996318+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:48:35.448519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1d","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.993556+0000","last_change":"2024-07-04T17:32:01.582827+0000","last_active":"2024-07-04T17:32:03.993556+0000","last_peered":"2024-07-04T17:32:03.993556+0000","last_clean":"2024-07-04T17:32:03.993556+0000","last_became_active":"2024-07-04T17:32:01.582511+0000","last_became_peered":"2024-07-04T17:32:01.582511+0000","last_unstale":"2024-07-04T17:32:03.993556+0000","last_undegraded":"2024-07-04T17:32:03.993556+0000","last_fullsized":"2024-07-04T17:32:03.993556+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:00.514922+0000","last_clean_scrub_stamp":"2024-07-04T17:32:00.514922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:43:00.246531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.571404+0000","last_change":"2024-07-04T17:32:02.606409+0000","last_active":"2024-07-04T17:32:03.571404+0000","last_peered":"2024-07-04T17:32:03.571404+0000","last_clean":"2024-07-04T17:32:03.571404+0000","last_became_active":"2024-07-04T17:32:02.605412+0000","last_became_peered":"2024-07-04T17:32:02.605412+0000","last_unstale":"2024-07-04T17:32:03.571404+0000","last_undegraded":"2024-07-04T17:32:03.571404+0000","last_fullsized":"2024-07-04T17:32:03.571404+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:43:48.709789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.984245+0000","last_change":"2024-07-04T17:31:42.137976+0000","last_active":"2024-07-04T17:32:03.984245+0000","last_peered":"2024-07-04T17:32:03.984245+0000","last_clean":"2024-07-04T17:32:03.984245+0000","last_became_active":"2024-07-04T17:31:42.137669+0000","last_became_peered":"2024-07-04T17:31:42.137669+0000","last_unstale":"2024-07-04T17:32:03.984245+0000","last_undegraded":"2024-07-04T17:32:03.984245+0000","last_fullsized":"2024-07-04T17:32:03.984245+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:07:25.661148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.573500+0000","last_change":"2024-07-04T17:32:02.610008+0000","last_active":"2024-07-04T17:32:03.573500+0000","last_peered":"2024-07-04T17:32:03.573500+0000","last_clean":"2024-07-04T17:32:03.573500+0000","last_became_active":"2024-07-04T17:32:02.609513+0000","last_became_peered":"2024-07-04T17:32:02.609513+0000","last_unstale":"2024-07-04T17:32:03.573500+0000","last_undegraded":"2024-07-04T17:32:03.573500+0000","last_fullsized":"2024-07-04T17:32:03.573500+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:14:23.536331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.990819+0000","last_change":"2024-07-04T17:31:42.150544+0000","last_active":"2024-07-04T17:32:03.990819+0000","last_peered":"2024-07-04T17:32:03.990819+0000","last_clean":"2024-07-04T17:32:03.990819+0000","last_became_active":"2024-07-04T17:31:42.150217+0000","last_became_peered":"2024-07-04T17:31:42.150217+0000","last_unstale":"2024-07-04T17:32:03.990819+0000","last_undegraded":"2024-07-04T17:32:03.990819+0000","last_fullsized":"2024-07-04T17:32:03.990819+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:50:45.348242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.983824+0000","last_change":"2024-07-04T17:32:02.610213+0000","last_active":"2024-07-04T17:32:03.983824+0000","last_peered":"2024-07-04T17:32:03.983824+0000","last_clean":"2024-07-04T17:32:03.983824+0000","last_became_active":"2024-07-04T17:32:02.609644+0000","last_became_peered":"2024-07-04T17:32:02.609644+0000","last_unstale":"2024-07-04T17:32:03.983824+0000","last_undegraded":"2024-07-04T17:32:03.983824+0000","last_fullsized":"2024-07-04T17:32:03.983824+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:44:33.949283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.572770+0000","last_change":"2024-07-04T17:31:42.115990+0000","last_active":"2024-07-04T17:32:03.572770+0000","last_peered":"2024-07-04T17:32:03.572770+0000","last_clean":"2024-07-04T17:32:03.572770+0000","last_became_active":"2024-07-04T17:31:42.115413+0000","last_became_peered":"2024-07-04T17:31:42.115413+0000","last_unstale":"2024-07-04T17:32:03.572770+0000","last_undegraded":"2024-07-04T17:32:03.572770+0000","last_fullsized":"2024-07-04T17:32:03.572770+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:36:22.090921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.983972+0000","last_change":"2024-07-04T17:32:02.621707+0000","last_active":"2024-07-04T17:32:03.983972+0000","last_peered":"2024-07-04T17:32:03.983972+0000","last_clean":"2024-07-04T17:32:03.983972+0000","last_became_active":"2024-07-04T17:32:02.620726+0000","last_became_peered":"2024-07-04T17:32:02.620726+0000","last_unstale":"2024-07-04T17:32:03.983972+0000","last_undegraded":"2024-07-04T17:32:03.983972+0000","last_fullsized":"2024-07-04T17:32:03.983972+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:02:45.804456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.991317+0000","last_change":"2024-07-04T17:31:42.111650+0000","last_active":"2024-07-04T17:32:03.991317+0000","last_peered":"2024-07-04T17:32:03.991317+0000","last_clean":"2024-07-04T17:32:03.991317+0000","last_became_active":"2024-07-04T17:31:42.111045+0000","last_became_peered":"2024-07-04T17:31:42.111045+0000","last_unstale":"2024-07-04T17:32:03.991317+0000","last_undegraded":"2024-07-04T17:32:03.991317+0000","last_fullsized":"2024-07-04T17:32:03.991317+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:11:39.466653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.995441+0000","last_change":"2024-07-04T17:32:02.600963+0000","last_active":"2024-07-04T17:32:03.995441+0000","last_peered":"2024-07-04T17:32:03.995441+0000","last_clean":"2024-07-04T17:32:03.995441+0000","last_became_active":"2024-07-04T17:32:02.598879+0000","last_became_peered":"2024-07-04T17:32:02.598879+0000","last_unstale":"2024-07-04T17:32:03.995441+0000","last_undegraded":"2024-07-04T17:32:03.995441+0000","last_fullsized":"2024-07-04T17:32:03.995441+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:45:52.984880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.992208+0000","last_change":"2024-07-04T17:31:42.139174+0000","last_active":"2024-07-04T17:32:03.992208+0000","last_peered":"2024-07-04T17:32:03.992208+0000","last_clean":"2024-07-04T17:32:03.992208+0000","last_became_active":"2024-07-04T17:31:42.138397+0000","last_became_peered":"2024-07-04T17:31:42.138397+0000","last_unstale":"2024-07-04T17:32:03.992208+0000","last_undegraded":"2024-07-04T17:32:03.992208+0000","last_fullsized":"2024-07-04T17:32:03.992208+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:21:29.625345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.995947+0000","last_change":"2024-07-04T17:32:02.608116+0000","last_active":"2024-07-04T17:32:03.995947+0000","last_peered":"2024-07-04T17:32:03.995947+0000","last_clean":"2024-07-04T17:32:03.995947+0000","last_became_active":"2024-07-04T17:32:02.606418+0000","last_became_peered":"2024-07-04T17:32:02.606418+0000","last_unstale":"2024-07-04T17:32:03.995947+0000","last_undegraded":"2024-07-04T17:32:03.995947+0000","last_fullsized":"2024-07-04T17:32:03.995947+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:23:11.566340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.994796+0000","last_change":"2024-07-04T17:31:42.130865+0000","last_active":"2024-07-04T17:32:03.994796+0000","last_peered":"2024-07-04T17:32:03.994796+0000","last_clean":"2024-07-04T17:32:03.994796+0000","last_became_active":"2024-07-04T17:31:42.130064+0000","last_became_peered":"2024-07-04T17:31:42.130064+0000","last_unstale":"2024-07-04T17:32:03.994796+0000","last_undegraded":"2024-07-04T17:32:03.994796+0000","last_fullsized":"2024-07-04T17:32:03.994796+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:54:40.425975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.987927+0000","last_change":"2024-07-04T17:32:02.610171+0000","last_active":"2024-07-04T17:32:03.987927+0000","last_peered":"2024-07-04T17:32:03.987927+0000","last_clean":"2024-07-04T17:32:03.987927+0000","last_became_active":"2024-07-04T17:32:02.609746+0000","last_became_peered":"2024-07-04T17:32:02.609746+0000","last_unstale":"2024-07-04T17:32:03.987927+0000","last_undegraded":"2024-07-04T17:32:03.987927+0000","last_fullsized":"2024-07-04T17:32:03.987927+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:33:45.577830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.996183+0000","last_change":"2024-07-04T17:31:42.142475+0000","last_active":"2024-07-04T17:32:03.996183+0000","last_peered":"2024-07-04T17:32:03.996183+0000","last_clean":"2024-07-04T17:32:03.996183+0000","last_became_active":"2024-07-04T17:31:42.142140+0000","last_became_peered":"2024-07-04T17:31:42.142140+0000","last_unstale":"2024-07-04T17:32:03.996183+0000","last_undegraded":"2024-07-04T17:32:03.996183+0000","last_fullsized":"2024-07-04T17:32:03.996183+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:12:40.862972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.574392+0000","last_change":"2024-07-04T17:32:02.605912+0000","last_active":"2024-07-04T17:32:03.574392+0000","last_peered":"2024-07-04T17:32:03.574392+0000","last_clean":"2024-07-04T17:32:03.574392+0000","last_became_active":"2024-07-04T17:32:02.605106+0000","last_became_peered":"2024-07-04T17:32:02.605106+0000","last_unstale":"2024-07-04T17:32:03.574392+0000","last_undegraded":"2024-07-04T17:32:03.574392+0000","last_fullsized":"2024-07-04T17:32:03.574392+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:51:10.170000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.576255+0000","last_change":"2024-07-04T17:31:42.129239+0000","last_active":"2024-07-04T17:32:03.576255+0000","last_peered":"2024-07-04T17:32:03.576255+0000","last_clean":"2024-07-04T17:32:03.576255+0000","last_became_active":"2024-07-04T17:31:42.127715+0000","last_became_peered":"2024-07-04T17:31:42.127715+0000","last_unstale":"2024-07-04T17:32:03.576255+0000","last_undegraded":"2024-07-04T17:32:03.576255+0000","last_fullsized":"2024-07-04T17:32:03.576255+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:51:01.891207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.989840+0000","last_change":"2024-07-04T17:32:02.607876+0000","last_active":"2024-07-04T17:32:03.989840+0000","last_peered":"2024-07-04T17:32:03.989840+0000","last_clean":"2024-07-04T17:32:03.989840+0000","last_became_active":"2024-07-04T17:32:02.607602+0000","last_became_peered":"2024-07-04T17:32:02.607602+0000","last_unstale":"2024-07-04T17:32:03.989840+0000","last_undegraded":"2024-07-04T17:32:03.989840+0000","last_fullsized":"2024-07-04T17:32:03.989840+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:07:22.204970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.571956+0000","last_change":"2024-07-04T17:31:42.126765+0000","last_active":"2024-07-04T17:32:03.571956+0000","last_peered":"2024-07-04T17:32:03.571956+0000","last_clean":"2024-07-04T17:32:03.571956+0000","last_became_active":"2024-07-04T17:31:42.126443+0000","last_became_peered":"2024-07-04T17:31:42.126443+0000","last_unstale":"2024-07-04T17:32:03.571956+0000","last_undegraded":"2024-07-04T17:32:03.571956+0000","last_fullsized":"2024-07-04T17:32:03.571956+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:34:36.071106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.990193+0000","last_change":"2024-07-04T17:32:02.608154+0000","last_active":"2024-07-04T17:32:03.990193+0000","last_peered":"2024-07-04T17:32:03.990193+0000","last_clean":"2024-07-04T17:32:03.990193+0000","last_became_active":"2024-07-04T17:32:02.606091+0000","last_became_peered":"2024-07-04T17:32:02.606091+0000","last_unstale":"2024-07-04T17:32:03.990193+0000","last_undegraded":"2024-07-04T17:32:03.990193+0000","last_fullsized":"2024-07-04T17:32:03.990193+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:47:34.307879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.990195+0000","last_change":"2024-07-04T17:31:42.153157+0000","last_active":"2024-07-04T17:32:03.990195+0000","last_peered":"2024-07-04T17:32:03.990195+0000","last_clean":"2024-07-04T17:32:03.990195+0000","last_became_active":"2024-07-04T17:31:42.150615+0000","last_became_peered":"2024-07-04T17:31:42.150615+0000","last_unstale":"2024-07-04T17:32:03.990195+0000","last_undegraded":"2024-07-04T17:32:03.990195+0000","last_fullsized":"2024-07-04T17:32:03.990195+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:50:49.706615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.988575+0000","last_change":"2024-07-04T17:32:02.609690+0000","last_active":"2024-07-04T17:32:03.988575+0000","last_peered":"2024-07-04T17:32:03.988575+0000","last_clean":"2024-07-04T17:32:03.988575+0000","last_became_active":"2024-07-04T17:32:02.608877+0000","last_became_peered":"2024-07-04T17:32:02.608877+0000","last_unstale":"2024-07-04T17:32:03.988575+0000","last_undegraded":"2024-07-04T17:32:03.988575+0000","last_fullsized":"2024-07-04T17:32:03.988575+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:37:21.373771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.572548+0000","last_change":"2024-07-04T17:31:42.116203+0000","last_active":"2024-07-04T17:32:03.572548+0000","last_peered":"2024-07-04T17:32:03.572548+0000","last_clean":"2024-07-04T17:32:03.572548+0000","last_became_active":"2024-07-04T17:31:42.115668+0000","last_became_peered":"2024-07-04T17:31:42.115668+0000","last_unstale":"2024-07-04T17:32:03.572548+0000","last_undegraded":"2024-07-04T17:32:03.572548+0000","last_fullsized":"2024-07-04T17:32:03.572548+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:06:36.600932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.984941+0000","last_change":"2024-07-04T17:32:02.620244+0000","last_active":"2024-07-04T17:32:03.984941+0000","last_peered":"2024-07-04T17:32:03.984941+0000","last_clean":"2024-07-04T17:32:03.984941+0000","last_became_active":"2024-07-04T17:32:02.619072+0000","last_became_peered":"2024-07-04T17:32:02.619072+0000","last_unstale":"2024-07-04T17:32:03.984941+0000","last_undegraded":"2024-07-04T17:32:03.984941+0000","last_fullsized":"2024-07-04T17:32:03.984941+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:13:14.107024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.573772+0000","last_change":"2024-07-04T17:31:42.127926+0000","last_active":"2024-07-04T17:32:03.573772+0000","last_peered":"2024-07-04T17:32:03.573772+0000","last_clean":"2024-07-04T17:32:03.573772+0000","last_became_active":"2024-07-04T17:31:42.127596+0000","last_became_peered":"2024-07-04T17:31:42.127596+0000","last_unstale":"2024-07-04T17:32:03.573772+0000","last_undegraded":"2024-07-04T17:32:03.573772+0000","last_fullsized":"2024-07-04T17:32:03.573772+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:20:44.775608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.995249+0000","last_change":"2024-07-04T17:32:02.601231+0000","last_active":"2024-07-04T17:32:03.995249+0000","last_peered":"2024-07-04T17:32:03.995249+0000","last_clean":"2024-07-04T17:32:03.995249+0000","last_became_active":"2024-07-04T17:32:02.599308+0000","last_became_peered":"2024-07-04T17:32:02.599308+0000","last_unstale":"2024-07-04T17:32:03.995249+0000","last_undegraded":"2024-07-04T17:32:03.995249+0000","last_fullsized":"2024-07-04T17:32:03.995249+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:46:29.471735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.988717+0000","last_change":"2024-07-04T17:31:42.141851+0000","last_active":"2024-07-04T17:32:03.988717+0000","last_peered":"2024-07-04T17:32:03.988717+0000","last_clean":"2024-07-04T17:32:03.988717+0000","last_became_active":"2024-07-04T17:31:42.139731+0000","last_became_peered":"2024-07-04T17:31:42.139731+0000","last_unstale":"2024-07-04T17:32:03.988717+0000","last_undegraded":"2024-07-04T17:32:03.988717+0000","last_fullsized":"2024-07-04T17:32:03.988717+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:46:33.797585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.990452+0000","last_change":"2024-07-04T17:32:02.611518+0000","last_active":"2024-07-04T17:32:03.990452+0000","last_peered":"2024-07-04T17:32:03.990452+0000","last_clean":"2024-07-04T17:32:03.990452+0000","last_became_active":"2024-07-04T17:32:02.610937+0000","last_became_peered":"2024-07-04T17:32:02.610937+0000","last_unstale":"2024-07-04T17:32:03.990452+0000","last_undegraded":"2024-07-04T17:32:03.990452+0000","last_fullsized":"2024-07-04T17:32:03.990452+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:12:42.384308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.994611+0000","last_change":"2024-07-04T17:31:42.143453+0000","last_active":"2024-07-04T17:32:03.994611+0000","last_peered":"2024-07-04T17:32:03.994611+0000","last_clean":"2024-07-04T17:32:03.994611+0000","last_became_active":"2024-07-04T17:31:42.142952+0000","last_became_peered":"2024-07-04T17:31:42.142952+0000","last_unstale":"2024-07-04T17:32:03.994611+0000","last_undegraded":"2024-07-04T17:32:03.994611+0000","last_fullsized":"2024-07-04T17:32:03.994611+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:21:03.395798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.988994+0000","last_change":"2024-07-04T17:32:02.591934+0000","last_active":"2024-07-04T17:32:03.988994+0000","last_peered":"2024-07-04T17:32:03.988994+0000","last_clean":"2024-07-04T17:32:03.988994+0000","last_became_active":"2024-07-04T17:32:02.577168+0000","last_became_peered":"2024-07-04T17:32:02.577168+0000","last_unstale":"2024-07-04T17:32:03.988994+0000","last_undegraded":"2024-07-04T17:32:03.988994+0000","last_fullsized":"2024-07-04T17:32:03.988994+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:43:36.215875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.996264+0000","last_change":"2024-07-04T17:31:42.131382+0000","last_active":"2024-07-04T17:32:03.996264+0000","last_peered":"2024-07-04T17:32:03.996264+0000","last_clean":"2024-07-04T17:32:03.996264+0000","last_became_active":"2024-07-04T17:31:42.131053+0000","last_became_peered":"2024-07-04T17:31:42.131053+0000","last_unstale":"2024-07-04T17:32:03.996264+0000","last_undegraded":"2024-07-04T17:32:03.996264+0000","last_fullsized":"2024-07-04T17:32:03.996264+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:52:52.277217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.991537+0000","last_change":"2024-07-04T17:32:02.603900+0000","last_active":"2024-07-04T17:32:03.991537+0000","last_peered":"2024-07-04T17:32:03.991537+0000","last_clean":"2024-07-04T17:32:03.991537+0000","last_became_active":"2024-07-04T17:32:02.602708+0000","last_became_peered":"2024-07-04T17:32:02.602708+0000","last_unstale":"2024-07-04T17:32:03.991537+0000","last_undegraded":"2024-07-04T17:32:03.991537+0000","last_fullsized":"2024-07-04T17:32:03.991537+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:34:06.894952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.990825+0000","last_change":"2024-07-04T17:31:42.127722+0000","last_active":"2024-07-04T17:32:03.990825+0000","last_peered":"2024-07-04T17:32:03.990825+0000","last_clean":"2024-07-04T17:32:03.990825+0000","last_became_active":"2024-07-04T17:31:42.127401+0000","last_became_peered":"2024-07-04T17:31:42.127401+0000","last_unstale":"2024-07-04T17:32:03.990825+0000","last_undegraded":"2024-07-04T17:32:03.990825+0000","last_fullsized":"2024-07-04T17:32:03.990825+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:08:43.721732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.984630+0000","last_change":"2024-07-04T17:32:02.578104+0000","last_active":"2024-07-04T17:32:03.984630+0000","last_peered":"2024-07-04T17:32:03.984630+0000","last_clean":"2024-07-04T17:32:03.984630+0000","last_became_active":"2024-07-04T17:32:02.577752+0000","last_became_peered":"2024-07-04T17:32:02.577752+0000","last_unstale":"2024-07-04T17:32:03.984630+0000","last_undegraded":"2024-07-04T17:32:03.984630+0000","last_fullsized":"2024-07-04T17:32:03.984630+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:40:49.324334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.987225+0000","last_change":"2024-07-04T17:31:42.124307+0000","last_active":"2024-07-04T17:32:03.987225+0000","last_peered":"2024-07-04T17:32:03.987225+0000","last_clean":"2024-07-04T17:32:03.987225+0000","last_became_active":"2024-07-04T17:31:42.121958+0000","last_became_peered":"2024-07-04T17:31:42.121958+0000","last_unstale":"2024-07-04T17:32:03.987225+0000","last_undegraded":"2024-07-04T17:32:03.987225+0000","last_fullsized":"2024-07-04T17:32:03.987225+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:14:03.072947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.573073+0000","last_change":"2024-07-04T17:32:02.596561+0000","last_active":"2024-07-04T17:32:03.573073+0000","last_peered":"2024-07-04T17:32:03.573073+0000","last_clean":"2024-07-04T17:32:03.573073+0000","last_became_active":"2024-07-04T17:32:02.596059+0000","last_became_peered":"2024-07-04T17:32:02.596059+0000","last_unstale":"2024-07-04T17:32:03.573073+0000","last_undegraded":"2024-07-04T17:32:03.573073+0000","last_fullsized":"2024-07-04T17:32:03.573073+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:19:30.318182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.984451+0000","last_change":"2024-07-04T17:31:42.126095+0000","last_active":"2024-07-04T17:32:03.984451+0000","last_peered":"2024-07-04T17:32:03.984451+0000","last_clean":"2024-07-04T17:32:03.984451+0000","last_became_active":"2024-07-04T17:31:42.125783+0000","last_became_peered":"2024-07-04T17:31:42.125783+0000","last_unstale":"2024-07-04T17:32:03.984451+0000","last_undegraded":"2024-07-04T17:32:03.984451+0000","last_fullsized":"2024-07-04T17:32:03.984451+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:01:18.486569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.571314+0000","last_change":"2024-07-04T17:32:02.583841+0000","last_active":"2024-07-04T17:32:03.571314+0000","last_peered":"2024-07-04T17:32:03.571314+0000","last_clean":"2024-07-04T17:32:03.571314+0000","last_became_active":"2024-07-04T17:32:02.583156+0000","last_became_peered":"2024-07-04T17:32:02.583156+0000","last_unstale":"2024-07-04T17:32:03.571314+0000","last_undegraded":"2024-07-04T17:32:03.571314+0000","last_fullsized":"2024-07-04T17:32:03.571314+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:23:11.566340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.993014+0000","last_change":"2024-07-04T17:31:42.137702+0000","last_active":"2024-07-04T17:32:03.993014+0000","last_peered":"2024-07-04T17:32:03.993014+0000","last_clean":"2024-07-04T17:32:03.993014+0000","last_became_active":"2024-07-04T17:31:42.134081+0000","last_became_peered":"2024-07-04T17:31:42.134081+0000","last_unstale":"2024-07-04T17:32:03.993014+0000","last_undegraded":"2024-07-04T17:32:03.993014+0000","last_fullsized":"2024-07-04T17:32:03.993014+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:26:29.769982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.571075+0000","last_change":"2024-07-04T17:32:02.605034+0000","last_active":"2024-07-04T17:32:03.571075+0000","last_peered":"2024-07-04T17:32:03.571075+0000","last_clean":"2024-07-04T17:32:03.571075+0000","last_became_active":"2024-07-04T17:32:02.603145+0000","last_became_peered":"2024-07-04T17:32:02.603145+0000","last_unstale":"2024-07-04T17:32:03.571075+0000","last_undegraded":"2024-07-04T17:32:03.571075+0000","last_fullsized":"2024-07-04T17:32:03.571075+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:45:52.984880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.573585+0000","last_change":"2024-07-04T17:31:42.132759+0000","last_active":"2024-07-04T17:32:03.573585+0000","last_peered":"2024-07-04T17:32:03.573585+0000","last_clean":"2024-07-04T17:32:03.573585+0000","last_became_active":"2024-07-04T17:31:42.132493+0000","last_became_peered":"2024-07-04T17:31:42.132493+0000","last_unstale":"2024-07-04T17:32:03.573585+0000","last_undegraded":"2024-07-04T17:32:03.573585+0000","last_fullsized":"2024-07-04T17:32:03.573585+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:41:07.944963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.995061+0000","last_change":"2024-07-04T17:32:02.618897+0000","last_active":"2024-07-04T17:32:03.995061+0000","last_peered":"2024-07-04T17:32:03.995061+0000","last_clean":"2024-07-04T17:32:03.995061+0000","last_became_active":"2024-07-04T17:32:02.618566+0000","last_became_peered":"2024-07-04T17:32:02.618566+0000","last_unstale":"2024-07-04T17:32:03.995061+0000","last_undegraded":"2024-07-04T17:32:03.995061+0000","last_fullsized":"2024-07-04T17:32:03.995061+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:05:00.682546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.992216+0000","last_change":"2024-07-04T17:31:42.110720+0000","last_active":"2024-07-04T17:32:03.992216+0000","last_peered":"2024-07-04T17:32:03.992216+0000","last_clean":"2024-07-04T17:32:03.992216+0000","last_became_active":"2024-07-04T17:31:42.110428+0000","last_became_peered":"2024-07-04T17:31:42.110428+0000","last_unstale":"2024-07-04T17:32:03.992216+0000","last_undegraded":"2024-07-04T17:32:03.992216+0000","last_fullsized":"2024-07-04T17:32:03.992216+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:03:49.795002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.576725+0000","last_change":"2024-07-04T17:32:02.610969+0000","last_active":"2024-07-04T17:32:03.576725+0000","last_peered":"2024-07-04T17:32:03.576725+0000","last_clean":"2024-07-04T17:32:03.576725+0000","last_became_active":"2024-07-04T17:32:02.604273+0000","last_became_peered":"2024-07-04T17:32:02.604273+0000","last_unstale":"2024-07-04T17:32:03.576725+0000","last_undegraded":"2024-07-04T17:32:03.576725+0000","last_fullsized":"2024-07-04T17:32:03.576725+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:11:59.930038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.993871+0000","last_change":"2024-07-04T17:31:42.138904+0000","last_active":"2024-07-04T17:32:03.993871+0000","last_peered":"2024-07-04T17:32:03.993871+0000","last_clean":"2024-07-04T17:32:03.993871+0000","last_became_active":"2024-07-04T17:31:42.138265+0000","last_became_peered":"2024-07-04T17:31:42.138265+0000","last_unstale":"2024-07-04T17:32:03.993871+0000","last_undegraded":"2024-07-04T17:32:03.993871+0000","last_fullsized":"2024-07-04T17:32:03.993871+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:26.894424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.991430+0000","last_change":"2024-07-04T17:32:02.622111+0000","last_active":"2024-07-04T17:32:03.991430+0000","last_peered":"2024-07-04T17:32:03.991430+0000","last_clean":"2024-07-04T17:32:03.991430+0000","last_became_active":"2024-07-04T17:32:02.621810+0000","last_became_peered":"2024-07-04T17:32:02.621810+0000","last_unstale":"2024-07-04T17:32:03.991430+0000","last_undegraded":"2024-07-04T17:32:03.991430+0000","last_fullsized":"2024-07-04T17:32:03.991430+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:12:45.045568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.996025+0000","last_change":"2024-07-04T17:31:42.109492+0000","last_active":"2024-07-04T17:32:03.996025+0000","last_peered":"2024-07-04T17:32:03.996025+0000","last_clean":"2024-07-04T17:32:03.996025+0000","last_became_active":"2024-07-04T17:31:42.108502+0000","last_became_peered":"2024-07-04T17:31:42.108502+0000","last_unstale":"2024-07-04T17:32:03.996025+0000","last_undegraded":"2024-07-04T17:32:03.996025+0000","last_fullsized":"2024-07-04T17:32:03.996025+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:39:41.707779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.990944+0000","last_change":"2024-07-04T17:32:02.604036+0000","last_active":"2024-07-04T17:32:03.990944+0000","last_peered":"2024-07-04T17:32:03.990944+0000","last_clean":"2024-07-04T17:32:03.990944+0000","last_became_active":"2024-07-04T17:32:02.603062+0000","last_became_peered":"2024-07-04T17:32:02.603062+0000","last_unstale":"2024-07-04T17:32:03.990944+0000","last_undegraded":"2024-07-04T17:32:03.990944+0000","last_fullsized":"2024-07-04T17:32:03.990944+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:34:12.680957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.994685+0000","last_change":"2024-07-04T17:31:42.134470+0000","last_active":"2024-07-04T17:32:03.994685+0000","last_peered":"2024-07-04T17:32:03.994685+0000","last_clean":"2024-07-04T17:32:03.994685+0000","last_became_active":"2024-07-04T17:31:42.134096+0000","last_became_peered":"2024-07-04T17:31:42.134096+0000","last_unstale":"2024-07-04T17:32:03.994685+0000","last_undegraded":"2024-07-04T17:32:03.994685+0000","last_fullsized":"2024-07-04T17:32:03.994685+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:07:42.785561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.989640+0000","last_change":"2024-07-04T17:32:02.603154+0000","last_active":"2024-07-04T17:32:03.989640+0000","last_peered":"2024-07-04T17:32:03.989640+0000","last_clean":"2024-07-04T17:32:03.989640+0000","last_became_active":"2024-07-04T17:32:02.602818+0000","last_became_peered":"2024-07-04T17:32:02.602818+0000","last_unstale":"2024-07-04T17:32:03.989640+0000","last_undegraded":"2024-07-04T17:32:03.989640+0000","last_fullsized":"2024-07-04T17:32:03.989640+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:49:39.286469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.571746+0000","last_change":"2024-07-04T17:31:42.127048+0000","last_active":"2024-07-04T17:32:03.571746+0000","last_peered":"2024-07-04T17:32:03.571746+0000","last_clean":"2024-07-04T17:32:03.571746+0000","last_became_active":"2024-07-04T17:31:42.126695+0000","last_became_peered":"2024-07-04T17:31:42.126695+0000","last_unstale":"2024-07-04T17:32:03.571746+0000","last_undegraded":"2024-07-04T17:32:03.571746+0000","last_fullsized":"2024-07-04T17:32:03.571746+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:46:15.072312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.572934+0000","last_change":"2024-07-04T17:32:02.610521+0000","last_active":"2024-07-04T17:32:03.572934+0000","last_peered":"2024-07-04T17:32:03.572934+0000","last_clean":"2024-07-04T17:32:03.572934+0000","last_became_active":"2024-07-04T17:32:02.609949+0000","last_became_peered":"2024-07-04T17:32:02.609949+0000","last_unstale":"2024-07-04T17:32:03.572934+0000","last_undegraded":"2024-07-04T17:32:03.572934+0000","last_fullsized":"2024-07-04T17:32:03.572934+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:23:26.532448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.571807+0000","last_change":"2024-07-04T17:31:42.131777+0000","last_active":"2024-07-04T17:32:03.571807+0000","last_peered":"2024-07-04T17:32:03.571807+0000","last_clean":"2024-07-04T17:32:03.571807+0000","last_became_active":"2024-07-04T17:31:42.127915+0000","last_became_peered":"2024-07-04T17:31:42.127915+0000","last_unstale":"2024-07-04T17:32:03.571807+0000","last_undegraded":"2024-07-04T17:32:03.571807+0000","last_fullsized":"2024-07-04T17:32:03.571807+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:14:13.315623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.570740+0000","last_change":"2024-07-04T17:32:02.584120+0000","last_active":"2024-07-04T17:32:03.570740+0000","last_peered":"2024-07-04T17:32:03.570740+0000","last_clean":"2024-07-04T17:32:03.570740+0000","last_became_active":"2024-07-04T17:32:02.583539+0000","last_became_peered":"2024-07-04T17:32:02.583539+0000","last_unstale":"2024-07-04T17:32:03.570740+0000","last_undegraded":"2024-07-04T17:32:03.570740+0000","last_fullsized":"2024-07-04T17:32:03.570740+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:05:00.682546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.985575+0000","last_change":"2024-07-04T17:31:42.146322+0000","last_active":"2024-07-04T17:32:03.985575+0000","last_peered":"2024-07-04T17:32:03.985575+0000","last_clean":"2024-07-04T17:32:03.985575+0000","last_became_active":"2024-07-04T17:31:42.146004+0000","last_became_peered":"2024-07-04T17:31:42.146004+0000","last_unstale":"2024-07-04T17:32:03.985575+0000","last_undegraded":"2024-07-04T17:32:03.985575+0000","last_fullsized":"2024-07-04T17:32:03.985575+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:10:27.612193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.571048+0000","last_change":"2024-07-04T17:32:02.614219+0000","last_active":"2024-07-04T17:32:03.571048+0000","last_peered":"2024-07-04T17:32:03.571048+0000","last_clean":"2024-07-04T17:32:03.571048+0000","last_became_active":"2024-07-04T17:32:02.613331+0000","last_became_peered":"2024-07-04T17:32:02.613331+0000","last_unstale":"2024-07-04T17:32:03.571048+0000","last_undegraded":"2024-07-04T17:32:03.571048+0000","last_fullsized":"2024-07-04T17:32:03.571048+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:46:29.471735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.573443+0000","last_change":"2024-07-04T17:31:42.118035+0000","last_active":"2024-07-04T17:32:03.573443+0000","last_peered":"2024-07-04T17:32:03.573443+0000","last_clean":"2024-07-04T17:32:03.573443+0000","last_became_active":"2024-07-04T17:31:42.117722+0000","last_became_peered":"2024-07-04T17:31:42.117722+0000","last_unstale":"2024-07-04T17:32:03.573443+0000","last_undegraded":"2024-07-04T17:32:03.573443+0000","last_fullsized":"2024-07-04T17:32:03.573443+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:17:18.803619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.994869+0000","last_change":"2024-07-04T17:32:02.605817+0000","last_active":"2024-07-04T17:32:03.994869+0000","last_peered":"2024-07-04T17:32:03.994869+0000","last_clean":"2024-07-04T17:32:03.994869+0000","last_became_active":"2024-07-04T17:32:02.603201+0000","last_became_peered":"2024-07-04T17:32:02.603201+0000","last_unstale":"2024-07-04T17:32:03.994869+0000","last_undegraded":"2024-07-04T17:32:03.994869+0000","last_fullsized":"2024-07-04T17:32:03.994869+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:30:29.511665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.992079+0000","last_change":"2024-07-04T17:31:42.148476+0000","last_active":"2024-07-04T17:32:03.992079+0000","last_peered":"2024-07-04T17:32:03.992079+0000","last_clean":"2024-07-04T17:32:03.992079+0000","last_became_active":"2024-07-04T17:31:42.148083+0000","last_became_peered":"2024-07-04T17:31:42.148083+0000","last_unstale":"2024-07-04T17:32:03.992079+0000","last_undegraded":"2024-07-04T17:32:03.992079+0000","last_fullsized":"2024-07-04T17:32:03.992079+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:19:47.089588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.3f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.576559+0000","last_change":"2024-07-04T17:32:02.609478+0000","last_active":"2024-07-04T17:32:03.576559+0000","last_peered":"2024-07-04T17:32:03.576559+0000","last_clean":"2024-07-04T17:32:03.576559+0000","last_became_active":"2024-07-04T17:32:02.603062+0000","last_became_peered":"2024-07-04T17:32:02.603062+0000","last_unstale":"2024-07-04T17:32:03.576559+0000","last_undegraded":"2024-07-04T17:32:03.576559+0000","last_fullsized":"2024-07-04T17:32:03.576559+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:30:37.885691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.994489+0000","last_change":"2024-07-04T17:31:42.146020+0000","last_active":"2024-07-04T17:32:03.994489+0000","last_peered":"2024-07-04T17:32:03.994489+0000","last_clean":"2024-07-04T17:32:03.994489+0000","last_became_active":"2024-07-04T17:31:42.143682+0000","last_became_peered":"2024-07-04T17:31:42.143682+0000","last_unstale":"2024-07-04T17:32:03.994489+0000","last_undegraded":"2024-07-04T17:32:03.994489+0000","last_fullsized":"2024-07-04T17:32:03.994489+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:06:33.198375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.574022+0000","last_change":"2024-07-04T17:32:02.581917+0000","last_active":"2024-07-04T17:32:03.574022+0000","last_peered":"2024-07-04T17:32:03.574022+0000","last_clean":"2024-07-04T17:32:03.574022+0000","last_became_active":"2024-07-04T17:32:02.581630+0000","last_became_peered":"2024-07-04T17:32:02.581630+0000","last_unstale":"2024-07-04T17:32:03.574022+0000","last_undegraded":"2024-07-04T17:32:03.574022+0000","last_fullsized":"2024-07-04T17:32:03.574022+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:45:45.913107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.575361+0000","last_change":"2024-07-04T17:31:42.132420+0000","last_active":"2024-07-04T17:32:03.575361+0000","last_peered":"2024-07-04T17:32:03.575361+0000","last_clean":"2024-07-04T17:32:03.575361+0000","last_became_active":"2024-07-04T17:31:42.132031+0000","last_became_peered":"2024-07-04T17:31:42.132031+0000","last_unstale":"2024-07-04T17:32:03.575361+0000","last_undegraded":"2024-07-04T17:32:03.575361+0000","last_fullsized":"2024-07-04T17:32:03.575361+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:41:23.204256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.990322+0000","last_change":"2024-07-04T17:32:02.604131+0000","last_active":"2024-07-04T17:32:03.990322+0000","last_peered":"2024-07-04T17:32:03.990322+0000","last_clean":"2024-07-04T17:32:03.990322+0000","last_became_active":"2024-07-04T17:32:02.603425+0000","last_became_peered":"2024-07-04T17:32:02.603425+0000","last_unstale":"2024-07-04T17:32:03.990322+0000","last_undegraded":"2024-07-04T17:32:03.990322+0000","last_fullsized":"2024-07-04T17:32:03.990322+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:32:01.527353+0000","last_clean_scrub_stamp":"2024-07-04T17:32:01.527353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:05:44.906220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3f","version":"0'0","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-07-04T17:32:03.994680+0000","last_change":"2024-07-04T17:31:42.147550+0000","last_active":"2024-07-04T17:32:03.994680+0000","last_peered":"2024-07-04T17:32:03.994680+0000","last_clean":"2024-07-04T17:32:03.994680+0000","last_became_active":"2024-07-04T17:31:42.146638+0000","last_became_peered":"2024-07-04T17:31:42.146638+0000","last_unstale":"2024-07-04T17:32:03.994680+0000","last_undegraded":"2024-07-04T17:32:03.994680+0000","last_fullsized":"2024-07-04T17:32:03.994680+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:31:41.063968+0000","last_clean_scrub_stamp":"2024-07-04T17:31:41.063968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:17:39.808949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":4107,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":356,"num_read_kb":926,"num_write":139,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738440,"num_pgs":128,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36316,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335524,"statfs":{"total":96636764160,"available":96599576576,"internally_reserved":0,"allocated":995328,"data_stored":385750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738441,"num_pgs":103,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36928,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334912,"statfs":{"total":96636764160,"available":96598949888,"internally_reserved":0,"allocated":1617920,"data_stored":980653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36928,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334912,"statfs":{"total":96636764160,"available":96598949888,"internally_reserved":0,"allocated":1617920,"data_stored":980585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:32:07.929 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-04T17:32:07.929 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:32:08.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.261+0000 7f4a64e00640 1 -- 172.21.15.42:0/2280189592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a6006cc80 msgr2=0x7f4a600b3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:08.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.261+0000 7f4a64e00640 1 --2- 172.21.15.42:0/2280189592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a6006cc80 0x7f4a600b3650 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c005fd0 tx=0x7f4a4c02a5f0 comp rx=0 tx=0).stop 2024-07-04T17:32:08.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.261+0000 7f4a64e00640 1 -- 172.21.15.42:0/2280189592 shutdown_connections 2024-07-04T17:32:08.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.261+0000 7f4a64e00640 1 --2- 172.21.15.42:0/2280189592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a6006cc80 0x7f4a600b3650 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:08.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.261+0000 7f4a64e00640 1 -- 172.21.15.42:0/2280189592 >> 172.21.15.42:0/2280189592 conn(0x7f4a600bf700 msgr2=0x7f4a600af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:08.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.261+0000 7f4a64e00640 1 -- 172.21.15.42:0/2280189592 shutdown_connections 2024-07-04T17:32:08.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.261+0000 7f4a64e00640 1 -- 172.21.15.42:0/2280189592 wait complete. 2024-07-04T17:32:08.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.263+0000 7f4a64e00640 1 Processor -- start 2024-07-04T17:32:08.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.264+0000 7f4a64e00640 1 -- start start 2024-07-04T17:32:08.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.265+0000 7f4a64e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a6006cc80 0x7f4a601da9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:08.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.265+0000 7f4a64e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4a601463b0 con 0x7f4a6006cc80 2024-07-04T17:32:08.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.265+0000 7f4a5de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a6006cc80 0x7f4a601da9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:08.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.265+0000 7f4a5de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a6006cc80 0x7f4a601da9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35444/0 (socket says 172.21.15.42:35444) 2024-07-04T17:32:08.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.265+0000 7f4a5de00640 1 -- 172.21.15.42:0/133370320 learned_addr learned my addr 172.21.15.42:0/133370320 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:08.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.266+0000 7f4a5de00640 1 -- 172.21.15.42:0/133370320 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a600b6580 con 0x7f4a6006cc80 2024-07-04T17:32:08.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.266+0000 7f4a5de00640 1 --2- 172.21.15.42:0/133370320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a6006cc80 0x7f4a601da9c0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f4a5000c380 tx=0x7f4a5000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:08.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.266+0000 7f4a56000640 1 -- 172.21.15.42:0/133370320 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a50017070 con 0x7f4a6006cc80 2024-07-04T17:32:08.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.266+0000 7f4a56000640 1 -- 172.21.15.42:0/133370320 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a500052f0 con 0x7f4a6006cc80 2024-07-04T17:32:08.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.266+0000 7f4a56000640 1 -- 172.21.15.42:0/133370320 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a500035f0 con 0x7f4a6006cc80 2024-07-04T17:32:08.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.266+0000 7f4a64e00640 1 -- 172.21.15.42:0/133370320 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a60143810 con 0x7f4a6006cc80 2024-07-04T17:32:08.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.267+0000 7f4a64e00640 1 -- 172.21.15.42:0/133370320 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a60143310 con 0x7f4a6006cc80 2024-07-04T17:32:08.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.268+0000 7f4a64e00640 1 -- 172.21.15.42:0/133370320 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a60143a40 con 0x7f4a6006cc80 2024-07-04T17:32:08.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.268+0000 7f4a56000640 1 -- 172.21.15.42:0/133370320 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7f4a5001a070 con 0x7f4a6006cc80 2024-07-04T17:32:08.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.268+0000 7f4a56000640 1 --2- 172.21.15.42:0/133370320 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f4a4003d3d0 0x7f4a4003f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:08.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.269+0000 7f4a56000640 1 -- 172.21.15.42:0/133370320 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4303+0+0 (secure 0 0 0) 0x7f4a50050dc0 con 0x7f4a6006cc80 2024-07-04T17:32:08.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.269+0000 7f4a5d400640 1 --2- 172.21.15.42:0/133370320 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f4a4003d3d0 0x7f4a4003f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:08.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.270+0000 7f4a5d400640 1 --2- 172.21.15.42:0/133370320 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f4a4003d3d0 0x7f4a4003f8e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c033000 tx=0x7f4a4c004730 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:08.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.274+0000 7f4a56000640 1 -- 172.21.15.42:0/133370320 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f4a600b06c0 con 0x7f4a6006cc80 2024-07-04T17:32:08.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.524+0000 7f4a64e00640 1 -- 172.21.15.42:0/133370320 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a600b3650 con 0x7f4a6006cc80 2024-07-04T17:32:08.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.526+0000 7f4a56000640 1 -- 172.21.15.42:0/133370320 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+7357 (secure 0 0 0) 0x7f4a50016170 con 0x7f4a6006cc80 2024-07-04T17:32:08.526 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:32:08.526 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":77,"btime":"2024-07-04T17:32:05:576854+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/244742173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":244742173},{"type":"v1","addr":"172.21.15.42:6825","nonce":244742173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5265,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/973681058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":973681058},{"type":"v1","addr":"172.21.15.42:6827","nonce":973681058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/480649366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":480649366},{"type":"v1","addr":"172.21.15.42:6829","nonce":480649366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:31:41.664414+0000","modified":"2024-07-04T17:31:44.709115+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5277":{"gid":5277,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6833/3777807763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":3777807763},{"type":"v1","addr":"172.21.15.42:6833","nonce":3777807763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5277,"qdb_cluster":[5277]},"id":8},{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:32:02.103946+0000","modified":"2024-07-04T17:32:05.184521+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5276":{"gid":5276,"name":"d","rank":0,"incarnation":73,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6831/327636003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":327636003},{"type":"v1","addr":"172.21.15.42:6831","nonce":327636003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5276,"qdb_cluster":[5276]},"id":9}]} 2024-07-04T17:32:08.527 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 77 2024-07-04T17:32:08.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.529+0000 7f4a64e00640 1 -- 172.21.15.42:0/133370320 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f4a4003d3d0 msgr2=0x7f4a4003f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:08.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.529+0000 7f4a64e00640 1 --2- 172.21.15.42:0/133370320 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f4a4003d3d0 0x7f4a4003f8e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c033000 tx=0x7f4a4c004730 comp rx=0 tx=0).stop 2024-07-04T17:32:08.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.529+0000 7f4a64e00640 1 -- 172.21.15.42:0/133370320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a6006cc80 msgr2=0x7f4a601da9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:08.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.529+0000 7f4a64e00640 1 --2- 172.21.15.42:0/133370320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a6006cc80 0x7f4a601da9c0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f4a5000c380 tx=0x7f4a5000c840 comp rx=0 tx=0).stop 2024-07-04T17:32:08.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.529+0000 7f4a64e00640 1 -- 172.21.15.42:0/133370320 shutdown_connections 2024-07-04T17:32:08.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.529+0000 7f4a64e00640 1 --2- 172.21.15.42:0/133370320 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7f4a4003d3d0 0x7f4a4003f8e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:08.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.529+0000 7f4a64e00640 1 --2- 172.21.15.42:0/133370320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a6006cc80 0x7f4a601da9c0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:08.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.529+0000 7f4a64e00640 1 -- 172.21.15.42:0/133370320 >> 172.21.15.42:0/133370320 conn(0x7f4a600bf700 msgr2=0x7f4a60075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:08.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.529+0000 7f4a64e00640 1 -- 172.21.15.42:0/133370320 shutdown_connections 2024-07-04T17:32:08.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:08.529+0000 7f4a64e00640 1 -- 172.21.15.42:0/133370320 wait complete. 2024-07-04T17:32:09.059 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 77, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-04T17:32:02.103946+0000', 'modified': '2024-07-04T17:32:05.184521+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5276}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5276': {'gid': 5276, 'name': 'd', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.42:6831/327636003', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6830', 'nonce': 327636003}, {'type': 'v1', 'addr': '172.21.15.42:6831', 'nonce': 327636003}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'backup_fs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5276, 'qdb_cluster': [5276]} 2024-07-04T17:32:09.059 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-04T17:32:09.059 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-07-04T17:32:09.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.361+0000 7fb7b6600640 1 -- 172.21.15.42:0/4270792045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7b006c020 msgr2=0x7fb7b014a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:09.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.361+0000 7fb7b6600640 1 --2- 172.21.15.42:0/4270792045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7b006c020 0x7fb7b014a9e0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fb79c005fd0 tx=0x7fb79c02a5f0 comp rx=0 tx=0).stop 2024-07-04T17:32:09.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.362+0000 7fb7b6600640 1 -- 172.21.15.42:0/4270792045 shutdown_connections 2024-07-04T17:32:09.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.362+0000 7fb7b6600640 1 --2- 172.21.15.42:0/4270792045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7b006c020 0x7fb7b014a9e0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:09.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.362+0000 7fb7b6600640 1 -- 172.21.15.42:0/4270792045 >> 172.21.15.42:0/4270792045 conn(0x7fb7b00bebc0 msgr2=0x7fb7b00befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:09.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.362+0000 7fb7b6600640 1 -- 172.21.15.42:0/4270792045 shutdown_connections 2024-07-04T17:32:09.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.362+0000 7fb7b6600640 1 -- 172.21.15.42:0/4270792045 wait complete. 2024-07-04T17:32:09.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.364+0000 7fb7b6600640 1 Processor -- start 2024-07-04T17:32:09.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.364+0000 7fb7b6600640 1 -- start start 2024-07-04T17:32:09.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.365+0000 7fb7b6600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7b006c020 0x7fb7b0072b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:09.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.365+0000 7fb7b6600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b014b550 con 0x7fb7b006c020 2024-07-04T17:32:09.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.366+0000 7fb7b5200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7b006c020 0x7fb7b0072b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:09.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.366+0000 7fb7b5200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7b006c020 0x7fb7b0072b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35456/0 (socket says 172.21.15.42:35456) 2024-07-04T17:32:09.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.366+0000 7fb7b5200640 1 -- 172.21.15.42:0/215875137 learned_addr learned my addr 172.21.15.42:0/215875137 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:09.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.366+0000 7fb7b5200640 1 -- 172.21.15.42:0/215875137 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b0070060 con 0x7fb7b006c020 2024-07-04T17:32:09.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.367+0000 7fb7b5200640 1 --2- 172.21.15.42:0/215875137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7b006c020 0x7fb7b0072b60 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4001eb0 tx=0x7fb7a400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:09.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.367+0000 7fb7acc00640 1 -- 172.21.15.42:0/215875137 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7a4017070 con 0x7fb7b006c020 2024-07-04T17:32:09.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.367+0000 7fb7acc00640 1 -- 172.21.15.42:0/215875137 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7a40052f0 con 0x7fb7b006c020 2024-07-04T17:32:09.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.367+0000 7fb7b6600640 1 -- 172.21.15.42:0/215875137 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7b0149990 con 0x7fb7b006c020 2024-07-04T17:32:09.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.367+0000 7fb7acc00640 1 -- 172.21.15.42:0/215875137 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7a40035f0 con 0x7fb7b006c020 2024-07-04T17:32:09.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.367+0000 7fb7b6600640 1 -- 172.21.15.42:0/215875137 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7b006fe70 con 0x7fb7b006c020 2024-07-04T17:32:09.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.369+0000 7fb7acc00640 1 -- 172.21.15.42:0/215875137 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7fb7a401a070 con 0x7fb7b006c020 2024-07-04T17:32:09.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.369+0000 7fb7acc00640 1 --2- 172.21.15.42:0/215875137 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fb79003d380 0x7fb79003f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:09.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.369+0000 7fb7acc00640 1 -- 172.21.15.42:0/215875137 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4303+0+0 (secure 0 0 0) 0x7fb7a4050d30 con 0x7fb7b006c020 2024-07-04T17:32:09.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.369+0000 7fb7afe00640 1 --2- 172.21.15.42:0/215875137 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fb79003d380 0x7fb79003f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:09.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.370+0000 7fb7afe00640 1 --2- 172.21.15.42:0/215875137 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fb79003d380 0x7fb79003f890 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb79c033000 tx=0x7fb79c004730 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:09.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.371+0000 7fb7beb96740 1 Processor -- start 2024-07-04T17:32:09.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.372+0000 7fb7beb96740 1 -- start start 2024-07-04T17:32:09.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.373+0000 7fb7beb96740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d96f31270 0x555d972d7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:09.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.373+0000 7fb7beb96740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x555d972485b0 con 0x555d96f31270 2024-07-04T17:32:09.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.373+0000 7fb7b5200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d96f31270 0x555d972d7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:09.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.373+0000 7fb7b5200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d96f31270 0x555d972d7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35470/0 (socket says 172.21.15.42:35470) 2024-07-04T17:32:09.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.373+0000 7fb7b5200640 1 -- 172.21.15.42:0/3812427592 learned_addr learned my addr 172.21.15.42:0/3812427592 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:09.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.373+0000 7fb7b5200640 1 -- 172.21.15.42:0/3812427592 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d96f9c850 con 0x555d96f31270 2024-07-04T17:32:09.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.374+0000 7fb7b5200640 1 --2- 172.21.15.42:0/3812427592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d96f31270 0x555d972d7d30 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4007bf0 tx=0x7fb7a400e3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be953eed766e48d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:09.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.374+0000 7fb7a1600640 1 -- 172.21.15.42:0/3812427592 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7a401be00 con 0x555d96f31270 2024-07-04T17:32:09.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.374+0000 7fb7a1600640 1 -- 172.21.15.42:0/3812427592 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7a401edf0 con 0x555d96f31270 2024-07-04T17:32:09.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.374+0000 7fb7a1600640 1 -- 172.21.15.42:0/3812427592 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7a40486f0 con 0x555d96f31270 2024-07-04T17:32:09.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.375+0000 7fb7beb96740 1 -- 172.21.15.42:0/3812427592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d96f31270 msgr2=0x555d972d7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:09.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.375+0000 7fb7beb96740 1 --2- 172.21.15.42:0/3812427592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d96f31270 0x555d972d7d30 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4007bf0 tx=0x7fb7a400e3a0 comp rx=0 tx=0).stop 2024-07-04T17:32:09.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.375+0000 7fb7beb96740 1 -- 172.21.15.42:0/3812427592 shutdown_connections 2024-07-04T17:32:09.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.375+0000 7fb7beb96740 1 --2- 172.21.15.42:0/3812427592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d96f31270 0x555d972d7d30 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:09.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.375+0000 7fb7beb96740 1 -- 172.21.15.42:0/3812427592 >> 172.21.15.42:0/3812427592 conn(0x555d96f253c0 msgr2=0x555d972d64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:09.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.375+0000 7fb7beb96740 1 -- 172.21.15.42:0/3812427592 shutdown_connections 2024-07-04T17:32:09.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.375+0000 7fb7beb96740 1 -- 172.21.15.42:0/3812427592 wait complete. 2024-07-04T17:32:09.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.376+0000 7fb7beb96740 1 -- start start 2024-07-04T17:32:09.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.376+0000 7fb7beb96740 1 Processor -- start 2024-07-04T17:32:09.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.377+0000 7fb7beb96740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d96f31270 0x555d972d28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:09.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.377+0000 7fb7beb96740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x555d9723bb90 con 0x555d96f31270 2024-07-04T17:32:09.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.377+0000 7fb7beb96740 10 client.0 _ll_register_callbacks cb 0x555d971e08a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T17:32:09.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.378+0000 7fb7b5200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d96f31270 0x555d972d28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:09.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.378+0000 7fb7b5200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d96f31270 0x555d972d28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35480/0 (socket says 172.21.15.42:35480) 2024-07-04T17:32:09.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.378+0000 7fb7b5200640 1 -- 172.21.15.42:0/1522903975 learned_addr learned my addr 172.21.15.42:0/1522903975 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:09.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.378+0000 7fb7b5200640 1 -- 172.21.15.42:0/1522903975 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d970136b0 con 0x555d96f31270 2024-07-04T17:32:09.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.378+0000 7fb7b5200640 1 --2- 172.21.15.42:0/1522903975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d96f31270 0x555d972d28f0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4005040 tx=0x7fb7a404f780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6409e9517d43d4eb server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:09.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.378+0000 7fb796000640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-07-04T17:32:09.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.378+0000 7fb796000640 1 -- 172.21.15.42:0/1522903975 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7a4047070 con 0x555d96f31270 2024-07-04T17:32:09.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.379+0000 7fb796000640 1 -- 172.21.15.42:0/1522903975 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 4303+0+0 (secure 0 0 0) 0x7fb7a4048ad0 con 0x555d96f31270 2024-07-04T17:32:09.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.379+0000 7fb7beb96740 1 -- 172.21.15.42:0/1522903975 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555d971d86c0 con 0x555d96f31270 2024-07-04T17:32:09.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.379+0000 7fb796000640 1 -- 172.21.15.42:0/1522903975 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fb7a402a050 con 0x555d96f31270 2024-07-04T17:32:09.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.379+0000 7fb7beb96740 10 client.5388 fetch_fsmap learned FSMap version 77 2024-07-04T17:32:09.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.379+0000 7fb7beb96740 1 -- 172.21.15.42:0/1522903975 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x555d972fa520 con 0x555d96f31270 2024-07-04T17:32:09.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.379+0000 7fb796000640 1 -- 172.21.15.42:0/1522903975 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x555d971d86c0 con 0x555d96f31270 2024-07-04T17:32:09.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.379+0000 7fb796000640 1 -- 172.21.15.42:0/1522903975 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 77) ==== 4754+0+0 (secure 0 0 0) 0x7fb7a402d020 con 0x555d96f31270 2024-07-04T17:32:09.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.379+0000 7fb7beb96740 10 client.5388 fetch_fsmap finished waiting for FSMap version 77 2024-07-04T17:32:09.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.379+0000 7fb7beb96740 10 client.5388 resolve_mds: resolved name 'd' to daemon mds.d 2024-07-04T17:32:09.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.379+0000 7fb7beb96740 20 client.5388 populate_metadata read hostname 'smithi042' 2024-07-04T17:32:09.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.379+0000 7fb7beb96740 1 --2- 172.21.15.42:0/1522903975 >> [v2:172.21.15.42:6830/327636003,v1:172.21.15.42:6831/327636003] conn(0x555d96f083b0 0x555d972ff980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:09.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.379+0000 7fb7beb96740 4 client.5388 mds_command: new command op to 5276 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T17:32:09.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.379+0000 7fb7beb96740 1 -- 172.21.15.42:0/1522903975 --> [v2:172.21.15.42:6830/327636003,v1:172.21.15.42:6831/327636003] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d9724a500 con 0x555d96f083b0 2024-07-04T17:32:09.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.380+0000 7fb7afe00640 1 --2- 172.21.15.42:0/1522903975 >> [v2:172.21.15.42:6830/327636003,v1:172.21.15.42:6831/327636003] conn(0x555d96f083b0 0x555d972ff980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:09.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.380+0000 7fb7afe00640 1 --2- 172.21.15.42:0/1522903975 >> [v2:172.21.15.42:6830/327636003,v1:172.21.15.42:6831/327636003] conn(0x555d96f083b0 0x555d972ff980 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=854e17d77ec87f13 server_cookie=2a4b6bf0d209f3b3 in_seq=0 out_seq=0 2024-07-04T17:32:09.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.380+0000 7fb796000640 10 client.5388 ms_handle_connect on v2:172.21.15.42:6830/327636003 2024-07-04T17:32:09.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.381+0000 7fb796000640 1 -- 172.21.15.42:0/1522903975 <== mds.0 v2:172.21.15.42:6830/327636003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555d9724a500 con 0x555d96f083b0 2024-07-04T17:32:09.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.381+0000 7fb796000640 10 client.5388 handle_command_reply: tid=0 2024-07-04T17:32:09.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.381+0000 7fb7beb96740 1 client.5388 shutdown 2024-07-04T17:32:09.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.381+0000 7fb7beb96740 1 -- 172.21.15.42:0/1522903975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d96f31270 msgr2=0x555d972d28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:09.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.381+0000 7fb7beb96740 1 --2- 172.21.15.42:0/1522903975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d96f31270 0x555d972d28f0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4005040 tx=0x7fb7a404f780 comp rx=0 tx=0).stop 2024-07-04T17:32:09.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.382+0000 7fb7beb96740 1 -- 172.21.15.42:0/1522903975 shutdown_connections 2024-07-04T17:32:09.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.382+0000 7fb7beb96740 1 --2- 172.21.15.42:0/1522903975 >> [v2:172.21.15.42:6830/327636003,v1:172.21.15.42:6831/327636003] conn(0x555d96f083b0 0x555d972ff980 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:09.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.382+0000 7fb7beb96740 1 --2- 172.21.15.42:0/1522903975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d96f31270 0x555d972d28f0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:09.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.382+0000 7fb7beb96740 1 -- 172.21.15.42:0/1522903975 >> 172.21.15.42:0/1522903975 conn(0x555d96f253c0 msgr2=0x555d972d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:09.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.382+0000 7fb796000640 0 client.5388 ms_handle_reset on v2:172.21.15.42:6830/327636003 2024-07-04T17:32:09.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.382+0000 7fb7beb96740 1 -- 172.21.15.42:0/1522903975 shutdown_connections 2024-07-04T17:32:09.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.382+0000 7fb7beb96740 1 -- 172.21.15.42:0/1522903975 wait complete. 2024-07-04T17:32:09.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.382+0000 7fb7beb96740 20 client.5388 trim_cache size 0 max 16384 2024-07-04T17:32:09.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.402+0000 7fb7beb96740 1 Processor -- start 2024-07-04T17:32:09.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.402+0000 7fb7beb96740 1 -- start start 2024-07-04T17:32:09.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.403+0000 7fb7beb96740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d96f31270 0x555d972cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:09.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.403+0000 7fb7beb96740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b00a9620 con 0x555d96f31270 2024-07-04T17:32:09.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.403+0000 7fb7b5200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d96f31270 0x555d972cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:09.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.403+0000 7fb7b5200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d96f31270 0x555d972cd7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35482/0 (socket says 172.21.15.42:35482) 2024-07-04T17:32:09.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.403+0000 7fb7b5200640 1 -- 172.21.15.42:0/4179790251 learned_addr learned my addr 172.21.15.42:0/4179790251 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:09.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.404+0000 7fb7b5200640 1 -- 172.21.15.42:0/4179790251 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d972f09c0 con 0x555d96f31270 2024-07-04T17:32:09.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.404+0000 7fb7b5200640 1 --2- 172.21.15.42:0/4179790251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d96f31270 0x555d972cd7a0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400cd60 tx=0x7fb7a4003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564f581c0d072b39 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:09.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.404+0000 7fb7a0c00640 1 -- 172.21.15.42:0/4179790251 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7a4047070 con 0x555d96f31270 2024-07-04T17:32:09.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.404+0000 7fb7a0c00640 1 -- 172.21.15.42:0/4179790251 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7a40489a0 con 0x555d96f31270 2024-07-04T17:32:09.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.404+0000 7fb7a0c00640 1 -- 172.21.15.42:0/4179790251 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7a4003970 con 0x555d96f31270 2024-07-04T17:32:09.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.405+0000 7fb7beb96740 1 -- 172.21.15.42:0/4179790251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d96f31270 msgr2=0x555d972cd7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:09.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.405+0000 7fb7beb96740 1 --2- 172.21.15.42:0/4179790251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d96f31270 0x555d972cd7a0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400cd60 tx=0x7fb7a4003f90 comp rx=0 tx=0).stop 2024-07-04T17:32:09.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.405+0000 7fb7beb96740 1 -- 172.21.15.42:0/4179790251 shutdown_connections 2024-07-04T17:32:09.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.405+0000 7fb7beb96740 1 --2- 172.21.15.42:0/4179790251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d96f31270 0x555d972cd7a0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:09.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.405+0000 7fb7beb96740 1 -- 172.21.15.42:0/4179790251 >> 172.21.15.42:0/4179790251 conn(0x555d96f253c0 msgr2=0x555d973037c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:09.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.405+0000 7fb7beb96740 1 -- 172.21.15.42:0/4179790251 shutdown_connections 2024-07-04T17:32:09.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.405+0000 7fb7beb96740 1 -- 172.21.15.42:0/4179790251 wait complete. 2024-07-04T17:32:09.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.406+0000 7fb7beb96740 1 -- start start 2024-07-04T17:32:09.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.406+0000 7fb7beb96740 1 Processor -- start 2024-07-04T17:32:09.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.408+0000 7fb7beb96740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d96f31270 0x555d97329840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:09.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.408+0000 7fb7beb96740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x555d97237b50 con 0x555d96f31270 2024-07-04T17:32:09.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.408+0000 7fb7beb96740 10 client.0 _ll_register_callbacks cb 0x7fb7a40102e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T17:32:09.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.408+0000 7fb7b5200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d96f31270 0x555d97329840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:09.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.408+0000 7fb7b5200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d96f31270 0x555d97329840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35490/0 (socket says 172.21.15.42:35490) 2024-07-04T17:32:09.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.408+0000 7fb7b5200640 1 -- 172.21.15.42:0/2153615277 learned_addr learned my addr 172.21.15.42:0/2153615277 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:09.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.408+0000 7fb7b5200640 1 -- 172.21.15.42:0/2153615277 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d972d5ef0 con 0x555d96f31270 2024-07-04T17:32:09.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.408+0000 7fb7b5200640 1 --2- 172.21.15.42:0/2153615277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d96f31270 0x555d97329840 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4007fd0 tx=0x7fb7a4003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa05df1626f71ed server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:09.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.408+0000 7fb796000640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-07-04T17:32:09.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.409+0000 7fb796000640 1 -- 172.21.15.42:0/2153615277 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7a4047070 con 0x555d96f31270 2024-07-04T17:32:09.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.409+0000 7fb796000640 1 -- 172.21.15.42:0/2153615277 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 4303+0+0 (secure 0 0 0) 0x7fb7a4016170 con 0x555d96f31270 2024-07-04T17:32:09.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.409+0000 7fb7beb96740 1 -- 172.21.15.42:0/2153615277 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7a40489a0 con 0x555d96f31270 2024-07-04T17:32:09.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.409+0000 7fb796000640 1 -- 172.21.15.42:0/2153615277 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fb7a40163c0 con 0x555d96f31270 2024-07-04T17:32:09.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.409+0000 7fb7beb96740 10 client.5390 fetch_fsmap learned FSMap version 77 2024-07-04T17:32:09.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.409+0000 7fb7beb96740 1 -- 172.21.15.42:0/2153615277 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x555d972d5ef0 con 0x555d96f31270 2024-07-04T17:32:09.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.409+0000 7fb796000640 1 -- 172.21.15.42:0/2153615277 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7a40489a0 con 0x555d96f31270 2024-07-04T17:32:09.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.409+0000 7fb796000640 1 -- 172.21.15.42:0/2153615277 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 77) ==== 4754+0+0 (secure 0 0 0) 0x7fb7a402d030 con 0x555d96f31270 2024-07-04T17:32:09.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.409+0000 7fb7beb96740 10 client.5390 fetch_fsmap finished waiting for FSMap version 77 2024-07-04T17:32:09.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.409+0000 7fb7beb96740 10 client.5390 resolve_mds: resolved name 'd' to daemon mds.d 2024-07-04T17:32:09.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.409+0000 7fb7beb96740 20 client.5390 populate_metadata read hostname 'smithi042' 2024-07-04T17:32:09.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.410+0000 7fb7beb96740 1 --2- 172.21.15.42:0/2153615277 >> [v2:172.21.15.42:6830/327636003,v1:172.21.15.42:6831/327636003] conn(0x555d96eefe40 0x555d973832c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:09.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.410+0000 7fb7beb96740 4 client.5390 mds_command: new command op to 5276 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-04T17:32:09.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.410+0000 7fb7beb96740 1 -- 172.21.15.42:0/2153615277 --> [v2:172.21.15.42:6830/327636003,v1:172.21.15.42:6831/327636003] -- command(tid 0: {"prefix": "status"}) -- 0x555d972fd720 con 0x555d96eefe40 2024-07-04T17:32:09.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.410+0000 7fb7afe00640 1 --2- 172.21.15.42:0/2153615277 >> [v2:172.21.15.42:6830/327636003,v1:172.21.15.42:6831/327636003] conn(0x555d96eefe40 0x555d973832c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:09.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.410+0000 7fb7afe00640 1 --2- 172.21.15.42:0/2153615277 >> [v2:172.21.15.42:6830/327636003,v1:172.21.15.42:6831/327636003] conn(0x555d96eefe40 0x555d973832c0 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8980382c5bd15af3 server_cookie=200c839b2af6fffa in_seq=0 out_seq=0 2024-07-04T17:32:09.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.410+0000 7fb796000640 10 client.5390 ms_handle_connect on v2:172.21.15.42:6830/327636003 2024-07-04T17:32:09.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.411+0000 7fb796000640 1 -- 172.21.15.42:0/2153615277 <== mds.0 v2:172.21.15.42:6830/327636003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x555d972fd720 con 0x555d96eefe40 2024-07-04T17:32:09.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.411+0000 7fb796000640 10 client.5390 handle_command_reply: tid=0 2024-07-04T17:32:09.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.411+0000 7fb7beb96740 1 client.5390 shutdown 2024-07-04T17:32:09.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.411+0000 7fb7beb96740 1 -- 172.21.15.42:0/2153615277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d96f31270 msgr2=0x555d97329840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:09.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.411+0000 7fb7beb96740 1 --2- 172.21.15.42:0/2153615277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d96f31270 0x555d97329840 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4007fd0 tx=0x7fb7a4003e00 comp rx=0 tx=0).stop 2024-07-04T17:32:09.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.411+0000 7fb7beb96740 1 -- 172.21.15.42:0/2153615277 shutdown_connections 2024-07-04T17:32:09.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.411+0000 7fb7beb96740 1 --2- 172.21.15.42:0/2153615277 >> [v2:172.21.15.42:6830/327636003,v1:172.21.15.42:6831/327636003] conn(0x555d96eefe40 0x555d973832c0 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:09.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.411+0000 7fb7beb96740 1 --2- 172.21.15.42:0/2153615277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555d96f31270 0x555d97329840 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:09.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.411+0000 7fb796000640 0 client.5390 ms_handle_reset on v2:172.21.15.42:6830/327636003 2024-07-04T17:32:09.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.411+0000 7fb7beb96740 1 -- 172.21.15.42:0/2153615277 >> 172.21.15.42:0/2153615277 conn(0x555d96f253c0 msgr2=0x555d972ec9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:09.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.411+0000 7fb7beb96740 1 -- 172.21.15.42:0/2153615277 shutdown_connections 2024-07-04T17:32:09.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.411+0000 7fb7beb96740 1 -- 172.21.15.42:0/2153615277 wait complete. 2024-07-04T17:32:09.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.411+0000 7fb7beb96740 20 client.5390 trim_cache size 0 max 16384 2024-07-04T17:32:09.414 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:32:09.414 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "be4f300a-93ef-4489-b540-37ece39b0aac", 2024-07-04T17:32:09.414 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-07-04T17:32:09.414 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 5276, 2024-07-04T17:32:09.414 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-07-04T17:32:09.414 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-07-04T17:32:09.415 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "backup_fs", 2024-07-04T17:32:09.415 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 6.8652826869999997, 2024-07-04T17:32:09.415 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 77, 2024-07-04T17:32:09.415 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 96, 2024-07-04T17:32:09.415 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 96, 2024-07-04T17:32:09.415 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 32.128151236999997 2024-07-04T17:32:09.415 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:32:09.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.412+0000 7fb7b6600640 1 -- 172.21.15.42:0/215875137 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fb79003d380 msgr2=0x7fb79003f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:09.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.412+0000 7fb7b6600640 1 --2- 172.21.15.42:0/215875137 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fb79003d380 0x7fb79003f890 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb79c033000 tx=0x7fb79c004730 comp rx=0 tx=0).stop 2024-07-04T17:32:09.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.412+0000 7fb7b6600640 1 -- 172.21.15.42:0/215875137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7b006c020 msgr2=0x7fb7b0072b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:09.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.412+0000 7fb7b6600640 1 --2- 172.21.15.42:0/215875137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7b006c020 0x7fb7b0072b60 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4001eb0 tx=0x7fb7a400c810 comp rx=0 tx=0).stop 2024-07-04T17:32:09.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.412+0000 7fb7b6600640 1 -- 172.21.15.42:0/215875137 shutdown_connections 2024-07-04T17:32:09.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.412+0000 7fb7b6600640 1 --2- 172.21.15.42:0/215875137 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fb79003d380 0x7fb79003f890 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:09.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.412+0000 7fb7b6600640 1 --2- 172.21.15.42:0/215875137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7b006c020 0x7fb7b0072b60 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:09.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.412+0000 7fb7b6600640 1 -- 172.21.15.42:0/215875137 >> 172.21.15.42:0/215875137 conn(0x7fb7b00bebc0 msgr2=0x7fb7b00b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:09.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.412+0000 7fb7b6600640 1 -- 172.21.15.42:0/215875137 shutdown_connections 2024-07-04T17:32:09.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.412+0000 7fb7b6600640 1 -- 172.21.15.42:0/215875137 wait complete. 2024-07-04T17:32:09.436 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable mirroring 2024-07-04T17:32:09.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.730+0000 7fd523e00640 1 -- 172.21.15.42:0/3657978875 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd51c0b1c10 msgr2=0x7fd51c0b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:09.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.730+0000 7fd523e00640 1 --2- 172.21.15.42:0/3657978875 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd51c0b1c10 0x7fd51c0b30d0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fd50c005fd0 tx=0x7fd50c02bb50 comp rx=0 tx=0).stop 2024-07-04T17:32:09.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.730+0000 7fd523e00640 1 -- 172.21.15.42:0/3657978875 shutdown_connections 2024-07-04T17:32:09.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.730+0000 7fd523e00640 1 --2- 172.21.15.42:0/3657978875 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd51c0b1c10 0x7fd51c0b30d0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:09.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.731+0000 7fd523e00640 1 -- 172.21.15.42:0/3657978875 >> 172.21.15.42:0/3657978875 conn(0x7fd51c06fe40 msgr2=0x7fd51c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:09.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.731+0000 7fd523e00640 1 -- 172.21.15.42:0/3657978875 shutdown_connections 2024-07-04T17:32:09.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.731+0000 7fd523e00640 1 -- 172.21.15.42:0/3657978875 wait complete. 2024-07-04T17:32:09.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.733+0000 7fd523e00640 1 Processor -- start 2024-07-04T17:32:09.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.733+0000 7fd523e00640 1 -- start start 2024-07-04T17:32:09.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.734+0000 7fd523e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd51c070d60 0x7fd51c0ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:09.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.734+0000 7fd523e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c0b2a40 con 0x7fd51c070d60 2024-07-04T17:32:09.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.735+0000 7fd521400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd51c070d60 0x7fd51c0ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:09.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.735+0000 7fd521400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd51c070d60 0x7fd51c0ae200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35506/0 (socket says 172.21.15.42:35506) 2024-07-04T17:32:09.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.735+0000 7fd521400640 1 -- 172.21.15.42:0/2443327274 learned_addr learned my addr 172.21.15.42:0/2443327274 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:09.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.735+0000 7fd521400640 1 -- 172.21.15.42:0/2443327274 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd51c0b5ad0 con 0x7fd51c070d60 2024-07-04T17:32:09.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.735+0000 7fd521400640 1 --2- 172.21.15.42:0/2443327274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd51c070d60 0x7fd51c0ae200 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fd50c006010 tx=0x7fd50c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:09.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.736+0000 7fd519600640 1 -- 172.21.15.42:0/2443327274 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd50c004380 con 0x7fd51c070d60 2024-07-04T17:32:09.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.736+0000 7fd519600640 1 -- 172.21.15.42:0/2443327274 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd50c004520 con 0x7fd51c070d60 2024-07-04T17:32:09.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.736+0000 7fd523e00640 1 -- 172.21.15.42:0/2443327274 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd51c0b69e0 con 0x7fd51c070d60 2024-07-04T17:32:09.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.736+0000 7fd519600640 1 -- 172.21.15.42:0/2443327274 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd50c02e480 con 0x7fd51c070d60 2024-07-04T17:32:09.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.736+0000 7fd523e00640 1 -- 172.21.15.42:0/2443327274 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd51c0b5280 con 0x7fd51c070d60 2024-07-04T17:32:09.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.737+0000 7fd523e00640 1 -- 172.21.15.42:0/2443327274 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd51c0b30d0 con 0x7fd51c070d60 2024-07-04T17:32:09.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.737+0000 7fd519600640 1 -- 172.21.15.42:0/2443327274 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) ==== 49513+0+0 (secure 0 0 0) 0x7fd50c02e620 con 0x7fd51c070d60 2024-07-04T17:32:09.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.738+0000 7fd519600640 1 --2- 172.21.15.42:0/2443327274 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fd4fc03d3d0 0x7fd4fc03f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:09.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.738+0000 7fd519600640 1 -- 172.21.15.42:0/2443327274 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4303+0+0 (secure 0 0 0) 0x7fd50c073420 con 0x7fd51c070d60 2024-07-04T17:32:09.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.739+0000 7fd520a00640 1 --2- 172.21.15.42:0/2443327274 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fd4fc03d3d0 0x7fd4fc03f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:09.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.739+0000 7fd520a00640 1 --2- 172.21.15.42:0/2443327274 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fd4fc03d3d0 0x7fd4fc03f8e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd508007fd0 tx=0x7fd508007920 comp rx=0 tx=0).ready entity=mgr.5173 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:09.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:09.743+0000 7fd519600640 1 -- 172.21.15.42:0/2443327274 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fd50c033070 con 0x7fd51c070d60 2024-07-04T17:32:10.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:10.014+0000 7fd523e00640 1 -- 172.21.15.42:0/2443327274 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "mirroring"} v 0) -- 0x7fd51c0b5880 con 0x7fd51c070d60 2024-07-04T17:32:10.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:10.608+0000 7fd519600640 1 -- 172.21.15.42:0/2443327274 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "mirroring"}]=0 v33) ==== 88+0+0 (secure 0 0 0) 0x7fd50c0730d0 con 0x7fd51c070d60 2024-07-04T17:32:10.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:10.613+0000 7fd523e00640 1 -- 172.21.15.42:0/2443327274 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fd4fc03d3d0 msgr2=0x7fd4fc03f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:10.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:10.613+0000 7fd523e00640 1 --2- 172.21.15.42:0/2443327274 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fd4fc03d3d0 0x7fd4fc03f8e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd508007fd0 tx=0x7fd508007920 comp rx=0 tx=0).stop 2024-07-04T17:32:10.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:10.613+0000 7fd523e00640 1 -- 172.21.15.42:0/2443327274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd51c070d60 msgr2=0x7fd51c0ae200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:10.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:10.613+0000 7fd523e00640 1 --2- 172.21.15.42:0/2443327274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd51c070d60 0x7fd51c0ae200 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fd50c006010 tx=0x7fd50c003f10 comp rx=0 tx=0).stop 2024-07-04T17:32:10.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:10.614+0000 7fd523e00640 1 -- 172.21.15.42:0/2443327274 shutdown_connections 2024-07-04T17:32:10.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:10.614+0000 7fd523e00640 1 --2- 172.21.15.42:0/2443327274 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fd4fc03d3d0 0x7fd4fc03f8e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:10.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:10.614+0000 7fd523e00640 1 --2- 172.21.15.42:0/2443327274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd51c070d60 0x7fd51c0ae200 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:10.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:10.614+0000 7fd523e00640 1 -- 172.21.15.42:0/2443327274 >> 172.21.15.42:0/2443327274 conn(0x7fd51c06fe40 msgr2=0x7fd51c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:10.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:10.614+0000 7fd523e00640 1 -- 172.21.15.42:0/2443327274 shutdown_connections 2024-07-04T17:32:10.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:10.615+0000 7fd523e00640 1 -- 172.21.15.42:0/2443327274 wait complete. 2024-07-04T17:32:10.649 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:32:10.649 DEBUG:teuthology.orchestra.run.smithi042:counter dump for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok counter dump 2024-07-04T17:32:10.877 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:32:10.878 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker": [ 2024-07-04T17:32:10.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:10.878 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:32:10.878 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "0" 2024-07-04T17:32:10.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:10.878 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:10.878 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:32:10.878 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:32:10.878 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:10.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:10.879 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:10.879 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:32:10.879 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "1" 2024-07-04T17:32:10.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:10.879 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:10.879 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:32:10.880 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:32:10.880 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:10.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:10.880 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:10.880 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:32:10.880 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "2" 2024-07-04T17:32:10.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:10.880 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:10.880 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:32:10.881 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:32:10.881 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:10.881 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:10.881 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:10.881 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": [ 2024-07-04T17:32:10.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:10.882 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:10.882 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:10.882 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 74, 2024-07-04T17:32:10.882 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 4, 2024-07-04T17:32:10.882 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 302496, 2024-07-04T17:32:10.882 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 800, 2024-07-04T17:32:10.883 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:32:10.883 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:32:10.883 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.027895627, 2024-07-04T17:32:10.883 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.002916926, 2024-07-04T17:32:10.883 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.015390205, 2024-07-04T17:32:10.883 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.008186273, 2024-07-04T17:32:10.884 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:32:10.884 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 4, 2024-07-04T17:32:10.884 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.004367369, 2024-07-04T17:32:10.884 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.001091842 2024-07-04T17:32:10.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:10.885 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:32:10.885 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:10.885 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:32:10.885 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:32:10.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:10.885 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 302496, 2024-07-04T17:32:10.886 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 800 2024-07-04T17:32:10.886 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:10.886 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:10.886 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:10.886 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": [ 2024-07-04T17:32:10.886 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:10.887 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:10.887 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:10.887 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 6, 2024-07-04T17:32:10.887 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 77, 2024-07-04T17:32:10.887 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 864, 2024-07-04T17:32:10.888 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 499952, 2024-07-04T17:32:10.888 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 14, 2024-07-04T17:32:10.888 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:32:10.888 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.031688610, 2024-07-04T17:32:10.888 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.011358322, 2024-07-04T17:32:10.889 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.010120530, 2024-07-04T17:32:10.889 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:32:10.889 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:32:10.889 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 71, 2024-07-04T17:32:10.889 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.050165643, 2024-07-04T17:32:10.889 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000706558 2024-07-04T17:32:10.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:10.890 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:32:10.890 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:10.890 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:32:10.890 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:32:10.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:10.891 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 864, 2024-07-04T17:32:10.891 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 499952 2024-07-04T17:32:10.891 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:10.891 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:10.891 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:10.892 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": [ 2024-07-04T17:32:10.892 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:10.892 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:10.892 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:10.892 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 37, 2024-07-04T17:32:10.893 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 7, 2024-07-04T17:32:10.893 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 1443712, 2024-07-04T17:32:10.893 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 1264, 2024-07-04T17:32:10.893 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:32:10.893 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:32:10.894 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.042612882, 2024-07-04T17:32:10.894 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.002966445, 2024-07-04T17:32:10.894 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.072581308, 2024-07-04T17:32:10.894 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:32:10.894 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:32:10.895 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 7, 2024-07-04T17:32:10.895 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.003249255, 2024-07-04T17:32:10.895 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000464179 2024-07-04T17:32:10.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:10.895 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:32:10.895 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:10.896 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:32:10.896 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:32:10.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:10.896 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 1443712, 2024-07-04T17:32:10.896 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 1264 2024-07-04T17:32:10.896 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:10.897 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:10.897 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:10.897 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": [ 2024-07-04T17:32:10.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:10.897 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:10.898 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:10.898 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-07-04T17:32:10.898 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-07-04T17:32:10.898 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:10.898 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:10.899 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:10.899 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror": [ 2024-07-04T17:32:10.899 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:10.899 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:10.899 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:10.899 INFO:teuthology.orchestra.run.smithi042.stdout: "mirrored_filesystems": 0, 2024-07-04T17:32:10.900 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_enable_failures": 0 2024-07-04T17:32:10.900 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:10.900 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:10.900 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:10.900 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": [ 2024-07-04T17:32:10.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:10.901 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:10.901 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:10.901 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-07-04T17:32:10.901 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-07-04T17:32:10.901 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T17:32:10.902 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-07-04T17:32:10.902 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T17:32:10.902 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-07-04T17:32:10.902 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T17:32:10.902 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T17:32:10.903 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T17:32:10.903 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T17:32:10.903 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T17:32:10.903 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-07-04T17:32:10.903 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T17:32:10.903 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T17:32:10.904 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-07-04T17:32:10.904 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-07-04T17:32:10.904 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-07-04T17:32:10.904 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-07-04T17:32:10.904 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T17:32:10.904 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T17:32:10.905 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T17:32:10.905 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T17:32:10.905 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T17:32:10.905 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-07-04T17:32:10.905 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-07-04T17:32:10.905 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-07-04T17:32:10.906 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T17:32:10.906 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T17:32:10.906 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-07-04T17:32:10.906 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-07-04T17:32:10.906 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-07-04T17:32:10.907 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-07-04T17:32:10.907 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T17:32:10.907 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-07-04T17:32:10.907 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T17:32:10.907 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-07-04T17:32:10.907 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 149186, 2024-07-04T17:32:10.908 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 26, 2024-07-04T17:32:10.908 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-07-04T17:32:10.908 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-07-04T17:32:10.908 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-07-04T17:32:10.908 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-07-04T17:32:10.908 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-07-04T17:32:10.908 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-07-04T17:32:10.909 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-07-04T17:32:10.909 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-07-04T17:32:10.909 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 4592, 2024-07-04T17:32:10.909 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 37, 2024-07-04T17:32:10.909 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T17:32:10.910 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-07-04T17:32:10.910 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-07-04T17:32:10.910 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-07-04T17:32:10.910 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 0, 2024-07-04T17:32:10.910 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 0, 2024-07-04T17:32:10.910 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-07-04T17:32:10.910 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-07-04T17:32:10.911 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-07-04T17:32:10.911 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-07-04T17:32:10.911 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:10.911 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:10.911 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:10.911 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter": [ 2024-07-04T17:32:10.911 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:10.911 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:10.911 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:10.911 INFO:teuthology.orchestra.run.smithi042.stdout: "op_active": 0, 2024-07-04T17:32:10.911 INFO:teuthology.orchestra.run.smithi042.stdout: "op_laggy": 0, 2024-07-04T17:32:10.911 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send": 0, 2024-07-04T17:32:10.912 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send_bytes": 0, 2024-07-04T17:32:10.912 INFO:teuthology.orchestra.run.smithi042.stdout: "op_resend": 0, 2024-07-04T17:32:10.912 INFO:teuthology.orchestra.run.smithi042.stdout: "op_reply": 0, 2024-07-04T17:32:10.912 INFO:teuthology.orchestra.run.smithi042.stdout: "op_latency": { 2024-07-04T17:32:10.912 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:10.912 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:32:10.912 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:32:10.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:10.912 INFO:teuthology.orchestra.run.smithi042.stdout: "op_inflight": 0, 2024-07-04T17:32:10.912 INFO:teuthology.orchestra.run.smithi042.stdout: "oplen_avg": { 2024-07-04T17:32:10.912 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:10.912 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0 2024-07-04T17:32:10.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:10.912 INFO:teuthology.orchestra.run.smithi042.stdout: "op": 0, 2024-07-04T17:32:10.913 INFO:teuthology.orchestra.run.smithi042.stdout: "op_r": 0, 2024-07-04T17:32:10.913 INFO:teuthology.orchestra.run.smithi042.stdout: "op_w": 0, 2024-07-04T17:32:10.913 INFO:teuthology.orchestra.run.smithi042.stdout: "op_rmw": 0, 2024-07-04T17:32:10.913 INFO:teuthology.orchestra.run.smithi042.stdout: "op_pg": 0, 2024-07-04T17:32:10.913 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_stat": 0, 2024-07-04T17:32:10.913 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_create": 0, 2024-07-04T17:32:10.913 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_read": 0, 2024-07-04T17:32:10.913 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_write": 0, 2024-07-04T17:32:10.913 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writefull": 0, 2024-07-04T17:32:10.913 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writesame": 0, 2024-07-04T17:32:10.914 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_append": 0, 2024-07-04T17:32:10.914 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_zero": 0, 2024-07-04T17:32:10.914 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_truncate": 0, 2024-07-04T17:32:10.914 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_delete": 0, 2024-07-04T17:32:10.914 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_mapext": 0, 2024-07-04T17:32:10.914 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_sparse_read": 0, 2024-07-04T17:32:10.914 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_clonerange": 0, 2024-07-04T17:32:10.914 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_getxattr": 0, 2024-07-04T17:32:10.914 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_setxattr": 0, 2024-07-04T17:32:10.914 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_cmpxattr": 0, 2024-07-04T17:32:10.914 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_rmxattr": 0, 2024-07-04T17:32:10.914 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_resetxattrs": 0, 2024-07-04T17:32:10.915 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_call": 0, 2024-07-04T17:32:10.915 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_watch": 0, 2024-07-04T17:32:10.915 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_notify": 0, 2024-07-04T17:32:10.915 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T17:32:10.915 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls": 0, 2024-07-04T17:32:10.915 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls_filter": 0, 2024-07-04T17:32:10.915 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_other": 0, 2024-07-04T17:32:10.915 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_active": 0, 2024-07-04T17:32:10.915 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_send": 0, 2024-07-04T17:32:10.915 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_resend": 0, 2024-07-04T17:32:10.915 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_ping": 0, 2024-07-04T17:32:10.915 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_active": 0, 2024-07-04T17:32:10.915 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_send": 0, 2024-07-04T17:32:10.915 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_resend": 0, 2024-07-04T17:32:10.916 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_active": 0, 2024-07-04T17:32:10.916 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_send": 0, 2024-07-04T17:32:10.916 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_resend": 0, 2024-07-04T17:32:10.916 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_active": 0, 2024-07-04T17:32:10.916 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_send": 0, 2024-07-04T17:32:10.916 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_resend": 0, 2024-07-04T17:32:10.916 INFO:teuthology.orchestra.run.smithi042.stdout: "command_active": 0, 2024-07-04T17:32:10.916 INFO:teuthology.orchestra.run.smithi042.stdout: "command_send": 0, 2024-07-04T17:32:10.916 INFO:teuthology.orchestra.run.smithi042.stdout: "command_resend": 0, 2024-07-04T17:32:10.916 INFO:teuthology.orchestra.run.smithi042.stdout: "map_epoch": 0, 2024-07-04T17:32:10.916 INFO:teuthology.orchestra.run.smithi042.stdout: "map_full": 0, 2024-07-04T17:32:10.916 INFO:teuthology.orchestra.run.smithi042.stdout: "map_inc": 0, 2024-07-04T17:32:10.916 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_sessions": 0, 2024-07-04T17:32:10.916 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_open": 0, 2024-07-04T17:32:10.917 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_close": 0, 2024-07-04T17:32:10.917 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_laggy": 0, 2024-07-04T17:32:10.917 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_wr": 0, 2024-07-04T17:32:10.917 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_rd": 0, 2024-07-04T17:32:10.917 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_del": 0 2024-07-04T17:32:10.917 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:10.917 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:10.917 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:10.917 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-client": [ 2024-07-04T17:32:10.917 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:10.917 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:10.917 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:10.918 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:32:10.918 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:32:10.918 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:32:10.918 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 74, 2024-07-04T17:32:10.918 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 292388, 2024-07-04T17:32:10.918 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:32:10.918 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 74, 2024-07-04T17:32:10.918 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:32:10.918 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:32:10.918 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 74, 2024-07-04T17:32:10.918 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 292388, 2024-07-04T17:32:10.918 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:32:10.918 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:10.919 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:32:10.919 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:32:10.919 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:10.919 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:10.919 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:10.919 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:10.919 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-radosclient": [ 2024-07-04T17:32:10.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:10.919 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:10.919 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:10.919 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:32:10.919 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:32:10.919 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:32:10.919 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 40, 2024-07-04T17:32:10.919 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 1438223, 2024-07-04T17:32:10.920 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:32:10.920 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 40, 2024-07-04T17:32:10.920 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:32:10.920 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:32:10.920 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 40, 2024-07-04T17:32:10.920 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 1438223, 2024-07-04T17:32:10.920 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:32:10.920 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:10.920 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:32:10.920 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:32:10.920 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:10.920 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:10.920 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:10.920 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:10.920 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_bytes": [ 2024-07-04T17:32:10.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:10.921 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:10.921 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:10.921 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:32:10.921 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:32:10.921 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:32:10.921 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:32:10.921 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:32:10.921 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:32:10.921 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:32:10.922 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:32:10.922 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:32:10.922 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:32:10.922 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:32:10.922 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:32:10.922 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:10.922 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:32:10.922 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:32:10.922 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:10.922 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:10.922 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:10.923 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:10.923 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_ops": [ 2024-07-04T17:32:10.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:10.923 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:10.923 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:10.923 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:32:10.923 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1024, 2024-07-04T17:32:10.923 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:32:10.923 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:32:10.923 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:32:10.923 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:32:10.923 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:32:10.923 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:32:10.924 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:32:10.924 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:32:10.924 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:32:10.924 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:32:10.924 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:10.924 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:32:10.924 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:32:10.924 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:10.924 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:10.924 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:10.924 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-07-04T17:32:10.924 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:32:10.926 DEBUG:tasks.cephfs.test_mirroring:command returned={ "AsyncMessenger::Worker": [ { "labels": { "id": "0" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "1" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "2" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } } ], "AsyncMessenger::Worker-0": [ { "labels": {}, "counters": { "msgr_recv_messages": 74, "msgr_send_messages": 4, "msgr_recv_bytes": 302496, "msgr_send_bytes": 800, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.027895627, "msgr_running_send_time": 0.002916926, "msgr_running_recv_time": 0.015390205, "msgr_running_fast_dispatch_time": 0.008186273, "msgr_send_messages_queue_lat": { "avgcount": 4, "sum": 0.004367369, "avgtime": 0.001091842 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 302496, "msgr_send_encrypted_bytes": 800 } } ], "AsyncMessenger::Worker-1": [ { "labels": {}, "counters": { "msgr_recv_messages": 6, "msgr_send_messages": 77, "msgr_recv_bytes": 864, "msgr_send_bytes": 499952, "msgr_created_connections": 14, "msgr_active_connections": 1, "msgr_running_total_time": 0.031688610, "msgr_running_send_time": 0.011358322, "msgr_running_recv_time": 0.010120530, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 71, "sum": 0.050165643, "avgtime": 0.000706558 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 864, "msgr_send_encrypted_bytes": 499952 } } ], "AsyncMessenger::Worker-2": [ { "labels": {}, "counters": { "msgr_recv_messages": 37, "msgr_send_messages": 7, "msgr_recv_bytes": 1443712, "msgr_send_bytes": 1264, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.042612882, "msgr_running_send_time": 0.002966445, "msgr_running_recv_time": 0.072581308, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 7, "sum": 0.003249255, "avgtime": 0.000464179 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 1443712, "msgr_send_encrypted_bytes": 1264 } } ], "cct": [ { "labels": {}, "counters": { "total_workers": 0, "unhealthy_workers": 0 } } ], "cephfs_mirror": [ { "labels": {}, "counters": { "mirrored_filesystems": 0, "mirror_enable_failures": 0 } } ], "mempool": [ { "labels": {}, "counters": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_items": 0, "bluefs_bytes": 0, "bluefs_items": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "buffer_anon_bytes": 149186, "buffer_anon_items": 26, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 4592, "osdmap_items": 37, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "mds_co_bytes": 0, "mds_co_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 } } ], "objecter": [ { "labels": {}, "counters": { "op_active": 0, "op_laggy": 0, "op_send": 0, "op_send_bytes": 0, "op_resend": 0, "op_reply": 0, "op_latency": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "op_inflight": 0, "oplen_avg": { "avgcount": 0, "sum": 0 }, "op": 0, "op_r": 0, "op_w": 0, "op_rmw": 0, "op_pg": 0, "osdop_stat": 0, "osdop_create": 0, "osdop_read": 0, "osdop_write": 0, "osdop_writefull": 0, "osdop_writesame": 0, "osdop_append": 0, "osdop_zero": 0, "osdop_truncate": 0, "osdop_delete": 0, "osdop_mapext": 0, "osdop_sparse_read": 0, "osdop_clonerange": 0, "osdop_getxattr": 0, "osdop_setxattr": 0, "osdop_cmpxattr": 0, "osdop_rmxattr": 0, "osdop_resetxattrs": 0, "osdop_call": 0, "osdop_watch": 0, "osdop_notify": 0, "osdop_src_cmpxattr": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_other": 0, "linger_active": 0, "linger_send": 0, "linger_resend": 0, "linger_ping": 0, "poolop_active": 0, "poolop_send": 0, "poolop_resend": 0, "poolstat_active": 0, "poolstat_send": 0, "poolstat_resend": 0, "statfs_active": 0, "statfs_send": 0, "statfs_resend": 0, "command_active": 0, "command_send": 0, "command_resend": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "osd_sessions": 0, "osd_session_open": 0, "osd_session_close": 0, "osd_laggy": 0, "omap_wr": 0, "omap_rd": 0, "omap_del": 0 } } ], "throttle-msgr_dispatch_throttler-client": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 74, "get_sum": 292388, "get_or_fail_fail": 0, "get_or_fail_success": 74, "take": 0, "take_sum": 0, "put": 74, "put_sum": 292388, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-msgr_dispatch_throttler-radosclient": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 40, "get_sum": 1438223, "get_or_fail_fail": 0, "get_or_fail_success": 40, "take": 0, "take_sum": 0, "put": 40, "put_sum": 1438223, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_bytes": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_ops": [ { "labels": {}, "counters": { "val": 0, "max": 1024, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ] } 2024-07-04T17:32:10.926 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snapshot mirror enable cephfs 2024-07-04T17:32:11.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.212+0000 7fac2f400640 1 -- 172.21.15.42:0/3117440184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac280bc4e0 msgr2=0x7fac280c3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:11.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.212+0000 7fac2f400640 1 --2- 172.21.15.42:0/3117440184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac280bc4e0 0x7fac280c3cb0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0097b0 tx=0x7fac1c02a530 comp rx=0 tx=0).stop 2024-07-04T17:32:11.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.212+0000 7fac2f400640 1 -- 172.21.15.42:0/3117440184 shutdown_connections 2024-07-04T17:32:11.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.212+0000 7fac2f400640 1 --2- 172.21.15.42:0/3117440184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac280bc4e0 0x7fac280c3cb0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:11.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.212+0000 7fac2f400640 1 -- 172.21.15.42:0/3117440184 >> 172.21.15.42:0/3117440184 conn(0x7fac2806e2f0 msgr2=0x7fac2806e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:11.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.212+0000 7fac2f400640 1 -- 172.21.15.42:0/3117440184 shutdown_connections 2024-07-04T17:32:11.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.212+0000 7fac2f400640 1 -- 172.21.15.42:0/3117440184 wait complete. 2024-07-04T17:32:11.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.214+0000 7fac2f400640 1 Processor -- start 2024-07-04T17:32:11.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.214+0000 7fac2f400640 1 -- start start 2024-07-04T17:32:11.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.216+0000 7fac2f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac280bc4e0 0x7fac2813d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:11.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.216+0000 7fac2f400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fac280b7810 con 0x7fac280bc4e0 2024-07-04T17:32:11.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.216+0000 7fac2ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac280bc4e0 0x7fac2813d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:11.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.216+0000 7fac2ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac280bc4e0 0x7fac2813d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35538/0 (socket says 172.21.15.42:35538) 2024-07-04T17:32:11.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.216+0000 7fac2ca00640 1 -- 172.21.15.42:0/785606240 learned_addr learned my addr 172.21.15.42:0/785606240 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:11.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.217+0000 7fac2ca00640 1 -- 172.21.15.42:0/785606240 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac2813db70 con 0x7fac280bc4e0 2024-07-04T17:32:11.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.217+0000 7fac2ca00640 1 --2- 172.21.15.42:0/785606240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac280bc4e0 0x7fac2813d590 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0098d0 tx=0x7fac1c033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:11.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.217+0000 7fac24c00640 1 -- 172.21.15.42:0/785606240 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac1c031bc0 con 0x7fac280bc4e0 2024-07-04T17:32:11.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.217+0000 7fac24c00640 1 -- 172.21.15.42:0/785606240 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac1c031e50 con 0x7fac280bc4e0 2024-07-04T17:32:11.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.217+0000 7fac2f400640 1 -- 172.21.15.42:0/785606240 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac281e4190 con 0x7fac280bc4e0 2024-07-04T17:32:11.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.217+0000 7fac24c00640 1 -- 172.21.15.42:0/785606240 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac1c033200 con 0x7fac280bc4e0 2024-07-04T17:32:11.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.218+0000 7fac2f400640 1 -- 172.21.15.42:0/785606240 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac2813b900 con 0x7fac280bc4e0 2024-07-04T17:32:11.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.219+0000 7fac24c00640 1 -- 172.21.15.42:0/785606240 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 33) ==== 49526+0+0 (secure 0 0 0) 0x7fac1c0333a0 con 0x7fac280bc4e0 2024-07-04T17:32:11.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.219+0000 7fac2f400640 1 -- 172.21.15.42:0/785606240 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac280c3cb0 con 0x7fac280bc4e0 2024-07-04T17:32:11.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.219+0000 7fac24c00640 1 --2- 172.21.15.42:0/785606240 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fac0803d420 0x7fac0803f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:11.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.219+0000 7fac24c00640 1 -- 172.21.15.42:0/785606240 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4303+0+0 (secure 0 0 0) 0x7fac1c073060 con 0x7fac280bc4e0 2024-07-04T17:32:11.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.219+0000 7fac27e00640 1 -- 172.21.15.42:0/785606240 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fac0803d420 msgr2=0x7fac0803f930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1139872311 2024-07-04T17:32:11.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.220+0000 7fac27e00640 1 --2- 172.21.15.42:0/785606240 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fac0803d420 0x7fac0803f930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:32:11.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.224+0000 7fac24c00640 1 -- 172.21.15.42:0/785606240 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fac1c03f030 con 0x7fac280bc4e0 2024-07-04T17:32:11.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.421+0000 7fac27e00640 1 -- 172.21.15.42:0/785606240 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fac0803d420 msgr2=0x7fac0803f930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1139872311 2024-07-04T17:32:11.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.421+0000 7fac27e00640 1 --2- 172.21.15.42:0/785606240 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fac0803d420 0x7fac0803f930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:32:11.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.452+0000 7fac2f400640 1 -- 172.21.15.42:0/785606240 --> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] -- mgr_command(tid 0: {"prefix": "fs snapshot mirror enable", "fs_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7fac280c3670 con 0x7fac0803d420 2024-07-04T17:32:11.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.822+0000 7fac27e00640 1 -- 172.21.15.42:0/785606240 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fac0803d420 msgr2=0x7fac0803f930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1139872311 2024-07-04T17:32:11.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:11.822+0000 7fac27e00640 1 --2- 172.21.15.42:0/785606240 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fac0803d420 0x7fac0803f930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-07-04T17:32:12.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:12.623+0000 7fac27e00640 1 -- 172.21.15.42:0/785606240 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fac0803d420 msgr2=0x7fac0803f930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1139872311 2024-07-04T17:32:12.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:12.623+0000 7fac27e00640 1 --2- 172.21.15.42:0/785606240 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fac0803d420 0x7fac0803f930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-07-04T17:32:14.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:14.225+0000 7fac27e00640 1 -- 172.21.15.42:0/785606240 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fac0803d420 msgr2=0x7fac0803f930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1139872311 2024-07-04T17:32:14.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:14.225+0000 7fac27e00640 1 --2- 172.21.15.42:0/785606240 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fac0803d420 0x7fac0803f930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-07-04T17:32:15.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:15.323+0000 7fac24c00640 1 -- 172.21.15.42:0/785606240 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 34) ==== 49135+0+0 (secure 0 0 0) 0x7fac1c0035f0 con 0x7fac280bc4e0 2024-07-04T17:32:15.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:15.324+0000 7fac24c00640 1 -- 172.21.15.42:0/785606240 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fac0803d420 msgr2=0x7fac0803f930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:32:15.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:15.324+0000 7fac24c00640 1 --2- 172.21.15.42:0/785606240 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fac0803d420 0x7fac0803f930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:15.401 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:15.401+0000 7fb39fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:15.402 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:15.401+0000 7fb39fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:15.402 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:15.401+0000 7fb39fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:15.402 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:15.401+0000 7fb39fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:15.402 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:15.401+0000 7fb39fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:15.402 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:15.401+0000 7fb39b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:15.402 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:15.401+0000 7fb39b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:15.402 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:15.401+0000 7fb39b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:15.402 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:15.401+0000 7fb39b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:15.402 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:15.401+0000 7fb39b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:15.421 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:15.421+0000 7fb39b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:15.421 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:15.421+0000 7fb39b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:15.422 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:15.421+0000 7fb39b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:15.422 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:15.421+0000 7fb39b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:15.422 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:15.421+0000 7fb39b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:15.422 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:15.422+0000 7fb39fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:15.422 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:15.422+0000 7fb39fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:15.422 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:15.422+0000 7fb39fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:15.423 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:15.422+0000 7fb39fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:15.423 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:15.422+0000 7fb39fc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:16.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:16.327+0000 7fac24c00640 1 -- 172.21.15.42:0/785606240 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mgrmap(e 35) ==== 49262+0+0 (secure 0 0 0) 0x7fac1c058e20 con 0x7fac280bc4e0 2024-07-04T17:32:16.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:16.327+0000 7fac24c00640 1 --2- 172.21.15.42:0/785606240 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7fac08046a70 0x7fac08048e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:16.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:16.328+0000 7fac24c00640 1 -- 172.21.15.42:0/785606240 --> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] -- mgr_command(tid 0: {"prefix": "fs snapshot mirror enable", "fs_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7fac280c3670 con 0x7fac08046a70 2024-07-04T17:32:16.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:16.333+0000 7fac27e00640 1 --2- 172.21.15.42:0/785606240 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7fac08046a70 0x7fac08048e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:16.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:16.338+0000 7fac27e00640 1 --2- 172.21.15.42:0/785606240 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7fac08046a70 0x7fac08048e90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fac14000c00 tx=0x7fac14007af0 comp rx=0 tx=0).ready entity=mgr.5396 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:17.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:17.326+0000 7fac24c00640 1 -- 172.21.15.42:0/785606240 <== mgr.5396 v2:172.21.15.42:6834/1323178695 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fac280c3670 con 0x7fac08046a70 2024-07-04T17:32:17.327 INFO:teuthology.orchestra.run.smithi042.stdout:{} 2024-07-04T17:32:17.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:17.330+0000 7fac2f400640 1 -- 172.21.15.42:0/785606240 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7fac08046a70 msgr2=0x7fac08048e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:17.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:17.330+0000 7fac2f400640 1 --2- 172.21.15.42:0/785606240 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7fac08046a70 0x7fac08048e90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fac14000c00 tx=0x7fac14007af0 comp rx=0 tx=0).stop 2024-07-04T17:32:17.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:17.330+0000 7fac2f400640 1 -- 172.21.15.42:0/785606240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac280bc4e0 msgr2=0x7fac2813d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:17.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:17.330+0000 7fac2f400640 1 --2- 172.21.15.42:0/785606240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac280bc4e0 0x7fac2813d590 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0098d0 tx=0x7fac1c033e40 comp rx=0 tx=0).stop 2024-07-04T17:32:17.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:17.330+0000 7fac2f400640 1 -- 172.21.15.42:0/785606240 shutdown_connections 2024-07-04T17:32:17.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:17.330+0000 7fac2f400640 1 --2- 172.21.15.42:0/785606240 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7fac08046a70 0x7fac08048e90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:17.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:17.330+0000 7fac2f400640 1 --2- 172.21.15.42:0/785606240 >> [v2:172.21.15.42:6834/1139872311,v1:172.21.15.42:6835/1139872311] conn(0x7fac0803d420 0x7fac0803f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:17.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:17.330+0000 7fac2f400640 1 --2- 172.21.15.42:0/785606240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fac280bc4e0 0x7fac2813d590 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:17.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:17.330+0000 7fac2f400640 1 -- 172.21.15.42:0/785606240 >> 172.21.15.42:0/785606240 conn(0x7fac2806e2f0 msgr2=0x7fac280bab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:17.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:17.330+0000 7fac2f400640 1 -- 172.21.15.42:0/785606240 shutdown_connections 2024-07-04T17:32:17.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:17.330+0000 7fac2f400640 1 -- 172.21.15.42:0/785606240 wait complete. 2024-07-04T17:32:27.363 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:32:27.364 DEBUG:teuthology.orchestra.run.smithi042:mirror status for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@8 2024-07-04T17:32:27.611 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:32:27.611 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_inst": "172.21.15.42:0/2096108670", 2024-07-04T17:32:27.611 INFO:teuthology.orchestra.run.smithi042.stdout: "peers": {}, 2024-07-04T17:32:27.612 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_dirs": { 2024-07-04T17:32:27.612 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_count": 0 2024-07-04T17:32:27.612 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.612 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:32:27.628 DEBUG:tasks.cephfs.test_mirroring:command returned={ "rados_inst": "172.21.15.42:0/2096108670", "peers": {}, "snap_dirs": { "dir_count": 0 } } 2024-07-04T17:32:27.629 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:32:27.629 DEBUG:teuthology.orchestra.run.smithi042:counter dump for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok counter dump 2024-07-04T17:32:27.871 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:32:27.871 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker": [ 2024-07-04T17:32:27.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:27.872 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:32:27.872 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "0" 2024-07-04T17:32:27.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:27.872 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:27.872 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:32:27.872 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:32:27.872 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:27.872 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:27.872 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:32:27.872 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "1" 2024-07-04T17:32:27.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:27.873 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:27.873 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:32:27.873 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:32:27.873 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:27.873 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:27.873 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:32:27.873 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "2" 2024-07-04T17:32:27.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:27.873 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:27.873 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:32:27.873 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:32:27.874 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.874 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.874 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:27.874 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": [ 2024-07-04T17:32:27.874 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:27.874 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:27.874 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:27.874 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 75, 2024-07-04T17:32:27.874 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 4, 2024-07-04T17:32:27.874 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 307392, 2024-07-04T17:32:27.874 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 800, 2024-07-04T17:32:27.874 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:32:27.875 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:32:27.875 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.028679265, 2024-07-04T17:32:27.875 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.003170141, 2024-07-04T17:32:27.875 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.015694758, 2024-07-04T17:32:27.875 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.008302702, 2024-07-04T17:32:27.875 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:32:27.875 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 4, 2024-07-04T17:32:27.875 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.004367369, 2024-07-04T17:32:27.875 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.001091842 2024-07-04T17:32:27.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:27.875 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:32:27.875 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:27.875 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:32:27.876 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:32:27.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:27.876 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 307392, 2024-07-04T17:32:27.876 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 800 2024-07-04T17:32:27.876 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.876 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.876 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:27.876 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": [ 2024-07-04T17:32:27.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:27.876 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:27.876 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:27.876 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 7, 2024-07-04T17:32:27.877 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 81, 2024-07-04T17:32:27.877 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 1008, 2024-07-04T17:32:27.877 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 581104, 2024-07-04T17:32:27.877 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 15, 2024-07-04T17:32:27.877 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 2, 2024-07-04T17:32:27.877 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.036220743, 2024-07-04T17:32:27.877 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.012131766, 2024-07-04T17:32:27.877 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.011984045, 2024-07-04T17:32:27.877 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:32:27.877 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:32:27.877 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 75, 2024-07-04T17:32:27.877 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.062115163, 2024-07-04T17:32:27.877 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000828202 2024-07-04T17:32:27.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:27.878 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:32:27.878 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:27.878 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:32:27.878 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:32:27.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:27.878 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 1008, 2024-07-04T17:32:27.878 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 581104 2024-07-04T17:32:27.878 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.878 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.878 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:27.878 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": [ 2024-07-04T17:32:27.878 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:27.878 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:27.879 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:27.879 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 41, 2024-07-04T17:32:27.879 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 7, 2024-07-04T17:32:27.879 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 1641952, 2024-07-04T17:32:27.879 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 1264, 2024-07-04T17:32:27.879 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:32:27.879 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:32:27.879 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.047663729, 2024-07-04T17:32:27.879 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.003168493, 2024-07-04T17:32:27.879 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.081571777, 2024-07-04T17:32:27.879 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:32:27.879 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:32:27.879 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 7, 2024-07-04T17:32:27.879 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.003249255, 2024-07-04T17:32:27.880 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000464179 2024-07-04T17:32:27.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:27.880 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:32:27.880 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:27.880 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:32:27.880 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:32:27.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:27.880 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 1641952, 2024-07-04T17:32:27.880 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 1264 2024-07-04T17:32:27.880 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.880 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.881 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:27.881 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": [ 2024-07-04T17:32:27.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:27.881 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:27.881 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:27.881 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-07-04T17:32:27.881 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-07-04T17:32:27.881 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.881 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.881 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:27.881 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror": [ 2024-07-04T17:32:27.882 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:27.882 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:27.882 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:27.882 INFO:teuthology.orchestra.run.smithi042.stdout: "mirrored_filesystems": 1, 2024-07-04T17:32:27.882 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_enable_failures": 0 2024-07-04T17:32:27.882 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.882 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.882 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:27.882 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_mirrored_filesystems": [ 2024-07-04T17:32:27.883 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:27.883 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:32:27.883 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem": "cephfs" 2024-07-04T17:32:27.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:27.883 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:27.883 INFO:teuthology.orchestra.run.smithi042.stdout: "mirroring_peers": 0, 2024-07-04T17:32:27.883 INFO:teuthology.orchestra.run.smithi042.stdout: "directory_count": 0 2024-07-04T17:32:27.883 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.883 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.883 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:27.884 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": [ 2024-07-04T17:32:27.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:27.884 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:27.884 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:27.884 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-07-04T17:32:27.884 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-07-04T17:32:27.884 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T17:32:27.884 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-07-04T17:32:27.884 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T17:32:27.884 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-07-04T17:32:27.884 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T17:32:27.885 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T17:32:27.885 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T17:32:27.885 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T17:32:27.885 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T17:32:27.885 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-07-04T17:32:27.885 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T17:32:27.885 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T17:32:27.885 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-07-04T17:32:27.885 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-07-04T17:32:27.885 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-07-04T17:32:27.885 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-07-04T17:32:27.885 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T17:32:27.886 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T17:32:27.886 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T17:32:27.886 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T17:32:27.886 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T17:32:27.886 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-07-04T17:32:27.886 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-07-04T17:32:27.886 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-07-04T17:32:27.886 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T17:32:27.886 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T17:32:27.886 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-07-04T17:32:27.886 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-07-04T17:32:27.886 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-07-04T17:32:27.886 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-07-04T17:32:27.886 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T17:32:27.887 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-07-04T17:32:27.887 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T17:32:27.887 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-07-04T17:32:27.887 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 183519, 2024-07-04T17:32:27.887 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 55, 2024-07-04T17:32:27.887 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-07-04T17:32:27.887 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-07-04T17:32:27.887 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-07-04T17:32:27.887 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-07-04T17:32:27.887 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-07-04T17:32:27.887 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-07-04T17:32:27.887 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-07-04T17:32:27.887 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-07-04T17:32:27.888 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 18464, 2024-07-04T17:32:27.888 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 165, 2024-07-04T17:32:27.888 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T17:32:27.888 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-07-04T17:32:27.888 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-07-04T17:32:27.888 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-07-04T17:32:27.888 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 0, 2024-07-04T17:32:27.888 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 0, 2024-07-04T17:32:27.888 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-07-04T17:32:27.888 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-07-04T17:32:27.888 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-07-04T17:32:27.888 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-07-04T17:32:27.888 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.888 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.889 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:27.889 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter": [ 2024-07-04T17:32:27.889 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:27.889 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:27.889 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:27.889 INFO:teuthology.orchestra.run.smithi042.stdout: "op_active": 0, 2024-07-04T17:32:27.889 INFO:teuthology.orchestra.run.smithi042.stdout: "op_laggy": 0, 2024-07-04T17:32:27.889 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send": 0, 2024-07-04T17:32:27.889 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send_bytes": 0, 2024-07-04T17:32:27.889 INFO:teuthology.orchestra.run.smithi042.stdout: "op_resend": 0, 2024-07-04T17:32:27.889 INFO:teuthology.orchestra.run.smithi042.stdout: "op_reply": 0, 2024-07-04T17:32:27.889 INFO:teuthology.orchestra.run.smithi042.stdout: "op_latency": { 2024-07-04T17:32:27.889 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:27.890 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:32:27.890 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:32:27.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:27.890 INFO:teuthology.orchestra.run.smithi042.stdout: "op_inflight": 0, 2024-07-04T17:32:27.890 INFO:teuthology.orchestra.run.smithi042.stdout: "oplen_avg": { 2024-07-04T17:32:27.890 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:27.890 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0 2024-07-04T17:32:27.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:27.890 INFO:teuthology.orchestra.run.smithi042.stdout: "op": 0, 2024-07-04T17:32:27.890 INFO:teuthology.orchestra.run.smithi042.stdout: "op_r": 0, 2024-07-04T17:32:27.890 INFO:teuthology.orchestra.run.smithi042.stdout: "op_w": 0, 2024-07-04T17:32:27.890 INFO:teuthology.orchestra.run.smithi042.stdout: "op_rmw": 0, 2024-07-04T17:32:27.890 INFO:teuthology.orchestra.run.smithi042.stdout: "op_pg": 0, 2024-07-04T17:32:27.891 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_stat": 0, 2024-07-04T17:32:27.891 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_create": 0, 2024-07-04T17:32:27.891 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_read": 0, 2024-07-04T17:32:27.891 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_write": 0, 2024-07-04T17:32:27.891 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writefull": 0, 2024-07-04T17:32:27.891 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writesame": 0, 2024-07-04T17:32:27.891 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_append": 0, 2024-07-04T17:32:27.891 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_zero": 0, 2024-07-04T17:32:27.891 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_truncate": 0, 2024-07-04T17:32:27.891 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_delete": 0, 2024-07-04T17:32:27.891 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_mapext": 0, 2024-07-04T17:32:27.891 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_sparse_read": 0, 2024-07-04T17:32:27.891 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_clonerange": 0, 2024-07-04T17:32:27.892 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_getxattr": 0, 2024-07-04T17:32:27.892 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_setxattr": 0, 2024-07-04T17:32:27.892 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_cmpxattr": 0, 2024-07-04T17:32:27.892 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_rmxattr": 0, 2024-07-04T17:32:27.892 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_resetxattrs": 0, 2024-07-04T17:32:27.892 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_call": 0, 2024-07-04T17:32:27.892 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_watch": 0, 2024-07-04T17:32:27.892 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_notify": 0, 2024-07-04T17:32:27.892 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T17:32:27.892 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls": 0, 2024-07-04T17:32:27.892 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls_filter": 0, 2024-07-04T17:32:27.893 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_other": 0, 2024-07-04T17:32:27.893 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_active": 0, 2024-07-04T17:32:27.893 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_send": 0, 2024-07-04T17:32:27.893 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_resend": 0, 2024-07-04T17:32:27.893 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_ping": 0, 2024-07-04T17:32:27.893 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_active": 0, 2024-07-04T17:32:27.893 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_send": 0, 2024-07-04T17:32:27.893 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_resend": 0, 2024-07-04T17:32:27.893 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_active": 0, 2024-07-04T17:32:27.893 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_send": 0, 2024-07-04T17:32:27.893 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_resend": 0, 2024-07-04T17:32:27.893 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_active": 0, 2024-07-04T17:32:27.893 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_send": 0, 2024-07-04T17:32:27.894 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_resend": 0, 2024-07-04T17:32:27.894 INFO:teuthology.orchestra.run.smithi042.stdout: "command_active": 0, 2024-07-04T17:32:27.894 INFO:teuthology.orchestra.run.smithi042.stdout: "command_send": 0, 2024-07-04T17:32:27.894 INFO:teuthology.orchestra.run.smithi042.stdout: "command_resend": 0, 2024-07-04T17:32:27.894 INFO:teuthology.orchestra.run.smithi042.stdout: "map_epoch": 0, 2024-07-04T17:32:27.894 INFO:teuthology.orchestra.run.smithi042.stdout: "map_full": 0, 2024-07-04T17:32:27.894 INFO:teuthology.orchestra.run.smithi042.stdout: "map_inc": 0, 2024-07-04T17:32:27.894 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_sessions": 0, 2024-07-04T17:32:27.894 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_open": 0, 2024-07-04T17:32:27.894 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_close": 0, 2024-07-04T17:32:27.894 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_laggy": 0, 2024-07-04T17:32:27.894 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_wr": 0, 2024-07-04T17:32:27.894 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_rd": 0, 2024-07-04T17:32:27.895 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_del": 0 2024-07-04T17:32:27.895 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.895 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.895 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:27.895 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-client": [ 2024-07-04T17:32:27.895 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:27.895 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:27.895 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:27.895 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:32:27.895 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:32:27.895 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:32:27.895 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 75, 2024-07-04T17:32:27.895 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 297142, 2024-07-04T17:32:27.896 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:32:27.896 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 75, 2024-07-04T17:32:27.896 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:32:27.896 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:32:27.896 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 75, 2024-07-04T17:32:27.896 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 297142, 2024-07-04T17:32:27.896 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:32:27.896 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:27.896 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:32:27.896 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:32:27.896 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.896 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.896 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.896 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:27.897 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-radosclient": [ 2024-07-04T17:32:27.897 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:27.897 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:27.897 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:27.897 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:32:27.897 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:32:27.897 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:32:27.897 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 45, 2024-07-04T17:32:27.897 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 1635939, 2024-07-04T17:32:27.897 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:32:27.897 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 45, 2024-07-04T17:32:27.897 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:32:27.897 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:32:27.898 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 45, 2024-07-04T17:32:27.898 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 1635939, 2024-07-04T17:32:27.898 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:32:27.898 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:27.898 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:32:27.898 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:32:27.898 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.898 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.898 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.898 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:27.898 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_bytes": [ 2024-07-04T17:32:27.898 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:27.898 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:27.899 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:27.899 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:32:27.899 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:32:27.899 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:32:27.899 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:32:27.899 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:32:27.899 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:32:27.899 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:32:27.899 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:32:27.899 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:32:27.899 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:32:27.899 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:32:27.899 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:32:27.900 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:27.900 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:32:27.900 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:32:27.900 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.900 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.900 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.900 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:27.900 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_ops": [ 2024-07-04T17:32:27.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:27.900 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:27.900 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:27.900 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:32:27.900 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1024, 2024-07-04T17:32:27.900 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:32:27.901 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:32:27.901 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:32:27.901 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:32:27.901 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:32:27.901 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:32:27.901 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:32:27.901 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:32:27.901 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:32:27.901 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:32:27.901 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:27.901 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:32:27.901 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:32:27.901 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.901 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.901 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:27.902 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-07-04T17:32:27.902 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:32:27.903 DEBUG:tasks.cephfs.test_mirroring:command returned={ "AsyncMessenger::Worker": [ { "labels": { "id": "0" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "1" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "2" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } } ], "AsyncMessenger::Worker-0": [ { "labels": {}, "counters": { "msgr_recv_messages": 75, "msgr_send_messages": 4, "msgr_recv_bytes": 307392, "msgr_send_bytes": 800, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.028679265, "msgr_running_send_time": 0.003170141, "msgr_running_recv_time": 0.015694758, "msgr_running_fast_dispatch_time": 0.008302702, "msgr_send_messages_queue_lat": { "avgcount": 4, "sum": 0.004367369, "avgtime": 0.001091842 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 307392, "msgr_send_encrypted_bytes": 800 } } ], "AsyncMessenger::Worker-1": [ { "labels": {}, "counters": { "msgr_recv_messages": 7, "msgr_send_messages": 81, "msgr_recv_bytes": 1008, "msgr_send_bytes": 581104, "msgr_created_connections": 15, "msgr_active_connections": 2, "msgr_running_total_time": 0.036220743, "msgr_running_send_time": 0.012131766, "msgr_running_recv_time": 0.011984045, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 75, "sum": 0.062115163, "avgtime": 0.000828202 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 1008, "msgr_send_encrypted_bytes": 581104 } } ], "AsyncMessenger::Worker-2": [ { "labels": {}, "counters": { "msgr_recv_messages": 41, "msgr_send_messages": 7, "msgr_recv_bytes": 1641952, "msgr_send_bytes": 1264, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.047663729, "msgr_running_send_time": 0.003168493, "msgr_running_recv_time": 0.081571777, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 7, "sum": 0.003249255, "avgtime": 0.000464179 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 1641952, "msgr_send_encrypted_bytes": 1264 } } ], "cct": [ { "labels": {}, "counters": { "total_workers": 0, "unhealthy_workers": 0 } } ], "cephfs_mirror": [ { "labels": {}, "counters": { "mirrored_filesystems": 1, "mirror_enable_failures": 0 } } ], "cephfs_mirror_mirrored_filesystems": [ { "labels": { "filesystem": "cephfs" }, "counters": { "mirroring_peers": 0, "directory_count": 0 } } ], "mempool": [ { "labels": {}, "counters": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_items": 0, "bluefs_bytes": 0, "bluefs_items": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "buffer_anon_bytes": 183519, "buffer_anon_items": 55, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 18464, "osdmap_items": 165, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "mds_co_bytes": 0, "mds_co_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 } } ], "objecter": [ { "labels": {}, "counters": { "op_active": 0, "op_laggy": 0, "op_send": 0, "op_send_bytes": 0, "op_resend": 0, "op_reply": 0, "op_latency": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "op_inflight": 0, "oplen_avg": { "avgcount": 0, "sum": 0 }, "op": 0, "op_r": 0, "op_w": 0, "op_rmw": 0, "op_pg": 0, "osdop_stat": 0, "osdop_create": 0, "osdop_read": 0, "osdop_write": 0, "osdop_writefull": 0, "osdop_writesame": 0, "osdop_append": 0, "osdop_zero": 0, "osdop_truncate": 0, "osdop_delete": 0, "osdop_mapext": 0, "osdop_sparse_read": 0, "osdop_clonerange": 0, "osdop_getxattr": 0, "osdop_setxattr": 0, "osdop_cmpxattr": 0, "osdop_rmxattr": 0, "osdop_resetxattrs": 0, "osdop_call": 0, "osdop_watch": 0, "osdop_notify": 0, "osdop_src_cmpxattr": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_other": 0, "linger_active": 0, "linger_send": 0, "linger_resend": 0, "linger_ping": 0, "poolop_active": 0, "poolop_send": 0, "poolop_resend": 0, "poolstat_active": 0, "poolstat_send": 0, "poolstat_resend": 0, "statfs_active": 0, "statfs_send": 0, "statfs_resend": 0, "command_active": 0, "command_send": 0, "command_resend": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "osd_sessions": 0, "osd_session_open": 0, "osd_session_close": 0, "osd_laggy": 0, "omap_wr": 0, "omap_rd": 0, "omap_del": 0 } } ], "throttle-msgr_dispatch_throttler-client": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 75, "get_sum": 297142, "get_or_fail_fail": 0, "get_or_fail_success": 75, "take": 0, "take_sum": 0, "put": 75, "put_sum": 297142, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-msgr_dispatch_throttler-radosclient": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 45, "get_sum": 1635939, "get_or_fail_fail": 0, "get_or_fail_success": 45, "take": 0, "take_sum": 0, "put": 45, "put_sum": 1635939, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_bytes": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_ops": [ { "labels": {}, "counters": { "val": 0, "max": 1024, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ] } 2024-07-04T17:32:27.903 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:32:27.903 DEBUG:teuthology.orchestra.run.smithi042:mirror status for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@8 2024-07-04T17:32:28.144 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:32:28.144 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_inst": "172.21.15.42:0/2096108670", 2024-07-04T17:32:28.144 INFO:teuthology.orchestra.run.smithi042.stdout: "peers": {}, 2024-07-04T17:32:28.144 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_dirs": { 2024-07-04T17:32:28.144 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_count": 0 2024-07-04T17:32:28.145 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:28.145 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:32:28.162 DEBUG:tasks.cephfs.test_mirroring:command returned={ "rados_inst": "172.21.15.42:0/2096108670", "peers": {}, "snap_dirs": { "dir_count": 0 } } 2024-07-04T17:32:28.162 DEBUG:tasks.cephfs.test_mirroring:initial dir_count=0 2024-07-04T17:32:28.162 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snapshot mirror add cephfs ./d1 2024-07-04T17:32:28.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:28.467+0000 7f49cfe00640 1 -- 172.21.15.42:0/1204711349 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49d00ac5b0 msgr2=0x7f49d00b0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:28.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:28.467+0000 7f49cfe00640 1 --2- 172.21.15.42:0/1204711349 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49d00ac5b0 0x7f49d00b0db0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f49b800a8c0 tx=0x7f49b802b7d0 comp rx=0 tx=0).stop 2024-07-04T17:32:28.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:28.468+0000 7f49cfe00640 1 -- 172.21.15.42:0/1204711349 shutdown_connections 2024-07-04T17:32:28.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:28.468+0000 7f49cfe00640 1 --2- 172.21.15.42:0/1204711349 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49d00ac5b0 0x7f49d00b0db0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:28.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:28.468+0000 7f49cfe00640 1 -- 172.21.15.42:0/1204711349 >> 172.21.15.42:0/1204711349 conn(0x7f49d00afaa0 msgr2=0x7f49d00afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:28.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:28.468+0000 7f49cfe00640 1 -- 172.21.15.42:0/1204711349 shutdown_connections 2024-07-04T17:32:28.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:28.468+0000 7f49cfe00640 1 -- 172.21.15.42:0/1204711349 wait complete. 2024-07-04T17:32:28.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:28.470+0000 7f49cfe00640 1 Processor -- start 2024-07-04T17:32:28.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:28.470+0000 7f49cfe00640 1 -- start start 2024-07-04T17:32:28.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:28.471+0000 7f49cfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49d006da60 0x7f49d013d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:28.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:28.471+0000 7f49cfe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f49d00ae590 con 0x7f49d006da60 2024-07-04T17:32:28.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:28.472+0000 7f49cea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49d006da60 0x7f49d013d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:28.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:28.472+0000 7f49cea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49d006da60 0x7f49d013d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42866/0 (socket says 172.21.15.42:42866) 2024-07-04T17:32:28.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:28.472+0000 7f49cea00640 1 -- 172.21.15.42:0/553090428 learned_addr learned my addr 172.21.15.42:0/553090428 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:28.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:28.472+0000 7f49cea00640 1 -- 172.21.15.42:0/553090428 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d013cb40 con 0x7f49d006da60 2024-07-04T17:32:28.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:28.473+0000 7f49cea00640 1 --2- 172.21.15.42:0/553090428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49d006da60 0x7f49d013d020 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f49c000c3b0 tx=0x7f49c000c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:28.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:28.473+0000 7f49c6a00640 1 -- 172.21.15.42:0/553090428 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49c0011780 con 0x7f49d006da60 2024-07-04T17:32:28.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:28.473+0000 7f49c6a00640 1 -- 172.21.15.42:0/553090428 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49c0017070 con 0x7f49d006da60 2024-07-04T17:32:28.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:28.473+0000 7f49c6a00640 1 -- 172.21.15.42:0/553090428 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49c0011c00 con 0x7f49d006da60 2024-07-04T17:32:28.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:28.473+0000 7f49cfe00640 1 -- 172.21.15.42:0/553090428 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d0136a10 con 0x7f49d006da60 2024-07-04T17:32:28.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:28.473+0000 7f49cfe00640 1 -- 172.21.15.42:0/553090428 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d013c080 con 0x7f49d006da60 2024-07-04T17:32:28.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:28.474+0000 7f49cfe00640 1 -- 172.21.15.42:0/553090428 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d00b0db0 con 0x7f49d006da60 2024-07-04T17:32:28.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:28.474+0000 7f49c6a00640 1 -- 172.21.15.42:0/553090428 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) ==== 49681+0+0 (secure 0 0 0) 0x7f49c001c460 con 0x7f49d006da60 2024-07-04T17:32:28.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:28.475+0000 7f49c6a00640 1 --2- 172.21.15.42:0/553090428 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f49ac03d5d0 0x7f49ac03fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:28.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:28.475+0000 7f49c6a00640 1 -- 172.21.15.42:0/553090428 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4647+0+0 (secure 0 0 0) 0x7f49c0051bc0 con 0x7f49d006da60 2024-07-04T17:32:28.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:28.475+0000 7f49ce000640 1 --2- 172.21.15.42:0/553090428 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f49ac03d5d0 0x7f49ac03fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:28.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:28.476+0000 7f49ce000640 1 --2- 172.21.15.42:0/553090428 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f49ac03d5d0 0x7f49ac03fae0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f49b8036000 tx=0x7f49b8004730 comp rx=0 tx=0).ready entity=mgr.5396 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:28.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:28.481+0000 7f49c6a00640 1 -- 172.21.15.42:0/553090428 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f49c001c850 con 0x7f49d006da60 2024-07-04T17:32:28.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:28.715+0000 7f49cfe00640 1 -- 172.21.15.42:0/553090428 --> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] -- mgr_command(tid 0: {"prefix": "fs snapshot mirror add", "fs_name": "cephfs", "path": "./d1", "target": ["mon-mgr", ""]}) -- 0x7f49d0136160 con 0x7f49ac03d5d0 2024-07-04T17:32:28.717 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:28.717+0000 7fb3b7800640 -1 mgr.server reply reply (22) Invalid argument ./d1 should be an absolute path 2024-07-04T17:32:28.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:28.717+0000 7f49c6a00640 1 -- 172.21.15.42:0/553090428 <== mgr.5396 v2:172.21.15.42:6834/1323178695 1 ==== mgr_command_reply(tid 0: -22 ./d1 should be an absolute path) ==== 39+0+0 (secure 0 0 0) 0x7f49d0136160 con 0x7f49ac03d5d0 2024-07-04T17:32:28.718 INFO:teuthology.orchestra.run.smithi042.stderr:Error EINVAL: ./d1 should be an absolute path 2024-07-04T17:32:28.736 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-07-04T17:32:28.736 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:32:28.736 DEBUG:teuthology.orchestra.run.smithi042:counter dump for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok counter dump 2024-07-04T17:32:28.979 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:32:28.979 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker": [ 2024-07-04T17:32:28.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:28.980 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:32:28.980 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "0" 2024-07-04T17:32:28.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:28.980 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:28.980 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:32:28.980 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:32:28.980 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:28.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:28.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:28.980 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:32:28.980 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "1" 2024-07-04T17:32:28.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:28.981 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:28.981 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:32:28.981 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:32:28.981 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:28.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:28.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:28.981 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:32:28.981 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "2" 2024-07-04T17:32:28.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:28.981 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:28.981 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:32:28.981 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:32:28.981 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:28.982 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:28.982 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:28.982 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": [ 2024-07-04T17:32:28.982 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:28.982 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:28.982 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:28.982 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 75, 2024-07-04T17:32:28.982 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 4, 2024-07-04T17:32:28.982 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 307392, 2024-07-04T17:32:28.982 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 800, 2024-07-04T17:32:28.982 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:32:28.982 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:32:28.982 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.028679265, 2024-07-04T17:32:28.982 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.003170141, 2024-07-04T17:32:28.983 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.015694758, 2024-07-04T17:32:28.983 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.008302702, 2024-07-04T17:32:28.983 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:32:28.983 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 4, 2024-07-04T17:32:28.983 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.004367369, 2024-07-04T17:32:28.983 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.001091842 2024-07-04T17:32:28.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:28.983 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:32:28.983 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:28.983 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:32:28.983 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:32:28.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:28.983 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 307392, 2024-07-04T17:32:28.983 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 800 2024-07-04T17:32:28.983 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:28.984 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:28.984 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:28.984 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": [ 2024-07-04T17:32:28.984 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:28.984 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:28.984 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:28.984 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 7, 2024-07-04T17:32:28.984 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 81, 2024-07-04T17:32:28.984 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 1008, 2024-07-04T17:32:28.984 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 581104, 2024-07-04T17:32:28.984 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 15, 2024-07-04T17:32:28.984 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 2, 2024-07-04T17:32:28.984 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.036220743, 2024-07-04T17:32:28.984 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.012131766, 2024-07-04T17:32:28.984 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.011984045, 2024-07-04T17:32:28.985 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:32:28.985 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:32:28.985 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 75, 2024-07-04T17:32:28.985 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.062115163, 2024-07-04T17:32:28.985 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000828202 2024-07-04T17:32:28.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:28.985 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:32:28.985 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:28.985 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:32:28.985 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:32:28.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:28.985 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 1008, 2024-07-04T17:32:28.985 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 581104 2024-07-04T17:32:28.985 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:28.986 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:28.986 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:28.986 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": [ 2024-07-04T17:32:28.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:28.986 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:28.986 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:28.986 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 41, 2024-07-04T17:32:28.986 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 7, 2024-07-04T17:32:28.986 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 1641952, 2024-07-04T17:32:28.986 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 1264, 2024-07-04T17:32:28.986 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:32:28.986 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:32:28.986 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.047663729, 2024-07-04T17:32:28.987 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.003168493, 2024-07-04T17:32:28.987 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.081571777, 2024-07-04T17:32:28.987 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:32:28.987 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:32:28.987 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 7, 2024-07-04T17:32:28.987 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.003249255, 2024-07-04T17:32:28.987 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000464179 2024-07-04T17:32:28.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:28.987 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:32:28.987 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:28.987 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:32:28.987 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:32:28.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:28.987 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 1641952, 2024-07-04T17:32:28.988 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 1264 2024-07-04T17:32:28.988 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:28.988 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:28.988 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:28.988 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": [ 2024-07-04T17:32:28.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:28.988 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:28.988 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:28.988 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-07-04T17:32:28.988 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-07-04T17:32:28.988 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:28.988 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:28.988 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:28.988 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror": [ 2024-07-04T17:32:28.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:28.989 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:28.989 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:28.989 INFO:teuthology.orchestra.run.smithi042.stdout: "mirrored_filesystems": 1, 2024-07-04T17:32:28.989 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_enable_failures": 0 2024-07-04T17:32:28.989 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:28.989 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:28.989 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:28.989 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_mirrored_filesystems": [ 2024-07-04T17:32:28.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:28.989 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:32:28.989 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem": "cephfs" 2024-07-04T17:32:28.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:28.989 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:28.989 INFO:teuthology.orchestra.run.smithi042.stdout: "mirroring_peers": 0, 2024-07-04T17:32:28.990 INFO:teuthology.orchestra.run.smithi042.stdout: "directory_count": 0 2024-07-04T17:32:28.990 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:28.990 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:28.990 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:28.990 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": [ 2024-07-04T17:32:28.990 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:28.991 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:28.991 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:28.991 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-07-04T17:32:28.991 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-07-04T17:32:28.991 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T17:32:28.991 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-07-04T17:32:28.991 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T17:32:28.991 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-07-04T17:32:28.992 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T17:32:28.992 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T17:32:28.992 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T17:32:28.992 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T17:32:28.992 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T17:32:28.992 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-07-04T17:32:28.992 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T17:32:28.992 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T17:32:28.992 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-07-04T17:32:28.992 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-07-04T17:32:28.992 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-07-04T17:32:28.992 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-07-04T17:32:28.993 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T17:32:28.993 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T17:32:28.993 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T17:32:28.993 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T17:32:28.993 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T17:32:28.993 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-07-04T17:32:28.993 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-07-04T17:32:28.993 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-07-04T17:32:28.993 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T17:32:28.993 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T17:32:28.993 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-07-04T17:32:28.993 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-07-04T17:32:28.993 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-07-04T17:32:28.993 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-07-04T17:32:28.993 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T17:32:28.994 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-07-04T17:32:28.994 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T17:32:28.994 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-07-04T17:32:28.994 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 183519, 2024-07-04T17:32:28.994 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 55, 2024-07-04T17:32:28.994 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-07-04T17:32:28.994 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-07-04T17:32:28.994 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-07-04T17:32:28.994 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-07-04T17:32:28.994 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-07-04T17:32:28.994 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-07-04T17:32:28.994 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-07-04T17:32:28.994 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-07-04T17:32:28.995 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 18464, 2024-07-04T17:32:28.995 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 165, 2024-07-04T17:32:28.995 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T17:32:28.995 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-07-04T17:32:28.995 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-07-04T17:32:28.995 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-07-04T17:32:28.995 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 0, 2024-07-04T17:32:28.995 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 0, 2024-07-04T17:32:28.995 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-07-04T17:32:28.995 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-07-04T17:32:28.995 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-07-04T17:32:28.995 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-07-04T17:32:28.995 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:28.996 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:28.996 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:28.996 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter": [ 2024-07-04T17:32:28.996 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:28.996 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:28.996 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:28.996 INFO:teuthology.orchestra.run.smithi042.stdout: "op_active": 0, 2024-07-04T17:32:28.996 INFO:teuthology.orchestra.run.smithi042.stdout: "op_laggy": 0, 2024-07-04T17:32:28.996 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send": 0, 2024-07-04T17:32:28.996 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send_bytes": 0, 2024-07-04T17:32:28.996 INFO:teuthology.orchestra.run.smithi042.stdout: "op_resend": 0, 2024-07-04T17:32:28.996 INFO:teuthology.orchestra.run.smithi042.stdout: "op_reply": 0, 2024-07-04T17:32:28.996 INFO:teuthology.orchestra.run.smithi042.stdout: "op_latency": { 2024-07-04T17:32:28.996 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:28.997 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:32:28.997 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:32:28.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:28.997 INFO:teuthology.orchestra.run.smithi042.stdout: "op_inflight": 0, 2024-07-04T17:32:28.997 INFO:teuthology.orchestra.run.smithi042.stdout: "oplen_avg": { 2024-07-04T17:32:28.997 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:28.997 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0 2024-07-04T17:32:28.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:28.997 INFO:teuthology.orchestra.run.smithi042.stdout: "op": 0, 2024-07-04T17:32:28.997 INFO:teuthology.orchestra.run.smithi042.stdout: "op_r": 0, 2024-07-04T17:32:28.997 INFO:teuthology.orchestra.run.smithi042.stdout: "op_w": 0, 2024-07-04T17:32:28.997 INFO:teuthology.orchestra.run.smithi042.stdout: "op_rmw": 0, 2024-07-04T17:32:28.997 INFO:teuthology.orchestra.run.smithi042.stdout: "op_pg": 0, 2024-07-04T17:32:28.997 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_stat": 0, 2024-07-04T17:32:28.997 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_create": 0, 2024-07-04T17:32:28.998 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_read": 0, 2024-07-04T17:32:28.998 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_write": 0, 2024-07-04T17:32:28.998 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writefull": 0, 2024-07-04T17:32:28.998 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writesame": 0, 2024-07-04T17:32:28.998 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_append": 0, 2024-07-04T17:32:28.998 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_zero": 0, 2024-07-04T17:32:28.998 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_truncate": 0, 2024-07-04T17:32:28.998 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_delete": 0, 2024-07-04T17:32:28.998 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_mapext": 0, 2024-07-04T17:32:28.998 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_sparse_read": 0, 2024-07-04T17:32:28.998 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_clonerange": 0, 2024-07-04T17:32:28.998 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_getxattr": 0, 2024-07-04T17:32:28.998 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_setxattr": 0, 2024-07-04T17:32:28.999 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_cmpxattr": 0, 2024-07-04T17:32:28.999 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_rmxattr": 0, 2024-07-04T17:32:28.999 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_resetxattrs": 0, 2024-07-04T17:32:28.999 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_call": 0, 2024-07-04T17:32:28.999 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_watch": 0, 2024-07-04T17:32:28.999 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_notify": 0, 2024-07-04T17:32:28.999 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T17:32:28.999 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls": 0, 2024-07-04T17:32:28.999 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls_filter": 0, 2024-07-04T17:32:28.999 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_other": 0, 2024-07-04T17:32:28.999 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_active": 0, 2024-07-04T17:32:28.999 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_send": 0, 2024-07-04T17:32:28.999 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_resend": 0, 2024-07-04T17:32:28.999 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_ping": 0, 2024-07-04T17:32:29.000 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_active": 0, 2024-07-04T17:32:29.000 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_send": 0, 2024-07-04T17:32:29.000 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_resend": 0, 2024-07-04T17:32:29.000 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_active": 0, 2024-07-04T17:32:29.000 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_send": 0, 2024-07-04T17:32:29.000 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_resend": 0, 2024-07-04T17:32:29.000 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_active": 0, 2024-07-04T17:32:29.000 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_send": 0, 2024-07-04T17:32:29.000 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_resend": 0, 2024-07-04T17:32:29.000 INFO:teuthology.orchestra.run.smithi042.stdout: "command_active": 0, 2024-07-04T17:32:29.000 INFO:teuthology.orchestra.run.smithi042.stdout: "command_send": 0, 2024-07-04T17:32:29.000 INFO:teuthology.orchestra.run.smithi042.stdout: "command_resend": 0, 2024-07-04T17:32:29.000 INFO:teuthology.orchestra.run.smithi042.stdout: "map_epoch": 0, 2024-07-04T17:32:29.000 INFO:teuthology.orchestra.run.smithi042.stdout: "map_full": 0, 2024-07-04T17:32:29.001 INFO:teuthology.orchestra.run.smithi042.stdout: "map_inc": 0, 2024-07-04T17:32:29.001 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_sessions": 0, 2024-07-04T17:32:29.001 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_open": 0, 2024-07-04T17:32:29.001 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_close": 0, 2024-07-04T17:32:29.001 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_laggy": 0, 2024-07-04T17:32:29.001 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_wr": 0, 2024-07-04T17:32:29.001 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_rd": 0, 2024-07-04T17:32:29.001 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_del": 0 2024-07-04T17:32:29.001 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:29.001 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:29.001 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:29.001 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-client": [ 2024-07-04T17:32:29.001 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:29.001 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:29.001 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:29.002 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:32:29.002 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:32:29.002 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:32:29.002 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 75, 2024-07-04T17:32:29.002 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 297142, 2024-07-04T17:32:29.002 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:32:29.002 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 75, 2024-07-04T17:32:29.002 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:32:29.002 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:32:29.002 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 75, 2024-07-04T17:32:29.002 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 297142, 2024-07-04T17:32:29.002 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:32:29.002 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:29.002 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:32:29.002 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:32:29.003 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:29.003 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:29.003 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:29.003 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:29.003 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-radosclient": [ 2024-07-04T17:32:29.003 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:29.003 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:29.003 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:29.003 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:32:29.003 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:32:29.003 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:32:29.003 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 45, 2024-07-04T17:32:29.003 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 1635939, 2024-07-04T17:32:29.003 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:32:29.003 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 45, 2024-07-04T17:32:29.003 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:32:29.004 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:32:29.004 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 45, 2024-07-04T17:32:29.004 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 1635939, 2024-07-04T17:32:29.004 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:32:29.004 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:29.004 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:32:29.004 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:32:29.004 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:29.004 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:29.004 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:29.004 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:29.004 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_bytes": [ 2024-07-04T17:32:29.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:29.005 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:29.005 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:29.005 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:32:29.005 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:32:29.005 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:32:29.005 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:32:29.005 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:32:29.005 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:32:29.005 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:32:29.005 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:32:29.005 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:32:29.005 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:32:29.005 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:32:29.006 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:32:29.006 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:29.006 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:32:29.006 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:32:29.006 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:29.006 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:29.006 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:29.006 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:29.006 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_ops": [ 2024-07-04T17:32:29.006 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:29.006 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:29.006 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:29.006 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:32:29.007 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1024, 2024-07-04T17:32:29.007 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:32:29.007 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:32:29.007 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:32:29.007 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:32:29.007 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:32:29.007 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:32:29.007 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:32:29.007 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:32:29.007 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:32:29.007 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:32:29.007 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:29.007 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:32:29.007 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:32:29.007 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:29.008 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:29.008 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:29.008 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-07-04T17:32:29.008 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:32:29.009 DEBUG:tasks.cephfs.test_mirroring:command returned={ "AsyncMessenger::Worker": [ { "labels": { "id": "0" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "1" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "2" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } } ], "AsyncMessenger::Worker-0": [ { "labels": {}, "counters": { "msgr_recv_messages": 75, "msgr_send_messages": 4, "msgr_recv_bytes": 307392, "msgr_send_bytes": 800, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.028679265, "msgr_running_send_time": 0.003170141, "msgr_running_recv_time": 0.015694758, "msgr_running_fast_dispatch_time": 0.008302702, "msgr_send_messages_queue_lat": { "avgcount": 4, "sum": 0.004367369, "avgtime": 0.001091842 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 307392, "msgr_send_encrypted_bytes": 800 } } ], "AsyncMessenger::Worker-1": [ { "labels": {}, "counters": { "msgr_recv_messages": 7, "msgr_send_messages": 81, "msgr_recv_bytes": 1008, "msgr_send_bytes": 581104, "msgr_created_connections": 15, "msgr_active_connections": 2, "msgr_running_total_time": 0.036220743, "msgr_running_send_time": 0.012131766, "msgr_running_recv_time": 0.011984045, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 75, "sum": 0.062115163, "avgtime": 0.000828202 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 1008, "msgr_send_encrypted_bytes": 581104 } } ], "AsyncMessenger::Worker-2": [ { "labels": {}, "counters": { "msgr_recv_messages": 41, "msgr_send_messages": 7, "msgr_recv_bytes": 1641952, "msgr_send_bytes": 1264, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.047663729, "msgr_running_send_time": 0.003168493, "msgr_running_recv_time": 0.081571777, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 7, "sum": 0.003249255, "avgtime": 0.000464179 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 1641952, "msgr_send_encrypted_bytes": 1264 } } ], "cct": [ { "labels": {}, "counters": { "total_workers": 0, "unhealthy_workers": 0 } } ], "cephfs_mirror": [ { "labels": {}, "counters": { "mirrored_filesystems": 1, "mirror_enable_failures": 0 } } ], "cephfs_mirror_mirrored_filesystems": [ { "labels": { "filesystem": "cephfs" }, "counters": { "mirroring_peers": 0, "directory_count": 0 } } ], "mempool": [ { "labels": {}, "counters": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_items": 0, "bluefs_bytes": 0, "bluefs_items": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "buffer_anon_bytes": 183519, "buffer_anon_items": 55, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 18464, "osdmap_items": 165, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "mds_co_bytes": 0, "mds_co_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 } } ], "objecter": [ { "labels": {}, "counters": { "op_active": 0, "op_laggy": 0, "op_send": 0, "op_send_bytes": 0, "op_resend": 0, "op_reply": 0, "op_latency": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "op_inflight": 0, "oplen_avg": { "avgcount": 0, "sum": 0 }, "op": 0, "op_r": 0, "op_w": 0, "op_rmw": 0, "op_pg": 0, "osdop_stat": 0, "osdop_create": 0, "osdop_read": 0, "osdop_write": 0, "osdop_writefull": 0, "osdop_writesame": 0, "osdop_append": 0, "osdop_zero": 0, "osdop_truncate": 0, "osdop_delete": 0, "osdop_mapext": 0, "osdop_sparse_read": 0, "osdop_clonerange": 0, "osdop_getxattr": 0, "osdop_setxattr": 0, "osdop_cmpxattr": 0, "osdop_rmxattr": 0, "osdop_resetxattrs": 0, "osdop_call": 0, "osdop_watch": 0, "osdop_notify": 0, "osdop_src_cmpxattr": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_other": 0, "linger_active": 0, "linger_send": 0, "linger_resend": 0, "linger_ping": 0, "poolop_active": 0, "poolop_send": 0, "poolop_resend": 0, "poolstat_active": 0, "poolstat_send": 0, "poolstat_resend": 0, "statfs_active": 0, "statfs_send": 0, "statfs_resend": 0, "command_active": 0, "command_send": 0, "command_resend": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "osd_sessions": 0, "osd_session_open": 0, "osd_session_close": 0, "osd_laggy": 0, "omap_wr": 0, "omap_rd": 0, "omap_del": 0 } } ], "throttle-msgr_dispatch_throttler-client": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 75, "get_sum": 297142, "get_or_fail_fail": 0, "get_or_fail_success": 75, "take": 0, "take_sum": 0, "put": 75, "put_sum": 297142, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-msgr_dispatch_throttler-radosclient": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 45, "get_sum": 1635939, "get_or_fail_fail": 0, "get_or_fail_success": 45, "take": 0, "take_sum": 0, "put": 45, "put_sum": 1635939, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_bytes": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_ops": [ { "labels": {}, "counters": { "val": 0, "max": 1024, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ] } 2024-07-04T17:32:29.009 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snapshot mirror disable cephfs 2024-07-04T17:32:29.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:29.299+0000 7f7a35c00640 1 -- 172.21.15.42:0/1939113826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a3006d150 msgr2=0x7f7a30073a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:29.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:29.299+0000 7f7a35c00640 1 --2- 172.21.15.42:0/1939113826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a3006d150 0x7f7a30073a20 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f7a200097b0 tx=0x7f7a2002a530 comp rx=0 tx=0).stop 2024-07-04T17:32:29.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:29.299+0000 7f7a35c00640 1 -- 172.21.15.42:0/1939113826 shutdown_connections 2024-07-04T17:32:29.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:29.299+0000 7f7a35c00640 1 --2- 172.21.15.42:0/1939113826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a3006d150 0x7f7a30073a20 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:29.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:29.299+0000 7f7a35c00640 1 -- 172.21.15.42:0/1939113826 >> 172.21.15.42:0/1939113826 conn(0x7f7a3006c020 msgr2=0x7f7a3006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:29.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:29.299+0000 7f7a35c00640 1 -- 172.21.15.42:0/1939113826 shutdown_connections 2024-07-04T17:32:29.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:29.299+0000 7f7a35c00640 1 -- 172.21.15.42:0/1939113826 wait complete. 2024-07-04T17:32:29.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:29.302+0000 7f7a35c00640 1 Processor -- start 2024-07-04T17:32:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:29.302+0000 7f7a35c00640 1 -- start start 2024-07-04T17:32:29.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:29.303+0000 7f7a35c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a3006d150 0x7f7a300aca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:29.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:29.303+0000 7f7a35c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7a30071c40 con 0x7f7a3006d150 2024-07-04T17:32:29.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:29.303+0000 7f7a2ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a3006d150 0x7f7a300aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:29.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:29.303+0000 7f7a2ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a3006d150 0x7f7a300aca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42886/0 (socket says 172.21.15.42:42886) 2024-07-04T17:32:29.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:29.303+0000 7f7a2ea00640 1 -- 172.21.15.42:0/3627312683 learned_addr learned my addr 172.21.15.42:0/3627312683 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:29.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:29.304+0000 7f7a2ea00640 1 -- 172.21.15.42:0/3627312683 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a300ad460 con 0x7f7a3006d150 2024-07-04T17:32:29.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:29.304+0000 7f7a2ea00640 1 --2- 172.21.15.42:0/3627312683 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a3006d150 0x7f7a300aca50 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f7a200098d0 tx=0x7f7a20033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:29.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:29.304+0000 7f7a26a00640 1 -- 172.21.15.42:0/3627312683 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a20031a90 con 0x7f7a3006d150 2024-07-04T17:32:29.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:29.304+0000 7f7a26a00640 1 -- 172.21.15.42:0/3627312683 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a20031d20 con 0x7f7a3006d150 2024-07-04T17:32:29.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:29.304+0000 7f7a26a00640 1 -- 172.21.15.42:0/3627312683 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a20033070 con 0x7f7a3006d150 2024-07-04T17:32:29.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:29.305+0000 7f7a35c00640 1 -- 172.21.15.42:0/3627312683 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a300b0c00 con 0x7f7a3006d150 2024-07-04T17:32:29.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:29.305+0000 7f7a35c00640 1 -- 172.21.15.42:0/3627312683 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a300ad170 con 0x7f7a3006d150 2024-07-04T17:32:29.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:29.306+0000 7f7a35c00640 1 -- 172.21.15.42:0/3627312683 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a30073a20 con 0x7f7a3006d150 2024-07-04T17:32:29.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:29.306+0000 7f7a26a00640 1 -- 172.21.15.42:0/3627312683 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) ==== 49681+0+0 (secure 0 0 0) 0x7f7a20033210 con 0x7f7a3006d150 2024-07-04T17:32:29.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:29.307+0000 7f7a26a00640 1 --2- 172.21.15.42:0/3627312683 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f7a0c03d5d0 0x7f7a0c03fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:29.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:29.307+0000 7f7a26a00640 1 -- 172.21.15.42:0/3627312683 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4647+0+0 (secure 0 0 0) 0x7f7a200735d0 con 0x7f7a3006d150 2024-07-04T17:32:29.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:29.307+0000 7f7a2e000640 1 --2- 172.21.15.42:0/3627312683 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f7a0c03d5d0 0x7f7a0c03fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:29.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:29.308+0000 7f7a2e000640 1 --2- 172.21.15.42:0/3627312683 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f7a0c03d5d0 0x7f7a0c03fae0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7a18000f60 tx=0x7f7a18007a30 comp rx=0 tx=0).ready entity=mgr.5396 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:29.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:29.312+0000 7f7a26a00640 1 -- 172.21.15.42:0/3627312683 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f7a2003f030 con 0x7f7a3006d150 2024-07-04T17:32:29.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:29.542+0000 7f7a35c00640 1 -- 172.21.15.42:0/3627312683 --> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] -- mgr_command(tid 0: {"prefix": "fs snapshot mirror disable", "fs_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7f7a300b0dd0 con 0x7f7a0c03d5d0 2024-07-04T17:32:30.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:30.414+0000 7f7a26a00640 1 -- 172.21.15.42:0/3627312683 <== mgr.5396 v2:172.21.15.42:6834/1323178695 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f7a300b0dd0 con 0x7f7a0c03d5d0 2024-07-04T17:32:30.415 INFO:teuthology.orchestra.run.smithi042.stdout:{} 2024-07-04T17:32:30.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:30.417+0000 7f7a35c00640 1 -- 172.21.15.42:0/3627312683 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f7a0c03d5d0 msgr2=0x7f7a0c03fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:30.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:30.417+0000 7f7a35c00640 1 --2- 172.21.15.42:0/3627312683 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f7a0c03d5d0 0x7f7a0c03fae0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7a18000f60 tx=0x7f7a18007a30 comp rx=0 tx=0).stop 2024-07-04T17:32:30.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:30.418+0000 7f7a35c00640 1 -- 172.21.15.42:0/3627312683 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a3006d150 msgr2=0x7f7a300aca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:30.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:30.418+0000 7f7a35c00640 1 --2- 172.21.15.42:0/3627312683 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a3006d150 0x7f7a300aca50 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f7a200098d0 tx=0x7f7a20033c10 comp rx=0 tx=0).stop 2024-07-04T17:32:30.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:30.418+0000 7f7a35c00640 1 -- 172.21.15.42:0/3627312683 shutdown_connections 2024-07-04T17:32:30.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:30.418+0000 7f7a35c00640 1 --2- 172.21.15.42:0/3627312683 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f7a0c03d5d0 0x7f7a0c03fae0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:30.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:30.418+0000 7f7a35c00640 1 --2- 172.21.15.42:0/3627312683 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a3006d150 0x7f7a300aca50 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:30.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:30.418+0000 7f7a35c00640 1 -- 172.21.15.42:0/3627312683 >> 172.21.15.42:0/3627312683 conn(0x7f7a3006c020 msgr2=0x7f7a300b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:30.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:30.418+0000 7f7a35c00640 1 -- 172.21.15.42:0/3627312683 shutdown_connections 2024-07-04T17:32:30.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:30.418+0000 7f7a35c00640 1 -- 172.21.15.42:0/3627312683 wait complete. 2024-07-04T17:32:30.915 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-04T17:32:40.447 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:32:40.447 DEBUG:teuthology.orchestra.run.smithi042:mirror status for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@8 2024-07-04T17:32:40.698 INFO:teuthology.orchestra.run.smithi042.stderr:no valid command found; 10 closest matches: 2024-07-04T17:32:40.698 INFO:teuthology.orchestra.run.smithi042.stderr:0 2024-07-04T17:32:40.698 INFO:teuthology.orchestra.run.smithi042.stderr:1 2024-07-04T17:32:40.698 INFO:teuthology.orchestra.run.smithi042.stderr:2 2024-07-04T17:32:40.699 INFO:teuthology.orchestra.run.smithi042.stderr:abort 2024-07-04T17:32:40.699 INFO:teuthology.orchestra.run.smithi042.stderr:assert 2024-07-04T17:32:40.699 INFO:teuthology.orchestra.run.smithi042.stderr:config diff 2024-07-04T17:32:40.699 INFO:teuthology.orchestra.run.smithi042.stderr:config diff get 2024-07-04T17:32:40.699 INFO:teuthology.orchestra.run.smithi042.stderr:config get 2024-07-04T17:32:40.699 INFO:teuthology.orchestra.run.smithi042.stderr:config help [] 2024-07-04T17:32:40.699 INFO:teuthology.orchestra.run.smithi042.stderr:config set ... 2024-07-04T17:32:40.699 INFO:teuthology.orchestra.run.smithi042.stderr:admin_socket: invalid command 2024-07-04T17:32:40.708 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-07-04T17:32:40.709 WARNING:tasks.cephfs.test_mirroring:mirror daemon command with label "mirror status for fs: cephfs" failed: Command failed (mirror status for fs: cephfs) on smithi042 with status 22: 'ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@8' 2024-07-04T17:32:40.710 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:32:40.710 DEBUG:teuthology.orchestra.run.smithi042:counter dump for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok counter dump 2024-07-04T17:32:40.956 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:32:40.956 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker": [ 2024-07-04T17:32:40.956 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:40.956 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:32:40.956 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "0" 2024-07-04T17:32:40.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:40.956 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:40.956 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:32:40.956 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:32:40.956 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:40.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:40.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:40.957 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:32:40.957 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "1" 2024-07-04T17:32:40.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:40.957 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:40.957 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:32:40.957 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:32:40.957 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:40.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:40.957 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:40.957 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:32:40.957 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "2" 2024-07-04T17:32:40.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:40.958 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:40.958 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:32:40.958 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:32:40.958 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:40.958 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:40.958 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:40.958 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": [ 2024-07-04T17:32:40.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:40.958 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:40.958 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:40.958 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 77, 2024-07-04T17:32:40.959 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 5, 2024-07-04T17:32:40.959 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 312624, 2024-07-04T17:32:40.959 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 1136, 2024-07-04T17:32:40.959 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:32:40.959 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:32:40.959 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.029534251, 2024-07-04T17:32:40.959 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.003435523, 2024-07-04T17:32:40.959 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.016069836, 2024-07-04T17:32:40.959 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.008470794, 2024-07-04T17:32:40.959 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:32:40.959 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 5, 2024-07-04T17:32:40.959 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.004490948, 2024-07-04T17:32:40.959 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000898189 2024-07-04T17:32:40.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:40.960 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:32:40.960 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:40.960 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:32:40.960 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:32:40.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:40.960 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 312624, 2024-07-04T17:32:40.960 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 1136 2024-07-04T17:32:40.960 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:40.960 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:40.960 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:40.960 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": [ 2024-07-04T17:32:40.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:40.960 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:40.961 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:40.961 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 7, 2024-07-04T17:32:40.961 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 83, 2024-07-04T17:32:40.961 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 1008, 2024-07-04T17:32:40.961 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 581616, 2024-07-04T17:32:40.961 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 15, 2024-07-04T17:32:40.961 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 2, 2024-07-04T17:32:40.961 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.036552281, 2024-07-04T17:32:40.961 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.012419631, 2024-07-04T17:32:40.961 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.011984045, 2024-07-04T17:32:40.961 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:32:40.961 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:32:40.961 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 77, 2024-07-04T17:32:40.962 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.062365844, 2024-07-04T17:32:40.962 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000809946 2024-07-04T17:32:40.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:40.962 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:32:40.962 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:40.962 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:32:40.962 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:32:40.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:40.962 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 1008, 2024-07-04T17:32:40.962 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 581616 2024-07-04T17:32:40.962 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:40.963 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:40.963 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:40.963 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": [ 2024-07-04T17:32:40.963 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:40.963 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:40.963 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:40.963 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 42, 2024-07-04T17:32:40.963 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 8, 2024-07-04T17:32:40.963 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 1642720, 2024-07-04T17:32:40.963 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 1600, 2024-07-04T17:32:40.963 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:32:40.963 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:32:40.963 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.048142596, 2024-07-04T17:32:40.964 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.003433898, 2024-07-04T17:32:40.964 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.081750271, 2024-07-04T17:32:40.964 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:32:40.964 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:32:40.964 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 8, 2024-07-04T17:32:40.964 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.003350228, 2024-07-04T17:32:40.964 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000418778 2024-07-04T17:32:40.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:40.964 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:32:40.964 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:40.964 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:32:40.964 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:32:40.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:40.965 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 1642720, 2024-07-04T17:32:40.965 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 1600 2024-07-04T17:32:40.965 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:40.965 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:40.965 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:40.965 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": [ 2024-07-04T17:32:40.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:40.965 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:40.965 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:40.965 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-07-04T17:32:40.965 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-07-04T17:32:40.965 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:40.965 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:40.965 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:40.966 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror": [ 2024-07-04T17:32:40.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:40.966 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:40.966 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:40.966 INFO:teuthology.orchestra.run.smithi042.stdout: "mirrored_filesystems": 0, 2024-07-04T17:32:40.966 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_enable_failures": 0 2024-07-04T17:32:40.966 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:40.966 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:40.966 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:40.966 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": [ 2024-07-04T17:32:40.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:40.966 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:40.966 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:40.966 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-07-04T17:32:40.967 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-07-04T17:32:40.967 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T17:32:40.967 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-07-04T17:32:40.967 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T17:32:40.967 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-07-04T17:32:40.967 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T17:32:40.967 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T17:32:40.967 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T17:32:40.967 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T17:32:40.967 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T17:32:40.967 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-07-04T17:32:40.967 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T17:32:40.968 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T17:32:40.968 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-07-04T17:32:40.968 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-07-04T17:32:40.968 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-07-04T17:32:40.968 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-07-04T17:32:40.968 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T17:32:40.968 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T17:32:40.968 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T17:32:40.968 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T17:32:40.968 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T17:32:40.968 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-07-04T17:32:40.968 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-07-04T17:32:40.968 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-07-04T17:32:40.969 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T17:32:40.969 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T17:32:40.969 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-07-04T17:32:40.969 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-07-04T17:32:40.969 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-07-04T17:32:40.969 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-07-04T17:32:40.969 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T17:32:40.969 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-07-04T17:32:40.969 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T17:32:40.969 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-07-04T17:32:40.969 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 149842, 2024-07-04T17:32:40.969 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 28, 2024-07-04T17:32:40.969 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-07-04T17:32:40.969 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-07-04T17:32:40.970 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-07-04T17:32:40.970 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-07-04T17:32:40.970 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-07-04T17:32:40.970 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-07-04T17:32:40.970 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-07-04T17:32:40.970 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-07-04T17:32:40.970 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 4592, 2024-07-04T17:32:40.970 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 37, 2024-07-04T17:32:40.970 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T17:32:40.970 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-07-04T17:32:40.970 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-07-04T17:32:40.970 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-07-04T17:32:40.970 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 0, 2024-07-04T17:32:40.971 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 0, 2024-07-04T17:32:40.971 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-07-04T17:32:40.971 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-07-04T17:32:40.971 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-07-04T17:32:40.971 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-07-04T17:32:40.971 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:40.971 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:40.971 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:40.971 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter": [ 2024-07-04T17:32:40.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:40.971 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:40.971 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:40.971 INFO:teuthology.orchestra.run.smithi042.stdout: "op_active": 0, 2024-07-04T17:32:40.971 INFO:teuthology.orchestra.run.smithi042.stdout: "op_laggy": 0, 2024-07-04T17:32:40.971 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send": 0, 2024-07-04T17:32:40.972 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send_bytes": 0, 2024-07-04T17:32:40.972 INFO:teuthology.orchestra.run.smithi042.stdout: "op_resend": 0, 2024-07-04T17:32:40.972 INFO:teuthology.orchestra.run.smithi042.stdout: "op_reply": 0, 2024-07-04T17:32:40.972 INFO:teuthology.orchestra.run.smithi042.stdout: "op_latency": { 2024-07-04T17:32:40.972 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:40.972 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:32:40.972 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:32:40.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:40.972 INFO:teuthology.orchestra.run.smithi042.stdout: "op_inflight": 0, 2024-07-04T17:32:40.972 INFO:teuthology.orchestra.run.smithi042.stdout: "oplen_avg": { 2024-07-04T17:32:40.972 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:40.973 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0 2024-07-04T17:32:40.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:32:40.973 INFO:teuthology.orchestra.run.smithi042.stdout: "op": 0, 2024-07-04T17:32:40.973 INFO:teuthology.orchestra.run.smithi042.stdout: "op_r": 0, 2024-07-04T17:32:40.973 INFO:teuthology.orchestra.run.smithi042.stdout: "op_w": 0, 2024-07-04T17:32:40.973 INFO:teuthology.orchestra.run.smithi042.stdout: "op_rmw": 0, 2024-07-04T17:32:40.973 INFO:teuthology.orchestra.run.smithi042.stdout: "op_pg": 0, 2024-07-04T17:32:40.973 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_stat": 0, 2024-07-04T17:32:40.973 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_create": 0, 2024-07-04T17:32:40.973 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_read": 0, 2024-07-04T17:32:40.973 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_write": 0, 2024-07-04T17:32:40.973 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writefull": 0, 2024-07-04T17:32:40.974 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writesame": 0, 2024-07-04T17:32:40.974 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_append": 0, 2024-07-04T17:32:40.974 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_zero": 0, 2024-07-04T17:32:40.974 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_truncate": 0, 2024-07-04T17:32:40.974 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_delete": 0, 2024-07-04T17:32:40.974 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_mapext": 0, 2024-07-04T17:32:40.974 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_sparse_read": 0, 2024-07-04T17:32:40.974 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_clonerange": 0, 2024-07-04T17:32:40.974 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_getxattr": 0, 2024-07-04T17:32:40.974 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_setxattr": 0, 2024-07-04T17:32:40.974 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_cmpxattr": 0, 2024-07-04T17:32:40.974 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_rmxattr": 0, 2024-07-04T17:32:40.975 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_resetxattrs": 0, 2024-07-04T17:32:40.975 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_call": 0, 2024-07-04T17:32:40.975 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_watch": 0, 2024-07-04T17:32:40.975 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_notify": 0, 2024-07-04T17:32:40.975 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T17:32:40.975 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls": 0, 2024-07-04T17:32:40.975 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls_filter": 0, 2024-07-04T17:32:40.975 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_other": 0, 2024-07-04T17:32:40.975 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_active": 0, 2024-07-04T17:32:40.975 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_send": 0, 2024-07-04T17:32:40.975 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_resend": 0, 2024-07-04T17:32:40.975 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_ping": 0, 2024-07-04T17:32:40.975 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_active": 0, 2024-07-04T17:32:40.976 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_send": 0, 2024-07-04T17:32:40.976 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_resend": 0, 2024-07-04T17:32:40.976 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_active": 0, 2024-07-04T17:32:40.976 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_send": 0, 2024-07-04T17:32:40.976 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_resend": 0, 2024-07-04T17:32:40.976 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_active": 0, 2024-07-04T17:32:40.976 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_send": 0, 2024-07-04T17:32:40.976 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_resend": 0, 2024-07-04T17:32:40.976 INFO:teuthology.orchestra.run.smithi042.stdout: "command_active": 0, 2024-07-04T17:32:40.976 INFO:teuthology.orchestra.run.smithi042.stdout: "command_send": 0, 2024-07-04T17:32:40.976 INFO:teuthology.orchestra.run.smithi042.stdout: "command_resend": 0, 2024-07-04T17:32:40.976 INFO:teuthology.orchestra.run.smithi042.stdout: "map_epoch": 0, 2024-07-04T17:32:40.976 INFO:teuthology.orchestra.run.smithi042.stdout: "map_full": 0, 2024-07-04T17:32:40.976 INFO:teuthology.orchestra.run.smithi042.stdout: "map_inc": 0, 2024-07-04T17:32:40.977 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_sessions": 0, 2024-07-04T17:32:40.977 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_open": 0, 2024-07-04T17:32:40.977 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_close": 0, 2024-07-04T17:32:40.977 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_laggy": 0, 2024-07-04T17:32:40.977 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_wr": 0, 2024-07-04T17:32:40.977 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_rd": 0, 2024-07-04T17:32:40.977 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_del": 0 2024-07-04T17:32:40.977 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:40.977 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:40.977 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:40.977 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-client": [ 2024-07-04T17:32:40.977 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:40.977 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:40.977 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:40.978 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:32:40.978 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:32:40.978 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:32:40.978 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 77, 2024-07-04T17:32:40.978 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 302102, 2024-07-04T17:32:40.978 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:32:40.978 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 77, 2024-07-04T17:32:40.978 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:32:40.978 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:32:40.978 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 77, 2024-07-04T17:32:40.978 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 302102, 2024-07-04T17:32:40.978 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:32:40.978 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:40.979 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:32:40.979 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:32:40.979 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:40.979 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:40.979 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:40.979 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:40.979 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-radosclient": [ 2024-07-04T17:32:40.979 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:40.979 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:40.979 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:40.979 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:32:40.979 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:32:40.980 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:32:40.980 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 46, 2024-07-04T17:32:40.980 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 1636567, 2024-07-04T17:32:40.980 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:32:40.980 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 46, 2024-07-04T17:32:40.980 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:32:40.980 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:32:40.980 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 46, 2024-07-04T17:32:40.980 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 1636567, 2024-07-04T17:32:40.980 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:32:40.980 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:40.980 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:32:40.981 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:32:40.981 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:40.981 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:40.981 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:40.981 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:40.981 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_bytes": [ 2024-07-04T17:32:40.981 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:40.981 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:40.981 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:40.981 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:32:40.981 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:32:40.981 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:32:40.981 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:32:40.982 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:32:40.982 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:32:40.982 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:32:40.982 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:32:40.982 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:32:40.982 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:32:40.982 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:32:40.982 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:32:40.982 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:40.983 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:32:40.983 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:32:40.983 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:40.983 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:40.983 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:40.983 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:32:40.983 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_ops": [ 2024-07-04T17:32:40.983 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:32:40.983 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:32:40.983 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:32:40.983 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:32:40.983 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1024, 2024-07-04T17:32:40.983 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:32:40.984 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:32:40.984 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:32:40.984 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:32:40.984 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:32:40.984 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:32:40.984 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:32:40.984 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:32:40.984 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:32:40.984 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:32:40.984 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:32:40.984 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:32:40.985 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:32:40.985 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:40.985 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:40.985 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:32:40.985 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-07-04T17:32:40.985 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:32:40.986 DEBUG:tasks.cephfs.test_mirroring:command returned={ "AsyncMessenger::Worker": [ { "labels": { "id": "0" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "1" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "2" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } } ], "AsyncMessenger::Worker-0": [ { "labels": {}, "counters": { "msgr_recv_messages": 77, "msgr_send_messages": 5, "msgr_recv_bytes": 312624, "msgr_send_bytes": 1136, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.029534251, "msgr_running_send_time": 0.003435523, "msgr_running_recv_time": 0.016069836, "msgr_running_fast_dispatch_time": 0.008470794, "msgr_send_messages_queue_lat": { "avgcount": 5, "sum": 0.004490948, "avgtime": 0.000898189 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 312624, "msgr_send_encrypted_bytes": 1136 } } ], "AsyncMessenger::Worker-1": [ { "labels": {}, "counters": { "msgr_recv_messages": 7, "msgr_send_messages": 83, "msgr_recv_bytes": 1008, "msgr_send_bytes": 581616, "msgr_created_connections": 15, "msgr_active_connections": 2, "msgr_running_total_time": 0.036552281, "msgr_running_send_time": 0.012419631, "msgr_running_recv_time": 0.011984045, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 77, "sum": 0.062365844, "avgtime": 0.000809946 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 1008, "msgr_send_encrypted_bytes": 581616 } } ], "AsyncMessenger::Worker-2": [ { "labels": {}, "counters": { "msgr_recv_messages": 42, "msgr_send_messages": 8, "msgr_recv_bytes": 1642720, "msgr_send_bytes": 1600, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.048142596, "msgr_running_send_time": 0.003433898, "msgr_running_recv_time": 0.081750271, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 8, "sum": 0.003350228, "avgtime": 0.000418778 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 1642720, "msgr_send_encrypted_bytes": 1600 } } ], "cct": [ { "labels": {}, "counters": { "total_workers": 0, "unhealthy_workers": 0 } } ], "cephfs_mirror": [ { "labels": {}, "counters": { "mirrored_filesystems": 0, "mirror_enable_failures": 0 } } ], "mempool": [ { "labels": {}, "counters": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_items": 0, "bluefs_bytes": 0, "bluefs_items": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "buffer_anon_bytes": 149842, "buffer_anon_items": 28, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 4592, "osdmap_items": 37, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "mds_co_bytes": 0, "mds_co_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 } } ], "objecter": [ { "labels": {}, "counters": { "op_active": 0, "op_laggy": 0, "op_send": 0, "op_send_bytes": 0, "op_resend": 0, "op_reply": 0, "op_latency": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "op_inflight": 0, "oplen_avg": { "avgcount": 0, "sum": 0 }, "op": 0, "op_r": 0, "op_w": 0, "op_rmw": 0, "op_pg": 0, "osdop_stat": 0, "osdop_create": 0, "osdop_read": 0, "osdop_write": 0, "osdop_writefull": 0, "osdop_writesame": 0, "osdop_append": 0, "osdop_zero": 0, "osdop_truncate": 0, "osdop_delete": 0, "osdop_mapext": 0, "osdop_sparse_read": 0, "osdop_clonerange": 0, "osdop_getxattr": 0, "osdop_setxattr": 0, "osdop_cmpxattr": 0, "osdop_rmxattr": 0, "osdop_resetxattrs": 0, "osdop_call": 0, "osdop_watch": 0, "osdop_notify": 0, "osdop_src_cmpxattr": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_other": 0, "linger_active": 0, "linger_send": 0, "linger_resend": 0, "linger_ping": 0, "poolop_active": 0, "poolop_send": 0, "poolop_resend": 0, "poolstat_active": 0, "poolstat_send": 0, "poolstat_resend": 0, "statfs_active": 0, "statfs_send": 0, "statfs_resend": 0, "command_active": 0, "command_send": 0, "command_resend": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "osd_sessions": 0, "osd_session_open": 0, "osd_session_close": 0, "osd_laggy": 0, "omap_wr": 0, "omap_rd": 0, "omap_del": 0 } } ], "throttle-msgr_dispatch_throttler-client": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 77, "get_sum": 302102, "get_or_fail_fail": 0, "get_or_fail_success": 77, "take": 0, "take_sum": 0, "put": 77, "put_sum": 302102, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-msgr_dispatch_throttler-radosclient": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 46, "get_sum": 1636567, "get_or_fail_fail": 0, "get_or_fail_success": 46, "take": 0, "take_sum": 0, "put": 46, "put_sum": 1636567, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_bytes": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_ops": [ { "labels": {}, "counters": { "val": 0, "max": 1024, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ] } 2024-07-04T17:32:40.987 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable mirroring 2024-07-04T17:32:41.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:41.284+0000 7fb21bc00640 1 -- 172.21.15.42:0/2649572355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2140b24a0 msgr2=0x7fb214070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:41.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:41.284+0000 7fb21bc00640 1 --2- 172.21.15.42:0/2649572355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2140b24a0 0x7fb214070cf0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fb208005fd0 tx=0x7fb20802af80 comp rx=0 tx=0).stop 2024-07-04T17:32:41.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:41.284+0000 7fb21bc00640 1 -- 172.21.15.42:0/2649572355 shutdown_connections 2024-07-04T17:32:41.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:41.284+0000 7fb21bc00640 1 --2- 172.21.15.42:0/2649572355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2140b24a0 0x7fb214070cf0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:41.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:41.284+0000 7fb21bc00640 1 -- 172.21.15.42:0/2649572355 >> 172.21.15.42:0/2649572355 conn(0x7fb21406fe40 msgr2=0x7fb21406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:41.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:41.285+0000 7fb21bc00640 1 -- 172.21.15.42:0/2649572355 shutdown_connections 2024-07-04T17:32:41.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:41.285+0000 7fb21bc00640 1 -- 172.21.15.42:0/2649572355 wait complete. 2024-07-04T17:32:41.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:41.287+0000 7fb21bc00640 1 Processor -- start 2024-07-04T17:32:41.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:41.287+0000 7fb21bc00640 1 -- start start 2024-07-04T17:32:41.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:41.288+0000 7fb21bc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2140b1500 0x7fb2140bfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:41.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:41.288+0000 7fb21bc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb214071d70 con 0x7fb2140b1500 2024-07-04T17:32:41.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:41.288+0000 7fb219200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2140b1500 0x7fb2140bfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:41.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:41.288+0000 7fb219200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2140b1500 0x7fb2140bfaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40610/0 (socket says 172.21.15.42:40610) 2024-07-04T17:32:41.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:41.288+0000 7fb219200640 1 -- 172.21.15.42:0/641461097 learned_addr learned my addr 172.21.15.42:0/641461097 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:41.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:41.289+0000 7fb219200640 1 -- 172.21.15.42:0/641461097 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2140bd710 con 0x7fb2140b1500 2024-07-04T17:32:41.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:41.289+0000 7fb219200640 1 --2- 172.21.15.42:0/641461097 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2140b1500 0x7fb2140bfaa0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fb208006010 tx=0x7fb208009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:41.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:41.289+0000 7fb210c00640 1 -- 172.21.15.42:0/641461097 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb208040020 con 0x7fb2140b1500 2024-07-04T17:32:41.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:41.289+0000 7fb210c00640 1 -- 172.21.15.42:0/641461097 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb208004040 con 0x7fb2140b1500 2024-07-04T17:32:41.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:41.289+0000 7fb210c00640 1 -- 172.21.15.42:0/641461097 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb208038040 con 0x7fb2140b1500 2024-07-04T17:32:41.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:41.289+0000 7fb21bc00640 1 -- 172.21.15.42:0/641461097 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2140bf190 con 0x7fb2140b1500 2024-07-04T17:32:41.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:41.290+0000 7fb21bc00640 1 -- 172.21.15.42:0/641461097 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2140bd430 con 0x7fb2140b1500 2024-07-04T17:32:41.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:41.291+0000 7fb21bc00640 1 -- 172.21.15.42:0/641461097 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb214070cf0 con 0x7fb2140b1500 2024-07-04T17:32:41.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:41.291+0000 7fb210c00640 1 -- 172.21.15.42:0/641461097 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) ==== 49681+0+0 (secure 0 0 0) 0x7fb208043020 con 0x7fb2140b1500 2024-07-04T17:32:41.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:41.292+0000 7fb210c00640 1 --2- 172.21.15.42:0/641461097 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7fb1f403d5d0 0x7fb1f403fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:41.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:41.292+0000 7fb210c00640 1 -- 172.21.15.42:0/641461097 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 4690+0+0 (secure 0 0 0) 0x7fb2080728b0 con 0x7fb2140b1500 2024-07-04T17:32:41.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:41.292+0000 7fb213e00640 1 --2- 172.21.15.42:0/641461097 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7fb1f403d5d0 0x7fb1f403fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:41.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:41.293+0000 7fb213e00640 1 --2- 172.21.15.42:0/641461097 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7fb1f403d5d0 0x7fb1f403fae0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb200001500 tx=0x7fb200007a30 comp rx=0 tx=0).ready entity=mgr.5396 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:41.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:41.298+0000 7fb210c00640 1 -- 172.21.15.42:0/641461097 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fb20804a030 con 0x7fb2140b1500 2024-07-04T17:32:41.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:41.563+0000 7fb21bc00640 1 -- 172.21.15.42:0/641461097 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "mirroring"} v 0) -- 0x7fb2141481d0 con 0x7fb2140b1500 2024-07-04T17:32:42.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.441+0000 7fb210c00640 1 -- 172.21.15.42:0/641461097 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "mirroring"}]=0 v38) ==== 89+0+0 (secure 0 0 0) 0x7fb208041e50 con 0x7fb2140b1500 2024-07-04T17:32:42.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.445+0000 7fb21bc00640 1 -- 172.21.15.42:0/641461097 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7fb1f403d5d0 msgr2=0x7fb1f403fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:42.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.445+0000 7fb21bc00640 1 --2- 172.21.15.42:0/641461097 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7fb1f403d5d0 0x7fb1f403fae0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb200001500 tx=0x7fb200007a30 comp rx=0 tx=0).stop 2024-07-04T17:32:42.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.445+0000 7fb21bc00640 1 -- 172.21.15.42:0/641461097 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2140b1500 msgr2=0x7fb2140bfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:42.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.445+0000 7fb21bc00640 1 --2- 172.21.15.42:0/641461097 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2140b1500 0x7fb2140bfaa0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fb208006010 tx=0x7fb208009900 comp rx=0 tx=0).stop 2024-07-04T17:32:42.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.446+0000 7fb21bc00640 1 -- 172.21.15.42:0/641461097 shutdown_connections 2024-07-04T17:32:42.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.446+0000 7fb21bc00640 1 --2- 172.21.15.42:0/641461097 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7fb1f403d5d0 0x7fb1f403fae0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:42.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.446+0000 7fb21bc00640 1 --2- 172.21.15.42:0/641461097 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2140b1500 0x7fb2140bfaa0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:42.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.446+0000 7fb21bc00640 1 -- 172.21.15.42:0/641461097 >> 172.21.15.42:0/641461097 conn(0x7fb21406fe40 msgr2=0x7fb21406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:42.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.447+0000 7fb21bc00640 1 -- 172.21.15.42:0/641461097 shutdown_connections 2024-07-04T17:32:42.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.447+0000 7fb21bc00640 1 -- 172.21.15.42:0/641461097 wait complete. 2024-07-04T17:32:42.481 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-04T17:32:42.481 DEBUG:teuthology.orchestra.run.smithi042:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-04T17:32:42.583 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-04T17:32:42.583 INFO:tasks.cephfs.mount:Terminating background process 2024-07-04T17:32:42.583 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:32:42.583 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:32:42.633 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:32:42.633 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:32:42.674 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-04T17:32:42.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.991+0000 7f40f3c00640 1 -- 172.21.15.42:0/2418707162 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40ec0bc040 msgr2=0x7f40ec0bd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:42.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.991+0000 7f40f3c00640 1 --2- 172.21.15.42:0/2418707162 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40ec0bc040 0x7f40ec0bd2d0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f40dc0097b0 tx=0x7f40dc02a540 comp rx=0 tx=0).stop 2024-07-04T17:32:42.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.991+0000 7f40f3c00640 1 -- 172.21.15.42:0/2418707162 shutdown_connections 2024-07-04T17:32:42.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.991+0000 7f40f3c00640 1 --2- 172.21.15.42:0/2418707162 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40ec0bc040 0x7f40ec0bd2d0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:42.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.991+0000 7f40f3c00640 1 -- 172.21.15.42:0/2418707162 >> 172.21.15.42:0/2418707162 conn(0x7f40ec06c940 msgr2=0x7f40ec06ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:42.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.991+0000 7f40f3c00640 1 -- 172.21.15.42:0/2418707162 shutdown_connections 2024-07-04T17:32:42.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.991+0000 7f40f3c00640 1 -- 172.21.15.42:0/2418707162 wait complete. 2024-07-04T17:32:42.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.993+0000 7f40f3c00640 1 Processor -- start 2024-07-04T17:32:42.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.994+0000 7f40f3c00640 1 -- start start 2024-07-04T17:32:42.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.995+0000 7f40f3c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40ec0bc040 0x7f40ec0c0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:42.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.995+0000 7f40f3c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec0b7530 con 0x7f40ec0bc040 2024-07-04T17:32:42.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.995+0000 7f40f1200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40ec0bc040 0x7f40ec0c0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:42.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.995+0000 7f40f1200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40ec0bc040 0x7f40ec0c0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40626/0 (socket says 172.21.15.42:40626) 2024-07-04T17:32:42.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.995+0000 7f40f1200640 1 -- 172.21.15.42:0/4090553698 learned_addr learned my addr 172.21.15.42:0/4090553698 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:42.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.996+0000 7f40f1200640 1 -- 172.21.15.42:0/4090553698 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40ec0bf0c0 con 0x7f40ec0bc040 2024-07-04T17:32:42.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.996+0000 7f40f1200640 1 --2- 172.21.15.42:0/4090553698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40ec0bc040 0x7f40ec0c0580 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f40dc0098d0 tx=0x7f40dc033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:42.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.996+0000 7f40e8c00640 1 -- 172.21.15.42:0/4090553698 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40dc031bc0 con 0x7f40ec0bc040 2024-07-04T17:32:42.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.996+0000 7f40e8c00640 1 -- 172.21.15.42:0/4090553698 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40dc031e50 con 0x7f40ec0bc040 2024-07-04T17:32:42.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.996+0000 7f40e8c00640 1 -- 172.21.15.42:0/4090553698 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40dc033200 con 0x7f40ec0bc040 2024-07-04T17:32:42.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.996+0000 7f40f3c00640 1 -- 172.21.15.42:0/4090553698 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40ec0c0b50 con 0x7f40ec0bc040 2024-07-04T17:32:42.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.997+0000 7f40f3c00640 1 -- 172.21.15.42:0/4090553698 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40ec0be3c0 con 0x7f40ec0bc040 2024-07-04T17:32:42.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.998+0000 7f40f3c00640 1 -- 172.21.15.42:0/4090553698 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40ec0b7c00 con 0x7f40ec0bc040 2024-07-04T17:32:42.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.998+0000 7f40e8c00640 1 -- 172.21.15.42:0/4090553698 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 49668+0+0 (secure 0 0 0) 0x7f40dc0333a0 con 0x7f40ec0bc040 2024-07-04T17:32:42.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.998+0000 7f40e8c00640 1 --2- 172.21.15.42:0/4090553698 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f40c803d580 0x7f40c803fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:42.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.998+0000 7f40e8c00640 1 -- 172.21.15.42:0/4090553698 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 4690+0+0 (secure 0 0 0) 0x7f40dc073460 con 0x7f40ec0bc040 2024-07-04T17:32:43.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.999+0000 7f40ebe00640 1 -- 172.21.15.42:0/4090553698 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f40c803d580 msgr2=0x7f40c803fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1323178695 2024-07-04T17:32:43.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:42.999+0000 7f40ebe00640 1 --2- 172.21.15.42:0/4090553698 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f40c803d580 0x7f40c803fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:32:43.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.003+0000 7f40e8c00640 1 -- 172.21.15.42:0/4090553698 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f40dc03f030 con 0x7f40ec0bc040 2024-07-04T17:32:43.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.200+0000 7f40ebe00640 1 -- 172.21.15.42:0/4090553698 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f40c803d580 msgr2=0x7f40c803fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1323178695 2024-07-04T17:32:43.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.200+0000 7f40ebe00640 1 --2- 172.21.15.42:0/4090553698 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f40c803d580 0x7f40c803fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:32:43.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.224+0000 7f40f3c00640 1 -- 172.21.15.42:0/4090553698 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f40ec069b70 con 0x7f40ec0bc040 2024-07-04T17:32:43.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.225+0000 7f40e8c00640 1 -- 172.21.15.42:0/4090553698 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 9 entries v98) ==== 80+0+526 (secure 0 0 0) 0x7f40dc033690 con 0x7f40ec0bc040 2024-07-04T17:32:43.226 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2096108670 2024-07-04T18:32:31.377755+0000 2024-07-04T17:32:43.226 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1641580783 2024-07-05T17:32:15.307330+0000 2024-07-04T17:32:43.226 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1212024809 2024-07-05T17:32:15.307330+0000 2024-07-04T17:32:43.226 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1089649627 2024-07-05T17:32:15.307330+0000 2024-07-04T17:32:43.226 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/257615823 2024-07-05T17:32:15.307330+0000 2024-07-04T17:32:43.226 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2580772079 2024-07-05T17:32:15.307330+0000 2024-07-04T17:32:43.226 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/611598150 2024-07-05T17:32:15.307330+0000 2024-07-04T17:32:43.226 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6835/1139872311 2024-07-05T17:32:15.307330+0000 2024-07-04T17:32:43.227 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6834/1139872311 2024-07-05T17:32:15.307330+0000 2024-07-04T17:32:43.227 INFO:teuthology.orchestra.run.smithi042.stderr:listed 9 entries 2024-07-04T17:32:43.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.227+0000 7f40f3c00640 1 -- 172.21.15.42:0/4090553698 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f40c803d580 msgr2=0x7f40c803fa90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:32:43.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.227+0000 7f40f3c00640 1 --2- 172.21.15.42:0/4090553698 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f40c803d580 0x7f40c803fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:43.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.227+0000 7f40f3c00640 1 -- 172.21.15.42:0/4090553698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40ec0bc040 msgr2=0x7f40ec0c0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:43.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.227+0000 7f40f3c00640 1 --2- 172.21.15.42:0/4090553698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40ec0bc040 0x7f40ec0c0580 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f40dc0098d0 tx=0x7f40dc033e40 comp rx=0 tx=0).stop 2024-07-04T17:32:43.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.227+0000 7f40f3c00640 1 -- 172.21.15.42:0/4090553698 shutdown_connections 2024-07-04T17:32:43.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.227+0000 7f40f3c00640 1 --2- 172.21.15.42:0/4090553698 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f40c803d580 0x7f40c803fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:43.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.227+0000 7f40f3c00640 1 --2- 172.21.15.42:0/4090553698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40ec0bc040 0x7f40ec0c0580 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:43.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.227+0000 7f40f3c00640 1 -- 172.21.15.42:0/4090553698 >> 172.21.15.42:0/4090553698 conn(0x7f40ec06c940 msgr2=0x7f40ec143200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:43.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.227+0000 7f40f3c00640 1 -- 172.21.15.42:0/4090553698 shutdown_connections 2024-07-04T17:32:43.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.227+0000 7f40f3c00640 1 -- 172.21.15.42:0/4090553698 wait complete. 2024-07-04T17:32:43.255 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi042.front.sepia.ceph.com... 2024-07-04T17:32:43.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:32:43.256 DEBUG:teuthology.orchestra.run.smithi042:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-07-04T17:32:43.330 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[60830]: fuse finished with error 0 and tester_r 0 2024-07-04T17:32:43.335 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-07-04T17:32:43.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:32:43.335 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-04T17:32:43.499 INFO:tasks.cephfs.mount:Terminating background process 2024-07-04T17:32:43.499 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:32:43.499 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:32:43.594 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:32:43.594 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:32:43.644 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-04T17:32:43.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.909+0000 7fd18f600640 1 -- 172.21.15.42:0/658480207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1880b38d0 msgr2=0x7fd1880b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:43.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.909+0000 7fd18f600640 1 --2- 172.21.15.42:0/658480207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1880b38d0 0x7fd1880b2d20 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fd17c005fd0 tx=0x7fd17c02a5f0 comp rx=0 tx=0).stop 2024-07-04T17:32:43.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.909+0000 7fd18f600640 1 -- 172.21.15.42:0/658480207 shutdown_connections 2024-07-04T17:32:43.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.909+0000 7fd18f600640 1 --2- 172.21.15.42:0/658480207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1880b38d0 0x7fd1880b2d20 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:43.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.909+0000 7fd18f600640 1 -- 172.21.15.42:0/658480207 >> 172.21.15.42:0/658480207 conn(0x7fd1880bf700 msgr2=0x7fd1880af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:43.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.909+0000 7fd18f600640 1 -- 172.21.15.42:0/658480207 shutdown_connections 2024-07-04T17:32:43.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.909+0000 7fd18f600640 1 -- 172.21.15.42:0/658480207 wait complete. 2024-07-04T17:32:43.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.911+0000 7fd18f600640 1 Processor -- start 2024-07-04T17:32:43.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.911+0000 7fd18f600640 1 -- start start 2024-07-04T17:32:43.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.912+0000 7fd18f600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1880b01f0 0x7fd188141a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:43.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.912+0000 7fd18f600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd18806cf50 con 0x7fd1880b01f0 2024-07-04T17:32:43.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.913+0000 7fd18cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1880b01f0 0x7fd188141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:43.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.913+0000 7fd18cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1880b01f0 0x7fd188141a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37710/0 (socket says 172.21.15.42:37710) 2024-07-04T17:32:43.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.913+0000 7fd18cc00640 1 -- 172.21.15.42:0/1412246743 learned_addr learned my addr 172.21.15.42:0/1412246743 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:43.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.913+0000 7fd18cc00640 1 -- 172.21.15.42:0/1412246743 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1881447c0 con 0x7fd1880b01f0 2024-07-04T17:32:43.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.914+0000 7fd18cc00640 1 --2- 172.21.15.42:0/1412246743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1880b01f0 0x7fd188141a10 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fd17c009850 tx=0x7fd17c0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:43.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.914+0000 7fd184c00640 1 -- 172.21.15.42:0/1412246743 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd17c03e030 con 0x7fd1880b01f0 2024-07-04T17:32:43.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.914+0000 7fd184c00640 1 -- 172.21.15.42:0/1412246743 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd17c0035f0 con 0x7fd1880b01f0 2024-07-04T17:32:43.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.914+0000 7fd184c00640 1 -- 172.21.15.42:0/1412246743 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd17c035040 con 0x7fd1880b01f0 2024-07-04T17:32:43.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.914+0000 7fd18f600640 1 -- 172.21.15.42:0/1412246743 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1881d5280 con 0x7fd1880b01f0 2024-07-04T17:32:43.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.914+0000 7fd18f600640 1 -- 172.21.15.42:0/1412246743 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd188144490 con 0x7fd1880b01f0 2024-07-04T17:32:43.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.915+0000 7fd18f600640 1 -- 172.21.15.42:0/1412246743 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1880b2d20 con 0x7fd1880b01f0 2024-07-04T17:32:43.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.916+0000 7fd184c00640 1 -- 172.21.15.42:0/1412246743 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 49668+0+0 (secure 0 0 0) 0x7fd17c041030 con 0x7fd1880b01f0 2024-07-04T17:32:43.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.916+0000 7fd184c00640 1 --2- 172.21.15.42:0/1412246743 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7fd16803d580 0x7fd16803fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:43.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.916+0000 7fd184c00640 1 -- 172.21.15.42:0/1412246743 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 4690+0+0 (secure 0 0 0) 0x7fd17c03f070 con 0x7fd1880b01f0 2024-07-04T17:32:43.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.916+0000 7fd187e00640 1 -- 172.21.15.42:0/1412246743 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7fd16803d580 msgr2=0x7fd16803fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1323178695 2024-07-04T17:32:43.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.916+0000 7fd187e00640 1 --2- 172.21.15.42:0/1412246743 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7fd16803d580 0x7fd16803fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:32:43.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:43.922+0000 7fd184c00640 1 -- 172.21.15.42:0/1412246743 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fd17c04bd60 con 0x7fd1880b01f0 2024-07-04T17:32:44.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.117+0000 7fd187e00640 1 -- 172.21.15.42:0/1412246743 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7fd16803d580 msgr2=0x7fd16803fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1323178695 2024-07-04T17:32:44.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.117+0000 7fd187e00640 1 --2- 172.21.15.42:0/1412246743 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7fd16803d580 0x7fd16803fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:32:44.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.176+0000 7fd18f600640 1 -- 172.21.15.42:0/1412246743 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd1881d5450 con 0x7fd1880b01f0 2024-07-04T17:32:44.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.177+0000 7fd184c00640 1 -- 172.21.15.42:0/1412246743 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 9 entries v98) ==== 80+0+526 (secure 0 0 0) 0x7fd17c072030 con 0x7fd1880b01f0 2024-07-04T17:32:44.178 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2096108670 2024-07-04T18:32:31.377755+0000 2024-07-04T17:32:44.178 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1641580783 2024-07-05T17:32:15.307330+0000 2024-07-04T17:32:44.178 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1212024809 2024-07-05T17:32:15.307330+0000 2024-07-04T17:32:44.178 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1089649627 2024-07-05T17:32:15.307330+0000 2024-07-04T17:32:44.179 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/257615823 2024-07-05T17:32:15.307330+0000 2024-07-04T17:32:44.179 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2580772079 2024-07-05T17:32:15.307330+0000 2024-07-04T17:32:44.179 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/611598150 2024-07-05T17:32:15.307330+0000 2024-07-04T17:32:44.179 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6835/1139872311 2024-07-05T17:32:15.307330+0000 2024-07-04T17:32:44.179 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6834/1139872311 2024-07-05T17:32:15.307330+0000 2024-07-04T17:32:44.179 INFO:teuthology.orchestra.run.smithi042.stderr:listed 9 entries 2024-07-04T17:32:44.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.179+0000 7fd18f600640 1 -- 172.21.15.42:0/1412246743 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7fd16803d580 msgr2=0x7fd16803fa90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:32:44.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.179+0000 7fd18f600640 1 --2- 172.21.15.42:0/1412246743 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7fd16803d580 0x7fd16803fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:44.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.180+0000 7fd18f600640 1 -- 172.21.15.42:0/1412246743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1880b01f0 msgr2=0x7fd188141a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:44.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.180+0000 7fd18f600640 1 --2- 172.21.15.42:0/1412246743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1880b01f0 0x7fd188141a10 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fd17c009850 tx=0x7fd17c0049d0 comp rx=0 tx=0).stop 2024-07-04T17:32:44.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.180+0000 7fd18f600640 1 -- 172.21.15.42:0/1412246743 shutdown_connections 2024-07-04T17:32:44.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.180+0000 7fd18f600640 1 --2- 172.21.15.42:0/1412246743 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7fd16803d580 0x7fd16803fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:44.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.180+0000 7fd18f600640 1 --2- 172.21.15.42:0/1412246743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1880b01f0 0x7fd188141a10 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:44.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.180+0000 7fd18f600640 1 -- 172.21.15.42:0/1412246743 >> 172.21.15.42:0/1412246743 conn(0x7fd1880bf700 msgr2=0x7fd1880aaf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:44.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.180+0000 7fd18f600640 1 -- 172.21.15.42:0/1412246743 shutdown_connections 2024-07-04T17:32:44.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.180+0000 7fd18f600640 1 -- 172.21.15.42:0/1412246743 wait complete. 2024-07-04T17:32:44.209 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi042.front.sepia.ceph.com... 2024-07-04T17:32:44.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:32:44.209 DEBUG:teuthology.orchestra.run.smithi042:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-07-04T17:32:44.282 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi042.stderr:ceph-fuse[61299]: fuse finished with error 0 and tester_r 0 2024-07-04T17:32:44.286 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-07-04T17:32:44.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:32:44.286 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-07-04T17:32:44.369 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:32:44.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.687+0000 7f0aa6200640 1 -- 172.21.15.42:0/1150811894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0aa0141590 msgr2=0x7f0aa01419a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:44.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.687+0000 7f0aa6200640 1 --2- 172.21.15.42:0/1150811894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0aa0141590 0x7f0aa01419a0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f0a880097b0 tx=0x7f0a8802a540 comp rx=0 tx=0).stop 2024-07-04T17:32:44.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.688+0000 7f0aa6200640 1 -- 172.21.15.42:0/1150811894 shutdown_connections 2024-07-04T17:32:44.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.688+0000 7f0aa6200640 1 --2- 172.21.15.42:0/1150811894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0aa0141590 0x7f0aa01419a0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:44.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.688+0000 7f0aa6200640 1 -- 172.21.15.42:0/1150811894 >> 172.21.15.42:0/1150811894 conn(0x7f0aa006c940 msgr2=0x7f0aa006ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:44.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.688+0000 7f0aa6200640 1 -- 172.21.15.42:0/1150811894 shutdown_connections 2024-07-04T17:32:44.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.688+0000 7f0aa6200640 1 -- 172.21.15.42:0/1150811894 wait complete. 2024-07-04T17:32:44.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.690+0000 7f0aa6200640 1 Processor -- start 2024-07-04T17:32:44.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.690+0000 7f0aa6200640 1 -- start start 2024-07-04T17:32:44.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.691+0000 7f0aa6200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0aa00bc040 0x7f0aa00b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:44.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.691+0000 7f0aa6200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa00b7530 con 0x7f0aa00bc040 2024-07-04T17:32:44.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.692+0000 7f0a9f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0aa00bc040 0x7f0aa00b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:44.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.692+0000 7f0a9f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0aa00bc040 0x7f0aa00b2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37732/0 (socket says 172.21.15.42:37732) 2024-07-04T17:32:44.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.692+0000 7f0a9f400640 1 -- 172.21.15.42:0/897894037 learned_addr learned my addr 172.21.15.42:0/897894037 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:44.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.692+0000 7f0a9f400640 1 -- 172.21.15.42:0/897894037 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa00c4210 con 0x7f0aa00bc040 2024-07-04T17:32:44.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.692+0000 7f0a9f400640 1 --2- 172.21.15.42:0/897894037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0aa00bc040 0x7f0aa00b2ce0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f0a9000c320 tx=0x7f0a9000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:44.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.693+0000 7f0a97400640 1 -- 172.21.15.42:0/897894037 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a900116c0 con 0x7f0aa00bc040 2024-07-04T17:32:44.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.693+0000 7f0a97400640 1 -- 172.21.15.42:0/897894037 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a90017070 con 0x7f0aa00bc040 2024-07-04T17:32:44.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.693+0000 7f0a97400640 1 -- 172.21.15.42:0/897894037 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a90011b40 con 0x7f0aa00bc040 2024-07-04T17:32:44.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.693+0000 7f0aa6200640 1 -- 172.21.15.42:0/897894037 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aa01e45a0 con 0x7f0aa00bc040 2024-07-04T17:32:44.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.693+0000 7f0aa6200640 1 -- 172.21.15.42:0/897894037 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aa00c3c40 con 0x7f0aa00bc040 2024-07-04T17:32:44.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.694+0000 7f0aa6200640 1 -- 172.21.15.42:0/897894037 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aa00b7bb0 con 0x7f0aa00bc040 2024-07-04T17:32:44.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.694+0000 7f0a97400640 1 -- 172.21.15.42:0/897894037 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 49668+0+0 (secure 0 0 0) 0x7f0a90011ce0 con 0x7f0aa00bc040 2024-07-04T17:32:44.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.695+0000 7f0a97400640 1 --2- 172.21.15.42:0/897894037 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f0a7c03d580 0x7f0a7c03fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:44.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.695+0000 7f0a97400640 1 -- 172.21.15.42:0/897894037 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 4690+0+0 (secure 0 0 0) 0x7f0a90051b10 con 0x7f0aa00bc040 2024-07-04T17:32:44.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.695+0000 7f0a97e00640 1 -- 172.21.15.42:0/897894037 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f0a7c03d580 msgr2=0x7f0a7c03fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1323178695 2024-07-04T17:32:44.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.695+0000 7f0a97e00640 1 --2- 172.21.15.42:0/897894037 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f0a7c03d580 0x7f0a7c03fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:32:44.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.700+0000 7f0a97400640 1 -- 172.21.15.42:0/897894037 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f0a9001ac80 con 0x7f0aa00bc040 2024-07-04T17:32:44.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.897+0000 7f0a97e00640 1 -- 172.21.15.42:0/897894037 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f0a7c03d580 msgr2=0x7f0a7c03fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1323178695 2024-07-04T17:32:44.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.897+0000 7f0a97e00640 1 --2- 172.21.15.42:0/897894037 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f0a7c03d580 0x7f0a7c03fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:32:44.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.955+0000 7f0aa6200640 1 -- 172.21.15.42:0/897894037 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0aa00c43e0 con 0x7f0aa00bc040 2024-07-04T17:32:44.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.956+0000 7f0a97400640 1 -- 172.21.15.42:0/897894037 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+7357 (secure 0 0 0) 0x7f0a900102e0 con 0x7f0aa00bc040 2024-07-04T17:32:44.957 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:32:44.958 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":79,"btime":"2024-07-04T17:32:30:398730+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/244742173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":244742173},{"type":"v1","addr":"172.21.15.42:6825","nonce":244742173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5265,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/973681058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":973681058},{"type":"v1","addr":"172.21.15.42:6827","nonce":973681058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/480649366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":480649366},{"type":"v1","addr":"172.21.15.42:6829","nonce":480649366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:31:41.664414+0000","modified":"2024-07-04T17:32:29.545650+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5277":{"gid":5277,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6833/3777807763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":3777807763},{"type":"v1","addr":"172.21.15.42:6833","nonce":3777807763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5277,"qdb_cluster":[5277]},"id":8},{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:32:02.103946+0000","modified":"2024-07-04T17:32:05.184521+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5276":{"gid":5276,"name":"d","rank":0,"incarnation":73,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6831/327636003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":327636003},{"type":"v1","addr":"172.21.15.42:6831","nonce":327636003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5276,"qdb_cluster":[5276]},"id":9}]} 2024-07-04T17:32:44.958 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 79 2024-07-04T17:32:44.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.959+0000 7f0aa6200640 1 -- 172.21.15.42:0/897894037 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f0a7c03d580 msgr2=0x7f0a7c03fa90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:32:44.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.959+0000 7f0aa6200640 1 --2- 172.21.15.42:0/897894037 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f0a7c03d580 0x7f0a7c03fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:44.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.959+0000 7f0aa6200640 1 -- 172.21.15.42:0/897894037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0aa00bc040 msgr2=0x7f0aa00b2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:44.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.959+0000 7f0aa6200640 1 --2- 172.21.15.42:0/897894037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0aa00bc040 0x7f0aa00b2ce0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f0a9000c320 tx=0x7f0a9000c7e0 comp rx=0 tx=0).stop 2024-07-04T17:32:44.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.959+0000 7f0aa6200640 1 -- 172.21.15.42:0/897894037 shutdown_connections 2024-07-04T17:32:44.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.959+0000 7f0aa6200640 1 --2- 172.21.15.42:0/897894037 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f0a7c03d580 0x7f0a7c03fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:44.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.959+0000 7f0aa6200640 1 --2- 172.21.15.42:0/897894037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0aa00bc040 0x7f0aa00b2ce0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:44.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.959+0000 7f0aa6200640 1 -- 172.21.15.42:0/897894037 >> 172.21.15.42:0/897894037 conn(0x7f0aa006c940 msgr2=0x7f0aa0145cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:44.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.959+0000 7f0aa6200640 1 -- 172.21.15.42:0/897894037 shutdown_connections 2024-07-04T17:32:44.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:44.959+0000 7f0aa6200640 1 -- 172.21.15.42:0/897894037 wait complete. 2024-07-04T17:32:44.987 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:32:45.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.259+0000 7f8d49800640 1 -- 172.21.15.42:0/865016281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d44068660 msgr2=0x7f8d440b1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:45.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.259+0000 7f8d49800640 1 --2- 172.21.15.42:0/865016281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d44068660 0x7f8d440b1c90 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f8d340097b0 tx=0x7f8d3402a540 comp rx=0 tx=0).stop 2024-07-04T17:32:45.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.260+0000 7f8d49800640 1 -- 172.21.15.42:0/865016281 shutdown_connections 2024-07-04T17:32:45.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.260+0000 7f8d49800640 1 --2- 172.21.15.42:0/865016281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d44068660 0x7f8d440b1c90 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:45.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.260+0000 7f8d49800640 1 -- 172.21.15.42:0/865016281 >> 172.21.15.42:0/865016281 conn(0x7f8d440bfe40 msgr2=0x7f8d440b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:45.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.260+0000 7f8d49800640 1 -- 172.21.15.42:0/865016281 shutdown_connections 2024-07-04T17:32:45.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.260+0000 7f8d49800640 1 -- 172.21.15.42:0/865016281 wait complete. 2024-07-04T17:32:45.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.262+0000 7f8d49800640 1 Processor -- start 2024-07-04T17:32:45.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.262+0000 7f8d49800640 1 -- start start 2024-07-04T17:32:45.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.263+0000 7f8d49800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d44068660 0x7f8d441d7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:45.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.263+0000 7f8d49800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8d440b1320 con 0x7f8d44068660 2024-07-04T17:32:45.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.263+0000 7f8d42a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d44068660 0x7f8d441d7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:45.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.263+0000 7f8d42a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d44068660 0x7f8d441d7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37746/0 (socket says 172.21.15.42:37746) 2024-07-04T17:32:45.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.263+0000 7f8d42a00640 1 -- 172.21.15.42:0/328352787 learned_addr learned my addr 172.21.15.42:0/328352787 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:45.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.264+0000 7f8d42a00640 1 -- 172.21.15.42:0/328352787 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d440b68b0 con 0x7f8d44068660 2024-07-04T17:32:45.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.264+0000 7f8d42a00640 1 --2- 172.21.15.42:0/328352787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d44068660 0x7f8d441d7140 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f8d34009780 tx=0x7f8d34033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:45.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.265+0000 7f8d3aa00640 1 -- 172.21.15.42:0/328352787 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d34031bc0 con 0x7f8d44068660 2024-07-04T17:32:45.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.265+0000 7f8d3aa00640 1 -- 172.21.15.42:0/328352787 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d34031e50 con 0x7f8d44068660 2024-07-04T17:32:45.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.265+0000 7f8d3aa00640 1 -- 172.21.15.42:0/328352787 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d34033200 con 0x7f8d44068660 2024-07-04T17:32:45.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.265+0000 7f8d49800640 1 -- 172.21.15.42:0/328352787 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d440b82d0 con 0x7f8d44068660 2024-07-04T17:32:45.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.265+0000 7f8d49800640 1 -- 172.21.15.42:0/328352787 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d440b6530 con 0x7f8d44068660 2024-07-04T17:32:45.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.266+0000 7f8d49800640 1 -- 172.21.15.42:0/328352787 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d440b19d0 con 0x7f8d44068660 2024-07-04T17:32:45.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.266+0000 7f8d3aa00640 1 -- 172.21.15.42:0/328352787 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 49668+0+0 (secure 0 0 0) 0x7f8d340333a0 con 0x7f8d44068660 2024-07-04T17:32:45.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.266+0000 7f8d3aa00640 1 --2- 172.21.15.42:0/328352787 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f8d2003d580 0x7f8d2003fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:45.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.267+0000 7f8d3aa00640 1 -- 172.21.15.42:0/328352787 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 4690+0+0 (secure 0 0 0) 0x7f8d34073550 con 0x7f8d44068660 2024-07-04T17:32:45.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.267+0000 7f8d3be00640 1 -- 172.21.15.42:0/328352787 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f8d2003d580 msgr2=0x7f8d2003fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1323178695 2024-07-04T17:32:45.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.267+0000 7f8d3be00640 1 --2- 172.21.15.42:0/328352787 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f8d2003d580 0x7f8d2003fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:32:45.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.271+0000 7f8d3aa00640 1 -- 172.21.15.42:0/328352787 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f8d3403f030 con 0x7f8d44068660 2024-07-04T17:32:45.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.468+0000 7f8d3be00640 1 -- 172.21.15.42:0/328352787 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f8d2003d580 msgr2=0x7f8d2003fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1323178695 2024-07-04T17:32:45.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.468+0000 7f8d3be00640 1 --2- 172.21.15.42:0/328352787 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f8d2003d580 0x7f8d2003fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:32:45.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.520+0000 7f8d49800640 1 -- 172.21.15.42:0/328352787 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d440b6a80 con 0x7f8d44068660 2024-07-04T17:32:45.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.521+0000 7f8d3aa00640 1 -- 172.21.15.42:0/328352787 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+7357 (secure 0 0 0) 0x7f8d34031660 con 0x7f8d44068660 2024-07-04T17:32:45.522 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:32:45.522 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":79,"btime":"2024-07-04T17:32:30:398730+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/244742173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":244742173},{"type":"v1","addr":"172.21.15.42:6825","nonce":244742173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5265,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/973681058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":973681058},{"type":"v1","addr":"172.21.15.42:6827","nonce":973681058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/480649366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":480649366},{"type":"v1","addr":"172.21.15.42:6829","nonce":480649366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:31:41.664414+0000","modified":"2024-07-04T17:32:29.545650+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5277":{"gid":5277,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6833/3777807763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":3777807763},{"type":"v1","addr":"172.21.15.42:6833","nonce":3777807763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5277,"qdb_cluster":[5277]},"id":8},{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:32:02.103946+0000","modified":"2024-07-04T17:32:05.184521+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5276":{"gid":5276,"name":"d","rank":0,"incarnation":73,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6831/327636003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":327636003},{"type":"v1","addr":"172.21.15.42:6831","nonce":327636003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5276,"qdb_cluster":[5276]},"id":9}]} 2024-07-04T17:32:45.523 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 79 2024-07-04T17:32:45.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.524+0000 7f8d49800640 1 -- 172.21.15.42:0/328352787 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f8d2003d580 msgr2=0x7f8d2003fa90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:32:45.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.524+0000 7f8d49800640 1 --2- 172.21.15.42:0/328352787 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f8d2003d580 0x7f8d2003fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:45.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.524+0000 7f8d49800640 1 -- 172.21.15.42:0/328352787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d44068660 msgr2=0x7f8d441d7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:45.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.524+0000 7f8d49800640 1 --2- 172.21.15.42:0/328352787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d44068660 0x7f8d441d7140 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f8d34009780 tx=0x7f8d34033e40 comp rx=0 tx=0).stop 2024-07-04T17:32:45.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.524+0000 7f8d49800640 1 -- 172.21.15.42:0/328352787 shutdown_connections 2024-07-04T17:32:45.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.524+0000 7f8d49800640 1 --2- 172.21.15.42:0/328352787 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f8d2003d580 0x7f8d2003fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:45.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.524+0000 7f8d49800640 1 --2- 172.21.15.42:0/328352787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d44068660 0x7f8d441d7140 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:45.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.524+0000 7f8d49800640 1 -- 172.21.15.42:0/328352787 >> 172.21.15.42:0/328352787 conn(0x7f8d440bfe40 msgr2=0x7f8d4406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:45.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.524+0000 7f8d49800640 1 -- 172.21.15.42:0/328352787 shutdown_connections 2024-07-04T17:32:45.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.524+0000 7f8d49800640 1 -- 172.21.15.42:0/328352787 wait complete. 2024-07-04T17:32:45.554 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:32:45.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.826+0000 7f92d3400640 1 -- 172.21.15.42:0/3127964691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92cc140e70 msgr2=0x7f92cc0bb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:45.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.826+0000 7f92d3400640 1 --2- 172.21.15.42:0/3127964691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92cc140e70 0x7f92cc0bb1f0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f92c0005fd0 tx=0x7f92c002a5f0 comp rx=0 tx=0).stop 2024-07-04T17:32:45.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.826+0000 7f92d3400640 1 -- 172.21.15.42:0/3127964691 shutdown_connections 2024-07-04T17:32:45.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.826+0000 7f92d3400640 1 --2- 172.21.15.42:0/3127964691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92cc140e70 0x7f92cc0bb1f0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:45.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.826+0000 7f92d3400640 1 -- 172.21.15.42:0/3127964691 >> 172.21.15.42:0/3127964691 conn(0x7f92cc0bf700 msgr2=0x7f92cc0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:45.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.826+0000 7f92d3400640 1 -- 172.21.15.42:0/3127964691 shutdown_connections 2024-07-04T17:32:45.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.826+0000 7f92d3400640 1 -- 172.21.15.42:0/3127964691 wait complete. 2024-07-04T17:32:45.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.828+0000 7f92d3400640 1 Processor -- start 2024-07-04T17:32:45.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.828+0000 7f92d3400640 1 -- start start 2024-07-04T17:32:45.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.829+0000 7f92d3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92cc0a9fa0 0x7f92cc1d6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:45.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.829+0000 7f92d3400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f92cc0aae30 con 0x7f92cc0a9fa0 2024-07-04T17:32:45.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.830+0000 7f92d0a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92cc0a9fa0 0x7f92cc1d6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:45.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.830+0000 7f92d0a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92cc0a9fa0 0x7f92cc1d6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37774/0 (socket says 172.21.15.42:37774) 2024-07-04T17:32:45.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.830+0000 7f92d0a00640 1 -- 172.21.15.42:0/1913250280 learned_addr learned my addr 172.21.15.42:0/1913250280 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:45.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.830+0000 7f92d0a00640 1 -- 172.21.15.42:0/1913250280 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92cc140120 con 0x7f92cc0a9fa0 2024-07-04T17:32:45.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.831+0000 7f92d0a00640 1 --2- 172.21.15.42:0/1913250280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92cc0a9fa0 0x7f92cc1d6590 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f92c0009850 tx=0x7f92c00049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:45.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.831+0000 7f92c8c00640 1 -- 172.21.15.42:0/1913250280 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92c003e030 con 0x7f92cc0a9fa0 2024-07-04T17:32:45.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.831+0000 7f92c8c00640 1 -- 172.21.15.42:0/1913250280 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f92c00035f0 con 0x7f92cc0a9fa0 2024-07-04T17:32:45.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.831+0000 7f92c8c00640 1 -- 172.21.15.42:0/1913250280 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92c0035040 con 0x7f92cc0a9fa0 2024-07-04T17:32:45.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.831+0000 7f92d3400640 1 -- 172.21.15.42:0/1913250280 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92cc1403c0 con 0x7f92cc0a9fa0 2024-07-04T17:32:45.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.831+0000 7f92d3400640 1 -- 172.21.15.42:0/1913250280 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92cc1408e0 con 0x7f92cc0a9fa0 2024-07-04T17:32:45.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.832+0000 7f92d3400640 1 -- 172.21.15.42:0/1913250280 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92cc0bb1f0 con 0x7f92cc0a9fa0 2024-07-04T17:32:45.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.833+0000 7f92c8c00640 1 -- 172.21.15.42:0/1913250280 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 49668+0+0 (secure 0 0 0) 0x7f92c0041030 con 0x7f92cc0a9fa0 2024-07-04T17:32:45.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.833+0000 7f92c8c00640 1 --2- 172.21.15.42:0/1913250280 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f92b003d580 0x7f92b003fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:45.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.833+0000 7f92c8c00640 1 -- 172.21.15.42:0/1913250280 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 4690+0+0 (secure 0 0 0) 0x7f92c003f070 con 0x7f92cc0a9fa0 2024-07-04T17:32:45.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.834+0000 7f92cbe00640 1 -- 172.21.15.42:0/1913250280 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f92b003d580 msgr2=0x7f92b003fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1323178695 2024-07-04T17:32:45.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.834+0000 7f92cbe00640 1 --2- 172.21.15.42:0/1913250280 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f92b003d580 0x7f92b003fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:32:45.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:45.838+0000 7f92c8c00640 1 -- 172.21.15.42:0/1913250280 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f92c004bd60 con 0x7f92cc0a9fa0 2024-07-04T17:32:46.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.035+0000 7f92cbe00640 1 -- 172.21.15.42:0/1913250280 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f92b003d580 msgr2=0x7f92b003fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1323178695 2024-07-04T17:32:46.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.035+0000 7f92cbe00640 1 --2- 172.21.15.42:0/1913250280 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f92b003d580 0x7f92b003fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:32:46.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.047+0000 7f92d3400640 1 -- 172.21.15.42:0/1913250280 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f92cc1d4230 con 0x7f92cc0a9fa0 2024-07-04T17:32:46.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.051+0000 7f92c8c00640 1 -- 172.21.15.42:0/1913250280 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+15015 (secure 0 0 0) 0x7f92c0030cc0 con 0x7f92cc0a9fa0 2024-07-04T17:32:46.052 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:32:46.052 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":98,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:32:31.406901+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:31:39.727656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-07-04T17:31:40.615610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:31:59.635881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.4099999666213989,"score_stable":1.4099999666213989,"optimal_score":1,"raw_score_acting":1.4099999666213989,"raw_score_stable":1.4099999666213989,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:32:01.086110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.0800000429153442,"score_stable":1.0800000429153442,"optimal_score":1,"raw_score_acting":1.0800000429153442,"raw_score_stable":1.0800000429153442,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/2096108670":"2024-07-04T18:32:31.377755+0000","172.21.15.42:0/1641580783":"2024-07-05T17:32:15.307330+0000","172.21.15.42:0/1212024809":"2024-07-05T17:32:15.307330+0000","172.21.15.42:0/1089649627":"2024-07-05T17:32:15.307330+0000","172.21.15.42:0/257615823":"2024-07-05T17:32:15.307330+0000","172.21.15.42:0/2580772079":"2024-07-05T17:32:15.307330+0000","172.21.15.42:0/611598150":"2024-07-05T17:32:15.307330+0000","172.21.15.42:6835/1139872311":"2024-07-05T17:32:15.307330+0000","172.21.15.42:6834/1139872311":"2024-07-05T17:32:15.307330+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:32:46.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.053+0000 7f92d3400640 1 -- 172.21.15.42:0/1913250280 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f92b003d580 msgr2=0x7f92b003fa90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:32:46.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.053+0000 7f92d3400640 1 --2- 172.21.15.42:0/1913250280 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f92b003d580 0x7f92b003fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:46.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.054+0000 7f92d3400640 1 -- 172.21.15.42:0/1913250280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92cc0a9fa0 msgr2=0x7f92cc1d6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:46.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.054+0000 7f92d3400640 1 --2- 172.21.15.42:0/1913250280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92cc0a9fa0 0x7f92cc1d6590 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f92c0009850 tx=0x7f92c00049d0 comp rx=0 tx=0).stop 2024-07-04T17:32:46.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.054+0000 7f92d3400640 1 -- 172.21.15.42:0/1913250280 shutdown_connections 2024-07-04T17:32:46.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.054+0000 7f92d3400640 1 --2- 172.21.15.42:0/1913250280 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f92b003d580 0x7f92b003fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:46.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.054+0000 7f92d3400640 1 --2- 172.21.15.42:0/1913250280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92cc0a9fa0 0x7f92cc1d6590 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:46.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.054+0000 7f92d3400640 1 -- 172.21.15.42:0/1913250280 >> 172.21.15.42:0/1913250280 conn(0x7f92cc0bf700 msgr2=0x7f92cc0b0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:46.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.054+0000 7f92d3400640 1 -- 172.21.15.42:0/1913250280 shutdown_connections 2024-07-04T17:32:46.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.054+0000 7f92d3400640 1 -- 172.21.15.42:0/1913250280 wait complete. 2024-07-04T17:32:46.081 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-04T17:32:46.081 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:32:46.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.358+0000 7ff978a00640 1 -- 172.21.15.42:0/3700284186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9740b5ae0 msgr2=0x7ff9740c38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:46.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.358+0000 7ff978a00640 1 --2- 172.21.15.42:0/3700284186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9740b5ae0 0x7ff9740c38b0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7ff96c005fd0 tx=0x7ff96c02a8b0 comp rx=0 tx=0).stop 2024-07-04T17:32:46.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.358+0000 7ff978a00640 1 -- 172.21.15.42:0/3700284186 shutdown_connections 2024-07-04T17:32:46.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.358+0000 7ff978a00640 1 --2- 172.21.15.42:0/3700284186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9740b5ae0 0x7ff9740c38b0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:46.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.358+0000 7ff978a00640 1 -- 172.21.15.42:0/3700284186 >> 172.21.15.42:0/3700284186 conn(0x7ff97406dd10 msgr2=0x7ff97406e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:46.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.358+0000 7ff978a00640 1 -- 172.21.15.42:0/3700284186 shutdown_connections 2024-07-04T17:32:46.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.358+0000 7ff978a00640 1 -- 172.21.15.42:0/3700284186 wait complete. 2024-07-04T17:32:46.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.360+0000 7ff978a00640 1 Processor -- start 2024-07-04T17:32:46.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.360+0000 7ff978a00640 1 -- start start 2024-07-04T17:32:46.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.362+0000 7ff978a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9740b5ae0 0x7ff9740ba990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:46.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.362+0000 7ff978a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff9740b76d0 con 0x7ff9740b5ae0 2024-07-04T17:32:46.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.362+0000 7ff972800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9740b5ae0 0x7ff9740ba990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:46.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.362+0000 7ff972800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9740b5ae0 0x7ff9740ba990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37790/0 (socket says 172.21.15.42:37790) 2024-07-04T17:32:46.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.362+0000 7ff972800640 1 -- 172.21.15.42:0/42226062 learned_addr learned my addr 172.21.15.42:0/42226062 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:46.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.362+0000 7ff972800640 1 -- 172.21.15.42:0/42226062 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9740bbb50 con 0x7ff9740b5ae0 2024-07-04T17:32:46.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.363+0000 7ff972800640 1 --2- 172.21.15.42:0/42226062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9740b5ae0 0x7ff9740ba990 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7ff96c006010 tx=0x7ff96c033b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:46.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.363+0000 7ff96aa00640 1 -- 172.21.15.42:0/42226062 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff96c0319f0 con 0x7ff9740b5ae0 2024-07-04T17:32:46.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.363+0000 7ff96aa00640 1 -- 172.21.15.42:0/42226062 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff96c031b90 con 0x7ff9740b5ae0 2024-07-04T17:32:46.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.363+0000 7ff96aa00640 1 -- 172.21.15.42:0/42226062 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff96c033070 con 0x7ff9740b5ae0 2024-07-04T17:32:46.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.363+0000 7ff978a00640 1 -- 172.21.15.42:0/42226062 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff97413b750 con 0x7ff9740b5ae0 2024-07-04T17:32:46.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.363+0000 7ff978a00640 1 -- 172.21.15.42:0/42226062 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff97413db20 con 0x7ff9740b5ae0 2024-07-04T17:32:46.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.364+0000 7ff96aa00640 1 -- 172.21.15.42:0/42226062 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 49668+0+0 (secure 0 0 0) 0x7ff96c033210 con 0x7ff9740b5ae0 2024-07-04T17:32:46.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.364+0000 7ff978a00640 1 -- 172.21.15.42:0/42226062 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9740b7d30 con 0x7ff9740b5ae0 2024-07-04T17:32:46.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.365+0000 7ff96aa00640 1 --2- 172.21.15.42:0/42226062 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7ff95003d580 0x7ff95003fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:46.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.365+0000 7ff96aa00640 1 -- 172.21.15.42:0/42226062 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 4690+0+0 (secure 0 0 0) 0x7ff96c073680 con 0x7ff9740b5ae0 2024-07-04T17:32:46.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.365+0000 7ff971e00640 1 -- 172.21.15.42:0/42226062 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7ff95003d580 msgr2=0x7ff95003fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1323178695 2024-07-04T17:32:46.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.365+0000 7ff971e00640 1 --2- 172.21.15.42:0/42226062 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7ff95003d580 0x7ff95003fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:32:46.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.370+0000 7ff96aa00640 1 -- 172.21.15.42:0/42226062 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7ff96c03f030 con 0x7ff9740b5ae0 2024-07-04T17:32:46.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.566+0000 7ff971e00640 1 -- 172.21.15.42:0/42226062 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7ff95003d580 msgr2=0x7ff95003fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1323178695 2024-07-04T17:32:46.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.567+0000 7ff971e00640 1 --2- 172.21.15.42:0/42226062 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7ff95003d580 0x7ff95003fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:32:46.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.620+0000 7ff978a00640 1 -- 172.21.15.42:0/42226062 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff974068dc0 con 0x7ff9740b5ae0 2024-07-04T17:32:46.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.622+0000 7ff96aa00640 1 -- 172.21.15.42:0/42226062 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+7357 (secure 0 0 0) 0x7ff96c071e20 con 0x7ff9740b5ae0 2024-07-04T17:32:46.623 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:32:46.623 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":79,"btime":"2024-07-04T17:32:30:398730+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/244742173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":244742173},{"type":"v1","addr":"172.21.15.42:6825","nonce":244742173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5265,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/973681058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":973681058},{"type":"v1","addr":"172.21.15.42:6827","nonce":973681058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/480649366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":480649366},{"type":"v1","addr":"172.21.15.42:6829","nonce":480649366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:31:41.664414+0000","modified":"2024-07-04T17:32:29.545650+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5277":{"gid":5277,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6833/3777807763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":3777807763},{"type":"v1","addr":"172.21.15.42:6833","nonce":3777807763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5277,"qdb_cluster":[5277]},"id":8},{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:32:02.103946+0000","modified":"2024-07-04T17:32:05.184521+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5276":{"gid":5276,"name":"d","rank":0,"incarnation":73,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6831/327636003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":327636003},{"type":"v1","addr":"172.21.15.42:6831","nonce":327636003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5276,"qdb_cluster":[5276]},"id":9}]} 2024-07-04T17:32:46.623 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 79 2024-07-04T17:32:46.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.624+0000 7ff978a00640 1 -- 172.21.15.42:0/42226062 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7ff95003d580 msgr2=0x7ff95003fa90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:32:46.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.624+0000 7ff978a00640 1 --2- 172.21.15.42:0/42226062 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7ff95003d580 0x7ff95003fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:46.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.624+0000 7ff978a00640 1 -- 172.21.15.42:0/42226062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9740b5ae0 msgr2=0x7ff9740ba990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:46.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.624+0000 7ff978a00640 1 --2- 172.21.15.42:0/42226062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9740b5ae0 0x7ff9740ba990 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7ff96c006010 tx=0x7ff96c033b10 comp rx=0 tx=0).stop 2024-07-04T17:32:46.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.625+0000 7ff978a00640 1 -- 172.21.15.42:0/42226062 shutdown_connections 2024-07-04T17:32:46.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.625+0000 7ff978a00640 1 --2- 172.21.15.42:0/42226062 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7ff95003d580 0x7ff95003fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:46.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.625+0000 7ff978a00640 1 --2- 172.21.15.42:0/42226062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9740b5ae0 0x7ff9740ba990 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:46.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.625+0000 7ff978a00640 1 -- 172.21.15.42:0/42226062 >> 172.21.15.42:0/42226062 conn(0x7ff97406dd10 msgr2=0x7ff9740c1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:46.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.625+0000 7ff978a00640 1 -- 172.21.15.42:0/42226062 shutdown_connections 2024-07-04T17:32:46.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.625+0000 7ff978a00640 1 -- 172.21.15.42:0/42226062 wait complete. 2024-07-04T17:32:46.662 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:32:46.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.935+0000 7f6b1fe00640 1 -- 172.21.15.42:0/384044032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b200bda60 msgr2=0x7f6b20073820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:46.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.935+0000 7f6b1fe00640 1 --2- 172.21.15.42:0/384044032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b200bda60 0x7f6b20073820 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f6b10005fd0 tx=0x7f6b1002aac0 comp rx=0 tx=0).stop 2024-07-04T17:32:46.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.935+0000 7f6b1fe00640 1 -- 172.21.15.42:0/384044032 shutdown_connections 2024-07-04T17:32:46.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.935+0000 7f6b1fe00640 1 --2- 172.21.15.42:0/384044032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b200bda60 0x7f6b20073820 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:46.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.935+0000 7f6b1fe00640 1 -- 172.21.15.42:0/384044032 >> 172.21.15.42:0/384044032 conn(0x7f6b20071a70 msgr2=0x7f6b200c0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:46.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.935+0000 7f6b1fe00640 1 -- 172.21.15.42:0/384044032 shutdown_connections 2024-07-04T17:32:46.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.935+0000 7f6b1fe00640 1 -- 172.21.15.42:0/384044032 wait complete. 2024-07-04T17:32:46.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.937+0000 7f6b1fe00640 1 Processor -- start 2024-07-04T17:32:46.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.938+0000 7f6b1fe00640 1 -- start start 2024-07-04T17:32:46.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.939+0000 7f6b1fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b20072970 0x7f6b200c0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:46.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.939+0000 7f6b1fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2014ead0 con 0x7f6b20072970 2024-07-04T17:32:46.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.939+0000 7f6b1ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b20072970 0x7f6b200c0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:46.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.939+0000 7f6b1ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b20072970 0x7f6b200c0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37810/0 (socket says 172.21.15.42:37810) 2024-07-04T17:32:46.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.939+0000 7f6b1ea00640 1 -- 172.21.15.42:0/1628679612 learned_addr learned my addr 172.21.15.42:0/1628679612 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:46.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.940+0000 7f6b1ea00640 1 -- 172.21.15.42:0/1628679612 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b2006e820 con 0x7f6b20072970 2024-07-04T17:32:46.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.940+0000 7f6b1ea00640 1 --2- 172.21.15.42:0/1628679612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b20072970 0x7f6b200c0590 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f6b10006010 tx=0x7f6b10003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:46.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.940+0000 7f6b16a00640 1 -- 172.21.15.42:0/1628679612 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b10044070 con 0x7f6b20072970 2024-07-04T17:32:46.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.940+0000 7f6b16a00640 1 -- 172.21.15.42:0/1628679612 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b10004250 con 0x7f6b20072970 2024-07-04T17:32:46.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.940+0000 7f6b16a00640 1 -- 172.21.15.42:0/1628679612 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b10004590 con 0x7f6b20072970 2024-07-04T17:32:46.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.940+0000 7f6b1fe00640 1 -- 172.21.15.42:0/1628679612 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b2006ed00 con 0x7f6b20072970 2024-07-04T17:32:46.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.941+0000 7f6b1fe00640 1 -- 172.21.15.42:0/1628679612 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b2006e310 con 0x7f6b20072970 2024-07-04T17:32:46.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.942+0000 7f6b1fe00640 1 -- 172.21.15.42:0/1628679612 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b2014beb0 con 0x7f6b20072970 2024-07-04T17:32:46.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.942+0000 7f6b16a00640 1 -- 172.21.15.42:0/1628679612 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) ==== 49668+0+0 (secure 0 0 0) 0x7f6b10047070 con 0x7f6b20072970 2024-07-04T17:32:46.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.943+0000 7f6b16a00640 1 --2- 172.21.15.42:0/1628679612 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f6b0003d580 0x7f6b0003fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:46.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.943+0000 7f6b16a00640 1 -- 172.21.15.42:0/1628679612 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 4690+0+0 (secure 0 0 0) 0x7f6b1003c070 con 0x7f6b20072970 2024-07-04T17:32:46.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.943+0000 7f6b1e000640 1 -- 172.21.15.42:0/1628679612 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f6b0003d580 msgr2=0x7f6b0003fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1323178695 2024-07-04T17:32:46.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.943+0000 7f6b1e000640 1 --2- 172.21.15.42:0/1628679612 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f6b0003d580 0x7f6b0003fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:32:46.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:46.948+0000 7f6b16a00640 1 -- 172.21.15.42:0/1628679612 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f6b2014beb0 con 0x7f6b20072970 2024-07-04T17:32:47.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.144+0000 7f6b1e000640 1 -- 172.21.15.42:0/1628679612 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f6b0003d580 msgr2=0x7f6b0003fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1323178695 2024-07-04T17:32:47.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.145+0000 7f6b1e000640 1 --2- 172.21.15.42:0/1628679612 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f6b0003d580 0x7f6b0003fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:32:47.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.197+0000 7f6b1fe00640 1 -- 172.21.15.42:0/1628679612 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b200be330 con 0x7f6b20072970 2024-07-04T17:32:47.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.199+0000 7f6b16a00640 1 -- 172.21.15.42:0/1628679612 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+7357 (secure 0 0 0) 0x7f6b200be330 con 0x7f6b20072970 2024-07-04T17:32:47.200 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:32:47.200 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":79,"btime":"2024-07-04T17:32:30:398730+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/244742173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":244742173},{"type":"v1","addr":"172.21.15.42:6825","nonce":244742173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5265,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/973681058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":973681058},{"type":"v1","addr":"172.21.15.42:6827","nonce":973681058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/480649366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":480649366},{"type":"v1","addr":"172.21.15.42:6829","nonce":480649366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:31:41.664414+0000","modified":"2024-07-04T17:32:29.545650+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5277":{"gid":5277,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6833/3777807763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":3777807763},{"type":"v1","addr":"172.21.15.42:6833","nonce":3777807763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5277,"qdb_cluster":[5277]},"id":8},{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:32:02.103946+0000","modified":"2024-07-04T17:32:05.184521+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5276":{"gid":5276,"name":"d","rank":0,"incarnation":73,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6831/327636003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":327636003},{"type":"v1","addr":"172.21.15.42:6831","nonce":327636003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5276,"qdb_cluster":[5276]},"id":9}]} 2024-07-04T17:32:47.200 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 79 2024-07-04T17:32:47.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.201+0000 7f6b1fe00640 1 -- 172.21.15.42:0/1628679612 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f6b0003d580 msgr2=0x7f6b0003fa90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:32:47.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.201+0000 7f6b1fe00640 1 --2- 172.21.15.42:0/1628679612 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f6b0003d580 0x7f6b0003fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:47.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.201+0000 7f6b1fe00640 1 -- 172.21.15.42:0/1628679612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b20072970 msgr2=0x7f6b200c0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:47.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.201+0000 7f6b1fe00640 1 --2- 172.21.15.42:0/1628679612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b20072970 0x7f6b200c0590 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f6b10006010 tx=0x7f6b10003f10 comp rx=0 tx=0).stop 2024-07-04T17:32:47.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.202+0000 7f6b1fe00640 1 -- 172.21.15.42:0/1628679612 shutdown_connections 2024-07-04T17:32:47.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.202+0000 7f6b1fe00640 1 --2- 172.21.15.42:0/1628679612 >> [v2:172.21.15.42:6834/1323178695,v1:172.21.15.42:6835/1323178695] conn(0x7f6b0003d580 0x7f6b0003fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:47.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.202+0000 7f6b1fe00640 1 --2- 172.21.15.42:0/1628679612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b20072970 0x7f6b200c0590 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:47.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.202+0000 7f6b1fe00640 1 -- 172.21.15.42:0/1628679612 >> 172.21.15.42:0/1628679612 conn(0x7f6b20071a70 msgr2=0x7f6b2014c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:47.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.202+0000 7f6b1fe00640 1 -- 172.21.15.42:0/1628679612 shutdown_connections 2024-07-04T17:32:47.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.202+0000 7f6b1fe00640 1 -- 172.21.15.42:0/1628679612 wait complete. 2024-07-04T17:32:47.228 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:32:47.330 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:47.329+0000 7fa5dae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:47.330 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:47.329+0000 7fa5dae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:47.330 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:47.329+0000 7fa5dae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:47.330 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:47.329+0000 7fa5dae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:47.330 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:47.329+0000 7fa5dae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:47.331 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:47.330+0000 7fa5d7200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:47.331 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:47.330+0000 7fa5d7200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:47.331 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:47.330+0000 7fa5d7200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:47.331 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:47.330+0000 7fa5d7200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:47.331 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:47.330+0000 7fa5d7200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:47.343 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:47.342+0000 7fa5d7200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:47.343 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:47.342+0000 7fa5d7200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:47.343 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:47.342+0000 7fa5d7200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:47.343 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:47.342+0000 7fa5d7200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:47.343 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:47.342+0000 7fa5d7200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:47.345 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:47.343+0000 7fa5dae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:47.345 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:47.343+0000 7fa5dae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:47.345 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:47.343+0000 7fa5dae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:47.345 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:47.343+0000 7fa5dae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:47.345 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:32:47.343+0000 7fa5dae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:32:47.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.497+0000 7f4b6de00640 1 -- 172.21.15.42:0/1191660101 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b680b38b0 msgr2=0x7f4b680b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:47.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.497+0000 7f4b6de00640 1 --2- 172.21.15.42:0/1191660101 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b680b38b0 0x7f4b680b2d20 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f4b58005fd0 tx=0x7f4b5802a5f0 comp rx=0 tx=0).stop 2024-07-04T17:32:47.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.497+0000 7f4b6de00640 1 -- 172.21.15.42:0/1191660101 shutdown_connections 2024-07-04T17:32:47.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.497+0000 7f4b6de00640 1 --2- 172.21.15.42:0/1191660101 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b680b38b0 0x7f4b680b2d20 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:47.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.497+0000 7f4b6de00640 1 -- 172.21.15.42:0/1191660101 >> 172.21.15.42:0/1191660101 conn(0x7f4b680bf700 msgr2=0x7f4b680af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:47.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.497+0000 7f4b6de00640 1 -- 172.21.15.42:0/1191660101 shutdown_connections 2024-07-04T17:32:47.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.497+0000 7f4b6de00640 1 -- 172.21.15.42:0/1191660101 wait complete. 2024-07-04T17:32:47.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.499+0000 7f4b6de00640 1 Processor -- start 2024-07-04T17:32:47.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.499+0000 7f4b6de00640 1 -- start start 2024-07-04T17:32:47.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.500+0000 7f4b6de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b680b1e00 0x7f4b681d4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:47.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.500+0000 7f4b6de00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4b680b06e0 con 0x7f4b680b1e00 2024-07-04T17:32:47.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.501+0000 7f4b66a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b680b1e00 0x7f4b681d4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:47.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.501+0000 7f4b66a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b680b1e00 0x7f4b681d4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37984/0 (socket says 172.21.15.42:37984) 2024-07-04T17:32:47.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.501+0000 7f4b66a00640 1 -- 172.21.15.42:0/2180923282 learned_addr learned my addr 172.21.15.42:0/2180923282 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:47.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.501+0000 7f4b66a00640 1 -- 172.21.15.42:0/2180923282 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b681d6740 con 0x7f4b680b1e00 2024-07-04T17:32:47.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.501+0000 7f4b66a00640 1 --2- 172.21.15.42:0/2180923282 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b680b1e00 0x7f4b681d4df0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f4b58009850 tx=0x7f4b580049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:47.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.502+0000 7f4b5f400640 1 -- 172.21.15.42:0/2180923282 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b5803e030 con 0x7f4b680b1e00 2024-07-04T17:32:47.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.502+0000 7f4b5f400640 1 -- 172.21.15.42:0/2180923282 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b580035f0 con 0x7f4b680b1e00 2024-07-04T17:32:47.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.502+0000 7f4b6de00640 1 -- 172.21.15.42:0/2180923282 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b680aa540 con 0x7f4b680b1e00 2024-07-04T17:32:47.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.502+0000 7f4b5f400640 1 -- 172.21.15.42:0/2180923282 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b58035040 con 0x7f4b680b1e00 2024-07-04T17:32:47.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.502+0000 7f4b6de00640 1 -- 172.21.15.42:0/2180923282 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b681d5330 con 0x7f4b680b1e00 2024-07-04T17:32:47.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.503+0000 7f4b6de00640 1 -- 172.21.15.42:0/2180923282 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b6806c590 con 0x7f4b680b1e00 2024-07-04T17:32:47.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.503+0000 7f4b5f400640 1 -- 172.21.15.42:0/2180923282 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 39) ==== 49122+0+0 (secure 0 0 0) 0x7f4b58041030 con 0x7f4b680b1e00 2024-07-04T17:32:47.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.504+0000 7f4b5f400640 1 -- 172.21.15.42:0/2180923282 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5163+0+0 (secure 0 0 0) 0x7f4b5803f070 con 0x7f4b680b1e00 2024-07-04T17:32:47.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.509+0000 7f4b5f400640 1 -- 172.21.15.42:0/2180923282 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f4b58042cc0 con 0x7f4b680b1e00 2024-07-04T17:32:47.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.721+0000 7f4b6de00640 1 -- 172.21.15.42:0/2180923282 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4b68069b70 con 0x7f4b680b1e00 2024-07-04T17:32:47.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.723+0000 7f4b5f400640 1 -- 172.21.15.42:0/2180923282 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+15702 (secure 0 0 0) 0x7f4b5804ba40 con 0x7f4b680b1e00 2024-07-04T17:32:47.724 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:32:47.724 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":99,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:32:47.238406+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:31:39.727656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-07-04T17:31:40.615610+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:31:59.635881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.4099999666213989,"score_stable":1.4099999666213989,"optimal_score":1,"raw_score_acting":1.4099999666213989,"raw_score_stable":1.4099999666213989,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:32:01.086110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.0800000429153442,"score_stable":1.0800000429153442,"optimal_score":1,"raw_score_acting":1.0800000429153442,"raw_score_stable":1.0800000429153442,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6835/1323178695":"2024-07-05T17:32:47.238337+0000","172.21.15.42:6834/1323178695":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/3479187327":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/143103576":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/1626565787":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/1439263513":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/2943589644":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/2854152871":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/4179523139":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/2096108670":"2024-07-04T18:32:31.377755+0000","172.21.15.42:0/1641580783":"2024-07-05T17:32:15.307330+0000","172.21.15.42:0/3208988074":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/1212024809":"2024-07-05T17:32:15.307330+0000","172.21.15.42:0/1089649627":"2024-07-05T17:32:15.307330+0000","172.21.15.42:0/257615823":"2024-07-05T17:32:15.307330+0000","172.21.15.42:0/2624550914":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/2580772079":"2024-07-05T17:32:15.307330+0000","172.21.15.42:0/611598150":"2024-07-05T17:32:15.307330+0000","172.21.15.42:6835/1139872311":"2024-07-05T17:32:15.307330+0000","172.21.15.42:6834/1139872311":"2024-07-05T17:32:15.307330+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:32:47.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.726+0000 7f4b6de00640 1 -- 172.21.15.42:0/2180923282 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b680b1e00 msgr2=0x7f4b681d4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:47.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.726+0000 7f4b6de00640 1 --2- 172.21.15.42:0/2180923282 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b680b1e00 0x7f4b681d4df0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f4b58009850 tx=0x7f4b580049d0 comp rx=0 tx=0).stop 2024-07-04T17:32:47.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.726+0000 7f4b6de00640 1 -- 172.21.15.42:0/2180923282 shutdown_connections 2024-07-04T17:32:47.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.726+0000 7f4b6de00640 1 --2- 172.21.15.42:0/2180923282 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b680b1e00 0x7f4b681d4df0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:47.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.726+0000 7f4b6de00640 1 -- 172.21.15.42:0/2180923282 >> 172.21.15.42:0/2180923282 conn(0x7f4b680bf700 msgr2=0x7f4b681412f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:47.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.726+0000 7f4b6de00640 1 -- 172.21.15.42:0/2180923282 shutdown_connections 2024-07-04T17:32:47.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:47.726+0000 7f4b6de00640 1 -- 172.21.15.42:0/2180923282 wait complete. 2024-07-04T17:32:47.751 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-07-04T17:32:48.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.029+0000 7f5f3e800640 1 -- 172.21.15.42:0/4064778285 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f380bad70 msgr2=0x7f5f3806c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:48.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.029+0000 7f5f3e800640 1 --2- 172.21.15.42:0/4064778285 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f380bad70 0x7f5f3806c2c0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f5f28005fd0 tx=0x7f5f2802b250 comp rx=0 tx=0).stop 2024-07-04T17:32:48.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.029+0000 7f5f3e800640 1 -- 172.21.15.42:0/4064778285 shutdown_connections 2024-07-04T17:32:48.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.029+0000 7f5f3e800640 1 --2- 172.21.15.42:0/4064778285 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f380bad70 0x7f5f3806c2c0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:48.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.029+0000 7f5f3e800640 1 -- 172.21.15.42:0/4064778285 >> 172.21.15.42:0/4064778285 conn(0x7f5f380bebc0 msgr2=0x7f5f380befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:48.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.029+0000 7f5f3e800640 1 -- 172.21.15.42:0/4064778285 shutdown_connections 2024-07-04T17:32:48.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.029+0000 7f5f3e800640 1 -- 172.21.15.42:0/4064778285 wait complete. 2024-07-04T17:32:48.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.031+0000 7f5f3e800640 1 Processor -- start 2024-07-04T17:32:48.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.031+0000 7f5f3e800640 1 -- start start 2024-07-04T17:32:48.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.032+0000 7f5f3e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f380b5650 0x7f5f381d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:48.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.032+0000 7f5f3e800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5f380b27c0 con 0x7f5f380b5650 2024-07-04T17:32:48.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.032+0000 7f5f3d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f380b5650 0x7f5f381d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:48.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.032+0000 7f5f3d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f380b5650 0x7f5f381d3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37990/0 (socket says 172.21.15.42:37990) 2024-07-04T17:32:48.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.032+0000 7f5f3d400640 1 -- 172.21.15.42:0/3053686607 learned_addr learned my addr 172.21.15.42:0/3053686607 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:48.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.033+0000 7f5f3d400640 1 -- 172.21.15.42:0/3053686607 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f380b6be0 con 0x7f5f380b5650 2024-07-04T17:32:48.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.033+0000 7f5f3d400640 1 --2- 172.21.15.42:0/3053686607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f380b5650 0x7f5f381d3c80 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f5f2802b840 tx=0x7f5f28003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:48.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.033+0000 7f5f35600640 1 -- 172.21.15.42:0/3053686607 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f2802be60 con 0x7f5f380b5650 2024-07-04T17:32:48.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.033+0000 7f5f35600640 1 -- 172.21.15.42:0/3053686607 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f28003c40 con 0x7f5f380b5650 2024-07-04T17:32:48.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.033+0000 7f5f35600640 1 -- 172.21.15.42:0/3053686607 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f2800a0d0 con 0x7f5f380b5650 2024-07-04T17:32:48.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.033+0000 7f5f3e800640 1 -- 172.21.15.42:0/3053686607 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f381dca20 con 0x7f5f380b5650 2024-07-04T17:32:48.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.034+0000 7f5f3e800640 1 -- 172.21.15.42:0/3053686607 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f381d43a0 con 0x7f5f380b5650 2024-07-04T17:32:48.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.035+0000 7f5f3e800640 1 -- 172.21.15.42:0/3053686607 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f3806c2c0 con 0x7f5f380b5650 2024-07-04T17:32:48.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.035+0000 7f5f35600640 1 -- 172.21.15.42:0/3053686607 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 39) ==== 49122+0+0 (secure 0 0 0) 0x7f5f2800a380 con 0x7f5f380b5650 2024-07-04T17:32:48.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.035+0000 7f5f35600640 1 -- 172.21.15.42:0/3053686607 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5163+0+0 (secure 0 0 0) 0x7f5f28071760 con 0x7f5f380b5650 2024-07-04T17:32:48.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.040+0000 7f5f35600640 1 -- 172.21.15.42:0/3053686607 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f5f28045360 con 0x7f5f380b5650 2024-07-04T17:32:48.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.254+0000 7f5f35600640 1 -- 172.21.15.42:0/3053686607 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 40) ==== 49249+0+0 (secure 0 0 0) 0x7f5f2803d970 con 0x7f5f380b5650 2024-07-04T17:32:48.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.254+0000 7f5f35600640 1 --2- 172.21.15.42:0/3053686607 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f5f18044190 0x7f5f180465b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:48.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.256+0000 7f5f3ca00640 1 --2- 172.21.15.42:0/3053686607 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f5f18044190 0x7f5f180465b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:48.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.260+0000 7f5f3ca00640 1 --2- 172.21.15.42:0/3053686607 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f5f18044190 0x7f5f180465b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5f24001500 tx=0x7f5f24007a30 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:48.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.290+0000 7f5f3e800640 1 -- 172.21.15.42:0/3053686607 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5f381dcc50 con 0x7f5f380b5650 2024-07-04T17:32:48.292 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-07-04T17:32:48.291+0000 7f8616e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-04T17:32:48.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.293+0000 7f5f35600640 1 -- 172.21.15.42:0/3053686607 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v80) ==== 161+0+0 (secure 0 0 0) 0x7f5f28031300 con 0x7f5f380b5650 2024-07-04T17:32:48.294 INFO:teuthology.orchestra.run.smithi042.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-04T17:32:48.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.296+0000 7f5f3e800640 1 -- 172.21.15.42:0/3053686607 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f5f18044190 msgr2=0x7f5f180465b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:48.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.296+0000 7f5f3e800640 1 --2- 172.21.15.42:0/3053686607 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f5f18044190 0x7f5f180465b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5f24001500 tx=0x7f5f24007a30 comp rx=0 tx=0).stop 2024-07-04T17:32:48.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.296+0000 7f5f3e800640 1 -- 172.21.15.42:0/3053686607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f380b5650 msgr2=0x7f5f381d3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:48.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.296+0000 7f5f3e800640 1 --2- 172.21.15.42:0/3053686607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f380b5650 0x7f5f381d3c80 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f5f2802b840 tx=0x7f5f28003820 comp rx=0 tx=0).stop 2024-07-04T17:32:48.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.297+0000 7f5f3e800640 1 -- 172.21.15.42:0/3053686607 shutdown_connections 2024-07-04T17:32:48.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.297+0000 7f5f3e800640 1 --2- 172.21.15.42:0/3053686607 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f5f18044190 0x7f5f180465b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:48.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.297+0000 7f5f3e800640 1 --2- 172.21.15.42:0/3053686607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f380b5650 0x7f5f381d3c80 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:48.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.297+0000 7f5f3e800640 1 -- 172.21.15.42:0/3053686607 >> 172.21.15.42:0/3053686607 conn(0x7f5f380bebc0 msgr2=0x7f5f380b0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:48.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.297+0000 7f5f3e800640 1 -- 172.21.15.42:0/3053686607 shutdown_connections 2024-07-04T17:32:48.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.297+0000 7f5f3e800640 1 -- 172.21.15.42:0/3053686607 wait complete. 2024-07-04T17:32:48.315 INFO:tasks.ceph.mds.a.smithi042.stderr: -5552> 2024-07-04T17:31:44.037+0000 7fdfe2a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-04T17:32:48.326 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-07-04T17:32:48.455 INFO:tasks.ceph.mds.a.smithi042.stdout:starting mds.a at 2024-07-04T17:32:48.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.595+0000 7faa6ca00640 1 -- 172.21.15.42:0/1258750967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa68070cf0 msgr2=0x7faa68149ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:48.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.595+0000 7faa6ca00640 1 --2- 172.21.15.42:0/1258750967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa68070cf0 0x7faa68149ea0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7faa54005fd0 tx=0x7faa5402ae00 comp rx=0 tx=0).stop 2024-07-04T17:32:48.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.595+0000 7faa6ca00640 1 -- 172.21.15.42:0/1258750967 shutdown_connections 2024-07-04T17:32:48.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.595+0000 7faa6ca00640 1 --2- 172.21.15.42:0/1258750967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa68070cf0 0x7faa68149ea0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:48.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.595+0000 7faa6ca00640 1 -- 172.21.15.42:0/1258750967 >> 172.21.15.42:0/1258750967 conn(0x7faa6806fe40 msgr2=0x7faa6806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:48.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.595+0000 7faa6ca00640 1 -- 172.21.15.42:0/1258750967 shutdown_connections 2024-07-04T17:32:48.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.595+0000 7faa6ca00640 1 -- 172.21.15.42:0/1258750967 wait complete. 2024-07-04T17:32:48.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.597+0000 7faa6ca00640 1 Processor -- start 2024-07-04T17:32:48.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.598+0000 7faa6ca00640 1 -- start start 2024-07-04T17:32:48.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.599+0000 7faa6ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa68070cf0 0x7faa681d4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:48.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.599+0000 7faa6ca00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faa68071ef0 con 0x7faa68070cf0 2024-07-04T17:32:48.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.599+0000 7faa65e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa68070cf0 0x7faa681d4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:48.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.599+0000 7faa65e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa68070cf0 0x7faa681d4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38020/0 (socket says 172.21.15.42:38020) 2024-07-04T17:32:48.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.599+0000 7faa65e00640 1 -- 172.21.15.42:0/3589918562 learned_addr learned my addr 172.21.15.42:0/3589918562 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:48.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.600+0000 7faa65e00640 1 -- 172.21.15.42:0/3589918562 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa680be1d0 con 0x7faa68070cf0 2024-07-04T17:32:48.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.600+0000 7faa65e00640 1 --2- 172.21.15.42:0/3589918562 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa68070cf0 0x7faa681d4c90 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7faa5800c340 tx=0x7faa5800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:48.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.600+0000 7faa5e000640 1 -- 172.21.15.42:0/3589918562 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa58017070 con 0x7faa68070cf0 2024-07-04T17:32:48.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.601+0000 7faa5e000640 1 -- 172.21.15.42:0/3589918562 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa580052f0 con 0x7faa68070cf0 2024-07-04T17:32:48.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.601+0000 7faa5e000640 1 -- 172.21.15.42:0/3589918562 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa580035f0 con 0x7faa68070cf0 2024-07-04T17:32:48.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.601+0000 7faa6ca00640 1 -- 172.21.15.42:0/3589918562 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa6813d850 con 0x7faa68070cf0 2024-07-04T17:32:48.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.601+0000 7faa6ca00640 1 -- 172.21.15.42:0/3589918562 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa681453b0 con 0x7faa68070cf0 2024-07-04T17:32:48.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.602+0000 7faa6ca00640 1 -- 172.21.15.42:0/3589918562 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa68071950 con 0x7faa68070cf0 2024-07-04T17:32:48.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.602+0000 7faa5e000640 1 -- 172.21.15.42:0/3589918562 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 40) ==== 49249+0+0 (secure 0 0 0) 0x7faa5801a070 con 0x7faa68070cf0 2024-07-04T17:32:48.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.603+0000 7faa5e000640 1 --2- 172.21.15.42:0/3589918562 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7faa4803d100 0x7faa4803f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:48.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.603+0000 7faa5e000640 1 -- 172.21.15.42:0/3589918562 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5249+0+0 (secure 0 0 0) 0x7faa58050bd0 con 0x7faa68070cf0 2024-07-04T17:32:48.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.603+0000 7faa65400640 1 --2- 172.21.15.42:0/3589918562 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7faa4803d100 0x7faa4803f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:48.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.604+0000 7faa65400640 1 --2- 172.21.15.42:0/3589918562 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7faa4803d100 0x7faa4803f610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7faa54034000 tx=0x7faa54004730 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:48.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.608+0000 7faa5e000640 1 -- 172.21.15.42:0/3589918562 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7faa68071950 con 0x7faa68070cf0 2024-07-04T17:32:48.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:48.863+0000 7faa6ca00640 1 -- 172.21.15.42:0/3589918562 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7faa68149ea0 con 0x7faa68070cf0 2024-07-04T17:32:49.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.451+0000 7faa5e000640 1 -- 172.21.15.42:0/3589918562 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v81) ==== 104+0+0 (secure 0 0 0) 0x7faa58011630 con 0x7faa68070cf0 2024-07-04T17:32:49.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.455+0000 7faa6ca00640 1 -- 172.21.15.42:0/3589918562 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7faa4803d100 msgr2=0x7faa4803f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:49.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.455+0000 7faa6ca00640 1 --2- 172.21.15.42:0/3589918562 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7faa4803d100 0x7faa4803f610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7faa54034000 tx=0x7faa54004730 comp rx=0 tx=0).stop 2024-07-04T17:32:49.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.455+0000 7faa6ca00640 1 -- 172.21.15.42:0/3589918562 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa68070cf0 msgr2=0x7faa681d4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:49.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.455+0000 7faa6ca00640 1 --2- 172.21.15.42:0/3589918562 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa68070cf0 0x7faa681d4c90 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7faa5800c340 tx=0x7faa5800c800 comp rx=0 tx=0).stop 2024-07-04T17:32:49.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.456+0000 7faa6ca00640 1 -- 172.21.15.42:0/3589918562 shutdown_connections 2024-07-04T17:32:49.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.456+0000 7faa6ca00640 1 --2- 172.21.15.42:0/3589918562 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7faa4803d100 0x7faa4803f610 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:49.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.456+0000 7faa6ca00640 1 --2- 172.21.15.42:0/3589918562 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa68070cf0 0x7faa681d4c90 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:49.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.456+0000 7faa6ca00640 1 -- 172.21.15.42:0/3589918562 >> 172.21.15.42:0/3589918562 conn(0x7faa6806fe40 msgr2=0x7faa681493b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:49.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.456+0000 7faa6ca00640 1 -- 172.21.15.42:0/3589918562 shutdown_connections 2024-07-04T17:32:49.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.456+0000 7faa6ca00640 1 -- 172.21.15.42:0/3589918562 wait complete. 2024-07-04T17:32:49.487 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-04T17:32:49.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.780+0000 7efe53400640 1 -- 172.21.15.42:0/1611805735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe4c06dc80 msgr2=0x7efe4c0a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:49.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.780+0000 7efe53400640 1 --2- 172.21.15.42:0/1611805735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe4c06dc80 0x7efe4c0a94d0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7efe40005fd0 tx=0x7efe4002a5f0 comp rx=0 tx=0).stop 2024-07-04T17:32:49.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.780+0000 7efe53400640 1 -- 172.21.15.42:0/1611805735 shutdown_connections 2024-07-04T17:32:49.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.780+0000 7efe53400640 1 --2- 172.21.15.42:0/1611805735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe4c06dc80 0x7efe4c0a94d0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:49.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.780+0000 7efe53400640 1 -- 172.21.15.42:0/1611805735 >> 172.21.15.42:0/1611805735 conn(0x7efe4c06c020 msgr2=0x7efe4c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:49.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.780+0000 7efe53400640 1 -- 172.21.15.42:0/1611805735 shutdown_connections 2024-07-04T17:32:49.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.780+0000 7efe53400640 1 -- 172.21.15.42:0/1611805735 wait complete. 2024-07-04T17:32:49.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.783+0000 7efe53400640 1 Processor -- start 2024-07-04T17:32:49.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.783+0000 7efe53400640 1 -- start start 2024-07-04T17:32:49.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.785+0000 7efe53400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe4c06dc80 0x7efe4c142040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:49.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.785+0000 7efe53400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c074100 con 0x7efe4c06dc80 2024-07-04T17:32:49.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.785+0000 7efe50a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe4c06dc80 0x7efe4c142040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:49.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.786+0000 7efe50a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe4c06dc80 0x7efe4c142040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38064/0 (socket says 172.21.15.42:38064) 2024-07-04T17:32:49.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.786+0000 7efe50a00640 1 -- 172.21.15.42:0/4053607658 learned_addr learned my addr 172.21.15.42:0/4053607658 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:49.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.786+0000 7efe50a00640 1 -- 172.21.15.42:0/4053607658 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe4c152830 con 0x7efe4c06dc80 2024-07-04T17:32:49.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.786+0000 7efe50a00640 1 --2- 172.21.15.42:0/4053607658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe4c06dc80 0x7efe4c142040 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7efe40009850 tx=0x7efe400049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:49.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.787+0000 7efe48c00640 1 -- 172.21.15.42:0/4053607658 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe4003e030 con 0x7efe4c06dc80 2024-07-04T17:32:49.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.787+0000 7efe48c00640 1 -- 172.21.15.42:0/4053607658 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe400035f0 con 0x7efe4c06dc80 2024-07-04T17:32:49.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.787+0000 7efe48c00640 1 -- 172.21.15.42:0/4053607658 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe40035040 con 0x7efe4c06dc80 2024-07-04T17:32:49.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.787+0000 7efe53400640 1 -- 172.21.15.42:0/4053607658 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe4c152390 con 0x7efe4c06dc80 2024-07-04T17:32:49.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.787+0000 7efe53400640 1 -- 172.21.15.42:0/4053607658 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe4c133c30 con 0x7efe4c06dc80 2024-07-04T17:32:49.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.788+0000 7efe53400640 1 -- 172.21.15.42:0/4053607658 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe4c0a94d0 con 0x7efe4c06dc80 2024-07-04T17:32:49.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.788+0000 7efe48c00640 1 -- 172.21.15.42:0/4053607658 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 41) ==== 49559+0+0 (secure 0 0 0) 0x7efe40041030 con 0x7efe4c06dc80 2024-07-04T17:32:49.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.789+0000 7efe48c00640 1 --2- 172.21.15.42:0/4053607658 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7efe3003d460 0x7efe3003f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:49.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.789+0000 7efe48c00640 1 -- 172.21.15.42:0/4053607658 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5249+0+0 (secure 0 0 0) 0x7efe4003f070 con 0x7efe4c06dc80 2024-07-04T17:32:49.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.789+0000 7efe4be00640 1 --2- 172.21.15.42:0/4053607658 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7efe3003d460 0x7efe3003f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:49.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.790+0000 7efe4be00640 1 --2- 172.21.15.42:0/4053607658 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7efe3003d460 0x7efe3003f970 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efe3c001560 tx=0x7efe3c0079f0 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:49.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:49.794+0000 7efe48c00640 1 -- 172.21.15.42:0/4053607658 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7efe4004bd20 con 0x7efe4c06dc80 2024-07-04T17:32:50.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.006+0000 7efe53400640 1 -- 172.21.15.42:0/4053607658 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7efe4c069b70 con 0x7efe4c06dc80 2024-07-04T17:32:50.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.467+0000 7efe48c00640 1 -- 172.21.15.42:0/4053607658 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v101) ==== 181+0+0 (secure 0 0 0) 0x7efe40030c90 con 0x7efe4c06dc80 2024-07-04T17:32:50.469 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' removed 2024-07-04T17:32:50.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.471+0000 7efe53400640 1 -- 172.21.15.42:0/4053607658 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7efe3003d460 msgr2=0x7efe3003f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:50.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.471+0000 7efe53400640 1 --2- 172.21.15.42:0/4053607658 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7efe3003d460 0x7efe3003f970 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efe3c001560 tx=0x7efe3c0079f0 comp rx=0 tx=0).stop 2024-07-04T17:32:50.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.471+0000 7efe53400640 1 -- 172.21.15.42:0/4053607658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe4c06dc80 msgr2=0x7efe4c142040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:50.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.471+0000 7efe53400640 1 --2- 172.21.15.42:0/4053607658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe4c06dc80 0x7efe4c142040 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7efe40009850 tx=0x7efe400049d0 comp rx=0 tx=0).stop 2024-07-04T17:32:50.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.472+0000 7efe53400640 1 -- 172.21.15.42:0/4053607658 shutdown_connections 2024-07-04T17:32:50.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.472+0000 7efe53400640 1 --2- 172.21.15.42:0/4053607658 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7efe3003d460 0x7efe3003f970 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:50.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.472+0000 7efe53400640 1 --2- 172.21.15.42:0/4053607658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe4c06dc80 0x7efe4c142040 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:50.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.472+0000 7efe53400640 1 -- 172.21.15.42:0/4053607658 >> 172.21.15.42:0/4053607658 conn(0x7efe4c06c020 msgr2=0x7efe4c13e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:50.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.472+0000 7efe53400640 1 -- 172.21.15.42:0/4053607658 shutdown_connections 2024-07-04T17:32:50.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.473+0000 7efe53400640 1 -- 172.21.15.42:0/4053607658 wait complete. 2024-07-04T17:32:50.518 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-04T17:32:50.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.807+0000 7f9b5e800640 1 -- 172.21.15.42:0/4145320181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b580ab2f0 msgr2=0x7f9b580a9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:50.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.807+0000 7f9b5e800640 1 --2- 172.21.15.42:0/4145320181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b580ab2f0 0x7f9b580a9990 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f9b48005fd0 tx=0x7f9b4802aac0 comp rx=0 tx=0).stop 2024-07-04T17:32:50.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.808+0000 7f9b5e800640 1 -- 172.21.15.42:0/4145320181 shutdown_connections 2024-07-04T17:32:50.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.808+0000 7f9b5e800640 1 --2- 172.21.15.42:0/4145320181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b580ab2f0 0x7f9b580a9990 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:50.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.808+0000 7f9b5e800640 1 -- 172.21.15.42:0/4145320181 >> 172.21.15.42:0/4145320181 conn(0x7f9b5806c020 msgr2=0x7f9b5806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:50.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.808+0000 7f9b5e800640 1 -- 172.21.15.42:0/4145320181 shutdown_connections 2024-07-04T17:32:50.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.808+0000 7f9b5e800640 1 -- 172.21.15.42:0/4145320181 wait complete. 2024-07-04T17:32:50.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.810+0000 7f9b5e800640 1 Processor -- start 2024-07-04T17:32:50.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.810+0000 7f9b5e800640 1 -- start start 2024-07-04T17:32:50.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.812+0000 7f9b5e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b5806d070 0x7f9b580bbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:50.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.812+0000 7f9b5e800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5813f430 con 0x7f9b5806d070 2024-07-04T17:32:50.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.812+0000 7f9b57400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b5806d070 0x7f9b580bbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:50.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.812+0000 7f9b57400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b5806d070 0x7f9b580bbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38082/0 (socket says 172.21.15.42:38082) 2024-07-04T17:32:50.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.812+0000 7f9b57400640 1 -- 172.21.15.42:0/1501654765 learned_addr learned my addr 172.21.15.42:0/1501654765 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:50.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.812+0000 7f9b57400640 1 -- 172.21.15.42:0/1501654765 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b580ae490 con 0x7f9b5806d070 2024-07-04T17:32:50.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.813+0000 7f9b57400640 1 --2- 172.21.15.42:0/1501654765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b5806d070 0x7f9b580bbe00 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f9b48006010 tx=0x7f9b48003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:50.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.813+0000 7f9b4f400640 1 -- 172.21.15.42:0/1501654765 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b48044070 con 0x7f9b5806d070 2024-07-04T17:32:50.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.813+0000 7f9b4f400640 1 -- 172.21.15.42:0/1501654765 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b48004250 con 0x7f9b5806d070 2024-07-04T17:32:50.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.813+0000 7f9b5e800640 1 -- 172.21.15.42:0/1501654765 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b581e4350 con 0x7f9b5806d070 2024-07-04T17:32:50.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.813+0000 7f9b4f400640 1 -- 172.21.15.42:0/1501654765 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b48004590 con 0x7f9b5806d070 2024-07-04T17:32:50.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.813+0000 7f9b5e800640 1 -- 172.21.15.42:0/1501654765 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b580bb5c0 con 0x7f9b5806d070 2024-07-04T17:32:50.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.815+0000 7f9b5e800640 1 -- 172.21.15.42:0/1501654765 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b580ae810 con 0x7f9b5806d070 2024-07-04T17:32:50.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.815+0000 7f9b4f400640 1 -- 172.21.15.42:0/1501654765 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 41) ==== 49559+0+0 (secure 0 0 0) 0x7f9b48047070 con 0x7f9b5806d070 2024-07-04T17:32:50.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.815+0000 7f9b4f400640 1 --2- 172.21.15.42:0/1501654765 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f9b3403d460 0x7f9b3403f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:50.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.816+0000 7f9b4f400640 1 -- 172.21.15.42:0/1501654765 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 4811+0+0 (secure 0 0 0) 0x7f9b4803c070 con 0x7f9b5806d070 2024-07-04T17:32:50.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.816+0000 7f9b56a00640 1 --2- 172.21.15.42:0/1501654765 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f9b3403d460 0x7f9b3403f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:50.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.816+0000 7f9b56a00640 1 --2- 172.21.15.42:0/1501654765 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f9b3403d460 0x7f9b3403f970 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9b40001560 tx=0x7f9b400079f0 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:50.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:50.821+0000 7f9b4f400640 1 -- 172.21.15.42:0/1501654765 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f9b48032030 con 0x7f9b5806d070 2024-07-04T17:32:51.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.031+0000 7f9b5e800640 1 -- 172.21.15.42:0/1501654765 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9b58140c80 con 0x7f9b5806d070 2024-07-04T17:32:51.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.470+0000 7f9b4f400640 1 -- 172.21.15.42:0/1501654765 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v102) ==== 169+0+0 (secure 0 0 0) 0x7f9b48047360 con 0x7f9b5806d070 2024-07-04T17:32:51.471 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' removed 2024-07-04T17:32:51.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.473+0000 7f9b5e800640 1 -- 172.21.15.42:0/1501654765 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f9b3403d460 msgr2=0x7f9b3403f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:51.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.473+0000 7f9b5e800640 1 --2- 172.21.15.42:0/1501654765 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f9b3403d460 0x7f9b3403f970 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9b40001560 tx=0x7f9b400079f0 comp rx=0 tx=0).stop 2024-07-04T17:32:51.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.473+0000 7f9b5e800640 1 -- 172.21.15.42:0/1501654765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b5806d070 msgr2=0x7f9b580bbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:51.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.473+0000 7f9b5e800640 1 --2- 172.21.15.42:0/1501654765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b5806d070 0x7f9b580bbe00 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f9b48006010 tx=0x7f9b48003f10 comp rx=0 tx=0).stop 2024-07-04T17:32:51.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.474+0000 7f9b5e800640 1 -- 172.21.15.42:0/1501654765 shutdown_connections 2024-07-04T17:32:51.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.474+0000 7f9b5e800640 1 --2- 172.21.15.42:0/1501654765 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f9b3403d460 0x7f9b3403f970 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:51.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.474+0000 7f9b5e800640 1 --2- 172.21.15.42:0/1501654765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b5806d070 0x7f9b580bbe00 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:51.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.474+0000 7f9b5e800640 1 -- 172.21.15.42:0/1501654765 >> 172.21.15.42:0/1501654765 conn(0x7f9b5806c020 msgr2=0x7f9b58070e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:51.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.474+0000 7f9b5e800640 1 -- 172.21.15.42:0/1501654765 shutdown_connections 2024-07-04T17:32:51.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.474+0000 7f9b5e800640 1 -- 172.21.15.42:0/1501654765 wait complete. 2024-07-04T17:32:51.516 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:32:51.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.793+0000 7f217be00640 1 -- 172.21.15.42:0/385148641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f217c0adac0 msgr2=0x7f217c06e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:51.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.793+0000 7f217be00640 1 --2- 172.21.15.42:0/385148641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f217c0adac0 0x7f217c06e120 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f2168005fd0 tx=0x7f216802a5f0 comp rx=0 tx=0).stop 2024-07-04T17:32:51.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.793+0000 7f217be00640 1 -- 172.21.15.42:0/385148641 shutdown_connections 2024-07-04T17:32:51.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.793+0000 7f217be00640 1 --2- 172.21.15.42:0/385148641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f217c0adac0 0x7f217c06e120 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:51.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.793+0000 7f217be00640 1 -- 172.21.15.42:0/385148641 >> 172.21.15.42:0/385148641 conn(0x7f217c071960 msgr2=0x7f217c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:51.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.793+0000 7f217be00640 1 -- 172.21.15.42:0/385148641 shutdown_connections 2024-07-04T17:32:51.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.793+0000 7f217be00640 1 -- 172.21.15.42:0/385148641 wait complete. 2024-07-04T17:32:51.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.795+0000 7f217be00640 1 Processor -- start 2024-07-04T17:32:51.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.796+0000 7f217be00640 1 -- start start 2024-07-04T17:32:51.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.797+0000 7f217be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f217c0adac0 0x7f217c0bc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:51.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.797+0000 7f217be00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f217c06e660 con 0x7f217c0adac0 2024-07-04T17:32:51.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.797+0000 7f217aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f217c0adac0 0x7f217c0bc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:51.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.797+0000 7f217aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f217c0adac0 0x7f217c0bc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38106/0 (socket says 172.21.15.42:38106) 2024-07-04T17:32:51.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.797+0000 7f217aa00640 1 -- 172.21.15.42:0/2620314164 learned_addr learned my addr 172.21.15.42:0/2620314164 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:51.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.797+0000 7f217aa00640 1 -- 172.21.15.42:0/2620314164 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f217c1e33c0 con 0x7f217c0adac0 2024-07-04T17:32:51.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.798+0000 7f217aa00640 1 --2- 172.21.15.42:0/2620314164 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f217c0adac0 0x7f217c0bc410 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f216c00c2f0 tx=0x7f216c00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:51.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.798+0000 7f2172a00640 1 -- 172.21.15.42:0/2620314164 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f216c017070 con 0x7f217c0adac0 2024-07-04T17:32:51.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.798+0000 7f2172a00640 1 -- 172.21.15.42:0/2620314164 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f216c0052f0 con 0x7f217c0adac0 2024-07-04T17:32:51.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.798+0000 7f217be00640 1 -- 172.21.15.42:0/2620314164 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f217c0bc950 con 0x7f217c0adac0 2024-07-04T17:32:51.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.798+0000 7f2172a00640 1 -- 172.21.15.42:0/2620314164 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f216c0035f0 con 0x7f217c0adac0 2024-07-04T17:32:51.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.798+0000 7f217be00640 1 -- 172.21.15.42:0/2620314164 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f217c1e3ed0 con 0x7f217c0adac0 2024-07-04T17:32:51.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.800+0000 7f217be00640 1 -- 172.21.15.42:0/2620314164 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f217c06ed20 con 0x7f217c0adac0 2024-07-04T17:32:51.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.800+0000 7f2172a00640 1 -- 172.21.15.42:0/2620314164 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7f216c01a070 con 0x7f217c0adac0 2024-07-04T17:32:51.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.800+0000 7f2172a00640 1 --2- 172.21.15.42:0/2620314164 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f215c03d4f0 0x7f215c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:51.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.800+0000 7f2172a00640 1 -- 172.21.15.42:0/2620314164 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 4397+0+0 (secure 0 0 0) 0x7f216c050ee0 con 0x7f217c0adac0 2024-07-04T17:32:51.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.801+0000 7f217a000640 1 --2- 172.21.15.42:0/2620314164 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f215c03d4f0 0x7f215c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:51.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.801+0000 7f217a000640 1 --2- 172.21.15.42:0/2620314164 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f215c03d4f0 0x7f215c03fa00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2168033000 tx=0x7f2168004730 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:51.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:51.806+0000 7f2172a00640 1 -- 172.21.15.42:0/2620314164 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f216c0102e0 con 0x7f217c0adac0 2024-07-04T17:32:52.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.053+0000 7f217be00640 1 -- 172.21.15.42:0/2620314164 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f217c0afcd0 con 0x7f217c0adac0 2024-07-04T17:32:52.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.054+0000 7f2172a00640 1 -- 172.21.15.42:0/2620314164 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+6042 (secure 0 0 0) 0x7f216c01bdf0 con 0x7f217c0adac0 2024-07-04T17:32:52.055 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:32:52.055 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":81,"btime":"2024-07-04T17:32:49:450042+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/244742173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":244742173},{"type":"v1","addr":"172.21.15.42:6825","nonce":244742173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5265,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/973681058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":973681058},{"type":"v1","addr":"172.21.15.42:6827","nonce":973681058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/480649366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":480649366},{"type":"v1","addr":"172.21.15.42:6829","nonce":480649366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5477,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6833/2552884551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2552884551},{"type":"v1","addr":"172.21.15.42:6833","nonce":2552884551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:32:02.103946+0000","modified":"2024-07-04T17:32:05.184521+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5276":{"gid":5276,"name":"d","rank":0,"incarnation":73,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6831/327636003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":327636003},{"type":"v1","addr":"172.21.15.42:6831","nonce":327636003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5276,"qdb_cluster":[5276]},"id":9}]} 2024-07-04T17:32:52.056 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 81 2024-07-04T17:32:52.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.057+0000 7f217be00640 1 -- 172.21.15.42:0/2620314164 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f215c03d4f0 msgr2=0x7f215c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:52.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.057+0000 7f217be00640 1 --2- 172.21.15.42:0/2620314164 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f215c03d4f0 0x7f215c03fa00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2168033000 tx=0x7f2168004730 comp rx=0 tx=0).stop 2024-07-04T17:32:52.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.057+0000 7f217be00640 1 -- 172.21.15.42:0/2620314164 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f217c0adac0 msgr2=0x7f217c0bc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:52.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.057+0000 7f217be00640 1 --2- 172.21.15.42:0/2620314164 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f217c0adac0 0x7f217c0bc410 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f216c00c2f0 tx=0x7f216c00c7b0 comp rx=0 tx=0).stop 2024-07-04T17:32:52.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.057+0000 7f217be00640 1 -- 172.21.15.42:0/2620314164 shutdown_connections 2024-07-04T17:32:52.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.057+0000 7f217be00640 1 --2- 172.21.15.42:0/2620314164 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f215c03d4f0 0x7f215c03fa00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:52.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.057+0000 7f217be00640 1 --2- 172.21.15.42:0/2620314164 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f217c0adac0 0x7f217c0bc410 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:52.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.057+0000 7f217be00640 1 -- 172.21.15.42:0/2620314164 >> 172.21.15.42:0/2620314164 conn(0x7f217c071960 msgr2=0x7f217c1d9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:52.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.057+0000 7f217be00640 1 -- 172.21.15.42:0/2620314164 shutdown_connections 2024-07-04T17:32:52.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.057+0000 7f217be00640 1 -- 172.21.15.42:0/2620314164 wait complete. 2024-07-04T17:32:52.083 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:32:52.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.358+0000 7f3596000640 1 -- 172.21.15.42:0/530526491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35900b2d20 msgr2=0x7f35900b37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:52.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.358+0000 7f3596000640 1 --2- 172.21.15.42:0/530526491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35900b2d20 0x7f35900b37d0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f3580005fd0 tx=0x7f358002a5f0 comp rx=0 tx=0).stop 2024-07-04T17:32:52.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.358+0000 7f3596000640 1 -- 172.21.15.42:0/530526491 shutdown_connections 2024-07-04T17:32:52.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.358+0000 7f3596000640 1 --2- 172.21.15.42:0/530526491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35900b2d20 0x7f35900b37d0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:52.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.358+0000 7f3596000640 1 -- 172.21.15.42:0/530526491 >> 172.21.15.42:0/530526491 conn(0x7f35900bf700 msgr2=0x7f35900af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:52.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.359+0000 7f3596000640 1 -- 172.21.15.42:0/530526491 shutdown_connections 2024-07-04T17:32:52.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.359+0000 7f3596000640 1 -- 172.21.15.42:0/530526491 wait complete. 2024-07-04T17:32:52.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.361+0000 7f3596000640 1 Processor -- start 2024-07-04T17:32:52.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.361+0000 7f3596000640 1 -- start start 2024-07-04T17:32:52.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.362+0000 7f3596000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35900b01f0 0x7f35900aa320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:52.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.362+0000 7f3596000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f359006cc20 con 0x7f35900b01f0 2024-07-04T17:32:52.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.363+0000 7f358f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35900b01f0 0x7f35900aa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:52.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.363+0000 7f358f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35900b01f0 0x7f35900aa320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38128/0 (socket says 172.21.15.42:38128) 2024-07-04T17:32:52.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.363+0000 7f358f400640 1 -- 172.21.15.42:0/2835120531 learned_addr learned my addr 172.21.15.42:0/2835120531 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:52.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.363+0000 7f358f400640 1 -- 172.21.15.42:0/2835120531 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35900aa910 con 0x7f35900b01f0 2024-07-04T17:32:52.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.364+0000 7f358f400640 1 --2- 172.21.15.42:0/2835120531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35900b01f0 0x7f35900aa320 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f3580034040 tx=0x7f35800049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:52.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.364+0000 7f3587400640 1 -- 172.21.15.42:0/2835120531 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f358003e030 con 0x7f35900b01f0 2024-07-04T17:32:52.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.364+0000 7f3587400640 1 -- 172.21.15.42:0/2835120531 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35800035f0 con 0x7f35900b01f0 2024-07-04T17:32:52.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.364+0000 7f3596000640 1 -- 172.21.15.42:0/2835120531 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35900ad0b0 con 0x7f35900b01f0 2024-07-04T17:32:52.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.364+0000 7f3587400640 1 -- 172.21.15.42:0/2835120531 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3580035040 con 0x7f35900b01f0 2024-07-04T17:32:52.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.364+0000 7f3596000640 1 -- 172.21.15.42:0/2835120531 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35900aafd0 con 0x7f35900b01f0 2024-07-04T17:32:52.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.365+0000 7f3596000640 1 -- 172.21.15.42:0/2835120531 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f359006d2c0 con 0x7f35900b01f0 2024-07-04T17:32:52.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.365+0000 7f3587400640 1 -- 172.21.15.42:0/2835120531 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7f3580041030 con 0x7f35900b01f0 2024-07-04T17:32:52.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.366+0000 7f3587400640 1 --2- 172.21.15.42:0/2835120531 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f357003d4f0 0x7f357003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:52.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.366+0000 7f3587400640 1 -- 172.21.15.42:0/2835120531 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 4397+0+0 (secure 0 0 0) 0x7f358003f070 con 0x7f35900b01f0 2024-07-04T17:32:52.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.366+0000 7f358ea00640 1 --2- 172.21.15.42:0/2835120531 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f357003d4f0 0x7f357003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:52.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.367+0000 7f358ea00640 1 --2- 172.21.15.42:0/2835120531 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f357003d4f0 0x7f357003fa00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f357c001560 tx=0x7f357c0079f0 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:52.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.371+0000 7f3587400640 1 -- 172.21.15.42:0/2835120531 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f358004bd60 con 0x7f35900b01f0 2024-07-04T17:32:52.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.582+0000 7f3596000640 1 -- 172.21.15.42:0/2835120531 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f35900af6e0 con 0x7f35900b01f0 2024-07-04T17:32:52.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.585+0000 7f3587400640 1 -- 172.21.15.42:0/2835120531 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+11997 (secure 0 0 0) 0x7f3580031400 con 0x7f35900b01f0 2024-07-04T17:32:52.586 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:32:52.586 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":102,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:32:51.468757+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:31:59.635881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.4099999666213989,"score_stable":1.4099999666213989,"optimal_score":1,"raw_score_acting":1.4099999666213989,"raw_score_stable":1.4099999666213989,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:32:01.086110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.0800000429153442,"score_stable":1.0800000429153442,"optimal_score":1,"raw_score_acting":1.0800000429153442,"raw_score_stable":1.0800000429153442,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6832/3777807763":"2024-07-05T17:32:48.292733+0000","172.21.15.42:6835/1323178695":"2024-07-05T17:32:47.238337+0000","172.21.15.42:6834/1323178695":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/3479187327":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/143103576":"2024-07-05T17:32:47.238337+0000","172.21.15.42:6833/3777807763":"2024-07-05T17:32:48.292733+0000","172.21.15.42:0/1626565787":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/1439263513":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/2943589644":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/2854152871":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/4179523139":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/2096108670":"2024-07-04T18:32:31.377755+0000","172.21.15.42:0/1641580783":"2024-07-05T17:32:15.307330+0000","172.21.15.42:0/3208988074":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/1212024809":"2024-07-05T17:32:15.307330+0000","172.21.15.42:0/1089649627":"2024-07-05T17:32:15.307330+0000","172.21.15.42:0/257615823":"2024-07-05T17:32:15.307330+0000","172.21.15.42:0/2624550914":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/2580772079":"2024-07-05T17:32:15.307330+0000","172.21.15.42:0/611598150":"2024-07-05T17:32:15.307330+0000","172.21.15.42:6835/1139872311":"2024-07-05T17:32:15.307330+0000","172.21.15.42:6834/1139872311":"2024-07-05T17:32:15.307330+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:32:52.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.587+0000 7f3596000640 1 -- 172.21.15.42:0/2835120531 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f357003d4f0 msgr2=0x7f357003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:52.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.587+0000 7f3596000640 1 --2- 172.21.15.42:0/2835120531 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f357003d4f0 0x7f357003fa00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f357c001560 tx=0x7f357c0079f0 comp rx=0 tx=0).stop 2024-07-04T17:32:52.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.587+0000 7f3596000640 1 -- 172.21.15.42:0/2835120531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35900b01f0 msgr2=0x7f35900aa320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:52.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.587+0000 7f3596000640 1 --2- 172.21.15.42:0/2835120531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35900b01f0 0x7f35900aa320 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f3580034040 tx=0x7f35800049d0 comp rx=0 tx=0).stop 2024-07-04T17:32:52.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.588+0000 7f3596000640 1 -- 172.21.15.42:0/2835120531 shutdown_connections 2024-07-04T17:32:52.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.588+0000 7f3596000640 1 --2- 172.21.15.42:0/2835120531 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f357003d4f0 0x7f357003fa00 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:52.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.588+0000 7f3596000640 1 --2- 172.21.15.42:0/2835120531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35900b01f0 0x7f35900aa320 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:52.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.588+0000 7f3596000640 1 -- 172.21.15.42:0/2835120531 >> 172.21.15.42:0/2835120531 conn(0x7f35900bf700 msgr2=0x7f359013f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:52.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.588+0000 7f3596000640 1 -- 172.21.15.42:0/2835120531 shutdown_connections 2024-07-04T17:32:52.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.588+0000 7f3596000640 1 -- 172.21.15.42:0/2835120531 wait complete. 2024-07-04T17:32:52.615 INFO:tasks.cephfs.filesystem:Destroying file system backup_fs and related pools 2024-07-04T17:32:52.616 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:32:52.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.903+0000 7ff5b9c00640 1 -- 172.21.15.42:0/2552404697 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5b40b2d20 msgr2=0x7ff5b40bb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:52.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.903+0000 7ff5b9c00640 1 --2- 172.21.15.42:0/2552404697 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5b40b2d20 0x7ff5b40bb1f0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7ff59c005fd0 tx=0x7ff59c02a560 comp rx=0 tx=0).stop 2024-07-04T17:32:52.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.904+0000 7ff5b9c00640 1 -- 172.21.15.42:0/2552404697 shutdown_connections 2024-07-04T17:32:52.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.904+0000 7ff5b9c00640 1 --2- 172.21.15.42:0/2552404697 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5b40b2d20 0x7ff5b40bb1f0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:52.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.904+0000 7ff5b9c00640 1 -- 172.21.15.42:0/2552404697 >> 172.21.15.42:0/2552404697 conn(0x7ff5b40bf700 msgr2=0x7ff5b40af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:52.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.904+0000 7ff5b9c00640 1 -- 172.21.15.42:0/2552404697 shutdown_connections 2024-07-04T17:32:52.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.904+0000 7ff5b9c00640 1 -- 172.21.15.42:0/2552404697 wait complete. 2024-07-04T17:32:52.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.907+0000 7ff5b9c00640 1 Processor -- start 2024-07-04T17:32:52.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.907+0000 7ff5b9c00640 1 -- start start 2024-07-04T17:32:52.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.908+0000 7ff5b9c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5b40b2d20 0x7ff5b413c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:52.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.908+0000 7ff5b9c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b406c020 con 0x7ff5b40b2d20 2024-07-04T17:32:52.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.909+0000 7ff5b2a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5b40b2d20 0x7ff5b413c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:52.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.909+0000 7ff5b2a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5b40b2d20 0x7ff5b413c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38156/0 (socket says 172.21.15.42:38156) 2024-07-04T17:32:52.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.909+0000 7ff5b2a00640 1 -- 172.21.15.42:0/2013460308 learned_addr learned my addr 172.21.15.42:0/2013460308 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:52.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.909+0000 7ff5b2a00640 1 -- 172.21.15.42:0/2013460308 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b41e4050 con 0x7ff5b40b2d20 2024-07-04T17:32:52.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.909+0000 7ff5b2a00640 1 --2- 172.21.15.42:0/2013460308 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5b40b2d20 0x7ff5b413c530 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7ff5a400c380 tx=0x7ff5a400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:52.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.910+0000 7ff5aaa00640 1 -- 172.21.15.42:0/2013460308 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5a4017070 con 0x7ff5b40b2d20 2024-07-04T17:32:52.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.910+0000 7ff5aaa00640 1 -- 172.21.15.42:0/2013460308 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5a40052f0 con 0x7ff5b40b2d20 2024-07-04T17:32:52.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.910+0000 7ff5aaa00640 1 -- 172.21.15.42:0/2013460308 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5a40035f0 con 0x7ff5b40b2d20 2024-07-04T17:32:52.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.910+0000 7ff5b9c00640 1 -- 172.21.15.42:0/2013460308 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5b413ca70 con 0x7ff5b40b2d20 2024-07-04T17:32:52.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.910+0000 7ff5b9c00640 1 -- 172.21.15.42:0/2013460308 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5b413e600 con 0x7ff5b40b2d20 2024-07-04T17:32:52.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.911+0000 7ff5aaa00640 1 -- 172.21.15.42:0/2013460308 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7ff5a401a070 con 0x7ff5b40b2d20 2024-07-04T17:32:52.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.911+0000 7ff5b9c00640 1 -- 172.21.15.42:0/2013460308 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5b413e7d0 con 0x7ff5b40b2d20 2024-07-04T17:32:52.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.912+0000 7ff5aaa00640 1 --2- 172.21.15.42:0/2013460308 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7ff59003d4f0 0x7ff59003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:52.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.912+0000 7ff5aaa00640 1 -- 172.21.15.42:0/2013460308 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 4397+0+0 (secure 0 0 0) 0x7ff5a4050fe0 con 0x7ff5b40b2d20 2024-07-04T17:32:52.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.912+0000 7ff5b2000640 1 --2- 172.21.15.42:0/2013460308 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7ff59003d4f0 0x7ff59003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:52.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.913+0000 7ff5b2000640 1 --2- 172.21.15.42:0/2013460308 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7ff59003d4f0 0x7ff59003fa00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff59c033000 tx=0x7ff59c004730 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:52.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:52.917+0000 7ff5aaa00640 1 -- 172.21.15.42:0/2013460308 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7ff5b413e7d0 con 0x7ff5b40b2d20 2024-07-04T17:32:53.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.156+0000 7ff5b9c00640 1 -- 172.21.15.42:0/2013460308 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5b406c710 con 0x7ff5b40b2d20 2024-07-04T17:32:53.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.157+0000 7ff5aaa00640 1 -- 172.21.15.42:0/2013460308 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+6042 (secure 0 0 0) 0x7ff5a4016170 con 0x7ff5b40b2d20 2024-07-04T17:32:53.159 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:32:53.159 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":81,"btime":"2024-07-04T17:32:49:450042+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/244742173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":244742173},{"type":"v1","addr":"172.21.15.42:6825","nonce":244742173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5265,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/973681058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":973681058},{"type":"v1","addr":"172.21.15.42:6827","nonce":973681058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/480649366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":480649366},{"type":"v1","addr":"172.21.15.42:6829","nonce":480649366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5477,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6833/2552884551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2552884551},{"type":"v1","addr":"172.21.15.42:6833","nonce":2552884551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:32:02.103946+0000","modified":"2024-07-04T17:32:05.184521+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5276":{"gid":5276,"name":"d","rank":0,"incarnation":73,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6831/327636003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":327636003},{"type":"v1","addr":"172.21.15.42:6831","nonce":327636003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5276,"qdb_cluster":[5276]},"id":9}]} 2024-07-04T17:32:53.159 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 81 2024-07-04T17:32:53.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.160+0000 7ff5b9c00640 1 -- 172.21.15.42:0/2013460308 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7ff59003d4f0 msgr2=0x7ff59003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:53.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.160+0000 7ff5b9c00640 1 --2- 172.21.15.42:0/2013460308 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7ff59003d4f0 0x7ff59003fa00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff59c033000 tx=0x7ff59c004730 comp rx=0 tx=0).stop 2024-07-04T17:32:53.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.160+0000 7ff5b9c00640 1 -- 172.21.15.42:0/2013460308 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5b40b2d20 msgr2=0x7ff5b413c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:53.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.160+0000 7ff5b9c00640 1 --2- 172.21.15.42:0/2013460308 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5b40b2d20 0x7ff5b413c530 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7ff5a400c380 tx=0x7ff5a400c840 comp rx=0 tx=0).stop 2024-07-04T17:32:53.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.160+0000 7ff5b9c00640 1 -- 172.21.15.42:0/2013460308 shutdown_connections 2024-07-04T17:32:53.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.160+0000 7ff5b9c00640 1 --2- 172.21.15.42:0/2013460308 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7ff59003d4f0 0x7ff59003fa00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:53.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.160+0000 7ff5b9c00640 1 --2- 172.21.15.42:0/2013460308 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5b40b2d20 0x7ff5b413c530 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:53.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.160+0000 7ff5b9c00640 1 -- 172.21.15.42:0/2013460308 >> 172.21.15.42:0/2013460308 conn(0x7ff5b40bf700 msgr2=0x7ff5b4075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:53.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.160+0000 7ff5b9c00640 1 -- 172.21.15.42:0/2013460308 shutdown_connections 2024-07-04T17:32:53.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.161+0000 7ff5b9c00640 1 -- 172.21.15.42:0/2013460308 wait complete. 2024-07-04T17:32:53.186 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:32:53.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.469+0000 7ff2b1400640 1 -- 172.21.15.42:0/1010850099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2ac06c020 msgr2=0x7ff2ac14a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:53.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.469+0000 7ff2b1400640 1 --2- 172.21.15.42:0/1010850099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2ac06c020 0x7ff2ac14a9e0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7ff298005fd0 tx=0x7ff29802a5f0 comp rx=0 tx=0).stop 2024-07-04T17:32:53.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.469+0000 7ff2b1400640 1 -- 172.21.15.42:0/1010850099 shutdown_connections 2024-07-04T17:32:53.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.469+0000 7ff2b1400640 1 --2- 172.21.15.42:0/1010850099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2ac06c020 0x7ff2ac14a9e0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:53.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.469+0000 7ff2b1400640 1 -- 172.21.15.42:0/1010850099 >> 172.21.15.42:0/1010850099 conn(0x7ff2ac0bebc0 msgr2=0x7ff2ac0befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:53.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.469+0000 7ff2b1400640 1 -- 172.21.15.42:0/1010850099 shutdown_connections 2024-07-04T17:32:53.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.469+0000 7ff2b1400640 1 -- 172.21.15.42:0/1010850099 wait complete. 2024-07-04T17:32:53.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.471+0000 7ff2b1400640 1 Processor -- start 2024-07-04T17:32:53.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.471+0000 7ff2b1400640 1 -- start start 2024-07-04T17:32:53.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.473+0000 7ff2b1400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2ac06c020 0x7ff2ac1d5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:53.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.473+0000 7ff2b1400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ac14b550 con 0x7ff2ac06c020 2024-07-04T17:32:53.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.473+0000 7ff2abe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2ac06c020 0x7ff2ac1d5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:53.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.473+0000 7ff2abe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2ac06c020 0x7ff2ac1d5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38180/0 (socket says 172.21.15.42:38180) 2024-07-04T17:32:53.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.473+0000 7ff2abe00640 1 -- 172.21.15.42:0/2603060081 learned_addr learned my addr 172.21.15.42:0/2603060081 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:53.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.473+0000 7ff2abe00640 1 -- 172.21.15.42:0/2603060081 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2ac1d8fc0 con 0x7ff2ac06c020 2024-07-04T17:32:53.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.474+0000 7ff2abe00640 1 --2- 172.21.15.42:0/2603060081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2ac06c020 0x7ff2ac1d5a60 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7ff29c001eb0 tx=0x7ff29c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:53.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.474+0000 7ff2a3e00640 1 -- 172.21.15.42:0/2603060081 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff29c017070 con 0x7ff2ac06c020 2024-07-04T17:32:53.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.474+0000 7ff2a3e00640 1 -- 172.21.15.42:0/2603060081 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff29c0052f0 con 0x7ff2ac06c020 2024-07-04T17:32:53.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.474+0000 7ff2a3e00640 1 -- 172.21.15.42:0/2603060081 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff29c0035f0 con 0x7ff2ac06c020 2024-07-04T17:32:53.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.474+0000 7ff2b1400640 1 -- 172.21.15.42:0/2603060081 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2ac0b1070 con 0x7ff2ac06c020 2024-07-04T17:32:53.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.474+0000 7ff2b1400640 1 -- 172.21.15.42:0/2603060081 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2ac1d7c80 con 0x7ff2ac06c020 2024-07-04T17:32:53.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.475+0000 7ff2b1400640 1 -- 172.21.15.42:0/2603060081 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2ac14bc80 con 0x7ff2ac06c020 2024-07-04T17:32:53.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.476+0000 7ff2a3e00640 1 -- 172.21.15.42:0/2603060081 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7ff29c01a070 con 0x7ff2ac06c020 2024-07-04T17:32:53.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.476+0000 7ff2a3e00640 1 --2- 172.21.15.42:0/2603060081 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7ff28c03d4f0 0x7ff28c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:53.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.476+0000 7ff2a3e00640 1 -- 172.21.15.42:0/2603060081 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 4397+0+0 (secure 0 0 0) 0x7ff29c050f20 con 0x7ff2ac06c020 2024-07-04T17:32:53.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.476+0000 7ff2ab400640 1 --2- 172.21.15.42:0/2603060081 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7ff28c03d4f0 0x7ff28c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:53.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.477+0000 7ff2ab400640 1 --2- 172.21.15.42:0/2603060081 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7ff28c03d4f0 0x7ff28c03fa00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff298033000 tx=0x7ff298004730 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:53.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.481+0000 7ff2a3e00640 1 -- 172.21.15.42:0/2603060081 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7ff29c0102e0 con 0x7ff2ac06c020 2024-07-04T17:32:53.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.733+0000 7ff2b1400640 1 -- 172.21.15.42:0/2603060081 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2ac14f540 con 0x7ff2ac06c020 2024-07-04T17:32:53.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.734+0000 7ff2a3e00640 1 -- 172.21.15.42:0/2603060081 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+6042 (secure 0 0 0) 0x7ff29c011910 con 0x7ff2ac06c020 2024-07-04T17:32:53.735 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:32:53.735 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":81,"btime":"2024-07-04T17:32:49:450042+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/244742173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":244742173},{"type":"v1","addr":"172.21.15.42:6825","nonce":244742173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5265,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/973681058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":973681058},{"type":"v1","addr":"172.21.15.42:6827","nonce":973681058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/480649366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":480649366},{"type":"v1","addr":"172.21.15.42:6829","nonce":480649366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5477,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6833/2552884551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2552884551},{"type":"v1","addr":"172.21.15.42:6833","nonce":2552884551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:32:02.103946+0000","modified":"2024-07-04T17:32:05.184521+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5276":{"gid":5276,"name":"d","rank":0,"incarnation":73,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6831/327636003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":327636003},{"type":"v1","addr":"172.21.15.42:6831","nonce":327636003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5276,"qdb_cluster":[5276]},"id":9}]} 2024-07-04T17:32:53.736 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 81 2024-07-04T17:32:53.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.736+0000 7ff2b1400640 1 -- 172.21.15.42:0/2603060081 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7ff28c03d4f0 msgr2=0x7ff28c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:53.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.737+0000 7ff2b1400640 1 --2- 172.21.15.42:0/2603060081 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7ff28c03d4f0 0x7ff28c03fa00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff298033000 tx=0x7ff298004730 comp rx=0 tx=0).stop 2024-07-04T17:32:53.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.737+0000 7ff2b1400640 1 -- 172.21.15.42:0/2603060081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2ac06c020 msgr2=0x7ff2ac1d5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:53.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.737+0000 7ff2b1400640 1 --2- 172.21.15.42:0/2603060081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2ac06c020 0x7ff2ac1d5a60 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7ff29c001eb0 tx=0x7ff29c00c810 comp rx=0 tx=0).stop 2024-07-04T17:32:53.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.737+0000 7ff2b1400640 1 -- 172.21.15.42:0/2603060081 shutdown_connections 2024-07-04T17:32:53.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.737+0000 7ff2b1400640 1 --2- 172.21.15.42:0/2603060081 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7ff28c03d4f0 0x7ff28c03fa00 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:53.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.737+0000 7ff2b1400640 1 --2- 172.21.15.42:0/2603060081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2ac06c020 0x7ff2ac1d5a60 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:53.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.737+0000 7ff2b1400640 1 -- 172.21.15.42:0/2603060081 >> 172.21.15.42:0/2603060081 conn(0x7ff2ac0bebc0 msgr2=0x7ff2ac0b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:53.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.737+0000 7ff2b1400640 1 -- 172.21.15.42:0/2603060081 shutdown_connections 2024-07-04T17:32:53.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:53.737+0000 7ff2b1400640 1 -- 172.21.15.42:0/2603060081 wait complete. 2024-07-04T17:32:53.762 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:32:54.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.046+0000 7f4560400640 1 -- 172.21.15.42:0/4032895976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45580b38b0 msgr2=0x7f45580b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:54.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.046+0000 7f4560400640 1 --2- 172.21.15.42:0/4032895976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45580b38b0 0x7f45580b2d20 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f4550005fd0 tx=0x7f455002aac0 comp rx=0 tx=0).stop 2024-07-04T17:32:54.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.046+0000 7f4560400640 1 -- 172.21.15.42:0/4032895976 shutdown_connections 2024-07-04T17:32:54.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.046+0000 7f4560400640 1 --2- 172.21.15.42:0/4032895976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45580b38b0 0x7f45580b2d20 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:54.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.046+0000 7f4560400640 1 -- 172.21.15.42:0/4032895976 >> 172.21.15.42:0/4032895976 conn(0x7f45580bf700 msgr2=0x7f45580af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:54.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.046+0000 7f4560400640 1 -- 172.21.15.42:0/4032895976 shutdown_connections 2024-07-04T17:32:54.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.046+0000 7f4560400640 1 -- 172.21.15.42:0/4032895976 wait complete. 2024-07-04T17:32:54.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.049+0000 7f4560400640 1 Processor -- start 2024-07-04T17:32:54.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.049+0000 7f4560400640 1 -- start start 2024-07-04T17:32:54.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.051+0000 7f4560400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45580b1e00 0x7f45580aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:54.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.051+0000 7f4560400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f45580b06e0 con 0x7f45580b1e00 2024-07-04T17:32:54.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.052+0000 7f455da00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45580b1e00 0x7f45580aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:54.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.052+0000 7f455da00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45580b1e00 0x7f45580aedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35228/0 (socket says 172.21.15.42:35228) 2024-07-04T17:32:54.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.052+0000 7f455da00640 1 -- 172.21.15.42:0/600630599 learned_addr learned my addr 172.21.15.42:0/600630599 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:54.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.052+0000 7f455da00640 1 -- 172.21.15.42:0/600630599 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45580ac720 con 0x7f45580b1e00 2024-07-04T17:32:54.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.052+0000 7f455da00640 1 --2- 172.21.15.42:0/600630599 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45580b1e00 0x7f45580aedb0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f4550006010 tx=0x7f4550003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:54.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.053+0000 7f4555600640 1 -- 172.21.15.42:0/600630599 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4550044070 con 0x7f45580b1e00 2024-07-04T17:32:54.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.053+0000 7f4555600640 1 -- 172.21.15.42:0/600630599 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4550004250 con 0x7f45580b1e00 2024-07-04T17:32:54.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.053+0000 7f4555600640 1 -- 172.21.15.42:0/600630599 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45500045d0 con 0x7f45580b1e00 2024-07-04T17:32:54.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.053+0000 7f4560400640 1 -- 172.21.15.42:0/600630599 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45580ae1a0 con 0x7f45580b1e00 2024-07-04T17:32:54.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.053+0000 7f4560400640 1 -- 172.21.15.42:0/600630599 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45580acc20 con 0x7f45580b1e00 2024-07-04T17:32:54.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.054+0000 7f4560400640 1 -- 172.21.15.42:0/600630599 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45580ace50 con 0x7f45580b1e00 2024-07-04T17:32:54.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.054+0000 7f4555600640 1 -- 172.21.15.42:0/600630599 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7f4550047070 con 0x7f45580b1e00 2024-07-04T17:32:54.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.055+0000 7f4555600640 1 --2- 172.21.15.42:0/600630599 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f453803d4f0 0x7f453803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:54.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.055+0000 7f4555600640 1 -- 172.21.15.42:0/600630599 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 4397+0+0 (secure 0 0 0) 0x7f455003c070 con 0x7f45580b1e00 2024-07-04T17:32:54.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.055+0000 7f455d000640 1 --2- 172.21.15.42:0/600630599 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f453803d4f0 0x7f453803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:54.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.056+0000 7f455d000640 1 --2- 172.21.15.42:0/600630599 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f453803d4f0 0x7f453803fa00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f455806ed80 tx=0x7f45440079f0 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:54.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.060+0000 7f4555600640 1 -- 172.21.15.42:0/600630599 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f4550032030 con 0x7f45580b1e00 2024-07-04T17:32:54.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.272+0000 7f4560400640 1 -- 172.21.15.42:0/600630599 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f45580af560 con 0x7f45580b1e00 2024-07-04T17:32:54.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.273+0000 7f4555600640 1 -- 172.21.15.42:0/600630599 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+11997 (secure 0 0 0) 0x7f4550047360 con 0x7f45580b1e00 2024-07-04T17:32:54.274 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:32:54.275 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":102,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:32:51.468757+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:31:59.635881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.4099999666213989,"score_stable":1.4099999666213989,"optimal_score":1,"raw_score_acting":1.4099999666213989,"raw_score_stable":1.4099999666213989,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:32:01.086110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.0800000429153442,"score_stable":1.0800000429153442,"optimal_score":1,"raw_score_acting":1.0800000429153442,"raw_score_stable":1.0800000429153442,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6832/3777807763":"2024-07-05T17:32:48.292733+0000","172.21.15.42:6835/1323178695":"2024-07-05T17:32:47.238337+0000","172.21.15.42:6834/1323178695":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/3479187327":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/143103576":"2024-07-05T17:32:47.238337+0000","172.21.15.42:6833/3777807763":"2024-07-05T17:32:48.292733+0000","172.21.15.42:0/1626565787":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/1439263513":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/2943589644":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/2854152871":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/4179523139":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/2096108670":"2024-07-04T18:32:31.377755+0000","172.21.15.42:0/1641580783":"2024-07-05T17:32:15.307330+0000","172.21.15.42:0/3208988074":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/1212024809":"2024-07-05T17:32:15.307330+0000","172.21.15.42:0/1089649627":"2024-07-05T17:32:15.307330+0000","172.21.15.42:0/257615823":"2024-07-05T17:32:15.307330+0000","172.21.15.42:0/2624550914":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/2580772079":"2024-07-05T17:32:15.307330+0000","172.21.15.42:0/611598150":"2024-07-05T17:32:15.307330+0000","172.21.15.42:6835/1139872311":"2024-07-05T17:32:15.307330+0000","172.21.15.42:6834/1139872311":"2024-07-05T17:32:15.307330+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:32:54.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.276+0000 7f4560400640 1 -- 172.21.15.42:0/600630599 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f453803d4f0 msgr2=0x7f453803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:54.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.276+0000 7f4560400640 1 --2- 172.21.15.42:0/600630599 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f453803d4f0 0x7f453803fa00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f455806ed80 tx=0x7f45440079f0 comp rx=0 tx=0).stop 2024-07-04T17:32:54.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.276+0000 7f4560400640 1 -- 172.21.15.42:0/600630599 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45580b1e00 msgr2=0x7f45580aedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:54.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.276+0000 7f4560400640 1 --2- 172.21.15.42:0/600630599 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45580b1e00 0x7f45580aedb0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f4550006010 tx=0x7f4550003f10 comp rx=0 tx=0).stop 2024-07-04T17:32:54.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.276+0000 7f4560400640 1 -- 172.21.15.42:0/600630599 shutdown_connections 2024-07-04T17:32:54.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.276+0000 7f4560400640 1 --2- 172.21.15.42:0/600630599 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f453803d4f0 0x7f453803fa00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:54.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.276+0000 7f4560400640 1 --2- 172.21.15.42:0/600630599 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45580b1e00 0x7f45580aedb0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:54.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.276+0000 7f4560400640 1 -- 172.21.15.42:0/600630599 >> 172.21.15.42:0/600630599 conn(0x7f45580bf700 msgr2=0x7f45580b0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:54.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.276+0000 7f4560400640 1 -- 172.21.15.42:0/600630599 shutdown_connections 2024-07-04T17:32:54.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.276+0000 7f4560400640 1 -- 172.21.15.42:0/600630599 wait complete. 2024-07-04T17:32:54.304 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail backup_fs 2024-07-04T17:32:54.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.598+0000 7f1a07e00640 1 -- 172.21.15.42:0/230150378 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a0806c2c0 msgr2=0x7f1a080b5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:54.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.598+0000 7f1a07e00640 1 --2- 172.21.15.42:0/230150378 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a0806c2c0 0x7f1a080b5650 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f19f0005fd0 tx=0x7f19f002ae00 comp rx=0 tx=0).stop 2024-07-04T17:32:54.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.598+0000 7f1a07e00640 1 -- 172.21.15.42:0/230150378 shutdown_connections 2024-07-04T17:32:54.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.598+0000 7f1a07e00640 1 --2- 172.21.15.42:0/230150378 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a0806c2c0 0x7f1a080b5650 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:54.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.598+0000 7f1a07e00640 1 -- 172.21.15.42:0/230150378 >> 172.21.15.42:0/230150378 conn(0x7f1a080bebc0 msgr2=0x7f1a080befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:54.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.599+0000 7f1a07e00640 1 -- 172.21.15.42:0/230150378 shutdown_connections 2024-07-04T17:32:54.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.599+0000 7f1a07e00640 1 -- 172.21.15.42:0/230150378 wait complete. 2024-07-04T17:32:54.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.601+0000 7f1a07e00640 1 Processor -- start 2024-07-04T17:32:54.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.602+0000 7f1a07e00640 1 -- start start 2024-07-04T17:32:54.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.603+0000 7f1a07e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a0806c2c0 0x7f1a081d3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:54.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.603+0000 7f1a07e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1a080b2810 con 0x7f1a0806c2c0 2024-07-04T17:32:54.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.604+0000 7f1a06a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a0806c2c0 0x7f1a081d3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:54.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.604+0000 7f1a06a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a0806c2c0 0x7f1a081d3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35238/0 (socket says 172.21.15.42:35238) 2024-07-04T17:32:54.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.604+0000 7f1a06a00640 1 -- 172.21.15.42:0/1501369179 learned_addr learned my addr 172.21.15.42:0/1501369179 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:54.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.604+0000 7f1a06a00640 1 -- 172.21.15.42:0/1501369179 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a081d32b0 con 0x7f1a0806c2c0 2024-07-04T17:32:54.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.604+0000 7f1a06a00640 1 --2- 172.21.15.42:0/1501369179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a0806c2c0 0x7f1a081d3770 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f19f800c700 tx=0x7f19f800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:54.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.605+0000 7f19ffe00640 1 -- 172.21.15.42:0/1501369179 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19f8003970 con 0x7f1a0806c2c0 2024-07-04T17:32:54.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.605+0000 7f19ffe00640 1 -- 172.21.15.42:0/1501369179 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19f8003b10 con 0x7f1a0806c2c0 2024-07-04T17:32:54.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.605+0000 7f19ffe00640 1 -- 172.21.15.42:0/1501369179 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19f800a820 con 0x7f1a0806c2c0 2024-07-04T17:32:54.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.605+0000 7f1a07e00640 1 -- 172.21.15.42:0/1501369179 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a081e75f0 con 0x7f1a0806c2c0 2024-07-04T17:32:54.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.605+0000 7f1a07e00640 1 -- 172.21.15.42:0/1501369179 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a0813f3a0 con 0x7f1a0806c2c0 2024-07-04T17:32:54.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.606+0000 7f19ffe00640 1 -- 172.21.15.42:0/1501369179 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7f19f800a9c0 con 0x7f1a0806c2c0 2024-07-04T17:32:54.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.606+0000 7f1a07e00640 1 -- 172.21.15.42:0/1501369179 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a080b5650 con 0x7f1a0806c2c0 2024-07-04T17:32:54.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.607+0000 7f19ffe00640 1 --2- 172.21.15.42:0/1501369179 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f19e403d4f0 0x7f19e403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:54.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.607+0000 7f19ffe00640 1 -- 172.21.15.42:0/1501369179 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 4397+0+0 (secure 0 0 0) 0x7f19f8051ba0 con 0x7f1a0806c2c0 2024-07-04T17:32:54.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.607+0000 7f19fe000640 1 --2- 172.21.15.42:0/1501369179 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f19e403d4f0 0x7f19e403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:54.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.608+0000 7f19fe000640 1 --2- 172.21.15.42:0/1501369179 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f19e403d4f0 0x7f19e403fa00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f19f0034000 tx=0x7f19f0004730 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:54.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.612+0000 7f19ffe00640 1 -- 172.21.15.42:0/1501369179 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f1a0806c6d0 con 0x7f1a0806c2c0 2024-07-04T17:32:54.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.863+0000 7f1a07e00640 1 -- 172.21.15.42:0/1501369179 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "backup_fs"} v 0) -- 0x7f1a080b1550 con 0x7f1a0806c2c0 2024-07-04T17:32:54.865 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-07-04T17:32:54.864+0000 7f8616e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-04T17:32:54.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.867+0000 7f19ffe00640 1 -- 172.21.15.42:0/1501369179 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "backup_fs"}]=0 backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v82) ==== 167+0+0 (secure 0 0 0) 0x7f19f800d150 con 0x7f1a0806c2c0 2024-07-04T17:32:54.868 INFO:teuthology.orchestra.run.smithi042.stderr:backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-04T17:32:54.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.870+0000 7f1a07e00640 1 -- 172.21.15.42:0/1501369179 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f19e403d4f0 msgr2=0x7f19e403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:54.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.870+0000 7f1a07e00640 1 --2- 172.21.15.42:0/1501369179 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f19e403d4f0 0x7f19e403fa00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f19f0034000 tx=0x7f19f0004730 comp rx=0 tx=0).stop 2024-07-04T17:32:54.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.870+0000 7f1a07e00640 1 -- 172.21.15.42:0/1501369179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a0806c2c0 msgr2=0x7f1a081d3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:54.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.870+0000 7f1a07e00640 1 --2- 172.21.15.42:0/1501369179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a0806c2c0 0x7f1a081d3770 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f19f800c700 tx=0x7f19f800cbc0 comp rx=0 tx=0).stop 2024-07-04T17:32:54.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.870+0000 7f1a07e00640 1 -- 172.21.15.42:0/1501369179 shutdown_connections 2024-07-04T17:32:54.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.870+0000 7f1a07e00640 1 --2- 172.21.15.42:0/1501369179 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f19e403d4f0 0x7f19e403fa00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:54.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.870+0000 7f1a07e00640 1 --2- 172.21.15.42:0/1501369179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a0806c2c0 0x7f1a081d3770 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:54.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.870+0000 7f1a07e00640 1 -- 172.21.15.42:0/1501369179 >> 172.21.15.42:0/1501369179 conn(0x7f1a080bebc0 msgr2=0x7f1a08142c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:54.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.871+0000 7f1a07e00640 1 -- 172.21.15.42:0/1501369179 shutdown_connections 2024-07-04T17:32:54.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:54.871+0000 7f1a07e00640 1 -- 172.21.15.42:0/1501369179 wait complete. 2024-07-04T17:32:54.884 INFO:tasks.ceph.mds.d.smithi042.stderr: -3219> 2024-07-04T17:32:04.039+0000 7f50a6600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-04T17:32:54.898 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm backup_fs --yes-i-really-mean-it 2024-07-04T17:32:55.031 INFO:tasks.ceph.mds.d.smithi042.stdout:starting mds.d at 2024-07-04T17:32:55.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.159+0000 7fbd4b200640 1 -- 172.21.15.42:0/812014594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd4c0ac5b0 msgr2=0x7fbd4c0b0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:55.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.159+0000 7fbd4b200640 1 --2- 172.21.15.42:0/812014594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd4c0ac5b0 0x7fbd4c0b0db0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c009230 tx=0x7fbd3c02a480 comp rx=0 tx=0).stop 2024-07-04T17:32:55.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.159+0000 7fbd4b200640 1 -- 172.21.15.42:0/812014594 shutdown_connections 2024-07-04T17:32:55.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.159+0000 7fbd4b200640 1 --2- 172.21.15.42:0/812014594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd4c0ac5b0 0x7fbd4c0b0db0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:55.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.159+0000 7fbd4b200640 1 -- 172.21.15.42:0/812014594 >> 172.21.15.42:0/812014594 conn(0x7fbd4c0afaa0 msgr2=0x7fbd4c0afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:55.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.159+0000 7fbd4b200640 1 -- 172.21.15.42:0/812014594 shutdown_connections 2024-07-04T17:32:55.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.159+0000 7fbd4b200640 1 -- 172.21.15.42:0/812014594 wait complete. 2024-07-04T17:32:55.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.161+0000 7fbd4b200640 1 Processor -- start 2024-07-04T17:32:55.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.161+0000 7fbd4b200640 1 -- start start 2024-07-04T17:32:55.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.163+0000 7fbd4b200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd4c0ac5b0 0x7fbd4c0b5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:55.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.163+0000 7fbd4b200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c0ae640 con 0x7fbd4c0ac5b0 2024-07-04T17:32:55.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.163+0000 7fbd49e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd4c0ac5b0 0x7fbd4c0b5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:55.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.163+0000 7fbd49e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd4c0ac5b0 0x7fbd4c0b5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35276/0 (socket says 172.21.15.42:35276) 2024-07-04T17:32:55.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.163+0000 7fbd49e00640 1 -- 172.21.15.42:0/3515864179 learned_addr learned my addr 172.21.15.42:0/3515864179 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:55.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.163+0000 7fbd49e00640 1 -- 172.21.15.42:0/3515864179 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4c1d8cb0 con 0x7fbd4c0ac5b0 2024-07-04T17:32:55.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.164+0000 7fbd49e00640 1 --2- 172.21.15.42:0/3515864179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd4c0ac5b0 0x7fbd4c0b5ce0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c009290 tx=0x7fbd3c033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:55.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.164+0000 7fbd42000640 1 -- 172.21.15.42:0/3515864179 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd3c0319b0 con 0x7fbd4c0ac5b0 2024-07-04T17:32:55.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.164+0000 7fbd42000640 1 -- 172.21.15.42:0/3515864179 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd3c031c40 con 0x7fbd4c0ac5b0 2024-07-04T17:32:55.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.164+0000 7fbd42000640 1 -- 172.21.15.42:0/3515864179 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd3c033070 con 0x7fbd4c0ac5b0 2024-07-04T17:32:55.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.164+0000 7fbd4b200640 1 -- 172.21.15.42:0/3515864179 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd4c1d9190 con 0x7fbd4c0ac5b0 2024-07-04T17:32:55.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.164+0000 7fbd4b200640 1 -- 172.21.15.42:0/3515864179 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd4c13d360 con 0x7fbd4c0ac5b0 2024-07-04T17:32:55.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.165+0000 7fbd4b200640 1 -- 172.21.15.42:0/3515864179 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd4c0b0db0 con 0x7fbd4c0ac5b0 2024-07-04T17:32:55.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.166+0000 7fbd42000640 1 -- 172.21.15.42:0/3515864179 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7fbd3c033210 con 0x7fbd4c0ac5b0 2024-07-04T17:32:55.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.166+0000 7fbd42000640 1 --2- 172.21.15.42:0/3515864179 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fbd2803d4f0 0x7fbd2803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:55.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.166+0000 7fbd42000640 1 -- 172.21.15.42:0/3515864179 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 4483+0+0 (secure 0 0 0) 0x7fbd3c073170 con 0x7fbd4c0ac5b0 2024-07-04T17:32:55.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.166+0000 7fbd49400640 1 --2- 172.21.15.42:0/3515864179 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fbd2803d4f0 0x7fbd2803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:55.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.167+0000 7fbd49400640 1 --2- 172.21.15.42:0/3515864179 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fbd2803d4f0 0x7fbd2803fa00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbd34002020 tx=0x7fbd34001670 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.172+0000 7fbd42000640 1 -- 172.21.15.42:0/3515864179 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fbd3c03f030 con 0x7fbd4c0ac5b0 2024-07-04T17:32:55.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.418+0000 7fbd4b200640 1 -- 172.21.15.42:0/3515864179 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true} v 0) -- 0x7fbd4c1d8e80 con 0x7fbd4c0ac5b0 2024-07-04T17:32:55.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.869+0000 7fbd42000640 1 -- 172.21.15.42:0/3515864179 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true}]=0 v83) ==== 107+0+0 (secure 0 0 0) 0x7fbd3c048df0 con 0x7fbd4c0ac5b0 2024-07-04T17:32:55.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.872+0000 7fbd4b200640 1 -- 172.21.15.42:0/3515864179 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fbd2803d4f0 msgr2=0x7fbd2803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:55.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.872+0000 7fbd4b200640 1 --2- 172.21.15.42:0/3515864179 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fbd2803d4f0 0x7fbd2803fa00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbd34002020 tx=0x7fbd34001670 comp rx=0 tx=0).stop 2024-07-04T17:32:55.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.872+0000 7fbd4b200640 1 -- 172.21.15.42:0/3515864179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd4c0ac5b0 msgr2=0x7fbd4c0b5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:55.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.872+0000 7fbd4b200640 1 --2- 172.21.15.42:0/3515864179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd4c0ac5b0 0x7fbd4c0b5ce0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c009290 tx=0x7fbd3c033ad0 comp rx=0 tx=0).stop 2024-07-04T17:32:55.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.872+0000 7fbd4b200640 1 -- 172.21.15.42:0/3515864179 shutdown_connections 2024-07-04T17:32:55.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.872+0000 7fbd4b200640 1 --2- 172.21.15.42:0/3515864179 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fbd2803d4f0 0x7fbd2803fa00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:55.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.872+0000 7fbd4b200640 1 --2- 172.21.15.42:0/3515864179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd4c0ac5b0 0x7fbd4c0b5ce0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:55.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.872+0000 7fbd4b200640 1 -- 172.21.15.42:0/3515864179 >> 172.21.15.42:0/3515864179 conn(0x7fbd4c0afaa0 msgr2=0x7fbd4c0aec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:55.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.872+0000 7fbd4b200640 1 -- 172.21.15.42:0/3515864179 shutdown_connections 2024-07-04T17:32:55.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:55.872+0000 7fbd4b200640 1 -- 172.21.15.42:0/3515864179 wait complete. 2024-07-04T17:32:55.905 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_metadata backup_fs_metadata --yes-i-really-really-mean-it 2024-07-04T17:32:56.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.198+0000 7fd96e200640 1 -- 172.21.15.42:0/3682015954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9680abe40 msgr2=0x7fd9680aa900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:56.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.198+0000 7fd96e200640 1 --2- 172.21.15.42:0/3682015954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9680abe40 0x7fd9680aa900 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fd954005fd0 tx=0x7fd95402a560 comp rx=0 tx=0).stop 2024-07-04T17:32:56.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.198+0000 7fd96e200640 1 -- 172.21.15.42:0/3682015954 shutdown_connections 2024-07-04T17:32:56.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.198+0000 7fd96e200640 1 --2- 172.21.15.42:0/3682015954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9680abe40 0x7fd9680aa900 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:56.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.198+0000 7fd96e200640 1 -- 172.21.15.42:0/3682015954 >> 172.21.15.42:0/3682015954 conn(0x7fd9680afaa0 msgr2=0x7fd9680afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:56.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.198+0000 7fd96e200640 1 -- 172.21.15.42:0/3682015954 shutdown_connections 2024-07-04T17:32:56.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.198+0000 7fd96e200640 1 -- 172.21.15.42:0/3682015954 wait complete. 2024-07-04T17:32:56.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.200+0000 7fd96e200640 1 Processor -- start 2024-07-04T17:32:56.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.200+0000 7fd96e200640 1 -- start start 2024-07-04T17:32:56.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.202+0000 7fd96e200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9680701d0 0x7fd9680adb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:56.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.202+0000 7fd96e200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd968073ed0 con 0x7fd9680701d0 2024-07-04T17:32:56.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.202+0000 7fd96ce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9680701d0 0x7fd9680adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:56.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.202+0000 7fd96ce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9680701d0 0x7fd9680adb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35302/0 (socket says 172.21.15.42:35302) 2024-07-04T17:32:56.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.202+0000 7fd96ce00640 1 -- 172.21.15.42:0/2110131777 learned_addr learned my addr 172.21.15.42:0/2110131777 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:56.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.202+0000 7fd96ce00640 1 -- 172.21.15.42:0/2110131777 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9680ad3f0 con 0x7fd9680701d0 2024-07-04T17:32:56.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.203+0000 7fd96ce00640 1 --2- 172.21.15.42:0/2110131777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9680701d0 0x7fd9680adb00 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00b340 tx=0x7fd95c00b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:56.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.203+0000 7fd964c00640 1 -- 172.21.15.42:0/2110131777 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd95c015030 con 0x7fd9680701d0 2024-07-04T17:32:56.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.203+0000 7fd964c00640 1 -- 172.21.15.42:0/2110131777 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd95c004240 con 0x7fd9680701d0 2024-07-04T17:32:56.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.203+0000 7fd96e200640 1 -- 172.21.15.42:0/2110131777 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9681499c0 con 0x7fd9680701d0 2024-07-04T17:32:56.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.203+0000 7fd964c00640 1 -- 172.21.15.42:0/2110131777 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd95c011630 con 0x7fd9680701d0 2024-07-04T17:32:56.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.203+0000 7fd96e200640 1 -- 172.21.15.42:0/2110131777 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9680ad0e0 con 0x7fd9680701d0 2024-07-04T17:32:56.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.205+0000 7fd96e200640 1 -- 172.21.15.42:0/2110131777 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9680008d0 con 0x7fd9680701d0 2024-07-04T17:32:56.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.205+0000 7fd964c00640 1 -- 172.21.15.42:0/2110131777 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7fd95c018070 con 0x7fd9680701d0 2024-07-04T17:32:56.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.205+0000 7fd964c00640 1 --2- 172.21.15.42:0/2110131777 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fd94803d4f0 0x7fd94803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:56.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.205+0000 7fd964c00640 1 -- 172.21.15.42:0/2110131777 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 4483+0+0 (secure 0 0 0) 0x7fd95c016070 con 0x7fd9680701d0 2024-07-04T17:32:56.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.206+0000 7fd967e00640 1 --2- 172.21.15.42:0/2110131777 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fd94803d4f0 0x7fd94803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:56.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.206+0000 7fd967e00640 1 --2- 172.21.15.42:0/2110131777 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fd94803d4f0 0x7fd94803fa00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd954033000 tx=0x7fd954004730 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:56.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.211+0000 7fd964c00640 1 -- 172.21.15.42:0/2110131777 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fd95c015e00 con 0x7fd9680701d0 2024-07-04T17:32:56.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.424+0000 7fd96e200640 1 -- 172.21.15.42:0/2110131777 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd96814a2a0 con 0x7fd9680701d0 2024-07-04T17:32:56.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.879+0000 7fd964c00640 1 -- 172.21.15.42:0/2110131777 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_metadata' removed v104) ==== 190+0+0 (secure 0 0 0) 0x7fd95c010ca0 con 0x7fd9680701d0 2024-07-04T17:32:56.881 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'backup_fs_metadata' removed 2024-07-04T17:32:56.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.883+0000 7fd96e200640 1 -- 172.21.15.42:0/2110131777 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fd94803d4f0 msgr2=0x7fd94803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:56.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.883+0000 7fd96e200640 1 --2- 172.21.15.42:0/2110131777 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fd94803d4f0 0x7fd94803fa00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd954033000 tx=0x7fd954004730 comp rx=0 tx=0).stop 2024-07-04T17:32:56.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.883+0000 7fd96e200640 1 -- 172.21.15.42:0/2110131777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9680701d0 msgr2=0x7fd9680adb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:56.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.883+0000 7fd96e200640 1 --2- 172.21.15.42:0/2110131777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9680701d0 0x7fd9680adb00 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00b340 tx=0x7fd95c00b800 comp rx=0 tx=0).stop 2024-07-04T17:32:56.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.884+0000 7fd96e200640 1 -- 172.21.15.42:0/2110131777 shutdown_connections 2024-07-04T17:32:56.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.884+0000 7fd96e200640 1 --2- 172.21.15.42:0/2110131777 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fd94803d4f0 0x7fd94803fa00 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:56.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.884+0000 7fd96e200640 1 --2- 172.21.15.42:0/2110131777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9680701d0 0x7fd9680adb00 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:56.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.884+0000 7fd96e200640 1 -- 172.21.15.42:0/2110131777 >> 172.21.15.42:0/2110131777 conn(0x7fd9680afaa0 msgr2=0x7fd9681d6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:56.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.884+0000 7fd96e200640 1 -- 172.21.15.42:0/2110131777 shutdown_connections 2024-07-04T17:32:56.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:56.884+0000 7fd96e200640 1 -- 172.21.15.42:0/2110131777 wait complete. 2024-07-04T17:32:56.924 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_data backup_fs_data --yes-i-really-really-mean-it 2024-07-04T17:32:57.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.209+0000 7fa861c00640 1 -- 172.21.15.42:0/138881409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa85c0a94d0 msgr2=0x7fa85c0b1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:57.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.209+0000 7fa861c00640 1 --2- 172.21.15.42:0/138881409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa85c0a94d0 0x7fa85c0b1100 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fa84c009230 tx=0x7fa84c02a480 comp rx=0 tx=0).stop 2024-07-04T17:32:57.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.209+0000 7fa861c00640 1 -- 172.21.15.42:0/138881409 shutdown_connections 2024-07-04T17:32:57.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.209+0000 7fa861c00640 1 --2- 172.21.15.42:0/138881409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa85c0a94d0 0x7fa85c0b1100 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:57.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.209+0000 7fa861c00640 1 -- 172.21.15.42:0/138881409 >> 172.21.15.42:0/138881409 conn(0x7fa85c06c020 msgr2=0x7fa85c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:57.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.210+0000 7fa861c00640 1 -- 172.21.15.42:0/138881409 shutdown_connections 2024-07-04T17:32:57.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.210+0000 7fa861c00640 1 -- 172.21.15.42:0/138881409 wait complete. 2024-07-04T17:32:57.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.212+0000 7fa861c00640 1 Processor -- start 2024-07-04T17:32:57.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.212+0000 7fa861c00640 1 -- start start 2024-07-04T17:32:57.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.213+0000 7fa861c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa85c073b50 0x7fa85c1d82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:57.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.213+0000 7fa861c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c072740 con 0x7fa85c073b50 2024-07-04T17:32:57.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.214+0000 7fa85aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa85c073b50 0x7fa85c1d82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:57.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.214+0000 7fa85aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa85c073b50 0x7fa85c1d82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35318/0 (socket says 172.21.15.42:35318) 2024-07-04T17:32:57.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.214+0000 7fa85aa00640 1 -- 172.21.15.42:0/586622110 learned_addr learned my addr 172.21.15.42:0/586622110 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:57.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.214+0000 7fa85aa00640 1 -- 172.21.15.42:0/586622110 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85c140690 con 0x7fa85c073b50 2024-07-04T17:32:57.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.214+0000 7fa85aa00640 1 --2- 172.21.15.42:0/586622110 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa85c073b50 0x7fa85c1d82d0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fa84c009290 tx=0x7fa84c033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:57.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.215+0000 7fa852a00640 1 -- 172.21.15.42:0/586622110 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa84c0319b0 con 0x7fa85c073b50 2024-07-04T17:32:57.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.215+0000 7fa852a00640 1 -- 172.21.15.42:0/586622110 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa84c031c40 con 0x7fa85c073b50 2024-07-04T17:32:57.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.215+0000 7fa852a00640 1 -- 172.21.15.42:0/586622110 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa84c033070 con 0x7fa85c073b50 2024-07-04T17:32:57.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.215+0000 7fa861c00640 1 -- 172.21.15.42:0/586622110 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa85c137ab0 con 0x7fa85c073b50 2024-07-04T17:32:57.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.215+0000 7fa861c00640 1 -- 172.21.15.42:0/586622110 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa85c0aac50 con 0x7fa85c073b50 2024-07-04T17:32:57.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.216+0000 7fa861c00640 1 -- 172.21.15.42:0/586622110 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa85c0b1100 con 0x7fa85c073b50 2024-07-04T17:32:57.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.216+0000 7fa852a00640 1 -- 172.21.15.42:0/586622110 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7fa84c033210 con 0x7fa85c073b50 2024-07-04T17:32:57.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.217+0000 7fa852a00640 1 --2- 172.21.15.42:0/586622110 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fa83c03d4f0 0x7fa83c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:57.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.217+0000 7fa852a00640 1 -- 172.21.15.42:0/586622110 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 4039+0+0 (secure 0 0 0) 0x7fa84c072fe0 con 0x7fa85c073b50 2024-07-04T17:32:57.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.217+0000 7fa85a000640 1 --2- 172.21.15.42:0/586622110 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fa83c03d4f0 0x7fa83c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:57.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.218+0000 7fa85a000640 1 --2- 172.21.15.42:0/586622110 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fa83c03d4f0 0x7fa83c03fa00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa848007fd0 tx=0x7fa848007920 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:57.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.222+0000 7fa852a00640 1 -- 172.21.15.42:0/586622110 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fa84c03f030 con 0x7fa85c073b50 2024-07-04T17:32:57.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.439+0000 7fa861c00640 1 -- 172.21.15.42:0/586622110 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa85c069b70 con 0x7fa85c073b50 2024-07-04T17:32:57.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.887+0000 7fa852a00640 1 -- 172.21.15.42:0/586622110 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_data' removed v105) ==== 178+0+0 (secure 0 0 0) 0x7fa84c0335b0 con 0x7fa85c073b50 2024-07-04T17:32:57.888 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'backup_fs_data' removed 2024-07-04T17:32:57.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.891+0000 7fa861c00640 1 -- 172.21.15.42:0/586622110 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fa83c03d4f0 msgr2=0x7fa83c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:57.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.892+0000 7fa861c00640 1 --2- 172.21.15.42:0/586622110 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fa83c03d4f0 0x7fa83c03fa00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa848007fd0 tx=0x7fa848007920 comp rx=0 tx=0).stop 2024-07-04T17:32:57.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.892+0000 7fa861c00640 1 -- 172.21.15.42:0/586622110 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa85c073b50 msgr2=0x7fa85c1d82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:57.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.892+0000 7fa861c00640 1 --2- 172.21.15.42:0/586622110 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa85c073b50 0x7fa85c1d82d0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fa84c009290 tx=0x7fa84c033ad0 comp rx=0 tx=0).stop 2024-07-04T17:32:57.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.895+0000 7fa861c00640 1 -- 172.21.15.42:0/586622110 shutdown_connections 2024-07-04T17:32:57.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.895+0000 7fa861c00640 1 --2- 172.21.15.42:0/586622110 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fa83c03d4f0 0x7fa83c03fa00 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:57.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.895+0000 7fa861c00640 1 --2- 172.21.15.42:0/586622110 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa85c073b50 0x7fa85c1d82d0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:57.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.895+0000 7fa861c00640 1 -- 172.21.15.42:0/586622110 >> 172.21.15.42:0/586622110 conn(0x7fa85c06c020 msgr2=0x7fa85c075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:57.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.896+0000 7fa861c00640 1 -- 172.21.15.42:0/586622110 shutdown_connections 2024-07-04T17:32:57.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:57.896+0000 7fa861c00640 1 -- 172.21.15.42:0/586622110 wait complete. 2024-07-04T17:32:57.943 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 22 2024-07-04T17:32:58.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.209+0000 7f64ee000640 1 -- 172.21.15.42:0/3364930815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64e80b1760 msgr2=0x7f64e80b8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:58.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.209+0000 7f64ee000640 1 --2- 172.21.15.42:0/3364930815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64e80b1760 0x7f64e80b8da0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f64dc0097b0 tx=0x7f64dc02a540 comp rx=0 tx=0).stop 2024-07-04T17:32:58.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.209+0000 7f64ee000640 1 -- 172.21.15.42:0/3364930815 shutdown_connections 2024-07-04T17:32:58.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.209+0000 7f64ee000640 1 --2- 172.21.15.42:0/3364930815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64e80b1760 0x7f64e80b8da0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:58.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.209+0000 7f64ee000640 1 -- 172.21.15.42:0/3364930815 >> 172.21.15.42:0/3364930815 conn(0x7f64e80bebc0 msgr2=0x7f64e80befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:58.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.209+0000 7f64ee000640 1 -- 172.21.15.42:0/3364930815 shutdown_connections 2024-07-04T17:32:58.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.209+0000 7f64ee000640 1 -- 172.21.15.42:0/3364930815 wait complete. 2024-07-04T17:32:58.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.212+0000 7f64ee000640 1 Processor -- start 2024-07-04T17:32:58.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.212+0000 7f64ee000640 1 -- start start 2024-07-04T17:32:58.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.213+0000 7f64ee000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64e806c020 0x7f64e81d5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:58.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.213+0000 7f64ee000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f64e814eed0 con 0x7f64e806c020 2024-07-04T17:32:58.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.214+0000 7f64ecc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64e806c020 0x7f64e81d5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:58.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.214+0000 7f64ecc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64e806c020 0x7f64e81d5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35332/0 (socket says 172.21.15.42:35332) 2024-07-04T17:32:58.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.214+0000 7f64ecc00640 1 -- 172.21.15.42:0/1212913544 learned_addr learned my addr 172.21.15.42:0/1212913544 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:58.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.214+0000 7f64ecc00640 1 -- 172.21.15.42:0/1212913544 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e81d7490 con 0x7f64e806c020 2024-07-04T17:32:58.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.215+0000 7f64ecc00640 1 --2- 172.21.15.42:0/1212913544 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64e806c020 0x7f64e81d5dd0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f64dc0098d0 tx=0x7f64dc033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:58.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.215+0000 7f64e4c00640 1 -- 172.21.15.42:0/1212913544 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64dc031bc0 con 0x7f64e806c020 2024-07-04T17:32:58.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.215+0000 7f64e4c00640 1 -- 172.21.15.42:0/1212913544 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64dc031e50 con 0x7f64e806c020 2024-07-04T17:32:58.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.215+0000 7f64e4c00640 1 -- 172.21.15.42:0/1212913544 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64dc033200 con 0x7f64e806c020 2024-07-04T17:32:58.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.215+0000 7f64ee000640 1 -- 172.21.15.42:0/1212913544 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64e8145990 con 0x7f64e806c020 2024-07-04T17:32:58.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.215+0000 7f64ee000640 1 -- 172.21.15.42:0/1212913544 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64e8148030 con 0x7f64e806c020 2024-07-04T17:32:58.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.217+0000 7f64e4c00640 1 -- 172.21.15.42:0/1212913544 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7f64dc0333a0 con 0x7f64e806c020 2024-07-04T17:32:58.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.217+0000 7f64ee000640 1 -- 172.21.15.42:0/1212913544 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64e80b8da0 con 0x7f64e806c020 2024-07-04T17:32:58.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.217+0000 7f64e4c00640 1 --2- 172.21.15.42:0/1212913544 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f64c803d4f0 0x7f64c803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:58.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.217+0000 7f64e4c00640 1 -- 172.21.15.42:0/1212913544 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 3619+0+0 (secure 0 0 0) 0x7f64dc072eb0 con 0x7f64e806c020 2024-07-04T17:32:58.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.218+0000 7f64e7e00640 1 --2- 172.21.15.42:0/1212913544 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f64c803d4f0 0x7f64c803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:58.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.219+0000 7f64e7e00640 1 --2- 172.21.15.42:0/1212913544 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f64c803d4f0 0x7f64c803fa00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f64e80b8760 tx=0x7f64d4007a90 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:58.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.223+0000 7f64e4c00640 1 -- 172.21.15.42:0/1212913544 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f64dc03f030 con 0x7f64e806c020 2024-07-04T17:32:58.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.435+0000 7f64ee000640 1 -- 172.21.15.42:0/1212913544 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config reset", "num": 22} v 0) -- 0x7f64e8068e30 con 0x7f64e806c020 2024-07-04T17:32:58.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.441+0000 7f64e4c00640 1 -- 172.21.15.42:0/1212913544 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 22}]=0 v30) ==== 71+0+0 (secure 0 0 0) 0x7f64dc033690 con 0x7f64e806c020 2024-07-04T17:32:58.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.443+0000 7f64ee000640 1 -- 172.21.15.42:0/1212913544 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f64c803d4f0 msgr2=0x7f64c803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:58.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.443+0000 7f64ee000640 1 --2- 172.21.15.42:0/1212913544 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f64c803d4f0 0x7f64c803fa00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f64e80b8760 tx=0x7f64d4007a90 comp rx=0 tx=0).stop 2024-07-04T17:32:58.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.443+0000 7f64ee000640 1 -- 172.21.15.42:0/1212913544 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64e806c020 msgr2=0x7f64e81d5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:58.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.443+0000 7f64ee000640 1 --2- 172.21.15.42:0/1212913544 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64e806c020 0x7f64e81d5dd0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f64dc0098d0 tx=0x7f64dc033e40 comp rx=0 tx=0).stop 2024-07-04T17:32:58.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.443+0000 7f64ee000640 1 -- 172.21.15.42:0/1212913544 shutdown_connections 2024-07-04T17:32:58.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.443+0000 7f64ee000640 1 --2- 172.21.15.42:0/1212913544 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f64c803d4f0 0x7f64c803fa00 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:58.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.443+0000 7f64ee000640 1 --2- 172.21.15.42:0/1212913544 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64e806c020 0x7f64e81d5dd0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:58.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.443+0000 7f64ee000640 1 -- 172.21.15.42:0/1212913544 >> 172.21.15.42:0/1212913544 conn(0x7f64e80bebc0 msgr2=0x7f64e80bbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:58.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.444+0000 7f64ee000640 1 -- 172.21.15.42:0/1212913544 shutdown_connections 2024-07-04T17:32:58.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.444+0000 7f64ee000640 1 -- 172.21.15.42:0/1212913544 wait complete. 2024-07-04T17:32:58.469 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_mirroring.TestMirroring.test_add_relative_directory_path' 2024-07-04T17:32:58.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.763+0000 7f3207400640 1 -- 172.21.15.42:0/1007712544 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32000b0cd0 msgr2=0x7f32000b34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:58.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.763+0000 7f3207400640 1 --2- 172.21.15.42:0/1007712544 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32000b0cd0 0x7f32000b34b0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00a8c0 tx=0x7f31ec02b860 comp rx=0 tx=0).stop 2024-07-04T17:32:58.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.763+0000 7f3207400640 1 -- 172.21.15.42:0/1007712544 shutdown_connections 2024-07-04T17:32:58.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.763+0000 7f3207400640 1 --2- 172.21.15.42:0/1007712544 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32000b0cd0 0x7f32000b34b0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:58.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.763+0000 7f3207400640 1 -- 172.21.15.42:0/1007712544 >> 172.21.15.42:0/1007712544 conn(0x7f32000bf700 msgr2=0x7f32000af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:58.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.763+0000 7f3207400640 1 -- 172.21.15.42:0/1007712544 shutdown_connections 2024-07-04T17:32:58.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.763+0000 7f3207400640 1 -- 172.21.15.42:0/1007712544 wait complete. 2024-07-04T17:32:58.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.765+0000 7f3207400640 1 Processor -- start 2024-07-04T17:32:58.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.766+0000 7f3207400640 1 -- start start 2024-07-04T17:32:58.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.767+0000 7f3207400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32000abdd0 0x7f3200072100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:58.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.767+0000 7f3207400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f32000b03a0 con 0x7f32000abdd0 2024-07-04T17:32:58.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.767+0000 7f3204a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32000abdd0 0x7f3200072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:58.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.767+0000 7f3204a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32000abdd0 0x7f3200072100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35350/0 (socket says 172.21.15.42:35350) 2024-07-04T17:32:58.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.767+0000 7f3204a00640 1 -- 172.21.15.42:0/1955546050 learned_addr learned my addr 172.21.15.42:0/1955546050 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:32:58.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.768+0000 7f3204a00640 1 -- 172.21.15.42:0/1955546050 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32000b8900 con 0x7f32000abdd0 2024-07-04T17:32:58.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.768+0000 7f3204a00640 1 --2- 172.21.15.42:0/1955546050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32000abdd0 0x7f3200072100 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f31f400c6b0 tx=0x7f31f400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:58.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.768+0000 7f31fcc00640 1 -- 172.21.15.42:0/1955546050 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31f4016020 con 0x7f32000abdd0 2024-07-04T17:32:58.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.769+0000 7f3207400640 1 -- 172.21.15.42:0/1955546050 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32000726d0 con 0x7f32000abdd0 2024-07-04T17:32:58.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.769+0000 7f31fcc00640 1 -- 172.21.15.42:0/1955546050 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31f400a5c0 con 0x7f32000abdd0 2024-07-04T17:32:58.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.769+0000 7f31fcc00640 1 -- 172.21.15.42:0/1955546050 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31f400a960 con 0x7f32000abdd0 2024-07-04T17:32:58.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.769+0000 7f3207400640 1 -- 172.21.15.42:0/1955546050 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32001dad70 con 0x7f32000abdd0 2024-07-04T17:32:58.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.770+0000 7f3207400640 1 -- 172.21.15.42:0/1955546050 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32000b2da0 con 0x7f32000abdd0 2024-07-04T17:32:58.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.770+0000 7f31fcc00640 1 -- 172.21.15.42:0/1955546050 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7f31f4019020 con 0x7f32000abdd0 2024-07-04T17:32:58.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.770+0000 7f31fcc00640 1 --2- 172.21.15.42:0/1955546050 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f31e003d4a0 0x7f31e003f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:32:58.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.771+0000 7f31fcc00640 1 -- 172.21.15.42:0/1955546050 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 3619+0+0 (secure 0 0 0) 0x7f31f40515c0 con 0x7f32000abdd0 2024-07-04T17:32:58.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.771+0000 7f31ffe00640 1 --2- 172.21.15.42:0/1955546050 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f31e003d4a0 0x7f31e003f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:32:58.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.772+0000 7f31ffe00640 1 --2- 172.21.15.42:0/1955546050 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f31e003d4a0 0x7f31e003f9b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f31ec036000 tx=0x7f31ec004730 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:32:58.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:58.776+0000 7f31fcc00640 1 -- 172.21.15.42:0/1955546050 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f31f4020030 con 0x7f32000abdd0 2024-07-04T17:32:59.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:59.064+0000 7f3207400640 1 -- 172.21.15.42:0/1955546050 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_mirroring.TestMirroring.test_add_relative_directory_path"]} v 0) -- 0x7f32000bdce0 con 0x7f32000abdd0 2024-07-04T17:32:59.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:59.896+0000 7f31fcc00640 1 -- 172.21.15.42:0/1955546050 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_mirroring.TestMirroring.test_add_relative_directory_path"]}]=0 v356) ==== 153+0+0 (secure 0 0 0) 0x7f32000b10e0 con 0x7f32000abdd0 2024-07-04T17:32:59.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:59.899+0000 7f3207400640 1 -- 172.21.15.42:0/1955546050 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f31e003d4a0 msgr2=0x7f31e003f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:59.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:59.899+0000 7f3207400640 1 --2- 172.21.15.42:0/1955546050 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f31e003d4a0 0x7f31e003f9b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f31ec036000 tx=0x7f31ec004730 comp rx=0 tx=0).stop 2024-07-04T17:32:59.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:59.899+0000 7f3207400640 1 -- 172.21.15.42:0/1955546050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32000abdd0 msgr2=0x7f3200072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:32:59.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:59.899+0000 7f3207400640 1 --2- 172.21.15.42:0/1955546050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32000abdd0 0x7f3200072100 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f31f400c6b0 tx=0x7f31f400cb70 comp rx=0 tx=0).stop 2024-07-04T17:32:59.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:59.900+0000 7f3207400640 1 -- 172.21.15.42:0/1955546050 shutdown_connections 2024-07-04T17:32:59.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:59.900+0000 7f3207400640 1 --2- 172.21.15.42:0/1955546050 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f31e003d4a0 0x7f31e003f9b0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:59.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:59.900+0000 7f3207400640 1 --2- 172.21.15.42:0/1955546050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32000abdd0 0x7f3200072100 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:32:59.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:59.900+0000 7f3207400640 1 -- 172.21.15.42:0/1955546050 >> 172.21.15.42:0/1955546050 conn(0x7f32000bf700 msgr2=0x7f32000aaba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:32:59.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:59.901+0000 7f3207400640 1 -- 172.21.15.42:0/1955546050 shutdown_connections 2024-07-04T17:32:59.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:32:59.901+0000 7f3207400640 1 -- 172.21.15.42:0/1955546050 wait complete. 2024-07-04T17:32:59.933 INFO:tasks.cephfs_test_runner:test_add_relative_directory_path (tasks.cephfs.test_mirroring.TestMirroring) ... ok 2024-07-04T17:32:59.934 INFO:tasks.cephfs_test_runner:Starting test: test_basic_mirror_commands (tasks.cephfs.test_mirroring.TestMirroring) 2024-07-04T17:32:59.934 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:33:00.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.229+0000 7f7889200640 1 -- 172.21.15.42:0/2058930805 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78840b3030 msgr2=0x7f78840b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:00.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.229+0000 7f7889200640 1 --2- 172.21.15.42:0/2058930805 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78840b3030 0x7f78840b01f0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f787000b2b0 tx=0x7f787002ad10 comp rx=0 tx=0).stop 2024-07-04T17:33:00.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.229+0000 7f7889200640 1 -- 172.21.15.42:0/2058930805 shutdown_connections 2024-07-04T17:33:00.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.229+0000 7f7889200640 1 --2- 172.21.15.42:0/2058930805 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78840b3030 0x7f78840b01f0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:00.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.229+0000 7f7889200640 1 -- 172.21.15.42:0/2058930805 >> 172.21.15.42:0/2058930805 conn(0x7f78840bf700 msgr2=0x7f78840af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:00.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.229+0000 7f7889200640 1 -- 172.21.15.42:0/2058930805 shutdown_connections 2024-07-04T17:33:00.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.229+0000 7f7889200640 1 -- 172.21.15.42:0/2058930805 wait complete. 2024-07-04T17:33:00.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.231+0000 7f7889200640 1 Processor -- start 2024-07-04T17:33:00.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.231+0000 7f7889200640 1 -- start start 2024-07-04T17:33:00.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.233+0000 7f7889200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f788406cea0 0x7f7884149410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:00.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.233+0000 7f7889200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f788406edd0 con 0x7f788406cea0 2024-07-04T17:33:00.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.233+0000 7f7881e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f788406cea0 0x7f7884149410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:00.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.233+0000 7f7881e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f788406cea0 0x7f7884149410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35366/0 (socket says 172.21.15.42:35366) 2024-07-04T17:33:00.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.233+0000 7f7881e00640 1 -- 172.21.15.42:0/1574644336 learned_addr learned my addr 172.21.15.42:0/1574644336 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:00.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.233+0000 7f7881e00640 1 -- 172.21.15.42:0/1574644336 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7884073f00 con 0x7f788406cea0 2024-07-04T17:33:00.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.234+0000 7f7881e00640 1 --2- 172.21.15.42:0/1574644336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f788406cea0 0x7f7884149410 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f787400c320 tx=0x7f787400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:00.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.234+0000 7f787a000640 1 -- 172.21.15.42:0/1574644336 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78740116c0 con 0x7f788406cea0 2024-07-04T17:33:00.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.234+0000 7f787a000640 1 -- 172.21.15.42:0/1574644336 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7874017070 con 0x7f788406cea0 2024-07-04T17:33:00.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.234+0000 7f787a000640 1 -- 172.21.15.42:0/1574644336 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7874011b40 con 0x7f788406cea0 2024-07-04T17:33:00.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.234+0000 7f7889200640 1 -- 172.21.15.42:0/1574644336 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78841dd590 con 0x7f788406cea0 2024-07-04T17:33:00.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.234+0000 7f7889200640 1 -- 172.21.15.42:0/1574644336 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78840710c0 con 0x7f788406cea0 2024-07-04T17:33:00.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.235+0000 7f7889200640 1 -- 172.21.15.42:0/1574644336 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78840b01f0 con 0x7f788406cea0 2024-07-04T17:33:00.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.236+0000 7f787a000640 1 -- 172.21.15.42:0/1574644336 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7f7874011ce0 con 0x7f788406cea0 2024-07-04T17:33:00.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.236+0000 7f787a000640 1 --2- 172.21.15.42:0/1574644336 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f786403d4f0 0x7f786403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:00.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.237+0000 7f787a000640 1 -- 172.21.15.42:0/1574644336 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 3619+0+0 (secure 0 0 0) 0x7f7874051750 con 0x7f788406cea0 2024-07-04T17:33:00.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.237+0000 7f7881400640 1 --2- 172.21.15.42:0/1574644336 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f786403d4f0 0x7f786403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:00.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.238+0000 7f7881400640 1 --2- 172.21.15.42:0/1574644336 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f786403d4f0 0x7f786403fa00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7870035000 tx=0x7f7870004730 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:00.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.241+0000 7f787a000640 1 -- 172.21.15.42:0/1574644336 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f787400a140 con 0x7f788406cea0 2024-07-04T17:33:00.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.456+0000 7f7889200640 1 -- 172.21.15.42:0/1574644336 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f788414cb90 con 0x7f788406cea0 2024-07-04T17:33:00.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.458+0000 7f787a000640 1 -- 172.21.15.42:0/1574644336 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+8277 (secure 0 0 0) 0x7f787401ac60 con 0x7f788406cea0 2024-07-04T17:33:00.460 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:33:00.460 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":105,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:32:57.881485+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6831/327636003":"2024-07-05T17:32:54.865413+0000","172.21.15.42:6830/327636003":"2024-07-05T17:32:54.865413+0000","172.21.15.42:6832/3777807763":"2024-07-05T17:32:48.292733+0000","172.21.15.42:6835/1323178695":"2024-07-05T17:32:47.238337+0000","172.21.15.42:6834/1323178695":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/3479187327":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/143103576":"2024-07-05T17:32:47.238337+0000","172.21.15.42:6833/3777807763":"2024-07-05T17:32:48.292733+0000","172.21.15.42:0/1626565787":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/1439263513":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/2943589644":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/2854152871":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/4179523139":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/2096108670":"2024-07-04T18:32:31.377755+0000","172.21.15.42:0/1641580783":"2024-07-05T17:32:15.307330+0000","172.21.15.42:0/3208988074":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/1212024809":"2024-07-05T17:32:15.307330+0000","172.21.15.42:0/1089649627":"2024-07-05T17:32:15.307330+0000","172.21.15.42:0/257615823":"2024-07-05T17:32:15.307330+0000","172.21.15.42:0/2624550914":"2024-07-05T17:32:47.238337+0000","172.21.15.42:0/2580772079":"2024-07-05T17:32:15.307330+0000","172.21.15.42:0/611598150":"2024-07-05T17:32:15.307330+0000","172.21.15.42:6835/1139872311":"2024-07-05T17:32:15.307330+0000","172.21.15.42:6834/1139872311":"2024-07-05T17:32:15.307330+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:33:00.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.461+0000 7f7889200640 1 -- 172.21.15.42:0/1574644336 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f786403d4f0 msgr2=0x7f786403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:00.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.461+0000 7f7889200640 1 --2- 172.21.15.42:0/1574644336 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f786403d4f0 0x7f786403fa00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7870035000 tx=0x7f7870004730 comp rx=0 tx=0).stop 2024-07-04T17:33:00.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.461+0000 7f7889200640 1 -- 172.21.15.42:0/1574644336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f788406cea0 msgr2=0x7f7884149410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:00.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.461+0000 7f7889200640 1 --2- 172.21.15.42:0/1574644336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f788406cea0 0x7f7884149410 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f787400c320 tx=0x7f787400c7e0 comp rx=0 tx=0).stop 2024-07-04T17:33:00.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.462+0000 7f7889200640 1 -- 172.21.15.42:0/1574644336 shutdown_connections 2024-07-04T17:33:00.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.462+0000 7f7889200640 1 --2- 172.21.15.42:0/1574644336 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f786403d4f0 0x7f786403fa00 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:00.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.462+0000 7f7889200640 1 --2- 172.21.15.42:0/1574644336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f788406cea0 0x7f7884149410 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:00.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.462+0000 7f7889200640 1 -- 172.21.15.42:0/1574644336 >> 172.21.15.42:0/1574644336 conn(0x7f78840bf700 msgr2=0x7f78840bce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:00.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.462+0000 7f7889200640 1 -- 172.21.15.42:0/1574644336 shutdown_connections 2024-07-04T17:33:00.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.462+0000 7f7889200640 1 -- 172.21.15.42:0/1574644336 wait complete. 2024-07-04T17:33:00.490 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-04T17:26:03.297630+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-04T17:33:00.490 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-04T17:33:00.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.777+0000 7fe5b8400640 1 -- 172.21.15.42:0/1862703154 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5b00a94d0 msgr2=0x7fe5b00b1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:00.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.777+0000 7fe5b8400640 1 --2- 172.21.15.42:0/1862703154 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5b00a94d0 0x7fe5b00b1100 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fe5a40097b0 tx=0x7fe5a402a530 comp rx=0 tx=0).stop 2024-07-04T17:33:00.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.777+0000 7fe5b8400640 1 -- 172.21.15.42:0/1862703154 shutdown_connections 2024-07-04T17:33:00.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.777+0000 7fe5b8400640 1 --2- 172.21.15.42:0/1862703154 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5b00a94d0 0x7fe5b00b1100 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:00.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.777+0000 7fe5b8400640 1 -- 172.21.15.42:0/1862703154 >> 172.21.15.42:0/1862703154 conn(0x7fe5b006c020 msgr2=0x7fe5b006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:00.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.777+0000 7fe5b8400640 1 -- 172.21.15.42:0/1862703154 shutdown_connections 2024-07-04T17:33:00.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.777+0000 7fe5b8400640 1 -- 172.21.15.42:0/1862703154 wait complete. 2024-07-04T17:33:00.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.779+0000 7fe5b8400640 1 Processor -- start 2024-07-04T17:33:00.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.779+0000 7fe5b8400640 1 -- start start 2024-07-04T17:33:00.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.781+0000 7fe5b8400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5b0073b50 0x7fe5b00b3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:00.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.781+0000 7fe5b8400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b0072740 con 0x7fe5b0073b50 2024-07-04T17:33:00.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.781+0000 7fe5b5a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5b0073b50 0x7fe5b00b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:00.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.781+0000 7fe5b5a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5b0073b50 0x7fe5b00b3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35392/0 (socket says 172.21.15.42:35392) 2024-07-04T17:33:00.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.781+0000 7fe5b5a00640 1 -- 172.21.15.42:0/1165304828 learned_addr learned my addr 172.21.15.42:0/1165304828 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:00.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.782+0000 7fe5b5a00640 1 -- 172.21.15.42:0/1165304828 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b0132ea0 con 0x7fe5b0073b50 2024-07-04T17:33:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.782+0000 7fe5b5a00640 1 --2- 172.21.15.42:0/1165304828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5b0073b50 0x7fe5b00b3450 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fe5a40098d0 tx=0x7fe5a4033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.782+0000 7fe5ad600640 1 -- 172.21.15.42:0/1165304828 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5a4031880 con 0x7fe5b0073b50 2024-07-04T17:33:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.782+0000 7fe5ad600640 1 -- 172.21.15.42:0/1165304828 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe5a4031b10 con 0x7fe5b0073b50 2024-07-04T17:33:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.782+0000 7fe5b8400640 1 -- 172.21.15.42:0/1165304828 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5b0139d50 con 0x7fe5b0073b50 2024-07-04T17:33:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.782+0000 7fe5ad600640 1 -- 172.21.15.42:0/1165304828 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5a4031df0 con 0x7fe5b0073b50 2024-07-04T17:33:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.783+0000 7fe5b8400640 1 -- 172.21.15.42:0/1165304828 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5b00b3990 con 0x7fe5b0073b50 2024-07-04T17:33:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.784+0000 7fe5b8400640 1 -- 172.21.15.42:0/1165304828 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5b00b1100 con 0x7fe5b0073b50 2024-07-04T17:33:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.784+0000 7fe5ad600640 1 -- 172.21.15.42:0/1165304828 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7fe5a4033070 con 0x7fe5b0073b50 2024-07-04T17:33:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.785+0000 7fe5ad600640 1 --2- 172.21.15.42:0/1165304828 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fe59003d4f0 0x7fe59003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.785+0000 7fe5ad600640 1 -- 172.21.15.42:0/1165304828 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 3619+0+0 (secure 0 0 0) 0x7fe5a4072e20 con 0x7fe5b0073b50 2024-07-04T17:33:00.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.785+0000 7fe5b5000640 1 --2- 172.21.15.42:0/1165304828 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fe59003d4f0 0x7fe59003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:00.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.786+0000 7fe5b5000640 1 --2- 172.21.15.42:0/1165304828 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fe59003d4f0 0x7fe59003fa00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe59c007fd0 tx=0x7fe59c007920 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:00.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.790+0000 7fe5ad600640 1 -- 172.21.15.42:0/1165304828 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fe5a403f030 con 0x7fe5b0073b50 2024-07-04T17:33:00.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.998+0000 7fe5b8400640 1 -- 172.21.15.42:0/1165304828 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe5b0153de0 con 0x7fe5b0073b50 2024-07-04T17:33:01.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:00.999+0000 7fe5ad600640 1 -- 172.21.15.42:0/1165304828 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) ==== 93+0+10 (secure 0 0 0) 0x7fe5a403a3a0 con 0x7fe5b0073b50 2024-07-04T17:33:01.000 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-07-04T17:33:01.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.002+0000 7fe5b8400640 1 -- 172.21.15.42:0/1165304828 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fe59003d4f0 msgr2=0x7fe59003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:01.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.002+0000 7fe5b8400640 1 --2- 172.21.15.42:0/1165304828 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fe59003d4f0 0x7fe59003fa00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe59c007fd0 tx=0x7fe59c007920 comp rx=0 tx=0).stop 2024-07-04T17:33:01.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.002+0000 7fe5b8400640 1 -- 172.21.15.42:0/1165304828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5b0073b50 msgr2=0x7fe5b00b3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:01.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.002+0000 7fe5b8400640 1 --2- 172.21.15.42:0/1165304828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5b0073b50 0x7fe5b00b3450 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fe5a40098d0 tx=0x7fe5a4033ad0 comp rx=0 tx=0).stop 2024-07-04T17:33:01.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.002+0000 7fe5b8400640 1 -- 172.21.15.42:0/1165304828 shutdown_connections 2024-07-04T17:33:01.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.002+0000 7fe5b8400640 1 --2- 172.21.15.42:0/1165304828 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fe59003d4f0 0x7fe59003fa00 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:01.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.002+0000 7fe5b8400640 1 --2- 172.21.15.42:0/1165304828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5b0073b50 0x7fe5b00b3450 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:01.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.002+0000 7fe5b8400640 1 -- 172.21.15.42:0/1165304828 >> 172.21.15.42:0/1165304828 conn(0x7fe5b006c020 msgr2=0x7fe5b00afde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:01.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.003+0000 7fe5b8400640 1 -- 172.21.15.42:0/1165304828 shutdown_connections 2024-07-04T17:33:01.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.003+0000 7fe5b8400640 1 -- 172.21.15.42:0/1165304828 wait complete. 2024-07-04T17:33:01.030 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_mirroring.TestMirroring.test_basic_mirror_commands' 2024-07-04T17:33:01.042 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-04T17:33:01.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.308+0000 7f3dc1200640 1 -- 172.21.15.42:0/2519800000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3dbc0b02b0 msgr2=0x7f3dbc0b3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:01.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.308+0000 7f3dc1200640 1 --2- 172.21.15.42:0/2519800000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3dbc0b02b0 0x7f3dbc0b3650 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f3dac0097b0 tx=0x7f3dac02a540 comp rx=0 tx=0).stop 2024-07-04T17:33:01.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.308+0000 7f3dc1200640 1 -- 172.21.15.42:0/2519800000 shutdown_connections 2024-07-04T17:33:01.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.308+0000 7f3dc1200640 1 --2- 172.21.15.42:0/2519800000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3dbc0b02b0 0x7f3dbc0b3650 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:01.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.308+0000 7f3dc1200640 1 -- 172.21.15.42:0/2519800000 >> 172.21.15.42:0/2519800000 conn(0x7f3dbc0bf700 msgr2=0x7f3dbc0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:01.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.308+0000 7f3dc1200640 1 -- 172.21.15.42:0/2519800000 shutdown_connections 2024-07-04T17:33:01.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.308+0000 7f3dc1200640 1 -- 172.21.15.42:0/2519800000 wait complete. 2024-07-04T17:33:01.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.310+0000 7f3dc1200640 1 Processor -- start 2024-07-04T17:33:01.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.311+0000 7f3dc1200640 1 -- start start 2024-07-04T17:33:01.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.312+0000 7f3dc1200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3dbc0b02b0 0x7f3dbc0af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:01.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.312+0000 7f3dc1200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3dbc1463b0 con 0x7f3dbc0b02b0 2024-07-04T17:33:01.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.312+0000 7f3db9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3dbc0b02b0 0x7f3dbc0af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:01.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.312+0000 7f3db9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3dbc0b02b0 0x7f3dbc0af0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35406/0 (socket says 172.21.15.42:35406) 2024-07-04T17:33:01.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.312+0000 7f3db9e00640 1 -- 172.21.15.42:0/1144533398 learned_addr learned my addr 172.21.15.42:0/1144533398 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:01.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.313+0000 7f3db9e00640 1 -- 172.21.15.42:0/1144533398 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dbc0b5250 con 0x7f3dbc0b02b0 2024-07-04T17:33:01.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.313+0000 7f3db9e00640 1 --2- 172.21.15.42:0/1144533398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3dbc0b02b0 0x7f3dbc0af0b0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f3dac0098d0 tx=0x7f3dac033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:01.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.313+0000 7f3db2000640 1 -- 172.21.15.42:0/1144533398 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3dac031bc0 con 0x7f3dbc0b02b0 2024-07-04T17:33:01.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.314+0000 7f3db2000640 1 -- 172.21.15.42:0/1144533398 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3dac031e50 con 0x7f3dbc0b02b0 2024-07-04T17:33:01.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.314+0000 7f3db2000640 1 -- 172.21.15.42:0/1144533398 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3dac033200 con 0x7f3dbc0b02b0 2024-07-04T17:33:01.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.314+0000 7f3dc1200640 1 -- 172.21.15.42:0/1144533398 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dbc0b5d60 con 0x7f3dbc0b02b0 2024-07-04T17:33:01.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.314+0000 7f3dc1200640 1 -- 172.21.15.42:0/1144533398 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dbc0b5660 con 0x7f3dbc0b02b0 2024-07-04T17:33:01.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.315+0000 7f3dc1200640 1 -- 172.21.15.42:0/1144533398 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dbc0b3650 con 0x7f3dbc0b02b0 2024-07-04T17:33:01.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.315+0000 7f3db2000640 1 -- 172.21.15.42:0/1144533398 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7f3dac0333a0 con 0x7f3dbc0b02b0 2024-07-04T17:33:01.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.316+0000 7f3db2000640 1 --2- 172.21.15.42:0/1144533398 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f3d9c03d4f0 0x7f3d9c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:01.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.316+0000 7f3db2000640 1 -- 172.21.15.42:0/1144533398 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 3619+0+0 (secure 0 0 0) 0x7f3dac072f70 con 0x7f3dbc0b02b0 2024-07-04T17:33:01.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.316+0000 7f3db9400640 1 --2- 172.21.15.42:0/1144533398 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f3d9c03d4f0 0x7f3d9c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:01.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.317+0000 7f3db9400640 1 --2- 172.21.15.42:0/1144533398 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f3d9c03d4f0 0x7f3d9c03fa00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3da8002040 tx=0x7f3da8001650 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:01.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.321+0000 7f3db2000640 1 -- 172.21.15.42:0/1144533398 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f3dbc0b5660 con 0x7f3dbc0b02b0 2024-07-04T17:33:01.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.609+0000 7f3dc1200640 1 -- 172.21.15.42:0/1144533398 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_mirroring.TestMirroring.test_basic_mirror_commands"]} v 0) -- 0x7f3dbc0b6be0 con 0x7f3dbc0b02b0 2024-07-04T17:33:01.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.904+0000 7f3db2000640 1 -- 172.21.15.42:0/1144533398 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_mirroring.TestMirroring.test_basic_mirror_commands"]}]=0 v358) ==== 150+0+0 (secure 0 0 0) 0x7f3dac03f030 con 0x7f3dbc0b02b0 2024-07-04T17:33:01.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.907+0000 7f3dc1200640 1 -- 172.21.15.42:0/1144533398 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f3d9c03d4f0 msgr2=0x7f3d9c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:01.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.907+0000 7f3dc1200640 1 --2- 172.21.15.42:0/1144533398 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f3d9c03d4f0 0x7f3d9c03fa00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3da8002040 tx=0x7f3da8001650 comp rx=0 tx=0).stop 2024-07-04T17:33:01.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.907+0000 7f3dc1200640 1 -- 172.21.15.42:0/1144533398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3dbc0b02b0 msgr2=0x7f3dbc0af0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:01.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.907+0000 7f3dc1200640 1 --2- 172.21.15.42:0/1144533398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3dbc0b02b0 0x7f3dbc0af0b0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f3dac0098d0 tx=0x7f3dac033e40 comp rx=0 tx=0).stop 2024-07-04T17:33:01.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.908+0000 7f3dc1200640 1 -- 172.21.15.42:0/1144533398 shutdown_connections 2024-07-04T17:33:01.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.908+0000 7f3dc1200640 1 --2- 172.21.15.42:0/1144533398 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f3d9c03d4f0 0x7f3d9c03fa00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:01.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.908+0000 7f3dc1200640 1 --2- 172.21.15.42:0/1144533398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3dbc0b02b0 0x7f3dbc0af0b0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:01.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.908+0000 7f3dc1200640 1 -- 172.21.15.42:0/1144533398 >> 172.21.15.42:0/1144533398 conn(0x7f3dbc0bf700 msgr2=0x7f3dbc1455f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:01.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.908+0000 7f3dc1200640 1 -- 172.21.15.42:0/1144533398 shutdown_connections 2024-07-04T17:33:01.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:01.908+0000 7f3dc1200640 1 -- 172.21.15.42:0/1144533398 wait complete. 2024-07-04T17:33:01.944 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-07-04T17:33:02.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.230+0000 7fe595c00640 1 -- 172.21.15.42:0/132816923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe590140e70 msgr2=0x7fe5900bb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:02.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.230+0000 7fe595c00640 1 --2- 172.21.15.42:0/132816923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe590140e70 0x7fe5900bb1f0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fe5800097b0 tx=0x7fe58002a530 comp rx=0 tx=0).stop 2024-07-04T17:33:02.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.230+0000 7fe595c00640 1 -- 172.21.15.42:0/132816923 shutdown_connections 2024-07-04T17:33:02.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.230+0000 7fe595c00640 1 --2- 172.21.15.42:0/132816923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe590140e70 0x7fe5900bb1f0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:02.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.230+0000 7fe595c00640 1 -- 172.21.15.42:0/132816923 >> 172.21.15.42:0/132816923 conn(0x7fe5900bf700 msgr2=0x7fe5900af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:02.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.230+0000 7fe595c00640 1 -- 172.21.15.42:0/132816923 shutdown_connections 2024-07-04T17:33:02.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.231+0000 7fe595c00640 1 -- 172.21.15.42:0/132816923 wait complete. 2024-07-04T17:33:02.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.233+0000 7fe595c00640 1 Processor -- start 2024-07-04T17:33:02.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.233+0000 7fe595c00640 1 -- start start 2024-07-04T17:33:02.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.234+0000 7fe595c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5900a9fa0 0x7fe5900aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:02.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.234+0000 7fe595c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe5900aae50 con 0x7fe5900a9fa0 2024-07-04T17:33:02.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.234+0000 7fe58ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5900a9fa0 0x7fe5900aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:02.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.234+0000 7fe58ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5900a9fa0 0x7fe5900aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35436/0 (socket says 172.21.15.42:35436) 2024-07-04T17:33:02.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.234+0000 7fe58ea00640 1 -- 172.21.15.42:0/2353470327 learned_addr learned my addr 172.21.15.42:0/2353470327 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:02.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.235+0000 7fe58ea00640 1 -- 172.21.15.42:0/2353470327 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5901e6800 con 0x7fe5900a9fa0 2024-07-04T17:33:02.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.235+0000 7fe58ea00640 1 --2- 172.21.15.42:0/2353470327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5900a9fa0 0x7fe5900aed60 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fe5800098d0 tx=0x7fe580033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:02.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.236+0000 7fe586a00640 1 -- 172.21.15.42:0/2353470327 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe580031a90 con 0x7fe5900a9fa0 2024-07-04T17:33:02.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.236+0000 7fe586a00640 1 -- 172.21.15.42:0/2353470327 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe580031d20 con 0x7fe5900a9fa0 2024-07-04T17:33:02.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.236+0000 7fe586a00640 1 -- 172.21.15.42:0/2353470327 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5800330d0 con 0x7fe5900a9fa0 2024-07-04T17:33:02.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.236+0000 7fe595c00640 1 -- 172.21.15.42:0/2353470327 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe590140120 con 0x7fe5900a9fa0 2024-07-04T17:33:02.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.236+0000 7fe595c00640 1 -- 172.21.15.42:0/2353470327 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5900ae9e0 con 0x7fe5900a9fa0 2024-07-04T17:33:02.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.237+0000 7fe595c00640 1 -- 172.21.15.42:0/2353470327 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5900ab510 con 0x7fe5900a9fa0 2024-07-04T17:33:02.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.237+0000 7fe586a00640 1 -- 172.21.15.42:0/2353470327 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7fe580033270 con 0x7fe5900a9fa0 2024-07-04T17:33:02.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.238+0000 7fe586a00640 1 --2- 172.21.15.42:0/2353470327 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fe57003d4f0 0x7fe57003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:02.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.238+0000 7fe586a00640 1 -- 172.21.15.42:0/2353470327 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 3619+0+0 (secure 0 0 0) 0x7fe580073000 con 0x7fe5900a9fa0 2024-07-04T17:33:02.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.238+0000 7fe58e000640 1 --2- 172.21.15.42:0/2353470327 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fe57003d4f0 0x7fe57003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:02.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.239+0000 7fe58e000640 1 --2- 172.21.15.42:0/2353470327 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fe57003d4f0 0x7fe57003fa00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe57c000f60 tx=0x7fe57c007a30 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:02.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.243+0000 7fe586a00640 1 -- 172.21.15.42:0/2353470327 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fe58003f030 con 0x7fe5900a9fa0 2024-07-04T17:33:02.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.451+0000 7fe595c00640 1 -- 172.21.15.42:0/2353470327 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fe590140350 con 0x7fe5900a9fa0 2024-07-04T17:33:02.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.453+0000 7fe586a00640 1 -- 172.21.15.42:0/2353470327 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v30) ==== 59+0+1628 (secure 0 0 0) 0x7fe580003b10 con 0x7fe5900a9fa0 2024-07-04T17:33:02.454 INFO:teuthology.orchestra.run.smithi042.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-07-04T17:33:02.454 INFO:teuthology.orchestra.run.smithi042.stdout:mon advanced mon_op_complaint_time 120 2024-07-04T17:33:02.454 INFO:teuthology.orchestra.run.smithi042.stdout:mon dev mon_warn_on_pool_no_app false 2024-07-04T17:33:02.454 INFO:teuthology.orchestra.run.smithi042.stdout:osd advanced osd_op_complaint_time 180.000000 2024-07-04T17:33:02.454 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3541.919687 2024-07-04T17:33:02.454 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3558.529833 2024-07-04T17:33:02.455 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3688.441103 2024-07-04T17:33:02.455 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced debug_mds 20/20 2024-07-04T17:33:02.455 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced debug_mds_balancer 20/20 2024-07-04T17:33:02.455 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced debug_ms 1/1 2024-07-04T17:33:02.455 INFO:teuthology.orchestra.run.smithi042.stdout:mds dev mds_debug_frag true 2024-07-04T17:33:02.455 INFO:teuthology.orchestra.run.smithi042.stdout:mds dev mds_debug_scatterstat true 2024-07-04T17:33:02.455 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced mds_op_complaint_time 180.000000 2024-07-04T17:33:02.455 INFO:teuthology.orchestra.run.smithi042.stdout:mds dev mds_verify_scatter true 2024-07-04T17:33:02.455 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced osd_op_complaint_time 180.000000 2024-07-04T17:33:02.455 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced rados_mon_op_timeout 900 2024-07-04T17:33:02.455 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced rados_osd_op_timeout 900 2024-07-04T17:33:02.455 INFO:teuthology.orchestra.run.smithi042.stdout:client advanced client_mount_timeout 600 2024-07-04T17:33:02.455 INFO:teuthology.orchestra.run.smithi042.stdout:client advanced debug_client 20/20 2024-07-04T17:33:02.456 INFO:teuthology.orchestra.run.smithi042.stdout:client advanced debug_ms 1/1 2024-07-04T17:33:02.456 INFO:teuthology.orchestra.run.smithi042.stdout:client advanced rados_mon_op_timeout 900 2024-07-04T17:33:02.456 INFO:teuthology.orchestra.run.smithi042.stdout:client advanced rados_osd_op_timeout 900 2024-07-04T17:33:02.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.455+0000 7fe595c00640 1 -- 172.21.15.42:0/2353470327 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fe57003d4f0 msgr2=0x7fe57003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:02.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.455+0000 7fe595c00640 1 --2- 172.21.15.42:0/2353470327 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fe57003d4f0 0x7fe57003fa00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe57c000f60 tx=0x7fe57c007a30 comp rx=0 tx=0).stop 2024-07-04T17:33:02.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.455+0000 7fe595c00640 1 -- 172.21.15.42:0/2353470327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5900a9fa0 msgr2=0x7fe5900aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:02.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.455+0000 7fe595c00640 1 --2- 172.21.15.42:0/2353470327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5900a9fa0 0x7fe5900aed60 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fe5800098d0 tx=0x7fe580033c10 comp rx=0 tx=0).stop 2024-07-04T17:33:02.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.455+0000 7fe595c00640 1 -- 172.21.15.42:0/2353470327 shutdown_connections 2024-07-04T17:33:02.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.455+0000 7fe595c00640 1 --2- 172.21.15.42:0/2353470327 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fe57003d4f0 0x7fe57003fa00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:02.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.455+0000 7fe595c00640 1 --2- 172.21.15.42:0/2353470327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5900a9fa0 0x7fe5900aed60 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:02.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.455+0000 7fe595c00640 1 -- 172.21.15.42:0/2353470327 >> 172.21.15.42:0/2353470327 conn(0x7fe5900bf700 msgr2=0x7fe5900b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:02.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.455+0000 7fe595c00640 1 -- 172.21.15.42:0/2353470327 shutdown_connections 2024-07-04T17:33:02.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.456+0000 7fe595c00640 1 -- 172.21.15.42:0/2353470327 wait complete. 2024-07-04T17:33:02.482 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-07-04T17:33:02.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.769+0000 7fbb76e00640 1 -- 172.21.15.42:0/13109387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb700a94d0 msgr2=0x7fbb700b1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:02.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.769+0000 7fbb76e00640 1 --2- 172.21.15.42:0/13109387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb700a94d0 0x7fbb700b1100 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fbb6000b2b0 tx=0x7fbb6002ad10 comp rx=0 tx=0).stop 2024-07-04T17:33:02.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.770+0000 7fbb76e00640 1 -- 172.21.15.42:0/13109387 shutdown_connections 2024-07-04T17:33:02.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.770+0000 7fbb76e00640 1 --2- 172.21.15.42:0/13109387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb700a94d0 0x7fbb700b1100 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:02.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.770+0000 7fbb76e00640 1 -- 172.21.15.42:0/13109387 >> 172.21.15.42:0/13109387 conn(0x7fbb7006c020 msgr2=0x7fbb7006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:02.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.770+0000 7fbb76e00640 1 -- 172.21.15.42:0/13109387 shutdown_connections 2024-07-04T17:33:02.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.770+0000 7fbb76e00640 1 -- 172.21.15.42:0/13109387 wait complete. 2024-07-04T17:33:02.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.772+0000 7fbb76e00640 1 Processor -- start 2024-07-04T17:33:02.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.772+0000 7fbb76e00640 1 -- start start 2024-07-04T17:33:02.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.773+0000 7fbb76e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb70073b50 0x7fbb701e3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:02.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.773+0000 7fbb76e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbb70072740 con 0x7fbb70073b50 2024-07-04T17:33:02.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.774+0000 7fbb6fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb70073b50 0x7fbb701e3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:02.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.774+0000 7fbb6fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb70073b50 0x7fbb701e3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35450/0 (socket says 172.21.15.42:35450) 2024-07-04T17:33:02.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.774+0000 7fbb6fe00640 1 -- 172.21.15.42:0/3838555040 learned_addr learned my addr 172.21.15.42:0/3838555040 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:02.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.774+0000 7fbb6fe00640 1 -- 172.21.15.42:0/3838555040 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb701e36d0 con 0x7fbb70073b50 2024-07-04T17:33:02.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.775+0000 7fbb6fe00640 1 --2- 172.21.15.42:0/3838555040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb70073b50 0x7fbb701e3e60 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fbb60000c00 tx=0x7fbb600097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:02.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.775+0000 7fbb6cc00640 1 -- 172.21.15.42:0/3838555040 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb60038040 con 0x7fbb70073b50 2024-07-04T17:33:02.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.775+0000 7fbb6cc00640 1 -- 172.21.15.42:0/3838555040 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb60041070 con 0x7fbb70073b50 2024-07-04T17:33:02.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.775+0000 7fbb6cc00640 1 -- 172.21.15.42:0/3838555040 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb60004420 con 0x7fbb70073b50 2024-07-04T17:33:02.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.775+0000 7fbb76e00640 1 -- 172.21.15.42:0/3838555040 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb701e3960 con 0x7fbb70073b50 2024-07-04T17:33:02.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.775+0000 7fbb76e00640 1 -- 172.21.15.42:0/3838555040 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb700adde0 con 0x7fbb70073b50 2024-07-04T17:33:02.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.776+0000 7fbb76e00640 1 -- 172.21.15.42:0/3838555040 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb700b1100 con 0x7fbb70073b50 2024-07-04T17:33:02.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.776+0000 7fbb6cc00640 1 -- 172.21.15.42:0/3838555040 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7fbb600035f0 con 0x7fbb70073b50 2024-07-04T17:33:02.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.777+0000 7fbb6cc00640 1 --2- 172.21.15.42:0/3838555040 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fbb4c03d4f0 0x7fbb4c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:02.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.777+0000 7fbb6cc00640 1 -- 172.21.15.42:0/3838555040 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 3619+0+0 (secure 0 0 0) 0x7fbb600721f0 con 0x7fbb70073b50 2024-07-04T17:33:02.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.777+0000 7fbb67e00640 1 --2- 172.21.15.42:0/3838555040 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fbb4c03d4f0 0x7fbb4c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:02.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.778+0000 7fbb67e00640 1 --2- 172.21.15.42:0/3838555040 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fbb4c03d4f0 0x7fbb4c03fa00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbb70073660 tx=0x7fbb58007a90 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:02.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.782+0000 7fbb6cc00640 1 -- 172.21.15.42:0/3838555040 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fbb600453c0 con 0x7fbb70073b50 2024-07-04T17:33:02.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.990+0000 7fbb76e00640 1 -- 172.21.15.42:0/3838555040 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fbb70069b70 con 0x7fbb70073b50 2024-07-04T17:33:02.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:02.997+0000 7fbb6cc00640 1 -- 172.21.15.42:0/3838555040 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v31)=0 v31) ==== 122+0+0 (secure 0 0 0) 0x7fbb60033cf0 con 0x7fbb70073b50 2024-07-04T17:33:03.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.000+0000 7fbb76e00640 1 -- 172.21.15.42:0/3838555040 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fbb4c03d4f0 msgr2=0x7fbb4c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:03.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.000+0000 7fbb76e00640 1 --2- 172.21.15.42:0/3838555040 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fbb4c03d4f0 0x7fbb4c03fa00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbb70073660 tx=0x7fbb58007a90 comp rx=0 tx=0).stop 2024-07-04T17:33:03.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.000+0000 7fbb76e00640 1 -- 172.21.15.42:0/3838555040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb70073b50 msgr2=0x7fbb701e3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:03.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.000+0000 7fbb76e00640 1 --2- 172.21.15.42:0/3838555040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb70073b50 0x7fbb701e3e60 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fbb60000c00 tx=0x7fbb600097d0 comp rx=0 tx=0).stop 2024-07-04T17:33:03.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.000+0000 7fbb76e00640 1 -- 172.21.15.42:0/3838555040 shutdown_connections 2024-07-04T17:33:03.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.000+0000 7fbb76e00640 1 --2- 172.21.15.42:0/3838555040 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fbb4c03d4f0 0x7fbb4c03fa00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:03.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.000+0000 7fbb76e00640 1 --2- 172.21.15.42:0/3838555040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb70073b50 0x7fbb701e3e60 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:03.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.000+0000 7fbb76e00640 1 -- 172.21.15.42:0/3838555040 >> 172.21.15.42:0/3838555040 conn(0x7fbb7006c020 msgr2=0x7fbb700afde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:03.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.000+0000 7fbb76e00640 1 -- 172.21.15.42:0/3838555040 shutdown_connections 2024-07-04T17:33:03.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.000+0000 7fbb76e00640 1 -- 172.21.15.42:0/3838555040 wait complete. 2024-07-04T17:33:03.027 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-04T17:33:03.027 DEBUG:teuthology.orchestra.run.smithi042:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-04T17:33:03.132 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-04T17:33:03.132 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:33:03.132 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:33:03.185 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-07-04T17:33:03.186 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:33:03.186 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:33:03.239 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-07-04T17:33:03.239 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-07-04T17:33:03.239 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:33:03.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.530+0000 7fd97d400640 1 -- 172.21.15.42:0/1684342171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9780b0cd0 msgr2=0x7fd9780b34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:03.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.530+0000 7fd97d400640 1 --2- 172.21.15.42:0/1684342171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9780b0cd0 0x7fd9780b34b0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fd96000a8c0 tx=0x7fd96002b860 comp rx=0 tx=0).stop 2024-07-04T17:33:03.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.530+0000 7fd97d400640 1 -- 172.21.15.42:0/1684342171 shutdown_connections 2024-07-04T17:33:03.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.530+0000 7fd97d400640 1 --2- 172.21.15.42:0/1684342171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9780b0cd0 0x7fd9780b34b0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:03.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.530+0000 7fd97d400640 1 -- 172.21.15.42:0/1684342171 >> 172.21.15.42:0/1684342171 conn(0x7fd9780bf700 msgr2=0x7fd9780af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:03.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.530+0000 7fd97d400640 1 -- 172.21.15.42:0/1684342171 shutdown_connections 2024-07-04T17:33:03.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.530+0000 7fd97d400640 1 -- 172.21.15.42:0/1684342171 wait complete. 2024-07-04T17:33:03.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.532+0000 7fd97d400640 1 Processor -- start 2024-07-04T17:33:03.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.532+0000 7fd97d400640 1 -- start start 2024-07-04T17:33:03.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.534+0000 7fd97d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9780abdd0 0x7fd978072100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:03.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.534+0000 7fd97d400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd9780b03a0 con 0x7fd9780abdd0 2024-07-04T17:33:03.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.534+0000 7fd976800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9780abdd0 0x7fd978072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:03.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.534+0000 7fd976800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9780abdd0 0x7fd978072100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35466/0 (socket says 172.21.15.42:35466) 2024-07-04T17:33:03.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.534+0000 7fd976800640 1 -- 172.21.15.42:0/1107885742 learned_addr learned my addr 172.21.15.42:0/1107885742 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:03.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.534+0000 7fd976800640 1 -- 172.21.15.42:0/1107885742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9780b8900 con 0x7fd9780abdd0 2024-07-04T17:33:03.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.535+0000 7fd976800640 1 --2- 172.21.15.42:0/1107885742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9780abdd0 0x7fd978072100 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fd96800c6b0 tx=0x7fd96800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:03.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.535+0000 7fd96ea00640 1 -- 172.21.15.42:0/1107885742 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd968016020 con 0x7fd9780abdd0 2024-07-04T17:33:03.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.535+0000 7fd96ea00640 1 -- 172.21.15.42:0/1107885742 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd96800a5c0 con 0x7fd9780abdd0 2024-07-04T17:33:03.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.535+0000 7fd96ea00640 1 -- 172.21.15.42:0/1107885742 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd96800a960 con 0x7fd9780abdd0 2024-07-04T17:33:03.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.535+0000 7fd97d400640 1 -- 172.21.15.42:0/1107885742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9780726d0 con 0x7fd9780abdd0 2024-07-04T17:33:03.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.535+0000 7fd97d400640 1 -- 172.21.15.42:0/1107885742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9781dad70 con 0x7fd9780abdd0 2024-07-04T17:33:03.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.536+0000 7fd97d400640 1 -- 172.21.15.42:0/1107885742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9780b2da0 con 0x7fd9780abdd0 2024-07-04T17:33:03.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.537+0000 7fd96ea00640 1 -- 172.21.15.42:0/1107885742 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7fd968019020 con 0x7fd9780abdd0 2024-07-04T17:33:03.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.537+0000 7fd96ea00640 1 --2- 172.21.15.42:0/1107885742 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fd95403d4f0 0x7fd95403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:03.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.537+0000 7fd96ea00640 1 -- 172.21.15.42:0/1107885742 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 3619+0+0 (secure 0 0 0) 0x7fd9680518f0 con 0x7fd9780abdd0 2024-07-04T17:33:03.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.538+0000 7fd96fe00640 1 --2- 172.21.15.42:0/1107885742 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fd95403d4f0 0x7fd95403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:03.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.538+0000 7fd96fe00640 1 --2- 172.21.15.42:0/1107885742 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fd95403d4f0 0x7fd95403fa00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd960036000 tx=0x7fd960004730 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:03.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.542+0000 7fd96ea00640 1 -- 172.21.15.42:0/1107885742 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fd968020020 con 0x7fd9780abdd0 2024-07-04T17:33:03.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.794+0000 7fd97d400640 1 -- 172.21.15.42:0/1107885742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9780bdce0 con 0x7fd9780abdd0 2024-07-04T17:33:03.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.795+0000 7fd96ea00640 1 -- 172.21.15.42:0/1107885742 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+4727 (secure 0 0 0) 0x7fd9780b10e0 con 0x7fd9780abdd0 2024-07-04T17:33:03.797 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:33:03.797 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":83,"btime":"2024-07-04T17:32:55:868384+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5262,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/244742173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":244742173},{"type":"v1","addr":"172.21.15.42:6825","nonce":244742173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":5265,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/973681058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":973681058},{"type":"v1","addr":"172.21.15.42:6827","nonce":973681058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/480649366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":480649366},{"type":"v1","addr":"172.21.15.42:6829","nonce":480649366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5477,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6833/2552884551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":2552884551},{"type":"v1","addr":"172.21.15.42:6833","nonce":2552884551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":5502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/1001952349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1001952349},{"type":"v1","addr":"172.21.15.42:6831","nonce":1001952349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[]} 2024-07-04T17:33:03.797 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 83 2024-07-04T17:33:03.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.798+0000 7fd97d400640 1 -- 172.21.15.42:0/1107885742 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fd95403d4f0 msgr2=0x7fd95403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:03.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.798+0000 7fd97d400640 1 --2- 172.21.15.42:0/1107885742 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fd95403d4f0 0x7fd95403fa00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd960036000 tx=0x7fd960004730 comp rx=0 tx=0).stop 2024-07-04T17:33:03.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.798+0000 7fd97d400640 1 -- 172.21.15.42:0/1107885742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9780abdd0 msgr2=0x7fd978072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:03.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.798+0000 7fd97d400640 1 --2- 172.21.15.42:0/1107885742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9780abdd0 0x7fd978072100 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fd96800c6b0 tx=0x7fd96800cb70 comp rx=0 tx=0).stop 2024-07-04T17:33:03.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.798+0000 7fd97d400640 1 -- 172.21.15.42:0/1107885742 shutdown_connections 2024-07-04T17:33:03.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.798+0000 7fd97d400640 1 --2- 172.21.15.42:0/1107885742 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fd95403d4f0 0x7fd95403fa00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:03.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.799+0000 7fd97d400640 1 --2- 172.21.15.42:0/1107885742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9780abdd0 0x7fd978072100 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:03.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.799+0000 7fd97d400640 1 -- 172.21.15.42:0/1107885742 >> 172.21.15.42:0/1107885742 conn(0x7fd9780bf700 msgr2=0x7fd9780aaba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:03.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.799+0000 7fd97d400640 1 -- 172.21.15.42:0/1107885742 shutdown_connections 2024-07-04T17:33:03.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:03.799+0000 7fd97d400640 1 -- 172.21.15.42:0/1107885742 wait complete. 2024-07-04T17:33:03.827 INFO:tasks.ceph.mds.a:Restarting daemon 2024-07-04T17:33:03.827 INFO:tasks.ceph.mds.a:Stopping old one... 2024-07-04T17:33:03.827 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-07-04T17:33:03.827 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:33:03.827 INFO:tasks.ceph.mds.b:Restarting daemon 2024-07-04T17:33:03.827 INFO:tasks.ceph.mds.b:Stopping old one... 2024-07-04T17:33:03.828 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-07-04T17:33:03.828 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:33:03.828 INFO:tasks.ceph.mds.c:Restarting daemon 2024-07-04T17:33:03.828 INFO:tasks.ceph.mds.c:Stopping old one... 2024-07-04T17:33:03.828 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-07-04T17:33:03.828 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:33:03.828 INFO:tasks.ceph.mds.d:Restarting daemon 2024-07-04T17:33:03.829 INFO:tasks.ceph.mds.d:Stopping old one... 2024-07-04T17:33:03.829 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-07-04T17:33:03.829 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:33:03.829 INFO:tasks.ceph.mds.e:Restarting daemon 2024-07-04T17:33:03.829 INFO:tasks.ceph.mds.e:Stopping old one... 2024-07-04T17:33:03.829 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-07-04T17:33:03.829 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:33:03.855 INFO:tasks.ceph.mds.b:Stopped 2024-07-04T17:33:03.855 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-07-04T17:33:03.860 INFO:tasks.ceph.mds.a:Stopped 2024-07-04T17:33:03.860 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-07-04T17:33:03.864 INFO:tasks.ceph.mds.c:Stopped 2024-07-04T17:33:03.864 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-07-04T17:33:03.867 INFO:tasks.ceph.mds.b:Started 2024-07-04T17:33:03.868 DEBUG:teuthology.parallel:result is None 2024-07-04T17:33:03.874 INFO:tasks.ceph.mds.a:Started 2024-07-04T17:33:03.874 DEBUG:teuthology.parallel:result is None 2024-07-04T17:33:03.875 INFO:tasks.ceph.mds.d:Stopped 2024-07-04T17:33:03.875 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-07-04T17:33:03.876 INFO:tasks.ceph.mds.e:Stopped 2024-07-04T17:33:03.876 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-07-04T17:33:03.879 INFO:tasks.ceph.mds.c:Started 2024-07-04T17:33:03.879 DEBUG:teuthology.parallel:result is None 2024-07-04T17:33:03.882 INFO:tasks.ceph.mds.d:Started 2024-07-04T17:33:03.883 DEBUG:teuthology.parallel:result is None 2024-07-04T17:33:03.885 INFO:tasks.ceph.mds.e:Started 2024-07-04T17:33:03.885 DEBUG:teuthology.parallel:result is None 2024-07-04T17:33:03.886 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-07-04T17:33:04.140 INFO:tasks.ceph.mds.e.smithi042.stdout:starting mds.e at 2024-07-04T17:33:04.184 INFO:tasks.ceph.mds.a.smithi042.stdout:starting mds.a at 2024-07-04T17:33:04.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:04.185+0000 7f59a3400640 1 -- 172.21.15.42:0/1255432590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f599c0b38b0 msgr2=0x7f599c0b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:04.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:04.185+0000 7f59a3400640 1 --2- 172.21.15.42:0/1255432590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f599c0b38b0 0x7f599c0b2d20 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f59900097b0 tx=0x7f599002a4a0 comp rx=0 tx=0).stop 2024-07-04T17:33:04.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:04.185+0000 7f59a3400640 1 -- 172.21.15.42:0/1255432590 shutdown_connections 2024-07-04T17:33:04.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:04.185+0000 7f59a3400640 1 --2- 172.21.15.42:0/1255432590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f599c0b38b0 0x7f599c0b2d20 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:04.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:04.185+0000 7f59a3400640 1 -- 172.21.15.42:0/1255432590 >> 172.21.15.42:0/1255432590 conn(0x7f599c0bf700 msgr2=0x7f599c0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:04.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:04.185+0000 7f59a3400640 1 -- 172.21.15.42:0/1255432590 shutdown_connections 2024-07-04T17:33:04.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:04.185+0000 7f59a3400640 1 -- 172.21.15.42:0/1255432590 wait complete. 2024-07-04T17:33:04.187 INFO:tasks.ceph.mds.d.smithi042.stdout:starting mds.d at 2024-07-04T17:33:04.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:04.188+0000 7f59a3400640 1 Processor -- start 2024-07-04T17:33:04.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:04.188+0000 7f59a3400640 1 -- start start 2024-07-04T17:33:04.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:04.189+0000 7f59a3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f599c0b1e00 0x7f599c0aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:04.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:04.189+0000 7f59a3400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f599c0b06e0 con 0x7f599c0b1e00 2024-07-04T17:33:04.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:04.189+0000 7f59a0a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f599c0b1e00 0x7f599c0aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:04.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:04.190+0000 7f59a0a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f599c0b1e00 0x7f599c0aeb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32862/0 (socket says 172.21.15.42:32862) 2024-07-04T17:33:04.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:04.190+0000 7f59a0a00640 1 -- 172.21.15.42:0/4043858682 learned_addr learned my addr 172.21.15.42:0/4043858682 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:04.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:04.190+0000 7f59a0a00640 1 -- 172.21.15.42:0/4043858682 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f599c1e7630 con 0x7f599c0b1e00 2024-07-04T17:33:04.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:04.191+0000 7f59a0a00640 1 --2- 172.21.15.42:0/4043858682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f599c0b1e00 0x7f599c0aeb50 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f59900098d0 tx=0x7f5990033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:04.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:04.191+0000 7f5998c00640 1 -- 172.21.15.42:0/4043858682 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5990031a90 con 0x7f599c0b1e00 2024-07-04T17:33:04.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:04.191+0000 7f5998c00640 1 -- 172.21.15.42:0/4043858682 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5990031d20 con 0x7f599c0b1e00 2024-07-04T17:33:04.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:04.191+0000 7f5998c00640 1 -- 172.21.15.42:0/4043858682 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5990033070 con 0x7f599c0b1e00 2024-07-04T17:33:04.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:04.191+0000 7f59a3400640 1 -- 172.21.15.42:0/4043858682 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f599c1e7820 con 0x7f599c0b1e00 2024-07-04T17:33:04.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:04.191+0000 7f59a3400640 1 -- 172.21.15.42:0/4043858682 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f599c0700b0 con 0x7f599c0b1e00 2024-07-04T17:33:04.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:04.193+0000 7f59a3400640 1 -- 172.21.15.42:0/4043858682 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f599c06c5d0 con 0x7f599c0b1e00 2024-07-04T17:33:04.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:04.193+0000 7f5998c00640 1 -- 172.21.15.42:0/4043858682 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7f5990033210 con 0x7f599c0b1e00 2024-07-04T17:33:04.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:04.194+0000 7f5998c00640 1 --2- 172.21.15.42:0/4043858682 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f597c03d540 0x7f597c03fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:04.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:04.194+0000 7f5998c00640 1 -- 172.21.15.42:0/4043858682 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 3619+0+0 (secure 0 0 0) 0x7f5990072ca0 con 0x7f599c0b1e00 2024-07-04T17:33:04.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:04.195+0000 7f599be00640 1 --2- 172.21.15.42:0/4043858682 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f597c03d540 0x7f597c03fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:04.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:04.196+0000 7f599be00640 1 --2- 172.21.15.42:0/4043858682 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f597c03d540 0x7f597c03fa50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f599c06ed80 tx=0x7f59880079b0 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:04.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:04.200+0000 7f5998c00640 1 -- 172.21.15.42:0/4043858682 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f599003f030 con 0x7f599c0b1e00 2024-07-04T17:33:04.205 INFO:tasks.ceph.mds.b.smithi042.stdout:starting mds.b at 2024-07-04T17:33:04.208 INFO:tasks.ceph.mds.c.smithi042.stdout:starting mds.c at 2024-07-04T17:33:04.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:04.418+0000 7f59a3400640 1 -- 172.21.15.42:0/4043858682 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f599c069b70 con 0x7f599c0b1e00 2024-07-04T17:33:05.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.016+0000 7f5998c00640 1 -- 172.21.15.42:0/4043858682 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v107) ==== 97+0+0 (secure 0 0 0) 0x7f59900335b0 con 0x7f599c0b1e00 2024-07-04T17:33:05.018 INFO:teuthology.orchestra.run.smithi042.stderr: removed all blocklist entries 2024-07-04T17:33:05.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.020+0000 7f59a3400640 1 -- 172.21.15.42:0/4043858682 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f597c03d540 msgr2=0x7f597c03fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:05.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.020+0000 7f59a3400640 1 --2- 172.21.15.42:0/4043858682 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f597c03d540 0x7f597c03fa50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f599c06ed80 tx=0x7f59880079b0 comp rx=0 tx=0).stop 2024-07-04T17:33:05.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.020+0000 7f59a3400640 1 -- 172.21.15.42:0/4043858682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f599c0b1e00 msgr2=0x7f599c0aeb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:05.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.020+0000 7f59a3400640 1 --2- 172.21.15.42:0/4043858682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f599c0b1e00 0x7f599c0aeb50 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f59900098d0 tx=0x7f5990033c10 comp rx=0 tx=0).stop 2024-07-04T17:33:05.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.020+0000 7f59a3400640 1 -- 172.21.15.42:0/4043858682 shutdown_connections 2024-07-04T17:33:05.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.020+0000 7f59a3400640 1 --2- 172.21.15.42:0/4043858682 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f597c03d540 0x7f597c03fa50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:05.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.020+0000 7f59a3400640 1 --2- 172.21.15.42:0/4043858682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f599c0b1e00 0x7f599c0aeb50 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:05.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.020+0000 7f59a3400640 1 -- 172.21.15.42:0/4043858682 >> 172.21.15.42:0/4043858682 conn(0x7f599c0bf700 msgr2=0x7f599c0b0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:05.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.020+0000 7f59a3400640 1 -- 172.21.15.42:0/4043858682 shutdown_connections 2024-07-04T17:33:05.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.020+0000 7f59a3400640 1 -- 172.21.15.42:0/4043858682 wait complete. 2024-07-04T17:33:05.051 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-07-04T17:33:05.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.334+0000 7efda4c00640 1 -- 172.21.15.42:0/2948479153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efda00bbff0 msgr2=0x7efda00bd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:05.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.334+0000 7efda4c00640 1 --2- 172.21.15.42:0/2948479153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efda00bbff0 0x7efda00bd370 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7efd90005fd0 tx=0x7efd9002aac0 comp rx=0 tx=0).stop 2024-07-04T17:33:05.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.334+0000 7efda4c00640 1 -- 172.21.15.42:0/2948479153 shutdown_connections 2024-07-04T17:33:05.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.334+0000 7efda4c00640 1 --2- 172.21.15.42:0/2948479153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efda00bbff0 0x7efda00bd370 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:05.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.334+0000 7efda4c00640 1 -- 172.21.15.42:0/2948479153 >> 172.21.15.42:0/2948479153 conn(0x7efda006c9a0 msgr2=0x7efda006de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:05.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.335+0000 7efda4c00640 1 -- 172.21.15.42:0/2948479153 shutdown_connections 2024-07-04T17:33:05.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.335+0000 7efda4c00640 1 -- 172.21.15.42:0/2948479153 wait complete. 2024-07-04T17:33:05.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.337+0000 7efda4c00640 1 Processor -- start 2024-07-04T17:33:05.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.337+0000 7efda4c00640 1 -- start start 2024-07-04T17:33:05.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.339+0000 7efda4c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efda00bbff0 0x7efda01d9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:05.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.339+0000 7efda4c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efda00b7530 con 0x7efda00bbff0 2024-07-04T17:33:05.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.339+0000 7efd9de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efda00bbff0 0x7efda01d9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:05.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.339+0000 7efd9de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efda00bbff0 0x7efda01d9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32946/0 (socket says 172.21.15.42:32946) 2024-07-04T17:33:05.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.340+0000 7efd9de00640 1 -- 172.21.15.42:0/115585817 learned_addr learned my addr 172.21.15.42:0/115585817 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:05.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.340+0000 7efd9de00640 1 -- 172.21.15.42:0/115585817 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda00c3110 con 0x7efda00bbff0 2024-07-04T17:33:05.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.340+0000 7efd9de00640 1 --2- 172.21.15.42:0/115585817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efda00bbff0 0x7efda01d9400 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7efd90006010 tx=0x7efd90003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:05.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.341+0000 7efd96000640 1 -- 172.21.15.42:0/115585817 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd90044070 con 0x7efda00bbff0 2024-07-04T17:33:05.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.341+0000 7efd96000640 1 -- 172.21.15.42:0/115585817 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd90004250 con 0x7efda00bbff0 2024-07-04T17:33:05.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.341+0000 7efd96000640 1 -- 172.21.15.42:0/115585817 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd900045d0 con 0x7efda00bbff0 2024-07-04T17:33:05.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.341+0000 7efda4c00640 1 -- 172.21.15.42:0/115585817 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda00c33b0 con 0x7efda00bbff0 2024-07-04T17:33:05.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.341+0000 7efda4c00640 1 -- 172.21.15.42:0/115585817 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda01d9940 con 0x7efda00bbff0 2024-07-04T17:33:05.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.342+0000 7efda4c00640 1 -- 172.21.15.42:0/115585817 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efda00bcc90 con 0x7efda00bbff0 2024-07-04T17:33:05.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.342+0000 7efd96000640 1 -- 172.21.15.42:0/115585817 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7efd90047070 con 0x7efda00bbff0 2024-07-04T17:33:05.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.343+0000 7efd96000640 1 --2- 172.21.15.42:0/115585817 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7efd7c03d4f0 0x7efd7c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:05.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.343+0000 7efd96000640 1 -- 172.21.15.42:0/115585817 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 2587+0+0 (secure 0 0 0) 0x7efd9003c070 con 0x7efda00bbff0 2024-07-04T17:33:05.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.343+0000 7efd9d400640 1 --2- 172.21.15.42:0/115585817 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7efd7c03d4f0 0x7efd7c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:05.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.344+0000 7efd9d400640 1 --2- 172.21.15.42:0/115585817 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7efd7c03d4f0 0x7efd7c03fa00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efd88001560 tx=0x7efd880079f0 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:05.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.348+0000 7efd96000640 1 -- 172.21.15.42:0/115585817 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7efda00bcc90 con 0x7efda00bbff0 2024-07-04T17:33:05.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.634+0000 7efda4c00640 1 -- 172.21.15.42:0/115585817 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7efda00b7b70 con 0x7efda00bbff0 2024-07-04T17:33:05.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.635+0000 7efd96000640 1 -- 172.21.15.42:0/115585817 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+4064 (secure 0 0 0) 0x7efd90032030 con 0x7efda00bbff0 2024-07-04T17:33:05.637 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:33:05.637 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:33:05.637 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_dump": [ 2024-07-04T17:33:05.637 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:05.637 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.a", 2024-07-04T17:33:05.637 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAM24Zm2VMEFxAAQqdcGux6Ker7jEr1dGAfzA==", 2024-07-04T17:33:05.637 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:33:05.637 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:33:05.637 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:33:05.637 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:33:05.638 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:33:05.638 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:05.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:05.638 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:05.638 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.b", 2024-07-04T17:33:05.638 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAM24ZmS/S7KxAAfjTZrLYR9qLxuTvd2zsCcQ==", 2024-07-04T17:33:05.638 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:33:05.638 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:33:05.638 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:33:05.638 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:33:05.638 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:33:05.638 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:05.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:05.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:05.639 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.c", 2024-07-04T17:33:05.639 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAN24Zm2JrRBBAAFMYIM2fLsr9EOfcyh3tZpA==", 2024-07-04T17:33:05.639 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:33:05.639 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:33:05.639 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:33:05.639 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:33:05.639 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:33:05.639 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:05.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:05.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:05.639 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.d", 2024-07-04T17:33:05.640 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAN24ZmPCGLFxAAhsh3XaAHCyauCGxh/3DFsw==", 2024-07-04T17:33:05.640 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:33:05.640 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:33:05.640 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:33:05.640 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:33:05.640 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:33:05.640 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:05.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:05.640 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:05.640 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.e", 2024-07-04T17:33:05.640 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAN24ZmVIaSKhAAhoBdDalwQ4eoo0rqb6p0Vw==", 2024-07-04T17:33:05.640 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:33:05.640 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:33:05.641 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:33:05.641 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:33:05.641 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:33:05.641 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:05.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:05.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:05.641 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.0", 2024-07-04T17:33:05.641 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAQ24Zm4spcCBAA4kTvvR7IOVa01K5ZCrJ35w==", 2024-07-04T17:33:05.641 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:33:05.641 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-07-04T17:33:05.641 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-07-04T17:33:05.641 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:33:05.641 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:05.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:05.641 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:05.642 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.1", 2024-07-04T17:33:05.642 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAT24Zmz1sfGRAAgfH5eIiQrufnPq6F94rHWg==", 2024-07-04T17:33:05.642 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:33:05.642 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-07-04T17:33:05.642 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-07-04T17:33:05.642 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:33:05.642 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:05.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:05.642 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:05.642 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.2", 2024-07-04T17:33:05.642 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAW24ZmJdQHORAAEjAM9OBqSIRhX148GC6pcw==", 2024-07-04T17:33:05.642 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:33:05.643 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-07-04T17:33:05.643 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-07-04T17:33:05.643 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:33:05.643 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:05.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:05.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:05.643 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.0", 2024-07-04T17:33:05.643 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAO24ZmUMmYABAAh4pNlkb9JsHQtK8uCJFQCA==", 2024-07-04T17:33:05.643 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:33:05.643 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:33:05.643 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow r", 2024-07-04T17:33:05.643 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-07-04T17:33:05.643 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:05.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:05.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:05.644 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.1", 2024-07-04T17:33:05.644 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAO24Zm0pAJEBAAntDeQ/oWDlJ+3xC7q5EQnQ==", 2024-07-04T17:33:05.644 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:33:05.644 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:33:05.644 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-07-04T17:33:05.644 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow rw", 2024-07-04T17:33:05.644 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rwx" 2024-07-04T17:33:05.644 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:05.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:05.644 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:05.644 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.admin", 2024-07-04T17:33:05.645 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAL24ZmMe3iHhAAFciH9cbUAIHLSAJvXy30EQ==", 2024-07-04T17:33:05.645 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:33:05.645 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-07-04T17:33:05.645 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:33:05.645 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:33:05.645 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:33:05.645 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:05.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:05.645 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:05.645 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.mirror", 2024-07-04T17:33:05.645 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAO24ZmBJzBHhAAR4Zg36YTtVizxm/g/Cdk0w==", 2024-07-04T17:33:05.645 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:33:05.645 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow r", 2024-07-04T17:33:05.646 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-07-04T17:33:05.646 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "profile cephfs-mirror", 2024-07-04T17:33:05.646 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rw tag cephfs metadata=*, allow r tag cephfs data=*" 2024-07-04T17:33:05.646 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:05.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:05.646 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:05.646 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.mirror_remote", 2024-07-04T17:33:05.646 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAO24Zm+BsqLhAA19GVuEjfWQeMe/fSc+dmoA==", 2024-07-04T17:33:05.646 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:33:05.646 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow rwps", 2024-07-04T17:33:05.647 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-07-04T17:33:05.647 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow r", 2024-07-04T17:33:05.647 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rw tag cephfs *=*" 2024-07-04T17:33:05.647 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:05.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:05.647 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:05.647 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mgr.x", 2024-07-04T17:33:05.647 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAM24Zm47tDCBAAjszudEEB4QHLlWqPXyZc2g==", 2024-07-04T17:33:05.647 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:33:05.647 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-07-04T17:33:05.647 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile mgr", 2024-07-04T17:33:05.647 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:33:05.647 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:05.647 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:05.648 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-07-04T17:33:05.648 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:33:05.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.638+0000 7efda4c00640 1 -- 172.21.15.42:0/115585817 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7efd7c03d4f0 msgr2=0x7efd7c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:05.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.638+0000 7efda4c00640 1 --2- 172.21.15.42:0/115585817 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7efd7c03d4f0 0x7efd7c03fa00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efd88001560 tx=0x7efd880079f0 comp rx=0 tx=0).stop 2024-07-04T17:33:05.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.638+0000 7efda4c00640 1 -- 172.21.15.42:0/115585817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efda00bbff0 msgr2=0x7efda01d9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:05.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.638+0000 7efda4c00640 1 --2- 172.21.15.42:0/115585817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efda00bbff0 0x7efda01d9400 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7efd90006010 tx=0x7efd90003f10 comp rx=0 tx=0).stop 2024-07-04T17:33:05.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.638+0000 7efda4c00640 1 -- 172.21.15.42:0/115585817 shutdown_connections 2024-07-04T17:33:05.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.638+0000 7efda4c00640 1 --2- 172.21.15.42:0/115585817 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7efd7c03d4f0 0x7efd7c03fa00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:05.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.639+0000 7efda4c00640 1 --2- 172.21.15.42:0/115585817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efda00bbff0 0x7efda01d9400 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:05.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.639+0000 7efda4c00640 1 -- 172.21.15.42:0/115585817 >> 172.21.15.42:0/115585817 conn(0x7efda006c9a0 msgr2=0x7efda00740b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:05.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.639+0000 7efda4c00640 1 -- 172.21.15.42:0/115585817 shutdown_connections 2024-07-04T17:33:05.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.639+0000 7efda4c00640 1 -- 172.21.15.42:0/115585817 wait complete. 2024-07-04T17:33:05.665 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-04T17:33:05.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.958+0000 7f164e800640 1 -- 172.21.15.42:0/1305584721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16480b2d20 msgr2=0x7f16480b37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:05.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.958+0000 7f164e800640 1 --2- 172.21.15.42:0/1305584721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16480b2d20 0x7f16480b37d0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f1638005fd0 tx=0x7f163802aac0 comp rx=0 tx=0).stop 2024-07-04T17:33:05.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.958+0000 7f164e800640 1 -- 172.21.15.42:0/1305584721 shutdown_connections 2024-07-04T17:33:05.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.958+0000 7f164e800640 1 --2- 172.21.15.42:0/1305584721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16480b2d20 0x7f16480b37d0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:05.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.958+0000 7f164e800640 1 -- 172.21.15.42:0/1305584721 >> 172.21.15.42:0/1305584721 conn(0x7f16480bf700 msgr2=0x7f16480af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:05.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.958+0000 7f164e800640 1 -- 172.21.15.42:0/1305584721 shutdown_connections 2024-07-04T17:33:05.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.958+0000 7f164e800640 1 -- 172.21.15.42:0/1305584721 wait complete. 2024-07-04T17:33:05.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.961+0000 7f164e800640 1 Processor -- start 2024-07-04T17:33:05.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.961+0000 7f164e800640 1 -- start start 2024-07-04T17:33:05.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.962+0000 7f164e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16480b01f0 0x7f1648141530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:05.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.962+0000 7f164e800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f164806cc20 con 0x7f16480b01f0 2024-07-04T17:33:05.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.962+0000 7f1647400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16480b01f0 0x7f1648141530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:05.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.962+0000 7f1647400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16480b01f0 0x7f1648141530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32972/0 (socket says 172.21.15.42:32972) 2024-07-04T17:33:05.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.962+0000 7f1647400640 1 -- 172.21.15.42:0/3167281346 learned_addr learned my addr 172.21.15.42:0/3167281346 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:05.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.963+0000 7f1647400640 1 -- 172.21.15.42:0/3167281346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16480ac990 con 0x7f16480b01f0 2024-07-04T17:33:05.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.963+0000 7f1647400640 1 --2- 172.21.15.42:0/3167281346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16480b01f0 0x7f1648141530 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f1638006010 tx=0x7f1638003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:05.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.964+0000 7f163fe00640 1 -- 172.21.15.42:0/3167281346 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1638044070 con 0x7f16480b01f0 2024-07-04T17:33:05.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.964+0000 7f163fe00640 1 -- 172.21.15.42:0/3167281346 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1638004250 con 0x7f16480b01f0 2024-07-04T17:33:05.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.964+0000 7f163fe00640 1 -- 172.21.15.42:0/3167281346 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16380045d0 con 0x7f16480b01f0 2024-07-04T17:33:05.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.964+0000 7f164e800640 1 -- 172.21.15.42:0/3167281346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16481e7780 con 0x7f16480b01f0 2024-07-04T17:33:05.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.964+0000 7f164e800640 1 -- 172.21.15.42:0/3167281346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16480ace70 con 0x7f16480b01f0 2024-07-04T17:33:05.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.965+0000 7f164e800640 1 -- 172.21.15.42:0/3167281346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f164806d2c0 con 0x7f16480b01f0 2024-07-04T17:33:05.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.965+0000 7f163fe00640 1 -- 172.21.15.42:0/3167281346 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7f1638047070 con 0x7f16480b01f0 2024-07-04T17:33:05.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.966+0000 7f163fe00640 1 --2- 172.21.15.42:0/3167281346 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f162403d4f0 0x7f162403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:05.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.966+0000 7f163fe00640 1 -- 172.21.15.42:0/3167281346 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 2587+0+0 (secure 0 0 0) 0x7f163803c070 con 0x7f16480b01f0 2024-07-04T17:33:05.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.966+0000 7f163ea00640 1 --2- 172.21.15.42:0/3167281346 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f162403d4f0 0x7f162403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.967+0000 7f163ea00640 1 --2- 172.21.15.42:0/3167281346 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f162403d4f0 0x7f162403fa00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1630001560 tx=0x7f16300079f0 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:05.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:05.971+0000 7f163fe00640 1 -- 172.21.15.42:0/3167281346 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f1638032030 con 0x7f16480b01f0 2024-07-04T17:33:06.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.192+0000 7f164e800640 1 -- 172.21.15.42:0/3167281346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f16480af6e0 con 0x7f16480b01f0 2024-07-04T17:33:06.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.193+0000 7f163fe00640 1 -- 172.21.15.42:0/3167281346 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v110) ==== 84+0+65 (secure 0 0 0) 0x7f1638047360 con 0x7f16480b01f0 2024-07-04T17:33:06.194 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:33:06.194 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-07-04T17:33:06.194 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:06.195 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 1, 2024-07-04T17:33:06.195 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": ".mgr" 2024-07-04T17:33:06.195 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:06.195 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-07-04T17:33:06.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.195+0000 7f164e800640 1 -- 172.21.15.42:0/3167281346 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f162403d4f0 msgr2=0x7f162403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:06.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.195+0000 7f164e800640 1 --2- 172.21.15.42:0/3167281346 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f162403d4f0 0x7f162403fa00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1630001560 tx=0x7f16300079f0 comp rx=0 tx=0).stop 2024-07-04T17:33:06.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.196+0000 7f164e800640 1 -- 172.21.15.42:0/3167281346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16480b01f0 msgr2=0x7f1648141530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:06.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.196+0000 7f164e800640 1 --2- 172.21.15.42:0/3167281346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16480b01f0 0x7f1648141530 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f1638006010 tx=0x7f1638003f10 comp rx=0 tx=0).stop 2024-07-04T17:33:06.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.196+0000 7f164e800640 1 -- 172.21.15.42:0/3167281346 shutdown_connections 2024-07-04T17:33:06.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.196+0000 7f164e800640 1 --2- 172.21.15.42:0/3167281346 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f162403d4f0 0x7f162403fa00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:06.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.196+0000 7f164e800640 1 --2- 172.21.15.42:0/3167281346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16480b01f0 0x7f1648141530 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:06.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.196+0000 7f164e800640 1 -- 172.21.15.42:0/3167281346 >> 172.21.15.42:0/3167281346 conn(0x7f16480bf700 msgr2=0x7f16480bc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:06.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.196+0000 7f164e800640 1 -- 172.21.15.42:0/3167281346 shutdown_connections 2024-07-04T17:33:06.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.196+0000 7f164e800640 1 -- 172.21.15.42:0/3167281346 wait complete. 2024-07-04T17:33:06.223 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-04T17:33:06.224 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-04T17:33:06.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.520+0000 7fb544a00640 1 -- 172.21.15.42:0/3259754821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb540073a20 msgr2=0x7fb54013e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:06.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.520+0000 7fb544a00640 1 --2- 172.21.15.42:0/3259754821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb540073a20 0x7fb54013e1a0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fb528005fd0 tx=0x7fb52802a5f0 comp rx=0 tx=0).stop 2024-07-04T17:33:06.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.520+0000 7fb544a00640 1 -- 172.21.15.42:0/3259754821 shutdown_connections 2024-07-04T17:33:06.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.520+0000 7fb544a00640 1 --2- 172.21.15.42:0/3259754821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb540073a20 0x7fb54013e1a0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:06.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.520+0000 7fb544a00640 1 -- 172.21.15.42:0/3259754821 >> 172.21.15.42:0/3259754821 conn(0x7fb54006c020 msgr2=0x7fb54006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:06.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.520+0000 7fb544a00640 1 -- 172.21.15.42:0/3259754821 shutdown_connections 2024-07-04T17:33:06.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.520+0000 7fb544a00640 1 -- 172.21.15.42:0/3259754821 wait complete. 2024-07-04T17:33:06.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.523+0000 7fb544a00640 1 Processor -- start 2024-07-04T17:33:06.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.523+0000 7fb544a00640 1 -- start start 2024-07-04T17:33:06.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.524+0000 7fb544a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb54006d150 0x7fb540144ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:06.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.524+0000 7fb544a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb540071c40 con 0x7fb54006d150 2024-07-04T17:33:06.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.524+0000 7fb53de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb54006d150 0x7fb540144ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:06.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.524+0000 7fb53de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb54006d150 0x7fb540144ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33004/0 (socket says 172.21.15.42:33004) 2024-07-04T17:33:06.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.524+0000 7fb53de00640 1 -- 172.21.15.42:0/568838220 learned_addr learned my addr 172.21.15.42:0/568838220 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:06.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.525+0000 7fb53de00640 1 -- 172.21.15.42:0/568838220 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5401452b0 con 0x7fb54006d150 2024-07-04T17:33:06.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.525+0000 7fb53de00640 1 --2- 172.21.15.42:0/568838220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb54006d150 0x7fb540144ca0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fb53000c700 tx=0x7fb53000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:06.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.525+0000 7fb536000640 1 -- 172.21.15.42:0/568838220 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb530003970 con 0x7fb54006d150 2024-07-04T17:33:06.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.525+0000 7fb536000640 1 -- 172.21.15.42:0/568838220 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb530003b10 con 0x7fb54006d150 2024-07-04T17:33:06.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.525+0000 7fb536000640 1 -- 172.21.15.42:0/568838220 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb53000a820 con 0x7fb54006d150 2024-07-04T17:33:06.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.525+0000 7fb544a00640 1 -- 172.21.15.42:0/568838220 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5401e7710 con 0x7fb54006d150 2024-07-04T17:33:06.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.526+0000 7fb544a00640 1 -- 172.21.15.42:0/568838220 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5401459b0 con 0x7fb54006d150 2024-07-04T17:33:06.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.527+0000 7fb544a00640 1 -- 172.21.15.42:0/568838220 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb54013e1a0 con 0x7fb54006d150 2024-07-04T17:33:06.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.527+0000 7fb536000640 1 -- 172.21.15.42:0/568838220 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7fb53000a9c0 con 0x7fb54006d150 2024-07-04T17:33:06.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.527+0000 7fb536000640 1 --2- 172.21.15.42:0/568838220 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fb51c03d4f0 0x7fb51c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:06.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.528+0000 7fb536000640 1 -- 172.21.15.42:0/568838220 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 2587+0+0 (secure 0 0 0) 0x7fb530051490 con 0x7fb54006d150 2024-07-04T17:33:06.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.528+0000 7fb53d400640 1 --2- 172.21.15.42:0/568838220 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fb51c03d4f0 0x7fb51c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:06.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.529+0000 7fb53d400640 1 --2- 172.21.15.42:0/568838220 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fb51c03d4f0 0x7fb51c03fa00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb528033000 tx=0x7fb528004730 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:06.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.533+0000 7fb536000640 1 -- 172.21.15.42:0/568838220 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fb54006d560 con 0x7fb54006d150 2024-07-04T17:33:06.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:06.746+0000 7fb544a00640 1 -- 172.21.15.42:0/568838220 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb540069b70 con 0x7fb54006d150 2024-07-04T17:33:07.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.044+0000 7fb536000640 1 -- 172.21.15.42:0/568838220 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v111) ==== 138+0+0 (secure 0 0 0) 0x7fb53000d150 con 0x7fb54006d150 2024-07-04T17:33:07.046 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' created 2024-07-04T17:33:07.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.048+0000 7fb544a00640 1 -- 172.21.15.42:0/568838220 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fb51c03d4f0 msgr2=0x7fb51c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:07.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.049+0000 7fb544a00640 1 --2- 172.21.15.42:0/568838220 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fb51c03d4f0 0x7fb51c03fa00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb528033000 tx=0x7fb528004730 comp rx=0 tx=0).stop 2024-07-04T17:33:07.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.049+0000 7fb544a00640 1 -- 172.21.15.42:0/568838220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb54006d150 msgr2=0x7fb540144ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:07.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.049+0000 7fb544a00640 1 --2- 172.21.15.42:0/568838220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb54006d150 0x7fb540144ca0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fb53000c700 tx=0x7fb53000cbc0 comp rx=0 tx=0).stop 2024-07-04T17:33:07.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.049+0000 7fb544a00640 1 -- 172.21.15.42:0/568838220 shutdown_connections 2024-07-04T17:33:07.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.049+0000 7fb544a00640 1 --2- 172.21.15.42:0/568838220 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fb51c03d4f0 0x7fb51c03fa00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:07.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.049+0000 7fb544a00640 1 --2- 172.21.15.42:0/568838220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb54006d150 0x7fb540144ca0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:07.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.049+0000 7fb544a00640 1 -- 172.21.15.42:0/568838220 >> 172.21.15.42:0/568838220 conn(0x7fb54006c020 msgr2=0x7fb5400b05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:07.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.049+0000 7fb544a00640 1 -- 172.21.15.42:0/568838220 shutdown_connections 2024-07-04T17:33:07.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.049+0000 7fb544a00640 1 -- 172.21.15.42:0/568838220 wait complete. 2024-07-04T17:33:07.085 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-04T17:33:07.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.369+0000 7f237ba00640 1 -- 172.21.15.42:0/1493716574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23740ab310 msgr2=0x7f23740a9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:07.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.369+0000 7f237ba00640 1 --2- 172.21.15.42:0/1493716574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23740ab310 0x7f23740a9990 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f2368005fd0 tx=0x7f236802a5f0 comp rx=0 tx=0).stop 2024-07-04T17:33:07.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.370+0000 7f237ba00640 1 -- 172.21.15.42:0/1493716574 shutdown_connections 2024-07-04T17:33:07.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.370+0000 7f237ba00640 1 --2- 172.21.15.42:0/1493716574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23740ab310 0x7f23740a9990 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:07.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.370+0000 7f237ba00640 1 -- 172.21.15.42:0/1493716574 >> 172.21.15.42:0/1493716574 conn(0x7f237406c020 msgr2=0x7f237406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:07.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.370+0000 7f237ba00640 1 -- 172.21.15.42:0/1493716574 shutdown_connections 2024-07-04T17:33:07.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.370+0000 7f237ba00640 1 -- 172.21.15.42:0/1493716574 wait complete. 2024-07-04T17:33:07.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.372+0000 7f237ba00640 1 Processor -- start 2024-07-04T17:33:07.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.372+0000 7f237ba00640 1 -- start start 2024-07-04T17:33:07.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.373+0000 7f237ba00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23740aa6f0 0x7f2374143a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:07.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.373+0000 7f237ba00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2374072f10 con 0x7f23740aa6f0 2024-07-04T17:33:07.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.374+0000 7f2379000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23740aa6f0 0x7f2374143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:07.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.374+0000 7f2379000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23740aa6f0 0x7f2374143a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33022/0 (socket says 172.21.15.42:33022) 2024-07-04T17:33:07.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.374+0000 7f2379000640 1 -- 172.21.15.42:0/2882915205 learned_addr learned my addr 172.21.15.42:0/2882915205 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:07.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.374+0000 7f2379000640 1 -- 172.21.15.42:0/2882915205 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23740ac280 con 0x7f23740aa6f0 2024-07-04T17:33:07.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.375+0000 7f2379000640 1 --2- 172.21.15.42:0/2882915205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23740aa6f0 0x7f2374143a40 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f2368009850 tx=0x7f23680049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:07.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.375+0000 7f2370c00640 1 -- 172.21.15.42:0/2882915205 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f236803e030 con 0x7f23740aa6f0 2024-07-04T17:33:07.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.375+0000 7f2370c00640 1 -- 172.21.15.42:0/2882915205 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23680035f0 con 0x7f23740aa6f0 2024-07-04T17:33:07.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.375+0000 7f2370c00640 1 -- 172.21.15.42:0/2882915205 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2368035040 con 0x7f23740aa6f0 2024-07-04T17:33:07.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.375+0000 7f237ba00640 1 -- 172.21.15.42:0/2882915205 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23740add20 con 0x7f23740aa6f0 2024-07-04T17:33:07.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.375+0000 7f237ba00640 1 -- 172.21.15.42:0/2882915205 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23740aba00 con 0x7f23740aa6f0 2024-07-04T17:33:07.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.376+0000 7f237ba00640 1 -- 172.21.15.42:0/2882915205 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2374073550 con 0x7f23740aa6f0 2024-07-04T17:33:07.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.377+0000 7f2370c00640 1 -- 172.21.15.42:0/2882915205 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7f2368041030 con 0x7f23740aa6f0 2024-07-04T17:33:07.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.377+0000 7f2370c00640 1 --2- 172.21.15.42:0/2882915205 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f235403d540 0x7f235403fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:07.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.377+0000 7f2370c00640 1 -- 172.21.15.42:0/2882915205 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 2957+0+0 (secure 0 0 0) 0x7f236803f070 con 0x7f23740aa6f0 2024-07-04T17:33:07.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.378+0000 7f2373e00640 1 --2- 172.21.15.42:0/2882915205 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f235403d540 0x7f235403fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:07.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.378+0000 7f2373e00640 1 --2- 172.21.15.42:0/2882915205 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f235403d540 0x7f235403fa50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f237406f2d0 tx=0x7f23600079f0 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:07.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.383+0000 7f2370c00640 1 -- 172.21.15.42:0/2882915205 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f236804bd60 con 0x7f23740aa6f0 2024-07-04T17:33:07.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:07.595+0000 7f237ba00640 1 -- 172.21.15.42:0/2882915205 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f23740ac450 con 0x7f23740aa6f0 2024-07-04T17:33:08.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.059+0000 7f2370c00640 1 -- 172.21.15.42:0/2882915205 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v112) ==== 170+0+0 (secure 0 0 0) 0x7f2368030cc0 con 0x7f23740aa6f0 2024-07-04T17:33:08.061 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' created 2024-07-04T17:33:08.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.063+0000 7f237ba00640 1 -- 172.21.15.42:0/2882915205 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f235403d540 msgr2=0x7f235403fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:08.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.063+0000 7f237ba00640 1 --2- 172.21.15.42:0/2882915205 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f235403d540 0x7f235403fa50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f237406f2d0 tx=0x7f23600079f0 comp rx=0 tx=0).stop 2024-07-04T17:33:08.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.063+0000 7f237ba00640 1 -- 172.21.15.42:0/2882915205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23740aa6f0 msgr2=0x7f2374143a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:08.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.063+0000 7f237ba00640 1 --2- 172.21.15.42:0/2882915205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23740aa6f0 0x7f2374143a40 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f2368009850 tx=0x7f23680049d0 comp rx=0 tx=0).stop 2024-07-04T17:33:08.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.063+0000 7f237ba00640 1 -- 172.21.15.42:0/2882915205 shutdown_connections 2024-07-04T17:33:08.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.063+0000 7f237ba00640 1 --2- 172.21.15.42:0/2882915205 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f235403d540 0x7f235403fa50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:08.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.063+0000 7f237ba00640 1 --2- 172.21.15.42:0/2882915205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23740aa6f0 0x7f2374143a40 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:08.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.063+0000 7f237ba00640 1 -- 172.21.15.42:0/2882915205 >> 172.21.15.42:0/2882915205 conn(0x7f237406c020 msgr2=0x7f23740b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:08.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.065+0000 7f237ba00640 1 -- 172.21.15.42:0/2882915205 shutdown_connections 2024-07-04T17:33:08.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.066+0000 7f237ba00640 1 -- 172.21.15.42:0/2882915205 wait complete. 2024-07-04T17:33:08.118 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-04T17:33:08.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.378+0000 7fc7e4600640 1 -- 172.21.15.42:0/334970025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7dc0b1100 msgr2=0x7fc7dc13f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:08.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.378+0000 7fc7e4600640 1 --2- 172.21.15.42:0/334970025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7dc0b1100 0x7fc7dc13f800 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fc7c80097b0 tx=0x7fc7c802a540 comp rx=0 tx=0).stop 2024-07-04T17:33:08.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.379+0000 7fc7e4600640 1 -- 172.21.15.42:0/334970025 shutdown_connections 2024-07-04T17:33:08.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.379+0000 7fc7e4600640 1 --2- 172.21.15.42:0/334970025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7dc0b1100 0x7fc7dc13f800 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:08.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.379+0000 7fc7e4600640 1 -- 172.21.15.42:0/334970025 >> 172.21.15.42:0/334970025 conn(0x7fc7dc06c020 msgr2=0x7fc7dc06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:08.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.379+0000 7fc7e4600640 1 -- 172.21.15.42:0/334970025 shutdown_connections 2024-07-04T17:33:08.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.379+0000 7fc7e4600640 1 -- 172.21.15.42:0/334970025 wait complete. 2024-07-04T17:33:08.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.381+0000 7fc7e4600640 1 Processor -- start 2024-07-04T17:33:08.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.381+0000 7fc7e4600640 1 -- start start 2024-07-04T17:33:08.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.383+0000 7fc7e4600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7dc0a94d0 0x7fc7dc0aa920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:08.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.383+0000 7fc7e4600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc7dc1401e0 con 0x7fc7dc0a94d0 2024-07-04T17:33:08.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.383+0000 7fc7e1c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7dc0a94d0 0x7fc7dc0aa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:08.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.383+0000 7fc7e1c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7dc0a94d0 0x7fc7dc0aa920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33052/0 (socket says 172.21.15.42:33052) 2024-07-04T17:33:08.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.383+0000 7fc7e1c00640 1 -- 172.21.15.42:0/1810187306 learned_addr learned my addr 172.21.15.42:0/1810187306 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:08.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.383+0000 7fc7e1c00640 1 -- 172.21.15.42:0/1810187306 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7dc1e2640 con 0x7fc7dc0a94d0 2024-07-04T17:33:08.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.384+0000 7fc7e1c00640 1 --2- 172.21.15.42:0/1810187306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7dc0a94d0 0x7fc7dc0aa920 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc00c6b0 tx=0x7fc7cc00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:08.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.384+0000 7fc7d9600640 1 -- 172.21.15.42:0/1810187306 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7cc007e10 con 0x7fc7dc0a94d0 2024-07-04T17:33:08.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.384+0000 7fc7e4600640 1 -- 172.21.15.42:0/1810187306 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7dc0acc30 con 0x7fc7dc0a94d0 2024-07-04T17:33:08.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.384+0000 7fc7d9600640 1 -- 172.21.15.42:0/1810187306 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7cc017070 con 0x7fc7dc0a94d0 2024-07-04T17:33:08.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.384+0000 7fc7d9600640 1 -- 172.21.15.42:0/1810187306 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7cc006b50 con 0x7fc7dc0a94d0 2024-07-04T17:33:08.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.384+0000 7fc7e4600640 1 -- 172.21.15.42:0/1810187306 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7dc1d54b0 con 0x7fc7dc0a94d0 2024-07-04T17:33:08.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.386+0000 7fc7e4600640 1 -- 172.21.15.42:0/1810187306 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7dc0aae60 con 0x7fc7dc0a94d0 2024-07-04T17:33:08.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.386+0000 7fc7d9600640 1 -- 172.21.15.42:0/1810187306 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7fc7cc006cf0 con 0x7fc7dc0a94d0 2024-07-04T17:33:08.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.386+0000 7fc7d9600640 1 --2- 172.21.15.42:0/1810187306 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fc7bc03d4a0 0x7fc7bc03f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:08.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.387+0000 7fc7d9600640 1 -- 172.21.15.42:0/1810187306 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 3339+0+0 (secure 0 0 0) 0x7fc7cc051600 con 0x7fc7dc0a94d0 2024-07-04T17:33:08.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.387+0000 7fc7e1200640 1 --2- 172.21.15.42:0/1810187306 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fc7bc03d4a0 0x7fc7bc03f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:08.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.388+0000 7fc7e1200640 1 --2- 172.21.15.42:0/1810187306 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fc7bc03d4a0 0x7fc7bc03f9b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8034000 tx=0x7fc7c8004730 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:08.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.393+0000 7fc7d9600640 1 -- 172.21.15.42:0/1810187306 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fc7dc0aae60 con 0x7fc7dc0a94d0 2024-07-04T17:33:08.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:08.647+0000 7fc7e4600640 1 -- 172.21.15.42:0/1810187306 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc7dc13f800 con 0x7fc7dc0a94d0 2024-07-04T17:33:09.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.061+0000 7fc7d9600640 1 -- 172.21.15.42:0/1810187306 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v86) ==== 174+0+0 (secure 0 0 0) 0x7fc7dc1e2640 con 0x7fc7dc0a94d0 2024-07-04T17:33:09.063 INFO:teuthology.orchestra.run.smithi042.stderr:new fs with metadata pool 20 and data pool 21 2024-07-04T17:33:09.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.066+0000 7fc7e4600640 1 -- 172.21.15.42:0/1810187306 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fc7bc03d4a0 msgr2=0x7fc7bc03f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:09.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.066+0000 7fc7e4600640 1 --2- 172.21.15.42:0/1810187306 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fc7bc03d4a0 0x7fc7bc03f9b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8034000 tx=0x7fc7c8004730 comp rx=0 tx=0).stop 2024-07-04T17:33:09.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.066+0000 7fc7e4600640 1 -- 172.21.15.42:0/1810187306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7dc0a94d0 msgr2=0x7fc7dc0aa920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:09.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.066+0000 7fc7e4600640 1 --2- 172.21.15.42:0/1810187306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7dc0a94d0 0x7fc7dc0aa920 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc00c6b0 tx=0x7fc7cc00cb70 comp rx=0 tx=0).stop 2024-07-04T17:33:09.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.066+0000 7fc7e4600640 1 -- 172.21.15.42:0/1810187306 shutdown_connections 2024-07-04T17:33:09.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.066+0000 7fc7e4600640 1 --2- 172.21.15.42:0/1810187306 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fc7bc03d4a0 0x7fc7bc03f9b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:09.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.066+0000 7fc7e4600640 1 --2- 172.21.15.42:0/1810187306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7dc0a94d0 0x7fc7dc0aa920 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:09.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.066+0000 7fc7e4600640 1 -- 172.21.15.42:0/1810187306 >> 172.21.15.42:0/1810187306 conn(0x7fc7dc06c020 msgr2=0x7fc7dc0b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:09.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.066+0000 7fc7e4600640 1 -- 172.21.15.42:0/1810187306 shutdown_connections 2024-07-04T17:33:09.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.066+0000 7fc7e4600640 1 -- 172.21.15.42:0/1810187306 wait complete. 2024-07-04T17:33:09.119 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:33:09.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.374+0000 7f542de00640 1 -- 172.21.15.42:0/2503911693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5428073f80 msgr2=0x7f54280b2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:09.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.374+0000 7f542de00640 1 --2- 172.21.15.42:0/2503911693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5428073f80 0x7f54280b2be0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f5428074560 tx=0x7f541802a5f0 comp rx=0 tx=0).stop 2024-07-04T17:33:09.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.374+0000 7f542de00640 1 -- 172.21.15.42:0/2503911693 shutdown_connections 2024-07-04T17:33:09.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.374+0000 7f542de00640 1 --2- 172.21.15.42:0/2503911693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5428073f80 0x7f54280b2be0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:09.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.374+0000 7f542de00640 1 -- 172.21.15.42:0/2503911693 >> 172.21.15.42:0/2503911693 conn(0x7f5428071960 msgr2=0x7f542806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:09.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.374+0000 7f542de00640 1 -- 172.21.15.42:0/2503911693 shutdown_connections 2024-07-04T17:33:09.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.374+0000 7f542de00640 1 -- 172.21.15.42:0/2503911693 wait complete. 2024-07-04T17:33:09.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.377+0000 7f542de00640 1 Processor -- start 2024-07-04T17:33:09.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.377+0000 7f542de00640 1 -- start start 2024-07-04T17:33:09.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.379+0000 7f542de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5428073f80 0x7f54280b97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:09.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.379+0000 7f542de00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f542814f360 con 0x7f5428073f80 2024-07-04T17:33:09.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.379+0000 7f542ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5428073f80 0x7f54280b97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:09.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.379+0000 7f542ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5428073f80 0x7f54280b97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33072/0 (socket says 172.21.15.42:33072) 2024-07-04T17:33:09.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.379+0000 7f542ca00640 1 -- 172.21.15.42:0/2058796044 learned_addr learned my addr 172.21.15.42:0/2058796044 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:09.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.379+0000 7f542ca00640 1 -- 172.21.15.42:0/2058796044 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54280bbec0 con 0x7f5428073f80 2024-07-04T17:33:09.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.380+0000 7f542ca00640 1 --2- 172.21.15.42:0/2058796044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5428073f80 0x7f54280b97c0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f541c00c380 tx=0x7f541c00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:09.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.380+0000 7f5424c00640 1 -- 172.21.15.42:0/2058796044 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f541c017070 con 0x7f5428073f80 2024-07-04T17:33:09.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.380+0000 7f5424c00640 1 -- 172.21.15.42:0/2058796044 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f541c0052f0 con 0x7f5428073f80 2024-07-04T17:33:09.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.380+0000 7f542de00640 1 -- 172.21.15.42:0/2058796044 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54281dabb0 con 0x7f5428073f80 2024-07-04T17:33:09.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.380+0000 7f5424c00640 1 -- 172.21.15.42:0/2058796044 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f541c0035f0 con 0x7f5428073f80 2024-07-04T17:33:09.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.380+0000 7f542de00640 1 -- 172.21.15.42:0/2058796044 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54280b9ee0 con 0x7f5428073f80 2024-07-04T17:33:09.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.381+0000 7f542de00640 1 -- 172.21.15.42:0/2058796044 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54280bbcf0 con 0x7f5428073f80 2024-07-04T17:33:09.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.382+0000 7f5424c00640 1 -- 172.21.15.42:0/2058796044 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7f541c01a070 con 0x7f5428073f80 2024-07-04T17:33:09.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.382+0000 7f5424c00640 1 --2- 172.21.15.42:0/2058796044 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f540c03d4f0 0x7f540c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:09.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.383+0000 7f5424c00640 1 -- 172.21.15.42:0/2058796044 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 3439+0+0 (secure 0 0 0) 0x7f541c050c40 con 0x7f5428073f80 2024-07-04T17:33:09.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.383+0000 7f5427e00640 1 --2- 172.21.15.42:0/2058796044 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f540c03d4f0 0x7f540c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:09.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.383+0000 7f5427e00640 1 --2- 172.21.15.42:0/2058796044 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f540c03d4f0 0x7f540c03fa00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5418033000 tx=0x7f5418004730 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:09.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.391+0000 7f5424c00640 1 -- 172.21.15.42:0/2058796044 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f541c01be00 con 0x7f5428073f80 2024-07-04T17:33:09.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.603+0000 7f542de00640 1 -- 172.21.15.42:0/2058796044 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f54281dad80 con 0x7f5428073f80 2024-07-04T17:33:09.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.605+0000 7f5424c00640 1 -- 172.21.15.42:0/2058796044 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+10625 (secure 0 0 0) 0x7f541c011910 con 0x7f5428073f80 2024-07-04T17:33:09.606 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:33:09.607 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":113,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:33:08.649642+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:33:06.769774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-07-04T17:33:07.613572+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:33:09.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.608+0000 7f542de00640 1 -- 172.21.15.42:0/2058796044 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f540c03d4f0 msgr2=0x7f540c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:09.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.608+0000 7f542de00640 1 --2- 172.21.15.42:0/2058796044 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f540c03d4f0 0x7f540c03fa00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5418033000 tx=0x7f5418004730 comp rx=0 tx=0).stop 2024-07-04T17:33:09.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.608+0000 7f542de00640 1 -- 172.21.15.42:0/2058796044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5428073f80 msgr2=0x7f54280b97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:09.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.608+0000 7f542de00640 1 --2- 172.21.15.42:0/2058796044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5428073f80 0x7f54280b97c0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f541c00c380 tx=0x7f541c00c840 comp rx=0 tx=0).stop 2024-07-04T17:33:09.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.608+0000 7f542de00640 1 -- 172.21.15.42:0/2058796044 shutdown_connections 2024-07-04T17:33:09.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.608+0000 7f542de00640 1 --2- 172.21.15.42:0/2058796044 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f540c03d4f0 0x7f540c03fa00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:09.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.608+0000 7f542de00640 1 --2- 172.21.15.42:0/2058796044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5428073f80 0x7f54280b97c0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:09.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.608+0000 7f542de00640 1 -- 172.21.15.42:0/2058796044 >> 172.21.15.42:0/2058796044 conn(0x7f5428071960 msgr2=0x7f542814d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:09.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.608+0000 7f542de00640 1 -- 172.21.15.42:0/2058796044 shutdown_connections 2024-07-04T17:33:09.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.608+0000 7f542de00640 1 -- 172.21.15.42:0/2058796044 wait complete. 2024-07-04T17:33:09.634 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:33:09.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.914+0000 7fbcbfe00640 1 -- 172.21.15.42:0/1739190276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbcc00b20c0 msgr2=0x7fbcc00b27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:09.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.914+0000 7fbcbfe00640 1 --2- 172.21.15.42:0/1739190276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbcc00b20c0 0x7fbcc00b27c0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0005fd0 tx=0x7fbcb002b9c0 comp rx=0 tx=0).stop 2024-07-04T17:33:09.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.914+0000 7fbcbfe00640 1 -- 172.21.15.42:0/1739190276 shutdown_connections 2024-07-04T17:33:09.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.914+0000 7fbcbfe00640 1 --2- 172.21.15.42:0/1739190276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbcc00b20c0 0x7fbcc00b27c0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:09.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.914+0000 7fbcbfe00640 1 -- 172.21.15.42:0/1739190276 >> 172.21.15.42:0/1739190276 conn(0x7fbcc00bebc0 msgr2=0x7fbcc00befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:09.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.914+0000 7fbcbfe00640 1 -- 172.21.15.42:0/1739190276 shutdown_connections 2024-07-04T17:33:09.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.914+0000 7fbcbfe00640 1 -- 172.21.15.42:0/1739190276 wait complete. 2024-07-04T17:33:09.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.916+0000 7fbcbfe00640 1 Processor -- start 2024-07-04T17:33:09.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.917+0000 7fbcbfe00640 1 -- start start 2024-07-04T17:33:09.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.918+0000 7fbcbfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbcc00b20c0 0x7fbcc01381a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:09.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.918+0000 7fbcbfe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc00b5480 con 0x7fbcc00b20c0 2024-07-04T17:33:09.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.918+0000 7fbcbea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbcc00b20c0 0x7fbcc01381a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:09.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.918+0000 7fbcbea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbcc00b20c0 0x7fbcc01381a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33088/0 (socket says 172.21.15.42:33088) 2024-07-04T17:33:09.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.918+0000 7fbcbea00640 1 -- 172.21.15.42:0/1167780608 learned_addr learned my addr 172.21.15.42:0/1167780608 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:09.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.919+0000 7fbcbea00640 1 -- 172.21.15.42:0/1167780608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc01397f0 con 0x7fbcc00b20c0 2024-07-04T17:33:09.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.919+0000 7fbcbea00640 1 --2- 172.21.15.42:0/1167780608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbcc00b20c0 0x7fbcc01381a0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000a960 tx=0x7fbcb00049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:09.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.919+0000 7fbcb6a00640 1 -- 172.21.15.42:0/1167780608 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcb0034070 con 0x7fbcc00b20c0 2024-07-04T17:33:09.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.919+0000 7fbcb6a00640 1 -- 172.21.15.42:0/1167780608 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbcb0004ed0 con 0x7fbcc00b20c0 2024-07-04T17:33:09.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.919+0000 7fbcb6a00640 1 -- 172.21.15.42:0/1167780608 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcb0003760 con 0x7fbcc00b20c0 2024-07-04T17:33:09.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.919+0000 7fbcbfe00640 1 -- 172.21.15.42:0/1167780608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc01e7f00 con 0x7fbcc00b20c0 2024-07-04T17:33:09.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.919+0000 7fbcbfe00640 1 -- 172.21.15.42:0/1167780608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc0139e70 con 0x7fbcc00b20c0 2024-07-04T17:33:09.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.920+0000 7fbcbfe00640 1 -- 172.21.15.42:0/1167780608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcc00bf460 con 0x7fbcc00b20c0 2024-07-04T17:33:09.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.921+0000 7fbcb6a00640 1 -- 172.21.15.42:0/1167780608 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7fbcb0003900 con 0x7fbcc00b20c0 2024-07-04T17:33:09.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.922+0000 7fbcb6a00640 1 --2- 172.21.15.42:0/1167780608 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fbc9c03d4f0 0x7fbc9c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:09.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.922+0000 7fbcb6a00640 1 -- 172.21.15.42:0/1167780608 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 3439+0+0 (secure 0 0 0) 0x7fbcb0072c90 con 0x7fbcc00b20c0 2024-07-04T17:33:09.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.922+0000 7fbcbe000640 1 --2- 172.21.15.42:0/1167780608 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fbc9c03d4f0 0x7fbc9c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:09.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.923+0000 7fbcbe000640 1 --2- 172.21.15.42:0/1167780608 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fbc9c03d4f0 0x7fbc9c03fa00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbca8001c60 tx=0x7fbca8007d70 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:09.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:09.927+0000 7fbcb6a00640 1 -- 172.21.15.42:0/1167780608 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fbcc0139e70 con 0x7fbcc00b20c0 2024-07-04T17:33:10.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.148+0000 7fbcbfe00640 1 -- 172.21.15.42:0/1167780608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbcc00b27c0 con 0x7fbcc00b20c0 2024-07-04T17:33:10.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.150+0000 7fbcb6a00640 1 -- 172.21.15.42:0/1167780608 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) ==== 74+0+10612 (secure 0 0 0) 0x7fbcb00403e0 con 0x7fbcc00b20c0 2024-07-04T17:33:10.152 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:33:10.152 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":114,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:33:10.076784+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:33:06.769774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-07-04T17:33:07.613572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:33:10.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.154+0000 7fbcbfe00640 1 -- 172.21.15.42:0/1167780608 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fbc9c03d4f0 msgr2=0x7fbc9c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:10.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.154+0000 7fbcbfe00640 1 --2- 172.21.15.42:0/1167780608 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fbc9c03d4f0 0x7fbc9c03fa00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbca8001c60 tx=0x7fbca8007d70 comp rx=0 tx=0).stop 2024-07-04T17:33:10.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.154+0000 7fbcbfe00640 1 -- 172.21.15.42:0/1167780608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbcc00b20c0 msgr2=0x7fbcc01381a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:10.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.154+0000 7fbcbfe00640 1 --2- 172.21.15.42:0/1167780608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbcc00b20c0 0x7fbcc01381a0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000a960 tx=0x7fbcb00049d0 comp rx=0 tx=0).stop 2024-07-04T17:33:10.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.154+0000 7fbcbfe00640 1 -- 172.21.15.42:0/1167780608 shutdown_connections 2024-07-04T17:33:10.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.154+0000 7fbcbfe00640 1 --2- 172.21.15.42:0/1167780608 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fbc9c03d4f0 0x7fbc9c03fa00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:10.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.154+0000 7fbcbfe00640 1 --2- 172.21.15.42:0/1167780608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbcc00b20c0 0x7fbcc01381a0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:10.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.154+0000 7fbcbfe00640 1 -- 172.21.15.42:0/1167780608 >> 172.21.15.42:0/1167780608 conn(0x7fbcc00bebc0 msgr2=0x7fbcc00b95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:10.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.154+0000 7fbcbfe00640 1 -- 172.21.15.42:0/1167780608 shutdown_connections 2024-07-04T17:33:10.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.154+0000 7fbcbfe00640 1 -- 172.21.15.42:0/1167780608 wait complete. 2024-07-04T17:33:10.184 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-07-04T17:33:10.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.461+0000 7feeea600640 1 -- 172.21.15.42:0/381466027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feee40b25d0 msgr2=0x7feee4074310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:10.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.461+0000 7feeea600640 1 --2- 172.21.15.42:0/381466027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feee40b25d0 0x7feee4074310 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7feed8005fd0 tx=0x7feed802a5f0 comp rx=0 tx=0).stop 2024-07-04T17:33:10.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.462+0000 7feeea600640 1 -- 172.21.15.42:0/381466027 shutdown_connections 2024-07-04T17:33:10.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.462+0000 7feeea600640 1 --2- 172.21.15.42:0/381466027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feee40b25d0 0x7feee4074310 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:10.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.462+0000 7feeea600640 1 -- 172.21.15.42:0/381466027 >> 172.21.15.42:0/381466027 conn(0x7feee40c02b0 msgr2=0x7feee40b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:10.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.462+0000 7feeea600640 1 -- 172.21.15.42:0/381466027 shutdown_connections 2024-07-04T17:33:10.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.462+0000 7feeea600640 1 -- 172.21.15.42:0/381466027 wait complete. 2024-07-04T17:33:10.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.464+0000 7feeea600640 1 Processor -- start 2024-07-04T17:33:10.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.464+0000 7feeea600640 1 -- start start 2024-07-04T17:33:10.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.465+0000 7feeea600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feee40b12e0 0x7feee40bc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:10.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.465+0000 7feeea600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7feee40b3290 con 0x7feee40b12e0 2024-07-04T17:33:10.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.466+0000 7feee9200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feee40b12e0 0x7feee40bc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:10.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.466+0000 7feee9200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feee40b12e0 0x7feee40bc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33098/0 (socket says 172.21.15.42:33098) 2024-07-04T17:33:10.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.466+0000 7feee9200640 1 -- 172.21.15.42:0/1512177051 learned_addr learned my addr 172.21.15.42:0/1512177051 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:10.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.466+0000 7feee9200640 1 -- 172.21.15.42:0/1512177051 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee413cea0 con 0x7feee40b12e0 2024-07-04T17:33:10.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.466+0000 7feee9200640 1 --2- 172.21.15.42:0/1512177051 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feee40b12e0 0x7feee40bc920 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7feed8034040 tx=0x7feed80049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:10.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.467+0000 7feee0c00640 1 -- 172.21.15.42:0/1512177051 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feed803e030 con 0x7feee40b12e0 2024-07-04T17:33:10.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.467+0000 7feee0c00640 1 -- 172.21.15.42:0/1512177051 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feed80035f0 con 0x7feee40b12e0 2024-07-04T17:33:10.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.467+0000 7feee0c00640 1 -- 172.21.15.42:0/1512177051 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feed8035040 con 0x7feee40b12e0 2024-07-04T17:33:10.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.467+0000 7feeea600640 1 -- 172.21.15.42:0/1512177051 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feee40b7bd0 con 0x7feee40b12e0 2024-07-04T17:33:10.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.467+0000 7feeea600640 1 -- 172.21.15.42:0/1512177051 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feee40b7850 con 0x7feee40b12e0 2024-07-04T17:33:10.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.468+0000 7feeea600640 1 -- 172.21.15.42:0/1512177051 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feee40b3960 con 0x7feee40b12e0 2024-07-04T17:33:10.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.468+0000 7feee0c00640 1 -- 172.21.15.42:0/1512177051 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7feed8041030 con 0x7feee40b12e0 2024-07-04T17:33:10.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.469+0000 7feee0c00640 1 --2- 172.21.15.42:0/1512177051 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7feec803d4f0 0x7feec803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:10.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.469+0000 7feee0c00640 1 -- 172.21.15.42:0/1512177051 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 3439+0+0 (secure 0 0 0) 0x7feed803f070 con 0x7feee40b12e0 2024-07-04T17:33:10.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.469+0000 7feee3e00640 1 --2- 172.21.15.42:0/1512177051 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7feec803d4f0 0x7feec803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:10.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.470+0000 7feee3e00640 1 --2- 172.21.15.42:0/1512177051 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7feec803d4f0 0x7feec803fa00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7feee4145360 tx=0x7feed40079f0 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:10.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.474+0000 7feee0c00640 1 -- 172.21.15.42:0/1512177051 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7feed804bd60 con 0x7feee40b12e0 2024-07-04T17:33:10.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:10.716+0000 7feeea600640 1 -- 172.21.15.42:0/1512177051 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7feee4068e30 con 0x7feee40b12e0 2024-07-04T17:33:11.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.093+0000 7feee0c00640 1 -- 172.21.15.42:0/1512177051 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v89) ==== 118+0+0 (secure 0 0 0) 0x7feed8072110 con 0x7feee40b12e0 2024-07-04T17:33:11.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.096+0000 7feeea600640 1 -- 172.21.15.42:0/1512177051 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7feec803d4f0 msgr2=0x7feec803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:11.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.096+0000 7feeea600640 1 --2- 172.21.15.42:0/1512177051 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7feec803d4f0 0x7feec803fa00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7feee4145360 tx=0x7feed40079f0 comp rx=0 tx=0).stop 2024-07-04T17:33:11.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.096+0000 7feeea600640 1 -- 172.21.15.42:0/1512177051 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feee40b12e0 msgr2=0x7feee40bc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:11.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.096+0000 7feeea600640 1 --2- 172.21.15.42:0/1512177051 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feee40b12e0 0x7feee40bc920 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7feed8034040 tx=0x7feed80049d0 comp rx=0 tx=0).stop 2024-07-04T17:33:11.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.097+0000 7feeea600640 1 -- 172.21.15.42:0/1512177051 shutdown_connections 2024-07-04T17:33:11.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.097+0000 7feeea600640 1 --2- 172.21.15.42:0/1512177051 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7feec803d4f0 0x7feec803fa00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:11.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.097+0000 7feeea600640 1 --2- 172.21.15.42:0/1512177051 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feee40b12e0 0x7feee40bc920 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:11.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.097+0000 7feeea600640 1 -- 172.21.15.42:0/1512177051 >> 172.21.15.42:0/1512177051 conn(0x7feee40c02b0 msgr2=0x7feee406d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:11.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.097+0000 7feeea600640 1 -- 172.21.15.42:0/1512177051 shutdown_connections 2024-07-04T17:33:11.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.097+0000 7feeea600640 1 -- 172.21.15.42:0/1512177051 wait complete. 2024-07-04T17:33:11.142 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-04T17:33:11.142 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-07-04T17:33:11.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.420+0000 7fd7f1800640 1 -- 172.21.15.42:0/2276880185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7ec06d150 msgr2=0x7fd7ec073a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:11.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.420+0000 7fd7f1800640 1 --2- 172.21.15.42:0/2276880185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7ec06d150 0x7fd7ec073a20 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc005fd0 tx=0x7fd7dc02a5f0 comp rx=0 tx=0).stop 2024-07-04T17:33:11.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.420+0000 7fd7f1800640 1 -- 172.21.15.42:0/2276880185 shutdown_connections 2024-07-04T17:33:11.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.420+0000 7fd7f1800640 1 --2- 172.21.15.42:0/2276880185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7ec06d150 0x7fd7ec073a20 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:11.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.420+0000 7fd7f1800640 1 -- 172.21.15.42:0/2276880185 >> 172.21.15.42:0/2276880185 conn(0x7fd7ec06c020 msgr2=0x7fd7ec06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:11.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.421+0000 7fd7f1800640 1 -- 172.21.15.42:0/2276880185 shutdown_connections 2024-07-04T17:33:11.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.421+0000 7fd7f1800640 1 -- 172.21.15.42:0/2276880185 wait complete. 2024-07-04T17:33:11.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.423+0000 7fd7f1800640 1 Processor -- start 2024-07-04T17:33:11.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.424+0000 7fd7f1800640 1 -- start start 2024-07-04T17:33:11.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.425+0000 7fd7f1800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7ec06d150 0x7fd7ec0aed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:11.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.426+0000 7fd7f1800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ec071c40 con 0x7fd7ec06d150 2024-07-04T17:33:11.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.426+0000 7fd7eaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7ec06d150 0x7fd7ec0aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:11.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.426+0000 7fd7eaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7ec06d150 0x7fd7ec0aed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33126/0 (socket says 172.21.15.42:33126) 2024-07-04T17:33:11.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.426+0000 7fd7eaa00640 1 -- 172.21.15.42:0/4046622307 learned_addr learned my addr 172.21.15.42:0/4046622307 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:11.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.426+0000 7fd7eaa00640 1 -- 172.21.15.42:0/4046622307 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7ec1d5d70 con 0x7fd7ec06d150 2024-07-04T17:33:11.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.427+0000 7fd7eaa00640 1 --2- 172.21.15.42:0/4046622307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7ec06d150 0x7fd7ec0aed30 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc009850 tx=0x7fd7dc0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:11.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.427+0000 7fd7e2a00640 1 -- 172.21.15.42:0/4046622307 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7dc03e030 con 0x7fd7ec06d150 2024-07-04T17:33:11.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.427+0000 7fd7e2a00640 1 -- 172.21.15.42:0/4046622307 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7dc0035f0 con 0x7fd7ec06d150 2024-07-04T17:33:11.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.427+0000 7fd7e2a00640 1 -- 172.21.15.42:0/4046622307 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7dc035040 con 0x7fd7ec06d150 2024-07-04T17:33:11.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.427+0000 7fd7f1800640 1 -- 172.21.15.42:0/4046622307 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7ec1d4140 con 0x7fd7ec06d150 2024-07-04T17:33:11.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.427+0000 7fd7f1800640 1 -- 172.21.15.42:0/4046622307 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7ec13ca30 con 0x7fd7ec06d150 2024-07-04T17:33:11.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.429+0000 7fd7e2a00640 1 -- 172.21.15.42:0/4046622307 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7fd7dc041030 con 0x7fd7ec06d150 2024-07-04T17:33:11.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.429+0000 7fd7f1800640 1 -- 172.21.15.42:0/4046622307 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7ec1d4310 con 0x7fd7ec06d150 2024-07-04T17:33:11.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.430+0000 7fd7e2a00640 1 --2- 172.21.15.42:0/4046622307 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fd7cc03d4f0 0x7fd7cc03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:11.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.430+0000 7fd7e2a00640 1 -- 172.21.15.42:0/4046622307 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 3439+0+0 (secure 0 0 0) 0x7fd7dc03f070 con 0x7fd7ec06d150 2024-07-04T17:33:11.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.430+0000 7fd7ea000640 1 --2- 172.21.15.42:0/4046622307 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fd7cc03d4f0 0x7fd7cc03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:11.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.431+0000 7fd7ea000640 1 --2- 172.21.15.42:0/4046622307 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fd7cc03d4f0 0x7fd7cc03fa00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8001560 tx=0x7fd7d80079f0 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:11.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.436+0000 7fd7e2a00640 1 -- 172.21.15.42:0/4046622307 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fd7dc04bd60 con 0x7fd7ec06d150 2024-07-04T17:33:11.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:11.685+0000 7fd7f1800640 1 -- 172.21.15.42:0/4046622307 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd7ec073a20 con 0x7fd7ec06d150 2024-07-04T17:33:12.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.103+0000 7fd7e2a00640 1 -- 172.21.15.42:0/4046622307 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v90) ==== 122+0+0 (secure 0 0 0) 0x7fd7dc0720b0 con 0x7fd7ec06d150 2024-07-04T17:33:12.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.107+0000 7fd7f1800640 1 -- 172.21.15.42:0/4046622307 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fd7cc03d4f0 msgr2=0x7fd7cc03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:12.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.107+0000 7fd7f1800640 1 --2- 172.21.15.42:0/4046622307 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fd7cc03d4f0 0x7fd7cc03fa00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8001560 tx=0x7fd7d80079f0 comp rx=0 tx=0).stop 2024-07-04T17:33:12.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.107+0000 7fd7f1800640 1 -- 172.21.15.42:0/4046622307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7ec06d150 msgr2=0x7fd7ec0aed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:12.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.107+0000 7fd7f1800640 1 --2- 172.21.15.42:0/4046622307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7ec06d150 0x7fd7ec0aed30 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc009850 tx=0x7fd7dc0049d0 comp rx=0 tx=0).stop 2024-07-04T17:33:12.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.107+0000 7fd7f1800640 1 -- 172.21.15.42:0/4046622307 shutdown_connections 2024-07-04T17:33:12.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.107+0000 7fd7f1800640 1 --2- 172.21.15.42:0/4046622307 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fd7cc03d4f0 0x7fd7cc03fa00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:12.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.107+0000 7fd7f1800640 1 --2- 172.21.15.42:0/4046622307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7ec06d150 0x7fd7ec0aed30 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:12.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.107+0000 7fd7f1800640 1 -- 172.21.15.42:0/4046622307 >> 172.21.15.42:0/4046622307 conn(0x7fd7ec06c020 msgr2=0x7fd7ec140560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:12.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.107+0000 7fd7f1800640 1 -- 172.21.15.42:0/4046622307 shutdown_connections 2024-07-04T17:33:12.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.108+0000 7fd7f1800640 1 -- 172.21.15.42:0/4046622307 wait complete. 2024-07-04T17:33:12.139 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-07-04T17:33:12.139 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:33:12.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.435+0000 7f9d98a00640 1 -- 172.21.15.42:0/4114232485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d940b1d20 msgr2=0x7f9d940b6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:12.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.435+0000 7f9d98a00640 1 --2- 172.21.15.42:0/4114232485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d940b1d20 0x7f9d940b6230 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c005fd0 tx=0x7f9d7c02a5f0 comp rx=0 tx=0).stop 2024-07-04T17:33:12.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.436+0000 7f9d98a00640 1 -- 172.21.15.42:0/4114232485 shutdown_connections 2024-07-04T17:33:12.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.436+0000 7f9d98a00640 1 --2- 172.21.15.42:0/4114232485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d940b1d20 0x7f9d940b6230 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:12.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.436+0000 7f9d98a00640 1 -- 172.21.15.42:0/4114232485 >> 172.21.15.42:0/4114232485 conn(0x7f9d940bebc0 msgr2=0x7f9d940befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:12.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.436+0000 7f9d98a00640 1 -- 172.21.15.42:0/4114232485 shutdown_connections 2024-07-04T17:33:12.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.436+0000 7f9d98a00640 1 -- 172.21.15.42:0/4114232485 wait complete. 2024-07-04T17:33:12.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.438+0000 7f9d98a00640 1 Processor -- start 2024-07-04T17:33:12.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.438+0000 7f9d98a00640 1 -- start start 2024-07-04T17:33:12.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.439+0000 7f9d98a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d940b1d20 0x7f9d941428f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:12.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.439+0000 7f9d98a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9d940b8b40 con 0x7f9d940b1d20 2024-07-04T17:33:12.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.440+0000 7f9d93400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d940b1d20 0x7f9d941428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:12.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.440+0000 7f9d93400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d940b1d20 0x7f9d941428f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33142/0 (socket says 172.21.15.42:33142) 2024-07-04T17:33:12.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.440+0000 7f9d93400640 1 -- 172.21.15.42:0/840258654 learned_addr learned my addr 172.21.15.42:0/840258654 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:12.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.440+0000 7f9d93400640 1 -- 172.21.15.42:0/840258654 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d941d3990 con 0x7f9d940b1d20 2024-07-04T17:33:12.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.440+0000 7f9d93400640 1 --2- 172.21.15.42:0/840258654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d940b1d20 0x7f9d941428f0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400c380 tx=0x7f9d8400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:12.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.441+0000 7f9d8be00640 1 -- 172.21.15.42:0/840258654 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d84017070 con 0x7f9d940b1d20 2024-07-04T17:33:12.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.441+0000 7f9d8be00640 1 -- 172.21.15.42:0/840258654 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d840052f0 con 0x7f9d940b1d20 2024-07-04T17:33:12.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.441+0000 7f9d8be00640 1 -- 172.21.15.42:0/840258654 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d840035f0 con 0x7f9d940b1d20 2024-07-04T17:33:12.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.441+0000 7f9d98a00640 1 -- 172.21.15.42:0/840258654 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d941d5090 con 0x7f9d940b1d20 2024-07-04T17:33:12.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.441+0000 7f9d98a00640 1 -- 172.21.15.42:0/840258654 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d941d8c10 con 0x7f9d940b1d20 2024-07-04T17:33:12.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.442+0000 7f9d98a00640 1 -- 172.21.15.42:0/840258654 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d940b91d0 con 0x7f9d940b1d20 2024-07-04T17:33:12.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.442+0000 7f9d8be00640 1 -- 172.21.15.42:0/840258654 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7f9d8401a070 con 0x7f9d940b1d20 2024-07-04T17:33:12.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.443+0000 7f9d8be00640 1 --2- 172.21.15.42:0/840258654 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f9d7003d4f0 0x7f9d7003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:12.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.443+0000 7f9d8be00640 1 -- 172.21.15.42:0/840258654 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 3439+0+0 (secure 0 0 0) 0x7f9d84050c40 con 0x7f9d940b1d20 2024-07-04T17:33:12.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.443+0000 7f9d8aa00640 1 --2- 172.21.15.42:0/840258654 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f9d7003d4f0 0x7f9d7003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:12.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.444+0000 7f9d8aa00640 1 --2- 172.21.15.42:0/840258654 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f9d7003d4f0 0x7f9d7003fa00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c033000 tx=0x7f9d7c004730 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:12.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.448+0000 7f9d8be00640 1 -- 172.21.15.42:0/840258654 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f9d9413ec10 con 0x7f9d940b1d20 2024-07-04T17:33:12.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.694+0000 7f9d98a00640 1 -- 172.21.15.42:0/840258654 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d94068e30 con 0x7f9d940b1d20 2024-07-04T17:33:12.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.695+0000 7f9d8be00640 1 -- 172.21.15.42:0/840258654 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+6034 (secure 0 0 0) 0x7f9d84016170 con 0x7f9d940b1d20 2024-07-04T17:33:12.696 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:33:12.696 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":90,"btime":"2024-07-04T17:33:12:103002+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5536,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/68857121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":68857121},{"type":"v1","addr":"172.21.15.42:6825","nonce":68857121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":5544,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6828/20990456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":20990456},{"type":"v1","addr":"172.21.15.42:6828","nonce":20990456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":5545,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/428501060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6827","nonce":428501060},{"type":"v1","addr":"172.21.15.42:6829","nonce":428501060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":5549,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/3224527035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":3224527035},{"type":"v1","addr":"172.21.15.42:6831","nonce":3224527035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:33:08.649551+0000","modified":"2024-07-04T17:33:11.687748+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5550":{"gid":5550,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6833/605786437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":605786437},{"type":"v1","addr":"172.21.15.42:6833","nonce":605786437}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5550,"qdb_cluster":[5550]},"id":10}]} 2024-07-04T17:33:12.696 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 90 2024-07-04T17:33:12.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.697+0000 7f9d98a00640 1 -- 172.21.15.42:0/840258654 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f9d7003d4f0 msgr2=0x7f9d7003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:12.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.697+0000 7f9d98a00640 1 --2- 172.21.15.42:0/840258654 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f9d7003d4f0 0x7f9d7003fa00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c033000 tx=0x7f9d7c004730 comp rx=0 tx=0).stop 2024-07-04T17:33:12.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.697+0000 7f9d98a00640 1 -- 172.21.15.42:0/840258654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d940b1d20 msgr2=0x7f9d941428f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:12.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.697+0000 7f9d98a00640 1 --2- 172.21.15.42:0/840258654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d940b1d20 0x7f9d941428f0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400c380 tx=0x7f9d8400c840 comp rx=0 tx=0).stop 2024-07-04T17:33:12.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.697+0000 7f9d98a00640 1 -- 172.21.15.42:0/840258654 shutdown_connections 2024-07-04T17:33:12.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.697+0000 7f9d98a00640 1 --2- 172.21.15.42:0/840258654 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f9d7003d4f0 0x7f9d7003fa00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:12.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.697+0000 7f9d98a00640 1 --2- 172.21.15.42:0/840258654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d940b1d20 0x7f9d941428f0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:12.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.697+0000 7f9d98a00640 1 -- 172.21.15.42:0/840258654 >> 172.21.15.42:0/840258654 conn(0x7f9d940bebc0 msgr2=0x7f9d940b6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:12.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.697+0000 7f9d98a00640 1 -- 172.21.15.42:0/840258654 shutdown_connections 2024-07-04T17:33:12.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.697+0000 7f9d98a00640 1 -- 172.21.15.42:0/840258654 wait complete. 2024-07-04T17:33:12.723 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:33:12.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.996+0000 7f3a1d600640 1 -- 172.21.15.42:0/2767520598 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a180bad70 msgr2=0x7f3a1806c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:12.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.996+0000 7f3a1d600640 1 --2- 172.21.15.42:0/2767520598 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a180bad70 0x7f3a1806c2c0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f3a000097b0 tx=0x7f3a0002a4b0 comp rx=0 tx=0).stop 2024-07-04T17:33:12.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.997+0000 7f3a1d600640 1 -- 172.21.15.42:0/2767520598 shutdown_connections 2024-07-04T17:33:12.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.997+0000 7f3a1d600640 1 --2- 172.21.15.42:0/2767520598 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a180bad70 0x7f3a1806c2c0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:12.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.997+0000 7f3a1d600640 1 -- 172.21.15.42:0/2767520598 >> 172.21.15.42:0/2767520598 conn(0x7f3a180bebc0 msgr2=0x7f3a180befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:12.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.997+0000 7f3a1d600640 1 -- 172.21.15.42:0/2767520598 shutdown_connections 2024-07-04T17:33:12.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.997+0000 7f3a1d600640 1 -- 172.21.15.42:0/2767520598 wait complete. 2024-07-04T17:33:13.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.999+0000 7f3a1d600640 1 Processor -- start 2024-07-04T17:33:13.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:12.999+0000 7f3a1d600640 1 -- start start 2024-07-04T17:33:13.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.000+0000 7f3a1d600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a1806d580 0x7f3a181d4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:13.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.000+0000 7f3a1d600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3a180b2810 con 0x7f3a1806d580 2024-07-04T17:33:13.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.000+0000 7f3a17e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a1806d580 0x7f3a181d4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:13.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.000+0000 7f3a17e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a1806d580 0x7f3a181d4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33168/0 (socket says 172.21.15.42:33168) 2024-07-04T17:33:13.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.001+0000 7f3a17e00640 1 -- 172.21.15.42:0/3959839109 learned_addr learned my addr 172.21.15.42:0/3959839109 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:13.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.001+0000 7f3a17e00640 1 -- 172.21.15.42:0/3959839109 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a18070ee0 con 0x7f3a1806d580 2024-07-04T17:33:13.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.001+0000 7f3a17e00640 1 --2- 172.21.15.42:0/3959839109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a1806d580 0x7f3a181d4ca0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f3a0800c320 tx=0x7f3a0800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:13.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.001+0000 7f3a14c00640 1 -- 172.21.15.42:0/3959839109 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a080116c0 con 0x7f3a1806d580 2024-07-04T17:33:13.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.001+0000 7f3a14c00640 1 -- 172.21.15.42:0/3959839109 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a08017070 con 0x7f3a1806d580 2024-07-04T17:33:13.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.001+0000 7f3a14c00640 1 -- 172.21.15.42:0/3959839109 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a08011b40 con 0x7f3a1806d580 2024-07-04T17:33:13.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.002+0000 7f3a1d600640 1 -- 172.21.15.42:0/3959839109 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a181d5270 con 0x7f3a1806d580 2024-07-04T17:33:13.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.002+0000 7f3a1d600640 1 -- 172.21.15.42:0/3959839109 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a18070c60 con 0x7f3a1806d580 2024-07-04T17:33:13.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.003+0000 7f3a1d600640 1 -- 172.21.15.42:0/3959839109 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a1806c2c0 con 0x7f3a1806d580 2024-07-04T17:33:13.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.003+0000 7f3a14c00640 1 -- 172.21.15.42:0/3959839109 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7f3a08011ce0 con 0x7f3a1806d580 2024-07-04T17:33:13.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.004+0000 7f3a14c00640 1 --2- 172.21.15.42:0/3959839109 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f39f403d4f0 0x7f39f403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:13.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.004+0000 7f3a14c00640 1 -- 172.21.15.42:0/3959839109 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 3439+0+0 (secure 0 0 0) 0x7f3a08051650 con 0x7f3a1806d580 2024-07-04T17:33:13.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.004+0000 7f3a0fe00640 1 --2- 172.21.15.42:0/3959839109 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f39f403d4f0 0x7f39f403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:13.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.005+0000 7f3a0fe00640 1 --2- 172.21.15.42:0/3959839109 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f39f403d4f0 0x7f39f403fa00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3a00034000 tx=0x7f3a00004730 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:13.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.009+0000 7f3a14c00640 1 -- 172.21.15.42:0/3959839109 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f3a18070c60 con 0x7f3a1806d580 2024-07-04T17:33:13.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.211+0000 7f3a1d600640 1 -- 172.21.15.42:0/3959839109 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3a18071100 con 0x7f3a1806d580 2024-07-04T17:33:13.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.213+0000 7f3a14c00640 1 -- 172.21.15.42:0/3959839109 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) ==== 74+0+10612 (secure 0 0 0) 0x7f3a0800a140 con 0x7f3a1806d580 2024-07-04T17:33:13.214 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:33:13.214 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":114,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:33:10.076784+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:33:06.769774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-07-04T17:33:07.613572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:33:13.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.215+0000 7f3a1d600640 1 -- 172.21.15.42:0/3959839109 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f39f403d4f0 msgr2=0x7f39f403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:13.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.215+0000 7f3a1d600640 1 --2- 172.21.15.42:0/3959839109 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f39f403d4f0 0x7f39f403fa00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3a00034000 tx=0x7f3a00004730 comp rx=0 tx=0).stop 2024-07-04T17:33:13.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.215+0000 7f3a1d600640 1 -- 172.21.15.42:0/3959839109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a1806d580 msgr2=0x7f3a181d4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:13.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.215+0000 7f3a1d600640 1 --2- 172.21.15.42:0/3959839109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a1806d580 0x7f3a181d4ca0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f3a0800c320 tx=0x7f3a0800c7e0 comp rx=0 tx=0).stop 2024-07-04T17:33:13.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.215+0000 7f3a1d600640 1 -- 172.21.15.42:0/3959839109 shutdown_connections 2024-07-04T17:33:13.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.215+0000 7f3a1d600640 1 --2- 172.21.15.42:0/3959839109 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f39f403d4f0 0x7f39f403fa00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:13.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.215+0000 7f3a1d600640 1 --2- 172.21.15.42:0/3959839109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a1806d580 0x7f3a181d4ca0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:13.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.215+0000 7f3a1d600640 1 -- 172.21.15.42:0/3959839109 >> 172.21.15.42:0/3959839109 conn(0x7f3a180bebc0 msgr2=0x7f3a180bb480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:13.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.215+0000 7f3a1d600640 1 -- 172.21.15.42:0/3959839109 shutdown_connections 2024-07-04T17:33:13.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.215+0000 7f3a1d600640 1 -- 172.21.15.42:0/3959839109 wait complete. 2024-07-04T17:33:13.241 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-04T17:33:13.241 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:33:13.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.534+0000 7f4b42c00640 1 -- 172.21.15.42:0/3315647559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b3c0abdd0 msgr2=0x7f4b3c0b0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:13.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.534+0000 7f4b42c00640 1 --2- 172.21.15.42:0/3315647559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b3c0abdd0 0x7f4b3c0b0cd0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c005fd0 tx=0x7f4b2c02b930 comp rx=0 tx=0).stop 2024-07-04T17:33:13.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.535+0000 7f4b42c00640 1 -- 172.21.15.42:0/3315647559 shutdown_connections 2024-07-04T17:33:13.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.535+0000 7f4b42c00640 1 --2- 172.21.15.42:0/3315647559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b3c0abdd0 0x7f4b3c0b0cd0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:13.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.535+0000 7f4b42c00640 1 -- 172.21.15.42:0/3315647559 >> 172.21.15.42:0/3315647559 conn(0x7f4b3c0bf700 msgr2=0x7f4b3c0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:13.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.535+0000 7f4b42c00640 1 -- 172.21.15.42:0/3315647559 shutdown_connections 2024-07-04T17:33:13.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.535+0000 7f4b42c00640 1 -- 172.21.15.42:0/3315647559 wait complete. 2024-07-04T17:33:13.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.537+0000 7f4b42c00640 1 Processor -- start 2024-07-04T17:33:13.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.538+0000 7f4b42c00640 1 -- start start 2024-07-04T17:33:13.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.539+0000 7f4b42c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b3c0a9d20 0x7f4b3c0ae920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:13.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.539+0000 7f4b42c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c0b03a0 con 0x7f4b3c0a9d20 2024-07-04T17:33:13.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.540+0000 7f4b3be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b3c0a9d20 0x7f4b3c0ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:13.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.540+0000 7f4b3be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b3c0a9d20 0x7f4b3c0ae920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33192/0 (socket says 172.21.15.42:33192) 2024-07-04T17:33:13.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.540+0000 7f4b3be00640 1 -- 172.21.15.42:0/4249213989 learned_addr learned my addr 172.21.15.42:0/4249213989 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:13.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.540+0000 7f4b3be00640 1 -- 172.21.15.42:0/4249213989 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b3c140d10 con 0x7f4b3c0a9d20 2024-07-04T17:33:13.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.540+0000 7f4b3be00640 1 --2- 172.21.15.42:0/4249213989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b3c0a9d20 0x7f4b3c0ae920 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c006010 tx=0x7f4b2c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:13.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.541+0000 7f4b33e00640 1 -- 172.21.15.42:0/4249213989 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b2c004c20 con 0x7f4b3c0a9d20 2024-07-04T17:33:13.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.541+0000 7f4b33e00640 1 -- 172.21.15.42:0/4249213989 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b2c004eb0 con 0x7f4b3c0a9d20 2024-07-04T17:33:13.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.541+0000 7f4b42c00640 1 -- 172.21.15.42:0/4249213989 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b3c1e4580 con 0x7f4b3c0a9d20 2024-07-04T17:33:13.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.541+0000 7f4b33e00640 1 -- 172.21.15.42:0/4249213989 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b2c0317c0 con 0x7f4b3c0a9d20 2024-07-04T17:33:13.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.541+0000 7f4b42c00640 1 -- 172.21.15.42:0/4249213989 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b3c140240 con 0x7f4b3c0a9d20 2024-07-04T17:33:13.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.542+0000 7f4b33e00640 1 -- 172.21.15.42:0/4249213989 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7f4b2c042020 con 0x7f4b3c0a9d20 2024-07-04T17:33:13.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.542+0000 7f4b42c00640 1 -- 172.21.15.42:0/4249213989 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b3c0b0cd0 con 0x7f4b3c0a9d20 2024-07-04T17:33:13.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.543+0000 7f4b33e00640 1 --2- 172.21.15.42:0/4249213989 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f4b1c03d4f0 0x7f4b1c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:13.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.543+0000 7f4b33e00640 1 -- 172.21.15.42:0/4249213989 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 3439+0+0 (secure 0 0 0) 0x7f4b2c036070 con 0x7f4b3c0a9d20 2024-07-04T17:33:13.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.543+0000 7f4b3b400640 1 --2- 172.21.15.42:0/4249213989 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f4b1c03d4f0 0x7f4b1c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:13.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.544+0000 7f4b3b400640 1 --2- 172.21.15.42:0/4249213989 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f4b1c03d4f0 0x7f4b1c03fa00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c143c70 tx=0x7f4b280079f0 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:13.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.548+0000 7f4b33e00640 1 -- 172.21.15.42:0/4249213989 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f4b2c031960 con 0x7f4b3c0a9d20 2024-07-04T17:33:13.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.760+0000 7f4b42c00640 1 -- 172.21.15.42:0/4249213989 --> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4b3c140070 con 0x7f4b1c03d4f0 2024-07-04T17:33:13.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.776+0000 7f4b33e00640 1 -- 172.21.15.42:0/4249213989 <== mgr.5434 v2:172.21.15.42:6834/4130303917 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246494 (secure 0 0 0) 0x7f4b3c140070 con 0x7f4b1c03d4f0 2024-07-04T17:33:13.781 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:33:13.783 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:33:13.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.781+0000 7f4b42c00640 1 -- 172.21.15.42:0/4249213989 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f4b1c03d4f0 msgr2=0x7f4b1c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:13.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.781+0000 7f4b42c00640 1 --2- 172.21.15.42:0/4249213989 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f4b1c03d4f0 0x7f4b1c03fa00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c143c70 tx=0x7f4b280079f0 comp rx=0 tx=0).stop 2024-07-04T17:33:13.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.781+0000 7f4b42c00640 1 -- 172.21.15.42:0/4249213989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b3c0a9d20 msgr2=0x7f4b3c0ae920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:13.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.781+0000 7f4b42c00640 1 --2- 172.21.15.42:0/4249213989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b3c0a9d20 0x7f4b3c0ae920 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c006010 tx=0x7f4b2c0045e0 comp rx=0 tx=0).stop 2024-07-04T17:33:13.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.781+0000 7f4b42c00640 1 -- 172.21.15.42:0/4249213989 shutdown_connections 2024-07-04T17:33:13.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.781+0000 7f4b42c00640 1 --2- 172.21.15.42:0/4249213989 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f4b1c03d4f0 0x7f4b1c03fa00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:13.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.781+0000 7f4b42c00640 1 --2- 172.21.15.42:0/4249213989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b3c0a9d20 0x7f4b3c0ae920 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:13.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.781+0000 7f4b42c00640 1 -- 172.21.15.42:0/4249213989 >> 172.21.15.42:0/4249213989 conn(0x7f4b3c0bf700 msgr2=0x7f4b3c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:13.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.781+0000 7f4b42c00640 1 -- 172.21.15.42:0/4249213989 shutdown_connections 2024-07-04T17:33:13.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:13.781+0000 7f4b42c00640 1 -- 172.21.15.42:0/4249213989 wait complete. 2024-07-04T17:33:13.809 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-07-04T17:33:13.274921+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":476,"num_read_kb":1278,"num_write":208,"num_write_kb":1120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":110664,"kb_used_data":4632,"kb_used_omap":4,"kb_used_meta":105979,"kb_avail":283004856,"statfs":{"total":289910292480,"available":289796972544,"internally_reserved":0,"allocated":4743168,"data_stored":2571250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":108522831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001662"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.631240+0000","last_change":"2024-07-04T17:33:09.110137+0000","last_active":"2024-07-04T17:33:10.631240+0000","last_peered":"2024-07-04T17:33:10.631240+0000","last_clean":"2024-07-04T17:33:10.631240+0000","last_became_active":"2024-07-04T17:33:09.109336+0000","last_became_peered":"2024-07-04T17:33:09.109336+0000","last_unstale":"2024-07-04T17:33:10.631240+0000","last_undegraded":"2024-07-04T17:33:10.631240+0000","last_fullsized":"2024-07-04T17:33:10.631240+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:09.113117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.127044+0000","last_change":"2024-07-04T17:33:09.113135+0000","last_active":"2024-07-04T17:33:10.127044+0000","last_peered":"2024-07-04T17:33:10.127044+0000","last_clean":"2024-07-04T17:33:10.127044+0000","last_became_active":"2024-07-04T17:33:09.112824+0000","last_became_peered":"2024-07-04T17:33:09.112824+0000","last_unstale":"2024-07-04T17:33:10.127044+0000","last_undegraded":"2024-07-04T17:33:10.127044+0000","last_fullsized":"2024-07-04T17:33:10.127044+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:25:36.254957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.630037+0000","last_change":"2024-07-04T17:33:09.126404+0000","last_active":"2024-07-04T17:33:10.630037+0000","last_peered":"2024-07-04T17:33:10.630037+0000","last_clean":"2024-07-04T17:33:10.630037+0000","last_became_active":"2024-07-04T17:33:09.126069+0000","last_became_peered":"2024-07-04T17:33:09.126069+0000","last_unstale":"2024-07-04T17:33:10.630037+0000","last_undegraded":"2024-07-04T17:33:10.630037+0000","last_fullsized":"2024-07-04T17:33:10.630037+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:54:17.261721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.089766+0000","last_change":"2024-07-04T17:33:09.104422+0000","last_active":"2024-07-04T17:33:10.089766+0000","last_peered":"2024-07-04T17:33:10.089766+0000","last_clean":"2024-07-04T17:33:10.089766+0000","last_became_active":"2024-07-04T17:33:09.104112+0000","last_became_peered":"2024-07-04T17:33:09.104112+0000","last_unstale":"2024-07-04T17:33:10.089766+0000","last_undegraded":"2024-07-04T17:33:10.089766+0000","last_fullsized":"2024-07-04T17:33:10.089766+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:50:45.806246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.629366+0000","last_change":"2024-07-04T17:33:09.111717+0000","last_active":"2024-07-04T17:33:10.629366+0000","last_peered":"2024-07-04T17:33:10.629366+0000","last_clean":"2024-07-04T17:33:10.629366+0000","last_became_active":"2024-07-04T17:33:09.110270+0000","last_became_peered":"2024-07-04T17:33:09.110270+0000","last_unstale":"2024-07-04T17:33:10.629366+0000","last_undegraded":"2024-07-04T17:33:10.629366+0000","last_fullsized":"2024-07-04T17:33:10.629366+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:18:53.013321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.628774+0000","last_change":"2024-07-04T17:33:09.108466+0000","last_active":"2024-07-04T17:33:10.628774+0000","last_peered":"2024-07-04T17:33:10.628774+0000","last_clean":"2024-07-04T17:33:10.628774+0000","last_became_active":"2024-07-04T17:33:09.108132+0000","last_became_peered":"2024-07-04T17:33:09.108132+0000","last_unstale":"2024-07-04T17:33:10.628774+0000","last_undegraded":"2024-07-04T17:33:10.628774+0000","last_fullsized":"2024-07-04T17:33:10.628774+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:51:32.675966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.126131+0000","last_change":"2024-07-04T17:33:09.108992+0000","last_active":"2024-07-04T17:33:10.126131+0000","last_peered":"2024-07-04T17:33:10.126131+0000","last_clean":"2024-07-04T17:33:10.126131+0000","last_became_active":"2024-07-04T17:33:09.107307+0000","last_became_peered":"2024-07-04T17:33:09.107307+0000","last_unstale":"2024-07-04T17:33:10.126131+0000","last_undegraded":"2024-07-04T17:33:10.126131+0000","last_fullsized":"2024-07-04T17:33:10.126131+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:32:07.600844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.628705+0000","last_change":"2024-07-04T17:33:09.127659+0000","last_active":"2024-07-04T17:33:10.628705+0000","last_peered":"2024-07-04T17:33:10.628705+0000","last_clean":"2024-07-04T17:33:10.628705+0000","last_became_active":"2024-07-04T17:33:09.126591+0000","last_became_peered":"2024-07-04T17:33:09.126591+0000","last_unstale":"2024-07-04T17:33:10.628705+0000","last_undegraded":"2024-07-04T17:33:10.628705+0000","last_fullsized":"2024-07-04T17:33:10.628705+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:44:07.778739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.631732+0000","last_change":"2024-07-04T17:33:09.110971+0000","last_active":"2024-07-04T17:33:10.631732+0000","last_peered":"2024-07-04T17:33:10.631732+0000","last_clean":"2024-07-04T17:33:10.631732+0000","last_became_active":"2024-07-04T17:33:09.109775+0000","last_became_peered":"2024-07-04T17:33:09.109775+0000","last_unstale":"2024-07-04T17:33:10.631732+0000","last_undegraded":"2024-07-04T17:33:10.631732+0000","last_fullsized":"2024-07-04T17:33:10.631732+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:27:36.573799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.630135+0000","last_change":"2024-07-04T17:33:09.127783+0000","last_active":"2024-07-04T17:33:10.630135+0000","last_peered":"2024-07-04T17:33:10.630135+0000","last_clean":"2024-07-04T17:33:10.630135+0000","last_became_active":"2024-07-04T17:33:09.125335+0000","last_became_peered":"2024-07-04T17:33:09.125335+0000","last_unstale":"2024-07-04T17:33:10.630135+0000","last_undegraded":"2024-07-04T17:33:10.630135+0000","last_fullsized":"2024-07-04T17:33:10.630135+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:29:15.785171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.630552+0000","last_change":"2024-07-04T17:33:09.113535+0000","last_active":"2024-07-04T17:33:10.630552+0000","last_peered":"2024-07-04T17:33:10.630552+0000","last_clean":"2024-07-04T17:33:10.630552+0000","last_became_active":"2024-07-04T17:33:09.113127+0000","last_became_peered":"2024-07-04T17:33:09.113127+0000","last_unstale":"2024-07-04T17:33:10.630552+0000","last_undegraded":"2024-07-04T17:33:10.630552+0000","last_fullsized":"2024-07-04T17:33:10.630552+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:14:08.620654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.127257+0000","last_change":"2024-07-04T17:33:09.112587+0000","last_active":"2024-07-04T17:33:10.127257+0000","last_peered":"2024-07-04T17:33:10.127257+0000","last_clean":"2024-07-04T17:33:10.127257+0000","last_became_active":"2024-07-04T17:33:09.112063+0000","last_became_peered":"2024-07-04T17:33:09.112063+0000","last_unstale":"2024-07-04T17:33:10.127257+0000","last_undegraded":"2024-07-04T17:33:10.127257+0000","last_fullsized":"2024-07-04T17:33:10.127257+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:59:39.105749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.629414+0000","last_change":"2024-07-04T17:33:09.110573+0000","last_active":"2024-07-04T17:33:10.629414+0000","last_peered":"2024-07-04T17:33:10.629414+0000","last_clean":"2024-07-04T17:33:10.629414+0000","last_became_active":"2024-07-04T17:33:09.109386+0000","last_became_peered":"2024-07-04T17:33:09.109386+0000","last_unstale":"2024-07-04T17:33:10.629414+0000","last_undegraded":"2024-07-04T17:33:10.629414+0000","last_fullsized":"2024-07-04T17:33:10.629414+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:33:02.261126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.126377+0000","last_change":"2024-07-04T17:33:09.119181+0000","last_active":"2024-07-04T17:33:10.126377+0000","last_peered":"2024-07-04T17:33:10.126377+0000","last_clean":"2024-07-04T17:33:10.126377+0000","last_became_active":"2024-07-04T17:33:09.118791+0000","last_became_peered":"2024-07-04T17:33:09.118791+0000","last_unstale":"2024-07-04T17:33:10.126377+0000","last_undegraded":"2024-07-04T17:33:10.126377+0000","last_fullsized":"2024-07-04T17:33:10.126377+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:08:30.473430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.628486+0000","last_change":"2024-07-04T17:33:09.111491+0000","last_active":"2024-07-04T17:33:10.628486+0000","last_peered":"2024-07-04T17:33:10.628486+0000","last_clean":"2024-07-04T17:33:10.628486+0000","last_became_active":"2024-07-04T17:33:09.111095+0000","last_became_peered":"2024-07-04T17:33:09.111095+0000","last_unstale":"2024-07-04T17:33:10.628486+0000","last_undegraded":"2024-07-04T17:33:10.628486+0000","last_fullsized":"2024-07-04T17:33:10.628486+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:49:13.666901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.088629+0000","last_change":"2024-07-04T17:33:09.101446+0000","last_active":"2024-07-04T17:33:10.088629+0000","last_peered":"2024-07-04T17:33:10.088629+0000","last_clean":"2024-07-04T17:33:10.088629+0000","last_became_active":"2024-07-04T17:33:09.100644+0000","last_became_peered":"2024-07-04T17:33:09.100644+0000","last_unstale":"2024-07-04T17:33:10.088629+0000","last_undegraded":"2024-07-04T17:33:10.088629+0000","last_fullsized":"2024-07-04T17:33:10.088629+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:27:36.573799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.091539+0000","last_change":"2024-07-04T17:33:09.124916+0000","last_active":"2024-07-04T17:33:10.091539+0000","last_peered":"2024-07-04T17:33:10.091539+0000","last_clean":"2024-07-04T17:33:10.091539+0000","last_became_active":"2024-07-04T17:33:09.124557+0000","last_became_peered":"2024-07-04T17:33:09.124557+0000","last_unstale":"2024-07-04T17:33:10.091539+0000","last_undegraded":"2024-07-04T17:33:10.091539+0000","last_fullsized":"2024-07-04T17:33:10.091539+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:10:26.737295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.630661+0000","last_change":"2024-07-04T17:33:09.115371+0000","last_active":"2024-07-04T17:33:10.630661+0000","last_peered":"2024-07-04T17:33:10.630661+0000","last_clean":"2024-07-04T17:33:10.630661+0000","last_became_active":"2024-07-04T17:33:09.114560+0000","last_became_peered":"2024-07-04T17:33:09.114560+0000","last_unstale":"2024-07-04T17:33:10.630661+0000","last_undegraded":"2024-07-04T17:33:10.630661+0000","last_fullsized":"2024-07-04T17:33:10.630661+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:39:29.413889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.631021+0000","last_change":"2024-07-04T17:33:09.113151+0000","last_active":"2024-07-04T17:33:10.631021+0000","last_peered":"2024-07-04T17:33:10.631021+0000","last_clean":"2024-07-04T17:33:10.631021+0000","last_became_active":"2024-07-04T17:33:09.112228+0000","last_became_peered":"2024-07-04T17:33:09.112228+0000","last_unstale":"2024-07-04T17:33:10.631021+0000","last_undegraded":"2024-07-04T17:33:10.631021+0000","last_fullsized":"2024-07-04T17:33:10.631021+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:16:18.941587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.089375+0000","last_change":"2024-07-04T17:33:09.118128+0000","last_active":"2024-07-04T17:33:10.089375+0000","last_peered":"2024-07-04T17:33:10.089375+0000","last_clean":"2024-07-04T17:33:10.089375+0000","last_became_active":"2024-07-04T17:33:09.117473+0000","last_became_peered":"2024-07-04T17:33:09.117473+0000","last_unstale":"2024-07-04T17:33:10.089375+0000","last_undegraded":"2024-07-04T17:33:10.089375+0000","last_fullsized":"2024-07-04T17:33:10.089375+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:06:51.425997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.089170+0000","last_change":"2024-07-04T17:33:09.127162+0000","last_active":"2024-07-04T17:33:10.089170+0000","last_peered":"2024-07-04T17:33:10.089170+0000","last_clean":"2024-07-04T17:33:10.089170+0000","last_became_active":"2024-07-04T17:33:09.123389+0000","last_became_peered":"2024-07-04T17:33:09.123389+0000","last_unstale":"2024-07-04T17:33:10.089170+0000","last_undegraded":"2024-07-04T17:33:10.089170+0000","last_fullsized":"2024-07-04T17:33:10.089170+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:33:17.955726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.126685+0000","last_change":"2024-07-04T17:33:09.118567+0000","last_active":"2024-07-04T17:33:10.126685+0000","last_peered":"2024-07-04T17:33:10.126685+0000","last_clean":"2024-07-04T17:33:10.126685+0000","last_became_active":"2024-07-04T17:33:09.117227+0000","last_became_peered":"2024-07-04T17:33:09.117227+0000","last_unstale":"2024-07-04T17:33:10.126685+0000","last_undegraded":"2024-07-04T17:33:10.126685+0000","last_fullsized":"2024-07-04T17:33:10.126685+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:42:43.320358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.089134+0000","last_change":"2024-07-04T17:33:09.131948+0000","last_active":"2024-07-04T17:33:10.089134+0000","last_peered":"2024-07-04T17:33:10.089134+0000","last_clean":"2024-07-04T17:33:10.089134+0000","last_became_active":"2024-07-04T17:33:09.121717+0000","last_became_peered":"2024-07-04T17:33:09.121717+0000","last_unstale":"2024-07-04T17:33:10.089134+0000","last_undegraded":"2024-07-04T17:33:10.089134+0000","last_fullsized":"2024-07-04T17:33:10.089134+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:05:18.766887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.628993+0000","last_change":"2024-07-04T17:33:09.124489+0000","last_active":"2024-07-04T17:33:10.628993+0000","last_peered":"2024-07-04T17:33:10.628993+0000","last_clean":"2024-07-04T17:33:10.628993+0000","last_became_active":"2024-07-04T17:33:09.122889+0000","last_became_peered":"2024-07-04T17:33:09.122889+0000","last_unstale":"2024-07-04T17:33:10.628993+0000","last_undegraded":"2024-07-04T17:33:10.628993+0000","last_fullsized":"2024-07-04T17:33:10.628993+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:01:35.821887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.091644+0000","last_change":"2024-07-04T17:33:09.118397+0000","last_active":"2024-07-04T17:33:10.091644+0000","last_peered":"2024-07-04T17:33:10.091644+0000","last_clean":"2024-07-04T17:33:10.091644+0000","last_became_active":"2024-07-04T17:33:09.117509+0000","last_became_peered":"2024-07-04T17:33:09.117509+0000","last_unstale":"2024-07-04T17:33:10.091644+0000","last_undegraded":"2024-07-04T17:33:10.091644+0000","last_fullsized":"2024-07-04T17:33:10.091644+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:37:23.202824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.630761+0000","last_change":"2024-07-04T17:33:09.115358+0000","last_active":"2024-07-04T17:33:10.630761+0000","last_peered":"2024-07-04T17:33:10.630761+0000","last_clean":"2024-07-04T17:33:10.630761+0000","last_became_active":"2024-07-04T17:33:09.113650+0000","last_became_peered":"2024-07-04T17:33:09.113650+0000","last_unstale":"2024-07-04T17:33:10.630761+0000","last_undegraded":"2024-07-04T17:33:10.630761+0000","last_fullsized":"2024-07-04T17:33:10.630761+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:34:20.388421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.090827+0000","last_change":"2024-07-04T17:33:09.130921+0000","last_active":"2024-07-04T17:33:10.090827+0000","last_peered":"2024-07-04T17:33:10.090827+0000","last_clean":"2024-07-04T17:33:10.090827+0000","last_became_active":"2024-07-04T17:33:09.123521+0000","last_became_peered":"2024-07-04T17:33:09.123521+0000","last_unstale":"2024-07-04T17:33:10.090827+0000","last_undegraded":"2024-07-04T17:33:10.090827+0000","last_fullsized":"2024-07-04T17:33:10.090827+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:13:51.565345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.127454+0000","last_change":"2024-07-04T17:33:09.114604+0000","last_active":"2024-07-04T17:33:10.127454+0000","last_peered":"2024-07-04T17:33:10.127454+0000","last_clean":"2024-07-04T17:33:10.127454+0000","last_became_active":"2024-07-04T17:33:09.114294+0000","last_became_peered":"2024-07-04T17:33:09.114294+0000","last_unstale":"2024-07-04T17:33:10.127454+0000","last_undegraded":"2024-07-04T17:33:10.127454+0000","last_fullsized":"2024-07-04T17:33:10.127454+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:34:50.754812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.629965+0000","last_change":"2024-07-04T17:33:09.124878+0000","last_active":"2024-07-04T17:33:10.629965+0000","last_peered":"2024-07-04T17:33:10.629965+0000","last_clean":"2024-07-04T17:33:10.629965+0000","last_became_active":"2024-07-04T17:33:09.124119+0000","last_became_peered":"2024-07-04T17:33:09.124119+0000","last_unstale":"2024-07-04T17:33:10.629965+0000","last_undegraded":"2024-07-04T17:33:10.629965+0000","last_fullsized":"2024-07-04T17:33:10.629965+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:18:22.081730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.126587+0000","last_change":"2024-07-04T17:33:09.104697+0000","last_active":"2024-07-04T17:33:10.126587+0000","last_peered":"2024-07-04T17:33:10.126587+0000","last_clean":"2024-07-04T17:33:10.126587+0000","last_became_active":"2024-07-04T17:33:09.103509+0000","last_became_peered":"2024-07-04T17:33:09.103509+0000","last_unstale":"2024-07-04T17:33:10.126587+0000","last_undegraded":"2024-07-04T17:33:10.126587+0000","last_fullsized":"2024-07-04T17:33:10.126587+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:05:17.882764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.126381+0000","last_change":"2024-07-04T17:33:09.104054+0000","last_active":"2024-07-04T17:33:10.126381+0000","last_peered":"2024-07-04T17:33:10.126381+0000","last_clean":"2024-07-04T17:33:10.126381+0000","last_became_active":"2024-07-04T17:33:09.103713+0000","last_became_peered":"2024-07-04T17:33:09.103713+0000","last_unstale":"2024-07-04T17:33:10.126381+0000","last_undegraded":"2024-07-04T17:33:10.126381+0000","last_fullsized":"2024-07-04T17:33:10.126381+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:27:24.257586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.088710+0000","last_change":"2024-07-04T17:33:09.122343+0000","last_active":"2024-07-04T17:33:10.088710+0000","last_peered":"2024-07-04T17:33:10.088710+0000","last_clean":"2024-07-04T17:33:10.088710+0000","last_became_active":"2024-07-04T17:33:09.121982+0000","last_became_peered":"2024-07-04T17:33:09.121982+0000","last_unstale":"2024-07-04T17:33:10.088710+0000","last_undegraded":"2024-07-04T17:33:10.088710+0000","last_fullsized":"2024-07-04T17:33:10.088710+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:55:41.686107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.631147+0000","last_change":"2024-07-04T17:33:09.115618+0000","last_active":"2024-07-04T17:33:10.631147+0000","last_peered":"2024-07-04T17:33:10.631147+0000","last_clean":"2024-07-04T17:33:10.631147+0000","last_became_active":"2024-07-04T17:33:09.115047+0000","last_became_peered":"2024-07-04T17:33:09.115047+0000","last_unstale":"2024-07-04T17:33:10.631147+0000","last_undegraded":"2024-07-04T17:33:10.631147+0000","last_fullsized":"2024-07-04T17:33:10.631147+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:38:58.383535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"113'4","reported_seq":18,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.632103+0000","last_change":"2024-07-04T17:33:08.117620+0000","last_active":"2024-07-04T17:33:10.632103+0000","last_peered":"2024-07-04T17:33:10.632103+0000","last_clean":"2024-07-04T17:33:10.632103+0000","last_became_active":"2024-07-04T17:33:08.117097+0000","last_became_peered":"2024-07-04T17:33:08.117097+0000","last_unstale":"2024-07-04T17:33:10.632103+0000","last_undegraded":"2024-07-04T17:33:10.632103+0000","last_fullsized":"2024-07-04T17:33:10.632103+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:00:27.380504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.092013+0000","last_change":"2024-07-04T17:33:09.133070+0000","last_active":"2024-07-04T17:33:10.092013+0000","last_peered":"2024-07-04T17:33:10.092013+0000","last_clean":"2024-07-04T17:33:10.092013+0000","last_became_active":"2024-07-04T17:33:09.132634+0000","last_became_peered":"2024-07-04T17:33:09.132634+0000","last_unstale":"2024-07-04T17:33:10.092013+0000","last_undegraded":"2024-07-04T17:33:10.092013+0000","last_fullsized":"2024-07-04T17:33:10.092013+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:42:13.182154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1e","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.127169+0000","last_change":"2024-07-04T17:33:08.105107+0000","last_active":"2024-07-04T17:33:10.127169+0000","last_peered":"2024-07-04T17:33:10.127169+0000","last_clean":"2024-07-04T17:33:10.127169+0000","last_became_active":"2024-07-04T17:33:08.104670+0000","last_became_peered":"2024-07-04T17:33:08.104670+0000","last_unstale":"2024-07-04T17:33:10.127169+0000","last_undegraded":"2024-07-04T17:33:10.127169+0000","last_fullsized":"2024-07-04T17:33:10.127169+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:46:04.479168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.630165+0000","last_change":"2024-07-04T17:33:09.107265+0000","last_active":"2024-07-04T17:33:10.630165+0000","last_peered":"2024-07-04T17:33:10.630165+0000","last_clean":"2024-07-04T17:33:10.630165+0000","last_became_active":"2024-07-04T17:33:09.106633+0000","last_became_peered":"2024-07-04T17:33:09.106633+0000","last_unstale":"2024-07-04T17:33:10.630165+0000","last_undegraded":"2024-07-04T17:33:10.630165+0000","last_fullsized":"2024-07-04T17:33:10.630165+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:05:30.327143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.091565+0000","last_change":"2024-07-04T17:33:08.089824+0000","last_active":"2024-07-04T17:33:10.091565+0000","last_peered":"2024-07-04T17:33:10.091565+0000","last_clean":"2024-07-04T17:33:10.091565+0000","last_became_active":"2024-07-04T17:33:08.084113+0000","last_became_peered":"2024-07-04T17:33:08.084113+0000","last_unstale":"2024-07-04T17:33:10.091565+0000","last_undegraded":"2024-07-04T17:33:10.091565+0000","last_fullsized":"2024-07-04T17:33:10.091565+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:39:25.915226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.091057+0000","last_change":"2024-07-04T17:33:09.129582+0000","last_active":"2024-07-04T17:33:10.091057+0000","last_peered":"2024-07-04T17:33:10.091057+0000","last_clean":"2024-07-04T17:33:10.091057+0000","last_became_active":"2024-07-04T17:33:09.127839+0000","last_became_peered":"2024-07-04T17:33:09.127839+0000","last_unstale":"2024-07-04T17:33:10.091057+0000","last_undegraded":"2024-07-04T17:33:10.091057+0000","last_fullsized":"2024-07-04T17:33:10.091057+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:35:13.414729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.090676+0000","last_change":"2024-07-04T17:33:08.108862+0000","last_active":"2024-07-04T17:33:10.090676+0000","last_peered":"2024-07-04T17:33:10.090676+0000","last_clean":"2024-07-04T17:33:10.090676+0000","last_became_active":"2024-07-04T17:33:08.107761+0000","last_became_peered":"2024-07-04T17:33:08.107761+0000","last_unstale":"2024-07-04T17:33:10.090676+0000","last_undegraded":"2024-07-04T17:33:10.090676+0000","last_fullsized":"2024-07-04T17:33:10.090676+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:15:53.250590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.126769+0000","last_change":"2024-07-04T17:33:09.118310+0000","last_active":"2024-07-04T17:33:10.126769+0000","last_peered":"2024-07-04T17:33:10.126769+0000","last_clean":"2024-07-04T17:33:10.126769+0000","last_became_active":"2024-07-04T17:33:09.116852+0000","last_became_peered":"2024-07-04T17:33:09.116852+0000","last_unstale":"2024-07-04T17:33:10.126769+0000","last_undegraded":"2024-07-04T17:33:10.126769+0000","last_fullsized":"2024-07-04T17:33:10.126769+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:41:21.265187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.1b","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.091201+0000","last_change":"2024-07-04T17:33:08.105642+0000","last_active":"2024-07-04T17:33:10.091201+0000","last_peered":"2024-07-04T17:33:10.091201+0000","last_clean":"2024-07-04T17:33:10.091201+0000","last_became_active":"2024-07-04T17:33:08.105297+0000","last_became_peered":"2024-07-04T17:33:08.105297+0000","last_unstale":"2024-07-04T17:33:10.091201+0000","last_undegraded":"2024-07-04T17:33:10.091201+0000","last_fullsized":"2024-07-04T17:33:10.091201+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:50:34.893649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.089384+0000","last_change":"2024-07-04T17:33:09.116075+0000","last_active":"2024-07-04T17:33:10.089384+0000","last_peered":"2024-07-04T17:33:10.089384+0000","last_clean":"2024-07-04T17:33:10.089384+0000","last_became_active":"2024-07-04T17:33:09.115705+0000","last_became_peered":"2024-07-04T17:33:09.115705+0000","last_unstale":"2024-07-04T17:33:10.089384+0000","last_undegraded":"2024-07-04T17:33:10.089384+0000","last_fullsized":"2024-07-04T17:33:10.089384+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:52:50.422683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.629511+0000","last_change":"2024-07-04T17:33:08.116547+0000","last_active":"2024-07-04T17:33:10.629511+0000","last_peered":"2024-07-04T17:33:10.629511+0000","last_clean":"2024-07-04T17:33:10.629511+0000","last_became_active":"2024-07-04T17:33:08.116095+0000","last_became_peered":"2024-07-04T17:33:08.116095+0000","last_unstale":"2024-07-04T17:33:10.629511+0000","last_undegraded":"2024-07-04T17:33:10.629511+0000","last_fullsized":"2024-07-04T17:33:10.629511+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:31:44.812381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.629249+0000","last_change":"2024-07-04T17:33:09.125116+0000","last_active":"2024-07-04T17:33:10.629249+0000","last_peered":"2024-07-04T17:33:10.629249+0000","last_clean":"2024-07-04T17:33:10.629249+0000","last_became_active":"2024-07-04T17:33:09.124103+0000","last_became_peered":"2024-07-04T17:33:09.124103+0000","last_unstale":"2024-07-04T17:33:10.629249+0000","last_undegraded":"2024-07-04T17:33:10.629249+0000","last_fullsized":"2024-07-04T17:33:10.629249+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:27:31.156139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.090675+0000","last_change":"2024-07-04T17:33:08.096904+0000","last_active":"2024-07-04T17:33:10.090675+0000","last_peered":"2024-07-04T17:33:10.090675+0000","last_clean":"2024-07-04T17:33:10.090675+0000","last_became_active":"2024-07-04T17:33:08.096590+0000","last_became_peered":"2024-07-04T17:33:08.096590+0000","last_unstale":"2024-07-04T17:33:10.090675+0000","last_undegraded":"2024-07-04T17:33:10.090675+0000","last_fullsized":"2024-07-04T17:33:10.090675+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:39:51.529018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.091219+0000","last_change":"2024-07-04T17:33:09.100755+0000","last_active":"2024-07-04T17:33:10.091219+0000","last_peered":"2024-07-04T17:33:10.091219+0000","last_clean":"2024-07-04T17:33:10.091219+0000","last_became_active":"2024-07-04T17:33:09.100389+0000","last_became_peered":"2024-07-04T17:33:09.100389+0000","last_unstale":"2024-07-04T17:33:10.091219+0000","last_undegraded":"2024-07-04T17:33:10.091219+0000","last_fullsized":"2024-07-04T17:33:10.091219+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:38:04.457302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.128545+0000","last_change":"2024-07-04T17:33:08.106647+0000","last_active":"2024-07-04T17:33:10.128545+0000","last_peered":"2024-07-04T17:33:10.128545+0000","last_clean":"2024-07-04T17:33:10.128545+0000","last_became_active":"2024-07-04T17:33:08.106277+0000","last_became_peered":"2024-07-04T17:33:08.106277+0000","last_unstale":"2024-07-04T17:33:10.128545+0000","last_undegraded":"2024-07-04T17:33:10.128545+0000","last_fullsized":"2024-07-04T17:33:10.128545+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:47:04.050710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.090776+0000","last_change":"2024-07-04T17:33:09.128294+0000","last_active":"2024-07-04T17:33:10.090776+0000","last_peered":"2024-07-04T17:33:10.090776+0000","last_clean":"2024-07-04T17:33:10.090776+0000","last_became_active":"2024-07-04T17:33:09.123696+0000","last_became_peered":"2024-07-04T17:33:09.123696+0000","last_unstale":"2024-07-04T17:33:10.090776+0000","last_undegraded":"2024-07-04T17:33:10.090776+0000","last_fullsized":"2024-07-04T17:33:10.090776+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:07:05.456492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.632103+0000","last_change":"2024-07-04T17:33:08.100600+0000","last_active":"2024-07-04T17:33:10.632103+0000","last_peered":"2024-07-04T17:33:10.632103+0000","last_clean":"2024-07-04T17:33:10.632103+0000","last_became_active":"2024-07-04T17:33:08.099352+0000","last_became_peered":"2024-07-04T17:33:08.099352+0000","last_unstale":"2024-07-04T17:33:10.632103+0000","last_undegraded":"2024-07-04T17:33:10.632103+0000","last_fullsized":"2024-07-04T17:33:10.632103+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:04:33.482388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.128669+0000","last_change":"2024-07-04T17:33:09.109950+0000","last_active":"2024-07-04T17:33:10.128669+0000","last_peered":"2024-07-04T17:33:10.128669+0000","last_clean":"2024-07-04T17:33:10.128669+0000","last_became_active":"2024-07-04T17:33:09.108850+0000","last_became_peered":"2024-07-04T17:33:09.108850+0000","last_unstale":"2024-07-04T17:33:10.128669+0000","last_undegraded":"2024-07-04T17:33:10.128669+0000","last_fullsized":"2024-07-04T17:33:10.128669+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:22:58.651803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.631462+0000","last_change":"2024-07-04T17:33:08.117619+0000","last_active":"2024-07-04T17:33:10.631462+0000","last_peered":"2024-07-04T17:33:10.631462+0000","last_clean":"2024-07-04T17:33:10.631462+0000","last_became_active":"2024-07-04T17:33:08.117033+0000","last_became_peered":"2024-07-04T17:33:08.117033+0000","last_unstale":"2024-07-04T17:33:10.631462+0000","last_undegraded":"2024-07-04T17:33:10.631462+0000","last_fullsized":"2024-07-04T17:33:10.631462+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:41:18.737857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.091601+0000","last_change":"2024-07-04T17:33:09.129855+0000","last_active":"2024-07-04T17:33:10.091601+0000","last_peered":"2024-07-04T17:33:10.091601+0000","last_clean":"2024-07-04T17:33:10.091601+0000","last_became_active":"2024-07-04T17:33:09.127858+0000","last_became_peered":"2024-07-04T17:33:09.127858+0000","last_unstale":"2024-07-04T17:33:10.091601+0000","last_undegraded":"2024-07-04T17:33:10.091601+0000","last_fullsized":"2024-07-04T17:33:10.091601+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:35:51.815415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"113'6","reported_seq":20,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.127766+0000","last_change":"2024-07-04T17:33:08.105018+0000","last_active":"2024-07-04T17:33:10.127766+0000","last_peered":"2024-07-04T17:33:10.127766+0000","last_clean":"2024-07-04T17:33:10.127766+0000","last_became_active":"2024-07-04T17:33:08.104685+0000","last_became_peered":"2024-07-04T17:33:08.104685+0000","last_unstale":"2024-07-04T17:33:10.127766+0000","last_undegraded":"2024-07-04T17:33:10.127766+0000","last_fullsized":"2024-07-04T17:33:10.127766+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:13:39.096736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.089655+0000","last_change":"2024-07-04T17:33:09.122352+0000","last_active":"2024-07-04T17:33:10.089655+0000","last_peered":"2024-07-04T17:33:10.089655+0000","last_clean":"2024-07-04T17:33:10.089655+0000","last_became_active":"2024-07-04T17:33:09.121662+0000","last_became_peered":"2024-07-04T17:33:09.121662+0000","last_unstale":"2024-07-04T17:33:10.089655+0000","last_undegraded":"2024-07-04T17:33:10.089655+0000","last_fullsized":"2024-07-04T17:33:10.089655+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:38:27.893548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.630328+0000","last_change":"2024-07-04T17:33:08.100243+0000","last_active":"2024-07-04T17:33:10.630328+0000","last_peered":"2024-07-04T17:33:10.630328+0000","last_clean":"2024-07-04T17:33:10.630328+0000","last_became_active":"2024-07-04T17:33:08.099860+0000","last_became_peered":"2024-07-04T17:33:08.099860+0000","last_unstale":"2024-07-04T17:33:10.630328+0000","last_undegraded":"2024-07-04T17:33:10.630328+0000","last_fullsized":"2024-07-04T17:33:10.630328+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:44:16.021572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.629956+0000","last_change":"2024-07-04T17:33:09.124609+0000","last_active":"2024-07-04T17:33:10.629956+0000","last_peered":"2024-07-04T17:33:10.629956+0000","last_clean":"2024-07-04T17:33:10.629956+0000","last_became_active":"2024-07-04T17:33:09.123720+0000","last_became_peered":"2024-07-04T17:33:09.123720+0000","last_unstale":"2024-07-04T17:33:10.629956+0000","last_undegraded":"2024-07-04T17:33:10.629956+0000","last_fullsized":"2024-07-04T17:33:10.629956+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:59:00.559158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.127073+0000","last_change":"2024-07-04T17:33:08.092450+0000","last_active":"2024-07-04T17:33:10.127073+0000","last_peered":"2024-07-04T17:33:10.127073+0000","last_clean":"2024-07-04T17:33:10.127073+0000","last_became_active":"2024-07-04T17:33:08.092088+0000","last_became_peered":"2024-07-04T17:33:08.092088+0000","last_unstale":"2024-07-04T17:33:10.127073+0000","last_undegraded":"2024-07-04T17:33:10.127073+0000","last_fullsized":"2024-07-04T17:33:10.127073+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:54:46.681088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.089354+0000","last_change":"2024-07-04T17:33:09.101721+0000","last_active":"2024-07-04T17:33:10.089354+0000","last_peered":"2024-07-04T17:33:10.089354+0000","last_clean":"2024-07-04T17:33:10.089354+0000","last_became_active":"2024-07-04T17:33:09.101084+0000","last_became_peered":"2024-07-04T17:33:09.101084+0000","last_unstale":"2024-07-04T17:33:10.089354+0000","last_undegraded":"2024-07-04T17:33:10.089354+0000","last_fullsized":"2024-07-04T17:33:10.089354+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:35:19.200734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.628612+0000","last_change":"2024-07-04T17:33:08.098833+0000","last_active":"2024-07-04T17:33:10.628612+0000","last_peered":"2024-07-04T17:33:10.628612+0000","last_clean":"2024-07-04T17:33:10.628612+0000","last_became_active":"2024-07-04T17:33:08.098492+0000","last_became_peered":"2024-07-04T17:33:08.098492+0000","last_unstale":"2024-07-04T17:33:10.628612+0000","last_undegraded":"2024-07-04T17:33:10.628612+0000","last_fullsized":"2024-07-04T17:33:10.628612+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:40:56.816879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.090378+0000","last_change":"2024-07-04T17:33:09.131477+0000","last_active":"2024-07-04T17:33:10.090378+0000","last_peered":"2024-07-04T17:33:10.090378+0000","last_clean":"2024-07-04T17:33:10.090378+0000","last_became_active":"2024-07-04T17:33:09.122889+0000","last_became_peered":"2024-07-04T17:33:09.122889+0000","last_unstale":"2024-07-04T17:33:10.090378+0000","last_undegraded":"2024-07-04T17:33:10.090378+0000","last_fullsized":"2024-07-04T17:33:10.090378+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:48:40.827656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"113'8","reported_seq":22,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.090899+0000","last_change":"2024-07-04T17:33:08.094956+0000","last_active":"2024-07-04T17:33:10.090899+0000","last_peered":"2024-07-04T17:33:10.090899+0000","last_clean":"2024-07-04T17:33:10.090899+0000","last_became_active":"2024-07-04T17:33:08.090486+0000","last_became_peered":"2024-07-04T17:33:08.090486+0000","last_unstale":"2024-07-04T17:33:10.090899+0000","last_undegraded":"2024-07-04T17:33:10.090899+0000","last_fullsized":"2024-07-04T17:33:10.090899+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:05:13.908213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.631246+0000","last_change":"2024-07-04T17:33:09.128035+0000","last_active":"2024-07-04T17:33:10.631246+0000","last_peered":"2024-07-04T17:33:10.631246+0000","last_clean":"2024-07-04T17:33:10.631246+0000","last_became_active":"2024-07-04T17:33:09.125335+0000","last_became_peered":"2024-07-04T17:33:09.125335+0000","last_unstale":"2024-07-04T17:33:10.631246+0000","last_undegraded":"2024-07-04T17:33:10.631246+0000","last_fullsized":"2024-07-04T17:33:10.631246+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:35:53.125670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.092351+0000","last_change":"2024-07-04T17:33:08.096006+0000","last_active":"2024-07-04T17:33:10.092351+0000","last_peered":"2024-07-04T17:33:10.092351+0000","last_clean":"2024-07-04T17:33:10.092351+0000","last_became_active":"2024-07-04T17:33:08.095685+0000","last_became_peered":"2024-07-04T17:33:08.095685+0000","last_unstale":"2024-07-04T17:33:10.092351+0000","last_undegraded":"2024-07-04T17:33:10.092351+0000","last_fullsized":"2024-07-04T17:33:10.092351+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:58:57.013662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.092079+0000","last_change":"2024-07-04T17:33:09.118421+0000","last_active":"2024-07-04T17:33:10.092079+0000","last_peered":"2024-07-04T17:33:10.092079+0000","last_clean":"2024-07-04T17:33:10.092079+0000","last_became_active":"2024-07-04T17:33:09.117653+0000","last_became_peered":"2024-07-04T17:33:09.117653+0000","last_unstale":"2024-07-04T17:33:10.092079+0000","last_undegraded":"2024-07-04T17:33:10.092079+0000","last_fullsized":"2024-07-04T17:33:10.092079+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:59:43.093156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.631515+0000","last_change":"2024-07-04T17:33:08.113131+0000","last_active":"2024-07-04T17:33:10.631515+0000","last_peered":"2024-07-04T17:33:10.631515+0000","last_clean":"2024-07-04T17:33:10.631515+0000","last_became_active":"2024-07-04T17:33:08.112337+0000","last_became_peered":"2024-07-04T17:33:08.112337+0000","last_unstale":"2024-07-04T17:33:10.631515+0000","last_undegraded":"2024-07-04T17:33:10.631515+0000","last_fullsized":"2024-07-04T17:33:10.631515+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:45:29.198114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.630701+0000","last_change":"2024-07-04T17:33:09.107000+0000","last_active":"2024-07-04T17:33:10.630701+0000","last_peered":"2024-07-04T17:33:10.630701+0000","last_clean":"2024-07-04T17:33:10.630701+0000","last_became_active":"2024-07-04T17:33:09.106204+0000","last_became_peered":"2024-07-04T17:33:09.106204+0000","last_unstale":"2024-07-04T17:33:10.630701+0000","last_undegraded":"2024-07-04T17:33:10.630701+0000","last_fullsized":"2024-07-04T17:33:10.630701+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:22:24.403011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.091704+0000","last_change":"2024-07-04T17:33:08.105392+0000","last_active":"2024-07-04T17:33:10.091704+0000","last_peered":"2024-07-04T17:33:10.091704+0000","last_clean":"2024-07-04T17:33:10.091704+0000","last_became_active":"2024-07-04T17:33:08.105067+0000","last_became_peered":"2024-07-04T17:33:08.105067+0000","last_unstale":"2024-07-04T17:33:10.091704+0000","last_undegraded":"2024-07-04T17:33:10.091704+0000","last_fullsized":"2024-07-04T17:33:10.091704+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:10:49.236238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.631540+0000","last_change":"2024-07-04T17:33:09.126710+0000","last_active":"2024-07-04T17:33:10.631540+0000","last_peered":"2024-07-04T17:33:10.631540+0000","last_clean":"2024-07-04T17:33:10.631540+0000","last_became_active":"2024-07-04T17:33:09.122975+0000","last_became_peered":"2024-07-04T17:33:09.122975+0000","last_unstale":"2024-07-04T17:33:10.631540+0000","last_undegraded":"2024-07-04T17:33:10.631540+0000","last_fullsized":"2024-07-04T17:33:10.631540+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:48:57.200238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.631877+0000","last_change":"2024-07-04T17:33:08.114916+0000","last_active":"2024-07-04T17:33:10.631877+0000","last_peered":"2024-07-04T17:33:10.631877+0000","last_clean":"2024-07-04T17:33:10.631877+0000","last_became_active":"2024-07-04T17:33:08.114455+0000","last_became_peered":"2024-07-04T17:33:08.114455+0000","last_unstale":"2024-07-04T17:33:10.631877+0000","last_undegraded":"2024-07-04T17:33:10.631877+0000","last_fullsized":"2024-07-04T17:33:10.631877+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:09:08.764721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.126957+0000","last_change":"2024-07-04T17:33:09.105710+0000","last_active":"2024-07-04T17:33:10.126957+0000","last_peered":"2024-07-04T17:33:10.126957+0000","last_clean":"2024-07-04T17:33:10.126957+0000","last_became_active":"2024-07-04T17:33:09.101819+0000","last_became_peered":"2024-07-04T17:33:09.101819+0000","last_unstale":"2024-07-04T17:33:10.126957+0000","last_undegraded":"2024-07-04T17:33:10.126957+0000","last_fullsized":"2024-07-04T17:33:10.126957+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:47:15.892916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.129007+0000","last_change":"2024-07-04T17:33:08.108404+0000","last_active":"2024-07-04T17:33:10.129007+0000","last_peered":"2024-07-04T17:33:10.129007+0000","last_clean":"2024-07-04T17:33:10.129007+0000","last_became_active":"2024-07-04T17:33:08.107387+0000","last_became_peered":"2024-07-04T17:33:08.107387+0000","last_unstale":"2024-07-04T17:33:10.129007+0000","last_undegraded":"2024-07-04T17:33:10.129007+0000","last_fullsized":"2024-07-04T17:33:10.129007+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:10:32.480743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.128952+0000","last_change":"2024-07-04T17:33:09.102491+0000","last_active":"2024-07-04T17:33:10.128952+0000","last_peered":"2024-07-04T17:33:10.128952+0000","last_clean":"2024-07-04T17:33:10.128952+0000","last_became_active":"2024-07-04T17:33:09.102170+0000","last_became_peered":"2024-07-04T17:33:09.102170+0000","last_unstale":"2024-07-04T17:33:10.128952+0000","last_undegraded":"2024-07-04T17:33:10.128952+0000","last_fullsized":"2024-07-04T17:33:10.128952+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:03:31.387343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.090123+0000","last_change":"2024-07-04T17:33:08.109294+0000","last_active":"2024-07-04T17:33:10.090123+0000","last_peered":"2024-07-04T17:33:10.090123+0000","last_clean":"2024-07-04T17:33:10.090123+0000","last_became_active":"2024-07-04T17:33:08.108944+0000","last_became_peered":"2024-07-04T17:33:08.108944+0000","last_unstale":"2024-07-04T17:33:10.090123+0000","last_undegraded":"2024-07-04T17:33:10.090123+0000","last_fullsized":"2024-07-04T17:33:10.090123+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:15:36.989862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.126703+0000","last_change":"2024-07-04T17:33:09.111629+0000","last_active":"2024-07-04T17:33:10.126703+0000","last_peered":"2024-07-04T17:33:10.126703+0000","last_clean":"2024-07-04T17:33:10.126703+0000","last_became_active":"2024-07-04T17:33:09.111332+0000","last_became_peered":"2024-07-04T17:33:09.111332+0000","last_unstale":"2024-07-04T17:33:10.126703+0000","last_undegraded":"2024-07-04T17:33:10.126703+0000","last_fullsized":"2024-07-04T17:33:10.126703+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:44:43.830281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.090536+0000","last_change":"2024-07-04T17:33:08.097164+0000","last_active":"2024-07-04T17:33:10.090536+0000","last_peered":"2024-07-04T17:33:10.090536+0000","last_clean":"2024-07-04T17:33:10.090536+0000","last_became_active":"2024-07-04T17:33:08.096841+0000","last_became_peered":"2024-07-04T17:33:08.096841+0000","last_unstale":"2024-07-04T17:33:10.090536+0000","last_undegraded":"2024-07-04T17:33:10.090536+0000","last_fullsized":"2024-07-04T17:33:10.090536+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:24:50.491277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.126703+0000","last_change":"2024-07-04T17:33:09.097586+0000","last_active":"2024-07-04T17:33:10.126703+0000","last_peered":"2024-07-04T17:33:10.126703+0000","last_clean":"2024-07-04T17:33:10.126703+0000","last_became_active":"2024-07-04T17:33:09.096586+0000","last_became_peered":"2024-07-04T17:33:09.096586+0000","last_unstale":"2024-07-04T17:33:10.126703+0000","last_undegraded":"2024-07-04T17:33:10.126703+0000","last_fullsized":"2024-07-04T17:33:10.126703+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:21:53.318774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.091046+0000","last_change":"2024-07-04T17:33:08.094584+0000","last_active":"2024-07-04T17:33:10.091046+0000","last_peered":"2024-07-04T17:33:10.091046+0000","last_clean":"2024-07-04T17:33:10.091046+0000","last_became_active":"2024-07-04T17:33:08.090067+0000","last_became_peered":"2024-07-04T17:33:08.090067+0000","last_unstale":"2024-07-04T17:33:10.091046+0000","last_undegraded":"2024-07-04T17:33:10.091046+0000","last_fullsized":"2024-07-04T17:33:10.091046+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:17:57.214638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.091398+0000","last_change":"2024-07-04T17:33:09.131493+0000","last_active":"2024-07-04T17:33:10.091398+0000","last_peered":"2024-07-04T17:33:10.091398+0000","last_clean":"2024-07-04T17:33:10.091398+0000","last_became_active":"2024-07-04T17:33:09.131132+0000","last_became_peered":"2024-07-04T17:33:09.131132+0000","last_unstale":"2024-07-04T17:33:10.091398+0000","last_undegraded":"2024-07-04T17:33:10.091398+0000","last_fullsized":"2024-07-04T17:33:10.091398+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:56:22.629300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.092649+0000","last_change":"2024-07-04T17:33:08.105850+0000","last_active":"2024-07-04T17:33:10.092649+0000","last_peered":"2024-07-04T17:33:10.092649+0000","last_clean":"2024-07-04T17:33:10.092649+0000","last_became_active":"2024-07-04T17:33:08.104987+0000","last_became_peered":"2024-07-04T17:33:08.104987+0000","last_unstale":"2024-07-04T17:33:10.092649+0000","last_undegraded":"2024-07-04T17:33:10.092649+0000","last_fullsized":"2024-07-04T17:33:10.092649+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:36:17.000989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.092449+0000","last_change":"2024-07-04T17:33:09.118693+0000","last_active":"2024-07-04T17:33:10.092449+0000","last_peered":"2024-07-04T17:33:10.092449+0000","last_clean":"2024-07-04T17:33:10.092449+0000","last_became_active":"2024-07-04T17:33:09.117928+0000","last_became_peered":"2024-07-04T17:33:09.117928+0000","last_unstale":"2024-07-04T17:33:10.092449+0000","last_undegraded":"2024-07-04T17:33:10.092449+0000","last_fullsized":"2024-07-04T17:33:10.092449+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:38:35.544115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"114'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.631708+0000","last_change":"2024-07-04T17:33:08.116106+0000","last_active":"2024-07-04T17:33:10.631708+0000","last_peered":"2024-07-04T17:33:10.631708+0000","last_clean":"2024-07-04T17:33:10.631708+0000","last_became_active":"2024-07-04T17:33:08.115757+0000","last_became_peered":"2024-07-04T17:33:08.115757+0000","last_unstale":"2024-07-04T17:33:10.631708+0000","last_undegraded":"2024-07-04T17:33:10.631708+0000","last_fullsized":"2024-07-04T17:33:10.631708+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:16:30.420529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.631205+0000","last_change":"2024-07-04T17:33:09.098889+0000","last_active":"2024-07-04T17:33:10.631205+0000","last_peered":"2024-07-04T17:33:10.631205+0000","last_clean":"2024-07-04T17:33:10.631205+0000","last_became_active":"2024-07-04T17:33:09.098564+0000","last_became_peered":"2024-07-04T17:33:09.098564+0000","last_unstale":"2024-07-04T17:33:10.631205+0000","last_undegraded":"2024-07-04T17:33:10.631205+0000","last_fullsized":"2024-07-04T17:33:10.631205+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:06:27.253525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.632086+0000","last_change":"2024-07-04T17:33:08.114022+0000","last_active":"2024-07-04T17:33:10.632086+0000","last_peered":"2024-07-04T17:33:10.632086+0000","last_clean":"2024-07-04T17:33:10.632086+0000","last_became_active":"2024-07-04T17:33:08.112033+0000","last_became_peered":"2024-07-04T17:33:08.112033+0000","last_unstale":"2024-07-04T17:33:10.632086+0000","last_undegraded":"2024-07-04T17:33:10.632086+0000","last_fullsized":"2024-07-04T17:33:10.632086+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:00:33.769816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.631621+0000","last_change":"2024-07-04T17:33:09.113463+0000","last_active":"2024-07-04T17:33:10.631621+0000","last_peered":"2024-07-04T17:33:10.631621+0000","last_clean":"2024-07-04T17:33:10.631621+0000","last_became_active":"2024-07-04T17:33:09.112826+0000","last_became_peered":"2024-07-04T17:33:09.112826+0000","last_unstale":"2024-07-04T17:33:10.631621+0000","last_undegraded":"2024-07-04T17:33:10.631621+0000","last_fullsized":"2024-07-04T17:33:10.631621+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:26:59.083650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.631999+0000","last_change":"2024-07-04T17:33:08.114768+0000","last_active":"2024-07-04T17:33:10.631999+0000","last_peered":"2024-07-04T17:33:10.631999+0000","last_clean":"2024-07-04T17:33:10.631999+0000","last_became_active":"2024-07-04T17:33:08.113902+0000","last_became_peered":"2024-07-04T17:33:08.113902+0000","last_unstale":"2024-07-04T17:33:10.631999+0000","last_undegraded":"2024-07-04T17:33:10.631999+0000","last_fullsized":"2024-07-04T17:33:10.631999+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:58:34.807900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.089969+0000","last_change":"2024-07-04T17:33:09.128723+0000","last_active":"2024-07-04T17:33:10.089969+0000","last_peered":"2024-07-04T17:33:10.089969+0000","last_clean":"2024-07-04T17:33:10.089969+0000","last_became_active":"2024-07-04T17:33:09.128217+0000","last_became_peered":"2024-07-04T17:33:09.128217+0000","last_unstale":"2024-07-04T17:33:10.089969+0000","last_undegraded":"2024-07-04T17:33:10.089969+0000","last_fullsized":"2024-07-04T17:33:10.089969+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:08:28.724747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.129017+0000","last_change":"2024-07-04T17:33:08.108382+0000","last_active":"2024-07-04T17:33:10.129017+0000","last_peered":"2024-07-04T17:33:10.129017+0000","last_clean":"2024-07-04T17:33:10.129017+0000","last_became_active":"2024-07-04T17:33:08.107656+0000","last_became_peered":"2024-07-04T17:33:08.107656+0000","last_unstale":"2024-07-04T17:33:10.129017+0000","last_undegraded":"2024-07-04T17:33:10.129017+0000","last_fullsized":"2024-07-04T17:33:10.129017+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:55:46.289096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.091064+0000","last_change":"2024-07-04T17:33:09.129070+0000","last_active":"2024-07-04T17:33:10.091064+0000","last_peered":"2024-07-04T17:33:10.091064+0000","last_clean":"2024-07-04T17:33:10.091064+0000","last_became_active":"2024-07-04T17:33:09.128751+0000","last_became_peered":"2024-07-04T17:33:09.128751+0000","last_unstale":"2024-07-04T17:33:10.091064+0000","last_undegraded":"2024-07-04T17:33:10.091064+0000","last_fullsized":"2024-07-04T17:33:10.091064+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:39:44.077373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.127426+0000","last_change":"2024-07-04T17:33:08.096375+0000","last_active":"2024-07-04T17:33:10.127426+0000","last_peered":"2024-07-04T17:33:10.127426+0000","last_clean":"2024-07-04T17:33:10.127426+0000","last_became_active":"2024-07-04T17:33:08.096060+0000","last_became_peered":"2024-07-04T17:33:08.096060+0000","last_unstale":"2024-07-04T17:33:10.127426+0000","last_undegraded":"2024-07-04T17:33:10.127426+0000","last_fullsized":"2024-07-04T17:33:10.127426+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:08:19.414085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.090019+0000","last_change":"2024-07-04T17:33:09.107349+0000","last_active":"2024-07-04T17:33:10.090019+0000","last_peered":"2024-07-04T17:33:10.090019+0000","last_clean":"2024-07-04T17:33:10.090019+0000","last_became_active":"2024-07-04T17:33:09.104954+0000","last_became_peered":"2024-07-04T17:33:09.104954+0000","last_unstale":"2024-07-04T17:33:10.090019+0000","last_undegraded":"2024-07-04T17:33:10.090019+0000","last_fullsized":"2024-07-04T17:33:10.090019+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:19:28.456336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.127031+0000","last_change":"2024-07-04T17:33:08.106155+0000","last_active":"2024-07-04T17:33:10.127031+0000","last_peered":"2024-07-04T17:33:10.127031+0000","last_clean":"2024-07-04T17:33:10.127031+0000","last_became_active":"2024-07-04T17:33:08.104825+0000","last_became_peered":"2024-07-04T17:33:08.104825+0000","last_unstale":"2024-07-04T17:33:10.127031+0000","last_undegraded":"2024-07-04T17:33:10.127031+0000","last_fullsized":"2024-07-04T17:33:10.127031+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:15:59.977728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.126732+0000","last_change":"2024-07-04T17:33:09.107749+0000","last_active":"2024-07-04T17:33:10.126732+0000","last_peered":"2024-07-04T17:33:10.126732+0000","last_clean":"2024-07-04T17:33:10.126732+0000","last_became_active":"2024-07-04T17:33:09.106400+0000","last_became_peered":"2024-07-04T17:33:09.106400+0000","last_unstale":"2024-07-04T17:33:10.126732+0000","last_undegraded":"2024-07-04T17:33:10.126732+0000","last_fullsized":"2024-07-04T17:33:10.126732+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:22:04.584333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.091166+0000","last_change":"2024-07-04T17:33:08.109770+0000","last_active":"2024-07-04T17:33:10.091166+0000","last_peered":"2024-07-04T17:33:10.091166+0000","last_clean":"2024-07-04T17:33:10.091166+0000","last_became_active":"2024-07-04T17:33:08.108245+0000","last_became_peered":"2024-07-04T17:33:08.108245+0000","last_unstale":"2024-07-04T17:33:10.091166+0000","last_undegraded":"2024-07-04T17:33:10.091166+0000","last_fullsized":"2024-07-04T17:33:10.091166+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:30:01.785284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.127795+0000","last_change":"2024-07-04T17:33:09.099278+0000","last_active":"2024-07-04T17:33:10.127795+0000","last_peered":"2024-07-04T17:33:10.127795+0000","last_clean":"2024-07-04T17:33:10.127795+0000","last_became_active":"2024-07-04T17:33:09.098752+0000","last_became_peered":"2024-07-04T17:33:09.098752+0000","last_unstale":"2024-07-04T17:33:10.127795+0000","last_undegraded":"2024-07-04T17:33:10.127795+0000","last_fullsized":"2024-07-04T17:33:10.127795+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:34:51.476685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"113'3","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.128334+0000","last_change":"2024-07-04T17:33:08.092774+0000","last_active":"2024-07-04T17:33:10.128334+0000","last_peered":"2024-07-04T17:33:10.128334+0000","last_clean":"2024-07-04T17:33:10.128334+0000","last_became_active":"2024-07-04T17:33:08.092404+0000","last_became_peered":"2024-07-04T17:33:08.092404+0000","last_unstale":"2024-07-04T17:33:10.128334+0000","last_undegraded":"2024-07-04T17:33:10.128334+0000","last_fullsized":"2024-07-04T17:33:10.128334+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:48:02.318598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"100'101","reported_seq":448,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.629308+0000","last_change":"2024-07-04T17:26:04.953559+0000","last_active":"2024-07-04T17:33:10.629308+0000","last_peered":"2024-07-04T17:33:10.629308+0000","last_clean":"2024-07-04T17:33:10.629308+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:33:10.629308+0000","last_undegraded":"2024-07-04T17:33:10.629308+0000","last_fullsized":"2024-07-04T17:33:10.629308+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_clean_scrub_stamp":"2024-07-04T17:26:03.933432+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.064181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":476,"num_read_kb":1278,"num_write":163,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":476,"num_read_kb":1278,"num_write":163,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101,"ondisk_log_size":101,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738453,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36504,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335336,"statfs":{"total":96636764160,"available":96599384064,"internally_reserved":0,"allocated":1187840,"data_stored":464184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738454,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37072,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334768,"statfs":{"total":96636764160,"available":96598802432,"internally_reserved":0,"allocated":1773568,"data_stored":1053042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37088,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334752,"statfs":{"total":96636764160,"available":96598786048,"internally_reserved":0,"allocated":1781760,"data_stored":1054024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:33:13.813 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:33:14.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.149+0000 7fbd23200640 1 -- 172.21.15.42:0/3470710872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd240bad70 msgr2=0x7fbd2406c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:14.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.149+0000 7fbd23200640 1 --2- 172.21.15.42:0/3470710872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd240bad70 0x7fbd2406c2c0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fbd10005fd0 tx=0x7fbd1002b890 comp rx=0 tx=0).stop 2024-07-04T17:33:14.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.149+0000 7fbd23200640 1 -- 172.21.15.42:0/3470710872 shutdown_connections 2024-07-04T17:33:14.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.149+0000 7fbd23200640 1 --2- 172.21.15.42:0/3470710872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd240bad70 0x7fbd2406c2c0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:14.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.149+0000 7fbd23200640 1 -- 172.21.15.42:0/3470710872 >> 172.21.15.42:0/3470710872 conn(0x7fbd240bebc0 msgr2=0x7fbd240befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:14.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.150+0000 7fbd23200640 1 -- 172.21.15.42:0/3470710872 shutdown_connections 2024-07-04T17:33:14.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.150+0000 7fbd23200640 1 -- 172.21.15.42:0/3470710872 wait complete. 2024-07-04T17:33:14.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.152+0000 7fbd23200640 1 Processor -- start 2024-07-04T17:33:14.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.152+0000 7fbd23200640 1 -- start start 2024-07-04T17:33:14.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.154+0000 7fbd23200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd2406d580 0x7fbd240b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:14.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.154+0000 7fbd23200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbd240b2810 con 0x7fbd2406d580 2024-07-04T17:33:14.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.154+0000 7fbd21e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd2406d580 0x7fbd240b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:14.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.154+0000 7fbd21e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd2406d580 0x7fbd240b7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58774/0 (socket says 172.21.15.42:58774) 2024-07-04T17:33:14.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.154+0000 7fbd21e00640 1 -- 172.21.15.42:0/2469032385 learned_addr learned my addr 172.21.15.42:0/2469032385 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:14.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.155+0000 7fbd21e00640 1 -- 172.21.15.42:0/2469032385 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd241d2ec0 con 0x7fbd2406d580 2024-07-04T17:33:14.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.155+0000 7fbd21e00640 1 --2- 172.21.15.42:0/2469032385 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd2406d580 0x7fbd240b7db0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fbd1400c320 tx=0x7fbd1400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:14.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.156+0000 7fbd1a000640 1 -- 172.21.15.42:0/2469032385 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd14017070 con 0x7fbd2406d580 2024-07-04T17:33:14.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.156+0000 7fbd1a000640 1 -- 172.21.15.42:0/2469032385 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd14005310 con 0x7fbd2406d580 2024-07-04T17:33:14.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.156+0000 7fbd23200640 1 -- 172.21.15.42:0/2469032385 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd241d33c0 con 0x7fbd2406d580 2024-07-04T17:33:14.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.156+0000 7fbd1a000640 1 -- 172.21.15.42:0/2469032385 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd140035f0 con 0x7fbd2406d580 2024-07-04T17:33:14.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.156+0000 7fbd23200640 1 -- 172.21.15.42:0/2469032385 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd241d2b40 con 0x7fbd2406d580 2024-07-04T17:33:14.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.157+0000 7fbd23200640 1 -- 172.21.15.42:0/2469032385 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd2406c2c0 con 0x7fbd2406d580 2024-07-04T17:33:14.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.157+0000 7fbd1a000640 1 -- 172.21.15.42:0/2469032385 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7fbd1401a070 con 0x7fbd2406d580 2024-07-04T17:33:14.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.158+0000 7fbd1a000640 1 --2- 172.21.15.42:0/2469032385 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fbd0403d4f0 0x7fbd0403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:14.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.158+0000 7fbd1a000640 1 -- 172.21.15.42:0/2469032385 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 3439+0+0 (secure 0 0 0) 0x7fbd14050c20 con 0x7fbd2406d580 2024-07-04T17:33:14.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.158+0000 7fbd21400640 1 --2- 172.21.15.42:0/2469032385 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fbd0403d4f0 0x7fbd0403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:14.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.159+0000 7fbd21400640 1 --2- 172.21.15.42:0/2469032385 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fbd0403d4f0 0x7fbd0403fa00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbd10035000 tx=0x7fbd10004730 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:14.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.163+0000 7fbd1a000640 1 -- 172.21.15.42:0/2469032385 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fbd1401bc60 con 0x7fbd2406d580 2024-07-04T17:33:14.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.374+0000 7fbd23200640 1 -- 172.21.15.42:0/2469032385 --> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbd24068e30 con 0x7fbd0403d4f0 2024-07-04T17:33:14.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.381+0000 7fbd1a000640 1 -- 172.21.15.42:0/2469032385 <== mgr.5434 v2:172.21.15.42:6834/4130303917 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246494 (secure 0 0 0) 0x7fbd24068e30 con 0x7fbd0403d4f0 2024-07-04T17:33:14.385 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:33:14.387 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:33:14.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.386+0000 7fbd23200640 1 -- 172.21.15.42:0/2469032385 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fbd0403d4f0 msgr2=0x7fbd0403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:14.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.386+0000 7fbd23200640 1 --2- 172.21.15.42:0/2469032385 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fbd0403d4f0 0x7fbd0403fa00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbd10035000 tx=0x7fbd10004730 comp rx=0 tx=0).stop 2024-07-04T17:33:14.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.386+0000 7fbd23200640 1 -- 172.21.15.42:0/2469032385 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd2406d580 msgr2=0x7fbd240b7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:14.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.386+0000 7fbd23200640 1 --2- 172.21.15.42:0/2469032385 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd2406d580 0x7fbd240b7db0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fbd1400c320 tx=0x7fbd1400c7e0 comp rx=0 tx=0).stop 2024-07-04T17:33:14.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.387+0000 7fbd23200640 1 -- 172.21.15.42:0/2469032385 shutdown_connections 2024-07-04T17:33:14.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.387+0000 7fbd23200640 1 --2- 172.21.15.42:0/2469032385 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fbd0403d4f0 0x7fbd0403fa00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:14.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.387+0000 7fbd23200640 1 --2- 172.21.15.42:0/2469032385 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd2406d580 0x7fbd240b7db0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:14.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.387+0000 7fbd23200640 1 -- 172.21.15.42:0/2469032385 >> 172.21.15.42:0/2469032385 conn(0x7fbd240bebc0 msgr2=0x7fbd241da010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:14.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.387+0000 7fbd23200640 1 -- 172.21.15.42:0/2469032385 shutdown_connections 2024-07-04T17:33:14.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.387+0000 7fbd23200640 1 -- 172.21.15.42:0/2469032385 wait complete. 2024-07-04T17:33:14.412 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-07-04T17:33:13.274921+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":476,"num_read_kb":1278,"num_write":208,"num_write_kb":1120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":110664,"kb_used_data":4632,"kb_used_omap":4,"kb_used_meta":105979,"kb_avail":283004856,"statfs":{"total":289910292480,"available":289796972544,"internally_reserved":0,"allocated":4743168,"data_stored":2571250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":108522831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001662"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.631240+0000","last_change":"2024-07-04T17:33:09.110137+0000","last_active":"2024-07-04T17:33:10.631240+0000","last_peered":"2024-07-04T17:33:10.631240+0000","last_clean":"2024-07-04T17:33:10.631240+0000","last_became_active":"2024-07-04T17:33:09.109336+0000","last_became_peered":"2024-07-04T17:33:09.109336+0000","last_unstale":"2024-07-04T17:33:10.631240+0000","last_undegraded":"2024-07-04T17:33:10.631240+0000","last_fullsized":"2024-07-04T17:33:10.631240+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:09.113117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.127044+0000","last_change":"2024-07-04T17:33:09.113135+0000","last_active":"2024-07-04T17:33:10.127044+0000","last_peered":"2024-07-04T17:33:10.127044+0000","last_clean":"2024-07-04T17:33:10.127044+0000","last_became_active":"2024-07-04T17:33:09.112824+0000","last_became_peered":"2024-07-04T17:33:09.112824+0000","last_unstale":"2024-07-04T17:33:10.127044+0000","last_undegraded":"2024-07-04T17:33:10.127044+0000","last_fullsized":"2024-07-04T17:33:10.127044+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:25:36.254957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.630037+0000","last_change":"2024-07-04T17:33:09.126404+0000","last_active":"2024-07-04T17:33:10.630037+0000","last_peered":"2024-07-04T17:33:10.630037+0000","last_clean":"2024-07-04T17:33:10.630037+0000","last_became_active":"2024-07-04T17:33:09.126069+0000","last_became_peered":"2024-07-04T17:33:09.126069+0000","last_unstale":"2024-07-04T17:33:10.630037+0000","last_undegraded":"2024-07-04T17:33:10.630037+0000","last_fullsized":"2024-07-04T17:33:10.630037+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:54:17.261721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.089766+0000","last_change":"2024-07-04T17:33:09.104422+0000","last_active":"2024-07-04T17:33:10.089766+0000","last_peered":"2024-07-04T17:33:10.089766+0000","last_clean":"2024-07-04T17:33:10.089766+0000","last_became_active":"2024-07-04T17:33:09.104112+0000","last_became_peered":"2024-07-04T17:33:09.104112+0000","last_unstale":"2024-07-04T17:33:10.089766+0000","last_undegraded":"2024-07-04T17:33:10.089766+0000","last_fullsized":"2024-07-04T17:33:10.089766+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:50:45.806246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.629366+0000","last_change":"2024-07-04T17:33:09.111717+0000","last_active":"2024-07-04T17:33:10.629366+0000","last_peered":"2024-07-04T17:33:10.629366+0000","last_clean":"2024-07-04T17:33:10.629366+0000","last_became_active":"2024-07-04T17:33:09.110270+0000","last_became_peered":"2024-07-04T17:33:09.110270+0000","last_unstale":"2024-07-04T17:33:10.629366+0000","last_undegraded":"2024-07-04T17:33:10.629366+0000","last_fullsized":"2024-07-04T17:33:10.629366+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:18:53.013321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.628774+0000","last_change":"2024-07-04T17:33:09.108466+0000","last_active":"2024-07-04T17:33:10.628774+0000","last_peered":"2024-07-04T17:33:10.628774+0000","last_clean":"2024-07-04T17:33:10.628774+0000","last_became_active":"2024-07-04T17:33:09.108132+0000","last_became_peered":"2024-07-04T17:33:09.108132+0000","last_unstale":"2024-07-04T17:33:10.628774+0000","last_undegraded":"2024-07-04T17:33:10.628774+0000","last_fullsized":"2024-07-04T17:33:10.628774+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:51:32.675966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.126131+0000","last_change":"2024-07-04T17:33:09.108992+0000","last_active":"2024-07-04T17:33:10.126131+0000","last_peered":"2024-07-04T17:33:10.126131+0000","last_clean":"2024-07-04T17:33:10.126131+0000","last_became_active":"2024-07-04T17:33:09.107307+0000","last_became_peered":"2024-07-04T17:33:09.107307+0000","last_unstale":"2024-07-04T17:33:10.126131+0000","last_undegraded":"2024-07-04T17:33:10.126131+0000","last_fullsized":"2024-07-04T17:33:10.126131+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:32:07.600844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.628705+0000","last_change":"2024-07-04T17:33:09.127659+0000","last_active":"2024-07-04T17:33:10.628705+0000","last_peered":"2024-07-04T17:33:10.628705+0000","last_clean":"2024-07-04T17:33:10.628705+0000","last_became_active":"2024-07-04T17:33:09.126591+0000","last_became_peered":"2024-07-04T17:33:09.126591+0000","last_unstale":"2024-07-04T17:33:10.628705+0000","last_undegraded":"2024-07-04T17:33:10.628705+0000","last_fullsized":"2024-07-04T17:33:10.628705+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:44:07.778739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.631732+0000","last_change":"2024-07-04T17:33:09.110971+0000","last_active":"2024-07-04T17:33:10.631732+0000","last_peered":"2024-07-04T17:33:10.631732+0000","last_clean":"2024-07-04T17:33:10.631732+0000","last_became_active":"2024-07-04T17:33:09.109775+0000","last_became_peered":"2024-07-04T17:33:09.109775+0000","last_unstale":"2024-07-04T17:33:10.631732+0000","last_undegraded":"2024-07-04T17:33:10.631732+0000","last_fullsized":"2024-07-04T17:33:10.631732+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:27:36.573799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.630135+0000","last_change":"2024-07-04T17:33:09.127783+0000","last_active":"2024-07-04T17:33:10.630135+0000","last_peered":"2024-07-04T17:33:10.630135+0000","last_clean":"2024-07-04T17:33:10.630135+0000","last_became_active":"2024-07-04T17:33:09.125335+0000","last_became_peered":"2024-07-04T17:33:09.125335+0000","last_unstale":"2024-07-04T17:33:10.630135+0000","last_undegraded":"2024-07-04T17:33:10.630135+0000","last_fullsized":"2024-07-04T17:33:10.630135+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:29:15.785171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.630552+0000","last_change":"2024-07-04T17:33:09.113535+0000","last_active":"2024-07-04T17:33:10.630552+0000","last_peered":"2024-07-04T17:33:10.630552+0000","last_clean":"2024-07-04T17:33:10.630552+0000","last_became_active":"2024-07-04T17:33:09.113127+0000","last_became_peered":"2024-07-04T17:33:09.113127+0000","last_unstale":"2024-07-04T17:33:10.630552+0000","last_undegraded":"2024-07-04T17:33:10.630552+0000","last_fullsized":"2024-07-04T17:33:10.630552+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:14:08.620654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.127257+0000","last_change":"2024-07-04T17:33:09.112587+0000","last_active":"2024-07-04T17:33:10.127257+0000","last_peered":"2024-07-04T17:33:10.127257+0000","last_clean":"2024-07-04T17:33:10.127257+0000","last_became_active":"2024-07-04T17:33:09.112063+0000","last_became_peered":"2024-07-04T17:33:09.112063+0000","last_unstale":"2024-07-04T17:33:10.127257+0000","last_undegraded":"2024-07-04T17:33:10.127257+0000","last_fullsized":"2024-07-04T17:33:10.127257+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:59:39.105749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.629414+0000","last_change":"2024-07-04T17:33:09.110573+0000","last_active":"2024-07-04T17:33:10.629414+0000","last_peered":"2024-07-04T17:33:10.629414+0000","last_clean":"2024-07-04T17:33:10.629414+0000","last_became_active":"2024-07-04T17:33:09.109386+0000","last_became_peered":"2024-07-04T17:33:09.109386+0000","last_unstale":"2024-07-04T17:33:10.629414+0000","last_undegraded":"2024-07-04T17:33:10.629414+0000","last_fullsized":"2024-07-04T17:33:10.629414+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:33:02.261126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.126377+0000","last_change":"2024-07-04T17:33:09.119181+0000","last_active":"2024-07-04T17:33:10.126377+0000","last_peered":"2024-07-04T17:33:10.126377+0000","last_clean":"2024-07-04T17:33:10.126377+0000","last_became_active":"2024-07-04T17:33:09.118791+0000","last_became_peered":"2024-07-04T17:33:09.118791+0000","last_unstale":"2024-07-04T17:33:10.126377+0000","last_undegraded":"2024-07-04T17:33:10.126377+0000","last_fullsized":"2024-07-04T17:33:10.126377+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:08:30.473430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.628486+0000","last_change":"2024-07-04T17:33:09.111491+0000","last_active":"2024-07-04T17:33:10.628486+0000","last_peered":"2024-07-04T17:33:10.628486+0000","last_clean":"2024-07-04T17:33:10.628486+0000","last_became_active":"2024-07-04T17:33:09.111095+0000","last_became_peered":"2024-07-04T17:33:09.111095+0000","last_unstale":"2024-07-04T17:33:10.628486+0000","last_undegraded":"2024-07-04T17:33:10.628486+0000","last_fullsized":"2024-07-04T17:33:10.628486+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:49:13.666901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.088629+0000","last_change":"2024-07-04T17:33:09.101446+0000","last_active":"2024-07-04T17:33:10.088629+0000","last_peered":"2024-07-04T17:33:10.088629+0000","last_clean":"2024-07-04T17:33:10.088629+0000","last_became_active":"2024-07-04T17:33:09.100644+0000","last_became_peered":"2024-07-04T17:33:09.100644+0000","last_unstale":"2024-07-04T17:33:10.088629+0000","last_undegraded":"2024-07-04T17:33:10.088629+0000","last_fullsized":"2024-07-04T17:33:10.088629+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:27:36.573799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.091539+0000","last_change":"2024-07-04T17:33:09.124916+0000","last_active":"2024-07-04T17:33:10.091539+0000","last_peered":"2024-07-04T17:33:10.091539+0000","last_clean":"2024-07-04T17:33:10.091539+0000","last_became_active":"2024-07-04T17:33:09.124557+0000","last_became_peered":"2024-07-04T17:33:09.124557+0000","last_unstale":"2024-07-04T17:33:10.091539+0000","last_undegraded":"2024-07-04T17:33:10.091539+0000","last_fullsized":"2024-07-04T17:33:10.091539+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:10:26.737295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.630661+0000","last_change":"2024-07-04T17:33:09.115371+0000","last_active":"2024-07-04T17:33:10.630661+0000","last_peered":"2024-07-04T17:33:10.630661+0000","last_clean":"2024-07-04T17:33:10.630661+0000","last_became_active":"2024-07-04T17:33:09.114560+0000","last_became_peered":"2024-07-04T17:33:09.114560+0000","last_unstale":"2024-07-04T17:33:10.630661+0000","last_undegraded":"2024-07-04T17:33:10.630661+0000","last_fullsized":"2024-07-04T17:33:10.630661+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:39:29.413889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.631021+0000","last_change":"2024-07-04T17:33:09.113151+0000","last_active":"2024-07-04T17:33:10.631021+0000","last_peered":"2024-07-04T17:33:10.631021+0000","last_clean":"2024-07-04T17:33:10.631021+0000","last_became_active":"2024-07-04T17:33:09.112228+0000","last_became_peered":"2024-07-04T17:33:09.112228+0000","last_unstale":"2024-07-04T17:33:10.631021+0000","last_undegraded":"2024-07-04T17:33:10.631021+0000","last_fullsized":"2024-07-04T17:33:10.631021+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:16:18.941587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.089375+0000","last_change":"2024-07-04T17:33:09.118128+0000","last_active":"2024-07-04T17:33:10.089375+0000","last_peered":"2024-07-04T17:33:10.089375+0000","last_clean":"2024-07-04T17:33:10.089375+0000","last_became_active":"2024-07-04T17:33:09.117473+0000","last_became_peered":"2024-07-04T17:33:09.117473+0000","last_unstale":"2024-07-04T17:33:10.089375+0000","last_undegraded":"2024-07-04T17:33:10.089375+0000","last_fullsized":"2024-07-04T17:33:10.089375+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:06:51.425997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.089170+0000","last_change":"2024-07-04T17:33:09.127162+0000","last_active":"2024-07-04T17:33:10.089170+0000","last_peered":"2024-07-04T17:33:10.089170+0000","last_clean":"2024-07-04T17:33:10.089170+0000","last_became_active":"2024-07-04T17:33:09.123389+0000","last_became_peered":"2024-07-04T17:33:09.123389+0000","last_unstale":"2024-07-04T17:33:10.089170+0000","last_undegraded":"2024-07-04T17:33:10.089170+0000","last_fullsized":"2024-07-04T17:33:10.089170+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:33:17.955726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.126685+0000","last_change":"2024-07-04T17:33:09.118567+0000","last_active":"2024-07-04T17:33:10.126685+0000","last_peered":"2024-07-04T17:33:10.126685+0000","last_clean":"2024-07-04T17:33:10.126685+0000","last_became_active":"2024-07-04T17:33:09.117227+0000","last_became_peered":"2024-07-04T17:33:09.117227+0000","last_unstale":"2024-07-04T17:33:10.126685+0000","last_undegraded":"2024-07-04T17:33:10.126685+0000","last_fullsized":"2024-07-04T17:33:10.126685+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:42:43.320358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.089134+0000","last_change":"2024-07-04T17:33:09.131948+0000","last_active":"2024-07-04T17:33:10.089134+0000","last_peered":"2024-07-04T17:33:10.089134+0000","last_clean":"2024-07-04T17:33:10.089134+0000","last_became_active":"2024-07-04T17:33:09.121717+0000","last_became_peered":"2024-07-04T17:33:09.121717+0000","last_unstale":"2024-07-04T17:33:10.089134+0000","last_undegraded":"2024-07-04T17:33:10.089134+0000","last_fullsized":"2024-07-04T17:33:10.089134+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:05:18.766887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.628993+0000","last_change":"2024-07-04T17:33:09.124489+0000","last_active":"2024-07-04T17:33:10.628993+0000","last_peered":"2024-07-04T17:33:10.628993+0000","last_clean":"2024-07-04T17:33:10.628993+0000","last_became_active":"2024-07-04T17:33:09.122889+0000","last_became_peered":"2024-07-04T17:33:09.122889+0000","last_unstale":"2024-07-04T17:33:10.628993+0000","last_undegraded":"2024-07-04T17:33:10.628993+0000","last_fullsized":"2024-07-04T17:33:10.628993+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:01:35.821887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.091644+0000","last_change":"2024-07-04T17:33:09.118397+0000","last_active":"2024-07-04T17:33:10.091644+0000","last_peered":"2024-07-04T17:33:10.091644+0000","last_clean":"2024-07-04T17:33:10.091644+0000","last_became_active":"2024-07-04T17:33:09.117509+0000","last_became_peered":"2024-07-04T17:33:09.117509+0000","last_unstale":"2024-07-04T17:33:10.091644+0000","last_undegraded":"2024-07-04T17:33:10.091644+0000","last_fullsized":"2024-07-04T17:33:10.091644+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:37:23.202824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.630761+0000","last_change":"2024-07-04T17:33:09.115358+0000","last_active":"2024-07-04T17:33:10.630761+0000","last_peered":"2024-07-04T17:33:10.630761+0000","last_clean":"2024-07-04T17:33:10.630761+0000","last_became_active":"2024-07-04T17:33:09.113650+0000","last_became_peered":"2024-07-04T17:33:09.113650+0000","last_unstale":"2024-07-04T17:33:10.630761+0000","last_undegraded":"2024-07-04T17:33:10.630761+0000","last_fullsized":"2024-07-04T17:33:10.630761+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:34:20.388421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.090827+0000","last_change":"2024-07-04T17:33:09.130921+0000","last_active":"2024-07-04T17:33:10.090827+0000","last_peered":"2024-07-04T17:33:10.090827+0000","last_clean":"2024-07-04T17:33:10.090827+0000","last_became_active":"2024-07-04T17:33:09.123521+0000","last_became_peered":"2024-07-04T17:33:09.123521+0000","last_unstale":"2024-07-04T17:33:10.090827+0000","last_undegraded":"2024-07-04T17:33:10.090827+0000","last_fullsized":"2024-07-04T17:33:10.090827+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:13:51.565345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.127454+0000","last_change":"2024-07-04T17:33:09.114604+0000","last_active":"2024-07-04T17:33:10.127454+0000","last_peered":"2024-07-04T17:33:10.127454+0000","last_clean":"2024-07-04T17:33:10.127454+0000","last_became_active":"2024-07-04T17:33:09.114294+0000","last_became_peered":"2024-07-04T17:33:09.114294+0000","last_unstale":"2024-07-04T17:33:10.127454+0000","last_undegraded":"2024-07-04T17:33:10.127454+0000","last_fullsized":"2024-07-04T17:33:10.127454+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:34:50.754812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.629965+0000","last_change":"2024-07-04T17:33:09.124878+0000","last_active":"2024-07-04T17:33:10.629965+0000","last_peered":"2024-07-04T17:33:10.629965+0000","last_clean":"2024-07-04T17:33:10.629965+0000","last_became_active":"2024-07-04T17:33:09.124119+0000","last_became_peered":"2024-07-04T17:33:09.124119+0000","last_unstale":"2024-07-04T17:33:10.629965+0000","last_undegraded":"2024-07-04T17:33:10.629965+0000","last_fullsized":"2024-07-04T17:33:10.629965+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:18:22.081730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.126587+0000","last_change":"2024-07-04T17:33:09.104697+0000","last_active":"2024-07-04T17:33:10.126587+0000","last_peered":"2024-07-04T17:33:10.126587+0000","last_clean":"2024-07-04T17:33:10.126587+0000","last_became_active":"2024-07-04T17:33:09.103509+0000","last_became_peered":"2024-07-04T17:33:09.103509+0000","last_unstale":"2024-07-04T17:33:10.126587+0000","last_undegraded":"2024-07-04T17:33:10.126587+0000","last_fullsized":"2024-07-04T17:33:10.126587+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:05:17.882764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.126381+0000","last_change":"2024-07-04T17:33:09.104054+0000","last_active":"2024-07-04T17:33:10.126381+0000","last_peered":"2024-07-04T17:33:10.126381+0000","last_clean":"2024-07-04T17:33:10.126381+0000","last_became_active":"2024-07-04T17:33:09.103713+0000","last_became_peered":"2024-07-04T17:33:09.103713+0000","last_unstale":"2024-07-04T17:33:10.126381+0000","last_undegraded":"2024-07-04T17:33:10.126381+0000","last_fullsized":"2024-07-04T17:33:10.126381+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:27:24.257586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.088710+0000","last_change":"2024-07-04T17:33:09.122343+0000","last_active":"2024-07-04T17:33:10.088710+0000","last_peered":"2024-07-04T17:33:10.088710+0000","last_clean":"2024-07-04T17:33:10.088710+0000","last_became_active":"2024-07-04T17:33:09.121982+0000","last_became_peered":"2024-07-04T17:33:09.121982+0000","last_unstale":"2024-07-04T17:33:10.088710+0000","last_undegraded":"2024-07-04T17:33:10.088710+0000","last_fullsized":"2024-07-04T17:33:10.088710+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:55:41.686107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.631147+0000","last_change":"2024-07-04T17:33:09.115618+0000","last_active":"2024-07-04T17:33:10.631147+0000","last_peered":"2024-07-04T17:33:10.631147+0000","last_clean":"2024-07-04T17:33:10.631147+0000","last_became_active":"2024-07-04T17:33:09.115047+0000","last_became_peered":"2024-07-04T17:33:09.115047+0000","last_unstale":"2024-07-04T17:33:10.631147+0000","last_undegraded":"2024-07-04T17:33:10.631147+0000","last_fullsized":"2024-07-04T17:33:10.631147+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:38:58.383535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"113'4","reported_seq":18,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.632103+0000","last_change":"2024-07-04T17:33:08.117620+0000","last_active":"2024-07-04T17:33:10.632103+0000","last_peered":"2024-07-04T17:33:10.632103+0000","last_clean":"2024-07-04T17:33:10.632103+0000","last_became_active":"2024-07-04T17:33:08.117097+0000","last_became_peered":"2024-07-04T17:33:08.117097+0000","last_unstale":"2024-07-04T17:33:10.632103+0000","last_undegraded":"2024-07-04T17:33:10.632103+0000","last_fullsized":"2024-07-04T17:33:10.632103+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:00:27.380504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.092013+0000","last_change":"2024-07-04T17:33:09.133070+0000","last_active":"2024-07-04T17:33:10.092013+0000","last_peered":"2024-07-04T17:33:10.092013+0000","last_clean":"2024-07-04T17:33:10.092013+0000","last_became_active":"2024-07-04T17:33:09.132634+0000","last_became_peered":"2024-07-04T17:33:09.132634+0000","last_unstale":"2024-07-04T17:33:10.092013+0000","last_undegraded":"2024-07-04T17:33:10.092013+0000","last_fullsized":"2024-07-04T17:33:10.092013+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:42:13.182154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1e","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.127169+0000","last_change":"2024-07-04T17:33:08.105107+0000","last_active":"2024-07-04T17:33:10.127169+0000","last_peered":"2024-07-04T17:33:10.127169+0000","last_clean":"2024-07-04T17:33:10.127169+0000","last_became_active":"2024-07-04T17:33:08.104670+0000","last_became_peered":"2024-07-04T17:33:08.104670+0000","last_unstale":"2024-07-04T17:33:10.127169+0000","last_undegraded":"2024-07-04T17:33:10.127169+0000","last_fullsized":"2024-07-04T17:33:10.127169+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:46:04.479168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.630165+0000","last_change":"2024-07-04T17:33:09.107265+0000","last_active":"2024-07-04T17:33:10.630165+0000","last_peered":"2024-07-04T17:33:10.630165+0000","last_clean":"2024-07-04T17:33:10.630165+0000","last_became_active":"2024-07-04T17:33:09.106633+0000","last_became_peered":"2024-07-04T17:33:09.106633+0000","last_unstale":"2024-07-04T17:33:10.630165+0000","last_undegraded":"2024-07-04T17:33:10.630165+0000","last_fullsized":"2024-07-04T17:33:10.630165+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:05:30.327143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.091565+0000","last_change":"2024-07-04T17:33:08.089824+0000","last_active":"2024-07-04T17:33:10.091565+0000","last_peered":"2024-07-04T17:33:10.091565+0000","last_clean":"2024-07-04T17:33:10.091565+0000","last_became_active":"2024-07-04T17:33:08.084113+0000","last_became_peered":"2024-07-04T17:33:08.084113+0000","last_unstale":"2024-07-04T17:33:10.091565+0000","last_undegraded":"2024-07-04T17:33:10.091565+0000","last_fullsized":"2024-07-04T17:33:10.091565+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:39:25.915226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.091057+0000","last_change":"2024-07-04T17:33:09.129582+0000","last_active":"2024-07-04T17:33:10.091057+0000","last_peered":"2024-07-04T17:33:10.091057+0000","last_clean":"2024-07-04T17:33:10.091057+0000","last_became_active":"2024-07-04T17:33:09.127839+0000","last_became_peered":"2024-07-04T17:33:09.127839+0000","last_unstale":"2024-07-04T17:33:10.091057+0000","last_undegraded":"2024-07-04T17:33:10.091057+0000","last_fullsized":"2024-07-04T17:33:10.091057+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:35:13.414729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.090676+0000","last_change":"2024-07-04T17:33:08.108862+0000","last_active":"2024-07-04T17:33:10.090676+0000","last_peered":"2024-07-04T17:33:10.090676+0000","last_clean":"2024-07-04T17:33:10.090676+0000","last_became_active":"2024-07-04T17:33:08.107761+0000","last_became_peered":"2024-07-04T17:33:08.107761+0000","last_unstale":"2024-07-04T17:33:10.090676+0000","last_undegraded":"2024-07-04T17:33:10.090676+0000","last_fullsized":"2024-07-04T17:33:10.090676+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:15:53.250590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.126769+0000","last_change":"2024-07-04T17:33:09.118310+0000","last_active":"2024-07-04T17:33:10.126769+0000","last_peered":"2024-07-04T17:33:10.126769+0000","last_clean":"2024-07-04T17:33:10.126769+0000","last_became_active":"2024-07-04T17:33:09.116852+0000","last_became_peered":"2024-07-04T17:33:09.116852+0000","last_unstale":"2024-07-04T17:33:10.126769+0000","last_undegraded":"2024-07-04T17:33:10.126769+0000","last_fullsized":"2024-07-04T17:33:10.126769+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:41:21.265187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.1b","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.091201+0000","last_change":"2024-07-04T17:33:08.105642+0000","last_active":"2024-07-04T17:33:10.091201+0000","last_peered":"2024-07-04T17:33:10.091201+0000","last_clean":"2024-07-04T17:33:10.091201+0000","last_became_active":"2024-07-04T17:33:08.105297+0000","last_became_peered":"2024-07-04T17:33:08.105297+0000","last_unstale":"2024-07-04T17:33:10.091201+0000","last_undegraded":"2024-07-04T17:33:10.091201+0000","last_fullsized":"2024-07-04T17:33:10.091201+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:50:34.893649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.089384+0000","last_change":"2024-07-04T17:33:09.116075+0000","last_active":"2024-07-04T17:33:10.089384+0000","last_peered":"2024-07-04T17:33:10.089384+0000","last_clean":"2024-07-04T17:33:10.089384+0000","last_became_active":"2024-07-04T17:33:09.115705+0000","last_became_peered":"2024-07-04T17:33:09.115705+0000","last_unstale":"2024-07-04T17:33:10.089384+0000","last_undegraded":"2024-07-04T17:33:10.089384+0000","last_fullsized":"2024-07-04T17:33:10.089384+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:52:50.422683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.629511+0000","last_change":"2024-07-04T17:33:08.116547+0000","last_active":"2024-07-04T17:33:10.629511+0000","last_peered":"2024-07-04T17:33:10.629511+0000","last_clean":"2024-07-04T17:33:10.629511+0000","last_became_active":"2024-07-04T17:33:08.116095+0000","last_became_peered":"2024-07-04T17:33:08.116095+0000","last_unstale":"2024-07-04T17:33:10.629511+0000","last_undegraded":"2024-07-04T17:33:10.629511+0000","last_fullsized":"2024-07-04T17:33:10.629511+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:31:44.812381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.629249+0000","last_change":"2024-07-04T17:33:09.125116+0000","last_active":"2024-07-04T17:33:10.629249+0000","last_peered":"2024-07-04T17:33:10.629249+0000","last_clean":"2024-07-04T17:33:10.629249+0000","last_became_active":"2024-07-04T17:33:09.124103+0000","last_became_peered":"2024-07-04T17:33:09.124103+0000","last_unstale":"2024-07-04T17:33:10.629249+0000","last_undegraded":"2024-07-04T17:33:10.629249+0000","last_fullsized":"2024-07-04T17:33:10.629249+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:27:31.156139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.090675+0000","last_change":"2024-07-04T17:33:08.096904+0000","last_active":"2024-07-04T17:33:10.090675+0000","last_peered":"2024-07-04T17:33:10.090675+0000","last_clean":"2024-07-04T17:33:10.090675+0000","last_became_active":"2024-07-04T17:33:08.096590+0000","last_became_peered":"2024-07-04T17:33:08.096590+0000","last_unstale":"2024-07-04T17:33:10.090675+0000","last_undegraded":"2024-07-04T17:33:10.090675+0000","last_fullsized":"2024-07-04T17:33:10.090675+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:39:51.529018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.091219+0000","last_change":"2024-07-04T17:33:09.100755+0000","last_active":"2024-07-04T17:33:10.091219+0000","last_peered":"2024-07-04T17:33:10.091219+0000","last_clean":"2024-07-04T17:33:10.091219+0000","last_became_active":"2024-07-04T17:33:09.100389+0000","last_became_peered":"2024-07-04T17:33:09.100389+0000","last_unstale":"2024-07-04T17:33:10.091219+0000","last_undegraded":"2024-07-04T17:33:10.091219+0000","last_fullsized":"2024-07-04T17:33:10.091219+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:38:04.457302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.128545+0000","last_change":"2024-07-04T17:33:08.106647+0000","last_active":"2024-07-04T17:33:10.128545+0000","last_peered":"2024-07-04T17:33:10.128545+0000","last_clean":"2024-07-04T17:33:10.128545+0000","last_became_active":"2024-07-04T17:33:08.106277+0000","last_became_peered":"2024-07-04T17:33:08.106277+0000","last_unstale":"2024-07-04T17:33:10.128545+0000","last_undegraded":"2024-07-04T17:33:10.128545+0000","last_fullsized":"2024-07-04T17:33:10.128545+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:47:04.050710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.090776+0000","last_change":"2024-07-04T17:33:09.128294+0000","last_active":"2024-07-04T17:33:10.090776+0000","last_peered":"2024-07-04T17:33:10.090776+0000","last_clean":"2024-07-04T17:33:10.090776+0000","last_became_active":"2024-07-04T17:33:09.123696+0000","last_became_peered":"2024-07-04T17:33:09.123696+0000","last_unstale":"2024-07-04T17:33:10.090776+0000","last_undegraded":"2024-07-04T17:33:10.090776+0000","last_fullsized":"2024-07-04T17:33:10.090776+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:07:05.456492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.632103+0000","last_change":"2024-07-04T17:33:08.100600+0000","last_active":"2024-07-04T17:33:10.632103+0000","last_peered":"2024-07-04T17:33:10.632103+0000","last_clean":"2024-07-04T17:33:10.632103+0000","last_became_active":"2024-07-04T17:33:08.099352+0000","last_became_peered":"2024-07-04T17:33:08.099352+0000","last_unstale":"2024-07-04T17:33:10.632103+0000","last_undegraded":"2024-07-04T17:33:10.632103+0000","last_fullsized":"2024-07-04T17:33:10.632103+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:04:33.482388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.128669+0000","last_change":"2024-07-04T17:33:09.109950+0000","last_active":"2024-07-04T17:33:10.128669+0000","last_peered":"2024-07-04T17:33:10.128669+0000","last_clean":"2024-07-04T17:33:10.128669+0000","last_became_active":"2024-07-04T17:33:09.108850+0000","last_became_peered":"2024-07-04T17:33:09.108850+0000","last_unstale":"2024-07-04T17:33:10.128669+0000","last_undegraded":"2024-07-04T17:33:10.128669+0000","last_fullsized":"2024-07-04T17:33:10.128669+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:22:58.651803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.631462+0000","last_change":"2024-07-04T17:33:08.117619+0000","last_active":"2024-07-04T17:33:10.631462+0000","last_peered":"2024-07-04T17:33:10.631462+0000","last_clean":"2024-07-04T17:33:10.631462+0000","last_became_active":"2024-07-04T17:33:08.117033+0000","last_became_peered":"2024-07-04T17:33:08.117033+0000","last_unstale":"2024-07-04T17:33:10.631462+0000","last_undegraded":"2024-07-04T17:33:10.631462+0000","last_fullsized":"2024-07-04T17:33:10.631462+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:41:18.737857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.091601+0000","last_change":"2024-07-04T17:33:09.129855+0000","last_active":"2024-07-04T17:33:10.091601+0000","last_peered":"2024-07-04T17:33:10.091601+0000","last_clean":"2024-07-04T17:33:10.091601+0000","last_became_active":"2024-07-04T17:33:09.127858+0000","last_became_peered":"2024-07-04T17:33:09.127858+0000","last_unstale":"2024-07-04T17:33:10.091601+0000","last_undegraded":"2024-07-04T17:33:10.091601+0000","last_fullsized":"2024-07-04T17:33:10.091601+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:35:51.815415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"113'6","reported_seq":20,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.127766+0000","last_change":"2024-07-04T17:33:08.105018+0000","last_active":"2024-07-04T17:33:10.127766+0000","last_peered":"2024-07-04T17:33:10.127766+0000","last_clean":"2024-07-04T17:33:10.127766+0000","last_became_active":"2024-07-04T17:33:08.104685+0000","last_became_peered":"2024-07-04T17:33:08.104685+0000","last_unstale":"2024-07-04T17:33:10.127766+0000","last_undegraded":"2024-07-04T17:33:10.127766+0000","last_fullsized":"2024-07-04T17:33:10.127766+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:13:39.096736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.089655+0000","last_change":"2024-07-04T17:33:09.122352+0000","last_active":"2024-07-04T17:33:10.089655+0000","last_peered":"2024-07-04T17:33:10.089655+0000","last_clean":"2024-07-04T17:33:10.089655+0000","last_became_active":"2024-07-04T17:33:09.121662+0000","last_became_peered":"2024-07-04T17:33:09.121662+0000","last_unstale":"2024-07-04T17:33:10.089655+0000","last_undegraded":"2024-07-04T17:33:10.089655+0000","last_fullsized":"2024-07-04T17:33:10.089655+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:38:27.893548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.630328+0000","last_change":"2024-07-04T17:33:08.100243+0000","last_active":"2024-07-04T17:33:10.630328+0000","last_peered":"2024-07-04T17:33:10.630328+0000","last_clean":"2024-07-04T17:33:10.630328+0000","last_became_active":"2024-07-04T17:33:08.099860+0000","last_became_peered":"2024-07-04T17:33:08.099860+0000","last_unstale":"2024-07-04T17:33:10.630328+0000","last_undegraded":"2024-07-04T17:33:10.630328+0000","last_fullsized":"2024-07-04T17:33:10.630328+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:44:16.021572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.629956+0000","last_change":"2024-07-04T17:33:09.124609+0000","last_active":"2024-07-04T17:33:10.629956+0000","last_peered":"2024-07-04T17:33:10.629956+0000","last_clean":"2024-07-04T17:33:10.629956+0000","last_became_active":"2024-07-04T17:33:09.123720+0000","last_became_peered":"2024-07-04T17:33:09.123720+0000","last_unstale":"2024-07-04T17:33:10.629956+0000","last_undegraded":"2024-07-04T17:33:10.629956+0000","last_fullsized":"2024-07-04T17:33:10.629956+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:59:00.559158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.127073+0000","last_change":"2024-07-04T17:33:08.092450+0000","last_active":"2024-07-04T17:33:10.127073+0000","last_peered":"2024-07-04T17:33:10.127073+0000","last_clean":"2024-07-04T17:33:10.127073+0000","last_became_active":"2024-07-04T17:33:08.092088+0000","last_became_peered":"2024-07-04T17:33:08.092088+0000","last_unstale":"2024-07-04T17:33:10.127073+0000","last_undegraded":"2024-07-04T17:33:10.127073+0000","last_fullsized":"2024-07-04T17:33:10.127073+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:54:46.681088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.089354+0000","last_change":"2024-07-04T17:33:09.101721+0000","last_active":"2024-07-04T17:33:10.089354+0000","last_peered":"2024-07-04T17:33:10.089354+0000","last_clean":"2024-07-04T17:33:10.089354+0000","last_became_active":"2024-07-04T17:33:09.101084+0000","last_became_peered":"2024-07-04T17:33:09.101084+0000","last_unstale":"2024-07-04T17:33:10.089354+0000","last_undegraded":"2024-07-04T17:33:10.089354+0000","last_fullsized":"2024-07-04T17:33:10.089354+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:35:19.200734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.628612+0000","last_change":"2024-07-04T17:33:08.098833+0000","last_active":"2024-07-04T17:33:10.628612+0000","last_peered":"2024-07-04T17:33:10.628612+0000","last_clean":"2024-07-04T17:33:10.628612+0000","last_became_active":"2024-07-04T17:33:08.098492+0000","last_became_peered":"2024-07-04T17:33:08.098492+0000","last_unstale":"2024-07-04T17:33:10.628612+0000","last_undegraded":"2024-07-04T17:33:10.628612+0000","last_fullsized":"2024-07-04T17:33:10.628612+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:40:56.816879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.090378+0000","last_change":"2024-07-04T17:33:09.131477+0000","last_active":"2024-07-04T17:33:10.090378+0000","last_peered":"2024-07-04T17:33:10.090378+0000","last_clean":"2024-07-04T17:33:10.090378+0000","last_became_active":"2024-07-04T17:33:09.122889+0000","last_became_peered":"2024-07-04T17:33:09.122889+0000","last_unstale":"2024-07-04T17:33:10.090378+0000","last_undegraded":"2024-07-04T17:33:10.090378+0000","last_fullsized":"2024-07-04T17:33:10.090378+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:48:40.827656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"113'8","reported_seq":22,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.090899+0000","last_change":"2024-07-04T17:33:08.094956+0000","last_active":"2024-07-04T17:33:10.090899+0000","last_peered":"2024-07-04T17:33:10.090899+0000","last_clean":"2024-07-04T17:33:10.090899+0000","last_became_active":"2024-07-04T17:33:08.090486+0000","last_became_peered":"2024-07-04T17:33:08.090486+0000","last_unstale":"2024-07-04T17:33:10.090899+0000","last_undegraded":"2024-07-04T17:33:10.090899+0000","last_fullsized":"2024-07-04T17:33:10.090899+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:05:13.908213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.631246+0000","last_change":"2024-07-04T17:33:09.128035+0000","last_active":"2024-07-04T17:33:10.631246+0000","last_peered":"2024-07-04T17:33:10.631246+0000","last_clean":"2024-07-04T17:33:10.631246+0000","last_became_active":"2024-07-04T17:33:09.125335+0000","last_became_peered":"2024-07-04T17:33:09.125335+0000","last_unstale":"2024-07-04T17:33:10.631246+0000","last_undegraded":"2024-07-04T17:33:10.631246+0000","last_fullsized":"2024-07-04T17:33:10.631246+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:35:53.125670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.092351+0000","last_change":"2024-07-04T17:33:08.096006+0000","last_active":"2024-07-04T17:33:10.092351+0000","last_peered":"2024-07-04T17:33:10.092351+0000","last_clean":"2024-07-04T17:33:10.092351+0000","last_became_active":"2024-07-04T17:33:08.095685+0000","last_became_peered":"2024-07-04T17:33:08.095685+0000","last_unstale":"2024-07-04T17:33:10.092351+0000","last_undegraded":"2024-07-04T17:33:10.092351+0000","last_fullsized":"2024-07-04T17:33:10.092351+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:58:57.013662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.092079+0000","last_change":"2024-07-04T17:33:09.118421+0000","last_active":"2024-07-04T17:33:10.092079+0000","last_peered":"2024-07-04T17:33:10.092079+0000","last_clean":"2024-07-04T17:33:10.092079+0000","last_became_active":"2024-07-04T17:33:09.117653+0000","last_became_peered":"2024-07-04T17:33:09.117653+0000","last_unstale":"2024-07-04T17:33:10.092079+0000","last_undegraded":"2024-07-04T17:33:10.092079+0000","last_fullsized":"2024-07-04T17:33:10.092079+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:59:43.093156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.631515+0000","last_change":"2024-07-04T17:33:08.113131+0000","last_active":"2024-07-04T17:33:10.631515+0000","last_peered":"2024-07-04T17:33:10.631515+0000","last_clean":"2024-07-04T17:33:10.631515+0000","last_became_active":"2024-07-04T17:33:08.112337+0000","last_became_peered":"2024-07-04T17:33:08.112337+0000","last_unstale":"2024-07-04T17:33:10.631515+0000","last_undegraded":"2024-07-04T17:33:10.631515+0000","last_fullsized":"2024-07-04T17:33:10.631515+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:45:29.198114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.630701+0000","last_change":"2024-07-04T17:33:09.107000+0000","last_active":"2024-07-04T17:33:10.630701+0000","last_peered":"2024-07-04T17:33:10.630701+0000","last_clean":"2024-07-04T17:33:10.630701+0000","last_became_active":"2024-07-04T17:33:09.106204+0000","last_became_peered":"2024-07-04T17:33:09.106204+0000","last_unstale":"2024-07-04T17:33:10.630701+0000","last_undegraded":"2024-07-04T17:33:10.630701+0000","last_fullsized":"2024-07-04T17:33:10.630701+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:22:24.403011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"113'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.091704+0000","last_change":"2024-07-04T17:33:08.105392+0000","last_active":"2024-07-04T17:33:10.091704+0000","last_peered":"2024-07-04T17:33:10.091704+0000","last_clean":"2024-07-04T17:33:10.091704+0000","last_became_active":"2024-07-04T17:33:08.105067+0000","last_became_peered":"2024-07-04T17:33:08.105067+0000","last_unstale":"2024-07-04T17:33:10.091704+0000","last_undegraded":"2024-07-04T17:33:10.091704+0000","last_fullsized":"2024-07-04T17:33:10.091704+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:10:49.236238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.631540+0000","last_change":"2024-07-04T17:33:09.126710+0000","last_active":"2024-07-04T17:33:10.631540+0000","last_peered":"2024-07-04T17:33:10.631540+0000","last_clean":"2024-07-04T17:33:10.631540+0000","last_became_active":"2024-07-04T17:33:09.122975+0000","last_became_peered":"2024-07-04T17:33:09.122975+0000","last_unstale":"2024-07-04T17:33:10.631540+0000","last_undegraded":"2024-07-04T17:33:10.631540+0000","last_fullsized":"2024-07-04T17:33:10.631540+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:48:57.200238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.631877+0000","last_change":"2024-07-04T17:33:08.114916+0000","last_active":"2024-07-04T17:33:10.631877+0000","last_peered":"2024-07-04T17:33:10.631877+0000","last_clean":"2024-07-04T17:33:10.631877+0000","last_became_active":"2024-07-04T17:33:08.114455+0000","last_became_peered":"2024-07-04T17:33:08.114455+0000","last_unstale":"2024-07-04T17:33:10.631877+0000","last_undegraded":"2024-07-04T17:33:10.631877+0000","last_fullsized":"2024-07-04T17:33:10.631877+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:09:08.764721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.126957+0000","last_change":"2024-07-04T17:33:09.105710+0000","last_active":"2024-07-04T17:33:10.126957+0000","last_peered":"2024-07-04T17:33:10.126957+0000","last_clean":"2024-07-04T17:33:10.126957+0000","last_became_active":"2024-07-04T17:33:09.101819+0000","last_became_peered":"2024-07-04T17:33:09.101819+0000","last_unstale":"2024-07-04T17:33:10.126957+0000","last_undegraded":"2024-07-04T17:33:10.126957+0000","last_fullsized":"2024-07-04T17:33:10.126957+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:47:15.892916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.129007+0000","last_change":"2024-07-04T17:33:08.108404+0000","last_active":"2024-07-04T17:33:10.129007+0000","last_peered":"2024-07-04T17:33:10.129007+0000","last_clean":"2024-07-04T17:33:10.129007+0000","last_became_active":"2024-07-04T17:33:08.107387+0000","last_became_peered":"2024-07-04T17:33:08.107387+0000","last_unstale":"2024-07-04T17:33:10.129007+0000","last_undegraded":"2024-07-04T17:33:10.129007+0000","last_fullsized":"2024-07-04T17:33:10.129007+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:10:32.480743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.128952+0000","last_change":"2024-07-04T17:33:09.102491+0000","last_active":"2024-07-04T17:33:10.128952+0000","last_peered":"2024-07-04T17:33:10.128952+0000","last_clean":"2024-07-04T17:33:10.128952+0000","last_became_active":"2024-07-04T17:33:09.102170+0000","last_became_peered":"2024-07-04T17:33:09.102170+0000","last_unstale":"2024-07-04T17:33:10.128952+0000","last_undegraded":"2024-07-04T17:33:10.128952+0000","last_fullsized":"2024-07-04T17:33:10.128952+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:03:31.387343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.090123+0000","last_change":"2024-07-04T17:33:08.109294+0000","last_active":"2024-07-04T17:33:10.090123+0000","last_peered":"2024-07-04T17:33:10.090123+0000","last_clean":"2024-07-04T17:33:10.090123+0000","last_became_active":"2024-07-04T17:33:08.108944+0000","last_became_peered":"2024-07-04T17:33:08.108944+0000","last_unstale":"2024-07-04T17:33:10.090123+0000","last_undegraded":"2024-07-04T17:33:10.090123+0000","last_fullsized":"2024-07-04T17:33:10.090123+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:15:36.989862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.126703+0000","last_change":"2024-07-04T17:33:09.111629+0000","last_active":"2024-07-04T17:33:10.126703+0000","last_peered":"2024-07-04T17:33:10.126703+0000","last_clean":"2024-07-04T17:33:10.126703+0000","last_became_active":"2024-07-04T17:33:09.111332+0000","last_became_peered":"2024-07-04T17:33:09.111332+0000","last_unstale":"2024-07-04T17:33:10.126703+0000","last_undegraded":"2024-07-04T17:33:10.126703+0000","last_fullsized":"2024-07-04T17:33:10.126703+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:44:43.830281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.090536+0000","last_change":"2024-07-04T17:33:08.097164+0000","last_active":"2024-07-04T17:33:10.090536+0000","last_peered":"2024-07-04T17:33:10.090536+0000","last_clean":"2024-07-04T17:33:10.090536+0000","last_became_active":"2024-07-04T17:33:08.096841+0000","last_became_peered":"2024-07-04T17:33:08.096841+0000","last_unstale":"2024-07-04T17:33:10.090536+0000","last_undegraded":"2024-07-04T17:33:10.090536+0000","last_fullsized":"2024-07-04T17:33:10.090536+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:24:50.491277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.126703+0000","last_change":"2024-07-04T17:33:09.097586+0000","last_active":"2024-07-04T17:33:10.126703+0000","last_peered":"2024-07-04T17:33:10.126703+0000","last_clean":"2024-07-04T17:33:10.126703+0000","last_became_active":"2024-07-04T17:33:09.096586+0000","last_became_peered":"2024-07-04T17:33:09.096586+0000","last_unstale":"2024-07-04T17:33:10.126703+0000","last_undegraded":"2024-07-04T17:33:10.126703+0000","last_fullsized":"2024-07-04T17:33:10.126703+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:21:53.318774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.091046+0000","last_change":"2024-07-04T17:33:08.094584+0000","last_active":"2024-07-04T17:33:10.091046+0000","last_peered":"2024-07-04T17:33:10.091046+0000","last_clean":"2024-07-04T17:33:10.091046+0000","last_became_active":"2024-07-04T17:33:08.090067+0000","last_became_peered":"2024-07-04T17:33:08.090067+0000","last_unstale":"2024-07-04T17:33:10.091046+0000","last_undegraded":"2024-07-04T17:33:10.091046+0000","last_fullsized":"2024-07-04T17:33:10.091046+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:17:57.214638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.091398+0000","last_change":"2024-07-04T17:33:09.131493+0000","last_active":"2024-07-04T17:33:10.091398+0000","last_peered":"2024-07-04T17:33:10.091398+0000","last_clean":"2024-07-04T17:33:10.091398+0000","last_became_active":"2024-07-04T17:33:09.131132+0000","last_became_peered":"2024-07-04T17:33:09.131132+0000","last_unstale":"2024-07-04T17:33:10.091398+0000","last_undegraded":"2024-07-04T17:33:10.091398+0000","last_fullsized":"2024-07-04T17:33:10.091398+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:56:22.629300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"113'2","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.092649+0000","last_change":"2024-07-04T17:33:08.105850+0000","last_active":"2024-07-04T17:33:10.092649+0000","last_peered":"2024-07-04T17:33:10.092649+0000","last_clean":"2024-07-04T17:33:10.092649+0000","last_became_active":"2024-07-04T17:33:08.104987+0000","last_became_peered":"2024-07-04T17:33:08.104987+0000","last_unstale":"2024-07-04T17:33:10.092649+0000","last_undegraded":"2024-07-04T17:33:10.092649+0000","last_fullsized":"2024-07-04T17:33:10.092649+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:36:17.000989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.092449+0000","last_change":"2024-07-04T17:33:09.118693+0000","last_active":"2024-07-04T17:33:10.092449+0000","last_peered":"2024-07-04T17:33:10.092449+0000","last_clean":"2024-07-04T17:33:10.092449+0000","last_became_active":"2024-07-04T17:33:09.117928+0000","last_became_peered":"2024-07-04T17:33:09.117928+0000","last_unstale":"2024-07-04T17:33:10.092449+0000","last_undegraded":"2024-07-04T17:33:10.092449+0000","last_fullsized":"2024-07-04T17:33:10.092449+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:38:35.544115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"114'1","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.631708+0000","last_change":"2024-07-04T17:33:08.116106+0000","last_active":"2024-07-04T17:33:10.631708+0000","last_peered":"2024-07-04T17:33:10.631708+0000","last_clean":"2024-07-04T17:33:10.631708+0000","last_became_active":"2024-07-04T17:33:08.115757+0000","last_became_peered":"2024-07-04T17:33:08.115757+0000","last_unstale":"2024-07-04T17:33:10.631708+0000","last_undegraded":"2024-07-04T17:33:10.631708+0000","last_fullsized":"2024-07-04T17:33:10.631708+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:16:30.420529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.631205+0000","last_change":"2024-07-04T17:33:09.098889+0000","last_active":"2024-07-04T17:33:10.631205+0000","last_peered":"2024-07-04T17:33:10.631205+0000","last_clean":"2024-07-04T17:33:10.631205+0000","last_became_active":"2024-07-04T17:33:09.098564+0000","last_became_peered":"2024-07-04T17:33:09.098564+0000","last_unstale":"2024-07-04T17:33:10.631205+0000","last_undegraded":"2024-07-04T17:33:10.631205+0000","last_fullsized":"2024-07-04T17:33:10.631205+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:06:27.253525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.632086+0000","last_change":"2024-07-04T17:33:08.114022+0000","last_active":"2024-07-04T17:33:10.632086+0000","last_peered":"2024-07-04T17:33:10.632086+0000","last_clean":"2024-07-04T17:33:10.632086+0000","last_became_active":"2024-07-04T17:33:08.112033+0000","last_became_peered":"2024-07-04T17:33:08.112033+0000","last_unstale":"2024-07-04T17:33:10.632086+0000","last_undegraded":"2024-07-04T17:33:10.632086+0000","last_fullsized":"2024-07-04T17:33:10.632086+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:00:33.769816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.631621+0000","last_change":"2024-07-04T17:33:09.113463+0000","last_active":"2024-07-04T17:33:10.631621+0000","last_peered":"2024-07-04T17:33:10.631621+0000","last_clean":"2024-07-04T17:33:10.631621+0000","last_became_active":"2024-07-04T17:33:09.112826+0000","last_became_peered":"2024-07-04T17:33:09.112826+0000","last_unstale":"2024-07-04T17:33:10.631621+0000","last_undegraded":"2024-07-04T17:33:10.631621+0000","last_fullsized":"2024-07-04T17:33:10.631621+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:26:59.083650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.631999+0000","last_change":"2024-07-04T17:33:08.114768+0000","last_active":"2024-07-04T17:33:10.631999+0000","last_peered":"2024-07-04T17:33:10.631999+0000","last_clean":"2024-07-04T17:33:10.631999+0000","last_became_active":"2024-07-04T17:33:08.113902+0000","last_became_peered":"2024-07-04T17:33:08.113902+0000","last_unstale":"2024-07-04T17:33:10.631999+0000","last_undegraded":"2024-07-04T17:33:10.631999+0000","last_fullsized":"2024-07-04T17:33:10.631999+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:58:34.807900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.089969+0000","last_change":"2024-07-04T17:33:09.128723+0000","last_active":"2024-07-04T17:33:10.089969+0000","last_peered":"2024-07-04T17:33:10.089969+0000","last_clean":"2024-07-04T17:33:10.089969+0000","last_became_active":"2024-07-04T17:33:09.128217+0000","last_became_peered":"2024-07-04T17:33:09.128217+0000","last_unstale":"2024-07-04T17:33:10.089969+0000","last_undegraded":"2024-07-04T17:33:10.089969+0000","last_fullsized":"2024-07-04T17:33:10.089969+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:08:28.724747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.129017+0000","last_change":"2024-07-04T17:33:08.108382+0000","last_active":"2024-07-04T17:33:10.129017+0000","last_peered":"2024-07-04T17:33:10.129017+0000","last_clean":"2024-07-04T17:33:10.129017+0000","last_became_active":"2024-07-04T17:33:08.107656+0000","last_became_peered":"2024-07-04T17:33:08.107656+0000","last_unstale":"2024-07-04T17:33:10.129017+0000","last_undegraded":"2024-07-04T17:33:10.129017+0000","last_fullsized":"2024-07-04T17:33:10.129017+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:55:46.289096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.091064+0000","last_change":"2024-07-04T17:33:09.129070+0000","last_active":"2024-07-04T17:33:10.091064+0000","last_peered":"2024-07-04T17:33:10.091064+0000","last_clean":"2024-07-04T17:33:10.091064+0000","last_became_active":"2024-07-04T17:33:09.128751+0000","last_became_peered":"2024-07-04T17:33:09.128751+0000","last_unstale":"2024-07-04T17:33:10.091064+0000","last_undegraded":"2024-07-04T17:33:10.091064+0000","last_fullsized":"2024-07-04T17:33:10.091064+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:39:44.077373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.127426+0000","last_change":"2024-07-04T17:33:08.096375+0000","last_active":"2024-07-04T17:33:10.127426+0000","last_peered":"2024-07-04T17:33:10.127426+0000","last_clean":"2024-07-04T17:33:10.127426+0000","last_became_active":"2024-07-04T17:33:08.096060+0000","last_became_peered":"2024-07-04T17:33:08.096060+0000","last_unstale":"2024-07-04T17:33:10.127426+0000","last_undegraded":"2024-07-04T17:33:10.127426+0000","last_fullsized":"2024-07-04T17:33:10.127426+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:08:19.414085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.090019+0000","last_change":"2024-07-04T17:33:09.107349+0000","last_active":"2024-07-04T17:33:10.090019+0000","last_peered":"2024-07-04T17:33:10.090019+0000","last_clean":"2024-07-04T17:33:10.090019+0000","last_became_active":"2024-07-04T17:33:09.104954+0000","last_became_peered":"2024-07-04T17:33:09.104954+0000","last_unstale":"2024-07-04T17:33:10.090019+0000","last_undegraded":"2024-07-04T17:33:10.090019+0000","last_fullsized":"2024-07-04T17:33:10.090019+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:19:28.456336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.127031+0000","last_change":"2024-07-04T17:33:08.106155+0000","last_active":"2024-07-04T17:33:10.127031+0000","last_peered":"2024-07-04T17:33:10.127031+0000","last_clean":"2024-07-04T17:33:10.127031+0000","last_became_active":"2024-07-04T17:33:08.104825+0000","last_became_peered":"2024-07-04T17:33:08.104825+0000","last_unstale":"2024-07-04T17:33:10.127031+0000","last_undegraded":"2024-07-04T17:33:10.127031+0000","last_fullsized":"2024-07-04T17:33:10.127031+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:15:59.977728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.126732+0000","last_change":"2024-07-04T17:33:09.107749+0000","last_active":"2024-07-04T17:33:10.126732+0000","last_peered":"2024-07-04T17:33:10.126732+0000","last_clean":"2024-07-04T17:33:10.126732+0000","last_became_active":"2024-07-04T17:33:09.106400+0000","last_became_peered":"2024-07-04T17:33:09.106400+0000","last_unstale":"2024-07-04T17:33:10.126732+0000","last_undegraded":"2024-07-04T17:33:10.126732+0000","last_fullsized":"2024-07-04T17:33:10.126732+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:22:04.584333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.091166+0000","last_change":"2024-07-04T17:33:08.109770+0000","last_active":"2024-07-04T17:33:10.091166+0000","last_peered":"2024-07-04T17:33:10.091166+0000","last_clean":"2024-07-04T17:33:10.091166+0000","last_became_active":"2024-07-04T17:33:08.108245+0000","last_became_peered":"2024-07-04T17:33:08.108245+0000","last_unstale":"2024-07-04T17:33:10.091166+0000","last_undegraded":"2024-07-04T17:33:10.091166+0000","last_fullsized":"2024-07-04T17:33:10.091166+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:30:01.785284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.127795+0000","last_change":"2024-07-04T17:33:09.099278+0000","last_active":"2024-07-04T17:33:10.127795+0000","last_peered":"2024-07-04T17:33:10.127795+0000","last_clean":"2024-07-04T17:33:10.127795+0000","last_became_active":"2024-07-04T17:33:09.098752+0000","last_became_peered":"2024-07-04T17:33:09.098752+0000","last_unstale":"2024-07-04T17:33:10.127795+0000","last_undegraded":"2024-07-04T17:33:10.127795+0000","last_fullsized":"2024-07-04T17:33:10.127795+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:34:51.476685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"113'3","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.128334+0000","last_change":"2024-07-04T17:33:08.092774+0000","last_active":"2024-07-04T17:33:10.128334+0000","last_peered":"2024-07-04T17:33:10.128334+0000","last_clean":"2024-07-04T17:33:10.128334+0000","last_became_active":"2024-07-04T17:33:08.092404+0000","last_became_peered":"2024-07-04T17:33:08.092404+0000","last_unstale":"2024-07-04T17:33:10.128334+0000","last_undegraded":"2024-07-04T17:33:10.128334+0000","last_fullsized":"2024-07-04T17:33:10.128334+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:48:02.318598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"100'101","reported_seq":448,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-07-04T17:33:10.629308+0000","last_change":"2024-07-04T17:26:04.953559+0000","last_active":"2024-07-04T17:33:10.629308+0000","last_peered":"2024-07-04T17:33:10.629308+0000","last_clean":"2024-07-04T17:33:10.629308+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:33:10.629308+0000","last_undegraded":"2024-07-04T17:33:10.629308+0000","last_fullsized":"2024-07-04T17:33:10.629308+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_clean_scrub_stamp":"2024-07-04T17:26:03.933432+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.064181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":476,"num_read_kb":1278,"num_write":163,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":476,"num_read_kb":1278,"num_write":163,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101,"ondisk_log_size":101,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738453,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36504,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335336,"statfs":{"total":96636764160,"available":96599384064,"internally_reserved":0,"allocated":1187840,"data_stored":464184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738454,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37072,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334768,"statfs":{"total":96636764160,"available":96598802432,"internally_reserved":0,"allocated":1773568,"data_stored":1053042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37088,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334752,"statfs":{"total":96636764160,"available":96598786048,"internally_reserved":0,"allocated":1781760,"data_stored":1054024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:33:14.416 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-04T17:33:14.416 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-07-04T17:33:14.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.750+0000 7fc174400640 1 -- 172.21.15.42:0/280447528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc16c0a99a0 msgr2=0x7fc16c06d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:14.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.750+0000 7fc174400640 1 --2- 172.21.15.42:0/280447528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc16c0a99a0 0x7fc16c06d120 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fc158005fd0 tx=0x7fc15802a5f0 comp rx=0 tx=0).stop 2024-07-04T17:33:14.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.750+0000 7fc174400640 1 -- 172.21.15.42:0/280447528 shutdown_connections 2024-07-04T17:33:14.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.750+0000 7fc174400640 1 --2- 172.21.15.42:0/280447528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc16c0a99a0 0x7fc16c06d120 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:14.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.750+0000 7fc174400640 1 -- 172.21.15.42:0/280447528 >> 172.21.15.42:0/280447528 conn(0x7fc16c06c020 msgr2=0x7fc16c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:14.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.750+0000 7fc174400640 1 -- 172.21.15.42:0/280447528 shutdown_connections 2024-07-04T17:33:14.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.750+0000 7fc174400640 1 -- 172.21.15.42:0/280447528 wait complete. 2024-07-04T17:33:14.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.752+0000 7fc174400640 1 Processor -- start 2024-07-04T17:33:14.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.752+0000 7fc174400640 1 -- start start 2024-07-04T17:33:14.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.753+0000 7fc174400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc16c0a99a0 0x7fc16c0b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:14.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.753+0000 7fc174400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c13f540 con 0x7fc16c0a99a0 2024-07-04T17:33:14.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.754+0000 7fc171a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc16c0a99a0 0x7fc16c0b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:14.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.754+0000 7fc171a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc16c0a99a0 0x7fc16c0b08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58792/0 (socket says 172.21.15.42:58792) 2024-07-04T17:33:14.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.754+0000 7fc171a00640 1 -- 172.21.15.42:0/375307337 learned_addr learned my addr 172.21.15.42:0/375307337 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:14.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.754+0000 7fc171a00640 1 -- 172.21.15.42:0/375307337 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc16c0adae0 con 0x7fc16c0a99a0 2024-07-04T17:33:14.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.755+0000 7fc171a00640 1 --2- 172.21.15.42:0/375307337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc16c0a99a0 0x7fc16c0b08f0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fc16000c320 tx=0x7fc16000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:14.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.755+0000 7fc169600640 1 -- 172.21.15.42:0/375307337 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc160017070 con 0x7fc16c0a99a0 2024-07-04T17:33:14.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.755+0000 7fc169600640 1 -- 172.21.15.42:0/375307337 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1600052f0 con 0x7fc16c0a99a0 2024-07-04T17:33:14.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.755+0000 7fc169600640 1 -- 172.21.15.42:0/375307337 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1600035f0 con 0x7fc16c0a99a0 2024-07-04T17:33:14.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.755+0000 7fc174400640 1 -- 172.21.15.42:0/375307337 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc16c0aec80 con 0x7fc16c0a99a0 2024-07-04T17:33:14.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.755+0000 7fc174400640 1 -- 172.21.15.42:0/375307337 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc16c1dd5b0 con 0x7fc16c0a99a0 2024-07-04T17:33:14.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.757+0000 7fc169600640 1 -- 172.21.15.42:0/375307337 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7fc16001a070 con 0x7fc16c0a99a0 2024-07-04T17:33:14.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.756+0000 7fc174400640 1 -- 172.21.15.42:0/375307337 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc16c0b1ae0 con 0x7fc16c0a99a0 2024-07-04T17:33:14.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.757+0000 7fc169600640 1 --2- 172.21.15.42:0/375307337 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fc14c03d4f0 0x7fc14c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:14.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.757+0000 7fc169600640 1 -- 172.21.15.42:0/375307337 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 3439+0+0 (secure 0 0 0) 0x7fc160050b10 con 0x7fc16c0a99a0 2024-07-04T17:33:14.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.757+0000 7fc171000640 1 --2- 172.21.15.42:0/375307337 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fc14c03d4f0 0x7fc14c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:14.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.758+0000 7fc171000640 1 --2- 172.21.15.42:0/375307337 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fc14c03d4f0 0x7fc14c03fa00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc158033000 tx=0x7fc158004730 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:14.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:14.763+0000 7fc169600640 1 -- 172.21.15.42:0/375307337 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fc16c0b1ae0 con 0x7fc16c0a99a0 2024-07-04T17:33:15.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.055+0000 7fc174400640 1 -- 172.21.15.42:0/375307337 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fc16c13fc50 con 0x7fc16c0a99a0 2024-07-04T17:33:15.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.060+0000 7fc169600640 1 -- 172.21.15.42:0/375307337 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7fc16c1dd5b0 con 0x7fc16c0a99a0 2024-07-04T17:33:15.061 INFO:teuthology.orchestra.run.smithi042.stdout:[client.0] 2024-07-04T17:33:15.061 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAO24ZmUMmYABAAh4pNlkb9JsHQtK8uCJFQCA== 2024-07-04T17:33:15.061 INFO:teuthology.orchestra.run.smithi042.stdout: caps mds = "allow" 2024-07-04T17:33:15.062 INFO:teuthology.orchestra.run.smithi042.stdout: caps mon = "allow r" 2024-07-04T17:33:15.062 INFO:teuthology.orchestra.run.smithi042.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-07-04T17:33:15.062 INFO:teuthology.orchestra.run.smithi042.stderr:updated caps for client.0 2024-07-04T17:33:15.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.063+0000 7fc174400640 1 -- 172.21.15.42:0/375307337 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fc14c03d4f0 msgr2=0x7fc14c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:15.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.063+0000 7fc174400640 1 --2- 172.21.15.42:0/375307337 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fc14c03d4f0 0x7fc14c03fa00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc158033000 tx=0x7fc158004730 comp rx=0 tx=0).stop 2024-07-04T17:33:15.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.063+0000 7fc174400640 1 -- 172.21.15.42:0/375307337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc16c0a99a0 msgr2=0x7fc16c0b08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:15.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.063+0000 7fc174400640 1 --2- 172.21.15.42:0/375307337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc16c0a99a0 0x7fc16c0b08f0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fc16000c320 tx=0x7fc16000c7e0 comp rx=0 tx=0).stop 2024-07-04T17:33:15.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.063+0000 7fc174400640 1 -- 172.21.15.42:0/375307337 shutdown_connections 2024-07-04T17:33:15.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.063+0000 7fc174400640 1 --2- 172.21.15.42:0/375307337 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fc14c03d4f0 0x7fc14c03fa00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:15.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.063+0000 7fc174400640 1 --2- 172.21.15.42:0/375307337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc16c0a99a0 0x7fc16c0b08f0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:15.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.063+0000 7fc174400640 1 -- 172.21.15.42:0/375307337 >> 172.21.15.42:0/375307337 conn(0x7fc16c06c020 msgr2=0x7fc16c13e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:15.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.063+0000 7fc174400640 1 -- 172.21.15.42:0/375307337 shutdown_connections 2024-07-04T17:33:15.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.063+0000 7fc174400640 1 -- 172.21.15.42:0/375307337 wait complete. 2024-07-04T17:33:15.093 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:33:15.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.383+0000 7fc6c5e00640 1 -- 172.21.15.42:0/1225248838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6c014abc0 msgr2=0x7fc6c006c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:15.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.383+0000 7fc6c5e00640 1 --2- 172.21.15.42:0/1225248838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6c014abc0 0x7fc6c006c020 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fc6b00097b0 tx=0x7fc6b002a540 comp rx=0 tx=0).stop 2024-07-04T17:33:15.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.383+0000 7fc6c5e00640 1 -- 172.21.15.42:0/1225248838 shutdown_connections 2024-07-04T17:33:15.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.383+0000 7fc6c5e00640 1 --2- 172.21.15.42:0/1225248838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6c014abc0 0x7fc6c006c020 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:15.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.383+0000 7fc6c5e00640 1 -- 172.21.15.42:0/1225248838 >> 172.21.15.42:0/1225248838 conn(0x7fc6c00bebc0 msgr2=0x7fc6c00befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:15.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.383+0000 7fc6c5e00640 1 -- 172.21.15.42:0/1225248838 shutdown_connections 2024-07-04T17:33:15.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.383+0000 7fc6c5e00640 1 -- 172.21.15.42:0/1225248838 wait complete. 2024-07-04T17:33:15.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.385+0000 7fc6c5e00640 1 Processor -- start 2024-07-04T17:33:15.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.385+0000 7fc6c5e00640 1 -- start start 2024-07-04T17:33:15.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.386+0000 7fc6c5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6c014abc0 0x7fc6c01e45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:15.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.387+0000 7fc6c5e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c014b8f0 con 0x7fc6c014abc0 2024-07-04T17:33:15.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.387+0000 7fc6c4a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6c014abc0 0x7fc6c01e45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:15.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.387+0000 7fc6c4a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6c014abc0 0x7fc6c01e45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58796/0 (socket says 172.21.15.42:58796) 2024-07-04T17:33:15.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.387+0000 7fc6c4a00640 1 -- 172.21.15.42:0/652642766 learned_addr learned my addr 172.21.15.42:0/652642766 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:15.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.387+0000 7fc6c4a00640 1 -- 172.21.15.42:0/652642766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c00a9650 con 0x7fc6c014abc0 2024-07-04T17:33:15.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.388+0000 7fc6c4a00640 1 --2- 172.21.15.42:0/652642766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6c014abc0 0x7fc6c01e45f0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fc6b00098d0 tx=0x7fc6b0033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:15.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.388+0000 7fc6bd600640 1 -- 172.21.15.42:0/652642766 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6b0031bc0 con 0x7fc6c014abc0 2024-07-04T17:33:15.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.388+0000 7fc6bd600640 1 -- 172.21.15.42:0/652642766 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6b0031e50 con 0x7fc6c014abc0 2024-07-04T17:33:15.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.388+0000 7fc6bd600640 1 -- 172.21.15.42:0/652642766 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6b0033200 con 0x7fc6c014abc0 2024-07-04T17:33:15.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.388+0000 7fc6c5e00640 1 -- 172.21.15.42:0/652642766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6c01e4b50 con 0x7fc6c014abc0 2024-07-04T17:33:15.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.388+0000 7fc6c5e00640 1 -- 172.21.15.42:0/652642766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6c01483f0 con 0x7fc6c014abc0 2024-07-04T17:33:15.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.389+0000 7fc6c5e00640 1 -- 172.21.15.42:0/652642766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6c014bfa0 con 0x7fc6c014abc0 2024-07-04T17:33:15.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.389+0000 7fc6bd600640 1 -- 172.21.15.42:0/652642766 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7fc6b00333a0 con 0x7fc6c014abc0 2024-07-04T17:33:15.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.390+0000 7fc6bd600640 1 --2- 172.21.15.42:0/652642766 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fc69c03d4f0 0x7fc69c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:15.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.390+0000 7fc6bd600640 1 -- 172.21.15.42:0/652642766 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 3439+0+0 (secure 0 0 0) 0x7fc6b0072eb0 con 0x7fc6c014abc0 2024-07-04T17:33:15.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.390+0000 7fc6b7e00640 1 --2- 172.21.15.42:0/652642766 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fc69c03d4f0 0x7fc69c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:15.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.391+0000 7fc6b7e00640 1 --2- 172.21.15.42:0/652642766 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fc69c03d4f0 0x7fc69c03fa00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc6c00b4f00 tx=0x7fc6a8007a90 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:15.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.395+0000 7fc6bd600640 1 -- 172.21.15.42:0/652642766 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fc6b003f030 con 0x7fc6c014abc0 2024-07-04T17:33:15.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.652+0000 7fc6c5e00640 1 -- 172.21.15.42:0/652642766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6c0068e30 con 0x7fc6c014abc0 2024-07-04T17:33:15.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.654+0000 7fc6bd600640 1 -- 172.21.15.42:0/652642766 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+6034 (secure 0 0 0) 0x7fc6b0033690 con 0x7fc6c014abc0 2024-07-04T17:33:15.655 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:33:15.655 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":90,"btime":"2024-07-04T17:33:12:103002+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5536,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/68857121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":68857121},{"type":"v1","addr":"172.21.15.42:6825","nonce":68857121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":5544,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6828/20990456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":20990456},{"type":"v1","addr":"172.21.15.42:6828","nonce":20990456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":5545,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/428501060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6827","nonce":428501060},{"type":"v1","addr":"172.21.15.42:6829","nonce":428501060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":5549,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/3224527035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":3224527035},{"type":"v1","addr":"172.21.15.42:6831","nonce":3224527035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:33:08.649551+0000","modified":"2024-07-04T17:33:11.687748+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5550":{"gid":5550,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6833/605786437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":605786437},{"type":"v1","addr":"172.21.15.42:6833","nonce":605786437}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5550,"qdb_cluster":[5550]},"id":10}]} 2024-07-04T17:33:15.656 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 90 2024-07-04T17:33:15.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.656+0000 7fc6c5e00640 1 -- 172.21.15.42:0/652642766 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fc69c03d4f0 msgr2=0x7fc69c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:15.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.656+0000 7fc6c5e00640 1 --2- 172.21.15.42:0/652642766 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fc69c03d4f0 0x7fc69c03fa00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc6c00b4f00 tx=0x7fc6a8007a90 comp rx=0 tx=0).stop 2024-07-04T17:33:15.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.656+0000 7fc6c5e00640 1 -- 172.21.15.42:0/652642766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6c014abc0 msgr2=0x7fc6c01e45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:15.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.656+0000 7fc6c5e00640 1 --2- 172.21.15.42:0/652642766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6c014abc0 0x7fc6c01e45f0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fc6b00098d0 tx=0x7fc6b0033e40 comp rx=0 tx=0).stop 2024-07-04T17:33:15.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.657+0000 7fc6c5e00640 1 -- 172.21.15.42:0/652642766 shutdown_connections 2024-07-04T17:33:15.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.657+0000 7fc6c5e00640 1 --2- 172.21.15.42:0/652642766 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fc69c03d4f0 0x7fc69c03fa00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:15.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.657+0000 7fc6c5e00640 1 --2- 172.21.15.42:0/652642766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6c014abc0 0x7fc6c01e45f0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:15.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.657+0000 7fc6c5e00640 1 -- 172.21.15.42:0/652642766 >> 172.21.15.42:0/652642766 conn(0x7fc6c00bebc0 msgr2=0x7fc6c00bbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:15.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.657+0000 7fc6c5e00640 1 -- 172.21.15.42:0/652642766 shutdown_connections 2024-07-04T17:33:15.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:15.657+0000 7fc6c5e00640 1 -- 172.21.15.42:0/652642766 wait complete. 2024-07-04T17:33:16.187 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-04T17:33:08.649551+0000', 'modified': '2024-07-04T17:33:11.687748+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5550}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5550': {'gid': 5550, 'name': 'c', 'rank': 0, 'incarnation': 86, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.42:6833/605786437', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6832', 'nonce': 605786437}, {'type': 'v1', 'addr': '172.21.15.42:6833', 'nonce': 605786437}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5550, 'qdb_cluster': [5550]} 2024-07-04T17:33:16.187 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-04T17:33:16.187 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-07-04T17:33:16.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.492+0000 7f8f53200640 1 -- 172.21.15.42:0/988729778 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f5406c200 msgr2=0x7f8f5414ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:16.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.492+0000 7f8f53200640 1 --2- 172.21.15.42:0/988729778 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f5406c200 0x7f8f5414ef10 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f8f44005fd0 tx=0x7f8f4402a560 comp rx=0 tx=0).stop 2024-07-04T17:33:16.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.493+0000 7f8f53200640 1 -- 172.21.15.42:0/988729778 shutdown_connections 2024-07-04T17:33:16.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.493+0000 7f8f53200640 1 --2- 172.21.15.42:0/988729778 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f5406c200 0x7f8f5414ef10 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:16.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.493+0000 7f8f53200640 1 -- 172.21.15.42:0/988729778 >> 172.21.15.42:0/988729778 conn(0x7f8f540bebc0 msgr2=0x7f8f540befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:16.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.493+0000 7f8f53200640 1 -- 172.21.15.42:0/988729778 shutdown_connections 2024-07-04T17:33:16.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.493+0000 7f8f53200640 1 -- 172.21.15.42:0/988729778 wait complete. 2024-07-04T17:33:16.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.495+0000 7f8f53200640 1 Processor -- start 2024-07-04T17:33:16.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.495+0000 7f8f53200640 1 -- start start 2024-07-04T17:33:16.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.496+0000 7f8f53200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f5406c200 0x7f8f541d7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:16.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.497+0000 7f8f53200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5414ae50 con 0x7f8f5406c200 2024-07-04T17:33:16.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.497+0000 7f8f51e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f5406c200 0x7f8f541d7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:16.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.497+0000 7f8f51e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f5406c200 0x7f8f541d7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58814/0 (socket says 172.21.15.42:58814) 2024-07-04T17:33:16.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.497+0000 7f8f51e00640 1 -- 172.21.15.42:0/1961641400 learned_addr learned my addr 172.21.15.42:0/1961641400 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:16.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.497+0000 7f8f51e00640 1 -- 172.21.15.42:0/1961641400 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f540bd000 con 0x7f8f5406c200 2024-07-04T17:33:16.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.498+0000 7f8f51e00640 1 --2- 172.21.15.42:0/1961641400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f5406c200 0x7f8f541d7e70 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f8f44034040 tx=0x7f8f440049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:16.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.498+0000 7f8f4a000640 1 -- 172.21.15.42:0/1961641400 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f4403e030 con 0x7f8f5406c200 2024-07-04T17:33:16.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.498+0000 7f8f4a000640 1 -- 172.21.15.42:0/1961641400 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f440035f0 con 0x7f8f5406c200 2024-07-04T17:33:16.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.498+0000 7f8f4a000640 1 -- 172.21.15.42:0/1961641400 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f44035040 con 0x7f8f5406c200 2024-07-04T17:33:16.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.498+0000 7f8f53200640 1 -- 172.21.15.42:0/1961641400 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f54070100 con 0x7f8f5406c200 2024-07-04T17:33:16.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.498+0000 7f8f53200640 1 -- 172.21.15.42:0/1961641400 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f540bcba0 con 0x7f8f5406c200 2024-07-04T17:33:16.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.500+0000 7f8f4a000640 1 -- 172.21.15.42:0/1961641400 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7f8f44041030 con 0x7f8f5406c200 2024-07-04T17:33:16.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.500+0000 7f8f4a000640 1 --2- 172.21.15.42:0/1961641400 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f8f3003d4f0 0x7f8f3003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:16.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.500+0000 7f8f4a000640 1 -- 172.21.15.42:0/1961641400 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 3439+0+0 (secure 0 0 0) 0x7f8f4403f070 con 0x7f8f5406c200 2024-07-04T17:33:16.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.501+0000 7f8f4be00640 1 --2- 172.21.15.42:0/1961641400 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f8f3003d4f0 0x7f8f3003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:16.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.501+0000 7f8f4be00640 1 --2- 172.21.15.42:0/1961641400 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f8f3003d4f0 0x7f8f3003fa00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8f540bac80 tx=0x7f8f3c007a30 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:16.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.503+0000 7f8f60241740 1 Processor -- start 2024-07-04T17:33:16.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.503+0000 7f8f60241740 1 -- start start 2024-07-04T17:33:16.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.504+0000 7f8f60241740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604fd7fe270 0x5604fdba4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:16.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.504+0000 7f8f60241740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5604fdb155b0 con 0x5604fd7fe270 2024-07-04T17:33:16.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.504+0000 7f8f51e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604fd7fe270 0x5604fdba4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:16.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.504+0000 7f8f51e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604fd7fe270 0x5604fdba4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58828/0 (socket says 172.21.15.42:58828) 2024-07-04T17:33:16.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.504+0000 7f8f51e00640 1 -- 172.21.15.42:0/2164142556 learned_addr learned my addr 172.21.15.42:0/2164142556 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:16.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.505+0000 7f8f51e00640 1 -- 172.21.15.42:0/2164142556 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604fd869850 con 0x5604fd7fe270 2024-07-04T17:33:16.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.505+0000 7f8f51e00640 1 --2- 172.21.15.42:0/2164142556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604fd7fe270 0x5604fdba4d30 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f8f4404bac0 tx=0x7f8f4402e260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906b73aabb420e77 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:16.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.505+0000 7f8f42a00640 1 -- 172.21.15.42:0/2164142556 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f44048040 con 0x5604fd7fe270 2024-07-04T17:33:16.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.505+0000 7f8f42a00640 1 -- 172.21.15.42:0/2164142556 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f4406f070 con 0x5604fd7fe270 2024-07-04T17:33:16.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.505+0000 7f8f42a00640 1 -- 172.21.15.42:0/2164142556 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f44031570 con 0x5604fd7fe270 2024-07-04T17:33:16.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.506+0000 7f8f60241740 1 -- 172.21.15.42:0/2164142556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604fd7fe270 msgr2=0x5604fdba4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:16.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.506+0000 7f8f60241740 1 --2- 172.21.15.42:0/2164142556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604fd7fe270 0x5604fdba4d30 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f8f4404bac0 tx=0x7f8f4402e260 comp rx=0 tx=0).stop 2024-07-04T17:33:16.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.506+0000 7f8f60241740 1 -- 172.21.15.42:0/2164142556 shutdown_connections 2024-07-04T17:33:16.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.506+0000 7f8f60241740 1 --2- 172.21.15.42:0/2164142556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604fd7fe270 0x5604fdba4d30 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:16.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.506+0000 7f8f60241740 1 -- 172.21.15.42:0/2164142556 >> 172.21.15.42:0/2164142556 conn(0x5604fd7f23c0 msgr2=0x5604fdba34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:16.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.506+0000 7f8f60241740 1 -- 172.21.15.42:0/2164142556 shutdown_connections 2024-07-04T17:33:16.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.506+0000 7f8f60241740 1 -- 172.21.15.42:0/2164142556 wait complete. 2024-07-04T17:33:16.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.507+0000 7f8f60241740 1 -- start start 2024-07-04T17:33:16.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.507+0000 7f8f60241740 1 Processor -- start 2024-07-04T17:33:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.509+0000 7f8f60241740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604fd7fe270 0x5604fdbe78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.509+0000 7f8f60241740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5604fdb08b90 con 0x5604fd7fe270 2024-07-04T17:33:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.509+0000 7f8f60241740 10 client.0 _ll_register_callbacks cb 0x5604fdaad8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T17:33:16.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.509+0000 7f8f51e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604fd7fe270 0x5604fdbe78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:16.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.509+0000 7f8f51e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604fd7fe270 0x5604fdbe78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58836/0 (socket says 172.21.15.42:58836) 2024-07-04T17:33:16.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.509+0000 7f8f51e00640 1 -- 172.21.15.42:0/1003925844 learned_addr learned my addr 172.21.15.42:0/1003925844 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:16.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.509+0000 7f8f51e00640 1 -- 172.21.15.42:0/1003925844 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604fd8e06b0 con 0x5604fd7fe270 2024-07-04T17:33:16.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.510+0000 7f8f51e00640 1 --2- 172.21.15.42:0/1003925844 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604fd7fe270 0x5604fdbe78c0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f8f4406c040 tx=0x7f8f44030ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2957761fb4f62e2 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:16.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.510+0000 7f8f37400640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-07-04T17:33:16.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.510+0000 7f8f37400640 1 -- 172.21.15.42:0/1003925844 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f44048050 con 0x5604fd7fe270 2024-07-04T17:33:16.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.510+0000 7f8f37400640 1 -- 172.21.15.42:0/1003925844 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 3439+0+0 (secure 0 0 0) 0x7f8f44003a10 con 0x5604fd7fe270 2024-07-04T17:33:16.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.510+0000 7f8f60241740 1 -- 172.21.15.42:0/1003925844 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5604fdaa56c0 con 0x5604fd7fe270 2024-07-04T17:33:16.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.510+0000 7f8f37400640 1 -- 172.21.15.42:0/1003925844 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f8f440303a0 con 0x5604fd7fe270 2024-07-04T17:33:16.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.510+0000 7f8f60241740 10 client.5591 fetch_fsmap learned FSMap version 90 2024-07-04T17:33:16.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.510+0000 7f8f60241740 1 -- 172.21.15.42:0/1003925844 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x5604fdbc0930 con 0x5604fd7fe270 2024-07-04T17:33:16.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.511+0000 7f8f37400640 1 -- 172.21.15.42:0/1003925844 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5604fdaa56c0 con 0x5604fd7fe270 2024-07-04T17:33:16.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.511+0000 7f8f37400640 1 -- 172.21.15.42:0/1003925844 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 90) ==== 4107+0+0 (secure 0 0 0) 0x7f8f440710d0 con 0x5604fd7fe270 2024-07-04T17:33:16.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.511+0000 7f8f60241740 10 client.5591 fetch_fsmap finished waiting for FSMap version 90 2024-07-04T17:33:16.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.511+0000 7f8f60241740 10 client.5591 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-04T17:33:16.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.511+0000 7f8f60241740 20 client.5591 populate_metadata read hostname 'smithi042' 2024-07-04T17:33:16.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.511+0000 7f8f60241740 1 --2- 172.21.15.42:0/1003925844 >> [v2:172.21.15.42:6832/605786437,v1:172.21.15.42:6833/605786437] conn(0x5604fd7d53b0 0x5604fdbcc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:16.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.511+0000 7f8f60241740 4 client.5591 mds_command: new command op to 5550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T17:33:16.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.511+0000 7f8f60241740 1 -- 172.21.15.42:0/1003925844 --> [v2:172.21.15.42:6832/605786437,v1:172.21.15.42:6833/605786437] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604fdb17500 con 0x5604fd7d53b0 2024-07-04T17:33:16.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.511+0000 7f8f4be00640 1 --2- 172.21.15.42:0/1003925844 >> [v2:172.21.15.42:6832/605786437,v1:172.21.15.42:6833/605786437] conn(0x5604fd7d53b0 0x5604fdbcc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:16.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.512+0000 7f8f4be00640 1 --2- 172.21.15.42:0/1003925844 >> [v2:172.21.15.42:6832/605786437,v1:172.21.15.42:6833/605786437] conn(0x5604fd7d53b0 0x5604fdbcc9f0 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4ec4416f326de8b server_cookie=10519c07fa6f84b4 in_seq=0 out_seq=0 2024-07-04T17:33:16.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.512+0000 7f8f37400640 10 client.5591 ms_handle_connect on v2:172.21.15.42:6832/605786437 2024-07-04T17:33:16.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.513+0000 7f8f37400640 1 -- 172.21.15.42:0/1003925844 <== mds.0 v2:172.21.15.42:6832/605786437 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5604fdb17500 con 0x5604fd7d53b0 2024-07-04T17:33:16.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.513+0000 7f8f37400640 10 client.5591 handle_command_reply: tid=0 2024-07-04T17:33:16.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.513+0000 7f8f60241740 1 client.5591 shutdown 2024-07-04T17:33:16.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.513+0000 7f8f60241740 1 -- 172.21.15.42:0/1003925844 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604fd7fe270 msgr2=0x5604fdbe78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:16.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.513+0000 7f8f60241740 1 --2- 172.21.15.42:0/1003925844 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604fd7fe270 0x5604fdbe78c0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f8f4406c040 tx=0x7f8f44030ef0 comp rx=0 tx=0).stop 2024-07-04T17:33:16.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.513+0000 7f8f60241740 1 -- 172.21.15.42:0/1003925844 shutdown_connections 2024-07-04T17:33:16.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.513+0000 7f8f60241740 1 --2- 172.21.15.42:0/1003925844 >> [v2:172.21.15.42:6832/605786437,v1:172.21.15.42:6833/605786437] conn(0x5604fd7d53b0 0x5604fdbcc9f0 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:16.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.513+0000 7f8f60241740 1 --2- 172.21.15.42:0/1003925844 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604fd7fe270 0x5604fdbe78c0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:16.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.513+0000 7f8f60241740 1 -- 172.21.15.42:0/1003925844 >> 172.21.15.42:0/1003925844 conn(0x5604fd7f23c0 msgr2=0x5604fdbd0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:16.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.513+0000 7f8f37400640 0 client.5591 ms_handle_reset on v2:172.21.15.42:6832/605786437 2024-07-04T17:33:16.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.513+0000 7f8f60241740 1 -- 172.21.15.42:0/1003925844 shutdown_connections 2024-07-04T17:33:16.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.514+0000 7f8f60241740 1 -- 172.21.15.42:0/1003925844 wait complete. 2024-07-04T17:33:16.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.514+0000 7f8f60241740 20 client.5591 trim_cache size 0 max 16384 2024-07-04T17:33:16.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.535+0000 7f8f60241740 1 Processor -- start 2024-07-04T17:33:16.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.535+0000 7f8f60241740 1 -- start start 2024-07-04T17:33:16.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.536+0000 7f8f60241740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604fd7fe270 0x5604fdbbfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:16.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.536+0000 7f8f60241740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8f541ead30 con 0x5604fd7fe270 2024-07-04T17:33:16.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.536+0000 7f8f51e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604fd7fe270 0x5604fdbbfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:16.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.536+0000 7f8f51e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604fd7fe270 0x5604fdbbfe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58842/0 (socket says 172.21.15.42:58842) 2024-07-04T17:33:16.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.536+0000 7f8f51e00640 1 -- 172.21.15.42:0/80560832 learned_addr learned my addr 172.21.15.42:0/80560832 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:16.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.537+0000 7f8f51e00640 1 -- 172.21.15.42:0/80560832 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604fdba50a0 con 0x5604fd7fe270 2024-07-04T17:33:16.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.537+0000 7f8f51e00640 1 --2- 172.21.15.42:0/80560832 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604fd7fe270 0x5604fdbbfe10 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f8f44000c00 tx=0x7f8f44031610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe6797de8cd6a74 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:16.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.537+0000 7f8f42000640 1 -- 172.21.15.42:0/80560832 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f44048040 con 0x5604fd7fe270 2024-07-04T17:33:16.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.537+0000 7f8f42000640 1 -- 172.21.15.42:0/80560832 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f4405c020 con 0x5604fd7fe270 2024-07-04T17:33:16.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.537+0000 7f8f42000640 1 -- 172.21.15.42:0/80560832 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f44003980 con 0x5604fd7fe270 2024-07-04T17:33:16.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.538+0000 7f8f60241740 1 -- 172.21.15.42:0/80560832 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604fd7fe270 msgr2=0x5604fdbbfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:16.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.538+0000 7f8f60241740 1 --2- 172.21.15.42:0/80560832 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604fd7fe270 0x5604fdbbfe10 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f8f44000c00 tx=0x7f8f44031610 comp rx=0 tx=0).stop 2024-07-04T17:33:16.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.538+0000 7f8f60241740 1 -- 172.21.15.42:0/80560832 shutdown_connections 2024-07-04T17:33:16.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.538+0000 7f8f60241740 1 --2- 172.21.15.42:0/80560832 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604fd7fe270 0x5604fdbbfe10 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:16.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.538+0000 7f8f60241740 1 -- 172.21.15.42:0/80560832 >> 172.21.15.42:0/80560832 conn(0x5604fd7f23c0 msgr2=0x5604fdbd0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:16.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.538+0000 7f8f60241740 1 -- 172.21.15.42:0/80560832 shutdown_connections 2024-07-04T17:33:16.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.538+0000 7f8f60241740 1 -- 172.21.15.42:0/80560832 wait complete. 2024-07-04T17:33:16.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.539+0000 7f8f60241740 1 -- start start 2024-07-04T17:33:16.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.539+0000 7f8f60241740 1 Processor -- start 2024-07-04T17:33:16.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.541+0000 7f8f60241740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604fd7fe270 0x5604fdba05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:16.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.541+0000 7f8f60241740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5604fdb04b50 con 0x5604fd7fe270 2024-07-04T17:33:16.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.541+0000 7f8f60241740 10 client.0 _ll_register_callbacks cb 0x7f8f440038d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T17:33:16.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.541+0000 7f8f51e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604fd7fe270 0x5604fdba05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:16.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.541+0000 7f8f51e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604fd7fe270 0x5604fdba05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58854/0 (socket says 172.21.15.42:58854) 2024-07-04T17:33:16.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.541+0000 7f8f51e00640 1 -- 172.21.15.42:0/1806431963 learned_addr learned my addr 172.21.15.42:0/1806431963 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:16.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.541+0000 7f8f51e00640 1 -- 172.21.15.42:0/1806431963 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604fdbf2670 con 0x5604fd7fe270 2024-07-04T17:33:16.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.542+0000 7f8f51e00640 1 --2- 172.21.15.42:0/1806431963 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604fd7fe270 0x5604fdba05f0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f8f44004bb0 tx=0x7f8f44005010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9852edc27a3d54 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:16.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.542+0000 7f8f37400640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-07-04T17:33:16.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.542+0000 7f8f37400640 1 -- 172.21.15.42:0/1806431963 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f44076070 con 0x5604fd7fe270 2024-07-04T17:33:16.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.542+0000 7f8f37400640 1 -- 172.21.15.42:0/1806431963 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 3439+0+0 (secure 0 0 0) 0x7f8f4405d070 con 0x5604fd7fe270 2024-07-04T17:33:16.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.542+0000 7f8f60241740 1 -- 172.21.15.42:0/1806431963 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f4405c020 con 0x5604fd7fe270 2024-07-04T17:33:16.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.542+0000 7f8f37400640 1 -- 172.21.15.42:0/1806431963 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f8f44080050 con 0x5604fd7fe270 2024-07-04T17:33:16.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.542+0000 7f8f60241740 10 client.5593 fetch_fsmap learned FSMap version 90 2024-07-04T17:33:16.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.542+0000 7f8f60241740 1 -- 172.21.15.42:0/1806431963 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x5604fdbf2670 con 0x5604fd7fe270 2024-07-04T17:33:16.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.543+0000 7f8f37400640 1 -- 172.21.15.42:0/1806431963 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f4405c020 con 0x5604fd7fe270 2024-07-04T17:33:16.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.543+0000 7f8f37400640 1 -- 172.21.15.42:0/1806431963 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 90) ==== 4107+0+0 (secure 0 0 0) 0x7f8f44042bc0 con 0x5604fd7fe270 2024-07-04T17:33:16.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.543+0000 7f8f60241740 10 client.5593 fetch_fsmap finished waiting for FSMap version 90 2024-07-04T17:33:16.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.543+0000 7f8f60241740 10 client.5593 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-04T17:33:16.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.543+0000 7f8f60241740 20 client.5593 populate_metadata read hostname 'smithi042' 2024-07-04T17:33:16.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.543+0000 7f8f60241740 1 --2- 172.21.15.42:0/1806431963 >> [v2:172.21.15.42:6832/605786437,v1:172.21.15.42:6833/605786437] conn(0x5604fd7bce40 0x5604fdc4eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:16.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.543+0000 7f8f60241740 4 client.5593 mds_command: new command op to 5550 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-04T17:33:16.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.543+0000 7f8f60241740 1 -- 172.21.15.42:0/1806431963 --> [v2:172.21.15.42:6832/605786437,v1:172.21.15.42:6833/605786437] -- command(tid 0: {"prefix": "status"}) -- 0x5604fdbc9340 con 0x5604fd7bce40 2024-07-04T17:33:16.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.543+0000 7f8f4be00640 1 --2- 172.21.15.42:0/1806431963 >> [v2:172.21.15.42:6832/605786437,v1:172.21.15.42:6833/605786437] conn(0x5604fd7bce40 0x5604fdc4eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:16.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.544+0000 7f8f4be00640 1 --2- 172.21.15.42:0/1806431963 >> [v2:172.21.15.42:6832/605786437,v1:172.21.15.42:6833/605786437] conn(0x5604fd7bce40 0x5604fdc4eda0 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aef48219b144b0d server_cookie=706b18edccb8ba5b in_seq=0 out_seq=0 2024-07-04T17:33:16.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.544+0000 7f8f37400640 10 client.5593 ms_handle_connect on v2:172.21.15.42:6832/605786437 2024-07-04T17:33:16.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.544+0000 7f8f37400640 1 -- 172.21.15.42:0/1806431963 <== mds.0 v2:172.21.15.42:6832/605786437 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5604fdbc9340 con 0x5604fd7bce40 2024-07-04T17:33:16.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.544+0000 7f8f37400640 10 client.5593 handle_command_reply: tid=0 2024-07-04T17:33:16.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.544+0000 7f8f60241740 1 client.5593 shutdown 2024-07-04T17:33:16.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.544+0000 7f8f60241740 1 -- 172.21.15.42:0/1806431963 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604fd7fe270 msgr2=0x5604fdba05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:16.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.544+0000 7f8f60241740 1 --2- 172.21.15.42:0/1806431963 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604fd7fe270 0x5604fdba05f0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f8f44004bb0 tx=0x7f8f44005010 comp rx=0 tx=0).stop 2024-07-04T17:33:16.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.544+0000 7f8f60241740 1 -- 172.21.15.42:0/1806431963 shutdown_connections 2024-07-04T17:33:16.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.544+0000 7f8f60241740 1 --2- 172.21.15.42:0/1806431963 >> [v2:172.21.15.42:6832/605786437,v1:172.21.15.42:6833/605786437] conn(0x5604fd7bce40 0x5604fdc4eda0 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:16.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.544+0000 7f8f60241740 1 --2- 172.21.15.42:0/1806431963 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5604fd7fe270 0x5604fdba05f0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:16.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.544+0000 7f8f60241740 1 -- 172.21.15.42:0/1806431963 >> 172.21.15.42:0/1806431963 conn(0x5604fd7f23c0 msgr2=0x5604fdbbfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:16.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.544+0000 7f8f37400640 0 client.5593 ms_handle_reset on v2:172.21.15.42:6832/605786437 2024-07-04T17:33:16.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.544+0000 7f8f60241740 1 -- 172.21.15.42:0/1806431963 shutdown_connections 2024-07-04T17:33:16.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.545+0000 7f8f60241740 1 -- 172.21.15.42:0/1806431963 wait complete. 2024-07-04T17:33:16.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.545+0000 7f8f60241740 20 client.5593 trim_cache size 0 max 16384 2024-07-04T17:33:16.548 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:33:16.548 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "be4f300a-93ef-4489-b540-37ece39b0aac", 2024-07-04T17:33:16.548 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-07-04T17:33:16.548 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 5550, 2024-07-04T17:33:16.548 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-07-04T17:33:16.549 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-07-04T17:33:16.549 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-07-04T17:33:16.549 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 7.4798874399999997, 2024-07-04T17:33:16.549 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 90, 2024-07-04T17:33:16.549 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 114, 2024-07-04T17:33:16.549 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 114, 2024-07-04T17:33:16.549 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 12.332423023 2024-07-04T17:33:16.549 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:33:16.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.545+0000 7f8f53200640 1 -- 172.21.15.42:0/1961641400 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f8f3003d4f0 msgr2=0x7f8f3003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:16.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.545+0000 7f8f53200640 1 --2- 172.21.15.42:0/1961641400 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f8f3003d4f0 0x7f8f3003fa00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8f540bac80 tx=0x7f8f3c007a30 comp rx=0 tx=0).stop 2024-07-04T17:33:16.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.545+0000 7f8f53200640 1 -- 172.21.15.42:0/1961641400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f5406c200 msgr2=0x7f8f541d7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:16.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.545+0000 7f8f53200640 1 --2- 172.21.15.42:0/1961641400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f5406c200 0x7f8f541d7e70 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f8f44034040 tx=0x7f8f440049d0 comp rx=0 tx=0).stop 2024-07-04T17:33:16.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.545+0000 7f8f53200640 1 -- 172.21.15.42:0/1961641400 shutdown_connections 2024-07-04T17:33:16.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.545+0000 7f8f53200640 1 --2- 172.21.15.42:0/1961641400 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f8f3003d4f0 0x7f8f3003fa00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:16.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.545+0000 7f8f53200640 1 --2- 172.21.15.42:0/1961641400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f5406c200 0x7f8f541d7e70 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:16.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.545+0000 7f8f53200640 1 -- 172.21.15.42:0/1961641400 >> 172.21.15.42:0/1961641400 conn(0x7f8f540bebc0 msgr2=0x7f8f540bbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:16.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.546+0000 7f8f53200640 1 -- 172.21.15.42:0/1961641400 shutdown_connections 2024-07-04T17:33:16.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.546+0000 7f8f53200640 1 -- 172.21.15.42:0/1961641400 wait complete. 2024-07-04T17:33:16.573 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-04T17:33:16.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.861+0000 7ff2bfe00640 1 -- 172.21.15.42:0/24487932 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2c0141dd0 msgr2=0x7ff2c00b8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:16.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.861+0000 7ff2bfe00640 1 --2- 172.21.15.42:0/24487932 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2c0141dd0 0x7ff2c00b8e60 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac005fd0 tx=0x7ff2ac02a5f0 comp rx=0 tx=0).stop 2024-07-04T17:33:16.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.862+0000 7ff2bfe00640 1 -- 172.21.15.42:0/24487932 shutdown_connections 2024-07-04T17:33:16.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.862+0000 7ff2bfe00640 1 --2- 172.21.15.42:0/24487932 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2c0141dd0 0x7ff2c00b8e60 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:16.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.862+0000 7ff2bfe00640 1 -- 172.21.15.42:0/24487932 >> 172.21.15.42:0/24487932 conn(0x7ff2c00bebc0 msgr2=0x7ff2c00befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:16.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.862+0000 7ff2bfe00640 1 -- 172.21.15.42:0/24487932 shutdown_connections 2024-07-04T17:33:16.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.862+0000 7ff2bfe00640 1 -- 172.21.15.42:0/24487932 wait complete. 2024-07-04T17:33:16.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.865+0000 7ff2bfe00640 1 Processor -- start 2024-07-04T17:33:16.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.865+0000 7ff2bfe00640 1 -- start start 2024-07-04T17:33:16.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.866+0000 7ff2bfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2c006cf90 0x7ff2c00b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:16.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.866+0000 7ff2bfe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c00babe0 con 0x7ff2c006cf90 2024-07-04T17:33:16.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.867+0000 7ff2bea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2c006cf90 0x7ff2c00b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:16.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.867+0000 7ff2bea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2c006cf90 0x7ff2c00b18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58880/0 (socket says 172.21.15.42:58880) 2024-07-04T17:33:16.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.867+0000 7ff2bea00640 1 -- 172.21.15.42:0/2084612753 learned_addr learned my addr 172.21.15.42:0/2084612753 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:16.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.868+0000 7ff2bea00640 1 -- 172.21.15.42:0/2084612753 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c00acd30 con 0x7ff2c006cf90 2024-07-04T17:33:16.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.868+0000 7ff2bea00640 1 --2- 172.21.15.42:0/2084612753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2c006cf90 0x7ff2c00b18e0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0001eb0 tx=0x7ff2b000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:16.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.868+0000 7ff2b6a00640 1 -- 172.21.15.42:0/2084612753 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2b0017070 con 0x7ff2c006cf90 2024-07-04T17:33:16.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.868+0000 7ff2b6a00640 1 -- 172.21.15.42:0/2084612753 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2b00052f0 con 0x7ff2c006cf90 2024-07-04T17:33:16.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.869+0000 7ff2bfe00640 1 -- 172.21.15.42:0/2084612753 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c01e4320 con 0x7ff2c006cf90 2024-07-04T17:33:16.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.869+0000 7ff2b6a00640 1 -- 172.21.15.42:0/2084612753 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2b00035f0 con 0x7ff2c006cf90 2024-07-04T17:33:16.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.869+0000 7ff2bfe00640 1 -- 172.21.15.42:0/2084612753 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c0141380 con 0x7ff2c006cf90 2024-07-04T17:33:16.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.870+0000 7ff2bfe00640 1 -- 172.21.15.42:0/2084612753 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c00bb1d0 con 0x7ff2c006cf90 2024-07-04T17:33:16.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.870+0000 7ff2b6a00640 1 -- 172.21.15.42:0/2084612753 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7ff2b001a070 con 0x7ff2c006cf90 2024-07-04T17:33:16.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.871+0000 7ff2b6a00640 1 --2- 172.21.15.42:0/2084612753 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7ff2a003d540 0x7ff2a003fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:16.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.871+0000 7ff2b6a00640 1 -- 172.21.15.42:0/2084612753 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 3439+0+0 (secure 0 0 0) 0x7ff2b0050ba0 con 0x7ff2c006cf90 2024-07-04T17:33:16.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.872+0000 7ff2be000640 1 --2- 172.21.15.42:0/2084612753 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7ff2a003d540 0x7ff2a003fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:16.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.873+0000 7ff2be000640 1 --2- 172.21.15.42:0/2084612753 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7ff2a003d540 0x7ff2a003fa50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac033000 tx=0x7ff2ac004730 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:16.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:16.879+0000 7ff2b6a00640 1 -- 172.21.15.42:0/2084612753 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7ff2b0016170 con 0x7ff2c006cf90 2024-07-04T17:33:17.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:17.140+0000 7ff2bfe00640 1 -- 172.21.15.42:0/2084612753 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff2c0068e30 con 0x7ff2c006cf90 2024-07-04T17:33:17.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:17.141+0000 7ff2b6a00640 1 -- 172.21.15.42:0/2084612753 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v90) ==== 53+0+73 (secure 0 0 0) 0x7ff2b00199b0 con 0x7ff2c006cf90 2024-07-04T17:33:17.142 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-04T17:33:17.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:17.143+0000 7ff2bfe00640 1 -- 172.21.15.42:0/2084612753 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7ff2a003d540 msgr2=0x7ff2a003fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:17.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:17.143+0000 7ff2bfe00640 1 --2- 172.21.15.42:0/2084612753 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7ff2a003d540 0x7ff2a003fa50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac033000 tx=0x7ff2ac004730 comp rx=0 tx=0).stop 2024-07-04T17:33:17.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:17.143+0000 7ff2bfe00640 1 -- 172.21.15.42:0/2084612753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2c006cf90 msgr2=0x7ff2c00b18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:17.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:17.143+0000 7ff2bfe00640 1 --2- 172.21.15.42:0/2084612753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2c006cf90 0x7ff2c00b18e0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0001eb0 tx=0x7ff2b000c810 comp rx=0 tx=0).stop 2024-07-04T17:33:17.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:17.143+0000 7ff2bfe00640 1 -- 172.21.15.42:0/2084612753 shutdown_connections 2024-07-04T17:33:17.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:17.143+0000 7ff2bfe00640 1 --2- 172.21.15.42:0/2084612753 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7ff2a003d540 0x7ff2a003fa50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:17.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:17.143+0000 7ff2bfe00640 1 --2- 172.21.15.42:0/2084612753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2c006cf90 0x7ff2c00b18e0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:17.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:17.143+0000 7ff2bfe00640 1 -- 172.21.15.42:0/2084612753 >> 172.21.15.42:0/2084612753 conn(0x7ff2c00bebc0 msgr2=0x7ff2c00b9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:17.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:17.144+0000 7ff2bfe00640 1 -- 172.21.15.42:0/2084612753 shutdown_connections 2024-07-04T17:33:17.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:17.144+0000 7ff2bfe00640 1 -- 172.21.15.42:0/2084612753 wait complete. 2024-07-04T17:33:17.173 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-07-04T17:33:17.173 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-04T17:33:17.173 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2024-07-04T17:33:17.173 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-04T17:33:17.173 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-04T17:33:17.173 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-07-04T17:33:17.174 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-07-04T17:33:17.174 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-04T17:33:17.174 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-04T17:33:17.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:33:17.174 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2024-07-04T17:33:17.228 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-04T17:33:17.228 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-04T17:33:17.228 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-07-04T17:33:17.228 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:33:17.228 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-07-04T17:33:17.228 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:33:17.228 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:33:17.228 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2024-07-04T17:33:17.229 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f0 2024-07-04T17:33:17.229 INFO:teuthology.orchestra.run.smithi042.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:33:17.229 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2024-07-04T17:33:17.229 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f1 2024-07-04T17:33:17.229 INFO:teuthology.orchestra.run.smithi042.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-04T17:33:17.229 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:33:17.229 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f0 2024-07-04T17:33:17.229 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-07-04T17:33:17.229 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 42173sec preferred_lft 42173sec 2024-07-04T17:33:17.229 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::ec4:7aff:febd:1440/64 scope link noprefixroute 2024-07-04T17:33:17.229 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:33:17.229 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:33:17.229 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:33:17.229 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f1 2024-07-04T17:33:17.230 INFO:teuthology.orchestra.run.smithi042.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-04T17:33:17.230 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 32:75:d6:f1:31:e1 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:33:17.230 INFO:teuthology.orchestra.run.smithi042.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-04T17:33:17.230 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:33:17.230 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::8839:a9ff:fe74:17f5/64 scope link 2024-07-04T17:33:17.230 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:33:17.230 INFO:teuthology.orchestra.run.smithi042.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-04T17:33:17.230 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether e6:ab:57:fb:ca:9e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:33:17.230 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::e4ab:57ff:fefb:ca9e/64 scope link 2024-07-04T17:33:17.230 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:33:17.231 INFO:teuthology.orchestra.run.smithi042.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-04T17:33:17.231 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 32:75:d6:f1:31:e1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:33:17.231 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::3075:d6ff:fef1:31e1/64 scope link 2024-07-04T17:33:17.231 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:33:17.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:33:17.231 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-04T17:33:17.307 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-07-04T17:33:17.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:33:17.312 DEBUG:teuthology.orchestra.run.smithi042:> route 2024-07-04T17:33:17.367 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2024-07-04T17:33:17.368 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-04T17:33:17.368 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-04T17:33:17.368 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-04T17:33:17.368 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-04T17:33:17.368 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-04T17:33:17.368 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-07-04T17:33:17.368 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-04T17:33:17.368 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-04T17:33:17.368 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-04T17:33:17.368 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-07-04T17:33:17.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-04T17:33:17.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:33:17.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:33:17.595 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-07-04T17:33:17.644 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-04T17:33:17.644 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2024-07-04T17:33:17.645 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-04T17:33:17.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:33:17.645 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.0 up 2024-07-04T17:33:17.732 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-07-04T17:33:17.732 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-04T17:33:17.732 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-04T17:33:17.789 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-04T17:33:17.789 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-04T17:33:17.790 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-04T17:33:17.842 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:33:17.931 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:33:17.993 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:33:17.993 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:33:17.993 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:33:17.993 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:33:17.994 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:33:17.994 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:33:17.994 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:33:17.994 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:33:17.994 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:33:17.994 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:33:17.994 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:33:17.994 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:33:17.994 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:33:17.994 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:33:17.994 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:33:17.994 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:33:17.995 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:33:17.995 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:33:17.995 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:33:17.995 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:33:17.995 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:33:17.995 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:33:17.995 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:33:17.995 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:33:17.995 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:33:17.995 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:33:17.995 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:33:17.995 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:33:17.995 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:33:17.996 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:33:17.996 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:33:17.996 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:33:17.996 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:33:17.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:33:17.997 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:33:18.052 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-07-04T17:33:18.052 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-07-04T17:33:18.057 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:33:18.126 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:33:18.168 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:33:18.168 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:33:18.168 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:33:18.168 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:33:18.168 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:33:18.168 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:33:18.169 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:33:18.169 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:33:18.169 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:33:18.169 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:33:18.169 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:33:18.169 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:33:18.169 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:33:18.169 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:33:18.169 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:33:18.169 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:33:18.169 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:33:18.169 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:33:18.169 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:33:18.169 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:33:18.170 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:33:18.170 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:33:18.170 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:33:18.170 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:33:18.170 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:33:18.170 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:33:18.170 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:33:18.170 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:33:18.170 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:33:18.170 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:33:18.170 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:33:18.170 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:33:18.170 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:33:18.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:33:18.171 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:33:18.309 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:2024-07-04T17:33:18.308+0000 7fa97c55a480 -1 init, newargv = 0x7fa968004580 newargc=15 2024-07-04T17:33:18.309 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[65641]: starting ceph client 2024-07-04T17:33:18.317 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[65641]: starting fuse 2024-07-04T17:33:19.360 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:33:19.456 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:33:19.513 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:33:19.513 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:33:19.513 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:33:19.513 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:33:19.513 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:33:19.513 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:33:19.513 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:33:19.513 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:33:19.514 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:33:19.514 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:33:19.514 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:33:19.514 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:33:19.514 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:33:19.514 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:33:19.514 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:33:19.514 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:33:19.514 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:33:19.514 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:33:19.514 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:33:19.514 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:33:19.514 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:33:19.515 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:33:19.515 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:33:19.515 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:33:19.515 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:33:19.515 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:33:19.515 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:33:19.515 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:33:19.515 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:33:19.515 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:33:19.515 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:33:19.515 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:33:19.516 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:33:19.516 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-07-04T17:33:19.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:33:19.517 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:33:19.570 INFO:teuthology.orchestra.run.smithi042.stdout:42 2024-07-04T17:33:19.570 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-07-04T17:33:19.570 DEBUG:teuthology.orchestra.run.smithi042:> sudo stdin-killer -- python3 -c ' 2024-07-04T17:33:19.570 DEBUG:teuthology.orchestra.run.smithi042:> import glob 2024-07-04T17:33:19.571 DEBUG:teuthology.orchestra.run.smithi042:> import re 2024-07-04T17:33:19.571 DEBUG:teuthology.orchestra.run.smithi042:> import os 2024-07-04T17:33:19.571 DEBUG:teuthology.orchestra.run.smithi042:> import subprocess 2024-07-04T17:33:19.571 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:33:19.571 DEBUG:teuthology.orchestra.run.smithi042:> def _find_admin_socket(client_name): 2024-07-04T17:33:19.571 DEBUG:teuthology.orchestra.run.smithi042:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-07-04T17:33:19.571 DEBUG:teuthology.orchestra.run.smithi042:> files = glob.glob(asok_path) 2024-07-04T17:33:19.571 DEBUG:teuthology.orchestra.run.smithi042:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-07-04T17:33:19.571 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:33:19.571 DEBUG:teuthology.orchestra.run.smithi042:> # Given a non-glob path, it better be there 2024-07-04T17:33:19.571 DEBUG:teuthology.orchestra.run.smithi042:> if "*" not in asok_path: 2024-07-04T17:33:19.571 DEBUG:teuthology.orchestra.run.smithi042:> assert(len(files) == 1) 2024-07-04T17:33:19.572 DEBUG:teuthology.orchestra.run.smithi042:> return files[0] 2024-07-04T17:33:19.572 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:33:19.572 DEBUG:teuthology.orchestra.run.smithi042:> for f in files: 2024-07-04T17:33:19.572 DEBUG:teuthology.orchestra.run.smithi042:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-07-04T17:33:19.572 DEBUG:teuthology.orchestra.run.smithi042:> if os.path.exists("/proc/{0}".format(pid)): 2024-07-04T17:33:19.572 DEBUG:teuthology.orchestra.run.smithi042:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-07-04T17:33:19.572 DEBUG:teuthology.orchestra.run.smithi042:> contents = proc_f.read() 2024-07-04T17:33:19.572 DEBUG:teuthology.orchestra.run.smithi042:> if mountpoint in contents: 2024-07-04T17:33:19.572 DEBUG:teuthology.orchestra.run.smithi042:> return f 2024-07-04T17:33:19.572 DEBUG:teuthology.orchestra.run.smithi042:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-07-04T17:33:19.572 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:33:19.572 DEBUG:teuthology.orchestra.run.smithi042:> print(_find_admin_socket("client.0")) 2024-07-04T17:33:19.572 DEBUG:teuthology.orchestra.run.smithi042:> ' 2024-07-04T17:33:19.750 INFO:teuthology.orchestra.run.smithi042.stdout:/var/run/ceph/ceph-client.0.65641.asok 2024-07-04T17:33:19.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:33:19.769 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.65641.asok 2024-07-04T17:33:19.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:33:19.769 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.65641.asok status 2024-07-04T17:33:20.079 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:33:20.080 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": { 2024-07-04T17:33:20.080 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_sha1": "9dba1e836b4756bf3a32be2a4936f371dff55008", 2024-07-04T17:33:20.080 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.0.0-4667-g9dba1e83 (9dba1e836b4756bf3a32be2a4936f371dff55008) squid (dev)", 2024-07-04T17:33:20.080 INFO:teuthology.orchestra.run.smithi042.stdout: "entity_id": "0", 2024-07-04T17:33:20.080 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": "smithi042", 2024-07-04T17:33:20.080 INFO:teuthology.orchestra.run.smithi042.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-07-04T17:33:20.080 INFO:teuthology.orchestra.run.smithi042.stdout: "pid": "65641", 2024-07-04T17:33:20.080 INFO:teuthology.orchestra.run.smithi042.stdout: "root": "/" 2024-07-04T17:33:20.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:20.080 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_count": 0, 2024-07-04T17:33:20.081 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_pinned_count": 0, 2024-07-04T17:33:20.081 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 5598, 2024-07-04T17:33:20.081 INFO:teuthology.orchestra.run.smithi042.stdout: "inst": { 2024-07-04T17:33:20.081 INFO:teuthology.orchestra.run.smithi042.stdout: "name": { 2024-07-04T17:33:20.081 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "client", 2024-07-04T17:33:20.081 INFO:teuthology.orchestra.run.smithi042.stdout: "num": 5598 2024-07-04T17:33:20.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:20.081 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-07-04T17:33:20.081 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-07-04T17:33:20.081 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-07-04T17:33:20.081 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1436416871 2024-07-04T17:33:20.081 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:20.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:20.082 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-07-04T17:33:20.082 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-07-04T17:33:20.082 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-07-04T17:33:20.082 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1436416871 2024-07-04T17:33:20.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:20.082 INFO:teuthology.orchestra.run.smithi042.stdout: "inst_str": "client.5598 192.168.0.1:0/1436416871", 2024-07-04T17:33:20.082 INFO:teuthology.orchestra.run.smithi042.stdout: "addr_str": "192.168.0.1:0/1436416871", 2024-07-04T17:33:20.082 INFO:teuthology.orchestra.run.smithi042.stdout: "inode_count": 1, 2024-07-04T17:33:20.082 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_epoch": 90, 2024-07-04T17:33:20.082 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch": 114, 2024-07-04T17:33:20.082 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch_barrier": 0, 2024-07-04T17:33:20.082 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklisted": false, 2024-07-04T17:33:20.082 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs" 2024-07-04T17:33:20.083 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:33:20.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:33:20.102 DEBUG:teuthology.orchestra.run.smithi042:> stat --file-system '--printf=%T 2024-07-04T17:33:20.102 DEBUG:teuthology.orchestra.run.smithi042:> ' -- /home/ubuntu/cephtest/mnt.0 2024-07-04T17:33:20.161 INFO:teuthology.orchestra.run.smithi042.stdout:fuseblk 2024-07-04T17:33:20.161 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-07-04T17:33:20.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:33:20.161 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-04T17:33:20.251 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-04T17:33:20.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.541+0000 7f8423600640 1 -- 172.21.15.42:0/4274326460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f841c0b2d20 msgr2=0x7f841c0b37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:20.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.541+0000 7f8423600640 1 --2- 172.21.15.42:0/4274326460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f841c0b2d20 0x7f841c0b37d0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f840c005fd0 tx=0x7f840c02a5f0 comp rx=0 tx=0).stop 2024-07-04T17:33:20.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.541+0000 7f8423600640 1 -- 172.21.15.42:0/4274326460 shutdown_connections 2024-07-04T17:33:20.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.541+0000 7f8423600640 1 --2- 172.21.15.42:0/4274326460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f841c0b2d20 0x7f841c0b37d0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:20.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.541+0000 7f8423600640 1 -- 172.21.15.42:0/4274326460 >> 172.21.15.42:0/4274326460 conn(0x7f841c0bf700 msgr2=0x7f841c0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:20.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.541+0000 7f8423600640 1 -- 172.21.15.42:0/4274326460 shutdown_connections 2024-07-04T17:33:20.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.541+0000 7f8423600640 1 -- 172.21.15.42:0/4274326460 wait complete. 2024-07-04T17:33:20.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.544+0000 7f8423600640 1 Processor -- start 2024-07-04T17:33:20.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.544+0000 7f8423600640 1 -- start start 2024-07-04T17:33:20.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.545+0000 7f8423600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f841c0b01f0 0x7f841c13d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:20.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.545+0000 7f8423600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f841c06cc20 con 0x7f841c0b01f0 2024-07-04T17:33:20.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.545+0000 7f8420c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f841c0b01f0 0x7f841c13d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:20.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.546+0000 7f8420c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f841c0b01f0 0x7f841c13d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58900/0 (socket says 172.21.15.42:58900) 2024-07-04T17:33:20.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.546+0000 7f8420c00640 1 -- 172.21.15.42:0/3572118428 learned_addr learned my addr 172.21.15.42:0/3572118428 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:20.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.546+0000 7f8420c00640 1 -- 172.21.15.42:0/3572118428 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841c0ab2d0 con 0x7f841c0b01f0 2024-07-04T17:33:20.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.546+0000 7f8420c00640 1 --2- 172.21.15.42:0/3572118428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f841c0b01f0 0x7f841c13d520 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f840c009850 tx=0x7f840c0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:20.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.547+0000 7f8419600640 1 -- 172.21.15.42:0/3572118428 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f840c03e030 con 0x7f841c0b01f0 2024-07-04T17:33:20.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.547+0000 7f8419600640 1 -- 172.21.15.42:0/3572118428 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f840c0035f0 con 0x7f841c0b01f0 2024-07-04T17:33:20.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.547+0000 7f8419600640 1 -- 172.21.15.42:0/3572118428 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f840c035040 con 0x7f841c0b01f0 2024-07-04T17:33:20.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.547+0000 7f8423600640 1 -- 172.21.15.42:0/3572118428 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f841c0a95e0 con 0x7f841c0b01f0 2024-07-04T17:33:20.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.547+0000 7f8423600640 1 -- 172.21.15.42:0/3572118428 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f841c1dcf80 con 0x7f841c0b01f0 2024-07-04T17:33:20.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.548+0000 7f8419600640 1 -- 172.21.15.42:0/3572118428 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7f840c041030 con 0x7f841c0b01f0 2024-07-04T17:33:20.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.548+0000 7f8423600640 1 -- 172.21.15.42:0/3572118428 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f841c06d2c0 con 0x7f841c0b01f0 2024-07-04T17:33:20.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.549+0000 7f8419600640 1 --2- 172.21.15.42:0/3572118428 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f83f803d540 0x7f83f803fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:20.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.549+0000 7f8419600640 1 -- 172.21.15.42:0/3572118428 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 3439+0+0 (secure 0 0 0) 0x7f840c03f070 con 0x7f841c0b01f0 2024-07-04T17:33:20.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.549+0000 7f8413e00640 1 --2- 172.21.15.42:0/3572118428 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f83f803d540 0x7f83f803fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:20.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.550+0000 7f8413e00640 1 --2- 172.21.15.42:0/3572118428 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f83f803d540 0x7f83f803fa50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8404001560 tx=0x7f84040079f0 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:20.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.554+0000 7f8419600640 1 -- 172.21.15.42:0/3572118428 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f840c04bd60 con 0x7f841c0b01f0 2024-07-04T17:33:20.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.804+0000 7f8423600640 1 -- 172.21.15.42:0/3572118428 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f841c0af6e0 con 0x7f841c0b01f0 2024-07-04T17:33:20.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.805+0000 7f8419600640 1 -- 172.21.15.42:0/3572118428 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v90) ==== 53+0+73 (secure 0 0 0) 0x7f840c030cc0 con 0x7f841c0b01f0 2024-07-04T17:33:20.806 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-04T17:33:20.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.807+0000 7f8423600640 1 -- 172.21.15.42:0/3572118428 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f83f803d540 msgr2=0x7f83f803fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:20.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.807+0000 7f8423600640 1 --2- 172.21.15.42:0/3572118428 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f83f803d540 0x7f83f803fa50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8404001560 tx=0x7f84040079f0 comp rx=0 tx=0).stop 2024-07-04T17:33:20.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.807+0000 7f8423600640 1 -- 172.21.15.42:0/3572118428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f841c0b01f0 msgr2=0x7f841c13d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:20.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.807+0000 7f8423600640 1 --2- 172.21.15.42:0/3572118428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f841c0b01f0 0x7f841c13d520 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f840c009850 tx=0x7f840c0049d0 comp rx=0 tx=0).stop 2024-07-04T17:33:20.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.808+0000 7f8423600640 1 -- 172.21.15.42:0/3572118428 shutdown_connections 2024-07-04T17:33:20.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.808+0000 7f8423600640 1 --2- 172.21.15.42:0/3572118428 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f83f803d540 0x7f83f803fa50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:20.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.808+0000 7f8423600640 1 --2- 172.21.15.42:0/3572118428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f841c0b01f0 0x7f841c13d520 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:20.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.808+0000 7f8423600640 1 -- 172.21.15.42:0/3572118428 >> 172.21.15.42:0/3572118428 conn(0x7f841c0bf700 msgr2=0x7f841c0be730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:20.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.808+0000 7f8423600640 1 -- 172.21.15.42:0/3572118428 shutdown_connections 2024-07-04T17:33:20.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:20.808+0000 7f8423600640 1 -- 172.21.15.42:0/3572118428 wait complete. 2024-07-04T17:33:20.835 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-07-04T17:33:20.835 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-04T17:33:20.836 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2024-07-04T17:33:20.836 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-07-04T17:33:20.836 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-07-04T17:33:20.836 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-07-04T17:33:20.836 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-07-04T17:33:20.836 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-04T17:33:20.836 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-07-04T17:33:20.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:33:20.836 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2024-07-04T17:33:20.888 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-04T17:33:20.889 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-04T17:33:20.889 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-07-04T17:33:20.889 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:33:20.889 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-07-04T17:33:20.889 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:33:20.890 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:33:20.890 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2024-07-04T17:33:20.890 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f0 2024-07-04T17:33:20.890 INFO:teuthology.orchestra.run.smithi042.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:33:20.890 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2024-07-04T17:33:20.890 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f1 2024-07-04T17:33:20.890 INFO:teuthology.orchestra.run.smithi042.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-04T17:33:20.890 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:33:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f0 2024-07-04T17:33:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-07-04T17:33:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 42169sec preferred_lft 42169sec 2024-07-04T17:33:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::ec4:7aff:febd:1440/64 scope link noprefixroute 2024-07-04T17:33:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:33:20.891 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:33:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:33:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f1 2024-07-04T17:33:20.891 INFO:teuthology.orchestra.run.smithi042.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-04T17:33:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 32:75:d6:f1:31:e1 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:33:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-04T17:33:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:33:20.891 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::8839:a9ff:fe74:17f5/64 scope link 2024-07-04T17:33:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:33:20.892 INFO:teuthology.orchestra.run.smithi042.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-04T17:33:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether e6:ab:57:fb:ca:9e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:33:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::e4ab:57ff:fefb:ca9e/64 scope link 2024-07-04T17:33:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:33:20.892 INFO:teuthology.orchestra.run.smithi042.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-04T17:33:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 32:75:d6:f1:31:e1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:33:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::3075:d6ff:fef1:31e1/64 scope link 2024-07-04T17:33:20.892 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:33:20.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:33:20.893 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-04T17:33:20.975 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-07-04T17:33:20.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:33:20.979 DEBUG:teuthology.orchestra.run.smithi042:> route 2024-07-04T17:33:21.039 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2024-07-04T17:33:21.039 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-04T17:33:21.039 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-04T17:33:21.039 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-04T17:33:21.039 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-04T17:33:21.040 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-04T17:33:21.041 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-07-04T17:33:21.041 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-04T17:33:21.041 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-04T17:33:21.041 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-04T17:33:21.041 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-07-04T17:33:21.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-04T17:33:21.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:33:21.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:33:21.268 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-07-04T17:33:21.321 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-04T17:33:21.321 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2024-07-04T17:33:21.321 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-07-04T17:33:21.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:33:21.322 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.1 up 2024-07-04T17:33:21.407 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-07-04T17:33:21.408 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-04T17:33:21.408 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-07-04T17:33:21.465 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-07-04T17:33:21.465 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-04T17:33:21.465 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-07-04T17:33:21.520 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:33:21.596 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:33:21.648 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:33:21.649 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:33:21.649 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:33:21.649 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:33:21.649 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:33:21.649 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:33:21.649 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:33:21.649 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:33:21.649 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:33:21.649 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:33:21.649 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:33:21.649 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:33:21.649 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:33:21.650 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:33:21.650 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:33:21.650 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:33:21.650 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:33:21.650 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:33:21.650 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:33:21.650 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:33:21.650 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:33:21.650 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:33:21.650 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:33:21.651 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:33:21.651 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:33:21.651 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:33:21.651 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:33:21.651 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:33:21.651 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:33:21.651 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:33:21.651 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:33:21.651 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:33:21.651 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:33:21.651 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-07-04T17:33:21.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:33:21.652 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:33:21.707 INFO:teuthology.orchestra.run.smithi042.stdout:42 2024-07-04T17:33:21.708 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-07-04T17:33:21.708 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-07-04T17:33:21.714 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:33:21.785 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:33:21.825 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:33:21.825 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:33:21.825 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:33:21.825 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:33:21.825 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:33:21.825 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:33:21.826 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:33:21.826 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:33:21.826 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:33:21.826 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:33:21.826 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:33:21.826 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:33:21.826 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:33:21.826 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:33:21.826 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:33:21.826 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:33:21.826 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:33:21.826 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:33:21.826 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:33:21.827 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:33:21.827 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:33:21.827 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:33:21.827 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:33:21.827 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:33:21.827 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:33:21.827 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:33:21.827 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:33:21.827 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:33:21.828 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:33:21.828 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:33:21.828 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:33:21.828 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:33:21.828 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:33:21.828 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-07-04T17:33:21.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:33:21.829 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:33:21.911 INFO:teuthology.orchestra.run.smithi042.stdout:42 2024-07-04T17:33:21.958 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi042.stderr:2024-07-04T17:33:21.957+0000 7f1ca8e93480 -1 init, newargv = 0x7f1c94004580 newargc=15 2024-07-04T17:33:21.958 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi042.stderr:ceph-fuse[66110]: starting ceph client 2024-07-04T17:33:21.965 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi042.stderr:ceph-fuse[66110]: starting fuse 2024-07-04T17:33:23.013 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:33:23.111 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:33:23.169 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:33:23.169 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:33:23.170 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:33:23.170 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:33:23.170 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:33:23.170 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:33:23.170 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:33:23.170 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:33:23.170 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:33:23.170 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:33:23.170 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:33:23.170 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:33:23.170 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:33:23.170 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:33:23.170 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:33:23.170 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:33:23.171 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:33:23.171 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:33:23.171 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:33:23.171 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:33:23.171 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:33:23.171 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:33:23.171 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:33:23.171 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:33:23.171 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:33:23.171 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:33:23.171 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:33:23.171 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:33:23.171 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:33:23.171 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:33:23.172 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:33:23.172 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:33:23.172 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:33:23.172 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-07-04T17:33:23.172 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.1 2024-07-04T17:33:23.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:33:23.173 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:33:23.227 INFO:teuthology.orchestra.run.smithi042.stdout:42 2024-07-04T17:33:23.227 INFO:teuthology.orchestra.run.smithi042.stdout:43 2024-07-04T17:33:23.228 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 43] 2024-07-04T17:33:23.228 DEBUG:teuthology.orchestra.run.smithi042:> sudo stdin-killer -- python3 -c ' 2024-07-04T17:33:23.228 DEBUG:teuthology.orchestra.run.smithi042:> import glob 2024-07-04T17:33:23.228 DEBUG:teuthology.orchestra.run.smithi042:> import re 2024-07-04T17:33:23.228 DEBUG:teuthology.orchestra.run.smithi042:> import os 2024-07-04T17:33:23.228 DEBUG:teuthology.orchestra.run.smithi042:> import subprocess 2024-07-04T17:33:23.228 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:33:23.228 DEBUG:teuthology.orchestra.run.smithi042:> def _find_admin_socket(client_name): 2024-07-04T17:33:23.228 DEBUG:teuthology.orchestra.run.smithi042:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-07-04T17:33:23.228 DEBUG:teuthology.orchestra.run.smithi042:> files = glob.glob(asok_path) 2024-07-04T17:33:23.228 DEBUG:teuthology.orchestra.run.smithi042:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-07-04T17:33:23.228 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:33:23.228 DEBUG:teuthology.orchestra.run.smithi042:> # Given a non-glob path, it better be there 2024-07-04T17:33:23.228 DEBUG:teuthology.orchestra.run.smithi042:> if "*" not in asok_path: 2024-07-04T17:33:23.229 DEBUG:teuthology.orchestra.run.smithi042:> assert(len(files) == 1) 2024-07-04T17:33:23.229 DEBUG:teuthology.orchestra.run.smithi042:> return files[0] 2024-07-04T17:33:23.229 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:33:23.229 DEBUG:teuthology.orchestra.run.smithi042:> for f in files: 2024-07-04T17:33:23.229 DEBUG:teuthology.orchestra.run.smithi042:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-07-04T17:33:23.229 DEBUG:teuthology.orchestra.run.smithi042:> if os.path.exists("/proc/{0}".format(pid)): 2024-07-04T17:33:23.229 DEBUG:teuthology.orchestra.run.smithi042:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-07-04T17:33:23.229 DEBUG:teuthology.orchestra.run.smithi042:> contents = proc_f.read() 2024-07-04T17:33:23.229 DEBUG:teuthology.orchestra.run.smithi042:> if mountpoint in contents: 2024-07-04T17:33:23.229 DEBUG:teuthology.orchestra.run.smithi042:> return f 2024-07-04T17:33:23.229 DEBUG:teuthology.orchestra.run.smithi042:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-07-04T17:33:23.229 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:33:23.229 DEBUG:teuthology.orchestra.run.smithi042:> print(_find_admin_socket("client.1")) 2024-07-04T17:33:23.229 DEBUG:teuthology.orchestra.run.smithi042:> ' 2024-07-04T17:33:23.403 INFO:teuthology.orchestra.run.smithi042.stdout:/var/run/ceph/ceph-client.1.66110.asok 2024-07-04T17:33:23.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:33:23.422 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.66110.asok 2024-07-04T17:33:23.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:33:23.422 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.66110.asok status 2024-07-04T17:33:23.739 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:33:23.739 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": { 2024-07-04T17:33:23.739 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_sha1": "9dba1e836b4756bf3a32be2a4936f371dff55008", 2024-07-04T17:33:23.740 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.0.0-4667-g9dba1e83 (9dba1e836b4756bf3a32be2a4936f371dff55008) squid (dev)", 2024-07-04T17:33:23.740 INFO:teuthology.orchestra.run.smithi042.stdout: "entity_id": "1", 2024-07-04T17:33:23.740 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": "smithi042", 2024-07-04T17:33:23.740 INFO:teuthology.orchestra.run.smithi042.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-07-04T17:33:23.740 INFO:teuthology.orchestra.run.smithi042.stdout: "pid": "66110", 2024-07-04T17:33:23.740 INFO:teuthology.orchestra.run.smithi042.stdout: "root": "/" 2024-07-04T17:33:23.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:23.740 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_count": 0, 2024-07-04T17:33:23.740 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_pinned_count": 0, 2024-07-04T17:33:23.740 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 5604, 2024-07-04T17:33:23.740 INFO:teuthology.orchestra.run.smithi042.stdout: "inst": { 2024-07-04T17:33:23.740 INFO:teuthology.orchestra.run.smithi042.stdout: "name": { 2024-07-04T17:33:23.740 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "client", 2024-07-04T17:33:23.741 INFO:teuthology.orchestra.run.smithi042.stdout: "num": 5604 2024-07-04T17:33:23.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:23.741 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-07-04T17:33:23.741 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-07-04T17:33:23.741 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.2:0", 2024-07-04T17:33:23.741 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1488568924 2024-07-04T17:33:23.741 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:23.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:23.741 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-07-04T17:33:23.741 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-07-04T17:33:23.741 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.2:0", 2024-07-04T17:33:23.741 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1488568924 2024-07-04T17:33:23.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:23.741 INFO:teuthology.orchestra.run.smithi042.stdout: "inst_str": "client.5604 192.168.0.2:0/1488568924", 2024-07-04T17:33:23.742 INFO:teuthology.orchestra.run.smithi042.stdout: "addr_str": "192.168.0.2:0/1488568924", 2024-07-04T17:33:23.742 INFO:teuthology.orchestra.run.smithi042.stdout: "inode_count": 1, 2024-07-04T17:33:23.742 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_epoch": 90, 2024-07-04T17:33:23.742 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch": 114, 2024-07-04T17:33:23.742 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch_barrier": 0, 2024-07-04T17:33:23.742 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklisted": false, 2024-07-04T17:33:23.742 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs" 2024-07-04T17:33:23.742 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:33:23.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:33:23.763 DEBUG:teuthology.orchestra.run.smithi042:> stat --file-system '--printf=%T 2024-07-04T17:33:23.763 DEBUG:teuthology.orchestra.run.smithi042:> ' -- /home/ubuntu/cephtest/mnt.1 2024-07-04T17:33:23.816 INFO:teuthology.orchestra.run.smithi042.stdout:fuseblk 2024-07-04T17:33:23.816 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-07-04T17:33:23.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:33:23.816 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-07-04T17:33:23.903 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-07-04T17:33:24.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:24.206+0000 7f85d3c00640 1 -- 172.21.15.42:0/940392120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85cc0b1100 msgr2=0x7f85cc13f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:24.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:24.206+0000 7f85d3c00640 1 --2- 172.21.15.42:0/940392120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85cc0b1100 0x7f85cc13f800 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f85b8009230 tx=0x7f85b802a480 comp rx=0 tx=0).stop 2024-07-04T17:33:24.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:24.206+0000 7f85d3c00640 1 -- 172.21.15.42:0/940392120 shutdown_connections 2024-07-04T17:33:24.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:24.206+0000 7f85d3c00640 1 --2- 172.21.15.42:0/940392120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85cc0b1100 0x7f85cc13f800 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:24.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:24.206+0000 7f85d3c00640 1 -- 172.21.15.42:0/940392120 >> 172.21.15.42:0/940392120 conn(0x7f85cc06c020 msgr2=0x7f85cc06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:24.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:24.206+0000 7f85d3c00640 1 -- 172.21.15.42:0/940392120 shutdown_connections 2024-07-04T17:33:24.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:24.206+0000 7f85d3c00640 1 -- 172.21.15.42:0/940392120 wait complete. 2024-07-04T17:33:24.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:24.208+0000 7f85d3c00640 1 Processor -- start 2024-07-04T17:33:24.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:24.208+0000 7f85d3c00640 1 -- start start 2024-07-04T17:33:24.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:24.210+0000 7f85d3c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85cc0a94d0 0x7f85cc141de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:24.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:24.210+0000 7f85d3c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc1401e0 con 0x7f85cc0a94d0 2024-07-04T17:33:24.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:24.210+0000 7f85d1200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85cc0a94d0 0x7f85cc141de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:24.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:24.210+0000 7f85d1200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85cc0a94d0 0x7f85cc141de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42778/0 (socket says 172.21.15.42:42778) 2024-07-04T17:33:24.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:24.210+0000 7f85d1200640 1 -- 172.21.15.42:0/2279914651 learned_addr learned my addr 172.21.15.42:0/2279914651 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:24.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:24.211+0000 7f85d1200640 1 -- 172.21.15.42:0/2279914651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85cc1d37b0 con 0x7f85cc0a94d0 2024-07-04T17:33:24.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:24.211+0000 7f85d1200640 1 --2- 172.21.15.42:0/2279914651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85cc0a94d0 0x7f85cc141de0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f85bc00c6b0 tx=0x7f85bc00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:24.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:24.211+0000 7f85c8c00640 1 -- 172.21.15.42:0/2279914651 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85bc007e10 con 0x7f85cc0a94d0 2024-07-04T17:33:24.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:24.211+0000 7f85c8c00640 1 -- 172.21.15.42:0/2279914651 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f85bc017070 con 0x7f85cc0a94d0 2024-07-04T17:33:24.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:24.211+0000 7f85d3c00640 1 -- 172.21.15.42:0/2279914651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85cc1d2c90 con 0x7f85cc0a94d0 2024-07-04T17:33:24.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:24.211+0000 7f85c8c00640 1 -- 172.21.15.42:0/2279914651 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85bc006b50 con 0x7f85cc0a94d0 2024-07-04T17:33:24.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:24.211+0000 7f85d3c00640 1 -- 172.21.15.42:0/2279914651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85cc1d3e30 con 0x7f85cc0a94d0 2024-07-04T17:33:24.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:24.213+0000 7f85d3c00640 1 -- 172.21.15.42:0/2279914651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85cc13f800 con 0x7f85cc0a94d0 2024-07-04T17:33:24.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:24.213+0000 7f85c8c00640 1 -- 172.21.15.42:0/2279914651 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7f85bc006cf0 con 0x7f85cc0a94d0 2024-07-04T17:33:24.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:24.213+0000 7f85c8c00640 1 --2- 172.21.15.42:0/2279914651 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f85ac03d4f0 0x7f85ac03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:24.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:24.214+0000 7f85c8c00640 1 -- 172.21.15.42:0/2279914651 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 3439+0+0 (secure 0 0 0) 0x7f85bc0540d0 con 0x7f85cc0a94d0 2024-07-04T17:33:24.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:24.214+0000 7f85cbe00640 1 --2- 172.21.15.42:0/2279914651 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f85ac03d4f0 0x7f85ac03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:24.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:24.215+0000 7f85cbe00640 1 --2- 172.21.15.42:0/2279914651 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f85ac03d4f0 0x7f85ac03fa00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f85b8034000 tx=0x7f85b8004730 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:24.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:24.219+0000 7f85c8c00640 1 -- 172.21.15.42:0/2279914651 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f85cc0b1510 con 0x7f85cc0a94d0 2024-07-04T17:33:24.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:24.471+0000 7f85d3c00640 1 -- 172.21.15.42:0/2279914651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f85cc0b42b0 con 0x7f85cc0a94d0 2024-07-04T17:33:25.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.348+0000 7f85c8c00640 1 -- 172.21.15.42:0/2279914651 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v91) ==== 136+0+0 (secure 0 0 0) 0x7f85bc0072e0 con 0x7f85cc0a94d0 2024-07-04T17:33:25.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.351+0000 7f85d3c00640 1 -- 172.21.15.42:0/2279914651 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f85ac03d4f0 msgr2=0x7f85ac03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:25.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.351+0000 7f85d3c00640 1 --2- 172.21.15.42:0/2279914651 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f85ac03d4f0 0x7f85ac03fa00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f85b8034000 tx=0x7f85b8004730 comp rx=0 tx=0).stop 2024-07-04T17:33:25.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.351+0000 7f85d3c00640 1 -- 172.21.15.42:0/2279914651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85cc0a94d0 msgr2=0x7f85cc141de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:25.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.351+0000 7f85d3c00640 1 --2- 172.21.15.42:0/2279914651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85cc0a94d0 0x7f85cc141de0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f85bc00c6b0 tx=0x7f85bc00cb70 comp rx=0 tx=0).stop 2024-07-04T17:33:25.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.352+0000 7f85d3c00640 1 -- 172.21.15.42:0/2279914651 shutdown_connections 2024-07-04T17:33:25.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.352+0000 7f85d3c00640 1 --2- 172.21.15.42:0/2279914651 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f85ac03d4f0 0x7f85ac03fa00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:25.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.352+0000 7f85d3c00640 1 --2- 172.21.15.42:0/2279914651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85cc0a94d0 0x7f85cc141de0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:25.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.352+0000 7f85d3c00640 1 -- 172.21.15.42:0/2279914651 >> 172.21.15.42:0/2279914651 conn(0x7f85cc06c020 msgr2=0x7f85cc075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:25.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.352+0000 7f85d3c00640 1 -- 172.21.15.42:0/2279914651 shutdown_connections 2024-07-04T17:33:25.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.352+0000 7f85d3c00640 1 -- 172.21.15.42:0/2279914651 wait complete. 2024-07-04T17:33:25.388 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-04T17:33:25.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.679+0000 7fc62da00640 1 -- 172.21.15.42:0/2870100942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6280b2d20 msgr2=0x7fc6280b37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:25.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.679+0000 7fc62da00640 1 --2- 172.21.15.42:0/2870100942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6280b2d20 0x7fc6280b37d0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fc618005fd0 tx=0x7fc61802a8b0 comp rx=0 tx=0).stop 2024-07-04T17:33:25.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.679+0000 7fc62da00640 1 -- 172.21.15.42:0/2870100942 shutdown_connections 2024-07-04T17:33:25.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.679+0000 7fc62da00640 1 --2- 172.21.15.42:0/2870100942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6280b2d20 0x7fc6280b37d0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:25.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.679+0000 7fc62da00640 1 -- 172.21.15.42:0/2870100942 >> 172.21.15.42:0/2870100942 conn(0x7fc6280bf700 msgr2=0x7fc6280af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:25.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.679+0000 7fc62da00640 1 -- 172.21.15.42:0/2870100942 shutdown_connections 2024-07-04T17:33:25.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.679+0000 7fc62da00640 1 -- 172.21.15.42:0/2870100942 wait complete. 2024-07-04T17:33:25.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.681+0000 7fc62da00640 1 Processor -- start 2024-07-04T17:33:25.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.682+0000 7fc62da00640 1 -- start start 2024-07-04T17:33:25.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.683+0000 7fc62da00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6280b01f0 0x7fc6281418f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:25.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.683+0000 7fc62da00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc62806cc20 con 0x7fc6280b01f0 2024-07-04T17:33:25.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.683+0000 7fc626a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6280b01f0 0x7fc6281418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:25.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.683+0000 7fc626a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6280b01f0 0x7fc6281418f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42802/0 (socket says 172.21.15.42:42802) 2024-07-04T17:33:25.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.683+0000 7fc626a00640 1 -- 172.21.15.42:0/1552504231 learned_addr learned my addr 172.21.15.42:0/1552504231 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:25.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.684+0000 7fc626a00640 1 -- 172.21.15.42:0/1552504231 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc628141410 con 0x7fc6280b01f0 2024-07-04T17:33:25.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.684+0000 7fc626a00640 1 --2- 172.21.15.42:0/1552504231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6280b01f0 0x7fc6281418f0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fc618009c50 tx=0x7fc618031670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:25.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.685+0000 7fc61ea00640 1 -- 172.21.15.42:0/1552504231 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6180318c0 con 0x7fc6280b01f0 2024-07-04T17:33:25.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.685+0000 7fc62da00640 1 -- 172.21.15.42:0/1552504231 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6281d7a40 con 0x7fc6280b01f0 2024-07-04T17:33:25.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.685+0000 7fc61ea00640 1 -- 172.21.15.42:0/1552504231 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc618031b50 con 0x7fc6280b01f0 2024-07-04T17:33:25.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.685+0000 7fc61ea00640 1 -- 172.21.15.42:0/1552504231 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc618033070 con 0x7fc6280b01f0 2024-07-04T17:33:25.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.685+0000 7fc62da00640 1 -- 172.21.15.42:0/1552504231 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc62813f000 con 0x7fc6280b01f0 2024-07-04T17:33:25.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.686+0000 7fc62da00640 1 -- 172.21.15.42:0/1552504231 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc62806d2c0 con 0x7fc6280b01f0 2024-07-04T17:33:25.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.686+0000 7fc61ea00640 1 -- 172.21.15.42:0/1552504231 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7fc618033210 con 0x7fc6280b01f0 2024-07-04T17:33:25.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.687+0000 7fc61ea00640 1 --2- 172.21.15.42:0/1552504231 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fc60803d4a0 0x7fc60803f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:25.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.687+0000 7fc61ea00640 1 -- 172.21.15.42:0/1552504231 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 3439+0+0 (secure 0 0 0) 0x7fc618072f40 con 0x7fc6280b01f0 2024-07-04T17:33:25.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.687+0000 7fc626000640 1 --2- 172.21.15.42:0/1552504231 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fc60803d4a0 0x7fc60803f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:25.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.688+0000 7fc626000640 1 --2- 172.21.15.42:0/1552504231 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fc60803d4a0 0x7fc60803f9b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc614007fd0 tx=0x7fc614007920 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:25.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.692+0000 7fc61ea00640 1 -- 172.21.15.42:0/1552504231 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fc61803f030 con 0x7fc6280b01f0 2024-07-04T17:33:25.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.904+0000 7fc62da00640 1 -- 172.21.15.42:0/1552504231 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc6280af6e0 con 0x7fc6280b01f0 2024-07-04T17:33:25.906 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:33:25.907 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-07-04T17:33:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 1, 2024-07-04T17:33:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": ".mgr" 2024-07-04T17:33:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 20, 2024-07-04T17:33:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": "cephfs_metadata" 2024-07-04T17:33:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 21, 2024-07-04T17:33:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": "cephfs_data" 2024-07-04T17:33:25.907 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:25.907 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-07-04T17:33:25.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.905+0000 7fc61ea00640 1 -- 172.21.15.42:0/1552504231 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v114) ==== 84+0+209 (secure 0 0 0) 0x7fc618033500 con 0x7fc6280b01f0 2024-07-04T17:33:25.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.908+0000 7fc62da00640 1 -- 172.21.15.42:0/1552504231 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fc60803d4a0 msgr2=0x7fc60803f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:25.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.908+0000 7fc62da00640 1 --2- 172.21.15.42:0/1552504231 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fc60803d4a0 0x7fc60803f9b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc614007fd0 tx=0x7fc614007920 comp rx=0 tx=0).stop 2024-07-04T17:33:25.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.908+0000 7fc62da00640 1 -- 172.21.15.42:0/1552504231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6280b01f0 msgr2=0x7fc6281418f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:25.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.908+0000 7fc62da00640 1 --2- 172.21.15.42:0/1552504231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6280b01f0 0x7fc6281418f0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fc618009c50 tx=0x7fc618031670 comp rx=0 tx=0).stop 2024-07-04T17:33:25.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.908+0000 7fc62da00640 1 -- 172.21.15.42:0/1552504231 shutdown_connections 2024-07-04T17:33:25.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.908+0000 7fc62da00640 1 --2- 172.21.15.42:0/1552504231 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fc60803d4a0 0x7fc60803f9b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:25.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.908+0000 7fc62da00640 1 --2- 172.21.15.42:0/1552504231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6280b01f0 0x7fc6281418f0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:25.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.908+0000 7fc62da00640 1 -- 172.21.15.42:0/1552504231 >> 172.21.15.42:0/1552504231 conn(0x7fc6280bf700 msgr2=0x7fc6280bc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:25.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.909+0000 7fc62da00640 1 -- 172.21.15.42:0/1552504231 shutdown_connections 2024-07-04T17:33:25.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:25.909+0000 7fc62da00640 1 -- 172.21.15.42:0/1552504231 wait complete. 2024-07-04T17:33:25.935 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'backup_fs' 2024-07-04T17:33:25.935 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_metadata --pg_num_min 64 2024-07-04T17:33:26.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:26.234+0000 7fb464400640 1 -- 172.21.15.42:0/407833381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb45c0aaa50 msgr2=0x7fb45c0a9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:26.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:26.234+0000 7fb464400640 1 --2- 172.21.15.42:0/407833381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb45c0aaa50 0x7fb45c0a9f30 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fb4480097b0 tx=0x7fb44802a530 comp rx=0 tx=0).stop 2024-07-04T17:33:26.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:26.234+0000 7fb464400640 1 -- 172.21.15.42:0/407833381 shutdown_connections 2024-07-04T17:33:26.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:26.234+0000 7fb464400640 1 --2- 172.21.15.42:0/407833381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb45c0aaa50 0x7fb45c0a9f30 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:26.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:26.234+0000 7fb464400640 1 -- 172.21.15.42:0/407833381 >> 172.21.15.42:0/407833381 conn(0x7fb45c0bf700 msgr2=0x7fb45c0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:26.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:26.234+0000 7fb464400640 1 -- 172.21.15.42:0/407833381 shutdown_connections 2024-07-04T17:33:26.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:26.235+0000 7fb464400640 1 -- 172.21.15.42:0/407833381 wait complete. 2024-07-04T17:33:26.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:26.237+0000 7fb464400640 1 Processor -- start 2024-07-04T17:33:26.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:26.237+0000 7fb464400640 1 -- start start 2024-07-04T17:33:26.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:26.238+0000 7fb464400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb45c0aaa50 0x7fb45c1d84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:26.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:26.238+0000 7fb464400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c06c020 con 0x7fb45c0aaa50 2024-07-04T17:33:26.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:26.238+0000 7fb461a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb45c0aaa50 0x7fb45c1d84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:26.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:26.239+0000 7fb461a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb45c0aaa50 0x7fb45c1d84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42820/0 (socket says 172.21.15.42:42820) 2024-07-04T17:33:26.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:26.239+0000 7fb461a00640 1 -- 172.21.15.42:0/2989645754 learned_addr learned my addr 172.21.15.42:0/2989645754 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:26.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:26.239+0000 7fb461a00640 1 -- 172.21.15.42:0/2989645754 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb45c1e1330 con 0x7fb45c0aaa50 2024-07-04T17:33:26.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:26.239+0000 7fb461a00640 1 --2- 172.21.15.42:0/2989645754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb45c0aaa50 0x7fb45c1d84b0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fb45000c3b0 tx=0x7fb45000c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:26.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:26.239+0000 7fb459600640 1 -- 172.21.15.42:0/2989645754 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb450011780 con 0x7fb45c0aaa50 2024-07-04T17:33:26.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:26.240+0000 7fb459600640 1 -- 172.21.15.42:0/2989645754 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb450017070 con 0x7fb45c0aaa50 2024-07-04T17:33:26.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:26.240+0000 7fb464400640 1 -- 172.21.15.42:0/2989645754 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb45c0ad6a0 con 0x7fb45c0aaa50 2024-07-04T17:33:26.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:26.240+0000 7fb459600640 1 -- 172.21.15.42:0/2989645754 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb450011c00 con 0x7fb45c0aaa50 2024-07-04T17:33:26.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:26.240+0000 7fb464400640 1 -- 172.21.15.42:0/2989645754 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb45c136840 con 0x7fb45c0aaa50 2024-07-04T17:33:26.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:26.241+0000 7fb464400640 1 -- 172.21.15.42:0/2989645754 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb45c0a9b10 con 0x7fb45c0aaa50 2024-07-04T17:33:26.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:26.241+0000 7fb459600640 1 -- 172.21.15.42:0/2989645754 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7fb45001c460 con 0x7fb45c0aaa50 2024-07-04T17:33:26.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:26.242+0000 7fb459600640 1 --2- 172.21.15.42:0/2989645754 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fb43c03d4a0 0x7fb43c03f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:26.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:26.242+0000 7fb459600640 1 -- 172.21.15.42:0/2989645754 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 3439+0+0 (secure 0 0 0) 0x7fb450051550 con 0x7fb45c0aaa50 2024-07-04T17:33:26.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:26.242+0000 7fb461000640 1 --2- 172.21.15.42:0/2989645754 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fb43c03d4a0 0x7fb43c03f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:26.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:26.243+0000 7fb461000640 1 --2- 172.21.15.42:0/2989645754 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fb43c03d4a0 0x7fb43c03f9b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb448034000 tx=0x7fb448004730 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:26.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:26.247+0000 7fb459600640 1 -- 172.21.15.42:0/2989645754 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fb45c0a9b10 con 0x7fb45c0aaa50 2024-07-04T17:33:26.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:26.456+0000 7fb464400640 1 -- 172.21.15.42:0/2989645754 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64} v 0) -- 0x7fb45c06c710 con 0x7fb45c0aaa50 2024-07-04T17:33:27.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.363+0000 7fb459600640 1 -- 172.21.15.42:0/2989645754 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64}]=0 pool 'backup_fs_metadata' created v115) ==== 144+0+0 (secure 0 0 0) 0x7fb450011e20 con 0x7fb45c0aaa50 2024-07-04T17:33:27.364 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'backup_fs_metadata' created 2024-07-04T17:33:27.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.367+0000 7fb464400640 1 -- 172.21.15.42:0/2989645754 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fb43c03d4a0 msgr2=0x7fb43c03f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:27.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.367+0000 7fb464400640 1 --2- 172.21.15.42:0/2989645754 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fb43c03d4a0 0x7fb43c03f9b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb448034000 tx=0x7fb448004730 comp rx=0 tx=0).stop 2024-07-04T17:33:27.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.367+0000 7fb464400640 1 -- 172.21.15.42:0/2989645754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb45c0aaa50 msgr2=0x7fb45c1d84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:27.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.367+0000 7fb464400640 1 --2- 172.21.15.42:0/2989645754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb45c0aaa50 0x7fb45c1d84b0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fb45000c3b0 tx=0x7fb45000c870 comp rx=0 tx=0).stop 2024-07-04T17:33:27.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.368+0000 7fb464400640 1 -- 172.21.15.42:0/2989645754 shutdown_connections 2024-07-04T17:33:27.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.368+0000 7fb464400640 1 --2- 172.21.15.42:0/2989645754 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fb43c03d4a0 0x7fb43c03f9b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:27.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.368+0000 7fb464400640 1 --2- 172.21.15.42:0/2989645754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb45c0aaa50 0x7fb45c1d84b0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:27.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.368+0000 7fb464400640 1 -- 172.21.15.42:0/2989645754 >> 172.21.15.42:0/2989645754 conn(0x7fb45c0bf700 msgr2=0x7fb45c0be150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:27.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.369+0000 7fb464400640 1 -- 172.21.15.42:0/2989645754 shutdown_connections 2024-07-04T17:33:27.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.369+0000 7fb464400640 1 -- 172.21.15.42:0/2989645754 wait complete. 2024-07-04T17:33:27.412 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-04T17:33:27.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.683+0000 7f610b200640 1 -- 172.21.15.42:0/2265846077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f610c070700 msgr2=0x7f610c1416d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:27.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.683+0000 7f610b200640 1 --2- 172.21.15.42:0/2265846077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f610c070700 0x7f610c1416d0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f60f40097b0 tx=0x7f60f402a540 comp rx=0 tx=0).stop 2024-07-04T17:33:27.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.683+0000 7f610b200640 1 -- 172.21.15.42:0/2265846077 shutdown_connections 2024-07-04T17:33:27.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.683+0000 7f610b200640 1 --2- 172.21.15.42:0/2265846077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f610c070700 0x7f610c1416d0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:27.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.683+0000 7f610b200640 1 -- 172.21.15.42:0/2265846077 >> 172.21.15.42:0/2265846077 conn(0x7f610c06c020 msgr2=0x7f610c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:27.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.683+0000 7f610b200640 1 -- 172.21.15.42:0/2265846077 shutdown_connections 2024-07-04T17:33:27.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.683+0000 7f610b200640 1 -- 172.21.15.42:0/2265846077 wait complete. 2024-07-04T17:33:27.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.686+0000 7f610b200640 1 Processor -- start 2024-07-04T17:33:27.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.686+0000 7f610b200640 1 -- start start 2024-07-04T17:33:27.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.688+0000 7f610b200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f610c070700 0x7f610c1d6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:27.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.688+0000 7f610b200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f610c0738f0 con 0x7f610c070700 2024-07-04T17:33:27.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.688+0000 7f6109e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f610c070700 0x7f610c1d6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:27.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.688+0000 7f6109e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f610c070700 0x7f610c1d6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42834/0 (socket says 172.21.15.42:42834) 2024-07-04T17:33:27.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.688+0000 7f6109e00640 1 -- 172.21.15.42:0/3220976032 learned_addr learned my addr 172.21.15.42:0/3220976032 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:27.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.689+0000 7f6109e00640 1 -- 172.21.15.42:0/3220976032 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f610c13b1d0 con 0x7f610c070700 2024-07-04T17:33:27.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.689+0000 7f6109e00640 1 --2- 172.21.15.42:0/3220976032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f610c070700 0x7f610c1d6f60 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f60fc00c380 tx=0x7f60fc00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:27.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.689+0000 7f6102000640 1 -- 172.21.15.42:0/3220976032 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60fc0116c0 con 0x7f610c070700 2024-07-04T17:33:27.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.690+0000 7f6102000640 1 -- 172.21.15.42:0/3220976032 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60fc017070 con 0x7f610c070700 2024-07-04T17:33:27.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.690+0000 7f610b200640 1 -- 172.21.15.42:0/3220976032 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f610c0aeff0 con 0x7f610c070700 2024-07-04T17:33:27.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.690+0000 7f6102000640 1 -- 172.21.15.42:0/3220976032 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60fc011b20 con 0x7f610c070700 2024-07-04T17:33:27.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.690+0000 7f610b200640 1 -- 172.21.15.42:0/3220976032 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f610c1e1de0 con 0x7f610c070700 2024-07-04T17:33:27.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.691+0000 7f610b200640 1 -- 172.21.15.42:0/3220976032 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f610c074000 con 0x7f610c070700 2024-07-04T17:33:27.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.691+0000 7f6102000640 1 -- 172.21.15.42:0/3220976032 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7f60fc01c460 con 0x7f610c070700 2024-07-04T17:33:27.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.692+0000 7f6102000640 1 --2- 172.21.15.42:0/3220976032 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f60e803d4a0 0x7f60e803f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:27.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.692+0000 7f6102000640 1 -- 172.21.15.42:0/3220976032 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 3812+0+0 (secure 0 0 0) 0x7f60fc051730 con 0x7f610c070700 2024-07-04T17:33:27.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.692+0000 7f6103e00640 1 --2- 172.21.15.42:0/3220976032 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f60e803d4a0 0x7f60e803f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:27.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.693+0000 7f6103e00640 1 --2- 172.21.15.42:0/3220976032 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f60e803d4a0 0x7f60e803f9b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f60f4034000 tx=0x7f60f4004730 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:27.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.698+0000 7f6102000640 1 -- 172.21.15.42:0/3220976032 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f610c13b1d0 con 0x7f610c070700 2024-07-04T17:33:27.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:27.911+0000 7f610b200640 1 -- 172.21.15.42:0/3220976032 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f610c1408e0 con 0x7f610c070700 2024-07-04T17:33:28.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.367+0000 7f6102000640 1 -- 172.21.15.42:0/3220976032 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'backup_fs_data' created v116) ==== 176+0+0 (secure 0 0 0) 0x7f60fc011cc0 con 0x7f610c070700 2024-07-04T17:33:28.369 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'backup_fs_data' created 2024-07-04T17:33:28.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.372+0000 7f610b200640 1 -- 172.21.15.42:0/3220976032 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f60e803d4a0 msgr2=0x7f60e803f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:28.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.372+0000 7f610b200640 1 --2- 172.21.15.42:0/3220976032 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f60e803d4a0 0x7f60e803f9b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f60f4034000 tx=0x7f60f4004730 comp rx=0 tx=0).stop 2024-07-04T17:33:28.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.372+0000 7f610b200640 1 -- 172.21.15.42:0/3220976032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f610c070700 msgr2=0x7f610c1d6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:28.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.372+0000 7f610b200640 1 --2- 172.21.15.42:0/3220976032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f610c070700 0x7f610c1d6f60 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f60fc00c380 tx=0x7f60fc00c840 comp rx=0 tx=0).stop 2024-07-04T17:33:28.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.372+0000 7f610b200640 1 -- 172.21.15.42:0/3220976032 shutdown_connections 2024-07-04T17:33:28.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.372+0000 7f610b200640 1 --2- 172.21.15.42:0/3220976032 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f60e803d4a0 0x7f60e803f9b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:28.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.372+0000 7f610b200640 1 --2- 172.21.15.42:0/3220976032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f610c070700 0x7f610c1d6f60 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:28.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.372+0000 7f610b200640 1 -- 172.21.15.42:0/3220976032 >> 172.21.15.42:0/3220976032 conn(0x7f610c06c020 msgr2=0x7f610c13d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:28.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.373+0000 7f610b200640 1 -- 172.21.15.42:0/3220976032 shutdown_connections 2024-07-04T17:33:28.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.373+0000 7f610b200640 1 -- 172.21.15.42:0/3220976032 wait complete. 2024-07-04T17:33:28.419 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new backup_fs backup_fs_metadata backup_fs_data 2024-07-04T17:33:28.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.683+0000 7f2f1e200640 1 -- 172.21.15.42:0/817838890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f180b0390 msgr2=0x7f2f18149cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:28.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.683+0000 7f2f1e200640 1 --2- 172.21.15.42:0/817838890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f180b0390 0x7f2f18149cb0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f2f04005fd0 tx=0x7f2f0402b010 comp rx=0 tx=0).stop 2024-07-04T17:33:28.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.683+0000 7f2f1e200640 1 -- 172.21.15.42:0/817838890 shutdown_connections 2024-07-04T17:33:28.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.683+0000 7f2f1e200640 1 --2- 172.21.15.42:0/817838890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f180b0390 0x7f2f18149cb0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:28.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.683+0000 7f2f1e200640 1 -- 172.21.15.42:0/817838890 >> 172.21.15.42:0/817838890 conn(0x7f2f180bf4d0 msgr2=0x7f2f180af870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:28.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.683+0000 7f2f1e200640 1 -- 172.21.15.42:0/817838890 shutdown_connections 2024-07-04T17:33:28.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.683+0000 7f2f1e200640 1 -- 172.21.15.42:0/817838890 wait complete. 2024-07-04T17:33:28.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.686+0000 7f2f1e200640 1 Processor -- start 2024-07-04T17:33:28.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.686+0000 7f2f1e200640 1 -- start start 2024-07-04T17:33:28.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.687+0000 7f2f1e200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f180aff80 0x7f2f18072bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:28.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.687+0000 7f2f1e200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2f180b25d0 con 0x7f2f180aff80 2024-07-04T17:33:28.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.687+0000 7f2f17400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f180aff80 0x7f2f18072bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:28.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.687+0000 7f2f17400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f180aff80 0x7f2f18072bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42848/0 (socket says 172.21.15.42:42848) 2024-07-04T17:33:28.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.687+0000 7f2f17400640 1 -- 172.21.15.42:0/914881980 learned_addr learned my addr 172.21.15.42:0/914881980 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:28.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.688+0000 7f2f17400640 1 -- 172.21.15.42:0/914881980 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f181e79e0 con 0x7f2f180aff80 2024-07-04T17:33:28.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.688+0000 7f2f17400640 1 --2- 172.21.15.42:0/914881980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f180aff80 0x7f2f18072bb0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f2f0800c380 tx=0x7f2f0800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:28.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.689+0000 7f2f0f400640 1 -- 172.21.15.42:0/914881980 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f08017070 con 0x7f2f180aff80 2024-07-04T17:33:28.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.689+0000 7f2f0f400640 1 -- 172.21.15.42:0/914881980 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f080052f0 con 0x7f2f180aff80 2024-07-04T17:33:28.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.689+0000 7f2f0f400640 1 -- 172.21.15.42:0/914881980 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f080035f0 con 0x7f2f180aff80 2024-07-04T17:33:28.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.689+0000 7f2f1e200640 1 -- 172.21.15.42:0/914881980 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f180732d0 con 0x7f2f180aff80 2024-07-04T17:33:28.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.689+0000 7f2f1e200640 1 -- 172.21.15.42:0/914881980 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f18072650 con 0x7f2f180aff80 2024-07-04T17:33:28.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.690+0000 7f2f1e200640 1 -- 172.21.15.42:0/914881980 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f181e7bb0 con 0x7f2f180aff80 2024-07-04T17:33:28.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.691+0000 7f2f0f400640 1 -- 172.21.15.42:0/914881980 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7f2f0801a070 con 0x7f2f180aff80 2024-07-04T17:33:28.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.691+0000 7f2f0f400640 1 --2- 172.21.15.42:0/914881980 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f2ef803d4f0 0x7f2ef803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:28.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.691+0000 7f2f0f400640 1 -- 172.21.15.42:0/914881980 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 4197+0+0 (secure 0 0 0) 0x7f2f08050f40 con 0x7f2f180aff80 2024-07-04T17:33:28.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.692+0000 7f2f16a00640 1 --2- 172.21.15.42:0/914881980 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f2ef803d4f0 0x7f2ef803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:28.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.693+0000 7f2f16a00640 1 --2- 172.21.15.42:0/914881980 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f2ef803d4f0 0x7f2ef803fa00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2f04035000 tx=0x7f2f04004ab0 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:28.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.697+0000 7f2f0f400640 1 -- 172.21.15.42:0/914881980 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f2f08016170 con 0x7f2f180aff80 2024-07-04T17:33:28.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:28.942+0000 7f2f1e200640 1 -- 172.21.15.42:0/914881980 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"} v 0) -- 0x7f2f181d31c0 con 0x7f2f180aff80 2024-07-04T17:33:29.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.394+0000 7f2f0f400640 1 -- 172.21.15.42:0/914881980 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"}]=0 new fs with metadata pool 22 and data pool 23 v92) ==== 183+0+0 (secure 0 0 0) 0x7f2f0804f900 con 0x7f2f180aff80 2024-07-04T17:33:29.401 INFO:teuthology.orchestra.run.smithi042.stderr:new fs with metadata pool 22 and data pool 23 2024-07-04T17:33:29.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.409+0000 7f2f1e200640 1 -- 172.21.15.42:0/914881980 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f2ef803d4f0 msgr2=0x7f2ef803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:29.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.409+0000 7f2f1e200640 1 --2- 172.21.15.42:0/914881980 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f2ef803d4f0 0x7f2ef803fa00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2f04035000 tx=0x7f2f04004ab0 comp rx=0 tx=0).stop 2024-07-04T17:33:29.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.409+0000 7f2f1e200640 1 -- 172.21.15.42:0/914881980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f180aff80 msgr2=0x7f2f18072bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:29.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.409+0000 7f2f1e200640 1 --2- 172.21.15.42:0/914881980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f180aff80 0x7f2f18072bb0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f2f0800c380 tx=0x7f2f0800c840 comp rx=0 tx=0).stop 2024-07-04T17:33:29.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.409+0000 7f2f1e200640 1 -- 172.21.15.42:0/914881980 shutdown_connections 2024-07-04T17:33:29.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.409+0000 7f2f1e200640 1 --2- 172.21.15.42:0/914881980 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f2ef803d4f0 0x7f2ef803fa00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:29.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.409+0000 7f2f1e200640 1 --2- 172.21.15.42:0/914881980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f180aff80 0x7f2f18072bb0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:29.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.409+0000 7f2f1e200640 1 -- 172.21.15.42:0/914881980 >> 172.21.15.42:0/914881980 conn(0x7f2f180bf4d0 msgr2=0x7f2f180bad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:29.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.411+0000 7f2f1e200640 1 -- 172.21.15.42:0/914881980 shutdown_connections 2024-07-04T17:33:29.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.411+0000 7f2f1e200640 1 -- 172.21.15.42:0/914881980 wait complete. 2024-07-04T17:33:29.465 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:33:29.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.719+0000 7f257ae00640 1 -- 172.21.15.42:0/483165448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25740b2d20 msgr2=0x7f25740b37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:29.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.719+0000 7f257ae00640 1 --2- 172.21.15.42:0/483165448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25740b2d20 0x7f25740b37d0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f25600097b0 tx=0x7f256002a530 comp rx=0 tx=0).stop 2024-07-04T17:33:29.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.719+0000 7f257ae00640 1 -- 172.21.15.42:0/483165448 shutdown_connections 2024-07-04T17:33:29.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.719+0000 7f257ae00640 1 --2- 172.21.15.42:0/483165448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25740b2d20 0x7f25740b37d0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:29.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.720+0000 7f257ae00640 1 -- 172.21.15.42:0/483165448 >> 172.21.15.42:0/483165448 conn(0x7f25740bf700 msgr2=0x7f25740af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:29.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.720+0000 7f257ae00640 1 -- 172.21.15.42:0/483165448 shutdown_connections 2024-07-04T17:33:29.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.720+0000 7f257ae00640 1 -- 172.21.15.42:0/483165448 wait complete. 2024-07-04T17:33:29.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.722+0000 7f257ae00640 1 Processor -- start 2024-07-04T17:33:29.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.722+0000 7f257ae00640 1 -- start start 2024-07-04T17:33:29.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.723+0000 7f257ae00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25740b01f0 0x7f25740ac5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:29.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.723+0000 7f257ae00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f257406cc20 con 0x7f25740b01f0 2024-07-04T17:33:29.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.723+0000 7f2573e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25740b01f0 0x7f25740ac5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:29.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.723+0000 7f2573e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25740b01f0 0x7f25740ac5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42858/0 (socket says 172.21.15.42:42858) 2024-07-04T17:33:29.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.723+0000 7f2573e00640 1 -- 172.21.15.42:0/1070170413 learned_addr learned my addr 172.21.15.42:0/1070170413 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:29.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.724+0000 7f2573e00640 1 -- 172.21.15.42:0/1070170413 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25740adad0 con 0x7f25740b01f0 2024-07-04T17:33:29.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.724+0000 7f2573e00640 1 --2- 172.21.15.42:0/1070170413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25740b01f0 0x7f25740ac5f0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f25600098d0 tx=0x7f2560033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:29.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.724+0000 7f256be00640 1 -- 172.21.15.42:0/1070170413 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2560031880 con 0x7f25740b01f0 2024-07-04T17:33:29.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.724+0000 7f256be00640 1 -- 172.21.15.42:0/1070170413 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2560031b10 con 0x7f25740b01f0 2024-07-04T17:33:29.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.724+0000 7f256be00640 1 -- 172.21.15.42:0/1070170413 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2560031df0 con 0x7f25740b01f0 2024-07-04T17:33:29.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.724+0000 7f257ae00640 1 -- 172.21.15.42:0/1070170413 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25740aa6c0 con 0x7f25740b01f0 2024-07-04T17:33:29.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.725+0000 7f257ae00640 1 -- 172.21.15.42:0/1070170413 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25741e0fc0 con 0x7f25740b01f0 2024-07-04T17:33:29.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.726+0000 7f257ae00640 1 -- 172.21.15.42:0/1070170413 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f257406d2c0 con 0x7f25740b01f0 2024-07-04T17:33:29.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.726+0000 7f256be00640 1 -- 172.21.15.42:0/1070170413 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7f2560033070 con 0x7f25740b01f0 2024-07-04T17:33:29.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.726+0000 7f256be00640 1 --2- 172.21.15.42:0/1070170413 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f255403d4f0 0x7f255403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:29.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.726+0000 7f256be00640 1 -- 172.21.15.42:0/1070170413 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 4303+0+0 (secure 0 0 0) 0x7f2560072ff0 con 0x7f25740b01f0 2024-07-04T17:33:29.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.727+0000 7f2573400640 1 --2- 172.21.15.42:0/1070170413 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f255403d4f0 0x7f255403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:29.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.727+0000 7f2573400640 1 --2- 172.21.15.42:0/1070170413 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f255403d4f0 0x7f255403fa00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2564007fd0 tx=0x7f2564007920 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:29.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.732+0000 7f256be00640 1 -- 172.21.15.42:0/1070170413 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f256003f030 con 0x7f25740b01f0 2024-07-04T17:33:29.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.937+0000 7f257ae00640 1 -- 172.21.15.42:0/1070170413 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25740af6e0 con 0x7f25740b01f0 2024-07-04T17:33:29.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.939+0000 7f256be00640 1 -- 172.21.15.42:0/1070170413 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) ==== 74+0+14471 (secure 0 0 0) 0x7f256003a3a0 con 0x7f25740b01f0 2024-07-04T17:33:29.941 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:33:29.941 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":117,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:33:28.944405+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:33:06.769774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-07-04T17:33:07.613572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:33:26.476351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:33:27.929016+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1699999570846558,"score_stable":1.1699999570846558,"optimal_score":1,"raw_score_acting":1.1699999570846558,"raw_score_stable":1.1699999570846558,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:33:29.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.941+0000 7f257ae00640 1 -- 172.21.15.42:0/1070170413 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f255403d4f0 msgr2=0x7f255403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:29.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.941+0000 7f257ae00640 1 --2- 172.21.15.42:0/1070170413 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f255403d4f0 0x7f255403fa00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2564007fd0 tx=0x7f2564007920 comp rx=0 tx=0).stop 2024-07-04T17:33:29.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.942+0000 7f257ae00640 1 -- 172.21.15.42:0/1070170413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25740b01f0 msgr2=0x7f25740ac5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:29.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.942+0000 7f257ae00640 1 --2- 172.21.15.42:0/1070170413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25740b01f0 0x7f25740ac5f0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f25600098d0 tx=0x7f2560033ad0 comp rx=0 tx=0).stop 2024-07-04T17:33:29.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.942+0000 7f257ae00640 1 -- 172.21.15.42:0/1070170413 shutdown_connections 2024-07-04T17:33:29.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.942+0000 7f257ae00640 1 --2- 172.21.15.42:0/1070170413 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f255403d4f0 0x7f255403fa00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:29.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.942+0000 7f257ae00640 1 --2- 172.21.15.42:0/1070170413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25740b01f0 0x7f25740ac5f0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:29.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.942+0000 7f257ae00640 1 -- 172.21.15.42:0/1070170413 >> 172.21.15.42:0/1070170413 conn(0x7f25740bf700 msgr2=0x7f25740bce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:29.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.942+0000 7f257ae00640 1 -- 172.21.15.42:0/1070170413 shutdown_connections 2024-07-04T17:33:29.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:29.942+0000 7f257ae00640 1 -- 172.21.15.42:0/1070170413 wait complete. 2024-07-04T17:33:29.968 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:33:30.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.228+0000 7fb973200640 1 -- 172.21.15.42:0/1717560260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb97406d580 msgr2=0x7fb97406c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:30.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.228+0000 7fb973200640 1 --2- 172.21.15.42:0/1717560260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb97406d580 0x7fb97406c2c0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fb9640097b0 tx=0x7fb96402a4a0 comp rx=0 tx=0).stop 2024-07-04T17:33:30.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.228+0000 7fb973200640 1 -- 172.21.15.42:0/1717560260 shutdown_connections 2024-07-04T17:33:30.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.228+0000 7fb973200640 1 --2- 172.21.15.42:0/1717560260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb97406d580 0x7fb97406c2c0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:30.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.228+0000 7fb973200640 1 -- 172.21.15.42:0/1717560260 >> 172.21.15.42:0/1717560260 conn(0x7fb9740bebc0 msgr2=0x7fb9740befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:30.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.228+0000 7fb973200640 1 -- 172.21.15.42:0/1717560260 shutdown_connections 2024-07-04T17:33:30.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.228+0000 7fb973200640 1 -- 172.21.15.42:0/1717560260 wait complete. 2024-07-04T17:33:30.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.230+0000 7fb973200640 1 Processor -- start 2024-07-04T17:33:30.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.230+0000 7fb973200640 1 -- start start 2024-07-04T17:33:30.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.231+0000 7fb973200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb97406d580 0x7fb9741d6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:30.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.231+0000 7fb973200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb9740b2810 con 0x7fb97406d580 2024-07-04T17:33:30.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.232+0000 7fb971e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb97406d580 0x7fb9741d6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:30.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.232+0000 7fb971e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb97406d580 0x7fb9741d6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42880/0 (socket says 172.21.15.42:42880) 2024-07-04T17:33:30.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.232+0000 7fb971e00640 1 -- 172.21.15.42:0/4061062778 learned_addr learned my addr 172.21.15.42:0/4061062778 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:30.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.232+0000 7fb971e00640 1 -- 172.21.15.42:0/4061062778 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9741e3770 con 0x7fb97406d580 2024-07-04T17:33:30.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.232+0000 7fb971e00640 1 --2- 172.21.15.42:0/4061062778 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb97406d580 0x7fb9741d6ab0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fb9640098d0 tx=0x7fb964033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:30.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.233+0000 7fb96a000640 1 -- 172.21.15.42:0/4061062778 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb964031bc0 con 0x7fb97406d580 2024-07-04T17:33:30.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.233+0000 7fb96a000640 1 -- 172.21.15.42:0/4061062778 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb964031e50 con 0x7fb97406d580 2024-07-04T17:33:30.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.233+0000 7fb96a000640 1 -- 172.21.15.42:0/4061062778 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb964033200 con 0x7fb97406d580 2024-07-04T17:33:30.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.233+0000 7fb973200640 1 -- 172.21.15.42:0/4061062778 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9740aeb70 con 0x7fb97406d580 2024-07-04T17:33:30.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.233+0000 7fb973200640 1 -- 172.21.15.42:0/4061062778 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9741d6ff0 con 0x7fb97406d580 2024-07-04T17:33:30.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.234+0000 7fb973200640 1 -- 172.21.15.42:0/4061062778 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb97406c2c0 con 0x7fb97406d580 2024-07-04T17:33:30.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.234+0000 7fb96a000640 1 -- 172.21.15.42:0/4061062778 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7fb9640333a0 con 0x7fb97406d580 2024-07-04T17:33:30.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.235+0000 7fb96a000640 1 --2- 172.21.15.42:0/4061062778 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fb95403d4f0 0x7fb95403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:30.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.235+0000 7fb96a000640 1 -- 172.21.15.42:0/4061062778 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 4303+0+0 (secure 0 0 0) 0x7fb964073210 con 0x7fb97406d580 2024-07-04T17:33:30.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.236+0000 7fb971400640 1 --2- 172.21.15.42:0/4061062778 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fb95403d4f0 0x7fb95403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:30.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.236+0000 7fb971400640 1 --2- 172.21.15.42:0/4061062778 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fb95403d4f0 0x7fb95403fa00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb9600015c0 tx=0x7fb960007a90 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:30.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.240+0000 7fb96a000640 1 -- 172.21.15.42:0/4061062778 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fb96403f030 con 0x7fb97406d580 2024-07-04T17:33:30.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.463+0000 7fb973200640 1 -- 172.21.15.42:0/4061062778 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb9740b9260 con 0x7fb97406d580 2024-07-04T17:33:30.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.465+0000 7fb96a000640 1 -- 172.21.15.42:0/4061062778 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+14458 (secure 0 0 0) 0x7fb964033690 con 0x7fb97406d580 2024-07-04T17:33:30.467 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:33:30.467 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":118,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:33:30.396257+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:33:06.769774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-07-04T17:33:07.613572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:33:26.476351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:33:27.929016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1699999570846558,"score_stable":1.1699999570846558,"optimal_score":1,"raw_score_acting":1.1699999570846558,"raw_score_stable":1.1699999570846558,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:33:30.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.468+0000 7fb973200640 1 -- 172.21.15.42:0/4061062778 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fb95403d4f0 msgr2=0x7fb95403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:30.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.468+0000 7fb973200640 1 --2- 172.21.15.42:0/4061062778 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fb95403d4f0 0x7fb95403fa00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb9600015c0 tx=0x7fb960007a90 comp rx=0 tx=0).stop 2024-07-04T17:33:30.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.468+0000 7fb973200640 1 -- 172.21.15.42:0/4061062778 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb97406d580 msgr2=0x7fb9741d6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:30.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.468+0000 7fb973200640 1 --2- 172.21.15.42:0/4061062778 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb97406d580 0x7fb9741d6ab0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fb9640098d0 tx=0x7fb964033e40 comp rx=0 tx=0).stop 2024-07-04T17:33:30.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.468+0000 7fb973200640 1 -- 172.21.15.42:0/4061062778 shutdown_connections 2024-07-04T17:33:30.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.468+0000 7fb973200640 1 --2- 172.21.15.42:0/4061062778 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fb95403d4f0 0x7fb95403fa00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:30.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.468+0000 7fb973200640 1 --2- 172.21.15.42:0/4061062778 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb97406d580 0x7fb9741d6ab0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:30.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.468+0000 7fb973200640 1 -- 172.21.15.42:0/4061062778 >> 172.21.15.42:0/4061062778 conn(0x7fb9740bebc0 msgr2=0x7fb9740bb480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:30.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.468+0000 7fb973200640 1 -- 172.21.15.42:0/4061062778 shutdown_connections 2024-07-04T17:33:30.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.468+0000 7fb973200640 1 -- 172.21.15.42:0/4061062778 wait complete. 2024-07-04T17:33:30.495 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs standby_count_wanted 0 2024-07-04T17:33:30.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.779+0000 7f0b5b600640 1 -- 172.21.15.42:0/3974632921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b540ab2f0 msgr2=0x7f0b540a9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:30.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.779+0000 7f0b5b600640 1 --2- 172.21.15.42:0/3974632921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b540ab2f0 0x7f0b540a9990 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f0b480097b0 tx=0x7f0b4802a530 comp rx=0 tx=0).stop 2024-07-04T17:33:30.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.779+0000 7f0b5b600640 1 -- 172.21.15.42:0/3974632921 shutdown_connections 2024-07-04T17:33:30.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.779+0000 7f0b5b600640 1 --2- 172.21.15.42:0/3974632921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b540ab2f0 0x7f0b540a9990 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:30.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.779+0000 7f0b5b600640 1 -- 172.21.15.42:0/3974632921 >> 172.21.15.42:0/3974632921 conn(0x7f0b5406c020 msgr2=0x7f0b5406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:30.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.779+0000 7f0b5b600640 1 -- 172.21.15.42:0/3974632921 shutdown_connections 2024-07-04T17:33:30.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.779+0000 7f0b5b600640 1 -- 172.21.15.42:0/3974632921 wait complete. 2024-07-04T17:33:30.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.782+0000 7f0b5b600640 1 Processor -- start 2024-07-04T17:33:30.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.782+0000 7f0b5b600640 1 -- start start 2024-07-04T17:33:30.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.784+0000 7f0b5b600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b5406d110 0x7f0b540bca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:30.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.784+0000 7f0b5b600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5413f430 con 0x7f0b5406d110 2024-07-04T17:33:30.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.784+0000 7f0b58c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b5406d110 0x7f0b540bca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:30.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.784+0000 7f0b58c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b5406d110 0x7f0b540bca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42904/0 (socket says 172.21.15.42:42904) 2024-07-04T17:33:30.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.784+0000 7f0b58c00640 1 -- 172.21.15.42:0/1315065483 learned_addr learned my addr 172.21.15.42:0/1315065483 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:30.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.784+0000 7f0b58c00640 1 -- 172.21.15.42:0/1315065483 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b540ba7b0 con 0x7f0b5406d110 2024-07-04T17:33:30.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.785+0000 7f0b58c00640 1 --2- 172.21.15.42:0/1315065483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b5406d110 0x7f0b540bca90 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f0b480098d0 tx=0x7f0b48033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:30.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.785+0000 7f0b50c00640 1 -- 172.21.15.42:0/1315065483 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b48031880 con 0x7f0b5406d110 2024-07-04T17:33:30.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.785+0000 7f0b50c00640 1 -- 172.21.15.42:0/1315065483 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b48031b10 con 0x7f0b5406d110 2024-07-04T17:33:30.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.785+0000 7f0b5b600640 1 -- 172.21.15.42:0/1315065483 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b540ba400 con 0x7f0b5406d110 2024-07-04T17:33:30.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.785+0000 7f0b50c00640 1 -- 172.21.15.42:0/1315065483 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b48031df0 con 0x7f0b5406d110 2024-07-04T17:33:30.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.785+0000 7f0b5b600640 1 -- 172.21.15.42:0/1315065483 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b540bac80 con 0x7f0b5406d110 2024-07-04T17:33:30.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.787+0000 7f0b5b600640 1 -- 172.21.15.42:0/1315065483 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b54144870 con 0x7f0b5406d110 2024-07-04T17:33:30.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.787+0000 7f0b50c00640 1 -- 172.21.15.42:0/1315065483 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7f0b48033070 con 0x7f0b5406d110 2024-07-04T17:33:30.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.788+0000 7f0b50c00640 1 --2- 172.21.15.42:0/1315065483 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f0b3403d4a0 0x7f0b3403f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:30.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.788+0000 7f0b50c00640 1 -- 172.21.15.42:0/1315065483 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 4303+0+0 (secure 0 0 0) 0x7f0b48073110 con 0x7f0b5406d110 2024-07-04T17:33:30.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.788+0000 7f0b53e00640 1 --2- 172.21.15.42:0/1315065483 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f0b3403d4a0 0x7f0b3403f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:30.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.789+0000 7f0b53e00640 1 --2- 172.21.15.42:0/1315065483 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f0b3403d4a0 0x7f0b3403f9b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0b400015e0 tx=0x7f0b40001290 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:30.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:30.793+0000 7f0b50c00640 1 -- 172.21.15.42:0/1315065483 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f0b54144870 con 0x7f0b5406d110 2024-07-04T17:33:31.026 INFO:tasks.ceph.mds.b.smithi042.stderr:2024-07-04T17:33:31.025+0000 7fd0bf600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-04T17:33:31.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.035+0000 7f0b5b600640 1 -- 172.21.15.42:0/1315065483 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f0b540ba980 con 0x7f0b5406d110 2024-07-04T17:33:31.184 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-04T17:33:31.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.422+0000 7f0b50c00640 1 -- 172.21.15.42:0/1315065483 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"}]=0 v95) ==== 121+0+0 (secure 0 0 0) 0x7f0b540ba980 con 0x7f0b5406d110 2024-07-04T17:33:31.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.426+0000 7f0b5b600640 1 -- 172.21.15.42:0/1315065483 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f0b3403d4a0 msgr2=0x7f0b3403f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:31.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.426+0000 7f0b5b600640 1 --2- 172.21.15.42:0/1315065483 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f0b3403d4a0 0x7f0b3403f9b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0b400015e0 tx=0x7f0b40001290 comp rx=0 tx=0).stop 2024-07-04T17:33:31.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.426+0000 7f0b5b600640 1 -- 172.21.15.42:0/1315065483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b5406d110 msgr2=0x7f0b540bca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:31.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.426+0000 7f0b5b600640 1 --2- 172.21.15.42:0/1315065483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b5406d110 0x7f0b540bca90 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f0b480098d0 tx=0x7f0b48033ad0 comp rx=0 tx=0).stop 2024-07-04T17:33:31.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.426+0000 7f0b5b600640 1 -- 172.21.15.42:0/1315065483 shutdown_connections 2024-07-04T17:33:31.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.426+0000 7f0b5b600640 1 --2- 172.21.15.42:0/1315065483 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f0b3403d4a0 0x7f0b3403f9b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:31.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.426+0000 7f0b5b600640 1 --2- 172.21.15.42:0/1315065483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b5406d110 0x7f0b540bca90 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:31.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.426+0000 7f0b5b600640 1 -- 172.21.15.42:0/1315065483 >> 172.21.15.42:0/1315065483 conn(0x7f0b5406c020 msgr2=0x7f0b540bf8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:31.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.426+0000 7f0b5b600640 1 -- 172.21.15.42:0/1315065483 shutdown_connections 2024-07-04T17:33:31.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.427+0000 7f0b5b600640 1 -- 172.21.15.42:0/1315065483 wait complete. 2024-07-04T17:33:31.460 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-04T17:33:31.460 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs allow_standby_replay false 2024-07-04T17:33:31.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.753+0000 7f8499c00640 1 -- 172.21.15.42:0/987452473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84940b0db0 msgr2=0x7f84940adf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:31.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.753+0000 7f8499c00640 1 --2- 172.21.15.42:0/987452473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84940b0db0 0x7f84940adf70 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f847c0097b0 tx=0x7f847c02a540 comp rx=0 tx=0).stop 2024-07-04T17:33:31.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.753+0000 7f8499c00640 1 -- 172.21.15.42:0/987452473 shutdown_connections 2024-07-04T17:33:31.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.753+0000 7f8499c00640 1 --2- 172.21.15.42:0/987452473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84940b0db0 0x7f84940adf70 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:31.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.753+0000 7f8499c00640 1 -- 172.21.15.42:0/987452473 >> 172.21.15.42:0/987452473 conn(0x7f84940afaa0 msgr2=0x7f84940afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:31.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.753+0000 7f8499c00640 1 -- 172.21.15.42:0/987452473 shutdown_connections 2024-07-04T17:33:31.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.754+0000 7f8499c00640 1 -- 172.21.15.42:0/987452473 wait complete. 2024-07-04T17:33:31.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.756+0000 7f8499c00640 1 Processor -- start 2024-07-04T17:33:31.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.756+0000 7f8499c00640 1 -- start start 2024-07-04T17:33:31.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.757+0000 7f8499c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84940acac0 0x7f84941d5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:31.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.757+0000 7f8499c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f84940ae630 con 0x7f84940acac0 2024-07-04T17:33:31.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.757+0000 7f8493e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84940acac0 0x7f84941d5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:31.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.757+0000 7f8493e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84940acac0 0x7f84941d5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42924/0 (socket says 172.21.15.42:42924) 2024-07-04T17:33:31.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.757+0000 7f8493e00640 1 -- 172.21.15.42:0/1078545110 learned_addr learned my addr 172.21.15.42:0/1078545110 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:31.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.758+0000 7f8493e00640 1 -- 172.21.15.42:0/1078545110 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8494146aa0 con 0x7f84940acac0 2024-07-04T17:33:31.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.758+0000 7f8493e00640 1 --2- 172.21.15.42:0/1078545110 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84940acac0 0x7f84941d5500 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f848400cb30 tx=0x7f84840021c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:31.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.758+0000 7f8490c00640 1 -- 172.21.15.42:0/1078545110 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8484016020 con 0x7f84940acac0 2024-07-04T17:33:31.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.758+0000 7f8499c00640 1 -- 172.21.15.42:0/1078545110 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84941e3e50 con 0x7f84940acac0 2024-07-04T17:33:31.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.759+0000 7f8499c00640 1 -- 172.21.15.42:0/1078545110 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84941443d0 con 0x7f84940acac0 2024-07-04T17:33:31.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.759+0000 7f8490c00640 1 -- 172.21.15.42:0/1078545110 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f848400aba0 con 0x7f84940acac0 2024-07-04T17:33:31.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.759+0000 7f8490c00640 1 -- 172.21.15.42:0/1078545110 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8484012420 con 0x7f84940acac0 2024-07-04T17:33:31.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.760+0000 7f8499c00640 1 -- 172.21.15.42:0/1078545110 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f846c005290 con 0x7f84940acac0 2024-07-04T17:33:31.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.760+0000 7f8490c00640 1 -- 172.21.15.42:0/1078545110 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7f8484019020 con 0x7f84940acac0 2024-07-04T17:33:31.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.761+0000 7f8490c00640 1 --2- 172.21.15.42:0/1078545110 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f847003d4f0 0x7f847003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:31.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.761+0000 7f8490c00640 1 -- 172.21.15.42:0/1078545110 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 4303+0+0 (secure 0 0 0) 0x7f84840522c0 con 0x7f84940acac0 2024-07-04T17:33:31.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.761+0000 7f848b400640 1 --2- 172.21.15.42:0/1078545110 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f847003d4f0 0x7f847003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:31.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.762+0000 7f848b400640 1 --2- 172.21.15.42:0/1078545110 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f847003d4f0 0x7f847003fa00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f847c034000 tx=0x7f847c004730 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:31.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:31.766+0000 7f8490c00640 1 -- 172.21.15.42:0/1078545110 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f8484020020 con 0x7f84940acac0 2024-07-04T17:33:32.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.023+0000 7f8499c00640 1 -- 172.21.15.42:0/1078545110 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f846c005580 con 0x7f84940acac0 2024-07-04T17:33:32.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.423+0000 7f8490c00640 1 -- 172.21.15.42:0/1078545110 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"}]=0 v96) ==== 125+0+0 (secure 0 0 0) 0x7f8484052080 con 0x7f84940acac0 2024-07-04T17:33:32.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.427+0000 7f8499c00640 1 -- 172.21.15.42:0/1078545110 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f847003d4f0 msgr2=0x7f847003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:32.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.427+0000 7f8499c00640 1 --2- 172.21.15.42:0/1078545110 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f847003d4f0 0x7f847003fa00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f847c034000 tx=0x7f847c004730 comp rx=0 tx=0).stop 2024-07-04T17:33:32.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.427+0000 7f8499c00640 1 -- 172.21.15.42:0/1078545110 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84940acac0 msgr2=0x7f84941d5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:32.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.427+0000 7f8499c00640 1 --2- 172.21.15.42:0/1078545110 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84940acac0 0x7f84941d5500 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f848400cb30 tx=0x7f84840021c0 comp rx=0 tx=0).stop 2024-07-04T17:33:32.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.427+0000 7f8499c00640 1 -- 172.21.15.42:0/1078545110 shutdown_connections 2024-07-04T17:33:32.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.427+0000 7f8499c00640 1 --2- 172.21.15.42:0/1078545110 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f847003d4f0 0x7f847003fa00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:32.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.427+0000 7f8499c00640 1 --2- 172.21.15.42:0/1078545110 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84940acac0 0x7f84941d5500 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:32.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.427+0000 7f8499c00640 1 -- 172.21.15.42:0/1078545110 >> 172.21.15.42:0/1078545110 conn(0x7f84940afaa0 msgr2=0x7f8494075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:32.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.428+0000 7f8499c00640 1 -- 172.21.15.42:0/1078545110 shutdown_connections 2024-07-04T17:33:32.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.428+0000 7f8499c00640 1 -- 172.21.15.42:0/1078545110 wait complete. 2024-07-04T17:33:32.457 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'backup_fs' 2024-07-04T17:33:32.457 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:33:32.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.743+0000 7fd9d8600640 1 -- 172.21.15.42:0/2948087686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9d00b3030 msgr2=0x7fd9d00b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:32.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.743+0000 7fd9d8600640 1 --2- 172.21.15.42:0/2948087686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9d00b3030 0x7fd9d00b01f0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc00a8c0 tx=0x7fd9bc02b860 comp rx=0 tx=0).stop 2024-07-04T17:33:32.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.744+0000 7fd9d8600640 1 -- 172.21.15.42:0/2948087686 shutdown_connections 2024-07-04T17:33:32.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.744+0000 7fd9d8600640 1 --2- 172.21.15.42:0/2948087686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9d00b3030 0x7fd9d00b01f0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:32.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.744+0000 7fd9d8600640 1 -- 172.21.15.42:0/2948087686 >> 172.21.15.42:0/2948087686 conn(0x7fd9d00bf700 msgr2=0x7fd9d00af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:32.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.744+0000 7fd9d8600640 1 -- 172.21.15.42:0/2948087686 shutdown_connections 2024-07-04T17:33:32.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.744+0000 7fd9d8600640 1 -- 172.21.15.42:0/2948087686 wait complete. 2024-07-04T17:33:32.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.746+0000 7fd9d8600640 1 Processor -- start 2024-07-04T17:33:32.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.746+0000 7fd9d8600640 1 -- start start 2024-07-04T17:33:32.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.747+0000 7fd9d8600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9d006cea0 0x7fd9d0144470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:32.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.747+0000 7fd9d8600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d006ef50 con 0x7fd9d006cea0 2024-07-04T17:33:32.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.748+0000 7fd9d5c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9d006cea0 0x7fd9d0144470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:32.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.748+0000 7fd9d5c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9d006cea0 0x7fd9d0144470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42930/0 (socket says 172.21.15.42:42930) 2024-07-04T17:33:32.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.748+0000 7fd9d5c00640 1 -- 172.21.15.42:0/1229271147 learned_addr learned my addr 172.21.15.42:0/1229271147 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:32.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.748+0000 7fd9d5c00640 1 -- 172.21.15.42:0/1229271147 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d0149590 con 0x7fd9d006cea0 2024-07-04T17:33:32.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.748+0000 7fd9d5c00640 1 --2- 172.21.15.42:0/1229271147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9d006cea0 0x7fd9d0144470 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fd9c400c380 tx=0x7fd9c400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:32.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.749+0000 7fd9cd600640 1 -- 172.21.15.42:0/1229271147 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9c40116c0 con 0x7fd9d006cea0 2024-07-04T17:33:32.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.749+0000 7fd9cd600640 1 -- 172.21.15.42:0/1229271147 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9c4017070 con 0x7fd9d006cea0 2024-07-04T17:33:32.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.749+0000 7fd9d8600640 1 -- 172.21.15.42:0/1229271147 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9d0146b60 con 0x7fd9d006cea0 2024-07-04T17:33:32.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.749+0000 7fd9cd600640 1 -- 172.21.15.42:0/1229271147 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9c4011b20 con 0x7fd9d006cea0 2024-07-04T17:33:32.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.749+0000 7fd9d8600640 1 -- 172.21.15.42:0/1229271147 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9d01479a0 con 0x7fd9d006cea0 2024-07-04T17:33:32.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.750+0000 7fd9d8600640 1 -- 172.21.15.42:0/1229271147 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9d00b01f0 con 0x7fd9d006cea0 2024-07-04T17:33:32.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.750+0000 7fd9cd600640 1 -- 172.21.15.42:0/1229271147 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7fd9c4011cc0 con 0x7fd9d006cea0 2024-07-04T17:33:32.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.751+0000 7fd9cd600640 1 --2- 172.21.15.42:0/1229271147 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fd9b003d4f0 0x7fd9b003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:32.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.751+0000 7fd9cd600640 1 -- 172.21.15.42:0/1229271147 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 4303+0+0 (secure 0 0 0) 0x7fd9c40519f0 con 0x7fd9d006cea0 2024-07-04T17:33:32.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.751+0000 7fd9d5200640 1 --2- 172.21.15.42:0/1229271147 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fd9b003d4f0 0x7fd9b003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:32.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.752+0000 7fd9d5200640 1 --2- 172.21.15.42:0/1229271147 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fd9b003d4f0 0x7fd9b003fa00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc036000 tx=0x7fd9bc004730 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:32.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:32.756+0000 7fd9cd600640 1 -- 172.21.15.42:0/1229271147 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fd9c4010cf0 con 0x7fd9d006cea0 2024-07-04T17:33:33.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.010+0000 7fd9d8600640 1 -- 172.21.15.42:0/1229271147 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9d0069b70 con 0x7fd9d006cea0 2024-07-04T17:33:33.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.011+0000 7fd9cd600640 1 -- 172.21.15.42:0/1229271147 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+7354 (secure 0 0 0) 0x7fd9c4019310 con 0x7fd9d006cea0 2024-07-04T17:33:33.013 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:33:33.013 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":96,"btime":"2024-07-04T17:33:32:423100+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5536,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/68857121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":68857121},{"type":"v1","addr":"172.21.15.42:6825","nonce":68857121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":5544,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6828/20990456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":20990456},{"type":"v1","addr":"172.21.15.42:6828","nonce":20990456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":5545,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/428501060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6827","nonce":428501060},{"type":"v1","addr":"172.21.15.42:6829","nonce":428501060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:33:08.649551+0000","modified":"2024-07-04T17:33:11.687748+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5550":{"gid":5550,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6833/605786437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":605786437},{"type":"v1","addr":"172.21.15.42:6833","nonce":605786437}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5550,"qdb_cluster":[5550]},"id":10},{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:33:28.944269+0000","modified":"2024-07-04T17:33:32.025418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5549},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5549":{"gid":5549,"name":"b","rank":0,"incarnation":92,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6831/3224527035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":3224527035},{"type":"v1","addr":"172.21.15.42:6831","nonce":3224527035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5549,"qdb_cluster":[5549]},"id":11}]} 2024-07-04T17:33:33.013 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 96 2024-07-04T17:33:33.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.014+0000 7fd9d8600640 1 -- 172.21.15.42:0/1229271147 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fd9b003d4f0 msgr2=0x7fd9b003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:33.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.014+0000 7fd9d8600640 1 --2- 172.21.15.42:0/1229271147 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fd9b003d4f0 0x7fd9b003fa00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc036000 tx=0x7fd9bc004730 comp rx=0 tx=0).stop 2024-07-04T17:33:33.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.014+0000 7fd9d8600640 1 -- 172.21.15.42:0/1229271147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9d006cea0 msgr2=0x7fd9d0144470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:33.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.014+0000 7fd9d8600640 1 --2- 172.21.15.42:0/1229271147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9d006cea0 0x7fd9d0144470 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fd9c400c380 tx=0x7fd9c400c840 comp rx=0 tx=0).stop 2024-07-04T17:33:33.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.014+0000 7fd9d8600640 1 -- 172.21.15.42:0/1229271147 shutdown_connections 2024-07-04T17:33:33.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.014+0000 7fd9d8600640 1 --2- 172.21.15.42:0/1229271147 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fd9b003d4f0 0x7fd9b003fa00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:33.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.014+0000 7fd9d8600640 1 --2- 172.21.15.42:0/1229271147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9d006cea0 0x7fd9d0144470 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:33.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.014+0000 7fd9d8600640 1 -- 172.21.15.42:0/1229271147 >> 172.21.15.42:0/1229271147 conn(0x7fd9d00bf700 msgr2=0x7fd9d00aa3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:33.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.014+0000 7fd9d8600640 1 -- 172.21.15.42:0/1229271147 shutdown_connections 2024-07-04T17:33:33.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.014+0000 7fd9d8600640 1 -- 172.21.15.42:0/1229271147 wait complete. 2024-07-04T17:33:33.042 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:33:33.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.333+0000 7fe21be00640 1 -- 172.21.15.42:0/4017954005 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe21406cea0 msgr2=0x7fe2140b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:33.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.333+0000 7fe21be00640 1 --2- 172.21.15.42:0/4017954005 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe21406cea0 0x7fe2140b3030 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fe20000a8c0 tx=0x7fe20002b860 comp rx=0 tx=0).stop 2024-07-04T17:33:33.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.333+0000 7fe21be00640 1 -- 172.21.15.42:0/4017954005 shutdown_connections 2024-07-04T17:33:33.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.333+0000 7fe21be00640 1 --2- 172.21.15.42:0/4017954005 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe21406cea0 0x7fe2140b3030 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:33.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.333+0000 7fe21be00640 1 -- 172.21.15.42:0/4017954005 >> 172.21.15.42:0/4017954005 conn(0x7fe2140bf700 msgr2=0x7fe2140af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:33.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.333+0000 7fe21be00640 1 -- 172.21.15.42:0/4017954005 shutdown_connections 2024-07-04T17:33:33.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.333+0000 7fe21be00640 1 -- 172.21.15.42:0/4017954005 wait complete. 2024-07-04T17:33:33.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.335+0000 7fe21be00640 1 Processor -- start 2024-07-04T17:33:33.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.336+0000 7fe21be00640 1 -- start start 2024-07-04T17:33:33.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.337+0000 7fe21be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe21406cea0 0x7fe2140ad640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:33.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.337+0000 7fe21be00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe2140b0370 con 0x7fe21406cea0 2024-07-04T17:33:33.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.337+0000 7fe219400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe21406cea0 0x7fe2140ad640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:33.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.337+0000 7fe219400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe21406cea0 0x7fe2140ad640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42944/0 (socket says 172.21.15.42:42944) 2024-07-04T17:33:33.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.337+0000 7fe219400640 1 -- 172.21.15.42:0/2531387838 learned_addr learned my addr 172.21.15.42:0/2531387838 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:33.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.338+0000 7fe219400640 1 -- 172.21.15.42:0/2531387838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2141e3420 con 0x7fe21406cea0 2024-07-04T17:33:33.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.338+0000 7fe219400640 1 --2- 172.21.15.42:0/2531387838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe21406cea0 0x7fe2140ad640 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fe20800c3b0 tx=0x7fe20800c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:33.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.338+0000 7fe211600640 1 -- 172.21.15.42:0/2531387838 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe208011780 con 0x7fe21406cea0 2024-07-04T17:33:33.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.338+0000 7fe211600640 1 -- 172.21.15.42:0/2531387838 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe208017070 con 0x7fe21406cea0 2024-07-04T17:33:33.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.338+0000 7fe211600640 1 -- 172.21.15.42:0/2531387838 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe208011c00 con 0x7fe21406cea0 2024-07-04T17:33:33.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.338+0000 7fe21be00640 1 -- 172.21.15.42:0/2531387838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe21406ff30 con 0x7fe21406cea0 2024-07-04T17:33:33.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.338+0000 7fe21be00640 1 -- 172.21.15.42:0/2531387838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2141daae0 con 0x7fe21406cea0 2024-07-04T17:33:33.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.340+0000 7fe21be00640 1 -- 172.21.15.42:0/2531387838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe214070160 con 0x7fe21406cea0 2024-07-04T17:33:33.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.340+0000 7fe211600640 1 -- 172.21.15.42:0/2531387838 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7fe20801c460 con 0x7fe21406cea0 2024-07-04T17:33:33.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.341+0000 7fe211600640 1 --2- 172.21.15.42:0/2531387838 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fe1f403d4f0 0x7fe1f403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:33.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.341+0000 7fe211600640 1 -- 172.21.15.42:0/2531387838 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 4303+0+0 (secure 0 0 0) 0x7fe2080519a0 con 0x7fe21406cea0 2024-07-04T17:33:33.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.341+0000 7fe218a00640 1 --2- 172.21.15.42:0/2531387838 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fe1f403d4f0 0x7fe1f403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:33.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.342+0000 7fe218a00640 1 --2- 172.21.15.42:0/2531387838 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fe1f403d4f0 0x7fe1f403fa00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe200036000 tx=0x7fe200004730 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:33.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.345+0000 7fe211600640 1 -- 172.21.15.42:0/2531387838 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fe208011da0 con 0x7fe21406cea0 2024-07-04T17:33:33.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.561+0000 7fe21be00640 1 -- 172.21.15.42:0/2531387838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2140b3030 con 0x7fe21406cea0 2024-07-04T17:33:33.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.564+0000 7fe211600640 1 -- 172.21.15.42:0/2531387838 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+14458 (secure 0 0 0) 0x7fe20801c800 con 0x7fe21406cea0 2024-07-04T17:33:33.566 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:33:33.566 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":118,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:33:30.396257+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:33:06.769774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-07-04T17:33:07.613572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:33:26.476351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:33:27.929016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1699999570846558,"score_stable":1.1699999570846558,"optimal_score":1,"raw_score_acting":1.1699999570846558,"raw_score_stable":1.1699999570846558,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:33:33.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.568+0000 7fe21be00640 1 -- 172.21.15.42:0/2531387838 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fe1f403d4f0 msgr2=0x7fe1f403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:33.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.568+0000 7fe21be00640 1 --2- 172.21.15.42:0/2531387838 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fe1f403d4f0 0x7fe1f403fa00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe200036000 tx=0x7fe200004730 comp rx=0 tx=0).stop 2024-07-04T17:33:33.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.568+0000 7fe21be00640 1 -- 172.21.15.42:0/2531387838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe21406cea0 msgr2=0x7fe2140ad640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:33.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.568+0000 7fe21be00640 1 --2- 172.21.15.42:0/2531387838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe21406cea0 0x7fe2140ad640 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fe20800c3b0 tx=0x7fe20800c870 comp rx=0 tx=0).stop 2024-07-04T17:33:33.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.568+0000 7fe21be00640 1 -- 172.21.15.42:0/2531387838 shutdown_connections 2024-07-04T17:33:33.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.568+0000 7fe21be00640 1 --2- 172.21.15.42:0/2531387838 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7fe1f403d4f0 0x7fe1f403fa00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:33.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.568+0000 7fe21be00640 1 --2- 172.21.15.42:0/2531387838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe21406cea0 0x7fe2140ad640 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:33.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.568+0000 7fe21be00640 1 -- 172.21.15.42:0/2531387838 >> 172.21.15.42:0/2531387838 conn(0x7fe2140bf700 msgr2=0x7fe2140aec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:33.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.568+0000 7fe21be00640 1 -- 172.21.15.42:0/2531387838 shutdown_connections 2024-07-04T17:33:33.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.568+0000 7fe21be00640 1 -- 172.21.15.42:0/2531387838 wait complete. 2024-07-04T17:33:33.595 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-04T17:33:33.595 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:33:33.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.887+0000 7f7400200640 1 -- 172.21.15.42:0/2752753480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73f80b2e00 msgr2=0x7f73f80b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:33.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.887+0000 7f7400200640 1 --2- 172.21.15.42:0/2752753480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73f80b2e00 0x7f73f80b37b0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f73ec0097b0 tx=0x7f73ec02a540 comp rx=0 tx=0).stop 2024-07-04T17:33:33.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.888+0000 7f7400200640 1 -- 172.21.15.42:0/2752753480 shutdown_connections 2024-07-04T17:33:33.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.888+0000 7f7400200640 1 --2- 172.21.15.42:0/2752753480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73f80b2e00 0x7f73f80b37b0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:33.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.888+0000 7f7400200640 1 -- 172.21.15.42:0/2752753480 >> 172.21.15.42:0/2752753480 conn(0x7f73f80bf700 msgr2=0x7f73f80af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:33.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.888+0000 7f7400200640 1 -- 172.21.15.42:0/2752753480 shutdown_connections 2024-07-04T17:33:33.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.888+0000 7f7400200640 1 -- 172.21.15.42:0/2752753480 wait complete. 2024-07-04T17:33:33.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.890+0000 7f7400200640 1 Processor -- start 2024-07-04T17:33:33.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.890+0000 7f7400200640 1 -- start start 2024-07-04T17:33:33.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.891+0000 7f7400200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73f80b01f0 0x7f73f80aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:33.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.891+0000 7f7400200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f73f806ce30 con 0x7f73f80b01f0 2024-07-04T17:33:33.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.891+0000 7f73fd800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73f80b01f0 0x7f73f80aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:33.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.891+0000 7f73fd800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73f80b01f0 0x7f73f80aebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40770/0 (socket says 172.21.15.42:40770) 2024-07-04T17:33:33.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.891+0000 7f73fd800640 1 -- 172.21.15.42:0/3485034687 learned_addr learned my addr 172.21.15.42:0/3485034687 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:33.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.892+0000 7f73fd800640 1 -- 172.21.15.42:0/3485034687 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f80af280 con 0x7f73f80b01f0 2024-07-04T17:33:33.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.892+0000 7f73fd800640 1 --2- 172.21.15.42:0/3485034687 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73f80b01f0 0x7f73f80aebe0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f73ec0098d0 tx=0x7f73ec033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:33.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.892+0000 7f73f5600640 1 -- 172.21.15.42:0/3485034687 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73ec031bc0 con 0x7f73f80b01f0 2024-07-04T17:33:33.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.892+0000 7f73f5600640 1 -- 172.21.15.42:0/3485034687 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73ec031e50 con 0x7f73f80b01f0 2024-07-04T17:33:33.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.892+0000 7f73f5600640 1 -- 172.21.15.42:0/3485034687 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73ec033200 con 0x7f73f80b01f0 2024-07-04T17:33:33.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.892+0000 7f7400200640 1 -- 172.21.15.42:0/3485034687 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f8072420 con 0x7f73f80b01f0 2024-07-04T17:33:33.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.893+0000 7f7400200640 1 -- 172.21.15.42:0/3485034687 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f80720f0 con 0x7f73f80b01f0 2024-07-04T17:33:33.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.894+0000 7f7400200640 1 -- 172.21.15.42:0/3485034687 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73f806d4e0 con 0x7f73f80b01f0 2024-07-04T17:33:33.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.894+0000 7f73f5600640 1 -- 172.21.15.42:0/3485034687 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7f73ec0333a0 con 0x7f73f80b01f0 2024-07-04T17:33:33.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.895+0000 7f73f5600640 1 --2- 172.21.15.42:0/3485034687 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f73d803d4f0 0x7f73d803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:33.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.895+0000 7f73f5600640 1 -- 172.21.15.42:0/3485034687 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 4303+0+0 (secure 0 0 0) 0x7f73ec0732a0 con 0x7f73f80b01f0 2024-07-04T17:33:33.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.895+0000 7f73fce00640 1 --2- 172.21.15.42:0/3485034687 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f73d803d4f0 0x7f73d803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:33.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.896+0000 7f73fce00640 1 --2- 172.21.15.42:0/3485034687 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f73d803d4f0 0x7f73d803fa00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f73e40015c0 tx=0x7f73e4007a90 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:33.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:33.900+0000 7f73f5600640 1 -- 172.21.15.42:0/3485034687 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f73ec03f030 con 0x7f73f80b01f0 2024-07-04T17:33:34.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.106+0000 7f7400200640 1 -- 172.21.15.42:0/3485034687 --> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f73f8069b70 con 0x7f73d803d4f0 2024-07-04T17:33:34.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.120+0000 7f73f5600640 1 -- 172.21.15.42:0/3485034687 <== mgr.5434 v2:172.21.15.42:6834/4130303917 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+483259 (secure 0 0 0) 0x7f73f8069b70 con 0x7f73d803d4f0 2024-07-04T17:33:34.129 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:33:34.132 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:33:34.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.129+0000 7f7400200640 1 -- 172.21.15.42:0/3485034687 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f73d803d4f0 msgr2=0x7f73d803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:34.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.129+0000 7f7400200640 1 --2- 172.21.15.42:0/3485034687 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f73d803d4f0 0x7f73d803fa00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f73e40015c0 tx=0x7f73e4007a90 comp rx=0 tx=0).stop 2024-07-04T17:33:34.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.129+0000 7f7400200640 1 -- 172.21.15.42:0/3485034687 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73f80b01f0 msgr2=0x7f73f80aebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:34.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.129+0000 7f7400200640 1 --2- 172.21.15.42:0/3485034687 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73f80b01f0 0x7f73f80aebe0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f73ec0098d0 tx=0x7f73ec033e40 comp rx=0 tx=0).stop 2024-07-04T17:33:34.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.129+0000 7f7400200640 1 -- 172.21.15.42:0/3485034687 shutdown_connections 2024-07-04T17:33:34.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.129+0000 7f7400200640 1 --2- 172.21.15.42:0/3485034687 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f73d803d4f0 0x7f73d803fa00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:34.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.129+0000 7f7400200640 1 --2- 172.21.15.42:0/3485034687 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73f80b01f0 0x7f73f80aebe0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:34.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.129+0000 7f7400200640 1 -- 172.21.15.42:0/3485034687 >> 172.21.15.42:0/3485034687 conn(0x7f73f80bf700 msgr2=0x7f73f81415f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:34.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.130+0000 7f7400200640 1 -- 172.21.15.42:0/3485034687 shutdown_connections 2024-07-04T17:33:34.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.130+0000 7f7400200640 1 -- 172.21.15.42:0/3485034687 wait complete. 2024-07-04T17:33:34.155 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":45,"stamp":"2024-07-04T17:33:33.281782+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596885,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":476,"num_read_kb":1278,"num_write":255,"num_write_kb":1137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":181,"ondisk_log_size":181,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":379,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":110860,"kb_used_data":4828,"kb_used_omap":4,"kb_used_meta":105979,"kb_avail":283004660,"statfs":{"total":289910292480,"available":289796771840,"internally_reserved":0,"allocated":4943872,"data_stored":2637853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":108522831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.916028"},"pg_stats":[{"pgid":"23.3d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.436130+0000","last_change":"2024-07-04T17:33:29.452643+0000","last_active":"2024-07-04T17:33:30.436130+0000","last_peered":"2024-07-04T17:33:30.436130+0000","last_clean":"2024-07-04T17:33:30.436130+0000","last_became_active":"2024-07-04T17:33:29.451332+0000","last_became_peered":"2024-07-04T17:33:29.451332+0000","last_unstale":"2024-07-04T17:33:30.436130+0000","last_undegraded":"2024-07-04T17:33:30.436130+0000","last_fullsized":"2024-07-04T17:33:30.436130+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:24:32.296539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3f","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.413580+0000","last_change":"2024-07-04T17:33:09.110137+0000","last_active":"2024-07-04T17:33:30.413580+0000","last_peered":"2024-07-04T17:33:30.413580+0000","last_clean":"2024-07-04T17:33:30.413580+0000","last_became_active":"2024-07-04T17:33:09.109336+0000","last_became_peered":"2024-07-04T17:33:09.109336+0000","last_unstale":"2024-07-04T17:33:30.413580+0000","last_undegraded":"2024-07-04T17:33:30.413580+0000","last_fullsized":"2024-07-04T17:33:30.413580+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:09.113117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.414866+0000","last_change":"2024-07-04T17:33:29.452645+0000","last_active":"2024-07-04T17:33:30.414866+0000","last_peered":"2024-07-04T17:33:30.414866+0000","last_clean":"2024-07-04T17:33:30.414866+0000","last_became_active":"2024-07-04T17:33:29.452065+0000","last_became_peered":"2024-07-04T17:33:29.452065+0000","last_unstale":"2024-07-04T17:33:30.414866+0000","last_undegraded":"2024-07-04T17:33:30.414866+0000","last_fullsized":"2024-07-04T17:33:30.414866+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:51:15.248651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.448862+0000","last_change":"2024-07-04T17:33:09.113135+0000","last_active":"2024-07-04T17:33:30.448862+0000","last_peered":"2024-07-04T17:33:30.448862+0000","last_clean":"2024-07-04T17:33:30.448862+0000","last_became_active":"2024-07-04T17:33:09.112824+0000","last_became_peered":"2024-07-04T17:33:09.112824+0000","last_unstale":"2024-07-04T17:33:30.448862+0000","last_undegraded":"2024-07-04T17:33:30.448862+0000","last_fullsized":"2024-07-04T17:33:30.448862+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:25:36.254957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.3f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.439081+0000","last_change":"2024-07-04T17:33:29.450485+0000","last_active":"2024-07-04T17:33:30.439081+0000","last_peered":"2024-07-04T17:33:30.439081+0000","last_clean":"2024-07-04T17:33:30.439081+0000","last_became_active":"2024-07-04T17:33:29.450123+0000","last_became_peered":"2024-07-04T17:33:29.450123+0000","last_unstale":"2024-07-04T17:33:30.439081+0000","last_undegraded":"2024-07-04T17:33:30.439081+0000","last_fullsized":"2024-07-04T17:33:30.439081+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:57:29.422317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.413950+0000","last_change":"2024-07-04T17:33:09.126404+0000","last_active":"2024-07-04T17:33:30.413950+0000","last_peered":"2024-07-04T17:33:30.413950+0000","last_clean":"2024-07-04T17:33:30.413950+0000","last_became_active":"2024-07-04T17:33:09.126069+0000","last_became_peered":"2024-07-04T17:33:09.126069+0000","last_unstale":"2024-07-04T17:33:30.413950+0000","last_undegraded":"2024-07-04T17:33:30.413950+0000","last_fullsized":"2024-07-04T17:33:30.413950+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:54:17.261721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.435743+0000","last_change":"2024-07-04T17:33:29.456026+0000","last_active":"2024-07-04T17:33:30.435743+0000","last_peered":"2024-07-04T17:33:30.435743+0000","last_clean":"2024-07-04T17:33:30.435743+0000","last_became_active":"2024-07-04T17:33:29.455689+0000","last_became_peered":"2024-07-04T17:33:29.455689+0000","last_unstale":"2024-07-04T17:33:30.435743+0000","last_undegraded":"2024-07-04T17:33:30.435743+0000","last_fullsized":"2024-07-04T17:33:30.435743+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:47:36.209825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.436526+0000","last_change":"2024-07-04T17:33:09.104422+0000","last_active":"2024-07-04T17:33:30.436526+0000","last_peered":"2024-07-04T17:33:30.436526+0000","last_clean":"2024-07-04T17:33:30.436526+0000","last_became_active":"2024-07-04T17:33:09.104112+0000","last_became_peered":"2024-07-04T17:33:09.104112+0000","last_unstale":"2024-07-04T17:33:30.436526+0000","last_undegraded":"2024-07-04T17:33:30.436526+0000","last_fullsized":"2024-07-04T17:33:30.436526+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:50:45.806246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444066+0000","last_change":"2024-07-04T17:33:29.436496+0000","last_active":"2024-07-04T17:33:30.444066+0000","last_peered":"2024-07-04T17:33:30.444066+0000","last_clean":"2024-07-04T17:33:30.444066+0000","last_became_active":"2024-07-04T17:33:29.436176+0000","last_became_peered":"2024-07-04T17:33:29.436176+0000","last_unstale":"2024-07-04T17:33:30.444066+0000","last_undegraded":"2024-07-04T17:33:30.444066+0000","last_fullsized":"2024-07-04T17:33:30.444066+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:48:31.824457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.412655+0000","last_change":"2024-07-04T17:33:09.111717+0000","last_active":"2024-07-04T17:33:30.412655+0000","last_peered":"2024-07-04T17:33:30.412655+0000","last_clean":"2024-07-04T17:33:30.412655+0000","last_became_active":"2024-07-04T17:33:09.110270+0000","last_became_peered":"2024-07-04T17:33:09.110270+0000","last_unstale":"2024-07-04T17:33:30.412655+0000","last_undegraded":"2024-07-04T17:33:30.412655+0000","last_fullsized":"2024-07-04T17:33:30.412655+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:18:53.013321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444157+0000","last_change":"2024-07-04T17:33:29.435180+0000","last_active":"2024-07-04T17:33:30.444157+0000","last_peered":"2024-07-04T17:33:30.444157+0000","last_clean":"2024-07-04T17:33:30.444157+0000","last_became_active":"2024-07-04T17:33:29.429212+0000","last_became_peered":"2024-07-04T17:33:29.429212+0000","last_unstale":"2024-07-04T17:33:30.444157+0000","last_undegraded":"2024-07-04T17:33:30.444157+0000","last_fullsized":"2024-07-04T17:33:30.444157+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:25:44.819023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.412205+0000","last_change":"2024-07-04T17:33:09.108466+0000","last_active":"2024-07-04T17:33:30.412205+0000","last_peered":"2024-07-04T17:33:30.412205+0000","last_clean":"2024-07-04T17:33:30.412205+0000","last_became_active":"2024-07-04T17:33:09.108132+0000","last_became_peered":"2024-07-04T17:33:09.108132+0000","last_unstale":"2024-07-04T17:33:30.412205+0000","last_undegraded":"2024-07-04T17:33:30.412205+0000","last_fullsized":"2024-07-04T17:33:30.412205+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:51:32.675966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.413460+0000","last_change":"2024-07-04T17:33:29.430628+0000","last_active":"2024-07-04T17:33:30.413460+0000","last_peered":"2024-07-04T17:33:30.413460+0000","last_clean":"2024-07-04T17:33:30.413460+0000","last_became_active":"2024-07-04T17:33:29.430257+0000","last_became_peered":"2024-07-04T17:33:29.430257+0000","last_unstale":"2024-07-04T17:33:30.413460+0000","last_undegraded":"2024-07-04T17:33:30.413460+0000","last_fullsized":"2024-07-04T17:33:30.413460+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:55:08.001999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444598+0000","last_change":"2024-07-04T17:33:09.108992+0000","last_active":"2024-07-04T17:33:30.444598+0000","last_peered":"2024-07-04T17:33:30.444598+0000","last_clean":"2024-07-04T17:33:30.444598+0000","last_became_active":"2024-07-04T17:33:09.107307+0000","last_became_peered":"2024-07-04T17:33:09.107307+0000","last_unstale":"2024-07-04T17:33:30.444598+0000","last_undegraded":"2024-07-04T17:33:30.444598+0000","last_fullsized":"2024-07-04T17:33:30.444598+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:32:07.600844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.434130+0000","last_change":"2024-07-04T17:33:29.449127+0000","last_active":"2024-07-04T17:33:30.434130+0000","last_peered":"2024-07-04T17:33:30.434130+0000","last_clean":"2024-07-04T17:33:30.434130+0000","last_became_active":"2024-07-04T17:33:29.448718+0000","last_became_peered":"2024-07-04T17:33:29.448718+0000","last_unstale":"2024-07-04T17:33:30.434130+0000","last_undegraded":"2024-07-04T17:33:30.434130+0000","last_fullsized":"2024-07-04T17:33:30.434130+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:41:41.582097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.412546+0000","last_change":"2024-07-04T17:33:09.127659+0000","last_active":"2024-07-04T17:33:30.412546+0000","last_peered":"2024-07-04T17:33:30.412546+0000","last_clean":"2024-07-04T17:33:30.412546+0000","last_became_active":"2024-07-04T17:33:09.126591+0000","last_became_peered":"2024-07-04T17:33:09.126591+0000","last_unstale":"2024-07-04T17:33:30.412546+0000","last_undegraded":"2024-07-04T17:33:30.412546+0000","last_fullsized":"2024-07-04T17:33:30.412546+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:44:07.778739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.413407+0000","last_change":"2024-07-04T17:33:29.460618+0000","last_active":"2024-07-04T17:33:30.413407+0000","last_peered":"2024-07-04T17:33:30.413407+0000","last_clean":"2024-07-04T17:33:30.413407+0000","last_became_active":"2024-07-04T17:33:29.460215+0000","last_became_peered":"2024-07-04T17:33:29.460215+0000","last_unstale":"2024-07-04T17:33:30.413407+0000","last_undegraded":"2024-07-04T17:33:30.413407+0000","last_fullsized":"2024-07-04T17:33:30.413407+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:46:25.800079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.413757+0000","last_change":"2024-07-04T17:33:09.110971+0000","last_active":"2024-07-04T17:33:30.413757+0000","last_peered":"2024-07-04T17:33:30.413757+0000","last_clean":"2024-07-04T17:33:30.413757+0000","last_became_active":"2024-07-04T17:33:09.109775+0000","last_became_peered":"2024-07-04T17:33:09.109775+0000","last_unstale":"2024-07-04T17:33:30.413757+0000","last_undegraded":"2024-07-04T17:33:30.413757+0000","last_fullsized":"2024-07-04T17:33:30.413757+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:27:36.573799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.415780+0000","last_change":"2024-07-04T17:33:29.443487+0000","last_active":"2024-07-04T17:33:30.415780+0000","last_peered":"2024-07-04T17:33:30.415780+0000","last_clean":"2024-07-04T17:33:30.415780+0000","last_became_active":"2024-07-04T17:33:29.443160+0000","last_became_peered":"2024-07-04T17:33:29.443160+0000","last_unstale":"2024-07-04T17:33:30.415780+0000","last_undegraded":"2024-07-04T17:33:30.415780+0000","last_fullsized":"2024-07-04T17:33:30.415780+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:29:22.695717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.414901+0000","last_change":"2024-07-04T17:33:09.127783+0000","last_active":"2024-07-04T17:33:30.414901+0000","last_peered":"2024-07-04T17:33:30.414901+0000","last_clean":"2024-07-04T17:33:30.414901+0000","last_became_active":"2024-07-04T17:33:09.125335+0000","last_became_peered":"2024-07-04T17:33:09.125335+0000","last_unstale":"2024-07-04T17:33:30.414901+0000","last_undegraded":"2024-07-04T17:33:30.414901+0000","last_fullsized":"2024-07-04T17:33:30.414901+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:29:15.785171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.447780+0000","last_change":"2024-07-04T17:33:29.445956+0000","last_active":"2024-07-04T17:33:30.447780+0000","last_peered":"2024-07-04T17:33:30.447780+0000","last_clean":"2024-07-04T17:33:30.447780+0000","last_became_active":"2024-07-04T17:33:29.445245+0000","last_became_peered":"2024-07-04T17:33:29.445245+0000","last_unstale":"2024-07-04T17:33:30.447780+0000","last_undegraded":"2024-07-04T17:33:30.447780+0000","last_fullsized":"2024-07-04T17:33:30.447780+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:23:14.047614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.414185+0000","last_change":"2024-07-04T17:33:09.113535+0000","last_active":"2024-07-04T17:33:30.414185+0000","last_peered":"2024-07-04T17:33:30.414185+0000","last_clean":"2024-07-04T17:33:30.414185+0000","last_became_active":"2024-07-04T17:33:09.113127+0000","last_became_peered":"2024-07-04T17:33:09.113127+0000","last_unstale":"2024-07-04T17:33:30.414185+0000","last_undegraded":"2024-07-04T17:33:30.414185+0000","last_fullsized":"2024-07-04T17:33:30.414185+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:14:08.620654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.446960+0000","last_change":"2024-07-04T17:33:29.433615+0000","last_active":"2024-07-04T17:33:30.446960+0000","last_peered":"2024-07-04T17:33:30.446960+0000","last_clean":"2024-07-04T17:33:30.446960+0000","last_became_active":"2024-07-04T17:33:29.432557+0000","last_became_peered":"2024-07-04T17:33:29.432557+0000","last_unstale":"2024-07-04T17:33:30.446960+0000","last_undegraded":"2024-07-04T17:33:30.446960+0000","last_fullsized":"2024-07-04T17:33:30.446960+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:58:57.336702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.446374+0000","last_change":"2024-07-04T17:33:09.112587+0000","last_active":"2024-07-04T17:33:30.446374+0000","last_peered":"2024-07-04T17:33:30.446374+0000","last_clean":"2024-07-04T17:33:30.446374+0000","last_became_active":"2024-07-04T17:33:09.112063+0000","last_became_peered":"2024-07-04T17:33:09.112063+0000","last_unstale":"2024-07-04T17:33:30.446374+0000","last_undegraded":"2024-07-04T17:33:30.446374+0000","last_fullsized":"2024-07-04T17:33:30.446374+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:59:39.105749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444127+0000","last_change":"2024-07-04T17:33:29.443001+0000","last_active":"2024-07-04T17:33:30.444127+0000","last_peered":"2024-07-04T17:33:30.444127+0000","last_clean":"2024-07-04T17:33:30.444127+0000","last_became_active":"2024-07-04T17:33:29.442427+0000","last_became_peered":"2024-07-04T17:33:29.442427+0000","last_unstale":"2024-07-04T17:33:30.444127+0000","last_undegraded":"2024-07-04T17:33:30.444127+0000","last_fullsized":"2024-07-04T17:33:30.444127+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:14:10.024394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.412821+0000","last_change":"2024-07-04T17:33:09.110573+0000","last_active":"2024-07-04T17:33:30.412821+0000","last_peered":"2024-07-04T17:33:30.412821+0000","last_clean":"2024-07-04T17:33:30.412821+0000","last_became_active":"2024-07-04T17:33:09.109386+0000","last_became_peered":"2024-07-04T17:33:09.109386+0000","last_unstale":"2024-07-04T17:33:30.412821+0000","last_undegraded":"2024-07-04T17:33:30.412821+0000","last_fullsized":"2024-07-04T17:33:30.412821+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:33:02.261126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444083+0000","last_change":"2024-07-04T17:33:29.438831+0000","last_active":"2024-07-04T17:33:30.444083+0000","last_peered":"2024-07-04T17:33:30.444083+0000","last_clean":"2024-07-04T17:33:30.444083+0000","last_became_active":"2024-07-04T17:33:29.438485+0000","last_became_peered":"2024-07-04T17:33:29.438485+0000","last_unstale":"2024-07-04T17:33:30.444083+0000","last_undegraded":"2024-07-04T17:33:30.444083+0000","last_fullsized":"2024-07-04T17:33:30.444083+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:14:36.275709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.445410+0000","last_change":"2024-07-04T17:33:09.119181+0000","last_active":"2024-07-04T17:33:30.445410+0000","last_peered":"2024-07-04T17:33:30.445410+0000","last_clean":"2024-07-04T17:33:30.445410+0000","last_became_active":"2024-07-04T17:33:09.118791+0000","last_became_peered":"2024-07-04T17:33:09.118791+0000","last_unstale":"2024-07-04T17:33:30.445410+0000","last_undegraded":"2024-07-04T17:33:30.445410+0000","last_fullsized":"2024-07-04T17:33:30.445410+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:08:30.473430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.447063+0000","last_change":"2024-07-04T17:33:29.422486+0000","last_active":"2024-07-04T17:33:30.447063+0000","last_peered":"2024-07-04T17:33:30.447063+0000","last_clean":"2024-07-04T17:33:30.447063+0000","last_became_active":"2024-07-04T17:33:29.420743+0000","last_became_peered":"2024-07-04T17:33:29.420743+0000","last_unstale":"2024-07-04T17:33:30.447063+0000","last_undegraded":"2024-07-04T17:33:30.447063+0000","last_fullsized":"2024-07-04T17:33:30.447063+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:35:34.544233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.412057+0000","last_change":"2024-07-04T17:33:09.111491+0000","last_active":"2024-07-04T17:33:30.412057+0000","last_peered":"2024-07-04T17:33:30.412057+0000","last_clean":"2024-07-04T17:33:30.412057+0000","last_became_active":"2024-07-04T17:33:09.111095+0000","last_became_peered":"2024-07-04T17:33:09.111095+0000","last_unstale":"2024-07-04T17:33:30.412057+0000","last_undegraded":"2024-07-04T17:33:30.412057+0000","last_fullsized":"2024-07-04T17:33:30.412057+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:49:13.666901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444105+0000","last_change":"2024-07-04T17:33:29.446498+0000","last_active":"2024-07-04T17:33:30.444105+0000","last_peered":"2024-07-04T17:33:30.444105+0000","last_clean":"2024-07-04T17:33:30.444105+0000","last_became_active":"2024-07-04T17:33:29.446127+0000","last_became_peered":"2024-07-04T17:33:29.446127+0000","last_unstale":"2024-07-04T17:33:30.444105+0000","last_undegraded":"2024-07-04T17:33:30.444105+0000","last_fullsized":"2024-07-04T17:33:30.444105+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:45:31.742540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.433639+0000","last_change":"2024-07-04T17:33:09.101446+0000","last_active":"2024-07-04T17:33:30.433639+0000","last_peered":"2024-07-04T17:33:30.433639+0000","last_clean":"2024-07-04T17:33:30.433639+0000","last_became_active":"2024-07-04T17:33:09.100644+0000","last_became_peered":"2024-07-04T17:33:09.100644+0000","last_unstale":"2024-07-04T17:33:30.433639+0000","last_undegraded":"2024-07-04T17:33:30.433639+0000","last_fullsized":"2024-07-04T17:33:30.433639+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:27:36.573799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.438186+0000","last_change":"2024-07-04T17:33:29.442123+0000","last_active":"2024-07-04T17:33:30.438186+0000","last_peered":"2024-07-04T17:33:30.438186+0000","last_clean":"2024-07-04T17:33:30.438186+0000","last_became_active":"2024-07-04T17:33:29.441776+0000","last_became_peered":"2024-07-04T17:33:29.441776+0000","last_unstale":"2024-07-04T17:33:30.438186+0000","last_undegraded":"2024-07-04T17:33:30.438186+0000","last_fullsized":"2024-07-04T17:33:30.438186+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:00:42.445313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.439148+0000","last_change":"2024-07-04T17:33:09.124916+0000","last_active":"2024-07-04T17:33:30.439148+0000","last_peered":"2024-07-04T17:33:30.439148+0000","last_clean":"2024-07-04T17:33:30.439148+0000","last_became_active":"2024-07-04T17:33:09.124557+0000","last_became_peered":"2024-07-04T17:33:09.124557+0000","last_unstale":"2024-07-04T17:33:30.439148+0000","last_undegraded":"2024-07-04T17:33:30.439148+0000","last_fullsized":"2024-07-04T17:33:30.439148+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:10:26.737295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.435469+0000","last_change":"2024-07-04T17:33:29.451690+0000","last_active":"2024-07-04T17:33:30.435469+0000","last_peered":"2024-07-04T17:33:30.435469+0000","last_clean":"2024-07-04T17:33:30.435469+0000","last_became_active":"2024-07-04T17:33:29.451059+0000","last_became_peered":"2024-07-04T17:33:29.451059+0000","last_unstale":"2024-07-04T17:33:30.435469+0000","last_undegraded":"2024-07-04T17:33:30.435469+0000","last_fullsized":"2024-07-04T17:33:30.435469+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:35:11.071722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.415085+0000","last_change":"2024-07-04T17:33:09.115371+0000","last_active":"2024-07-04T17:33:30.415085+0000","last_peered":"2024-07-04T17:33:30.415085+0000","last_clean":"2024-07-04T17:33:30.415085+0000","last_became_active":"2024-07-04T17:33:09.114560+0000","last_became_peered":"2024-07-04T17:33:09.114560+0000","last_unstale":"2024-07-04T17:33:30.415085+0000","last_undegraded":"2024-07-04T17:33:30.415085+0000","last_fullsized":"2024-07-04T17:33:30.415085+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:39:29.413889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.439734+0000","last_change":"2024-07-04T17:33:29.450333+0000","last_active":"2024-07-04T17:33:30.439734+0000","last_peered":"2024-07-04T17:33:30.439734+0000","last_clean":"2024-07-04T17:33:30.439734+0000","last_became_active":"2024-07-04T17:33:29.449918+0000","last_became_peered":"2024-07-04T17:33:29.449918+0000","last_unstale":"2024-07-04T17:33:30.439734+0000","last_undegraded":"2024-07-04T17:33:30.439734+0000","last_fullsized":"2024-07-04T17:33:30.439734+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:44:58.705067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.414480+0000","last_change":"2024-07-04T17:33:09.113151+0000","last_active":"2024-07-04T17:33:30.414480+0000","last_peered":"2024-07-04T17:33:30.414480+0000","last_clean":"2024-07-04T17:33:30.414480+0000","last_became_active":"2024-07-04T17:33:09.112228+0000","last_became_peered":"2024-07-04T17:33:09.112228+0000","last_unstale":"2024-07-04T17:33:30.414480+0000","last_undegraded":"2024-07-04T17:33:30.414480+0000","last_fullsized":"2024-07-04T17:33:30.414480+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:16:18.941587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.414686+0000","last_change":"2024-07-04T17:33:29.447697+0000","last_active":"2024-07-04T17:33:30.414686+0000","last_peered":"2024-07-04T17:33:30.414686+0000","last_clean":"2024-07-04T17:33:30.414686+0000","last_became_active":"2024-07-04T17:33:29.447194+0000","last_became_peered":"2024-07-04T17:33:29.447194+0000","last_unstale":"2024-07-04T17:33:30.414686+0000","last_undegraded":"2024-07-04T17:33:30.414686+0000","last_fullsized":"2024-07-04T17:33:30.414686+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:22:48.866984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.436742+0000","last_change":"2024-07-04T17:33:09.118128+0000","last_active":"2024-07-04T17:33:30.436742+0000","last_peered":"2024-07-04T17:33:30.436742+0000","last_clean":"2024-07-04T17:33:30.436742+0000","last_became_active":"2024-07-04T17:33:09.117473+0000","last_became_peered":"2024-07-04T17:33:09.117473+0000","last_unstale":"2024-07-04T17:33:30.436742+0000","last_undegraded":"2024-07-04T17:33:30.436742+0000","last_fullsized":"2024-07-04T17:33:30.436742+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:06:51.425997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.434772+0000","last_change":"2024-07-04T17:33:29.462800+0000","last_active":"2024-07-04T17:33:30.434772+0000","last_peered":"2024-07-04T17:33:30.434772+0000","last_clean":"2024-07-04T17:33:30.434772+0000","last_became_active":"2024-07-04T17:33:29.462567+0000","last_became_peered":"2024-07-04T17:33:29.462567+0000","last_unstale":"2024-07-04T17:33:30.434772+0000","last_undegraded":"2024-07-04T17:33:30.434772+0000","last_fullsized":"2024-07-04T17:33:30.434772+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:45:04.147190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.435764+0000","last_change":"2024-07-04T17:33:09.127162+0000","last_active":"2024-07-04T17:33:30.435764+0000","last_peered":"2024-07-04T17:33:30.435764+0000","last_clean":"2024-07-04T17:33:30.435764+0000","last_became_active":"2024-07-04T17:33:09.123389+0000","last_became_peered":"2024-07-04T17:33:09.123389+0000","last_unstale":"2024-07-04T17:33:30.435764+0000","last_undegraded":"2024-07-04T17:33:30.435764+0000","last_fullsized":"2024-07-04T17:33:30.435764+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:33:17.955726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.412632+0000","last_change":"2024-07-04T17:33:29.434386+0000","last_active":"2024-07-04T17:33:30.412632+0000","last_peered":"2024-07-04T17:33:30.412632+0000","last_clean":"2024-07-04T17:33:30.412632+0000","last_became_active":"2024-07-04T17:33:29.431792+0000","last_became_peered":"2024-07-04T17:33:29.431792+0000","last_unstale":"2024-07-04T17:33:30.412632+0000","last_undegraded":"2024-07-04T17:33:30.412632+0000","last_fullsized":"2024-07-04T17:33:30.412632+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:48:23.639508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.445498+0000","last_change":"2024-07-04T17:33:09.118567+0000","last_active":"2024-07-04T17:33:30.445498+0000","last_peered":"2024-07-04T17:33:30.445498+0000","last_clean":"2024-07-04T17:33:30.445498+0000","last_became_active":"2024-07-04T17:33:09.117227+0000","last_became_peered":"2024-07-04T17:33:09.117227+0000","last_unstale":"2024-07-04T17:33:30.445498+0000","last_undegraded":"2024-07-04T17:33:30.445498+0000","last_fullsized":"2024-07-04T17:33:30.445498+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:42:43.320358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.436762+0000","last_change":"2024-07-04T17:33:29.452959+0000","last_active":"2024-07-04T17:33:30.436762+0000","last_peered":"2024-07-04T17:33:30.436762+0000","last_clean":"2024-07-04T17:33:30.436762+0000","last_became_active":"2024-07-04T17:33:29.451873+0000","last_became_peered":"2024-07-04T17:33:29.451873+0000","last_unstale":"2024-07-04T17:33:30.436762+0000","last_undegraded":"2024-07-04T17:33:30.436762+0000","last_fullsized":"2024-07-04T17:33:30.436762+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:37:16.694086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.434211+0000","last_change":"2024-07-04T17:33:09.131948+0000","last_active":"2024-07-04T17:33:30.434211+0000","last_peered":"2024-07-04T17:33:30.434211+0000","last_clean":"2024-07-04T17:33:30.434211+0000","last_became_active":"2024-07-04T17:33:09.121717+0000","last_became_peered":"2024-07-04T17:33:09.121717+0000","last_unstale":"2024-07-04T17:33:30.434211+0000","last_undegraded":"2024-07-04T17:33:30.434211+0000","last_fullsized":"2024-07-04T17:33:30.434211+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:05:18.766887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.411993+0000","last_change":"2024-07-04T17:33:29.456681+0000","last_active":"2024-07-04T17:33:30.411993+0000","last_peered":"2024-07-04T17:33:30.411993+0000","last_clean":"2024-07-04T17:33:30.411993+0000","last_became_active":"2024-07-04T17:33:29.456262+0000","last_became_peered":"2024-07-04T17:33:29.456262+0000","last_unstale":"2024-07-04T17:33:30.411993+0000","last_undegraded":"2024-07-04T17:33:30.411993+0000","last_fullsized":"2024-07-04T17:33:30.411993+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:16:21.298639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.412705+0000","last_change":"2024-07-04T17:33:09.124489+0000","last_active":"2024-07-04T17:33:30.412705+0000","last_peered":"2024-07-04T17:33:30.412705+0000","last_clean":"2024-07-04T17:33:30.412705+0000","last_became_active":"2024-07-04T17:33:09.122889+0000","last_became_peered":"2024-07-04T17:33:09.122889+0000","last_unstale":"2024-07-04T17:33:30.412705+0000","last_undegraded":"2024-07-04T17:33:30.412705+0000","last_fullsized":"2024-07-04T17:33:30.412705+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:01:35.821887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.413627+0000","last_change":"2024-07-04T17:33:29.460554+0000","last_active":"2024-07-04T17:33:30.413627+0000","last_peered":"2024-07-04T17:33:30.413627+0000","last_clean":"2024-07-04T17:33:30.413627+0000","last_became_active":"2024-07-04T17:33:29.459577+0000","last_became_peered":"2024-07-04T17:33:29.459577+0000","last_unstale":"2024-07-04T17:33:30.413627+0000","last_undegraded":"2024-07-04T17:33:30.413627+0000","last_fullsized":"2024-07-04T17:33:30.413627+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:08:40.734996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.439265+0000","last_change":"2024-07-04T17:33:09.118397+0000","last_active":"2024-07-04T17:33:30.439265+0000","last_peered":"2024-07-04T17:33:30.439265+0000","last_clean":"2024-07-04T17:33:30.439265+0000","last_became_active":"2024-07-04T17:33:09.117509+0000","last_became_peered":"2024-07-04T17:33:09.117509+0000","last_unstale":"2024-07-04T17:33:30.439265+0000","last_undegraded":"2024-07-04T17:33:30.439265+0000","last_fullsized":"2024-07-04T17:33:30.439265+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:37:23.202824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.415159+0000","last_change":"2024-07-04T17:33:29.452392+0000","last_active":"2024-07-04T17:33:30.415159+0000","last_peered":"2024-07-04T17:33:30.415159+0000","last_clean":"2024-07-04T17:33:30.415159+0000","last_became_active":"2024-07-04T17:33:29.451644+0000","last_became_peered":"2024-07-04T17:33:29.451644+0000","last_unstale":"2024-07-04T17:33:30.415159+0000","last_undegraded":"2024-07-04T17:33:30.415159+0000","last_fullsized":"2024-07-04T17:33:30.415159+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:37:06.907567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.415132+0000","last_change":"2024-07-04T17:33:09.115358+0000","last_active":"2024-07-04T17:33:30.415132+0000","last_peered":"2024-07-04T17:33:30.415132+0000","last_clean":"2024-07-04T17:33:30.415132+0000","last_became_active":"2024-07-04T17:33:09.113650+0000","last_became_peered":"2024-07-04T17:33:09.113650+0000","last_unstale":"2024-07-04T17:33:30.415132+0000","last_undegraded":"2024-07-04T17:33:30.415132+0000","last_fullsized":"2024-07-04T17:33:30.415132+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:34:20.388421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.414069+0000","last_change":"2024-07-04T17:33:29.442803+0000","last_active":"2024-07-04T17:33:30.414069+0000","last_peered":"2024-07-04T17:33:30.414069+0000","last_clean":"2024-07-04T17:33:30.414069+0000","last_became_active":"2024-07-04T17:33:29.442473+0000","last_became_peered":"2024-07-04T17:33:29.442473+0000","last_unstale":"2024-07-04T17:33:30.414069+0000","last_undegraded":"2024-07-04T17:33:30.414069+0000","last_fullsized":"2024-07-04T17:33:30.414069+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:49:43.858447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.436722+0000","last_change":"2024-07-04T17:33:09.130921+0000","last_active":"2024-07-04T17:33:30.436722+0000","last_peered":"2024-07-04T17:33:30.436722+0000","last_clean":"2024-07-04T17:33:30.436722+0000","last_became_active":"2024-07-04T17:33:09.123521+0000","last_became_peered":"2024-07-04T17:33:09.123521+0000","last_unstale":"2024-07-04T17:33:30.436722+0000","last_undegraded":"2024-07-04T17:33:30.436722+0000","last_fullsized":"2024-07-04T17:33:30.436722+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:13:51.565345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.447258+0000","last_change":"2024-07-04T17:33:29.444474+0000","last_active":"2024-07-04T17:33:30.447258+0000","last_peered":"2024-07-04T17:33:30.447258+0000","last_clean":"2024-07-04T17:33:30.447258+0000","last_became_active":"2024-07-04T17:33:29.444096+0000","last_became_peered":"2024-07-04T17:33:29.444096+0000","last_unstale":"2024-07-04T17:33:30.447258+0000","last_undegraded":"2024-07-04T17:33:30.447258+0000","last_fullsized":"2024-07-04T17:33:30.447258+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:48:54.442775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.446331+0000","last_change":"2024-07-04T17:33:09.114604+0000","last_active":"2024-07-04T17:33:30.446331+0000","last_peered":"2024-07-04T17:33:30.446331+0000","last_clean":"2024-07-04T17:33:30.446331+0000","last_became_active":"2024-07-04T17:33:09.114294+0000","last_became_peered":"2024-07-04T17:33:09.114294+0000","last_unstale":"2024-07-04T17:33:30.446331+0000","last_undegraded":"2024-07-04T17:33:30.446331+0000","last_fullsized":"2024-07-04T17:33:30.446331+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:34:50.754812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.433534+0000","last_change":"2024-07-04T17:33:29.453534+0000","last_active":"2024-07-04T17:33:30.433534+0000","last_peered":"2024-07-04T17:33:30.433534+0000","last_clean":"2024-07-04T17:33:30.433534+0000","last_became_active":"2024-07-04T17:33:29.448780+0000","last_became_peered":"2024-07-04T17:33:29.448780+0000","last_unstale":"2024-07-04T17:33:30.433534+0000","last_undegraded":"2024-07-04T17:33:30.433534+0000","last_fullsized":"2024-07-04T17:33:30.433534+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:27:44.574496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.412993+0000","last_change":"2024-07-04T17:33:09.124878+0000","last_active":"2024-07-04T17:33:30.412993+0000","last_peered":"2024-07-04T17:33:30.412993+0000","last_clean":"2024-07-04T17:33:30.412993+0000","last_became_active":"2024-07-04T17:33:09.124119+0000","last_became_peered":"2024-07-04T17:33:09.124119+0000","last_unstale":"2024-07-04T17:33:30.412993+0000","last_undegraded":"2024-07-04T17:33:30.412993+0000","last_fullsized":"2024-07-04T17:33:30.412993+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:18:22.081730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444421+0000","last_change":"2024-07-04T17:33:29.426691+0000","last_active":"2024-07-04T17:33:30.444421+0000","last_peered":"2024-07-04T17:33:30.444421+0000","last_clean":"2024-07-04T17:33:30.444421+0000","last_became_active":"2024-07-04T17:33:29.426009+0000","last_became_peered":"2024-07-04T17:33:29.426009+0000","last_unstale":"2024-07-04T17:33:30.444421+0000","last_undegraded":"2024-07-04T17:33:30.444421+0000","last_fullsized":"2024-07-04T17:33:30.444421+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:57:14.024066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.445613+0000","last_change":"2024-07-04T17:33:09.104697+0000","last_active":"2024-07-04T17:33:30.445613+0000","last_peered":"2024-07-04T17:33:30.445613+0000","last_clean":"2024-07-04T17:33:30.445613+0000","last_became_active":"2024-07-04T17:33:09.103509+0000","last_became_peered":"2024-07-04T17:33:09.103509+0000","last_unstale":"2024-07-04T17:33:30.445613+0000","last_undegraded":"2024-07-04T17:33:30.445613+0000","last_fullsized":"2024-07-04T17:33:30.445613+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:05:17.882764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.436344+0000","last_change":"2024-07-04T17:33:29.452531+0000","last_active":"2024-07-04T17:33:30.436344+0000","last_peered":"2024-07-04T17:33:30.436344+0000","last_clean":"2024-07-04T17:33:30.436344+0000","last_became_active":"2024-07-04T17:33:29.452204+0000","last_became_peered":"2024-07-04T17:33:29.452204+0000","last_unstale":"2024-07-04T17:33:30.436344+0000","last_undegraded":"2024-07-04T17:33:30.436344+0000","last_fullsized":"2024-07-04T17:33:30.436344+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:41:05.853324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444824+0000","last_change":"2024-07-04T17:33:09.104054+0000","last_active":"2024-07-04T17:33:30.444824+0000","last_peered":"2024-07-04T17:33:30.444824+0000","last_clean":"2024-07-04T17:33:30.444824+0000","last_became_active":"2024-07-04T17:33:09.103713+0000","last_became_peered":"2024-07-04T17:33:09.103713+0000","last_unstale":"2024-07-04T17:33:30.444824+0000","last_undegraded":"2024-07-04T17:33:30.444824+0000","last_fullsized":"2024-07-04T17:33:30.444824+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:27:24.257586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444539+0000","last_change":"2024-07-04T17:33:29.444892+0000","last_active":"2024-07-04T17:33:30.444539+0000","last_peered":"2024-07-04T17:33:30.444539+0000","last_clean":"2024-07-04T17:33:30.444539+0000","last_became_active":"2024-07-04T17:33:29.444506+0000","last_became_peered":"2024-07-04T17:33:29.444506+0000","last_unstale":"2024-07-04T17:33:30.444539+0000","last_undegraded":"2024-07-04T17:33:30.444539+0000","last_fullsized":"2024-07-04T17:33:30.444539+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:02:54.958549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.433811+0000","last_change":"2024-07-04T17:33:09.122343+0000","last_active":"2024-07-04T17:33:30.433811+0000","last_peered":"2024-07-04T17:33:30.433811+0000","last_clean":"2024-07-04T17:33:30.433811+0000","last_became_active":"2024-07-04T17:33:09.121982+0000","last_became_peered":"2024-07-04T17:33:09.121982+0000","last_unstale":"2024-07-04T17:33:30.433811+0000","last_undegraded":"2024-07-04T17:33:30.433811+0000","last_fullsized":"2024-07-04T17:33:30.433811+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:55:41.686107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.435981+0000","last_change":"2024-07-04T17:33:29.453791+0000","last_active":"2024-07-04T17:33:30.435981+0000","last_peered":"2024-07-04T17:33:30.435981+0000","last_clean":"2024-07-04T17:33:30.435981+0000","last_became_active":"2024-07-04T17:33:29.453492+0000","last_became_peered":"2024-07-04T17:33:29.453492+0000","last_unstale":"2024-07-04T17:33:30.435981+0000","last_undegraded":"2024-07-04T17:33:30.435981+0000","last_fullsized":"2024-07-04T17:33:30.435981+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:35:11.793595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.413868+0000","last_change":"2024-07-04T17:33:28.430109+0000","last_active":"2024-07-04T17:33:30.413868+0000","last_peered":"2024-07-04T17:33:30.413868+0000","last_clean":"2024-07-04T17:33:30.413868+0000","last_became_active":"2024-07-04T17:33:28.428355+0000","last_became_peered":"2024-07-04T17:33:28.428355+0000","last_unstale":"2024-07-04T17:33:30.413868+0000","last_undegraded":"2024-07-04T17:33:30.413868+0000","last_fullsized":"2024-07-04T17:33:30.413868+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:10:37.736526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.415214+0000","last_change":"2024-07-04T17:33:09.115618+0000","last_active":"2024-07-04T17:33:30.415214+0000","last_peered":"2024-07-04T17:33:30.415214+0000","last_clean":"2024-07-04T17:33:30.415214+0000","last_became_active":"2024-07-04T17:33:09.115047+0000","last_became_peered":"2024-07-04T17:33:09.115047+0000","last_unstale":"2024-07-04T17:33:30.415214+0000","last_undegraded":"2024-07-04T17:33:30.415214+0000","last_fullsized":"2024-07-04T17:33:30.415214+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:38:58.383535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"113'4","reported_seq":26,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.414050+0000","last_change":"2024-07-04T17:33:08.117620+0000","last_active":"2024-07-04T17:33:30.414050+0000","last_peered":"2024-07-04T17:33:30.414050+0000","last_clean":"2024-07-04T17:33:30.414050+0000","last_became_active":"2024-07-04T17:33:08.117097+0000","last_became_peered":"2024-07-04T17:33:08.117097+0000","last_unstale":"2024-07-04T17:33:30.414050+0000","last_undegraded":"2024-07-04T17:33:30.414050+0000","last_fullsized":"2024-07-04T17:33:30.414050+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:00:27.380504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.447978+0000","last_change":"2024-07-04T17:33:29.438701+0000","last_active":"2024-07-04T17:33:30.447978+0000","last_peered":"2024-07-04T17:33:30.447978+0000","last_clean":"2024-07-04T17:33:30.447978+0000","last_became_active":"2024-07-04T17:33:29.427189+0000","last_became_peered":"2024-07-04T17:33:29.427189+0000","last_unstale":"2024-07-04T17:33:30.447978+0000","last_undegraded":"2024-07-04T17:33:30.447978+0000","last_fullsized":"2024-07-04T17:33:30.447978+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:29:19.275486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1c","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.415560+0000","last_change":"2024-07-04T17:33:28.421141+0000","last_active":"2024-07-04T17:33:30.415560+0000","last_peered":"2024-07-04T17:33:30.415560+0000","last_clean":"2024-07-04T17:33:30.415560+0000","last_became_active":"2024-07-04T17:33:28.420812+0000","last_became_peered":"2024-07-04T17:33:28.420812+0000","last_unstale":"2024-07-04T17:33:30.415560+0000","last_undegraded":"2024-07-04T17:33:30.415560+0000","last_fullsized":"2024-07-04T17:33:30.415560+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:08:37.812504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.439375+0000","last_change":"2024-07-04T17:33:09.133070+0000","last_active":"2024-07-04T17:33:30.439375+0000","last_peered":"2024-07-04T17:33:30.439375+0000","last_clean":"2024-07-04T17:33:30.439375+0000","last_became_active":"2024-07-04T17:33:09.132634+0000","last_became_peered":"2024-07-04T17:33:09.132634+0000","last_unstale":"2024-07-04T17:33:30.439375+0000","last_undegraded":"2024-07-04T17:33:30.439375+0000","last_fullsized":"2024-07-04T17:33:30.439375+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:42:13.182154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1e","version":"113'1","reported_seq":23,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.448851+0000","last_change":"2024-07-04T17:33:08.105107+0000","last_active":"2024-07-04T17:33:30.448851+0000","last_peered":"2024-07-04T17:33:30.448851+0000","last_clean":"2024-07-04T17:33:30.448851+0000","last_became_active":"2024-07-04T17:33:08.104670+0000","last_became_peered":"2024-07-04T17:33:08.104670+0000","last_unstale":"2024-07-04T17:33:30.448851+0000","last_undegraded":"2024-07-04T17:33:30.448851+0000","last_fullsized":"2024-07-04T17:33:30.448851+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:46:04.479168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.447524+0000","last_change":"2024-07-04T17:33:29.433713+0000","last_active":"2024-07-04T17:33:30.447524+0000","last_peered":"2024-07-04T17:33:30.447524+0000","last_clean":"2024-07-04T17:33:30.447524+0000","last_became_active":"2024-07-04T17:33:29.433373+0000","last_became_peered":"2024-07-04T17:33:29.433373+0000","last_unstale":"2024-07-04T17:33:30.447524+0000","last_undegraded":"2024-07-04T17:33:30.447524+0000","last_fullsized":"2024-07-04T17:33:30.447524+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:55:35.551291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"117'4","reported_seq":18,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.438539+0000","last_change":"2024-07-04T17:33:28.432934+0000","last_active":"2024-07-04T17:33:30.438539+0000","last_peered":"2024-07-04T17:33:30.438539+0000","last_clean":"2024-07-04T17:33:30.438539+0000","last_became_active":"2024-07-04T17:33:28.432353+0000","last_became_peered":"2024-07-04T17:33:28.432353+0000","last_unstale":"2024-07-04T17:33:30.438539+0000","last_undegraded":"2024-07-04T17:33:30.438539+0000","last_fullsized":"2024-07-04T17:33:30.438539+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:43:23.051302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.415936+0000","last_change":"2024-07-04T17:33:09.107265+0000","last_active":"2024-07-04T17:33:30.415936+0000","last_peered":"2024-07-04T17:33:30.415936+0000","last_clean":"2024-07-04T17:33:30.415936+0000","last_became_active":"2024-07-04T17:33:09.106633+0000","last_became_peered":"2024-07-04T17:33:09.106633+0000","last_unstale":"2024-07-04T17:33:30.415936+0000","last_undegraded":"2024-07-04T17:33:30.415936+0000","last_fullsized":"2024-07-04T17:33:30.415936+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:05:30.327143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"113'2","reported_seq":24,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.438292+0000","last_change":"2024-07-04T17:33:08.089824+0000","last_active":"2024-07-04T17:33:30.438292+0000","last_peered":"2024-07-04T17:33:30.438292+0000","last_clean":"2024-07-04T17:33:30.438292+0000","last_became_active":"2024-07-04T17:33:08.084113+0000","last_became_peered":"2024-07-04T17:33:08.084113+0000","last_unstale":"2024-07-04T17:33:30.438292+0000","last_undegraded":"2024-07-04T17:33:30.438292+0000","last_fullsized":"2024-07-04T17:33:30.438292+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:39:25.915226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.438462+0000","last_change":"2024-07-04T17:33:29.443535+0000","last_active":"2024-07-04T17:33:30.438462+0000","last_peered":"2024-07-04T17:33:30.438462+0000","last_clean":"2024-07-04T17:33:30.438462+0000","last_became_active":"2024-07-04T17:33:29.443073+0000","last_became_peered":"2024-07-04T17:33:29.443073+0000","last_unstale":"2024-07-04T17:33:30.438462+0000","last_undegraded":"2024-07-04T17:33:30.438462+0000","last_fullsized":"2024-07-04T17:33:30.438462+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:33:11.025783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1e","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.447614+0000","last_change":"2024-07-04T17:33:28.408215+0000","last_active":"2024-07-04T17:33:30.447614+0000","last_peered":"2024-07-04T17:33:30.447614+0000","last_clean":"2024-07-04T17:33:30.447614+0000","last_became_active":"2024-07-04T17:33:28.407844+0000","last_became_peered":"2024-07-04T17:33:28.407844+0000","last_unstale":"2024-07-04T17:33:30.447614+0000","last_undegraded":"2024-07-04T17:33:30.447614+0000","last_fullsized":"2024-07-04T17:33:30.447614+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:17:40.690935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.436619+0000","last_change":"2024-07-04T17:33:09.129582+0000","last_active":"2024-07-04T17:33:30.436619+0000","last_peered":"2024-07-04T17:33:30.436619+0000","last_clean":"2024-07-04T17:33:30.436619+0000","last_became_active":"2024-07-04T17:33:09.127839+0000","last_became_peered":"2024-07-04T17:33:09.127839+0000","last_unstale":"2024-07-04T17:33:30.436619+0000","last_undegraded":"2024-07-04T17:33:30.436619+0000","last_fullsized":"2024-07-04T17:33:30.436619+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:35:13.414729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"113'2","reported_seq":24,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.436890+0000","last_change":"2024-07-04T17:33:08.108862+0000","last_active":"2024-07-04T17:33:30.436890+0000","last_peered":"2024-07-04T17:33:30.436890+0000","last_clean":"2024-07-04T17:33:30.436890+0000","last_became_active":"2024-07-04T17:33:08.107761+0000","last_became_peered":"2024-07-04T17:33:08.107761+0000","last_unstale":"2024-07-04T17:33:30.436890+0000","last_undegraded":"2024-07-04T17:33:30.436890+0000","last_fullsized":"2024-07-04T17:33:30.436890+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:15:53.250590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.411488+0000","last_change":"2024-07-04T17:33:29.424087+0000","last_active":"2024-07-04T17:33:30.411488+0000","last_peered":"2024-07-04T17:33:30.411488+0000","last_clean":"2024-07-04T17:33:30.411488+0000","last_became_active":"2024-07-04T17:33:29.417359+0000","last_became_peered":"2024-07-04T17:33:29.417359+0000","last_unstale":"2024-07-04T17:33:30.411488+0000","last_undegraded":"2024-07-04T17:33:30.411488+0000","last_fullsized":"2024-07-04T17:33:30.411488+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:56:07.610007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.433946+0000","last_change":"2024-07-04T17:33:28.424837+0000","last_active":"2024-07-04T17:33:30.433946+0000","last_peered":"2024-07-04T17:33:30.433946+0000","last_clean":"2024-07-04T17:33:30.433946+0000","last_became_active":"2024-07-04T17:33:28.424496+0000","last_became_peered":"2024-07-04T17:33:28.424496+0000","last_unstale":"2024-07-04T17:33:30.433946+0000","last_undegraded":"2024-07-04T17:33:30.433946+0000","last_fullsized":"2024-07-04T17:33:30.433946+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:47:24.359112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.445853+0000","last_change":"2024-07-04T17:33:09.118310+0000","last_active":"2024-07-04T17:33:30.445853+0000","last_peered":"2024-07-04T17:33:30.445853+0000","last_clean":"2024-07-04T17:33:30.445853+0000","last_became_active":"2024-07-04T17:33:09.116852+0000","last_became_peered":"2024-07-04T17:33:09.116852+0000","last_unstale":"2024-07-04T17:33:30.445853+0000","last_undegraded":"2024-07-04T17:33:30.445853+0000","last_fullsized":"2024-07-04T17:33:30.445853+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:41:21.265187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.1b","version":"113'1","reported_seq":23,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.438224+0000","last_change":"2024-07-04T17:33:08.105642+0000","last_active":"2024-07-04T17:33:30.438224+0000","last_peered":"2024-07-04T17:33:30.438224+0000","last_clean":"2024-07-04T17:33:30.438224+0000","last_became_active":"2024-07-04T17:33:08.105297+0000","last_became_peered":"2024-07-04T17:33:08.105297+0000","last_unstale":"2024-07-04T17:33:30.438224+0000","last_undegraded":"2024-07-04T17:33:30.438224+0000","last_fullsized":"2024-07-04T17:33:30.438224+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:50:34.893649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.433487+0000","last_change":"2024-07-04T17:33:29.455771+0000","last_active":"2024-07-04T17:33:30.433487+0000","last_peered":"2024-07-04T17:33:30.433487+0000","last_clean":"2024-07-04T17:33:30.433487+0000","last_became_active":"2024-07-04T17:33:29.455458+0000","last_became_peered":"2024-07-04T17:33:29.455458+0000","last_unstale":"2024-07-04T17:33:30.433487+0000","last_undegraded":"2024-07-04T17:33:30.433487+0000","last_fullsized":"2024-07-04T17:33:30.433487+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:08:50.790340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444532+0000","last_change":"2024-07-04T17:33:28.408122+0000","last_active":"2024-07-04T17:33:30.444532+0000","last_peered":"2024-07-04T17:33:30.444532+0000","last_clean":"2024-07-04T17:33:30.444532+0000","last_became_active":"2024-07-04T17:33:28.407784+0000","last_became_peered":"2024-07-04T17:33:28.407784+0000","last_unstale":"2024-07-04T17:33:30.444532+0000","last_undegraded":"2024-07-04T17:33:30.444532+0000","last_fullsized":"2024-07-04T17:33:30.444532+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:33:01.100217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.435962+0000","last_change":"2024-07-04T17:33:09.116075+0000","last_active":"2024-07-04T17:33:30.435962+0000","last_peered":"2024-07-04T17:33:30.435962+0000","last_clean":"2024-07-04T17:33:30.435962+0000","last_became_active":"2024-07-04T17:33:09.115705+0000","last_became_peered":"2024-07-04T17:33:09.115705+0000","last_unstale":"2024-07-04T17:33:30.435962+0000","last_undegraded":"2024-07-04T17:33:30.435962+0000","last_fullsized":"2024-07-04T17:33:30.435962+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:52:50.422683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"113'1","reported_seq":23,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.412996+0000","last_change":"2024-07-04T17:33:08.116547+0000","last_active":"2024-07-04T17:33:30.412996+0000","last_peered":"2024-07-04T17:33:30.412996+0000","last_clean":"2024-07-04T17:33:30.412996+0000","last_became_active":"2024-07-04T17:33:08.116095+0000","last_became_peered":"2024-07-04T17:33:08.116095+0000","last_unstale":"2024-07-04T17:33:30.412996+0000","last_undegraded":"2024-07-04T17:33:30.412996+0000","last_fullsized":"2024-07-04T17:33:30.412996+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:31:44.812381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444370+0000","last_change":"2024-07-04T17:33:29.433466+0000","last_active":"2024-07-04T17:33:30.444370+0000","last_peered":"2024-07-04T17:33:30.444370+0000","last_clean":"2024-07-04T17:33:30.444370+0000","last_became_active":"2024-07-04T17:33:29.424893+0000","last_became_peered":"2024-07-04T17:33:29.424893+0000","last_unstale":"2024-07-04T17:33:30.444370+0000","last_undegraded":"2024-07-04T17:33:30.444370+0000","last_fullsized":"2024-07-04T17:33:30.444370+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:29:45.081943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.435225+0000","last_change":"2024-07-04T17:33:28.417247+0000","last_active":"2024-07-04T17:33:30.435225+0000","last_peered":"2024-07-04T17:33:30.435225+0000","last_clean":"2024-07-04T17:33:30.435225+0000","last_became_active":"2024-07-04T17:33:28.416942+0000","last_became_peered":"2024-07-04T17:33:28.416942+0000","last_unstale":"2024-07-04T17:33:30.435225+0000","last_undegraded":"2024-07-04T17:33:30.435225+0000","last_fullsized":"2024-07-04T17:33:30.435225+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:37:41.663967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.412807+0000","last_change":"2024-07-04T17:33:09.125116+0000","last_active":"2024-07-04T17:33:30.412807+0000","last_peered":"2024-07-04T17:33:30.412807+0000","last_clean":"2024-07-04T17:33:30.412807+0000","last_became_active":"2024-07-04T17:33:09.124103+0000","last_became_peered":"2024-07-04T17:33:09.124103+0000","last_unstale":"2024-07-04T17:33:30.412807+0000","last_undegraded":"2024-07-04T17:33:30.412807+0000","last_fullsized":"2024-07-04T17:33:30.412807+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:27:31.156139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.435099+0000","last_change":"2024-07-04T17:33:08.096904+0000","last_active":"2024-07-04T17:33:30.435099+0000","last_peered":"2024-07-04T17:33:30.435099+0000","last_clean":"2024-07-04T17:33:30.435099+0000","last_became_active":"2024-07-04T17:33:08.096590+0000","last_became_peered":"2024-07-04T17:33:08.096590+0000","last_unstale":"2024-07-04T17:33:30.435099+0000","last_undegraded":"2024-07-04T17:33:30.435099+0000","last_fullsized":"2024-07-04T17:33:30.435099+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:39:51.529018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.411926+0000","last_change":"2024-07-04T17:33:29.430845+0000","last_active":"2024-07-04T17:33:30.411926+0000","last_peered":"2024-07-04T17:33:30.411926+0000","last_clean":"2024-07-04T17:33:30.411926+0000","last_became_active":"2024-07-04T17:33:29.430473+0000","last_became_peered":"2024-07-04T17:33:29.430473+0000","last_unstale":"2024-07-04T17:33:30.411926+0000","last_undegraded":"2024-07-04T17:33:30.411926+0000","last_fullsized":"2024-07-04T17:33:30.411926+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:14:00.417647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.411871+0000","last_change":"2024-07-04T17:33:28.424065+0000","last_active":"2024-07-04T17:33:30.411871+0000","last_peered":"2024-07-04T17:33:30.411871+0000","last_clean":"2024-07-04T17:33:30.411871+0000","last_became_active":"2024-07-04T17:33:28.423732+0000","last_became_peered":"2024-07-04T17:33:28.423732+0000","last_unstale":"2024-07-04T17:33:30.411871+0000","last_undegraded":"2024-07-04T17:33:30.411871+0000","last_fullsized":"2024-07-04T17:33:30.411871+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:28:03.208000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.434349+0000","last_change":"2024-07-04T17:33:09.100755+0000","last_active":"2024-07-04T17:33:30.434349+0000","last_peered":"2024-07-04T17:33:30.434349+0000","last_clean":"2024-07-04T17:33:30.434349+0000","last_became_active":"2024-07-04T17:33:09.100389+0000","last_became_peered":"2024-07-04T17:33:09.100389+0000","last_unstale":"2024-07-04T17:33:30.434349+0000","last_undegraded":"2024-07-04T17:33:30.434349+0000","last_fullsized":"2024-07-04T17:33:30.434349+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:38:04.457302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.445604+0000","last_change":"2024-07-04T17:33:08.106647+0000","last_active":"2024-07-04T17:33:30.445604+0000","last_peered":"2024-07-04T17:33:30.445604+0000","last_clean":"2024-07-04T17:33:30.445604+0000","last_became_active":"2024-07-04T17:33:08.106277+0000","last_became_peered":"2024-07-04T17:33:08.106277+0000","last_unstale":"2024-07-04T17:33:30.445604+0000","last_undegraded":"2024-07-04T17:33:30.445604+0000","last_fullsized":"2024-07-04T17:33:30.445604+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:47:04.050710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.445698+0000","last_change":"2024-07-04T17:33:29.433719+0000","last_active":"2024-07-04T17:33:30.445698+0000","last_peered":"2024-07-04T17:33:30.445698+0000","last_clean":"2024-07-04T17:33:30.445698+0000","last_became_active":"2024-07-04T17:33:29.433351+0000","last_became_peered":"2024-07-04T17:33:29.433351+0000","last_unstale":"2024-07-04T17:33:30.445698+0000","last_undegraded":"2024-07-04T17:33:30.445698+0000","last_fullsized":"2024-07-04T17:33:30.445698+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:05:24.141848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.15","version":"117'3","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.436055+0000","last_change":"2024-07-04T17:33:28.430046+0000","last_active":"2024-07-04T17:33:30.436055+0000","last_peered":"2024-07-04T17:33:30.436055+0000","last_clean":"2024-07-04T17:33:30.436055+0000","last_became_active":"2024-07-04T17:33:28.429140+0000","last_became_peered":"2024-07-04T17:33:28.429140+0000","last_unstale":"2024-07-04T17:33:30.436055+0000","last_undegraded":"2024-07-04T17:33:30.436055+0000","last_fullsized":"2024-07-04T17:33:30.436055+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:47:01.462455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.438802+0000","last_change":"2024-07-04T17:33:09.128294+0000","last_active":"2024-07-04T17:33:30.438802+0000","last_peered":"2024-07-04T17:33:30.438802+0000","last_clean":"2024-07-04T17:33:30.438802+0000","last_became_active":"2024-07-04T17:33:09.123696+0000","last_became_peered":"2024-07-04T17:33:09.123696+0000","last_unstale":"2024-07-04T17:33:30.438802+0000","last_undegraded":"2024-07-04T17:33:30.438802+0000","last_fullsized":"2024-07-04T17:33:30.438802+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:07:05.456492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.414224+0000","last_change":"2024-07-04T17:33:08.100600+0000","last_active":"2024-07-04T17:33:30.414224+0000","last_peered":"2024-07-04T17:33:30.414224+0000","last_clean":"2024-07-04T17:33:30.414224+0000","last_became_active":"2024-07-04T17:33:08.099352+0000","last_became_peered":"2024-07-04T17:33:08.099352+0000","last_unstale":"2024-07-04T17:33:30.414224+0000","last_undegraded":"2024-07-04T17:33:30.414224+0000","last_fullsized":"2024-07-04T17:33:30.414224+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:04:33.482388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.435960+0000","last_change":"2024-07-04T17:33:29.452926+0000","last_active":"2024-07-04T17:33:30.435960+0000","last_peered":"2024-07-04T17:33:30.435960+0000","last_clean":"2024-07-04T17:33:30.435960+0000","last_became_active":"2024-07-04T17:33:29.452322+0000","last_became_peered":"2024-07-04T17:33:29.452322+0000","last_unstale":"2024-07-04T17:33:30.435960+0000","last_undegraded":"2024-07-04T17:33:30.435960+0000","last_fullsized":"2024-07-04T17:33:30.435960+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:23:18.968713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.14","version":"117'6","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.415753+0000","last_change":"2024-07-04T17:33:28.429133+0000","last_active":"2024-07-04T17:33:30.415753+0000","last_peered":"2024-07-04T17:33:30.415753+0000","last_clean":"2024-07-04T17:33:30.415753+0000","last_became_active":"2024-07-04T17:33:28.428803+0000","last_became_peered":"2024-07-04T17:33:28.428803+0000","last_unstale":"2024-07-04T17:33:30.415753+0000","last_undegraded":"2024-07-04T17:33:30.415753+0000","last_fullsized":"2024-07-04T17:33:30.415753+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:53:44.447019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.447846+0000","last_change":"2024-07-04T17:33:09.109950+0000","last_active":"2024-07-04T17:33:30.447846+0000","last_peered":"2024-07-04T17:33:30.447846+0000","last_clean":"2024-07-04T17:33:30.447846+0000","last_became_active":"2024-07-04T17:33:09.108850+0000","last_became_peered":"2024-07-04T17:33:09.108850+0000","last_unstale":"2024-07-04T17:33:30.447846+0000","last_undegraded":"2024-07-04T17:33:30.447846+0000","last_fullsized":"2024-07-04T17:33:30.447846+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:22:58.651803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.415604+0000","last_change":"2024-07-04T17:33:08.117619+0000","last_active":"2024-07-04T17:33:30.415604+0000","last_peered":"2024-07-04T17:33:30.415604+0000","last_clean":"2024-07-04T17:33:30.415604+0000","last_became_active":"2024-07-04T17:33:08.117033+0000","last_became_peered":"2024-07-04T17:33:08.117033+0000","last_unstale":"2024-07-04T17:33:30.415604+0000","last_undegraded":"2024-07-04T17:33:30.415604+0000","last_fullsized":"2024-07-04T17:33:30.415604+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:41:18.737857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.439260+0000","last_change":"2024-07-04T17:33:29.452800+0000","last_active":"2024-07-04T17:33:30.439260+0000","last_peered":"2024-07-04T17:33:30.439260+0000","last_clean":"2024-07-04T17:33:30.439260+0000","last_became_active":"2024-07-04T17:33:29.451858+0000","last_became_peered":"2024-07-04T17:33:29.451858+0000","last_unstale":"2024-07-04T17:33:30.439260+0000","last_undegraded":"2024-07-04T17:33:30.439260+0000","last_fullsized":"2024-07-04T17:33:30.439260+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:07:07.479809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.435901+0000","last_change":"2024-07-04T17:33:28.425749+0000","last_active":"2024-07-04T17:33:30.435901+0000","last_peered":"2024-07-04T17:33:30.435901+0000","last_clean":"2024-07-04T17:33:30.435901+0000","last_became_active":"2024-07-04T17:33:28.423428+0000","last_became_peered":"2024-07-04T17:33:28.423428+0000","last_unstale":"2024-07-04T17:33:30.435901+0000","last_undegraded":"2024-07-04T17:33:30.435901+0000","last_fullsized":"2024-07-04T17:33:30.435901+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:10:05.668947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.436962+0000","last_change":"2024-07-04T17:33:09.129855+0000","last_active":"2024-07-04T17:33:30.436962+0000","last_peered":"2024-07-04T17:33:30.436962+0000","last_clean":"2024-07-04T17:33:30.436962+0000","last_became_active":"2024-07-04T17:33:09.127858+0000","last_became_peered":"2024-07-04T17:33:09.127858+0000","last_unstale":"2024-07-04T17:33:30.436962+0000","last_undegraded":"2024-07-04T17:33:30.436962+0000","last_fullsized":"2024-07-04T17:33:30.436962+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:35:51.815415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"114'8","reported_seq":30,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.446761+0000","last_change":"2024-07-04T17:33:08.105018+0000","last_active":"2024-07-04T17:33:30.446761+0000","last_peered":"2024-07-04T17:33:30.446761+0000","last_clean":"2024-07-04T17:33:30.446761+0000","last_became_active":"2024-07-04T17:33:08.104685+0000","last_became_peered":"2024-07-04T17:33:08.104685+0000","last_unstale":"2024-07-04T17:33:30.446761+0000","last_undegraded":"2024-07-04T17:33:30.446761+0000","last_fullsized":"2024-07-04T17:33:30.446761+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:13:39.096736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3309,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.412236+0000","last_change":"2024-07-04T17:33:29.435558+0000","last_active":"2024-07-04T17:33:30.412236+0000","last_peered":"2024-07-04T17:33:30.412236+0000","last_clean":"2024-07-04T17:33:30.412236+0000","last_became_active":"2024-07-04T17:33:29.432192+0000","last_became_peered":"2024-07-04T17:33:29.432192+0000","last_unstale":"2024-07-04T17:33:30.412236+0000","last_undegraded":"2024-07-04T17:33:30.412236+0000","last_fullsized":"2024-07-04T17:33:30.412236+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:10:53.801654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.411856+0000","last_change":"2024-07-04T17:33:28.408617+0000","last_active":"2024-07-04T17:33:30.411856+0000","last_peered":"2024-07-04T17:33:30.411856+0000","last_clean":"2024-07-04T17:33:30.411856+0000","last_became_active":"2024-07-04T17:33:28.408267+0000","last_became_peered":"2024-07-04T17:33:28.408267+0000","last_unstale":"2024-07-04T17:33:30.411856+0000","last_undegraded":"2024-07-04T17:33:30.411856+0000","last_fullsized":"2024-07-04T17:33:30.411856+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:47:45.392135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.435092+0000","last_change":"2024-07-04T17:33:09.122352+0000","last_active":"2024-07-04T17:33:30.435092+0000","last_peered":"2024-07-04T17:33:30.435092+0000","last_clean":"2024-07-04T17:33:30.435092+0000","last_became_active":"2024-07-04T17:33:09.121662+0000","last_became_peered":"2024-07-04T17:33:09.121662+0000","last_unstale":"2024-07-04T17:33:30.435092+0000","last_undegraded":"2024-07-04T17:33:30.435092+0000","last_fullsized":"2024-07-04T17:33:30.435092+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:38:27.893548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"113'2","reported_seq":24,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.413633+0000","last_change":"2024-07-04T17:33:08.100243+0000","last_active":"2024-07-04T17:33:30.413633+0000","last_peered":"2024-07-04T17:33:30.413633+0000","last_clean":"2024-07-04T17:33:30.413633+0000","last_became_active":"2024-07-04T17:33:08.099860+0000","last_became_peered":"2024-07-04T17:33:08.099860+0000","last_unstale":"2024-07-04T17:33:30.413633+0000","last_undegraded":"2024-07-04T17:33:30.413633+0000","last_fullsized":"2024-07-04T17:33:30.413633+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:44:16.021572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444462+0000","last_change":"2024-07-04T17:33:29.423569+0000","last_active":"2024-07-04T17:33:30.444462+0000","last_peered":"2024-07-04T17:33:30.444462+0000","last_clean":"2024-07-04T17:33:30.444462+0000","last_became_active":"2024-07-04T17:33:29.423198+0000","last_became_peered":"2024-07-04T17:33:29.423198+0000","last_unstale":"2024-07-04T17:33:30.444462+0000","last_undegraded":"2024-07-04T17:33:30.444462+0000","last_fullsized":"2024-07-04T17:33:30.444462+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:45:53.608714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.10","version":"117'8","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.411671+0000","last_change":"2024-07-04T17:33:28.409675+0000","last_active":"2024-07-04T17:33:30.411671+0000","last_peered":"2024-07-04T17:33:30.411671+0000","last_clean":"2024-07-04T17:33:30.411671+0000","last_became_active":"2024-07-04T17:33:28.408564+0000","last_became_peered":"2024-07-04T17:33:28.408564+0000","last_unstale":"2024-07-04T17:33:30.411671+0000","last_undegraded":"2024-07-04T17:33:30.411671+0000","last_fullsized":"2024-07-04T17:33:30.411671+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:25:43.381033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.413153+0000","last_change":"2024-07-04T17:33:09.124609+0000","last_active":"2024-07-04T17:33:30.413153+0000","last_peered":"2024-07-04T17:33:30.413153+0000","last_clean":"2024-07-04T17:33:30.413153+0000","last_became_active":"2024-07-04T17:33:09.123720+0000","last_became_peered":"2024-07-04T17:33:09.123720+0000","last_unstale":"2024-07-04T17:33:30.413153+0000","last_undegraded":"2024-07-04T17:33:30.413153+0000","last_fullsized":"2024-07-04T17:33:30.413153+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:59:00.559158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.446072+0000","last_change":"2024-07-04T17:33:08.092450+0000","last_active":"2024-07-04T17:33:30.446072+0000","last_peered":"2024-07-04T17:33:30.446072+0000","last_clean":"2024-07-04T17:33:30.446072+0000","last_became_active":"2024-07-04T17:33:08.092088+0000","last_became_peered":"2024-07-04T17:33:08.092088+0000","last_unstale":"2024-07-04T17:33:30.446072+0000","last_undegraded":"2024-07-04T17:33:30.446072+0000","last_fullsized":"2024-07-04T17:33:30.446072+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:54:46.681088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.434866+0000","last_change":"2024-07-04T17:33:29.463209+0000","last_active":"2024-07-04T17:33:30.434866+0000","last_peered":"2024-07-04T17:33:30.434866+0000","last_clean":"2024-07-04T17:33:30.434866+0000","last_became_active":"2024-07-04T17:33:29.462348+0000","last_became_peered":"2024-07-04T17:33:29.462348+0000","last_unstale":"2024-07-04T17:33:30.434866+0000","last_undegraded":"2024-07-04T17:33:30.434866+0000","last_fullsized":"2024-07-04T17:33:30.434866+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:22:13.635684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.13","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.412020+0000","last_change":"2024-07-04T17:33:28.429010+0000","last_active":"2024-07-04T17:33:30.412020+0000","last_peered":"2024-07-04T17:33:30.412020+0000","last_clean":"2024-07-04T17:33:30.412020+0000","last_became_active":"2024-07-04T17:33:28.428679+0000","last_became_peered":"2024-07-04T17:33:28.428679+0000","last_unstale":"2024-07-04T17:33:30.412020+0000","last_undegraded":"2024-07-04T17:33:30.412020+0000","last_fullsized":"2024-07-04T17:33:30.412020+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:00:02.397558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.433960+0000","last_change":"2024-07-04T17:33:09.101721+0000","last_active":"2024-07-04T17:33:30.433960+0000","last_peered":"2024-07-04T17:33:30.433960+0000","last_clean":"2024-07-04T17:33:30.433960+0000","last_became_active":"2024-07-04T17:33:09.101084+0000","last_became_peered":"2024-07-04T17:33:09.101084+0000","last_unstale":"2024-07-04T17:33:30.433960+0000","last_undegraded":"2024-07-04T17:33:30.433960+0000","last_fullsized":"2024-07-04T17:33:30.433960+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:35:19.200734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.412226+0000","last_change":"2024-07-04T17:33:08.098833+0000","last_active":"2024-07-04T17:33:30.412226+0000","last_peered":"2024-07-04T17:33:30.412226+0000","last_clean":"2024-07-04T17:33:30.412226+0000","last_became_active":"2024-07-04T17:33:08.098492+0000","last_became_peered":"2024-07-04T17:33:08.098492+0000","last_unstale":"2024-07-04T17:33:30.412226+0000","last_undegraded":"2024-07-04T17:33:30.412226+0000","last_fullsized":"2024-07-04T17:33:30.412226+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:40:56.816879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.445646+0000","last_change":"2024-07-04T17:33:29.421732+0000","last_active":"2024-07-04T17:33:30.445646+0000","last_peered":"2024-07-04T17:33:30.445646+0000","last_clean":"2024-07-04T17:33:30.445646+0000","last_became_active":"2024-07-04T17:33:29.420326+0000","last_became_peered":"2024-07-04T17:33:29.420326+0000","last_unstale":"2024-07-04T17:33:30.445646+0000","last_undegraded":"2024-07-04T17:33:30.445646+0000","last_fullsized":"2024-07-04T17:33:30.445646+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:47:40.618326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444637+0000","last_change":"2024-07-04T17:33:28.407378+0000","last_active":"2024-07-04T17:33:30.444637+0000","last_peered":"2024-07-04T17:33:30.444637+0000","last_clean":"2024-07-04T17:33:30.444637+0000","last_became_active":"2024-07-04T17:33:28.406935+0000","last_became_peered":"2024-07-04T17:33:28.406935+0000","last_unstale":"2024-07-04T17:33:30.444637+0000","last_undegraded":"2024-07-04T17:33:30.444637+0000","last_fullsized":"2024-07-04T17:33:30.444637+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:53:02.453078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.434844+0000","last_change":"2024-07-04T17:33:09.131477+0000","last_active":"2024-07-04T17:33:30.434844+0000","last_peered":"2024-07-04T17:33:30.434844+0000","last_clean":"2024-07-04T17:33:30.434844+0000","last_became_active":"2024-07-04T17:33:09.122889+0000","last_became_peered":"2024-07-04T17:33:09.122889+0000","last_unstale":"2024-07-04T17:33:30.434844+0000","last_undegraded":"2024-07-04T17:33:30.434844+0000","last_fullsized":"2024-07-04T17:33:30.434844+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:48:40.827656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"113'8","reported_seq":30,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.434199+0000","last_change":"2024-07-04T17:33:08.094956+0000","last_active":"2024-07-04T17:33:30.434199+0000","last_peered":"2024-07-04T17:33:30.434199+0000","last_clean":"2024-07-04T17:33:30.434199+0000","last_became_active":"2024-07-04T17:33:08.090486+0000","last_became_peered":"2024-07-04T17:33:08.090486+0000","last_unstale":"2024-07-04T17:33:30.434199+0000","last_undegraded":"2024-07-04T17:33:30.434199+0000","last_fullsized":"2024-07-04T17:33:30.434199+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:05:13.908213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.415400+0000","last_change":"2024-07-04T17:33:29.435810+0000","last_active":"2024-07-04T17:33:30.415400+0000","last_peered":"2024-07-04T17:33:30.415400+0000","last_clean":"2024-07-04T17:33:30.415400+0000","last_became_active":"2024-07-04T17:33:29.434719+0000","last_became_peered":"2024-07-04T17:33:29.434719+0000","last_unstale":"2024-07-04T17:33:30.415400+0000","last_undegraded":"2024-07-04T17:33:30.415400+0000","last_fullsized":"2024-07-04T17:33:30.415400+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:53:04.956029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.414354+0000","last_change":"2024-07-04T17:33:28.422613+0000","last_active":"2024-07-04T17:33:30.414354+0000","last_peered":"2024-07-04T17:33:30.414354+0000","last_clean":"2024-07-04T17:33:30.414354+0000","last_became_active":"2024-07-04T17:33:28.422278+0000","last_became_peered":"2024-07-04T17:33:28.422278+0000","last_unstale":"2024-07-04T17:33:30.414354+0000","last_undegraded":"2024-07-04T17:33:30.414354+0000","last_fullsized":"2024-07-04T17:33:30.414354+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:11:29.473901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.415583+0000","last_change":"2024-07-04T17:33:09.128035+0000","last_active":"2024-07-04T17:33:30.415583+0000","last_peered":"2024-07-04T17:33:30.415583+0000","last_clean":"2024-07-04T17:33:30.415583+0000","last_became_active":"2024-07-04T17:33:09.125335+0000","last_became_peered":"2024-07-04T17:33:09.125335+0000","last_unstale":"2024-07-04T17:33:30.415583+0000","last_undegraded":"2024-07-04T17:33:30.415583+0000","last_fullsized":"2024-07-04T17:33:30.415583+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:35:53.125670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"113'1","reported_seq":23,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.439669+0000","last_change":"2024-07-04T17:33:08.096006+0000","last_active":"2024-07-04T17:33:30.439669+0000","last_peered":"2024-07-04T17:33:30.439669+0000","last_clean":"2024-07-04T17:33:30.439669+0000","last_became_active":"2024-07-04T17:33:08.095685+0000","last_became_peered":"2024-07-04T17:33:08.095685+0000","last_unstale":"2024-07-04T17:33:30.439669+0000","last_undegraded":"2024-07-04T17:33:30.439669+0000","last_fullsized":"2024-07-04T17:33:30.439669+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:58:57.013662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.447786+0000","last_change":"2024-07-04T17:33:29.428235+0000","last_active":"2024-07-04T17:33:30.447786+0000","last_peered":"2024-07-04T17:33:30.447786+0000","last_clean":"2024-07-04T17:33:30.447786+0000","last_became_active":"2024-07-04T17:33:29.426766+0000","last_became_peered":"2024-07-04T17:33:29.426766+0000","last_unstale":"2024-07-04T17:33:30.447786+0000","last_undegraded":"2024-07-04T17:33:30.447786+0000","last_fullsized":"2024-07-04T17:33:30.447786+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:16:01.565064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.445673+0000","last_change":"2024-07-04T17:33:28.421961+0000","last_active":"2024-07-04T17:33:30.445673+0000","last_peered":"2024-07-04T17:33:30.445673+0000","last_clean":"2024-07-04T17:33:30.445673+0000","last_became_active":"2024-07-04T17:33:28.421617+0000","last_became_peered":"2024-07-04T17:33:28.421617+0000","last_unstale":"2024-07-04T17:33:30.445673+0000","last_undegraded":"2024-07-04T17:33:30.445673+0000","last_fullsized":"2024-07-04T17:33:30.445673+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:34:01.546503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.439398+0000","last_change":"2024-07-04T17:33:09.118421+0000","last_active":"2024-07-04T17:33:30.439398+0000","last_peered":"2024-07-04T17:33:30.439398+0000","last_clean":"2024-07-04T17:33:30.439398+0000","last_became_active":"2024-07-04T17:33:09.117653+0000","last_became_peered":"2024-07-04T17:33:09.117653+0000","last_unstale":"2024-07-04T17:33:30.439398+0000","last_undegraded":"2024-07-04T17:33:30.439398+0000","last_fullsized":"2024-07-04T17:33:30.439398+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:59:43.093156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"113'2","reported_seq":24,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.415677+0000","last_change":"2024-07-04T17:33:08.113131+0000","last_active":"2024-07-04T17:33:30.415677+0000","last_peered":"2024-07-04T17:33:30.415677+0000","last_clean":"2024-07-04T17:33:30.415677+0000","last_became_active":"2024-07-04T17:33:08.112337+0000","last_became_peered":"2024-07-04T17:33:08.112337+0000","last_unstale":"2024-07-04T17:33:30.415677+0000","last_undegraded":"2024-07-04T17:33:30.415677+0000","last_fullsized":"2024-07-04T17:33:30.415677+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:45:29.198114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.414886+0000","last_change":"2024-07-04T17:33:29.447485+0000","last_active":"2024-07-04T17:33:30.414886+0000","last_peered":"2024-07-04T17:33:30.414886+0000","last_clean":"2024-07-04T17:33:30.414886+0000","last_became_active":"2024-07-04T17:33:29.447144+0000","last_became_peered":"2024-07-04T17:33:29.447144+0000","last_unstale":"2024-07-04T17:33:30.414886+0000","last_undegraded":"2024-07-04T17:33:30.414886+0000","last_fullsized":"2024-07-04T17:33:30.414886+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:06:53.353893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.413302+0000","last_change":"2024-07-04T17:33:28.423301+0000","last_active":"2024-07-04T17:33:30.413302+0000","last_peered":"2024-07-04T17:33:30.413302+0000","last_clean":"2024-07-04T17:33:30.413302+0000","last_became_active":"2024-07-04T17:33:28.421659+0000","last_became_peered":"2024-07-04T17:33:28.421659+0000","last_unstale":"2024-07-04T17:33:30.413302+0000","last_undegraded":"2024-07-04T17:33:30.413302+0000","last_fullsized":"2024-07-04T17:33:30.413302+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:41:17.683702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.415940+0000","last_change":"2024-07-04T17:33:09.107000+0000","last_active":"2024-07-04T17:33:30.415940+0000","last_peered":"2024-07-04T17:33:30.415940+0000","last_clean":"2024-07-04T17:33:30.415940+0000","last_became_active":"2024-07-04T17:33:09.106204+0000","last_became_peered":"2024-07-04T17:33:09.106204+0000","last_unstale":"2024-07-04T17:33:30.415940+0000","last_undegraded":"2024-07-04T17:33:30.415940+0000","last_fullsized":"2024-07-04T17:33:30.415940+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:22:24.403011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"113'1","reported_seq":23,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.438327+0000","last_change":"2024-07-04T17:33:08.105392+0000","last_active":"2024-07-04T17:33:30.438327+0000","last_peered":"2024-07-04T17:33:30.438327+0000","last_clean":"2024-07-04T17:33:30.438327+0000","last_became_active":"2024-07-04T17:33:08.105067+0000","last_became_peered":"2024-07-04T17:33:08.105067+0000","last_unstale":"2024-07-04T17:33:30.438327+0000","last_undegraded":"2024-07-04T17:33:30.438327+0000","last_fullsized":"2024-07-04T17:33:30.438327+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:10:49.236238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.438724+0000","last_change":"2024-07-04T17:33:29.449293+0000","last_active":"2024-07-04T17:33:30.438724+0000","last_peered":"2024-07-04T17:33:30.438724+0000","last_clean":"2024-07-04T17:33:30.438724+0000","last_became_active":"2024-07-04T17:33:29.448949+0000","last_became_peered":"2024-07-04T17:33:29.448949+0000","last_unstale":"2024-07-04T17:33:30.438724+0000","last_undegraded":"2024-07-04T17:33:30.438724+0000","last_fullsized":"2024-07-04T17:33:30.438724+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:03:45.183273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.e","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.447730+0000","last_change":"2024-07-04T17:33:28.422730+0000","last_active":"2024-07-04T17:33:30.447730+0000","last_peered":"2024-07-04T17:33:30.447730+0000","last_clean":"2024-07-04T17:33:30.447730+0000","last_became_active":"2024-07-04T17:33:28.421927+0000","last_became_peered":"2024-07-04T17:33:28.421927+0000","last_unstale":"2024-07-04T17:33:30.447730+0000","last_undegraded":"2024-07-04T17:33:30.447730+0000","last_fullsized":"2024-07-04T17:33:30.447730+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:04:56.556554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.414511+0000","last_change":"2024-07-04T17:33:09.126710+0000","last_active":"2024-07-04T17:33:30.414511+0000","last_peered":"2024-07-04T17:33:30.414511+0000","last_clean":"2024-07-04T17:33:30.414511+0000","last_became_active":"2024-07-04T17:33:09.122975+0000","last_became_peered":"2024-07-04T17:33:09.122975+0000","last_unstale":"2024-07-04T17:33:30.414511+0000","last_undegraded":"2024-07-04T17:33:30.414511+0000","last_fullsized":"2024-07-04T17:33:30.414511+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:48:57.200238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.416155+0000","last_change":"2024-07-04T17:33:08.114916+0000","last_active":"2024-07-04T17:33:30.416155+0000","last_peered":"2024-07-04T17:33:30.416155+0000","last_clean":"2024-07-04T17:33:30.416155+0000","last_became_active":"2024-07-04T17:33:08.114455+0000","last_became_peered":"2024-07-04T17:33:08.114455+0000","last_unstale":"2024-07-04T17:33:30.416155+0000","last_undegraded":"2024-07-04T17:33:30.416155+0000","last_fullsized":"2024-07-04T17:33:30.416155+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:09:08.764721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.433630+0000","last_change":"2024-07-04T17:33:29.441891+0000","last_active":"2024-07-04T17:33:30.433630+0000","last_peered":"2024-07-04T17:33:30.433630+0000","last_clean":"2024-07-04T17:33:30.433630+0000","last_became_active":"2024-07-04T17:33:29.441260+0000","last_became_peered":"2024-07-04T17:33:29.441260+0000","last_unstale":"2024-07-04T17:33:30.433630+0000","last_undegraded":"2024-07-04T17:33:30.433630+0000","last_fullsized":"2024-07-04T17:33:30.433630+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:05:38.199674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.411418+0000","last_change":"2024-07-04T17:33:28.432124+0000","last_active":"2024-07-04T17:33:30.411418+0000","last_peered":"2024-07-04T17:33:30.411418+0000","last_clean":"2024-07-04T17:33:30.411418+0000","last_became_active":"2024-07-04T17:33:28.428245+0000","last_became_peered":"2024-07-04T17:33:28.428245+0000","last_unstale":"2024-07-04T17:33:30.411418+0000","last_undegraded":"2024-07-04T17:33:30.411418+0000","last_fullsized":"2024-07-04T17:33:30.411418+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:05:52.733963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.445909+0000","last_change":"2024-07-04T17:33:09.105710+0000","last_active":"2024-07-04T17:33:30.445909+0000","last_peered":"2024-07-04T17:33:30.445909+0000","last_clean":"2024-07-04T17:33:30.445909+0000","last_became_active":"2024-07-04T17:33:09.101819+0000","last_became_peered":"2024-07-04T17:33:09.101819+0000","last_unstale":"2024-07-04T17:33:30.445909+0000","last_undegraded":"2024-07-04T17:33:30.445909+0000","last_fullsized":"2024-07-04T17:33:30.445909+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:47:15.892916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"113'2","reported_seq":24,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.446875+0000","last_change":"2024-07-04T17:33:08.108404+0000","last_active":"2024-07-04T17:33:30.446875+0000","last_peered":"2024-07-04T17:33:30.446875+0000","last_clean":"2024-07-04T17:33:30.446875+0000","last_became_active":"2024-07-04T17:33:08.107387+0000","last_became_peered":"2024-07-04T17:33:08.107387+0000","last_unstale":"2024-07-04T17:33:30.446875+0000","last_undegraded":"2024-07-04T17:33:30.446875+0000","last_fullsized":"2024-07-04T17:33:30.446875+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:10:32.480743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.433670+0000","last_change":"2024-07-04T17:33:29.454218+0000","last_active":"2024-07-04T17:33:30.433670+0000","last_peered":"2024-07-04T17:33:30.433670+0000","last_clean":"2024-07-04T17:33:30.433670+0000","last_became_active":"2024-07-04T17:33:29.453913+0000","last_became_peered":"2024-07-04T17:33:29.453913+0000","last_unstale":"2024-07-04T17:33:30.433670+0000","last_undegraded":"2024-07-04T17:33:30.433670+0000","last_fullsized":"2024-07-04T17:33:30.433670+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:22:24.901243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.445804+0000","last_change":"2024-07-04T17:33:28.410838+0000","last_active":"2024-07-04T17:33:30.445804+0000","last_peered":"2024-07-04T17:33:30.445804+0000","last_clean":"2024-07-04T17:33:30.445804+0000","last_became_active":"2024-07-04T17:33:28.410528+0000","last_became_peered":"2024-07-04T17:33:28.410528+0000","last_unstale":"2024-07-04T17:33:30.445804+0000","last_undegraded":"2024-07-04T17:33:30.445804+0000","last_fullsized":"2024-07-04T17:33:30.445804+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:10:40.346118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.446672+0000","last_change":"2024-07-04T17:33:09.102491+0000","last_active":"2024-07-04T17:33:30.446672+0000","last_peered":"2024-07-04T17:33:30.446672+0000","last_clean":"2024-07-04T17:33:30.446672+0000","last_became_active":"2024-07-04T17:33:09.102170+0000","last_became_peered":"2024-07-04T17:33:09.102170+0000","last_unstale":"2024-07-04T17:33:30.446672+0000","last_undegraded":"2024-07-04T17:33:30.446672+0000","last_fullsized":"2024-07-04T17:33:30.446672+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:03:31.387343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.435546+0000","last_change":"2024-07-04T17:33:08.109294+0000","last_active":"2024-07-04T17:33:30.435546+0000","last_peered":"2024-07-04T17:33:30.435546+0000","last_clean":"2024-07-04T17:33:30.435546+0000","last_became_active":"2024-07-04T17:33:08.108944+0000","last_became_peered":"2024-07-04T17:33:08.108944+0000","last_unstale":"2024-07-04T17:33:30.435546+0000","last_undegraded":"2024-07-04T17:33:30.435546+0000","last_fullsized":"2024-07-04T17:33:30.435546+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:15:36.989862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.434321+0000","last_change":"2024-07-04T17:33:29.452093+0000","last_active":"2024-07-04T17:33:30.434321+0000","last_peered":"2024-07-04T17:33:30.434321+0000","last_clean":"2024-07-04T17:33:30.434321+0000","last_became_active":"2024-07-04T17:33:29.451428+0000","last_became_peered":"2024-07-04T17:33:29.451428+0000","last_unstale":"2024-07-04T17:33:30.434321+0000","last_undegraded":"2024-07-04T17:33:30.434321+0000","last_fullsized":"2024-07-04T17:33:30.434321+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:43:03.637268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.b","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.412206+0000","last_change":"2024-07-04T17:33:28.428048+0000","last_active":"2024-07-04T17:33:30.412206+0000","last_peered":"2024-07-04T17:33:30.412206+0000","last_clean":"2024-07-04T17:33:30.412206+0000","last_became_active":"2024-07-04T17:33:28.427693+0000","last_became_peered":"2024-07-04T17:33:28.427693+0000","last_unstale":"2024-07-04T17:33:30.412206+0000","last_undegraded":"2024-07-04T17:33:30.412206+0000","last_fullsized":"2024-07-04T17:33:30.412206+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:38:54.848517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.445206+0000","last_change":"2024-07-04T17:33:09.111629+0000","last_active":"2024-07-04T17:33:30.445206+0000","last_peered":"2024-07-04T17:33:30.445206+0000","last_clean":"2024-07-04T17:33:30.445206+0000","last_became_active":"2024-07-04T17:33:09.111332+0000","last_became_peered":"2024-07-04T17:33:09.111332+0000","last_unstale":"2024-07-04T17:33:30.445206+0000","last_undegraded":"2024-07-04T17:33:30.445206+0000","last_fullsized":"2024-07-04T17:33:30.445206+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:44:43.830281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.434965+0000","last_change":"2024-07-04T17:33:08.097164+0000","last_active":"2024-07-04T17:33:30.434965+0000","last_peered":"2024-07-04T17:33:30.434965+0000","last_clean":"2024-07-04T17:33:30.434965+0000","last_became_active":"2024-07-04T17:33:08.096841+0000","last_became_peered":"2024-07-04T17:33:08.096841+0000","last_unstale":"2024-07-04T17:33:30.434965+0000","last_undegraded":"2024-07-04T17:33:30.434965+0000","last_fullsized":"2024-07-04T17:33:30.434965+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:24:50.491277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.445807+0000","last_change":"2024-07-04T17:33:29.423657+0000","last_active":"2024-07-04T17:33:30.445807+0000","last_peered":"2024-07-04T17:33:30.445807+0000","last_clean":"2024-07-04T17:33:30.445807+0000","last_became_active":"2024-07-04T17:33:29.419856+0000","last_became_peered":"2024-07-04T17:33:29.419856+0000","last_unstale":"2024-07-04T17:33:30.445807+0000","last_undegraded":"2024-07-04T17:33:30.445807+0000","last_fullsized":"2024-07-04T17:33:30.445807+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:00:59.531635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444735+0000","last_change":"2024-07-04T17:33:28.406714+0000","last_active":"2024-07-04T17:33:30.444735+0000","last_peered":"2024-07-04T17:33:30.444735+0000","last_clean":"2024-07-04T17:33:30.444735+0000","last_became_active":"2024-07-04T17:33:28.406342+0000","last_became_peered":"2024-07-04T17:33:28.406342+0000","last_unstale":"2024-07-04T17:33:30.444735+0000","last_undegraded":"2024-07-04T17:33:30.444735+0000","last_fullsized":"2024-07-04T17:33:30.444735+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:42:18.264655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444996+0000","last_change":"2024-07-04T17:33:09.097586+0000","last_active":"2024-07-04T17:33:30.444996+0000","last_peered":"2024-07-04T17:33:30.444996+0000","last_clean":"2024-07-04T17:33:30.444996+0000","last_became_active":"2024-07-04T17:33:09.096586+0000","last_became_peered":"2024-07-04T17:33:09.096586+0000","last_unstale":"2024-07-04T17:33:30.444996+0000","last_undegraded":"2024-07-04T17:33:30.444996+0000","last_fullsized":"2024-07-04T17:33:30.444996+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:21:53.318774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.434416+0000","last_change":"2024-07-04T17:33:08.094584+0000","last_active":"2024-07-04T17:33:30.434416+0000","last_peered":"2024-07-04T17:33:30.434416+0000","last_clean":"2024-07-04T17:33:30.434416+0000","last_became_active":"2024-07-04T17:33:08.090067+0000","last_became_peered":"2024-07-04T17:33:08.090067+0000","last_unstale":"2024-07-04T17:33:30.434416+0000","last_undegraded":"2024-07-04T17:33:30.434416+0000","last_fullsized":"2024-07-04T17:33:30.434416+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:17:57.214638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.436090+0000","last_change":"2024-07-04T17:33:29.463129+0000","last_active":"2024-07-04T17:33:30.436090+0000","last_peered":"2024-07-04T17:33:30.436090+0000","last_clean":"2024-07-04T17:33:30.436090+0000","last_became_active":"2024-07-04T17:33:29.462631+0000","last_became_peered":"2024-07-04T17:33:29.462631+0000","last_unstale":"2024-07-04T17:33:30.436090+0000","last_undegraded":"2024-07-04T17:33:30.436090+0000","last_fullsized":"2024-07-04T17:33:30.436090+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:03:51.704253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.436434+0000","last_change":"2024-07-04T17:33:28.430344+0000","last_active":"2024-07-04T17:33:30.436434+0000","last_peered":"2024-07-04T17:33:30.436434+0000","last_clean":"2024-07-04T17:33:30.436434+0000","last_became_active":"2024-07-04T17:33:28.429590+0000","last_became_peered":"2024-07-04T17:33:28.429590+0000","last_unstale":"2024-07-04T17:33:30.436434+0000","last_undegraded":"2024-07-04T17:33:30.436434+0000","last_fullsized":"2024-07-04T17:33:30.436434+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:05:09.258795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.438904+0000","last_change":"2024-07-04T17:33:09.131493+0000","last_active":"2024-07-04T17:33:30.438904+0000","last_peered":"2024-07-04T17:33:30.438904+0000","last_clean":"2024-07-04T17:33:30.438904+0000","last_became_active":"2024-07-04T17:33:09.131132+0000","last_became_peered":"2024-07-04T17:33:09.131132+0000","last_unstale":"2024-07-04T17:33:30.438904+0000","last_undegraded":"2024-07-04T17:33:30.438904+0000","last_fullsized":"2024-07-04T17:33:30.438904+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:56:22.629300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"113'2","reported_seq":24,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.439860+0000","last_change":"2024-07-04T17:33:08.105850+0000","last_active":"2024-07-04T17:33:30.439860+0000","last_peered":"2024-07-04T17:33:30.439860+0000","last_clean":"2024-07-04T17:33:30.439860+0000","last_became_active":"2024-07-04T17:33:08.104987+0000","last_became_peered":"2024-07-04T17:33:08.104987+0000","last_unstale":"2024-07-04T17:33:30.439860+0000","last_undegraded":"2024-07-04T17:33:30.439860+0000","last_fullsized":"2024-07-04T17:33:30.439860+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:36:17.000989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.413823+0000","last_change":"2024-07-04T17:33:29.460185+0000","last_active":"2024-07-04T17:33:30.413823+0000","last_peered":"2024-07-04T17:33:30.413823+0000","last_clean":"2024-07-04T17:33:30.413823+0000","last_became_active":"2024-07-04T17:33:29.459735+0000","last_became_peered":"2024-07-04T17:33:29.459735+0000","last_unstale":"2024-07-04T17:33:30.413823+0000","last_undegraded":"2024-07-04T17:33:30.413823+0000","last_fullsized":"2024-07-04T17:33:30.413823+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:21:36.261030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.436156+0000","last_change":"2024-07-04T17:33:28.418849+0000","last_active":"2024-07-04T17:33:30.436156+0000","last_peered":"2024-07-04T17:33:30.436156+0000","last_clean":"2024-07-04T17:33:30.436156+0000","last_became_active":"2024-07-04T17:33:28.418515+0000","last_became_peered":"2024-07-04T17:33:28.418515+0000","last_unstale":"2024-07-04T17:33:30.436156+0000","last_undegraded":"2024-07-04T17:33:30.436156+0000","last_fullsized":"2024-07-04T17:33:30.436156+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:10:52.789145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.439540+0000","last_change":"2024-07-04T17:33:09.118693+0000","last_active":"2024-07-04T17:33:30.439540+0000","last_peered":"2024-07-04T17:33:30.439540+0000","last_clean":"2024-07-04T17:33:30.439540+0000","last_became_active":"2024-07-04T17:33:09.117928+0000","last_became_peered":"2024-07-04T17:33:09.117928+0000","last_unstale":"2024-07-04T17:33:30.439540+0000","last_undegraded":"2024-07-04T17:33:30.439540+0000","last_fullsized":"2024-07-04T17:33:30.439540+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:38:35.544115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"114'1","reported_seq":23,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.415736+0000","last_change":"2024-07-04T17:33:08.116106+0000","last_active":"2024-07-04T17:33:30.415736+0000","last_peered":"2024-07-04T17:33:30.415736+0000","last_clean":"2024-07-04T17:33:30.415736+0000","last_became_active":"2024-07-04T17:33:08.115757+0000","last_became_peered":"2024-07-04T17:33:08.115757+0000","last_unstale":"2024-07-04T17:33:30.415736+0000","last_undegraded":"2024-07-04T17:33:30.415736+0000","last_fullsized":"2024-07-04T17:33:30.415736+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:16:30.420529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.447628+0000","last_change":"2024-07-04T17:33:29.444582+0000","last_active":"2024-07-04T17:33:30.447628+0000","last_peered":"2024-07-04T17:33:30.447628+0000","last_clean":"2024-07-04T17:33:30.447628+0000","last_became_active":"2024-07-04T17:33:29.440219+0000","last_became_peered":"2024-07-04T17:33:29.440219+0000","last_unstale":"2024-07-04T17:33:30.447628+0000","last_undegraded":"2024-07-04T17:33:30.447628+0000","last_fullsized":"2024-07-04T17:33:30.447628+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:07:12.694107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.7","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.438694+0000","last_change":"2024-07-04T17:33:28.435061+0000","last_active":"2024-07-04T17:33:30.438694+0000","last_peered":"2024-07-04T17:33:30.438694+0000","last_clean":"2024-07-04T17:33:30.438694+0000","last_became_active":"2024-07-04T17:33:28.434639+0000","last_became_peered":"2024-07-04T17:33:28.434639+0000","last_unstale":"2024-07-04T17:33:30.438694+0000","last_undegraded":"2024-07-04T17:33:30.438694+0000","last_fullsized":"2024-07-04T17:33:30.438694+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:44:17.694191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.416011+0000","last_change":"2024-07-04T17:33:09.098889+0000","last_active":"2024-07-04T17:33:30.416011+0000","last_peered":"2024-07-04T17:33:30.416011+0000","last_clean":"2024-07-04T17:33:30.416011+0000","last_became_active":"2024-07-04T17:33:09.098564+0000","last_became_peered":"2024-07-04T17:33:09.098564+0000","last_unstale":"2024-07-04T17:33:30.416011+0000","last_undegraded":"2024-07-04T17:33:30.416011+0000","last_fullsized":"2024-07-04T17:33:30.416011+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:06:27.253525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.415057+0000","last_change":"2024-07-04T17:33:08.114022+0000","last_active":"2024-07-04T17:33:30.415057+0000","last_peered":"2024-07-04T17:33:30.415057+0000","last_clean":"2024-07-04T17:33:30.415057+0000","last_became_active":"2024-07-04T17:33:08.112033+0000","last_became_peered":"2024-07-04T17:33:08.112033+0000","last_unstale":"2024-07-04T17:33:30.415057+0000","last_undegraded":"2024-07-04T17:33:30.415057+0000","last_fullsized":"2024-07-04T17:33:30.415057+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:00:33.769816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.447448+0000","last_change":"2024-07-04T17:33:29.444970+0000","last_active":"2024-07-04T17:33:30.447448+0000","last_peered":"2024-07-04T17:33:30.447448+0000","last_clean":"2024-07-04T17:33:30.447448+0000","last_became_active":"2024-07-04T17:33:29.439990+0000","last_became_peered":"2024-07-04T17:33:29.439990+0000","last_unstale":"2024-07-04T17:33:30.447448+0000","last_undegraded":"2024-07-04T17:33:30.447448+0000","last_fullsized":"2024-07-04T17:33:30.447448+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:11:57.484633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"118'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.436382+0000","last_change":"2024-07-04T17:33:28.435388+0000","last_active":"2024-07-04T17:33:30.436382+0000","last_peered":"2024-07-04T17:33:30.436382+0000","last_clean":"2024-07-04T17:33:30.436382+0000","last_became_active":"2024-07-04T17:33:28.434662+0000","last_became_peered":"2024-07-04T17:33:28.434662+0000","last_unstale":"2024-07-04T17:33:30.436382+0000","last_undegraded":"2024-07-04T17:33:30.436382+0000","last_fullsized":"2024-07-04T17:33:30.436382+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:06:37.633452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.414668+0000","last_change":"2024-07-04T17:33:09.113463+0000","last_active":"2024-07-04T17:33:30.414668+0000","last_peered":"2024-07-04T17:33:30.414668+0000","last_clean":"2024-07-04T17:33:30.414668+0000","last_became_active":"2024-07-04T17:33:09.112826+0000","last_became_peered":"2024-07-04T17:33:09.112826+0000","last_unstale":"2024-07-04T17:33:30.414668+0000","last_undegraded":"2024-07-04T17:33:30.414668+0000","last_fullsized":"2024-07-04T17:33:30.414668+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:26:59.083650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.416154+0000","last_change":"2024-07-04T17:33:08.114768+0000","last_active":"2024-07-04T17:33:30.416154+0000","last_peered":"2024-07-04T17:33:30.416154+0000","last_clean":"2024-07-04T17:33:30.416154+0000","last_became_active":"2024-07-04T17:33:08.113902+0000","last_became_peered":"2024-07-04T17:33:08.113902+0000","last_unstale":"2024-07-04T17:33:30.416154+0000","last_undegraded":"2024-07-04T17:33:30.416154+0000","last_fullsized":"2024-07-04T17:33:30.416154+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:58:34.807900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444713+0000","last_change":"2024-07-04T17:33:29.412043+0000","last_active":"2024-07-04T17:33:30.444713+0000","last_peered":"2024-07-04T17:33:30.444713+0000","last_clean":"2024-07-04T17:33:30.444713+0000","last_became_active":"2024-07-04T17:33:29.411648+0000","last_became_peered":"2024-07-04T17:33:29.411648+0000","last_unstale":"2024-07-04T17:33:30.444713+0000","last_undegraded":"2024-07-04T17:33:30.444713+0000","last_fullsized":"2024-07-04T17:33:30.444713+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:27:12.466261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.433740+0000","last_change":"2024-07-04T17:33:28.424058+0000","last_active":"2024-07-04T17:33:30.433740+0000","last_peered":"2024-07-04T17:33:30.433740+0000","last_clean":"2024-07-04T17:33:30.433740+0000","last_became_active":"2024-07-04T17:33:28.423741+0000","last_became_peered":"2024-07-04T17:33:28.423741+0000","last_unstale":"2024-07-04T17:33:30.433740+0000","last_undegraded":"2024-07-04T17:33:30.433740+0000","last_fullsized":"2024-07-04T17:33:30.433740+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:41:17.683702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.435414+0000","last_change":"2024-07-04T17:33:09.128723+0000","last_active":"2024-07-04T17:33:30.435414+0000","last_peered":"2024-07-04T17:33:30.435414+0000","last_clean":"2024-07-04T17:33:30.435414+0000","last_became_active":"2024-07-04T17:33:09.128217+0000","last_became_peered":"2024-07-04T17:33:09.128217+0000","last_unstale":"2024-07-04T17:33:30.435414+0000","last_undegraded":"2024-07-04T17:33:30.435414+0000","last_fullsized":"2024-07-04T17:33:30.435414+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:08:28.724747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.447061+0000","last_change":"2024-07-04T17:33:08.108382+0000","last_active":"2024-07-04T17:33:30.447061+0000","last_peered":"2024-07-04T17:33:30.447061+0000","last_clean":"2024-07-04T17:33:30.447061+0000","last_became_active":"2024-07-04T17:33:08.107656+0000","last_became_peered":"2024-07-04T17:33:08.107656+0000","last_unstale":"2024-07-04T17:33:30.447061+0000","last_undegraded":"2024-07-04T17:33:30.447061+0000","last_fullsized":"2024-07-04T17:33:30.447061+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:55:46.289096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.435127+0000","last_change":"2024-07-04T17:33:29.451047+0000","last_active":"2024-07-04T17:33:30.435127+0000","last_peered":"2024-07-04T17:33:30.435127+0000","last_clean":"2024-07-04T17:33:30.435127+0000","last_became_active":"2024-07-04T17:33:29.450715+0000","last_became_peered":"2024-07-04T17:33:29.450715+0000","last_unstale":"2024-07-04T17:33:30.435127+0000","last_undegraded":"2024-07-04T17:33:30.435127+0000","last_fullsized":"2024-07-04T17:33:30.435127+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:59:59.422659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.433434+0000","last_change":"2024-07-04T17:33:28.417708+0000","last_active":"2024-07-04T17:33:30.433434+0000","last_peered":"2024-07-04T17:33:30.433434+0000","last_clean":"2024-07-04T17:33:30.433434+0000","last_became_active":"2024-07-04T17:33:28.417409+0000","last_became_peered":"2024-07-04T17:33:28.417409+0000","last_unstale":"2024-07-04T17:33:30.433434+0000","last_undegraded":"2024-07-04T17:33:30.433434+0000","last_fullsized":"2024-07-04T17:33:30.433434+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:45:26.883263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.436976+0000","last_change":"2024-07-04T17:33:09.129070+0000","last_active":"2024-07-04T17:33:30.436976+0000","last_peered":"2024-07-04T17:33:30.436976+0000","last_clean":"2024-07-04T17:33:30.436976+0000","last_became_active":"2024-07-04T17:33:09.128751+0000","last_became_peered":"2024-07-04T17:33:09.128751+0000","last_unstale":"2024-07-04T17:33:30.436976+0000","last_undegraded":"2024-07-04T17:33:30.436976+0000","last_fullsized":"2024-07-04T17:33:30.436976+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:39:44.077373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.446237+0000","last_change":"2024-07-04T17:33:08.096375+0000","last_active":"2024-07-04T17:33:30.446237+0000","last_peered":"2024-07-04T17:33:30.446237+0000","last_clean":"2024-07-04T17:33:30.446237+0000","last_became_active":"2024-07-04T17:33:08.096060+0000","last_became_peered":"2024-07-04T17:33:08.096060+0000","last_unstale":"2024-07-04T17:33:30.446237+0000","last_undegraded":"2024-07-04T17:33:30.446237+0000","last_fullsized":"2024-07-04T17:33:30.446237+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:08:19.414085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444566+0000","last_change":"2024-07-04T17:33:29.429275+0000","last_active":"2024-07-04T17:33:30.444566+0000","last_peered":"2024-07-04T17:33:30.444566+0000","last_clean":"2024-07-04T17:33:30.444566+0000","last_became_active":"2024-07-04T17:33:29.424470+0000","last_became_peered":"2024-07-04T17:33:29.424470+0000","last_unstale":"2024-07-04T17:33:30.444566+0000","last_undegraded":"2024-07-04T17:33:30.444566+0000","last_fullsized":"2024-07-04T17:33:30.444566+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:36:56.447125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.412239+0000","last_change":"2024-07-04T17:33:28.430003+0000","last_active":"2024-07-04T17:33:30.412239+0000","last_peered":"2024-07-04T17:33:30.412239+0000","last_clean":"2024-07-04T17:33:30.412239+0000","last_became_active":"2024-07-04T17:33:28.429687+0000","last_became_peered":"2024-07-04T17:33:28.429687+0000","last_unstale":"2024-07-04T17:33:30.412239+0000","last_undegraded":"2024-07-04T17:33:30.412239+0000","last_fullsized":"2024-07-04T17:33:30.412239+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:12:14.054912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.434768+0000","last_change":"2024-07-04T17:33:09.107349+0000","last_active":"2024-07-04T17:33:30.434768+0000","last_peered":"2024-07-04T17:33:30.434768+0000","last_clean":"2024-07-04T17:33:30.434768+0000","last_became_active":"2024-07-04T17:33:09.104954+0000","last_became_peered":"2024-07-04T17:33:09.104954+0000","last_unstale":"2024-07-04T17:33:30.434768+0000","last_undegraded":"2024-07-04T17:33:30.434768+0000","last_fullsized":"2024-07-04T17:33:30.434768+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:19:28.456336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.446338+0000","last_change":"2024-07-04T17:33:08.106155+0000","last_active":"2024-07-04T17:33:30.446338+0000","last_peered":"2024-07-04T17:33:30.446338+0000","last_clean":"2024-07-04T17:33:30.446338+0000","last_became_active":"2024-07-04T17:33:08.104825+0000","last_became_peered":"2024-07-04T17:33:08.104825+0000","last_unstale":"2024-07-04T17:33:30.446338+0000","last_undegraded":"2024-07-04T17:33:30.446338+0000","last_fullsized":"2024-07-04T17:33:30.446338+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:15:59.977728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.435036+0000","last_change":"2024-07-04T17:33:29.450723+0000","last_active":"2024-07-04T17:33:30.435036+0000","last_peered":"2024-07-04T17:33:30.435036+0000","last_clean":"2024-07-04T17:33:30.435036+0000","last_became_active":"2024-07-04T17:33:29.449529+0000","last_became_peered":"2024-07-04T17:33:29.449529+0000","last_unstale":"2024-07-04T17:33:30.435036+0000","last_undegraded":"2024-07-04T17:33:30.435036+0000","last_fullsized":"2024-07-04T17:33:30.435036+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:25:56.571866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.412435+0000","last_change":"2024-07-04T17:33:28.429840+0000","last_active":"2024-07-04T17:33:30.412435+0000","last_peered":"2024-07-04T17:33:30.412435+0000","last_clean":"2024-07-04T17:33:30.412435+0000","last_became_active":"2024-07-04T17:33:28.429539+0000","last_became_peered":"2024-07-04T17:33:28.429539+0000","last_unstale":"2024-07-04T17:33:30.412435+0000","last_undegraded":"2024-07-04T17:33:30.412435+0000","last_fullsized":"2024-07-04T17:33:30.412435+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:47:35.483539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.445166+0000","last_change":"2024-07-04T17:33:09.107749+0000","last_active":"2024-07-04T17:33:30.445166+0000","last_peered":"2024-07-04T17:33:30.445166+0000","last_clean":"2024-07-04T17:33:30.445166+0000","last_became_active":"2024-07-04T17:33:09.106400+0000","last_became_peered":"2024-07-04T17:33:09.106400+0000","last_unstale":"2024-07-04T17:33:30.445166+0000","last_undegraded":"2024-07-04T17:33:30.445166+0000","last_fullsized":"2024-07-04T17:33:30.445166+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:22:04.584333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.434601+0000","last_change":"2024-07-04T17:33:08.109770+0000","last_active":"2024-07-04T17:33:30.434601+0000","last_peered":"2024-07-04T17:33:30.434601+0000","last_clean":"2024-07-04T17:33:30.434601+0000","last_became_active":"2024-07-04T17:33:08.108245+0000","last_became_peered":"2024-07-04T17:33:08.108245+0000","last_unstale":"2024-07-04T17:33:30.434601+0000","last_undegraded":"2024-07-04T17:33:30.434601+0000","last_fullsized":"2024-07-04T17:33:30.434601+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:30:01.785284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.438092+0000","last_change":"2024-07-04T17:33:29.452049+0000","last_active":"2024-07-04T17:33:30.438092+0000","last_peered":"2024-07-04T17:33:30.438092+0000","last_clean":"2024-07-04T17:33:30.438092+0000","last_became_active":"2024-07-04T17:33:29.451712+0000","last_became_peered":"2024-07-04T17:33:29.451712+0000","last_unstale":"2024-07-04T17:33:30.438092+0000","last_undegraded":"2024-07-04T17:33:30.438092+0000","last_fullsized":"2024-07-04T17:33:30.438092+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:28:10.840046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.413257+0000","last_change":"2024-07-04T17:33:28.422561+0000","last_active":"2024-07-04T17:33:30.413257+0000","last_peered":"2024-07-04T17:33:30.413257+0000","last_clean":"2024-07-04T17:33:30.413257+0000","last_became_active":"2024-07-04T17:33:28.420301+0000","last_became_peered":"2024-07-04T17:33:28.420301+0000","last_unstale":"2024-07-04T17:33:30.413257+0000","last_undegraded":"2024-07-04T17:33:30.413257+0000","last_fullsized":"2024-07-04T17:33:30.413257+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:45:26.883263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.446598+0000","last_change":"2024-07-04T17:33:09.099278+0000","last_active":"2024-07-04T17:33:30.446598+0000","last_peered":"2024-07-04T17:33:30.446598+0000","last_clean":"2024-07-04T17:33:30.446598+0000","last_became_active":"2024-07-04T17:33:09.098752+0000","last_became_peered":"2024-07-04T17:33:09.098752+0000","last_unstale":"2024-07-04T17:33:30.446598+0000","last_undegraded":"2024-07-04T17:33:30.446598+0000","last_fullsized":"2024-07-04T17:33:30.446598+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:34:51.476685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"113'3","reported_seq":25,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.449201+0000","last_change":"2024-07-04T17:33:08.092774+0000","last_active":"2024-07-04T17:33:30.449201+0000","last_peered":"2024-07-04T17:33:30.449201+0000","last_clean":"2024-07-04T17:33:30.449201+0000","last_became_active":"2024-07-04T17:33:08.092404+0000","last_became_peered":"2024-07-04T17:33:08.092404+0000","last_unstale":"2024-07-04T17:33:30.449201+0000","last_undegraded":"2024-07-04T17:33:30.449201+0000","last_fullsized":"2024-07-04T17:33:30.449201+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:48:02.318598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"100'101","reported_seq":456,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.413183+0000","last_change":"2024-07-04T17:26:04.953559+0000","last_active":"2024-07-04T17:33:30.413183+0000","last_peered":"2024-07-04T17:33:30.413183+0000","last_clean":"2024-07-04T17:33:30.413183+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:33:30.413183+0000","last_undegraded":"2024-07-04T17:33:30.413183+0000","last_fullsized":"2024-07-04T17:33:30.413183+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_clean_scrub_stamp":"2024-07-04T17:26:03.933432+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.064181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":476,"num_read_kb":1278,"num_write":163,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":4107,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":476,"num_read_kb":1278,"num_write":163,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101,"ondisk_log_size":101,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738457,"num_pgs":132,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36568,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335272,"statfs":{"total":96636764160,"available":96599318528,"internally_reserved":0,"allocated":1257472,"data_stored":487662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738458,"num_pgs":126,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37148,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334692,"statfs":{"total":96636764160,"available":96598724608,"internally_reserved":0,"allocated":1847296,"data_stored":1074333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":121,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37144,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334696,"statfs":{"total":96636764160,"available":96598728704,"internally_reserved":0,"allocated":1839104,"data_stored":1075858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:33:34.164 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:33:34.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.492+0000 7f6196000640 1 -- 172.21.15.42:0/509609830 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6188031a50 con 0x7f61980bc040 2024-07-04T17:33:34.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.492+0000 7f619e400640 1 -- 172.21.15.42:0/509609830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61980bc040 msgr2=0x7f61980bd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:34.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.492+0000 7f619e400640 1 --2- 172.21.15.42:0/509609830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61980bc040 0x7f61980bd2d0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f61880097b0 tx=0x7f618802a540 comp rx=0 tx=0).stop 2024-07-04T17:33:34.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.493+0000 7f619e400640 1 -- 172.21.15.42:0/509609830 shutdown_connections 2024-07-04T17:33:34.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.493+0000 7f619e400640 1 --2- 172.21.15.42:0/509609830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61980bc040 0x7f61980bd2d0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:34.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.493+0000 7f619e400640 1 -- 172.21.15.42:0/509609830 >> 172.21.15.42:0/509609830 conn(0x7f619806c940 msgr2=0x7f619806ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:34.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.493+0000 7f619e400640 1 -- 172.21.15.42:0/509609830 shutdown_connections 2024-07-04T17:33:34.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.493+0000 7f619e400640 1 -- 172.21.15.42:0/509609830 wait complete. 2024-07-04T17:33:34.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.495+0000 7f619e400640 1 Processor -- start 2024-07-04T17:33:34.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.495+0000 7f619e400640 1 -- start start 2024-07-04T17:33:34.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.497+0000 7f619e400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61980bc040 0x7f61981d92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:34.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.497+0000 7f619e400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f61980b7530 con 0x7f61980bc040 2024-07-04T17:33:34.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.497+0000 7f6197400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61980bc040 0x7f61981d92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:34.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.497+0000 7f6197400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61980bc040 0x7f61981d92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40782/0 (socket says 172.21.15.42:40782) 2024-07-04T17:33:34.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.497+0000 7f6197400640 1 -- 172.21.15.42:0/1029176320 learned_addr learned my addr 172.21.15.42:0/1029176320 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:34.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.498+0000 7f6197400640 1 -- 172.21.15.42:0/1029176320 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61981e3720 con 0x7f61980bc040 2024-07-04T17:33:34.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.498+0000 7f6197400640 1 --2- 172.21.15.42:0/1029176320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61980bc040 0x7f61981d92b0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f61880098d0 tx=0x7f6188033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:34.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.499+0000 7f618f400640 1 -- 172.21.15.42:0/1029176320 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6188031bc0 con 0x7f61980bc040 2024-07-04T17:33:34.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.499+0000 7f618f400640 1 -- 172.21.15.42:0/1029176320 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6188031e50 con 0x7f61980bc040 2024-07-04T17:33:34.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.499+0000 7f618f400640 1 -- 172.21.15.42:0/1029176320 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6188033200 con 0x7f61980bc040 2024-07-04T17:33:34.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.499+0000 7f619e400640 1 -- 172.21.15.42:0/1029176320 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61981e39a0 con 0x7f61980bc040 2024-07-04T17:33:34.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.499+0000 7f619e400640 1 -- 172.21.15.42:0/1029176320 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61981d9a60 con 0x7f61980bc040 2024-07-04T17:33:34.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.500+0000 7f619e400640 1 -- 172.21.15.42:0/1029176320 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61980b7c00 con 0x7f61980bc040 2024-07-04T17:33:34.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.500+0000 7f618f400640 1 -- 172.21.15.42:0/1029176320 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7f61880333a0 con 0x7f61980bc040 2024-07-04T17:33:34.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.501+0000 7f618f400640 1 --2- 172.21.15.42:0/1029176320 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f617403d4f0 0x7f617403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:34.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.501+0000 7f618f400640 1 -- 172.21.15.42:0/1029176320 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 4303+0+0 (secure 0 0 0) 0x7f61880731e0 con 0x7f61980bc040 2024-07-04T17:33:34.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.501+0000 7f6196a00640 1 --2- 172.21.15.42:0/1029176320 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f617403d4f0 0x7f617403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:34.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.502+0000 7f6196a00640 1 --2- 172.21.15.42:0/1029176320 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f617403d4f0 0x7f617403fa00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6180001560 tx=0x7f6180007a90 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:34.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.506+0000 7f618f400640 1 -- 172.21.15.42:0/1029176320 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f618803f030 con 0x7f61980bc040 2024-07-04T17:33:34.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.715+0000 7f619e400640 1 -- 172.21.15.42:0/1029176320 --> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f61980c0410 con 0x7f617403d4f0 2024-07-04T17:33:34.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.724+0000 7f618f400640 1 -- 172.21.15.42:0/1029176320 <== mgr.5434 v2:172.21.15.42:6834/4130303917 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+483259 (secure 0 0 0) 0x7f61980c0410 con 0x7f617403d4f0 2024-07-04T17:33:34.729 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:33:34.738 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:33:34.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.737+0000 7f619e400640 1 -- 172.21.15.42:0/1029176320 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f617403d4f0 msgr2=0x7f617403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:34.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.737+0000 7f619e400640 1 --2- 172.21.15.42:0/1029176320 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f617403d4f0 0x7f617403fa00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6180001560 tx=0x7f6180007a90 comp rx=0 tx=0).stop 2024-07-04T17:33:34.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.737+0000 7f619e400640 1 -- 172.21.15.42:0/1029176320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61980bc040 msgr2=0x7f61981d92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:34.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.737+0000 7f619e400640 1 --2- 172.21.15.42:0/1029176320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61980bc040 0x7f61981d92b0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f61880098d0 tx=0x7f6188033e40 comp rx=0 tx=0).stop 2024-07-04T17:33:34.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.737+0000 7f619e400640 1 -- 172.21.15.42:0/1029176320 shutdown_connections 2024-07-04T17:33:34.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.737+0000 7f619e400640 1 --2- 172.21.15.42:0/1029176320 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f617403d4f0 0x7f617403fa00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:34.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.737+0000 7f619e400640 1 --2- 172.21.15.42:0/1029176320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61980bc040 0x7f61981d92b0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:34.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.737+0000 7f619e400640 1 -- 172.21.15.42:0/1029176320 >> 172.21.15.42:0/1029176320 conn(0x7f619806c940 msgr2=0x7f61980b65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:34.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.737+0000 7f619e400640 1 -- 172.21.15.42:0/1029176320 shutdown_connections 2024-07-04T17:33:34.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:34.737+0000 7f619e400640 1 -- 172.21.15.42:0/1029176320 wait complete. 2024-07-04T17:33:34.764 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":45,"stamp":"2024-07-04T17:33:33.281782+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596885,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":476,"num_read_kb":1278,"num_write":255,"num_write_kb":1137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":181,"ondisk_log_size":181,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":379,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":110860,"kb_used_data":4828,"kb_used_omap":4,"kb_used_meta":105979,"kb_avail":283004660,"statfs":{"total":289910292480,"available":289796771840,"internally_reserved":0,"allocated":4943872,"data_stored":2637853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":108522831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.916028"},"pg_stats":[{"pgid":"23.3d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.436130+0000","last_change":"2024-07-04T17:33:29.452643+0000","last_active":"2024-07-04T17:33:30.436130+0000","last_peered":"2024-07-04T17:33:30.436130+0000","last_clean":"2024-07-04T17:33:30.436130+0000","last_became_active":"2024-07-04T17:33:29.451332+0000","last_became_peered":"2024-07-04T17:33:29.451332+0000","last_unstale":"2024-07-04T17:33:30.436130+0000","last_undegraded":"2024-07-04T17:33:30.436130+0000","last_fullsized":"2024-07-04T17:33:30.436130+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:24:32.296539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3f","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.413580+0000","last_change":"2024-07-04T17:33:09.110137+0000","last_active":"2024-07-04T17:33:30.413580+0000","last_peered":"2024-07-04T17:33:30.413580+0000","last_clean":"2024-07-04T17:33:30.413580+0000","last_became_active":"2024-07-04T17:33:09.109336+0000","last_became_peered":"2024-07-04T17:33:09.109336+0000","last_unstale":"2024-07-04T17:33:30.413580+0000","last_undegraded":"2024-07-04T17:33:30.413580+0000","last_fullsized":"2024-07-04T17:33:30.413580+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:09.113117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.414866+0000","last_change":"2024-07-04T17:33:29.452645+0000","last_active":"2024-07-04T17:33:30.414866+0000","last_peered":"2024-07-04T17:33:30.414866+0000","last_clean":"2024-07-04T17:33:30.414866+0000","last_became_active":"2024-07-04T17:33:29.452065+0000","last_became_peered":"2024-07-04T17:33:29.452065+0000","last_unstale":"2024-07-04T17:33:30.414866+0000","last_undegraded":"2024-07-04T17:33:30.414866+0000","last_fullsized":"2024-07-04T17:33:30.414866+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:51:15.248651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.448862+0000","last_change":"2024-07-04T17:33:09.113135+0000","last_active":"2024-07-04T17:33:30.448862+0000","last_peered":"2024-07-04T17:33:30.448862+0000","last_clean":"2024-07-04T17:33:30.448862+0000","last_became_active":"2024-07-04T17:33:09.112824+0000","last_became_peered":"2024-07-04T17:33:09.112824+0000","last_unstale":"2024-07-04T17:33:30.448862+0000","last_undegraded":"2024-07-04T17:33:30.448862+0000","last_fullsized":"2024-07-04T17:33:30.448862+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:25:36.254957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.3f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.439081+0000","last_change":"2024-07-04T17:33:29.450485+0000","last_active":"2024-07-04T17:33:30.439081+0000","last_peered":"2024-07-04T17:33:30.439081+0000","last_clean":"2024-07-04T17:33:30.439081+0000","last_became_active":"2024-07-04T17:33:29.450123+0000","last_became_peered":"2024-07-04T17:33:29.450123+0000","last_unstale":"2024-07-04T17:33:30.439081+0000","last_undegraded":"2024-07-04T17:33:30.439081+0000","last_fullsized":"2024-07-04T17:33:30.439081+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:57:29.422317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.413950+0000","last_change":"2024-07-04T17:33:09.126404+0000","last_active":"2024-07-04T17:33:30.413950+0000","last_peered":"2024-07-04T17:33:30.413950+0000","last_clean":"2024-07-04T17:33:30.413950+0000","last_became_active":"2024-07-04T17:33:09.126069+0000","last_became_peered":"2024-07-04T17:33:09.126069+0000","last_unstale":"2024-07-04T17:33:30.413950+0000","last_undegraded":"2024-07-04T17:33:30.413950+0000","last_fullsized":"2024-07-04T17:33:30.413950+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:54:17.261721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.435743+0000","last_change":"2024-07-04T17:33:29.456026+0000","last_active":"2024-07-04T17:33:30.435743+0000","last_peered":"2024-07-04T17:33:30.435743+0000","last_clean":"2024-07-04T17:33:30.435743+0000","last_became_active":"2024-07-04T17:33:29.455689+0000","last_became_peered":"2024-07-04T17:33:29.455689+0000","last_unstale":"2024-07-04T17:33:30.435743+0000","last_undegraded":"2024-07-04T17:33:30.435743+0000","last_fullsized":"2024-07-04T17:33:30.435743+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:47:36.209825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.436526+0000","last_change":"2024-07-04T17:33:09.104422+0000","last_active":"2024-07-04T17:33:30.436526+0000","last_peered":"2024-07-04T17:33:30.436526+0000","last_clean":"2024-07-04T17:33:30.436526+0000","last_became_active":"2024-07-04T17:33:09.104112+0000","last_became_peered":"2024-07-04T17:33:09.104112+0000","last_unstale":"2024-07-04T17:33:30.436526+0000","last_undegraded":"2024-07-04T17:33:30.436526+0000","last_fullsized":"2024-07-04T17:33:30.436526+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:50:45.806246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444066+0000","last_change":"2024-07-04T17:33:29.436496+0000","last_active":"2024-07-04T17:33:30.444066+0000","last_peered":"2024-07-04T17:33:30.444066+0000","last_clean":"2024-07-04T17:33:30.444066+0000","last_became_active":"2024-07-04T17:33:29.436176+0000","last_became_peered":"2024-07-04T17:33:29.436176+0000","last_unstale":"2024-07-04T17:33:30.444066+0000","last_undegraded":"2024-07-04T17:33:30.444066+0000","last_fullsized":"2024-07-04T17:33:30.444066+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:48:31.824457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.412655+0000","last_change":"2024-07-04T17:33:09.111717+0000","last_active":"2024-07-04T17:33:30.412655+0000","last_peered":"2024-07-04T17:33:30.412655+0000","last_clean":"2024-07-04T17:33:30.412655+0000","last_became_active":"2024-07-04T17:33:09.110270+0000","last_became_peered":"2024-07-04T17:33:09.110270+0000","last_unstale":"2024-07-04T17:33:30.412655+0000","last_undegraded":"2024-07-04T17:33:30.412655+0000","last_fullsized":"2024-07-04T17:33:30.412655+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:18:53.013321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444157+0000","last_change":"2024-07-04T17:33:29.435180+0000","last_active":"2024-07-04T17:33:30.444157+0000","last_peered":"2024-07-04T17:33:30.444157+0000","last_clean":"2024-07-04T17:33:30.444157+0000","last_became_active":"2024-07-04T17:33:29.429212+0000","last_became_peered":"2024-07-04T17:33:29.429212+0000","last_unstale":"2024-07-04T17:33:30.444157+0000","last_undegraded":"2024-07-04T17:33:30.444157+0000","last_fullsized":"2024-07-04T17:33:30.444157+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:25:44.819023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.412205+0000","last_change":"2024-07-04T17:33:09.108466+0000","last_active":"2024-07-04T17:33:30.412205+0000","last_peered":"2024-07-04T17:33:30.412205+0000","last_clean":"2024-07-04T17:33:30.412205+0000","last_became_active":"2024-07-04T17:33:09.108132+0000","last_became_peered":"2024-07-04T17:33:09.108132+0000","last_unstale":"2024-07-04T17:33:30.412205+0000","last_undegraded":"2024-07-04T17:33:30.412205+0000","last_fullsized":"2024-07-04T17:33:30.412205+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:51:32.675966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.413460+0000","last_change":"2024-07-04T17:33:29.430628+0000","last_active":"2024-07-04T17:33:30.413460+0000","last_peered":"2024-07-04T17:33:30.413460+0000","last_clean":"2024-07-04T17:33:30.413460+0000","last_became_active":"2024-07-04T17:33:29.430257+0000","last_became_peered":"2024-07-04T17:33:29.430257+0000","last_unstale":"2024-07-04T17:33:30.413460+0000","last_undegraded":"2024-07-04T17:33:30.413460+0000","last_fullsized":"2024-07-04T17:33:30.413460+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:55:08.001999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444598+0000","last_change":"2024-07-04T17:33:09.108992+0000","last_active":"2024-07-04T17:33:30.444598+0000","last_peered":"2024-07-04T17:33:30.444598+0000","last_clean":"2024-07-04T17:33:30.444598+0000","last_became_active":"2024-07-04T17:33:09.107307+0000","last_became_peered":"2024-07-04T17:33:09.107307+0000","last_unstale":"2024-07-04T17:33:30.444598+0000","last_undegraded":"2024-07-04T17:33:30.444598+0000","last_fullsized":"2024-07-04T17:33:30.444598+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:32:07.600844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.434130+0000","last_change":"2024-07-04T17:33:29.449127+0000","last_active":"2024-07-04T17:33:30.434130+0000","last_peered":"2024-07-04T17:33:30.434130+0000","last_clean":"2024-07-04T17:33:30.434130+0000","last_became_active":"2024-07-04T17:33:29.448718+0000","last_became_peered":"2024-07-04T17:33:29.448718+0000","last_unstale":"2024-07-04T17:33:30.434130+0000","last_undegraded":"2024-07-04T17:33:30.434130+0000","last_fullsized":"2024-07-04T17:33:30.434130+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:41:41.582097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.412546+0000","last_change":"2024-07-04T17:33:09.127659+0000","last_active":"2024-07-04T17:33:30.412546+0000","last_peered":"2024-07-04T17:33:30.412546+0000","last_clean":"2024-07-04T17:33:30.412546+0000","last_became_active":"2024-07-04T17:33:09.126591+0000","last_became_peered":"2024-07-04T17:33:09.126591+0000","last_unstale":"2024-07-04T17:33:30.412546+0000","last_undegraded":"2024-07-04T17:33:30.412546+0000","last_fullsized":"2024-07-04T17:33:30.412546+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:44:07.778739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.413407+0000","last_change":"2024-07-04T17:33:29.460618+0000","last_active":"2024-07-04T17:33:30.413407+0000","last_peered":"2024-07-04T17:33:30.413407+0000","last_clean":"2024-07-04T17:33:30.413407+0000","last_became_active":"2024-07-04T17:33:29.460215+0000","last_became_peered":"2024-07-04T17:33:29.460215+0000","last_unstale":"2024-07-04T17:33:30.413407+0000","last_undegraded":"2024-07-04T17:33:30.413407+0000","last_fullsized":"2024-07-04T17:33:30.413407+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:46:25.800079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.413757+0000","last_change":"2024-07-04T17:33:09.110971+0000","last_active":"2024-07-04T17:33:30.413757+0000","last_peered":"2024-07-04T17:33:30.413757+0000","last_clean":"2024-07-04T17:33:30.413757+0000","last_became_active":"2024-07-04T17:33:09.109775+0000","last_became_peered":"2024-07-04T17:33:09.109775+0000","last_unstale":"2024-07-04T17:33:30.413757+0000","last_undegraded":"2024-07-04T17:33:30.413757+0000","last_fullsized":"2024-07-04T17:33:30.413757+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:27:36.573799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.415780+0000","last_change":"2024-07-04T17:33:29.443487+0000","last_active":"2024-07-04T17:33:30.415780+0000","last_peered":"2024-07-04T17:33:30.415780+0000","last_clean":"2024-07-04T17:33:30.415780+0000","last_became_active":"2024-07-04T17:33:29.443160+0000","last_became_peered":"2024-07-04T17:33:29.443160+0000","last_unstale":"2024-07-04T17:33:30.415780+0000","last_undegraded":"2024-07-04T17:33:30.415780+0000","last_fullsized":"2024-07-04T17:33:30.415780+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:29:22.695717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.414901+0000","last_change":"2024-07-04T17:33:09.127783+0000","last_active":"2024-07-04T17:33:30.414901+0000","last_peered":"2024-07-04T17:33:30.414901+0000","last_clean":"2024-07-04T17:33:30.414901+0000","last_became_active":"2024-07-04T17:33:09.125335+0000","last_became_peered":"2024-07-04T17:33:09.125335+0000","last_unstale":"2024-07-04T17:33:30.414901+0000","last_undegraded":"2024-07-04T17:33:30.414901+0000","last_fullsized":"2024-07-04T17:33:30.414901+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:29:15.785171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.447780+0000","last_change":"2024-07-04T17:33:29.445956+0000","last_active":"2024-07-04T17:33:30.447780+0000","last_peered":"2024-07-04T17:33:30.447780+0000","last_clean":"2024-07-04T17:33:30.447780+0000","last_became_active":"2024-07-04T17:33:29.445245+0000","last_became_peered":"2024-07-04T17:33:29.445245+0000","last_unstale":"2024-07-04T17:33:30.447780+0000","last_undegraded":"2024-07-04T17:33:30.447780+0000","last_fullsized":"2024-07-04T17:33:30.447780+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:23:14.047614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.414185+0000","last_change":"2024-07-04T17:33:09.113535+0000","last_active":"2024-07-04T17:33:30.414185+0000","last_peered":"2024-07-04T17:33:30.414185+0000","last_clean":"2024-07-04T17:33:30.414185+0000","last_became_active":"2024-07-04T17:33:09.113127+0000","last_became_peered":"2024-07-04T17:33:09.113127+0000","last_unstale":"2024-07-04T17:33:30.414185+0000","last_undegraded":"2024-07-04T17:33:30.414185+0000","last_fullsized":"2024-07-04T17:33:30.414185+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:14:08.620654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.446960+0000","last_change":"2024-07-04T17:33:29.433615+0000","last_active":"2024-07-04T17:33:30.446960+0000","last_peered":"2024-07-04T17:33:30.446960+0000","last_clean":"2024-07-04T17:33:30.446960+0000","last_became_active":"2024-07-04T17:33:29.432557+0000","last_became_peered":"2024-07-04T17:33:29.432557+0000","last_unstale":"2024-07-04T17:33:30.446960+0000","last_undegraded":"2024-07-04T17:33:30.446960+0000","last_fullsized":"2024-07-04T17:33:30.446960+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:58:57.336702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.446374+0000","last_change":"2024-07-04T17:33:09.112587+0000","last_active":"2024-07-04T17:33:30.446374+0000","last_peered":"2024-07-04T17:33:30.446374+0000","last_clean":"2024-07-04T17:33:30.446374+0000","last_became_active":"2024-07-04T17:33:09.112063+0000","last_became_peered":"2024-07-04T17:33:09.112063+0000","last_unstale":"2024-07-04T17:33:30.446374+0000","last_undegraded":"2024-07-04T17:33:30.446374+0000","last_fullsized":"2024-07-04T17:33:30.446374+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:59:39.105749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444127+0000","last_change":"2024-07-04T17:33:29.443001+0000","last_active":"2024-07-04T17:33:30.444127+0000","last_peered":"2024-07-04T17:33:30.444127+0000","last_clean":"2024-07-04T17:33:30.444127+0000","last_became_active":"2024-07-04T17:33:29.442427+0000","last_became_peered":"2024-07-04T17:33:29.442427+0000","last_unstale":"2024-07-04T17:33:30.444127+0000","last_undegraded":"2024-07-04T17:33:30.444127+0000","last_fullsized":"2024-07-04T17:33:30.444127+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:14:10.024394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.412821+0000","last_change":"2024-07-04T17:33:09.110573+0000","last_active":"2024-07-04T17:33:30.412821+0000","last_peered":"2024-07-04T17:33:30.412821+0000","last_clean":"2024-07-04T17:33:30.412821+0000","last_became_active":"2024-07-04T17:33:09.109386+0000","last_became_peered":"2024-07-04T17:33:09.109386+0000","last_unstale":"2024-07-04T17:33:30.412821+0000","last_undegraded":"2024-07-04T17:33:30.412821+0000","last_fullsized":"2024-07-04T17:33:30.412821+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:33:02.261126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444083+0000","last_change":"2024-07-04T17:33:29.438831+0000","last_active":"2024-07-04T17:33:30.444083+0000","last_peered":"2024-07-04T17:33:30.444083+0000","last_clean":"2024-07-04T17:33:30.444083+0000","last_became_active":"2024-07-04T17:33:29.438485+0000","last_became_peered":"2024-07-04T17:33:29.438485+0000","last_unstale":"2024-07-04T17:33:30.444083+0000","last_undegraded":"2024-07-04T17:33:30.444083+0000","last_fullsized":"2024-07-04T17:33:30.444083+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:14:36.275709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.445410+0000","last_change":"2024-07-04T17:33:09.119181+0000","last_active":"2024-07-04T17:33:30.445410+0000","last_peered":"2024-07-04T17:33:30.445410+0000","last_clean":"2024-07-04T17:33:30.445410+0000","last_became_active":"2024-07-04T17:33:09.118791+0000","last_became_peered":"2024-07-04T17:33:09.118791+0000","last_unstale":"2024-07-04T17:33:30.445410+0000","last_undegraded":"2024-07-04T17:33:30.445410+0000","last_fullsized":"2024-07-04T17:33:30.445410+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:08:30.473430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.447063+0000","last_change":"2024-07-04T17:33:29.422486+0000","last_active":"2024-07-04T17:33:30.447063+0000","last_peered":"2024-07-04T17:33:30.447063+0000","last_clean":"2024-07-04T17:33:30.447063+0000","last_became_active":"2024-07-04T17:33:29.420743+0000","last_became_peered":"2024-07-04T17:33:29.420743+0000","last_unstale":"2024-07-04T17:33:30.447063+0000","last_undegraded":"2024-07-04T17:33:30.447063+0000","last_fullsized":"2024-07-04T17:33:30.447063+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:35:34.544233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.412057+0000","last_change":"2024-07-04T17:33:09.111491+0000","last_active":"2024-07-04T17:33:30.412057+0000","last_peered":"2024-07-04T17:33:30.412057+0000","last_clean":"2024-07-04T17:33:30.412057+0000","last_became_active":"2024-07-04T17:33:09.111095+0000","last_became_peered":"2024-07-04T17:33:09.111095+0000","last_unstale":"2024-07-04T17:33:30.412057+0000","last_undegraded":"2024-07-04T17:33:30.412057+0000","last_fullsized":"2024-07-04T17:33:30.412057+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:49:13.666901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444105+0000","last_change":"2024-07-04T17:33:29.446498+0000","last_active":"2024-07-04T17:33:30.444105+0000","last_peered":"2024-07-04T17:33:30.444105+0000","last_clean":"2024-07-04T17:33:30.444105+0000","last_became_active":"2024-07-04T17:33:29.446127+0000","last_became_peered":"2024-07-04T17:33:29.446127+0000","last_unstale":"2024-07-04T17:33:30.444105+0000","last_undegraded":"2024-07-04T17:33:30.444105+0000","last_fullsized":"2024-07-04T17:33:30.444105+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:45:31.742540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.433639+0000","last_change":"2024-07-04T17:33:09.101446+0000","last_active":"2024-07-04T17:33:30.433639+0000","last_peered":"2024-07-04T17:33:30.433639+0000","last_clean":"2024-07-04T17:33:30.433639+0000","last_became_active":"2024-07-04T17:33:09.100644+0000","last_became_peered":"2024-07-04T17:33:09.100644+0000","last_unstale":"2024-07-04T17:33:30.433639+0000","last_undegraded":"2024-07-04T17:33:30.433639+0000","last_fullsized":"2024-07-04T17:33:30.433639+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:27:36.573799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.438186+0000","last_change":"2024-07-04T17:33:29.442123+0000","last_active":"2024-07-04T17:33:30.438186+0000","last_peered":"2024-07-04T17:33:30.438186+0000","last_clean":"2024-07-04T17:33:30.438186+0000","last_became_active":"2024-07-04T17:33:29.441776+0000","last_became_peered":"2024-07-04T17:33:29.441776+0000","last_unstale":"2024-07-04T17:33:30.438186+0000","last_undegraded":"2024-07-04T17:33:30.438186+0000","last_fullsized":"2024-07-04T17:33:30.438186+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:00:42.445313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.439148+0000","last_change":"2024-07-04T17:33:09.124916+0000","last_active":"2024-07-04T17:33:30.439148+0000","last_peered":"2024-07-04T17:33:30.439148+0000","last_clean":"2024-07-04T17:33:30.439148+0000","last_became_active":"2024-07-04T17:33:09.124557+0000","last_became_peered":"2024-07-04T17:33:09.124557+0000","last_unstale":"2024-07-04T17:33:30.439148+0000","last_undegraded":"2024-07-04T17:33:30.439148+0000","last_fullsized":"2024-07-04T17:33:30.439148+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:10:26.737295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.435469+0000","last_change":"2024-07-04T17:33:29.451690+0000","last_active":"2024-07-04T17:33:30.435469+0000","last_peered":"2024-07-04T17:33:30.435469+0000","last_clean":"2024-07-04T17:33:30.435469+0000","last_became_active":"2024-07-04T17:33:29.451059+0000","last_became_peered":"2024-07-04T17:33:29.451059+0000","last_unstale":"2024-07-04T17:33:30.435469+0000","last_undegraded":"2024-07-04T17:33:30.435469+0000","last_fullsized":"2024-07-04T17:33:30.435469+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:35:11.071722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.415085+0000","last_change":"2024-07-04T17:33:09.115371+0000","last_active":"2024-07-04T17:33:30.415085+0000","last_peered":"2024-07-04T17:33:30.415085+0000","last_clean":"2024-07-04T17:33:30.415085+0000","last_became_active":"2024-07-04T17:33:09.114560+0000","last_became_peered":"2024-07-04T17:33:09.114560+0000","last_unstale":"2024-07-04T17:33:30.415085+0000","last_undegraded":"2024-07-04T17:33:30.415085+0000","last_fullsized":"2024-07-04T17:33:30.415085+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:39:29.413889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.439734+0000","last_change":"2024-07-04T17:33:29.450333+0000","last_active":"2024-07-04T17:33:30.439734+0000","last_peered":"2024-07-04T17:33:30.439734+0000","last_clean":"2024-07-04T17:33:30.439734+0000","last_became_active":"2024-07-04T17:33:29.449918+0000","last_became_peered":"2024-07-04T17:33:29.449918+0000","last_unstale":"2024-07-04T17:33:30.439734+0000","last_undegraded":"2024-07-04T17:33:30.439734+0000","last_fullsized":"2024-07-04T17:33:30.439734+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:44:58.705067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.414480+0000","last_change":"2024-07-04T17:33:09.113151+0000","last_active":"2024-07-04T17:33:30.414480+0000","last_peered":"2024-07-04T17:33:30.414480+0000","last_clean":"2024-07-04T17:33:30.414480+0000","last_became_active":"2024-07-04T17:33:09.112228+0000","last_became_peered":"2024-07-04T17:33:09.112228+0000","last_unstale":"2024-07-04T17:33:30.414480+0000","last_undegraded":"2024-07-04T17:33:30.414480+0000","last_fullsized":"2024-07-04T17:33:30.414480+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:16:18.941587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.414686+0000","last_change":"2024-07-04T17:33:29.447697+0000","last_active":"2024-07-04T17:33:30.414686+0000","last_peered":"2024-07-04T17:33:30.414686+0000","last_clean":"2024-07-04T17:33:30.414686+0000","last_became_active":"2024-07-04T17:33:29.447194+0000","last_became_peered":"2024-07-04T17:33:29.447194+0000","last_unstale":"2024-07-04T17:33:30.414686+0000","last_undegraded":"2024-07-04T17:33:30.414686+0000","last_fullsized":"2024-07-04T17:33:30.414686+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:22:48.866984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.436742+0000","last_change":"2024-07-04T17:33:09.118128+0000","last_active":"2024-07-04T17:33:30.436742+0000","last_peered":"2024-07-04T17:33:30.436742+0000","last_clean":"2024-07-04T17:33:30.436742+0000","last_became_active":"2024-07-04T17:33:09.117473+0000","last_became_peered":"2024-07-04T17:33:09.117473+0000","last_unstale":"2024-07-04T17:33:30.436742+0000","last_undegraded":"2024-07-04T17:33:30.436742+0000","last_fullsized":"2024-07-04T17:33:30.436742+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:06:51.425997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.434772+0000","last_change":"2024-07-04T17:33:29.462800+0000","last_active":"2024-07-04T17:33:30.434772+0000","last_peered":"2024-07-04T17:33:30.434772+0000","last_clean":"2024-07-04T17:33:30.434772+0000","last_became_active":"2024-07-04T17:33:29.462567+0000","last_became_peered":"2024-07-04T17:33:29.462567+0000","last_unstale":"2024-07-04T17:33:30.434772+0000","last_undegraded":"2024-07-04T17:33:30.434772+0000","last_fullsized":"2024-07-04T17:33:30.434772+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:45:04.147190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.435764+0000","last_change":"2024-07-04T17:33:09.127162+0000","last_active":"2024-07-04T17:33:30.435764+0000","last_peered":"2024-07-04T17:33:30.435764+0000","last_clean":"2024-07-04T17:33:30.435764+0000","last_became_active":"2024-07-04T17:33:09.123389+0000","last_became_peered":"2024-07-04T17:33:09.123389+0000","last_unstale":"2024-07-04T17:33:30.435764+0000","last_undegraded":"2024-07-04T17:33:30.435764+0000","last_fullsized":"2024-07-04T17:33:30.435764+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:33:17.955726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.412632+0000","last_change":"2024-07-04T17:33:29.434386+0000","last_active":"2024-07-04T17:33:30.412632+0000","last_peered":"2024-07-04T17:33:30.412632+0000","last_clean":"2024-07-04T17:33:30.412632+0000","last_became_active":"2024-07-04T17:33:29.431792+0000","last_became_peered":"2024-07-04T17:33:29.431792+0000","last_unstale":"2024-07-04T17:33:30.412632+0000","last_undegraded":"2024-07-04T17:33:30.412632+0000","last_fullsized":"2024-07-04T17:33:30.412632+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:48:23.639508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.445498+0000","last_change":"2024-07-04T17:33:09.118567+0000","last_active":"2024-07-04T17:33:30.445498+0000","last_peered":"2024-07-04T17:33:30.445498+0000","last_clean":"2024-07-04T17:33:30.445498+0000","last_became_active":"2024-07-04T17:33:09.117227+0000","last_became_peered":"2024-07-04T17:33:09.117227+0000","last_unstale":"2024-07-04T17:33:30.445498+0000","last_undegraded":"2024-07-04T17:33:30.445498+0000","last_fullsized":"2024-07-04T17:33:30.445498+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:42:43.320358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.436762+0000","last_change":"2024-07-04T17:33:29.452959+0000","last_active":"2024-07-04T17:33:30.436762+0000","last_peered":"2024-07-04T17:33:30.436762+0000","last_clean":"2024-07-04T17:33:30.436762+0000","last_became_active":"2024-07-04T17:33:29.451873+0000","last_became_peered":"2024-07-04T17:33:29.451873+0000","last_unstale":"2024-07-04T17:33:30.436762+0000","last_undegraded":"2024-07-04T17:33:30.436762+0000","last_fullsized":"2024-07-04T17:33:30.436762+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:37:16.694086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.434211+0000","last_change":"2024-07-04T17:33:09.131948+0000","last_active":"2024-07-04T17:33:30.434211+0000","last_peered":"2024-07-04T17:33:30.434211+0000","last_clean":"2024-07-04T17:33:30.434211+0000","last_became_active":"2024-07-04T17:33:09.121717+0000","last_became_peered":"2024-07-04T17:33:09.121717+0000","last_unstale":"2024-07-04T17:33:30.434211+0000","last_undegraded":"2024-07-04T17:33:30.434211+0000","last_fullsized":"2024-07-04T17:33:30.434211+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:05:18.766887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.411993+0000","last_change":"2024-07-04T17:33:29.456681+0000","last_active":"2024-07-04T17:33:30.411993+0000","last_peered":"2024-07-04T17:33:30.411993+0000","last_clean":"2024-07-04T17:33:30.411993+0000","last_became_active":"2024-07-04T17:33:29.456262+0000","last_became_peered":"2024-07-04T17:33:29.456262+0000","last_unstale":"2024-07-04T17:33:30.411993+0000","last_undegraded":"2024-07-04T17:33:30.411993+0000","last_fullsized":"2024-07-04T17:33:30.411993+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:16:21.298639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.412705+0000","last_change":"2024-07-04T17:33:09.124489+0000","last_active":"2024-07-04T17:33:30.412705+0000","last_peered":"2024-07-04T17:33:30.412705+0000","last_clean":"2024-07-04T17:33:30.412705+0000","last_became_active":"2024-07-04T17:33:09.122889+0000","last_became_peered":"2024-07-04T17:33:09.122889+0000","last_unstale":"2024-07-04T17:33:30.412705+0000","last_undegraded":"2024-07-04T17:33:30.412705+0000","last_fullsized":"2024-07-04T17:33:30.412705+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:01:35.821887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.413627+0000","last_change":"2024-07-04T17:33:29.460554+0000","last_active":"2024-07-04T17:33:30.413627+0000","last_peered":"2024-07-04T17:33:30.413627+0000","last_clean":"2024-07-04T17:33:30.413627+0000","last_became_active":"2024-07-04T17:33:29.459577+0000","last_became_peered":"2024-07-04T17:33:29.459577+0000","last_unstale":"2024-07-04T17:33:30.413627+0000","last_undegraded":"2024-07-04T17:33:30.413627+0000","last_fullsized":"2024-07-04T17:33:30.413627+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:08:40.734996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.439265+0000","last_change":"2024-07-04T17:33:09.118397+0000","last_active":"2024-07-04T17:33:30.439265+0000","last_peered":"2024-07-04T17:33:30.439265+0000","last_clean":"2024-07-04T17:33:30.439265+0000","last_became_active":"2024-07-04T17:33:09.117509+0000","last_became_peered":"2024-07-04T17:33:09.117509+0000","last_unstale":"2024-07-04T17:33:30.439265+0000","last_undegraded":"2024-07-04T17:33:30.439265+0000","last_fullsized":"2024-07-04T17:33:30.439265+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:37:23.202824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.415159+0000","last_change":"2024-07-04T17:33:29.452392+0000","last_active":"2024-07-04T17:33:30.415159+0000","last_peered":"2024-07-04T17:33:30.415159+0000","last_clean":"2024-07-04T17:33:30.415159+0000","last_became_active":"2024-07-04T17:33:29.451644+0000","last_became_peered":"2024-07-04T17:33:29.451644+0000","last_unstale":"2024-07-04T17:33:30.415159+0000","last_undegraded":"2024-07-04T17:33:30.415159+0000","last_fullsized":"2024-07-04T17:33:30.415159+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:37:06.907567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.415132+0000","last_change":"2024-07-04T17:33:09.115358+0000","last_active":"2024-07-04T17:33:30.415132+0000","last_peered":"2024-07-04T17:33:30.415132+0000","last_clean":"2024-07-04T17:33:30.415132+0000","last_became_active":"2024-07-04T17:33:09.113650+0000","last_became_peered":"2024-07-04T17:33:09.113650+0000","last_unstale":"2024-07-04T17:33:30.415132+0000","last_undegraded":"2024-07-04T17:33:30.415132+0000","last_fullsized":"2024-07-04T17:33:30.415132+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:34:20.388421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.414069+0000","last_change":"2024-07-04T17:33:29.442803+0000","last_active":"2024-07-04T17:33:30.414069+0000","last_peered":"2024-07-04T17:33:30.414069+0000","last_clean":"2024-07-04T17:33:30.414069+0000","last_became_active":"2024-07-04T17:33:29.442473+0000","last_became_peered":"2024-07-04T17:33:29.442473+0000","last_unstale":"2024-07-04T17:33:30.414069+0000","last_undegraded":"2024-07-04T17:33:30.414069+0000","last_fullsized":"2024-07-04T17:33:30.414069+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:49:43.858447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.436722+0000","last_change":"2024-07-04T17:33:09.130921+0000","last_active":"2024-07-04T17:33:30.436722+0000","last_peered":"2024-07-04T17:33:30.436722+0000","last_clean":"2024-07-04T17:33:30.436722+0000","last_became_active":"2024-07-04T17:33:09.123521+0000","last_became_peered":"2024-07-04T17:33:09.123521+0000","last_unstale":"2024-07-04T17:33:30.436722+0000","last_undegraded":"2024-07-04T17:33:30.436722+0000","last_fullsized":"2024-07-04T17:33:30.436722+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:13:51.565345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.447258+0000","last_change":"2024-07-04T17:33:29.444474+0000","last_active":"2024-07-04T17:33:30.447258+0000","last_peered":"2024-07-04T17:33:30.447258+0000","last_clean":"2024-07-04T17:33:30.447258+0000","last_became_active":"2024-07-04T17:33:29.444096+0000","last_became_peered":"2024-07-04T17:33:29.444096+0000","last_unstale":"2024-07-04T17:33:30.447258+0000","last_undegraded":"2024-07-04T17:33:30.447258+0000","last_fullsized":"2024-07-04T17:33:30.447258+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:48:54.442775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.446331+0000","last_change":"2024-07-04T17:33:09.114604+0000","last_active":"2024-07-04T17:33:30.446331+0000","last_peered":"2024-07-04T17:33:30.446331+0000","last_clean":"2024-07-04T17:33:30.446331+0000","last_became_active":"2024-07-04T17:33:09.114294+0000","last_became_peered":"2024-07-04T17:33:09.114294+0000","last_unstale":"2024-07-04T17:33:30.446331+0000","last_undegraded":"2024-07-04T17:33:30.446331+0000","last_fullsized":"2024-07-04T17:33:30.446331+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:34:50.754812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.433534+0000","last_change":"2024-07-04T17:33:29.453534+0000","last_active":"2024-07-04T17:33:30.433534+0000","last_peered":"2024-07-04T17:33:30.433534+0000","last_clean":"2024-07-04T17:33:30.433534+0000","last_became_active":"2024-07-04T17:33:29.448780+0000","last_became_peered":"2024-07-04T17:33:29.448780+0000","last_unstale":"2024-07-04T17:33:30.433534+0000","last_undegraded":"2024-07-04T17:33:30.433534+0000","last_fullsized":"2024-07-04T17:33:30.433534+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:27:44.574496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.412993+0000","last_change":"2024-07-04T17:33:09.124878+0000","last_active":"2024-07-04T17:33:30.412993+0000","last_peered":"2024-07-04T17:33:30.412993+0000","last_clean":"2024-07-04T17:33:30.412993+0000","last_became_active":"2024-07-04T17:33:09.124119+0000","last_became_peered":"2024-07-04T17:33:09.124119+0000","last_unstale":"2024-07-04T17:33:30.412993+0000","last_undegraded":"2024-07-04T17:33:30.412993+0000","last_fullsized":"2024-07-04T17:33:30.412993+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:18:22.081730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444421+0000","last_change":"2024-07-04T17:33:29.426691+0000","last_active":"2024-07-04T17:33:30.444421+0000","last_peered":"2024-07-04T17:33:30.444421+0000","last_clean":"2024-07-04T17:33:30.444421+0000","last_became_active":"2024-07-04T17:33:29.426009+0000","last_became_peered":"2024-07-04T17:33:29.426009+0000","last_unstale":"2024-07-04T17:33:30.444421+0000","last_undegraded":"2024-07-04T17:33:30.444421+0000","last_fullsized":"2024-07-04T17:33:30.444421+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:57:14.024066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.445613+0000","last_change":"2024-07-04T17:33:09.104697+0000","last_active":"2024-07-04T17:33:30.445613+0000","last_peered":"2024-07-04T17:33:30.445613+0000","last_clean":"2024-07-04T17:33:30.445613+0000","last_became_active":"2024-07-04T17:33:09.103509+0000","last_became_peered":"2024-07-04T17:33:09.103509+0000","last_unstale":"2024-07-04T17:33:30.445613+0000","last_undegraded":"2024-07-04T17:33:30.445613+0000","last_fullsized":"2024-07-04T17:33:30.445613+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:05:17.882764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.436344+0000","last_change":"2024-07-04T17:33:29.452531+0000","last_active":"2024-07-04T17:33:30.436344+0000","last_peered":"2024-07-04T17:33:30.436344+0000","last_clean":"2024-07-04T17:33:30.436344+0000","last_became_active":"2024-07-04T17:33:29.452204+0000","last_became_peered":"2024-07-04T17:33:29.452204+0000","last_unstale":"2024-07-04T17:33:30.436344+0000","last_undegraded":"2024-07-04T17:33:30.436344+0000","last_fullsized":"2024-07-04T17:33:30.436344+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:41:05.853324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444824+0000","last_change":"2024-07-04T17:33:09.104054+0000","last_active":"2024-07-04T17:33:30.444824+0000","last_peered":"2024-07-04T17:33:30.444824+0000","last_clean":"2024-07-04T17:33:30.444824+0000","last_became_active":"2024-07-04T17:33:09.103713+0000","last_became_peered":"2024-07-04T17:33:09.103713+0000","last_unstale":"2024-07-04T17:33:30.444824+0000","last_undegraded":"2024-07-04T17:33:30.444824+0000","last_fullsized":"2024-07-04T17:33:30.444824+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:27:24.257586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444539+0000","last_change":"2024-07-04T17:33:29.444892+0000","last_active":"2024-07-04T17:33:30.444539+0000","last_peered":"2024-07-04T17:33:30.444539+0000","last_clean":"2024-07-04T17:33:30.444539+0000","last_became_active":"2024-07-04T17:33:29.444506+0000","last_became_peered":"2024-07-04T17:33:29.444506+0000","last_unstale":"2024-07-04T17:33:30.444539+0000","last_undegraded":"2024-07-04T17:33:30.444539+0000","last_fullsized":"2024-07-04T17:33:30.444539+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:02:54.958549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.433811+0000","last_change":"2024-07-04T17:33:09.122343+0000","last_active":"2024-07-04T17:33:30.433811+0000","last_peered":"2024-07-04T17:33:30.433811+0000","last_clean":"2024-07-04T17:33:30.433811+0000","last_became_active":"2024-07-04T17:33:09.121982+0000","last_became_peered":"2024-07-04T17:33:09.121982+0000","last_unstale":"2024-07-04T17:33:30.433811+0000","last_undegraded":"2024-07-04T17:33:30.433811+0000","last_fullsized":"2024-07-04T17:33:30.433811+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:55:41.686107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.435981+0000","last_change":"2024-07-04T17:33:29.453791+0000","last_active":"2024-07-04T17:33:30.435981+0000","last_peered":"2024-07-04T17:33:30.435981+0000","last_clean":"2024-07-04T17:33:30.435981+0000","last_became_active":"2024-07-04T17:33:29.453492+0000","last_became_peered":"2024-07-04T17:33:29.453492+0000","last_unstale":"2024-07-04T17:33:30.435981+0000","last_undegraded":"2024-07-04T17:33:30.435981+0000","last_fullsized":"2024-07-04T17:33:30.435981+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:35:11.793595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.413868+0000","last_change":"2024-07-04T17:33:28.430109+0000","last_active":"2024-07-04T17:33:30.413868+0000","last_peered":"2024-07-04T17:33:30.413868+0000","last_clean":"2024-07-04T17:33:30.413868+0000","last_became_active":"2024-07-04T17:33:28.428355+0000","last_became_peered":"2024-07-04T17:33:28.428355+0000","last_unstale":"2024-07-04T17:33:30.413868+0000","last_undegraded":"2024-07-04T17:33:30.413868+0000","last_fullsized":"2024-07-04T17:33:30.413868+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:10:37.736526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.415214+0000","last_change":"2024-07-04T17:33:09.115618+0000","last_active":"2024-07-04T17:33:30.415214+0000","last_peered":"2024-07-04T17:33:30.415214+0000","last_clean":"2024-07-04T17:33:30.415214+0000","last_became_active":"2024-07-04T17:33:09.115047+0000","last_became_peered":"2024-07-04T17:33:09.115047+0000","last_unstale":"2024-07-04T17:33:30.415214+0000","last_undegraded":"2024-07-04T17:33:30.415214+0000","last_fullsized":"2024-07-04T17:33:30.415214+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:38:58.383535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"113'4","reported_seq":26,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.414050+0000","last_change":"2024-07-04T17:33:08.117620+0000","last_active":"2024-07-04T17:33:30.414050+0000","last_peered":"2024-07-04T17:33:30.414050+0000","last_clean":"2024-07-04T17:33:30.414050+0000","last_became_active":"2024-07-04T17:33:08.117097+0000","last_became_peered":"2024-07-04T17:33:08.117097+0000","last_unstale":"2024-07-04T17:33:30.414050+0000","last_undegraded":"2024-07-04T17:33:30.414050+0000","last_fullsized":"2024-07-04T17:33:30.414050+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:00:27.380504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.447978+0000","last_change":"2024-07-04T17:33:29.438701+0000","last_active":"2024-07-04T17:33:30.447978+0000","last_peered":"2024-07-04T17:33:30.447978+0000","last_clean":"2024-07-04T17:33:30.447978+0000","last_became_active":"2024-07-04T17:33:29.427189+0000","last_became_peered":"2024-07-04T17:33:29.427189+0000","last_unstale":"2024-07-04T17:33:30.447978+0000","last_undegraded":"2024-07-04T17:33:30.447978+0000","last_fullsized":"2024-07-04T17:33:30.447978+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:29:19.275486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1c","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.415560+0000","last_change":"2024-07-04T17:33:28.421141+0000","last_active":"2024-07-04T17:33:30.415560+0000","last_peered":"2024-07-04T17:33:30.415560+0000","last_clean":"2024-07-04T17:33:30.415560+0000","last_became_active":"2024-07-04T17:33:28.420812+0000","last_became_peered":"2024-07-04T17:33:28.420812+0000","last_unstale":"2024-07-04T17:33:30.415560+0000","last_undegraded":"2024-07-04T17:33:30.415560+0000","last_fullsized":"2024-07-04T17:33:30.415560+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:08:37.812504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.439375+0000","last_change":"2024-07-04T17:33:09.133070+0000","last_active":"2024-07-04T17:33:30.439375+0000","last_peered":"2024-07-04T17:33:30.439375+0000","last_clean":"2024-07-04T17:33:30.439375+0000","last_became_active":"2024-07-04T17:33:09.132634+0000","last_became_peered":"2024-07-04T17:33:09.132634+0000","last_unstale":"2024-07-04T17:33:30.439375+0000","last_undegraded":"2024-07-04T17:33:30.439375+0000","last_fullsized":"2024-07-04T17:33:30.439375+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:42:13.182154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1e","version":"113'1","reported_seq":23,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.448851+0000","last_change":"2024-07-04T17:33:08.105107+0000","last_active":"2024-07-04T17:33:30.448851+0000","last_peered":"2024-07-04T17:33:30.448851+0000","last_clean":"2024-07-04T17:33:30.448851+0000","last_became_active":"2024-07-04T17:33:08.104670+0000","last_became_peered":"2024-07-04T17:33:08.104670+0000","last_unstale":"2024-07-04T17:33:30.448851+0000","last_undegraded":"2024-07-04T17:33:30.448851+0000","last_fullsized":"2024-07-04T17:33:30.448851+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:46:04.479168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.447524+0000","last_change":"2024-07-04T17:33:29.433713+0000","last_active":"2024-07-04T17:33:30.447524+0000","last_peered":"2024-07-04T17:33:30.447524+0000","last_clean":"2024-07-04T17:33:30.447524+0000","last_became_active":"2024-07-04T17:33:29.433373+0000","last_became_peered":"2024-07-04T17:33:29.433373+0000","last_unstale":"2024-07-04T17:33:30.447524+0000","last_undegraded":"2024-07-04T17:33:30.447524+0000","last_fullsized":"2024-07-04T17:33:30.447524+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:55:35.551291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"117'4","reported_seq":18,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.438539+0000","last_change":"2024-07-04T17:33:28.432934+0000","last_active":"2024-07-04T17:33:30.438539+0000","last_peered":"2024-07-04T17:33:30.438539+0000","last_clean":"2024-07-04T17:33:30.438539+0000","last_became_active":"2024-07-04T17:33:28.432353+0000","last_became_peered":"2024-07-04T17:33:28.432353+0000","last_unstale":"2024-07-04T17:33:30.438539+0000","last_undegraded":"2024-07-04T17:33:30.438539+0000","last_fullsized":"2024-07-04T17:33:30.438539+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:43:23.051302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.415936+0000","last_change":"2024-07-04T17:33:09.107265+0000","last_active":"2024-07-04T17:33:30.415936+0000","last_peered":"2024-07-04T17:33:30.415936+0000","last_clean":"2024-07-04T17:33:30.415936+0000","last_became_active":"2024-07-04T17:33:09.106633+0000","last_became_peered":"2024-07-04T17:33:09.106633+0000","last_unstale":"2024-07-04T17:33:30.415936+0000","last_undegraded":"2024-07-04T17:33:30.415936+0000","last_fullsized":"2024-07-04T17:33:30.415936+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:05:30.327143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"113'2","reported_seq":24,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.438292+0000","last_change":"2024-07-04T17:33:08.089824+0000","last_active":"2024-07-04T17:33:30.438292+0000","last_peered":"2024-07-04T17:33:30.438292+0000","last_clean":"2024-07-04T17:33:30.438292+0000","last_became_active":"2024-07-04T17:33:08.084113+0000","last_became_peered":"2024-07-04T17:33:08.084113+0000","last_unstale":"2024-07-04T17:33:30.438292+0000","last_undegraded":"2024-07-04T17:33:30.438292+0000","last_fullsized":"2024-07-04T17:33:30.438292+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:39:25.915226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.438462+0000","last_change":"2024-07-04T17:33:29.443535+0000","last_active":"2024-07-04T17:33:30.438462+0000","last_peered":"2024-07-04T17:33:30.438462+0000","last_clean":"2024-07-04T17:33:30.438462+0000","last_became_active":"2024-07-04T17:33:29.443073+0000","last_became_peered":"2024-07-04T17:33:29.443073+0000","last_unstale":"2024-07-04T17:33:30.438462+0000","last_undegraded":"2024-07-04T17:33:30.438462+0000","last_fullsized":"2024-07-04T17:33:30.438462+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:33:11.025783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1e","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.447614+0000","last_change":"2024-07-04T17:33:28.408215+0000","last_active":"2024-07-04T17:33:30.447614+0000","last_peered":"2024-07-04T17:33:30.447614+0000","last_clean":"2024-07-04T17:33:30.447614+0000","last_became_active":"2024-07-04T17:33:28.407844+0000","last_became_peered":"2024-07-04T17:33:28.407844+0000","last_unstale":"2024-07-04T17:33:30.447614+0000","last_undegraded":"2024-07-04T17:33:30.447614+0000","last_fullsized":"2024-07-04T17:33:30.447614+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:17:40.690935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.436619+0000","last_change":"2024-07-04T17:33:09.129582+0000","last_active":"2024-07-04T17:33:30.436619+0000","last_peered":"2024-07-04T17:33:30.436619+0000","last_clean":"2024-07-04T17:33:30.436619+0000","last_became_active":"2024-07-04T17:33:09.127839+0000","last_became_peered":"2024-07-04T17:33:09.127839+0000","last_unstale":"2024-07-04T17:33:30.436619+0000","last_undegraded":"2024-07-04T17:33:30.436619+0000","last_fullsized":"2024-07-04T17:33:30.436619+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:35:13.414729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"113'2","reported_seq":24,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.436890+0000","last_change":"2024-07-04T17:33:08.108862+0000","last_active":"2024-07-04T17:33:30.436890+0000","last_peered":"2024-07-04T17:33:30.436890+0000","last_clean":"2024-07-04T17:33:30.436890+0000","last_became_active":"2024-07-04T17:33:08.107761+0000","last_became_peered":"2024-07-04T17:33:08.107761+0000","last_unstale":"2024-07-04T17:33:30.436890+0000","last_undegraded":"2024-07-04T17:33:30.436890+0000","last_fullsized":"2024-07-04T17:33:30.436890+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:15:53.250590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.411488+0000","last_change":"2024-07-04T17:33:29.424087+0000","last_active":"2024-07-04T17:33:30.411488+0000","last_peered":"2024-07-04T17:33:30.411488+0000","last_clean":"2024-07-04T17:33:30.411488+0000","last_became_active":"2024-07-04T17:33:29.417359+0000","last_became_peered":"2024-07-04T17:33:29.417359+0000","last_unstale":"2024-07-04T17:33:30.411488+0000","last_undegraded":"2024-07-04T17:33:30.411488+0000","last_fullsized":"2024-07-04T17:33:30.411488+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:56:07.610007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.433946+0000","last_change":"2024-07-04T17:33:28.424837+0000","last_active":"2024-07-04T17:33:30.433946+0000","last_peered":"2024-07-04T17:33:30.433946+0000","last_clean":"2024-07-04T17:33:30.433946+0000","last_became_active":"2024-07-04T17:33:28.424496+0000","last_became_peered":"2024-07-04T17:33:28.424496+0000","last_unstale":"2024-07-04T17:33:30.433946+0000","last_undegraded":"2024-07-04T17:33:30.433946+0000","last_fullsized":"2024-07-04T17:33:30.433946+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:47:24.359112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.445853+0000","last_change":"2024-07-04T17:33:09.118310+0000","last_active":"2024-07-04T17:33:30.445853+0000","last_peered":"2024-07-04T17:33:30.445853+0000","last_clean":"2024-07-04T17:33:30.445853+0000","last_became_active":"2024-07-04T17:33:09.116852+0000","last_became_peered":"2024-07-04T17:33:09.116852+0000","last_unstale":"2024-07-04T17:33:30.445853+0000","last_undegraded":"2024-07-04T17:33:30.445853+0000","last_fullsized":"2024-07-04T17:33:30.445853+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:41:21.265187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.1b","version":"113'1","reported_seq":23,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.438224+0000","last_change":"2024-07-04T17:33:08.105642+0000","last_active":"2024-07-04T17:33:30.438224+0000","last_peered":"2024-07-04T17:33:30.438224+0000","last_clean":"2024-07-04T17:33:30.438224+0000","last_became_active":"2024-07-04T17:33:08.105297+0000","last_became_peered":"2024-07-04T17:33:08.105297+0000","last_unstale":"2024-07-04T17:33:30.438224+0000","last_undegraded":"2024-07-04T17:33:30.438224+0000","last_fullsized":"2024-07-04T17:33:30.438224+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:50:34.893649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.433487+0000","last_change":"2024-07-04T17:33:29.455771+0000","last_active":"2024-07-04T17:33:30.433487+0000","last_peered":"2024-07-04T17:33:30.433487+0000","last_clean":"2024-07-04T17:33:30.433487+0000","last_became_active":"2024-07-04T17:33:29.455458+0000","last_became_peered":"2024-07-04T17:33:29.455458+0000","last_unstale":"2024-07-04T17:33:30.433487+0000","last_undegraded":"2024-07-04T17:33:30.433487+0000","last_fullsized":"2024-07-04T17:33:30.433487+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:08:50.790340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444532+0000","last_change":"2024-07-04T17:33:28.408122+0000","last_active":"2024-07-04T17:33:30.444532+0000","last_peered":"2024-07-04T17:33:30.444532+0000","last_clean":"2024-07-04T17:33:30.444532+0000","last_became_active":"2024-07-04T17:33:28.407784+0000","last_became_peered":"2024-07-04T17:33:28.407784+0000","last_unstale":"2024-07-04T17:33:30.444532+0000","last_undegraded":"2024-07-04T17:33:30.444532+0000","last_fullsized":"2024-07-04T17:33:30.444532+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:33:01.100217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.435962+0000","last_change":"2024-07-04T17:33:09.116075+0000","last_active":"2024-07-04T17:33:30.435962+0000","last_peered":"2024-07-04T17:33:30.435962+0000","last_clean":"2024-07-04T17:33:30.435962+0000","last_became_active":"2024-07-04T17:33:09.115705+0000","last_became_peered":"2024-07-04T17:33:09.115705+0000","last_unstale":"2024-07-04T17:33:30.435962+0000","last_undegraded":"2024-07-04T17:33:30.435962+0000","last_fullsized":"2024-07-04T17:33:30.435962+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:52:50.422683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"113'1","reported_seq":23,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.412996+0000","last_change":"2024-07-04T17:33:08.116547+0000","last_active":"2024-07-04T17:33:30.412996+0000","last_peered":"2024-07-04T17:33:30.412996+0000","last_clean":"2024-07-04T17:33:30.412996+0000","last_became_active":"2024-07-04T17:33:08.116095+0000","last_became_peered":"2024-07-04T17:33:08.116095+0000","last_unstale":"2024-07-04T17:33:30.412996+0000","last_undegraded":"2024-07-04T17:33:30.412996+0000","last_fullsized":"2024-07-04T17:33:30.412996+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:31:44.812381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444370+0000","last_change":"2024-07-04T17:33:29.433466+0000","last_active":"2024-07-04T17:33:30.444370+0000","last_peered":"2024-07-04T17:33:30.444370+0000","last_clean":"2024-07-04T17:33:30.444370+0000","last_became_active":"2024-07-04T17:33:29.424893+0000","last_became_peered":"2024-07-04T17:33:29.424893+0000","last_unstale":"2024-07-04T17:33:30.444370+0000","last_undegraded":"2024-07-04T17:33:30.444370+0000","last_fullsized":"2024-07-04T17:33:30.444370+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:29:45.081943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.435225+0000","last_change":"2024-07-04T17:33:28.417247+0000","last_active":"2024-07-04T17:33:30.435225+0000","last_peered":"2024-07-04T17:33:30.435225+0000","last_clean":"2024-07-04T17:33:30.435225+0000","last_became_active":"2024-07-04T17:33:28.416942+0000","last_became_peered":"2024-07-04T17:33:28.416942+0000","last_unstale":"2024-07-04T17:33:30.435225+0000","last_undegraded":"2024-07-04T17:33:30.435225+0000","last_fullsized":"2024-07-04T17:33:30.435225+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:37:41.663967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.412807+0000","last_change":"2024-07-04T17:33:09.125116+0000","last_active":"2024-07-04T17:33:30.412807+0000","last_peered":"2024-07-04T17:33:30.412807+0000","last_clean":"2024-07-04T17:33:30.412807+0000","last_became_active":"2024-07-04T17:33:09.124103+0000","last_became_peered":"2024-07-04T17:33:09.124103+0000","last_unstale":"2024-07-04T17:33:30.412807+0000","last_undegraded":"2024-07-04T17:33:30.412807+0000","last_fullsized":"2024-07-04T17:33:30.412807+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:27:31.156139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.435099+0000","last_change":"2024-07-04T17:33:08.096904+0000","last_active":"2024-07-04T17:33:30.435099+0000","last_peered":"2024-07-04T17:33:30.435099+0000","last_clean":"2024-07-04T17:33:30.435099+0000","last_became_active":"2024-07-04T17:33:08.096590+0000","last_became_peered":"2024-07-04T17:33:08.096590+0000","last_unstale":"2024-07-04T17:33:30.435099+0000","last_undegraded":"2024-07-04T17:33:30.435099+0000","last_fullsized":"2024-07-04T17:33:30.435099+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:39:51.529018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.411926+0000","last_change":"2024-07-04T17:33:29.430845+0000","last_active":"2024-07-04T17:33:30.411926+0000","last_peered":"2024-07-04T17:33:30.411926+0000","last_clean":"2024-07-04T17:33:30.411926+0000","last_became_active":"2024-07-04T17:33:29.430473+0000","last_became_peered":"2024-07-04T17:33:29.430473+0000","last_unstale":"2024-07-04T17:33:30.411926+0000","last_undegraded":"2024-07-04T17:33:30.411926+0000","last_fullsized":"2024-07-04T17:33:30.411926+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:14:00.417647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.411871+0000","last_change":"2024-07-04T17:33:28.424065+0000","last_active":"2024-07-04T17:33:30.411871+0000","last_peered":"2024-07-04T17:33:30.411871+0000","last_clean":"2024-07-04T17:33:30.411871+0000","last_became_active":"2024-07-04T17:33:28.423732+0000","last_became_peered":"2024-07-04T17:33:28.423732+0000","last_unstale":"2024-07-04T17:33:30.411871+0000","last_undegraded":"2024-07-04T17:33:30.411871+0000","last_fullsized":"2024-07-04T17:33:30.411871+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:28:03.208000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.434349+0000","last_change":"2024-07-04T17:33:09.100755+0000","last_active":"2024-07-04T17:33:30.434349+0000","last_peered":"2024-07-04T17:33:30.434349+0000","last_clean":"2024-07-04T17:33:30.434349+0000","last_became_active":"2024-07-04T17:33:09.100389+0000","last_became_peered":"2024-07-04T17:33:09.100389+0000","last_unstale":"2024-07-04T17:33:30.434349+0000","last_undegraded":"2024-07-04T17:33:30.434349+0000","last_fullsized":"2024-07-04T17:33:30.434349+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:38:04.457302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.445604+0000","last_change":"2024-07-04T17:33:08.106647+0000","last_active":"2024-07-04T17:33:30.445604+0000","last_peered":"2024-07-04T17:33:30.445604+0000","last_clean":"2024-07-04T17:33:30.445604+0000","last_became_active":"2024-07-04T17:33:08.106277+0000","last_became_peered":"2024-07-04T17:33:08.106277+0000","last_unstale":"2024-07-04T17:33:30.445604+0000","last_undegraded":"2024-07-04T17:33:30.445604+0000","last_fullsized":"2024-07-04T17:33:30.445604+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:47:04.050710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.445698+0000","last_change":"2024-07-04T17:33:29.433719+0000","last_active":"2024-07-04T17:33:30.445698+0000","last_peered":"2024-07-04T17:33:30.445698+0000","last_clean":"2024-07-04T17:33:30.445698+0000","last_became_active":"2024-07-04T17:33:29.433351+0000","last_became_peered":"2024-07-04T17:33:29.433351+0000","last_unstale":"2024-07-04T17:33:30.445698+0000","last_undegraded":"2024-07-04T17:33:30.445698+0000","last_fullsized":"2024-07-04T17:33:30.445698+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:05:24.141848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.15","version":"117'3","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.436055+0000","last_change":"2024-07-04T17:33:28.430046+0000","last_active":"2024-07-04T17:33:30.436055+0000","last_peered":"2024-07-04T17:33:30.436055+0000","last_clean":"2024-07-04T17:33:30.436055+0000","last_became_active":"2024-07-04T17:33:28.429140+0000","last_became_peered":"2024-07-04T17:33:28.429140+0000","last_unstale":"2024-07-04T17:33:30.436055+0000","last_undegraded":"2024-07-04T17:33:30.436055+0000","last_fullsized":"2024-07-04T17:33:30.436055+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:47:01.462455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.438802+0000","last_change":"2024-07-04T17:33:09.128294+0000","last_active":"2024-07-04T17:33:30.438802+0000","last_peered":"2024-07-04T17:33:30.438802+0000","last_clean":"2024-07-04T17:33:30.438802+0000","last_became_active":"2024-07-04T17:33:09.123696+0000","last_became_peered":"2024-07-04T17:33:09.123696+0000","last_unstale":"2024-07-04T17:33:30.438802+0000","last_undegraded":"2024-07-04T17:33:30.438802+0000","last_fullsized":"2024-07-04T17:33:30.438802+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:07:05.456492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.414224+0000","last_change":"2024-07-04T17:33:08.100600+0000","last_active":"2024-07-04T17:33:30.414224+0000","last_peered":"2024-07-04T17:33:30.414224+0000","last_clean":"2024-07-04T17:33:30.414224+0000","last_became_active":"2024-07-04T17:33:08.099352+0000","last_became_peered":"2024-07-04T17:33:08.099352+0000","last_unstale":"2024-07-04T17:33:30.414224+0000","last_undegraded":"2024-07-04T17:33:30.414224+0000","last_fullsized":"2024-07-04T17:33:30.414224+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:04:33.482388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.435960+0000","last_change":"2024-07-04T17:33:29.452926+0000","last_active":"2024-07-04T17:33:30.435960+0000","last_peered":"2024-07-04T17:33:30.435960+0000","last_clean":"2024-07-04T17:33:30.435960+0000","last_became_active":"2024-07-04T17:33:29.452322+0000","last_became_peered":"2024-07-04T17:33:29.452322+0000","last_unstale":"2024-07-04T17:33:30.435960+0000","last_undegraded":"2024-07-04T17:33:30.435960+0000","last_fullsized":"2024-07-04T17:33:30.435960+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:23:18.968713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.14","version":"117'6","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.415753+0000","last_change":"2024-07-04T17:33:28.429133+0000","last_active":"2024-07-04T17:33:30.415753+0000","last_peered":"2024-07-04T17:33:30.415753+0000","last_clean":"2024-07-04T17:33:30.415753+0000","last_became_active":"2024-07-04T17:33:28.428803+0000","last_became_peered":"2024-07-04T17:33:28.428803+0000","last_unstale":"2024-07-04T17:33:30.415753+0000","last_undegraded":"2024-07-04T17:33:30.415753+0000","last_fullsized":"2024-07-04T17:33:30.415753+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:53:44.447019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.447846+0000","last_change":"2024-07-04T17:33:09.109950+0000","last_active":"2024-07-04T17:33:30.447846+0000","last_peered":"2024-07-04T17:33:30.447846+0000","last_clean":"2024-07-04T17:33:30.447846+0000","last_became_active":"2024-07-04T17:33:09.108850+0000","last_became_peered":"2024-07-04T17:33:09.108850+0000","last_unstale":"2024-07-04T17:33:30.447846+0000","last_undegraded":"2024-07-04T17:33:30.447846+0000","last_fullsized":"2024-07-04T17:33:30.447846+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:22:58.651803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.415604+0000","last_change":"2024-07-04T17:33:08.117619+0000","last_active":"2024-07-04T17:33:30.415604+0000","last_peered":"2024-07-04T17:33:30.415604+0000","last_clean":"2024-07-04T17:33:30.415604+0000","last_became_active":"2024-07-04T17:33:08.117033+0000","last_became_peered":"2024-07-04T17:33:08.117033+0000","last_unstale":"2024-07-04T17:33:30.415604+0000","last_undegraded":"2024-07-04T17:33:30.415604+0000","last_fullsized":"2024-07-04T17:33:30.415604+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:41:18.737857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.439260+0000","last_change":"2024-07-04T17:33:29.452800+0000","last_active":"2024-07-04T17:33:30.439260+0000","last_peered":"2024-07-04T17:33:30.439260+0000","last_clean":"2024-07-04T17:33:30.439260+0000","last_became_active":"2024-07-04T17:33:29.451858+0000","last_became_peered":"2024-07-04T17:33:29.451858+0000","last_unstale":"2024-07-04T17:33:30.439260+0000","last_undegraded":"2024-07-04T17:33:30.439260+0000","last_fullsized":"2024-07-04T17:33:30.439260+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:07:07.479809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.435901+0000","last_change":"2024-07-04T17:33:28.425749+0000","last_active":"2024-07-04T17:33:30.435901+0000","last_peered":"2024-07-04T17:33:30.435901+0000","last_clean":"2024-07-04T17:33:30.435901+0000","last_became_active":"2024-07-04T17:33:28.423428+0000","last_became_peered":"2024-07-04T17:33:28.423428+0000","last_unstale":"2024-07-04T17:33:30.435901+0000","last_undegraded":"2024-07-04T17:33:30.435901+0000","last_fullsized":"2024-07-04T17:33:30.435901+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:10:05.668947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.436962+0000","last_change":"2024-07-04T17:33:09.129855+0000","last_active":"2024-07-04T17:33:30.436962+0000","last_peered":"2024-07-04T17:33:30.436962+0000","last_clean":"2024-07-04T17:33:30.436962+0000","last_became_active":"2024-07-04T17:33:09.127858+0000","last_became_peered":"2024-07-04T17:33:09.127858+0000","last_unstale":"2024-07-04T17:33:30.436962+0000","last_undegraded":"2024-07-04T17:33:30.436962+0000","last_fullsized":"2024-07-04T17:33:30.436962+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:35:51.815415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"114'8","reported_seq":30,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.446761+0000","last_change":"2024-07-04T17:33:08.105018+0000","last_active":"2024-07-04T17:33:30.446761+0000","last_peered":"2024-07-04T17:33:30.446761+0000","last_clean":"2024-07-04T17:33:30.446761+0000","last_became_active":"2024-07-04T17:33:08.104685+0000","last_became_peered":"2024-07-04T17:33:08.104685+0000","last_unstale":"2024-07-04T17:33:30.446761+0000","last_undegraded":"2024-07-04T17:33:30.446761+0000","last_fullsized":"2024-07-04T17:33:30.446761+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:13:39.096736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3309,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.412236+0000","last_change":"2024-07-04T17:33:29.435558+0000","last_active":"2024-07-04T17:33:30.412236+0000","last_peered":"2024-07-04T17:33:30.412236+0000","last_clean":"2024-07-04T17:33:30.412236+0000","last_became_active":"2024-07-04T17:33:29.432192+0000","last_became_peered":"2024-07-04T17:33:29.432192+0000","last_unstale":"2024-07-04T17:33:30.412236+0000","last_undegraded":"2024-07-04T17:33:30.412236+0000","last_fullsized":"2024-07-04T17:33:30.412236+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:10:53.801654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.411856+0000","last_change":"2024-07-04T17:33:28.408617+0000","last_active":"2024-07-04T17:33:30.411856+0000","last_peered":"2024-07-04T17:33:30.411856+0000","last_clean":"2024-07-04T17:33:30.411856+0000","last_became_active":"2024-07-04T17:33:28.408267+0000","last_became_peered":"2024-07-04T17:33:28.408267+0000","last_unstale":"2024-07-04T17:33:30.411856+0000","last_undegraded":"2024-07-04T17:33:30.411856+0000","last_fullsized":"2024-07-04T17:33:30.411856+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:47:45.392135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.435092+0000","last_change":"2024-07-04T17:33:09.122352+0000","last_active":"2024-07-04T17:33:30.435092+0000","last_peered":"2024-07-04T17:33:30.435092+0000","last_clean":"2024-07-04T17:33:30.435092+0000","last_became_active":"2024-07-04T17:33:09.121662+0000","last_became_peered":"2024-07-04T17:33:09.121662+0000","last_unstale":"2024-07-04T17:33:30.435092+0000","last_undegraded":"2024-07-04T17:33:30.435092+0000","last_fullsized":"2024-07-04T17:33:30.435092+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:38:27.893548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"113'2","reported_seq":24,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.413633+0000","last_change":"2024-07-04T17:33:08.100243+0000","last_active":"2024-07-04T17:33:30.413633+0000","last_peered":"2024-07-04T17:33:30.413633+0000","last_clean":"2024-07-04T17:33:30.413633+0000","last_became_active":"2024-07-04T17:33:08.099860+0000","last_became_peered":"2024-07-04T17:33:08.099860+0000","last_unstale":"2024-07-04T17:33:30.413633+0000","last_undegraded":"2024-07-04T17:33:30.413633+0000","last_fullsized":"2024-07-04T17:33:30.413633+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:44:16.021572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444462+0000","last_change":"2024-07-04T17:33:29.423569+0000","last_active":"2024-07-04T17:33:30.444462+0000","last_peered":"2024-07-04T17:33:30.444462+0000","last_clean":"2024-07-04T17:33:30.444462+0000","last_became_active":"2024-07-04T17:33:29.423198+0000","last_became_peered":"2024-07-04T17:33:29.423198+0000","last_unstale":"2024-07-04T17:33:30.444462+0000","last_undegraded":"2024-07-04T17:33:30.444462+0000","last_fullsized":"2024-07-04T17:33:30.444462+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:45:53.608714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.10","version":"117'8","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.411671+0000","last_change":"2024-07-04T17:33:28.409675+0000","last_active":"2024-07-04T17:33:30.411671+0000","last_peered":"2024-07-04T17:33:30.411671+0000","last_clean":"2024-07-04T17:33:30.411671+0000","last_became_active":"2024-07-04T17:33:28.408564+0000","last_became_peered":"2024-07-04T17:33:28.408564+0000","last_unstale":"2024-07-04T17:33:30.411671+0000","last_undegraded":"2024-07-04T17:33:30.411671+0000","last_fullsized":"2024-07-04T17:33:30.411671+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:25:43.381033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.413153+0000","last_change":"2024-07-04T17:33:09.124609+0000","last_active":"2024-07-04T17:33:30.413153+0000","last_peered":"2024-07-04T17:33:30.413153+0000","last_clean":"2024-07-04T17:33:30.413153+0000","last_became_active":"2024-07-04T17:33:09.123720+0000","last_became_peered":"2024-07-04T17:33:09.123720+0000","last_unstale":"2024-07-04T17:33:30.413153+0000","last_undegraded":"2024-07-04T17:33:30.413153+0000","last_fullsized":"2024-07-04T17:33:30.413153+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:59:00.559158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.446072+0000","last_change":"2024-07-04T17:33:08.092450+0000","last_active":"2024-07-04T17:33:30.446072+0000","last_peered":"2024-07-04T17:33:30.446072+0000","last_clean":"2024-07-04T17:33:30.446072+0000","last_became_active":"2024-07-04T17:33:08.092088+0000","last_became_peered":"2024-07-04T17:33:08.092088+0000","last_unstale":"2024-07-04T17:33:30.446072+0000","last_undegraded":"2024-07-04T17:33:30.446072+0000","last_fullsized":"2024-07-04T17:33:30.446072+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:54:46.681088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.434866+0000","last_change":"2024-07-04T17:33:29.463209+0000","last_active":"2024-07-04T17:33:30.434866+0000","last_peered":"2024-07-04T17:33:30.434866+0000","last_clean":"2024-07-04T17:33:30.434866+0000","last_became_active":"2024-07-04T17:33:29.462348+0000","last_became_peered":"2024-07-04T17:33:29.462348+0000","last_unstale":"2024-07-04T17:33:30.434866+0000","last_undegraded":"2024-07-04T17:33:30.434866+0000","last_fullsized":"2024-07-04T17:33:30.434866+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:22:13.635684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.13","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.412020+0000","last_change":"2024-07-04T17:33:28.429010+0000","last_active":"2024-07-04T17:33:30.412020+0000","last_peered":"2024-07-04T17:33:30.412020+0000","last_clean":"2024-07-04T17:33:30.412020+0000","last_became_active":"2024-07-04T17:33:28.428679+0000","last_became_peered":"2024-07-04T17:33:28.428679+0000","last_unstale":"2024-07-04T17:33:30.412020+0000","last_undegraded":"2024-07-04T17:33:30.412020+0000","last_fullsized":"2024-07-04T17:33:30.412020+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:00:02.397558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.433960+0000","last_change":"2024-07-04T17:33:09.101721+0000","last_active":"2024-07-04T17:33:30.433960+0000","last_peered":"2024-07-04T17:33:30.433960+0000","last_clean":"2024-07-04T17:33:30.433960+0000","last_became_active":"2024-07-04T17:33:09.101084+0000","last_became_peered":"2024-07-04T17:33:09.101084+0000","last_unstale":"2024-07-04T17:33:30.433960+0000","last_undegraded":"2024-07-04T17:33:30.433960+0000","last_fullsized":"2024-07-04T17:33:30.433960+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:35:19.200734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.412226+0000","last_change":"2024-07-04T17:33:08.098833+0000","last_active":"2024-07-04T17:33:30.412226+0000","last_peered":"2024-07-04T17:33:30.412226+0000","last_clean":"2024-07-04T17:33:30.412226+0000","last_became_active":"2024-07-04T17:33:08.098492+0000","last_became_peered":"2024-07-04T17:33:08.098492+0000","last_unstale":"2024-07-04T17:33:30.412226+0000","last_undegraded":"2024-07-04T17:33:30.412226+0000","last_fullsized":"2024-07-04T17:33:30.412226+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:40:56.816879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.445646+0000","last_change":"2024-07-04T17:33:29.421732+0000","last_active":"2024-07-04T17:33:30.445646+0000","last_peered":"2024-07-04T17:33:30.445646+0000","last_clean":"2024-07-04T17:33:30.445646+0000","last_became_active":"2024-07-04T17:33:29.420326+0000","last_became_peered":"2024-07-04T17:33:29.420326+0000","last_unstale":"2024-07-04T17:33:30.445646+0000","last_undegraded":"2024-07-04T17:33:30.445646+0000","last_fullsized":"2024-07-04T17:33:30.445646+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:47:40.618326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444637+0000","last_change":"2024-07-04T17:33:28.407378+0000","last_active":"2024-07-04T17:33:30.444637+0000","last_peered":"2024-07-04T17:33:30.444637+0000","last_clean":"2024-07-04T17:33:30.444637+0000","last_became_active":"2024-07-04T17:33:28.406935+0000","last_became_peered":"2024-07-04T17:33:28.406935+0000","last_unstale":"2024-07-04T17:33:30.444637+0000","last_undegraded":"2024-07-04T17:33:30.444637+0000","last_fullsized":"2024-07-04T17:33:30.444637+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:53:02.453078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.434844+0000","last_change":"2024-07-04T17:33:09.131477+0000","last_active":"2024-07-04T17:33:30.434844+0000","last_peered":"2024-07-04T17:33:30.434844+0000","last_clean":"2024-07-04T17:33:30.434844+0000","last_became_active":"2024-07-04T17:33:09.122889+0000","last_became_peered":"2024-07-04T17:33:09.122889+0000","last_unstale":"2024-07-04T17:33:30.434844+0000","last_undegraded":"2024-07-04T17:33:30.434844+0000","last_fullsized":"2024-07-04T17:33:30.434844+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:48:40.827656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"113'8","reported_seq":30,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.434199+0000","last_change":"2024-07-04T17:33:08.094956+0000","last_active":"2024-07-04T17:33:30.434199+0000","last_peered":"2024-07-04T17:33:30.434199+0000","last_clean":"2024-07-04T17:33:30.434199+0000","last_became_active":"2024-07-04T17:33:08.090486+0000","last_became_peered":"2024-07-04T17:33:08.090486+0000","last_unstale":"2024-07-04T17:33:30.434199+0000","last_undegraded":"2024-07-04T17:33:30.434199+0000","last_fullsized":"2024-07-04T17:33:30.434199+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:05:13.908213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.415400+0000","last_change":"2024-07-04T17:33:29.435810+0000","last_active":"2024-07-04T17:33:30.415400+0000","last_peered":"2024-07-04T17:33:30.415400+0000","last_clean":"2024-07-04T17:33:30.415400+0000","last_became_active":"2024-07-04T17:33:29.434719+0000","last_became_peered":"2024-07-04T17:33:29.434719+0000","last_unstale":"2024-07-04T17:33:30.415400+0000","last_undegraded":"2024-07-04T17:33:30.415400+0000","last_fullsized":"2024-07-04T17:33:30.415400+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:53:04.956029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.414354+0000","last_change":"2024-07-04T17:33:28.422613+0000","last_active":"2024-07-04T17:33:30.414354+0000","last_peered":"2024-07-04T17:33:30.414354+0000","last_clean":"2024-07-04T17:33:30.414354+0000","last_became_active":"2024-07-04T17:33:28.422278+0000","last_became_peered":"2024-07-04T17:33:28.422278+0000","last_unstale":"2024-07-04T17:33:30.414354+0000","last_undegraded":"2024-07-04T17:33:30.414354+0000","last_fullsized":"2024-07-04T17:33:30.414354+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:11:29.473901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.415583+0000","last_change":"2024-07-04T17:33:09.128035+0000","last_active":"2024-07-04T17:33:30.415583+0000","last_peered":"2024-07-04T17:33:30.415583+0000","last_clean":"2024-07-04T17:33:30.415583+0000","last_became_active":"2024-07-04T17:33:09.125335+0000","last_became_peered":"2024-07-04T17:33:09.125335+0000","last_unstale":"2024-07-04T17:33:30.415583+0000","last_undegraded":"2024-07-04T17:33:30.415583+0000","last_fullsized":"2024-07-04T17:33:30.415583+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:35:53.125670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"113'1","reported_seq":23,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.439669+0000","last_change":"2024-07-04T17:33:08.096006+0000","last_active":"2024-07-04T17:33:30.439669+0000","last_peered":"2024-07-04T17:33:30.439669+0000","last_clean":"2024-07-04T17:33:30.439669+0000","last_became_active":"2024-07-04T17:33:08.095685+0000","last_became_peered":"2024-07-04T17:33:08.095685+0000","last_unstale":"2024-07-04T17:33:30.439669+0000","last_undegraded":"2024-07-04T17:33:30.439669+0000","last_fullsized":"2024-07-04T17:33:30.439669+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:58:57.013662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.447786+0000","last_change":"2024-07-04T17:33:29.428235+0000","last_active":"2024-07-04T17:33:30.447786+0000","last_peered":"2024-07-04T17:33:30.447786+0000","last_clean":"2024-07-04T17:33:30.447786+0000","last_became_active":"2024-07-04T17:33:29.426766+0000","last_became_peered":"2024-07-04T17:33:29.426766+0000","last_unstale":"2024-07-04T17:33:30.447786+0000","last_undegraded":"2024-07-04T17:33:30.447786+0000","last_fullsized":"2024-07-04T17:33:30.447786+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:16:01.565064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.445673+0000","last_change":"2024-07-04T17:33:28.421961+0000","last_active":"2024-07-04T17:33:30.445673+0000","last_peered":"2024-07-04T17:33:30.445673+0000","last_clean":"2024-07-04T17:33:30.445673+0000","last_became_active":"2024-07-04T17:33:28.421617+0000","last_became_peered":"2024-07-04T17:33:28.421617+0000","last_unstale":"2024-07-04T17:33:30.445673+0000","last_undegraded":"2024-07-04T17:33:30.445673+0000","last_fullsized":"2024-07-04T17:33:30.445673+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:34:01.546503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.439398+0000","last_change":"2024-07-04T17:33:09.118421+0000","last_active":"2024-07-04T17:33:30.439398+0000","last_peered":"2024-07-04T17:33:30.439398+0000","last_clean":"2024-07-04T17:33:30.439398+0000","last_became_active":"2024-07-04T17:33:09.117653+0000","last_became_peered":"2024-07-04T17:33:09.117653+0000","last_unstale":"2024-07-04T17:33:30.439398+0000","last_undegraded":"2024-07-04T17:33:30.439398+0000","last_fullsized":"2024-07-04T17:33:30.439398+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:59:43.093156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"113'2","reported_seq":24,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.415677+0000","last_change":"2024-07-04T17:33:08.113131+0000","last_active":"2024-07-04T17:33:30.415677+0000","last_peered":"2024-07-04T17:33:30.415677+0000","last_clean":"2024-07-04T17:33:30.415677+0000","last_became_active":"2024-07-04T17:33:08.112337+0000","last_became_peered":"2024-07-04T17:33:08.112337+0000","last_unstale":"2024-07-04T17:33:30.415677+0000","last_undegraded":"2024-07-04T17:33:30.415677+0000","last_fullsized":"2024-07-04T17:33:30.415677+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:45:29.198114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.414886+0000","last_change":"2024-07-04T17:33:29.447485+0000","last_active":"2024-07-04T17:33:30.414886+0000","last_peered":"2024-07-04T17:33:30.414886+0000","last_clean":"2024-07-04T17:33:30.414886+0000","last_became_active":"2024-07-04T17:33:29.447144+0000","last_became_peered":"2024-07-04T17:33:29.447144+0000","last_unstale":"2024-07-04T17:33:30.414886+0000","last_undegraded":"2024-07-04T17:33:30.414886+0000","last_fullsized":"2024-07-04T17:33:30.414886+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:06:53.353893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.413302+0000","last_change":"2024-07-04T17:33:28.423301+0000","last_active":"2024-07-04T17:33:30.413302+0000","last_peered":"2024-07-04T17:33:30.413302+0000","last_clean":"2024-07-04T17:33:30.413302+0000","last_became_active":"2024-07-04T17:33:28.421659+0000","last_became_peered":"2024-07-04T17:33:28.421659+0000","last_unstale":"2024-07-04T17:33:30.413302+0000","last_undegraded":"2024-07-04T17:33:30.413302+0000","last_fullsized":"2024-07-04T17:33:30.413302+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:41:17.683702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.415940+0000","last_change":"2024-07-04T17:33:09.107000+0000","last_active":"2024-07-04T17:33:30.415940+0000","last_peered":"2024-07-04T17:33:30.415940+0000","last_clean":"2024-07-04T17:33:30.415940+0000","last_became_active":"2024-07-04T17:33:09.106204+0000","last_became_peered":"2024-07-04T17:33:09.106204+0000","last_unstale":"2024-07-04T17:33:30.415940+0000","last_undegraded":"2024-07-04T17:33:30.415940+0000","last_fullsized":"2024-07-04T17:33:30.415940+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:22:24.403011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"113'1","reported_seq":23,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.438327+0000","last_change":"2024-07-04T17:33:08.105392+0000","last_active":"2024-07-04T17:33:30.438327+0000","last_peered":"2024-07-04T17:33:30.438327+0000","last_clean":"2024-07-04T17:33:30.438327+0000","last_became_active":"2024-07-04T17:33:08.105067+0000","last_became_peered":"2024-07-04T17:33:08.105067+0000","last_unstale":"2024-07-04T17:33:30.438327+0000","last_undegraded":"2024-07-04T17:33:30.438327+0000","last_fullsized":"2024-07-04T17:33:30.438327+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:10:49.236238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.438724+0000","last_change":"2024-07-04T17:33:29.449293+0000","last_active":"2024-07-04T17:33:30.438724+0000","last_peered":"2024-07-04T17:33:30.438724+0000","last_clean":"2024-07-04T17:33:30.438724+0000","last_became_active":"2024-07-04T17:33:29.448949+0000","last_became_peered":"2024-07-04T17:33:29.448949+0000","last_unstale":"2024-07-04T17:33:30.438724+0000","last_undegraded":"2024-07-04T17:33:30.438724+0000","last_fullsized":"2024-07-04T17:33:30.438724+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:03:45.183273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.e","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.447730+0000","last_change":"2024-07-04T17:33:28.422730+0000","last_active":"2024-07-04T17:33:30.447730+0000","last_peered":"2024-07-04T17:33:30.447730+0000","last_clean":"2024-07-04T17:33:30.447730+0000","last_became_active":"2024-07-04T17:33:28.421927+0000","last_became_peered":"2024-07-04T17:33:28.421927+0000","last_unstale":"2024-07-04T17:33:30.447730+0000","last_undegraded":"2024-07-04T17:33:30.447730+0000","last_fullsized":"2024-07-04T17:33:30.447730+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:04:56.556554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.414511+0000","last_change":"2024-07-04T17:33:09.126710+0000","last_active":"2024-07-04T17:33:30.414511+0000","last_peered":"2024-07-04T17:33:30.414511+0000","last_clean":"2024-07-04T17:33:30.414511+0000","last_became_active":"2024-07-04T17:33:09.122975+0000","last_became_peered":"2024-07-04T17:33:09.122975+0000","last_unstale":"2024-07-04T17:33:30.414511+0000","last_undegraded":"2024-07-04T17:33:30.414511+0000","last_fullsized":"2024-07-04T17:33:30.414511+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:48:57.200238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.416155+0000","last_change":"2024-07-04T17:33:08.114916+0000","last_active":"2024-07-04T17:33:30.416155+0000","last_peered":"2024-07-04T17:33:30.416155+0000","last_clean":"2024-07-04T17:33:30.416155+0000","last_became_active":"2024-07-04T17:33:08.114455+0000","last_became_peered":"2024-07-04T17:33:08.114455+0000","last_unstale":"2024-07-04T17:33:30.416155+0000","last_undegraded":"2024-07-04T17:33:30.416155+0000","last_fullsized":"2024-07-04T17:33:30.416155+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:09:08.764721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.433630+0000","last_change":"2024-07-04T17:33:29.441891+0000","last_active":"2024-07-04T17:33:30.433630+0000","last_peered":"2024-07-04T17:33:30.433630+0000","last_clean":"2024-07-04T17:33:30.433630+0000","last_became_active":"2024-07-04T17:33:29.441260+0000","last_became_peered":"2024-07-04T17:33:29.441260+0000","last_unstale":"2024-07-04T17:33:30.433630+0000","last_undegraded":"2024-07-04T17:33:30.433630+0000","last_fullsized":"2024-07-04T17:33:30.433630+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:05:38.199674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.411418+0000","last_change":"2024-07-04T17:33:28.432124+0000","last_active":"2024-07-04T17:33:30.411418+0000","last_peered":"2024-07-04T17:33:30.411418+0000","last_clean":"2024-07-04T17:33:30.411418+0000","last_became_active":"2024-07-04T17:33:28.428245+0000","last_became_peered":"2024-07-04T17:33:28.428245+0000","last_unstale":"2024-07-04T17:33:30.411418+0000","last_undegraded":"2024-07-04T17:33:30.411418+0000","last_fullsized":"2024-07-04T17:33:30.411418+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:05:52.733963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.445909+0000","last_change":"2024-07-04T17:33:09.105710+0000","last_active":"2024-07-04T17:33:30.445909+0000","last_peered":"2024-07-04T17:33:30.445909+0000","last_clean":"2024-07-04T17:33:30.445909+0000","last_became_active":"2024-07-04T17:33:09.101819+0000","last_became_peered":"2024-07-04T17:33:09.101819+0000","last_unstale":"2024-07-04T17:33:30.445909+0000","last_undegraded":"2024-07-04T17:33:30.445909+0000","last_fullsized":"2024-07-04T17:33:30.445909+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:47:15.892916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"113'2","reported_seq":24,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.446875+0000","last_change":"2024-07-04T17:33:08.108404+0000","last_active":"2024-07-04T17:33:30.446875+0000","last_peered":"2024-07-04T17:33:30.446875+0000","last_clean":"2024-07-04T17:33:30.446875+0000","last_became_active":"2024-07-04T17:33:08.107387+0000","last_became_peered":"2024-07-04T17:33:08.107387+0000","last_unstale":"2024-07-04T17:33:30.446875+0000","last_undegraded":"2024-07-04T17:33:30.446875+0000","last_fullsized":"2024-07-04T17:33:30.446875+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:10:32.480743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.433670+0000","last_change":"2024-07-04T17:33:29.454218+0000","last_active":"2024-07-04T17:33:30.433670+0000","last_peered":"2024-07-04T17:33:30.433670+0000","last_clean":"2024-07-04T17:33:30.433670+0000","last_became_active":"2024-07-04T17:33:29.453913+0000","last_became_peered":"2024-07-04T17:33:29.453913+0000","last_unstale":"2024-07-04T17:33:30.433670+0000","last_undegraded":"2024-07-04T17:33:30.433670+0000","last_fullsized":"2024-07-04T17:33:30.433670+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:22:24.901243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.445804+0000","last_change":"2024-07-04T17:33:28.410838+0000","last_active":"2024-07-04T17:33:30.445804+0000","last_peered":"2024-07-04T17:33:30.445804+0000","last_clean":"2024-07-04T17:33:30.445804+0000","last_became_active":"2024-07-04T17:33:28.410528+0000","last_became_peered":"2024-07-04T17:33:28.410528+0000","last_unstale":"2024-07-04T17:33:30.445804+0000","last_undegraded":"2024-07-04T17:33:30.445804+0000","last_fullsized":"2024-07-04T17:33:30.445804+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:10:40.346118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.446672+0000","last_change":"2024-07-04T17:33:09.102491+0000","last_active":"2024-07-04T17:33:30.446672+0000","last_peered":"2024-07-04T17:33:30.446672+0000","last_clean":"2024-07-04T17:33:30.446672+0000","last_became_active":"2024-07-04T17:33:09.102170+0000","last_became_peered":"2024-07-04T17:33:09.102170+0000","last_unstale":"2024-07-04T17:33:30.446672+0000","last_undegraded":"2024-07-04T17:33:30.446672+0000","last_fullsized":"2024-07-04T17:33:30.446672+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:03:31.387343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.435546+0000","last_change":"2024-07-04T17:33:08.109294+0000","last_active":"2024-07-04T17:33:30.435546+0000","last_peered":"2024-07-04T17:33:30.435546+0000","last_clean":"2024-07-04T17:33:30.435546+0000","last_became_active":"2024-07-04T17:33:08.108944+0000","last_became_peered":"2024-07-04T17:33:08.108944+0000","last_unstale":"2024-07-04T17:33:30.435546+0000","last_undegraded":"2024-07-04T17:33:30.435546+0000","last_fullsized":"2024-07-04T17:33:30.435546+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:15:36.989862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.434321+0000","last_change":"2024-07-04T17:33:29.452093+0000","last_active":"2024-07-04T17:33:30.434321+0000","last_peered":"2024-07-04T17:33:30.434321+0000","last_clean":"2024-07-04T17:33:30.434321+0000","last_became_active":"2024-07-04T17:33:29.451428+0000","last_became_peered":"2024-07-04T17:33:29.451428+0000","last_unstale":"2024-07-04T17:33:30.434321+0000","last_undegraded":"2024-07-04T17:33:30.434321+0000","last_fullsized":"2024-07-04T17:33:30.434321+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:43:03.637268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.b","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.412206+0000","last_change":"2024-07-04T17:33:28.428048+0000","last_active":"2024-07-04T17:33:30.412206+0000","last_peered":"2024-07-04T17:33:30.412206+0000","last_clean":"2024-07-04T17:33:30.412206+0000","last_became_active":"2024-07-04T17:33:28.427693+0000","last_became_peered":"2024-07-04T17:33:28.427693+0000","last_unstale":"2024-07-04T17:33:30.412206+0000","last_undegraded":"2024-07-04T17:33:30.412206+0000","last_fullsized":"2024-07-04T17:33:30.412206+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:38:54.848517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.445206+0000","last_change":"2024-07-04T17:33:09.111629+0000","last_active":"2024-07-04T17:33:30.445206+0000","last_peered":"2024-07-04T17:33:30.445206+0000","last_clean":"2024-07-04T17:33:30.445206+0000","last_became_active":"2024-07-04T17:33:09.111332+0000","last_became_peered":"2024-07-04T17:33:09.111332+0000","last_unstale":"2024-07-04T17:33:30.445206+0000","last_undegraded":"2024-07-04T17:33:30.445206+0000","last_fullsized":"2024-07-04T17:33:30.445206+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:44:43.830281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.434965+0000","last_change":"2024-07-04T17:33:08.097164+0000","last_active":"2024-07-04T17:33:30.434965+0000","last_peered":"2024-07-04T17:33:30.434965+0000","last_clean":"2024-07-04T17:33:30.434965+0000","last_became_active":"2024-07-04T17:33:08.096841+0000","last_became_peered":"2024-07-04T17:33:08.096841+0000","last_unstale":"2024-07-04T17:33:30.434965+0000","last_undegraded":"2024-07-04T17:33:30.434965+0000","last_fullsized":"2024-07-04T17:33:30.434965+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:24:50.491277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.445807+0000","last_change":"2024-07-04T17:33:29.423657+0000","last_active":"2024-07-04T17:33:30.445807+0000","last_peered":"2024-07-04T17:33:30.445807+0000","last_clean":"2024-07-04T17:33:30.445807+0000","last_became_active":"2024-07-04T17:33:29.419856+0000","last_became_peered":"2024-07-04T17:33:29.419856+0000","last_unstale":"2024-07-04T17:33:30.445807+0000","last_undegraded":"2024-07-04T17:33:30.445807+0000","last_fullsized":"2024-07-04T17:33:30.445807+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:00:59.531635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444735+0000","last_change":"2024-07-04T17:33:28.406714+0000","last_active":"2024-07-04T17:33:30.444735+0000","last_peered":"2024-07-04T17:33:30.444735+0000","last_clean":"2024-07-04T17:33:30.444735+0000","last_became_active":"2024-07-04T17:33:28.406342+0000","last_became_peered":"2024-07-04T17:33:28.406342+0000","last_unstale":"2024-07-04T17:33:30.444735+0000","last_undegraded":"2024-07-04T17:33:30.444735+0000","last_fullsized":"2024-07-04T17:33:30.444735+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:42:18.264655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444996+0000","last_change":"2024-07-04T17:33:09.097586+0000","last_active":"2024-07-04T17:33:30.444996+0000","last_peered":"2024-07-04T17:33:30.444996+0000","last_clean":"2024-07-04T17:33:30.444996+0000","last_became_active":"2024-07-04T17:33:09.096586+0000","last_became_peered":"2024-07-04T17:33:09.096586+0000","last_unstale":"2024-07-04T17:33:30.444996+0000","last_undegraded":"2024-07-04T17:33:30.444996+0000","last_fullsized":"2024-07-04T17:33:30.444996+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:21:53.318774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.434416+0000","last_change":"2024-07-04T17:33:08.094584+0000","last_active":"2024-07-04T17:33:30.434416+0000","last_peered":"2024-07-04T17:33:30.434416+0000","last_clean":"2024-07-04T17:33:30.434416+0000","last_became_active":"2024-07-04T17:33:08.090067+0000","last_became_peered":"2024-07-04T17:33:08.090067+0000","last_unstale":"2024-07-04T17:33:30.434416+0000","last_undegraded":"2024-07-04T17:33:30.434416+0000","last_fullsized":"2024-07-04T17:33:30.434416+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:17:57.214638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.436090+0000","last_change":"2024-07-04T17:33:29.463129+0000","last_active":"2024-07-04T17:33:30.436090+0000","last_peered":"2024-07-04T17:33:30.436090+0000","last_clean":"2024-07-04T17:33:30.436090+0000","last_became_active":"2024-07-04T17:33:29.462631+0000","last_became_peered":"2024-07-04T17:33:29.462631+0000","last_unstale":"2024-07-04T17:33:30.436090+0000","last_undegraded":"2024-07-04T17:33:30.436090+0000","last_fullsized":"2024-07-04T17:33:30.436090+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:03:51.704253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.436434+0000","last_change":"2024-07-04T17:33:28.430344+0000","last_active":"2024-07-04T17:33:30.436434+0000","last_peered":"2024-07-04T17:33:30.436434+0000","last_clean":"2024-07-04T17:33:30.436434+0000","last_became_active":"2024-07-04T17:33:28.429590+0000","last_became_peered":"2024-07-04T17:33:28.429590+0000","last_unstale":"2024-07-04T17:33:30.436434+0000","last_undegraded":"2024-07-04T17:33:30.436434+0000","last_fullsized":"2024-07-04T17:33:30.436434+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:05:09.258795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.438904+0000","last_change":"2024-07-04T17:33:09.131493+0000","last_active":"2024-07-04T17:33:30.438904+0000","last_peered":"2024-07-04T17:33:30.438904+0000","last_clean":"2024-07-04T17:33:30.438904+0000","last_became_active":"2024-07-04T17:33:09.131132+0000","last_became_peered":"2024-07-04T17:33:09.131132+0000","last_unstale":"2024-07-04T17:33:30.438904+0000","last_undegraded":"2024-07-04T17:33:30.438904+0000","last_fullsized":"2024-07-04T17:33:30.438904+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:56:22.629300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"113'2","reported_seq":24,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.439860+0000","last_change":"2024-07-04T17:33:08.105850+0000","last_active":"2024-07-04T17:33:30.439860+0000","last_peered":"2024-07-04T17:33:30.439860+0000","last_clean":"2024-07-04T17:33:30.439860+0000","last_became_active":"2024-07-04T17:33:08.104987+0000","last_became_peered":"2024-07-04T17:33:08.104987+0000","last_unstale":"2024-07-04T17:33:30.439860+0000","last_undegraded":"2024-07-04T17:33:30.439860+0000","last_fullsized":"2024-07-04T17:33:30.439860+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:36:17.000989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.413823+0000","last_change":"2024-07-04T17:33:29.460185+0000","last_active":"2024-07-04T17:33:30.413823+0000","last_peered":"2024-07-04T17:33:30.413823+0000","last_clean":"2024-07-04T17:33:30.413823+0000","last_became_active":"2024-07-04T17:33:29.459735+0000","last_became_peered":"2024-07-04T17:33:29.459735+0000","last_unstale":"2024-07-04T17:33:30.413823+0000","last_undegraded":"2024-07-04T17:33:30.413823+0000","last_fullsized":"2024-07-04T17:33:30.413823+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:21:36.261030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.436156+0000","last_change":"2024-07-04T17:33:28.418849+0000","last_active":"2024-07-04T17:33:30.436156+0000","last_peered":"2024-07-04T17:33:30.436156+0000","last_clean":"2024-07-04T17:33:30.436156+0000","last_became_active":"2024-07-04T17:33:28.418515+0000","last_became_peered":"2024-07-04T17:33:28.418515+0000","last_unstale":"2024-07-04T17:33:30.436156+0000","last_undegraded":"2024-07-04T17:33:30.436156+0000","last_fullsized":"2024-07-04T17:33:30.436156+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:10:52.789145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.439540+0000","last_change":"2024-07-04T17:33:09.118693+0000","last_active":"2024-07-04T17:33:30.439540+0000","last_peered":"2024-07-04T17:33:30.439540+0000","last_clean":"2024-07-04T17:33:30.439540+0000","last_became_active":"2024-07-04T17:33:09.117928+0000","last_became_peered":"2024-07-04T17:33:09.117928+0000","last_unstale":"2024-07-04T17:33:30.439540+0000","last_undegraded":"2024-07-04T17:33:30.439540+0000","last_fullsized":"2024-07-04T17:33:30.439540+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:38:35.544115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"114'1","reported_seq":23,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.415736+0000","last_change":"2024-07-04T17:33:08.116106+0000","last_active":"2024-07-04T17:33:30.415736+0000","last_peered":"2024-07-04T17:33:30.415736+0000","last_clean":"2024-07-04T17:33:30.415736+0000","last_became_active":"2024-07-04T17:33:08.115757+0000","last_became_peered":"2024-07-04T17:33:08.115757+0000","last_unstale":"2024-07-04T17:33:30.415736+0000","last_undegraded":"2024-07-04T17:33:30.415736+0000","last_fullsized":"2024-07-04T17:33:30.415736+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:16:30.420529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.447628+0000","last_change":"2024-07-04T17:33:29.444582+0000","last_active":"2024-07-04T17:33:30.447628+0000","last_peered":"2024-07-04T17:33:30.447628+0000","last_clean":"2024-07-04T17:33:30.447628+0000","last_became_active":"2024-07-04T17:33:29.440219+0000","last_became_peered":"2024-07-04T17:33:29.440219+0000","last_unstale":"2024-07-04T17:33:30.447628+0000","last_undegraded":"2024-07-04T17:33:30.447628+0000","last_fullsized":"2024-07-04T17:33:30.447628+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:07:12.694107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.7","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.438694+0000","last_change":"2024-07-04T17:33:28.435061+0000","last_active":"2024-07-04T17:33:30.438694+0000","last_peered":"2024-07-04T17:33:30.438694+0000","last_clean":"2024-07-04T17:33:30.438694+0000","last_became_active":"2024-07-04T17:33:28.434639+0000","last_became_peered":"2024-07-04T17:33:28.434639+0000","last_unstale":"2024-07-04T17:33:30.438694+0000","last_undegraded":"2024-07-04T17:33:30.438694+0000","last_fullsized":"2024-07-04T17:33:30.438694+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:44:17.694191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.416011+0000","last_change":"2024-07-04T17:33:09.098889+0000","last_active":"2024-07-04T17:33:30.416011+0000","last_peered":"2024-07-04T17:33:30.416011+0000","last_clean":"2024-07-04T17:33:30.416011+0000","last_became_active":"2024-07-04T17:33:09.098564+0000","last_became_peered":"2024-07-04T17:33:09.098564+0000","last_unstale":"2024-07-04T17:33:30.416011+0000","last_undegraded":"2024-07-04T17:33:30.416011+0000","last_fullsized":"2024-07-04T17:33:30.416011+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:06:27.253525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.415057+0000","last_change":"2024-07-04T17:33:08.114022+0000","last_active":"2024-07-04T17:33:30.415057+0000","last_peered":"2024-07-04T17:33:30.415057+0000","last_clean":"2024-07-04T17:33:30.415057+0000","last_became_active":"2024-07-04T17:33:08.112033+0000","last_became_peered":"2024-07-04T17:33:08.112033+0000","last_unstale":"2024-07-04T17:33:30.415057+0000","last_undegraded":"2024-07-04T17:33:30.415057+0000","last_fullsized":"2024-07-04T17:33:30.415057+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:00:33.769816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.447448+0000","last_change":"2024-07-04T17:33:29.444970+0000","last_active":"2024-07-04T17:33:30.447448+0000","last_peered":"2024-07-04T17:33:30.447448+0000","last_clean":"2024-07-04T17:33:30.447448+0000","last_became_active":"2024-07-04T17:33:29.439990+0000","last_became_peered":"2024-07-04T17:33:29.439990+0000","last_unstale":"2024-07-04T17:33:30.447448+0000","last_undegraded":"2024-07-04T17:33:30.447448+0000","last_fullsized":"2024-07-04T17:33:30.447448+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:11:57.484633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"118'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.436382+0000","last_change":"2024-07-04T17:33:28.435388+0000","last_active":"2024-07-04T17:33:30.436382+0000","last_peered":"2024-07-04T17:33:30.436382+0000","last_clean":"2024-07-04T17:33:30.436382+0000","last_became_active":"2024-07-04T17:33:28.434662+0000","last_became_peered":"2024-07-04T17:33:28.434662+0000","last_unstale":"2024-07-04T17:33:30.436382+0000","last_undegraded":"2024-07-04T17:33:30.436382+0000","last_fullsized":"2024-07-04T17:33:30.436382+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:06:37.633452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.414668+0000","last_change":"2024-07-04T17:33:09.113463+0000","last_active":"2024-07-04T17:33:30.414668+0000","last_peered":"2024-07-04T17:33:30.414668+0000","last_clean":"2024-07-04T17:33:30.414668+0000","last_became_active":"2024-07-04T17:33:09.112826+0000","last_became_peered":"2024-07-04T17:33:09.112826+0000","last_unstale":"2024-07-04T17:33:30.414668+0000","last_undegraded":"2024-07-04T17:33:30.414668+0000","last_fullsized":"2024-07-04T17:33:30.414668+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:26:59.083650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.416154+0000","last_change":"2024-07-04T17:33:08.114768+0000","last_active":"2024-07-04T17:33:30.416154+0000","last_peered":"2024-07-04T17:33:30.416154+0000","last_clean":"2024-07-04T17:33:30.416154+0000","last_became_active":"2024-07-04T17:33:08.113902+0000","last_became_peered":"2024-07-04T17:33:08.113902+0000","last_unstale":"2024-07-04T17:33:30.416154+0000","last_undegraded":"2024-07-04T17:33:30.416154+0000","last_fullsized":"2024-07-04T17:33:30.416154+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:58:34.807900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444713+0000","last_change":"2024-07-04T17:33:29.412043+0000","last_active":"2024-07-04T17:33:30.444713+0000","last_peered":"2024-07-04T17:33:30.444713+0000","last_clean":"2024-07-04T17:33:30.444713+0000","last_became_active":"2024-07-04T17:33:29.411648+0000","last_became_peered":"2024-07-04T17:33:29.411648+0000","last_unstale":"2024-07-04T17:33:30.444713+0000","last_undegraded":"2024-07-04T17:33:30.444713+0000","last_fullsized":"2024-07-04T17:33:30.444713+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:27:12.466261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.433740+0000","last_change":"2024-07-04T17:33:28.424058+0000","last_active":"2024-07-04T17:33:30.433740+0000","last_peered":"2024-07-04T17:33:30.433740+0000","last_clean":"2024-07-04T17:33:30.433740+0000","last_became_active":"2024-07-04T17:33:28.423741+0000","last_became_peered":"2024-07-04T17:33:28.423741+0000","last_unstale":"2024-07-04T17:33:30.433740+0000","last_undegraded":"2024-07-04T17:33:30.433740+0000","last_fullsized":"2024-07-04T17:33:30.433740+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:41:17.683702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.435414+0000","last_change":"2024-07-04T17:33:09.128723+0000","last_active":"2024-07-04T17:33:30.435414+0000","last_peered":"2024-07-04T17:33:30.435414+0000","last_clean":"2024-07-04T17:33:30.435414+0000","last_became_active":"2024-07-04T17:33:09.128217+0000","last_became_peered":"2024-07-04T17:33:09.128217+0000","last_unstale":"2024-07-04T17:33:30.435414+0000","last_undegraded":"2024-07-04T17:33:30.435414+0000","last_fullsized":"2024-07-04T17:33:30.435414+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:08:28.724747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.447061+0000","last_change":"2024-07-04T17:33:08.108382+0000","last_active":"2024-07-04T17:33:30.447061+0000","last_peered":"2024-07-04T17:33:30.447061+0000","last_clean":"2024-07-04T17:33:30.447061+0000","last_became_active":"2024-07-04T17:33:08.107656+0000","last_became_peered":"2024-07-04T17:33:08.107656+0000","last_unstale":"2024-07-04T17:33:30.447061+0000","last_undegraded":"2024-07-04T17:33:30.447061+0000","last_fullsized":"2024-07-04T17:33:30.447061+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:55:46.289096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.435127+0000","last_change":"2024-07-04T17:33:29.451047+0000","last_active":"2024-07-04T17:33:30.435127+0000","last_peered":"2024-07-04T17:33:30.435127+0000","last_clean":"2024-07-04T17:33:30.435127+0000","last_became_active":"2024-07-04T17:33:29.450715+0000","last_became_peered":"2024-07-04T17:33:29.450715+0000","last_unstale":"2024-07-04T17:33:30.435127+0000","last_undegraded":"2024-07-04T17:33:30.435127+0000","last_fullsized":"2024-07-04T17:33:30.435127+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:59:59.422659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.433434+0000","last_change":"2024-07-04T17:33:28.417708+0000","last_active":"2024-07-04T17:33:30.433434+0000","last_peered":"2024-07-04T17:33:30.433434+0000","last_clean":"2024-07-04T17:33:30.433434+0000","last_became_active":"2024-07-04T17:33:28.417409+0000","last_became_peered":"2024-07-04T17:33:28.417409+0000","last_unstale":"2024-07-04T17:33:30.433434+0000","last_undegraded":"2024-07-04T17:33:30.433434+0000","last_fullsized":"2024-07-04T17:33:30.433434+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:45:26.883263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.436976+0000","last_change":"2024-07-04T17:33:09.129070+0000","last_active":"2024-07-04T17:33:30.436976+0000","last_peered":"2024-07-04T17:33:30.436976+0000","last_clean":"2024-07-04T17:33:30.436976+0000","last_became_active":"2024-07-04T17:33:09.128751+0000","last_became_peered":"2024-07-04T17:33:09.128751+0000","last_unstale":"2024-07-04T17:33:30.436976+0000","last_undegraded":"2024-07-04T17:33:30.436976+0000","last_fullsized":"2024-07-04T17:33:30.436976+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:39:44.077373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.446237+0000","last_change":"2024-07-04T17:33:08.096375+0000","last_active":"2024-07-04T17:33:30.446237+0000","last_peered":"2024-07-04T17:33:30.446237+0000","last_clean":"2024-07-04T17:33:30.446237+0000","last_became_active":"2024-07-04T17:33:08.096060+0000","last_became_peered":"2024-07-04T17:33:08.096060+0000","last_unstale":"2024-07-04T17:33:30.446237+0000","last_undegraded":"2024-07-04T17:33:30.446237+0000","last_fullsized":"2024-07-04T17:33:30.446237+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:08:19.414085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.444566+0000","last_change":"2024-07-04T17:33:29.429275+0000","last_active":"2024-07-04T17:33:30.444566+0000","last_peered":"2024-07-04T17:33:30.444566+0000","last_clean":"2024-07-04T17:33:30.444566+0000","last_became_active":"2024-07-04T17:33:29.424470+0000","last_became_peered":"2024-07-04T17:33:29.424470+0000","last_unstale":"2024-07-04T17:33:30.444566+0000","last_undegraded":"2024-07-04T17:33:30.444566+0000","last_fullsized":"2024-07-04T17:33:30.444566+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:36:56.447125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.412239+0000","last_change":"2024-07-04T17:33:28.430003+0000","last_active":"2024-07-04T17:33:30.412239+0000","last_peered":"2024-07-04T17:33:30.412239+0000","last_clean":"2024-07-04T17:33:30.412239+0000","last_became_active":"2024-07-04T17:33:28.429687+0000","last_became_peered":"2024-07-04T17:33:28.429687+0000","last_unstale":"2024-07-04T17:33:30.412239+0000","last_undegraded":"2024-07-04T17:33:30.412239+0000","last_fullsized":"2024-07-04T17:33:30.412239+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:12:14.054912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.434768+0000","last_change":"2024-07-04T17:33:09.107349+0000","last_active":"2024-07-04T17:33:30.434768+0000","last_peered":"2024-07-04T17:33:30.434768+0000","last_clean":"2024-07-04T17:33:30.434768+0000","last_became_active":"2024-07-04T17:33:09.104954+0000","last_became_peered":"2024-07-04T17:33:09.104954+0000","last_unstale":"2024-07-04T17:33:30.434768+0000","last_undegraded":"2024-07-04T17:33:30.434768+0000","last_fullsized":"2024-07-04T17:33:30.434768+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:19:28.456336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.446338+0000","last_change":"2024-07-04T17:33:08.106155+0000","last_active":"2024-07-04T17:33:30.446338+0000","last_peered":"2024-07-04T17:33:30.446338+0000","last_clean":"2024-07-04T17:33:30.446338+0000","last_became_active":"2024-07-04T17:33:08.104825+0000","last_became_peered":"2024-07-04T17:33:08.104825+0000","last_unstale":"2024-07-04T17:33:30.446338+0000","last_undegraded":"2024-07-04T17:33:30.446338+0000","last_fullsized":"2024-07-04T17:33:30.446338+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:15:59.977728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.435036+0000","last_change":"2024-07-04T17:33:29.450723+0000","last_active":"2024-07-04T17:33:30.435036+0000","last_peered":"2024-07-04T17:33:30.435036+0000","last_clean":"2024-07-04T17:33:30.435036+0000","last_became_active":"2024-07-04T17:33:29.449529+0000","last_became_peered":"2024-07-04T17:33:29.449529+0000","last_unstale":"2024-07-04T17:33:30.435036+0000","last_undegraded":"2024-07-04T17:33:30.435036+0000","last_fullsized":"2024-07-04T17:33:30.435036+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:25:56.571866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.412435+0000","last_change":"2024-07-04T17:33:28.429840+0000","last_active":"2024-07-04T17:33:30.412435+0000","last_peered":"2024-07-04T17:33:30.412435+0000","last_clean":"2024-07-04T17:33:30.412435+0000","last_became_active":"2024-07-04T17:33:28.429539+0000","last_became_peered":"2024-07-04T17:33:28.429539+0000","last_unstale":"2024-07-04T17:33:30.412435+0000","last_undegraded":"2024-07-04T17:33:30.412435+0000","last_fullsized":"2024-07-04T17:33:30.412435+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:47:35.483539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.445166+0000","last_change":"2024-07-04T17:33:09.107749+0000","last_active":"2024-07-04T17:33:30.445166+0000","last_peered":"2024-07-04T17:33:30.445166+0000","last_clean":"2024-07-04T17:33:30.445166+0000","last_became_active":"2024-07-04T17:33:09.106400+0000","last_became_peered":"2024-07-04T17:33:09.106400+0000","last_unstale":"2024-07-04T17:33:30.445166+0000","last_undegraded":"2024-07-04T17:33:30.445166+0000","last_fullsized":"2024-07-04T17:33:30.445166+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:22:04.584333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.434601+0000","last_change":"2024-07-04T17:33:08.109770+0000","last_active":"2024-07-04T17:33:30.434601+0000","last_peered":"2024-07-04T17:33:30.434601+0000","last_clean":"2024-07-04T17:33:30.434601+0000","last_became_active":"2024-07-04T17:33:08.108245+0000","last_became_peered":"2024-07-04T17:33:08.108245+0000","last_unstale":"2024-07-04T17:33:30.434601+0000","last_undegraded":"2024-07-04T17:33:30.434601+0000","last_fullsized":"2024-07-04T17:33:30.434601+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:30:01.785284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.438092+0000","last_change":"2024-07-04T17:33:29.452049+0000","last_active":"2024-07-04T17:33:30.438092+0000","last_peered":"2024-07-04T17:33:30.438092+0000","last_clean":"2024-07-04T17:33:30.438092+0000","last_became_active":"2024-07-04T17:33:29.451712+0000","last_became_peered":"2024-07-04T17:33:29.451712+0000","last_unstale":"2024-07-04T17:33:30.438092+0000","last_undegraded":"2024-07-04T17:33:30.438092+0000","last_fullsized":"2024-07-04T17:33:30.438092+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:28.364040+0000","last_clean_scrub_stamp":"2024-07-04T17:33:28.364040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:28:10.840046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.413257+0000","last_change":"2024-07-04T17:33:28.422561+0000","last_active":"2024-07-04T17:33:30.413257+0000","last_peered":"2024-07-04T17:33:30.413257+0000","last_clean":"2024-07-04T17:33:30.413257+0000","last_became_active":"2024-07-04T17:33:28.420301+0000","last_became_peered":"2024-07-04T17:33:28.420301+0000","last_unstale":"2024-07-04T17:33:30.413257+0000","last_undegraded":"2024-07-04T17:33:30.413257+0000","last_fullsized":"2024-07-04T17:33:30.413257+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:27.351532+0000","last_clean_scrub_stamp":"2024-07-04T17:33:27.351532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:45:26.883263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.446598+0000","last_change":"2024-07-04T17:33:09.099278+0000","last_active":"2024-07-04T17:33:30.446598+0000","last_peered":"2024-07-04T17:33:30.446598+0000","last_clean":"2024-07-04T17:33:30.446598+0000","last_became_active":"2024-07-04T17:33:09.098752+0000","last_became_peered":"2024-07-04T17:33:09.098752+0000","last_unstale":"2024-07-04T17:33:30.446598+0000","last_undegraded":"2024-07-04T17:33:30.446598+0000","last_fullsized":"2024-07-04T17:33:30.446598+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:08.047130+0000","last_clean_scrub_stamp":"2024-07-04T17:33:08.047130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:34:51.476685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"113'3","reported_seq":25,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.449201+0000","last_change":"2024-07-04T17:33:08.092774+0000","last_active":"2024-07-04T17:33:30.449201+0000","last_peered":"2024-07-04T17:33:30.449201+0000","last_clean":"2024-07-04T17:33:30.449201+0000","last_became_active":"2024-07-04T17:33:08.092404+0000","last_became_peered":"2024-07-04T17:33:08.092404+0000","last_unstale":"2024-07-04T17:33:30.449201+0000","last_undegraded":"2024-07-04T17:33:30.449201+0000","last_fullsized":"2024-07-04T17:33:30.449201+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:33:07.043129+0000","last_clean_scrub_stamp":"2024-07-04T17:33:07.043129+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:48:02.318598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"100'101","reported_seq":456,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-07-04T17:33:30.413183+0000","last_change":"2024-07-04T17:26:04.953559+0000","last_active":"2024-07-04T17:33:30.413183+0000","last_peered":"2024-07-04T17:33:30.413183+0000","last_clean":"2024-07-04T17:33:30.413183+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:33:30.413183+0000","last_undegraded":"2024-07-04T17:33:30.413183+0000","last_fullsized":"2024-07-04T17:33:30.413183+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_clean_scrub_stamp":"2024-07-04T17:26:03.933432+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.064181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":476,"num_read_kb":1278,"num_write":163,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":4107,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":476,"num_read_kb":1278,"num_write":163,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101,"ondisk_log_size":101,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738457,"num_pgs":132,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36568,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335272,"statfs":{"total":96636764160,"available":96599318528,"internally_reserved":0,"allocated":1257472,"data_stored":487662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738458,"num_pgs":126,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37148,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334692,"statfs":{"total":96636764160,"available":96598724608,"internally_reserved":0,"allocated":1847296,"data_stored":1074333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":121,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37144,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334696,"statfs":{"total":96636764160,"available":96598728704,"internally_reserved":0,"allocated":1839104,"data_stored":1075858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:33:34.771 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-04T17:33:34.771 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:33:35.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.102+0000 7f7bca400640 1 -- 172.21.15.42:0/1894911727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7bc406d580 msgr2=0x7f7bc406c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:35.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.102+0000 7f7bca400640 1 --2- 172.21.15.42:0/1894911727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7bc406d580 0x7f7bc406c2c0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4005fd0 tx=0x7f7bb402a820 comp rx=0 tx=0).stop 2024-07-04T17:33:35.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.102+0000 7f7bca400640 1 -- 172.21.15.42:0/1894911727 shutdown_connections 2024-07-04T17:33:35.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.102+0000 7f7bca400640 1 --2- 172.21.15.42:0/1894911727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7bc406d580 0x7f7bc406c2c0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:35.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.102+0000 7f7bca400640 1 -- 172.21.15.42:0/1894911727 >> 172.21.15.42:0/1894911727 conn(0x7f7bc40bebc0 msgr2=0x7f7bc40befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:35.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.102+0000 7f7bca400640 1 -- 172.21.15.42:0/1894911727 shutdown_connections 2024-07-04T17:33:35.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.102+0000 7f7bca400640 1 -- 172.21.15.42:0/1894911727 wait complete. 2024-07-04T17:33:35.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.104+0000 7f7bca400640 1 Processor -- start 2024-07-04T17:33:35.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.105+0000 7f7bca400640 1 -- start start 2024-07-04T17:33:35.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.106+0000 7f7bca400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7bc406d580 0x7f7bc40bde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:35.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.106+0000 7f7bca400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc40b2810 con 0x7f7bc406d580 2024-07-04T17:33:35.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.106+0000 7f7bc9000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7bc406d580 0x7f7bc40bde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:35.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.106+0000 7f7bc9000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7bc406d580 0x7f7bc40bde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40792/0 (socket says 172.21.15.42:40792) 2024-07-04T17:33:35.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.106+0000 7f7bc9000640 1 -- 172.21.15.42:0/976951789 learned_addr learned my addr 172.21.15.42:0/976951789 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:35.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.107+0000 7f7bc9000640 1 -- 172.21.15.42:0/976951789 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc40afd40 con 0x7f7bc406d580 2024-07-04T17:33:35.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.107+0000 7f7bc9000640 1 --2- 172.21.15.42:0/976951789 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7bc406d580 0x7f7bc40bde10 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4009c50 tx=0x7f7bb4032dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:35.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.107+0000 7f7bc0c00640 1 -- 172.21.15.42:0/976951789 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bb4037050 con 0x7f7bc406d580 2024-07-04T17:33:35.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.107+0000 7f7bc0c00640 1 -- 172.21.15.42:0/976951789 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7bb4043070 con 0x7f7bc406d580 2024-07-04T17:33:35.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.108+0000 7f7bca400640 1 -- 172.21.15.42:0/976951789 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bc40afa50 con 0x7f7bc406d580 2024-07-04T17:33:35.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.108+0000 7f7bc0c00640 1 -- 172.21.15.42:0/976951789 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bb40043e0 con 0x7f7bc406d580 2024-07-04T17:33:35.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.108+0000 7f7bca400640 1 -- 172.21.15.42:0/976951789 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bc41dd100 con 0x7f7bc406d580 2024-07-04T17:33:35.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.109+0000 7f7bca400640 1 -- 172.21.15.42:0/976951789 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bc406c2c0 con 0x7f7bc406d580 2024-07-04T17:33:35.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.109+0000 7f7bc0c00640 1 -- 172.21.15.42:0/976951789 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7f7bb4004580 con 0x7f7bc406d580 2024-07-04T17:33:35.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.109+0000 7f7bc0c00640 1 --2- 172.21.15.42:0/976951789 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f7ba403d4f0 0x7f7ba403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:35.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.110+0000 7f7bc0c00640 1 -- 172.21.15.42:0/976951789 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 4303+0+0 (secure 0 0 0) 0x7f7bb402f070 con 0x7f7bc406d580 2024-07-04T17:33:35.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.110+0000 7f7bc3e00640 1 --2- 172.21.15.42:0/976951789 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f7ba403d4f0 0x7f7ba403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:35.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.111+0000 7f7bc3e00640 1 --2- 172.21.15.42:0/976951789 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f7ba403d4f0 0x7f7ba403fa00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0007fd0 tx=0x7f7bb0007920 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:35.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.115+0000 7f7bc0c00640 1 -- 172.21.15.42:0/976951789 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f7bb403bd70 con 0x7f7bc406d580 2024-07-04T17:33:35.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.364+0000 7f7bca400640 1 -- 172.21.15.42:0/976951789 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7bc4075ae0 con 0x7f7bc406d580 2024-07-04T17:33:35.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.366+0000 7f7bc0c00640 1 -- 172.21.15.42:0/976951789 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+7354 (secure 0 0 0) 0x7f7bb403dcd0 con 0x7f7bc406d580 2024-07-04T17:33:35.367 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:33:35.367 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":96,"btime":"2024-07-04T17:33:32:423100+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5536,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/68857121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":68857121},{"type":"v1","addr":"172.21.15.42:6825","nonce":68857121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":5544,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6828/20990456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":20990456},{"type":"v1","addr":"172.21.15.42:6828","nonce":20990456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":5545,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/428501060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6827","nonce":428501060},{"type":"v1","addr":"172.21.15.42:6829","nonce":428501060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:33:08.649551+0000","modified":"2024-07-04T17:33:11.687748+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5550":{"gid":5550,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6833/605786437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":605786437},{"type":"v1","addr":"172.21.15.42:6833","nonce":605786437}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5550,"qdb_cluster":[5550]},"id":10},{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:33:28.944269+0000","modified":"2024-07-04T17:33:32.025418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5549},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5549":{"gid":5549,"name":"b","rank":0,"incarnation":92,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6831/3224527035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":3224527035},{"type":"v1","addr":"172.21.15.42:6831","nonce":3224527035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5549,"qdb_cluster":[5549]},"id":11}]} 2024-07-04T17:33:35.368 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 96 2024-07-04T17:33:35.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.369+0000 7f7bca400640 1 -- 172.21.15.42:0/976951789 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f7ba403d4f0 msgr2=0x7f7ba403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:35.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.369+0000 7f7bca400640 1 --2- 172.21.15.42:0/976951789 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f7ba403d4f0 0x7f7ba403fa00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0007fd0 tx=0x7f7bb0007920 comp rx=0 tx=0).stop 2024-07-04T17:33:35.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.369+0000 7f7bca400640 1 -- 172.21.15.42:0/976951789 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7bc406d580 msgr2=0x7f7bc40bde10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:35.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.369+0000 7f7bca400640 1 --2- 172.21.15.42:0/976951789 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7bc406d580 0x7f7bc40bde10 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4009c50 tx=0x7f7bb4032dc0 comp rx=0 tx=0).stop 2024-07-04T17:33:35.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.369+0000 7f7bca400640 1 -- 172.21.15.42:0/976951789 shutdown_connections 2024-07-04T17:33:35.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.369+0000 7f7bca400640 1 --2- 172.21.15.42:0/976951789 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f7ba403d4f0 0x7f7ba403fa00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:35.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.369+0000 7f7bca400640 1 --2- 172.21.15.42:0/976951789 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7bc406d580 0x7f7bc40bde10 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:35.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.369+0000 7f7bca400640 1 -- 172.21.15.42:0/976951789 >> 172.21.15.42:0/976951789 conn(0x7f7bc40bebc0 msgr2=0x7f7bc40bbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:35.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.369+0000 7f7bca400640 1 -- 172.21.15.42:0/976951789 shutdown_connections 2024-07-04T17:33:35.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:35.369+0000 7f7bca400640 1 -- 172.21.15.42:0/976951789 wait complete. 2024-07-04T17:33:35.897 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 96, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-04T17:33:28.944269+0000', 'modified': '2024-07-04T17:33:32.025418+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5549}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5549': {'gid': 5549, 'name': 'b', 'rank': 0, 'incarnation': 92, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.42:6831/3224527035', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6830', 'nonce': 3224527035}, {'type': 'v1', 'addr': '172.21.15.42:6831', 'nonce': 3224527035}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'backup_fs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5549, 'qdb_cluster': [5549]} 2024-07-04T17:33:35.897 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-04T17:33:35.897 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-07-04T17:33:36.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.194+0000 7f4b24c00640 1 -- 172.21.15.42:0/3375334798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b200ba830 msgr2=0x7f4b200b7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:36.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.194+0000 7f4b24c00640 1 --2- 172.21.15.42:0/3375334798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b200ba830 0x7f4b200b7cc0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f4b080097b0 tx=0x7f4b0802a530 comp rx=0 tx=0).stop 2024-07-04T17:33:36.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.194+0000 7f4b24c00640 1 -- 172.21.15.42:0/3375334798 shutdown_connections 2024-07-04T17:33:36.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.194+0000 7f4b24c00640 1 --2- 172.21.15.42:0/3375334798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b200ba830 0x7f4b200b7cc0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:36.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.194+0000 7f4b24c00640 1 -- 172.21.15.42:0/3375334798 >> 172.21.15.42:0/3375334798 conn(0x7f4b200bebc0 msgr2=0x7f4b200befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:36.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.194+0000 7f4b24c00640 1 -- 172.21.15.42:0/3375334798 shutdown_connections 2024-07-04T17:33:36.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.194+0000 7f4b24c00640 1 -- 172.21.15.42:0/3375334798 wait complete. 2024-07-04T17:33:36.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.196+0000 7f4b24c00640 1 Processor -- start 2024-07-04T17:33:36.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.196+0000 7f4b24c00640 1 -- start start 2024-07-04T17:33:36.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.197+0000 7f4b24c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b200ba830 0x7f4b201d7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:36.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.198+0000 7f4b24c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4b200b1e30 con 0x7f4b200ba830 2024-07-04T17:33:36.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.198+0000 7f4b1f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b200ba830 0x7f4b201d7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:36.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.198+0000 7f4b1f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b200ba830 0x7f4b201d7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40808/0 (socket says 172.21.15.42:40808) 2024-07-04T17:33:36.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.198+0000 7f4b1f400640 1 -- 172.21.15.42:0/1055774968 learned_addr learned my addr 172.21.15.42:0/1055774968 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:36.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.198+0000 7f4b1f400640 1 -- 172.21.15.42:0/1055774968 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b200af7a0 con 0x7f4b200ba830 2024-07-04T17:33:36.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.199+0000 7f4b1f400640 1 --2- 172.21.15.42:0/1055774968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b200ba830 0x7f4b201d7420 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f4b080098d0 tx=0x7f4b08033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:36.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.199+0000 7f4b17400640 1 -- 172.21.15.42:0/1055774968 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b08031bc0 con 0x7f4b200ba830 2024-07-04T17:33:36.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.199+0000 7f4b17400640 1 -- 172.21.15.42:0/1055774968 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b08031e50 con 0x7f4b200ba830 2024-07-04T17:33:36.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.199+0000 7f4b17400640 1 -- 172.21.15.42:0/1055774968 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b08033260 con 0x7f4b200ba830 2024-07-04T17:33:36.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.199+0000 7f4b24c00640 1 -- 172.21.15.42:0/1055774968 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b200714e0 con 0x7f4b200ba830 2024-07-04T17:33:36.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.199+0000 7f4b24c00640 1 -- 172.21.15.42:0/1055774968 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b20073ae0 con 0x7f4b200ba830 2024-07-04T17:33:36.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.200+0000 7f4b17400640 1 -- 172.21.15.42:0/1055774968 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7f4b08033400 con 0x7f4b200ba830 2024-07-04T17:33:36.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.201+0000 7f4b17400640 1 --2- 172.21.15.42:0/1055774968 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f4afc03d4f0 0x7f4afc03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:36.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.201+0000 7f4b17400640 1 -- 172.21.15.42:0/1055774968 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 4303+0+0 (secure 0 0 0) 0x7f4b080732b0 con 0x7f4b200ba830 2024-07-04T17:33:36.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.201+0000 7f4b1ea00640 1 --2- 172.21.15.42:0/1055774968 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f4afc03d4f0 0x7f4afc03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:36.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.202+0000 7f4b1ea00640 1 --2- 172.21.15.42:0/1055774968 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f4afc03d4f0 0x7f4afc03fa00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4b10002040 tx=0x7f4b10001650 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:36.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.203+0000 7f4b2d1b8740 1 Processor -- start 2024-07-04T17:33:36.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.203+0000 7f4b2d1b8740 1 -- start start 2024-07-04T17:33:36.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.204+0000 7f4b2d1b8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ff4b813270 0x55ff4bbb9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:36.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.204+0000 7f4b2d1b8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55ff4bb2a5b0 con 0x55ff4b813270 2024-07-04T17:33:36.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.204+0000 7f4b1f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ff4b813270 0x55ff4bbb9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:36.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.205+0000 7f4b1f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ff4b813270 0x55ff4bbb9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40816/0 (socket says 172.21.15.42:40816) 2024-07-04T17:33:36.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.205+0000 7f4b1f400640 1 -- 172.21.15.42:0/2208007815 learned_addr learned my addr 172.21.15.42:0/2208007815 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:36.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.205+0000 7f4b1f400640 1 -- 172.21.15.42:0/2208007815 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff4b87e850 con 0x55ff4b813270 2024-07-04T17:33:36.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.205+0000 7f4b1f400640 1 --2- 172.21.15.42:0/2208007815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ff4b813270 0x55ff4bbb9780 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f4b08005e90 tx=0x7f4b080316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7bace28f3bf8c4 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:36.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.205+0000 7f4b0f400640 1 -- 172.21.15.42:0/2208007815 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b0806e070 con 0x55ff4b813270 2024-07-04T17:33:36.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.205+0000 7f4b0f400640 1 -- 172.21.15.42:0/2208007815 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b08003720 con 0x55ff4b813270 2024-07-04T17:33:36.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.206+0000 7f4b0f400640 1 -- 172.21.15.42:0/2208007815 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b0805f050 con 0x55ff4b813270 2024-07-04T17:33:36.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.206+0000 7f4b2d1b8740 1 -- 172.21.15.42:0/2208007815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ff4b813270 msgr2=0x55ff4bbb9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:36.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.206+0000 7f4b2d1b8740 1 --2- 172.21.15.42:0/2208007815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ff4b813270 0x55ff4bbb9780 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f4b08005e90 tx=0x7f4b080316f0 comp rx=0 tx=0).stop 2024-07-04T17:33:36.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.206+0000 7f4b2d1b8740 1 -- 172.21.15.42:0/2208007815 shutdown_connections 2024-07-04T17:33:36.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.206+0000 7f4b2d1b8740 1 --2- 172.21.15.42:0/2208007815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ff4b813270 0x55ff4bbb9780 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:36.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.206+0000 7f4b2d1b8740 1 -- 172.21.15.42:0/2208007815 >> 172.21.15.42:0/2208007815 conn(0x55ff4b8073c0 msgr2=0x55ff4bbb7fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:36.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.206+0000 7f4b2d1b8740 1 -- 172.21.15.42:0/2208007815 shutdown_connections 2024-07-04T17:33:36.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.207+0000 7f4b2d1b8740 1 -- 172.21.15.42:0/2208007815 wait complete. 2024-07-04T17:33:36.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.207+0000 7f4b2d1b8740 1 -- start start 2024-07-04T17:33:36.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.208+0000 7f4b2d1b8740 1 Processor -- start 2024-07-04T17:33:36.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.209+0000 7f4b2d1b8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ff4b813270 0x55ff4bbda550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:36.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.209+0000 7f4b2d1b8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55ff4bb1db90 con 0x55ff4b813270 2024-07-04T17:33:36.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.209+0000 7f4b2d1b8740 10 client.0 _ll_register_callbacks cb 0x55ff4bac28a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T17:33:36.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.209+0000 7f4b1f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ff4b813270 0x55ff4bbda550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:36.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.209+0000 7f4b1f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ff4b813270 0x55ff4bbda550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40822/0 (socket says 172.21.15.42:40822) 2024-07-04T17:33:36.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.209+0000 7f4b1f400640 1 -- 172.21.15.42:0/2567124191 learned_addr learned my addr 172.21.15.42:0/2567124191 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:36.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.210+0000 7f4b1f400640 1 -- 172.21.15.42:0/2567124191 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff4b8f56b0 con 0x55ff4b813270 2024-07-04T17:33:36.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.210+0000 7f4b1f400640 1 --2- 172.21.15.42:0/2567124191 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ff4b813270 0x55ff4bbda550 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f4b0803bfd0 tx=0x7f4b0806ae00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0b385803183e3 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:36.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.210+0000 7f4b03e00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-07-04T17:33:36.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.210+0000 7f4b03e00640 1 -- 172.21.15.42:0/2567124191 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b0806e070 con 0x55ff4b813270 2024-07-04T17:33:36.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.210+0000 7f4b03e00640 1 -- 172.21.15.42:0/2567124191 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 4303+0+0 (secure 0 0 0) 0x7f4b08037e30 con 0x55ff4b813270 2024-07-04T17:33:36.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.210+0000 7f4b2d1b8740 1 -- 172.21.15.42:0/2567124191 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ff4baba6c0 con 0x55ff4b813270 2024-07-04T17:33:36.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.211+0000 7f4b03e00640 1 -- 172.21.15.42:0/2567124191 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f4b0804bdd0 con 0x55ff4b813270 2024-07-04T17:33:36.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.211+0000 7f4b2d1b8740 10 client.5651 fetch_fsmap learned FSMap version 96 2024-07-04T17:33:36.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.211+0000 7f4b2d1b8740 1 -- 172.21.15.42:0/2567124191 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) -- 0x55ff4bbd5670 con 0x55ff4b813270 2024-07-04T17:33:36.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.211+0000 7f4b03e00640 1 -- 172.21.15.42:0/2567124191 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55ff4baba6c0 con 0x55ff4b813270 2024-07-04T17:33:36.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.211+0000 7f4b03e00640 1 -- 172.21.15.42:0/2567124191 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 96) ==== 4754+0+0 (secure 0 0 0) 0x7f4b08070300 con 0x55ff4b813270 2024-07-04T17:33:36.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.211+0000 7f4b2d1b8740 10 client.5651 fetch_fsmap finished waiting for FSMap version 96 2024-07-04T17:33:36.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.211+0000 7f4b2d1b8740 10 client.5651 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-04T17:33:36.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.211+0000 7f4b2d1b8740 20 client.5651 populate_metadata read hostname 'smithi042' 2024-07-04T17:33:36.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.211+0000 7f4b2d1b8740 1 --2- 172.21.15.42:0/2567124191 >> [v2:172.21.15.42:6830/3224527035,v1:172.21.15.42:6831/3224527035] conn(0x55ff4b7ea3b0 0x55ff4bbdd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:36.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.211+0000 7f4b2d1b8740 4 client.5651 mds_command: new command op to 5549 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T17:33:36.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.211+0000 7f4b2d1b8740 1 -- 172.21.15.42:0/2567124191 --> [v2:172.21.15.42:6830/3224527035,v1:172.21.15.42:6831/3224527035] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff4bb2c500 con 0x55ff4b7ea3b0 2024-07-04T17:33:36.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.211+0000 7f4b1ea00640 1 --2- 172.21.15.42:0/2567124191 >> [v2:172.21.15.42:6830/3224527035,v1:172.21.15.42:6831/3224527035] conn(0x55ff4b7ea3b0 0x55ff4bbdd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:36.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.212+0000 7f4b1ea00640 1 --2- 172.21.15.42:0/2567124191 >> [v2:172.21.15.42:6830/3224527035,v1:172.21.15.42:6831/3224527035] conn(0x55ff4b7ea3b0 0x55ff4bbdd6e0 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c731e8901e8439e server_cookie=8a626d93de88c246 in_seq=0 out_seq=0 2024-07-04T17:33:36.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.212+0000 7f4b03e00640 10 client.5651 ms_handle_connect on v2:172.21.15.42:6830/3224527035 2024-07-04T17:33:36.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.213+0000 7f4b03e00640 1 -- 172.21.15.42:0/2567124191 <== mds.0 v2:172.21.15.42:6830/3224527035 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ff4bb2c500 con 0x55ff4b7ea3b0 2024-07-04T17:33:36.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.213+0000 7f4b03e00640 10 client.5651 handle_command_reply: tid=0 2024-07-04T17:33:36.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.213+0000 7f4b2d1b8740 1 client.5651 shutdown 2024-07-04T17:33:36.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.213+0000 7f4b2d1b8740 1 -- 172.21.15.42:0/2567124191 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ff4b813270 msgr2=0x55ff4bbda550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:36.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.213+0000 7f4b2d1b8740 1 --2- 172.21.15.42:0/2567124191 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ff4b813270 0x55ff4bbda550 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f4b0803bfd0 tx=0x7f4b0806ae00 comp rx=0 tx=0).stop 2024-07-04T17:33:36.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.213+0000 7f4b2d1b8740 1 -- 172.21.15.42:0/2567124191 shutdown_connections 2024-07-04T17:33:36.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.213+0000 7f4b2d1b8740 1 --2- 172.21.15.42:0/2567124191 >> [v2:172.21.15.42:6830/3224527035,v1:172.21.15.42:6831/3224527035] conn(0x55ff4b7ea3b0 0x55ff4bbdd6e0 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:36.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.213+0000 7f4b2d1b8740 1 --2- 172.21.15.42:0/2567124191 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ff4b813270 0x55ff4bbda550 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:36.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.213+0000 7f4b2d1b8740 1 -- 172.21.15.42:0/2567124191 >> 172.21.15.42:0/2567124191 conn(0x55ff4b8073c0 msgr2=0x55ff4bbb3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:36.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.213+0000 7f4b03e00640 0 client.5651 ms_handle_reset on v2:172.21.15.42:6830/3224527035 2024-07-04T17:33:36.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.214+0000 7f4b2d1b8740 1 -- 172.21.15.42:0/2567124191 shutdown_connections 2024-07-04T17:33:36.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.214+0000 7f4b2d1b8740 1 -- 172.21.15.42:0/2567124191 wait complete. 2024-07-04T17:33:36.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.214+0000 7f4b2d1b8740 20 client.5651 trim_cache size 0 max 16384 2024-07-04T17:33:36.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.235+0000 7f4b2d1b8740 1 Processor -- start 2024-07-04T17:33:36.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.235+0000 7f4b2d1b8740 1 -- start start 2024-07-04T17:33:36.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.236+0000 7f4b2d1b8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ff4b813270 0x55ff4bbdcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:36.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.236+0000 7f4b2d1b8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4b200b7d10 con 0x55ff4b813270 2024-07-04T17:33:36.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.236+0000 7f4b1f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ff4b813270 0x55ff4bbdcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:36.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.236+0000 7f4b1f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ff4b813270 0x55ff4bbdcf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40824/0 (socket says 172.21.15.42:40824) 2024-07-04T17:33:36.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.236+0000 7f4b1f400640 1 -- 172.21.15.42:0/1979093208 learned_addr learned my addr 172.21.15.42:0/1979093208 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:36.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.237+0000 7f4b1f400640 1 -- 172.21.15.42:0/1979093208 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff4bbd5670 con 0x55ff4b813270 2024-07-04T17:33:36.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.237+0000 7f4b1f400640 1 --2- 172.21.15.42:0/1979093208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ff4b813270 0x55ff4bbdcf20 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f4b08033f20 tx=0x7f4b080316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbdeb39e844d7956 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:36.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.237+0000 7f4b0ea00640 1 -- 172.21.15.42:0/1979093208 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b0806e040 con 0x55ff4b813270 2024-07-04T17:33:36.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.237+0000 7f4b0ea00640 1 -- 172.21.15.42:0/1979093208 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b08071cb0 con 0x55ff4b813270 2024-07-04T17:33:36.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.237+0000 7f4b0ea00640 1 -- 172.21.15.42:0/1979093208 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b08070430 con 0x55ff4b813270 2024-07-04T17:33:36.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.238+0000 7f4b2d1b8740 1 -- 172.21.15.42:0/1979093208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ff4b813270 msgr2=0x55ff4bbdcf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:36.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.238+0000 7f4b2d1b8740 1 --2- 172.21.15.42:0/1979093208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ff4b813270 0x55ff4bbdcf20 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f4b08033f20 tx=0x7f4b080316f0 comp rx=0 tx=0).stop 2024-07-04T17:33:36.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.238+0000 7f4b2d1b8740 1 -- 172.21.15.42:0/1979093208 shutdown_connections 2024-07-04T17:33:36.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.238+0000 7f4b2d1b8740 1 --2- 172.21.15.42:0/1979093208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ff4b813270 0x55ff4bbdcf20 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:36.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.238+0000 7f4b2d1b8740 1 -- 172.21.15.42:0/1979093208 >> 172.21.15.42:0/1979093208 conn(0x55ff4b8073c0 msgr2=0x55ff4bbfccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:36.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.238+0000 7f4b2d1b8740 1 -- 172.21.15.42:0/1979093208 shutdown_connections 2024-07-04T17:33:36.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.238+0000 7f4b2d1b8740 1 -- 172.21.15.42:0/1979093208 wait complete. 2024-07-04T17:33:36.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.239+0000 7f4b2d1b8740 1 -- start start 2024-07-04T17:33:36.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.239+0000 7f4b2d1b8740 1 Processor -- start 2024-07-04T17:33:36.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.241+0000 7f4b2d1b8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ff4b813270 0x55ff4bc01a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:36.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.241+0000 7f4b2d1b8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55ff4bb19b50 con 0x55ff4b813270 2024-07-04T17:33:36.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.241+0000 7f4b2d1b8740 10 client.0 _ll_register_callbacks cb 0x7f4b08071990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T17:33:36.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.241+0000 7f4b1f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ff4b813270 0x55ff4bc01a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:36.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.241+0000 7f4b1f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ff4b813270 0x55ff4bc01a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40838/0 (socket says 172.21.15.42:40838) 2024-07-04T17:33:36.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.241+0000 7f4b1f400640 1 -- 172.21.15.42:0/1612249292 learned_addr learned my addr 172.21.15.42:0/1612249292 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:36.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.241+0000 7f4b1f400640 1 -- 172.21.15.42:0/1612249292 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff4bc01ff0 con 0x55ff4b813270 2024-07-04T17:33:36.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.242+0000 7f4b1f400640 1 --2- 172.21.15.42:0/1612249292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ff4b813270 0x55ff4bc01a90 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f4b0806af70 tx=0x7f4b080043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ef10420e6f3dcf server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:36.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.242+0000 7f4b03e00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-07-04T17:33:36.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.242+0000 7f4b03e00640 1 -- 172.21.15.42:0/1612249292 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b0806e040 con 0x55ff4b813270 2024-07-04T17:33:36.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.242+0000 7f4b03e00640 1 -- 172.21.15.42:0/1612249292 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 4303+0+0 (secure 0 0 0) 0x7f4b08030800 con 0x55ff4b813270 2024-07-04T17:33:36.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.242+0000 7f4b2d1b8740 1 -- 172.21.15.42:0/1612249292 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b08071cb0 con 0x55ff4b813270 2024-07-04T17:33:36.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.242+0000 7f4b03e00640 1 -- 172.21.15.42:0/1612249292 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f4b08030a80 con 0x55ff4b813270 2024-07-04T17:33:36.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.242+0000 7f4b2d1b8740 10 client.5653 fetch_fsmap learned FSMap version 96 2024-07-04T17:33:36.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.242+0000 7f4b2d1b8740 1 -- 172.21.15.42:0/1612249292 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) -- 0x55ff4bc01ff0 con 0x55ff4b813270 2024-07-04T17:33:36.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.242+0000 7f4b03e00640 1 -- 172.21.15.42:0/1612249292 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b08071cb0 con 0x55ff4b813270 2024-07-04T17:33:36.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.243+0000 7f4b03e00640 1 -- 172.21.15.42:0/1612249292 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 96) ==== 4754+0+0 (secure 0 0 0) 0x7f4b08030ca0 con 0x55ff4b813270 2024-07-04T17:33:36.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.243+0000 7f4b2d1b8740 10 client.5653 fetch_fsmap finished waiting for FSMap version 96 2024-07-04T17:33:36.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.243+0000 7f4b2d1b8740 10 client.5653 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-04T17:33:36.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.243+0000 7f4b2d1b8740 20 client.5653 populate_metadata read hostname 'smithi042' 2024-07-04T17:33:36.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.243+0000 7f4b2d1b8740 1 --2- 172.21.15.42:0/1612249292 >> [v2:172.21.15.42:6830/3224527035,v1:172.21.15.42:6831/3224527035] conn(0x55ff4b7d1e40 0x55ff4bbd8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:36.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.243+0000 7f4b2d1b8740 4 client.5653 mds_command: new command op to 5549 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-04T17:33:36.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.243+0000 7f4b2d1b8740 1 -- 172.21.15.42:0/1612249292 --> [v2:172.21.15.42:6830/3224527035,v1:172.21.15.42:6831/3224527035] -- command(tid 0: {"prefix": "status"}) -- 0x55ff4bb27fa0 con 0x55ff4b7d1e40 2024-07-04T17:33:36.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.243+0000 7f4b1ea00640 1 --2- 172.21.15.42:0/1612249292 >> [v2:172.21.15.42:6830/3224527035,v1:172.21.15.42:6831/3224527035] conn(0x55ff4b7d1e40 0x55ff4bbd8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:36.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.244+0000 7f4b1ea00640 1 --2- 172.21.15.42:0/1612249292 >> [v2:172.21.15.42:6830/3224527035,v1:172.21.15.42:6831/3224527035] conn(0x55ff4b7d1e40 0x55ff4bbd8850 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=553b528afe645b34 server_cookie=a40c7db64ff32e12 in_seq=0 out_seq=0 2024-07-04T17:33:36.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.244+0000 7f4b03e00640 10 client.5653 ms_handle_connect on v2:172.21.15.42:6830/3224527035 2024-07-04T17:33:36.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.244+0000 7f4b03e00640 1 -- 172.21.15.42:0/1612249292 <== mds.0 v2:172.21.15.42:6830/3224527035 1 ==== command_reply(tid 0: 0 ) ==== 8+0+336 (crc 0 0 0) 0x55ff4bb27fa0 con 0x55ff4b7d1e40 2024-07-04T17:33:36.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.244+0000 7f4b03e00640 10 client.5653 handle_command_reply: tid=0 2024-07-04T17:33:36.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.244+0000 7f4b2d1b8740 1 client.5653 shutdown 2024-07-04T17:33:36.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.244+0000 7f4b2d1b8740 1 -- 172.21.15.42:0/1612249292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ff4b813270 msgr2=0x55ff4bc01a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:36.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.244+0000 7f4b2d1b8740 1 --2- 172.21.15.42:0/1612249292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ff4b813270 0x55ff4bc01a90 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f4b0806af70 tx=0x7f4b080043f0 comp rx=0 tx=0).stop 2024-07-04T17:33:36.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.244+0000 7f4b2d1b8740 1 -- 172.21.15.42:0/1612249292 shutdown_connections 2024-07-04T17:33:36.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.244+0000 7f4b2d1b8740 1 --2- 172.21.15.42:0/1612249292 >> [v2:172.21.15.42:6830/3224527035,v1:172.21.15.42:6831/3224527035] conn(0x55ff4b7d1e40 0x55ff4bbd8850 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:36.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.244+0000 7f4b2d1b8740 1 --2- 172.21.15.42:0/1612249292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ff4b813270 0x55ff4bc01a90 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:36.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.244+0000 7f4b2d1b8740 1 -- 172.21.15.42:0/1612249292 >> 172.21.15.42:0/1612249292 conn(0x55ff4b8073c0 msgr2=0x55ff4bbdcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:36.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.244+0000 7f4b03e00640 0 client.5653 ms_handle_reset on v2:172.21.15.42:6830/3224527035 2024-07-04T17:33:36.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.244+0000 7f4b2d1b8740 1 -- 172.21.15.42:0/1612249292 shutdown_connections 2024-07-04T17:33:36.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.244+0000 7f4b2d1b8740 1 -- 172.21.15.42:0/1612249292 wait complete. 2024-07-04T17:33:36.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.245+0000 7f4b2d1b8740 20 client.5653 trim_cache size 0 max 16384 2024-07-04T17:33:36.248 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:33:36.248 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "be4f300a-93ef-4489-b540-37ece39b0aac", 2024-07-04T17:33:36.248 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-07-04T17:33:36.248 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 5549, 2024-07-04T17:33:36.248 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-07-04T17:33:36.249 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-07-04T17:33:36.249 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "backup_fs", 2024-07-04T17:33:36.249 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 6.8433936849999997, 2024-07-04T17:33:36.249 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 96, 2024-07-04T17:33:36.249 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 118, 2024-07-04T17:33:36.249 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 118, 2024-07-04T17:33:36.249 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 32.036474562999999 2024-07-04T17:33:36.249 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:33:36.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.245+0000 7f4b24c00640 1 -- 172.21.15.42:0/1055774968 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f4afc03d4f0 msgr2=0x7f4afc03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:36.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.245+0000 7f4b24c00640 1 --2- 172.21.15.42:0/1055774968 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f4afc03d4f0 0x7f4afc03fa00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4b10002040 tx=0x7f4b10001650 comp rx=0 tx=0).stop 2024-07-04T17:33:36.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.245+0000 7f4b24c00640 1 -- 172.21.15.42:0/1055774968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b200ba830 msgr2=0x7f4b201d7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:36.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.245+0000 7f4b24c00640 1 --2- 172.21.15.42:0/1055774968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b200ba830 0x7f4b201d7420 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f4b080098d0 tx=0x7f4b08033e40 comp rx=0 tx=0).stop 2024-07-04T17:33:36.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.245+0000 7f4b24c00640 1 -- 172.21.15.42:0/1055774968 shutdown_connections 2024-07-04T17:33:36.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.245+0000 7f4b24c00640 1 --2- 172.21.15.42:0/1055774968 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f4afc03d4f0 0x7f4afc03fa00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:36.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.245+0000 7f4b24c00640 1 --2- 172.21.15.42:0/1055774968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b200ba830 0x7f4b201d7420 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:36.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.245+0000 7f4b24c00640 1 -- 172.21.15.42:0/1055774968 >> 172.21.15.42:0/1055774968 conn(0x7f4b200bebc0 msgr2=0x7f4b20142fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:36.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.245+0000 7f4b24c00640 1 -- 172.21.15.42:0/1055774968 shutdown_connections 2024-07-04T17:33:36.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.245+0000 7f4b24c00640 1 -- 172.21.15.42:0/1055774968 wait complete. 2024-07-04T17:33:36.275 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable mirroring 2024-07-04T17:33:36.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.567+0000 7f34a0c00640 1 -- 172.21.15.42:0/2696910146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349c0734e0 msgr2=0x7f349c0b26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:36.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.567+0000 7f34a0c00640 1 --2- 172.21.15.42:0/2696910146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349c0734e0 0x7f349c0b26f0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f34880097b0 tx=0x7f348802a540 comp rx=0 tx=0).stop 2024-07-04T17:33:36.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.567+0000 7f34a0c00640 1 -- 172.21.15.42:0/2696910146 shutdown_connections 2024-07-04T17:33:36.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.567+0000 7f34a0c00640 1 --2- 172.21.15.42:0/2696910146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349c0734e0 0x7f349c0b26f0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:36.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.567+0000 7f34a0c00640 1 -- 172.21.15.42:0/2696910146 >> 172.21.15.42:0/2696910146 conn(0x7f349c06fe40 msgr2=0x7f349c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:36.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.568+0000 7f34a0c00640 1 -- 172.21.15.42:0/2696910146 shutdown_connections 2024-07-04T17:33:36.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.568+0000 7f34a0c00640 1 -- 172.21.15.42:0/2696910146 wait complete. 2024-07-04T17:33:36.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.570+0000 7f34a0c00640 1 Processor -- start 2024-07-04T17:33:36.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.570+0000 7f34a0c00640 1 -- start start 2024-07-04T17:33:36.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.571+0000 7f34a0c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349c072a60 0x7f349c1e6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:36.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.571+0000 7f34a0c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f349c140830 con 0x7f349c072a60 2024-07-04T17:33:36.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.571+0000 7f3499e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349c072a60 0x7f349c1e6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:36.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.571+0000 7f3499e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349c072a60 0x7f349c1e6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40866/0 (socket says 172.21.15.42:40866) 2024-07-04T17:33:36.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.571+0000 7f3499e00640 1 -- 172.21.15.42:0/3194866315 learned_addr learned my addr 172.21.15.42:0/3194866315 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:36.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.572+0000 7f3499e00640 1 -- 172.21.15.42:0/3194866315 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f349c1d4f90 con 0x7f349c072a60 2024-07-04T17:33:36.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.572+0000 7f3499e00640 1 --2- 172.21.15.42:0/3194866315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349c072a60 0x7f349c1e6c00 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f348c00cb40 tx=0x7f348c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:36.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.572+0000 7f3492000640 1 -- 172.21.15.42:0/3194866315 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f348c016020 con 0x7f349c072a60 2024-07-04T17:33:36.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.573+0000 7f3492000640 1 -- 172.21.15.42:0/3194866315 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f348c00ac30 con 0x7f349c072a60 2024-07-04T17:33:36.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.573+0000 7f34a0c00640 1 -- 172.21.15.42:0/3194866315 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f349c1e7160 con 0x7f349c072a60 2024-07-04T17:33:36.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.573+0000 7f3492000640 1 -- 172.21.15.42:0/3194866315 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f348c012420 con 0x7f349c072a60 2024-07-04T17:33:36.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.573+0000 7f34a0c00640 1 -- 172.21.15.42:0/3194866315 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f349c13c9d0 con 0x7f349c072a60 2024-07-04T17:33:36.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.574+0000 7f34a0c00640 1 -- 172.21.15.42:0/3194866315 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f349c0afbf0 con 0x7f349c072a60 2024-07-04T17:33:36.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.574+0000 7f3492000640 1 -- 172.21.15.42:0/3194866315 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 42) ==== 49615+0+0 (secure 0 0 0) 0x7f348c019020 con 0x7f349c072a60 2024-07-04T17:33:36.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.575+0000 7f3492000640 1 --2- 172.21.15.42:0/3194866315 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f347c03d4f0 0x7f347c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:36.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.575+0000 7f3492000640 1 -- 172.21.15.42:0/3194866315 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 4303+0+0 (secure 0 0 0) 0x7f348c0520c0 con 0x7f349c072a60 2024-07-04T17:33:36.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.575+0000 7f3499400640 1 --2- 172.21.15.42:0/3194866315 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f347c03d4f0 0x7f347c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:36.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.576+0000 7f3499400640 1 --2- 172.21.15.42:0/3194866315 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f347c03d4f0 0x7f347c03fa00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3488034000 tx=0x7f3488004730 comp rx=0 tx=0).ready entity=mgr.5434 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:36.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.580+0000 7f3492000640 1 -- 172.21.15.42:0/3194866315 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f349c0afbf0 con 0x7f349c072a60 2024-07-04T17:33:36.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:36.840+0000 7f34a0c00640 1 -- 172.21.15.42:0/3194866315 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "mirroring"} v 0) -- 0x7f349c13e070 con 0x7f349c072a60 2024-07-04T17:33:37.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:37.451+0000 7f3492000640 1 -- 172.21.15.42:0/3194866315 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "mirroring"}]=0 v43) ==== 88+0+0 (secure 0 0 0) 0x7f348c020030 con 0x7f349c072a60 2024-07-04T17:33:37.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:37.457+0000 7f34a0c00640 1 -- 172.21.15.42:0/3194866315 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f347c03d4f0 msgr2=0x7f347c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:37.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:37.457+0000 7f34a0c00640 1 --2- 172.21.15.42:0/3194866315 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f347c03d4f0 0x7f347c03fa00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3488034000 tx=0x7f3488004730 comp rx=0 tx=0).stop 2024-07-04T17:33:37.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:37.457+0000 7f34a0c00640 1 -- 172.21.15.42:0/3194866315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349c072a60 msgr2=0x7f349c1e6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:37.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:37.457+0000 7f34a0c00640 1 --2- 172.21.15.42:0/3194866315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349c072a60 0x7f349c1e6c00 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f348c00cb40 tx=0x7f348c00a060 comp rx=0 tx=0).stop 2024-07-04T17:33:37.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:37.458+0000 7f34a0c00640 1 -- 172.21.15.42:0/3194866315 shutdown_connections 2024-07-04T17:33:37.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:37.458+0000 7f34a0c00640 1 --2- 172.21.15.42:0/3194866315 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7f347c03d4f0 0x7f347c03fa00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:37.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:37.458+0000 7f34a0c00640 1 --2- 172.21.15.42:0/3194866315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349c072a60 0x7f349c1e6c00 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:37.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:37.458+0000 7f34a0c00640 1 -- 172.21.15.42:0/3194866315 >> 172.21.15.42:0/3194866315 conn(0x7f349c06fe40 msgr2=0x7f349c0b1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:37.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:37.459+0000 7f34a0c00640 1 -- 172.21.15.42:0/3194866315 shutdown_connections 2024-07-04T17:33:37.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:37.460+0000 7f34a0c00640 1 -- 172.21.15.42:0/3194866315 wait complete. 2024-07-04T17:33:37.493 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:33:37.493 DEBUG:teuthology.orchestra.run.smithi042:counter dump for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok counter dump 2024-07-04T17:33:37.713 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:33:37.713 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker": [ 2024-07-04T17:33:37.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:37.714 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:33:37.714 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "0" 2024-07-04T17:33:37.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:37.714 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:37.714 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:33:37.714 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:33:37.714 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:37.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:37.714 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:37.714 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:33:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "1" 2024-07-04T17:33:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:33:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:33:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:33:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "2" 2024-07-04T17:33:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:37.715 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:37.716 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:33:37.716 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:33:37.716 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:37.716 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:37.716 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:37.716 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": [ 2024-07-04T17:33:37.716 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:37.716 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:37.716 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:37.716 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 94, 2024-07-04T17:33:37.716 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 5, 2024-07-04T17:33:37.716 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 385488, 2024-07-04T17:33:37.716 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 1136, 2024-07-04T17:33:37.717 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:33:37.717 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:33:37.717 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.035685981, 2024-07-04T17:33:37.717 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.004100999, 2024-07-04T17:33:37.717 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.019059290, 2024-07-04T17:33:37.717 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.010637924, 2024-07-04T17:33:37.717 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:33:37.717 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 5, 2024-07-04T17:33:37.717 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.004490948, 2024-07-04T17:33:37.717 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000898189 2024-07-04T17:33:37.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:37.717 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:33:37.717 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:33:37.718 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:33:37.718 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:33:37.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:37.718 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 385488, 2024-07-04T17:33:37.718 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 1136 2024-07-04T17:33:37.718 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:37.718 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:37.718 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:37.718 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": [ 2024-07-04T17:33:37.718 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:37.718 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:37.718 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:37.718 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 8, 2024-07-04T17:33:37.719 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 97, 2024-07-04T17:33:37.719 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 1152, 2024-07-04T17:33:37.719 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 664608, 2024-07-04T17:33:37.719 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 18, 2024-07-04T17:33:37.719 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 3, 2024-07-04T17:33:37.719 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.042660590, 2024-07-04T17:33:37.719 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.014184783, 2024-07-04T17:33:37.719 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.014356959, 2024-07-04T17:33:37.719 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:33:37.719 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:33:37.719 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 89, 2024-07-04T17:33:37.719 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.072395044, 2024-07-04T17:33:37.719 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000813427 2024-07-04T17:33:37.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:37.720 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:33:37.720 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:33:37.720 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:33:37.720 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:33:37.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:37.720 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 1152, 2024-07-04T17:33:37.720 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 664608 2024-07-04T17:33:37.720 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:37.720 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:37.720 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:37.720 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": [ 2024-07-04T17:33:37.720 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:37.720 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:37.721 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:37.721 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 48, 2024-07-04T17:33:37.721 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 8, 2024-07-04T17:33:37.721 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 1940384, 2024-07-04T17:33:37.721 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 1600, 2024-07-04T17:33:37.721 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:33:37.721 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:33:37.721 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.055197692, 2024-07-04T17:33:37.721 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.003961334, 2024-07-04T17:33:37.721 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.093794757, 2024-07-04T17:33:37.721 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:33:37.721 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:33:37.721 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 8, 2024-07-04T17:33:37.722 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.003350228, 2024-07-04T17:33:37.722 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000418778 2024-07-04T17:33:37.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:37.722 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:33:37.722 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:33:37.722 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:33:37.722 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:33:37.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:37.722 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 1940384, 2024-07-04T17:33:37.722 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 1600 2024-07-04T17:33:37.722 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:37.722 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:37.722 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:37.723 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": [ 2024-07-04T17:33:37.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:37.723 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:37.723 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:37.723 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-07-04T17:33:37.723 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-07-04T17:33:37.723 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:37.723 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:37.723 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:37.723 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror": [ 2024-07-04T17:33:37.723 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:37.723 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:37.723 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:37.724 INFO:teuthology.orchestra.run.smithi042.stdout: "mirrored_filesystems": 0, 2024-07-04T17:33:37.724 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_enable_failures": 0 2024-07-04T17:33:37.724 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:37.724 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:37.724 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:37.724 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": [ 2024-07-04T17:33:37.724 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:37.724 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:37.724 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:37.724 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-07-04T17:33:37.724 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-07-04T17:33:37.724 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T17:33:37.724 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-07-04T17:33:37.724 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T17:33:37.724 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-07-04T17:33:37.725 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T17:33:37.725 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T17:33:37.725 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T17:33:37.725 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T17:33:37.725 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T17:33:37.725 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-07-04T17:33:37.725 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T17:33:37.725 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T17:33:37.725 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-07-04T17:33:37.725 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-07-04T17:33:37.725 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-07-04T17:33:37.725 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-07-04T17:33:37.725 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T17:33:37.725 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T17:33:37.725 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T17:33:37.726 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T17:33:37.726 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T17:33:37.726 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-07-04T17:33:37.726 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-07-04T17:33:37.726 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-07-04T17:33:37.726 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T17:33:37.726 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T17:33:37.726 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-07-04T17:33:37.726 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-07-04T17:33:37.726 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-07-04T17:33:37.726 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-07-04T17:33:37.726 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T17:33:37.726 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-07-04T17:33:37.727 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T17:33:37.727 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-07-04T17:33:37.727 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 149196, 2024-07-04T17:33:37.727 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 27, 2024-07-04T17:33:37.727 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-07-04T17:33:37.727 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-07-04T17:33:37.727 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-07-04T17:33:37.727 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-07-04T17:33:37.727 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-07-04T17:33:37.727 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-07-04T17:33:37.727 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-07-04T17:33:37.727 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-07-04T17:33:37.727 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 4592, 2024-07-04T17:33:37.728 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 37, 2024-07-04T17:33:37.728 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T17:33:37.728 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-07-04T17:33:37.728 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-07-04T17:33:37.728 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-07-04T17:33:37.728 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 0, 2024-07-04T17:33:37.728 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 0, 2024-07-04T17:33:37.728 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-07-04T17:33:37.728 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-07-04T17:33:37.728 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-07-04T17:33:37.728 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-07-04T17:33:37.728 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:37.728 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:37.729 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:37.729 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter": [ 2024-07-04T17:33:37.729 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:37.729 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:37.729 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:37.729 INFO:teuthology.orchestra.run.smithi042.stdout: "op_active": 0, 2024-07-04T17:33:37.729 INFO:teuthology.orchestra.run.smithi042.stdout: "op_laggy": 0, 2024-07-04T17:33:37.729 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send": 0, 2024-07-04T17:33:37.729 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send_bytes": 0, 2024-07-04T17:33:37.729 INFO:teuthology.orchestra.run.smithi042.stdout: "op_resend": 0, 2024-07-04T17:33:37.729 INFO:teuthology.orchestra.run.smithi042.stdout: "op_reply": 0, 2024-07-04T17:33:37.729 INFO:teuthology.orchestra.run.smithi042.stdout: "op_latency": { 2024-07-04T17:33:37.729 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:33:37.730 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:33:37.730 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:33:37.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:37.730 INFO:teuthology.orchestra.run.smithi042.stdout: "op_inflight": 0, 2024-07-04T17:33:37.730 INFO:teuthology.orchestra.run.smithi042.stdout: "oplen_avg": { 2024-07-04T17:33:37.730 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:33:37.730 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0 2024-07-04T17:33:37.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:37.730 INFO:teuthology.orchestra.run.smithi042.stdout: "op": 0, 2024-07-04T17:33:37.730 INFO:teuthology.orchestra.run.smithi042.stdout: "op_r": 0, 2024-07-04T17:33:37.730 INFO:teuthology.orchestra.run.smithi042.stdout: "op_w": 0, 2024-07-04T17:33:37.730 INFO:teuthology.orchestra.run.smithi042.stdout: "op_rmw": 0, 2024-07-04T17:33:37.730 INFO:teuthology.orchestra.run.smithi042.stdout: "op_pg": 0, 2024-07-04T17:33:37.730 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_stat": 0, 2024-07-04T17:33:37.731 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_create": 0, 2024-07-04T17:33:37.731 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_read": 0, 2024-07-04T17:33:37.731 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_write": 0, 2024-07-04T17:33:37.731 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writefull": 0, 2024-07-04T17:33:37.731 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writesame": 0, 2024-07-04T17:33:37.731 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_append": 0, 2024-07-04T17:33:37.731 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_zero": 0, 2024-07-04T17:33:37.731 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_truncate": 0, 2024-07-04T17:33:37.731 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_delete": 0, 2024-07-04T17:33:37.731 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_mapext": 0, 2024-07-04T17:33:37.731 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_sparse_read": 0, 2024-07-04T17:33:37.731 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_clonerange": 0, 2024-07-04T17:33:37.731 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_getxattr": 0, 2024-07-04T17:33:37.731 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_setxattr": 0, 2024-07-04T17:33:37.732 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_cmpxattr": 0, 2024-07-04T17:33:37.732 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_rmxattr": 0, 2024-07-04T17:33:37.732 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_resetxattrs": 0, 2024-07-04T17:33:37.732 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_call": 0, 2024-07-04T17:33:37.732 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_watch": 0, 2024-07-04T17:33:37.732 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_notify": 0, 2024-07-04T17:33:37.732 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T17:33:37.732 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls": 0, 2024-07-04T17:33:37.732 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls_filter": 0, 2024-07-04T17:33:37.732 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_other": 0, 2024-07-04T17:33:37.732 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_active": 0, 2024-07-04T17:33:37.732 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_send": 0, 2024-07-04T17:33:37.732 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_resend": 0, 2024-07-04T17:33:37.732 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_ping": 0, 2024-07-04T17:33:37.732 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_active": 0, 2024-07-04T17:33:37.733 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_send": 0, 2024-07-04T17:33:37.733 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_resend": 0, 2024-07-04T17:33:37.733 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_active": 0, 2024-07-04T17:33:37.733 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_send": 0, 2024-07-04T17:33:37.733 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_resend": 0, 2024-07-04T17:33:37.733 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_active": 0, 2024-07-04T17:33:37.733 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_send": 0, 2024-07-04T17:33:37.733 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_resend": 0, 2024-07-04T17:33:37.733 INFO:teuthology.orchestra.run.smithi042.stdout: "command_active": 0, 2024-07-04T17:33:37.733 INFO:teuthology.orchestra.run.smithi042.stdout: "command_send": 0, 2024-07-04T17:33:37.733 INFO:teuthology.orchestra.run.smithi042.stdout: "command_resend": 0, 2024-07-04T17:33:37.733 INFO:teuthology.orchestra.run.smithi042.stdout: "map_epoch": 0, 2024-07-04T17:33:37.733 INFO:teuthology.orchestra.run.smithi042.stdout: "map_full": 0, 2024-07-04T17:33:37.733 INFO:teuthology.orchestra.run.smithi042.stdout: "map_inc": 0, 2024-07-04T17:33:37.734 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_sessions": 0, 2024-07-04T17:33:37.734 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_open": 0, 2024-07-04T17:33:37.734 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_close": 0, 2024-07-04T17:33:37.734 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_laggy": 0, 2024-07-04T17:33:37.734 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_wr": 0, 2024-07-04T17:33:37.734 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_rd": 0, 2024-07-04T17:33:37.734 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_del": 0 2024-07-04T17:33:37.734 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:37.734 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:37.734 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:37.734 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-client": [ 2024-07-04T17:33:37.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:37.734 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:37.735 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:37.735 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:33:37.735 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:33:37.735 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:33:37.735 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 94, 2024-07-04T17:33:37.735 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 372643, 2024-07-04T17:33:37.735 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:33:37.735 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 94, 2024-07-04T17:33:37.735 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:33:37.735 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:33:37.735 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 94, 2024-07-04T17:33:37.735 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 372643, 2024-07-04T17:33:37.735 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:33:37.736 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:33:37.736 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:33:37.736 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:33:37.736 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:37.736 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:37.736 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:37.736 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:37.736 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-radosclient": [ 2024-07-04T17:33:37.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:37.736 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:37.736 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:37.736 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:33:37.736 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:33:37.737 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:33:37.737 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 53, 2024-07-04T17:33:37.737 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 1933421, 2024-07-04T17:33:37.737 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:33:37.737 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 53, 2024-07-04T17:33:37.737 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:33:37.737 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:33:37.737 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 53, 2024-07-04T17:33:37.737 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 1933421, 2024-07-04T17:33:37.737 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:33:37.737 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:33:37.737 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:33:37.737 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:33:37.737 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:37.737 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:37.737 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:37.738 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:37.738 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_bytes": [ 2024-07-04T17:33:37.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:37.738 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:37.738 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:37.738 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:33:37.738 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:33:37.738 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:33:37.738 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:33:37.738 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:33:37.738 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:33:37.738 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:33:37.738 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:33:37.739 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:33:37.739 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:33:37.739 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:33:37.739 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:33:37.739 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:33:37.739 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:33:37.739 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:33:37.739 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:37.739 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:37.739 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:37.739 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:37.739 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_ops": [ 2024-07-04T17:33:37.739 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:37.740 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:37.740 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:37.740 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:33:37.740 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1024, 2024-07-04T17:33:37.740 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:33:37.740 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:33:37.740 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:33:37.740 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:33:37.740 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:33:37.740 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:33:37.740 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:33:37.740 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:33:37.740 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:33:37.740 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:33:37.741 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:33:37.741 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:33:37.741 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:33:37.741 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:37.741 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:37.741 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:37.741 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-07-04T17:33:37.741 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:33:37.742 DEBUG:tasks.cephfs.test_mirroring:command returned={ "AsyncMessenger::Worker": [ { "labels": { "id": "0" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "1" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "2" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } } ], "AsyncMessenger::Worker-0": [ { "labels": {}, "counters": { "msgr_recv_messages": 94, "msgr_send_messages": 5, "msgr_recv_bytes": 385488, "msgr_send_bytes": 1136, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.035685981, "msgr_running_send_time": 0.004100999, "msgr_running_recv_time": 0.019059290, "msgr_running_fast_dispatch_time": 0.010637924, "msgr_send_messages_queue_lat": { "avgcount": 5, "sum": 0.004490948, "avgtime": 0.000898189 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 385488, "msgr_send_encrypted_bytes": 1136 } } ], "AsyncMessenger::Worker-1": [ { "labels": {}, "counters": { "msgr_recv_messages": 8, "msgr_send_messages": 97, "msgr_recv_bytes": 1152, "msgr_send_bytes": 664608, "msgr_created_connections": 18, "msgr_active_connections": 3, "msgr_running_total_time": 0.042660590, "msgr_running_send_time": 0.014184783, "msgr_running_recv_time": 0.014356959, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 89, "sum": 0.072395044, "avgtime": 0.000813427 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 1152, "msgr_send_encrypted_bytes": 664608 } } ], "AsyncMessenger::Worker-2": [ { "labels": {}, "counters": { "msgr_recv_messages": 48, "msgr_send_messages": 8, "msgr_recv_bytes": 1940384, "msgr_send_bytes": 1600, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.055197692, "msgr_running_send_time": 0.003961334, "msgr_running_recv_time": 0.093794757, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 8, "sum": 0.003350228, "avgtime": 0.000418778 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 1940384, "msgr_send_encrypted_bytes": 1600 } } ], "cct": [ { "labels": {}, "counters": { "total_workers": 0, "unhealthy_workers": 0 } } ], "cephfs_mirror": [ { "labels": {}, "counters": { "mirrored_filesystems": 0, "mirror_enable_failures": 0 } } ], "mempool": [ { "labels": {}, "counters": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_items": 0, "bluefs_bytes": 0, "bluefs_items": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "buffer_anon_bytes": 149196, "buffer_anon_items": 27, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 4592, "osdmap_items": 37, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "mds_co_bytes": 0, "mds_co_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 } } ], "objecter": [ { "labels": {}, "counters": { "op_active": 0, "op_laggy": 0, "op_send": 0, "op_send_bytes": 0, "op_resend": 0, "op_reply": 0, "op_latency": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "op_inflight": 0, "oplen_avg": { "avgcount": 0, "sum": 0 }, "op": 0, "op_r": 0, "op_w": 0, "op_rmw": 0, "op_pg": 0, "osdop_stat": 0, "osdop_create": 0, "osdop_read": 0, "osdop_write": 0, "osdop_writefull": 0, "osdop_writesame": 0, "osdop_append": 0, "osdop_zero": 0, "osdop_truncate": 0, "osdop_delete": 0, "osdop_mapext": 0, "osdop_sparse_read": 0, "osdop_clonerange": 0, "osdop_getxattr": 0, "osdop_setxattr": 0, "osdop_cmpxattr": 0, "osdop_rmxattr": 0, "osdop_resetxattrs": 0, "osdop_call": 0, "osdop_watch": 0, "osdop_notify": 0, "osdop_src_cmpxattr": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_other": 0, "linger_active": 0, "linger_send": 0, "linger_resend": 0, "linger_ping": 0, "poolop_active": 0, "poolop_send": 0, "poolop_resend": 0, "poolstat_active": 0, "poolstat_send": 0, "poolstat_resend": 0, "statfs_active": 0, "statfs_send": 0, "statfs_resend": 0, "command_active": 0, "command_send": 0, "command_resend": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "osd_sessions": 0, "osd_session_open": 0, "osd_session_close": 0, "osd_laggy": 0, "omap_wr": 0, "omap_rd": 0, "omap_del": 0 } } ], "throttle-msgr_dispatch_throttler-client": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 94, "get_sum": 372643, "get_or_fail_fail": 0, "get_or_fail_success": 94, "take": 0, "take_sum": 0, "put": 94, "put_sum": 372643, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-msgr_dispatch_throttler-radosclient": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 53, "get_sum": 1933421, "get_or_fail_fail": 0, "get_or_fail_success": 53, "take": 0, "take_sum": 0, "put": 53, "put_sum": 1933421, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_bytes": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_ops": [ { "labels": {}, "counters": { "val": 0, "max": 1024, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ] } 2024-07-04T17:33:37.742 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snapshot mirror enable cephfs 2024-07-04T17:33:38.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.013+0000 7efd14c00640 1 -- 172.21.15.42:0/1857580316 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd100a99e0 msgr2=0x7efd1006c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:38.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.013+0000 7efd14c00640 1 --2- 172.21.15.42:0/1857580316 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd100a99e0 0x7efd1006c9d0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7efcfc0097b0 tx=0x7efcfc02a530 comp rx=0 tx=0).stop 2024-07-04T17:33:38.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.013+0000 7efd14c00640 1 -- 172.21.15.42:0/1857580316 shutdown_connections 2024-07-04T17:33:38.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.013+0000 7efd14c00640 1 --2- 172.21.15.42:0/1857580316 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd100a99e0 0x7efd1006c9d0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:38.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.013+0000 7efd14c00640 1 -- 172.21.15.42:0/1857580316 >> 172.21.15.42:0/1857580316 conn(0x7efd10071cf0 msgr2=0x7efd1006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:38.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.013+0000 7efd14c00640 1 -- 172.21.15.42:0/1857580316 shutdown_connections 2024-07-04T17:33:38.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.013+0000 7efd14c00640 1 -- 172.21.15.42:0/1857580316 wait complete. 2024-07-04T17:33:38.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.015+0000 7efd14c00640 1 Processor -- start 2024-07-04T17:33:38.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.016+0000 7efd14c00640 1 -- start start 2024-07-04T17:33:38.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.017+0000 7efd14c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd10074700 0x7efd101d72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:38.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.017+0000 7efd14c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efd1013f4a0 con 0x7efd10074700 2024-07-04T17:33:38.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.017+0000 7efd0de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd10074700 0x7efd101d72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:38.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.017+0000 7efd0de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd10074700 0x7efd101d72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40896/0 (socket says 172.21.15.42:40896) 2024-07-04T17:33:38.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.017+0000 7efd0de00640 1 -- 172.21.15.42:0/504735464 learned_addr learned my addr 172.21.15.42:0/504735464 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:38.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.018+0000 7efd0de00640 1 -- 172.21.15.42:0/504735464 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd100b8730 con 0x7efd10074700 2024-07-04T17:33:38.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.018+0000 7efd0de00640 1 --2- 172.21.15.42:0/504735464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd10074700 0x7efd101d72a0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7efd0000c6b0 tx=0x7efd0000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:38.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.018+0000 7efd06000640 1 -- 172.21.15.42:0/504735464 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd00003aa0 con 0x7efd10074700 2024-07-04T17:33:38.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.018+0000 7efd06000640 1 -- 172.21.15.42:0/504735464 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd00003c60 con 0x7efd10074700 2024-07-04T17:33:38.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.018+0000 7efd06000640 1 -- 172.21.15.42:0/504735464 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd00005070 con 0x7efd10074700 2024-07-04T17:33:38.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.018+0000 7efd14c00640 1 -- 172.21.15.42:0/504735464 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd100baf90 con 0x7efd10074700 2024-07-04T17:33:38.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.019+0000 7efd14c00640 1 -- 172.21.15.42:0/504735464 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd100c1e80 con 0x7efd10074700 2024-07-04T17:33:38.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.020+0000 7efd06000640 1 -- 172.21.15.42:0/504735464 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 43) ==== 49628+0+0 (secure 0 0 0) 0x7efd00005230 con 0x7efd10074700 2024-07-04T17:33:38.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.020+0000 7efd14c00640 1 -- 172.21.15.42:0/504735464 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd1013fbb0 con 0x7efd10074700 2024-07-04T17:33:38.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.020+0000 7efd06000640 1 --2- 172.21.15.42:0/504735464 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7efcf003d540 0x7efcf003fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:38.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.020+0000 7efd06000640 1 -- 172.21.15.42:0/504735464 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 4303+0+0 (secure 0 0 0) 0x7efd00051af0 con 0x7efd10074700 2024-07-04T17:33:38.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.020+0000 7efd0d400640 1 -- 172.21.15.42:0/504735464 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7efcf003d540 msgr2=0x7efcf003fa50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/4130303917 2024-07-04T17:33:38.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.021+0000 7efd0d400640 1 --2- 172.21.15.42:0/504735464 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7efcf003d540 0x7efcf003fa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:33:38.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.030+0000 7efd06000640 1 -- 172.21.15.42:0/504735464 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7efd00005520 con 0x7efd10074700 2024-07-04T17:33:38.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.222+0000 7efd0d400640 1 -- 172.21.15.42:0/504735464 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7efcf003d540 msgr2=0x7efcf003fa50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/4130303917 2024-07-04T17:33:38.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.222+0000 7efd0d400640 1 --2- 172.21.15.42:0/504735464 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7efcf003d540 0x7efcf003fa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:33:38.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.258+0000 7efd14c00640 1 -- 172.21.15.42:0/504735464 --> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] -- mgr_command(tid 0: {"prefix": "fs snapshot mirror enable", "fs_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7efd10069b70 con 0x7efcf003d540 2024-07-04T17:33:38.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.623+0000 7efd0d400640 1 -- 172.21.15.42:0/504735464 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7efcf003d540 msgr2=0x7efcf003fa50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/4130303917 2024-07-04T17:33:38.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:38.623+0000 7efd0d400640 1 --2- 172.21.15.42:0/504735464 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7efcf003d540 0x7efcf003fa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-07-04T17:33:39.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:39.424+0000 7efd0d400640 1 -- 172.21.15.42:0/504735464 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7efcf003d540 msgr2=0x7efcf003fa50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/4130303917 2024-07-04T17:33:39.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:39.424+0000 7efd0d400640 1 --2- 172.21.15.42:0/504735464 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7efcf003d540 0x7efcf003fa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-07-04T17:33:41.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:41.026+0000 7efd0d400640 1 -- 172.21.15.42:0/504735464 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7efcf003d540 msgr2=0x7efcf003fa50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/4130303917 2024-07-04T17:33:41.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:41.026+0000 7efd0d400640 1 --2- 172.21.15.42:0/504735464 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7efcf003d540 0x7efcf003fa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-07-04T17:33:42.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:42.118+0000 7efd06000640 1 -- 172.21.15.42:0/504735464 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 44) ==== 49135+0+0 (secure 0 0 0) 0x7efd0000a660 con 0x7efd10074700 2024-07-04T17:33:42.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:42.118+0000 7efd06000640 1 -- 172.21.15.42:0/504735464 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7efcf003d540 msgr2=0x7efcf003fa50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:33:42.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:42.118+0000 7efd06000640 1 --2- 172.21.15.42:0/504735464 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7efcf003d540 0x7efcf003fa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:42.210 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:33:42.209+0000 7f2fbe400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:33:42.211 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:33:42.209+0000 7f2fbe400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:33:42.211 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:33:42.209+0000 7f2fbe400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:33:42.211 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:33:42.209+0000 7f2fbe400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:33:42.211 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:33:42.209+0000 7f2fbe400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:33:42.212 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:33:42.210+0000 7f2fc2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:33:42.212 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:33:42.210+0000 7f2fc2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:33:42.212 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:33:42.210+0000 7f2fc2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:33:42.213 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:33:42.210+0000 7f2fc2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:33:42.213 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:33:42.210+0000 7f2fc2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:33:42.231 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:33:42.230+0000 7f2fc2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:33:42.231 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:33:42.230+0000 7f2fc2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:33:42.231 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:33:42.230+0000 7f2fc2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:33:42.231 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:33:42.230+0000 7f2fc2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:33:42.232 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:33:42.230+0000 7f2fc2a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:33:42.232 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:33:42.230+0000 7f2fbe400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:33:42.232 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:33:42.230+0000 7f2fbe400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:33:42.232 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:33:42.230+0000 7f2fbe400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:33:42.232 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:33:42.230+0000 7f2fbe400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:33:42.232 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:33:42.230+0000 7f2fbe400640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:33:43.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:43.123+0000 7efd06000640 1 -- 172.21.15.42:0/504735464 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mgrmap(e 45) ==== 49262+0+0 (secure 0 0 0) 0x7efd000381f0 con 0x7efd10074700 2024-07-04T17:33:43.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:43.123+0000 7efd06000640 1 --2- 172.21.15.42:0/504735464 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7efcf0046b80 0x7efcf0048fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:43.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:43.123+0000 7efd06000640 1 -- 172.21.15.42:0/504735464 --> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] -- mgr_command(tid 0: {"prefix": "fs snapshot mirror enable", "fs_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7efd00014e80 con 0x7efcf0046b80 2024-07-04T17:33:43.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:43.127+0000 7efd0d400640 1 --2- 172.21.15.42:0/504735464 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7efcf0046b80 0x7efcf0048fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:43.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:43.140+0000 7efd0d400640 1 --2- 172.21.15.42:0/504735464 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7efcf0046b80 0x7efcf0048fa0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efcfc034000 tx=0x7efcfc004730 comp rx=0 tx=0).ready entity=mgr.5658 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:44.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:44.123+0000 7efd06000640 1 -- 172.21.15.42:0/504735464 <== mgr.5658 v2:172.21.15.42:6834/1310962821 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7efd00014e80 con 0x7efcf0046b80 2024-07-04T17:33:44.155 INFO:teuthology.orchestra.run.smithi042.stdout:{} 2024-07-04T17:33:44.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:44.127+0000 7efd14c00640 1 -- 172.21.15.42:0/504735464 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7efcf0046b80 msgr2=0x7efcf0048fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:44.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:44.127+0000 7efd14c00640 1 --2- 172.21.15.42:0/504735464 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7efcf0046b80 0x7efcf0048fa0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efcfc034000 tx=0x7efcfc004730 comp rx=0 tx=0).stop 2024-07-04T17:33:44.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:44.127+0000 7efd14c00640 1 -- 172.21.15.42:0/504735464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd10074700 msgr2=0x7efd101d72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:44.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:44.127+0000 7efd14c00640 1 --2- 172.21.15.42:0/504735464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd10074700 0x7efd101d72a0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7efd0000c6b0 tx=0x7efd0000cb70 comp rx=0 tx=0).stop 2024-07-04T17:33:44.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:44.128+0000 7efd14c00640 1 -- 172.21.15.42:0/504735464 shutdown_connections 2024-07-04T17:33:44.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:44.128+0000 7efd14c00640 1 --2- 172.21.15.42:0/504735464 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7efcf0046b80 0x7efcf0048fa0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:44.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:44.128+0000 7efd14c00640 1 --2- 172.21.15.42:0/504735464 >> [v2:172.21.15.42:6834/4130303917,v1:172.21.15.42:6835/4130303917] conn(0x7efcf003d540 0x7efcf003fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:44.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:44.128+0000 7efd14c00640 1 --2- 172.21.15.42:0/504735464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd10074700 0x7efd101d72a0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:44.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:44.128+0000 7efd14c00640 1 -- 172.21.15.42:0/504735464 >> 172.21.15.42:0/504735464 conn(0x7efd10071cf0 msgr2=0x7efd100b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:44.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:44.128+0000 7efd14c00640 1 -- 172.21.15.42:0/504735464 shutdown_connections 2024-07-04T17:33:44.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:44.128+0000 7efd14c00640 1 -- 172.21.15.42:0/504735464 wait complete. 2024-07-04T17:33:54.160 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:33:54.160 DEBUG:teuthology.orchestra.run.smithi042:mirror status for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@10 2024-07-04T17:33:54.410 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:33:54.410 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_inst": "172.21.15.42:0/2238986031", 2024-07-04T17:33:54.410 INFO:teuthology.orchestra.run.smithi042.stdout: "peers": {}, 2024-07-04T17:33:54.410 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_dirs": { 2024-07-04T17:33:54.411 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_count": 0 2024-07-04T17:33:54.411 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.411 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:33:54.428 DEBUG:tasks.cephfs.test_mirroring:command returned={ "rados_inst": "172.21.15.42:0/2238986031", "peers": {}, "snap_dirs": { "dir_count": 0 } } 2024-07-04T17:33:54.428 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:33:54.428 DEBUG:teuthology.orchestra.run.smithi042:counter dump for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok counter dump 2024-07-04T17:33:54.671 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:33:54.671 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker": [ 2024-07-04T17:33:54.671 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:54.671 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:33:54.671 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "0" 2024-07-04T17:33:54.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:54.671 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:54.672 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:33:54.672 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:33:54.672 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:54.672 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:54.672 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:33:54.672 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "1" 2024-07-04T17:33:54.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:54.672 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:54.672 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:33:54.672 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:33:54.672 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:54.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:54.673 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:33:54.673 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "2" 2024-07-04T17:33:54.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:54.673 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:54.673 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:33:54.673 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:33:54.673 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.673 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.673 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:54.673 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": [ 2024-07-04T17:33:54.674 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:54.674 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:54.674 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:54.674 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 95, 2024-07-04T17:33:54.674 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 5, 2024-07-04T17:33:54.674 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 390384, 2024-07-04T17:33:54.674 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 1136, 2024-07-04T17:33:54.674 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:33:54.674 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:33:54.674 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.036248320, 2024-07-04T17:33:54.674 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.004243662, 2024-07-04T17:33:54.675 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.019280115, 2024-07-04T17:33:54.675 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.010775880, 2024-07-04T17:33:54.675 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:33:54.675 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 5, 2024-07-04T17:33:54.675 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.004490948, 2024-07-04T17:33:54.675 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000898189 2024-07-04T17:33:54.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:54.675 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:33:54.675 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:33:54.675 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:33:54.675 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:33:54.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:54.676 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 390384, 2024-07-04T17:33:54.676 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 1136 2024-07-04T17:33:54.676 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.676 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.676 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:54.676 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": [ 2024-07-04T17:33:54.676 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:54.676 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:54.676 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:54.676 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 9, 2024-07-04T17:33:54.676 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 101, 2024-07-04T17:33:54.676 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 1296, 2024-07-04T17:33:54.676 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 745760, 2024-07-04T17:33:54.676 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 19, 2024-07-04T17:33:54.677 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 4, 2024-07-04T17:33:54.677 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.047089833, 2024-07-04T17:33:54.677 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.014883619, 2024-07-04T17:33:54.677 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.016278387, 2024-07-04T17:33:54.677 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:33:54.677 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:33:54.677 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 93, 2024-07-04T17:33:54.677 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.084642050, 2024-07-04T17:33:54.677 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000910129 2024-07-04T17:33:54.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:54.677 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:33:54.677 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:33:54.677 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:33:54.677 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:33:54.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:54.678 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 1296, 2024-07-04T17:33:54.678 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 745760 2024-07-04T17:33:54.678 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.678 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.678 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:54.678 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": [ 2024-07-04T17:33:54.678 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:54.678 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:54.678 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:54.678 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 52, 2024-07-04T17:33:54.678 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 8, 2024-07-04T17:33:54.678 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 2138624, 2024-07-04T17:33:54.678 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 1600, 2024-07-04T17:33:54.679 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:33:54.679 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:33:54.679 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.059358227, 2024-07-04T17:33:54.679 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.004105920, 2024-07-04T17:33:54.679 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.101541947, 2024-07-04T17:33:54.679 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:33:54.679 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:33:54.679 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 8, 2024-07-04T17:33:54.679 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.003350228, 2024-07-04T17:33:54.679 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000418778 2024-07-04T17:33:54.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:54.679 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:33:54.679 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:33:54.679 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:33:54.679 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:33:54.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:54.680 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 2138624, 2024-07-04T17:33:54.680 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 1600 2024-07-04T17:33:54.680 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.680 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.680 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:54.680 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": [ 2024-07-04T17:33:54.680 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:54.680 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:54.680 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:54.680 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-07-04T17:33:54.680 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-07-04T17:33:54.680 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.680 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.680 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:54.680 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror": [ 2024-07-04T17:33:54.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:54.681 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:54.681 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:54.681 INFO:teuthology.orchestra.run.smithi042.stdout: "mirrored_filesystems": 1, 2024-07-04T17:33:54.681 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_enable_failures": 0 2024-07-04T17:33:54.681 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.681 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.681 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:54.681 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_mirrored_filesystems": [ 2024-07-04T17:33:54.681 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:54.681 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:33:54.681 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem": "cephfs" 2024-07-04T17:33:54.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:54.681 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:54.681 INFO:teuthology.orchestra.run.smithi042.stdout: "mirroring_peers": 0, 2024-07-04T17:33:54.681 INFO:teuthology.orchestra.run.smithi042.stdout: "directory_count": 0 2024-07-04T17:33:54.682 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.682 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.682 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:54.682 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": [ 2024-07-04T17:33:54.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:54.682 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:54.682 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:54.682 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-07-04T17:33:54.682 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-07-04T17:33:54.682 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T17:33:54.682 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-07-04T17:33:54.682 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T17:33:54.682 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-07-04T17:33:54.683 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T17:33:54.683 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T17:33:54.683 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T17:33:54.683 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T17:33:54.683 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T17:33:54.683 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-07-04T17:33:54.683 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T17:33:54.683 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T17:33:54.683 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-07-04T17:33:54.683 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-07-04T17:33:54.683 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-07-04T17:33:54.683 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-07-04T17:33:54.683 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T17:33:54.683 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T17:33:54.683 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T17:33:54.683 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T17:33:54.684 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T17:33:54.684 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-07-04T17:33:54.684 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-07-04T17:33:54.684 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-07-04T17:33:54.684 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T17:33:54.684 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T17:33:54.684 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-07-04T17:33:54.684 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-07-04T17:33:54.684 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-07-04T17:33:54.684 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-07-04T17:33:54.684 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T17:33:54.684 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-07-04T17:33:54.684 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T17:33:54.684 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-07-04T17:33:54.684 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 175459, 2024-07-04T17:33:54.685 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 53, 2024-07-04T17:33:54.685 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-07-04T17:33:54.685 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-07-04T17:33:54.685 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-07-04T17:33:54.685 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-07-04T17:33:54.685 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-07-04T17:33:54.685 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-07-04T17:33:54.685 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-07-04T17:33:54.685 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-07-04T17:33:54.685 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 18720, 2024-07-04T17:33:54.685 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 169, 2024-07-04T17:33:54.685 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T17:33:54.685 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-07-04T17:33:54.686 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-07-04T17:33:54.686 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-07-04T17:33:54.686 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 0, 2024-07-04T17:33:54.686 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 0, 2024-07-04T17:33:54.686 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-07-04T17:33:54.686 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-07-04T17:33:54.686 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-07-04T17:33:54.686 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-07-04T17:33:54.686 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.686 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.686 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:54.686 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter": [ 2024-07-04T17:33:54.686 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:54.687 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:54.687 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:54.687 INFO:teuthology.orchestra.run.smithi042.stdout: "op_active": 0, 2024-07-04T17:33:54.687 INFO:teuthology.orchestra.run.smithi042.stdout: "op_laggy": 0, 2024-07-04T17:33:54.687 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send": 0, 2024-07-04T17:33:54.687 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send_bytes": 0, 2024-07-04T17:33:54.687 INFO:teuthology.orchestra.run.smithi042.stdout: "op_resend": 0, 2024-07-04T17:33:54.687 INFO:teuthology.orchestra.run.smithi042.stdout: "op_reply": 0, 2024-07-04T17:33:54.687 INFO:teuthology.orchestra.run.smithi042.stdout: "op_latency": { 2024-07-04T17:33:54.687 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:33:54.687 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:33:54.687 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:33:54.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:54.687 INFO:teuthology.orchestra.run.smithi042.stdout: "op_inflight": 0, 2024-07-04T17:33:54.688 INFO:teuthology.orchestra.run.smithi042.stdout: "oplen_avg": { 2024-07-04T17:33:54.688 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:33:54.688 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0 2024-07-04T17:33:54.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:54.688 INFO:teuthology.orchestra.run.smithi042.stdout: "op": 0, 2024-07-04T17:33:54.688 INFO:teuthology.orchestra.run.smithi042.stdout: "op_r": 0, 2024-07-04T17:33:54.688 INFO:teuthology.orchestra.run.smithi042.stdout: "op_w": 0, 2024-07-04T17:33:54.688 INFO:teuthology.orchestra.run.smithi042.stdout: "op_rmw": 0, 2024-07-04T17:33:54.688 INFO:teuthology.orchestra.run.smithi042.stdout: "op_pg": 0, 2024-07-04T17:33:54.688 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_stat": 0, 2024-07-04T17:33:54.688 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_create": 0, 2024-07-04T17:33:54.688 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_read": 0, 2024-07-04T17:33:54.688 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_write": 0, 2024-07-04T17:33:54.688 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writefull": 0, 2024-07-04T17:33:54.689 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writesame": 0, 2024-07-04T17:33:54.689 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_append": 0, 2024-07-04T17:33:54.689 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_zero": 0, 2024-07-04T17:33:54.689 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_truncate": 0, 2024-07-04T17:33:54.689 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_delete": 0, 2024-07-04T17:33:54.689 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_mapext": 0, 2024-07-04T17:33:54.689 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_sparse_read": 0, 2024-07-04T17:33:54.689 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_clonerange": 0, 2024-07-04T17:33:54.689 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_getxattr": 0, 2024-07-04T17:33:54.689 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_setxattr": 0, 2024-07-04T17:33:54.689 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_cmpxattr": 0, 2024-07-04T17:33:54.689 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_rmxattr": 0, 2024-07-04T17:33:54.689 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_resetxattrs": 0, 2024-07-04T17:33:54.690 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_call": 0, 2024-07-04T17:33:54.690 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_watch": 0, 2024-07-04T17:33:54.690 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_notify": 0, 2024-07-04T17:33:54.690 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T17:33:54.690 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls": 0, 2024-07-04T17:33:54.690 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls_filter": 0, 2024-07-04T17:33:54.690 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_other": 0, 2024-07-04T17:33:54.690 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_active": 0, 2024-07-04T17:33:54.690 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_send": 0, 2024-07-04T17:33:54.690 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_resend": 0, 2024-07-04T17:33:54.690 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_ping": 0, 2024-07-04T17:33:54.690 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_active": 0, 2024-07-04T17:33:54.690 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_send": 0, 2024-07-04T17:33:54.691 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_resend": 0, 2024-07-04T17:33:54.691 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_active": 0, 2024-07-04T17:33:54.691 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_send": 0, 2024-07-04T17:33:54.691 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_resend": 0, 2024-07-04T17:33:54.691 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_active": 0, 2024-07-04T17:33:54.691 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_send": 0, 2024-07-04T17:33:54.691 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_resend": 0, 2024-07-04T17:33:54.691 INFO:teuthology.orchestra.run.smithi042.stdout: "command_active": 0, 2024-07-04T17:33:54.691 INFO:teuthology.orchestra.run.smithi042.stdout: "command_send": 0, 2024-07-04T17:33:54.691 INFO:teuthology.orchestra.run.smithi042.stdout: "command_resend": 0, 2024-07-04T17:33:54.691 INFO:teuthology.orchestra.run.smithi042.stdout: "map_epoch": 0, 2024-07-04T17:33:54.691 INFO:teuthology.orchestra.run.smithi042.stdout: "map_full": 0, 2024-07-04T17:33:54.691 INFO:teuthology.orchestra.run.smithi042.stdout: "map_inc": 0, 2024-07-04T17:33:54.692 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_sessions": 0, 2024-07-04T17:33:54.692 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_open": 0, 2024-07-04T17:33:54.692 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_close": 0, 2024-07-04T17:33:54.692 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_laggy": 0, 2024-07-04T17:33:54.692 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_wr": 0, 2024-07-04T17:33:54.692 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_rd": 0, 2024-07-04T17:33:54.692 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_del": 0 2024-07-04T17:33:54.692 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.692 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.692 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:54.692 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-client": [ 2024-07-04T17:33:54.692 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:54.692 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:54.692 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:54.692 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:33:54.693 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:33:54.693 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:33:54.693 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 95, 2024-07-04T17:33:54.693 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 377397, 2024-07-04T17:33:54.693 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:33:54.693 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 95, 2024-07-04T17:33:54.693 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:33:54.693 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:33:54.693 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 95, 2024-07-04T17:33:54.693 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 377397, 2024-07-04T17:33:54.693 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:33:54.693 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:33:54.693 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:33:54.694 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:33:54.694 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.694 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.694 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.694 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:54.694 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-radosclient": [ 2024-07-04T17:33:54.694 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:54.694 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:54.694 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:54.694 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:33:54.694 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:33:54.694 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:33:54.695 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 58, 2024-07-04T17:33:54.695 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 2131137, 2024-07-04T17:33:54.695 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:33:54.695 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 58, 2024-07-04T17:33:54.695 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:33:54.695 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:33:54.695 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 58, 2024-07-04T17:33:54.695 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 2131137, 2024-07-04T17:33:54.695 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:33:54.695 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:33:54.695 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:33:54.695 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:33:54.695 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.695 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.696 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.696 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:54.696 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_bytes": [ 2024-07-04T17:33:54.696 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:54.696 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:54.696 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:54.696 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:33:54.696 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:33:54.696 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:33:54.696 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:33:54.696 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:33:54.696 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:33:54.696 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:33:54.697 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:33:54.697 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:33:54.697 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:33:54.697 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:33:54.697 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:33:54.697 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:33:54.697 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:33:54.697 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:33:54.697 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.697 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.697 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.697 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:54.698 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_ops": [ 2024-07-04T17:33:54.698 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:54.698 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:54.698 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:54.698 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:33:54.698 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1024, 2024-07-04T17:33:54.698 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:33:54.698 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:33:54.698 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:33:54.698 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:33:54.698 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:33:54.698 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:33:54.698 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:33:54.699 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:33:54.699 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:33:54.699 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:33:54.699 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:33:54.699 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:33:54.699 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:33:54.699 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.699 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.699 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.699 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-07-04T17:33:54.699 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:33:54.700 DEBUG:tasks.cephfs.test_mirroring:command returned={ "AsyncMessenger::Worker": [ { "labels": { "id": "0" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "1" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "2" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } } ], "AsyncMessenger::Worker-0": [ { "labels": {}, "counters": { "msgr_recv_messages": 95, "msgr_send_messages": 5, "msgr_recv_bytes": 390384, "msgr_send_bytes": 1136, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.036248320, "msgr_running_send_time": 0.004243662, "msgr_running_recv_time": 0.019280115, "msgr_running_fast_dispatch_time": 0.010775880, "msgr_send_messages_queue_lat": { "avgcount": 5, "sum": 0.004490948, "avgtime": 0.000898189 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 390384, "msgr_send_encrypted_bytes": 1136 } } ], "AsyncMessenger::Worker-1": [ { "labels": {}, "counters": { "msgr_recv_messages": 9, "msgr_send_messages": 101, "msgr_recv_bytes": 1296, "msgr_send_bytes": 745760, "msgr_created_connections": 19, "msgr_active_connections": 4, "msgr_running_total_time": 0.047089833, "msgr_running_send_time": 0.014883619, "msgr_running_recv_time": 0.016278387, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 93, "sum": 0.084642050, "avgtime": 0.000910129 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 1296, "msgr_send_encrypted_bytes": 745760 } } ], "AsyncMessenger::Worker-2": [ { "labels": {}, "counters": { "msgr_recv_messages": 52, "msgr_send_messages": 8, "msgr_recv_bytes": 2138624, "msgr_send_bytes": 1600, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.059358227, "msgr_running_send_time": 0.004105920, "msgr_running_recv_time": 0.101541947, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 8, "sum": 0.003350228, "avgtime": 0.000418778 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 2138624, "msgr_send_encrypted_bytes": 1600 } } ], "cct": [ { "labels": {}, "counters": { "total_workers": 0, "unhealthy_workers": 0 } } ], "cephfs_mirror": [ { "labels": {}, "counters": { "mirrored_filesystems": 1, "mirror_enable_failures": 0 } } ], "cephfs_mirror_mirrored_filesystems": [ { "labels": { "filesystem": "cephfs" }, "counters": { "mirroring_peers": 0, "directory_count": 0 } } ], "mempool": [ { "labels": {}, "counters": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_items": 0, "bluefs_bytes": 0, "bluefs_items": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "buffer_anon_bytes": 175459, "buffer_anon_items": 53, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 18720, "osdmap_items": 169, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "mds_co_bytes": 0, "mds_co_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 } } ], "objecter": [ { "labels": {}, "counters": { "op_active": 0, "op_laggy": 0, "op_send": 0, "op_send_bytes": 0, "op_resend": 0, "op_reply": 0, "op_latency": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "op_inflight": 0, "oplen_avg": { "avgcount": 0, "sum": 0 }, "op": 0, "op_r": 0, "op_w": 0, "op_rmw": 0, "op_pg": 0, "osdop_stat": 0, "osdop_create": 0, "osdop_read": 0, "osdop_write": 0, "osdop_writefull": 0, "osdop_writesame": 0, "osdop_append": 0, "osdop_zero": 0, "osdop_truncate": 0, "osdop_delete": 0, "osdop_mapext": 0, "osdop_sparse_read": 0, "osdop_clonerange": 0, "osdop_getxattr": 0, "osdop_setxattr": 0, "osdop_cmpxattr": 0, "osdop_rmxattr": 0, "osdop_resetxattrs": 0, "osdop_call": 0, "osdop_watch": 0, "osdop_notify": 0, "osdop_src_cmpxattr": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_other": 0, "linger_active": 0, "linger_send": 0, "linger_resend": 0, "linger_ping": 0, "poolop_active": 0, "poolop_send": 0, "poolop_resend": 0, "poolstat_active": 0, "poolstat_send": 0, "poolstat_resend": 0, "statfs_active": 0, "statfs_send": 0, "statfs_resend": 0, "command_active": 0, "command_send": 0, "command_resend": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "osd_sessions": 0, "osd_session_open": 0, "osd_session_close": 0, "osd_laggy": 0, "omap_wr": 0, "omap_rd": 0, "omap_del": 0 } } ], "throttle-msgr_dispatch_throttler-client": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 95, "get_sum": 377397, "get_or_fail_fail": 0, "get_or_fail_success": 95, "take": 0, "take_sum": 0, "put": 95, "put_sum": 377397, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-msgr_dispatch_throttler-radosclient": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 58, "get_sum": 2131137, "get_or_fail_fail": 0, "get_or_fail_success": 58, "take": 0, "take_sum": 0, "put": 58, "put_sum": 2131137, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_bytes": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_ops": [ { "labels": {}, "counters": { "val": 0, "max": 1024, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ] } 2024-07-04T17:33:54.701 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:33:54.701 DEBUG:teuthology.orchestra.run.smithi042:counter dump for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok counter dump 2024-07-04T17:33:54.943 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:33:54.943 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker": [ 2024-07-04T17:33:54.943 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:54.943 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:33:54.943 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "0" 2024-07-04T17:33:54.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:54.943 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:54.943 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:33:54.944 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:33:54.944 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:54.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:54.944 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:33:54.944 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "1" 2024-07-04T17:33:54.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:54.944 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:54.944 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:33:54.944 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:33:54.944 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:54.944 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:54.944 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:33:54.944 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "2" 2024-07-04T17:33:54.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:54.945 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:54.945 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:33:54.945 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:33:54.945 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.945 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.945 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:54.945 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": [ 2024-07-04T17:33:54.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:54.945 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:54.945 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:54.945 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 95, 2024-07-04T17:33:54.945 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 5, 2024-07-04T17:33:54.945 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 390384, 2024-07-04T17:33:54.945 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 1136, 2024-07-04T17:33:54.945 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:33:54.946 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:33:54.946 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.036248320, 2024-07-04T17:33:54.946 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.004243662, 2024-07-04T17:33:54.946 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.019280115, 2024-07-04T17:33:54.946 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.010775880, 2024-07-04T17:33:54.946 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:33:54.946 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 5, 2024-07-04T17:33:54.946 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.004490948, 2024-07-04T17:33:54.946 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000898189 2024-07-04T17:33:54.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:54.946 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:33:54.946 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:33:54.947 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:33:54.947 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:33:54.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:54.947 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 390384, 2024-07-04T17:33:54.947 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 1136 2024-07-04T17:33:54.947 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.947 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.947 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:54.947 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": [ 2024-07-04T17:33:54.947 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:54.947 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:54.947 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:54.947 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 9, 2024-07-04T17:33:54.947 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 101, 2024-07-04T17:33:54.947 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 1296, 2024-07-04T17:33:54.948 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 745760, 2024-07-04T17:33:54.948 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 19, 2024-07-04T17:33:54.948 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 4, 2024-07-04T17:33:54.948 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.047089833, 2024-07-04T17:33:54.948 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.014883619, 2024-07-04T17:33:54.948 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.016278387, 2024-07-04T17:33:54.948 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:33:54.948 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:33:54.948 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 93, 2024-07-04T17:33:54.948 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.084642050, 2024-07-04T17:33:54.948 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000910129 2024-07-04T17:33:54.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:54.948 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:33:54.948 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:33:54.948 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:33:54.948 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:33:54.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:54.949 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 1296, 2024-07-04T17:33:54.949 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 745760 2024-07-04T17:33:54.949 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.949 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.949 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:54.949 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": [ 2024-07-04T17:33:54.949 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:54.949 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:54.949 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:54.949 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 52, 2024-07-04T17:33:54.949 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 8, 2024-07-04T17:33:54.949 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 2138624, 2024-07-04T17:33:54.949 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 1600, 2024-07-04T17:33:54.949 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:33:54.950 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:33:54.950 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.059358227, 2024-07-04T17:33:54.950 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.004105920, 2024-07-04T17:33:54.950 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.101541947, 2024-07-04T17:33:54.950 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:33:54.950 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:33:54.950 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 8, 2024-07-04T17:33:54.950 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.003350228, 2024-07-04T17:33:54.950 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000418778 2024-07-04T17:33:54.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:54.950 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:33:54.950 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:33:54.950 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:33:54.951 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:33:54.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:54.951 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 2138624, 2024-07-04T17:33:54.951 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 1600 2024-07-04T17:33:54.951 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.951 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.951 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:54.951 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": [ 2024-07-04T17:33:54.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:54.951 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:54.951 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:54.952 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-07-04T17:33:54.952 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-07-04T17:33:54.952 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.952 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.952 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:54.952 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror": [ 2024-07-04T17:33:54.952 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:54.952 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:54.952 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:54.952 INFO:teuthology.orchestra.run.smithi042.stdout: "mirrored_filesystems": 1, 2024-07-04T17:33:54.952 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_enable_failures": 0 2024-07-04T17:33:54.952 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.952 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.952 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:54.953 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_mirrored_filesystems": [ 2024-07-04T17:33:54.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:54.953 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:33:54.953 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem": "cephfs" 2024-07-04T17:33:54.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:54.953 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:54.953 INFO:teuthology.orchestra.run.smithi042.stdout: "mirroring_peers": 0, 2024-07-04T17:33:54.953 INFO:teuthology.orchestra.run.smithi042.stdout: "directory_count": 0 2024-07-04T17:33:54.953 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.953 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.953 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:54.953 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": [ 2024-07-04T17:33:54.954 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:54.954 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:54.954 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:54.954 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-07-04T17:33:54.954 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-07-04T17:33:54.954 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T17:33:54.954 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-07-04T17:33:54.954 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T17:33:54.954 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-07-04T17:33:54.954 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T17:33:54.954 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T17:33:54.954 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T17:33:54.955 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T17:33:54.955 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T17:33:54.955 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-07-04T17:33:54.955 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T17:33:54.955 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T17:33:54.955 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-07-04T17:33:54.955 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-07-04T17:33:54.955 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-07-04T17:33:54.955 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-07-04T17:33:54.955 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T17:33:54.955 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T17:33:54.955 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T17:33:54.955 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T17:33:54.956 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T17:33:54.956 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-07-04T17:33:54.956 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-07-04T17:33:54.956 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-07-04T17:33:54.956 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T17:33:54.956 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T17:33:54.956 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-07-04T17:33:54.956 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-07-04T17:33:54.956 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-07-04T17:33:54.956 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-07-04T17:33:54.956 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T17:33:54.956 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-07-04T17:33:54.956 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T17:33:54.956 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-07-04T17:33:54.957 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 175459, 2024-07-04T17:33:54.957 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 53, 2024-07-04T17:33:54.957 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-07-04T17:33:54.957 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-07-04T17:33:54.957 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-07-04T17:33:54.957 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-07-04T17:33:54.957 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-07-04T17:33:54.957 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-07-04T17:33:54.957 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-07-04T17:33:54.957 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-07-04T17:33:54.957 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 18720, 2024-07-04T17:33:54.957 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 169, 2024-07-04T17:33:54.957 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T17:33:54.957 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-07-04T17:33:54.958 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-07-04T17:33:54.958 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-07-04T17:33:54.958 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 0, 2024-07-04T17:33:54.958 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 0, 2024-07-04T17:33:54.958 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-07-04T17:33:54.958 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-07-04T17:33:54.958 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-07-04T17:33:54.958 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-07-04T17:33:54.958 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.958 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.958 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:54.958 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter": [ 2024-07-04T17:33:54.958 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:54.959 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:54.959 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:54.959 INFO:teuthology.orchestra.run.smithi042.stdout: "op_active": 0, 2024-07-04T17:33:54.959 INFO:teuthology.orchestra.run.smithi042.stdout: "op_laggy": 0, 2024-07-04T17:33:54.959 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send": 0, 2024-07-04T17:33:54.959 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send_bytes": 0, 2024-07-04T17:33:54.959 INFO:teuthology.orchestra.run.smithi042.stdout: "op_resend": 0, 2024-07-04T17:33:54.959 INFO:teuthology.orchestra.run.smithi042.stdout: "op_reply": 0, 2024-07-04T17:33:54.959 INFO:teuthology.orchestra.run.smithi042.stdout: "op_latency": { 2024-07-04T17:33:54.959 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:33:54.959 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:33:54.959 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:33:54.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:54.960 INFO:teuthology.orchestra.run.smithi042.stdout: "op_inflight": 0, 2024-07-04T17:33:54.960 INFO:teuthology.orchestra.run.smithi042.stdout: "oplen_avg": { 2024-07-04T17:33:54.960 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:33:54.960 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0 2024-07-04T17:33:54.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:33:54.960 INFO:teuthology.orchestra.run.smithi042.stdout: "op": 0, 2024-07-04T17:33:54.960 INFO:teuthology.orchestra.run.smithi042.stdout: "op_r": 0, 2024-07-04T17:33:54.960 INFO:teuthology.orchestra.run.smithi042.stdout: "op_w": 0, 2024-07-04T17:33:54.960 INFO:teuthology.orchestra.run.smithi042.stdout: "op_rmw": 0, 2024-07-04T17:33:54.960 INFO:teuthology.orchestra.run.smithi042.stdout: "op_pg": 0, 2024-07-04T17:33:54.960 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_stat": 0, 2024-07-04T17:33:54.960 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_create": 0, 2024-07-04T17:33:54.961 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_read": 0, 2024-07-04T17:33:54.961 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_write": 0, 2024-07-04T17:33:54.961 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writefull": 0, 2024-07-04T17:33:54.961 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writesame": 0, 2024-07-04T17:33:54.961 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_append": 0, 2024-07-04T17:33:54.961 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_zero": 0, 2024-07-04T17:33:54.961 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_truncate": 0, 2024-07-04T17:33:54.961 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_delete": 0, 2024-07-04T17:33:54.961 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_mapext": 0, 2024-07-04T17:33:54.961 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_sparse_read": 0, 2024-07-04T17:33:54.961 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_clonerange": 0, 2024-07-04T17:33:54.961 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_getxattr": 0, 2024-07-04T17:33:54.962 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_setxattr": 0, 2024-07-04T17:33:54.962 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_cmpxattr": 0, 2024-07-04T17:33:54.962 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_rmxattr": 0, 2024-07-04T17:33:54.962 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_resetxattrs": 0, 2024-07-04T17:33:54.962 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_call": 0, 2024-07-04T17:33:54.962 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_watch": 0, 2024-07-04T17:33:54.962 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_notify": 0, 2024-07-04T17:33:54.962 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T17:33:54.962 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls": 0, 2024-07-04T17:33:54.962 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls_filter": 0, 2024-07-04T17:33:54.962 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_other": 0, 2024-07-04T17:33:54.962 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_active": 0, 2024-07-04T17:33:54.962 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_send": 0, 2024-07-04T17:33:54.963 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_resend": 0, 2024-07-04T17:33:54.963 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_ping": 0, 2024-07-04T17:33:54.963 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_active": 0, 2024-07-04T17:33:54.963 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_send": 0, 2024-07-04T17:33:54.963 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_resend": 0, 2024-07-04T17:33:54.963 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_active": 0, 2024-07-04T17:33:54.963 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_send": 0, 2024-07-04T17:33:54.963 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_resend": 0, 2024-07-04T17:33:54.963 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_active": 0, 2024-07-04T17:33:54.963 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_send": 0, 2024-07-04T17:33:54.963 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_resend": 0, 2024-07-04T17:33:54.963 INFO:teuthology.orchestra.run.smithi042.stdout: "command_active": 0, 2024-07-04T17:33:54.963 INFO:teuthology.orchestra.run.smithi042.stdout: "command_send": 0, 2024-07-04T17:33:54.963 INFO:teuthology.orchestra.run.smithi042.stdout: "command_resend": 0, 2024-07-04T17:33:54.964 INFO:teuthology.orchestra.run.smithi042.stdout: "map_epoch": 0, 2024-07-04T17:33:54.964 INFO:teuthology.orchestra.run.smithi042.stdout: "map_full": 0, 2024-07-04T17:33:54.964 INFO:teuthology.orchestra.run.smithi042.stdout: "map_inc": 0, 2024-07-04T17:33:54.964 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_sessions": 0, 2024-07-04T17:33:54.964 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_open": 0, 2024-07-04T17:33:54.964 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_close": 0, 2024-07-04T17:33:54.964 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_laggy": 0, 2024-07-04T17:33:54.964 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_wr": 0, 2024-07-04T17:33:54.964 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_rd": 0, 2024-07-04T17:33:54.964 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_del": 0 2024-07-04T17:33:54.964 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.964 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.964 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:54.964 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-client": [ 2024-07-04T17:33:54.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:54.965 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:54.965 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:54.965 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:33:54.965 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:33:54.965 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:33:54.965 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 95, 2024-07-04T17:33:54.965 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 377397, 2024-07-04T17:33:54.965 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:33:54.965 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 95, 2024-07-04T17:33:54.965 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:33:54.965 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:33:54.965 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 95, 2024-07-04T17:33:54.965 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 377397, 2024-07-04T17:33:54.965 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:33:54.966 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:33:54.966 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:33:54.966 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:33:54.966 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.966 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.966 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.966 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:54.966 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-radosclient": [ 2024-07-04T17:33:54.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:54.966 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:54.966 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:54.966 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:33:54.966 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:33:54.966 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:33:54.967 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 58, 2024-07-04T17:33:54.967 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 2131137, 2024-07-04T17:33:54.967 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:33:54.967 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 58, 2024-07-04T17:33:54.967 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:33:54.967 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:33:54.967 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 58, 2024-07-04T17:33:54.967 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 2131137, 2024-07-04T17:33:54.967 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:33:54.967 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:33:54.967 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:33:54.967 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:33:54.967 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.968 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.968 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.968 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:54.968 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_bytes": [ 2024-07-04T17:33:54.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:54.968 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:54.968 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:54.968 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:33:54.968 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:33:54.968 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:33:54.968 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:33:54.968 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:33:54.968 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:33:54.968 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:33:54.968 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:33:54.969 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:33:54.969 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:33:54.969 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:33:54.969 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:33:54.969 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:33:54.969 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:33:54.969 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:33:54.969 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.969 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.969 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.969 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:33:54.969 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_ops": [ 2024-07-04T17:33:54.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:33:54.969 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:33:54.969 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:33:54.969 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:33:54.970 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1024, 2024-07-04T17:33:54.970 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:33:54.970 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:33:54.970 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:33:54.970 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:33:54.970 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:33:54.970 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:33:54.970 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:33:54.970 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:33:54.970 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:33:54.970 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:33:54.970 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:33:54.970 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:33:54.970 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:33:54.970 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.970 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.971 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:33:54.971 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-07-04T17:33:54.971 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:33:54.972 DEBUG:tasks.cephfs.test_mirroring:command returned={ "AsyncMessenger::Worker": [ { "labels": { "id": "0" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "1" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "2" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } } ], "AsyncMessenger::Worker-0": [ { "labels": {}, "counters": { "msgr_recv_messages": 95, "msgr_send_messages": 5, "msgr_recv_bytes": 390384, "msgr_send_bytes": 1136, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.036248320, "msgr_running_send_time": 0.004243662, "msgr_running_recv_time": 0.019280115, "msgr_running_fast_dispatch_time": 0.010775880, "msgr_send_messages_queue_lat": { "avgcount": 5, "sum": 0.004490948, "avgtime": 0.000898189 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 390384, "msgr_send_encrypted_bytes": 1136 } } ], "AsyncMessenger::Worker-1": [ { "labels": {}, "counters": { "msgr_recv_messages": 9, "msgr_send_messages": 101, "msgr_recv_bytes": 1296, "msgr_send_bytes": 745760, "msgr_created_connections": 19, "msgr_active_connections": 4, "msgr_running_total_time": 0.047089833, "msgr_running_send_time": 0.014883619, "msgr_running_recv_time": 0.016278387, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 93, "sum": 0.084642050, "avgtime": 0.000910129 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 1296, "msgr_send_encrypted_bytes": 745760 } } ], "AsyncMessenger::Worker-2": [ { "labels": {}, "counters": { "msgr_recv_messages": 52, "msgr_send_messages": 8, "msgr_recv_bytes": 2138624, "msgr_send_bytes": 1600, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.059358227, "msgr_running_send_time": 0.004105920, "msgr_running_recv_time": 0.101541947, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 8, "sum": 0.003350228, "avgtime": 0.000418778 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 2138624, "msgr_send_encrypted_bytes": 1600 } } ], "cct": [ { "labels": {}, "counters": { "total_workers": 0, "unhealthy_workers": 0 } } ], "cephfs_mirror": [ { "labels": {}, "counters": { "mirrored_filesystems": 1, "mirror_enable_failures": 0 } } ], "cephfs_mirror_mirrored_filesystems": [ { "labels": { "filesystem": "cephfs" }, "counters": { "mirroring_peers": 0, "directory_count": 0 } } ], "mempool": [ { "labels": {}, "counters": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_items": 0, "bluefs_bytes": 0, "bluefs_items": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "buffer_anon_bytes": 175459, "buffer_anon_items": 53, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 18720, "osdmap_items": 169, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "mds_co_bytes": 0, "mds_co_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 } } ], "objecter": [ { "labels": {}, "counters": { "op_active": 0, "op_laggy": 0, "op_send": 0, "op_send_bytes": 0, "op_resend": 0, "op_reply": 0, "op_latency": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "op_inflight": 0, "oplen_avg": { "avgcount": 0, "sum": 0 }, "op": 0, "op_r": 0, "op_w": 0, "op_rmw": 0, "op_pg": 0, "osdop_stat": 0, "osdop_create": 0, "osdop_read": 0, "osdop_write": 0, "osdop_writefull": 0, "osdop_writesame": 0, "osdop_append": 0, "osdop_zero": 0, "osdop_truncate": 0, "osdop_delete": 0, "osdop_mapext": 0, "osdop_sparse_read": 0, "osdop_clonerange": 0, "osdop_getxattr": 0, "osdop_setxattr": 0, "osdop_cmpxattr": 0, "osdop_rmxattr": 0, "osdop_resetxattrs": 0, "osdop_call": 0, "osdop_watch": 0, "osdop_notify": 0, "osdop_src_cmpxattr": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_other": 0, "linger_active": 0, "linger_send": 0, "linger_resend": 0, "linger_ping": 0, "poolop_active": 0, "poolop_send": 0, "poolop_resend": 0, "poolstat_active": 0, "poolstat_send": 0, "poolstat_resend": 0, "statfs_active": 0, "statfs_send": 0, "statfs_resend": 0, "command_active": 0, "command_send": 0, "command_resend": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "osd_sessions": 0, "osd_session_open": 0, "osd_session_close": 0, "osd_laggy": 0, "omap_wr": 0, "omap_rd": 0, "omap_del": 0 } } ], "throttle-msgr_dispatch_throttler-client": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 95, "get_sum": 377397, "get_or_fail_fail": 0, "get_or_fail_success": 95, "take": 0, "take_sum": 0, "put": 95, "put_sum": 377397, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-msgr_dispatch_throttler-radosclient": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 58, "get_sum": 2131137, "get_or_fail_fail": 0, "get_or_fail_success": 58, "take": 0, "take_sum": 0, "put": 58, "put_sum": 2131137, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_bytes": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_ops": [ { "labels": {}, "counters": { "val": 0, "max": 1024, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ] } 2024-07-04T17:33:54.972 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snapshot mirror disable cephfs 2024-07-04T17:33:55.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:55.263+0000 7fa039e00640 1 -- 172.21.15.42:0/2255679805 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa03406dc80 msgr2=0x7fa0340a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:55.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:55.263+0000 7fa039e00640 1 --2- 172.21.15.42:0/2255679805 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa03406dc80 0x7fa0340a94d0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fa024005fd0 tx=0x7fa02402a560 comp rx=0 tx=0).stop 2024-07-04T17:33:55.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:55.263+0000 7fa039e00640 1 -- 172.21.15.42:0/2255679805 shutdown_connections 2024-07-04T17:33:55.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:55.263+0000 7fa039e00640 1 --2- 172.21.15.42:0/2255679805 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa03406dc80 0x7fa0340a94d0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:55.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:55.263+0000 7fa039e00640 1 -- 172.21.15.42:0/2255679805 >> 172.21.15.42:0/2255679805 conn(0x7fa03406c020 msgr2=0x7fa03406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:55.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:55.263+0000 7fa039e00640 1 -- 172.21.15.42:0/2255679805 shutdown_connections 2024-07-04T17:33:55.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:55.263+0000 7fa039e00640 1 -- 172.21.15.42:0/2255679805 wait complete. 2024-07-04T17:33:55.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:55.265+0000 7fa039e00640 1 Processor -- start 2024-07-04T17:33:55.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:55.265+0000 7fa039e00640 1 -- start start 2024-07-04T17:33:55.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:55.267+0000 7fa039e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa03406dc80 0x7fa0340ae780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:55.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:55.267+0000 7fa039e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa034072740 con 0x7fa03406dc80 2024-07-04T17:33:55.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:55.267+0000 7fa032a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa03406dc80 0x7fa0340ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:55.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:55.267+0000 7fa032a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa03406dc80 0x7fa0340ae780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35572/0 (socket says 172.21.15.42:35572) 2024-07-04T17:33:55.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:55.267+0000 7fa032a00640 1 -- 172.21.15.42:0/2775027315 learned_addr learned my addr 172.21.15.42:0/2775027315 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:33:55.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:55.268+0000 7fa032a00640 1 -- 172.21.15.42:0/2775027315 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0340ae260 con 0x7fa03406dc80 2024-07-04T17:33:55.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:55.268+0000 7fa032a00640 1 --2- 172.21.15.42:0/2775027315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa03406dc80 0x7fa0340ae780 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fa024009850 tx=0x7fa0240049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:55.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:55.268+0000 7fa02b400640 1 -- 172.21.15.42:0/2775027315 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa02403e030 con 0x7fa03406dc80 2024-07-04T17:33:55.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:55.268+0000 7fa02b400640 1 -- 172.21.15.42:0/2775027315 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0240035f0 con 0x7fa03406dc80 2024-07-04T17:33:55.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:55.268+0000 7fa02b400640 1 -- 172.21.15.42:0/2775027315 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa024035040 con 0x7fa03406dc80 2024-07-04T17:33:55.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:55.268+0000 7fa039e00640 1 -- 172.21.15.42:0/2775027315 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa03413b9a0 con 0x7fa03406dc80 2024-07-04T17:33:55.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:55.269+0000 7fa039e00640 1 -- 172.21.15.42:0/2775027315 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0340adcc0 con 0x7fa03406dc80 2024-07-04T17:33:55.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:55.270+0000 7fa02b400640 1 -- 172.21.15.42:0/2775027315 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 47) ==== 49681+0+0 (secure 0 0 0) 0x7fa024041030 con 0x7fa03406dc80 2024-07-04T17:33:55.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:55.270+0000 7fa039e00640 1 -- 172.21.15.42:0/2775027315 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0340008d0 con 0x7fa03406dc80 2024-07-04T17:33:55.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:55.270+0000 7fa02b400640 1 --2- 172.21.15.42:0/2775027315 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7fa01403d5d0 0x7fa01403fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:33:55.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:55.270+0000 7fa02b400640 1 -- 172.21.15.42:0/2775027315 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 4733+0+0 (secure 0 0 0) 0x7fa02403f070 con 0x7fa03406dc80 2024-07-04T17:33:55.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:55.271+0000 7fa032000640 1 --2- 172.21.15.42:0/2775027315 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7fa01403d5d0 0x7fa01403fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:33:55.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:55.271+0000 7fa032000640 1 --2- 172.21.15.42:0/2775027315 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7fa01403d5d0 0x7fa01403fae0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa020009670 tx=0x7fa020006bd0 comp rx=0 tx=0).ready entity=mgr.5658 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:33:55.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:55.276+0000 7fa02b400640 1 -- 172.21.15.42:0/2775027315 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fa0340b1510 con 0x7fa03406dc80 2024-07-04T17:33:55.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:55.502+0000 7fa039e00640 1 -- 172.21.15.42:0/2775027315 --> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] -- mgr_command(tid 0: {"prefix": "fs snapshot mirror disable", "fs_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7fa0340adaf0 con 0x7fa01403d5d0 2024-07-04T17:33:56.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:56.199+0000 7fa02b400640 1 -- 172.21.15.42:0/2775027315 <== mgr.5658 v2:172.21.15.42:6834/1310962821 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fa0340adaf0 con 0x7fa01403d5d0 2024-07-04T17:33:56.201 INFO:teuthology.orchestra.run.smithi042.stdout:{} 2024-07-04T17:33:56.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:56.202+0000 7fa039e00640 1 -- 172.21.15.42:0/2775027315 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7fa01403d5d0 msgr2=0x7fa01403fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:56.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:56.202+0000 7fa039e00640 1 --2- 172.21.15.42:0/2775027315 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7fa01403d5d0 0x7fa01403fae0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa020009670 tx=0x7fa020006bd0 comp rx=0 tx=0).stop 2024-07-04T17:33:56.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:56.202+0000 7fa039e00640 1 -- 172.21.15.42:0/2775027315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa03406dc80 msgr2=0x7fa0340ae780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:33:56.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:56.202+0000 7fa039e00640 1 --2- 172.21.15.42:0/2775027315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa03406dc80 0x7fa0340ae780 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fa024009850 tx=0x7fa0240049d0 comp rx=0 tx=0).stop 2024-07-04T17:33:56.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:56.203+0000 7fa039e00640 1 -- 172.21.15.42:0/2775027315 shutdown_connections 2024-07-04T17:33:56.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:56.203+0000 7fa039e00640 1 --2- 172.21.15.42:0/2775027315 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7fa01403d5d0 0x7fa01403fae0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:56.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:56.203+0000 7fa039e00640 1 --2- 172.21.15.42:0/2775027315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa03406dc80 0x7fa0340ae780 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:33:56.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:56.203+0000 7fa039e00640 1 -- 172.21.15.42:0/2775027315 >> 172.21.15.42:0/2775027315 conn(0x7fa03406c020 msgr2=0x7fa034070420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:33:56.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:56.203+0000 7fa039e00640 1 -- 172.21.15.42:0/2775027315 shutdown_connections 2024-07-04T17:33:56.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:33:56.203+0000 7fa039e00640 1 -- 172.21.15.42:0/2775027315 wait complete. 2024-07-04T17:34:01.307 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-04T17:34:06.237 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:34:06.237 DEBUG:teuthology.orchestra.run.smithi042:mirror status for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@10 2024-07-04T17:34:06.480 INFO:teuthology.orchestra.run.smithi042.stderr:no valid command found; 10 closest matches: 2024-07-04T17:34:06.480 INFO:teuthology.orchestra.run.smithi042.stderr:0 2024-07-04T17:34:06.480 INFO:teuthology.orchestra.run.smithi042.stderr:1 2024-07-04T17:34:06.480 INFO:teuthology.orchestra.run.smithi042.stderr:2 2024-07-04T17:34:06.480 INFO:teuthology.orchestra.run.smithi042.stderr:abort 2024-07-04T17:34:06.480 INFO:teuthology.orchestra.run.smithi042.stderr:assert 2024-07-04T17:34:06.480 INFO:teuthology.orchestra.run.smithi042.stderr:config diff 2024-07-04T17:34:06.480 INFO:teuthology.orchestra.run.smithi042.stderr:config diff get 2024-07-04T17:34:06.480 INFO:teuthology.orchestra.run.smithi042.stderr:config get 2024-07-04T17:34:06.480 INFO:teuthology.orchestra.run.smithi042.stderr:config help [] 2024-07-04T17:34:06.481 INFO:teuthology.orchestra.run.smithi042.stderr:config set ... 2024-07-04T17:34:06.481 INFO:teuthology.orchestra.run.smithi042.stderr:admin_socket: invalid command 2024-07-04T17:34:06.490 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-07-04T17:34:06.491 WARNING:tasks.cephfs.test_mirroring:mirror daemon command with label "mirror status for fs: cephfs" failed: Command failed (mirror status for fs: cephfs) on smithi042 with status 22: 'ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@10' 2024-07-04T17:34:06.491 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:34:06.491 DEBUG:teuthology.orchestra.run.smithi042:counter dump for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok counter dump 2024-07-04T17:34:06.733 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:34:06.733 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker": [ 2024-07-04T17:34:06.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:34:06.734 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:34:06.734 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "0" 2024-07-04T17:34:06.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:06.734 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:34:06.734 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:34:06.734 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:34:06.734 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:06.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:06.734 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:34:06.734 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:34:06.734 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "1" 2024-07-04T17:34:06.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:06.735 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:34:06.735 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:34:06.735 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:34:06.735 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:06.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:06.735 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:34:06.735 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:34:06.735 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "2" 2024-07-04T17:34:06.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:06.735 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:34:06.735 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:34:06.735 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:34:06.735 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:06.736 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:06.736 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:34:06.736 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": [ 2024-07-04T17:34:06.736 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:34:06.736 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:34:06.736 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:34:06.736 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 96, 2024-07-04T17:34:06.736 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 5, 2024-07-04T17:34:06.736 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 395280, 2024-07-04T17:34:06.736 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 1136, 2024-07-04T17:34:06.736 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:34:06.736 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:34:06.737 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.036962725, 2024-07-04T17:34:06.737 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.004433736, 2024-07-04T17:34:06.737 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.019570876, 2024-07-04T17:34:06.737 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.010911972, 2024-07-04T17:34:06.737 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:34:06.737 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 5, 2024-07-04T17:34:06.737 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.004490948, 2024-07-04T17:34:06.737 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000898189 2024-07-04T17:34:06.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:06.737 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:34:06.737 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:34:06.737 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:34:06.737 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:34:06.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:06.738 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 395280, 2024-07-04T17:34:06.738 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 1136 2024-07-04T17:34:06.738 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:06.738 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:06.738 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:34:06.738 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": [ 2024-07-04T17:34:06.738 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:34:06.738 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:34:06.738 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:34:06.738 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 9, 2024-07-04T17:34:06.738 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 103, 2024-07-04T17:34:06.739 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 1296, 2024-07-04T17:34:06.739 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 746272, 2024-07-04T17:34:06.739 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 19, 2024-07-04T17:34:06.739 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 4, 2024-07-04T17:34:06.739 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.047523730, 2024-07-04T17:34:06.739 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.015274974, 2024-07-04T17:34:06.739 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.016278387, 2024-07-04T17:34:06.739 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:34:06.739 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:34:06.739 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 95, 2024-07-04T17:34:06.739 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.084804159, 2024-07-04T17:34:06.739 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000892675 2024-07-04T17:34:06.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:06.739 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:34:06.740 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:34:06.740 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:34:06.740 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:34:06.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:06.740 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 1296, 2024-07-04T17:34:06.740 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 746272 2024-07-04T17:34:06.740 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:06.740 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:06.740 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:34:06.740 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": [ 2024-07-04T17:34:06.740 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:34:06.740 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:34:06.740 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:34:06.740 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 52, 2024-07-04T17:34:06.741 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 8, 2024-07-04T17:34:06.741 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 2138624, 2024-07-04T17:34:06.741 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 1600, 2024-07-04T17:34:06.741 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:34:06.741 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:34:06.741 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.059721608, 2024-07-04T17:34:06.741 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.004290389, 2024-07-04T17:34:06.741 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.101618704, 2024-07-04T17:34:06.741 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:34:06.741 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:34:06.741 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 8, 2024-07-04T17:34:06.741 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.003350228, 2024-07-04T17:34:06.741 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000418778 2024-07-04T17:34:06.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:06.741 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:34:06.742 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:34:06.742 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:34:06.742 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:34:06.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:06.742 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 2138624, 2024-07-04T17:34:06.742 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 1600 2024-07-04T17:34:06.742 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:06.742 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:06.742 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:34:06.742 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": [ 2024-07-04T17:34:06.742 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:34:06.742 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:34:06.742 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:34:06.742 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-07-04T17:34:06.743 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-07-04T17:34:06.743 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:06.743 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:06.743 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:34:06.743 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror": [ 2024-07-04T17:34:06.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:34:06.743 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:34:06.743 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:34:06.743 INFO:teuthology.orchestra.run.smithi042.stdout: "mirrored_filesystems": 0, 2024-07-04T17:34:06.743 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_enable_failures": 0 2024-07-04T17:34:06.743 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:06.743 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:06.744 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:34:06.744 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": [ 2024-07-04T17:34:06.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:34:06.744 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:34:06.744 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:34:06.744 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-07-04T17:34:06.744 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-07-04T17:34:06.744 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T17:34:06.744 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-07-04T17:34:06.744 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T17:34:06.744 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-07-04T17:34:06.744 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T17:34:06.745 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T17:34:06.745 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T17:34:06.745 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T17:34:06.745 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T17:34:06.745 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-07-04T17:34:06.745 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T17:34:06.745 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T17:34:06.745 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-07-04T17:34:06.745 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-07-04T17:34:06.745 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-07-04T17:34:06.745 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-07-04T17:34:06.745 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T17:34:06.745 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T17:34:06.745 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T17:34:06.746 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T17:34:06.746 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T17:34:06.746 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-07-04T17:34:06.746 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-07-04T17:34:06.746 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-07-04T17:34:06.746 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T17:34:06.746 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T17:34:06.746 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-07-04T17:34:06.746 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-07-04T17:34:06.746 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-07-04T17:34:06.746 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-07-04T17:34:06.746 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T17:34:06.747 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-07-04T17:34:06.747 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T17:34:06.747 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-07-04T17:34:06.747 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 149206, 2024-07-04T17:34:06.747 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 28, 2024-07-04T17:34:06.747 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-07-04T17:34:06.747 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-07-04T17:34:06.747 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-07-04T17:34:06.747 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-07-04T17:34:06.747 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-07-04T17:34:06.747 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-07-04T17:34:06.747 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-07-04T17:34:06.747 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-07-04T17:34:06.747 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 4592, 2024-07-04T17:34:06.747 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 37, 2024-07-04T17:34:06.748 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T17:34:06.748 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-07-04T17:34:06.748 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-07-04T17:34:06.748 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-07-04T17:34:06.748 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 0, 2024-07-04T17:34:06.748 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 0, 2024-07-04T17:34:06.748 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-07-04T17:34:06.748 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-07-04T17:34:06.748 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-07-04T17:34:06.748 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-07-04T17:34:06.748 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:06.748 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:06.748 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:34:06.748 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter": [ 2024-07-04T17:34:06.748 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:34:06.749 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:34:06.749 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:34:06.749 INFO:teuthology.orchestra.run.smithi042.stdout: "op_active": 0, 2024-07-04T17:34:06.749 INFO:teuthology.orchestra.run.smithi042.stdout: "op_laggy": 0, 2024-07-04T17:34:06.749 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send": 0, 2024-07-04T17:34:06.749 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send_bytes": 0, 2024-07-04T17:34:06.749 INFO:teuthology.orchestra.run.smithi042.stdout: "op_resend": 0, 2024-07-04T17:34:06.749 INFO:teuthology.orchestra.run.smithi042.stdout: "op_reply": 0, 2024-07-04T17:34:06.749 INFO:teuthology.orchestra.run.smithi042.stdout: "op_latency": { 2024-07-04T17:34:06.749 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:34:06.749 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:34:06.749 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:34:06.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:06.750 INFO:teuthology.orchestra.run.smithi042.stdout: "op_inflight": 0, 2024-07-04T17:34:06.750 INFO:teuthology.orchestra.run.smithi042.stdout: "oplen_avg": { 2024-07-04T17:34:06.750 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:34:06.750 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0 2024-07-04T17:34:06.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:06.750 INFO:teuthology.orchestra.run.smithi042.stdout: "op": 0, 2024-07-04T17:34:06.750 INFO:teuthology.orchestra.run.smithi042.stdout: "op_r": 0, 2024-07-04T17:34:06.750 INFO:teuthology.orchestra.run.smithi042.stdout: "op_w": 0, 2024-07-04T17:34:06.750 INFO:teuthology.orchestra.run.smithi042.stdout: "op_rmw": 0, 2024-07-04T17:34:06.750 INFO:teuthology.orchestra.run.smithi042.stdout: "op_pg": 0, 2024-07-04T17:34:06.750 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_stat": 0, 2024-07-04T17:34:06.750 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_create": 0, 2024-07-04T17:34:06.750 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_read": 0, 2024-07-04T17:34:06.750 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_write": 0, 2024-07-04T17:34:06.751 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writefull": 0, 2024-07-04T17:34:06.751 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writesame": 0, 2024-07-04T17:34:06.751 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_append": 0, 2024-07-04T17:34:06.751 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_zero": 0, 2024-07-04T17:34:06.751 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_truncate": 0, 2024-07-04T17:34:06.751 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_delete": 0, 2024-07-04T17:34:06.751 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_mapext": 0, 2024-07-04T17:34:06.751 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_sparse_read": 0, 2024-07-04T17:34:06.751 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_clonerange": 0, 2024-07-04T17:34:06.751 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_getxattr": 0, 2024-07-04T17:34:06.751 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_setxattr": 0, 2024-07-04T17:34:06.751 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_cmpxattr": 0, 2024-07-04T17:34:06.751 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_rmxattr": 0, 2024-07-04T17:34:06.752 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_resetxattrs": 0, 2024-07-04T17:34:06.752 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_call": 0, 2024-07-04T17:34:06.752 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_watch": 0, 2024-07-04T17:34:06.752 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_notify": 0, 2024-07-04T17:34:06.752 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T17:34:06.752 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls": 0, 2024-07-04T17:34:06.752 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls_filter": 0, 2024-07-04T17:34:06.752 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_other": 0, 2024-07-04T17:34:06.752 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_active": 0, 2024-07-04T17:34:06.752 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_send": 0, 2024-07-04T17:34:06.752 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_resend": 0, 2024-07-04T17:34:06.752 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_ping": 0, 2024-07-04T17:34:06.752 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_active": 0, 2024-07-04T17:34:06.752 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_send": 0, 2024-07-04T17:34:06.752 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_resend": 0, 2024-07-04T17:34:06.753 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_active": 0, 2024-07-04T17:34:06.753 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_send": 0, 2024-07-04T17:34:06.753 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_resend": 0, 2024-07-04T17:34:06.753 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_active": 0, 2024-07-04T17:34:06.753 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_send": 0, 2024-07-04T17:34:06.753 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_resend": 0, 2024-07-04T17:34:06.753 INFO:teuthology.orchestra.run.smithi042.stdout: "command_active": 0, 2024-07-04T17:34:06.753 INFO:teuthology.orchestra.run.smithi042.stdout: "command_send": 0, 2024-07-04T17:34:06.753 INFO:teuthology.orchestra.run.smithi042.stdout: "command_resend": 0, 2024-07-04T17:34:06.753 INFO:teuthology.orchestra.run.smithi042.stdout: "map_epoch": 0, 2024-07-04T17:34:06.753 INFO:teuthology.orchestra.run.smithi042.stdout: "map_full": 0, 2024-07-04T17:34:06.753 INFO:teuthology.orchestra.run.smithi042.stdout: "map_inc": 0, 2024-07-04T17:34:06.753 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_sessions": 0, 2024-07-04T17:34:06.753 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_open": 0, 2024-07-04T17:34:06.754 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_close": 0, 2024-07-04T17:34:06.754 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_laggy": 0, 2024-07-04T17:34:06.754 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_wr": 0, 2024-07-04T17:34:06.754 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_rd": 0, 2024-07-04T17:34:06.754 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_del": 0 2024-07-04T17:34:06.754 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:06.754 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:06.754 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:34:06.754 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-client": [ 2024-07-04T17:34:06.754 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:34:06.754 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:34:06.754 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:34:06.754 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:34:06.755 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:34:06.755 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:34:06.755 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 96, 2024-07-04T17:34:06.755 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 382151, 2024-07-04T17:34:06.755 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:34:06.755 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 96, 2024-07-04T17:34:06.755 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:34:06.755 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:34:06.755 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 96, 2024-07-04T17:34:06.755 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 382151, 2024-07-04T17:34:06.755 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:34:06.755 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:34:06.755 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:34:06.755 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:34:06.756 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:06.756 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:06.756 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:06.756 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:34:06.756 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-radosclient": [ 2024-07-04T17:34:06.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:34:06.756 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:34:06.756 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:34:06.756 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:34:06.756 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:34:06.756 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:34:06.756 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 58, 2024-07-04T17:34:06.756 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 2131137, 2024-07-04T17:34:06.756 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:34:06.756 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 58, 2024-07-04T17:34:06.757 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:34:06.757 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:34:06.757 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 58, 2024-07-04T17:34:06.757 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 2131137, 2024-07-04T17:34:06.757 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:34:06.757 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:34:06.757 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:34:06.757 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:34:06.757 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:06.757 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:06.757 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:06.757 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:34:06.757 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_bytes": [ 2024-07-04T17:34:06.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:34:06.758 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:34:06.758 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:34:06.758 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:34:06.758 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:34:06.758 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:34:06.758 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:34:06.758 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:34:06.758 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:34:06.758 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:34:06.758 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:34:06.758 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:34:06.758 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:34:06.758 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:34:06.759 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:34:06.759 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:34:06.759 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:34:06.759 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:34:06.759 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:06.759 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:06.759 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:06.759 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:34:06.759 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_ops": [ 2024-07-04T17:34:06.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:34:06.759 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:34:06.759 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:34:06.759 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:34:06.759 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1024, 2024-07-04T17:34:06.760 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:34:06.760 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:34:06.760 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:34:06.760 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:34:06.760 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:34:06.760 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:34:06.760 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:34:06.760 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:34:06.760 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:34:06.760 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:34:06.760 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:34:06.760 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:34:06.760 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:34:06.760 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:06.761 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:06.761 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:06.761 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-07-04T17:34:06.761 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:34:06.762 DEBUG:tasks.cephfs.test_mirroring:command returned={ "AsyncMessenger::Worker": [ { "labels": { "id": "0" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "1" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "2" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } } ], "AsyncMessenger::Worker-0": [ { "labels": {}, "counters": { "msgr_recv_messages": 96, "msgr_send_messages": 5, "msgr_recv_bytes": 395280, "msgr_send_bytes": 1136, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.036962725, "msgr_running_send_time": 0.004433736, "msgr_running_recv_time": 0.019570876, "msgr_running_fast_dispatch_time": 0.010911972, "msgr_send_messages_queue_lat": { "avgcount": 5, "sum": 0.004490948, "avgtime": 0.000898189 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 395280, "msgr_send_encrypted_bytes": 1136 } } ], "AsyncMessenger::Worker-1": [ { "labels": {}, "counters": { "msgr_recv_messages": 9, "msgr_send_messages": 103, "msgr_recv_bytes": 1296, "msgr_send_bytes": 746272, "msgr_created_connections": 19, "msgr_active_connections": 4, "msgr_running_total_time": 0.047523730, "msgr_running_send_time": 0.015274974, "msgr_running_recv_time": 0.016278387, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 95, "sum": 0.084804159, "avgtime": 0.000892675 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 1296, "msgr_send_encrypted_bytes": 746272 } } ], "AsyncMessenger::Worker-2": [ { "labels": {}, "counters": { "msgr_recv_messages": 52, "msgr_send_messages": 8, "msgr_recv_bytes": 2138624, "msgr_send_bytes": 1600, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.059721608, "msgr_running_send_time": 0.004290389, "msgr_running_recv_time": 0.101618704, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 8, "sum": 0.003350228, "avgtime": 0.000418778 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 2138624, "msgr_send_encrypted_bytes": 1600 } } ], "cct": [ { "labels": {}, "counters": { "total_workers": 0, "unhealthy_workers": 0 } } ], "cephfs_mirror": [ { "labels": {}, "counters": { "mirrored_filesystems": 0, "mirror_enable_failures": 0 } } ], "mempool": [ { "labels": {}, "counters": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_items": 0, "bluefs_bytes": 0, "bluefs_items": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "buffer_anon_bytes": 149206, "buffer_anon_items": 28, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 4592, "osdmap_items": 37, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "mds_co_bytes": 0, "mds_co_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 } } ], "objecter": [ { "labels": {}, "counters": { "op_active": 0, "op_laggy": 0, "op_send": 0, "op_send_bytes": 0, "op_resend": 0, "op_reply": 0, "op_latency": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "op_inflight": 0, "oplen_avg": { "avgcount": 0, "sum": 0 }, "op": 0, "op_r": 0, "op_w": 0, "op_rmw": 0, "op_pg": 0, "osdop_stat": 0, "osdop_create": 0, "osdop_read": 0, "osdop_write": 0, "osdop_writefull": 0, "osdop_writesame": 0, "osdop_append": 0, "osdop_zero": 0, "osdop_truncate": 0, "osdop_delete": 0, "osdop_mapext": 0, "osdop_sparse_read": 0, "osdop_clonerange": 0, "osdop_getxattr": 0, "osdop_setxattr": 0, "osdop_cmpxattr": 0, "osdop_rmxattr": 0, "osdop_resetxattrs": 0, "osdop_call": 0, "osdop_watch": 0, "osdop_notify": 0, "osdop_src_cmpxattr": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_other": 0, "linger_active": 0, "linger_send": 0, "linger_resend": 0, "linger_ping": 0, "poolop_active": 0, "poolop_send": 0, "poolop_resend": 0, "poolstat_active": 0, "poolstat_send": 0, "poolstat_resend": 0, "statfs_active": 0, "statfs_send": 0, "statfs_resend": 0, "command_active": 0, "command_send": 0, "command_resend": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "osd_sessions": 0, "osd_session_open": 0, "osd_session_close": 0, "osd_laggy": 0, "omap_wr": 0, "omap_rd": 0, "omap_del": 0 } } ], "throttle-msgr_dispatch_throttler-client": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 96, "get_sum": 382151, "get_or_fail_fail": 0, "get_or_fail_success": 96, "take": 0, "take_sum": 0, "put": 96, "put_sum": 382151, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-msgr_dispatch_throttler-radosclient": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 58, "get_sum": 2131137, "get_or_fail_fail": 0, "get_or_fail_success": 58, "take": 0, "take_sum": 0, "put": 58, "put_sum": 2131137, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_bytes": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_ops": [ { "labels": {}, "counters": { "val": 0, "max": 1024, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ] } 2024-07-04T17:34:06.762 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable mirroring 2024-07-04T17:34:07.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:07.062+0000 7f2c4ba00640 1 -- 172.21.15.42:0/4175291552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c440b24a0 msgr2=0x7f2c44070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:07.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:07.062+0000 7f2c4ba00640 1 --2- 172.21.15.42:0/4175291552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c440b24a0 0x7f2c44070cf0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f2c300097b0 tx=0x7f2c3002a4b0 comp rx=0 tx=0).stop 2024-07-04T17:34:07.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:07.062+0000 7f2c4ba00640 1 -- 172.21.15.42:0/4175291552 shutdown_connections 2024-07-04T17:34:07.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:07.062+0000 7f2c4ba00640 1 --2- 172.21.15.42:0/4175291552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c440b24a0 0x7f2c44070cf0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:07.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:07.062+0000 7f2c4ba00640 1 -- 172.21.15.42:0/4175291552 >> 172.21.15.42:0/4175291552 conn(0x7f2c4406fe40 msgr2=0x7f2c4406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:07.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:07.062+0000 7f2c4ba00640 1 -- 172.21.15.42:0/4175291552 shutdown_connections 2024-07-04T17:34:07.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:07.062+0000 7f2c4ba00640 1 -- 172.21.15.42:0/4175291552 wait complete. 2024-07-04T17:34:07.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:07.065+0000 7f2c4ba00640 1 Processor -- start 2024-07-04T17:34:07.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:07.065+0000 7f2c4ba00640 1 -- start start 2024-07-04T17:34:07.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:07.066+0000 7f2c4ba00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c4406dc80 0x7f2c44140390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:07.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:07.066+0000 7f2c4ba00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2c44071d70 con 0x7f2c4406dc80 2024-07-04T17:34:07.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:07.067+0000 7f2c49000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c4406dc80 0x7f2c44140390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:07.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:07.067+0000 7f2c49000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c4406dc80 0x7f2c44140390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49554/0 (socket says 172.21.15.42:49554) 2024-07-04T17:34:07.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:07.067+0000 7f2c49000640 1 -- 172.21.15.42:0/205088534 learned_addr learned my addr 172.21.15.42:0/205088534 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:07.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:07.067+0000 7f2c49000640 1 -- 172.21.15.42:0/205088534 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c44141970 con 0x7f2c4406dc80 2024-07-04T17:34:07.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:07.068+0000 7f2c49000640 1 --2- 172.21.15.42:0/205088534 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c4406dc80 0x7f2c44140390 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800c320 tx=0x7f2c3800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:07.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:07.068+0000 7f2c40c00640 1 -- 172.21.15.42:0/205088534 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c380116c0 con 0x7f2c4406dc80 2024-07-04T17:34:07.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:07.068+0000 7f2c40c00640 1 -- 172.21.15.42:0/205088534 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c38017070 con 0x7f2c4406dc80 2024-07-04T17:34:07.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:07.068+0000 7f2c40c00640 1 -- 172.21.15.42:0/205088534 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c38011b00 con 0x7f2c4406dc80 2024-07-04T17:34:07.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:07.068+0000 7f2c4ba00640 1 -- 172.21.15.42:0/205088534 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c4413c640 con 0x7f2c4406dc80 2024-07-04T17:34:07.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:07.068+0000 7f2c4ba00640 1 -- 172.21.15.42:0/205088534 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c4413e590 con 0x7f2c4406dc80 2024-07-04T17:34:07.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:07.069+0000 7f2c40c00640 1 -- 172.21.15.42:0/205088534 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 47) ==== 49681+0+0 (secure 0 0 0) 0x7f2c38011ca0 con 0x7f2c4406dc80 2024-07-04T17:34:07.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:07.069+0000 7f2c4ba00640 1 -- 172.21.15.42:0/205088534 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c44070cf0 con 0x7f2c4406dc80 2024-07-04T17:34:07.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:07.070+0000 7f2c40c00640 1 --2- 172.21.15.42:0/205088534 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f2c2403d5d0 0x7f2c2403fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:07.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:07.070+0000 7f2c40c00640 1 -- 172.21.15.42:0/205088534 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 4776+0+0 (secure 0 0 0) 0x7f2c38051c20 con 0x7f2c4406dc80 2024-07-04T17:34:07.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:07.070+0000 7f2c43e00640 1 --2- 172.21.15.42:0/205088534 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f2c2403d5d0 0x7f2c2403fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:07.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:07.071+0000 7f2c43e00640 1 --2- 172.21.15.42:0/205088534 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f2c2403d5d0 0x7f2c2403fae0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2c30034000 tx=0x7f2c30004730 comp rx=0 tx=0).ready entity=mgr.5658 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:07.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:07.076+0000 7f2c40c00640 1 -- 172.21.15.42:0/205088534 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f2c3800a140 con 0x7f2c4406dc80 2024-07-04T17:34:07.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:07.350+0000 7f2c4ba00640 1 -- 172.21.15.42:0/205088534 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "mirroring"} v 0) -- 0x7f2c440afbd0 con 0x7f2c4406dc80 2024-07-04T17:34:08.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.235+0000 7f2c40c00640 1 -- 172.21.15.42:0/205088534 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "mirroring"}]=0 v48) ==== 89+0+0 (secure 0 0 0) 0x7f2c38010cb0 con 0x7f2c4406dc80 2024-07-04T17:34:08.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.242+0000 7f2c4ba00640 1 -- 172.21.15.42:0/205088534 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f2c2403d5d0 msgr2=0x7f2c2403fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:08.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.242+0000 7f2c4ba00640 1 --2- 172.21.15.42:0/205088534 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f2c2403d5d0 0x7f2c2403fae0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2c30034000 tx=0x7f2c30004730 comp rx=0 tx=0).stop 2024-07-04T17:34:08.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.242+0000 7f2c4ba00640 1 -- 172.21.15.42:0/205088534 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c4406dc80 msgr2=0x7f2c44140390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:08.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.242+0000 7f2c4ba00640 1 --2- 172.21.15.42:0/205088534 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c4406dc80 0x7f2c44140390 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800c320 tx=0x7f2c3800c7e0 comp rx=0 tx=0).stop 2024-07-04T17:34:08.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.243+0000 7f2c4ba00640 1 -- 172.21.15.42:0/205088534 shutdown_connections 2024-07-04T17:34:08.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.243+0000 7f2c4ba00640 1 --2- 172.21.15.42:0/205088534 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f2c2403d5d0 0x7f2c2403fae0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:08.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.243+0000 7f2c4ba00640 1 --2- 172.21.15.42:0/205088534 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c4406dc80 0x7f2c44140390 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:08.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.243+0000 7f2c4ba00640 1 -- 172.21.15.42:0/205088534 >> 172.21.15.42:0/205088534 conn(0x7f2c4406fe40 msgr2=0x7f2c44072ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:08.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.243+0000 7f2c4ba00640 1 -- 172.21.15.42:0/205088534 shutdown_connections 2024-07-04T17:34:08.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.244+0000 7f2c4ba00640 1 -- 172.21.15.42:0/205088534 wait complete. 2024-07-04T17:34:08.278 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-04T17:34:08.278 DEBUG:teuthology.orchestra.run.smithi042:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-04T17:34:08.366 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-04T17:34:08.366 INFO:tasks.cephfs.mount:Terminating background process 2024-07-04T17:34:08.366 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:34:08.366 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:34:08.408 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:34:08.408 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:34:08.496 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-04T17:34:08.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.806+0000 7f12cd000640 1 -- 172.21.15.42:0/2892181982 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12c80a9fa0 msgr2=0x7f12c80b0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:08.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.806+0000 7f12cd000640 1 --2- 172.21.15.42:0/2892181982 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12c80a9fa0 0x7f12c80b0010 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f12b00097b0 tx=0x7f12b002a4a0 comp rx=0 tx=0).stop 2024-07-04T17:34:08.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.806+0000 7f12cd000640 1 -- 172.21.15.42:0/2892181982 shutdown_connections 2024-07-04T17:34:08.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.806+0000 7f12cd000640 1 --2- 172.21.15.42:0/2892181982 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12c80a9fa0 0x7f12c80b0010 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:08.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.806+0000 7f12cd000640 1 -- 172.21.15.42:0/2892181982 >> 172.21.15.42:0/2892181982 conn(0x7f12c80bf700 msgr2=0x7f12c80af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:08.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.807+0000 7f12cd000640 1 -- 172.21.15.42:0/2892181982 shutdown_connections 2024-07-04T17:34:08.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.807+0000 7f12cd000640 1 -- 172.21.15.42:0/2892181982 wait complete. 2024-07-04T17:34:08.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.809+0000 7f12cd000640 1 Processor -- start 2024-07-04T17:34:08.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.809+0000 7f12cd000640 1 -- start start 2024-07-04T17:34:08.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.810+0000 7f12cd000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12c80a9fa0 0x7f12c80ac410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:08.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.810+0000 7f12cd000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f12c80aae30 con 0x7f12c80a9fa0 2024-07-04T17:34:08.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.810+0000 7f12c5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12c80a9fa0 0x7f12c80ac410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:08.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.810+0000 7f12c5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12c80a9fa0 0x7f12c80ac410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49598/0 (socket says 172.21.15.42:49598) 2024-07-04T17:34:08.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.810+0000 7f12c5e00640 1 -- 172.21.15.42:0/3828861382 learned_addr learned my addr 172.21.15.42:0/3828861382 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:08.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.811+0000 7f12c5e00640 1 -- 172.21.15.42:0/3828861382 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c81d56c0 con 0x7f12c80a9fa0 2024-07-04T17:34:08.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.811+0000 7f12c5e00640 1 --2- 172.21.15.42:0/3828861382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12c80a9fa0 0x7f12c80ac410 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f12b800c320 tx=0x7f12b800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:08.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.811+0000 7f12be000640 1 -- 172.21.15.42:0/3828861382 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12b80116c0 con 0x7f12c80a9fa0 2024-07-04T17:34:08.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.811+0000 7f12be000640 1 -- 172.21.15.42:0/3828861382 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12b8017070 con 0x7f12c80a9fa0 2024-07-04T17:34:08.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.811+0000 7f12cd000640 1 -- 172.21.15.42:0/3828861382 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12c81d5350 con 0x7f12c80a9fa0 2024-07-04T17:34:08.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.811+0000 7f12be000640 1 -- 172.21.15.42:0/3828861382 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12b8011b40 con 0x7f12c80a9fa0 2024-07-04T17:34:08.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.811+0000 7f12cd000640 1 -- 172.21.15.42:0/3828861382 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12c80ac950 con 0x7f12c80a9fa0 2024-07-04T17:34:08.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.813+0000 7f12cd000640 1 -- 172.21.15.42:0/3828861382 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12c80ab520 con 0x7f12c80a9fa0 2024-07-04T17:34:08.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.813+0000 7f12be000640 1 -- 172.21.15.42:0/3828861382 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 48) ==== 49668+0+0 (secure 0 0 0) 0x7f12b8011ce0 con 0x7f12c80a9fa0 2024-07-04T17:34:08.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.813+0000 7f12be000640 1 --2- 172.21.15.42:0/3828861382 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f12a403d580 0x7f12a403fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:08.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.813+0000 7f12be000640 1 -- 172.21.15.42:0/3828861382 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 4776+0+0 (secure 0 0 0) 0x7f12b8051b20 con 0x7f12c80a9fa0 2024-07-04T17:34:08.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.814+0000 7f12bfe00640 1 -- 172.21.15.42:0/3828861382 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f12a403d580 msgr2=0x7f12a403fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1310962821 2024-07-04T17:34:08.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.814+0000 7f12bfe00640 1 --2- 172.21.15.42:0/3828861382 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f12a403d580 0x7f12a403fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:34:08.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:08.818+0000 7f12be000640 1 -- 172.21.15.42:0/3828861382 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f12b801ac80 con 0x7f12c80a9fa0 2024-07-04T17:34:09.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.015+0000 7f12bfe00640 1 -- 172.21.15.42:0/3828861382 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f12a403d580 msgr2=0x7f12a403fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1310962821 2024-07-04T17:34:09.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.015+0000 7f12bfe00640 1 --2- 172.21.15.42:0/3828861382 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f12a403d580 0x7f12a403fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:34:09.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.038+0000 7f12cd000640 1 -- 172.21.15.42:0/3828861382 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f12c8069b70 con 0x7f12c80a9fa0 2024-07-04T17:34:09.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.038+0000 7f12be000640 1 -- 172.21.15.42:0/3828861382 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 11 entries v120) ==== 81+0+643 (secure 0 0 0) 0x7f12b80102e0 con 0x7f12c80a9fa0 2024-07-04T17:34:09.039 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2238986031 2024-07-04T18:33:57.174196+0000 2024-07-04T17:34:09.040 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3738568139 2024-07-05T17:33:42.103205+0000 2024-07-04T17:34:09.040 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6835/4130303917 2024-07-05T17:33:42.103205+0000 2024-07-04T17:34:09.040 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6834/4130303917 2024-07-05T17:33:42.103205+0000 2024-07-04T17:34:09.040 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1148975990 2024-07-05T17:33:42.103205+0000 2024-07-04T17:34:09.040 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1387747213 2024-07-05T17:33:42.103205+0000 2024-07-04T17:34:09.040 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2342229102 2024-07-05T17:33:42.103205+0000 2024-07-04T17:34:09.040 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3100365373 2024-07-05T17:33:42.103205+0000 2024-07-04T17:34:09.040 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3283150650 2024-07-05T17:33:42.103205+0000 2024-07-04T17:34:09.040 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/701226807 2024-07-05T17:33:42.103205+0000 2024-07-04T17:34:09.040 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/4022678544 2024-07-05T17:33:42.103205+0000 2024-07-04T17:34:09.040 INFO:teuthology.orchestra.run.smithi042.stderr:listed 11 entries 2024-07-04T17:34:09.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.041+0000 7f12cd000640 1 -- 172.21.15.42:0/3828861382 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f12a403d580 msgr2=0x7f12a403fa90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:34:09.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.041+0000 7f12cd000640 1 --2- 172.21.15.42:0/3828861382 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f12a403d580 0x7f12a403fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:09.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.041+0000 7f12cd000640 1 -- 172.21.15.42:0/3828861382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12c80a9fa0 msgr2=0x7f12c80ac410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:09.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.041+0000 7f12cd000640 1 --2- 172.21.15.42:0/3828861382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12c80a9fa0 0x7f12c80ac410 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f12b800c320 tx=0x7f12b800c7e0 comp rx=0 tx=0).stop 2024-07-04T17:34:09.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.041+0000 7f12cd000640 1 -- 172.21.15.42:0/3828861382 shutdown_connections 2024-07-04T17:34:09.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.041+0000 7f12cd000640 1 --2- 172.21.15.42:0/3828861382 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f12a403d580 0x7f12a403fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:09.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.041+0000 7f12cd000640 1 --2- 172.21.15.42:0/3828861382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12c80a9fa0 0x7f12c80ac410 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:09.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.041+0000 7f12cd000640 1 -- 172.21.15.42:0/3828861382 >> 172.21.15.42:0/3828861382 conn(0x7f12c80bf700 msgr2=0x7f12c81d8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:09.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.041+0000 7f12cd000640 1 -- 172.21.15.42:0/3828861382 shutdown_connections 2024-07-04T17:34:09.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.042+0000 7f12cd000640 1 -- 172.21.15.42:0/3828861382 wait complete. 2024-07-04T17:34:09.071 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi042.front.sepia.ceph.com... 2024-07-04T17:34:09.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:34:09.071 DEBUG:teuthology.orchestra.run.smithi042:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-07-04T17:34:09.150 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[65641]: fuse finished with error 0 and tester_r 0 2024-07-04T17:34:09.156 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-07-04T17:34:09.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:34:09.156 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-04T17:34:09.299 INFO:tasks.cephfs.mount:Terminating background process 2024-07-04T17:34:09.299 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:34:09.299 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:34:09.347 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:34:09.348 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:34:09.439 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-04T17:34:09.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.704+0000 7fa148800640 1 -- 172.21.15.42:0/452807094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1400bc040 msgr2=0x7fa1400bd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:09.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.704+0000 7fa148800640 1 --2- 172.21.15.42:0/452807094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1400bc040 0x7fa1400bd2d0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fa134005fd0 tx=0x7fa13402a5f0 comp rx=0 tx=0).stop 2024-07-04T17:34:09.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.704+0000 7fa148800640 1 -- 172.21.15.42:0/452807094 shutdown_connections 2024-07-04T17:34:09.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.704+0000 7fa148800640 1 --2- 172.21.15.42:0/452807094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1400bc040 0x7fa1400bd2d0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:09.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.704+0000 7fa148800640 1 -- 172.21.15.42:0/452807094 >> 172.21.15.42:0/452807094 conn(0x7fa14006c940 msgr2=0x7fa14006ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:09.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.704+0000 7fa148800640 1 -- 172.21.15.42:0/452807094 shutdown_connections 2024-07-04T17:34:09.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.704+0000 7fa148800640 1 -- 172.21.15.42:0/452807094 wait complete. 2024-07-04T17:34:09.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.708+0000 7fa148800640 1 Processor -- start 2024-07-04T17:34:09.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.708+0000 7fa148800640 1 -- start start 2024-07-04T17:34:09.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.710+0000 7fa148800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1400bc040 0x7fa1400c0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:09.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.710+0000 7fa148800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa1400b7530 con 0x7fa1400bc040 2024-07-04T17:34:09.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.710+0000 7fa145e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1400bc040 0x7fa1400c0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:09.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.710+0000 7fa145e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1400bc040 0x7fa1400c0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49612/0 (socket says 172.21.15.42:49612) 2024-07-04T17:34:09.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.710+0000 7fa145e00640 1 -- 172.21.15.42:0/1284224163 learned_addr learned my addr 172.21.15.42:0/1284224163 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:09.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.711+0000 7fa145e00640 1 -- 172.21.15.42:0/1284224163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1400bf0c0 con 0x7fa1400bc040 2024-07-04T17:34:09.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.711+0000 7fa145e00640 1 --2- 172.21.15.42:0/1284224163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1400bc040 0x7fa1400c0580 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fa134009850 tx=0x7fa1340049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:09.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.711+0000 7fa13a000640 1 -- 172.21.15.42:0/1284224163 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa13403e030 con 0x7fa1400bc040 2024-07-04T17:34:09.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.711+0000 7fa13a000640 1 -- 172.21.15.42:0/1284224163 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1340035f0 con 0x7fa1400bc040 2024-07-04T17:34:09.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.711+0000 7fa148800640 1 -- 172.21.15.42:0/1284224163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1400c0b50 con 0x7fa1400bc040 2024-07-04T17:34:09.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.711+0000 7fa13a000640 1 -- 172.21.15.42:0/1284224163 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa134035040 con 0x7fa1400bc040 2024-07-04T17:34:09.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.711+0000 7fa148800640 1 -- 172.21.15.42:0/1284224163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1400be3c0 con 0x7fa1400bc040 2024-07-04T17:34:09.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.713+0000 7fa148800640 1 -- 172.21.15.42:0/1284224163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1400b7c00 con 0x7fa1400bc040 2024-07-04T17:34:09.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.713+0000 7fa13a000640 1 -- 172.21.15.42:0/1284224163 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 48) ==== 49668+0+0 (secure 0 0 0) 0x7fa134041030 con 0x7fa1400bc040 2024-07-04T17:34:09.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.714+0000 7fa13a000640 1 --2- 172.21.15.42:0/1284224163 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7fa12403d580 0x7fa12403fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:09.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.714+0000 7fa13a000640 1 -- 172.21.15.42:0/1284224163 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 4776+0+0 (secure 0 0 0) 0x7fa13403f070 con 0x7fa1400bc040 2024-07-04T17:34:09.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.714+0000 7fa145400640 1 -- 172.21.15.42:0/1284224163 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7fa12403d580 msgr2=0x7fa12403fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1310962821 2024-07-04T17:34:09.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.714+0000 7fa145400640 1 --2- 172.21.15.42:0/1284224163 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7fa12403d580 0x7fa12403fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:34:09.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.718+0000 7fa13a000640 1 -- 172.21.15.42:0/1284224163 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fa13404bd20 con 0x7fa1400bc040 2024-07-04T17:34:09.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.916+0000 7fa145400640 1 -- 172.21.15.42:0/1284224163 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7fa12403d580 msgr2=0x7fa12403fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1310962821 2024-07-04T17:34:09.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.916+0000 7fa145400640 1 --2- 172.21.15.42:0/1284224163 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7fa12403d580 0x7fa12403fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:34:09.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.941+0000 7fa148800640 1 -- 172.21.15.42:0/1284224163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa140069b70 con 0x7fa1400bc040 2024-07-04T17:34:09.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.942+0000 7fa13a000640 1 -- 172.21.15.42:0/1284224163 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 11 entries v120) ==== 81+0+643 (secure 0 0 0) 0x7fa134030c90 con 0x7fa1400bc040 2024-07-04T17:34:09.943 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2238986031 2024-07-04T18:33:57.174196+0000 2024-07-04T17:34:09.943 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3738568139 2024-07-05T17:33:42.103205+0000 2024-07-04T17:34:09.943 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6835/4130303917 2024-07-05T17:33:42.103205+0000 2024-07-04T17:34:09.943 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6834/4130303917 2024-07-05T17:33:42.103205+0000 2024-07-04T17:34:09.943 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1148975990 2024-07-05T17:33:42.103205+0000 2024-07-04T17:34:09.944 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1387747213 2024-07-05T17:33:42.103205+0000 2024-07-04T17:34:09.944 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2342229102 2024-07-05T17:33:42.103205+0000 2024-07-04T17:34:09.944 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3100365373 2024-07-05T17:33:42.103205+0000 2024-07-04T17:34:09.944 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3283150650 2024-07-05T17:33:42.103205+0000 2024-07-04T17:34:09.944 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/701226807 2024-07-05T17:33:42.103205+0000 2024-07-04T17:34:09.944 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/4022678544 2024-07-05T17:33:42.103205+0000 2024-07-04T17:34:09.944 INFO:teuthology.orchestra.run.smithi042.stderr:listed 11 entries 2024-07-04T17:34:09.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.944+0000 7fa148800640 1 -- 172.21.15.42:0/1284224163 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7fa12403d580 msgr2=0x7fa12403fa90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:34:09.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.944+0000 7fa148800640 1 --2- 172.21.15.42:0/1284224163 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7fa12403d580 0x7fa12403fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:09.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.944+0000 7fa148800640 1 -- 172.21.15.42:0/1284224163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1400bc040 msgr2=0x7fa1400c0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:09.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.944+0000 7fa148800640 1 --2- 172.21.15.42:0/1284224163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1400bc040 0x7fa1400c0580 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fa134009850 tx=0x7fa1340049d0 comp rx=0 tx=0).stop 2024-07-04T17:34:09.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.945+0000 7fa148800640 1 -- 172.21.15.42:0/1284224163 shutdown_connections 2024-07-04T17:34:09.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.945+0000 7fa148800640 1 --2- 172.21.15.42:0/1284224163 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7fa12403d580 0x7fa12403fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:09.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.945+0000 7fa148800640 1 --2- 172.21.15.42:0/1284224163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1400bc040 0x7fa1400c0580 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:09.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.945+0000 7fa148800640 1 -- 172.21.15.42:0/1284224163 >> 172.21.15.42:0/1284224163 conn(0x7fa14006c940 msgr2=0x7fa140143200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:09.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.945+0000 7fa148800640 1 -- 172.21.15.42:0/1284224163 shutdown_connections 2024-07-04T17:34:09.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:09.945+0000 7fa148800640 1 -- 172.21.15.42:0/1284224163 wait complete. 2024-07-04T17:34:09.971 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi042.front.sepia.ceph.com... 2024-07-04T17:34:09.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:34:09.971 DEBUG:teuthology.orchestra.run.smithi042:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-07-04T17:34:10.038 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi042.stderr:ceph-fuse[66110]: fuse finished with error 0 and tester_r 0 2024-07-04T17:34:10.043 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-07-04T17:34:10.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:34:10.043 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-07-04T17:34:10.150 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:34:10.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.428+0000 7f12ce400640 1 -- 172.21.15.42:0/1659527592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12c80bbef0 msgr2=0x7f12c81414a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:10.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.428+0000 7f12ce400640 1 --2- 172.21.15.42:0/1659527592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12c80bbef0 0x7f12c81414a0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f12b80097b0 tx=0x7f12b802a530 comp rx=0 tx=0).stop 2024-07-04T17:34:10.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.428+0000 7f12ce400640 1 -- 172.21.15.42:0/1659527592 shutdown_connections 2024-07-04T17:34:10.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.428+0000 7f12ce400640 1 --2- 172.21.15.42:0/1659527592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12c80bbef0 0x7f12c81414a0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:10.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.428+0000 7f12ce400640 1 -- 172.21.15.42:0/1659527592 >> 172.21.15.42:0/1659527592 conn(0x7f12c80688a0 msgr2=0x7f12c806db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:10.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.428+0000 7f12ce400640 1 -- 172.21.15.42:0/1659527592 shutdown_connections 2024-07-04T17:34:10.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.428+0000 7f12ce400640 1 -- 172.21.15.42:0/1659527592 wait complete. 2024-07-04T17:34:10.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.430+0000 7f12ce400640 1 Processor -- start 2024-07-04T17:34:10.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.430+0000 7f12ce400640 1 -- start start 2024-07-04T17:34:10.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.432+0000 7f12ce400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12c80bbef0 0x7f12c813faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:10.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.432+0000 7f12ce400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f12c80b79a0 con 0x7f12c80bbef0 2024-07-04T17:34:10.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.432+0000 7f12cd000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12c80bbef0 0x7f12c813faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:10.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.432+0000 7f12cd000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12c80bbef0 0x7f12c813faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49628/0 (socket says 172.21.15.42:49628) 2024-07-04T17:34:10.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.432+0000 7f12cd000640 1 -- 172.21.15.42:0/619880668 learned_addr learned my addr 172.21.15.42:0/619880668 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:10.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.433+0000 7f12cd000640 1 -- 172.21.15.42:0/619880668 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c80c01f0 con 0x7f12c80bbef0 2024-07-04T17:34:10.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.433+0000 7f12cd000640 1 --2- 172.21.15.42:0/619880668 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12c80bbef0 0x7f12c813faf0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f12b80098d0 tx=0x7f12b8033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:10.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.433+0000 7f12c4c00640 1 -- 172.21.15.42:0/619880668 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12b8031bc0 con 0x7f12c80bbef0 2024-07-04T17:34:10.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.433+0000 7f12c4c00640 1 -- 172.21.15.42:0/619880668 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12b8031e50 con 0x7f12c80bbef0 2024-07-04T17:34:10.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.433+0000 7f12c4c00640 1 -- 172.21.15.42:0/619880668 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12b8033260 con 0x7f12c80bbef0 2024-07-04T17:34:10.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.433+0000 7f12ce400640 1 -- 172.21.15.42:0/619880668 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12c81e39b0 con 0x7f12c80bbef0 2024-07-04T17:34:10.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.433+0000 7f12ce400640 1 -- 172.21.15.42:0/619880668 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12c81e3710 con 0x7f12c80bbef0 2024-07-04T17:34:10.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.434+0000 7f12ce400640 1 -- 172.21.15.42:0/619880668 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12c80b8060 con 0x7f12c80bbef0 2024-07-04T17:34:10.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.435+0000 7f12c4c00640 1 -- 172.21.15.42:0/619880668 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 48) ==== 49668+0+0 (secure 0 0 0) 0x7f12b8033400 con 0x7f12c80bbef0 2024-07-04T17:34:10.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.435+0000 7f12c4c00640 1 --2- 172.21.15.42:0/619880668 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f12a403d580 0x7f12a403fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:10.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.435+0000 7f12c4c00640 1 -- 172.21.15.42:0/619880668 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 4776+0+0 (secure 0 0 0) 0x7f12b80734b0 con 0x7f12c80bbef0 2024-07-04T17:34:10.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.435+0000 7f12c7e00640 1 -- 172.21.15.42:0/619880668 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f12a403d580 msgr2=0x7f12a403fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1310962821 2024-07-04T17:34:10.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.435+0000 7f12c7e00640 1 --2- 172.21.15.42:0/619880668 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f12a403d580 0x7f12a403fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:34:10.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.440+0000 7f12c4c00640 1 -- 172.21.15.42:0/619880668 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f12b803f030 con 0x7f12c80bbef0 2024-07-04T17:34:10.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.636+0000 7f12c7e00640 1 -- 172.21.15.42:0/619880668 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f12a403d580 msgr2=0x7f12a403fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1310962821 2024-07-04T17:34:10.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.636+0000 7f12c7e00640 1 --2- 172.21.15.42:0/619880668 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f12a403d580 0x7f12a403fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:34:10.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.689+0000 7f12ce400640 1 -- 172.21.15.42:0/619880668 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12c8068e30 con 0x7f12c80bbef0 2024-07-04T17:34:10.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.690+0000 7f12c4c00640 1 -- 172.21.15.42:0/619880668 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+7354 (secure 0 0 0) 0x7f12b80035f0 con 0x7f12c80bbef0 2024-07-04T17:34:10.691 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:34:10.691 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":98,"btime":"2024-07-04T17:33:56:185420+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5536,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/68857121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":68857121},{"type":"v1","addr":"172.21.15.42:6825","nonce":68857121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":5544,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6828/20990456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":20990456},{"type":"v1","addr":"172.21.15.42:6828","nonce":20990456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":5545,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/428501060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6827","nonce":428501060},{"type":"v1","addr":"172.21.15.42:6829","nonce":428501060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:33:08.649551+0000","modified":"2024-07-04T17:33:55.506934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5550":{"gid":5550,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6833/605786437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":605786437},{"type":"v1","addr":"172.21.15.42:6833","nonce":605786437}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5550,"qdb_cluster":[5550]},"id":10},{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:33:28.944269+0000","modified":"2024-07-04T17:33:32.025418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5549},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5549":{"gid":5549,"name":"b","rank":0,"incarnation":92,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6831/3224527035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":3224527035},{"type":"v1","addr":"172.21.15.42:6831","nonce":3224527035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5549,"qdb_cluster":[5549]},"id":11}]} 2024-07-04T17:34:10.692 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 98 2024-07-04T17:34:10.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.693+0000 7f12ce400640 1 -- 172.21.15.42:0/619880668 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f12a403d580 msgr2=0x7f12a403fa90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:34:10.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.693+0000 7f12ce400640 1 --2- 172.21.15.42:0/619880668 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f12a403d580 0x7f12a403fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:10.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.693+0000 7f12ce400640 1 -- 172.21.15.42:0/619880668 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12c80bbef0 msgr2=0x7f12c813faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:10.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.693+0000 7f12ce400640 1 --2- 172.21.15.42:0/619880668 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12c80bbef0 0x7f12c813faf0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f12b80098d0 tx=0x7f12b8033e40 comp rx=0 tx=0).stop 2024-07-04T17:34:10.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.693+0000 7f12ce400640 1 -- 172.21.15.42:0/619880668 shutdown_connections 2024-07-04T17:34:10.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.693+0000 7f12ce400640 1 --2- 172.21.15.42:0/619880668 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f12a403d580 0x7f12a403fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:10.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.693+0000 7f12ce400640 1 --2- 172.21.15.42:0/619880668 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12c80bbef0 0x7f12c813faf0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:10.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.693+0000 7f12ce400640 1 -- 172.21.15.42:0/619880668 >> 172.21.15.42:0/619880668 conn(0x7f12c80688a0 msgr2=0x7f12c80b6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:10.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.693+0000 7f12ce400640 1 -- 172.21.15.42:0/619880668 shutdown_connections 2024-07-04T17:34:10.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.693+0000 7f12ce400640 1 -- 172.21.15.42:0/619880668 wait complete. 2024-07-04T17:34:10.719 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:34:10.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.990+0000 7f05d5000640 1 -- 172.21.15.42:0/3446499975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05d00bcf90 msgr2=0x7f05d00bd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:10.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.990+0000 7f05d5000640 1 --2- 172.21.15.42:0/3446499975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05d00bcf90 0x7f05d00bd3a0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f05b80097b0 tx=0x7f05b802a540 comp rx=0 tx=0).stop 2024-07-04T17:34:10.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.990+0000 7f05d5000640 1 -- 172.21.15.42:0/3446499975 shutdown_connections 2024-07-04T17:34:10.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.990+0000 7f05d5000640 1 --2- 172.21.15.42:0/3446499975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05d00bcf90 0x7f05d00bd3a0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:10.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.990+0000 7f05d5000640 1 -- 172.21.15.42:0/3446499975 >> 172.21.15.42:0/3446499975 conn(0x7f05d00688a0 msgr2=0x7f05d006db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:10.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.990+0000 7f05d5000640 1 -- 172.21.15.42:0/3446499975 shutdown_connections 2024-07-04T17:34:10.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.990+0000 7f05d5000640 1 -- 172.21.15.42:0/3446499975 wait complete. 2024-07-04T17:34:10.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.992+0000 7f05d5000640 1 Processor -- start 2024-07-04T17:34:10.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.992+0000 7f05d5000640 1 -- start start 2024-07-04T17:34:10.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.993+0000 7f05d5000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05d00bbef0 0x7f05d00c2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:10.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.993+0000 7f05d5000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f05d00b79a0 con 0x7f05d00bbef0 2024-07-04T17:34:10.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.994+0000 7f05cf400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05d00bbef0 0x7f05d00c2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:10.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.994+0000 7f05cf400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05d00bbef0 0x7f05d00c2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49638/0 (socket says 172.21.15.42:49638) 2024-07-04T17:34:10.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.994+0000 7f05cf400640 1 -- 172.21.15.42:0/4220586644 learned_addr learned my addr 172.21.15.42:0/4220586644 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:10.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.994+0000 7f05cf400640 1 -- 172.21.15.42:0/4220586644 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d00c1fa0 con 0x7f05d00bbef0 2024-07-04T17:34:10.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.995+0000 7f05cf400640 1 --2- 172.21.15.42:0/4220586644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05d00bbef0 0x7f05d00c2480 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f05c000c320 tx=0x7f05c000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:10.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.995+0000 7f05c7400640 1 -- 172.21.15.42:0/4220586644 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05c00116c0 con 0x7f05d00bbef0 2024-07-04T17:34:10.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.995+0000 7f05c7400640 1 -- 172.21.15.42:0/4220586644 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05c0017070 con 0x7f05d00bbef0 2024-07-04T17:34:10.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.995+0000 7f05c7400640 1 -- 172.21.15.42:0/4220586644 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05c0011b00 con 0x7f05d00bbef0 2024-07-04T17:34:10.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.995+0000 7f05d5000640 1 -- 172.21.15.42:0/4220586644 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05d0136b40 con 0x7f05d00bbef0 2024-07-04T17:34:10.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.995+0000 7f05d5000640 1 -- 172.21.15.42:0/4220586644 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05d00c1c70 con 0x7f05d00bbef0 2024-07-04T17:34:10.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.996+0000 7f05d5000640 1 -- 172.21.15.42:0/4220586644 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05d00b8020 con 0x7f05d00bbef0 2024-07-04T17:34:10.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.996+0000 7f05c7400640 1 -- 172.21.15.42:0/4220586644 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 48) ==== 49668+0+0 (secure 0 0 0) 0x7f05c0011ca0 con 0x7f05d00bbef0 2024-07-04T17:34:10.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.997+0000 7f05c7400640 1 --2- 172.21.15.42:0/4220586644 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f05ac03d580 0x7f05ac03fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:10.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.997+0000 7f05c7400640 1 -- 172.21.15.42:0/4220586644 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 4776+0+0 (secure 0 0 0) 0x7f05c0051b30 con 0x7f05d00bbef0 2024-07-04T17:34:10.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.997+0000 7f05c7e00640 1 -- 172.21.15.42:0/4220586644 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f05ac03d580 msgr2=0x7f05ac03fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1310962821 2024-07-04T17:34:10.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:10.997+0000 7f05c7e00640 1 --2- 172.21.15.42:0/4220586644 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f05ac03d580 0x7f05ac03fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:34:11.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.002+0000 7f05c7400640 1 -- 172.21.15.42:0/4220586644 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f05d00bc300 con 0x7f05d00bbef0 2024-07-04T17:34:11.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.199+0000 7f05c7e00640 1 -- 172.21.15.42:0/4220586644 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f05ac03d580 msgr2=0x7f05ac03fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1310962821 2024-07-04T17:34:11.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.199+0000 7f05c7e00640 1 --2- 172.21.15.42:0/4220586644 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f05ac03d580 0x7f05ac03fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:34:11.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.251+0000 7f05d5000640 1 -- 172.21.15.42:0/4220586644 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05d0068e30 con 0x7f05d00bbef0 2024-07-04T17:34:11.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.252+0000 7f05c7400640 1 -- 172.21.15.42:0/4220586644 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+7354 (secure 0 0 0) 0x7f05c000a140 con 0x7f05d00bbef0 2024-07-04T17:34:11.254 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:34:11.254 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":98,"btime":"2024-07-04T17:33:56:185420+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5536,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/68857121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":68857121},{"type":"v1","addr":"172.21.15.42:6825","nonce":68857121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":5544,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6828/20990456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":20990456},{"type":"v1","addr":"172.21.15.42:6828","nonce":20990456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":5545,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/428501060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6827","nonce":428501060},{"type":"v1","addr":"172.21.15.42:6829","nonce":428501060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:33:08.649551+0000","modified":"2024-07-04T17:33:55.506934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5550":{"gid":5550,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6833/605786437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":605786437},{"type":"v1","addr":"172.21.15.42:6833","nonce":605786437}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5550,"qdb_cluster":[5550]},"id":10},{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:33:28.944269+0000","modified":"2024-07-04T17:33:32.025418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5549},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5549":{"gid":5549,"name":"b","rank":0,"incarnation":92,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6831/3224527035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":3224527035},{"type":"v1","addr":"172.21.15.42:6831","nonce":3224527035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5549,"qdb_cluster":[5549]},"id":11}]} 2024-07-04T17:34:11.254 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 98 2024-07-04T17:34:11.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.255+0000 7f05d5000640 1 -- 172.21.15.42:0/4220586644 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f05ac03d580 msgr2=0x7f05ac03fa90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:34:11.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.255+0000 7f05d5000640 1 --2- 172.21.15.42:0/4220586644 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f05ac03d580 0x7f05ac03fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:11.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.255+0000 7f05d5000640 1 -- 172.21.15.42:0/4220586644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05d00bbef0 msgr2=0x7f05d00c2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:11.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.255+0000 7f05d5000640 1 --2- 172.21.15.42:0/4220586644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05d00bbef0 0x7f05d00c2480 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f05c000c320 tx=0x7f05c000c7e0 comp rx=0 tx=0).stop 2024-07-04T17:34:11.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.255+0000 7f05d5000640 1 -- 172.21.15.42:0/4220586644 shutdown_connections 2024-07-04T17:34:11.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.255+0000 7f05d5000640 1 --2- 172.21.15.42:0/4220586644 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f05ac03d580 0x7f05ac03fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:11.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.255+0000 7f05d5000640 1 --2- 172.21.15.42:0/4220586644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05d00bbef0 0x7f05d00c2480 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:11.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.255+0000 7f05d5000640 1 -- 172.21.15.42:0/4220586644 >> 172.21.15.42:0/4220586644 conn(0x7f05d00688a0 msgr2=0x7f05d0075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:11.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.256+0000 7f05d5000640 1 -- 172.21.15.42:0/4220586644 shutdown_connections 2024-07-04T17:34:11.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.256+0000 7f05d5000640 1 -- 172.21.15.42:0/4220586644 wait complete. 2024-07-04T17:34:11.282 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:34:11.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.554+0000 7fa1cc600640 1 -- 172.21.15.42:0/2060646541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1c40bd100 msgr2=0x7fa1c4071f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:11.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.554+0000 7fa1cc600640 1 --2- 172.21.15.42:0/2060646541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1c40bd100 0x7fa1c4071f20 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0005fd0 tx=0x7fa1b002a560 comp rx=0 tx=0).stop 2024-07-04T17:34:11.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.554+0000 7fa1cc600640 1 -- 172.21.15.42:0/2060646541 shutdown_connections 2024-07-04T17:34:11.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.554+0000 7fa1cc600640 1 --2- 172.21.15.42:0/2060646541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1c40bd100 0x7fa1c4071f20 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:11.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.554+0000 7fa1cc600640 1 -- 172.21.15.42:0/2060646541 >> 172.21.15.42:0/2060646541 conn(0x7fa1c40c03e0 msgr2=0x7fa1c40c07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:11.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.554+0000 7fa1cc600640 1 -- 172.21.15.42:0/2060646541 shutdown_connections 2024-07-04T17:34:11.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.554+0000 7fa1cc600640 1 -- 172.21.15.42:0/2060646541 wait complete. 2024-07-04T17:34:11.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.558+0000 7fa1cc600640 1 Processor -- start 2024-07-04T17:34:11.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.558+0000 7fa1cc600640 1 -- start start 2024-07-04T17:34:11.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.560+0000 7fa1cc600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1c406d8a0 0x7fa1c41d6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:11.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.560+0000 7fa1cc600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c41486a0 con 0x7fa1c406d8a0 2024-07-04T17:34:11.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.560+0000 7fa1c9c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1c406d8a0 0x7fa1c41d6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:11.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.560+0000 7fa1c9c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1c406d8a0 0x7fa1c41d6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49650/0 (socket says 172.21.15.42:49650) 2024-07-04T17:34:11.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.560+0000 7fa1c9c00640 1 -- 172.21.15.42:0/1296736367 learned_addr learned my addr 172.21.15.42:0/1296736367 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:11.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.561+0000 7fa1c9c00640 1 -- 172.21.15.42:0/1296736367 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c4149000 con 0x7fa1c406d8a0 2024-07-04T17:34:11.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.561+0000 7fa1c9c00640 1 --2- 172.21.15.42:0/1296736367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1c406d8a0 0x7fa1c41d6c90 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8001eb0 tx=0x7fa1b800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:11.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.561+0000 7fa1c1600640 1 -- 172.21.15.42:0/1296736367 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1b8017070 con 0x7fa1c406d8a0 2024-07-04T17:34:11.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.561+0000 7fa1c1600640 1 -- 172.21.15.42:0/1296736367 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1b80052f0 con 0x7fa1c406d8a0 2024-07-04T17:34:11.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.561+0000 7fa1c1600640 1 -- 172.21.15.42:0/1296736367 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1b80035f0 con 0x7fa1c406d8a0 2024-07-04T17:34:11.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.561+0000 7fa1cc600640 1 -- 172.21.15.42:0/1296736367 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1c414b090 con 0x7fa1c406d8a0 2024-07-04T17:34:11.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.561+0000 7fa1cc600640 1 -- 172.21.15.42:0/1296736367 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1c414ae60 con 0x7fa1c406d8a0 2024-07-04T17:34:11.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.562+0000 7fa1cc600640 1 -- 172.21.15.42:0/1296736367 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1c40008d0 con 0x7fa1c406d8a0 2024-07-04T17:34:11.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.563+0000 7fa1c1600640 1 -- 172.21.15.42:0/1296736367 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 48) ==== 49668+0+0 (secure 0 0 0) 0x7fa1b801a070 con 0x7fa1c406d8a0 2024-07-04T17:34:11.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.563+0000 7fa1c1600640 1 --2- 172.21.15.42:0/1296736367 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7fa1a403d580 0x7fa1a403fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:11.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.563+0000 7fa1c1600640 1 -- 172.21.15.42:0/1296736367 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 4776+0+0 (secure 0 0 0) 0x7fa1b8051220 con 0x7fa1c406d8a0 2024-07-04T17:34:11.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.563+0000 7fa1c9200640 1 -- 172.21.15.42:0/1296736367 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7fa1a403d580 msgr2=0x7fa1a403fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1310962821 2024-07-04T17:34:11.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.563+0000 7fa1c9200640 1 --2- 172.21.15.42:0/1296736367 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7fa1a403d580 0x7fa1a403fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:34:11.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.569+0000 7fa1c1600640 1 -- 172.21.15.42:0/1296736367 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fa1b80102e0 con 0x7fa1c406d8a0 2024-07-04T17:34:11.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.764+0000 7fa1c9200640 1 -- 172.21.15.42:0/1296736367 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7fa1a403d580 msgr2=0x7fa1a403fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1310962821 2024-07-04T17:34:11.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.764+0000 7fa1c9200640 1 --2- 172.21.15.42:0/1296736367 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7fa1a403d580 0x7fa1a403fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:34:11.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.777+0000 7fa1cc600640 1 -- 172.21.15.42:0/1296736367 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa1c4148ce0 con 0x7fa1c406d8a0 2024-07-04T17:34:11.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.779+0000 7fa1c1600640 1 -- 172.21.15.42:0/1296736367 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) ==== 74+0+15144 (secure 0 0 0) 0x7fa1b8011910 con 0x7fa1c406d8a0 2024-07-04T17:34:11.781 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:34:11.782 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":120,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:33:57.194026+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:33:06.769774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-07-04T17:33:07.613572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:33:26.476351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:33:27.929016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1699999570846558,"score_stable":1.1699999570846558,"optimal_score":1,"raw_score_acting":1.1699999570846558,"raw_score_stable":1.1699999570846558,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/2238986031":"2024-07-04T18:33:57.174196+0000","172.21.15.42:0/3738568139":"2024-07-05T17:33:42.103205+0000","172.21.15.42:6835/4130303917":"2024-07-05T17:33:42.103205+0000","172.21.15.42:6834/4130303917":"2024-07-05T17:33:42.103205+0000","172.21.15.42:0/1148975990":"2024-07-05T17:33:42.103205+0000","172.21.15.42:0/1387747213":"2024-07-05T17:33:42.103205+0000","172.21.15.42:0/2342229102":"2024-07-05T17:33:42.103205+0000","172.21.15.42:0/3100365373":"2024-07-05T17:33:42.103205+0000","172.21.15.42:0/3283150650":"2024-07-05T17:33:42.103205+0000","172.21.15.42:0/701226807":"2024-07-05T17:33:42.103205+0000","172.21.15.42:0/4022678544":"2024-07-05T17:33:42.103205+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:34:11.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.781+0000 7fa1cc600640 1 -- 172.21.15.42:0/1296736367 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7fa1a403d580 msgr2=0x7fa1a403fa90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:34:11.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.781+0000 7fa1cc600640 1 --2- 172.21.15.42:0/1296736367 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7fa1a403d580 0x7fa1a403fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:11.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.781+0000 7fa1cc600640 1 -- 172.21.15.42:0/1296736367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1c406d8a0 msgr2=0x7fa1c41d6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:11.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.781+0000 7fa1cc600640 1 --2- 172.21.15.42:0/1296736367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1c406d8a0 0x7fa1c41d6c90 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8001eb0 tx=0x7fa1b800c810 comp rx=0 tx=0).stop 2024-07-04T17:34:11.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.781+0000 7fa1cc600640 1 -- 172.21.15.42:0/1296736367 shutdown_connections 2024-07-04T17:34:11.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.781+0000 7fa1cc600640 1 --2- 172.21.15.42:0/1296736367 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7fa1a403d580 0x7fa1a403fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:11.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.781+0000 7fa1cc600640 1 --2- 172.21.15.42:0/1296736367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1c406d8a0 0x7fa1c41d6c90 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:11.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.781+0000 7fa1cc600640 1 -- 172.21.15.42:0/1296736367 >> 172.21.15.42:0/1296736367 conn(0x7fa1c40c03e0 msgr2=0x7fa1c40be3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:11.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.781+0000 7fa1cc600640 1 -- 172.21.15.42:0/1296736367 shutdown_connections 2024-07-04T17:34:11.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:11.782+0000 7fa1cc600640 1 -- 172.21.15.42:0/1296736367 wait complete. 2024-07-04T17:34:11.807 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-04T17:34:11.807 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:34:12.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.078+0000 7f88e4c00640 1 -- 172.21.15.42:0/1816091236 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88e00b3030 msgr2=0x7f88e00b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:12.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.078+0000 7f88e4c00640 1 --2- 172.21.15.42:0/1816091236 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88e00b3030 0x7f88e00b01f0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f88c800b2b0 tx=0x7f88c802ad10 comp rx=0 tx=0).stop 2024-07-04T17:34:12.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.078+0000 7f88e4c00640 1 -- 172.21.15.42:0/1816091236 shutdown_connections 2024-07-04T17:34:12.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.078+0000 7f88e4c00640 1 --2- 172.21.15.42:0/1816091236 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88e00b3030 0x7f88e00b01f0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:12.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.078+0000 7f88e4c00640 1 -- 172.21.15.42:0/1816091236 >> 172.21.15.42:0/1816091236 conn(0x7f88e00bf700 msgr2=0x7f88e00af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:12.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.078+0000 7f88e4c00640 1 -- 172.21.15.42:0/1816091236 shutdown_connections 2024-07-04T17:34:12.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.078+0000 7f88e4c00640 1 -- 172.21.15.42:0/1816091236 wait complete. 2024-07-04T17:34:12.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.080+0000 7f88e4c00640 1 Processor -- start 2024-07-04T17:34:12.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.080+0000 7f88e4c00640 1 -- start start 2024-07-04T17:34:12.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.082+0000 7f88e4c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88e006cea0 0x7f88e01d7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:12.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.082+0000 7f88e4c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f88e006ef50 con 0x7f88e006cea0 2024-07-04T17:34:12.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.082+0000 7f88dde00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88e006cea0 0x7f88e01d7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:12.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.082+0000 7f88dde00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88e006cea0 0x7f88e01d7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49680/0 (socket says 172.21.15.42:49680) 2024-07-04T17:34:12.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.082+0000 7f88dde00640 1 -- 172.21.15.42:0/2580625913 learned_addr learned my addr 172.21.15.42:0/2580625913 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:12.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.082+0000 7f88dde00640 1 -- 172.21.15.42:0/2580625913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e00ae910 con 0x7f88e006cea0 2024-07-04T17:34:12.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.083+0000 7f88dde00640 1 --2- 172.21.15.42:0/2580625913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88e006cea0 0x7f88e01d7be0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f88d000c380 tx=0x7f88d000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:12.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.083+0000 7f88d6000640 1 -- 172.21.15.42:0/2580625913 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88d00116c0 con 0x7f88e006cea0 2024-07-04T17:34:12.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.083+0000 7f88d6000640 1 -- 172.21.15.42:0/2580625913 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88d0017070 con 0x7f88e006cea0 2024-07-04T17:34:12.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.083+0000 7f88d6000640 1 -- 172.21.15.42:0/2580625913 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88d0011b20 con 0x7f88e006cea0 2024-07-04T17:34:12.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.083+0000 7f88e4c00640 1 -- 172.21.15.42:0/2580625913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88e01dce40 con 0x7f88e006cea0 2024-07-04T17:34:12.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.083+0000 7f88e4c00640 1 -- 172.21.15.42:0/2580625913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88e01d77c0 con 0x7f88e006cea0 2024-07-04T17:34:12.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.084+0000 7f88e4c00640 1 -- 172.21.15.42:0/2580625913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88e00b01f0 con 0x7f88e006cea0 2024-07-04T17:34:12.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.085+0000 7f88d6000640 1 -- 172.21.15.42:0/2580625913 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 48) ==== 49668+0+0 (secure 0 0 0) 0x7f88d0011cc0 con 0x7f88e006cea0 2024-07-04T17:34:12.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.085+0000 7f88d6000640 1 --2- 172.21.15.42:0/2580625913 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f88bc03d580 0x7f88bc03fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:12.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.085+0000 7f88d6000640 1 -- 172.21.15.42:0/2580625913 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 4776+0+0 (secure 0 0 0) 0x7f88d0051ca0 con 0x7f88e006cea0 2024-07-04T17:34:12.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.085+0000 7f88d7e00640 1 -- 172.21.15.42:0/2580625913 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f88bc03d580 msgr2=0x7f88bc03fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1310962821 2024-07-04T17:34:12.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.086+0000 7f88d7e00640 1 --2- 172.21.15.42:0/2580625913 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f88bc03d580 0x7f88bc03fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:34:12.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.090+0000 7f88d6000640 1 -- 172.21.15.42:0/2580625913 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f88d000a140 con 0x7f88e006cea0 2024-07-04T17:34:12.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.287+0000 7f88d7e00640 1 -- 172.21.15.42:0/2580625913 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f88bc03d580 msgr2=0x7f88bc03fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1310962821 2024-07-04T17:34:12.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.287+0000 7f88d7e00640 1 --2- 172.21.15.42:0/2580625913 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f88bc03d580 0x7f88bc03fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:34:12.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.343+0000 7f88e4c00640 1 -- 172.21.15.42:0/2580625913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88e00aeae0 con 0x7f88e006cea0 2024-07-04T17:34:12.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.344+0000 7f88d6000640 1 -- 172.21.15.42:0/2580625913 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+7354 (secure 0 0 0) 0x7f88d0010cf0 con 0x7f88e006cea0 2024-07-04T17:34:12.344 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:34:12.344 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":98,"btime":"2024-07-04T17:33:56:185420+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5536,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/68857121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":68857121},{"type":"v1","addr":"172.21.15.42:6825","nonce":68857121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":5544,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6828/20990456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":20990456},{"type":"v1","addr":"172.21.15.42:6828","nonce":20990456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":5545,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/428501060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6827","nonce":428501060},{"type":"v1","addr":"172.21.15.42:6829","nonce":428501060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:33:08.649551+0000","modified":"2024-07-04T17:33:55.506934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5550":{"gid":5550,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6833/605786437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":605786437},{"type":"v1","addr":"172.21.15.42:6833","nonce":605786437}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5550,"qdb_cluster":[5550]},"id":10},{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:33:28.944269+0000","modified":"2024-07-04T17:33:32.025418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5549},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5549":{"gid":5549,"name":"b","rank":0,"incarnation":92,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6831/3224527035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":3224527035},{"type":"v1","addr":"172.21.15.42:6831","nonce":3224527035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5549,"qdb_cluster":[5549]},"id":11}]} 2024-07-04T17:34:12.345 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 98 2024-07-04T17:34:12.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.346+0000 7f88e4c00640 1 -- 172.21.15.42:0/2580625913 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f88bc03d580 msgr2=0x7f88bc03fa90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:34:12.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.346+0000 7f88e4c00640 1 --2- 172.21.15.42:0/2580625913 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f88bc03d580 0x7f88bc03fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:12.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.347+0000 7f88e4c00640 1 -- 172.21.15.42:0/2580625913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88e006cea0 msgr2=0x7f88e01d7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:12.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.347+0000 7f88e4c00640 1 --2- 172.21.15.42:0/2580625913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88e006cea0 0x7f88e01d7be0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f88d000c380 tx=0x7f88d000c840 comp rx=0 tx=0).stop 2024-07-04T17:34:12.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.347+0000 7f88e4c00640 1 -- 172.21.15.42:0/2580625913 shutdown_connections 2024-07-04T17:34:12.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.347+0000 7f88e4c00640 1 --2- 172.21.15.42:0/2580625913 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f88bc03d580 0x7f88bc03fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:12.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.347+0000 7f88e4c00640 1 --2- 172.21.15.42:0/2580625913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88e006cea0 0x7f88e01d7be0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:12.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.347+0000 7f88e4c00640 1 -- 172.21.15.42:0/2580625913 >> 172.21.15.42:0/2580625913 conn(0x7f88e00bf700 msgr2=0x7f88e014c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:12.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.347+0000 7f88e4c00640 1 -- 172.21.15.42:0/2580625913 shutdown_connections 2024-07-04T17:34:12.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.347+0000 7f88e4c00640 1 -- 172.21.15.42:0/2580625913 wait complete. 2024-07-04T17:34:12.374 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:34:12.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.645+0000 7f7a37400640 1 -- 172.21.15.42:0/772541388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a3006cea0 msgr2=0x7f7a300b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:12.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.645+0000 7f7a37400640 1 --2- 172.21.15.42:0/772541388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a3006cea0 0x7f7a300b3030 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000afd0 tx=0x7f7a2002b5b0 comp rx=0 tx=0).stop 2024-07-04T17:34:12.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.645+0000 7f7a37400640 1 -- 172.21.15.42:0/772541388 shutdown_connections 2024-07-04T17:34:12.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.645+0000 7f7a37400640 1 --2- 172.21.15.42:0/772541388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a3006cea0 0x7f7a300b3030 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:12.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.645+0000 7f7a37400640 1 -- 172.21.15.42:0/772541388 >> 172.21.15.42:0/772541388 conn(0x7f7a300bf700 msgr2=0x7f7a300af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:12.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.645+0000 7f7a37400640 1 -- 172.21.15.42:0/772541388 shutdown_connections 2024-07-04T17:34:12.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.646+0000 7f7a37400640 1 -- 172.21.15.42:0/772541388 wait complete. 2024-07-04T17:34:12.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.648+0000 7f7a37400640 1 Processor -- start 2024-07-04T17:34:12.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.648+0000 7f7a37400640 1 -- start start 2024-07-04T17:34:12.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.649+0000 7f7a37400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a3006cea0 0x7f7a30148d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:12.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.649+0000 7f7a37400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3006ef50 con 0x7f7a3006cea0 2024-07-04T17:34:12.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.649+0000 7f7a34a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a3006cea0 0x7f7a30148d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:12.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.649+0000 7f7a34a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a3006cea0 0x7f7a30148d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49694/0 (socket says 172.21.15.42:49694) 2024-07-04T17:34:12.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.649+0000 7f7a34a00640 1 -- 172.21.15.42:0/117377903 learned_addr learned my addr 172.21.15.42:0/117377903 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:12.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.650+0000 7f7a34a00640 1 -- 172.21.15.42:0/117377903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a30149570 con 0x7f7a3006cea0 2024-07-04T17:34:12.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.650+0000 7f7a34a00640 1 --2- 172.21.15.42:0/117377903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a3006cea0 0x7f7a30148d80 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f7a200076a0 tx=0x7f7a20003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:12.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.650+0000 7f7a2cc00640 1 -- 172.21.15.42:0/117377903 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a20037040 con 0x7f7a3006cea0 2024-07-04T17:34:12.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.650+0000 7f7a2cc00640 1 -- 172.21.15.42:0/117377903 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a20004420 con 0x7f7a3006cea0 2024-07-04T17:34:12.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.650+0000 7f7a37400640 1 -- 172.21.15.42:0/117377903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a300708a0 con 0x7f7a3006cea0 2024-07-04T17:34:12.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.650+0000 7f7a2cc00640 1 -- 172.21.15.42:0/117377903 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a20004700 con 0x7f7a3006cea0 2024-07-04T17:34:12.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.651+0000 7f7a37400640 1 -- 172.21.15.42:0/117377903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a301497b0 con 0x7f7a3006cea0 2024-07-04T17:34:12.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.652+0000 7f7a37400640 1 -- 172.21.15.42:0/117377903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a300b3030 con 0x7f7a3006cea0 2024-07-04T17:34:12.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.652+0000 7f7a2cc00640 1 -- 172.21.15.42:0/117377903 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 48) ==== 49668+0+0 (secure 0 0 0) 0x7f7a20042030 con 0x7f7a3006cea0 2024-07-04T17:34:12.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.652+0000 7f7a2cc00640 1 --2- 172.21.15.42:0/117377903 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f7a0c03d580 0x7f7a0c03fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:12.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.653+0000 7f7a2cc00640 1 -- 172.21.15.42:0/117377903 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 4776+0+0 (secure 0 0 0) 0x7f7a20040070 con 0x7f7a3006cea0 2024-07-04T17:34:12.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.653+0000 7f7a2fe00640 1 -- 172.21.15.42:0/117377903 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f7a0c03d580 msgr2=0x7f7a0c03fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1310962821 2024-07-04T17:34:12.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.653+0000 7f7a2fe00640 1 --2- 172.21.15.42:0/117377903 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f7a0c03d580 0x7f7a0c03fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:34:12.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.662+0000 7f7a2cc00640 1 -- 172.21.15.42:0/117377903 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f7a20075030 con 0x7f7a3006cea0 2024-07-04T17:34:12.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.854+0000 7f7a2fe00640 1 -- 172.21.15.42:0/117377903 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f7a0c03d580 msgr2=0x7f7a0c03fa90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/1310962821 2024-07-04T17:34:12.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.854+0000 7f7a2fe00640 1 --2- 172.21.15.42:0/117377903 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f7a0c03d580 0x7f7a0c03fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:34:12.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.912+0000 7f7a37400640 1 -- 172.21.15.42:0/117377903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a30070a70 con 0x7f7a3006cea0 2024-07-04T17:34:12.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.913+0000 7f7a2cc00640 1 -- 172.21.15.42:0/117377903 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+7354 (secure 0 0 0) 0x7f7a20049020 con 0x7f7a3006cea0 2024-07-04T17:34:12.913 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:34:12.913 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":98,"btime":"2024-07-04T17:33:56:185420+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5536,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/68857121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":68857121},{"type":"v1","addr":"172.21.15.42:6825","nonce":68857121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":5544,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6828/20990456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":20990456},{"type":"v1","addr":"172.21.15.42:6828","nonce":20990456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":5545,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/428501060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6827","nonce":428501060},{"type":"v1","addr":"172.21.15.42:6829","nonce":428501060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:33:08.649551+0000","modified":"2024-07-04T17:33:55.506934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5550},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5550":{"gid":5550,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6833/605786437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":605786437},{"type":"v1","addr":"172.21.15.42:6833","nonce":605786437}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5550,"qdb_cluster":[5550]},"id":10},{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:33:28.944269+0000","modified":"2024-07-04T17:33:32.025418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5549},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5549":{"gid":5549,"name":"b","rank":0,"incarnation":92,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6831/3224527035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":3224527035},{"type":"v1","addr":"172.21.15.42:6831","nonce":3224527035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5549,"qdb_cluster":[5549]},"id":11}]} 2024-07-04T17:34:12.913 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 98 2024-07-04T17:34:12.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.915+0000 7f7a37400640 1 -- 172.21.15.42:0/117377903 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f7a0c03d580 msgr2=0x7f7a0c03fa90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:34:12.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.915+0000 7f7a37400640 1 --2- 172.21.15.42:0/117377903 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f7a0c03d580 0x7f7a0c03fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:12.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.916+0000 7f7a37400640 1 -- 172.21.15.42:0/117377903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a3006cea0 msgr2=0x7f7a30148d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:12.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.916+0000 7f7a37400640 1 --2- 172.21.15.42:0/117377903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a3006cea0 0x7f7a30148d80 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f7a200076a0 tx=0x7f7a20003f80 comp rx=0 tx=0).stop 2024-07-04T17:34:12.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.916+0000 7f7a37400640 1 -- 172.21.15.42:0/117377903 shutdown_connections 2024-07-04T17:34:12.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.916+0000 7f7a37400640 1 --2- 172.21.15.42:0/117377903 >> [v2:172.21.15.42:6834/1310962821,v1:172.21.15.42:6835/1310962821] conn(0x7f7a0c03d580 0x7f7a0c03fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:12.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.916+0000 7f7a37400640 1 --2- 172.21.15.42:0/117377903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a3006cea0 0x7f7a30148d80 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:12.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.916+0000 7f7a37400640 1 -- 172.21.15.42:0/117377903 >> 172.21.15.42:0/117377903 conn(0x7f7a300bf700 msgr2=0x7f7a3014a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:12.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.916+0000 7f7a37400640 1 -- 172.21.15.42:0/117377903 shutdown_connections 2024-07-04T17:34:12.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:12.916+0000 7f7a37400640 1 -- 172.21.15.42:0/117377903 wait complete. 2024-07-04T17:34:12.945 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:34:13.169 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:34:13.169+0000 7f252c600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:34:13.170 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:34:13.169+0000 7f252c600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:34:13.170 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:34:13.169+0000 7f252c600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:34:13.170 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:34:13.169+0000 7f252c600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:34:13.170 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:34:13.169+0000 7f252c600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:34:13.170 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:34:13.169+0000 7f2528000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:34:13.170 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:34:13.169+0000 7f2528000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:34:13.170 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:34:13.169+0000 7f2528000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:34:13.170 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:34:13.169+0000 7f2528000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:34:13.170 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:34:13.169+0000 7f2528000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:34:13.185 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:34:13.185+0000 7f252c600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:34:13.185 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:34:13.185+0000 7f252c600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:34:13.185 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:34:13.185+0000 7f252c600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:34:13.185 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:34:13.185+0000 7f252c600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:34:13.185 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:34:13.185+0000 7f252c600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:34:13.185 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:34:13.185+0000 7f2528000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:34:13.186 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:34:13.185+0000 7f2528000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:34:13.186 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:34:13.185+0000 7f2528000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:34:13.186 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:34:13.185+0000 7f2528000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:34:13.186 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:34:13.185+0000 7f2528000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:34:13.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.230+0000 7f6957000640 1 -- 172.21.15.42:0/3881282618 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69500b38b0 msgr2=0x7f69500b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:13.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.230+0000 7f6957000640 1 --2- 172.21.15.42:0/3881282618 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69500b38b0 0x7f69500b2d20 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f69400097b0 tx=0x7f694002a540 comp rx=0 tx=0).stop 2024-07-04T17:34:13.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.230+0000 7f6957000640 1 -- 172.21.15.42:0/3881282618 shutdown_connections 2024-07-04T17:34:13.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.230+0000 7f6957000640 1 --2- 172.21.15.42:0/3881282618 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69500b38b0 0x7f69500b2d20 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:13.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.230+0000 7f6957000640 1 -- 172.21.15.42:0/3881282618 >> 172.21.15.42:0/3881282618 conn(0x7f69500bf700 msgr2=0x7f69500af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:13.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.230+0000 7f6957000640 1 -- 172.21.15.42:0/3881282618 shutdown_connections 2024-07-04T17:34:13.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.230+0000 7f6957000640 1 -- 172.21.15.42:0/3881282618 wait complete. 2024-07-04T17:34:13.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.232+0000 7f6957000640 1 Processor -- start 2024-07-04T17:34:13.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.232+0000 7f6957000640 1 -- start start 2024-07-04T17:34:13.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.233+0000 7f6957000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69500b1e00 0x7f69501d4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:13.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.233+0000 7f6957000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f69500b06e0 con 0x7f69500b1e00 2024-07-04T17:34:13.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.234+0000 7f694fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69500b1e00 0x7f69501d4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:13.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.234+0000 7f694fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69500b1e00 0x7f69501d4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49832/0 (socket says 172.21.15.42:49832) 2024-07-04T17:34:13.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.234+0000 7f694fe00640 1 -- 172.21.15.42:0/1128282669 learned_addr learned my addr 172.21.15.42:0/1128282669 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:13.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.234+0000 7f694fe00640 1 -- 172.21.15.42:0/1128282669 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69501d6740 con 0x7f69500b1e00 2024-07-04T17:34:13.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.234+0000 7f694fe00640 1 --2- 172.21.15.42:0/1128282669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69500b1e00 0x7f69501d4df0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f69400098d0 tx=0x7f6940033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:13.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.235+0000 7f694cc00640 1 -- 172.21.15.42:0/1128282669 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6940031bc0 con 0x7f69500b1e00 2024-07-04T17:34:13.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.235+0000 7f694cc00640 1 -- 172.21.15.42:0/1128282669 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6940031e50 con 0x7f69500b1e00 2024-07-04T17:34:13.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.235+0000 7f694cc00640 1 -- 172.21.15.42:0/1128282669 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6940033200 con 0x7f69500b1e00 2024-07-04T17:34:13.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.235+0000 7f6957000640 1 -- 172.21.15.42:0/1128282669 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69500aa540 con 0x7f69500b1e00 2024-07-04T17:34:13.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.235+0000 7f6957000640 1 -- 172.21.15.42:0/1128282669 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69501d5330 con 0x7f69500b1e00 2024-07-04T17:34:13.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.236+0000 7f6957000640 1 -- 172.21.15.42:0/1128282669 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f695006c590 con 0x7f69500b1e00 2024-07-04T17:34:13.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.236+0000 7f694cc00640 1 -- 172.21.15.42:0/1128282669 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 49) ==== 49122+0+0 (secure 0 0 0) 0x7f69400333a0 con 0x7f69500b1e00 2024-07-04T17:34:13.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.237+0000 7f694cc00640 1 -- 172.21.15.42:0/1128282669 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5249+0+0 (secure 0 0 0) 0x7f6940072c80 con 0x7f69500b1e00 2024-07-04T17:34:13.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.241+0000 7f694cc00640 1 -- 172.21.15.42:0/1128282669 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f694003f030 con 0x7f69500b1e00 2024-07-04T17:34:13.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.455+0000 7f6957000640 1 -- 172.21.15.42:0/1128282669 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6950069b70 con 0x7f69500b1e00 2024-07-04T17:34:13.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.458+0000 7f694cc00640 1 -- 172.21.15.42:0/1128282669 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+15830 (secure 0 0 0) 0x7f694003a3a0 con 0x7f69500b1e00 2024-07-04T17:34:13.459 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:34:13.459 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":121,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:34:13.066265+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:33:06.769774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-07-04T17:33:07.613572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:33:26.476351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:33:27.929016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1699999570846558,"score_stable":1.1699999570846558,"optimal_score":1,"raw_score_acting":1.1699999570846558,"raw_score_stable":1.1699999570846558,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/2864803316":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/397279469":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/1279811784":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/3718814387":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/1854623919":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/3279442351":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/1060612519":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/1246900581":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/130985999":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/2238986031":"2024-07-04T18:33:57.174196+0000","172.21.15.42:6835/1310962821":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/3738568139":"2024-07-05T17:33:42.103205+0000","172.21.15.42:6835/4130303917":"2024-07-05T17:33:42.103205+0000","172.21.15.42:6834/4130303917":"2024-07-05T17:33:42.103205+0000","172.21.15.42:0/1148975990":"2024-07-05T17:33:42.103205+0000","172.21.15.42:0/1387747213":"2024-07-05T17:33:42.103205+0000","172.21.15.42:0/2342229102":"2024-07-05T17:33:42.103205+0000","172.21.15.42:0/3100365373":"2024-07-05T17:33:42.103205+0000","172.21.15.42:6834/1310962821":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/3283150650":"2024-07-05T17:33:42.103205+0000","172.21.15.42:0/701226807":"2024-07-05T17:33:42.103205+0000","172.21.15.42:0/4022678544":"2024-07-05T17:33:42.103205+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:34:13.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.461+0000 7f6957000640 1 -- 172.21.15.42:0/1128282669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69500b1e00 msgr2=0x7f69501d4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:13.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.461+0000 7f6957000640 1 --2- 172.21.15.42:0/1128282669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69500b1e00 0x7f69501d4df0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f69400098d0 tx=0x7f6940033e40 comp rx=0 tx=0).stop 2024-07-04T17:34:13.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.461+0000 7f6957000640 1 -- 172.21.15.42:0/1128282669 shutdown_connections 2024-07-04T17:34:13.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.461+0000 7f6957000640 1 --2- 172.21.15.42:0/1128282669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69500b1e00 0x7f69501d4df0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:13.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.461+0000 7f6957000640 1 -- 172.21.15.42:0/1128282669 >> 172.21.15.42:0/1128282669 conn(0x7f69500bf700 msgr2=0x7f69501412f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:13.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.462+0000 7f6957000640 1 -- 172.21.15.42:0/1128282669 shutdown_connections 2024-07-04T17:34:13.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.462+0000 7f6957000640 1 -- 172.21.15.42:0/1128282669 wait complete. 2024-07-04T17:34:13.487 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-07-04T17:34:13.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.775+0000 7f1858000640 1 -- 172.21.15.42:0/117380167 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18500abdd0 msgr2=0x7f18500b0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:13.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.775+0000 7f1858000640 1 --2- 172.21.15.42:0/117380167 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18500abdd0 0x7f18500b0cd0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f1844005fd0 tx=0x7f184402b930 comp rx=0 tx=0).stop 2024-07-04T17:34:13.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.775+0000 7f1858000640 1 -- 172.21.15.42:0/117380167 shutdown_connections 2024-07-04T17:34:13.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.775+0000 7f1858000640 1 --2- 172.21.15.42:0/117380167 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18500abdd0 0x7f18500b0cd0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:13.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.775+0000 7f1858000640 1 -- 172.21.15.42:0/117380167 >> 172.21.15.42:0/117380167 conn(0x7f18500bf700 msgr2=0x7f18500af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:13.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.775+0000 7f1858000640 1 -- 172.21.15.42:0/117380167 shutdown_connections 2024-07-04T17:34:13.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.775+0000 7f1858000640 1 -- 172.21.15.42:0/117380167 wait complete. 2024-07-04T17:34:13.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.777+0000 7f1858000640 1 Processor -- start 2024-07-04T17:34:13.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.778+0000 7f1858000640 1 -- start start 2024-07-04T17:34:13.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.779+0000 7f1858000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18500a9d20 0x7f185013f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:13.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.779+0000 7f1858000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f18500b03a0 con 0x7f18500a9d20 2024-07-04T17:34:13.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.779+0000 7f1855600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18500a9d20 0x7f185013f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:13.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.779+0000 7f1855600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18500a9d20 0x7f185013f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49846/0 (socket says 172.21.15.42:49846) 2024-07-04T17:34:13.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.779+0000 7f1855600640 1 -- 172.21.15.42:0/2993873888 learned_addr learned my addr 172.21.15.42:0/2993873888 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:13.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.780+0000 7f1855600640 1 -- 172.21.15.42:0/2993873888 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1850073f20 con 0x7f18500a9d20 2024-07-04T17:34:13.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.780+0000 7f1855600640 1 --2- 172.21.15.42:0/2993873888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18500a9d20 0x7f185013f120 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f1844036040 tx=0x7f18440045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:13.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.780+0000 7f184d600640 1 -- 172.21.15.42:0/2993873888 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1844004ae0 con 0x7f18500a9d20 2024-07-04T17:34:13.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.780+0000 7f184d600640 1 -- 172.21.15.42:0/2993873888 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1844004d70 con 0x7f18500a9d20 2024-07-04T17:34:13.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.780+0000 7f184d600640 1 -- 172.21.15.42:0/2993873888 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1844031630 con 0x7f18500a9d20 2024-07-04T17:34:13.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.780+0000 7f1858000640 1 -- 172.21.15.42:0/2993873888 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1850071100 con 0x7f18500a9d20 2024-07-04T17:34:13.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.781+0000 7f1858000640 1 -- 172.21.15.42:0/2993873888 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18501406c0 con 0x7f18500a9d20 2024-07-04T17:34:13.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.782+0000 7f1858000640 1 -- 172.21.15.42:0/2993873888 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18500b0cd0 con 0x7f18500a9d20 2024-07-04T17:34:13.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.782+0000 7f184d600640 1 -- 172.21.15.42:0/2993873888 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 49) ==== 49122+0+0 (secure 0 0 0) 0x7f1844042020 con 0x7f18500a9d20 2024-07-04T17:34:13.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.782+0000 7f184d600640 1 -- 172.21.15.42:0/2993873888 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5249+0+0 (secure 0 0 0) 0x7f1844036070 con 0x7f18500a9d20 2024-07-04T17:34:13.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:13.787+0000 7f184d600640 1 -- 172.21.15.42:0/2993873888 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f184404dcf0 con 0x7f18500a9d20 2024-07-04T17:34:14.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.040+0000 7f1858000640 1 -- 172.21.15.42:0/2993873888 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f18500712d0 con 0x7f18500a9d20 2024-07-04T17:34:14.040 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-07-04T17:34:14.041+0000 7f8616e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-04T17:34:14.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.042+0000 7f184d600640 1 -- 172.21.15.42:0/2993873888 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v99) ==== 161+0+0 (secure 0 0 0) 0x7f1844031a60 con 0x7f18500a9d20 2024-07-04T17:34:14.043 INFO:teuthology.orchestra.run.smithi042.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-04T17:34:14.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.046+0000 7f1858000640 1 -- 172.21.15.42:0/2993873888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18500a9d20 msgr2=0x7f185013f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:14.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.046+0000 7f1858000640 1 --2- 172.21.15.42:0/2993873888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18500a9d20 0x7f185013f120 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f1844036040 tx=0x7f18440045e0 comp rx=0 tx=0).stop 2024-07-04T17:34:14.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.046+0000 7f1858000640 1 -- 172.21.15.42:0/2993873888 shutdown_connections 2024-07-04T17:34:14.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.046+0000 7f1858000640 1 --2- 172.21.15.42:0/2993873888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18500a9d20 0x7f185013f120 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:14.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.046+0000 7f1858000640 1 -- 172.21.15.42:0/2993873888 >> 172.21.15.42:0/2993873888 conn(0x7f18500bf700 msgr2=0x7f185006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:14.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.047+0000 7f1858000640 1 -- 172.21.15.42:0/2993873888 shutdown_connections 2024-07-04T17:34:14.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.047+0000 7f1858000640 1 -- 172.21.15.42:0/2993873888 wait complete. 2024-07-04T17:34:14.083 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-07-04T17:34:14.207 INFO:tasks.ceph.mds.c.smithi042.stdout:starting mds.c at 2024-07-04T17:34:14.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.349+0000 7f281ea00640 1 -- 172.21.15.42:0/1368273464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28180b1930 msgr2=0x7f28180b1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:14.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.349+0000 7f281ea00640 1 --2- 172.21.15.42:0/1368273464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28180b1930 0x7f28180b1d40 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f28080097b0 tx=0x7f280802a540 comp rx=0 tx=0).stop 2024-07-04T17:34:14.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.349+0000 7f281ea00640 1 -- 172.21.15.42:0/1368273464 shutdown_connections 2024-07-04T17:34:14.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.349+0000 7f281ea00640 1 --2- 172.21.15.42:0/1368273464 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28180b1930 0x7f28180b1d40 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:14.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.349+0000 7f281ea00640 1 -- 172.21.15.42:0/1368273464 >> 172.21.15.42:0/1368273464 conn(0x7f281806fe40 msgr2=0x7f281806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:14.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.349+0000 7f281ea00640 1 -- 172.21.15.42:0/1368273464 shutdown_connections 2024-07-04T17:34:14.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.349+0000 7f281ea00640 1 -- 172.21.15.42:0/1368273464 wait complete. 2024-07-04T17:34:14.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.352+0000 7f281ea00640 1 Processor -- start 2024-07-04T17:34:14.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.352+0000 7f281ea00640 1 -- start start 2024-07-04T17:34:14.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.353+0000 7f281ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28180b02b0 0x7f28181d78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:14.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.353+0000 7f281ea00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2818141790 con 0x7f28180b02b0 2024-07-04T17:34:14.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.353+0000 7f2817e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28180b02b0 0x7f28181d78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:14.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.354+0000 7f2817e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28180b02b0 0x7f28181d78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32824/0 (socket says 172.21.15.42:32824) 2024-07-04T17:34:14.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.354+0000 7f2817e00640 1 -- 172.21.15.42:0/3813550838 learned_addr learned my addr 172.21.15.42:0/3813550838 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:14.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.354+0000 7f2817e00640 1 -- 172.21.15.42:0/3813550838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281814fab0 con 0x7f28180b02b0 2024-07-04T17:34:14.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.354+0000 7f2817e00640 1 --2- 172.21.15.42:0/3813550838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28180b02b0 0x7f28181d78e0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f280400c3b0 tx=0x7f280400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:14.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.354+0000 7f280fe00640 1 -- 172.21.15.42:0/3813550838 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2804011780 con 0x7f28180b02b0 2024-07-04T17:34:14.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.355+0000 7f280fe00640 1 -- 172.21.15.42:0/3813550838 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2804017070 con 0x7f28180b02b0 2024-07-04T17:34:14.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.355+0000 7f281ea00640 1 -- 172.21.15.42:0/3813550838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2818072270 con 0x7f28180b02b0 2024-07-04T17:34:14.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.355+0000 7f280fe00640 1 -- 172.21.15.42:0/3813550838 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2804011c00 con 0x7f28180b02b0 2024-07-04T17:34:14.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.355+0000 7f281ea00640 1 -- 172.21.15.42:0/3813550838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2818071c30 con 0x7f28180b02b0 2024-07-04T17:34:14.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.356+0000 7f281ea00640 1 -- 172.21.15.42:0/3813550838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2818140930 con 0x7f28180b02b0 2024-07-04T17:34:14.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.356+0000 7f280fe00640 1 -- 172.21.15.42:0/3813550838 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 50) ==== 49249+0+0 (secure 0 0 0) 0x7f280401c460 con 0x7f28180b02b0 2024-07-04T17:34:14.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.357+0000 7f280fe00640 1 --2- 172.21.15.42:0/3813550838 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f27f803d100 0x7f27f803f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:14.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.357+0000 7f280fe00640 1 -- 172.21.15.42:0/3813550838 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5335+0+0 (secure 0 0 0) 0x7f28040516f0 con 0x7f28180b02b0 2024-07-04T17:34:14.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.357+0000 7f2817400640 1 --2- 172.21.15.42:0/3813550838 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f27f803d100 0x7f27f803f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:14.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.358+0000 7f2817400640 1 --2- 172.21.15.42:0/3813550838 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f27f803d100 0x7f27f803f610 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2808034000 tx=0x7f2808004730 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:14.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.362+0000 7f280fe00640 1 -- 172.21.15.42:0/3813550838 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f280400a140 con 0x7f28180b02b0 2024-07-04T17:34:14.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:14.606+0000 7f281ea00640 1 -- 172.21.15.42:0/3813550838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2818069b70 con 0x7f28180b02b0 2024-07-04T17:34:15.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.082+0000 7f280fe00640 1 -- 172.21.15.42:0/3813550838 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v100) ==== 104+0+0 (secure 0 0 0) 0x7f2804051030 con 0x7f28180b02b0 2024-07-04T17:34:15.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.086+0000 7f281ea00640 1 -- 172.21.15.42:0/3813550838 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f27f803d100 msgr2=0x7f27f803f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:15.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.086+0000 7f281ea00640 1 --2- 172.21.15.42:0/3813550838 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f27f803d100 0x7f27f803f610 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2808034000 tx=0x7f2808004730 comp rx=0 tx=0).stop 2024-07-04T17:34:15.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.086+0000 7f281ea00640 1 -- 172.21.15.42:0/3813550838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28180b02b0 msgr2=0x7f28181d78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:15.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.086+0000 7f281ea00640 1 --2- 172.21.15.42:0/3813550838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28180b02b0 0x7f28181d78e0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f280400c3b0 tx=0x7f280400c870 comp rx=0 tx=0).stop 2024-07-04T17:34:15.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.086+0000 7f281ea00640 1 -- 172.21.15.42:0/3813550838 shutdown_connections 2024-07-04T17:34:15.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.086+0000 7f281ea00640 1 --2- 172.21.15.42:0/3813550838 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f27f803d100 0x7f27f803f610 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:15.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.086+0000 7f281ea00640 1 --2- 172.21.15.42:0/3813550838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28180b02b0 0x7f28181d78e0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:15.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.086+0000 7f281ea00640 1 -- 172.21.15.42:0/3813550838 >> 172.21.15.42:0/3813550838 conn(0x7f281806fe40 msgr2=0x7f281813e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:15.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.086+0000 7f281ea00640 1 -- 172.21.15.42:0/3813550838 shutdown_connections 2024-07-04T17:34:15.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.086+0000 7f281ea00640 1 -- 172.21.15.42:0/3813550838 wait complete. 2024-07-04T17:34:15.116 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-04T17:34:15.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.393+0000 7efe6ce00640 1 -- 172.21.15.42:0/170101394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe680ac9d0 msgr2=0x7efe680aef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:15.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.393+0000 7efe6ce00640 1 --2- 172.21.15.42:0/170101394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe680ac9d0 0x7efe680aef80 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7efe50005fd0 tx=0x7efe5002a560 comp rx=0 tx=0).stop 2024-07-04T17:34:15.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.393+0000 7efe6ce00640 1 -- 172.21.15.42:0/170101394 shutdown_connections 2024-07-04T17:34:15.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.393+0000 7efe6ce00640 1 --2- 172.21.15.42:0/170101394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe680ac9d0 0x7efe680aef80 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:15.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.393+0000 7efe6ce00640 1 -- 172.21.15.42:0/170101394 >> 172.21.15.42:0/170101394 conn(0x7efe680afaa0 msgr2=0x7efe680afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:15.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.394+0000 7efe6ce00640 1 -- 172.21.15.42:0/170101394 shutdown_connections 2024-07-04T17:34:15.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.394+0000 7efe6ce00640 1 -- 172.21.15.42:0/170101394 wait complete. 2024-07-04T17:34:15.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.396+0000 7efe6ce00640 1 Processor -- start 2024-07-04T17:34:15.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.396+0000 7efe6ce00640 1 -- start start 2024-07-04T17:34:15.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.397+0000 7efe6ce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe680ac9d0 0x7efe68142cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:15.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.397+0000 7efe6ce00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efe681415a0 con 0x7efe680ac9d0 2024-07-04T17:34:15.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.397+0000 7efe67400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe680ac9d0 0x7efe68142cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:15.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.397+0000 7efe67400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe680ac9d0 0x7efe68142cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32848/0 (socket says 172.21.15.42:32848) 2024-07-04T17:34:15.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.397+0000 7efe67400640 1 -- 172.21.15.42:0/2075400793 learned_addr learned my addr 172.21.15.42:0/2075400793 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:15.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.398+0000 7efe67400640 1 -- 172.21.15.42:0/2075400793 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe680aaa70 con 0x7efe680ac9d0 2024-07-04T17:34:15.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.398+0000 7efe67400640 1 --2- 172.21.15.42:0/2075400793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe680ac9d0 0x7efe68142cf0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7efe5800c340 tx=0x7efe5800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:15.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.398+0000 7efe5f400640 1 -- 172.21.15.42:0/2075400793 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe58005410 con 0x7efe680ac9d0 2024-07-04T17:34:15.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.398+0000 7efe5f400640 1 -- 172.21.15.42:0/2075400793 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe58007030 con 0x7efe680ac9d0 2024-07-04T17:34:15.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.398+0000 7efe5f400640 1 -- 172.21.15.42:0/2075400793 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe580073f0 con 0x7efe680ac9d0 2024-07-04T17:34:15.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.398+0000 7efe6ce00640 1 -- 172.21.15.42:0/2075400793 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe680aad10 con 0x7efe680ac9d0 2024-07-04T17:34:15.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.399+0000 7efe6ce00640 1 -- 172.21.15.42:0/2075400793 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe68144790 con 0x7efe680ac9d0 2024-07-04T17:34:15.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.400+0000 7efe6ce00640 1 -- 172.21.15.42:0/2075400793 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe68141c70 con 0x7efe680ac9d0 2024-07-04T17:34:15.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.400+0000 7efe5f400640 1 -- 172.21.15.42:0/2075400793 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 51) ==== 49559+0+0 (secure 0 0 0) 0x7efe5801a020 con 0x7efe680ac9d0 2024-07-04T17:34:15.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.400+0000 7efe5f400640 1 --2- 172.21.15.42:0/2075400793 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7efe4403d460 0x7efe4403f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:15.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.400+0000 7efe5f400640 1 -- 172.21.15.42:0/2075400793 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5335+0+0 (secure 0 0 0) 0x7efe58051620 con 0x7efe680ac9d0 2024-07-04T17:34:15.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.401+0000 7efe5fe00640 1 --2- 172.21.15.42:0/2075400793 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7efe4403d460 0x7efe4403f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:15.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.401+0000 7efe5fe00640 1 --2- 172.21.15.42:0/2075400793 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7efe4403d460 0x7efe4403f970 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7efe50033000 tx=0x7efe50004730 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:15.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.405+0000 7efe5f400640 1 -- 172.21.15.42:0/2075400793 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7efe680b1280 con 0x7efe680ac9d0 2024-07-04T17:34:15.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:15.614+0000 7efe6ce00640 1 -- 172.21.15.42:0/2075400793 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7efe680aaee0 con 0x7efe680ac9d0 2024-07-04T17:34:16.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.268+0000 7efe5f400640 1 -- 172.21.15.42:0/2075400793 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v123) ==== 181+0+0 (secure 0 0 0) 0x7efe58007590 con 0x7efe680ac9d0 2024-07-04T17:34:16.268 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' removed 2024-07-04T17:34:16.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.271+0000 7efe6ce00640 1 -- 172.21.15.42:0/2075400793 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7efe4403d460 msgr2=0x7efe4403f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:16.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.271+0000 7efe6ce00640 1 --2- 172.21.15.42:0/2075400793 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7efe4403d460 0x7efe4403f970 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7efe50033000 tx=0x7efe50004730 comp rx=0 tx=0).stop 2024-07-04T17:34:16.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.271+0000 7efe6ce00640 1 -- 172.21.15.42:0/2075400793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe680ac9d0 msgr2=0x7efe68142cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:16.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.271+0000 7efe6ce00640 1 --2- 172.21.15.42:0/2075400793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe680ac9d0 0x7efe68142cf0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7efe5800c340 tx=0x7efe5800c800 comp rx=0 tx=0).stop 2024-07-04T17:34:16.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.272+0000 7efe6ce00640 1 -- 172.21.15.42:0/2075400793 shutdown_connections 2024-07-04T17:34:16.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.272+0000 7efe6ce00640 1 --2- 172.21.15.42:0/2075400793 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7efe4403d460 0x7efe4403f970 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:16.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.272+0000 7efe6ce00640 1 --2- 172.21.15.42:0/2075400793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe680ac9d0 0x7efe68142cf0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:16.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.272+0000 7efe6ce00640 1 -- 172.21.15.42:0/2075400793 >> 172.21.15.42:0/2075400793 conn(0x7efe680afaa0 msgr2=0x7efe6813f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:16.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.272+0000 7efe6ce00640 1 -- 172.21.15.42:0/2075400793 shutdown_connections 2024-07-04T17:34:16.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.272+0000 7efe6ce00640 1 -- 172.21.15.42:0/2075400793 wait complete. 2024-07-04T17:34:16.309 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-04T17:34:16.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.603+0000 7ffa3a200640 1 -- 172.21.15.42:0/2693334516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa34073470 msgr2=0x7ffa340a9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:16.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.603+0000 7ffa3a200640 1 --2- 172.21.15.42:0/2693334516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa34073470 0x7ffa340a9990 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c005fd0 tx=0x7ffa1c02a5f0 comp rx=0 tx=0).stop 2024-07-04T17:34:16.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.603+0000 7ffa3a200640 1 -- 172.21.15.42:0/2693334516 shutdown_connections 2024-07-04T17:34:16.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.603+0000 7ffa3a200640 1 --2- 172.21.15.42:0/2693334516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa34073470 0x7ffa340a9990 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:16.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.603+0000 7ffa3a200640 1 -- 172.21.15.42:0/2693334516 >> 172.21.15.42:0/2693334516 conn(0x7ffa3406c020 msgr2=0x7ffa3406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:16.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.603+0000 7ffa3a200640 1 -- 172.21.15.42:0/2693334516 shutdown_connections 2024-07-04T17:34:16.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.603+0000 7ffa3a200640 1 -- 172.21.15.42:0/2693334516 wait complete. 2024-07-04T17:34:16.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.606+0000 7ffa3a200640 1 Processor -- start 2024-07-04T17:34:16.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.606+0000 7ffa3a200640 1 -- start start 2024-07-04T17:34:16.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.608+0000 7ffa3a200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa34073470 0x7ffa341d67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:16.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.608+0000 7ffa3a200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3413f430 con 0x7ffa34073470 2024-07-04T17:34:16.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.608+0000 7ffa33400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa34073470 0x7ffa341d67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:16.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.608+0000 7ffa33400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa34073470 0x7ffa341d67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32860/0 (socket says 172.21.15.42:32860) 2024-07-04T17:34:16.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.608+0000 7ffa33400640 1 -- 172.21.15.42:0/3021732934 learned_addr learned my addr 172.21.15.42:0/3021732934 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:16.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.609+0000 7ffa33400640 1 -- 172.21.15.42:0/3021732934 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa340ac4b0 con 0x7ffa34073470 2024-07-04T17:34:16.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.609+0000 7ffa33400640 1 --2- 172.21.15.42:0/3021732934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa34073470 0x7ffa341d67c0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400c380 tx=0x7ffa2400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:16.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.609+0000 7ffa2b400640 1 -- 172.21.15.42:0/3021732934 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa24017070 con 0x7ffa34073470 2024-07-04T17:34:16.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.609+0000 7ffa2b400640 1 -- 172.21.15.42:0/3021732934 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa240052f0 con 0x7ffa34073470 2024-07-04T17:34:16.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.609+0000 7ffa2b400640 1 -- 172.21.15.42:0/3021732934 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa240035f0 con 0x7ffa34073470 2024-07-04T17:34:16.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.609+0000 7ffa3a200640 1 -- 172.21.15.42:0/3021732934 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa341e43b0 con 0x7ffa34073470 2024-07-04T17:34:16.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.610+0000 7ffa3a200640 1 -- 172.21.15.42:0/3021732934 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa340ab9c0 con 0x7ffa34073470 2024-07-04T17:34:16.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.611+0000 7ffa2b400640 1 -- 172.21.15.42:0/3021732934 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 51) ==== 49559+0+0 (secure 0 0 0) 0x7ffa2401a070 con 0x7ffa34073470 2024-07-04T17:34:16.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.611+0000 7ffa3a200640 1 -- 172.21.15.42:0/3021732934 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa341e3310 con 0x7ffa34073470 2024-07-04T17:34:16.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.611+0000 7ffa2b400640 1 --2- 172.21.15.42:0/3021732934 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7ffa1003d460 0x7ffa1003f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:16.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.611+0000 7ffa2b400640 1 -- 172.21.15.42:0/3021732934 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 4897+0+0 (secure 0 0 0) 0x7ffa240511c0 con 0x7ffa34073470 2024-07-04T17:34:16.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.612+0000 7ffa2be00640 1 --2- 172.21.15.42:0/3021732934 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7ffa1003d460 0x7ffa1003f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:16.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.613+0000 7ffa2be00640 1 --2- 172.21.15.42:0/3021732934 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7ffa1003d460 0x7ffa1003f970 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c033000 tx=0x7ffa1c004730 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:16.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.617+0000 7ffa2b400640 1 -- 172.21.15.42:0/3021732934 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7ffa24016170 con 0x7ffa34073470 2024-07-04T17:34:16.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:16.824+0000 7ffa3a200640 1 -- 172.21.15.42:0/3021732934 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ffa3413fb00 con 0x7ffa34073470 2024-07-04T17:34:17.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.276+0000 7ffa2b400640 1 -- 172.21.15.42:0/3021732934 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v124) ==== 169+0+0 (secure 0 0 0) 0x7ffa240192c0 con 0x7ffa34073470 2024-07-04T17:34:17.277 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' removed 2024-07-04T17:34:17.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.280+0000 7ffa3a200640 1 -- 172.21.15.42:0/3021732934 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7ffa1003d460 msgr2=0x7ffa1003f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:17.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.280+0000 7ffa3a200640 1 --2- 172.21.15.42:0/3021732934 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7ffa1003d460 0x7ffa1003f970 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c033000 tx=0x7ffa1c004730 comp rx=0 tx=0).stop 2024-07-04T17:34:17.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.280+0000 7ffa3a200640 1 -- 172.21.15.42:0/3021732934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa34073470 msgr2=0x7ffa341d67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:17.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.280+0000 7ffa3a200640 1 --2- 172.21.15.42:0/3021732934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa34073470 0x7ffa341d67c0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400c380 tx=0x7ffa2400c840 comp rx=0 tx=0).stop 2024-07-04T17:34:17.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.280+0000 7ffa3a200640 1 -- 172.21.15.42:0/3021732934 shutdown_connections 2024-07-04T17:34:17.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.280+0000 7ffa3a200640 1 --2- 172.21.15.42:0/3021732934 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7ffa1003d460 0x7ffa1003f970 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:17.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.280+0000 7ffa3a200640 1 --2- 172.21.15.42:0/3021732934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa34073470 0x7ffa341d67c0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:17.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.280+0000 7ffa3a200640 1 -- 172.21.15.42:0/3021732934 >> 172.21.15.42:0/3021732934 conn(0x7ffa3406c020 msgr2=0x7ffa3406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:17.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.281+0000 7ffa3a200640 1 -- 172.21.15.42:0/3021732934 shutdown_connections 2024-07-04T17:34:17.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.285+0000 7ffa3a200640 1 -- 172.21.15.42:0/3021732934 wait complete. 2024-07-04T17:34:17.325 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:34:17.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.590+0000 7f8b4d800640 1 -- 172.21.15.42:0/162243176 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b480b1d20 msgr2=0x7f8b480b6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:17.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.590+0000 7f8b4d800640 1 --2- 172.21.15.42:0/162243176 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b480b1d20 0x7f8b480b6230 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f8b300097b0 tx=0x7f8b3002a540 comp rx=0 tx=0).stop 2024-07-04T17:34:17.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.590+0000 7f8b4d800640 1 -- 172.21.15.42:0/162243176 shutdown_connections 2024-07-04T17:34:17.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.590+0000 7f8b4d800640 1 --2- 172.21.15.42:0/162243176 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b480b1d20 0x7f8b480b6230 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:17.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.590+0000 7f8b4d800640 1 -- 172.21.15.42:0/162243176 >> 172.21.15.42:0/162243176 conn(0x7f8b480bebc0 msgr2=0x7f8b480befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:17.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.591+0000 7f8b4d800640 1 -- 172.21.15.42:0/162243176 shutdown_connections 2024-07-04T17:34:17.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.591+0000 7f8b4d800640 1 -- 172.21.15.42:0/162243176 wait complete. 2024-07-04T17:34:17.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.593+0000 7f8b4d800640 1 Processor -- start 2024-07-04T17:34:17.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.593+0000 7f8b4d800640 1 -- start start 2024-07-04T17:34:17.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.594+0000 7f8b4d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b480b1d20 0x7f8b480ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:17.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.594+0000 7f8b4d800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8b480b8b40 con 0x7f8b480b1d20 2024-07-04T17:34:17.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.595+0000 7f8b47e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b480b1d20 0x7f8b480ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:17.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.595+0000 7f8b47e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b480b1d20 0x7f8b480ad8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32880/0 (socket says 172.21.15.42:32880) 2024-07-04T17:34:17.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.595+0000 7f8b47e00640 1 -- 172.21.15.42:0/3644174475 learned_addr learned my addr 172.21.15.42:0/3644174475 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:17.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.595+0000 7f8b47e00640 1 -- 172.21.15.42:0/3644174475 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b481dcc50 con 0x7f8b480b1d20 2024-07-04T17:34:17.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.595+0000 7f8b47e00640 1 --2- 172.21.15.42:0/3644174475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b480b1d20 0x7f8b480ad8e0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f8b3800c6b0 tx=0x7f8b3800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:17.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.596+0000 7f8b44c00640 1 -- 172.21.15.42:0/3644174475 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b38016020 con 0x7f8b480b1d20 2024-07-04T17:34:17.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.596+0000 7f8b44c00640 1 -- 172.21.15.42:0/3644174475 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b3800a5c0 con 0x7f8b480b1d20 2024-07-04T17:34:17.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.596+0000 7f8b44c00640 1 -- 172.21.15.42:0/3644174475 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b3800a960 con 0x7f8b480b1d20 2024-07-04T17:34:17.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.596+0000 7f8b4d800640 1 -- 172.21.15.42:0/3644174475 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b481d72f0 con 0x7f8b480b1d20 2024-07-04T17:34:17.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.596+0000 7f8b4d800640 1 -- 172.21.15.42:0/3644174475 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b48140ea0 con 0x7f8b480b1d20 2024-07-04T17:34:17.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.597+0000 7f8b4d800640 1 -- 172.21.15.42:0/3644174475 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b480b91d0 con 0x7f8b480b1d20 2024-07-04T17:34:17.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.597+0000 7f8b44c00640 1 -- 172.21.15.42:0/3644174475 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f8b38019020 con 0x7f8b480b1d20 2024-07-04T17:34:17.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.598+0000 7f8b44c00640 1 --2- 172.21.15.42:0/3644174475 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f8b2403d4f0 0x7f8b2403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:17.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.598+0000 7f8b44c00640 1 -- 172.21.15.42:0/3644174475 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 4483+0+0 (secure 0 0 0) 0x7f8b38051c50 con 0x7f8b480b1d20 2024-07-04T17:34:17.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.598+0000 7f8b3fe00640 1 --2- 172.21.15.42:0/3644174475 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f8b2403d4f0 0x7f8b2403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:17.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.599+0000 7f8b3fe00640 1 --2- 172.21.15.42:0/3644174475 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f8b2403d4f0 0x7f8b2403fa00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8b30034000 tx=0x7f8b30004730 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:17.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.603+0000 7f8b44c00640 1 -- 172.21.15.42:0/3644174475 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f8b38020020 con 0x7f8b480b1d20 2024-07-04T17:34:17.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.850+0000 7f8b4d800640 1 -- 172.21.15.42:0/3644174475 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b48068e30 con 0x7f8b480b1d20 2024-07-04T17:34:17.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.851+0000 7f8b44c00640 1 -- 172.21.15.42:0/3644174475 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+6036 (secure 0 0 0) 0x7f8b4813ec10 con 0x7f8b480b1d20 2024-07-04T17:34:17.852 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:34:17.852 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":100,"btime":"2024-07-04T17:34:15:076257+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5536,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/68857121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":68857121},{"type":"v1","addr":"172.21.15.42:6825","nonce":68857121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":5544,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6828/20990456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":20990456},{"type":"v1","addr":"172.21.15.42:6828","nonce":20990456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":5545,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/428501060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6827","nonce":428501060},{"type":"v1","addr":"172.21.15.42:6829","nonce":428501060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":5736,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6833/94781196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":94781196},{"type":"v1","addr":"172.21.15.42:6833","nonce":94781196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:33:28.944269+0000","modified":"2024-07-04T17:33:32.025418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5549},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5549":{"gid":5549,"name":"b","rank":0,"incarnation":92,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6831/3224527035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":3224527035},{"type":"v1","addr":"172.21.15.42:6831","nonce":3224527035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5549,"qdb_cluster":[5549]},"id":11}]} 2024-07-04T17:34:17.852 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 100 2024-07-04T17:34:17.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.854+0000 7f8b4d800640 1 -- 172.21.15.42:0/3644174475 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f8b2403d4f0 msgr2=0x7f8b2403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:17.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.854+0000 7f8b4d800640 1 --2- 172.21.15.42:0/3644174475 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f8b2403d4f0 0x7f8b2403fa00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8b30034000 tx=0x7f8b30004730 comp rx=0 tx=0).stop 2024-07-04T17:34:17.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.854+0000 7f8b4d800640 1 -- 172.21.15.42:0/3644174475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b480b1d20 msgr2=0x7f8b480ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:17.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.854+0000 7f8b4d800640 1 --2- 172.21.15.42:0/3644174475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b480b1d20 0x7f8b480ad8e0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f8b3800c6b0 tx=0x7f8b3800cb70 comp rx=0 tx=0).stop 2024-07-04T17:34:17.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.854+0000 7f8b4d800640 1 -- 172.21.15.42:0/3644174475 shutdown_connections 2024-07-04T17:34:17.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.854+0000 7f8b4d800640 1 --2- 172.21.15.42:0/3644174475 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f8b2403d4f0 0x7f8b2403fa00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:17.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.854+0000 7f8b4d800640 1 --2- 172.21.15.42:0/3644174475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b480b1d20 0x7f8b480ad8e0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:17.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.854+0000 7f8b4d800640 1 -- 172.21.15.42:0/3644174475 >> 172.21.15.42:0/3644174475 conn(0x7f8b480bebc0 msgr2=0x7f8b4806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:17.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.855+0000 7f8b4d800640 1 -- 172.21.15.42:0/3644174475 shutdown_connections 2024-07-04T17:34:17.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:17.855+0000 7f8b4d800640 1 -- 172.21.15.42:0/3644174475 wait complete. 2024-07-04T17:34:17.886 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:34:18.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.189+0000 7fa242200640 1 -- 172.21.15.42:0/1808442630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa23c068660 msgr2=0x7fa23c0b0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:18.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.189+0000 7fa242200640 1 --2- 172.21.15.42:0/1808442630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa23c068660 0x7fa23c0b0be0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fa224005fd0 tx=0x7fa22402a5f0 comp rx=0 tx=0).stop 2024-07-04T17:34:18.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.189+0000 7fa242200640 1 -- 172.21.15.42:0/1808442630 shutdown_connections 2024-07-04T17:34:18.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.189+0000 7fa242200640 1 --2- 172.21.15.42:0/1808442630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa23c068660 0x7fa23c0b0be0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:18.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.189+0000 7fa242200640 1 -- 172.21.15.42:0/1808442630 >> 172.21.15.42:0/1808442630 conn(0x7fa23c0bfe40 msgr2=0x7fa23c0b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:18.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.189+0000 7fa242200640 1 -- 172.21.15.42:0/1808442630 shutdown_connections 2024-07-04T17:34:18.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.189+0000 7fa242200640 1 -- 172.21.15.42:0/1808442630 wait complete. 2024-07-04T17:34:18.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.191+0000 7fa242200640 1 Processor -- start 2024-07-04T17:34:18.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.191+0000 7fa242200640 1 -- start start 2024-07-04T17:34:18.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.193+0000 7fa242200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa23c068660 0x7fa23c13a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:18.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.193+0000 7fa242200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa23c146360 con 0x7fa23c068660 2024-07-04T17:34:18.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.193+0000 7fa23b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa23c068660 0x7fa23c13a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:18.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.193+0000 7fa23b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa23c068660 0x7fa23c13a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32900/0 (socket says 172.21.15.42:32900) 2024-07-04T17:34:18.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.193+0000 7fa23b400640 1 -- 172.21.15.42:0/870151766 learned_addr learned my addr 172.21.15.42:0/870151766 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:18.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.193+0000 7fa23b400640 1 -- 172.21.15.42:0/870151766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa23c13c070 con 0x7fa23c068660 2024-07-04T17:34:18.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.194+0000 7fa23b400640 1 --2- 172.21.15.42:0/870151766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa23c068660 0x7fa23c13a320 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fa22c001eb0 tx=0x7fa22c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:18.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.194+0000 7fa233400640 1 -- 172.21.15.42:0/870151766 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa22c017070 con 0x7fa23c068660 2024-07-04T17:34:18.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.194+0000 7fa233400640 1 -- 172.21.15.42:0/870151766 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa22c0052f0 con 0x7fa23c068660 2024-07-04T17:34:18.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.194+0000 7fa233400640 1 -- 172.21.15.42:0/870151766 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa22c0035f0 con 0x7fa23c068660 2024-07-04T17:34:18.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.194+0000 7fa242200640 1 -- 172.21.15.42:0/870151766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa23c13bda0 con 0x7fa23c068660 2024-07-04T17:34:18.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.194+0000 7fa242200640 1 -- 172.21.15.42:0/870151766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa23c074450 con 0x7fa23c068660 2024-07-04T17:34:18.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.196+0000 7fa242200640 1 -- 172.21.15.42:0/870151766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa23c13a860 con 0x7fa23c068660 2024-07-04T17:34:18.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.196+0000 7fa233400640 1 -- 172.21.15.42:0/870151766 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7fa22c01a070 con 0x7fa23c068660 2024-07-04T17:34:18.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.196+0000 7fa233400640 1 --2- 172.21.15.42:0/870151766 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fa21803d4f0 0x7fa21803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:18.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.196+0000 7fa233400640 1 -- 172.21.15.42:0/870151766 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 4483+0+0 (secure 0 0 0) 0x7fa22c050fe0 con 0x7fa23c068660 2024-07-04T17:34:18.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.197+0000 7fa233e00640 1 --2- 172.21.15.42:0/870151766 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fa21803d4f0 0x7fa21803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:18.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.198+0000 7fa233e00640 1 --2- 172.21.15.42:0/870151766 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fa21803d4f0 0x7fa21803fa00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa224005fd0 tx=0x7fa224004730 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:18.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.202+0000 7fa233400640 1 -- 172.21.15.42:0/870151766 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fa22c0102e0 con 0x7fa23c068660 2024-07-04T17:34:18.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.414+0000 7fa242200640 1 -- 172.21.15.42:0/870151766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa23c0b0be0 con 0x7fa23c068660 2024-07-04T17:34:18.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.415+0000 7fa233400640 1 -- 172.21.15.42:0/870151766 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) ==== 74+0+12120 (secure 0 0 0) 0x7fa22c04f8e0 con 0x7fa23c068660 2024-07-04T17:34:18.415 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:34:18.416 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":124,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:34:17.268610+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:33:26.476351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:33:27.929016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1699999570846558,"score_stable":1.1699999570846558,"optimal_score":1,"raw_score_acting":1.1699999570846558,"raw_score_stable":1.1699999570846558,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6833/605786437":"2024-07-05T17:34:14.040930+0000","172.21.15.42:6832/605786437":"2024-07-05T17:34:14.040930+0000","172.21.15.42:0/2864803316":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/397279469":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/1279811784":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/3718814387":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/1854623919":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/3279442351":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/1060612519":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/1246900581":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/130985999":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/2238986031":"2024-07-04T18:33:57.174196+0000","172.21.15.42:6835/1310962821":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/3738568139":"2024-07-05T17:33:42.103205+0000","172.21.15.42:6835/4130303917":"2024-07-05T17:33:42.103205+0000","172.21.15.42:6834/4130303917":"2024-07-05T17:33:42.103205+0000","172.21.15.42:0/1148975990":"2024-07-05T17:33:42.103205+0000","172.21.15.42:0/1387747213":"2024-07-05T17:33:42.103205+0000","172.21.15.42:0/2342229102":"2024-07-05T17:33:42.103205+0000","172.21.15.42:0/3100365373":"2024-07-05T17:33:42.103205+0000","172.21.15.42:6834/1310962821":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/3283150650":"2024-07-05T17:33:42.103205+0000","172.21.15.42:0/701226807":"2024-07-05T17:33:42.103205+0000","172.21.15.42:0/4022678544":"2024-07-05T17:33:42.103205+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:34:18.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.418+0000 7fa242200640 1 -- 172.21.15.42:0/870151766 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fa21803d4f0 msgr2=0x7fa21803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:18.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.418+0000 7fa242200640 1 --2- 172.21.15.42:0/870151766 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fa21803d4f0 0x7fa21803fa00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa224005fd0 tx=0x7fa224004730 comp rx=0 tx=0).stop 2024-07-04T17:34:18.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.418+0000 7fa242200640 1 -- 172.21.15.42:0/870151766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa23c068660 msgr2=0x7fa23c13a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:18.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.418+0000 7fa242200640 1 --2- 172.21.15.42:0/870151766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa23c068660 0x7fa23c13a320 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fa22c001eb0 tx=0x7fa22c00c810 comp rx=0 tx=0).stop 2024-07-04T17:34:18.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.418+0000 7fa242200640 1 -- 172.21.15.42:0/870151766 shutdown_connections 2024-07-04T17:34:18.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.418+0000 7fa242200640 1 --2- 172.21.15.42:0/870151766 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fa21803d4f0 0x7fa21803fa00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:18.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.418+0000 7fa242200640 1 --2- 172.21.15.42:0/870151766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa23c068660 0x7fa23c13a320 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:18.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.418+0000 7fa242200640 1 -- 172.21.15.42:0/870151766 >> 172.21.15.42:0/870151766 conn(0x7fa23c0bfe40 msgr2=0x7fa23c145660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:18.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.419+0000 7fa242200640 1 -- 172.21.15.42:0/870151766 shutdown_connections 2024-07-04T17:34:18.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.419+0000 7fa242200640 1 -- 172.21.15.42:0/870151766 wait complete. 2024-07-04T17:34:18.446 INFO:tasks.cephfs.filesystem:Destroying file system backup_fs and related pools 2024-07-04T17:34:18.446 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:34:18.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.739+0000 7fee87000640 1 -- 172.21.15.42:0/520983615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee800bad70 msgr2=0x7fee8006c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:18.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.739+0000 7fee87000640 1 --2- 172.21.15.42:0/520983615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee800bad70 0x7fee8006c2c0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fee6c005fd0 tx=0x7fee6c02a560 comp rx=0 tx=0).stop 2024-07-04T17:34:18.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.739+0000 7fee87000640 1 -- 172.21.15.42:0/520983615 shutdown_connections 2024-07-04T17:34:18.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.739+0000 7fee87000640 1 --2- 172.21.15.42:0/520983615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee800bad70 0x7fee8006c2c0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:18.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.739+0000 7fee87000640 1 -- 172.21.15.42:0/520983615 >> 172.21.15.42:0/520983615 conn(0x7fee800bebc0 msgr2=0x7fee800befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:18.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.739+0000 7fee87000640 1 -- 172.21.15.42:0/520983615 shutdown_connections 2024-07-04T17:34:18.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.739+0000 7fee87000640 1 -- 172.21.15.42:0/520983615 wait complete. 2024-07-04T17:34:18.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.742+0000 7fee87000640 1 Processor -- start 2024-07-04T17:34:18.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.742+0000 7fee87000640 1 -- start start 2024-07-04T17:34:18.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.743+0000 7fee87000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee8006d580 0x7fee8013e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:18.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.743+0000 7fee87000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fee800b2810 con 0x7fee8006d580 2024-07-04T17:34:18.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.743+0000 7fee85c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee8006d580 0x7fee8013e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:18.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.743+0000 7fee85c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee8006d580 0x7fee8013e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32930/0 (socket says 172.21.15.42:32930) 2024-07-04T17:34:18.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.743+0000 7fee85c00640 1 -- 172.21.15.42:0/1630364226 learned_addr learned my addr 172.21.15.42:0/1630364226 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:18.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.744+0000 7fee85c00640 1 -- 172.21.15.42:0/1630364226 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee800739e0 con 0x7fee8006d580 2024-07-04T17:34:18.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.744+0000 7fee85c00640 1 --2- 172.21.15.42:0/1630364226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee8006d580 0x7fee8013e290 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fee7400c380 tx=0x7fee7400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:18.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.744+0000 7fee7d600640 1 -- 172.21.15.42:0/1630364226 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee74017070 con 0x7fee8006d580 2024-07-04T17:34:18.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.744+0000 7fee7d600640 1 -- 172.21.15.42:0/1630364226 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee740052f0 con 0x7fee8006d580 2024-07-04T17:34:18.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.744+0000 7fee7d600640 1 -- 172.21.15.42:0/1630364226 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee740035f0 con 0x7fee8006d580 2024-07-04T17:34:18.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.745+0000 7fee87000640 1 -- 172.21.15.42:0/1630364226 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee8013eda0 con 0x7fee8006d580 2024-07-04T17:34:18.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.745+0000 7fee87000640 1 -- 172.21.15.42:0/1630364226 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee80072070 con 0x7fee8006d580 2024-07-04T17:34:18.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.746+0000 7fee87000640 1 -- 172.21.15.42:0/1630364226 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee8006c2c0 con 0x7fee8006d580 2024-07-04T17:34:18.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.746+0000 7fee7d600640 1 -- 172.21.15.42:0/1630364226 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7fee7401a070 con 0x7fee8006d580 2024-07-04T17:34:18.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.747+0000 7fee7d600640 1 --2- 172.21.15.42:0/1630364226 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fee6003d4f0 0x7fee6003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:18.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.747+0000 7fee7d600640 1 -- 172.21.15.42:0/1630364226 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 4483+0+0 (secure 0 0 0) 0x7fee74051060 con 0x7fee8006d580 2024-07-04T17:34:18.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.747+0000 7fee85200640 1 --2- 172.21.15.42:0/1630364226 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fee6003d4f0 0x7fee6003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:18.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.748+0000 7fee85200640 1 --2- 172.21.15.42:0/1630364226 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fee6003d4f0 0x7fee6003fa00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fee6c033000 tx=0x7fee6c004730 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:18.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:18.752+0000 7fee7d600640 1 -- 172.21.15.42:0/1630364226 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fee74016170 con 0x7fee8006d580 2024-07-04T17:34:19.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.009+0000 7fee87000640 1 -- 172.21.15.42:0/1630364226 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee800b8ee0 con 0x7fee8006d580 2024-07-04T17:34:19.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.010+0000 7fee7d600640 1 -- 172.21.15.42:0/1630364226 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+6036 (secure 0 0 0) 0x7fee740192c0 con 0x7fee8006d580 2024-07-04T17:34:19.011 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:34:19.011 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":100,"btime":"2024-07-04T17:34:15:076257+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5536,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/68857121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":68857121},{"type":"v1","addr":"172.21.15.42:6825","nonce":68857121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":5544,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6828/20990456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":20990456},{"type":"v1","addr":"172.21.15.42:6828","nonce":20990456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":5545,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/428501060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6827","nonce":428501060},{"type":"v1","addr":"172.21.15.42:6829","nonce":428501060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":5736,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6833/94781196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":94781196},{"type":"v1","addr":"172.21.15.42:6833","nonce":94781196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:33:28.944269+0000","modified":"2024-07-04T17:33:32.025418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5549},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5549":{"gid":5549,"name":"b","rank":0,"incarnation":92,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6831/3224527035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":3224527035},{"type":"v1","addr":"172.21.15.42:6831","nonce":3224527035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5549,"qdb_cluster":[5549]},"id":11}]} 2024-07-04T17:34:19.011 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 100 2024-07-04T17:34:19.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.014+0000 7fee87000640 1 -- 172.21.15.42:0/1630364226 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fee6003d4f0 msgr2=0x7fee6003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:19.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.014+0000 7fee87000640 1 --2- 172.21.15.42:0/1630364226 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fee6003d4f0 0x7fee6003fa00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fee6c033000 tx=0x7fee6c004730 comp rx=0 tx=0).stop 2024-07-04T17:34:19.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.014+0000 7fee87000640 1 -- 172.21.15.42:0/1630364226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee8006d580 msgr2=0x7fee8013e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:19.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.014+0000 7fee87000640 1 --2- 172.21.15.42:0/1630364226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee8006d580 0x7fee8013e290 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fee7400c380 tx=0x7fee7400c840 comp rx=0 tx=0).stop 2024-07-04T17:34:19.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.014+0000 7fee87000640 1 -- 172.21.15.42:0/1630364226 shutdown_connections 2024-07-04T17:34:19.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.014+0000 7fee87000640 1 --2- 172.21.15.42:0/1630364226 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fee6003d4f0 0x7fee6003fa00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:19.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.014+0000 7fee87000640 1 --2- 172.21.15.42:0/1630364226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee8006d580 0x7fee8013e290 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:19.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.014+0000 7fee87000640 1 -- 172.21.15.42:0/1630364226 >> 172.21.15.42:0/1630364226 conn(0x7fee800bebc0 msgr2=0x7fee80142c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:19.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.014+0000 7fee87000640 1 -- 172.21.15.42:0/1630364226 shutdown_connections 2024-07-04T17:34:19.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.014+0000 7fee87000640 1 -- 172.21.15.42:0/1630364226 wait complete. 2024-07-04T17:34:19.041 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:34:19.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.330+0000 7f34a3000640 1 -- 172.21.15.42:0/1446230225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349c06cea0 msgr2=0x7f349c0b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:19.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.330+0000 7f34a3000640 1 --2- 172.21.15.42:0/1446230225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349c06cea0 0x7f349c0b3030 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f348c005fd0 tx=0x7f348c02b930 comp rx=0 tx=0).stop 2024-07-04T17:34:19.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.330+0000 7f34a3000640 1 -- 172.21.15.42:0/1446230225 shutdown_connections 2024-07-04T17:34:19.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.330+0000 7f34a3000640 1 --2- 172.21.15.42:0/1446230225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349c06cea0 0x7f349c0b3030 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:19.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.330+0000 7f34a3000640 1 -- 172.21.15.42:0/1446230225 >> 172.21.15.42:0/1446230225 conn(0x7f349c0bf700 msgr2=0x7f349c0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:19.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.331+0000 7f34a3000640 1 -- 172.21.15.42:0/1446230225 shutdown_connections 2024-07-04T17:34:19.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.331+0000 7f34a3000640 1 -- 172.21.15.42:0/1446230225 wait complete. 2024-07-04T17:34:19.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.333+0000 7f34a3000640 1 Processor -- start 2024-07-04T17:34:19.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.333+0000 7f34a3000640 1 -- start start 2024-07-04T17:34:19.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.335+0000 7f34a3000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349c06cea0 0x7f349c0ae530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:19.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.335+0000 7f34a3000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f349c06ef50 con 0x7f349c06cea0 2024-07-04T17:34:19.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.335+0000 7f349be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349c06cea0 0x7f349c0ae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:19.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.335+0000 7f349be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349c06cea0 0x7f349c0ae530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32948/0 (socket says 172.21.15.42:32948) 2024-07-04T17:34:19.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.335+0000 7f349be00640 1 -- 172.21.15.42:0/3855249763 learned_addr learned my addr 172.21.15.42:0/3855249763 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:19.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.336+0000 7f349be00640 1 -- 172.21.15.42:0/3855249763 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f349c0ac640 con 0x7f349c06cea0 2024-07-04T17:34:19.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.336+0000 7f349be00640 1 --2- 172.21.15.42:0/3855249763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349c06cea0 0x7f349c0ae530 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f348c00a960 tx=0x7f348c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:19.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.336+0000 7f3498c00640 1 -- 172.21.15.42:0/3855249763 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f348c03f070 con 0x7f349c06cea0 2024-07-04T17:34:19.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.336+0000 7f3498c00640 1 -- 172.21.15.42:0/3855249763 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f348c004c50 con 0x7f349c06cea0 2024-07-04T17:34:19.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.336+0000 7f3498c00640 1 -- 172.21.15.42:0/3855249763 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f348c004f50 con 0x7f349c06cea0 2024-07-04T17:34:19.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.337+0000 7f34a3000640 1 -- 172.21.15.42:0/3855249763 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f349c0aea70 con 0x7f349c06cea0 2024-07-04T17:34:19.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.337+0000 7f34a3000640 1 -- 172.21.15.42:0/3855249763 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f349c146b90 con 0x7f349c06cea0 2024-07-04T17:34:19.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.338+0000 7f34a3000640 1 -- 172.21.15.42:0/3855249763 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f349c0b3030 con 0x7f349c06cea0 2024-07-04T17:34:19.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.338+0000 7f3498c00640 1 -- 172.21.15.42:0/3855249763 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f348c038070 con 0x7f349c06cea0 2024-07-04T17:34:19.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.339+0000 7f3498c00640 1 --2- 172.21.15.42:0/3855249763 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f347803d4f0 0x7f347803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:19.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.339+0000 7f3498c00640 1 -- 172.21.15.42:0/3855249763 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 4483+0+0 (secure 0 0 0) 0x7f348c036070 con 0x7f349c06cea0 2024-07-04T17:34:19.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.339+0000 7f3493400640 1 --2- 172.21.15.42:0/3855249763 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f347803d4f0 0x7f347803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:19.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.340+0000 7f3493400640 1 --2- 172.21.15.42:0/3855249763 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f347803d4f0 0x7f347803fa00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3484001c60 tx=0x7f3484007d70 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:19.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.344+0000 7f3498c00640 1 -- 172.21.15.42:0/3855249763 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f348c03ace0 con 0x7f349c06cea0 2024-07-04T17:34:19.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.594+0000 7f34a3000640 1 -- 172.21.15.42:0/3855249763 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f349c0ac410 con 0x7f349c06cea0 2024-07-04T17:34:19.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.596+0000 7f3498c00640 1 -- 172.21.15.42:0/3855249763 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+6036 (secure 0 0 0) 0x7f348c04c3a0 con 0x7f349c06cea0 2024-07-04T17:34:19.596 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:34:19.596 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":100,"btime":"2024-07-04T17:34:15:076257+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5536,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/68857121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":68857121},{"type":"v1","addr":"172.21.15.42:6825","nonce":68857121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":5544,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6828/20990456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":20990456},{"type":"v1","addr":"172.21.15.42:6828","nonce":20990456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":5545,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/428501060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6827","nonce":428501060},{"type":"v1","addr":"172.21.15.42:6829","nonce":428501060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":5736,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6833/94781196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":94781196},{"type":"v1","addr":"172.21.15.42:6833","nonce":94781196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:33:28.944269+0000","modified":"2024-07-04T17:33:32.025418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5549},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5549":{"gid":5549,"name":"b","rank":0,"incarnation":92,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6831/3224527035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":3224527035},{"type":"v1","addr":"172.21.15.42:6831","nonce":3224527035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5549,"qdb_cluster":[5549]},"id":11}]} 2024-07-04T17:34:19.596 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 100 2024-07-04T17:34:19.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.598+0000 7f34a3000640 1 -- 172.21.15.42:0/3855249763 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f347803d4f0 msgr2=0x7f347803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:19.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.598+0000 7f34a3000640 1 --2- 172.21.15.42:0/3855249763 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f347803d4f0 0x7f347803fa00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3484001c60 tx=0x7f3484007d70 comp rx=0 tx=0).stop 2024-07-04T17:34:19.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.598+0000 7f34a3000640 1 -- 172.21.15.42:0/3855249763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349c06cea0 msgr2=0x7f349c0ae530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:19.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.598+0000 7f34a3000640 1 --2- 172.21.15.42:0/3855249763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349c06cea0 0x7f349c0ae530 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f348c00a960 tx=0x7f348c0045e0 comp rx=0 tx=0).stop 2024-07-04T17:34:19.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.598+0000 7f34a3000640 1 -- 172.21.15.42:0/3855249763 shutdown_connections 2024-07-04T17:34:19.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.598+0000 7f34a3000640 1 --2- 172.21.15.42:0/3855249763 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f347803d4f0 0x7f347803fa00 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:19.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.598+0000 7f34a3000640 1 --2- 172.21.15.42:0/3855249763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349c06cea0 0x7f349c0ae530 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:19.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.599+0000 7f34a3000640 1 -- 172.21.15.42:0/3855249763 >> 172.21.15.42:0/3855249763 conn(0x7f349c0bf700 msgr2=0x7f349c14ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:19.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.599+0000 7f34a3000640 1 -- 172.21.15.42:0/3855249763 shutdown_connections 2024-07-04T17:34:19.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.599+0000 7f34a3000640 1 -- 172.21.15.42:0/3855249763 wait complete. 2024-07-04T17:34:19.627 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:34:19.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.917+0000 7f2205800640 1 -- 172.21.15.42:0/3346609357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22000b37d0 msgr2=0x7f22000b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:19.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.917+0000 7f2205800640 1 --2- 172.21.15.42:0/3346609357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22000b37d0 0x7f22000b01f0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f21ec005fd0 tx=0x7f21ec02a5f0 comp rx=0 tx=0).stop 2024-07-04T17:34:19.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.917+0000 7f2205800640 1 -- 172.21.15.42:0/3346609357 shutdown_connections 2024-07-04T17:34:19.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.917+0000 7f2205800640 1 --2- 172.21.15.42:0/3346609357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22000b37d0 0x7f22000b01f0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:19.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.917+0000 7f2205800640 1 -- 172.21.15.42:0/3346609357 >> 172.21.15.42:0/3346609357 conn(0x7f22000bf700 msgr2=0x7f22000af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:19.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.917+0000 7f2205800640 1 -- 172.21.15.42:0/3346609357 shutdown_connections 2024-07-04T17:34:19.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.917+0000 7f2205800640 1 -- 172.21.15.42:0/3346609357 wait complete. 2024-07-04T17:34:19.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.920+0000 7f2205800640 1 Processor -- start 2024-07-04T17:34:19.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.920+0000 7f2205800640 1 -- start start 2024-07-04T17:34:19.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.921+0000 7f2205800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22000b2d20 0x7f2200070ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:19.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.921+0000 7f2205800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f220006ce30 con 0x7f22000b2d20 2024-07-04T17:34:19.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.922+0000 7f21fea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22000b2d20 0x7f2200070ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:19.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.922+0000 7f21fea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22000b2d20 0x7f2200070ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32966/0 (socket says 172.21.15.42:32966) 2024-07-04T17:34:19.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.922+0000 7f21fea00640 1 -- 172.21.15.42:0/2555681671 learned_addr learned my addr 172.21.15.42:0/2555681671 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:19.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.922+0000 7f21fea00640 1 -- 172.21.15.42:0/2555681671 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22000aecc0 con 0x7f22000b2d20 2024-07-04T17:34:19.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.922+0000 7f21fea00640 1 --2- 172.21.15.42:0/2555681671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22000b2d20 0x7f2200070ef0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f21f000c2f0 tx=0x7f21f000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:19.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.923+0000 7f21f6a00640 1 -- 172.21.15.42:0/2555681671 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21f0017070 con 0x7f22000b2d20 2024-07-04T17:34:19.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.923+0000 7f21f6a00640 1 -- 172.21.15.42:0/2555681671 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21f00052f0 con 0x7f22000b2d20 2024-07-04T17:34:19.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.923+0000 7f2205800640 1 -- 172.21.15.42:0/2555681671 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22000709a0 con 0x7f22000b2d20 2024-07-04T17:34:19.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.923+0000 7f21f6a00640 1 -- 172.21.15.42:0/2555681671 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21f00035f0 con 0x7f22000b2d20 2024-07-04T17:34:19.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.923+0000 7f2205800640 1 -- 172.21.15.42:0/2555681671 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f220013f1b0 con 0x7f22000b2d20 2024-07-04T17:34:19.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.924+0000 7f2205800640 1 -- 172.21.15.42:0/2555681671 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f220013f430 con 0x7f22000b2d20 2024-07-04T17:34:19.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.924+0000 7f21f6a00640 1 -- 172.21.15.42:0/2555681671 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f21f001a070 con 0x7f22000b2d20 2024-07-04T17:34:19.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.925+0000 7f21f6a00640 1 --2- 172.21.15.42:0/2555681671 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f21e003d4f0 0x7f21e003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:19.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.925+0000 7f21f6a00640 1 -- 172.21.15.42:0/2555681671 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 4483+0+0 (secure 0 0 0) 0x7f21f0050f40 con 0x7f22000b2d20 2024-07-04T17:34:19.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.925+0000 7f21fe000640 1 --2- 172.21.15.42:0/2555681671 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f21e003d4f0 0x7f21e003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:19.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.926+0000 7f21fe000640 1 --2- 172.21.15.42:0/2555681671 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f21e003d4f0 0x7f21e003fa00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f21ec033000 tx=0x7f21ec004730 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:19.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:19.930+0000 7f21f6a00640 1 -- 172.21.15.42:0/2555681671 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f21f00102e0 con 0x7f22000b2d20 2024-07-04T17:34:20.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.150+0000 7f2205800640 1 -- 172.21.15.42:0/2555681671 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f220006d490 con 0x7f22000b2d20 2024-07-04T17:34:20.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.153+0000 7f21f6a00640 1 -- 172.21.15.42:0/2555681671 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) ==== 74+0+12120 (secure 0 0 0) 0x7f21f001bdf0 con 0x7f22000b2d20 2024-07-04T17:34:20.153 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:34:20.154 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":124,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:34:17.268610+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:33:26.476351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:33:27.929016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1699999570846558,"score_stable":1.1699999570846558,"optimal_score":1,"raw_score_acting":1.1699999570846558,"raw_score_stable":1.1699999570846558,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6833/605786437":"2024-07-05T17:34:14.040930+0000","172.21.15.42:6832/605786437":"2024-07-05T17:34:14.040930+0000","172.21.15.42:0/2864803316":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/397279469":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/1279811784":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/3718814387":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/1854623919":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/3279442351":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/1060612519":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/1246900581":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/130985999":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/2238986031":"2024-07-04T18:33:57.174196+0000","172.21.15.42:6835/1310962821":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/3738568139":"2024-07-05T17:33:42.103205+0000","172.21.15.42:6835/4130303917":"2024-07-05T17:33:42.103205+0000","172.21.15.42:6834/4130303917":"2024-07-05T17:33:42.103205+0000","172.21.15.42:0/1148975990":"2024-07-05T17:33:42.103205+0000","172.21.15.42:0/1387747213":"2024-07-05T17:33:42.103205+0000","172.21.15.42:0/2342229102":"2024-07-05T17:33:42.103205+0000","172.21.15.42:0/3100365373":"2024-07-05T17:33:42.103205+0000","172.21.15.42:6834/1310962821":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/3283150650":"2024-07-05T17:33:42.103205+0000","172.21.15.42:0/701226807":"2024-07-05T17:33:42.103205+0000","172.21.15.42:0/4022678544":"2024-07-05T17:33:42.103205+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:34:20.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.156+0000 7f2205800640 1 -- 172.21.15.42:0/2555681671 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f21e003d4f0 msgr2=0x7f21e003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:20.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.156+0000 7f2205800640 1 --2- 172.21.15.42:0/2555681671 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f21e003d4f0 0x7f21e003fa00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f21ec033000 tx=0x7f21ec004730 comp rx=0 tx=0).stop 2024-07-04T17:34:20.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.156+0000 7f2205800640 1 -- 172.21.15.42:0/2555681671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22000b2d20 msgr2=0x7f2200070ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:20.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.156+0000 7f2205800640 1 --2- 172.21.15.42:0/2555681671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22000b2d20 0x7f2200070ef0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f21f000c2f0 tx=0x7f21f000c7b0 comp rx=0 tx=0).stop 2024-07-04T17:34:20.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.156+0000 7f2205800640 1 -- 172.21.15.42:0/2555681671 shutdown_connections 2024-07-04T17:34:20.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.156+0000 7f2205800640 1 --2- 172.21.15.42:0/2555681671 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f21e003d4f0 0x7f21e003fa00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:20.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.156+0000 7f2205800640 1 --2- 172.21.15.42:0/2555681671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f22000b2d20 0x7f2200070ef0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:20.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.156+0000 7f2205800640 1 -- 172.21.15.42:0/2555681671 >> 172.21.15.42:0/2555681671 conn(0x7f22000bf700 msgr2=0x7f22000bc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:20.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.157+0000 7f2205800640 1 -- 172.21.15.42:0/2555681671 shutdown_connections 2024-07-04T17:34:20.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.157+0000 7f2205800640 1 -- 172.21.15.42:0/2555681671 wait complete. 2024-07-04T17:34:20.187 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail backup_fs 2024-07-04T17:34:20.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.485+0000 7fe7f4e00640 1 -- 172.21.15.42:0/1513174399 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7f00ad8e0 msgr2=0x7fe7f00af1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:20.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.485+0000 7fe7f4e00640 1 --2- 172.21.15.42:0/1513174399 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7f00ad8e0 0x7fe7f00af1e0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80097b0 tx=0x7fe7d802a5d0 comp rx=0 tx=0).stop 2024-07-04T17:34:20.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.485+0000 7fe7f4e00640 1 -- 172.21.15.42:0/1513174399 shutdown_connections 2024-07-04T17:34:20.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.485+0000 7fe7f4e00640 1 --2- 172.21.15.42:0/1513174399 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7f00ad8e0 0x7fe7f00af1e0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:20.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.485+0000 7fe7f4e00640 1 -- 172.21.15.42:0/1513174399 >> 172.21.15.42:0/1513174399 conn(0x7fe7f006fec0 msgr2=0x7fe7f006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:20.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.485+0000 7fe7f4e00640 1 -- 172.21.15.42:0/1513174399 shutdown_connections 2024-07-04T17:34:20.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.485+0000 7fe7f4e00640 1 -- 172.21.15.42:0/1513174399 wait complete. 2024-07-04T17:34:20.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.488+0000 7fe7f4e00640 1 Processor -- start 2024-07-04T17:34:20.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.488+0000 7fe7f4e00640 1 -- start start 2024-07-04T17:34:20.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.490+0000 7fe7f4e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7f0068660 0x7fe7f01d9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:20.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.490+0000 7fe7f4e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f01413d0 con 0x7fe7f0068660 2024-07-04T17:34:20.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.490+0000 7fe7ef400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7f0068660 0x7fe7f01d9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:20.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.490+0000 7fe7ef400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7f0068660 0x7fe7f01d9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32984/0 (socket says 172.21.15.42:32984) 2024-07-04T17:34:20.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.490+0000 7fe7ef400640 1 -- 172.21.15.42:0/1632170974 learned_addr learned my addr 172.21.15.42:0/1632170974 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:20.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.491+0000 7fe7ef400640 1 -- 172.21.15.42:0/1632170974 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f01d89c0 con 0x7fe7f0068660 2024-07-04T17:34:20.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.491+0000 7fe7ef400640 1 --2- 172.21.15.42:0/1632170974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7f0068660 0x7fe7f01d9300 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000c380 tx=0x7fe7e000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:20.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.491+0000 7fe7e7400640 1 -- 172.21.15.42:0/1632170974 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7e00116c0 con 0x7fe7f0068660 2024-07-04T17:34:20.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.491+0000 7fe7e7400640 1 -- 172.21.15.42:0/1632170974 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7e0017070 con 0x7fe7f0068660 2024-07-04T17:34:20.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.491+0000 7fe7e7400640 1 -- 172.21.15.42:0/1632170974 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7e0011b20 con 0x7fe7f0068660 2024-07-04T17:34:20.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.491+0000 7fe7f4e00640 1 -- 172.21.15.42:0/1632170974 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f01e3ca0 con 0x7fe7f0068660 2024-07-04T17:34:20.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.492+0000 7fe7f4e00640 1 -- 172.21.15.42:0/1632170974 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f01d9040 con 0x7fe7f0068660 2024-07-04T17:34:20.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.493+0000 7fe7f4e00640 1 -- 172.21.15.42:0/1632170974 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7f01e3e70 con 0x7fe7f0068660 2024-07-04T17:34:20.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.493+0000 7fe7e7400640 1 -- 172.21.15.42:0/1632170974 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7fe7e0011cc0 con 0x7fe7f0068660 2024-07-04T17:34:20.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.494+0000 7fe7e7400640 1 --2- 172.21.15.42:0/1632170974 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fe7cc03d4f0 0x7fe7cc03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:20.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.494+0000 7fe7e7400640 1 -- 172.21.15.42:0/1632170974 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 4483+0+0 (secure 0 0 0) 0x7fe7e0051a50 con 0x7fe7f0068660 2024-07-04T17:34:20.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.494+0000 7fe7e7e00640 1 --2- 172.21.15.42:0/1632170974 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fe7cc03d4f0 0x7fe7cc03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:20.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.495+0000 7fe7e7e00640 1 --2- 172.21.15.42:0/1632170974 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fe7cc03d4f0 0x7fe7cc03fa00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8034000 tx=0x7fe7d8004730 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:20.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.499+0000 7fe7e7400640 1 -- 172.21.15.42:0/1632170974 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fe7f01d9040 con 0x7fe7f0068660 2024-07-04T17:34:20.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.758+0000 7fe7f4e00640 1 -- 172.21.15.42:0/1632170974 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "backup_fs"} v 0) -- 0x7fe7f0141a80 con 0x7fe7f0068660 2024-07-04T17:34:20.759 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-07-04T17:34:20.759+0000 7f8616e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-04T17:34:20.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.761+0000 7fe7e7400640 1 -- 172.21.15.42:0/1632170974 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "backup_fs"}]=0 backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v101) ==== 167+0+0 (secure 0 0 0) 0x7fe7e0010cf0 con 0x7fe7f0068660 2024-07-04T17:34:20.761 INFO:teuthology.orchestra.run.smithi042.stderr:backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-04T17:34:20.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.765+0000 7fe7f4e00640 1 -- 172.21.15.42:0/1632170974 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fe7cc03d4f0 msgr2=0x7fe7cc03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:20.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.765+0000 7fe7f4e00640 1 --2- 172.21.15.42:0/1632170974 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fe7cc03d4f0 0x7fe7cc03fa00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8034000 tx=0x7fe7d8004730 comp rx=0 tx=0).stop 2024-07-04T17:34:20.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.765+0000 7fe7f4e00640 1 -- 172.21.15.42:0/1632170974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7f0068660 msgr2=0x7fe7f01d9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:20.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.765+0000 7fe7f4e00640 1 --2- 172.21.15.42:0/1632170974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7f0068660 0x7fe7f01d9300 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000c380 tx=0x7fe7e000c840 comp rx=0 tx=0).stop 2024-07-04T17:34:20.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.765+0000 7fe7f4e00640 1 -- 172.21.15.42:0/1632170974 shutdown_connections 2024-07-04T17:34:20.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.765+0000 7fe7f4e00640 1 --2- 172.21.15.42:0/1632170974 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fe7cc03d4f0 0x7fe7cc03fa00 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:20.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.765+0000 7fe7f4e00640 1 --2- 172.21.15.42:0/1632170974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7f0068660 0x7fe7f01d9300 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:20.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.765+0000 7fe7f4e00640 1 -- 172.21.15.42:0/1632170974 >> 172.21.15.42:0/1632170974 conn(0x7fe7f006fec0 msgr2=0x7fe7f0073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:20.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.765+0000 7fe7f4e00640 1 -- 172.21.15.42:0/1632170974 shutdown_connections 2024-07-04T17:34:20.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:20.766+0000 7fe7f4e00640 1 -- 172.21.15.42:0/1632170974 wait complete. 2024-07-04T17:34:20.773 INFO:tasks.ceph.mds.b.smithi042.stderr: -3159> 2024-07-04T17:33:31.025+0000 7fd0bf600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-04T17:34:20.793 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm backup_fs --yes-i-really-mean-it 2024-07-04T17:34:20.902 INFO:tasks.ceph.mds.b.smithi042.stdout:starting mds.b at 2024-07-04T17:34:21.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.070+0000 7f4eb2a00640 1 -- 172.21.15.42:0/3417828651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4eac0bd790 msgr2=0x7f4eac0be2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:21.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.070+0000 7f4eb2a00640 1 --2- 172.21.15.42:0/3417828651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4eac0bd790 0x7f4eac0be2a0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c005fd0 tx=0x7f4e9c02a5f0 comp rx=0 tx=0).stop 2024-07-04T17:34:21.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.070+0000 7f4eb2a00640 1 -- 172.21.15.42:0/3417828651 shutdown_connections 2024-07-04T17:34:21.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.070+0000 7f4eb2a00640 1 --2- 172.21.15.42:0/3417828651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4eac0bd790 0x7f4eac0be2a0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:21.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.070+0000 7f4eb2a00640 1 -- 172.21.15.42:0/3417828651 >> 172.21.15.42:0/3417828651 conn(0x7f4eac06e6c0 msgr2=0x7f4eac06ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:21.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.070+0000 7f4eb2a00640 1 -- 172.21.15.42:0/3417828651 shutdown_connections 2024-07-04T17:34:21.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.070+0000 7f4eb2a00640 1 -- 172.21.15.42:0/3417828651 wait complete. 2024-07-04T17:34:21.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.072+0000 7f4eb2a00640 1 Processor -- start 2024-07-04T17:34:21.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.073+0000 7f4eb2a00640 1 -- start start 2024-07-04T17:34:21.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.074+0000 7f4eb2a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4eac0bc500 0x7f4eac0c3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:21.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.074+0000 7f4eb2a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac0b83b0 con 0x7f4eac0bc500 2024-07-04T17:34:21.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.074+0000 7f4eabe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4eac0bc500 0x7f4eac0c3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:21.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.074+0000 7f4eabe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4eac0bc500 0x7f4eac0c3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33004/0 (socket says 172.21.15.42:33004) 2024-07-04T17:34:21.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.074+0000 7f4eabe00640 1 -- 172.21.15.42:0/2150550689 learned_addr learned my addr 172.21.15.42:0/2150550689 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:21.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.075+0000 7f4eabe00640 1 -- 172.21.15.42:0/2150550689 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eac140e30 con 0x7f4eac0bc500 2024-07-04T17:34:21.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.075+0000 7f4eabe00640 1 --2- 172.21.15.42:0/2150550689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4eac0bc500 0x7f4eac0c3400 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c009850 tx=0x7f4e9c0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:21.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.075+0000 7f4ea3e00640 1 -- 172.21.15.42:0/2150550689 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e9c03e030 con 0x7f4eac0bc500 2024-07-04T17:34:21.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.075+0000 7f4ea3e00640 1 -- 172.21.15.42:0/2150550689 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e9c0035f0 con 0x7f4eac0bc500 2024-07-04T17:34:21.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.075+0000 7f4ea3e00640 1 -- 172.21.15.42:0/2150550689 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e9c035040 con 0x7f4eac0bc500 2024-07-04T17:34:21.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.075+0000 7f4eb2a00640 1 -- 172.21.15.42:0/2150550689 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eac1428e0 con 0x7f4eac0bc500 2024-07-04T17:34:21.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.075+0000 7f4eb2a00640 1 -- 172.21.15.42:0/2150550689 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eac1433a0 con 0x7f4eac0bc500 2024-07-04T17:34:21.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.077+0000 7f4eb2a00640 1 -- 172.21.15.42:0/2150550689 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eac142ab0 con 0x7f4eac0bc500 2024-07-04T17:34:21.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.077+0000 7f4ea3e00640 1 -- 172.21.15.42:0/2150550689 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f4e9c041030 con 0x7f4eac0bc500 2024-07-04T17:34:21.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.077+0000 7f4ea3e00640 1 --2- 172.21.15.42:0/2150550689 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f4e8803d4f0 0x7f4e8803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:21.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.077+0000 7f4ea3e00640 1 -- 172.21.15.42:0/2150550689 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 4569+0+0 (secure 0 0 0) 0x7f4e9c03f070 con 0x7f4eac0bc500 2024-07-04T17:34:21.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.078+0000 7f4eab400640 1 --2- 172.21.15.42:0/2150550689 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f4e8803d4f0 0x7f4e8803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:21.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.078+0000 7f4eab400640 1 --2- 172.21.15.42:0/2150550689 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f4e8803d4f0 0x7f4e8803fa00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4e94001c60 tx=0x7f4e94007d70 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:21.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.082+0000 7f4ea3e00640 1 -- 172.21.15.42:0/2150550689 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f4e9c04bd60 con 0x7f4eac0bc500 2024-07-04T17:34:21.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.334+0000 7f4eb2a00640 1 -- 172.21.15.42:0/2150550689 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true} v 0) -- 0x7f4eac0b8a60 con 0x7f4eac0bc500 2024-07-04T17:34:21.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.769+0000 7f4ea3e00640 1 -- 172.21.15.42:0/2150550689 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true}]=0 v102) ==== 107+0+0 (secure 0 0 0) 0x7f4e9c030cc0 con 0x7f4eac0bc500 2024-07-04T17:34:21.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.773+0000 7f4eb2a00640 1 -- 172.21.15.42:0/2150550689 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f4e8803d4f0 msgr2=0x7f4e8803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:21.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.773+0000 7f4eb2a00640 1 --2- 172.21.15.42:0/2150550689 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f4e8803d4f0 0x7f4e8803fa00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4e94001c60 tx=0x7f4e94007d70 comp rx=0 tx=0).stop 2024-07-04T17:34:21.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.773+0000 7f4eb2a00640 1 -- 172.21.15.42:0/2150550689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4eac0bc500 msgr2=0x7f4eac0c3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:21.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.773+0000 7f4eb2a00640 1 --2- 172.21.15.42:0/2150550689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4eac0bc500 0x7f4eac0c3400 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c009850 tx=0x7f4e9c0049d0 comp rx=0 tx=0).stop 2024-07-04T17:34:21.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.773+0000 7f4eb2a00640 1 -- 172.21.15.42:0/2150550689 shutdown_connections 2024-07-04T17:34:21.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.773+0000 7f4eb2a00640 1 --2- 172.21.15.42:0/2150550689 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f4e8803d4f0 0x7f4e8803fa00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:21.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.773+0000 7f4eb2a00640 1 --2- 172.21.15.42:0/2150550689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4eac0bc500 0x7f4eac0c3400 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:21.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.773+0000 7f4eb2a00640 1 -- 172.21.15.42:0/2150550689 >> 172.21.15.42:0/2150550689 conn(0x7f4eac06e6c0 msgr2=0x7f4eac075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:21.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.773+0000 7f4eb2a00640 1 -- 172.21.15.42:0/2150550689 shutdown_connections 2024-07-04T17:34:21.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:21.773+0000 7f4eb2a00640 1 -- 172.21.15.42:0/2150550689 wait complete. 2024-07-04T17:34:21.808 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_metadata backup_fs_metadata --yes-i-really-really-mean-it 2024-07-04T17:34:22.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:22.102+0000 7f00bc200640 1 -- 172.21.15.42:0/652272476 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00b4073a20 msgr2=0x7f00b413e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:22.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:22.102+0000 7f00bc200640 1 --2- 172.21.15.42:0/652272476 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00b4073a20 0x7f00b413e1a0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f00a4005fd0 tx=0x7f00a402a5f0 comp rx=0 tx=0).stop 2024-07-04T17:34:22.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:22.102+0000 7f00bc200640 1 -- 172.21.15.42:0/652272476 shutdown_connections 2024-07-04T17:34:22.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:22.102+0000 7f00bc200640 1 --2- 172.21.15.42:0/652272476 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00b4073a20 0x7f00b413e1a0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:22.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:22.102+0000 7f00bc200640 1 -- 172.21.15.42:0/652272476 >> 172.21.15.42:0/652272476 conn(0x7f00b406c020 msgr2=0x7f00b406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:22.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:22.102+0000 7f00bc200640 1 -- 172.21.15.42:0/652272476 shutdown_connections 2024-07-04T17:34:22.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:22.102+0000 7f00bc200640 1 -- 172.21.15.42:0/652272476 wait complete. 2024-07-04T17:34:22.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:22.104+0000 7f00bc200640 1 Processor -- start 2024-07-04T17:34:22.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:22.104+0000 7f00bc200640 1 -- start start 2024-07-04T17:34:22.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:22.105+0000 7f00bc200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00b406d070 0x7f00b40b84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:22.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:22.105+0000 7f00bc200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f00b4071c40 con 0x7f00b406d070 2024-07-04T17:34:22.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:22.106+0000 7f00b9800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00b406d070 0x7f00b40b84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:22.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:22.106+0000 7f00b9800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00b406d070 0x7f00b40b84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33014/0 (socket says 172.21.15.42:33014) 2024-07-04T17:34:22.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:22.106+0000 7f00b9800640 1 -- 172.21.15.42:0/3759906881 learned_addr learned my addr 172.21.15.42:0/3759906881 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:22.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:22.106+0000 7f00b9800640 1 -- 172.21.15.42:0/3759906881 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b40b8a30 con 0x7f00b406d070 2024-07-04T17:34:22.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:22.106+0000 7f00b9800640 1 --2- 172.21.15.42:0/3759906881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00b406d070 0x7f00b40b84d0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f00a800c380 tx=0x7f00a800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:22.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:22.107+0000 7f00b1600640 1 -- 172.21.15.42:0/3759906881 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00a8017070 con 0x7f00b406d070 2024-07-04T17:34:22.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:22.107+0000 7f00b1600640 1 -- 172.21.15.42:0/3759906881 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00a80052f0 con 0x7f00b406d070 2024-07-04T17:34:22.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:22.107+0000 7f00b1600640 1 -- 172.21.15.42:0/3759906881 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00a80035f0 con 0x7f00b406d070 2024-07-04T17:34:22.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:22.107+0000 7f00bc200640 1 -- 172.21.15.42:0/3759906881 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b40b60d0 con 0x7f00b406d070 2024-07-04T17:34:22.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:22.107+0000 7f00bc200640 1 -- 172.21.15.42:0/3759906881 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b40b6aa0 con 0x7f00b406d070 2024-07-04T17:34:22.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:22.108+0000 7f00bc200640 1 -- 172.21.15.42:0/3759906881 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00b413e1a0 con 0x7f00b406d070 2024-07-04T17:34:22.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:22.108+0000 7f00b1600640 1 -- 172.21.15.42:0/3759906881 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f00a801a070 con 0x7f00b406d070 2024-07-04T17:34:22.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:22.109+0000 7f00b1600640 1 --2- 172.21.15.42:0/3759906881 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f009803d4f0 0x7f009803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:22.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:22.109+0000 7f00b1600640 1 -- 172.21.15.42:0/3759906881 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 4569+0+0 (secure 0 0 0) 0x7f00a80510b0 con 0x7f00b406d070 2024-07-04T17:34:22.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:22.109+0000 7f00b8e00640 1 --2- 172.21.15.42:0/3759906881 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f009803d4f0 0x7f009803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:22.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:22.110+0000 7f00b8e00640 1 --2- 172.21.15.42:0/3759906881 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f009803d4f0 0x7f009803fa00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f00a4033000 tx=0x7f00a4004730 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:22.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:22.114+0000 7f00b1600640 1 -- 172.21.15.42:0/3759906881 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f00b406d480 con 0x7f00b406d070 2024-07-04T17:34:22.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:22.327+0000 7f00bc200640 1 -- 172.21.15.42:0/3759906881 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f00b4069b70 con 0x7f00b406d070 2024-07-04T17:34:23.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.298+0000 7f00b1600640 1 -- 172.21.15.42:0/3759906881 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_metadata' removed v126) ==== 190+0+0 (secure 0 0 0) 0x7f00a8016170 con 0x7f00b406d070 2024-07-04T17:34:23.298 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'backup_fs_metadata' removed 2024-07-04T17:34:23.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.302+0000 7f00bc200640 1 -- 172.21.15.42:0/3759906881 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f009803d4f0 msgr2=0x7f009803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:23.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.302+0000 7f00bc200640 1 --2- 172.21.15.42:0/3759906881 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f009803d4f0 0x7f009803fa00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f00a4033000 tx=0x7f00a4004730 comp rx=0 tx=0).stop 2024-07-04T17:34:23.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.302+0000 7f00bc200640 1 -- 172.21.15.42:0/3759906881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00b406d070 msgr2=0x7f00b40b84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:23.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.302+0000 7f00bc200640 1 --2- 172.21.15.42:0/3759906881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00b406d070 0x7f00b40b84d0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f00a800c380 tx=0x7f00a800c840 comp rx=0 tx=0).stop 2024-07-04T17:34:23.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.303+0000 7f00bc200640 1 -- 172.21.15.42:0/3759906881 shutdown_connections 2024-07-04T17:34:23.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.303+0000 7f00bc200640 1 --2- 172.21.15.42:0/3759906881 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f009803d4f0 0x7f009803fa00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:23.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.303+0000 7f00bc200640 1 --2- 172.21.15.42:0/3759906881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00b406d070 0x7f00b40b84d0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:23.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.303+0000 7f00bc200640 1 -- 172.21.15.42:0/3759906881 >> 172.21.15.42:0/3759906881 conn(0x7f00b406c020 msgr2=0x7f00b4141470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:23.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.303+0000 7f00bc200640 1 -- 172.21.15.42:0/3759906881 shutdown_connections 2024-07-04T17:34:23.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.303+0000 7f00bc200640 1 -- 172.21.15.42:0/3759906881 wait complete. 2024-07-04T17:34:23.341 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_data backup_fs_data --yes-i-really-really-mean-it 2024-07-04T17:34:23.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.631+0000 7f8162c00640 1 -- 172.21.15.42:0/811330647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f815c0b0db0 msgr2=0x7f815c0adf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:23.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.631+0000 7f8162c00640 1 --2- 172.21.15.42:0/811330647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f815c0b0db0 0x7f815c0adf70 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f8148005fd0 tx=0x7f814802a5f0 comp rx=0 tx=0).stop 2024-07-04T17:34:23.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.632+0000 7f8162c00640 1 -- 172.21.15.42:0/811330647 shutdown_connections 2024-07-04T17:34:23.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.632+0000 7f8162c00640 1 --2- 172.21.15.42:0/811330647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f815c0b0db0 0x7f815c0adf70 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:23.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.632+0000 7f8162c00640 1 -- 172.21.15.42:0/811330647 >> 172.21.15.42:0/811330647 conn(0x7f815c0afaa0 msgr2=0x7f815c0afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:23.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.632+0000 7f8162c00640 1 -- 172.21.15.42:0/811330647 shutdown_connections 2024-07-04T17:34:23.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.632+0000 7f8162c00640 1 -- 172.21.15.42:0/811330647 wait complete. 2024-07-04T17:34:23.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.634+0000 7f8162c00640 1 Processor -- start 2024-07-04T17:34:23.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.635+0000 7f8162c00640 1 -- start start 2024-07-04T17:34:23.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.636+0000 7f8162c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f815c0acac0 0x7f815c141d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:23.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.636+0000 7f8162c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f815c0ae630 con 0x7f815c0acac0 2024-07-04T17:34:23.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.637+0000 7f8161800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f815c0acac0 0x7f815c141d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:23.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.637+0000 7f8161800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f815c0acac0 0x7f815c141d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33026/0 (socket says 172.21.15.42:33026) 2024-07-04T17:34:23.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.637+0000 7f8161800640 1 -- 172.21.15.42:0/1334620773 learned_addr learned my addr 172.21.15.42:0/1334620773 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:23.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.637+0000 7f8161800640 1 -- 172.21.15.42:0/1334620773 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f815c070a30 con 0x7f815c0acac0 2024-07-04T17:34:23.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.638+0000 7f8161800640 1 --2- 172.21.15.42:0/1334620773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f815c0acac0 0x7f815c141d30 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f815000c340 tx=0x7f815000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:23.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.638+0000 7f8159600640 1 -- 172.21.15.42:0/1334620773 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8150005410 con 0x7f815c0acac0 2024-07-04T17:34:23.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.638+0000 7f8159600640 1 -- 172.21.15.42:0/1334620773 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8150007030 con 0x7f815c0acac0 2024-07-04T17:34:23.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.638+0000 7f8159600640 1 -- 172.21.15.42:0/1334620773 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81500073f0 con 0x7f815c0acac0 2024-07-04T17:34:23.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.638+0000 7f8162c00640 1 -- 172.21.15.42:0/1334620773 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f815c1d4c80 con 0x7f815c0acac0 2024-07-04T17:34:23.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.638+0000 7f8162c00640 1 -- 172.21.15.42:0/1334620773 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f815c1d5150 con 0x7f815c0acac0 2024-07-04T17:34:23.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.640+0000 7f8162c00640 1 -- 172.21.15.42:0/1334620773 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f815c075a40 con 0x7f815c0acac0 2024-07-04T17:34:23.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.640+0000 7f8159600640 1 -- 172.21.15.42:0/1334620773 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f815001a020 con 0x7f815c0acac0 2024-07-04T17:34:23.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.640+0000 7f8159600640 1 --2- 172.21.15.42:0/1334620773 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f813c03d520 0x7f813c03fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:23.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.640+0000 7f8159600640 1 -- 172.21.15.42:0/1334620773 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 4125+0+0 (secure 0 0 0) 0x7f81500512d0 con 0x7f815c0acac0 2024-07-04T17:34:23.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.641+0000 7f8160e00640 1 --2- 172.21.15.42:0/1334620773 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f813c03d520 0x7f813c03fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:23.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.642+0000 7f8160e00640 1 --2- 172.21.15.42:0/1334620773 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f813c03d520 0x7f813c03fa30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8148033000 tx=0x7f8148004730 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:23.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.646+0000 7f8159600640 1 -- 172.21.15.42:0/1334620773 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f815c075a40 con 0x7f815c0acac0 2024-07-04T17:34:23.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:23.856+0000 7f8162c00640 1 -- 172.21.15.42:0/1334620773 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f815c071420 con 0x7f815c0acac0 2024-07-04T17:34:24.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.300+0000 7f8159600640 1 -- 172.21.15.42:0/1334620773 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_data' removed v127) ==== 178+0+0 (secure 0 0 0) 0x7f815c1d5150 con 0x7f815c0acac0 2024-07-04T17:34:24.300 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'backup_fs_data' removed 2024-07-04T17:34:24.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.304+0000 7f8162c00640 1 -- 172.21.15.42:0/1334620773 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f813c03d520 msgr2=0x7f813c03fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:24.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.304+0000 7f8162c00640 1 --2- 172.21.15.42:0/1334620773 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f813c03d520 0x7f813c03fa30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8148033000 tx=0x7f8148004730 comp rx=0 tx=0).stop 2024-07-04T17:34:24.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.304+0000 7f8162c00640 1 -- 172.21.15.42:0/1334620773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f815c0acac0 msgr2=0x7f815c141d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:24.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.304+0000 7f8162c00640 1 --2- 172.21.15.42:0/1334620773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f815c0acac0 0x7f815c141d30 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f815000c340 tx=0x7f815000c800 comp rx=0 tx=0).stop 2024-07-04T17:34:24.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.308+0000 7f8162c00640 1 -- 172.21.15.42:0/1334620773 shutdown_connections 2024-07-04T17:34:24.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.308+0000 7f8162c00640 1 --2- 172.21.15.42:0/1334620773 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f813c03d520 0x7f813c03fa30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:24.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.308+0000 7f8162c00640 1 --2- 172.21.15.42:0/1334620773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f815c0acac0 0x7f815c141d30 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:24.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.308+0000 7f8162c00640 1 -- 172.21.15.42:0/1334620773 >> 172.21.15.42:0/1334620773 conn(0x7f815c0afaa0 msgr2=0x7f815c0b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:24.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.309+0000 7f8162c00640 1 -- 172.21.15.42:0/1334620773 shutdown_connections 2024-07-04T17:34:24.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.309+0000 7f8162c00640 1 -- 172.21.15.42:0/1334620773 wait complete. 2024-07-04T17:34:24.351 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 22 2024-07-04T17:34:24.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.622+0000 7fc03be00640 1 -- 172.21.15.42:0/2548473397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c06c020 msgr2=0x7fc03c14a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:24.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.622+0000 7fc03be00640 1 --2- 172.21.15.42:0/2548473397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c06c020 0x7fc03c14a9e0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fc0240097b0 tx=0x7fc02402a4d0 comp rx=0 tx=0).stop 2024-07-04T17:34:24.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.623+0000 7fc03be00640 1 -- 172.21.15.42:0/2548473397 shutdown_connections 2024-07-04T17:34:24.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.623+0000 7fc03be00640 1 --2- 172.21.15.42:0/2548473397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c06c020 0x7fc03c14a9e0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:24.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.623+0000 7fc03be00640 1 -- 172.21.15.42:0/2548473397 >> 172.21.15.42:0/2548473397 conn(0x7fc03c0bebc0 msgr2=0x7fc03c0befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:24.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.623+0000 7fc03be00640 1 -- 172.21.15.42:0/2548473397 shutdown_connections 2024-07-04T17:34:24.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.623+0000 7fc03be00640 1 -- 172.21.15.42:0/2548473397 wait complete. 2024-07-04T17:34:24.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.625+0000 7fc03be00640 1 Processor -- start 2024-07-04T17:34:24.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.625+0000 7fc03be00640 1 -- start start 2024-07-04T17:34:24.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.626+0000 7fc03be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c06c020 0x7fc03c14a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:24.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.626+0000 7fc03be00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc03c14b580 con 0x7fc03c06c020 2024-07-04T17:34:24.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.627+0000 7fc03aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c06c020 0x7fc03c14a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:24.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.627+0000 7fc03aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c06c020 0x7fc03c14a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51970/0 (socket says 172.21.15.42:51970) 2024-07-04T17:34:24.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.627+0000 7fc03aa00640 1 -- 172.21.15.42:0/1480910326 learned_addr learned my addr 172.21.15.42:0/1480910326 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:24.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.627+0000 7fc03aa00640 1 -- 172.21.15.42:0/1480910326 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc03c0739a0 con 0x7fc03c06c020 2024-07-04T17:34:24.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.627+0000 7fc03aa00640 1 --2- 172.21.15.42:0/1480910326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c06c020 0x7fc03c14a390 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00c450 tx=0x7fc02c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:24.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.628+0000 7fc033400640 1 -- 172.21.15.42:0/1480910326 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc02c0116c0 con 0x7fc03c06c020 2024-07-04T17:34:24.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.628+0000 7fc033400640 1 -- 172.21.15.42:0/1480910326 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc02c017070 con 0x7fc03c06c020 2024-07-04T17:34:24.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.628+0000 7fc033400640 1 -- 172.21.15.42:0/1480910326 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc02c011b20 con 0x7fc03c06c020 2024-07-04T17:34:24.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.628+0000 7fc03be00640 1 -- 172.21.15.42:0/1480910326 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc03c0721a0 con 0x7fc03c06c020 2024-07-04T17:34:24.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.628+0000 7fc03be00640 1 -- 172.21.15.42:0/1480910326 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc03c071ee0 con 0x7fc03c06c020 2024-07-04T17:34:24.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.629+0000 7fc03be00640 1 -- 172.21.15.42:0/1480910326 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc03c0758a0 con 0x7fc03c06c020 2024-07-04T17:34:24.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.629+0000 7fc033400640 1 -- 172.21.15.42:0/1480910326 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7fc02c011cc0 con 0x7fc03c06c020 2024-07-04T17:34:24.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.630+0000 7fc033400640 1 --2- 172.21.15.42:0/1480910326 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fc01803d4f0 0x7fc01803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:24.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.630+0000 7fc033400640 1 -- 172.21.15.42:0/1480910326 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 3705+0+0 (secure 0 0 0) 0x7fc02c0518d0 con 0x7fc03c06c020 2024-07-04T17:34:24.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.630+0000 7fc033e00640 1 --2- 172.21.15.42:0/1480910326 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fc01803d4f0 0x7fc01803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:24.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.631+0000 7fc033e00640 1 --2- 172.21.15.42:0/1480910326 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fc01803d4f0 0x7fc01803fa00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc024034000 tx=0x7fc024004730 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:24.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.635+0000 7fc033400640 1 -- 172.21.15.42:0/1480910326 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fc03c0758a0 con 0x7fc03c06c020 2024-07-04T17:34:24.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.866+0000 7fc03be00640 1 -- 172.21.15.42:0/1480910326 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config reset", "num": 22} v 0) -- 0x7fc03c14bcb0 con 0x7fc03c06c020 2024-07-04T17:34:24.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.871+0000 7fc033400640 1 -- 172.21.15.42:0/1480910326 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 22}]=0 v32) ==== 71+0+0 (secure 0 0 0) 0x7fc02c015d50 con 0x7fc03c06c020 2024-07-04T17:34:24.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.874+0000 7fc03be00640 1 -- 172.21.15.42:0/1480910326 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fc01803d4f0 msgr2=0x7fc01803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:24.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.874+0000 7fc03be00640 1 --2- 172.21.15.42:0/1480910326 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fc01803d4f0 0x7fc01803fa00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc024034000 tx=0x7fc024004730 comp rx=0 tx=0).stop 2024-07-04T17:34:24.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.874+0000 7fc03be00640 1 -- 172.21.15.42:0/1480910326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c06c020 msgr2=0x7fc03c14a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:24.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.874+0000 7fc03be00640 1 --2- 172.21.15.42:0/1480910326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c06c020 0x7fc03c14a390 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00c450 tx=0x7fc02c00c910 comp rx=0 tx=0).stop 2024-07-04T17:34:24.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.874+0000 7fc03be00640 1 -- 172.21.15.42:0/1480910326 shutdown_connections 2024-07-04T17:34:24.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.874+0000 7fc03be00640 1 --2- 172.21.15.42:0/1480910326 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fc01803d4f0 0x7fc01803fa00 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:24.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.874+0000 7fc03be00640 1 --2- 172.21.15.42:0/1480910326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c06c020 0x7fc03c14a390 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:24.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.874+0000 7fc03be00640 1 -- 172.21.15.42:0/1480910326 >> 172.21.15.42:0/1480910326 conn(0x7fc03c0bebc0 msgr2=0x7fc03c0b9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:24.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.874+0000 7fc03be00640 1 -- 172.21.15.42:0/1480910326 shutdown_connections 2024-07-04T17:34:24.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:24.874+0000 7fc03be00640 1 -- 172.21.15.42:0/1480910326 wait complete. 2024-07-04T17:34:24.900 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_mirroring.TestMirroring.test_basic_mirror_commands' 2024-07-04T17:34:25.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:25.193+0000 7f01a6200640 1 -- 172.21.15.42:0/4126271765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01a00bca50 msgr2=0x7f01a0073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:25.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:25.193+0000 7f01a6200640 1 --2- 172.21.15.42:0/4126271765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01a00bca50 0x7f01a0073e80 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f01900097b0 tx=0x7f019002a540 comp rx=0 tx=0).stop 2024-07-04T17:34:25.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:25.194+0000 7f01a6200640 1 -- 172.21.15.42:0/4126271765 shutdown_connections 2024-07-04T17:34:25.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:25.194+0000 7f01a6200640 1 --2- 172.21.15.42:0/4126271765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01a00bca50 0x7f01a0073e80 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:25.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:25.194+0000 7f01a6200640 1 -- 172.21.15.42:0/4126271765 >> 172.21.15.42:0/4126271765 conn(0x7f01a0070830 msgr2=0x7f01a00bfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:25.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:25.194+0000 7f01a6200640 1 -- 172.21.15.42:0/4126271765 shutdown_connections 2024-07-04T17:34:25.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:25.194+0000 7f01a6200640 1 -- 172.21.15.42:0/4126271765 wait complete. 2024-07-04T17:34:25.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:25.196+0000 7f01a6200640 1 Processor -- start 2024-07-04T17:34:25.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:25.196+0000 7f01a6200640 1 -- start start 2024-07-04T17:34:25.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:25.197+0000 7f01a6200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01a00bca50 0x7f01a014afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:25.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:25.197+0000 7f01a6200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f01a01485b0 con 0x7f01a00bca50 2024-07-04T17:34:25.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:25.198+0000 7f01a4e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01a00bca50 0x7f01a014afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:25.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:25.198+0000 7f01a4e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01a00bca50 0x7f01a014afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51984/0 (socket says 172.21.15.42:51984) 2024-07-04T17:34:25.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:25.198+0000 7f01a4e00640 1 -- 172.21.15.42:0/312319049 learned_addr learned my addr 172.21.15.42:0/312319049 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:25.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:25.198+0000 7f01a4e00640 1 -- 172.21.15.42:0/312319049 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a01e3670 con 0x7f01a00bca50 2024-07-04T17:34:25.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:25.199+0000 7f01a4e00640 1 --2- 172.21.15.42:0/312319049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01a00bca50 0x7f01a014afc0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f01900098d0 tx=0x7f0190033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:25.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:25.199+0000 7f019cc00640 1 -- 172.21.15.42:0/312319049 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0190031bc0 con 0x7f01a00bca50 2024-07-04T17:34:25.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:25.199+0000 7f019cc00640 1 -- 172.21.15.42:0/312319049 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0190031e50 con 0x7f01a00bca50 2024-07-04T17:34:25.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:25.199+0000 7f01a6200640 1 -- 172.21.15.42:0/312319049 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01a014b890 con 0x7f01a00bca50 2024-07-04T17:34:25.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:25.199+0000 7f019cc00640 1 -- 172.21.15.42:0/312319049 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0190033260 con 0x7f01a00bca50 2024-07-04T17:34:25.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:25.199+0000 7f01a6200640 1 -- 172.21.15.42:0/312319049 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01a014b500 con 0x7f01a00bca50 2024-07-04T17:34:25.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:25.200+0000 7f01a6200640 1 -- 172.21.15.42:0/312319049 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01a0148c70 con 0x7f01a00bca50 2024-07-04T17:34:25.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:25.200+0000 7f019cc00640 1 -- 172.21.15.42:0/312319049 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f0190033400 con 0x7f01a00bca50 2024-07-04T17:34:25.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:25.201+0000 7f019cc00640 1 --2- 172.21.15.42:0/312319049 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f018003d4f0 0x7f018003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:25.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:25.201+0000 7f019cc00640 1 -- 172.21.15.42:0/312319049 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 3705+0+0 (secure 0 0 0) 0x7f0190072f80 con 0x7f01a00bca50 2024-07-04T17:34:25.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:25.201+0000 7f019fe00640 1 --2- 172.21.15.42:0/312319049 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f018003d4f0 0x7f018003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:25.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:25.202+0000 7f019fe00640 1 --2- 172.21.15.42:0/312319049 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f018003d4f0 0x7f018003fa00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f01a00b2bc0 tx=0x7f018c001650 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:25.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:25.206+0000 7f019cc00640 1 -- 172.21.15.42:0/312319049 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f019003f030 con 0x7f01a00bca50 2024-07-04T17:34:25.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:25.509+0000 7f01a6200640 1 -- 172.21.15.42:0/312319049 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_mirroring.TestMirroring.test_basic_mirror_commands"]} v 0) -- 0x7f01a0146050 con 0x7f01a00bca50 2024-07-04T17:34:26.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.312+0000 7f019cc00640 1 -- 172.21.15.42:0/312319049 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_mirroring.TestMirroring.test_basic_mirror_commands"]}]=0 v428) ==== 147+0+0 (secure 0 0 0) 0x7f019003a3a0 con 0x7f01a00bca50 2024-07-04T17:34:26.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.316+0000 7f01a6200640 1 -- 172.21.15.42:0/312319049 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f018003d4f0 msgr2=0x7f018003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:26.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.316+0000 7f01a6200640 1 --2- 172.21.15.42:0/312319049 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f018003d4f0 0x7f018003fa00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f01a00b2bc0 tx=0x7f018c001650 comp rx=0 tx=0).stop 2024-07-04T17:34:26.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.316+0000 7f01a6200640 1 -- 172.21.15.42:0/312319049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01a00bca50 msgr2=0x7f01a014afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:26.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.316+0000 7f01a6200640 1 --2- 172.21.15.42:0/312319049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01a00bca50 0x7f01a014afc0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f01900098d0 tx=0x7f0190033e40 comp rx=0 tx=0).stop 2024-07-04T17:34:26.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.316+0000 7f01a6200640 1 -- 172.21.15.42:0/312319049 shutdown_connections 2024-07-04T17:34:26.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.317+0000 7f01a6200640 1 --2- 172.21.15.42:0/312319049 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f018003d4f0 0x7f018003fa00 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:26.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.317+0000 7f01a6200640 1 --2- 172.21.15.42:0/312319049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01a00bca50 0x7f01a014afc0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:26.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.317+0000 7f01a6200640 1 -- 172.21.15.42:0/312319049 >> 172.21.15.42:0/312319049 conn(0x7f01a0070830 msgr2=0x7f01a00b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:26.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.317+0000 7f01a6200640 1 -- 172.21.15.42:0/312319049 shutdown_connections 2024-07-04T17:34:26.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.317+0000 7f01a6200640 1 -- 172.21.15.42:0/312319049 wait complete. 2024-07-04T17:34:26.346 INFO:tasks.cephfs_test_runner:test_basic_mirror_commands (tasks.cephfs.test_mirroring.TestMirroring) ... ok 2024-07-04T17:34:26.346 INFO:tasks.cephfs_test_runner:Starting test: test_cephfs_mirror_blocklist (tasks.cephfs.test_mirroring.TestMirroring) 2024-07-04T17:34:26.347 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:34:26.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.639+0000 7f975be00640 1 -- 172.21.15.42:0/678572638 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97540bc040 msgr2=0x7f97540bd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:26.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.639+0000 7f975be00640 1 --2- 172.21.15.42:0/678572638 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97540bc040 0x7f97540bd2d0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f9744005fd0 tx=0x7f974402a8b0 comp rx=0 tx=0).stop 2024-07-04T17:34:26.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.639+0000 7f975be00640 1 -- 172.21.15.42:0/678572638 shutdown_connections 2024-07-04T17:34:26.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.639+0000 7f975be00640 1 --2- 172.21.15.42:0/678572638 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97540bc040 0x7f97540bd2d0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:26.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.639+0000 7f975be00640 1 -- 172.21.15.42:0/678572638 >> 172.21.15.42:0/678572638 conn(0x7f975406c940 msgr2=0x7f975406ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:26.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.640+0000 7f975be00640 1 -- 172.21.15.42:0/678572638 shutdown_connections 2024-07-04T17:34:26.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.640+0000 7f975be00640 1 -- 172.21.15.42:0/678572638 wait complete. 2024-07-04T17:34:26.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.642+0000 7f975be00640 1 Processor -- start 2024-07-04T17:34:26.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.642+0000 7f975be00640 1 -- start start 2024-07-04T17:34:26.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.643+0000 7f975be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97540bc040 0x7f97540bfae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:26.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.643+0000 7f975be00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f97540b7530 con 0x7f97540bc040 2024-07-04T17:34:26.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.643+0000 7f9759400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97540bc040 0x7f97540bfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:26.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.643+0000 7f9759400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97540bc040 0x7f97540bfae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51998/0 (socket says 172.21.15.42:51998) 2024-07-04T17:34:26.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.643+0000 7f9759400640 1 -- 172.21.15.42:0/2731613456 learned_addr learned my addr 172.21.15.42:0/2731613456 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:26.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.644+0000 7f9759400640 1 -- 172.21.15.42:0/2731613456 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97540c25e0 con 0x7f97540bc040 2024-07-04T17:34:26.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.644+0000 7f9759400640 1 --2- 172.21.15.42:0/2731613456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97540bc040 0x7f97540bfae0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f9744009c50 tx=0x7f9744031670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:26.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.644+0000 7f9751600640 1 -- 172.21.15.42:0/2731613456 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97440318c0 con 0x7f97540bc040 2024-07-04T17:34:26.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.644+0000 7f9751600640 1 -- 172.21.15.42:0/2731613456 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9744031b50 con 0x7f97540bc040 2024-07-04T17:34:26.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.644+0000 7f975be00640 1 -- 172.21.15.42:0/2731613456 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f975413fba0 con 0x7f97540bc040 2024-07-04T17:34:26.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.645+0000 7f9751600640 1 -- 172.21.15.42:0/2731613456 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9744033070 con 0x7f97540bc040 2024-07-04T17:34:26.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.645+0000 7f975be00640 1 -- 172.21.15.42:0/2731613456 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97540c1470 con 0x7f97540bc040 2024-07-04T17:34:26.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.646+0000 7f9751600640 1 -- 172.21.15.42:0/2731613456 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f9744033210 con 0x7f97540bc040 2024-07-04T17:34:26.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.646+0000 7f975be00640 1 -- 172.21.15.42:0/2731613456 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97540b7c00 con 0x7f97540bc040 2024-07-04T17:34:26.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.646+0000 7f9751600640 1 --2- 172.21.15.42:0/2731613456 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f973403d4f0 0x7f973403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:26.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.647+0000 7f9751600640 1 -- 172.21.15.42:0/2731613456 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 3705+0+0 (secure 0 0 0) 0x7f9744072ff0 con 0x7f97540bc040 2024-07-04T17:34:26.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.647+0000 7f9758a00640 1 --2- 172.21.15.42:0/2731613456 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f973403d4f0 0x7f973403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:26.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.647+0000 7f9758a00640 1 --2- 172.21.15.42:0/2731613456 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f973403d4f0 0x7f973403fa00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9740007fd0 tx=0x7f9740007920 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:26.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.652+0000 7f9751600640 1 -- 172.21.15.42:0/2731613456 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f974403f030 con 0x7f97540bc040 2024-07-04T17:34:26.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.856+0000 7f975be00640 1 -- 172.21.15.42:0/2731613456 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f97540c23b0 con 0x7f97540bc040 2024-07-04T17:34:26.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.857+0000 7f9751600640 1 -- 172.21.15.42:0/2731613456 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) ==== 74+0+8404 (secure 0 0 0) 0x7f974404ccf0 con 0x7f97540bc040 2024-07-04T17:34:26.857 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:34:26.857 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":127,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:34:24.298246+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6831/3224527035":"2024-07-05T17:34:20.759432+0000","172.21.15.42:6833/605786437":"2024-07-05T17:34:14.040930+0000","172.21.15.42:6830/3224527035":"2024-07-05T17:34:20.759432+0000","172.21.15.42:6832/605786437":"2024-07-05T17:34:14.040930+0000","172.21.15.42:0/2864803316":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/397279469":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/1279811784":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/3718814387":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/1854623919":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/3279442351":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/1060612519":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/1246900581":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/130985999":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/2238986031":"2024-07-04T18:33:57.174196+0000","172.21.15.42:6835/1310962821":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/3738568139":"2024-07-05T17:33:42.103205+0000","172.21.15.42:6835/4130303917":"2024-07-05T17:33:42.103205+0000","172.21.15.42:6834/4130303917":"2024-07-05T17:33:42.103205+0000","172.21.15.42:0/1148975990":"2024-07-05T17:33:42.103205+0000","172.21.15.42:0/1387747213":"2024-07-05T17:33:42.103205+0000","172.21.15.42:0/2342229102":"2024-07-05T17:33:42.103205+0000","172.21.15.42:0/3100365373":"2024-07-05T17:33:42.103205+0000","172.21.15.42:6834/1310962821":"2024-07-05T17:34:13.066085+0000","172.21.15.42:0/3283150650":"2024-07-05T17:33:42.103205+0000","172.21.15.42:0/701226807":"2024-07-05T17:33:42.103205+0000","172.21.15.42:0/4022678544":"2024-07-05T17:33:42.103205+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:34:26.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.860+0000 7f975be00640 1 -- 172.21.15.42:0/2731613456 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f973403d4f0 msgr2=0x7f973403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:26.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.860+0000 7f975be00640 1 --2- 172.21.15.42:0/2731613456 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f973403d4f0 0x7f973403fa00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9740007fd0 tx=0x7f9740007920 comp rx=0 tx=0).stop 2024-07-04T17:34:26.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.860+0000 7f975be00640 1 -- 172.21.15.42:0/2731613456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97540bc040 msgr2=0x7f97540bfae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:26.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.860+0000 7f975be00640 1 --2- 172.21.15.42:0/2731613456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97540bc040 0x7f97540bfae0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f9744009c50 tx=0x7f9744031670 comp rx=0 tx=0).stop 2024-07-04T17:34:26.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.860+0000 7f975be00640 1 -- 172.21.15.42:0/2731613456 shutdown_connections 2024-07-04T17:34:26.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.860+0000 7f975be00640 1 --2- 172.21.15.42:0/2731613456 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f973403d4f0 0x7f973403fa00 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:26.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.860+0000 7f975be00640 1 --2- 172.21.15.42:0/2731613456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97540bc040 0x7f97540bfae0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:26.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.860+0000 7f975be00640 1 -- 172.21.15.42:0/2731613456 >> 172.21.15.42:0/2731613456 conn(0x7f975406c940 msgr2=0x7f9754140eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:26.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.860+0000 7f975be00640 1 -- 172.21.15.42:0/2731613456 shutdown_connections 2024-07-04T17:34:26.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:26.860+0000 7f975be00640 1 -- 172.21.15.42:0/2731613456 wait complete. 2024-07-04T17:34:26.886 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-04T17:26:03.297630+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-04T17:34:26.887 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-04T17:34:27.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.183+0000 7f56b0c00640 1 -- 172.21.15.42:0/3954778425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56ac0b0220 msgr2=0x7f56ac06c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:27.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.183+0000 7f56b0c00640 1 --2- 172.21.15.42:0/3954778425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56ac0b0220 0x7f56ac06c760 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f56940097b0 tx=0x7f569402a4b0 comp rx=0 tx=0).stop 2024-07-04T17:34:27.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.183+0000 7f56b0c00640 1 -- 172.21.15.42:0/3954778425 shutdown_connections 2024-07-04T17:34:27.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.183+0000 7f56b0c00640 1 --2- 172.21.15.42:0/3954778425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56ac0b0220 0x7f56ac06c760 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:27.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.183+0000 7f56b0c00640 1 -- 172.21.15.42:0/3954778425 >> 172.21.15.42:0/3954778425 conn(0x7f56ac0bf4d0 msgr2=0x7f56ac0af870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:27.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.183+0000 7f56b0c00640 1 -- 172.21.15.42:0/3954778425 shutdown_connections 2024-07-04T17:34:27.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.183+0000 7f56b0c00640 1 -- 172.21.15.42:0/3954778425 wait complete. 2024-07-04T17:34:27.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.185+0000 7f56b0c00640 1 Processor -- start 2024-07-04T17:34:27.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.185+0000 7f56b0c00640 1 -- start start 2024-07-04T17:34:27.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.186+0000 7f56b0c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56ac0b0220 0x7f56ac0be5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:27.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.186+0000 7f56b0c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac14ba20 con 0x7f56ac0b0220 2024-07-04T17:34:27.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.187+0000 7f56a9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56ac0b0220 0x7f56ac0be5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:27.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.187+0000 7f56a9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56ac0b0220 0x7f56ac0be5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52026/0 (socket says 172.21.15.42:52026) 2024-07-04T17:34:27.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.187+0000 7f56a9e00640 1 -- 172.21.15.42:0/3646451326 learned_addr learned my addr 172.21.15.42:0/3646451326 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:27.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.187+0000 7f56a9e00640 1 -- 172.21.15.42:0/3646451326 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56ac071c60 con 0x7f56ac0b0220 2024-07-04T17:34:27.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.188+0000 7f56a9e00640 1 --2- 172.21.15.42:0/3646451326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56ac0b0220 0x7f56ac0be5d0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f569c00c320 tx=0x7f569c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:27.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.188+0000 7f56a2000640 1 -- 172.21.15.42:0/3646451326 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f569c0116c0 con 0x7f56ac0b0220 2024-07-04T17:34:27.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.188+0000 7f56a2000640 1 -- 172.21.15.42:0/3646451326 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f569c017070 con 0x7f56ac0b0220 2024-07-04T17:34:27.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.188+0000 7f56a2000640 1 -- 172.21.15.42:0/3646451326 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f569c011b00 con 0x7f56ac0b0220 2024-07-04T17:34:27.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.188+0000 7f56b0c00640 1 -- 172.21.15.42:0/3646451326 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56ac149770 con 0x7f56ac0b0220 2024-07-04T17:34:27.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.188+0000 7f56b0c00640 1 -- 172.21.15.42:0/3646451326 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56ac1d8f30 con 0x7f56ac0b0220 2024-07-04T17:34:27.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.189+0000 7f56b0c00640 1 -- 172.21.15.42:0/3646451326 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56ac14c070 con 0x7f56ac0b0220 2024-07-04T17:34:27.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.189+0000 7f56a2000640 1 -- 172.21.15.42:0/3646451326 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f569c011ca0 con 0x7f56ac0b0220 2024-07-04T17:34:27.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.190+0000 7f56a2000640 1 --2- 172.21.15.42:0/3646451326 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f568803d4f0 0x7f568803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:27.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.190+0000 7f56a2000640 1 -- 172.21.15.42:0/3646451326 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 3705+0+0 (secure 0 0 0) 0x7f569c051780 con 0x7f56ac0b0220 2024-07-04T17:34:27.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.190+0000 7f56a3e00640 1 --2- 172.21.15.42:0/3646451326 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f568803d4f0 0x7f568803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:27.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.191+0000 7f56a3e00640 1 --2- 172.21.15.42:0/3646451326 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f568803d4f0 0x7f568803fa00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5694034000 tx=0x7f5694004730 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:27.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.195+0000 7f56a2000640 1 -- 172.21.15.42:0/3646451326 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f56ac0b0630 con 0x7f56ac0b0220 2024-07-04T17:34:27.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.405+0000 7f56b0c00640 1 -- 172.21.15.42:0/3646451326 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f56ac149940 con 0x7f56ac0b0220 2024-07-04T17:34:27.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.406+0000 7f56a2000640 1 -- 172.21.15.42:0/3646451326 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) ==== 93+0+10 (secure 0 0 0) 0x7f569c00a140 con 0x7f56ac0b0220 2024-07-04T17:34:27.406 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-07-04T17:34:27.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.408+0000 7f56b0c00640 1 -- 172.21.15.42:0/3646451326 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f568803d4f0 msgr2=0x7f568803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:27.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.408+0000 7f56b0c00640 1 --2- 172.21.15.42:0/3646451326 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f568803d4f0 0x7f568803fa00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5694034000 tx=0x7f5694004730 comp rx=0 tx=0).stop 2024-07-04T17:34:27.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.408+0000 7f56b0c00640 1 -- 172.21.15.42:0/3646451326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56ac0b0220 msgr2=0x7f56ac0be5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:27.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.408+0000 7f56b0c00640 1 --2- 172.21.15.42:0/3646451326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56ac0b0220 0x7f56ac0be5d0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f569c00c320 tx=0x7f569c00c7e0 comp rx=0 tx=0).stop 2024-07-04T17:34:27.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.409+0000 7f56b0c00640 1 -- 172.21.15.42:0/3646451326 shutdown_connections 2024-07-04T17:34:27.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.409+0000 7f56b0c00640 1 --2- 172.21.15.42:0/3646451326 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f568803d4f0 0x7f568803fa00 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:27.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.409+0000 7f56b0c00640 1 --2- 172.21.15.42:0/3646451326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56ac0b0220 0x7f56ac0be5d0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:27.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.409+0000 7f56b0c00640 1 -- 172.21.15.42:0/3646451326 >> 172.21.15.42:0/3646451326 conn(0x7f56ac0bf4d0 msgr2=0x7f56ac0ba410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:27.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.409+0000 7f56b0c00640 1 -- 172.21.15.42:0/3646451326 shutdown_connections 2024-07-04T17:34:27.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.409+0000 7f56b0c00640 1 -- 172.21.15.42:0/3646451326 wait complete. 2024-07-04T17:34:27.437 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_mirroring.TestMirroring.test_cephfs_mirror_blocklist' 2024-07-04T17:34:27.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.721+0000 7f7a5ca00640 1 -- 172.21.15.42:0/1129907037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a580b1d20 msgr2=0x7f7a580b6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:27.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.721+0000 7f7a5ca00640 1 --2- 172.21.15.42:0/1129907037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a580b1d20 0x7f7a580b6230 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f7a440097b0 tx=0x7f7a4402a530 comp rx=0 tx=0).stop 2024-07-04T17:34:27.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.721+0000 7f7a5ca00640 1 -- 172.21.15.42:0/1129907037 shutdown_connections 2024-07-04T17:34:27.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.721+0000 7f7a5ca00640 1 --2- 172.21.15.42:0/1129907037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a580b1d20 0x7f7a580b6230 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:27.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.721+0000 7f7a5ca00640 1 -- 172.21.15.42:0/1129907037 >> 172.21.15.42:0/1129907037 conn(0x7f7a580bebc0 msgr2=0x7f7a580befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:27.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.721+0000 7f7a5ca00640 1 -- 172.21.15.42:0/1129907037 shutdown_connections 2024-07-04T17:34:27.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.721+0000 7f7a5ca00640 1 -- 172.21.15.42:0/1129907037 wait complete. 2024-07-04T17:34:27.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.723+0000 7f7a5ca00640 1 Processor -- start 2024-07-04T17:34:27.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.723+0000 7f7a5ca00640 1 -- start start 2024-07-04T17:34:27.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.724+0000 7f7a5ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a580b1d20 0x7f7a581d70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:27.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.724+0000 7f7a5ca00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7a580b8b40 con 0x7f7a580b1d20 2024-07-04T17:34:27.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.725+0000 7f7a57400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a580b1d20 0x7f7a581d70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:27.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.725+0000 7f7a57400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a580b1d20 0x7f7a581d70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52048/0 (socket says 172.21.15.42:52048) 2024-07-04T17:34:27.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.725+0000 7f7a57400640 1 -- 172.21.15.42:0/2596712322 learned_addr learned my addr 172.21.15.42:0/2596712322 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:27.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.725+0000 7f7a57400640 1 -- 172.21.15.42:0/2596712322 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a580ae720 con 0x7f7a580b1d20 2024-07-04T17:34:27.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.725+0000 7f7a57400640 1 --2- 172.21.15.42:0/2596712322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a580b1d20 0x7f7a581d70e0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800c6b0 tx=0x7f7a4800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:27.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.726+0000 7f7a4f400640 1 -- 172.21.15.42:0/2596712322 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a48016020 con 0x7f7a580b1d20 2024-07-04T17:34:27.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.726+0000 7f7a4f400640 1 -- 172.21.15.42:0/2596712322 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a4800a5c0 con 0x7f7a580b1d20 2024-07-04T17:34:27.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.726+0000 7f7a4f400640 1 -- 172.21.15.42:0/2596712322 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a4800a960 con 0x7f7a580b1d20 2024-07-04T17:34:27.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.726+0000 7f7a5ca00640 1 -- 172.21.15.42:0/2596712322 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a580ae9c0 con 0x7f7a580b1d20 2024-07-04T17:34:27.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.726+0000 7f7a5ca00640 1 -- 172.21.15.42:0/2596712322 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a581dcb90 con 0x7f7a580b1d20 2024-07-04T17:34:27.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.727+0000 7f7a5ca00640 1 -- 172.21.15.42:0/2596712322 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a580b91d0 con 0x7f7a580b1d20 2024-07-04T17:34:27.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.727+0000 7f7a4f400640 1 -- 172.21.15.42:0/2596712322 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f7a48019020 con 0x7f7a580b1d20 2024-07-04T17:34:27.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.728+0000 7f7a4f400640 1 --2- 172.21.15.42:0/2596712322 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f7a3803d4f0 0x7f7a3803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:27.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.728+0000 7f7a4f400640 1 -- 172.21.15.42:0/2596712322 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 3705+0+0 (secure 0 0 0) 0x7f7a48051900 con 0x7f7a580b1d20 2024-07-04T17:34:27.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.728+0000 7f7a56a00640 1 --2- 172.21.15.42:0/2596712322 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f7a3803d4f0 0x7f7a3803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:27.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.729+0000 7f7a56a00640 1 --2- 172.21.15.42:0/2596712322 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f7a3803d4f0 0x7f7a3803fa00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7a44034000 tx=0x7f7a44004730 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:27.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:27.732+0000 7f7a4f400640 1 -- 172.21.15.42:0/2596712322 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f7a48020020 con 0x7f7a580b1d20 2024-07-04T17:34:28.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.019+0000 7f7a5ca00640 1 -- 172.21.15.42:0/2596712322 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_mirroring.TestMirroring.test_cephfs_mirror_blocklist"]} v 0) -- 0x7f7a580aeb90 con 0x7f7a580b1d20 2024-07-04T17:34:28.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.320+0000 7f7a4f400640 1 -- 172.21.15.42:0/2596712322 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_mirroring.TestMirroring.test_cephfs_mirror_blocklist"]}]=0 v430) ==== 152+0+0 (secure 0 0 0) 0x7f7a5813ec10 con 0x7f7a580b1d20 2024-07-04T17:34:28.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.324+0000 7f7a5ca00640 1 -- 172.21.15.42:0/2596712322 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f7a3803d4f0 msgr2=0x7f7a3803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:28.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.324+0000 7f7a5ca00640 1 --2- 172.21.15.42:0/2596712322 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f7a3803d4f0 0x7f7a3803fa00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7a44034000 tx=0x7f7a44004730 comp rx=0 tx=0).stop 2024-07-04T17:34:28.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.324+0000 7f7a5ca00640 1 -- 172.21.15.42:0/2596712322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a580b1d20 msgr2=0x7f7a581d70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:28.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.324+0000 7f7a5ca00640 1 --2- 172.21.15.42:0/2596712322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a580b1d20 0x7f7a581d70e0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800c6b0 tx=0x7f7a4800cb70 comp rx=0 tx=0).stop 2024-07-04T17:34:28.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.324+0000 7f7a5ca00640 1 -- 172.21.15.42:0/2596712322 shutdown_connections 2024-07-04T17:34:28.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.324+0000 7f7a5ca00640 1 --2- 172.21.15.42:0/2596712322 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f7a3803d4f0 0x7f7a3803fa00 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:28.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.324+0000 7f7a5ca00640 1 --2- 172.21.15.42:0/2596712322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a580b1d20 0x7f7a581d70e0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:28.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.324+0000 7f7a5ca00640 1 -- 172.21.15.42:0/2596712322 >> 172.21.15.42:0/2596712322 conn(0x7f7a580bebc0 msgr2=0x7f7a5806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:28.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.325+0000 7f7a5ca00640 1 -- 172.21.15.42:0/2596712322 shutdown_connections 2024-07-04T17:34:28.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.325+0000 7f7a5ca00640 1 -- 172.21.15.42:0/2596712322 wait complete. 2024-07-04T17:34:28.357 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-07-04T17:34:28.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.654+0000 7f8865800640 1 -- 172.21.15.42:0/2194481550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f886006c2c0 msgr2=0x7f88600b5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:28.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.654+0000 7f8865800640 1 --2- 172.21.15.42:0/2194481550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f886006c2c0 0x7f88600b5650 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f884c005fd0 tx=0x7f884c02ae00 comp rx=0 tx=0).stop 2024-07-04T17:34:28.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.654+0000 7f8865800640 1 -- 172.21.15.42:0/2194481550 shutdown_connections 2024-07-04T17:34:28.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.654+0000 7f8865800640 1 --2- 172.21.15.42:0/2194481550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f886006c2c0 0x7f88600b5650 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:28.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.654+0000 7f8865800640 1 -- 172.21.15.42:0/2194481550 >> 172.21.15.42:0/2194481550 conn(0x7f88600bebc0 msgr2=0x7f88600befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:28.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.654+0000 7f8865800640 1 -- 172.21.15.42:0/2194481550 shutdown_connections 2024-07-04T17:34:28.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.654+0000 7f8865800640 1 -- 172.21.15.42:0/2194481550 wait complete. 2024-07-04T17:34:28.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.656+0000 7f8865800640 1 Processor -- start 2024-07-04T17:34:28.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.656+0000 7f8865800640 1 -- start start 2024-07-04T17:34:28.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.658+0000 7f8865800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f886006c2c0 0x7f88601d8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:28.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.658+0000 7f8865800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f88600b27c0 con 0x7f886006c2c0 2024-07-04T17:34:28.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.658+0000 7f885fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f886006c2c0 0x7f88601d8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:28.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.658+0000 7f885fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f886006c2c0 0x7f88601d8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52060/0 (socket says 172.21.15.42:52060) 2024-07-04T17:34:28.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.658+0000 7f885fe00640 1 -- 172.21.15.42:0/3755673326 learned_addr learned my addr 172.21.15.42:0/3755673326 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:28.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.658+0000 7f885fe00640 1 -- 172.21.15.42:0/3755673326 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88600aea80 con 0x7f886006c2c0 2024-07-04T17:34:28.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.659+0000 7f885fe00640 1 --2- 172.21.15.42:0/3755673326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f886006c2c0 0x7f88601d8f70 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f885000c320 tx=0x7f885000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:28.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.659+0000 7f8857e00640 1 -- 172.21.15.42:0/3755673326 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8850017070 con 0x7f886006c2c0 2024-07-04T17:34:28.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.659+0000 7f8857e00640 1 -- 172.21.15.42:0/3755673326 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8850005310 con 0x7f886006c2c0 2024-07-04T17:34:28.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.659+0000 7f8857e00640 1 -- 172.21.15.42:0/3755673326 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88500035f0 con 0x7f886006c2c0 2024-07-04T17:34:28.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.659+0000 7f8865800640 1 -- 172.21.15.42:0/3755673326 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88601d9540 con 0x7f886006c2c0 2024-07-04T17:34:28.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.659+0000 7f8865800640 1 -- 172.21.15.42:0/3755673326 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88601dc9a0 con 0x7f886006c2c0 2024-07-04T17:34:28.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.660+0000 7f8865800640 1 -- 172.21.15.42:0/3755673326 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88600ae870 con 0x7f886006c2c0 2024-07-04T17:34:28.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.661+0000 7f8857e00640 1 -- 172.21.15.42:0/3755673326 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f885001a070 con 0x7f886006c2c0 2024-07-04T17:34:28.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.661+0000 7f8857e00640 1 --2- 172.21.15.42:0/3755673326 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f884003d4f0 0x7f884003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:28.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.661+0000 7f8857e00640 1 -- 172.21.15.42:0/3755673326 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 3705+0+0 (secure 0 0 0) 0x7f8850050d30 con 0x7f886006c2c0 2024-07-04T17:34:28.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.662+0000 7f885f400640 1 --2- 172.21.15.42:0/3755673326 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f884003d4f0 0x7f884003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:28.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.662+0000 7f885f400640 1 --2- 172.21.15.42:0/3755673326 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f884003d4f0 0x7f884003fa00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f884c034000 tx=0x7f884c004730 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:28.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.666+0000 7f8857e00640 1 -- 172.21.15.42:0/3755673326 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f886006c6d0 con 0x7f886006c2c0 2024-07-04T17:34:28.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.876+0000 7f8865800640 1 -- 172.21.15.42:0/3755673326 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f88600b5650 con 0x7f886006c2c0 2024-07-04T17:34:28.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.879+0000 7f8857e00640 1 -- 172.21.15.42:0/3755673326 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v32) ==== 59+0+1628 (secure 0 0 0) 0x7f885001bc60 con 0x7f886006c2c0 2024-07-04T17:34:28.879 INFO:teuthology.orchestra.run.smithi042.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-07-04T17:34:28.879 INFO:teuthology.orchestra.run.smithi042.stdout:mon advanced mon_op_complaint_time 120 2024-07-04T17:34:28.879 INFO:teuthology.orchestra.run.smithi042.stdout:mon dev mon_warn_on_pool_no_app false 2024-07-04T17:34:28.879 INFO:teuthology.orchestra.run.smithi042.stdout:osd advanced osd_op_complaint_time 180.000000 2024-07-04T17:34:28.879 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3541.919687 2024-07-04T17:34:28.879 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3558.529833 2024-07-04T17:34:28.879 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3688.441103 2024-07-04T17:34:28.880 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced debug_mds 20/20 2024-07-04T17:34:28.880 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced debug_mds_balancer 20/20 2024-07-04T17:34:28.880 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced debug_ms 1/1 2024-07-04T17:34:28.880 INFO:teuthology.orchestra.run.smithi042.stdout:mds dev mds_debug_frag true 2024-07-04T17:34:28.880 INFO:teuthology.orchestra.run.smithi042.stdout:mds dev mds_debug_scatterstat true 2024-07-04T17:34:28.880 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced mds_op_complaint_time 180.000000 2024-07-04T17:34:28.880 INFO:teuthology.orchestra.run.smithi042.stdout:mds dev mds_verify_scatter true 2024-07-04T17:34:28.880 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced osd_op_complaint_time 180.000000 2024-07-04T17:34:28.880 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced rados_mon_op_timeout 900 2024-07-04T17:34:28.880 INFO:teuthology.orchestra.run.smithi042.stdout:mds advanced rados_osd_op_timeout 900 2024-07-04T17:34:28.880 INFO:teuthology.orchestra.run.smithi042.stdout:client advanced client_mount_timeout 600 2024-07-04T17:34:28.880 INFO:teuthology.orchestra.run.smithi042.stdout:client advanced debug_client 20/20 2024-07-04T17:34:28.881 INFO:teuthology.orchestra.run.smithi042.stdout:client advanced debug_ms 1/1 2024-07-04T17:34:28.881 INFO:teuthology.orchestra.run.smithi042.stdout:client advanced rados_mon_op_timeout 900 2024-07-04T17:34:28.881 INFO:teuthology.orchestra.run.smithi042.stdout:client advanced rados_osd_op_timeout 900 2024-07-04T17:34:28.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.882+0000 7f8865800640 1 -- 172.21.15.42:0/3755673326 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f884003d4f0 msgr2=0x7f884003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:28.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.882+0000 7f8865800640 1 --2- 172.21.15.42:0/3755673326 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f884003d4f0 0x7f884003fa00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f884c034000 tx=0x7f884c004730 comp rx=0 tx=0).stop 2024-07-04T17:34:28.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.882+0000 7f8865800640 1 -- 172.21.15.42:0/3755673326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f886006c2c0 msgr2=0x7f88601d8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:28.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.882+0000 7f8865800640 1 --2- 172.21.15.42:0/3755673326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f886006c2c0 0x7f88601d8f70 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f885000c320 tx=0x7f885000c7e0 comp rx=0 tx=0).stop 2024-07-04T17:34:28.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.882+0000 7f8865800640 1 -- 172.21.15.42:0/3755673326 shutdown_connections 2024-07-04T17:34:28.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.882+0000 7f8865800640 1 --2- 172.21.15.42:0/3755673326 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f884003d4f0 0x7f884003fa00 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:28.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.882+0000 7f8865800640 1 --2- 172.21.15.42:0/3755673326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f886006c2c0 0x7f88601d8f70 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:28.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.882+0000 7f8865800640 1 -- 172.21.15.42:0/3755673326 >> 172.21.15.42:0/3755673326 conn(0x7f88600bebc0 msgr2=0x7f88600befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:28.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.882+0000 7f8865800640 1 -- 172.21.15.42:0/3755673326 shutdown_connections 2024-07-04T17:34:28.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:28.882+0000 7f8865800640 1 -- 172.21.15.42:0/3755673326 wait complete. 2024-07-04T17:34:28.907 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-07-04T17:34:29.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.207+0000 7f2e72e00640 1 -- 172.21.15.42:0/4004205928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e6c0a99b0 msgr2=0x7f2e6c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:29.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.207+0000 7f2e72e00640 1 --2- 172.21.15.42:0/4004205928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e6c0a99b0 0x7f2e6c06d110 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f2e580097b0 tx=0x7f2e5802a540 comp rx=0 tx=0).stop 2024-07-04T17:34:29.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.208+0000 7f2e72e00640 1 -- 172.21.15.42:0/4004205928 shutdown_connections 2024-07-04T17:34:29.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.208+0000 7f2e72e00640 1 --2- 172.21.15.42:0/4004205928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e6c0a99b0 0x7f2e6c06d110 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:29.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.208+0000 7f2e72e00640 1 -- 172.21.15.42:0/4004205928 >> 172.21.15.42:0/4004205928 conn(0x7f2e6c06c020 msgr2=0x7f2e6c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:29.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.208+0000 7f2e72e00640 1 -- 172.21.15.42:0/4004205928 shutdown_connections 2024-07-04T17:34:29.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.208+0000 7f2e72e00640 1 -- 172.21.15.42:0/4004205928 wait complete. 2024-07-04T17:34:29.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.210+0000 7f2e72e00640 1 Processor -- start 2024-07-04T17:34:29.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.210+0000 7f2e72e00640 1 -- start start 2024-07-04T17:34:29.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.211+0000 7f2e72e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e6c0a99b0 0x7f2e6c1e39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:29.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.211+0000 7f2e72e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6c13f430 con 0x7f2e6c0a99b0 2024-07-04T17:34:29.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.211+0000 7f2e6be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e6c0a99b0 0x7f2e6c1e39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:29.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.212+0000 7f2e6be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e6c0a99b0 0x7f2e6c1e39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52090/0 (socket says 172.21.15.42:52090) 2024-07-04T17:34:29.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.212+0000 7f2e6be00640 1 -- 172.21.15.42:0/1206874808 learned_addr learned my addr 172.21.15.42:0/1206874808 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:29.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.212+0000 7f2e6be00640 1 -- 172.21.15.42:0/1206874808 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e6c0b7320 con 0x7f2e6c0a99b0 2024-07-04T17:34:29.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.212+0000 7f2e6be00640 1 --2- 172.21.15.42:0/1206874808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e6c0a99b0 0x7f2e6c1e39e0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c00cb40 tx=0x7f2e5c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:29.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.213+0000 7f2e63e00640 1 -- 172.21.15.42:0/1206874808 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e5c016020 con 0x7f2e6c0a99b0 2024-07-04T17:34:29.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.213+0000 7f2e63e00640 1 -- 172.21.15.42:0/1206874808 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e5c00ac30 con 0x7f2e6c0a99b0 2024-07-04T17:34:29.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.213+0000 7f2e72e00640 1 -- 172.21.15.42:0/1206874808 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e6c0b6570 con 0x7f2e6c0a99b0 2024-07-04T17:34:29.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.213+0000 7f2e63e00640 1 -- 172.21.15.42:0/1206874808 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e5c012420 con 0x7f2e6c0a99b0 2024-07-04T17:34:29.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.213+0000 7f2e72e00640 1 -- 172.21.15.42:0/1206874808 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e6c0b6f10 con 0x7f2e6c0a99b0 2024-07-04T17:34:29.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.214+0000 7f2e72e00640 1 -- 172.21.15.42:0/1206874808 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e6c13fa70 con 0x7f2e6c0a99b0 2024-07-04T17:34:29.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.214+0000 7f2e63e00640 1 -- 172.21.15.42:0/1206874808 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f2e5c019020 con 0x7f2e6c0a99b0 2024-07-04T17:34:29.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.215+0000 7f2e63e00640 1 --2- 172.21.15.42:0/1206874808 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f2e4c03d4f0 0x7f2e4c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:29.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.215+0000 7f2e63e00640 1 -- 172.21.15.42:0/1206874808 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 3705+0+0 (secure 0 0 0) 0x7f2e5c051dc0 con 0x7f2e6c0a99b0 2024-07-04T17:34:29.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.215+0000 7f2e6b400640 1 --2- 172.21.15.42:0/1206874808 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f2e4c03d4f0 0x7f2e4c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:29.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.216+0000 7f2e6b400640 1 --2- 172.21.15.42:0/1206874808 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f2e4c03d4f0 0x7f2e4c03fa00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2e58034000 tx=0x7f2e58004730 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:29.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.220+0000 7f2e63e00640 1 -- 172.21.15.42:0/1206874808 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f2e5c020030 con 0x7f2e6c0a99b0 2024-07-04T17:34:29.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.429+0000 7f2e72e00640 1 -- 172.21.15.42:0/1206874808 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2e6c1449c0 con 0x7f2e6c0a99b0 2024-07-04T17:34:29.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.433+0000 7f2e63e00640 1 -- 172.21.15.42:0/1206874808 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v33)=0 v33) ==== 122+0+0 (secure 0 0 0) 0x7f2e5c018cd0 con 0x7f2e6c0a99b0 2024-07-04T17:34:29.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.435+0000 7f2e72e00640 1 -- 172.21.15.42:0/1206874808 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f2e4c03d4f0 msgr2=0x7f2e4c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:29.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.435+0000 7f2e72e00640 1 --2- 172.21.15.42:0/1206874808 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f2e4c03d4f0 0x7f2e4c03fa00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2e58034000 tx=0x7f2e58004730 comp rx=0 tx=0).stop 2024-07-04T17:34:29.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.435+0000 7f2e72e00640 1 -- 172.21.15.42:0/1206874808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e6c0a99b0 msgr2=0x7f2e6c1e39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:29.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.435+0000 7f2e72e00640 1 --2- 172.21.15.42:0/1206874808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e6c0a99b0 0x7f2e6c1e39e0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c00cb40 tx=0x7f2e5c00a060 comp rx=0 tx=0).stop 2024-07-04T17:34:29.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.436+0000 7f2e72e00640 1 -- 172.21.15.42:0/1206874808 shutdown_connections 2024-07-04T17:34:29.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.436+0000 7f2e72e00640 1 --2- 172.21.15.42:0/1206874808 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f2e4c03d4f0 0x7f2e4c03fa00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:29.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.436+0000 7f2e72e00640 1 --2- 172.21.15.42:0/1206874808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e6c0a99b0 0x7f2e6c1e39e0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:29.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.436+0000 7f2e72e00640 1 -- 172.21.15.42:0/1206874808 >> 172.21.15.42:0/1206874808 conn(0x7f2e6c06c020 msgr2=0x7f2e6c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:29.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.436+0000 7f2e72e00640 1 -- 172.21.15.42:0/1206874808 shutdown_connections 2024-07-04T17:34:29.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.436+0000 7f2e72e00640 1 -- 172.21.15.42:0/1206874808 wait complete. 2024-07-04T17:34:29.461 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-04T17:34:29.461 DEBUG:teuthology.orchestra.run.smithi042:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-04T17:34:29.574 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-04T17:34:29.574 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:34:29.575 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:34:29.630 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-07-04T17:34:29.630 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:34:29.630 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:34:29.687 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-07-04T17:34:29.687 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-07-04T17:34:29.687 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:34:29.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.979+0000 7f3597400640 1 -- 172.21.15.42:0/842903169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35900b2d20 msgr2=0x7f35900b37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:29.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.979+0000 7f3597400640 1 --2- 172.21.15.42:0/842903169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35900b2d20 0x7f35900b37d0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f35800097b0 tx=0x7f358002a530 comp rx=0 tx=0).stop 2024-07-04T17:34:29.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.979+0000 7f3597400640 1 -- 172.21.15.42:0/842903169 shutdown_connections 2024-07-04T17:34:29.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.979+0000 7f3597400640 1 --2- 172.21.15.42:0/842903169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35900b2d20 0x7f35900b37d0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:29.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.979+0000 7f3597400640 1 -- 172.21.15.42:0/842903169 >> 172.21.15.42:0/842903169 conn(0x7f35900bf700 msgr2=0x7f35900af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:29.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.979+0000 7f3597400640 1 -- 172.21.15.42:0/842903169 shutdown_connections 2024-07-04T17:34:29.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.979+0000 7f3597400640 1 -- 172.21.15.42:0/842903169 wait complete. 2024-07-04T17:34:29.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.982+0000 7f3597400640 1 Processor -- start 2024-07-04T17:34:29.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.982+0000 7f3597400640 1 -- start start 2024-07-04T17:34:29.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.984+0000 7f3597400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35900b01f0 0x7f35901418f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:29.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.984+0000 7f3597400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f359006cc20 con 0x7f35900b01f0 2024-07-04T17:34:29.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.984+0000 7f3594a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35900b01f0 0x7f35901418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:29.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.984+0000 7f3594a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35900b01f0 0x7f35901418f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52116/0 (socket says 172.21.15.42:52116) 2024-07-04T17:34:29.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.984+0000 7f3594a00640 1 -- 172.21.15.42:0/2175368802 learned_addr learned my addr 172.21.15.42:0/2175368802 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:29.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.985+0000 7f3594a00640 1 -- 172.21.15.42:0/2175368802 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3590141410 con 0x7f35900b01f0 2024-07-04T17:34:29.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.985+0000 7f3594a00640 1 --2- 172.21.15.42:0/2175368802 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35900b01f0 0x7f35901418f0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f35800098d0 tx=0x7f3580033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:29.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.985+0000 7f358cc00640 1 -- 172.21.15.42:0/2175368802 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3580031880 con 0x7f35900b01f0 2024-07-04T17:34:29.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.985+0000 7f358cc00640 1 -- 172.21.15.42:0/2175368802 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3580031b10 con 0x7f35900b01f0 2024-07-04T17:34:29.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.985+0000 7f3597400640 1 -- 172.21.15.42:0/2175368802 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35901d7a40 con 0x7f35900b01f0 2024-07-04T17:34:29.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.985+0000 7f358cc00640 1 -- 172.21.15.42:0/2175368802 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3580031df0 con 0x7f35900b01f0 2024-07-04T17:34:29.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.985+0000 7f3597400640 1 -- 172.21.15.42:0/2175368802 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f359013f000 con 0x7f35900b01f0 2024-07-04T17:34:29.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.987+0000 7f358cc00640 1 -- 172.21.15.42:0/2175368802 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f3580033070 con 0x7f35900b01f0 2024-07-04T17:34:29.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.987+0000 7f3597400640 1 -- 172.21.15.42:0/2175368802 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f359006d2c0 con 0x7f35900b01f0 2024-07-04T17:34:29.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.987+0000 7f358cc00640 1 --2- 172.21.15.42:0/2175368802 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f357403d4f0 0x7f357403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:29.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.987+0000 7f358cc00640 1 -- 172.21.15.42:0/2175368802 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 3705+0+0 (secure 0 0 0) 0x7f3580072e20 con 0x7f35900b01f0 2024-07-04T17:34:29.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.988+0000 7f358fe00640 1 --2- 172.21.15.42:0/2175368802 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f357403d4f0 0x7f357403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:29.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.989+0000 7f358fe00640 1 --2- 172.21.15.42:0/2175368802 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f357403d4f0 0x7f357403fa00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3584007fd0 tx=0x7f3584007920 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:29.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:29.994+0000 7f358cc00640 1 -- 172.21.15.42:0/2175368802 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f358003f030 con 0x7f35900b01f0 2024-07-04T17:34:30.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.252+0000 7f3597400640 1 -- 172.21.15.42:0/2175368802 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35900af6e0 con 0x7f35900b01f0 2024-07-04T17:34:30.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.253+0000 7f358cc00640 1 -- 172.21.15.42:0/2175368802 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+4718 (secure 0 0 0) 0x7f3580003970 con 0x7f35900b01f0 2024-07-04T17:34:30.253 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:34:30.253 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":102,"btime":"2024-07-04T17:34:21:762542+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5536,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/68857121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":68857121},{"type":"v1","addr":"172.21.15.42:6825","nonce":68857121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":5544,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6828/20990456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":20990456},{"type":"v1","addr":"172.21.15.42:6828","nonce":20990456}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":5545,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/428501060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6827","nonce":428501060},{"type":"v1","addr":"172.21.15.42:6829","nonce":428501060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":5736,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6833/94781196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":94781196},{"type":"v1","addr":"172.21.15.42:6833","nonce":94781196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":100},{"gid":5768,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/2282924594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2282924594},{"type":"v1","addr":"172.21.15.42:6831","nonce":2282924594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102}],"filesystems":[]} 2024-07-04T17:34:30.254 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 102 2024-07-04T17:34:30.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.256+0000 7f3597400640 1 -- 172.21.15.42:0/2175368802 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f357403d4f0 msgr2=0x7f357403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:30.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.256+0000 7f3597400640 1 --2- 172.21.15.42:0/2175368802 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f357403d4f0 0x7f357403fa00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3584007fd0 tx=0x7f3584007920 comp rx=0 tx=0).stop 2024-07-04T17:34:30.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.256+0000 7f3597400640 1 -- 172.21.15.42:0/2175368802 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35900b01f0 msgr2=0x7f35901418f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:30.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.256+0000 7f3597400640 1 --2- 172.21.15.42:0/2175368802 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35900b01f0 0x7f35901418f0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f35800098d0 tx=0x7f3580033ad0 comp rx=0 tx=0).stop 2024-07-04T17:34:30.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.256+0000 7f3597400640 1 -- 172.21.15.42:0/2175368802 shutdown_connections 2024-07-04T17:34:30.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.256+0000 7f3597400640 1 --2- 172.21.15.42:0/2175368802 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f357403d4f0 0x7f357403fa00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:30.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.256+0000 7f3597400640 1 --2- 172.21.15.42:0/2175368802 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35900b01f0 0x7f35901418f0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:30.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.256+0000 7f3597400640 1 -- 172.21.15.42:0/2175368802 >> 172.21.15.42:0/2175368802 conn(0x7f35900bf700 msgr2=0x7f35900bc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:30.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.256+0000 7f3597400640 1 -- 172.21.15.42:0/2175368802 shutdown_connections 2024-07-04T17:34:30.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.257+0000 7f3597400640 1 -- 172.21.15.42:0/2175368802 wait complete. 2024-07-04T17:34:30.285 INFO:tasks.ceph.mds.a:Restarting daemon 2024-07-04T17:34:30.285 INFO:tasks.ceph.mds.a:Stopping old one... 2024-07-04T17:34:30.286 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-07-04T17:34:30.286 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:34:30.286 INFO:tasks.ceph.mds.b:Restarting daemon 2024-07-04T17:34:30.286 INFO:tasks.ceph.mds.b:Stopping old one... 2024-07-04T17:34:30.286 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-07-04T17:34:30.287 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:34:30.287 INFO:tasks.ceph.mds.c:Restarting daemon 2024-07-04T17:34:30.287 INFO:tasks.ceph.mds.c:Stopping old one... 2024-07-04T17:34:30.287 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-07-04T17:34:30.287 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:34:30.287 INFO:tasks.ceph.mds.d:Restarting daemon 2024-07-04T17:34:30.287 INFO:tasks.ceph.mds.d:Stopping old one... 2024-07-04T17:34:30.287 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-07-04T17:34:30.288 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:34:30.288 INFO:tasks.ceph.mds.e:Restarting daemon 2024-07-04T17:34:30.288 INFO:tasks.ceph.mds.e:Stopping old one... 2024-07-04T17:34:30.288 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-07-04T17:34:30.288 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:34:30.315 INFO:tasks.ceph.mds.c:Stopped 2024-07-04T17:34:30.315 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-07-04T17:34:30.318 INFO:tasks.ceph.mds.b:Stopped 2024-07-04T17:34:30.318 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-07-04T17:34:30.324 INFO:tasks.ceph.mds.a:Stopped 2024-07-04T17:34:30.325 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-07-04T17:34:30.327 INFO:tasks.ceph.mds.c:Started 2024-07-04T17:34:30.328 DEBUG:teuthology.parallel:result is None 2024-07-04T17:34:30.328 INFO:tasks.ceph.mds.d:Stopped 2024-07-04T17:34:30.328 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-07-04T17:34:30.329 INFO:tasks.ceph.mds.e:Stopped 2024-07-04T17:34:30.329 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-07-04T17:34:30.330 INFO:tasks.ceph.mds.b:Started 2024-07-04T17:34:30.330 DEBUG:teuthology.parallel:result is None 2024-07-04T17:34:30.333 INFO:tasks.ceph.mds.a:Started 2024-07-04T17:34:30.334 DEBUG:teuthology.parallel:result is None 2024-07-04T17:34:30.337 INFO:tasks.ceph.mds.d:Started 2024-07-04T17:34:30.337 DEBUG:teuthology.parallel:result is None 2024-07-04T17:34:30.340 INFO:tasks.ceph.mds.e:Started 2024-07-04T17:34:30.340 DEBUG:teuthology.parallel:result is None 2024-07-04T17:34:30.341 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-07-04T17:34:30.605 INFO:tasks.ceph.mds.e.smithi042.stdout:starting mds.e at 2024-07-04T17:34:30.625 INFO:tasks.ceph.mds.c.smithi042.stdout:starting mds.c at 2024-07-04T17:34:30.636 INFO:tasks.ceph.mds.b.smithi042.stdout:starting mds.b at 2024-07-04T17:34:30.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.652+0000 7f0277400640 1 -- 172.21.15.42:0/2576286558 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f027006cea0 msgr2=0x7f02700b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:30.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.652+0000 7f0277400640 1 --2- 172.21.15.42:0/2576286558 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f027006cea0 0x7f02700b3030 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f025c005fd0 tx=0x7f025c02b930 comp rx=0 tx=0).stop 2024-07-04T17:34:30.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.652+0000 7f0277400640 1 -- 172.21.15.42:0/2576286558 shutdown_connections 2024-07-04T17:34:30.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.652+0000 7f0277400640 1 --2- 172.21.15.42:0/2576286558 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f027006cea0 0x7f02700b3030 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:30.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.652+0000 7f0277400640 1 -- 172.21.15.42:0/2576286558 >> 172.21.15.42:0/2576286558 conn(0x7f02700bf700 msgr2=0x7f02700af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:30.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.653+0000 7f0277400640 1 -- 172.21.15.42:0/2576286558 shutdown_connections 2024-07-04T17:34:30.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.653+0000 7f0277400640 1 -- 172.21.15.42:0/2576286558 wait complete. 2024-07-04T17:34:30.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.655+0000 7f0277400640 1 Processor -- start 2024-07-04T17:34:30.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.655+0000 7f0277400640 1 -- start start 2024-07-04T17:34:30.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.656+0000 7f0277400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f027006cea0 0x7f02700bad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:30.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.657+0000 7f0277400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f02700b0370 con 0x7f027006cea0 2024-07-04T17:34:30.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.657+0000 7f0274a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f027006cea0 0x7f02700bad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:30.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.657+0000 7f0274a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f027006cea0 0x7f02700bad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52192/0 (socket says 172.21.15.42:52192) 2024-07-04T17:34:30.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.657+0000 7f0274a00640 1 -- 172.21.15.42:0/3033674163 learned_addr learned my addr 172.21.15.42:0/3033674163 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:30.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.657+0000 7f0274a00640 1 -- 172.21.15.42:0/3033674163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02701d3270 con 0x7f027006cea0 2024-07-04T17:34:30.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.658+0000 7f0274a00640 1 --2- 172.21.15.42:0/3033674163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f027006cea0 0x7f02700bad30 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f026400c380 tx=0x7f026400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:30.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.659+0000 7f026cc00640 1 -- 172.21.15.42:0/3033674163 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0264017070 con 0x7f027006cea0 2024-07-04T17:34:30.659 INFO:tasks.ceph.mds.a.smithi042.stdout:starting mds.a at 2024-07-04T17:34:30.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.659+0000 7f026cc00640 1 -- 172.21.15.42:0/3033674163 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02640052f0 con 0x7f027006cea0 2024-07-04T17:34:30.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.659+0000 7f026cc00640 1 -- 172.21.15.42:0/3033674163 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02640035f0 con 0x7f027006cea0 2024-07-04T17:34:30.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.659+0000 7f0277400640 1 -- 172.21.15.42:0/3033674163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02700ac5b0 con 0x7f027006cea0 2024-07-04T17:34:30.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.659+0000 7f0277400640 1 -- 172.21.15.42:0/3033674163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02700aecd0 con 0x7f027006cea0 2024-07-04T17:34:30.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.660+0000 7f0277400640 1 -- 172.21.15.42:0/3033674163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02700b3030 con 0x7f027006cea0 2024-07-04T17:34:30.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.660+0000 7f026cc00640 1 -- 172.21.15.42:0/3033674163 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f026401a070 con 0x7f027006cea0 2024-07-04T17:34:30.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.661+0000 7f026cc00640 1 --2- 172.21.15.42:0/3033674163 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f025003d540 0x7f025003fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:30.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.661+0000 7f026cc00640 1 -- 172.21.15.42:0/3033674163 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 3705+0+0 (secure 0 0 0) 0x7f0264050d30 con 0x7f027006cea0 2024-07-04T17:34:30.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.661+0000 7f026fe00640 1 --2- 172.21.15.42:0/3033674163 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f025003d540 0x7f025003fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:30.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.662+0000 7f026fe00640 1 --2- 172.21.15.42:0/3033674163 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f025003d540 0x7f025003fa50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f025c035000 tx=0x7f025c004730 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:30.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.667+0000 7f026cc00640 1 -- 172.21.15.42:0/3033674163 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f026401e0d0 con 0x7f027006cea0 2024-07-04T17:34:30.667 INFO:tasks.ceph.mds.d.smithi042.stdout:starting mds.d at 2024-07-04T17:34:30.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:30.881+0000 7f0277400640 1 -- 172.21.15.42:0/3033674163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f02701d82c0 con 0x7f027006cea0 2024-07-04T17:34:31.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.450+0000 7f026cc00640 1 -- 172.21.15.42:0/3033674163 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v129) ==== 97+0+0 (secure 0 0 0) 0x7f02640192c0 con 0x7f027006cea0 2024-07-04T17:34:31.451 INFO:teuthology.orchestra.run.smithi042.stderr: removed all blocklist entries 2024-07-04T17:34:31.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.454+0000 7f0277400640 1 -- 172.21.15.42:0/3033674163 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f025003d540 msgr2=0x7f025003fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:31.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.454+0000 7f0277400640 1 --2- 172.21.15.42:0/3033674163 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f025003d540 0x7f025003fa50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f025c035000 tx=0x7f025c004730 comp rx=0 tx=0).stop 2024-07-04T17:34:31.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.454+0000 7f0277400640 1 -- 172.21.15.42:0/3033674163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f027006cea0 msgr2=0x7f02700bad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:31.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.454+0000 7f0277400640 1 --2- 172.21.15.42:0/3033674163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f027006cea0 0x7f02700bad30 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f026400c380 tx=0x7f026400c840 comp rx=0 tx=0).stop 2024-07-04T17:34:31.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.454+0000 7f0277400640 1 -- 172.21.15.42:0/3033674163 shutdown_connections 2024-07-04T17:34:31.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.454+0000 7f0277400640 1 --2- 172.21.15.42:0/3033674163 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f025003d540 0x7f025003fa50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:31.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.454+0000 7f0277400640 1 --2- 172.21.15.42:0/3033674163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f027006cea0 0x7f02700bad30 secure :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f026400c380 tx=0x7f026400c840 comp rx=0 tx=0).stop 2024-07-04T17:34:31.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.454+0000 7f0277400640 1 -- 172.21.15.42:0/3033674163 >> 172.21.15.42:0/3033674163 conn(0x7f02700bf700 msgr2=0x7f027014b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:31.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.455+0000 7f0277400640 1 -- 172.21.15.42:0/3033674163 shutdown_connections 2024-07-04T17:34:31.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.455+0000 7f0277400640 1 -- 172.21.15.42:0/3033674163 wait complete. 2024-07-04T17:34:31.458 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-04T17:34:31.487 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-07-04T17:34:31.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.804+0000 7fb5ce400640 1 -- 172.21.15.42:0/2397374108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5c80ba8b0 msgr2=0x7fb5c80b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:31.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.804+0000 7fb5ce400640 1 --2- 172.21.15.42:0/2397374108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5c80ba8b0 0x7fb5c80b3550 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8005fd0 tx=0x7fb5b802a8b0 comp rx=0 tx=0).stop 2024-07-04T17:34:31.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.804+0000 7fb5ce400640 1 -- 172.21.15.42:0/2397374108 shutdown_connections 2024-07-04T17:34:31.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.804+0000 7fb5ce400640 1 --2- 172.21.15.42:0/2397374108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5c80ba8b0 0x7fb5c80b3550 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:31.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.804+0000 7fb5ce400640 1 -- 172.21.15.42:0/2397374108 >> 172.21.15.42:0/2397374108 conn(0x7fb5c80bebc0 msgr2=0x7fb5c80befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:31.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.804+0000 7fb5ce400640 1 -- 172.21.15.42:0/2397374108 shutdown_connections 2024-07-04T17:34:31.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.804+0000 7fb5ce400640 1 -- 172.21.15.42:0/2397374108 wait complete. 2024-07-04T17:34:31.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.806+0000 7fb5ce400640 1 Processor -- start 2024-07-04T17:34:31.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.806+0000 7fb5ce400640 1 -- start start 2024-07-04T17:34:31.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.807+0000 7fb5ce400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5c80b1d20 0x7fb5c8141b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:31.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.807+0000 7fb5ce400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c80b8b40 con 0x7fb5c80b1d20 2024-07-04T17:34:31.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.808+0000 7fb5cd000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5c80b1d20 0x7fb5c8141b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:31.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.808+0000 7fb5cd000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5c80b1d20 0x7fb5c8141b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52234/0 (socket says 172.21.15.42:52234) 2024-07-04T17:34:31.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.808+0000 7fb5cd000640 1 -- 172.21.15.42:0/3691300797 learned_addr learned my addr 172.21.15.42:0/3691300797 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:31.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.808+0000 7fb5cd000640 1 -- 172.21.15.42:0/3691300797 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c80be520 con 0x7fb5c80b1d20 2024-07-04T17:34:31.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.808+0000 7fb5cd000640 1 --2- 172.21.15.42:0/3691300797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5c80b1d20 0x7fb5c8141b20 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8006010 tx=0x7fb5b8033b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:31.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.809+0000 7fb5c5600640 1 -- 172.21.15.42:0/3691300797 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5b80319f0 con 0x7fb5c80b1d20 2024-07-04T17:34:31.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.809+0000 7fb5c5600640 1 -- 172.21.15.42:0/3691300797 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5b8031c80 con 0x7fb5c80b1d20 2024-07-04T17:34:31.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.809+0000 7fb5c5600640 1 -- 172.21.15.42:0/3691300797 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5b8033070 con 0x7fb5c80b1d20 2024-07-04T17:34:31.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.809+0000 7fb5ce400640 1 -- 172.21.15.42:0/3691300797 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c80be0f0 con 0x7fb5c80b1d20 2024-07-04T17:34:31.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.809+0000 7fb5ce400640 1 -- 172.21.15.42:0/3691300797 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c81d5a20 con 0x7fb5c80b1d20 2024-07-04T17:34:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.810+0000 7fb5ce400640 1 -- 172.21.15.42:0/3691300797 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5c80b9240 con 0x7fb5c80b1d20 2024-07-04T17:34:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.810+0000 7fb5c5600640 1 -- 172.21.15.42:0/3691300797 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7fb5b8033210 con 0x7fb5c80b1d20 2024-07-04T17:34:31.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.811+0000 7fb5c5600640 1 --2- 172.21.15.42:0/3691300797 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fb5a403d4a0 0x7fb5a403f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:31.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.811+0000 7fb5c5600640 1 -- 172.21.15.42:0/3691300797 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 2587+0+0 (secure 0 0 0) 0x7fb5b8072cb0 con 0x7fb5c80b1d20 2024-07-04T17:34:31.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.811+0000 7fb5bfe00640 1 --2- 172.21.15.42:0/3691300797 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fb5a403d4a0 0x7fb5a403f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:31.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.812+0000 7fb5bfe00640 1 --2- 172.21.15.42:0/3691300797 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fb5a403d4a0 0x7fb5a403f9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb5c81442c0 tx=0x7fb5b0007a90 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:31.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:31.816+0000 7fb5c5600640 1 -- 172.21.15.42:0/3691300797 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fb5b803f030 con 0x7fb5c80b1d20 2024-07-04T17:34:32.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.091+0000 7fb5ce400640 1 -- 172.21.15.42:0/3691300797 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fb5c80be6f0 con 0x7fb5c80b1d20 2024-07-04T17:34:32.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.092+0000 7fb5c5600640 1 -- 172.21.15.42:0/3691300797 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+4064 (secure 0 0 0) 0x7fb5c80b6940 con 0x7fb5c80b1d20 2024-07-04T17:34:32.092 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:34:32.093 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:34:32.093 INFO:teuthology.orchestra.run.smithi042.stdout: "auth_dump": [ 2024-07-04T17:34:32.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:34:32.093 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.a", 2024-07-04T17:34:32.093 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAM24Zm2VMEFxAAQqdcGux6Ker7jEr1dGAfzA==", 2024-07-04T17:34:32.093 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:34:32.093 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:34:32.093 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:34:32.093 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:34:32.093 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:34:32.093 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:32.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:32.093 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:34:32.094 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.b", 2024-07-04T17:34:32.094 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAM24ZmS/S7KxAAfjTZrLYR9qLxuTvd2zsCcQ==", 2024-07-04T17:34:32.094 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:34:32.094 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:34:32.094 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:34:32.094 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:34:32.094 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:34:32.094 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:32.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:32.094 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:34:32.094 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.c", 2024-07-04T17:34:32.094 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAN24Zm2JrRBBAAFMYIM2fLsr9EOfcyh3tZpA==", 2024-07-04T17:34:32.094 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:34:32.095 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:34:32.095 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:34:32.095 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:34:32.095 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:34:32.095 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:32.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:32.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:34:32.095 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.d", 2024-07-04T17:34:32.095 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAN24ZmPCGLFxAAhsh3XaAHCyauCGxh/3DFsw==", 2024-07-04T17:34:32.095 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:34:32.095 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:34:32.095 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:34:32.095 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:34:32.095 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:34:32.096 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:32.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:32.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:34:32.096 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mds.e", 2024-07-04T17:34:32.096 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAN24ZmVIaSKhAAhoBdDalwQ4eoo0rqb6p0Vw==", 2024-07-04T17:34:32.096 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:34:32.096 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:34:32.096 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:34:32.096 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:34:32.096 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:34:32.096 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:32.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:32.096 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:34:32.096 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.0", 2024-07-04T17:34:32.097 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAQ24Zm4spcCBAA4kTvvR7IOVa01K5ZCrJ35w==", 2024-07-04T17:34:32.097 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:34:32.097 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-07-04T17:34:32.097 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-07-04T17:34:32.097 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:34:32.097 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:32.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:32.097 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:34:32.097 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.1", 2024-07-04T17:34:32.097 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAT24Zmz1sfGRAAgfH5eIiQrufnPq6F94rHWg==", 2024-07-04T17:34:32.097 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:34:32.097 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-07-04T17:34:32.097 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-07-04T17:34:32.097 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:34:32.097 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:32.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:32.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:34:32.098 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "osd.2", 2024-07-04T17:34:32.098 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAW24ZmJdQHORAAEjAM9OBqSIRhX148GC6pcw==", 2024-07-04T17:34:32.098 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:34:32.098 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow profile osd", 2024-07-04T17:34:32.098 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile osd", 2024-07-04T17:34:32.098 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:34:32.098 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:32.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:32.098 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:34:32.098 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.0", 2024-07-04T17:34:32.098 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAO24ZmUMmYABAAh4pNlkb9JsHQtK8uCJFQCA==", 2024-07-04T17:34:32.098 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:34:32.099 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:34:32.099 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow r", 2024-07-04T17:34:32.099 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-07-04T17:34:32.099 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:32.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:32.099 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:34:32.099 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.1", 2024-07-04T17:34:32.099 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAO24Zm0pAJEBAAntDeQ/oWDlJ+3xC7q5EQnQ==", 2024-07-04T17:34:32.099 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:34:32.099 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow", 2024-07-04T17:34:32.099 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-07-04T17:34:32.099 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow rw", 2024-07-04T17:34:32.099 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rwx" 2024-07-04T17:34:32.099 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:32.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:32.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:34:32.100 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.admin", 2024-07-04T17:34:32.100 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAL24ZmMe3iHhAAFciH9cbUAIHLSAJvXy30EQ==", 2024-07-04T17:34:32.100 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:34:32.100 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-07-04T17:34:32.100 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow *", 2024-07-04T17:34:32.100 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow *", 2024-07-04T17:34:32.100 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:34:32.100 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:32.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:32.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:34:32.100 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.mirror", 2024-07-04T17:34:32.100 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAO24ZmBJzBHhAAR4Zg36YTtVizxm/g/Cdk0w==", 2024-07-04T17:34:32.101 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:34:32.101 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow r", 2024-07-04T17:34:32.101 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-07-04T17:34:32.101 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "profile cephfs-mirror", 2024-07-04T17:34:32.101 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rw tag cephfs metadata=*, allow r tag cephfs data=*" 2024-07-04T17:34:32.101 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:32.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:32.101 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:34:32.101 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "client.mirror_remote", 2024-07-04T17:34:32.101 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAO24Zm+BsqLhAA19GVuEjfWQeMe/fSc+dmoA==", 2024-07-04T17:34:32.101 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:34:32.101 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow rwps", 2024-07-04T17:34:32.101 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": "allow r", 2024-07-04T17:34:32.102 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow r", 2024-07-04T17:34:32.102 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow rw tag cephfs *=*" 2024-07-04T17:34:32.102 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:32.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:32.102 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:34:32.102 INFO:teuthology.orchestra.run.smithi042.stdout: "entity": "mgr.x", 2024-07-04T17:34:32.102 INFO:teuthology.orchestra.run.smithi042.stdout: "key": "AQAM24Zm47tDCBAAjszudEEB4QHLlWqPXyZc2g==", 2024-07-04T17:34:32.102 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": { 2024-07-04T17:34:32.102 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": "allow *", 2024-07-04T17:34:32.102 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": "allow profile mgr", 2024-07-04T17:34:32.102 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": "allow *" 2024-07-04T17:34:32.102 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:32.103 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:32.103 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-07-04T17:34:32.103 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:34:32.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.095+0000 7fb5ce400640 1 -- 172.21.15.42:0/3691300797 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fb5a403d4a0 msgr2=0x7fb5a403f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:32.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.095+0000 7fb5ce400640 1 --2- 172.21.15.42:0/3691300797 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fb5a403d4a0 0x7fb5a403f9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb5c81442c0 tx=0x7fb5b0007a90 comp rx=0 tx=0).stop 2024-07-04T17:34:32.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.095+0000 7fb5ce400640 1 -- 172.21.15.42:0/3691300797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5c80b1d20 msgr2=0x7fb5c8141b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:32.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.095+0000 7fb5ce400640 1 --2- 172.21.15.42:0/3691300797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5c80b1d20 0x7fb5c8141b20 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8006010 tx=0x7fb5b8033b10 comp rx=0 tx=0).stop 2024-07-04T17:34:32.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.095+0000 7fb5ce400640 1 -- 172.21.15.42:0/3691300797 shutdown_connections 2024-07-04T17:34:32.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.095+0000 7fb5ce400640 1 --2- 172.21.15.42:0/3691300797 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fb5a403d4a0 0x7fb5a403f9b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:32.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.095+0000 7fb5ce400640 1 --2- 172.21.15.42:0/3691300797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb5c80b1d20 0x7fb5c8141b20 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:32.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.095+0000 7fb5ce400640 1 -- 172.21.15.42:0/3691300797 >> 172.21.15.42:0/3691300797 conn(0x7fb5c80bebc0 msgr2=0x7fb5c8142f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:32.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.095+0000 7fb5ce400640 1 -- 172.21.15.42:0/3691300797 shutdown_connections 2024-07-04T17:34:32.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.095+0000 7fb5ce400640 1 -- 172.21.15.42:0/3691300797 wait complete. 2024-07-04T17:34:32.122 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-04T17:34:32.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.399+0000 7f1ba0e00640 1 -- 172.21.15.42:0/2567868598 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b9c0ba5d0 msgr2=0x7f1b9c0b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:32.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.399+0000 7f1ba0e00640 1 --2- 172.21.15.42:0/2567868598 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b9c0ba5d0 0x7f1b9c0b2110 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f1b840097b0 tx=0x7f1b8402a540 comp rx=0 tx=0).stop 2024-07-04T17:34:32.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.399+0000 7f1ba0e00640 1 -- 172.21.15.42:0/2567868598 shutdown_connections 2024-07-04T17:34:32.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.399+0000 7f1ba0e00640 1 --2- 172.21.15.42:0/2567868598 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b9c0ba5d0 0x7f1b9c0b2110 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:32.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.399+0000 7f1ba0e00640 1 -- 172.21.15.42:0/2567868598 >> 172.21.15.42:0/2567868598 conn(0x7f1b9c0bebc0 msgr2=0x7f1b9c0befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:32.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.399+0000 7f1ba0e00640 1 -- 172.21.15.42:0/2567868598 shutdown_connections 2024-07-04T17:34:32.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.399+0000 7f1ba0e00640 1 -- 172.21.15.42:0/2567868598 wait complete. 2024-07-04T17:34:32.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.401+0000 7f1ba0e00640 1 Processor -- start 2024-07-04T17:34:32.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.402+0000 7f1ba0e00640 1 -- start start 2024-07-04T17:34:32.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.403+0000 7f1ba0e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b9c0b8f80 0x7f1b9c1d8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:32.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.403+0000 7f1ba0e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c0b3380 con 0x7f1b9c0b8f80 2024-07-04T17:34:32.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.403+0000 7f1b9b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b9c0b8f80 0x7f1b9c1d8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:32.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.403+0000 7f1b9b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b9c0b8f80 0x7f1b9c1d8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52256/0 (socket says 172.21.15.42:52256) 2024-07-04T17:34:32.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.403+0000 7f1b9b400640 1 -- 172.21.15.42:0/1793337003 learned_addr learned my addr 172.21.15.42:0/1793337003 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:32.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.404+0000 7f1b9b400640 1 -- 172.21.15.42:0/1793337003 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b9c1e44e0 con 0x7f1b9c0b8f80 2024-07-04T17:34:32.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.404+0000 7f1b9b400640 1 --2- 172.21.15.42:0/1793337003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b9c0b8f80 0x7f1b9c1d8960 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00c380 tx=0x7f1b8c00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:32.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.404+0000 7f1b93400640 1 -- 172.21.15.42:0/1793337003 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b8c0116c0 con 0x7f1b9c0b8f80 2024-07-04T17:34:32.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.404+0000 7f1b93400640 1 -- 172.21.15.42:0/1793337003 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b8c017070 con 0x7f1b9c0b8f80 2024-07-04T17:34:32.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.404+0000 7f1ba0e00640 1 -- 172.21.15.42:0/1793337003 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b9c13e290 con 0x7f1b9c0b8f80 2024-07-04T17:34:32.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.404+0000 7f1b93400640 1 -- 172.21.15.42:0/1793337003 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b8c011b20 con 0x7f1b9c0b8f80 2024-07-04T17:34:32.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.405+0000 7f1ba0e00640 1 -- 172.21.15.42:0/1793337003 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b9c0a9bc0 con 0x7f1b9c0b8f80 2024-07-04T17:34:32.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.406+0000 7f1ba0e00640 1 -- 172.21.15.42:0/1793337003 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b9c0b3a30 con 0x7f1b9c0b8f80 2024-07-04T17:34:32.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.406+0000 7f1b93400640 1 -- 172.21.15.42:0/1793337003 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f1b8c011cc0 con 0x7f1b9c0b8f80 2024-07-04T17:34:32.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.406+0000 7f1b93400640 1 --2- 172.21.15.42:0/1793337003 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f1b7803d4f0 0x7f1b7803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:32.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.406+0000 7f1b93400640 1 -- 172.21.15.42:0/1793337003 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 2587+0+0 (secure 0 0 0) 0x7f1b8c051360 con 0x7f1b9c0b8f80 2024-07-04T17:34:32.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.407+0000 7f1b93e00640 1 --2- 172.21.15.42:0/1793337003 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f1b7803d4f0 0x7f1b7803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:32.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.407+0000 7f1b93e00640 1 --2- 172.21.15.42:0/1793337003 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f1b7803d4f0 0x7f1b7803fa00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1b84034000 tx=0x7f1b84004730 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:32.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.412+0000 7f1b93400640 1 -- 172.21.15.42:0/1793337003 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f1b8c00a140 con 0x7f1b9c0b8f80 2024-07-04T17:34:32.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.621+0000 7f1ba0e00640 1 -- 172.21.15.42:0/1793337003 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f1b9c13e4c0 con 0x7f1b9c0b8f80 2024-07-04T17:34:32.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.622+0000 7f1b93400640 1 -- 172.21.15.42:0/1793337003 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v132) ==== 84+0+65 (secure 0 0 0) 0x7f1b8c010cf0 con 0x7f1b9c0b8f80 2024-07-04T17:34:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:34:32.622 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-07-04T17:34:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:34:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 1, 2024-07-04T17:34:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": ".mgr" 2024-07-04T17:34:32.622 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:32.622 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-07-04T17:34:32.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.624+0000 7f1ba0e00640 1 -- 172.21.15.42:0/1793337003 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f1b7803d4f0 msgr2=0x7f1b7803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:32.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.624+0000 7f1ba0e00640 1 --2- 172.21.15.42:0/1793337003 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f1b7803d4f0 0x7f1b7803fa00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1b84034000 tx=0x7f1b84004730 comp rx=0 tx=0).stop 2024-07-04T17:34:32.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.624+0000 7f1ba0e00640 1 -- 172.21.15.42:0/1793337003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b9c0b8f80 msgr2=0x7f1b9c1d8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:32.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.624+0000 7f1ba0e00640 1 --2- 172.21.15.42:0/1793337003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b9c0b8f80 0x7f1b9c1d8960 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00c380 tx=0x7f1b8c00c840 comp rx=0 tx=0).stop 2024-07-04T17:34:32.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.625+0000 7f1ba0e00640 1 -- 172.21.15.42:0/1793337003 shutdown_connections 2024-07-04T17:34:32.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.625+0000 7f1ba0e00640 1 --2- 172.21.15.42:0/1793337003 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f1b7803d4f0 0x7f1b7803fa00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:32.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.625+0000 7f1ba0e00640 1 --2- 172.21.15.42:0/1793337003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b9c0b8f80 0x7f1b9c1d8960 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:32.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.625+0000 7f1ba0e00640 1 -- 172.21.15.42:0/1793337003 >> 172.21.15.42:0/1793337003 conn(0x7f1b9c0bebc0 msgr2=0x7f1b9c142c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:32.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.625+0000 7f1ba0e00640 1 -- 172.21.15.42:0/1793337003 shutdown_connections 2024-07-04T17:34:32.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.625+0000 7f1ba0e00640 1 -- 172.21.15.42:0/1793337003 wait complete. 2024-07-04T17:34:32.650 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-04T17:34:32.650 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-04T17:34:32.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.945+0000 7fa4bf200640 1 -- 172.21.15.42:0/2715798410 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4c00b0fe0 msgr2=0x7fa4c00adef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:32.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.945+0000 7fa4bf200640 1 --2- 172.21.15.42:0/2715798410 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4c00b0fe0 0x7fa4c00adef0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fa4a80097b0 tx=0x7fa4a802a530 comp rx=0 tx=0).stop 2024-07-04T17:34:32.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.945+0000 7fa4bf200640 1 -- 172.21.15.42:0/2715798410 shutdown_connections 2024-07-04T17:34:32.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.945+0000 7fa4bf200640 1 --2- 172.21.15.42:0/2715798410 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4c00b0fe0 0x7fa4c00adef0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:32.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.945+0000 7fa4bf200640 1 -- 172.21.15.42:0/2715798410 >> 172.21.15.42:0/2715798410 conn(0x7fa4c00afaa0 msgr2=0x7fa4c00afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:32.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.945+0000 7fa4bf200640 1 -- 172.21.15.42:0/2715798410 shutdown_connections 2024-07-04T17:34:32.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.945+0000 7fa4bf200640 1 -- 172.21.15.42:0/2715798410 wait complete. 2024-07-04T17:34:32.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.947+0000 7fa4bf200640 1 Processor -- start 2024-07-04T17:34:32.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.947+0000 7fa4bf200640 1 -- start start 2024-07-04T17:34:32.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.949+0000 7fa4bf200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4c00b0bd0 0x7fa4c013f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:32.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.949+0000 7fa4bf200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c00accd0 con 0x7fa4c00b0bd0 2024-07-04T17:34:32.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.949+0000 7fa4bde00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4c00b0bd0 0x7fa4c013f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:32.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.949+0000 7fa4bde00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4c00b0bd0 0x7fa4c013f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52278/0 (socket says 172.21.15.42:52278) 2024-07-04T17:34:32.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.949+0000 7fa4bde00640 1 -- 172.21.15.42:0/2204036423 learned_addr learned my addr 172.21.15.42:0/2204036423 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:32.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.949+0000 7fa4bde00640 1 -- 172.21.15.42:0/2204036423 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c0147180 con 0x7fa4c00b0bd0 2024-07-04T17:34:32.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.950+0000 7fa4bde00640 1 --2- 172.21.15.42:0/2204036423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4c00b0bd0 0x7fa4c013f0a0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000c6b0 tx=0x7fa4b000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:32.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.950+0000 7fa4b6a00640 1 -- 172.21.15.42:0/2204036423 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4b0003aa0 con 0x7fa4c00b0bd0 2024-07-04T17:34:32.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.950+0000 7fa4b6a00640 1 -- 172.21.15.42:0/2204036423 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4b0003c60 con 0x7fa4c00b0bd0 2024-07-04T17:34:32.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.950+0000 7fa4b6a00640 1 -- 172.21.15.42:0/2204036423 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4b0005070 con 0x7fa4c00b0bd0 2024-07-04T17:34:32.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.950+0000 7fa4bf200640 1 -- 172.21.15.42:0/2204036423 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4c01478d0 con 0x7fa4c00b0bd0 2024-07-04T17:34:32.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.950+0000 7fa4bf200640 1 -- 172.21.15.42:0/2204036423 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4c0145c80 con 0x7fa4c00b0bd0 2024-07-04T17:34:32.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.951+0000 7fa4bf200640 1 -- 172.21.15.42:0/2204036423 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4c00ad370 con 0x7fa4c00b0bd0 2024-07-04T17:34:32.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.951+0000 7fa4b6a00640 1 -- 172.21.15.42:0/2204036423 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7fa4b0005230 con 0x7fa4c00b0bd0 2024-07-04T17:34:32.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.952+0000 7fa4b6a00640 1 --2- 172.21.15.42:0/2204036423 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fa49c03d4f0 0x7fa49c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:32.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.952+0000 7fa4b6a00640 1 -- 172.21.15.42:0/2204036423 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 2587+0+0 (secure 0 0 0) 0x7fa4b0051440 con 0x7fa4c00b0bd0 2024-07-04T17:34:32.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.952+0000 7fa4b5400640 1 --2- 172.21.15.42:0/2204036423 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fa49c03d4f0 0x7fa49c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:32.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.953+0000 7fa4b5400640 1 --2- 172.21.15.42:0/2204036423 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fa49c03d4f0 0x7fa49c03fa00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8034000 tx=0x7fa4a8004730 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:32.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:32.957+0000 7fa4b6a00640 1 -- 172.21.15.42:0/2204036423 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fa4b0005700 con 0x7fa4c00b0bd0 2024-07-04T17:34:33.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.167+0000 7fa4bf200640 1 -- 172.21.15.42:0/2204036423 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa4c0068e30 con 0x7fa4c00b0bd0 2024-07-04T17:34:33.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.462+0000 7fa4b6a00640 1 -- 172.21.15.42:0/2204036423 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v133) ==== 138+0+0 (secure 0 0 0) 0x7fa4b000a660 con 0x7fa4c00b0bd0 2024-07-04T17:34:33.462 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' created 2024-07-04T17:34:33.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.465+0000 7fa4bf200640 1 -- 172.21.15.42:0/2204036423 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fa49c03d4f0 msgr2=0x7fa49c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:33.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.465+0000 7fa4bf200640 1 --2- 172.21.15.42:0/2204036423 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fa49c03d4f0 0x7fa49c03fa00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8034000 tx=0x7fa4a8004730 comp rx=0 tx=0).stop 2024-07-04T17:34:33.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.465+0000 7fa4bf200640 1 -- 172.21.15.42:0/2204036423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4c00b0bd0 msgr2=0x7fa4c013f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:33.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.465+0000 7fa4bf200640 1 --2- 172.21.15.42:0/2204036423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4c00b0bd0 0x7fa4c013f0a0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000c6b0 tx=0x7fa4b000cb70 comp rx=0 tx=0).stop 2024-07-04T17:34:33.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.466+0000 7fa4bf200640 1 -- 172.21.15.42:0/2204036423 shutdown_connections 2024-07-04T17:34:33.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.466+0000 7fa4bf200640 1 --2- 172.21.15.42:0/2204036423 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fa49c03d4f0 0x7fa49c03fa00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:33.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.466+0000 7fa4bf200640 1 --2- 172.21.15.42:0/2204036423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4c00b0bd0 0x7fa4c013f0a0 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:33.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.466+0000 7fa4bf200640 1 -- 172.21.15.42:0/2204036423 >> 172.21.15.42:0/2204036423 conn(0x7fa4c00afaa0 msgr2=0x7fa4c00aaeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:33.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.466+0000 7fa4bf200640 1 -- 172.21.15.42:0/2204036423 shutdown_connections 2024-07-04T17:34:33.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.466+0000 7fa4bf200640 1 -- 172.21.15.42:0/2204036423 wait complete. 2024-07-04T17:34:33.502 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-04T17:34:33.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.784+0000 7f3001c00640 1 -- 172.21.15.42:0/1398537395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ffc06da20 msgr2=0x7f2ffc0b1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:33.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.784+0000 7f3001c00640 1 --2- 172.21.15.42:0/1398537395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ffc06da20 0x7f2ffc0b1020 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00a8c0 tx=0x7f2fec02b860 comp rx=0 tx=0).stop 2024-07-04T17:34:33.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.784+0000 7f3001c00640 1 -- 172.21.15.42:0/1398537395 shutdown_connections 2024-07-04T17:34:33.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.784+0000 7f3001c00640 1 --2- 172.21.15.42:0/1398537395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ffc06da20 0x7f2ffc0b1020 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:33.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.784+0000 7f3001c00640 1 -- 172.21.15.42:0/1398537395 >> 172.21.15.42:0/1398537395 conn(0x7f2ffc06c020 msgr2=0x7f2ffc06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:33.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.785+0000 7f3001c00640 1 -- 172.21.15.42:0/1398537395 shutdown_connections 2024-07-04T17:34:33.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.785+0000 7f3001c00640 1 -- 172.21.15.42:0/1398537395 wait complete. 2024-07-04T17:34:33.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.787+0000 7f3001c00640 1 Processor -- start 2024-07-04T17:34:33.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.788+0000 7f3001c00640 1 -- start start 2024-07-04T17:34:33.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.789+0000 7f3001c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ffc06da20 0x7f2ffc070180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:33.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.789+0000 7f3001c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc1510e0 con 0x7f2ffc06da20 2024-07-04T17:34:33.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.790+0000 7f2ffbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ffc06da20 0x7f2ffc070180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:33.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.790+0000 7f2ffbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ffc06da20 0x7f2ffc070180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52292/0 (socket says 172.21.15.42:52292) 2024-07-04T17:34:33.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.790+0000 7f2ffbe00640 1 -- 172.21.15.42:0/3906260837 learned_addr learned my addr 172.21.15.42:0/3906260837 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:33.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.790+0000 7f2ffbe00640 1 -- 172.21.15.42:0/3906260837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ffc070770 con 0x7f2ffc06da20 2024-07-04T17:34:33.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.791+0000 7f2ffbe00640 1 --2- 172.21.15.42:0/3906260837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ffc06da20 0x7f2ffc070180 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f2fec034040 tx=0x7f2fec031b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:33.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.791+0000 7f2ff3e00640 1 -- 172.21.15.42:0/3906260837 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fec031d60 con 0x7f2ffc06da20 2024-07-04T17:34:33.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.791+0000 7f2ff3e00640 1 -- 172.21.15.42:0/3906260837 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fec0035f0 con 0x7f2ffc06da20 2024-07-04T17:34:33.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.791+0000 7f2ff3e00640 1 -- 172.21.15.42:0/3906260837 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fec0039b0 con 0x7f2ffc06da20 2024-07-04T17:34:33.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.791+0000 7f3001c00640 1 -- 172.21.15.42:0/3906260837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ffc0721c0 con 0x7f2ffc06da20 2024-07-04T17:34:33.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.791+0000 7f3001c00640 1 -- 172.21.15.42:0/3906260837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ffc1dcbe0 con 0x7f2ffc06da20 2024-07-04T17:34:33.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.792+0000 7f3001c00640 1 -- 172.21.15.42:0/3906260837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ffc0b1020 con 0x7f2ffc06da20 2024-07-04T17:34:33.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.792+0000 7f2ff3e00640 1 -- 172.21.15.42:0/3906260837 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f2fec003c60 con 0x7f2ffc06da20 2024-07-04T17:34:33.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.793+0000 7f2ff3e00640 1 --2- 172.21.15.42:0/3906260837 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f2fdc03d4a0 0x7f2fdc03f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:33.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.793+0000 7f2ff3e00640 1 -- 172.21.15.42:0/3906260837 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 2957+0+0 (secure 0 0 0) 0x7f2fec072b80 con 0x7f2ffc06da20 2024-07-04T17:34:33.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.793+0000 7f2ffb400640 1 --2- 172.21.15.42:0/3906260837 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f2fdc03d4a0 0x7f2fdc03f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:33.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.794+0000 7f2ffb400640 1 --2- 172.21.15.42:0/3906260837 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f2fdc03d4a0 0x7f2fdc03f9b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8007fd0 tx=0x7f2fe8007920 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:33.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:33.799+0000 7f2ff3e00640 1 -- 172.21.15.42:0/3906260837 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f2fec032cf0 con 0x7f2ffc06da20 2024-07-04T17:34:34.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.011+0000 7f3001c00640 1 -- 172.21.15.42:0/3906260837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2ffc14d900 con 0x7f2ffc06da20 2024-07-04T17:34:34.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.467+0000 7f2ff3e00640 1 -- 172.21.15.42:0/3906260837 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v134) ==== 170+0+0 (secure 0 0 0) 0x7f2fec044ac0 con 0x7f2ffc06da20 2024-07-04T17:34:34.467 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' created 2024-07-04T17:34:34.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.471+0000 7f3001c00640 1 -- 172.21.15.42:0/3906260837 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f2fdc03d4a0 msgr2=0x7f2fdc03f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:34.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.471+0000 7f3001c00640 1 --2- 172.21.15.42:0/3906260837 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f2fdc03d4a0 0x7f2fdc03f9b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8007fd0 tx=0x7f2fe8007920 comp rx=0 tx=0).stop 2024-07-04T17:34:34.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.471+0000 7f3001c00640 1 -- 172.21.15.42:0/3906260837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ffc06da20 msgr2=0x7f2ffc070180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:34.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.471+0000 7f3001c00640 1 --2- 172.21.15.42:0/3906260837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ffc06da20 0x7f2ffc070180 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f2fec034040 tx=0x7f2fec031b50 comp rx=0 tx=0).stop 2024-07-04T17:34:34.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.471+0000 7f3001c00640 1 -- 172.21.15.42:0/3906260837 shutdown_connections 2024-07-04T17:34:34.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.471+0000 7f3001c00640 1 --2- 172.21.15.42:0/3906260837 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f2fdc03d4a0 0x7f2fdc03f9b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:34.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.471+0000 7f3001c00640 1 --2- 172.21.15.42:0/3906260837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ffc06da20 0x7f2ffc070180 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:34.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.471+0000 7f3001c00640 1 -- 172.21.15.42:0/3906260837 >> 172.21.15.42:0/3906260837 conn(0x7f2ffc06c020 msgr2=0x7f2ffc13c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:34.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.472+0000 7f3001c00640 1 -- 172.21.15.42:0/3906260837 shutdown_connections 2024-07-04T17:34:34.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.473+0000 7f3001c00640 1 -- 172.21.15.42:0/3906260837 wait complete. 2024-07-04T17:34:34.518 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-04T17:34:34.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.784+0000 7fcc14a00640 1 -- 172.21.15.42:0/851256625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc100b0bd0 msgr2=0x7fcc10149c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:34.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.784+0000 7fcc14a00640 1 --2- 172.21.15.42:0/851256625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc100b0bd0 0x7fcc10149c20 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fcc000097b0 tx=0x7fcc0002a530 comp rx=0 tx=0).stop 2024-07-04T17:34:34.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.784+0000 7fcc14a00640 1 -- 172.21.15.42:0/851256625 shutdown_connections 2024-07-04T17:34:34.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.784+0000 7fcc14a00640 1 --2- 172.21.15.42:0/851256625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc100b0bd0 0x7fcc10149c20 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:34.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.784+0000 7fcc14a00640 1 -- 172.21.15.42:0/851256625 >> 172.21.15.42:0/851256625 conn(0x7fcc100afaa0 msgr2=0x7fcc100afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:34.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.784+0000 7fcc14a00640 1 -- 172.21.15.42:0/851256625 shutdown_connections 2024-07-04T17:34:34.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.784+0000 7fcc14a00640 1 -- 172.21.15.42:0/851256625 wait complete. 2024-07-04T17:34:34.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.786+0000 7fcc14a00640 1 Processor -- start 2024-07-04T17:34:34.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.787+0000 7fcc14a00640 1 -- start start 2024-07-04T17:34:34.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.788+0000 7fcc14a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc100b0bd0 0x7fcc101d51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:34.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.788+0000 7fcc14a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcc100accd0 con 0x7fcc100b0bd0 2024-07-04T17:34:34.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.788+0000 7fcc0e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc100b0bd0 0x7fcc101d51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:34.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.788+0000 7fcc0e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc100b0bd0 0x7fcc101d51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48168/0 (socket says 172.21.15.42:48168) 2024-07-04T17:34:34.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.788+0000 7fcc0e800640 1 -- 172.21.15.42:0/1247499860 learned_addr learned my addr 172.21.15.42:0/1247499860 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:34.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.789+0000 7fcc0e800640 1 -- 172.21.15.42:0/1247499860 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc101430a0 con 0x7fcc100b0bd0 2024-07-04T17:34:34.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.789+0000 7fcc0e800640 1 --2- 172.21.15.42:0/1247499860 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc100b0bd0 0x7fcc101d51e0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fcc000098d0 tx=0x7fcc00033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:34.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.789+0000 7fcc06a00640 1 -- 172.21.15.42:0/1247499860 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc00031880 con 0x7fcc100b0bd0 2024-07-04T17:34:34.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.789+0000 7fcc06a00640 1 -- 172.21.15.42:0/1247499860 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc00031b10 con 0x7fcc100b0bd0 2024-07-04T17:34:34.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.789+0000 7fcc06a00640 1 -- 172.21.15.42:0/1247499860 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc00031df0 con 0x7fcc100b0bd0 2024-07-04T17:34:34.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.789+0000 7fcc14a00640 1 -- 172.21.15.42:0/1247499860 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc101e3450 con 0x7fcc100b0bd0 2024-07-04T17:34:34.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.789+0000 7fcc14a00640 1 -- 172.21.15.42:0/1247499860 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc10142d40 con 0x7fcc100b0bd0 2024-07-04T17:34:34.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.791+0000 7fcc14a00640 1 -- 172.21.15.42:0/1247499860 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc100ad3a0 con 0x7fcc100b0bd0 2024-07-04T17:34:34.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.791+0000 7fcc06a00640 1 -- 172.21.15.42:0/1247499860 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7fcc00033070 con 0x7fcc100b0bd0 2024-07-04T17:34:34.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.792+0000 7fcc06a00640 1 --2- 172.21.15.42:0/1247499860 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fcbf003d4f0 0x7fcbf003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:34.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.792+0000 7fcc06a00640 1 -- 172.21.15.42:0/1247499860 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 3339+0+0 (secure 0 0 0) 0x7fcc00072c40 con 0x7fcc100b0bd0 2024-07-04T17:34:34.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.792+0000 7fcc0de00640 1 --2- 172.21.15.42:0/1247499860 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fcbf003d4f0 0x7fcbf003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:34.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.793+0000 7fcc0de00640 1 --2- 172.21.15.42:0/1247499860 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fcbf003d4f0 0x7fcbf003fa00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcc100b49b0 tx=0x7fcbfc001670 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:34.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:34.798+0000 7fcc06a00640 1 -- 172.21.15.42:0/1247499860 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fcc0003f030 con 0x7fcc100b0bd0 2024-07-04T17:34:35.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.048+0000 7fcc14a00640 1 -- 172.21.15.42:0/1247499860 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fcc100abbd0 con 0x7fcc100b0bd0 2024-07-04T17:34:35.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.485+0000 7fcc06a00640 1 -- 172.21.15.42:0/1247499860 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 25 v105) ==== 174+0+0 (secure 0 0 0) 0x7fcc0003a3a0 con 0x7fcc100b0bd0 2024-07-04T17:34:35.486 INFO:teuthology.orchestra.run.smithi042.stderr:new fs with metadata pool 24 and data pool 25 2024-07-04T17:34:35.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.489+0000 7fcc14a00640 1 -- 172.21.15.42:0/1247499860 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fcbf003d4f0 msgr2=0x7fcbf003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:35.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.489+0000 7fcc14a00640 1 --2- 172.21.15.42:0/1247499860 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fcbf003d4f0 0x7fcbf003fa00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcc100b49b0 tx=0x7fcbfc001670 comp rx=0 tx=0).stop 2024-07-04T17:34:35.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.489+0000 7fcc14a00640 1 -- 172.21.15.42:0/1247499860 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc100b0bd0 msgr2=0x7fcc101d51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:35.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.489+0000 7fcc14a00640 1 --2- 172.21.15.42:0/1247499860 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc100b0bd0 0x7fcc101d51e0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fcc000098d0 tx=0x7fcc00033ad0 comp rx=0 tx=0).stop 2024-07-04T17:34:35.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.489+0000 7fcc14a00640 1 -- 172.21.15.42:0/1247499860 shutdown_connections 2024-07-04T17:34:35.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.489+0000 7fcc14a00640 1 --2- 172.21.15.42:0/1247499860 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fcbf003d4f0 0x7fcbf003fa00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:35.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.489+0000 7fcc14a00640 1 --2- 172.21.15.42:0/1247499860 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc100b0bd0 0x7fcc101d51e0 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:35.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.489+0000 7fcc14a00640 1 -- 172.21.15.42:0/1247499860 >> 172.21.15.42:0/1247499860 conn(0x7fcc100afaa0 msgr2=0x7fcc10075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:35.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.489+0000 7fcc14a00640 1 -- 172.21.15.42:0/1247499860 shutdown_connections 2024-07-04T17:34:35.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.490+0000 7fcc14a00640 1 -- 172.21.15.42:0/1247499860 wait complete. 2024-07-04T17:34:35.537 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:34:35.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.795+0000 7f9614800640 1 -- 172.21.15.42:0/1264080386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f960c0abdd0 msgr2=0x7f960c0b0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:35.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.795+0000 7f9614800640 1 --2- 172.21.15.42:0/1264080386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f960c0abdd0 0x7f960c0b0cd0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f960000a8c0 tx=0x7f960002b860 comp rx=0 tx=0).stop 2024-07-04T17:34:35.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.796+0000 7f9614800640 1 -- 172.21.15.42:0/1264080386 shutdown_connections 2024-07-04T17:34:35.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.796+0000 7f9614800640 1 --2- 172.21.15.42:0/1264080386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f960c0abdd0 0x7f960c0b0cd0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:35.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.796+0000 7f9614800640 1 -- 172.21.15.42:0/1264080386 >> 172.21.15.42:0/1264080386 conn(0x7f960c0bf700 msgr2=0x7f960c0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:35.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.796+0000 7f9614800640 1 -- 172.21.15.42:0/1264080386 shutdown_connections 2024-07-04T17:34:35.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.796+0000 7f9614800640 1 -- 172.21.15.42:0/1264080386 wait complete. 2024-07-04T17:34:35.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.798+0000 7f9614800640 1 Processor -- start 2024-07-04T17:34:35.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.798+0000 7f9614800640 1 -- start start 2024-07-04T17:34:35.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.799+0000 7f9614800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f960c0a9d20 0x7f960c0ba7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:35.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.799+0000 7f9614800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f960c0b03a0 con 0x7f960c0a9d20 2024-07-04T17:34:35.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.799+0000 7f9611e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f960c0a9d20 0x7f960c0ba7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:35.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.799+0000 7f9611e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f960c0a9d20 0x7f960c0ba7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48192/0 (socket says 172.21.15.42:48192) 2024-07-04T17:34:35.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.799+0000 7f9611e00640 1 -- 172.21.15.42:0/1691999686 learned_addr learned my addr 172.21.15.42:0/1691999686 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:35.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.800+0000 7f9611e00640 1 -- 172.21.15.42:0/1691999686 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960c145100 con 0x7f960c0a9d20 2024-07-04T17:34:35.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.800+0000 7f9611e00640 1 --2- 172.21.15.42:0/1691999686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f960c0a9d20 0x7f960c0ba7b0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f9600000c00 tx=0x7f9600031c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:35.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.800+0000 7f9606000640 1 -- 172.21.15.42:0/1691999686 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96000035f0 con 0x7f960c0a9d20 2024-07-04T17:34:35.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.800+0000 7f9606000640 1 -- 172.21.15.42:0/1691999686 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9600003790 con 0x7f960c0a9d20 2024-07-04T17:34:35.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.800+0000 7f9606000640 1 -- 172.21.15.42:0/1691999686 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9600003b10 con 0x7f960c0a9d20 2024-07-04T17:34:35.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.800+0000 7f9614800640 1 -- 172.21.15.42:0/1691999686 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f960c146ba0 con 0x7f960c0a9d20 2024-07-04T17:34:35.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.801+0000 7f9614800640 1 -- 172.21.15.42:0/1691999686 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f960c145620 con 0x7f960c0a9d20 2024-07-04T17:34:35.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.802+0000 7f9614800640 1 -- 172.21.15.42:0/1691999686 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f960c0b0cd0 con 0x7f960c0a9d20 2024-07-04T17:34:35.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.802+0000 7f9606000640 1 -- 172.21.15.42:0/1691999686 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f9600003cb0 con 0x7f960c0a9d20 2024-07-04T17:34:35.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.802+0000 7f9606000640 1 --2- 172.21.15.42:0/1691999686 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f95ec03d4f0 0x7f95ec03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:35.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.802+0000 7f9606000640 1 -- 172.21.15.42:0/1691999686 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 3439+0+0 (secure 0 0 0) 0x7f9600072f40 con 0x7f960c0a9d20 2024-07-04T17:34:35.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.803+0000 7f9611400640 1 --2- 172.21.15.42:0/1691999686 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f95ec03d4f0 0x7f95ec03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:35.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.803+0000 7f9611400640 1 --2- 172.21.15.42:0/1691999686 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f95ec03d4f0 0x7f95ec03fa00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f960c143c70 tx=0x7f95f8007a90 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:35.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:35.808+0000 7f9606000640 1 -- 172.21.15.42:0/1691999686 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f9600071c00 con 0x7f960c0a9d20 2024-07-04T17:34:36.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.014+0000 7f9614800640 1 -- 172.21.15.42:0/1691999686 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f960c1458b0 con 0x7f960c0a9d20 2024-07-04T17:34:36.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.015+0000 7f9606000640 1 -- 172.21.15.42:0/1691999686 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) ==== 74+0+10625 (secure 0 0 0) 0x7f960004d580 con 0x7f960c0a9d20 2024-07-04T17:34:36.015 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:34:36.016 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":135,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:34:35.049743+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:34:33.191097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-07-04T17:34:34.033408+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:34:36.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.019+0000 7f9614800640 1 -- 172.21.15.42:0/1691999686 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f95ec03d4f0 msgr2=0x7f95ec03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:36.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.019+0000 7f9614800640 1 --2- 172.21.15.42:0/1691999686 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f95ec03d4f0 0x7f95ec03fa00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f960c143c70 tx=0x7f95f8007a90 comp rx=0 tx=0).stop 2024-07-04T17:34:36.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.019+0000 7f9614800640 1 -- 172.21.15.42:0/1691999686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f960c0a9d20 msgr2=0x7f960c0ba7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:36.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.019+0000 7f9614800640 1 --2- 172.21.15.42:0/1691999686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f960c0a9d20 0x7f960c0ba7b0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f9600000c00 tx=0x7f9600031c90 comp rx=0 tx=0).stop 2024-07-04T17:34:36.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.019+0000 7f9614800640 1 -- 172.21.15.42:0/1691999686 shutdown_connections 2024-07-04T17:34:36.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.019+0000 7f9614800640 1 --2- 172.21.15.42:0/1691999686 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f95ec03d4f0 0x7f95ec03fa00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:36.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.019+0000 7f9614800640 1 --2- 172.21.15.42:0/1691999686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f960c0a9d20 0x7f960c0ba7b0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:36.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.019+0000 7f9614800640 1 -- 172.21.15.42:0/1691999686 >> 172.21.15.42:0/1691999686 conn(0x7f960c0bf700 msgr2=0x7f960c0bcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:36.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.019+0000 7f9614800640 1 -- 172.21.15.42:0/1691999686 shutdown_connections 2024-07-04T17:34:36.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.019+0000 7f9614800640 1 -- 172.21.15.42:0/1691999686 wait complete. 2024-07-04T17:34:36.042 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:34:36.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.313+0000 7f2fe9a00640 1 -- 172.21.15.42:0/2064065256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fe40bad70 msgr2=0x7f2fe406c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:36.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.313+0000 7f2fe9a00640 1 --2- 172.21.15.42:0/2064065256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fe40bad70 0x7f2fe406c2c0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400b2b0 tx=0x7f2fd402ac80 comp rx=0 tx=0).stop 2024-07-04T17:34:36.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.314+0000 7f2fe9a00640 1 -- 172.21.15.42:0/2064065256 shutdown_connections 2024-07-04T17:34:36.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.314+0000 7f2fe9a00640 1 --2- 172.21.15.42:0/2064065256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fe40bad70 0x7f2fe406c2c0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:36.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.314+0000 7f2fe9a00640 1 -- 172.21.15.42:0/2064065256 >> 172.21.15.42:0/2064065256 conn(0x7f2fe40bebc0 msgr2=0x7f2fe40befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:36.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.314+0000 7f2fe9a00640 1 -- 172.21.15.42:0/2064065256 shutdown_connections 2024-07-04T17:34:36.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.314+0000 7f2fe9a00640 1 -- 172.21.15.42:0/2064065256 wait complete. 2024-07-04T17:34:36.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.316+0000 7f2fe9a00640 1 Processor -- start 2024-07-04T17:34:36.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.316+0000 7f2fe9a00640 1 -- start start 2024-07-04T17:34:36.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.317+0000 7f2fe9a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fe40b5650 0x7f2fe40bdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:36.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.317+0000 7f2fe9a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe40b27c0 con 0x7f2fe40b5650 2024-07-04T17:34:36.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.317+0000 7f2fe3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fe40b5650 0x7f2fe40bdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:36.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.317+0000 7f2fe3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fe40b5650 0x7f2fe40bdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48218/0 (socket says 172.21.15.42:48218) 2024-07-04T17:34:36.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.317+0000 7f2fe3e00640 1 -- 172.21.15.42:0/28834542 learned_addr learned my addr 172.21.15.42:0/28834542 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:36.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.318+0000 7f2fe3e00640 1 -- 172.21.15.42:0/28834542 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe40729e0 con 0x7f2fe40b5650 2024-07-04T17:34:36.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.318+0000 7f2fe3e00640 1 --2- 172.21.15.42:0/28834542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fe40b5650 0x7f2fe40bdc20 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4000c00 tx=0x7f2fd40097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:36.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.318+0000 7f2fe0c00640 1 -- 172.21.15.42:0/28834542 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fd4038040 con 0x7f2fe40b5650 2024-07-04T17:34:36.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.318+0000 7f2fe0c00640 1 -- 172.21.15.42:0/28834542 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fd4041070 con 0x7f2fe40b5650 2024-07-04T17:34:36.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.318+0000 7f2fe0c00640 1 -- 172.21.15.42:0/28834542 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fd40042f0 con 0x7f2fe40b5650 2024-07-04T17:34:36.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.318+0000 7f2fe9a00640 1 -- 172.21.15.42:0/28834542 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fe40a9e80 con 0x7f2fe40b5650 2024-07-04T17:34:36.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.319+0000 7f2fe9a00640 1 -- 172.21.15.42:0/28834542 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fe40a9b00 con 0x7f2fe40b5650 2024-07-04T17:34:36.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.320+0000 7f2fe9a00640 1 -- 172.21.15.42:0/28834542 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fe406c2c0 con 0x7f2fe40b5650 2024-07-04T17:34:36.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.320+0000 7f2fe0c00640 1 -- 172.21.15.42:0/28834542 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f2fd40035f0 con 0x7f2fe40b5650 2024-07-04T17:34:36.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.321+0000 7f2fe0c00640 1 --2- 172.21.15.42:0/28834542 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f2fc003d4f0 0x7f2fc003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:36.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.321+0000 7f2fe0c00640 1 -- 172.21.15.42:0/28834542 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 3439+0+0 (secure 0 0 0) 0x7f2fd4071ff0 con 0x7f2fe40b5650 2024-07-04T17:34:36.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.321+0000 7f2fdb400640 1 --2- 172.21.15.42:0/28834542 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f2fc003d4f0 0x7f2fc003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:36.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.322+0000 7f2fdb400640 1 --2- 172.21.15.42:0/28834542 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f2fc003d4f0 0x7f2fc003fa00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0015c0 tx=0x7f2fcc007a90 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:36.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.326+0000 7f2fe0c00640 1 -- 172.21.15.42:0/28834542 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f2fd4031e20 con 0x7f2fe40b5650 2024-07-04T17:34:36.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.539+0000 7f2fe9a00640 1 -- 172.21.15.42:0/28834542 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2fe4068e30 con 0x7f2fe40b5650 2024-07-04T17:34:36.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.541+0000 7f2fe0c00640 1 -- 172.21.15.42:0/28834542 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) ==== 74+0+10612 (secure 0 0 0) 0x7f2fd4003990 con 0x7f2fe40b5650 2024-07-04T17:34:36.541 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:34:36.541 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":136,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:34:36.498276+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:34:33.191097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-07-04T17:34:34.033408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:34:36.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.544+0000 7f2fe9a00640 1 -- 172.21.15.42:0/28834542 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f2fc003d4f0 msgr2=0x7f2fc003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:36.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.544+0000 7f2fe9a00640 1 --2- 172.21.15.42:0/28834542 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f2fc003d4f0 0x7f2fc003fa00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0015c0 tx=0x7f2fcc007a90 comp rx=0 tx=0).stop 2024-07-04T17:34:36.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.544+0000 7f2fe9a00640 1 -- 172.21.15.42:0/28834542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fe40b5650 msgr2=0x7f2fe40bdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:36.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.544+0000 7f2fe9a00640 1 --2- 172.21.15.42:0/28834542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fe40b5650 0x7f2fe40bdc20 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4000c00 tx=0x7f2fd40097d0 comp rx=0 tx=0).stop 2024-07-04T17:34:36.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.544+0000 7f2fe9a00640 1 -- 172.21.15.42:0/28834542 shutdown_connections 2024-07-04T17:34:36.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.544+0000 7f2fe9a00640 1 --2- 172.21.15.42:0/28834542 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f2fc003d4f0 0x7f2fc003fa00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:36.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.544+0000 7f2fe9a00640 1 --2- 172.21.15.42:0/28834542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fe40b5650 0x7f2fe40bdc20 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:36.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.544+0000 7f2fe9a00640 1 -- 172.21.15.42:0/28834542 >> 172.21.15.42:0/28834542 conn(0x7f2fe40bebc0 msgr2=0x7f2fe413e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:36.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.544+0000 7f2fe9a00640 1 -- 172.21.15.42:0/28834542 shutdown_connections 2024-07-04T17:34:36.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.544+0000 7f2fe9a00640 1 -- 172.21.15.42:0/28834542 wait complete. 2024-07-04T17:34:36.569 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-07-04T17:34:36.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.849+0000 7f66aee00640 1 -- 172.21.15.42:0/542812873 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66a80701d0 msgr2=0x7f66a80abe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:36.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.849+0000 7f66aee00640 1 --2- 172.21.15.42:0/542812873 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66a80701d0 0x7f66a80abe40 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f669c0097b0 tx=0x7f669c02a530 comp rx=0 tx=0).stop 2024-07-04T17:34:36.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.849+0000 7f66aee00640 1 -- 172.21.15.42:0/542812873 shutdown_connections 2024-07-04T17:34:36.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.849+0000 7f66aee00640 1 --2- 172.21.15.42:0/542812873 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66a80701d0 0x7f66a80abe40 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:36.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.849+0000 7f66aee00640 1 -- 172.21.15.42:0/542812873 >> 172.21.15.42:0/542812873 conn(0x7f66a80afaa0 msgr2=0x7f66a80afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:36.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.849+0000 7f66aee00640 1 -- 172.21.15.42:0/542812873 shutdown_connections 2024-07-04T17:34:36.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.849+0000 7f66aee00640 1 -- 172.21.15.42:0/542812873 wait complete. 2024-07-04T17:34:36.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.852+0000 7f66aee00640 1 Processor -- start 2024-07-04T17:34:36.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.852+0000 7f66aee00640 1 -- start start 2024-07-04T17:34:36.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.854+0000 7f66aee00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66a80701d0 0x7f66a80713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:36.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.854+0000 7f66aee00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f66a80ab190 con 0x7f66a80701d0 2024-07-04T17:34:36.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.854+0000 7f66ada00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66a80701d0 0x7f66a80713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:36.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.854+0000 7f66ada00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66a80701d0 0x7f66a80713c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48248/0 (socket says 172.21.15.42:48248) 2024-07-04T17:34:36.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.854+0000 7f66ada00640 1 -- 172.21.15.42:0/2708287675 learned_addr learned my addr 172.21.15.42:0/2708287675 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:36.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.855+0000 7f66ada00640 1 -- 172.21.15.42:0/2708287675 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a8070ea0 con 0x7f66a80701d0 2024-07-04T17:34:36.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.855+0000 7f66ada00640 1 --2- 172.21.15.42:0/2708287675 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66a80701d0 0x7f66a80713c0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f669c0098d0 tx=0x7f669c033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:36.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.855+0000 7f66a5600640 1 -- 172.21.15.42:0/2708287675 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f669c031880 con 0x7f66a80701d0 2024-07-04T17:34:36.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.855+0000 7f66a5600640 1 -- 172.21.15.42:0/2708287675 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f669c031b10 con 0x7f66a80701d0 2024-07-04T17:34:36.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.855+0000 7f66aee00640 1 -- 172.21.15.42:0/2708287675 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a8149f30 con 0x7f66a80701d0 2024-07-04T17:34:36.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.855+0000 7f66a5600640 1 -- 172.21.15.42:0/2708287675 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f669c031df0 con 0x7f66a80701d0 2024-07-04T17:34:36.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.855+0000 7f66aee00640 1 -- 172.21.15.42:0/2708287675 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a8070970 con 0x7f66a80701d0 2024-07-04T17:34:36.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.857+0000 7f66a5600640 1 -- 172.21.15.42:0/2708287675 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f669c033070 con 0x7f66a80701d0 2024-07-04T17:34:36.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.857+0000 7f66aee00640 1 -- 172.21.15.42:0/2708287675 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66a80abe40 con 0x7f66a80701d0 2024-07-04T17:34:36.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.857+0000 7f66a5600640 1 --2- 172.21.15.42:0/2708287675 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f668803d4f0 0x7f668803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:36.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.857+0000 7f66a5600640 1 -- 172.21.15.42:0/2708287675 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 3439+0+0 (secure 0 0 0) 0x7f669c072d10 con 0x7f66a80701d0 2024-07-04T17:34:36.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.858+0000 7f66ad000640 1 --2- 172.21.15.42:0/2708287675 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f668803d4f0 0x7f668803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:36.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.858+0000 7f66ad000640 1 --2- 172.21.15.42:0/2708287675 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f668803d4f0 0x7f668803fa00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6694007fd0 tx=0x7f6694007920 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:36.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:36.864+0000 7f66a5600640 1 -- 172.21.15.42:0/2708287675 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f669c03f030 con 0x7f66a80701d0 2024-07-04T17:34:37.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.108+0000 7f66aee00640 1 -- 172.21.15.42:0/2708287675 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f66a80aa6e0 con 0x7f66a80701d0 2024-07-04T17:34:37.459 INFO:tasks.ceph.mds.d.smithi042.stderr:2024-07-04T17:34:37.459+0000 7f624e600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-04T17:34:37.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.514+0000 7f66a5600640 1 -- 172.21.15.42:0/2708287675 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v107) ==== 118+0+0 (secure 0 0 0) 0x7f669c003970 con 0x7f66a80701d0 2024-07-04T17:34:37.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.518+0000 7f66aee00640 1 -- 172.21.15.42:0/2708287675 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f668803d4f0 msgr2=0x7f668803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:37.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.518+0000 7f66aee00640 1 --2- 172.21.15.42:0/2708287675 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f668803d4f0 0x7f668803fa00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6694007fd0 tx=0x7f6694007920 comp rx=0 tx=0).stop 2024-07-04T17:34:37.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.518+0000 7f66aee00640 1 -- 172.21.15.42:0/2708287675 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66a80701d0 msgr2=0x7f66a80713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:37.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.518+0000 7f66aee00640 1 --2- 172.21.15.42:0/2708287675 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66a80701d0 0x7f66a80713c0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f669c0098d0 tx=0x7f669c033ad0 comp rx=0 tx=0).stop 2024-07-04T17:34:37.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.518+0000 7f66aee00640 1 -- 172.21.15.42:0/2708287675 shutdown_connections 2024-07-04T17:34:37.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.518+0000 7f66aee00640 1 --2- 172.21.15.42:0/2708287675 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f668803d4f0 0x7f668803fa00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:37.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.518+0000 7f66aee00640 1 --2- 172.21.15.42:0/2708287675 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66a80701d0 0x7f66a80713c0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:37.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.518+0000 7f66aee00640 1 -- 172.21.15.42:0/2708287675 >> 172.21.15.42:0/2708287675 conn(0x7f66a80afaa0 msgr2=0x7f66a8147a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:37.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.519+0000 7f66aee00640 1 -- 172.21.15.42:0/2708287675 shutdown_connections 2024-07-04T17:34:37.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.519+0000 7f66aee00640 1 -- 172.21.15.42:0/2708287675 wait complete. 2024-07-04T17:34:37.552 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-04T17:34:37.552 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-07-04T17:34:37.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.845+0000 7f2339a00640 1 -- 172.21.15.42:0/631792704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f233406d130 msgr2=0x7f2334071d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:37.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.845+0000 7f2339a00640 1 --2- 172.21.15.42:0/631792704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f233406d130 0x7f2334071d90 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f231c005fd0 tx=0x7f231c02a8b0 comp rx=0 tx=0).stop 2024-07-04T17:34:37.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.845+0000 7f2339a00640 1 -- 172.21.15.42:0/631792704 shutdown_connections 2024-07-04T17:34:37.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.845+0000 7f2339a00640 1 --2- 172.21.15.42:0/631792704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f233406d130 0x7f2334071d90 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:37.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.845+0000 7f2339a00640 1 -- 172.21.15.42:0/631792704 >> 172.21.15.42:0/631792704 conn(0x7f233406c020 msgr2=0x7f233406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:37.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.845+0000 7f2339a00640 1 -- 172.21.15.42:0/631792704 shutdown_connections 2024-07-04T17:34:37.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.846+0000 7f2339a00640 1 -- 172.21.15.42:0/631792704 wait complete. 2024-07-04T17:34:37.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.847+0000 7f2339a00640 1 Processor -- start 2024-07-04T17:34:37.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.848+0000 7f2339a00640 1 -- start start 2024-07-04T17:34:37.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.849+0000 7f2339a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f233406d130 0x7f23340b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:37.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.849+0000 7f2339a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2334141900 con 0x7f233406d130 2024-07-04T17:34:37.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.849+0000 7f2332a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f233406d130 0x7f23340b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:37.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.849+0000 7f2332a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f233406d130 0x7f23340b6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48262/0 (socket says 172.21.15.42:48262) 2024-07-04T17:34:37.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.849+0000 7f2332a00640 1 -- 172.21.15.42:0/333247835 learned_addr learned my addr 172.21.15.42:0/333247835 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:37.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.850+0000 7f2332a00640 1 -- 172.21.15.42:0/333247835 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23340b7bf0 con 0x7f233406d130 2024-07-04T17:34:37.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.850+0000 7f2332a00640 1 --2- 172.21.15.42:0/333247835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f233406d130 0x7f23340b6120 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f2324001eb0 tx=0x7f232400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:37.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.850+0000 7f232aa00640 1 -- 172.21.15.42:0/333247835 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23240116c0 con 0x7f233406d130 2024-07-04T17:34:37.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.850+0000 7f232aa00640 1 -- 172.21.15.42:0/333247835 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2324017070 con 0x7f233406d130 2024-07-04T17:34:37.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.850+0000 7f232aa00640 1 -- 172.21.15.42:0/333247835 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2324011ae0 con 0x7f233406d130 2024-07-04T17:34:37.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.850+0000 7f2339a00640 1 -- 172.21.15.42:0/333247835 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23341e4580 con 0x7f233406d130 2024-07-04T17:34:37.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.851+0000 7f2339a00640 1 -- 172.21.15.42:0/333247835 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f233413e640 con 0x7f233406d130 2024-07-04T17:34:37.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.852+0000 7f232aa00640 1 -- 172.21.15.42:0/333247835 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f2324011c80 con 0x7f233406d130 2024-07-04T17:34:37.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.852+0000 7f2339a00640 1 -- 172.21.15.42:0/333247835 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f233413e8d0 con 0x7f233406d130 2024-07-04T17:34:37.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.853+0000 7f232aa00640 1 --2- 172.21.15.42:0/333247835 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f231003d4f0 0x7f231003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:37.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.853+0000 7f232aa00640 1 -- 172.21.15.42:0/333247835 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 3439+0+0 (secure 0 0 0) 0x7f2324051590 con 0x7f233406d130 2024-07-04T17:34:37.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.853+0000 7f2332000640 1 --2- 172.21.15.42:0/333247835 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f231003d4f0 0x7f231003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:37.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.854+0000 7f2332000640 1 --2- 172.21.15.42:0/333247835 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f231003d4f0 0x7f231003fa00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f231c034000 tx=0x7f231c004ab0 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:37.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:37.858+0000 7f232aa00640 1 -- 172.21.15.42:0/333247835 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f23340afc70 con 0x7f233406d130 2024-07-04T17:34:38.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.108+0000 7f2339a00640 1 -- 172.21.15.42:0/333247835 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f2334071d90 con 0x7f233406d130 2024-07-04T17:34:38.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.516+0000 7f232aa00640 1 -- 172.21.15.42:0/333247835 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v108) ==== 122+0+0 (secure 0 0 0) 0x7f232400a140 con 0x7f233406d130 2024-07-04T17:34:38.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.519+0000 7f2339a00640 1 -- 172.21.15.42:0/333247835 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f231003d4f0 msgr2=0x7f231003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:38.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.519+0000 7f2339a00640 1 --2- 172.21.15.42:0/333247835 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f231003d4f0 0x7f231003fa00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f231c034000 tx=0x7f231c004ab0 comp rx=0 tx=0).stop 2024-07-04T17:34:38.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.519+0000 7f2339a00640 1 -- 172.21.15.42:0/333247835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f233406d130 msgr2=0x7f23340b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:38.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.519+0000 7f2339a00640 1 --2- 172.21.15.42:0/333247835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f233406d130 0x7f23340b6120 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f2324001eb0 tx=0x7f232400c810 comp rx=0 tx=0).stop 2024-07-04T17:34:38.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.519+0000 7f2339a00640 1 -- 172.21.15.42:0/333247835 shutdown_connections 2024-07-04T17:34:38.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.519+0000 7f2339a00640 1 --2- 172.21.15.42:0/333247835 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f231003d4f0 0x7f231003fa00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:38.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.519+0000 7f2339a00640 1 --2- 172.21.15.42:0/333247835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f233406d130 0x7f23340b6120 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:38.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.519+0000 7f2339a00640 1 -- 172.21.15.42:0/333247835 >> 172.21.15.42:0/333247835 conn(0x7f233406c020 msgr2=0x7f2334144600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:38.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.520+0000 7f2339a00640 1 -- 172.21.15.42:0/333247835 shutdown_connections 2024-07-04T17:34:38.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.520+0000 7f2339a00640 1 -- 172.21.15.42:0/333247835 wait complete. 2024-07-04T17:34:38.552 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-07-04T17:34:38.553 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:34:38.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.836+0000 7ffbbee00640 1 -- 172.21.15.42:0/3386085972 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbb80b02b0 msgr2=0x7ffbb80b3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:38.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.836+0000 7ffbbee00640 1 --2- 172.21.15.42:0/3386085972 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbb80b02b0 0x7ffbb80b3650 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7ffba80097b0 tx=0x7ffba802a4b0 comp rx=0 tx=0).stop 2024-07-04T17:34:38.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.836+0000 7ffbbee00640 1 -- 172.21.15.42:0/3386085972 shutdown_connections 2024-07-04T17:34:38.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.836+0000 7ffbbee00640 1 --2- 172.21.15.42:0/3386085972 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbb80b02b0 0x7ffbb80b3650 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:38.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.836+0000 7ffbbee00640 1 -- 172.21.15.42:0/3386085972 >> 172.21.15.42:0/3386085972 conn(0x7ffbb80bf700 msgr2=0x7ffbb80af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:38.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.836+0000 7ffbbee00640 1 -- 172.21.15.42:0/3386085972 shutdown_connections 2024-07-04T17:34:38.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.836+0000 7ffbbee00640 1 -- 172.21.15.42:0/3386085972 wait complete. 2024-07-04T17:34:38.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.838+0000 7ffbbee00640 1 Processor -- start 2024-07-04T17:34:38.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.839+0000 7ffbbee00640 1 -- start start 2024-07-04T17:34:38.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.840+0000 7ffbbee00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbb80b02b0 0x7ffbb80b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:38.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.840+0000 7ffbbee00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb806c710 con 0x7ffbb80b02b0 2024-07-04T17:34:38.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.840+0000 7ffbb7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbb80b02b0 0x7ffbb80b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:38.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.840+0000 7ffbb7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbb80b02b0 0x7ffbb80b7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48284/0 (socket says 172.21.15.42:48284) 2024-07-04T17:34:38.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.840+0000 7ffbb7e00640 1 -- 172.21.15.42:0/1783959713 learned_addr learned my addr 172.21.15.42:0/1783959713 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:38.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.841+0000 7ffbb7e00640 1 -- 172.21.15.42:0/1783959713 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb80b9420 con 0x7ffbb80b02b0 2024-07-04T17:34:38.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.841+0000 7ffbb7e00640 1 --2- 172.21.15.42:0/1783959713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbb80b02b0 0x7ffbb80b7140 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7ffba80098d0 tx=0x7ffba8033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:38.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.841+0000 7ffbafe00640 1 -- 172.21.15.42:0/1783959713 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffba8031bc0 con 0x7ffbb80b02b0 2024-07-04T17:34:38.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.841+0000 7ffbafe00640 1 -- 172.21.15.42:0/1783959713 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffba8031e50 con 0x7ffbb80b02b0 2024-07-04T17:34:38.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.841+0000 7ffbafe00640 1 -- 172.21.15.42:0/1783959713 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffba8033260 con 0x7ffbb80b02b0 2024-07-04T17:34:38.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.841+0000 7ffbbee00640 1 -- 172.21.15.42:0/1783959713 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb81e4570 con 0x7ffbb80b02b0 2024-07-04T17:34:38.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.842+0000 7ffbbee00640 1 -- 172.21.15.42:0/1783959713 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb80ba900 con 0x7ffbb80b02b0 2024-07-04T17:34:38.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.843+0000 7ffbbee00640 1 -- 172.21.15.42:0/1783959713 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbb806cd50 con 0x7ffbb80b02b0 2024-07-04T17:34:38.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.843+0000 7ffbafe00640 1 -- 172.21.15.42:0/1783959713 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7ffba8033400 con 0x7ffbb80b02b0 2024-07-04T17:34:38.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.843+0000 7ffbafe00640 1 --2- 172.21.15.42:0/1783959713 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7ffb9803d540 0x7ffb9803fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:38.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.843+0000 7ffbafe00640 1 -- 172.21.15.42:0/1783959713 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 3439+0+0 (secure 0 0 0) 0x7ffba8072d60 con 0x7ffbb80b02b0 2024-07-04T17:34:38.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.844+0000 7ffbb7400640 1 --2- 172.21.15.42:0/1783959713 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7ffb9803d540 0x7ffb9803fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:38.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.844+0000 7ffbb7400640 1 --2- 172.21.15.42:0/1783959713 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7ffb9803d540 0x7ffb9803fa50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffba40015c0 tx=0x7ffba4007a90 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:38.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:38.849+0000 7ffbafe00640 1 -- 172.21.15.42:0/1783959713 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7ffba803f030 con 0x7ffbb80b02b0 2024-07-04T17:34:39.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.098+0000 7ffbbee00640 1 -- 172.21.15.42:0/1783959713 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffbb80be150 con 0x7ffbb80b02b0 2024-07-04T17:34:39.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.100+0000 7ffbafe00640 1 -- 172.21.15.42:0/1783959713 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+6059 (secure 0 0 0) 0x7ffba803a3a0 con 0x7ffbb80b02b0 2024-07-04T17:34:39.100 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:34:39.100 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":108,"btime":"2024-07-04T17:34:38:514215+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5803,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/1184311185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":1184311185},{"type":"v1","addr":"172.21.15.42:6825","nonce":1184311185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103},{"gid":5808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/1815554749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1815554749},{"type":"v1","addr":"172.21.15.42:6827","nonce":1815554749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":5809,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/2027504764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2027504764},{"type":"v1","addr":"172.21.15.42:6829","nonce":2027504764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":5817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/4064998265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":4064998265},{"type":"v1","addr":"172.21.15.42:6831","nonce":4064998265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:34:35.049685+0000","modified":"2024-07-04T17:34:38.110000+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5818},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5818":{"gid":5818,"name":"d","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6833/1863956336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1863956336},{"type":"v1","addr":"172.21.15.42:6833","nonce":1863956336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5818,"qdb_cluster":[5818]},"id":12}]} 2024-07-04T17:34:39.100 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 108 2024-07-04T17:34:39.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.103+0000 7ffbbee00640 1 -- 172.21.15.42:0/1783959713 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7ffb9803d540 msgr2=0x7ffb9803fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:39.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.103+0000 7ffbbee00640 1 --2- 172.21.15.42:0/1783959713 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7ffb9803d540 0x7ffb9803fa50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffba40015c0 tx=0x7ffba4007a90 comp rx=0 tx=0).stop 2024-07-04T17:34:39.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.103+0000 7ffbbee00640 1 -- 172.21.15.42:0/1783959713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbb80b02b0 msgr2=0x7ffbb80b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:39.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.103+0000 7ffbbee00640 1 --2- 172.21.15.42:0/1783959713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbb80b02b0 0x7ffbb80b7140 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7ffba80098d0 tx=0x7ffba8033e40 comp rx=0 tx=0).stop 2024-07-04T17:34:39.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.103+0000 7ffbbee00640 1 -- 172.21.15.42:0/1783959713 shutdown_connections 2024-07-04T17:34:39.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.103+0000 7ffbbee00640 1 --2- 172.21.15.42:0/1783959713 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7ffb9803d540 0x7ffb9803fa50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:39.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.103+0000 7ffbbee00640 1 --2- 172.21.15.42:0/1783959713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbb80b02b0 0x7ffbb80b7140 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:39.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.103+0000 7ffbbee00640 1 -- 172.21.15.42:0/1783959713 >> 172.21.15.42:0/1783959713 conn(0x7ffbb80bf700 msgr2=0x7ffbb8145320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:39.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.103+0000 7ffbbee00640 1 -- 172.21.15.42:0/1783959713 shutdown_connections 2024-07-04T17:34:39.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.103+0000 7ffbbee00640 1 -- 172.21.15.42:0/1783959713 wait complete. 2024-07-04T17:34:39.130 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:34:39.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.421+0000 7f8113400640 1 -- 172.21.15.42:0/100758173 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f810c0b3030 msgr2=0x7f810c0b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:39.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.421+0000 7f8113400640 1 --2- 172.21.15.42:0/100758173 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f810c0b3030 0x7f810c0b01f0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f80f80091e0 tx=0x7f80f802b590 comp rx=0 tx=0).stop 2024-07-04T17:34:39.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.421+0000 7f8113400640 1 -- 172.21.15.42:0/100758173 shutdown_connections 2024-07-04T17:34:39.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.421+0000 7f8113400640 1 --2- 172.21.15.42:0/100758173 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f810c0b3030 0x7f810c0b01f0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:39.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.421+0000 7f8113400640 1 -- 172.21.15.42:0/100758173 >> 172.21.15.42:0/100758173 conn(0x7f810c0bf700 msgr2=0x7f810c0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:39.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.422+0000 7f8113400640 1 -- 172.21.15.42:0/100758173 shutdown_connections 2024-07-04T17:34:39.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.422+0000 7f8113400640 1 -- 172.21.15.42:0/100758173 wait complete. 2024-07-04T17:34:39.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.424+0000 7f8113400640 1 Processor -- start 2024-07-04T17:34:39.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.424+0000 7f8113400640 1 -- start start 2024-07-04T17:34:39.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.425+0000 7f8113400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f810c06cea0 0x7f810c1d7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:39.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.425+0000 7f8113400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f810c06ef50 con 0x7f810c06cea0 2024-07-04T17:34:39.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.425+0000 7f8110a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f810c06cea0 0x7f810c1d7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:39.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.425+0000 7f8110a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f810c06cea0 0x7f810c1d7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48308/0 (socket says 172.21.15.42:48308) 2024-07-04T17:34:39.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.425+0000 7f8110a00640 1 -- 172.21.15.42:0/4172861766 learned_addr learned my addr 172.21.15.42:0/4172861766 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:39.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.426+0000 7f8110a00640 1 -- 172.21.15.42:0/4172861766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810c1d7260 con 0x7f810c06cea0 2024-07-04T17:34:39.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.426+0000 7f8110a00640 1 --2- 172.21.15.42:0/4172861766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f810c06cea0 0x7f810c1d7720 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f80fc00c320 tx=0x7f80fc00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:39.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.427+0000 7f8108c00640 1 -- 172.21.15.42:0/4172861766 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80fc0116c0 con 0x7f810c06cea0 2024-07-04T17:34:39.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.427+0000 7f8108c00640 1 -- 172.21.15.42:0/4172861766 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80fc017070 con 0x7f810c06cea0 2024-07-04T17:34:39.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.427+0000 7f8108c00640 1 -- 172.21.15.42:0/4172861766 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80fc011b40 con 0x7f810c06cea0 2024-07-04T17:34:39.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.427+0000 7f8113400640 1 -- 172.21.15.42:0/4172861766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f810c0bd5d0 con 0x7f810c06cea0 2024-07-04T17:34:39.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.427+0000 7f8113400640 1 -- 172.21.15.42:0/4172861766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f810c1dca60 con 0x7f810c06cea0 2024-07-04T17:34:39.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.428+0000 7f8108c00640 1 -- 172.21.15.42:0/4172861766 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f80fc011ce0 con 0x7f810c06cea0 2024-07-04T17:34:39.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.428+0000 7f8113400640 1 -- 172.21.15.42:0/4172861766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f810c1dcc90 con 0x7f810c06cea0 2024-07-04T17:34:39.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.429+0000 7f8108c00640 1 --2- 172.21.15.42:0/4172861766 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f80ec03d4f0 0x7f80ec03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:39.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.429+0000 7f8108c00640 1 -- 172.21.15.42:0/4172861766 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 3439+0+0 (secure 0 0 0) 0x7f80fc051630 con 0x7f810c06cea0 2024-07-04T17:34:39.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.429+0000 7f810be00640 1 --2- 172.21.15.42:0/4172861766 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f80ec03d4f0 0x7f80ec03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:39.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.430+0000 7f810be00640 1 --2- 172.21.15.42:0/4172861766 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f80ec03d4f0 0x7f80ec03fa00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f80f8035000 tx=0x7f80f8004730 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:39.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.434+0000 7f8108c00640 1 -- 172.21.15.42:0/4172861766 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f80fc01ac80 con 0x7f810c06cea0 2024-07-04T17:34:39.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.647+0000 7f8113400640 1 -- 172.21.15.42:0/4172861766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f810c0b01f0 con 0x7f810c06cea0 2024-07-04T17:34:39.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.650+0000 7f8108c00640 1 -- 172.21.15.42:0/4172861766 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) ==== 74+0+10612 (secure 0 0 0) 0x7f80fc0513d0 con 0x7f810c06cea0 2024-07-04T17:34:39.650 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:34:39.650 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":136,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:34:36.498276+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:34:33.191097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-07-04T17:34:34.033408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:34:39.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.652+0000 7f8113400640 1 -- 172.21.15.42:0/4172861766 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f80ec03d4f0 msgr2=0x7f80ec03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:39.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.653+0000 7f8113400640 1 --2- 172.21.15.42:0/4172861766 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f80ec03d4f0 0x7f80ec03fa00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f80f8035000 tx=0x7f80f8004730 comp rx=0 tx=0).stop 2024-07-04T17:34:39.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.653+0000 7f8113400640 1 -- 172.21.15.42:0/4172861766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f810c06cea0 msgr2=0x7f810c1d7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:39.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.653+0000 7f8113400640 1 --2- 172.21.15.42:0/4172861766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f810c06cea0 0x7f810c1d7720 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f80fc00c320 tx=0x7f80fc00c7e0 comp rx=0 tx=0).stop 2024-07-04T17:34:39.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.653+0000 7f8113400640 1 -- 172.21.15.42:0/4172861766 shutdown_connections 2024-07-04T17:34:39.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.653+0000 7f8113400640 1 --2- 172.21.15.42:0/4172861766 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f80ec03d4f0 0x7f80ec03fa00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:39.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.653+0000 7f8113400640 1 --2- 172.21.15.42:0/4172861766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f810c06cea0 0x7f810c1d7720 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:39.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.653+0000 7f8113400640 1 -- 172.21.15.42:0/4172861766 >> 172.21.15.42:0/4172861766 conn(0x7f810c0bf700 msgr2=0x7f810c0aa3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:39.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.653+0000 7f8113400640 1 -- 172.21.15.42:0/4172861766 shutdown_connections 2024-07-04T17:34:39.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.653+0000 7f8113400640 1 -- 172.21.15.42:0/4172861766 wait complete. 2024-07-04T17:34:39.681 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-04T17:34:39.681 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:34:39.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.964+0000 7f219d000640 1 -- 172.21.15.42:0/2781640535 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21980b2d20 msgr2=0x7f21980b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:39.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.964+0000 7f219d000640 1 --2- 172.21.15.42:0/2781640535 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21980b2d20 0x7f21980b01f0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f21800097b0 tx=0x7f218002a530 comp rx=0 tx=0).stop 2024-07-04T17:34:39.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.964+0000 7f219d000640 1 -- 172.21.15.42:0/2781640535 shutdown_connections 2024-07-04T17:34:39.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.964+0000 7f219d000640 1 --2- 172.21.15.42:0/2781640535 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21980b2d20 0x7f21980b01f0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:39.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.964+0000 7f219d000640 1 -- 172.21.15.42:0/2781640535 >> 172.21.15.42:0/2781640535 conn(0x7f21980bf700 msgr2=0x7f21980af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:39.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.964+0000 7f219d000640 1 -- 172.21.15.42:0/2781640535 shutdown_connections 2024-07-04T17:34:39.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.964+0000 7f219d000640 1 -- 172.21.15.42:0/2781640535 wait complete. 2024-07-04T17:34:39.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.966+0000 7f219d000640 1 Processor -- start 2024-07-04T17:34:39.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.966+0000 7f219d000640 1 -- start start 2024-07-04T17:34:39.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.968+0000 7f219d000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21980b2d20 0x7f21980b9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:39.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.968+0000 7f219d000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f219806cd40 con 0x7f21980b2d20 2024-07-04T17:34:39.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.968+0000 7f2195e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21980b2d20 0x7f21980b9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:39.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.968+0000 7f2195e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21980b2d20 0x7f21980b9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48330/0 (socket says 172.21.15.42:48330) 2024-07-04T17:34:39.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.968+0000 7f2195e00640 1 -- 172.21.15.42:0/837953157 learned_addr learned my addr 172.21.15.42:0/837953157 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:39.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.968+0000 7f2195e00640 1 -- 172.21.15.42:0/837953157 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f219813ef30 con 0x7f21980b2d20 2024-07-04T17:34:39.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.969+0000 7f2195e00640 1 --2- 172.21.15.42:0/837953157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21980b2d20 0x7f21980b9be0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f218800c380 tx=0x7f218800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:39.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.969+0000 7f218ea00640 1 -- 172.21.15.42:0/837953157 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21880107a0 con 0x7f21980b2d20 2024-07-04T17:34:39.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.969+0000 7f218ea00640 1 -- 172.21.15.42:0/837953157 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2188017070 con 0x7f21980b2d20 2024-07-04T17:34:39.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.969+0000 7f218ea00640 1 -- 172.21.15.42:0/837953157 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2188010c00 con 0x7f21980b2d20 2024-07-04T17:34:39.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.969+0000 7f219d000640 1 -- 172.21.15.42:0/837953157 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21980ae760 con 0x7f21980b2d20 2024-07-04T17:34:39.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.969+0000 7f219d000640 1 -- 172.21.15.42:0/837953157 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21980ac620 con 0x7f21980b2d20 2024-07-04T17:34:39.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.970+0000 7f219d000640 1 -- 172.21.15.42:0/837953157 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f219806d450 con 0x7f21980b2d20 2024-07-04T17:34:39.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.971+0000 7f218ea00640 1 -- 172.21.15.42:0/837953157 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f218801c460 con 0x7f21980b2d20 2024-07-04T17:34:39.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.971+0000 7f218ea00640 1 --2- 172.21.15.42:0/837953157 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f217403d4f0 0x7f217403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:39.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.971+0000 7f218ea00640 1 -- 172.21.15.42:0/837953157 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 3439+0+0 (secure 0 0 0) 0x7f2188051610 con 0x7f21980b2d20 2024-07-04T17:34:39.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.971+0000 7f218d400640 1 --2- 172.21.15.42:0/837953157 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f217403d4f0 0x7f217403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:39.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.972+0000 7f218d400640 1 --2- 172.21.15.42:0/837953157 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f217403d4f0 0x7f217403fa00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2180034000 tx=0x7f2180004730 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:39.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:39.977+0000 7f218ea00640 1 -- 172.21.15.42:0/837953157 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f218801c800 con 0x7f21980b2d20 2024-07-04T17:34:40.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.185+0000 7f219d000640 1 -- 172.21.15.42:0/837953157 --> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f21980acce0 con 0x7f217403d4f0 2024-07-04T17:34:40.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.191+0000 7f218ea00640 1 -- 172.21.15.42:0/837953157 <== mgr.5692 v2:172.21.15.42:6834/4064094064 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246494 (secure 0 0 0) 0x7f21980acce0 con 0x7f217403d4f0 2024-07-04T17:34:40.198 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:34:40.200 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:34:40.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.197+0000 7f219d000640 1 -- 172.21.15.42:0/837953157 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f217403d4f0 msgr2=0x7f217403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:40.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.197+0000 7f219d000640 1 --2- 172.21.15.42:0/837953157 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f217403d4f0 0x7f217403fa00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2180034000 tx=0x7f2180004730 comp rx=0 tx=0).stop 2024-07-04T17:34:40.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.197+0000 7f219d000640 1 -- 172.21.15.42:0/837953157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21980b2d20 msgr2=0x7f21980b9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:40.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.197+0000 7f219d000640 1 --2- 172.21.15.42:0/837953157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21980b2d20 0x7f21980b9be0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f218800c380 tx=0x7f218800c840 comp rx=0 tx=0).stop 2024-07-04T17:34:40.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.197+0000 7f219d000640 1 -- 172.21.15.42:0/837953157 shutdown_connections 2024-07-04T17:34:40.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.197+0000 7f219d000640 1 --2- 172.21.15.42:0/837953157 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f217403d4f0 0x7f217403fa00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:40.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.197+0000 7f219d000640 1 --2- 172.21.15.42:0/837953157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21980b2d20 0x7f21980b9be0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:40.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.197+0000 7f219d000640 1 -- 172.21.15.42:0/837953157 >> 172.21.15.42:0/837953157 conn(0x7f21980bf700 msgr2=0x7f21980bab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:40.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.198+0000 7f219d000640 1 -- 172.21.15.42:0/837953157 shutdown_connections 2024-07-04T17:34:40.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.198+0000 7f219d000640 1 -- 172.21.15.42:0/837953157 wait complete. 2024-07-04T17:34:40.225 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-07-04T17:34:39.108759+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":596,"num_read_kb":1630,"num_write":232,"num_write_kb":1120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":111356,"kb_used_data":5328,"kb_used_omap":4,"kb_used_meta":105979,"kb_avail":283004164,"statfs":{"total":289910292480,"available":289796263936,"internally_reserved":0,"allocated":5455872,"data_stored":2865163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":108522831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.645211"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.877030+0000","last_change":"2024-07-04T17:34:35.545191+0000","last_active":"2024-07-04T17:34:36.877030+0000","last_peered":"2024-07-04T17:34:36.877030+0000","last_clean":"2024-07-04T17:34:36.877030+0000","last_became_active":"2024-07-04T17:34:35.542859+0000","last_became_peered":"2024-07-04T17:34:35.542859+0000","last_unstale":"2024-07-04T17:34:36.877030+0000","last_undegraded":"2024-07-04T17:34:36.877030+0000","last_fullsized":"2024-07-04T17:34:36.877030+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:36:40.642306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876779+0000","last_change":"2024-07-04T17:34:35.541970+0000","last_active":"2024-07-04T17:34:36.876779+0000","last_peered":"2024-07-04T17:34:36.876779+0000","last_clean":"2024-07-04T17:34:36.876779+0000","last_became_active":"2024-07-04T17:34:35.541341+0000","last_became_peered":"2024-07-04T17:34:35.541341+0000","last_unstale":"2024-07-04T17:34:36.876779+0000","last_undegraded":"2024-07-04T17:34:36.876779+0000","last_fullsized":"2024-07-04T17:34:36.876779+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:06:30.239920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876729+0000","last_change":"2024-07-04T17:34:35.531108+0000","last_active":"2024-07-04T17:34:36.876729+0000","last_peered":"2024-07-04T17:34:36.876729+0000","last_clean":"2024-07-04T17:34:36.876729+0000","last_became_active":"2024-07-04T17:34:35.530755+0000","last_became_peered":"2024-07-04T17:34:35.530755+0000","last_unstale":"2024-07-04T17:34:36.876729+0000","last_undegraded":"2024-07-04T17:34:36.876729+0000","last_fullsized":"2024-07-04T17:34:36.876729+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:48:46.716399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876393+0000","last_change":"2024-07-04T17:34:35.534705+0000","last_active":"2024-07-04T17:34:36.876393+0000","last_peered":"2024-07-04T17:34:36.876393+0000","last_clean":"2024-07-04T17:34:36.876393+0000","last_became_active":"2024-07-04T17:34:35.532134+0000","last_became_peered":"2024-07-04T17:34:35.532134+0000","last_unstale":"2024-07-04T17:34:36.876393+0000","last_undegraded":"2024-07-04T17:34:36.876393+0000","last_fullsized":"2024-07-04T17:34:36.876393+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:04:01.056622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.149527+0000","last_change":"2024-07-04T17:34:35.529627+0000","last_active":"2024-07-04T17:34:37.149527+0000","last_peered":"2024-07-04T17:34:37.149527+0000","last_clean":"2024-07-04T17:34:37.149527+0000","last_became_active":"2024-07-04T17:34:35.528998+0000","last_became_peered":"2024-07-04T17:34:35.528998+0000","last_unstale":"2024-07-04T17:34:37.149527+0000","last_undegraded":"2024-07-04T17:34:37.149527+0000","last_fullsized":"2024-07-04T17:34:37.149527+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:53:22.203053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.875783+0000","last_change":"2024-07-04T17:34:35.517987+0000","last_active":"2024-07-04T17:34:36.875783+0000","last_peered":"2024-07-04T17:34:36.875783+0000","last_clean":"2024-07-04T17:34:36.875783+0000","last_became_active":"2024-07-04T17:34:35.517658+0000","last_became_peered":"2024-07-04T17:34:35.517658+0000","last_unstale":"2024-07-04T17:34:36.875783+0000","last_undegraded":"2024-07-04T17:34:36.875783+0000","last_fullsized":"2024-07-04T17:34:36.875783+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:49:37.922530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.148317+0000","last_change":"2024-07-04T17:34:35.555625+0000","last_active":"2024-07-04T17:34:37.148317+0000","last_peered":"2024-07-04T17:34:37.148317+0000","last_clean":"2024-07-04T17:34:37.148317+0000","last_became_active":"2024-07-04T17:34:35.554652+0000","last_became_peered":"2024-07-04T17:34:35.554652+0000","last_unstale":"2024-07-04T17:34:37.148317+0000","last_undegraded":"2024-07-04T17:34:37.148317+0000","last_fullsized":"2024-07-04T17:34:37.148317+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:47:42.975532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.875749+0000","last_change":"2024-07-04T17:34:35.543396+0000","last_active":"2024-07-04T17:34:36.875749+0000","last_peered":"2024-07-04T17:34:36.875749+0000","last_clean":"2024-07-04T17:34:36.875749+0000","last_became_active":"2024-07-04T17:34:35.542958+0000","last_became_peered":"2024-07-04T17:34:35.542958+0000","last_unstale":"2024-07-04T17:34:36.875749+0000","last_undegraded":"2024-07-04T17:34:36.875749+0000","last_fullsized":"2024-07-04T17:34:36.875749+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:37:46.927489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.877516+0000","last_change":"2024-07-04T17:34:35.543199+0000","last_active":"2024-07-04T17:34:36.877516+0000","last_peered":"2024-07-04T17:34:36.877516+0000","last_clean":"2024-07-04T17:34:36.877516+0000","last_became_active":"2024-07-04T17:34:35.537240+0000","last_became_peered":"2024-07-04T17:34:35.537240+0000","last_unstale":"2024-07-04T17:34:36.877516+0000","last_undegraded":"2024-07-04T17:34:36.877516+0000","last_fullsized":"2024-07-04T17:34:36.877516+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:17:07.663136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.516071+0000","last_change":"2024-07-04T17:34:35.552696+0000","last_active":"2024-07-04T17:34:36.516071+0000","last_peered":"2024-07-04T17:34:36.516071+0000","last_clean":"2024-07-04T17:34:36.516071+0000","last_became_active":"2024-07-04T17:34:35.552377+0000","last_became_peered":"2024-07-04T17:34:35.552377+0000","last_unstale":"2024-07-04T17:34:36.516071+0000","last_undegraded":"2024-07-04T17:34:36.516071+0000","last_fullsized":"2024-07-04T17:34:36.516071+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:54:28.842946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.150879+0000","last_change":"2024-07-04T17:34:35.535880+0000","last_active":"2024-07-04T17:34:37.150879+0000","last_peered":"2024-07-04T17:34:37.150879+0000","last_clean":"2024-07-04T17:34:37.150879+0000","last_became_active":"2024-07-04T17:34:35.535547+0000","last_became_peered":"2024-07-04T17:34:35.535547+0000","last_unstale":"2024-07-04T17:34:37.150879+0000","last_undegraded":"2024-07-04T17:34:37.150879+0000","last_fullsized":"2024-07-04T17:34:37.150879+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:22:16.634063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.514833+0000","last_change":"2024-07-04T17:34:35.561769+0000","last_active":"2024-07-04T17:34:36.514833+0000","last_peered":"2024-07-04T17:34:36.514833+0000","last_clean":"2024-07-04T17:34:36.514833+0000","last_became_active":"2024-07-04T17:34:35.560437+0000","last_became_peered":"2024-07-04T17:34:35.560437+0000","last_unstale":"2024-07-04T17:34:36.514833+0000","last_undegraded":"2024-07-04T17:34:36.514833+0000","last_fullsized":"2024-07-04T17:34:36.514833+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:52:11.652290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876275+0000","last_change":"2024-07-04T17:34:35.514882+0000","last_active":"2024-07-04T17:34:36.876275+0000","last_peered":"2024-07-04T17:34:36.876275+0000","last_clean":"2024-07-04T17:34:36.876275+0000","last_became_active":"2024-07-04T17:34:35.514560+0000","last_became_peered":"2024-07-04T17:34:35.514560+0000","last_unstale":"2024-07-04T17:34:36.876275+0000","last_undegraded":"2024-07-04T17:34:36.876275+0000","last_fullsized":"2024-07-04T17:34:36.876275+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:46:59.706787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.875838+0000","last_change":"2024-07-04T17:34:35.538102+0000","last_active":"2024-07-04T17:34:36.875838+0000","last_peered":"2024-07-04T17:34:36.875838+0000","last_clean":"2024-07-04T17:34:36.875838+0000","last_became_active":"2024-07-04T17:34:35.536203+0000","last_became_peered":"2024-07-04T17:34:35.536203+0000","last_unstale":"2024-07-04T17:34:36.875838+0000","last_undegraded":"2024-07-04T17:34:36.875838+0000","last_fullsized":"2024-07-04T17:34:36.875838+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:15:42.373781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.512868+0000","last_change":"2024-07-04T17:34:35.542229+0000","last_active":"2024-07-04T17:34:36.512868+0000","last_peered":"2024-07-04T17:34:36.512868+0000","last_clean":"2024-07-04T17:34:36.512868+0000","last_became_active":"2024-07-04T17:34:35.541754+0000","last_became_peered":"2024-07-04T17:34:35.541754+0000","last_unstale":"2024-07-04T17:34:36.512868+0000","last_undegraded":"2024-07-04T17:34:36.512868+0000","last_fullsized":"2024-07-04T17:34:36.512868+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:35:52.678989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.147957+0000","last_change":"2024-07-04T17:34:35.525814+0000","last_active":"2024-07-04T17:34:37.147957+0000","last_peered":"2024-07-04T17:34:37.147957+0000","last_clean":"2024-07-04T17:34:37.147957+0000","last_became_active":"2024-07-04T17:34:35.525479+0000","last_became_peered":"2024-07-04T17:34:35.525479+0000","last_unstale":"2024-07-04T17:34:37.147957+0000","last_undegraded":"2024-07-04T17:34:37.147957+0000","last_fullsized":"2024-07-04T17:34:37.147957+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:44:57.595263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.150012+0000","last_change":"2024-07-04T17:34:35.554702+0000","last_active":"2024-07-04T17:34:37.150012+0000","last_peered":"2024-07-04T17:34:37.150012+0000","last_clean":"2024-07-04T17:34:37.150012+0000","last_became_active":"2024-07-04T17:34:35.554205+0000","last_became_peered":"2024-07-04T17:34:35.554205+0000","last_unstale":"2024-07-04T17:34:37.150012+0000","last_undegraded":"2024-07-04T17:34:37.150012+0000","last_fullsized":"2024-07-04T17:34:37.150012+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:18:48.550958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.149338+0000","last_change":"2024-07-04T17:34:35.536019+0000","last_active":"2024-07-04T17:34:37.149338+0000","last_peered":"2024-07-04T17:34:37.149338+0000","last_clean":"2024-07-04T17:34:37.149338+0000","last_became_active":"2024-07-04T17:34:35.535705+0000","last_became_peered":"2024-07-04T17:34:35.535705+0000","last_unstale":"2024-07-04T17:34:37.149338+0000","last_undegraded":"2024-07-04T17:34:37.149338+0000","last_fullsized":"2024-07-04T17:34:37.149338+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:06:39.223609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.516771+0000","last_change":"2024-07-04T17:34:35.562071+0000","last_active":"2024-07-04T17:34:36.516771+0000","last_peered":"2024-07-04T17:34:36.516771+0000","last_clean":"2024-07-04T17:34:36.516771+0000","last_became_active":"2024-07-04T17:34:35.560220+0000","last_became_peered":"2024-07-04T17:34:35.560220+0000","last_unstale":"2024-07-04T17:34:36.516771+0000","last_undegraded":"2024-07-04T17:34:36.516771+0000","last_fullsized":"2024-07-04T17:34:36.516771+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:27:08.229969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.149675+0000","last_change":"2024-07-04T17:34:35.553714+0000","last_active":"2024-07-04T17:34:37.149675+0000","last_peered":"2024-07-04T17:34:37.149675+0000","last_clean":"2024-07-04T17:34:37.149675+0000","last_became_active":"2024-07-04T17:34:35.553384+0000","last_became_peered":"2024-07-04T17:34:35.553384+0000","last_unstale":"2024-07-04T17:34:37.149675+0000","last_undegraded":"2024-07-04T17:34:37.149675+0000","last_fullsized":"2024-07-04T17:34:37.149675+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:47:20.883963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.150040+0000","last_change":"2024-07-04T17:34:35.554950+0000","last_active":"2024-07-04T17:34:37.150040+0000","last_peered":"2024-07-04T17:34:37.150040+0000","last_clean":"2024-07-04T17:34:37.150040+0000","last_became_active":"2024-07-04T17:34:35.554259+0000","last_became_peered":"2024-07-04T17:34:35.554259+0000","last_unstale":"2024-07-04T17:34:37.150040+0000","last_undegraded":"2024-07-04T17:34:37.150040+0000","last_fullsized":"2024-07-04T17:34:37.150040+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:59:46.128637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.514038+0000","last_change":"2024-07-04T17:34:35.523276+0000","last_active":"2024-07-04T17:34:36.514038+0000","last_peered":"2024-07-04T17:34:36.514038+0000","last_clean":"2024-07-04T17:34:36.514038+0000","last_became_active":"2024-07-04T17:34:35.522611+0000","last_became_peered":"2024-07-04T17:34:35.522611+0000","last_unstale":"2024-07-04T17:34:36.514038+0000","last_undegraded":"2024-07-04T17:34:36.514038+0000","last_fullsized":"2024-07-04T17:34:36.514038+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:48:47.209945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.875725+0000","last_change":"2024-07-04T17:34:35.546941+0000","last_active":"2024-07-04T17:34:36.875725+0000","last_peered":"2024-07-04T17:34:36.875725+0000","last_clean":"2024-07-04T17:34:36.875725+0000","last_became_active":"2024-07-04T17:34:35.544163+0000","last_became_peered":"2024-07-04T17:34:35.544163+0000","last_unstale":"2024-07-04T17:34:36.875725+0000","last_undegraded":"2024-07-04T17:34:36.875725+0000","last_fullsized":"2024-07-04T17:34:36.875725+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:15:08.178810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.512521+0000","last_change":"2024-07-04T17:34:35.545420+0000","last_active":"2024-07-04T17:34:36.512521+0000","last_peered":"2024-07-04T17:34:36.512521+0000","last_clean":"2024-07-04T17:34:36.512521+0000","last_became_active":"2024-07-04T17:34:35.543935+0000","last_became_peered":"2024-07-04T17:34:35.543935+0000","last_unstale":"2024-07-04T17:34:36.512521+0000","last_undegraded":"2024-07-04T17:34:36.512521+0000","last_fullsized":"2024-07-04T17:34:36.512521+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:16:01.387094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.877366+0000","last_change":"2024-07-04T17:34:35.536388+0000","last_active":"2024-07-04T17:34:36.877366+0000","last_peered":"2024-07-04T17:34:36.877366+0000","last_clean":"2024-07-04T17:34:36.877366+0000","last_became_active":"2024-07-04T17:34:35.536052+0000","last_became_peered":"2024-07-04T17:34:35.536052+0000","last_unstale":"2024-07-04T17:34:36.877366+0000","last_undegraded":"2024-07-04T17:34:36.877366+0000","last_fullsized":"2024-07-04T17:34:36.877366+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:08:18.792180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.516187+0000","last_change":"2024-07-04T17:34:35.533907+0000","last_active":"2024-07-04T17:34:36.516187+0000","last_peered":"2024-07-04T17:34:36.516187+0000","last_clean":"2024-07-04T17:34:36.516187+0000","last_became_active":"2024-07-04T17:34:35.533596+0000","last_became_peered":"2024-07-04T17:34:35.533596+0000","last_unstale":"2024-07-04T17:34:36.516187+0000","last_undegraded":"2024-07-04T17:34:36.516187+0000","last_fullsized":"2024-07-04T17:34:36.516187+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:41:30.147907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876928+0000","last_change":"2024-07-04T17:34:35.537742+0000","last_active":"2024-07-04T17:34:36.876928+0000","last_peered":"2024-07-04T17:34:36.876928+0000","last_clean":"2024-07-04T17:34:36.876928+0000","last_became_active":"2024-07-04T17:34:35.533422+0000","last_became_peered":"2024-07-04T17:34:35.533422+0000","last_unstale":"2024-07-04T17:34:36.876928+0000","last_undegraded":"2024-07-04T17:34:36.876928+0000","last_fullsized":"2024-07-04T17:34:36.876928+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:02:05.629708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.149794+0000","last_change":"2024-07-04T17:34:35.553505+0000","last_active":"2024-07-04T17:34:37.149794+0000","last_peered":"2024-07-04T17:34:37.149794+0000","last_clean":"2024-07-04T17:34:37.149794+0000","last_became_active":"2024-07-04T17:34:35.553230+0000","last_became_peered":"2024-07-04T17:34:35.553230+0000","last_unstale":"2024-07-04T17:34:37.149794+0000","last_undegraded":"2024-07-04T17:34:37.149794+0000","last_fullsized":"2024-07-04T17:34:37.149794+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:49:38.920935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.516510+0000","last_change":"2024-07-04T17:34:35.524780+0000","last_active":"2024-07-04T17:34:36.516510+0000","last_peered":"2024-07-04T17:34:36.516510+0000","last_clean":"2024-07-04T17:34:36.516510+0000","last_became_active":"2024-07-04T17:34:35.524418+0000","last_became_peered":"2024-07-04T17:34:35.524418+0000","last_unstale":"2024-07-04T17:34:36.516510+0000","last_undegraded":"2024-07-04T17:34:36.516510+0000","last_fullsized":"2024-07-04T17:34:36.516510+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:16:31.650779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.514271+0000","last_change":"2024-07-04T17:34:35.522942+0000","last_active":"2024-07-04T17:34:36.514271+0000","last_peered":"2024-07-04T17:34:36.514271+0000","last_clean":"2024-07-04T17:34:36.514271+0000","last_became_active":"2024-07-04T17:34:35.522210+0000","last_became_peered":"2024-07-04T17:34:35.522210+0000","last_unstale":"2024-07-04T17:34:36.514271+0000","last_undegraded":"2024-07-04T17:34:36.514271+0000","last_fullsized":"2024-07-04T17:34:36.514271+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:03:46.125136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876271+0000","last_change":"2024-07-04T17:34:35.544485+0000","last_active":"2024-07-04T17:34:36.876271+0000","last_peered":"2024-07-04T17:34:36.876271+0000","last_clean":"2024-07-04T17:34:36.876271+0000","last_became_active":"2024-07-04T17:34:35.539252+0000","last_became_peered":"2024-07-04T17:34:35.539252+0000","last_unstale":"2024-07-04T17:34:36.876271+0000","last_undegraded":"2024-07-04T17:34:36.876271+0000","last_fullsized":"2024-07-04T17:34:36.876271+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:58:20.122138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.148332+0000","last_change":"2024-07-04T17:34:35.532311+0000","last_active":"2024-07-04T17:34:37.148332+0000","last_peered":"2024-07-04T17:34:37.148332+0000","last_clean":"2024-07-04T17:34:37.148332+0000","last_became_active":"2024-07-04T17:34:35.531704+0000","last_became_peered":"2024-07-04T17:34:35.531704+0000","last_unstale":"2024-07-04T17:34:37.148332+0000","last_undegraded":"2024-07-04T17:34:37.148332+0000","last_fullsized":"2024-07-04T17:34:37.148332+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:38:06.359176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876980+0000","last_change":"2024-07-04T17:34:35.536755+0000","last_active":"2024-07-04T17:34:36.876980+0000","last_peered":"2024-07-04T17:34:36.876980+0000","last_clean":"2024-07-04T17:34:36.876980+0000","last_became_active":"2024-07-04T17:34:35.535641+0000","last_became_peered":"2024-07-04T17:34:35.535641+0000","last_unstale":"2024-07-04T17:34:36.876980+0000","last_undegraded":"2024-07-04T17:34:36.876980+0000","last_fullsized":"2024-07-04T17:34:36.876980+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:00:03.434775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1f","version":"135'4","reported_seq":18,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.150606+0000","last_change":"2024-07-04T17:34:34.526194+0000","last_active":"2024-07-04T17:34:37.150606+0000","last_peered":"2024-07-04T17:34:37.150606+0000","last_clean":"2024-07-04T17:34:37.150606+0000","last_became_active":"2024-07-04T17:34:34.525749+0000","last_became_peered":"2024-07-04T17:34:34.525749+0000","last_unstale":"2024-07-04T17:34:37.150606+0000","last_undegraded":"2024-07-04T17:34:37.150606+0000","last_fullsized":"2024-07-04T17:34:37.150606+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:15:41.361322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.516393+0000","last_change":"2024-07-04T17:34:35.548372+0000","last_active":"2024-07-04T17:34:36.516393+0000","last_peered":"2024-07-04T17:34:36.516393+0000","last_clean":"2024-07-04T17:34:36.516393+0000","last_became_active":"2024-07-04T17:34:35.547829+0000","last_became_peered":"2024-07-04T17:34:35.547829+0000","last_unstale":"2024-07-04T17:34:36.516393+0000","last_undegraded":"2024-07-04T17:34:36.516393+0000","last_fullsized":"2024-07-04T17:34:36.516393+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:37:49.541089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"135'1","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.516376+0000","last_change":"2024-07-04T17:34:34.505687+0000","last_active":"2024-07-04T17:34:36.516376+0000","last_peered":"2024-07-04T17:34:36.516376+0000","last_clean":"2024-07-04T17:34:36.516376+0000","last_became_active":"2024-07-04T17:34:34.505337+0000","last_became_peered":"2024-07-04T17:34:34.505337+0000","last_unstale":"2024-07-04T17:34:36.516376+0000","last_undegraded":"2024-07-04T17:34:36.516376+0000","last_fullsized":"2024-07-04T17:34:36.516376+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:57:27.037624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876216+0000","last_change":"2024-07-04T17:34:35.531247+0000","last_active":"2024-07-04T17:34:36.876216+0000","last_peered":"2024-07-04T17:34:36.876216+0000","last_clean":"2024-07-04T17:34:36.876216+0000","last_became_active":"2024-07-04T17:34:35.530846+0000","last_became_peered":"2024-07-04T17:34:35.530846+0000","last_unstale":"2024-07-04T17:34:36.876216+0000","last_undegraded":"2024-07-04T17:34:36.876216+0000","last_fullsized":"2024-07-04T17:34:36.876216+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:30:51.180015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1d","version":"135'2","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.877257+0000","last_change":"2024-07-04T17:34:34.512001+0000","last_active":"2024-07-04T17:34:36.877257+0000","last_peered":"2024-07-04T17:34:36.877257+0000","last_clean":"2024-07-04T17:34:36.877257+0000","last_became_active":"2024-07-04T17:34:34.511480+0000","last_became_peered":"2024-07-04T17:34:34.511480+0000","last_unstale":"2024-07-04T17:34:36.877257+0000","last_undegraded":"2024-07-04T17:34:36.877257+0000","last_fullsized":"2024-07-04T17:34:36.877257+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:46:03.790681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.877115+0000","last_change":"2024-07-04T17:34:35.546422+0000","last_active":"2024-07-04T17:34:36.877115+0000","last_peered":"2024-07-04T17:34:36.877115+0000","last_clean":"2024-07-04T17:34:36.877115+0000","last_became_active":"2024-07-04T17:34:35.546061+0000","last_became_peered":"2024-07-04T17:34:35.546061+0000","last_unstale":"2024-07-04T17:34:36.877115+0000","last_undegraded":"2024-07-04T17:34:36.877115+0000","last_fullsized":"2024-07-04T17:34:36.877115+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:50:00.540847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1c","version":"135'2","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.515111+0000","last_change":"2024-07-04T17:34:34.506897+0000","last_active":"2024-07-04T17:34:36.515111+0000","last_peered":"2024-07-04T17:34:36.515111+0000","last_clean":"2024-07-04T17:34:36.515111+0000","last_became_active":"2024-07-04T17:34:34.506023+0000","last_became_peered":"2024-07-04T17:34:34.506023+0000","last_unstale":"2024-07-04T17:34:36.515111+0000","last_undegraded":"2024-07-04T17:34:36.515111+0000","last_fullsized":"2024-07-04T17:34:36.515111+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:36:57.170806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.514873+0000","last_change":"2024-07-04T17:34:35.515987+0000","last_active":"2024-07-04T17:34:36.514873+0000","last_peered":"2024-07-04T17:34:36.514873+0000","last_clean":"2024-07-04T17:34:36.514873+0000","last_became_active":"2024-07-04T17:34:35.515158+0000","last_became_peered":"2024-07-04T17:34:35.515158+0000","last_unstale":"2024-07-04T17:34:36.514873+0000","last_undegraded":"2024-07-04T17:34:36.514873+0000","last_fullsized":"2024-07-04T17:34:36.514873+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:39:44.537119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"135'1","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.151220+0000","last_change":"2024-07-04T17:34:34.526306+0000","last_active":"2024-07-04T17:34:37.151220+0000","last_peered":"2024-07-04T17:34:37.151220+0000","last_clean":"2024-07-04T17:34:37.151220+0000","last_became_active":"2024-07-04T17:34:34.525936+0000","last_became_peered":"2024-07-04T17:34:34.525936+0000","last_unstale":"2024-07-04T17:34:37.151220+0000","last_undegraded":"2024-07-04T17:34:37.151220+0000","last_fullsized":"2024-07-04T17:34:37.151220+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:58:19.109679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.150714+0000","last_change":"2024-07-04T17:34:35.529792+0000","last_active":"2024-07-04T17:34:37.150714+0000","last_peered":"2024-07-04T17:34:37.150714+0000","last_clean":"2024-07-04T17:34:37.150714+0000","last_became_active":"2024-07-04T17:34:35.529159+0000","last_became_peered":"2024-07-04T17:34:35.529159+0000","last_unstale":"2024-07-04T17:34:37.150714+0000","last_undegraded":"2024-07-04T17:34:37.150714+0000","last_fullsized":"2024-07-04T17:34:37.150714+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:19:05.515444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.1a","version":"135'1","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.149312+0000","last_change":"2024-07-04T17:34:34.520915+0000","last_active":"2024-07-04T17:34:37.149312+0000","last_peered":"2024-07-04T17:34:37.149312+0000","last_clean":"2024-07-04T17:34:37.149312+0000","last_became_active":"2024-07-04T17:34:34.520164+0000","last_became_peered":"2024-07-04T17:34:34.520164+0000","last_unstale":"2024-07-04T17:34:37.149312+0000","last_undegraded":"2024-07-04T17:34:37.149312+0000","last_fullsized":"2024-07-04T17:34:37.149312+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:15:07.166351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.148501+0000","last_change":"2024-07-04T17:34:35.528571+0000","last_active":"2024-07-04T17:34:37.148501+0000","last_peered":"2024-07-04T17:34:37.148501+0000","last_clean":"2024-07-04T17:34:37.148501+0000","last_became_active":"2024-07-04T17:34:35.528217+0000","last_became_peered":"2024-07-04T17:34:35.528217+0000","last_unstale":"2024-07-04T17:34:37.148501+0000","last_undegraded":"2024-07-04T17:34:37.148501+0000","last_fullsized":"2024-07-04T17:34:37.148501+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:25:31.311940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.148060+0000","last_change":"2024-07-04T17:34:34.520226+0000","last_active":"2024-07-04T17:34:37.148060+0000","last_peered":"2024-07-04T17:34:37.148060+0000","last_clean":"2024-07-04T17:34:37.148060+0000","last_became_active":"2024-07-04T17:34:34.519880+0000","last_became_peered":"2024-07-04T17:34:34.519880+0000","last_unstale":"2024-07-04T17:34:37.148060+0000","last_undegraded":"2024-07-04T17:34:37.148060+0000","last_fullsized":"2024-07-04T17:34:37.148060+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:30:01.808525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.877355+0000","last_change":"2024-07-04T17:34:35.541668+0000","last_active":"2024-07-04T17:34:36.877355+0000","last_peered":"2024-07-04T17:34:36.877355+0000","last_clean":"2024-07-04T17:34:36.877355+0000","last_became_active":"2024-07-04T17:34:35.537142+0000","last_became_peered":"2024-07-04T17:34:35.537142+0000","last_unstale":"2024-07-04T17:34:36.877355+0000","last_undegraded":"2024-07-04T17:34:36.877355+0000","last_fullsized":"2024-07-04T17:34:36.877355+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:56:41.649364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.877615+0000","last_change":"2024-07-04T17:34:34.516054+0000","last_active":"2024-07-04T17:34:36.877615+0000","last_peered":"2024-07-04T17:34:36.877615+0000","last_clean":"2024-07-04T17:34:36.877615+0000","last_became_active":"2024-07-04T17:34:34.514177+0000","last_became_peered":"2024-07-04T17:34:34.514177+0000","last_unstale":"2024-07-04T17:34:36.877615+0000","last_undegraded":"2024-07-04T17:34:36.877615+0000","last_fullsized":"2024-07-04T17:34:36.877615+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:18:46.789057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.877584+0000","last_change":"2024-07-04T17:34:35.546010+0000","last_active":"2024-07-04T17:34:36.877584+0000","last_peered":"2024-07-04T17:34:36.877584+0000","last_clean":"2024-07-04T17:34:36.877584+0000","last_became_active":"2024-07-04T17:34:35.536818+0000","last_became_peered":"2024-07-04T17:34:35.536818+0000","last_unstale":"2024-07-04T17:34:36.877584+0000","last_undegraded":"2024-07-04T17:34:36.877584+0000","last_fullsized":"2024-07-04T17:34:36.877584+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:24:20.145687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.149751+0000","last_change":"2024-07-04T17:34:34.526851+0000","last_active":"2024-07-04T17:34:37.149751+0000","last_peered":"2024-07-04T17:34:37.149751+0000","last_clean":"2024-07-04T17:34:37.149751+0000","last_became_active":"2024-07-04T17:34:34.526514+0000","last_became_peered":"2024-07-04T17:34:34.526514+0000","last_unstale":"2024-07-04T17:34:37.149751+0000","last_undegraded":"2024-07-04T17:34:37.149751+0000","last_fullsized":"2024-07-04T17:34:37.149751+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:37:45.915030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.515402+0000","last_change":"2024-07-04T17:34:35.524150+0000","last_active":"2024-07-04T17:34:36.515402+0000","last_peered":"2024-07-04T17:34:36.515402+0000","last_clean":"2024-07-04T17:34:36.515402+0000","last_became_active":"2024-07-04T17:34:35.523835+0000","last_became_peered":"2024-07-04T17:34:35.523835+0000","last_unstale":"2024-07-04T17:34:36.515402+0000","last_undegraded":"2024-07-04T17:34:36.515402+0000","last_fullsized":"2024-07-04T17:34:36.515402+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:14:55.953139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.15","version":"135'3","reported_seq":17,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.150842+0000","last_change":"2024-07-04T17:34:34.525906+0000","last_active":"2024-07-04T17:34:37.150842+0000","last_peered":"2024-07-04T17:34:37.150842+0000","last_clean":"2024-07-04T17:34:37.150842+0000","last_became_active":"2024-07-04T17:34:34.525545+0000","last_became_peered":"2024-07-04T17:34:34.525545+0000","last_unstale":"2024-07-04T17:34:37.150842+0000","last_undegraded":"2024-07-04T17:34:37.150842+0000","last_fullsized":"2024-07-04T17:34:37.150842+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:15:15.110008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.516890+0000","last_change":"2024-07-04T17:34:35.523356+0000","last_active":"2024-07-04T17:34:36.516890+0000","last_peered":"2024-07-04T17:34:36.516890+0000","last_clean":"2024-07-04T17:34:36.516890+0000","last_became_active":"2024-07-04T17:34:35.522584+0000","last_became_peered":"2024-07-04T17:34:35.522584+0000","last_unstale":"2024-07-04T17:34:36.516890+0000","last_undegraded":"2024-07-04T17:34:36.516890+0000","last_fullsized":"2024-07-04T17:34:36.516890+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:34:41.701054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"135'6","reported_seq":20,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876837+0000","last_change":"2024-07-04T17:34:34.513760+0000","last_active":"2024-07-04T17:34:36.876837+0000","last_peered":"2024-07-04T17:34:36.876837+0000","last_clean":"2024-07-04T17:34:36.876837+0000","last_became_active":"2024-07-04T17:34:34.512609+0000","last_became_peered":"2024-07-04T17:34:34.512609+0000","last_unstale":"2024-07-04T17:34:36.876837+0000","last_undegraded":"2024-07-04T17:34:36.876837+0000","last_fullsized":"2024-07-04T17:34:36.876837+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:08:12.565423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.875976+0000","last_change":"2024-07-04T17:34:35.537826+0000","last_active":"2024-07-04T17:34:36.875976+0000","last_peered":"2024-07-04T17:34:36.875976+0000","last_clean":"2024-07-04T17:34:36.875976+0000","last_became_active":"2024-07-04T17:34:35.535781+0000","last_became_peered":"2024-07-04T17:34:35.535781+0000","last_unstale":"2024-07-04T17:34:36.875976+0000","last_undegraded":"2024-07-04T17:34:36.875976+0000","last_fullsized":"2024-07-04T17:34:36.875976+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:26:50.917096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"135'2","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.514769+0000","last_change":"2024-07-04T17:34:34.507682+0000","last_active":"2024-07-04T17:34:36.514769+0000","last_peered":"2024-07-04T17:34:36.514769+0000","last_clean":"2024-07-04T17:34:36.514769+0000","last_became_active":"2024-07-04T17:34:34.507353+0000","last_became_peered":"2024-07-04T17:34:34.507353+0000","last_unstale":"2024-07-04T17:34:36.514769+0000","last_undegraded":"2024-07-04T17:34:36.514769+0000","last_fullsized":"2024-07-04T17:34:36.514769+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:47:18.975316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.151057+0000","last_change":"2024-07-04T17:34:35.529515+0000","last_active":"2024-07-04T17:34:37.151057+0000","last_peered":"2024-07-04T17:34:37.151057+0000","last_clean":"2024-07-04T17:34:37.151057+0000","last_became_active":"2024-07-04T17:34:35.528532+0000","last_became_peered":"2024-07-04T17:34:35.528532+0000","last_unstale":"2024-07-04T17:34:37.151057+0000","last_undegraded":"2024-07-04T17:34:37.151057+0000","last_fullsized":"2024-07-04T17:34:37.151057+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:33:17.769730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.515596+0000","last_change":"2024-07-04T17:34:34.512082+0000","last_active":"2024-07-04T17:34:36.515596+0000","last_peered":"2024-07-04T17:34:36.515596+0000","last_clean":"2024-07-04T17:34:36.515596+0000","last_became_active":"2024-07-04T17:34:34.511518+0000","last_became_peered":"2024-07-04T17:34:34.511518+0000","last_unstale":"2024-07-04T17:34:36.515596+0000","last_undegraded":"2024-07-04T17:34:36.515596+0000","last_fullsized":"2024-07-04T17:34:36.515596+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:34:32.553466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.513075+0000","last_change":"2024-07-04T17:34:35.539061+0000","last_active":"2024-07-04T17:34:36.513075+0000","last_peered":"2024-07-04T17:34:36.513075+0000","last_clean":"2024-07-04T17:34:36.513075+0000","last_became_active":"2024-07-04T17:34:35.538660+0000","last_became_peered":"2024-07-04T17:34:35.538660+0000","last_unstale":"2024-07-04T17:34:36.513075+0000","last_undegraded":"2024-07-04T17:34:36.513075+0000","last_fullsized":"2024-07-04T17:34:36.513075+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:36:20.868550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.513414+0000","last_change":"2024-07-04T17:34:34.513736+0000","last_active":"2024-07-04T17:34:36.513414+0000","last_peered":"2024-07-04T17:34:36.513414+0000","last_clean":"2024-07-04T17:34:36.513414+0000","last_became_active":"2024-07-04T17:34:34.512396+0000","last_became_peered":"2024-07-04T17:34:34.512396+0000","last_unstale":"2024-07-04T17:34:36.513414+0000","last_undegraded":"2024-07-04T17:34:36.513414+0000","last_fullsized":"2024-07-04T17:34:36.513414+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:25:30.776827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.513375+0000","last_change":"2024-07-04T17:34:35.540239+0000","last_active":"2024-07-04T17:34:36.513375+0000","last_peered":"2024-07-04T17:34:36.513375+0000","last_clean":"2024-07-04T17:34:36.513375+0000","last_became_active":"2024-07-04T17:34:35.539911+0000","last_became_peered":"2024-07-04T17:34:35.539911+0000","last_unstale":"2024-07-04T17:34:36.513375+0000","last_undegraded":"2024-07-04T17:34:36.513375+0000","last_fullsized":"2024-07-04T17:34:36.513375+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:03:47.021324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"135'8","reported_seq":22,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.148314+0000","last_change":"2024-07-04T17:34:34.526204+0000","last_active":"2024-07-04T17:34:37.148314+0000","last_peered":"2024-07-04T17:34:37.148314+0000","last_clean":"2024-07-04T17:34:37.148314+0000","last_became_active":"2024-07-04T17:34:34.525811+0000","last_became_peered":"2024-07-04T17:34:34.525811+0000","last_unstale":"2024-07-04T17:34:37.148314+0000","last_undegraded":"2024-07-04T17:34:37.148314+0000","last_fullsized":"2024-07-04T17:34:37.148314+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:49:36.910071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.516594+0000","last_change":"2024-07-04T17:34:35.530930+0000","last_active":"2024-07-04T17:34:36.516594+0000","last_peered":"2024-07-04T17:34:36.516594+0000","last_clean":"2024-07-04T17:34:36.516594+0000","last_became_active":"2024-07-04T17:34:35.527474+0000","last_became_peered":"2024-07-04T17:34:35.527474+0000","last_unstale":"2024-07-04T17:34:36.516594+0000","last_undegraded":"2024-07-04T17:34:36.516594+0000","last_fullsized":"2024-07-04T17:34:36.516594+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:51:21.803840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"135'1","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.150859+0000","last_change":"2024-07-04T17:34:34.526961+0000","last_active":"2024-07-04T17:34:37.150859+0000","last_peered":"2024-07-04T17:34:37.150859+0000","last_clean":"2024-07-04T17:34:37.150859+0000","last_became_active":"2024-07-04T17:34:34.526338+0000","last_became_peered":"2024-07-04T17:34:34.526338+0000","last_unstale":"2024-07-04T17:34:37.150859+0000","last_undegraded":"2024-07-04T17:34:37.150859+0000","last_fullsized":"2024-07-04T17:34:37.150859+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:46:36.828153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.516882+0000","last_change":"2024-07-04T17:34:35.546152+0000","last_active":"2024-07-04T17:34:36.516882+0000","last_peered":"2024-07-04T17:34:36.516882+0000","last_clean":"2024-07-04T17:34:36.516882+0000","last_became_active":"2024-07-04T17:34:35.545573+0000","last_became_peered":"2024-07-04T17:34:35.545573+0000","last_unstale":"2024-07-04T17:34:36.516882+0000","last_undegraded":"2024-07-04T17:34:36.516882+0000","last_fullsized":"2024-07-04T17:34:36.516882+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:20:21.078798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.e","version":"135'2","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.877128+0000","last_change":"2024-07-04T17:34:34.516533+0000","last_active":"2024-07-04T17:34:36.877128+0000","last_peered":"2024-07-04T17:34:36.877128+0000","last_clean":"2024-07-04T17:34:36.877128+0000","last_became_active":"2024-07-04T17:34:34.515492+0000","last_became_peered":"2024-07-04T17:34:34.515492+0000","last_unstale":"2024-07-04T17:34:36.877128+0000","last_undegraded":"2024-07-04T17:34:36.877128+0000","last_fullsized":"2024-07-04T17:34:36.877128+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:06:02.654676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876587+0000","last_change":"2024-07-04T17:34:35.544212+0000","last_active":"2024-07-04T17:34:36.876587+0000","last_peered":"2024-07-04T17:34:36.876587+0000","last_clean":"2024-07-04T17:34:36.876587+0000","last_became_active":"2024-07-04T17:34:35.543570+0000","last_became_peered":"2024-07-04T17:34:35.543570+0000","last_unstale":"2024-07-04T17:34:36.876587+0000","last_undegraded":"2024-07-04T17:34:36.876587+0000","last_fullsized":"2024-07-04T17:34:36.876587+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:38:02.545198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.d","version":"135'1","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.517109+0000","last_change":"2024-07-04T17:34:34.514159+0000","last_active":"2024-07-04T17:34:36.517109+0000","last_peered":"2024-07-04T17:34:36.517109+0000","last_clean":"2024-07-04T17:34:36.517109+0000","last_became_active":"2024-07-04T17:34:34.511974+0000","last_became_peered":"2024-07-04T17:34:34.511974+0000","last_unstale":"2024-07-04T17:34:36.517109+0000","last_undegraded":"2024-07-04T17:34:36.517109+0000","last_fullsized":"2024-07-04T17:34:36.517109+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:20:11.531736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.517145+0000","last_change":"2024-07-04T17:34:35.523503+0000","last_active":"2024-07-04T17:34:36.517145+0000","last_peered":"2024-07-04T17:34:36.517145+0000","last_clean":"2024-07-04T17:34:36.517145+0000","last_became_active":"2024-07-04T17:34:35.522995+0000","last_became_peered":"2024-07-04T17:34:35.522995+0000","last_unstale":"2024-07-04T17:34:36.517145+0000","last_undegraded":"2024-07-04T17:34:36.517145+0000","last_fullsized":"2024-07-04T17:34:36.517145+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:24:36.746418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.515481+0000","last_change":"2024-07-04T17:34:34.508754+0000","last_active":"2024-07-04T17:34:36.515481+0000","last_peered":"2024-07-04T17:34:36.515481+0000","last_clean":"2024-07-04T17:34:36.515481+0000","last_became_active":"2024-07-04T17:34:34.507614+0000","last_became_peered":"2024-07-04T17:34:34.507614+0000","last_unstale":"2024-07-04T17:34:36.515481+0000","last_undegraded":"2024-07-04T17:34:36.515481+0000","last_fullsized":"2024-07-04T17:34:36.515481+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:09:44.865693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.515411+0000","last_change":"2024-07-04T17:34:35.515740+0000","last_active":"2024-07-04T17:34:36.515411+0000","last_peered":"2024-07-04T17:34:36.515411+0000","last_clean":"2024-07-04T17:34:36.515411+0000","last_became_active":"2024-07-04T17:34:35.514774+0000","last_became_peered":"2024-07-04T17:34:35.514774+0000","last_unstale":"2024-07-04T17:34:36.515411+0000","last_undegraded":"2024-07-04T17:34:36.515411+0000","last_fullsized":"2024-07-04T17:34:36.515411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:54:35.373423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"135'2","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876412+0000","last_change":"2024-07-04T17:34:34.515650+0000","last_active":"2024-07-04T17:34:36.876412+0000","last_peered":"2024-07-04T17:34:36.876412+0000","last_clean":"2024-07-04T17:34:36.876412+0000","last_became_active":"2024-07-04T17:34:34.514347+0000","last_became_peered":"2024-07-04T17:34:34.514347+0000","last_unstale":"2024-07-04T17:34:36.876412+0000","last_undegraded":"2024-07-04T17:34:36.876412+0000","last_fullsized":"2024-07-04T17:34:36.876412+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:34:16.111397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.150994+0000","last_change":"2024-07-04T17:34:35.529352+0000","last_active":"2024-07-04T17:34:37.150994+0000","last_peered":"2024-07-04T17:34:37.150994+0000","last_clean":"2024-07-04T17:34:37.150994+0000","last_became_active":"2024-07-04T17:34:35.528095+0000","last_became_peered":"2024-07-04T17:34:35.528095+0000","last_unstale":"2024-07-04T17:34:37.150994+0000","last_undegraded":"2024-07-04T17:34:37.150994+0000","last_fullsized":"2024-07-04T17:34:37.150994+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:53:30.230859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.149721+0000","last_change":"2024-07-04T17:34:34.520687+0000","last_active":"2024-07-04T17:34:37.149721+0000","last_peered":"2024-07-04T17:34:37.149721+0000","last_clean":"2024-07-04T17:34:37.149721+0000","last_became_active":"2024-07-04T17:34:34.520328+0000","last_became_peered":"2024-07-04T17:34:34.520328+0000","last_unstale":"2024-07-04T17:34:37.149721+0000","last_undegraded":"2024-07-04T17:34:37.149721+0000","last_fullsized":"2024-07-04T17:34:37.149721+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:26:49.904637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.513134+0000","last_change":"2024-07-04T17:34:35.525251+0000","last_active":"2024-07-04T17:34:36.513134+0000","last_peered":"2024-07-04T17:34:36.513134+0000","last_clean":"2024-07-04T17:34:36.513134+0000","last_became_active":"2024-07-04T17:34:35.524606+0000","last_became_peered":"2024-07-04T17:34:35.524606+0000","last_unstale":"2024-07-04T17:34:36.513134+0000","last_undegraded":"2024-07-04T17:34:36.513134+0000","last_fullsized":"2024-07-04T17:34:36.513134+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:18:18.311591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876069+0000","last_change":"2024-07-04T17:34:34.515246+0000","last_active":"2024-07-04T17:34:36.876069+0000","last_peered":"2024-07-04T17:34:36.876069+0000","last_clean":"2024-07-04T17:34:36.876069+0000","last_became_active":"2024-07-04T17:34:34.514888+0000","last_became_peered":"2024-07-04T17:34:34.514888+0000","last_unstale":"2024-07-04T17:34:36.876069+0000","last_undegraded":"2024-07-04T17:34:36.876069+0000","last_fullsized":"2024-07-04T17:34:36.876069+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:35:07.644625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.148882+0000","last_change":"2024-07-04T17:34:35.541160+0000","last_active":"2024-07-04T17:34:37.148882+0000","last_peered":"2024-07-04T17:34:37.148882+0000","last_clean":"2024-07-04T17:34:37.148882+0000","last_became_active":"2024-07-04T17:34:35.540869+0000","last_became_peered":"2024-07-04T17:34:35.540869+0000","last_unstale":"2024-07-04T17:34:37.148882+0000","last_undegraded":"2024-07-04T17:34:37.148882+0000","last_fullsized":"2024-07-04T17:34:37.148882+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:41:18.824553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.148452+0000","last_change":"2024-07-04T17:34:34.521170+0000","last_active":"2024-07-04T17:34:37.148452+0000","last_peered":"2024-07-04T17:34:37.148452+0000","last_clean":"2024-07-04T17:34:37.148452+0000","last_became_active":"2024-07-04T17:34:34.520731+0000","last_became_peered":"2024-07-04T17:34:34.520731+0000","last_unstale":"2024-07-04T17:34:37.148452+0000","last_undegraded":"2024-07-04T17:34:37.148452+0000","last_fullsized":"2024-07-04T17:34:37.148452+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:30:50.167556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.149653+0000","last_change":"2024-07-04T17:34:35.534458+0000","last_active":"2024-07-04T17:34:37.149653+0000","last_peered":"2024-07-04T17:34:37.149653+0000","last_clean":"2024-07-04T17:34:37.149653+0000","last_became_active":"2024-07-04T17:34:35.534123+0000","last_became_peered":"2024-07-04T17:34:35.534123+0000","last_unstale":"2024-07-04T17:34:37.149653+0000","last_undegraded":"2024-07-04T17:34:37.149653+0000","last_fullsized":"2024-07-04T17:34:37.149653+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:55:02.890933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"135'2","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.517059+0000","last_change":"2024-07-04T17:34:34.516580+0000","last_active":"2024-07-04T17:34:36.517059+0000","last_peered":"2024-07-04T17:34:36.517059+0000","last_clean":"2024-07-04T17:34:36.517059+0000","last_became_active":"2024-07-04T17:34:34.513629+0000","last_became_peered":"2024-07-04T17:34:34.513629+0000","last_unstale":"2024-07-04T17:34:36.517059+0000","last_undegraded":"2024-07-04T17:34:36.517059+0000","last_fullsized":"2024-07-04T17:34:36.517059+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:31:26.410567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.517014+0000","last_change":"2024-07-04T17:34:35.545883+0000","last_active":"2024-07-04T17:34:36.517014+0000","last_peered":"2024-07-04T17:34:36.517014+0000","last_clean":"2024-07-04T17:34:36.517014+0000","last_became_active":"2024-07-04T17:34:35.545154+0000","last_became_peered":"2024-07-04T17:34:35.545154+0000","last_unstale":"2024-07-04T17:34:36.517014+0000","last_undegraded":"2024-07-04T17:34:36.517014+0000","last_fullsized":"2024-07-04T17:34:36.517014+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:36:58.592291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"135'1","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.516726+0000","last_change":"2024-07-04T17:34:34.505851+0000","last_active":"2024-07-04T17:34:36.516726+0000","last_peered":"2024-07-04T17:34:36.516726+0000","last_clean":"2024-07-04T17:34:36.516726+0000","last_became_active":"2024-07-04T17:34:34.505548+0000","last_became_peered":"2024-07-04T17:34:34.505548+0000","last_unstale":"2024-07-04T17:34:36.516726+0000","last_undegraded":"2024-07-04T17:34:36.516726+0000","last_fullsized":"2024-07-04T17:34:36.516726+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:17:04.987342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876683+0000","last_change":"2024-07-04T17:34:35.543872+0000","last_active":"2024-07-04T17:34:36.876683+0000","last_peered":"2024-07-04T17:34:36.876683+0000","last_clean":"2024-07-04T17:34:36.876683+0000","last_became_active":"2024-07-04T17:34:35.542863+0000","last_became_peered":"2024-07-04T17:34:35.542863+0000","last_unstale":"2024-07-04T17:34:36.876683+0000","last_undegraded":"2024-07-04T17:34:36.876683+0000","last_fullsized":"2024-07-04T17:34:36.876683+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:28:18.564334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.517238+0000","last_change":"2024-07-04T17:34:34.505554+0000","last_active":"2024-07-04T17:34:36.517238+0000","last_peered":"2024-07-04T17:34:36.517238+0000","last_clean":"2024-07-04T17:34:36.517238+0000","last_became_active":"2024-07-04T17:34:34.505151+0000","last_became_peered":"2024-07-04T17:34:34.505151+0000","last_unstale":"2024-07-04T17:34:36.517238+0000","last_undegraded":"2024-07-04T17:34:36.517238+0000","last_fullsized":"2024-07-04T17:34:36.517238+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:17:53.346868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.877230+0000","last_change":"2024-07-04T17:34:35.543911+0000","last_active":"2024-07-04T17:34:36.877230+0000","last_peered":"2024-07-04T17:34:36.877230+0000","last_clean":"2024-07-04T17:34:36.877230+0000","last_became_active":"2024-07-04T17:34:35.533364+0000","last_became_peered":"2024-07-04T17:34:35.533364+0000","last_unstale":"2024-07-04T17:34:36.877230+0000","last_undegraded":"2024-07-04T17:34:36.877230+0000","last_fullsized":"2024-07-04T17:34:36.877230+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:13:03.582706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.515601+0000","last_change":"2024-07-04T17:34:34.507394+0000","last_active":"2024-07-04T17:34:36.515601+0000","last_peered":"2024-07-04T17:34:36.515601+0000","last_clean":"2024-07-04T17:34:36.515601+0000","last_became_active":"2024-07-04T17:34:34.506560+0000","last_became_peered":"2024-07-04T17:34:34.506560+0000","last_unstale":"2024-07-04T17:34:36.515601+0000","last_undegraded":"2024-07-04T17:34:36.515601+0000","last_fullsized":"2024-07-04T17:34:36.515601+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:12:30.641841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876028+0000","last_change":"2024-07-04T17:34:35.543760+0000","last_active":"2024-07-04T17:34:36.876028+0000","last_peered":"2024-07-04T17:34:36.876028+0000","last_clean":"2024-07-04T17:34:36.876028+0000","last_became_active":"2024-07-04T17:34:35.533337+0000","last_became_peered":"2024-07-04T17:34:35.533337+0000","last_unstale":"2024-07-04T17:34:36.876028+0000","last_undegraded":"2024-07-04T17:34:36.876028+0000","last_fullsized":"2024-07-04T17:34:36.876028+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:15:16.122467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876366+0000","last_change":"2024-07-04T17:34:34.513417+0000","last_active":"2024-07-04T17:34:36.876366+0000","last_peered":"2024-07-04T17:34:36.876366+0000","last_clean":"2024-07-04T17:34:36.876366+0000","last_became_active":"2024-07-04T17:34:34.513092+0000","last_became_peered":"2024-07-04T17:34:34.513092+0000","last_unstale":"2024-07-04T17:34:36.876366+0000","last_undegraded":"2024-07-04T17:34:36.876366+0000","last_fullsized":"2024-07-04T17:34:36.876366+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:04:50.268887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.151142+0000","last_change":"2024-07-04T17:34:35.521764+0000","last_active":"2024-07-04T17:34:37.151142+0000","last_peered":"2024-07-04T17:34:37.151142+0000","last_clean":"2024-07-04T17:34:37.151142+0000","last_became_active":"2024-07-04T17:34:35.521455+0000","last_became_peered":"2024-07-04T17:34:35.521455+0000","last_unstale":"2024-07-04T17:34:37.151142+0000","last_undegraded":"2024-07-04T17:34:37.151142+0000","last_fullsized":"2024-07-04T17:34:37.151142+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:54:12.411851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876108+0000","last_change":"2024-07-04T17:34:34.518708+0000","last_active":"2024-07-04T17:34:36.876108+0000","last_peered":"2024-07-04T17:34:36.876108+0000","last_clean":"2024-07-04T17:34:36.876108+0000","last_became_active":"2024-07-04T17:34:34.518368+0000","last_became_peered":"2024-07-04T17:34:34.518368+0000","last_unstale":"2024-07-04T17:34:36.876108+0000","last_undegraded":"2024-07-04T17:34:36.876108+0000","last_fullsized":"2024-07-04T17:34:36.876108+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:58:34.507931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.875956+0000","last_change":"2024-07-04T17:34:35.520630+0000","last_active":"2024-07-04T17:34:36.875956+0000","last_peered":"2024-07-04T17:34:36.875956+0000","last_clean":"2024-07-04T17:34:36.875956+0000","last_became_active":"2024-07-04T17:34:35.520357+0000","last_became_peered":"2024-07-04T17:34:35.520357+0000","last_unstale":"2024-07-04T17:34:36.875956+0000","last_undegraded":"2024-07-04T17:34:36.875956+0000","last_fullsized":"2024-07-04T17:34:36.875956+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:46:37.840612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876189+0000","last_change":"2024-07-04T17:34:34.518104+0000","last_active":"2024-07-04T17:34:36.876189+0000","last_peered":"2024-07-04T17:34:36.876189+0000","last_clean":"2024-07-04T17:34:36.876189+0000","last_became_active":"2024-07-04T17:34:34.513906+0000","last_became_peered":"2024-07-04T17:34:34.513906+0000","last_unstale":"2024-07-04T17:34:36.876189+0000","last_undegraded":"2024-07-04T17:34:36.876189+0000","last_fullsized":"2024-07-04T17:34:36.876189+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:43:24.362777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.149307+0000","last_change":"2024-07-04T17:34:35.538841+0000","last_active":"2024-07-04T17:34:37.149307+0000","last_peered":"2024-07-04T17:34:37.149307+0000","last_clean":"2024-07-04T17:34:37.149307+0000","last_became_active":"2024-07-04T17:34:35.538551+0000","last_became_peered":"2024-07-04T17:34:35.538551+0000","last_unstale":"2024-07-04T17:34:37.149307+0000","last_undegraded":"2024-07-04T17:34:37.149307+0000","last_fullsized":"2024-07-04T17:34:37.149307+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:02:46.435969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876055+0000","last_change":"2024-07-04T17:34:34.514509+0000","last_active":"2024-07-04T17:34:36.876055+0000","last_peered":"2024-07-04T17:34:36.876055+0000","last_clean":"2024-07-04T17:34:36.876055+0000","last_became_active":"2024-07-04T17:34:34.511266+0000","last_became_peered":"2024-07-04T17:34:34.511266+0000","last_unstale":"2024-07-04T17:34:36.876055+0000","last_undegraded":"2024-07-04T17:34:36.876055+0000","last_fullsized":"2024-07-04T17:34:36.876055+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:18:52.279136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.513594+0000","last_change":"2024-07-04T17:34:35.515886+0000","last_active":"2024-07-04T17:34:36.513594+0000","last_peered":"2024-07-04T17:34:36.513594+0000","last_clean":"2024-07-04T17:34:36.513594+0000","last_became_active":"2024-07-04T17:34:35.515383+0000","last_became_peered":"2024-07-04T17:34:35.515383+0000","last_unstale":"2024-07-04T17:34:36.513594+0000","last_undegraded":"2024-07-04T17:34:36.513594+0000","last_fullsized":"2024-07-04T17:34:36.513594+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:04:52.389960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.513082+0000","last_change":"2024-07-04T17:34:34.515873+0000","last_active":"2024-07-04T17:34:36.513082+0000","last_peered":"2024-07-04T17:34:36.513082+0000","last_clean":"2024-07-04T17:34:36.513082+0000","last_became_active":"2024-07-04T17:34:34.515559+0000","last_became_peered":"2024-07-04T17:34:34.515559+0000","last_unstale":"2024-07-04T17:34:36.513082+0000","last_undegraded":"2024-07-04T17:34:36.513082+0000","last_fullsized":"2024-07-04T17:34:36.513082+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:21:52.711984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"122'115","reported_seq":541,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876189+0000","last_change":"2024-07-04T17:26:04.953559+0000","last_active":"2024-07-04T17:34:36.876189+0000","last_peered":"2024-07-04T17:34:36.876189+0000","last_clean":"2024-07-04T17:34:36.876189+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:34:36.876189+0000","last_undegraded":"2024-07-04T17:34:36.876189+0000","last_fullsized":"2024-07-04T17:34:36.876189+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_clean_scrub_stamp":"2024-07-04T17:26:03.933432+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.064181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1630,"num_write":187,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1630,"num_write":187,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738469,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36724,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335116,"statfs":{"total":96636764160,"available":96599158784,"internally_reserved":0,"allocated":1417216,"data_stored":560633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738470,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37308,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334532,"statfs":{"total":96636764160,"available":96598560768,"internally_reserved":0,"allocated":2011136,"data_stored":1151883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738468,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37324,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334516,"statfs":{"total":96636764160,"available":96598544384,"internally_reserved":0,"allocated":2027520,"data_stored":1152647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:34:40.230 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:34:40.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.567+0000 7f2ab9400640 1 -- 172.21.15.42:0/3056307893 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ab406edd0 msgr2=0x7f2ab40b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:40.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.567+0000 7f2ab9400640 1 --2- 172.21.15.42:0/3056307893 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ab406edd0 0x7f2ab40b3030 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f2aa40097b0 tx=0x7f2aa402a540 comp rx=0 tx=0).stop 2024-07-04T17:34:40.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.567+0000 7f2ab9400640 1 -- 172.21.15.42:0/3056307893 shutdown_connections 2024-07-04T17:34:40.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.567+0000 7f2ab9400640 1 --2- 172.21.15.42:0/3056307893 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ab406edd0 0x7f2ab40b3030 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:40.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.567+0000 7f2ab9400640 1 -- 172.21.15.42:0/3056307893 >> 172.21.15.42:0/3056307893 conn(0x7f2ab40bf700 msgr2=0x7f2ab40af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:40.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.567+0000 7f2ab9400640 1 -- 172.21.15.42:0/3056307893 shutdown_connections 2024-07-04T17:34:40.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.567+0000 7f2ab9400640 1 -- 172.21.15.42:0/3056307893 wait complete. 2024-07-04T17:34:40.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.569+0000 7f2ab9400640 1 Processor -- start 2024-07-04T17:34:40.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.569+0000 7f2ab9400640 1 -- start start 2024-07-04T17:34:40.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.571+0000 7f2ab9400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ab406cea0 0x7f2ab41d4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:40.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.571+0000 7f2ab9400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab40b0580 con 0x7f2ab406cea0 2024-07-04T17:34:40.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.571+0000 7f2ab2800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ab406cea0 0x7f2ab41d4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:40.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.571+0000 7f2ab2800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ab406cea0 0x7f2ab41d4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48348/0 (socket says 172.21.15.42:48348) 2024-07-04T17:34:40.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.571+0000 7f2ab2800640 1 -- 172.21.15.42:0/3664203017 learned_addr learned my addr 172.21.15.42:0/3664203017 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:40.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.571+0000 7f2ab2800640 1 -- 172.21.15.42:0/3664203017 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab4146aa0 con 0x7f2ab406cea0 2024-07-04T17:34:40.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.572+0000 7f2ab2800640 1 --2- 172.21.15.42:0/3664203017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ab406cea0 0x7f2ab41d4350 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f2aa40098d0 tx=0x7f2aa4033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:40.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.572+0000 7f2aaaa00640 1 -- 172.21.15.42:0/3664203017 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2aa4031bc0 con 0x7f2ab406cea0 2024-07-04T17:34:40.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.572+0000 7f2aaaa00640 1 -- 172.21.15.42:0/3664203017 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2aa4031e50 con 0x7f2ab406cea0 2024-07-04T17:34:40.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.572+0000 7f2ab9400640 1 -- 172.21.15.42:0/3664203017 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ab41d6f50 con 0x7f2ab406cea0 2024-07-04T17:34:40.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.572+0000 7f2aaaa00640 1 -- 172.21.15.42:0/3664203017 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2aa4033260 con 0x7f2ab406cea0 2024-07-04T17:34:40.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.572+0000 7f2ab9400640 1 -- 172.21.15.42:0/3664203017 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ab41d6c40 con 0x7f2ab406cea0 2024-07-04T17:34:40.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.574+0000 7f2ab9400640 1 -- 172.21.15.42:0/3664203017 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ab40b3030 con 0x7f2ab406cea0 2024-07-04T17:34:40.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.574+0000 7f2aaaa00640 1 -- 172.21.15.42:0/3664203017 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f2aa4033400 con 0x7f2ab406cea0 2024-07-04T17:34:40.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.574+0000 7f2aaaa00640 1 --2- 172.21.15.42:0/3664203017 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f2a9403d4a0 0x7f2a9403f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:40.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.574+0000 7f2aaaa00640 1 -- 172.21.15.42:0/3664203017 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 3439+0+0 (secure 0 0 0) 0x7f2aa4072e60 con 0x7f2ab406cea0 2024-07-04T17:34:40.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.575+0000 7f2ab1e00640 1 --2- 172.21.15.42:0/3664203017 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f2a9403d4a0 0x7f2a9403f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:40.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.576+0000 7f2ab1e00640 1 --2- 172.21.15.42:0/3664203017 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f2a9403d4a0 0x7f2a9403f9b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0002040 tx=0x7f2aa0001650 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:40.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.580+0000 7f2aaaa00640 1 -- 172.21.15.42:0/3664203017 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f2aa403f030 con 0x7f2ab406cea0 2024-07-04T17:34:40.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.785+0000 7f2ab9400640 1 -- 172.21.15.42:0/3664203017 --> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2ab41d4cc0 con 0x7f2a9403d4a0 2024-07-04T17:34:40.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.798+0000 7f2aaaa00640 1 -- 172.21.15.42:0/3664203017 <== mgr.5692 v2:172.21.15.42:6834/4064094064 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246494 (secure 0 0 0) 0x7f2ab41d4cc0 con 0x7f2a9403d4a0 2024-07-04T17:34:40.801 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:34:40.802 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:34:40.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.802+0000 7f2ab9400640 1 -- 172.21.15.42:0/3664203017 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f2a9403d4a0 msgr2=0x7f2a9403f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:40.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.802+0000 7f2ab9400640 1 --2- 172.21.15.42:0/3664203017 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f2a9403d4a0 0x7f2a9403f9b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0002040 tx=0x7f2aa0001650 comp rx=0 tx=0).stop 2024-07-04T17:34:40.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.802+0000 7f2ab9400640 1 -- 172.21.15.42:0/3664203017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ab406cea0 msgr2=0x7f2ab41d4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:40.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.802+0000 7f2ab9400640 1 --2- 172.21.15.42:0/3664203017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ab406cea0 0x7f2ab41d4350 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f2aa40098d0 tx=0x7f2aa4033e40 comp rx=0 tx=0).stop 2024-07-04T17:34:40.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.803+0000 7f2ab9400640 1 -- 172.21.15.42:0/3664203017 shutdown_connections 2024-07-04T17:34:40.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.803+0000 7f2ab9400640 1 --2- 172.21.15.42:0/3664203017 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f2a9403d4a0 0x7f2a9403f9b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:40.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.803+0000 7f2ab9400640 1 --2- 172.21.15.42:0/3664203017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ab406cea0 0x7f2ab41d4350 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:40.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.803+0000 7f2ab9400640 1 -- 172.21.15.42:0/3664203017 >> 172.21.15.42:0/3664203017 conn(0x7f2ab40bf700 msgr2=0x7f2ab40bb740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:40.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.803+0000 7f2ab9400640 1 -- 172.21.15.42:0/3664203017 shutdown_connections 2024-07-04T17:34:40.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:40.803+0000 7f2ab9400640 1 -- 172.21.15.42:0/3664203017 wait complete. 2024-07-04T17:34:40.831 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-07-04T17:34:39.108759+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":596,"num_read_kb":1630,"num_write":232,"num_write_kb":1120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":111356,"kb_used_data":5328,"kb_used_omap":4,"kb_used_meta":105979,"kb_avail":283004164,"statfs":{"total":289910292480,"available":289796263936,"internally_reserved":0,"allocated":5455872,"data_stored":2865163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":108522831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.645211"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.877030+0000","last_change":"2024-07-04T17:34:35.545191+0000","last_active":"2024-07-04T17:34:36.877030+0000","last_peered":"2024-07-04T17:34:36.877030+0000","last_clean":"2024-07-04T17:34:36.877030+0000","last_became_active":"2024-07-04T17:34:35.542859+0000","last_became_peered":"2024-07-04T17:34:35.542859+0000","last_unstale":"2024-07-04T17:34:36.877030+0000","last_undegraded":"2024-07-04T17:34:36.877030+0000","last_fullsized":"2024-07-04T17:34:36.877030+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:36:40.642306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876779+0000","last_change":"2024-07-04T17:34:35.541970+0000","last_active":"2024-07-04T17:34:36.876779+0000","last_peered":"2024-07-04T17:34:36.876779+0000","last_clean":"2024-07-04T17:34:36.876779+0000","last_became_active":"2024-07-04T17:34:35.541341+0000","last_became_peered":"2024-07-04T17:34:35.541341+0000","last_unstale":"2024-07-04T17:34:36.876779+0000","last_undegraded":"2024-07-04T17:34:36.876779+0000","last_fullsized":"2024-07-04T17:34:36.876779+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:06:30.239920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876729+0000","last_change":"2024-07-04T17:34:35.531108+0000","last_active":"2024-07-04T17:34:36.876729+0000","last_peered":"2024-07-04T17:34:36.876729+0000","last_clean":"2024-07-04T17:34:36.876729+0000","last_became_active":"2024-07-04T17:34:35.530755+0000","last_became_peered":"2024-07-04T17:34:35.530755+0000","last_unstale":"2024-07-04T17:34:36.876729+0000","last_undegraded":"2024-07-04T17:34:36.876729+0000","last_fullsized":"2024-07-04T17:34:36.876729+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:48:46.716399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876393+0000","last_change":"2024-07-04T17:34:35.534705+0000","last_active":"2024-07-04T17:34:36.876393+0000","last_peered":"2024-07-04T17:34:36.876393+0000","last_clean":"2024-07-04T17:34:36.876393+0000","last_became_active":"2024-07-04T17:34:35.532134+0000","last_became_peered":"2024-07-04T17:34:35.532134+0000","last_unstale":"2024-07-04T17:34:36.876393+0000","last_undegraded":"2024-07-04T17:34:36.876393+0000","last_fullsized":"2024-07-04T17:34:36.876393+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:04:01.056622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.149527+0000","last_change":"2024-07-04T17:34:35.529627+0000","last_active":"2024-07-04T17:34:37.149527+0000","last_peered":"2024-07-04T17:34:37.149527+0000","last_clean":"2024-07-04T17:34:37.149527+0000","last_became_active":"2024-07-04T17:34:35.528998+0000","last_became_peered":"2024-07-04T17:34:35.528998+0000","last_unstale":"2024-07-04T17:34:37.149527+0000","last_undegraded":"2024-07-04T17:34:37.149527+0000","last_fullsized":"2024-07-04T17:34:37.149527+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:53:22.203053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.875783+0000","last_change":"2024-07-04T17:34:35.517987+0000","last_active":"2024-07-04T17:34:36.875783+0000","last_peered":"2024-07-04T17:34:36.875783+0000","last_clean":"2024-07-04T17:34:36.875783+0000","last_became_active":"2024-07-04T17:34:35.517658+0000","last_became_peered":"2024-07-04T17:34:35.517658+0000","last_unstale":"2024-07-04T17:34:36.875783+0000","last_undegraded":"2024-07-04T17:34:36.875783+0000","last_fullsized":"2024-07-04T17:34:36.875783+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:49:37.922530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.148317+0000","last_change":"2024-07-04T17:34:35.555625+0000","last_active":"2024-07-04T17:34:37.148317+0000","last_peered":"2024-07-04T17:34:37.148317+0000","last_clean":"2024-07-04T17:34:37.148317+0000","last_became_active":"2024-07-04T17:34:35.554652+0000","last_became_peered":"2024-07-04T17:34:35.554652+0000","last_unstale":"2024-07-04T17:34:37.148317+0000","last_undegraded":"2024-07-04T17:34:37.148317+0000","last_fullsized":"2024-07-04T17:34:37.148317+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:47:42.975532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.875749+0000","last_change":"2024-07-04T17:34:35.543396+0000","last_active":"2024-07-04T17:34:36.875749+0000","last_peered":"2024-07-04T17:34:36.875749+0000","last_clean":"2024-07-04T17:34:36.875749+0000","last_became_active":"2024-07-04T17:34:35.542958+0000","last_became_peered":"2024-07-04T17:34:35.542958+0000","last_unstale":"2024-07-04T17:34:36.875749+0000","last_undegraded":"2024-07-04T17:34:36.875749+0000","last_fullsized":"2024-07-04T17:34:36.875749+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:37:46.927489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.877516+0000","last_change":"2024-07-04T17:34:35.543199+0000","last_active":"2024-07-04T17:34:36.877516+0000","last_peered":"2024-07-04T17:34:36.877516+0000","last_clean":"2024-07-04T17:34:36.877516+0000","last_became_active":"2024-07-04T17:34:35.537240+0000","last_became_peered":"2024-07-04T17:34:35.537240+0000","last_unstale":"2024-07-04T17:34:36.877516+0000","last_undegraded":"2024-07-04T17:34:36.877516+0000","last_fullsized":"2024-07-04T17:34:36.877516+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:17:07.663136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.516071+0000","last_change":"2024-07-04T17:34:35.552696+0000","last_active":"2024-07-04T17:34:36.516071+0000","last_peered":"2024-07-04T17:34:36.516071+0000","last_clean":"2024-07-04T17:34:36.516071+0000","last_became_active":"2024-07-04T17:34:35.552377+0000","last_became_peered":"2024-07-04T17:34:35.552377+0000","last_unstale":"2024-07-04T17:34:36.516071+0000","last_undegraded":"2024-07-04T17:34:36.516071+0000","last_fullsized":"2024-07-04T17:34:36.516071+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:54:28.842946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.150879+0000","last_change":"2024-07-04T17:34:35.535880+0000","last_active":"2024-07-04T17:34:37.150879+0000","last_peered":"2024-07-04T17:34:37.150879+0000","last_clean":"2024-07-04T17:34:37.150879+0000","last_became_active":"2024-07-04T17:34:35.535547+0000","last_became_peered":"2024-07-04T17:34:35.535547+0000","last_unstale":"2024-07-04T17:34:37.150879+0000","last_undegraded":"2024-07-04T17:34:37.150879+0000","last_fullsized":"2024-07-04T17:34:37.150879+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:22:16.634063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.514833+0000","last_change":"2024-07-04T17:34:35.561769+0000","last_active":"2024-07-04T17:34:36.514833+0000","last_peered":"2024-07-04T17:34:36.514833+0000","last_clean":"2024-07-04T17:34:36.514833+0000","last_became_active":"2024-07-04T17:34:35.560437+0000","last_became_peered":"2024-07-04T17:34:35.560437+0000","last_unstale":"2024-07-04T17:34:36.514833+0000","last_undegraded":"2024-07-04T17:34:36.514833+0000","last_fullsized":"2024-07-04T17:34:36.514833+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:52:11.652290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876275+0000","last_change":"2024-07-04T17:34:35.514882+0000","last_active":"2024-07-04T17:34:36.876275+0000","last_peered":"2024-07-04T17:34:36.876275+0000","last_clean":"2024-07-04T17:34:36.876275+0000","last_became_active":"2024-07-04T17:34:35.514560+0000","last_became_peered":"2024-07-04T17:34:35.514560+0000","last_unstale":"2024-07-04T17:34:36.876275+0000","last_undegraded":"2024-07-04T17:34:36.876275+0000","last_fullsized":"2024-07-04T17:34:36.876275+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:46:59.706787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.875838+0000","last_change":"2024-07-04T17:34:35.538102+0000","last_active":"2024-07-04T17:34:36.875838+0000","last_peered":"2024-07-04T17:34:36.875838+0000","last_clean":"2024-07-04T17:34:36.875838+0000","last_became_active":"2024-07-04T17:34:35.536203+0000","last_became_peered":"2024-07-04T17:34:35.536203+0000","last_unstale":"2024-07-04T17:34:36.875838+0000","last_undegraded":"2024-07-04T17:34:36.875838+0000","last_fullsized":"2024-07-04T17:34:36.875838+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:15:42.373781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.512868+0000","last_change":"2024-07-04T17:34:35.542229+0000","last_active":"2024-07-04T17:34:36.512868+0000","last_peered":"2024-07-04T17:34:36.512868+0000","last_clean":"2024-07-04T17:34:36.512868+0000","last_became_active":"2024-07-04T17:34:35.541754+0000","last_became_peered":"2024-07-04T17:34:35.541754+0000","last_unstale":"2024-07-04T17:34:36.512868+0000","last_undegraded":"2024-07-04T17:34:36.512868+0000","last_fullsized":"2024-07-04T17:34:36.512868+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:35:52.678989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.147957+0000","last_change":"2024-07-04T17:34:35.525814+0000","last_active":"2024-07-04T17:34:37.147957+0000","last_peered":"2024-07-04T17:34:37.147957+0000","last_clean":"2024-07-04T17:34:37.147957+0000","last_became_active":"2024-07-04T17:34:35.525479+0000","last_became_peered":"2024-07-04T17:34:35.525479+0000","last_unstale":"2024-07-04T17:34:37.147957+0000","last_undegraded":"2024-07-04T17:34:37.147957+0000","last_fullsized":"2024-07-04T17:34:37.147957+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:44:57.595263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.150012+0000","last_change":"2024-07-04T17:34:35.554702+0000","last_active":"2024-07-04T17:34:37.150012+0000","last_peered":"2024-07-04T17:34:37.150012+0000","last_clean":"2024-07-04T17:34:37.150012+0000","last_became_active":"2024-07-04T17:34:35.554205+0000","last_became_peered":"2024-07-04T17:34:35.554205+0000","last_unstale":"2024-07-04T17:34:37.150012+0000","last_undegraded":"2024-07-04T17:34:37.150012+0000","last_fullsized":"2024-07-04T17:34:37.150012+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:18:48.550958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.149338+0000","last_change":"2024-07-04T17:34:35.536019+0000","last_active":"2024-07-04T17:34:37.149338+0000","last_peered":"2024-07-04T17:34:37.149338+0000","last_clean":"2024-07-04T17:34:37.149338+0000","last_became_active":"2024-07-04T17:34:35.535705+0000","last_became_peered":"2024-07-04T17:34:35.535705+0000","last_unstale":"2024-07-04T17:34:37.149338+0000","last_undegraded":"2024-07-04T17:34:37.149338+0000","last_fullsized":"2024-07-04T17:34:37.149338+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:06:39.223609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.516771+0000","last_change":"2024-07-04T17:34:35.562071+0000","last_active":"2024-07-04T17:34:36.516771+0000","last_peered":"2024-07-04T17:34:36.516771+0000","last_clean":"2024-07-04T17:34:36.516771+0000","last_became_active":"2024-07-04T17:34:35.560220+0000","last_became_peered":"2024-07-04T17:34:35.560220+0000","last_unstale":"2024-07-04T17:34:36.516771+0000","last_undegraded":"2024-07-04T17:34:36.516771+0000","last_fullsized":"2024-07-04T17:34:36.516771+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:27:08.229969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.149675+0000","last_change":"2024-07-04T17:34:35.553714+0000","last_active":"2024-07-04T17:34:37.149675+0000","last_peered":"2024-07-04T17:34:37.149675+0000","last_clean":"2024-07-04T17:34:37.149675+0000","last_became_active":"2024-07-04T17:34:35.553384+0000","last_became_peered":"2024-07-04T17:34:35.553384+0000","last_unstale":"2024-07-04T17:34:37.149675+0000","last_undegraded":"2024-07-04T17:34:37.149675+0000","last_fullsized":"2024-07-04T17:34:37.149675+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:47:20.883963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.150040+0000","last_change":"2024-07-04T17:34:35.554950+0000","last_active":"2024-07-04T17:34:37.150040+0000","last_peered":"2024-07-04T17:34:37.150040+0000","last_clean":"2024-07-04T17:34:37.150040+0000","last_became_active":"2024-07-04T17:34:35.554259+0000","last_became_peered":"2024-07-04T17:34:35.554259+0000","last_unstale":"2024-07-04T17:34:37.150040+0000","last_undegraded":"2024-07-04T17:34:37.150040+0000","last_fullsized":"2024-07-04T17:34:37.150040+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:59:46.128637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.514038+0000","last_change":"2024-07-04T17:34:35.523276+0000","last_active":"2024-07-04T17:34:36.514038+0000","last_peered":"2024-07-04T17:34:36.514038+0000","last_clean":"2024-07-04T17:34:36.514038+0000","last_became_active":"2024-07-04T17:34:35.522611+0000","last_became_peered":"2024-07-04T17:34:35.522611+0000","last_unstale":"2024-07-04T17:34:36.514038+0000","last_undegraded":"2024-07-04T17:34:36.514038+0000","last_fullsized":"2024-07-04T17:34:36.514038+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:48:47.209945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.875725+0000","last_change":"2024-07-04T17:34:35.546941+0000","last_active":"2024-07-04T17:34:36.875725+0000","last_peered":"2024-07-04T17:34:36.875725+0000","last_clean":"2024-07-04T17:34:36.875725+0000","last_became_active":"2024-07-04T17:34:35.544163+0000","last_became_peered":"2024-07-04T17:34:35.544163+0000","last_unstale":"2024-07-04T17:34:36.875725+0000","last_undegraded":"2024-07-04T17:34:36.875725+0000","last_fullsized":"2024-07-04T17:34:36.875725+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:15:08.178810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.512521+0000","last_change":"2024-07-04T17:34:35.545420+0000","last_active":"2024-07-04T17:34:36.512521+0000","last_peered":"2024-07-04T17:34:36.512521+0000","last_clean":"2024-07-04T17:34:36.512521+0000","last_became_active":"2024-07-04T17:34:35.543935+0000","last_became_peered":"2024-07-04T17:34:35.543935+0000","last_unstale":"2024-07-04T17:34:36.512521+0000","last_undegraded":"2024-07-04T17:34:36.512521+0000","last_fullsized":"2024-07-04T17:34:36.512521+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:16:01.387094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.877366+0000","last_change":"2024-07-04T17:34:35.536388+0000","last_active":"2024-07-04T17:34:36.877366+0000","last_peered":"2024-07-04T17:34:36.877366+0000","last_clean":"2024-07-04T17:34:36.877366+0000","last_became_active":"2024-07-04T17:34:35.536052+0000","last_became_peered":"2024-07-04T17:34:35.536052+0000","last_unstale":"2024-07-04T17:34:36.877366+0000","last_undegraded":"2024-07-04T17:34:36.877366+0000","last_fullsized":"2024-07-04T17:34:36.877366+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:08:18.792180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.516187+0000","last_change":"2024-07-04T17:34:35.533907+0000","last_active":"2024-07-04T17:34:36.516187+0000","last_peered":"2024-07-04T17:34:36.516187+0000","last_clean":"2024-07-04T17:34:36.516187+0000","last_became_active":"2024-07-04T17:34:35.533596+0000","last_became_peered":"2024-07-04T17:34:35.533596+0000","last_unstale":"2024-07-04T17:34:36.516187+0000","last_undegraded":"2024-07-04T17:34:36.516187+0000","last_fullsized":"2024-07-04T17:34:36.516187+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:41:30.147907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876928+0000","last_change":"2024-07-04T17:34:35.537742+0000","last_active":"2024-07-04T17:34:36.876928+0000","last_peered":"2024-07-04T17:34:36.876928+0000","last_clean":"2024-07-04T17:34:36.876928+0000","last_became_active":"2024-07-04T17:34:35.533422+0000","last_became_peered":"2024-07-04T17:34:35.533422+0000","last_unstale":"2024-07-04T17:34:36.876928+0000","last_undegraded":"2024-07-04T17:34:36.876928+0000","last_fullsized":"2024-07-04T17:34:36.876928+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:02:05.629708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.149794+0000","last_change":"2024-07-04T17:34:35.553505+0000","last_active":"2024-07-04T17:34:37.149794+0000","last_peered":"2024-07-04T17:34:37.149794+0000","last_clean":"2024-07-04T17:34:37.149794+0000","last_became_active":"2024-07-04T17:34:35.553230+0000","last_became_peered":"2024-07-04T17:34:35.553230+0000","last_unstale":"2024-07-04T17:34:37.149794+0000","last_undegraded":"2024-07-04T17:34:37.149794+0000","last_fullsized":"2024-07-04T17:34:37.149794+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:49:38.920935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.516510+0000","last_change":"2024-07-04T17:34:35.524780+0000","last_active":"2024-07-04T17:34:36.516510+0000","last_peered":"2024-07-04T17:34:36.516510+0000","last_clean":"2024-07-04T17:34:36.516510+0000","last_became_active":"2024-07-04T17:34:35.524418+0000","last_became_peered":"2024-07-04T17:34:35.524418+0000","last_unstale":"2024-07-04T17:34:36.516510+0000","last_undegraded":"2024-07-04T17:34:36.516510+0000","last_fullsized":"2024-07-04T17:34:36.516510+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:16:31.650779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.514271+0000","last_change":"2024-07-04T17:34:35.522942+0000","last_active":"2024-07-04T17:34:36.514271+0000","last_peered":"2024-07-04T17:34:36.514271+0000","last_clean":"2024-07-04T17:34:36.514271+0000","last_became_active":"2024-07-04T17:34:35.522210+0000","last_became_peered":"2024-07-04T17:34:35.522210+0000","last_unstale":"2024-07-04T17:34:36.514271+0000","last_undegraded":"2024-07-04T17:34:36.514271+0000","last_fullsized":"2024-07-04T17:34:36.514271+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:03:46.125136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876271+0000","last_change":"2024-07-04T17:34:35.544485+0000","last_active":"2024-07-04T17:34:36.876271+0000","last_peered":"2024-07-04T17:34:36.876271+0000","last_clean":"2024-07-04T17:34:36.876271+0000","last_became_active":"2024-07-04T17:34:35.539252+0000","last_became_peered":"2024-07-04T17:34:35.539252+0000","last_unstale":"2024-07-04T17:34:36.876271+0000","last_undegraded":"2024-07-04T17:34:36.876271+0000","last_fullsized":"2024-07-04T17:34:36.876271+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:58:20.122138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.148332+0000","last_change":"2024-07-04T17:34:35.532311+0000","last_active":"2024-07-04T17:34:37.148332+0000","last_peered":"2024-07-04T17:34:37.148332+0000","last_clean":"2024-07-04T17:34:37.148332+0000","last_became_active":"2024-07-04T17:34:35.531704+0000","last_became_peered":"2024-07-04T17:34:35.531704+0000","last_unstale":"2024-07-04T17:34:37.148332+0000","last_undegraded":"2024-07-04T17:34:37.148332+0000","last_fullsized":"2024-07-04T17:34:37.148332+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:38:06.359176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876980+0000","last_change":"2024-07-04T17:34:35.536755+0000","last_active":"2024-07-04T17:34:36.876980+0000","last_peered":"2024-07-04T17:34:36.876980+0000","last_clean":"2024-07-04T17:34:36.876980+0000","last_became_active":"2024-07-04T17:34:35.535641+0000","last_became_peered":"2024-07-04T17:34:35.535641+0000","last_unstale":"2024-07-04T17:34:36.876980+0000","last_undegraded":"2024-07-04T17:34:36.876980+0000","last_fullsized":"2024-07-04T17:34:36.876980+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:00:03.434775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1f","version":"135'4","reported_seq":18,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.150606+0000","last_change":"2024-07-04T17:34:34.526194+0000","last_active":"2024-07-04T17:34:37.150606+0000","last_peered":"2024-07-04T17:34:37.150606+0000","last_clean":"2024-07-04T17:34:37.150606+0000","last_became_active":"2024-07-04T17:34:34.525749+0000","last_became_peered":"2024-07-04T17:34:34.525749+0000","last_unstale":"2024-07-04T17:34:37.150606+0000","last_undegraded":"2024-07-04T17:34:37.150606+0000","last_fullsized":"2024-07-04T17:34:37.150606+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:15:41.361322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.516393+0000","last_change":"2024-07-04T17:34:35.548372+0000","last_active":"2024-07-04T17:34:36.516393+0000","last_peered":"2024-07-04T17:34:36.516393+0000","last_clean":"2024-07-04T17:34:36.516393+0000","last_became_active":"2024-07-04T17:34:35.547829+0000","last_became_peered":"2024-07-04T17:34:35.547829+0000","last_unstale":"2024-07-04T17:34:36.516393+0000","last_undegraded":"2024-07-04T17:34:36.516393+0000","last_fullsized":"2024-07-04T17:34:36.516393+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:37:49.541089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"135'1","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.516376+0000","last_change":"2024-07-04T17:34:34.505687+0000","last_active":"2024-07-04T17:34:36.516376+0000","last_peered":"2024-07-04T17:34:36.516376+0000","last_clean":"2024-07-04T17:34:36.516376+0000","last_became_active":"2024-07-04T17:34:34.505337+0000","last_became_peered":"2024-07-04T17:34:34.505337+0000","last_unstale":"2024-07-04T17:34:36.516376+0000","last_undegraded":"2024-07-04T17:34:36.516376+0000","last_fullsized":"2024-07-04T17:34:36.516376+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:57:27.037624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876216+0000","last_change":"2024-07-04T17:34:35.531247+0000","last_active":"2024-07-04T17:34:36.876216+0000","last_peered":"2024-07-04T17:34:36.876216+0000","last_clean":"2024-07-04T17:34:36.876216+0000","last_became_active":"2024-07-04T17:34:35.530846+0000","last_became_peered":"2024-07-04T17:34:35.530846+0000","last_unstale":"2024-07-04T17:34:36.876216+0000","last_undegraded":"2024-07-04T17:34:36.876216+0000","last_fullsized":"2024-07-04T17:34:36.876216+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:30:51.180015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1d","version":"135'2","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.877257+0000","last_change":"2024-07-04T17:34:34.512001+0000","last_active":"2024-07-04T17:34:36.877257+0000","last_peered":"2024-07-04T17:34:36.877257+0000","last_clean":"2024-07-04T17:34:36.877257+0000","last_became_active":"2024-07-04T17:34:34.511480+0000","last_became_peered":"2024-07-04T17:34:34.511480+0000","last_unstale":"2024-07-04T17:34:36.877257+0000","last_undegraded":"2024-07-04T17:34:36.877257+0000","last_fullsized":"2024-07-04T17:34:36.877257+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:46:03.790681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.877115+0000","last_change":"2024-07-04T17:34:35.546422+0000","last_active":"2024-07-04T17:34:36.877115+0000","last_peered":"2024-07-04T17:34:36.877115+0000","last_clean":"2024-07-04T17:34:36.877115+0000","last_became_active":"2024-07-04T17:34:35.546061+0000","last_became_peered":"2024-07-04T17:34:35.546061+0000","last_unstale":"2024-07-04T17:34:36.877115+0000","last_undegraded":"2024-07-04T17:34:36.877115+0000","last_fullsized":"2024-07-04T17:34:36.877115+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:50:00.540847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1c","version":"135'2","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.515111+0000","last_change":"2024-07-04T17:34:34.506897+0000","last_active":"2024-07-04T17:34:36.515111+0000","last_peered":"2024-07-04T17:34:36.515111+0000","last_clean":"2024-07-04T17:34:36.515111+0000","last_became_active":"2024-07-04T17:34:34.506023+0000","last_became_peered":"2024-07-04T17:34:34.506023+0000","last_unstale":"2024-07-04T17:34:36.515111+0000","last_undegraded":"2024-07-04T17:34:36.515111+0000","last_fullsized":"2024-07-04T17:34:36.515111+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:36:57.170806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.514873+0000","last_change":"2024-07-04T17:34:35.515987+0000","last_active":"2024-07-04T17:34:36.514873+0000","last_peered":"2024-07-04T17:34:36.514873+0000","last_clean":"2024-07-04T17:34:36.514873+0000","last_became_active":"2024-07-04T17:34:35.515158+0000","last_became_peered":"2024-07-04T17:34:35.515158+0000","last_unstale":"2024-07-04T17:34:36.514873+0000","last_undegraded":"2024-07-04T17:34:36.514873+0000","last_fullsized":"2024-07-04T17:34:36.514873+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:39:44.537119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"135'1","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.151220+0000","last_change":"2024-07-04T17:34:34.526306+0000","last_active":"2024-07-04T17:34:37.151220+0000","last_peered":"2024-07-04T17:34:37.151220+0000","last_clean":"2024-07-04T17:34:37.151220+0000","last_became_active":"2024-07-04T17:34:34.525936+0000","last_became_peered":"2024-07-04T17:34:34.525936+0000","last_unstale":"2024-07-04T17:34:37.151220+0000","last_undegraded":"2024-07-04T17:34:37.151220+0000","last_fullsized":"2024-07-04T17:34:37.151220+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:58:19.109679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.150714+0000","last_change":"2024-07-04T17:34:35.529792+0000","last_active":"2024-07-04T17:34:37.150714+0000","last_peered":"2024-07-04T17:34:37.150714+0000","last_clean":"2024-07-04T17:34:37.150714+0000","last_became_active":"2024-07-04T17:34:35.529159+0000","last_became_peered":"2024-07-04T17:34:35.529159+0000","last_unstale":"2024-07-04T17:34:37.150714+0000","last_undegraded":"2024-07-04T17:34:37.150714+0000","last_fullsized":"2024-07-04T17:34:37.150714+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:19:05.515444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.1a","version":"135'1","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.149312+0000","last_change":"2024-07-04T17:34:34.520915+0000","last_active":"2024-07-04T17:34:37.149312+0000","last_peered":"2024-07-04T17:34:37.149312+0000","last_clean":"2024-07-04T17:34:37.149312+0000","last_became_active":"2024-07-04T17:34:34.520164+0000","last_became_peered":"2024-07-04T17:34:34.520164+0000","last_unstale":"2024-07-04T17:34:37.149312+0000","last_undegraded":"2024-07-04T17:34:37.149312+0000","last_fullsized":"2024-07-04T17:34:37.149312+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:15:07.166351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.148501+0000","last_change":"2024-07-04T17:34:35.528571+0000","last_active":"2024-07-04T17:34:37.148501+0000","last_peered":"2024-07-04T17:34:37.148501+0000","last_clean":"2024-07-04T17:34:37.148501+0000","last_became_active":"2024-07-04T17:34:35.528217+0000","last_became_peered":"2024-07-04T17:34:35.528217+0000","last_unstale":"2024-07-04T17:34:37.148501+0000","last_undegraded":"2024-07-04T17:34:37.148501+0000","last_fullsized":"2024-07-04T17:34:37.148501+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:25:31.311940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.148060+0000","last_change":"2024-07-04T17:34:34.520226+0000","last_active":"2024-07-04T17:34:37.148060+0000","last_peered":"2024-07-04T17:34:37.148060+0000","last_clean":"2024-07-04T17:34:37.148060+0000","last_became_active":"2024-07-04T17:34:34.519880+0000","last_became_peered":"2024-07-04T17:34:34.519880+0000","last_unstale":"2024-07-04T17:34:37.148060+0000","last_undegraded":"2024-07-04T17:34:37.148060+0000","last_fullsized":"2024-07-04T17:34:37.148060+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:30:01.808525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.877355+0000","last_change":"2024-07-04T17:34:35.541668+0000","last_active":"2024-07-04T17:34:36.877355+0000","last_peered":"2024-07-04T17:34:36.877355+0000","last_clean":"2024-07-04T17:34:36.877355+0000","last_became_active":"2024-07-04T17:34:35.537142+0000","last_became_peered":"2024-07-04T17:34:35.537142+0000","last_unstale":"2024-07-04T17:34:36.877355+0000","last_undegraded":"2024-07-04T17:34:36.877355+0000","last_fullsized":"2024-07-04T17:34:36.877355+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:56:41.649364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.877615+0000","last_change":"2024-07-04T17:34:34.516054+0000","last_active":"2024-07-04T17:34:36.877615+0000","last_peered":"2024-07-04T17:34:36.877615+0000","last_clean":"2024-07-04T17:34:36.877615+0000","last_became_active":"2024-07-04T17:34:34.514177+0000","last_became_peered":"2024-07-04T17:34:34.514177+0000","last_unstale":"2024-07-04T17:34:36.877615+0000","last_undegraded":"2024-07-04T17:34:36.877615+0000","last_fullsized":"2024-07-04T17:34:36.877615+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:18:46.789057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.877584+0000","last_change":"2024-07-04T17:34:35.546010+0000","last_active":"2024-07-04T17:34:36.877584+0000","last_peered":"2024-07-04T17:34:36.877584+0000","last_clean":"2024-07-04T17:34:36.877584+0000","last_became_active":"2024-07-04T17:34:35.536818+0000","last_became_peered":"2024-07-04T17:34:35.536818+0000","last_unstale":"2024-07-04T17:34:36.877584+0000","last_undegraded":"2024-07-04T17:34:36.877584+0000","last_fullsized":"2024-07-04T17:34:36.877584+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:24:20.145687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.149751+0000","last_change":"2024-07-04T17:34:34.526851+0000","last_active":"2024-07-04T17:34:37.149751+0000","last_peered":"2024-07-04T17:34:37.149751+0000","last_clean":"2024-07-04T17:34:37.149751+0000","last_became_active":"2024-07-04T17:34:34.526514+0000","last_became_peered":"2024-07-04T17:34:34.526514+0000","last_unstale":"2024-07-04T17:34:37.149751+0000","last_undegraded":"2024-07-04T17:34:37.149751+0000","last_fullsized":"2024-07-04T17:34:37.149751+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:37:45.915030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.515402+0000","last_change":"2024-07-04T17:34:35.524150+0000","last_active":"2024-07-04T17:34:36.515402+0000","last_peered":"2024-07-04T17:34:36.515402+0000","last_clean":"2024-07-04T17:34:36.515402+0000","last_became_active":"2024-07-04T17:34:35.523835+0000","last_became_peered":"2024-07-04T17:34:35.523835+0000","last_unstale":"2024-07-04T17:34:36.515402+0000","last_undegraded":"2024-07-04T17:34:36.515402+0000","last_fullsized":"2024-07-04T17:34:36.515402+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:14:55.953139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.15","version":"135'3","reported_seq":17,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.150842+0000","last_change":"2024-07-04T17:34:34.525906+0000","last_active":"2024-07-04T17:34:37.150842+0000","last_peered":"2024-07-04T17:34:37.150842+0000","last_clean":"2024-07-04T17:34:37.150842+0000","last_became_active":"2024-07-04T17:34:34.525545+0000","last_became_peered":"2024-07-04T17:34:34.525545+0000","last_unstale":"2024-07-04T17:34:37.150842+0000","last_undegraded":"2024-07-04T17:34:37.150842+0000","last_fullsized":"2024-07-04T17:34:37.150842+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:15:15.110008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.516890+0000","last_change":"2024-07-04T17:34:35.523356+0000","last_active":"2024-07-04T17:34:36.516890+0000","last_peered":"2024-07-04T17:34:36.516890+0000","last_clean":"2024-07-04T17:34:36.516890+0000","last_became_active":"2024-07-04T17:34:35.522584+0000","last_became_peered":"2024-07-04T17:34:35.522584+0000","last_unstale":"2024-07-04T17:34:36.516890+0000","last_undegraded":"2024-07-04T17:34:36.516890+0000","last_fullsized":"2024-07-04T17:34:36.516890+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:34:41.701054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"135'6","reported_seq":20,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876837+0000","last_change":"2024-07-04T17:34:34.513760+0000","last_active":"2024-07-04T17:34:36.876837+0000","last_peered":"2024-07-04T17:34:36.876837+0000","last_clean":"2024-07-04T17:34:36.876837+0000","last_became_active":"2024-07-04T17:34:34.512609+0000","last_became_peered":"2024-07-04T17:34:34.512609+0000","last_unstale":"2024-07-04T17:34:36.876837+0000","last_undegraded":"2024-07-04T17:34:36.876837+0000","last_fullsized":"2024-07-04T17:34:36.876837+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:08:12.565423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.875976+0000","last_change":"2024-07-04T17:34:35.537826+0000","last_active":"2024-07-04T17:34:36.875976+0000","last_peered":"2024-07-04T17:34:36.875976+0000","last_clean":"2024-07-04T17:34:36.875976+0000","last_became_active":"2024-07-04T17:34:35.535781+0000","last_became_peered":"2024-07-04T17:34:35.535781+0000","last_unstale":"2024-07-04T17:34:36.875976+0000","last_undegraded":"2024-07-04T17:34:36.875976+0000","last_fullsized":"2024-07-04T17:34:36.875976+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:26:50.917096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"135'2","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.514769+0000","last_change":"2024-07-04T17:34:34.507682+0000","last_active":"2024-07-04T17:34:36.514769+0000","last_peered":"2024-07-04T17:34:36.514769+0000","last_clean":"2024-07-04T17:34:36.514769+0000","last_became_active":"2024-07-04T17:34:34.507353+0000","last_became_peered":"2024-07-04T17:34:34.507353+0000","last_unstale":"2024-07-04T17:34:36.514769+0000","last_undegraded":"2024-07-04T17:34:36.514769+0000","last_fullsized":"2024-07-04T17:34:36.514769+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:47:18.975316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.151057+0000","last_change":"2024-07-04T17:34:35.529515+0000","last_active":"2024-07-04T17:34:37.151057+0000","last_peered":"2024-07-04T17:34:37.151057+0000","last_clean":"2024-07-04T17:34:37.151057+0000","last_became_active":"2024-07-04T17:34:35.528532+0000","last_became_peered":"2024-07-04T17:34:35.528532+0000","last_unstale":"2024-07-04T17:34:37.151057+0000","last_undegraded":"2024-07-04T17:34:37.151057+0000","last_fullsized":"2024-07-04T17:34:37.151057+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:33:17.769730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.515596+0000","last_change":"2024-07-04T17:34:34.512082+0000","last_active":"2024-07-04T17:34:36.515596+0000","last_peered":"2024-07-04T17:34:36.515596+0000","last_clean":"2024-07-04T17:34:36.515596+0000","last_became_active":"2024-07-04T17:34:34.511518+0000","last_became_peered":"2024-07-04T17:34:34.511518+0000","last_unstale":"2024-07-04T17:34:36.515596+0000","last_undegraded":"2024-07-04T17:34:36.515596+0000","last_fullsized":"2024-07-04T17:34:36.515596+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:34:32.553466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.513075+0000","last_change":"2024-07-04T17:34:35.539061+0000","last_active":"2024-07-04T17:34:36.513075+0000","last_peered":"2024-07-04T17:34:36.513075+0000","last_clean":"2024-07-04T17:34:36.513075+0000","last_became_active":"2024-07-04T17:34:35.538660+0000","last_became_peered":"2024-07-04T17:34:35.538660+0000","last_unstale":"2024-07-04T17:34:36.513075+0000","last_undegraded":"2024-07-04T17:34:36.513075+0000","last_fullsized":"2024-07-04T17:34:36.513075+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:36:20.868550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.513414+0000","last_change":"2024-07-04T17:34:34.513736+0000","last_active":"2024-07-04T17:34:36.513414+0000","last_peered":"2024-07-04T17:34:36.513414+0000","last_clean":"2024-07-04T17:34:36.513414+0000","last_became_active":"2024-07-04T17:34:34.512396+0000","last_became_peered":"2024-07-04T17:34:34.512396+0000","last_unstale":"2024-07-04T17:34:36.513414+0000","last_undegraded":"2024-07-04T17:34:36.513414+0000","last_fullsized":"2024-07-04T17:34:36.513414+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:25:30.776827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.513375+0000","last_change":"2024-07-04T17:34:35.540239+0000","last_active":"2024-07-04T17:34:36.513375+0000","last_peered":"2024-07-04T17:34:36.513375+0000","last_clean":"2024-07-04T17:34:36.513375+0000","last_became_active":"2024-07-04T17:34:35.539911+0000","last_became_peered":"2024-07-04T17:34:35.539911+0000","last_unstale":"2024-07-04T17:34:36.513375+0000","last_undegraded":"2024-07-04T17:34:36.513375+0000","last_fullsized":"2024-07-04T17:34:36.513375+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:03:47.021324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"135'8","reported_seq":22,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.148314+0000","last_change":"2024-07-04T17:34:34.526204+0000","last_active":"2024-07-04T17:34:37.148314+0000","last_peered":"2024-07-04T17:34:37.148314+0000","last_clean":"2024-07-04T17:34:37.148314+0000","last_became_active":"2024-07-04T17:34:34.525811+0000","last_became_peered":"2024-07-04T17:34:34.525811+0000","last_unstale":"2024-07-04T17:34:37.148314+0000","last_undegraded":"2024-07-04T17:34:37.148314+0000","last_fullsized":"2024-07-04T17:34:37.148314+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:49:36.910071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.516594+0000","last_change":"2024-07-04T17:34:35.530930+0000","last_active":"2024-07-04T17:34:36.516594+0000","last_peered":"2024-07-04T17:34:36.516594+0000","last_clean":"2024-07-04T17:34:36.516594+0000","last_became_active":"2024-07-04T17:34:35.527474+0000","last_became_peered":"2024-07-04T17:34:35.527474+0000","last_unstale":"2024-07-04T17:34:36.516594+0000","last_undegraded":"2024-07-04T17:34:36.516594+0000","last_fullsized":"2024-07-04T17:34:36.516594+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:51:21.803840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"135'1","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.150859+0000","last_change":"2024-07-04T17:34:34.526961+0000","last_active":"2024-07-04T17:34:37.150859+0000","last_peered":"2024-07-04T17:34:37.150859+0000","last_clean":"2024-07-04T17:34:37.150859+0000","last_became_active":"2024-07-04T17:34:34.526338+0000","last_became_peered":"2024-07-04T17:34:34.526338+0000","last_unstale":"2024-07-04T17:34:37.150859+0000","last_undegraded":"2024-07-04T17:34:37.150859+0000","last_fullsized":"2024-07-04T17:34:37.150859+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:46:36.828153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.516882+0000","last_change":"2024-07-04T17:34:35.546152+0000","last_active":"2024-07-04T17:34:36.516882+0000","last_peered":"2024-07-04T17:34:36.516882+0000","last_clean":"2024-07-04T17:34:36.516882+0000","last_became_active":"2024-07-04T17:34:35.545573+0000","last_became_peered":"2024-07-04T17:34:35.545573+0000","last_unstale":"2024-07-04T17:34:36.516882+0000","last_undegraded":"2024-07-04T17:34:36.516882+0000","last_fullsized":"2024-07-04T17:34:36.516882+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:20:21.078798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.e","version":"135'2","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.877128+0000","last_change":"2024-07-04T17:34:34.516533+0000","last_active":"2024-07-04T17:34:36.877128+0000","last_peered":"2024-07-04T17:34:36.877128+0000","last_clean":"2024-07-04T17:34:36.877128+0000","last_became_active":"2024-07-04T17:34:34.515492+0000","last_became_peered":"2024-07-04T17:34:34.515492+0000","last_unstale":"2024-07-04T17:34:36.877128+0000","last_undegraded":"2024-07-04T17:34:36.877128+0000","last_fullsized":"2024-07-04T17:34:36.877128+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:06:02.654676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876587+0000","last_change":"2024-07-04T17:34:35.544212+0000","last_active":"2024-07-04T17:34:36.876587+0000","last_peered":"2024-07-04T17:34:36.876587+0000","last_clean":"2024-07-04T17:34:36.876587+0000","last_became_active":"2024-07-04T17:34:35.543570+0000","last_became_peered":"2024-07-04T17:34:35.543570+0000","last_unstale":"2024-07-04T17:34:36.876587+0000","last_undegraded":"2024-07-04T17:34:36.876587+0000","last_fullsized":"2024-07-04T17:34:36.876587+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:38:02.545198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.d","version":"135'1","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.517109+0000","last_change":"2024-07-04T17:34:34.514159+0000","last_active":"2024-07-04T17:34:36.517109+0000","last_peered":"2024-07-04T17:34:36.517109+0000","last_clean":"2024-07-04T17:34:36.517109+0000","last_became_active":"2024-07-04T17:34:34.511974+0000","last_became_peered":"2024-07-04T17:34:34.511974+0000","last_unstale":"2024-07-04T17:34:36.517109+0000","last_undegraded":"2024-07-04T17:34:36.517109+0000","last_fullsized":"2024-07-04T17:34:36.517109+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:20:11.531736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.517145+0000","last_change":"2024-07-04T17:34:35.523503+0000","last_active":"2024-07-04T17:34:36.517145+0000","last_peered":"2024-07-04T17:34:36.517145+0000","last_clean":"2024-07-04T17:34:36.517145+0000","last_became_active":"2024-07-04T17:34:35.522995+0000","last_became_peered":"2024-07-04T17:34:35.522995+0000","last_unstale":"2024-07-04T17:34:36.517145+0000","last_undegraded":"2024-07-04T17:34:36.517145+0000","last_fullsized":"2024-07-04T17:34:36.517145+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:24:36.746418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.515481+0000","last_change":"2024-07-04T17:34:34.508754+0000","last_active":"2024-07-04T17:34:36.515481+0000","last_peered":"2024-07-04T17:34:36.515481+0000","last_clean":"2024-07-04T17:34:36.515481+0000","last_became_active":"2024-07-04T17:34:34.507614+0000","last_became_peered":"2024-07-04T17:34:34.507614+0000","last_unstale":"2024-07-04T17:34:36.515481+0000","last_undegraded":"2024-07-04T17:34:36.515481+0000","last_fullsized":"2024-07-04T17:34:36.515481+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:09:44.865693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.515411+0000","last_change":"2024-07-04T17:34:35.515740+0000","last_active":"2024-07-04T17:34:36.515411+0000","last_peered":"2024-07-04T17:34:36.515411+0000","last_clean":"2024-07-04T17:34:36.515411+0000","last_became_active":"2024-07-04T17:34:35.514774+0000","last_became_peered":"2024-07-04T17:34:35.514774+0000","last_unstale":"2024-07-04T17:34:36.515411+0000","last_undegraded":"2024-07-04T17:34:36.515411+0000","last_fullsized":"2024-07-04T17:34:36.515411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:54:35.373423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"135'2","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876412+0000","last_change":"2024-07-04T17:34:34.515650+0000","last_active":"2024-07-04T17:34:36.876412+0000","last_peered":"2024-07-04T17:34:36.876412+0000","last_clean":"2024-07-04T17:34:36.876412+0000","last_became_active":"2024-07-04T17:34:34.514347+0000","last_became_peered":"2024-07-04T17:34:34.514347+0000","last_unstale":"2024-07-04T17:34:36.876412+0000","last_undegraded":"2024-07-04T17:34:36.876412+0000","last_fullsized":"2024-07-04T17:34:36.876412+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:34:16.111397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.150994+0000","last_change":"2024-07-04T17:34:35.529352+0000","last_active":"2024-07-04T17:34:37.150994+0000","last_peered":"2024-07-04T17:34:37.150994+0000","last_clean":"2024-07-04T17:34:37.150994+0000","last_became_active":"2024-07-04T17:34:35.528095+0000","last_became_peered":"2024-07-04T17:34:35.528095+0000","last_unstale":"2024-07-04T17:34:37.150994+0000","last_undegraded":"2024-07-04T17:34:37.150994+0000","last_fullsized":"2024-07-04T17:34:37.150994+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:53:30.230859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.149721+0000","last_change":"2024-07-04T17:34:34.520687+0000","last_active":"2024-07-04T17:34:37.149721+0000","last_peered":"2024-07-04T17:34:37.149721+0000","last_clean":"2024-07-04T17:34:37.149721+0000","last_became_active":"2024-07-04T17:34:34.520328+0000","last_became_peered":"2024-07-04T17:34:34.520328+0000","last_unstale":"2024-07-04T17:34:37.149721+0000","last_undegraded":"2024-07-04T17:34:37.149721+0000","last_fullsized":"2024-07-04T17:34:37.149721+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:26:49.904637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.513134+0000","last_change":"2024-07-04T17:34:35.525251+0000","last_active":"2024-07-04T17:34:36.513134+0000","last_peered":"2024-07-04T17:34:36.513134+0000","last_clean":"2024-07-04T17:34:36.513134+0000","last_became_active":"2024-07-04T17:34:35.524606+0000","last_became_peered":"2024-07-04T17:34:35.524606+0000","last_unstale":"2024-07-04T17:34:36.513134+0000","last_undegraded":"2024-07-04T17:34:36.513134+0000","last_fullsized":"2024-07-04T17:34:36.513134+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:18:18.311591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876069+0000","last_change":"2024-07-04T17:34:34.515246+0000","last_active":"2024-07-04T17:34:36.876069+0000","last_peered":"2024-07-04T17:34:36.876069+0000","last_clean":"2024-07-04T17:34:36.876069+0000","last_became_active":"2024-07-04T17:34:34.514888+0000","last_became_peered":"2024-07-04T17:34:34.514888+0000","last_unstale":"2024-07-04T17:34:36.876069+0000","last_undegraded":"2024-07-04T17:34:36.876069+0000","last_fullsized":"2024-07-04T17:34:36.876069+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:35:07.644625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.148882+0000","last_change":"2024-07-04T17:34:35.541160+0000","last_active":"2024-07-04T17:34:37.148882+0000","last_peered":"2024-07-04T17:34:37.148882+0000","last_clean":"2024-07-04T17:34:37.148882+0000","last_became_active":"2024-07-04T17:34:35.540869+0000","last_became_peered":"2024-07-04T17:34:35.540869+0000","last_unstale":"2024-07-04T17:34:37.148882+0000","last_undegraded":"2024-07-04T17:34:37.148882+0000","last_fullsized":"2024-07-04T17:34:37.148882+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:41:18.824553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.148452+0000","last_change":"2024-07-04T17:34:34.521170+0000","last_active":"2024-07-04T17:34:37.148452+0000","last_peered":"2024-07-04T17:34:37.148452+0000","last_clean":"2024-07-04T17:34:37.148452+0000","last_became_active":"2024-07-04T17:34:34.520731+0000","last_became_peered":"2024-07-04T17:34:34.520731+0000","last_unstale":"2024-07-04T17:34:37.148452+0000","last_undegraded":"2024-07-04T17:34:37.148452+0000","last_fullsized":"2024-07-04T17:34:37.148452+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:30:50.167556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.149653+0000","last_change":"2024-07-04T17:34:35.534458+0000","last_active":"2024-07-04T17:34:37.149653+0000","last_peered":"2024-07-04T17:34:37.149653+0000","last_clean":"2024-07-04T17:34:37.149653+0000","last_became_active":"2024-07-04T17:34:35.534123+0000","last_became_peered":"2024-07-04T17:34:35.534123+0000","last_unstale":"2024-07-04T17:34:37.149653+0000","last_undegraded":"2024-07-04T17:34:37.149653+0000","last_fullsized":"2024-07-04T17:34:37.149653+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:55:02.890933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"135'2","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.517059+0000","last_change":"2024-07-04T17:34:34.516580+0000","last_active":"2024-07-04T17:34:36.517059+0000","last_peered":"2024-07-04T17:34:36.517059+0000","last_clean":"2024-07-04T17:34:36.517059+0000","last_became_active":"2024-07-04T17:34:34.513629+0000","last_became_peered":"2024-07-04T17:34:34.513629+0000","last_unstale":"2024-07-04T17:34:36.517059+0000","last_undegraded":"2024-07-04T17:34:36.517059+0000","last_fullsized":"2024-07-04T17:34:36.517059+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:31:26.410567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.517014+0000","last_change":"2024-07-04T17:34:35.545883+0000","last_active":"2024-07-04T17:34:36.517014+0000","last_peered":"2024-07-04T17:34:36.517014+0000","last_clean":"2024-07-04T17:34:36.517014+0000","last_became_active":"2024-07-04T17:34:35.545154+0000","last_became_peered":"2024-07-04T17:34:35.545154+0000","last_unstale":"2024-07-04T17:34:36.517014+0000","last_undegraded":"2024-07-04T17:34:36.517014+0000","last_fullsized":"2024-07-04T17:34:36.517014+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:36:58.592291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"135'1","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.516726+0000","last_change":"2024-07-04T17:34:34.505851+0000","last_active":"2024-07-04T17:34:36.516726+0000","last_peered":"2024-07-04T17:34:36.516726+0000","last_clean":"2024-07-04T17:34:36.516726+0000","last_became_active":"2024-07-04T17:34:34.505548+0000","last_became_peered":"2024-07-04T17:34:34.505548+0000","last_unstale":"2024-07-04T17:34:36.516726+0000","last_undegraded":"2024-07-04T17:34:36.516726+0000","last_fullsized":"2024-07-04T17:34:36.516726+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:17:04.987342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876683+0000","last_change":"2024-07-04T17:34:35.543872+0000","last_active":"2024-07-04T17:34:36.876683+0000","last_peered":"2024-07-04T17:34:36.876683+0000","last_clean":"2024-07-04T17:34:36.876683+0000","last_became_active":"2024-07-04T17:34:35.542863+0000","last_became_peered":"2024-07-04T17:34:35.542863+0000","last_unstale":"2024-07-04T17:34:36.876683+0000","last_undegraded":"2024-07-04T17:34:36.876683+0000","last_fullsized":"2024-07-04T17:34:36.876683+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:28:18.564334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.517238+0000","last_change":"2024-07-04T17:34:34.505554+0000","last_active":"2024-07-04T17:34:36.517238+0000","last_peered":"2024-07-04T17:34:36.517238+0000","last_clean":"2024-07-04T17:34:36.517238+0000","last_became_active":"2024-07-04T17:34:34.505151+0000","last_became_peered":"2024-07-04T17:34:34.505151+0000","last_unstale":"2024-07-04T17:34:36.517238+0000","last_undegraded":"2024-07-04T17:34:36.517238+0000","last_fullsized":"2024-07-04T17:34:36.517238+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:17:53.346868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.877230+0000","last_change":"2024-07-04T17:34:35.543911+0000","last_active":"2024-07-04T17:34:36.877230+0000","last_peered":"2024-07-04T17:34:36.877230+0000","last_clean":"2024-07-04T17:34:36.877230+0000","last_became_active":"2024-07-04T17:34:35.533364+0000","last_became_peered":"2024-07-04T17:34:35.533364+0000","last_unstale":"2024-07-04T17:34:36.877230+0000","last_undegraded":"2024-07-04T17:34:36.877230+0000","last_fullsized":"2024-07-04T17:34:36.877230+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:13:03.582706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.515601+0000","last_change":"2024-07-04T17:34:34.507394+0000","last_active":"2024-07-04T17:34:36.515601+0000","last_peered":"2024-07-04T17:34:36.515601+0000","last_clean":"2024-07-04T17:34:36.515601+0000","last_became_active":"2024-07-04T17:34:34.506560+0000","last_became_peered":"2024-07-04T17:34:34.506560+0000","last_unstale":"2024-07-04T17:34:36.515601+0000","last_undegraded":"2024-07-04T17:34:36.515601+0000","last_fullsized":"2024-07-04T17:34:36.515601+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:12:30.641841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876028+0000","last_change":"2024-07-04T17:34:35.543760+0000","last_active":"2024-07-04T17:34:36.876028+0000","last_peered":"2024-07-04T17:34:36.876028+0000","last_clean":"2024-07-04T17:34:36.876028+0000","last_became_active":"2024-07-04T17:34:35.533337+0000","last_became_peered":"2024-07-04T17:34:35.533337+0000","last_unstale":"2024-07-04T17:34:36.876028+0000","last_undegraded":"2024-07-04T17:34:36.876028+0000","last_fullsized":"2024-07-04T17:34:36.876028+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:15:16.122467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876366+0000","last_change":"2024-07-04T17:34:34.513417+0000","last_active":"2024-07-04T17:34:36.876366+0000","last_peered":"2024-07-04T17:34:36.876366+0000","last_clean":"2024-07-04T17:34:36.876366+0000","last_became_active":"2024-07-04T17:34:34.513092+0000","last_became_peered":"2024-07-04T17:34:34.513092+0000","last_unstale":"2024-07-04T17:34:36.876366+0000","last_undegraded":"2024-07-04T17:34:36.876366+0000","last_fullsized":"2024-07-04T17:34:36.876366+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:04:50.268887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.151142+0000","last_change":"2024-07-04T17:34:35.521764+0000","last_active":"2024-07-04T17:34:37.151142+0000","last_peered":"2024-07-04T17:34:37.151142+0000","last_clean":"2024-07-04T17:34:37.151142+0000","last_became_active":"2024-07-04T17:34:35.521455+0000","last_became_peered":"2024-07-04T17:34:35.521455+0000","last_unstale":"2024-07-04T17:34:37.151142+0000","last_undegraded":"2024-07-04T17:34:37.151142+0000","last_fullsized":"2024-07-04T17:34:37.151142+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:54:12.411851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876108+0000","last_change":"2024-07-04T17:34:34.518708+0000","last_active":"2024-07-04T17:34:36.876108+0000","last_peered":"2024-07-04T17:34:36.876108+0000","last_clean":"2024-07-04T17:34:36.876108+0000","last_became_active":"2024-07-04T17:34:34.518368+0000","last_became_peered":"2024-07-04T17:34:34.518368+0000","last_unstale":"2024-07-04T17:34:36.876108+0000","last_undegraded":"2024-07-04T17:34:36.876108+0000","last_fullsized":"2024-07-04T17:34:36.876108+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:58:34.507931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.875956+0000","last_change":"2024-07-04T17:34:35.520630+0000","last_active":"2024-07-04T17:34:36.875956+0000","last_peered":"2024-07-04T17:34:36.875956+0000","last_clean":"2024-07-04T17:34:36.875956+0000","last_became_active":"2024-07-04T17:34:35.520357+0000","last_became_peered":"2024-07-04T17:34:35.520357+0000","last_unstale":"2024-07-04T17:34:36.875956+0000","last_undegraded":"2024-07-04T17:34:36.875956+0000","last_fullsized":"2024-07-04T17:34:36.875956+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:46:37.840612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876189+0000","last_change":"2024-07-04T17:34:34.518104+0000","last_active":"2024-07-04T17:34:36.876189+0000","last_peered":"2024-07-04T17:34:36.876189+0000","last_clean":"2024-07-04T17:34:36.876189+0000","last_became_active":"2024-07-04T17:34:34.513906+0000","last_became_peered":"2024-07-04T17:34:34.513906+0000","last_unstale":"2024-07-04T17:34:36.876189+0000","last_undegraded":"2024-07-04T17:34:36.876189+0000","last_fullsized":"2024-07-04T17:34:36.876189+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:43:24.362777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:37.149307+0000","last_change":"2024-07-04T17:34:35.538841+0000","last_active":"2024-07-04T17:34:37.149307+0000","last_peered":"2024-07-04T17:34:37.149307+0000","last_clean":"2024-07-04T17:34:37.149307+0000","last_became_active":"2024-07-04T17:34:35.538551+0000","last_became_peered":"2024-07-04T17:34:35.538551+0000","last_unstale":"2024-07-04T17:34:37.149307+0000","last_undegraded":"2024-07-04T17:34:37.149307+0000","last_fullsized":"2024-07-04T17:34:37.149307+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:02:46.435969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876055+0000","last_change":"2024-07-04T17:34:34.514509+0000","last_active":"2024-07-04T17:34:36.876055+0000","last_peered":"2024-07-04T17:34:36.876055+0000","last_clean":"2024-07-04T17:34:36.876055+0000","last_became_active":"2024-07-04T17:34:34.511266+0000","last_became_peered":"2024-07-04T17:34:34.511266+0000","last_unstale":"2024-07-04T17:34:36.876055+0000","last_undegraded":"2024-07-04T17:34:36.876055+0000","last_fullsized":"2024-07-04T17:34:36.876055+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:18:52.279136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.513594+0000","last_change":"2024-07-04T17:34:35.515886+0000","last_active":"2024-07-04T17:34:36.513594+0000","last_peered":"2024-07-04T17:34:36.513594+0000","last_clean":"2024-07-04T17:34:36.513594+0000","last_became_active":"2024-07-04T17:34:35.515383+0000","last_became_peered":"2024-07-04T17:34:35.515383+0000","last_unstale":"2024-07-04T17:34:36.513594+0000","last_undegraded":"2024-07-04T17:34:36.513594+0000","last_fullsized":"2024-07-04T17:34:36.513594+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:04:52.389960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.513082+0000","last_change":"2024-07-04T17:34:34.515873+0000","last_active":"2024-07-04T17:34:36.513082+0000","last_peered":"2024-07-04T17:34:36.513082+0000","last_clean":"2024-07-04T17:34:36.513082+0000","last_became_active":"2024-07-04T17:34:34.515559+0000","last_became_peered":"2024-07-04T17:34:34.515559+0000","last_unstale":"2024-07-04T17:34:36.513082+0000","last_undegraded":"2024-07-04T17:34:36.513082+0000","last_fullsized":"2024-07-04T17:34:36.513082+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:21:52.711984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"122'115","reported_seq":541,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-07-04T17:34:36.876189+0000","last_change":"2024-07-04T17:26:04.953559+0000","last_active":"2024-07-04T17:34:36.876189+0000","last_peered":"2024-07-04T17:34:36.876189+0000","last_clean":"2024-07-04T17:34:36.876189+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:34:36.876189+0000","last_undegraded":"2024-07-04T17:34:36.876189+0000","last_fullsized":"2024-07-04T17:34:36.876189+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_clean_scrub_stamp":"2024-07-04T17:26:03.933432+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.064181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1630,"num_write":187,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1630,"num_write":187,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738469,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36724,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335116,"statfs":{"total":96636764160,"available":96599158784,"internally_reserved":0,"allocated":1417216,"data_stored":560633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738470,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37308,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334532,"statfs":{"total":96636764160,"available":96598560768,"internally_reserved":0,"allocated":2011136,"data_stored":1151883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738468,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37324,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334516,"statfs":{"total":96636764160,"available":96598544384,"internally_reserved":0,"allocated":2027520,"data_stored":1152647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:34:40.834 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-04T17:34:40.834 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-07-04T17:34:41.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.176+0000 7fe9b5000640 1 -- 172.21.15.42:0/4189463299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9b00a9fa0 msgr2=0x7fe9b00a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:41.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.176+0000 7fe9b5000640 1 --2- 172.21.15.42:0/4189463299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9b00a9fa0 0x7fe9b00a94d0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0005fd0 tx=0x7fe9a002aac0 comp rx=0 tx=0).stop 2024-07-04T17:34:41.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.176+0000 7fe9b5000640 1 -- 172.21.15.42:0/4189463299 shutdown_connections 2024-07-04T17:34:41.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.176+0000 7fe9b5000640 1 --2- 172.21.15.42:0/4189463299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9b00a9fa0 0x7fe9b00a94d0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:41.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.176+0000 7fe9b5000640 1 -- 172.21.15.42:0/4189463299 >> 172.21.15.42:0/4189463299 conn(0x7fe9b0070eb0 msgr2=0x7fe9b00b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:41.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.177+0000 7fe9b5000640 1 -- 172.21.15.42:0/4189463299 shutdown_connections 2024-07-04T17:34:41.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.177+0000 7fe9b5000640 1 -- 172.21.15.42:0/4189463299 wait complete. 2024-07-04T17:34:41.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.179+0000 7fe9b5000640 1 Processor -- start 2024-07-04T17:34:41.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.179+0000 7fe9b5000640 1 -- start start 2024-07-04T17:34:41.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.180+0000 7fe9b5000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9b00a9fa0 0x7fe9b006c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:41.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.180+0000 7fe9b5000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b013e190 con 0x7fe9b00a9fa0 2024-07-04T17:34:41.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.181+0000 7fe9af400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9b00a9fa0 0x7fe9b006c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:41.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.181+0000 7fe9af400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9b00a9fa0 0x7fe9b006c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48370/0 (socket says 172.21.15.42:48370) 2024-07-04T17:34:41.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.181+0000 7fe9af400640 1 -- 172.21.15.42:0/3909105829 learned_addr learned my addr 172.21.15.42:0/3909105829 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:41.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.181+0000 7fe9af400640 1 -- 172.21.15.42:0/3909105829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b01449b0 con 0x7fe9b00a9fa0 2024-07-04T17:34:41.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.181+0000 7fe9af400640 1 --2- 172.21.15.42:0/3909105829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9b00a9fa0 0x7fe9b006c2c0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0006010 tx=0x7fe9a0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:41.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.182+0000 7fe9a7e00640 1 -- 172.21.15.42:0/3909105829 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9a0044070 con 0x7fe9b00a9fa0 2024-07-04T17:34:41.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.182+0000 7fe9a7e00640 1 -- 172.21.15.42:0/3909105829 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9a0004250 con 0x7fe9b00a9fa0 2024-07-04T17:34:41.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.182+0000 7fe9b5000640 1 -- 172.21.15.42:0/3909105829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9b00acd50 con 0x7fe9b00a9fa0 2024-07-04T17:34:41.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.182+0000 7fe9a7e00640 1 -- 172.21.15.42:0/3909105829 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9a0004590 con 0x7fe9b00a9fa0 2024-07-04T17:34:41.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.182+0000 7fe9b5000640 1 -- 172.21.15.42:0/3909105829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9b01445f0 con 0x7fe9b00a9fa0 2024-07-04T17:34:41.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.183+0000 7fe9b5000640 1 -- 172.21.15.42:0/3909105829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9b013e860 con 0x7fe9b00a9fa0 2024-07-04T17:34:41.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.183+0000 7fe9a7e00640 1 -- 172.21.15.42:0/3909105829 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7fe9a0047070 con 0x7fe9b00a9fa0 2024-07-04T17:34:41.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.184+0000 7fe9a7e00640 1 --2- 172.21.15.42:0/3909105829 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fe98c03d4f0 0x7fe98c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:41.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.184+0000 7fe9a7e00640 1 -- 172.21.15.42:0/3909105829 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 3439+0+0 (secure 0 0 0) 0x7fe9a003c070 con 0x7fe9b00a9fa0 2024-07-04T17:34:41.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.184+0000 7fe9a6a00640 1 --2- 172.21.15.42:0/3909105829 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fe98c03d4f0 0x7fe98c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:41.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.185+0000 7fe9a6a00640 1 --2- 172.21.15.42:0/3909105829 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fe98c03d4f0 0x7fe98c03fa00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0074470 tx=0x7fe9980079f0 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:41.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.189+0000 7fe9a7e00640 1 -- 172.21.15.42:0/3909105829 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fe9a0032030 con 0x7fe9b00a9fa0 2024-07-04T17:34:41.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.472+0000 7fe9b5000640 1 -- 172.21.15.42:0/3909105829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fe9b0068e30 con 0x7fe9b00a9fa0 2024-07-04T17:34:41.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.475+0000 7fe9a7e00640 1 -- 172.21.15.42:0/3909105829 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7fe9a0047360 con 0x7fe9b00a9fa0 2024-07-04T17:34:41.475 INFO:teuthology.orchestra.run.smithi042.stdout:[client.0] 2024-07-04T17:34:41.475 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQAO24ZmUMmYABAAh4pNlkb9JsHQtK8uCJFQCA== 2024-07-04T17:34:41.475 INFO:teuthology.orchestra.run.smithi042.stdout: caps mds = "allow" 2024-07-04T17:34:41.475 INFO:teuthology.orchestra.run.smithi042.stdout: caps mon = "allow r" 2024-07-04T17:34:41.475 INFO:teuthology.orchestra.run.smithi042.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-07-04T17:34:41.475 INFO:teuthology.orchestra.run.smithi042.stderr:updated caps for client.0 2024-07-04T17:34:41.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.477+0000 7fe9b5000640 1 -- 172.21.15.42:0/3909105829 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fe98c03d4f0 msgr2=0x7fe98c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:41.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.477+0000 7fe9b5000640 1 --2- 172.21.15.42:0/3909105829 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fe98c03d4f0 0x7fe98c03fa00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0074470 tx=0x7fe9980079f0 comp rx=0 tx=0).stop 2024-07-04T17:34:41.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.477+0000 7fe9b5000640 1 -- 172.21.15.42:0/3909105829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9b00a9fa0 msgr2=0x7fe9b006c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:41.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.477+0000 7fe9b5000640 1 --2- 172.21.15.42:0/3909105829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9b00a9fa0 0x7fe9b006c2c0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0006010 tx=0x7fe9a0003f10 comp rx=0 tx=0).stop 2024-07-04T17:34:41.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.477+0000 7fe9b5000640 1 -- 172.21.15.42:0/3909105829 shutdown_connections 2024-07-04T17:34:41.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.477+0000 7fe9b5000640 1 --2- 172.21.15.42:0/3909105829 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fe98c03d4f0 0x7fe98c03fa00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:41.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.477+0000 7fe9b5000640 1 --2- 172.21.15.42:0/3909105829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9b00a9fa0 0x7fe9b006c2c0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:41.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.477+0000 7fe9b5000640 1 -- 172.21.15.42:0/3909105829 >> 172.21.15.42:0/3909105829 conn(0x7fe9b0070eb0 msgr2=0x7fe9b01404f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:41.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.477+0000 7fe9b5000640 1 -- 172.21.15.42:0/3909105829 shutdown_connections 2024-07-04T17:34:41.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.477+0000 7fe9b5000640 1 -- 172.21.15.42:0/3909105829 wait complete. 2024-07-04T17:34:41.506 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:34:41.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.782+0000 7f811f200640 1 -- 172.21.15.42:0/1302911195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81200b1760 msgr2=0x7f81200b8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:41.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.782+0000 7f811f200640 1 --2- 172.21.15.42:0/1302911195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81200b1760 0x7f81200b8da0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f81100097b0 tx=0x7f811002a4a0 comp rx=0 tx=0).stop 2024-07-04T17:34:41.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.782+0000 7f811f200640 1 -- 172.21.15.42:0/1302911195 shutdown_connections 2024-07-04T17:34:41.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.782+0000 7f811f200640 1 --2- 172.21.15.42:0/1302911195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81200b1760 0x7f81200b8da0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:41.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.782+0000 7f811f200640 1 -- 172.21.15.42:0/1302911195 >> 172.21.15.42:0/1302911195 conn(0x7f81200bebc0 msgr2=0x7f81200befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:41.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.782+0000 7f811f200640 1 -- 172.21.15.42:0/1302911195 shutdown_connections 2024-07-04T17:34:41.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.782+0000 7f811f200640 1 -- 172.21.15.42:0/1302911195 wait complete. 2024-07-04T17:34:41.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.784+0000 7f811f200640 1 Processor -- start 2024-07-04T17:34:41.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.784+0000 7f811f200640 1 -- start start 2024-07-04T17:34:41.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.785+0000 7f811f200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f812006c020 0x7f812014db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:41.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.785+0000 7f811f200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f812014c030 con 0x7f812006c020 2024-07-04T17:34:41.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.786+0000 7f811de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f812006c020 0x7f812014db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:41.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.786+0000 7f811de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f812006c020 0x7f812014db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48390/0 (socket says 172.21.15.42:48390) 2024-07-04T17:34:41.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.786+0000 7f811de00640 1 -- 172.21.15.42:0/1575045132 learned_addr learned my addr 172.21.15.42:0/1575045132 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:41.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.786+0000 7f811de00640 1 -- 172.21.15.42:0/1575045132 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8120072ec0 con 0x7f812006c020 2024-07-04T17:34:41.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.786+0000 7f811de00640 1 --2- 172.21.15.42:0/1575045132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f812006c020 0x7f812014db90 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f811002aa90 tx=0x7f8110033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:41.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.787+0000 7f8116a00640 1 -- 172.21.15.42:0/1575045132 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81100319b0 con 0x7f812006c020 2024-07-04T17:34:41.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.787+0000 7f8116a00640 1 -- 172.21.15.42:0/1575045132 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8110031c40 con 0x7f812006c020 2024-07-04T17:34:41.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.787+0000 7f8116a00640 1 -- 172.21.15.42:0/1575045132 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81100330d0 con 0x7f812006c020 2024-07-04T17:34:41.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.787+0000 7f811f200640 1 -- 172.21.15.42:0/1575045132 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81200739a0 con 0x7f812006c020 2024-07-04T17:34:41.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.787+0000 7f811f200640 1 -- 172.21.15.42:0/1575045132 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81201e0bb0 con 0x7f812006c020 2024-07-04T17:34:41.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.788+0000 7f811f200640 1 -- 172.21.15.42:0/1575045132 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f812014f320 con 0x7f812006c020 2024-07-04T17:34:41.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.788+0000 7f8116a00640 1 -- 172.21.15.42:0/1575045132 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f8110033270 con 0x7f812006c020 2024-07-04T17:34:41.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.789+0000 7f8116a00640 1 --2- 172.21.15.42:0/1575045132 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f80fc03d4f0 0x7f80fc03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:41.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.789+0000 7f8116a00640 1 -- 172.21.15.42:0/1575045132 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 3439+0+0 (secure 0 0 0) 0x7f8110072da0 con 0x7f812006c020 2024-07-04T17:34:41.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.789+0000 7f8115400640 1 --2- 172.21.15.42:0/1575045132 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f80fc03d4f0 0x7f80fc03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:41.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.790+0000 7f8115400640 1 --2- 172.21.15.42:0/1575045132 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f80fc03d4f0 0x7f80fc03fa00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8108007fd0 tx=0x7f8108007920 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:41.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:41.793+0000 7f8116a00640 1 -- 172.21.15.42:0/1575045132 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f811003f030 con 0x7f812006c020 2024-07-04T17:34:42.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.035+0000 7f811f200640 1 -- 172.21.15.42:0/1575045132 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8120073b70 con 0x7f812006c020 2024-07-04T17:34:42.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.036+0000 7f8116a00640 1 -- 172.21.15.42:0/1575045132 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+6059 (secure 0 0 0) 0x7f81200b92e0 con 0x7f812006c020 2024-07-04T17:34:42.036 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:34:42.037 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":108,"btime":"2024-07-04T17:34:38:514215+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5803,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/1184311185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":1184311185},{"type":"v1","addr":"172.21.15.42:6825","nonce":1184311185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103},{"gid":5808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/1815554749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1815554749},{"type":"v1","addr":"172.21.15.42:6827","nonce":1815554749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":5809,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/2027504764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2027504764},{"type":"v1","addr":"172.21.15.42:6829","nonce":2027504764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":5817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/4064998265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":4064998265},{"type":"v1","addr":"172.21.15.42:6831","nonce":4064998265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:34:35.049685+0000","modified":"2024-07-04T17:34:38.110000+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5818},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5818":{"gid":5818,"name":"d","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6833/1863956336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1863956336},{"type":"v1","addr":"172.21.15.42:6833","nonce":1863956336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5818,"qdb_cluster":[5818]},"id":12}]} 2024-07-04T17:34:42.037 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 108 2024-07-04T17:34:42.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.039+0000 7f811f200640 1 -- 172.21.15.42:0/1575045132 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f80fc03d4f0 msgr2=0x7f80fc03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:42.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.039+0000 7f811f200640 1 --2- 172.21.15.42:0/1575045132 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f80fc03d4f0 0x7f80fc03fa00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8108007fd0 tx=0x7f8108007920 comp rx=0 tx=0).stop 2024-07-04T17:34:42.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.039+0000 7f811f200640 1 -- 172.21.15.42:0/1575045132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f812006c020 msgr2=0x7f812014db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:42.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.039+0000 7f811f200640 1 --2- 172.21.15.42:0/1575045132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f812006c020 0x7f812014db90 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f811002aa90 tx=0x7f8110033ad0 comp rx=0 tx=0).stop 2024-07-04T17:34:42.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.039+0000 7f811f200640 1 -- 172.21.15.42:0/1575045132 shutdown_connections 2024-07-04T17:34:42.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.039+0000 7f811f200640 1 --2- 172.21.15.42:0/1575045132 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f80fc03d4f0 0x7f80fc03fa00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:42.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.039+0000 7f811f200640 1 --2- 172.21.15.42:0/1575045132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f812006c020 0x7f812014db90 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:42.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.039+0000 7f811f200640 1 -- 172.21.15.42:0/1575045132 >> 172.21.15.42:0/1575045132 conn(0x7f81200bebc0 msgr2=0x7f81200befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:42.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.039+0000 7f811f200640 1 -- 172.21.15.42:0/1575045132 shutdown_connections 2024-07-04T17:34:42.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.039+0000 7f811f200640 1 -- 172.21.15.42:0/1575045132 wait complete. 2024-07-04T17:34:42.565 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 108, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-04T17:34:35.049685+0000', 'modified': '2024-07-04T17:34:38.110000+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5818}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5818': {'gid': 5818, 'name': 'd', 'rank': 0, 'incarnation': 105, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.42:6833/1863956336', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6832', 'nonce': 1863956336}, {'type': 'v1', 'addr': '172.21.15.42:6833', 'nonce': 1863956336}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [25], 'metadata_pool': 24, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5818, 'qdb_cluster': [5818]} 2024-07-04T17:34:42.565 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-04T17:34:42.565 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-07-04T17:34:42.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.864+0000 7f5778a00640 1 -- 172.21.15.42:0/955789590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5758003a50 msgr2=0x7f5758003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:42.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.864+0000 7f5778a00640 1 --2- 172.21.15.42:0/955789590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5758003a50 0x7f5758003ea0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f5764009ad0 tx=0x7f5758005830 comp rx=0 tx=0).stop 2024-07-04T17:34:42.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.864+0000 7f5778a00640 1 -- 172.21.15.42:0/955789590 shutdown_connections 2024-07-04T17:34:42.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.864+0000 7f5778a00640 1 --2- 172.21.15.42:0/955789590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5758003a50 0x7f5758003ea0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:42.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.864+0000 7f5778a00640 1 --2- 172.21.15.42:0/955789590 >> v2:172.21.15.42:3300/0 conn(0x7f57740bad70 0x7f577406c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:42.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.864+0000 7f5778a00640 1 -- 172.21.15.42:0/955789590 >> 172.21.15.42:0/955789590 conn(0x7f57740bebc0 msgr2=0x7f57740befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:42.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.864+0000 7f5778a00640 1 -- 172.21.15.42:0/955789590 shutdown_connections 2024-07-04T17:34:42.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.864+0000 7f5778a00640 1 -- 172.21.15.42:0/955789590 wait complete. 2024-07-04T17:34:42.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.866+0000 7f5778a00640 1 Processor -- start 2024-07-04T17:34:42.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.866+0000 7f5778a00640 1 -- start start 2024-07-04T17:34:42.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.868+0000 7f5778a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5758003a50 0x7f577413ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:42.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.868+0000 7f5778a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f57740b27c0 con 0x7f5758003a50 2024-07-04T17:34:42.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.868+0000 7f5772800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5758003a50 0x7f577413ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:42.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.868+0000 7f5772800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5758003a50 0x7f577413ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48408/0 (socket says 172.21.15.42:48408) 2024-07-04T17:34:42.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.868+0000 7f5772800640 1 -- 172.21.15.42:0/2748219706 learned_addr learned my addr 172.21.15.42:0/2748219706 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:42.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.868+0000 7f5772800640 1 -- 172.21.15.42:0/2748219706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57740be4b0 con 0x7f5758003a50 2024-07-04T17:34:42.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.869+0000 7f5772800640 1 --2- 172.21.15.42:0/2748219706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5758003a50 0x7f577413ed90 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f5764009500 tx=0x7f5764033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:42.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.869+0000 7f576b400640 1 -- 172.21.15.42:0/2748219706 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5764038040 con 0x7f5758003a50 2024-07-04T17:34:42.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.869+0000 7f576b400640 1 -- 172.21.15.42:0/2748219706 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5764004240 con 0x7f5758003a50 2024-07-04T17:34:42.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.869+0000 7f576b400640 1 -- 172.21.15.42:0/2748219706 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5764033830 con 0x7f5758003a50 2024-07-04T17:34:42.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.869+0000 7f5778a00640 1 -- 172.21.15.42:0/2748219706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5774137600 con 0x7f5758003a50 2024-07-04T17:34:42.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.869+0000 7f5778a00640 1 -- 172.21.15.42:0/2748219706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5774136fb0 con 0x7f5758003a50 2024-07-04T17:34:42.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.871+0000 7f576b400640 1 -- 172.21.15.42:0/2748219706 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f5764045060 con 0x7f5758003a50 2024-07-04T17:34:42.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.871+0000 7f576b400640 1 --2- 172.21.15.42:0/2748219706 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f575003d4f0 0x7f575003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:42.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.872+0000 7f576b400640 1 -- 172.21.15.42:0/2748219706 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 3439+0+0 (secure 0 0 0) 0x7f5764071ed0 con 0x7f5758003a50 2024-07-04T17:34:42.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.872+0000 7f5769e00640 1 --2- 172.21.15.42:0/2748219706 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f575003d4f0 0x7f575003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:42.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.872+0000 7f5769e00640 1 --2- 172.21.15.42:0/2748219706 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f575003d4f0 0x7f575003fa00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f575c007860 tx=0x7f575c001530 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:42.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.873+0000 7f5780419740 1 Processor -- start 2024-07-04T17:34:42.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.874+0000 7f5780419740 1 -- start start 2024-07-04T17:34:42.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.875+0000 7f5780419740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559da3e70270 0x559da4216d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:42.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.875+0000 7f5780419740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559da41875b0 con 0x559da3e70270 2024-07-04T17:34:42.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.875+0000 7f5772800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559da3e70270 0x559da4216d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:42.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.875+0000 7f5772800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559da3e70270 0x559da4216d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48412/0 (socket says 172.21.15.42:48412) 2024-07-04T17:34:42.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.875+0000 7f5772800640 1 -- 172.21.15.42:0/2798349244 learned_addr learned my addr 172.21.15.42:0/2798349244 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:42.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.875+0000 7f5772800640 1 -- 172.21.15.42:0/2798349244 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559da3edb850 con 0x559da3e70270 2024-07-04T17:34:42.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.875+0000 7f5772800640 1 --2- 172.21.15.42:0/2798349244 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559da3e70270 0x559da4216d30 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f5764047c70 tx=0x7f5764031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd64ef4d85a53813 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:42.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.876+0000 7f5763000640 1 -- 172.21.15.42:0/2798349244 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f576406d070 con 0x559da3e70270 2024-07-04T17:34:42.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.876+0000 7f5763000640 1 -- 172.21.15.42:0/2798349244 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5764069b90 con 0x559da3e70270 2024-07-04T17:34:42.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.876+0000 7f5763000640 1 -- 172.21.15.42:0/2798349244 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5764031b70 con 0x559da3e70270 2024-07-04T17:34:42.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.876+0000 7f5780419740 1 -- 172.21.15.42:0/2798349244 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559da3e70270 msgr2=0x559da4216d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:42.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.876+0000 7f5780419740 1 --2- 172.21.15.42:0/2798349244 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559da3e70270 0x559da4216d30 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f5764047c70 tx=0x7f5764031400 comp rx=0 tx=0).stop 2024-07-04T17:34:42.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.876+0000 7f5780419740 1 -- 172.21.15.42:0/2798349244 shutdown_connections 2024-07-04T17:34:42.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.876+0000 7f5780419740 1 --2- 172.21.15.42:0/2798349244 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559da3e70270 0x559da4216d30 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:42.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.876+0000 7f5780419740 1 -- 172.21.15.42:0/2798349244 >> 172.21.15.42:0/2798349244 conn(0x559da3e643c0 msgr2=0x559da42154d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:42.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.877+0000 7f5780419740 1 -- 172.21.15.42:0/2798349244 shutdown_connections 2024-07-04T17:34:42.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.877+0000 7f5780419740 1 -- 172.21.15.42:0/2798349244 wait complete. 2024-07-04T17:34:42.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.877+0000 7f5780419740 1 -- start start 2024-07-04T17:34:42.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.878+0000 7f5780419740 1 Processor -- start 2024-07-04T17:34:42.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.879+0000 7f5780419740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559da3e70270 0x559da42598c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:42.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.879+0000 7f5780419740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559da417ab90 con 0x559da3e70270 2024-07-04T17:34:42.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.879+0000 7f5780419740 10 client.0 _ll_register_callbacks cb 0x559da411f8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T17:34:42.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.879+0000 7f5772800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559da3e70270 0x559da42598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:42.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.879+0000 7f5772800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559da3e70270 0x559da42598c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48416/0 (socket says 172.21.15.42:48416) 2024-07-04T17:34:42.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.879+0000 7f5772800640 1 -- 172.21.15.42:0/1750605291 learned_addr learned my addr 172.21.15.42:0/1750605291 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:42.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.880+0000 7f5772800640 1 -- 172.21.15.42:0/1750605291 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559da3f526b0 con 0x559da3e70270 2024-07-04T17:34:42.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.880+0000 7f5772800640 1 --2- 172.21.15.42:0/1750605291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559da3e70270 0x559da42598c0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f57640314b0 tx=0x7f5764031da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98f4d94f8227799 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:42.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.880+0000 7f5757400640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-07-04T17:34:42.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.880+0000 7f5757400640 1 -- 172.21.15.42:0/1750605291 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f576406d050 con 0x559da3e70270 2024-07-04T17:34:42.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.880+0000 7f5757400640 1 -- 172.21.15.42:0/1750605291 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 3439+0+0 (secure 0 0 0) 0x7f57640463b0 con 0x559da3e70270 2024-07-04T17:34:42.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.880+0000 7f5780419740 1 -- 172.21.15.42:0/1750605291 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559da41176c0 con 0x559da3e70270 2024-07-04T17:34:42.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.880+0000 7f5757400640 1 -- 172.21.15.42:0/1750605291 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) ==== 24+0+0 (secure 0 0 0) 0x7f57640465f0 con 0x559da3e70270 2024-07-04T17:34:42.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.880+0000 7f5780419740 10 client.5868 fetch_fsmap learned FSMap version 108 2024-07-04T17:34:42.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.880+0000 7f5780419740 1 -- 172.21.15.42:0/1750605291 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) -- 0x559da4232960 con 0x559da3e70270 2024-07-04T17:34:42.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.881+0000 7f5757400640 1 -- 172.21.15.42:0/1750605291 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x559da41176c0 con 0x559da3e70270 2024-07-04T17:34:42.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.881+0000 7f5757400640 1 -- 172.21.15.42:0/1750605291 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 108) ==== 4107+0+0 (secure 0 0 0) 0x7f576403b030 con 0x559da3e70270 2024-07-04T17:34:42.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.881+0000 7f5780419740 10 client.5868 fetch_fsmap finished waiting for FSMap version 108 2024-07-04T17:34:42.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.881+0000 7f5780419740 10 client.5868 resolve_mds: resolved name 'd' to daemon mds.d 2024-07-04T17:34:42.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.881+0000 7f5780419740 20 client.5868 populate_metadata read hostname 'smithi042' 2024-07-04T17:34:42.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.881+0000 7f5780419740 1 --2- 172.21.15.42:0/1750605291 >> [v2:172.21.15.42:6832/1863956336,v1:172.21.15.42:6833/1863956336] conn(0x559da3e473b0 0x559da423e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:42.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.881+0000 7f5780419740 4 client.5868 mds_command: new command op to 5818 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T17:34:42.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.881+0000 7f5780419740 1 -- 172.21.15.42:0/1750605291 --> [v2:172.21.15.42:6832/1863956336,v1:172.21.15.42:6833/1863956336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559da4189500 con 0x559da3e473b0 2024-07-04T17:34:42.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.881+0000 7f5769e00640 1 --2- 172.21.15.42:0/1750605291 >> [v2:172.21.15.42:6832/1863956336,v1:172.21.15.42:6833/1863956336] conn(0x559da3e473b0 0x559da423e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:42.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.882+0000 7f5769e00640 1 --2- 172.21.15.42:0/1750605291 >> [v2:172.21.15.42:6832/1863956336,v1:172.21.15.42:6833/1863956336] conn(0x559da3e473b0 0x559da423e9f0 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=617cfa49a235a1fd server_cookie=ec69681ec6c378f3 in_seq=0 out_seq=0 2024-07-04T17:34:42.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.882+0000 7f5757400640 10 client.5868 ms_handle_connect on v2:172.21.15.42:6832/1863956336 2024-07-04T17:34:42.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.883+0000 7f5757400640 1 -- 172.21.15.42:0/1750605291 <== mds.0 v2:172.21.15.42:6832/1863956336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559da4189500 con 0x559da3e473b0 2024-07-04T17:34:42.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.883+0000 7f5757400640 10 client.5868 handle_command_reply: tid=0 2024-07-04T17:34:42.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.883+0000 7f5780419740 1 client.5868 shutdown 2024-07-04T17:34:42.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.883+0000 7f5780419740 1 -- 172.21.15.42:0/1750605291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559da3e70270 msgr2=0x559da42598c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:42.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.883+0000 7f5780419740 1 --2- 172.21.15.42:0/1750605291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559da3e70270 0x559da42598c0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f57640314b0 tx=0x7f5764031da0 comp rx=0 tx=0).stop 2024-07-04T17:34:42.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.883+0000 7f5780419740 1 -- 172.21.15.42:0/1750605291 shutdown_connections 2024-07-04T17:34:42.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.883+0000 7f5780419740 1 --2- 172.21.15.42:0/1750605291 >> [v2:172.21.15.42:6832/1863956336,v1:172.21.15.42:6833/1863956336] conn(0x559da3e473b0 0x559da423e9f0 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:42.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.883+0000 7f5780419740 1 --2- 172.21.15.42:0/1750605291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559da3e70270 0x559da42598c0 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:42.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.883+0000 7f5780419740 1 -- 172.21.15.42:0/1750605291 >> 172.21.15.42:0/1750605291 conn(0x559da3e643c0 msgr2=0x559da4242f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:42.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.883+0000 7f5757400640 0 client.5868 ms_handle_reset on v2:172.21.15.42:6832/1863956336 2024-07-04T17:34:42.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.883+0000 7f5780419740 1 -- 172.21.15.42:0/1750605291 shutdown_connections 2024-07-04T17:34:42.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.883+0000 7f5780419740 1 -- 172.21.15.42:0/1750605291 wait complete. 2024-07-04T17:34:42.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.883+0000 7f5780419740 20 client.5868 trim_cache size 0 max 16384 2024-07-04T17:34:42.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.903+0000 7f5780419740 1 Processor -- start 2024-07-04T17:34:42.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.903+0000 7f5780419740 1 -- start start 2024-07-04T17:34:42.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.904+0000 7f5780419740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559da3e70270 0x559da4231b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:42.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.905+0000 7f5780419740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5774139130 con 0x559da3e70270 2024-07-04T17:34:42.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.905+0000 7f5772800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559da3e70270 0x559da4231b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:42.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.905+0000 7f5772800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559da3e70270 0x559da4231b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48432/0 (socket says 172.21.15.42:48432) 2024-07-04T17:34:42.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.905+0000 7f5772800640 1 -- 172.21.15.42:0/958988455 learned_addr learned my addr 172.21.15.42:0/958988455 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:42.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.905+0000 7f5772800640 1 -- 172.21.15.42:0/958988455 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559da42170a0 con 0x559da3e70270 2024-07-04T17:34:42.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.906+0000 7f5772800640 1 --2- 172.21.15.42:0/958988455 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559da3e70270 0x559da4231b80 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f576402b660 tx=0x7f57640036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3640326208577146 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:42.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.906+0000 7f5762600640 1 -- 172.21.15.42:0/958988455 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f576406d040 con 0x559da3e70270 2024-07-04T17:34:42.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.906+0000 7f5762600640 1 -- 172.21.15.42:0/958988455 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f576405a070 con 0x559da3e70270 2024-07-04T17:34:42.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.906+0000 7f5762600640 1 -- 172.21.15.42:0/958988455 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5764070b90 con 0x559da3e70270 2024-07-04T17:34:42.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.906+0000 7f5780419740 1 -- 172.21.15.42:0/958988455 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559da3e70270 msgr2=0x559da4231b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:42.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.906+0000 7f5780419740 1 --2- 172.21.15.42:0/958988455 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559da3e70270 0x559da4231b80 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f576402b660 tx=0x7f57640036f0 comp rx=0 tx=0).stop 2024-07-04T17:34:42.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.907+0000 7f5780419740 1 -- 172.21.15.42:0/958988455 shutdown_connections 2024-07-04T17:34:42.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.907+0000 7f5780419740 1 --2- 172.21.15.42:0/958988455 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559da3e70270 0x559da4231b80 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:42.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.907+0000 7f5780419740 1 -- 172.21.15.42:0/958988455 >> 172.21.15.42:0/958988455 conn(0x559da3e643c0 msgr2=0x559da420c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:42.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.907+0000 7f5780419740 1 -- 172.21.15.42:0/958988455 shutdown_connections 2024-07-04T17:34:42.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.907+0000 7f5780419740 1 -- 172.21.15.42:0/958988455 wait complete. 2024-07-04T17:34:42.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.907+0000 7f5780419740 1 -- start start 2024-07-04T17:34:42.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.908+0000 7f5780419740 1 Processor -- start 2024-07-04T17:34:42.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.909+0000 7f5780419740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559da3e70270 0x559da4212b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:42.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.909+0000 7f5780419740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559da4176b50 con 0x559da3e70270 2024-07-04T17:34:42.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.909+0000 7f5780419740 10 client.0 _ll_register_callbacks cb 0x7f57640699b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T17:34:42.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.909+0000 7f5772800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559da3e70270 0x559da4212b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:42.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.909+0000 7f5772800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559da3e70270 0x559da4212b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48448/0 (socket says 172.21.15.42:48448) 2024-07-04T17:34:42.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.909+0000 7f5772800640 1 -- 172.21.15.42:0/905750710 learned_addr learned my addr 172.21.15.42:0/905750710 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:42.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.910+0000 7f5772800640 1 -- 172.21.15.42:0/905750710 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559da42644f0 con 0x559da3e70270 2024-07-04T17:34:42.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.910+0000 7f5772800640 1 --2- 172.21.15.42:0/905750710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559da3e70270 0x559da4212b90 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f57640318d0 tx=0x7f5764070dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cedaae24b25668b8 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:42.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.910+0000 7f5757400640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-07-04T17:34:42.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.910+0000 7f5757400640 1 -- 172.21.15.42:0/905750710 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5764056070 con 0x559da3e70270 2024-07-04T17:34:42.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.910+0000 7f5757400640 1 -- 172.21.15.42:0/905750710 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 3439+0+0 (secure 0 0 0) 0x7f5764069640 con 0x559da3e70270 2024-07-04T17:34:42.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.910+0000 7f5780419740 1 -- 172.21.15.42:0/905750710 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f576405a070 con 0x559da3e70270 2024-07-04T17:34:42.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.910+0000 7f5757400640 1 -- 172.21.15.42:0/905750710 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) ==== 24+0+0 (secure 0 0 0) 0x7f576403baa0 con 0x559da3e70270 2024-07-04T17:34:42.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.910+0000 7f5780419740 10 client.5870 fetch_fsmap learned FSMap version 108 2024-07-04T17:34:42.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.910+0000 7f5780419740 1 -- 172.21.15.42:0/905750710 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) -- 0x559da42644f0 con 0x559da3e70270 2024-07-04T17:34:42.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.911+0000 7f5757400640 1 -- 172.21.15.42:0/905750710 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f576405a070 con 0x559da3e70270 2024-07-04T17:34:42.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.911+0000 7f5757400640 1 -- 172.21.15.42:0/905750710 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 108) ==== 4107+0+0 (secure 0 0 0) 0x7f5764003970 con 0x559da3e70270 2024-07-04T17:34:42.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.911+0000 7f5780419740 10 client.5870 fetch_fsmap finished waiting for FSMap version 108 2024-07-04T17:34:42.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.911+0000 7f5780419740 10 client.5870 resolve_mds: resolved name 'd' to daemon mds.d 2024-07-04T17:34:42.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.911+0000 7f5780419740 20 client.5870 populate_metadata read hostname 'smithi042' 2024-07-04T17:34:42.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.911+0000 7f5780419740 1 --2- 172.21.15.42:0/905750710 >> [v2:172.21.15.42:6832/1863956336,v1:172.21.15.42:6833/1863956336] conn(0x559da3e2ee40 0x559da42359a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:42.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.911+0000 7f5780419740 4 client.5870 mds_command: new command op to 5818 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-04T17:34:42.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.911+0000 7f5780419740 1 -- 172.21.15.42:0/905750710 --> [v2:172.21.15.42:6832/1863956336,v1:172.21.15.42:6833/1863956336] -- command(tid 0: {"prefix": "status"}) -- 0x559da423b340 con 0x559da3e2ee40 2024-07-04T17:34:42.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.911+0000 7f5769e00640 1 --2- 172.21.15.42:0/905750710 >> [v2:172.21.15.42:6832/1863956336,v1:172.21.15.42:6833/1863956336] conn(0x559da3e2ee40 0x559da42359a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:42.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.912+0000 7f5769e00640 1 --2- 172.21.15.42:0/905750710 >> [v2:172.21.15.42:6832/1863956336,v1:172.21.15.42:6833/1863956336] conn(0x559da3e2ee40 0x559da42359a0 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9118da7428fae1e8 server_cookie=9bafc28cf15eceff in_seq=0 out_seq=0 2024-07-04T17:34:42.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.912+0000 7f5757400640 10 client.5870 ms_handle_connect on v2:172.21.15.42:6832/1863956336 2024-07-04T17:34:42.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.912+0000 7f5757400640 1 -- 172.21.15.42:0/905750710 <== mds.0 v2:172.21.15.42:6832/1863956336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x559da423b340 con 0x559da3e2ee40 2024-07-04T17:34:42.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.912+0000 7f5757400640 10 client.5870 handle_command_reply: tid=0 2024-07-04T17:34:42.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.912+0000 7f5780419740 1 client.5870 shutdown 2024-07-04T17:34:42.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.912+0000 7f5780419740 1 -- 172.21.15.42:0/905750710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559da3e70270 msgr2=0x559da4212b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:42.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.912+0000 7f5780419740 1 --2- 172.21.15.42:0/905750710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559da3e70270 0x559da4212b90 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f57640318d0 tx=0x7f5764070dc0 comp rx=0 tx=0).stop 2024-07-04T17:34:42.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.912+0000 7f5780419740 1 -- 172.21.15.42:0/905750710 shutdown_connections 2024-07-04T17:34:42.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.912+0000 7f5780419740 1 --2- 172.21.15.42:0/905750710 >> [v2:172.21.15.42:6832/1863956336,v1:172.21.15.42:6833/1863956336] conn(0x559da3e2ee40 0x559da42359a0 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:42.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.912+0000 7f5780419740 1 --2- 172.21.15.42:0/905750710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559da3e70270 0x559da4212b90 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:42.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.912+0000 7f5780419740 1 -- 172.21.15.42:0/905750710 >> 172.21.15.42:0/905750710 conn(0x559da3e643c0 msgr2=0x559da4242430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:42.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.912+0000 7f5757400640 0 client.5870 ms_handle_reset on v2:172.21.15.42:6832/1863956336 2024-07-04T17:34:42.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.913+0000 7f5780419740 1 -- 172.21.15.42:0/905750710 shutdown_connections 2024-07-04T17:34:42.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.913+0000 7f5780419740 1 -- 172.21.15.42:0/905750710 wait complete. 2024-07-04T17:34:42.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.913+0000 7f5780419740 20 client.5870 trim_cache size 0 max 16384 2024-07-04T17:34:42.915 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:34:42.915 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "be4f300a-93ef-4489-b540-37ece39b0aac", 2024-07-04T17:34:42.915 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-07-04T17:34:42.915 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 5818, 2024-07-04T17:34:42.915 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-07-04T17:34:42.915 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-07-04T17:34:42.915 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-07-04T17:34:42.916 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 7.4248847519999996, 2024-07-04T17:34:42.916 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 108, 2024-07-04T17:34:42.916 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 136, 2024-07-04T17:34:42.916 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 136, 2024-07-04T17:34:42.916 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 12.238941672999999 2024-07-04T17:34:42.916 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:34:42.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.913+0000 7f5778a00640 1 -- 172.21.15.42:0/2748219706 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f575003d4f0 msgr2=0x7f575003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:42.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.913+0000 7f5778a00640 1 --2- 172.21.15.42:0/2748219706 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f575003d4f0 0x7f575003fa00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f575c007860 tx=0x7f575c001530 comp rx=0 tx=0).stop 2024-07-04T17:34:42.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.913+0000 7f5778a00640 1 -- 172.21.15.42:0/2748219706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5758003a50 msgr2=0x7f577413ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:42.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.913+0000 7f5778a00640 1 --2- 172.21.15.42:0/2748219706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5758003a50 0x7f577413ed90 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f5764009500 tx=0x7f5764033670 comp rx=0 tx=0).stop 2024-07-04T17:34:42.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.914+0000 7f5778a00640 1 -- 172.21.15.42:0/2748219706 shutdown_connections 2024-07-04T17:34:42.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.914+0000 7f5778a00640 1 --2- 172.21.15.42:0/2748219706 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f575003d4f0 0x7f575003fa00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:42.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.914+0000 7f5778a00640 1 --2- 172.21.15.42:0/2748219706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5758003a50 0x7f577413ed90 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:42.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.914+0000 7f5778a00640 1 -- 172.21.15.42:0/2748219706 >> 172.21.15.42:0/2748219706 conn(0x7f57740bebc0 msgr2=0x7f577413bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:42.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.914+0000 7f5778a00640 1 -- 172.21.15.42:0/2748219706 shutdown_connections 2024-07-04T17:34:42.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:42.914+0000 7f5778a00640 1 -- 172.21.15.42:0/2748219706 wait complete. 2024-07-04T17:34:42.937 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-04T17:34:43.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.224+0000 7fb720200640 1 -- 172.21.15.42:0/4042303834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7180b38d0 msgr2=0x7fb7180b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:43.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.224+0000 7fb720200640 1 --2- 172.21.15.42:0/4042303834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7180b38d0 0x7fb7180b2d20 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fb70c0097b0 tx=0x7fb70c02a530 comp rx=0 tx=0).stop 2024-07-04T17:34:43.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.224+0000 7fb720200640 1 -- 172.21.15.42:0/4042303834 shutdown_connections 2024-07-04T17:34:43.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.224+0000 7fb720200640 1 --2- 172.21.15.42:0/4042303834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7180b38d0 0x7fb7180b2d20 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:43.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.224+0000 7fb720200640 1 -- 172.21.15.42:0/4042303834 >> 172.21.15.42:0/4042303834 conn(0x7fb7180bf700 msgr2=0x7fb7180af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:43.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.224+0000 7fb720200640 1 -- 172.21.15.42:0/4042303834 shutdown_connections 2024-07-04T17:34:43.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.224+0000 7fb720200640 1 -- 172.21.15.42:0/4042303834 wait complete. 2024-07-04T17:34:43.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.226+0000 7fb720200640 1 Processor -- start 2024-07-04T17:34:43.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.227+0000 7fb720200640 1 -- start start 2024-07-04T17:34:43.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.228+0000 7fb720200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7180b01f0 0x7fb7180aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:43.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.228+0000 7fb720200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb71806cf50 con 0x7fb7180b01f0 2024-07-04T17:34:43.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.228+0000 7fb71d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7180b01f0 0x7fb7180aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:43.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.228+0000 7fb71d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7180b01f0 0x7fb7180aec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48466/0 (socket says 172.21.15.42:48466) 2024-07-04T17:34:43.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.228+0000 7fb71d800640 1 -- 172.21.15.42:0/2981947407 learned_addr learned my addr 172.21.15.42:0/2981947407 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:43.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.229+0000 7fb71d800640 1 -- 172.21.15.42:0/2981947407 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb718070d80 con 0x7fb7180b01f0 2024-07-04T17:34:43.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.229+0000 7fb71d800640 1 --2- 172.21.15.42:0/2981947407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7180b01f0 0x7fb7180aec00 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fb70c0098d0 tx=0x7fb70c033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:43.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.229+0000 7fb715600640 1 -- 172.21.15.42:0/2981947407 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb70c031bc0 con 0x7fb7180b01f0 2024-07-04T17:34:43.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.229+0000 7fb715600640 1 -- 172.21.15.42:0/2981947407 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb70c031e50 con 0x7fb7180b01f0 2024-07-04T17:34:43.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.229+0000 7fb720200640 1 -- 172.21.15.42:0/2981947407 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7180a9a70 con 0x7fb7180b01f0 2024-07-04T17:34:43.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.229+0000 7fb715600640 1 -- 172.21.15.42:0/2981947407 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb70c033200 con 0x7fb7180b01f0 2024-07-04T17:34:43.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.230+0000 7fb720200640 1 -- 172.21.15.42:0/2981947407 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7181e0f20 con 0x7fb7180b01f0 2024-07-04T17:34:43.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.231+0000 7fb720200640 1 -- 172.21.15.42:0/2981947407 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7180af140 con 0x7fb7180b01f0 2024-07-04T17:34:43.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.231+0000 7fb715600640 1 -- 172.21.15.42:0/2981947407 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7fb70c0333a0 con 0x7fb7180b01f0 2024-07-04T17:34:43.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.232+0000 7fb715600640 1 --2- 172.21.15.42:0/2981947407 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fb6fc03d4f0 0x7fb6fc03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:43.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.232+0000 7fb715600640 1 -- 172.21.15.42:0/2981947407 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 3439+0+0 (secure 0 0 0) 0x7fb70c072ef0 con 0x7fb7180b01f0 2024-07-04T17:34:43.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.232+0000 7fb71ce00640 1 --2- 172.21.15.42:0/2981947407 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fb6fc03d4f0 0x7fb6fc03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:43.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.233+0000 7fb71ce00640 1 --2- 172.21.15.42:0/2981947407 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fb6fc03d4f0 0x7fb6fc03fa00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb7080015c0 tx=0x7fb708007a90 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:43.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.237+0000 7fb715600640 1 -- 172.21.15.42:0/2981947407 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fb7180af140 con 0x7fb7180b01f0 2024-07-04T17:34:43.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.484+0000 7fb720200640 1 -- 172.21.15.42:0/2981947407 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb7180b2d20 con 0x7fb7180b01f0 2024-07-04T17:34:43.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.485+0000 7fb715600640 1 -- 172.21.15.42:0/2981947407 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v108) ==== 53+0+73 (secure 0 0 0) 0x7fb70c03f030 con 0x7fb7180b01f0 2024-07-04T17:34:43.486 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-04T17:34:43.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.488+0000 7fb720200640 1 -- 172.21.15.42:0/2981947407 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fb6fc03d4f0 msgr2=0x7fb6fc03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:43.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.488+0000 7fb720200640 1 --2- 172.21.15.42:0/2981947407 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fb6fc03d4f0 0x7fb6fc03fa00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb7080015c0 tx=0x7fb708007a90 comp rx=0 tx=0).stop 2024-07-04T17:34:43.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.488+0000 7fb720200640 1 -- 172.21.15.42:0/2981947407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7180b01f0 msgr2=0x7fb7180aec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:43.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.488+0000 7fb720200640 1 --2- 172.21.15.42:0/2981947407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7180b01f0 0x7fb7180aec00 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fb70c0098d0 tx=0x7fb70c033e40 comp rx=0 tx=0).stop 2024-07-04T17:34:43.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.488+0000 7fb720200640 1 -- 172.21.15.42:0/2981947407 shutdown_connections 2024-07-04T17:34:43.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.488+0000 7fb720200640 1 --2- 172.21.15.42:0/2981947407 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fb6fc03d4f0 0x7fb6fc03fa00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:43.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.488+0000 7fb720200640 1 --2- 172.21.15.42:0/2981947407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7180b01f0 0x7fb7180aec00 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:43.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.488+0000 7fb720200640 1 -- 172.21.15.42:0/2981947407 >> 172.21.15.42:0/2981947407 conn(0x7fb7180bf700 msgr2=0x7fb7180be520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:43.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.488+0000 7fb720200640 1 -- 172.21.15.42:0/2981947407 shutdown_connections 2024-07-04T17:34:43.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43.489+0000 7fb720200640 1 -- 172.21.15.42:0/2981947407 wait complete. 2024-07-04T17:34:43.513 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-07-04T17:34:43.513 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-04T17:34:43.513 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2024-07-04T17:34:43.514 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-04T17:34:43.514 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-04T17:34:43.514 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-07-04T17:34:43.514 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-07-04T17:34:43.514 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-04T17:34:43.514 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-04T17:34:43.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:34:43.515 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2024-07-04T17:34:43.570 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-04T17:34:43.570 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-04T17:34:43.570 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-07-04T17:34:43.570 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:34:43.570 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-07-04T17:34:43.570 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:34:43.570 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:34:43.571 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2024-07-04T17:34:43.571 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f0 2024-07-04T17:34:43.571 INFO:teuthology.orchestra.run.smithi042.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:34:43.571 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2024-07-04T17:34:43.571 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f1 2024-07-04T17:34:43.571 INFO:teuthology.orchestra.run.smithi042.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-04T17:34:43.571 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:34:43.571 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f0 2024-07-04T17:34:43.571 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-07-04T17:34:43.571 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 42086sec preferred_lft 42086sec 2024-07-04T17:34:43.571 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::ec4:7aff:febd:1440/64 scope link noprefixroute 2024-07-04T17:34:43.571 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:34:43.571 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:34:43.571 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:34:43.572 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f1 2024-07-04T17:34:43.572 INFO:teuthology.orchestra.run.smithi042.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-04T17:34:43.572 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 32:75:d6:f1:31:e1 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:34:43.572 INFO:teuthology.orchestra.run.smithi042.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-04T17:34:43.572 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:34:43.572 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::8839:a9ff:fe74:17f5/64 scope link 2024-07-04T17:34:43.572 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:34:43.572 INFO:teuthology.orchestra.run.smithi042.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-04T17:34:43.572 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether e6:ab:57:fb:ca:9e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:34:43.572 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::e4ab:57ff:fefb:ca9e/64 scope link 2024-07-04T17:34:43.572 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:34:43.572 INFO:teuthology.orchestra.run.smithi042.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-04T17:34:43.572 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 32:75:d6:f1:31:e1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:34:43.573 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::3075:d6ff:fef1:31e1/64 scope link 2024-07-04T17:34:43.573 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:34:43.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:34:43.573 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-04T17:34:43.648 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-07-04T17:34:43.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:34:43.653 DEBUG:teuthology.orchestra.run.smithi042:> route 2024-07-04T17:34:43.705 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2024-07-04T17:34:43.706 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-04T17:34:43.706 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-04T17:34:43.706 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-04T17:34:43.706 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-04T17:34:43.706 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-04T17:34:43.706 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-07-04T17:34:43.706 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-04T17:34:43.706 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-04T17:34:43.706 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-04T17:34:43.707 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-07-04T17:34:43.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-04T17:34:43.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:34:43.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:34:43.924 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-07-04T17:34:43.973 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-04T17:34:43.973 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2024-07-04T17:34:43.974 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-04T17:34:43.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:34:43.974 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.0 up 2024-07-04T17:34:44.059 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-07-04T17:34:44.059 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-04T17:34:44.059 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-04T17:34:44.118 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-04T17:34:44.118 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-04T17:34:44.118 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-04T17:34:44.172 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:34:44.258 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:34:44.316 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:34:44.316 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:34:44.316 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:34:44.317 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:34:44.317 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:34:44.317 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:34:44.317 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:34:44.317 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:34:44.317 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:34:44.317 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:34:44.317 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:34:44.317 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:34:44.317 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:34:44.317 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:34:44.317 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:34:44.317 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:34:44.318 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:34:44.318 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:34:44.318 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:34:44.318 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:34:44.318 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:34:44.318 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:34:44.318 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:34:44.318 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:34:44.318 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:34:44.318 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:34:44.318 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:34:44.319 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:34:44.319 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:34:44.319 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:34:44.319 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:34:44.319 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:34:44.319 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:34:44.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:34:44.319 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:34:44.375 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-07-04T17:34:44.376 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-07-04T17:34:44.381 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:34:44.449 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:34:44.489 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:34:44.489 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:34:44.489 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:34:44.490 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:34:44.490 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:34:44.490 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:34:44.490 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:34:44.490 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:34:44.490 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:34:44.490 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:34:44.490 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:34:44.490 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:34:44.490 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:34:44.490 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:34:44.490 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:34:44.490 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:34:44.490 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:34:44.491 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:34:44.491 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:34:44.491 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:34:44.491 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:34:44.491 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:34:44.491 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:34:44.491 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:34:44.491 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:34:44.491 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:34:44.491 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:34:44.491 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:34:44.491 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:34:44.492 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:34:44.492 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:34:44.492 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:34:44.492 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:34:44.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:34:44.493 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:34:44.624 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:2024-07-04T17:34:44.624+0000 7f766ae93480 -1 init, newargv = 0x7f764c001080 newargc=15 2024-07-04T17:34:44.624 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[70393]: starting ceph client 2024-07-04T17:34:44.631 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[70393]: starting fuse 2024-07-04T17:34:45.682 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:34:45.839 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:34:45.899 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:34:45.899 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:34:45.899 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:34:45.899 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:34:45.899 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:34:45.899 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:34:45.899 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:34:45.899 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:34:45.899 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:34:45.899 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:34:45.899 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:34:45.899 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:34:45.900 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:34:45.900 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:34:45.900 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:34:45.900 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:34:45.900 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:34:45.900 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:34:45.900 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:34:45.900 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:34:45.900 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:34:45.900 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:34:45.900 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:34:45.900 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:34:45.901 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:34:45.901 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:34:45.901 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:34:45.901 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:34:45.901 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:34:45.901 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:34:45.901 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:34:45.901 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:34:45.901 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:34:45.901 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-07-04T17:34:45.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:34:45.902 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:34:45.959 INFO:teuthology.orchestra.run.smithi042.stdout:42 2024-07-04T17:34:45.959 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-07-04T17:34:45.960 DEBUG:teuthology.orchestra.run.smithi042:> sudo stdin-killer -- python3 -c ' 2024-07-04T17:34:45.960 DEBUG:teuthology.orchestra.run.smithi042:> import glob 2024-07-04T17:34:45.960 DEBUG:teuthology.orchestra.run.smithi042:> import re 2024-07-04T17:34:45.960 DEBUG:teuthology.orchestra.run.smithi042:> import os 2024-07-04T17:34:45.960 DEBUG:teuthology.orchestra.run.smithi042:> import subprocess 2024-07-04T17:34:45.960 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:34:45.960 DEBUG:teuthology.orchestra.run.smithi042:> def _find_admin_socket(client_name): 2024-07-04T17:34:45.960 DEBUG:teuthology.orchestra.run.smithi042:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-07-04T17:34:45.960 DEBUG:teuthology.orchestra.run.smithi042:> files = glob.glob(asok_path) 2024-07-04T17:34:45.960 DEBUG:teuthology.orchestra.run.smithi042:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-07-04T17:34:45.960 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:34:45.960 DEBUG:teuthology.orchestra.run.smithi042:> # Given a non-glob path, it better be there 2024-07-04T17:34:45.961 DEBUG:teuthology.orchestra.run.smithi042:> if "*" not in asok_path: 2024-07-04T17:34:45.961 DEBUG:teuthology.orchestra.run.smithi042:> assert(len(files) == 1) 2024-07-04T17:34:45.961 DEBUG:teuthology.orchestra.run.smithi042:> return files[0] 2024-07-04T17:34:45.961 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:34:45.961 DEBUG:teuthology.orchestra.run.smithi042:> for f in files: 2024-07-04T17:34:45.961 DEBUG:teuthology.orchestra.run.smithi042:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-07-04T17:34:45.961 DEBUG:teuthology.orchestra.run.smithi042:> if os.path.exists("/proc/{0}".format(pid)): 2024-07-04T17:34:45.961 DEBUG:teuthology.orchestra.run.smithi042:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-07-04T17:34:45.961 DEBUG:teuthology.orchestra.run.smithi042:> contents = proc_f.read() 2024-07-04T17:34:45.961 DEBUG:teuthology.orchestra.run.smithi042:> if mountpoint in contents: 2024-07-04T17:34:45.961 DEBUG:teuthology.orchestra.run.smithi042:> return f 2024-07-04T17:34:45.961 DEBUG:teuthology.orchestra.run.smithi042:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-07-04T17:34:45.961 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:34:45.961 DEBUG:teuthology.orchestra.run.smithi042:> print(_find_admin_socket("client.0")) 2024-07-04T17:34:45.962 DEBUG:teuthology.orchestra.run.smithi042:> ' 2024-07-04T17:34:46.142 INFO:teuthology.orchestra.run.smithi042.stdout:/var/run/ceph/ceph-client.0.70393.asok 2024-07-04T17:34:46.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:34:46.161 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.70393.asok 2024-07-04T17:34:46.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:34:46.161 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.70393.asok status 2024-07-04T17:34:46.471 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:34:46.471 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": { 2024-07-04T17:34:46.471 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_sha1": "9dba1e836b4756bf3a32be2a4936f371dff55008", 2024-07-04T17:34:46.471 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.0.0-4667-g9dba1e83 (9dba1e836b4756bf3a32be2a4936f371dff55008) squid (dev)", 2024-07-04T17:34:46.471 INFO:teuthology.orchestra.run.smithi042.stdout: "entity_id": "0", 2024-07-04T17:34:46.471 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": "smithi042", 2024-07-04T17:34:46.471 INFO:teuthology.orchestra.run.smithi042.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-07-04T17:34:46.471 INFO:teuthology.orchestra.run.smithi042.stdout: "pid": "70393", 2024-07-04T17:34:46.471 INFO:teuthology.orchestra.run.smithi042.stdout: "root": "/" 2024-07-04T17:34:46.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:46.472 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_count": 0, 2024-07-04T17:34:46.472 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_pinned_count": 0, 2024-07-04T17:34:46.472 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 5875, 2024-07-04T17:34:46.472 INFO:teuthology.orchestra.run.smithi042.stdout: "inst": { 2024-07-04T17:34:46.472 INFO:teuthology.orchestra.run.smithi042.stdout: "name": { 2024-07-04T17:34:46.472 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "client", 2024-07-04T17:34:46.472 INFO:teuthology.orchestra.run.smithi042.stdout: "num": 5875 2024-07-04T17:34:46.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:46.472 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-07-04T17:34:46.472 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-07-04T17:34:46.472 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-07-04T17:34:46.472 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 143386260 2024-07-04T17:34:46.472 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:46.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:46.473 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-07-04T17:34:46.473 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-07-04T17:34:46.473 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-07-04T17:34:46.473 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 143386260 2024-07-04T17:34:46.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:46.473 INFO:teuthology.orchestra.run.smithi042.stdout: "inst_str": "client.5875 192.168.0.1:0/143386260", 2024-07-04T17:34:46.473 INFO:teuthology.orchestra.run.smithi042.stdout: "addr_str": "192.168.0.1:0/143386260", 2024-07-04T17:34:46.473 INFO:teuthology.orchestra.run.smithi042.stdout: "inode_count": 1, 2024-07-04T17:34:46.473 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_epoch": 108, 2024-07-04T17:34:46.474 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch": 136, 2024-07-04T17:34:46.474 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch_barrier": 0, 2024-07-04T17:34:46.474 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklisted": false, 2024-07-04T17:34:46.474 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs" 2024-07-04T17:34:46.474 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:34:46.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:34:46.492 DEBUG:teuthology.orchestra.run.smithi042:> stat --file-system '--printf=%T 2024-07-04T17:34:46.492 DEBUG:teuthology.orchestra.run.smithi042:> ' -- /home/ubuntu/cephtest/mnt.0 2024-07-04T17:34:46.547 INFO:teuthology.orchestra.run.smithi042.stdout:fuseblk 2024-07-04T17:34:46.548 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-07-04T17:34:46.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:34:46.548 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-04T17:34:46.638 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-07-04T17:34:46.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:46.930+0000 7fdda0a00640 1 -- 172.21.15.42:0/98898811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd9c06cea0 msgr2=0x7fdd9c0b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:46.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:46.930+0000 7fdda0a00640 1 --2- 172.21.15.42:0/98898811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd9c06cea0 0x7fdd9c0b3030 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fdd84005fd0 tx=0x7fdd8402ae80 comp rx=0 tx=0).stop 2024-07-04T17:34:46.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:46.930+0000 7fdda0a00640 1 -- 172.21.15.42:0/98898811 shutdown_connections 2024-07-04T17:34:46.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:46.930+0000 7fdda0a00640 1 --2- 172.21.15.42:0/98898811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd9c06cea0 0x7fdd9c0b3030 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:46.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:46.930+0000 7fdda0a00640 1 -- 172.21.15.42:0/98898811 >> 172.21.15.42:0/98898811 conn(0x7fdd9c0bf700 msgr2=0x7fdd9c0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:46.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:46.930+0000 7fdda0a00640 1 -- 172.21.15.42:0/98898811 shutdown_connections 2024-07-04T17:34:46.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:46.930+0000 7fdda0a00640 1 -- 172.21.15.42:0/98898811 wait complete. 2024-07-04T17:34:46.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:46.932+0000 7fdda0a00640 1 Processor -- start 2024-07-04T17:34:46.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:46.933+0000 7fdda0a00640 1 -- start start 2024-07-04T17:34:46.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:46.934+0000 7fdda0a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd9c06cea0 0x7fdd9c1487b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:46.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:46.934+0000 7fdda0a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9c06ef50 con 0x7fdd9c06cea0 2024-07-04T17:34:46.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:46.934+0000 7fdd99e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd9c06cea0 0x7fdd9c1487b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:46.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:46.934+0000 7fdd99e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd9c06cea0 0x7fdd9c1487b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33938/0 (socket says 172.21.15.42:33938) 2024-07-04T17:34:46.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:46.934+0000 7fdd99e00640 1 -- 172.21.15.42:0/2946103164 learned_addr learned my addr 172.21.15.42:0/2946103164 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:46.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:46.935+0000 7fdd99e00640 1 -- 172.21.15.42:0/2946103164 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd9c141450 con 0x7fdd9c06cea0 2024-07-04T17:34:46.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:46.935+0000 7fdd99e00640 1 --2- 172.21.15.42:0/2946103164 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd9c06cea0 0x7fdd9c1487b0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400b3c0 tx=0x7fdd8402cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:46.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:46.935+0000 7fdd92000640 1 -- 172.21.15.42:0/2946103164 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd84041020 con 0x7fdd9c06cea0 2024-07-04T17:34:46.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:46.935+0000 7fdd92000640 1 -- 172.21.15.42:0/2946103164 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd840095a0 con 0x7fdd9c06cea0 2024-07-04T17:34:46.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:46.935+0000 7fdd92000640 1 -- 172.21.15.42:0/2946103164 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd840098c0 con 0x7fdd9c06cea0 2024-07-04T17:34:46.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:46.935+0000 7fdda0a00640 1 -- 172.21.15.42:0/2946103164 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd9c141940 con 0x7fdd9c06cea0 2024-07-04T17:34:46.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:46.936+0000 7fdda0a00640 1 -- 172.21.15.42:0/2946103164 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd9c148560 con 0x7fdd9c06cea0 2024-07-04T17:34:46.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:46.937+0000 7fdda0a00640 1 -- 172.21.15.42:0/2946103164 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd9c141260 con 0x7fdd9c06cea0 2024-07-04T17:34:46.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:46.937+0000 7fdd92000640 1 -- 172.21.15.42:0/2946103164 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7fdd84044020 con 0x7fdd9c06cea0 2024-07-04T17:34:46.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:46.938+0000 7fdd92000640 1 --2- 172.21.15.42:0/2946103164 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fdd7803d4a0 0x7fdd7803f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:46.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:46.938+0000 7fdd92000640 1 -- 172.21.15.42:0/2946103164 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 3439+0+0 (secure 0 0 0) 0x7fdd84039040 con 0x7fdd9c06cea0 2024-07-04T17:34:46.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:46.938+0000 7fdd99400640 1 --2- 172.21.15.42:0/2946103164 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fdd7803d4a0 0x7fdd7803f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:46.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:46.939+0000 7fdd99400640 1 --2- 172.21.15.42:0/2946103164 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fdd7803d4a0 0x7fdd7803f9b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c001c60 tx=0x7fdd8c007d70 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:46.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:46.943+0000 7fdd92000640 1 -- 172.21.15.42:0/2946103164 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fdd8404b030 con 0x7fdd9c06cea0 2024-07-04T17:34:47.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:47.195+0000 7fdda0a00640 1 -- 172.21.15.42:0/2946103164 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdd9c141b10 con 0x7fdd9c06cea0 2024-07-04T17:34:47.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:47.196+0000 7fdd92000640 1 -- 172.21.15.42:0/2946103164 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v108) ==== 53+0+73 (secure 0 0 0) 0x7fdd84030e80 con 0x7fdd9c06cea0 2024-07-04T17:34:47.196 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-04T17:34:47.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:47.199+0000 7fdda0a00640 1 -- 172.21.15.42:0/2946103164 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fdd7803d4a0 msgr2=0x7fdd7803f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:47.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:47.199+0000 7fdda0a00640 1 --2- 172.21.15.42:0/2946103164 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fdd7803d4a0 0x7fdd7803f9b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c001c60 tx=0x7fdd8c007d70 comp rx=0 tx=0).stop 2024-07-04T17:34:47.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:47.199+0000 7fdda0a00640 1 -- 172.21.15.42:0/2946103164 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd9c06cea0 msgr2=0x7fdd9c1487b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:47.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:47.199+0000 7fdda0a00640 1 --2- 172.21.15.42:0/2946103164 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd9c06cea0 0x7fdd9c1487b0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400b3c0 tx=0x7fdd8402cd70 comp rx=0 tx=0).stop 2024-07-04T17:34:47.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:47.199+0000 7fdda0a00640 1 -- 172.21.15.42:0/2946103164 shutdown_connections 2024-07-04T17:34:47.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:47.199+0000 7fdda0a00640 1 --2- 172.21.15.42:0/2946103164 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fdd7803d4a0 0x7fdd7803f9b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:47.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:47.199+0000 7fdda0a00640 1 --2- 172.21.15.42:0/2946103164 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd9c06cea0 0x7fdd9c1487b0 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:47.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:47.199+0000 7fdda0a00640 1 -- 172.21.15.42:0/2946103164 >> 172.21.15.42:0/2946103164 conn(0x7fdd9c0bf700 msgr2=0x7fdd9c0aa3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:47.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:47.199+0000 7fdda0a00640 1 -- 172.21.15.42:0/2946103164 shutdown_connections 2024-07-04T17:34:47.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:47.199+0000 7fdda0a00640 1 -- 172.21.15.42:0/2946103164 wait complete. 2024-07-04T17:34:47.225 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-07-04T17:34:47.225 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-04T17:34:47.225 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2024-07-04T17:34:47.225 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-07-04T17:34:47.225 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-07-04T17:34:47.225 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-07-04T17:34:47.225 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-07-04T17:34:47.225 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-04T17:34:47.226 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-07-04T17:34:47.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:34:47.226 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2024-07-04T17:34:47.277 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-04T17:34:47.277 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-04T17:34:47.277 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-07-04T17:34:47.277 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:34:47.277 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-07-04T17:34:47.278 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:34:47.278 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:34:47.278 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2024-07-04T17:34:47.278 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f0 2024-07-04T17:34:47.278 INFO:teuthology.orchestra.run.smithi042.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:34:47.278 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2024-07-04T17:34:47.278 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f1 2024-07-04T17:34:47.278 INFO:teuthology.orchestra.run.smithi042.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-04T17:34:47.278 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:34:47.278 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f0 2024-07-04T17:34:47.278 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-07-04T17:34:47.278 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 42083sec preferred_lft 42083sec 2024-07-04T17:34:47.278 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::ec4:7aff:febd:1440/64 scope link noprefixroute 2024-07-04T17:34:47.278 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:34:47.279 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T17:34:47.279 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:34:47.279 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f1 2024-07-04T17:34:47.279 INFO:teuthology.orchestra.run.smithi042.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-07-04T17:34:47.279 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 32:75:d6:f1:31:e1 brd ff:ff:ff:ff:ff:ff 2024-07-04T17:34:47.279 INFO:teuthology.orchestra.run.smithi042.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-07-04T17:34:47.279 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:34:47.279 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::8839:a9ff:fe74:17f5/64 scope link 2024-07-04T17:34:47.279 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:34:47.279 INFO:teuthology.orchestra.run.smithi042.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-04T17:34:47.279 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether e6:ab:57:fb:ca:9e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:34:47.279 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::e4ab:57ff:fefb:ca9e/64 scope link 2024-07-04T17:34:47.279 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:34:47.279 INFO:teuthology.orchestra.run.smithi042.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-07-04T17:34:47.279 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 32:75:d6:f1:31:e1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:34:47.280 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::3075:d6ff:fef1:31e1/64 scope link 2024-07-04T17:34:47.280 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-07-04T17:34:47.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:34:47.280 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-04T17:34:47.358 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-07-04T17:34:47.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:34:47.362 DEBUG:teuthology.orchestra.run.smithi042:> route 2024-07-04T17:34:47.411 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2024-07-04T17:34:47.412 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-04T17:34:47.412 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-04T17:34:47.412 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-04T17:34:47.412 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-04T17:34:47.412 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-04T17:34:47.412 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-07-04T17:34:47.412 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-04T17:34:47.412 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-04T17:34:47.412 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-04T17:34:47.412 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-07-04T17:34:47.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-04T17:34:47.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:34:47.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:34:47.635 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-07-04T17:34:47.687 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-04T17:34:47.687 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2024-07-04T17:34:47.688 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-07-04T17:34:47.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:34:47.688 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.1 up 2024-07-04T17:34:47.775 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-07-04T17:34:47.775 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-04T17:34:47.775 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-07-04T17:34:47.826 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-07-04T17:34:47.827 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-04T17:34:47.827 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-07-04T17:34:47.873 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:34:47.962 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:34:48.017 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:34:48.017 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:34:48.017 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:34:48.017 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:34:48.017 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:34:48.018 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:34:48.018 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:34:48.018 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:34:48.018 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:34:48.018 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:34:48.018 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:34:48.018 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:34:48.018 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:34:48.018 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:34:48.018 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:34:48.018 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:34:48.018 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:34:48.018 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:34:48.019 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:34:48.019 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:34:48.019 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:34:48.019 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:34:48.019 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:34:48.019 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:34:48.019 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:34:48.019 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:34:48.019 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:34:48.019 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:34:48.019 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:34:48.019 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:34:48.019 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:34:48.019 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:34:48.019 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:34:48.019 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-07-04T17:34:48.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:34:48.021 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:34:48.077 INFO:teuthology.orchestra.run.smithi042.stdout:42 2024-07-04T17:34:48.077 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-07-04T17:34:48.077 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-07-04T17:34:48.082 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:34:48.150 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:34:48.192 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:34:48.192 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:34:48.192 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:34:48.192 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:34:48.192 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:34:48.192 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:34:48.192 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:34:48.192 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:34:48.192 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:34:48.192 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:34:48.192 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:34:48.193 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:34:48.193 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:34:48.193 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:34:48.193 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:34:48.193 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:34:48.193 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:34:48.193 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:34:48.193 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:34:48.193 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:34:48.193 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:34:48.193 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:34:48.193 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:34:48.193 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:34:48.193 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:34:48.194 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:34:48.194 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:34:48.194 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:34:48.194 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:34:48.194 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:34:48.194 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:34:48.194 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:34:48.194 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:34:48.194 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-07-04T17:34:48.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:34:48.195 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:34:48.289 INFO:teuthology.orchestra.run.smithi042.stdout:42 2024-07-04T17:34:48.330 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi042.stderr:2024-07-04T17:34:48.330+0000 7f70ee1ca480 -1 init, newargv = 0x7f70d4004640 newargc=15 2024-07-04T17:34:48.330 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi042.stderr:ceph-fuse[70862]: starting ceph client 2024-07-04T17:34:48.338 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi042.stderr:ceph-fuse[70862]: starting fuse 2024-07-04T17:34:49.391 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-07-04T17:34:49.478 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T17:34:49.535 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-07-04T17:34:49.535 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-07-04T17:34:49.536 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-07-04T17:34:49.536 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-07-04T17:34:49.536 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-07-04T17:34:49.536 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-07-04T17:34:49.536 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-07-04T17:34:49.536 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-07-04T17:34:49.536 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-07-04T17:34:49.536 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-07-04T17:34:49.536 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-07-04T17:34:49.536 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-07-04T17:34:49.536 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T17:34:49.536 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-07-04T17:34:49.536 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-07-04T17:34:49.536 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-07-04T17:34:49.537 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-07-04T17:34:49.537 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-07-04T17:34:49.537 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-07-04T17:34:49.537 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T17:34:49.537 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T17:34:49.537 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T17:34:49.537 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T17:34:49.537 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-07-04T17:34:49.537 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-07-04T17:34:49.537 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-0 2024-07-04T17:34:49.537 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-1 2024-07-04T17:34:49.537 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph/osd/ceph-2 2024-07-04T17:34:49.537 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-07-04T17:34:49.538 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:34:49.538 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:34:49.538 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:34:49.538 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:34:49.538 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.0 2024-07-04T17:34:49.538 INFO:teuthology.orchestra.run.smithi042.stdout:/home/ubuntu/cephtest/mnt.1 2024-07-04T17:34:49.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:34:49.539 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-07-04T17:34:49.593 INFO:teuthology.orchestra.run.smithi042.stdout:42 2024-07-04T17:34:49.593 INFO:teuthology.orchestra.run.smithi042.stdout:43 2024-07-04T17:34:49.594 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 43] 2024-07-04T17:34:49.594 DEBUG:teuthology.orchestra.run.smithi042:> sudo stdin-killer -- python3 -c ' 2024-07-04T17:34:49.594 DEBUG:teuthology.orchestra.run.smithi042:> import glob 2024-07-04T17:34:49.594 DEBUG:teuthology.orchestra.run.smithi042:> import re 2024-07-04T17:34:49.594 DEBUG:teuthology.orchestra.run.smithi042:> import os 2024-07-04T17:34:49.594 DEBUG:teuthology.orchestra.run.smithi042:> import subprocess 2024-07-04T17:34:49.594 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:34:49.594 DEBUG:teuthology.orchestra.run.smithi042:> def _find_admin_socket(client_name): 2024-07-04T17:34:49.594 DEBUG:teuthology.orchestra.run.smithi042:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-07-04T17:34:49.594 DEBUG:teuthology.orchestra.run.smithi042:> files = glob.glob(asok_path) 2024-07-04T17:34:49.595 DEBUG:teuthology.orchestra.run.smithi042:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-07-04T17:34:49.595 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:34:49.595 DEBUG:teuthology.orchestra.run.smithi042:> # Given a non-glob path, it better be there 2024-07-04T17:34:49.595 DEBUG:teuthology.orchestra.run.smithi042:> if "*" not in asok_path: 2024-07-04T17:34:49.595 DEBUG:teuthology.orchestra.run.smithi042:> assert(len(files) == 1) 2024-07-04T17:34:49.595 DEBUG:teuthology.orchestra.run.smithi042:> return files[0] 2024-07-04T17:34:49.595 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:34:49.595 DEBUG:teuthology.orchestra.run.smithi042:> for f in files: 2024-07-04T17:34:49.595 DEBUG:teuthology.orchestra.run.smithi042:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-07-04T17:34:49.595 DEBUG:teuthology.orchestra.run.smithi042:> if os.path.exists("/proc/{0}".format(pid)): 2024-07-04T17:34:49.595 DEBUG:teuthology.orchestra.run.smithi042:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-07-04T17:34:49.595 DEBUG:teuthology.orchestra.run.smithi042:> contents = proc_f.read() 2024-07-04T17:34:49.595 DEBUG:teuthology.orchestra.run.smithi042:> if mountpoint in contents: 2024-07-04T17:34:49.595 DEBUG:teuthology.orchestra.run.smithi042:> return f 2024-07-04T17:34:49.595 DEBUG:teuthology.orchestra.run.smithi042:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-07-04T17:34:49.595 DEBUG:teuthology.orchestra.run.smithi042:> 2024-07-04T17:34:49.595 DEBUG:teuthology.orchestra.run.smithi042:> print(_find_admin_socket("client.1")) 2024-07-04T17:34:49.596 DEBUG:teuthology.orchestra.run.smithi042:> ' 2024-07-04T17:34:49.774 INFO:teuthology.orchestra.run.smithi042.stdout:/var/run/ceph/ceph-client.1.70862.asok 2024-07-04T17:34:49.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T17:34:49.793 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.70862.asok 2024-07-04T17:34:49.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:34:49.794 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.70862.asok status 2024-07-04T17:34:50.103 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:34:50.103 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": { 2024-07-04T17:34:50.103 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_sha1": "9dba1e836b4756bf3a32be2a4936f371dff55008", 2024-07-04T17:34:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.0.0-4667-g9dba1e83 (9dba1e836b4756bf3a32be2a4936f371dff55008) squid (dev)", 2024-07-04T17:34:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: "entity_id": "1", 2024-07-04T17:34:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": "smithi042", 2024-07-04T17:34:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-07-04T17:34:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: "pid": "70862", 2024-07-04T17:34:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: "root": "/" 2024-07-04T17:34:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_count": 0, 2024-07-04T17:34:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_pinned_count": 0, 2024-07-04T17:34:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 5881, 2024-07-04T17:34:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: "inst": { 2024-07-04T17:34:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: "name": { 2024-07-04T17:34:50.104 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "client", 2024-07-04T17:34:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: "num": 5881 2024-07-04T17:34:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-07-04T17:34:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-07-04T17:34:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.2:0", 2024-07-04T17:34:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 685887892 2024-07-04T17:34:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-07-04T17:34:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-07-04T17:34:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.2:0", 2024-07-04T17:34:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 685887892 2024-07-04T17:34:50.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: "inst_str": "client.5881 192.168.0.2:0/685887892", 2024-07-04T17:34:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: "addr_str": "192.168.0.2:0/685887892", 2024-07-04T17:34:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: "inode_count": 1, 2024-07-04T17:34:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_epoch": 108, 2024-07-04T17:34:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch": 136, 2024-07-04T17:34:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch_barrier": 0, 2024-07-04T17:34:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklisted": false, 2024-07-04T17:34:50.106 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs" 2024-07-04T17:34:50.106 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:34:50.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:34:50.127 DEBUG:teuthology.orchestra.run.smithi042:> stat --file-system '--printf=%T 2024-07-04T17:34:50.127 DEBUG:teuthology.orchestra.run.smithi042:> ' -- /home/ubuntu/cephtest/mnt.1 2024-07-04T17:34:50.184 INFO:teuthology.orchestra.run.smithi042.stdout:fuseblk 2024-07-04T17:34:50.184 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-07-04T17:34:50.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:34:50.184 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-07-04T17:34:50.267 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-07-04T17:34:50.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:50.570+0000 7fcaa7e00640 1 -- 172.21.15.42:0/3063083276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcaa80ac9d0 msgr2=0x7fcaa80aef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:50.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:50.570+0000 7fcaa7e00640 1 --2- 172.21.15.42:0/3063083276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcaa80ac9d0 0x7fcaa80aef80 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fca900097b0 tx=0x7fca9002a4b0 comp rx=0 tx=0).stop 2024-07-04T17:34:50.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:50.570+0000 7fcaa7e00640 1 -- 172.21.15.42:0/3063083276 shutdown_connections 2024-07-04T17:34:50.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:50.570+0000 7fcaa7e00640 1 --2- 172.21.15.42:0/3063083276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcaa80ac9d0 0x7fcaa80aef80 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:50.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:50.570+0000 7fcaa7e00640 1 -- 172.21.15.42:0/3063083276 >> 172.21.15.42:0/3063083276 conn(0x7fcaa80afaa0 msgr2=0x7fcaa80afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:50.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:50.570+0000 7fcaa7e00640 1 -- 172.21.15.42:0/3063083276 shutdown_connections 2024-07-04T17:34:50.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:50.570+0000 7fcaa7e00640 1 -- 172.21.15.42:0/3063083276 wait complete. 2024-07-04T17:34:50.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:50.572+0000 7fcaa7e00640 1 Processor -- start 2024-07-04T17:34:50.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:50.572+0000 7fcaa7e00640 1 -- start start 2024-07-04T17:34:50.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:50.574+0000 7fcaa7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcaa80ac9d0 0x7fcaa8143100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:50.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:50.574+0000 7fcaa7e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa81415a0 con 0x7fcaa80ac9d0 2024-07-04T17:34:50.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:50.574+0000 7fcaa6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcaa80ac9d0 0x7fcaa8143100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:50.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:50.574+0000 7fcaa6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcaa80ac9d0 0x7fcaa8143100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33960/0 (socket says 172.21.15.42:33960) 2024-07-04T17:34:50.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:50.574+0000 7fcaa6a00640 1 -- 172.21.15.42:0/870551061 learned_addr learned my addr 172.21.15.42:0/870551061 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:50.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:50.575+0000 7fcaa6a00640 1 -- 172.21.15.42:0/870551061 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa80ab420 con 0x7fcaa80ac9d0 2024-07-04T17:34:50.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:50.575+0000 7fcaa6a00640 1 --2- 172.21.15.42:0/870551061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcaa80ac9d0 0x7fcaa8143100 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fca9800c6b0 tx=0x7fca9800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:50.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:50.575+0000 7fca9ea00640 1 -- 172.21.15.42:0/870551061 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca98007e10 con 0x7fcaa80ac9d0 2024-07-04T17:34:50.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:50.575+0000 7fca9ea00640 1 -- 172.21.15.42:0/870551061 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca98017070 con 0x7fcaa80ac9d0 2024-07-04T17:34:50.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:50.575+0000 7fca9ea00640 1 -- 172.21.15.42:0/870551061 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca98006b50 con 0x7fcaa80ac9d0 2024-07-04T17:34:50.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:50.575+0000 7fcaa7e00640 1 -- 172.21.15.42:0/870551061 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa80b5bc0 con 0x7fcaa80ac9d0 2024-07-04T17:34:50.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:50.575+0000 7fcaa7e00640 1 -- 172.21.15.42:0/870551061 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa8144ca0 con 0x7fcaa80ac9d0 2024-07-04T17:34:50.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:50.576+0000 7fcaa7e00640 1 -- 172.21.15.42:0/870551061 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaa813e960 con 0x7fcaa80ac9d0 2024-07-04T17:34:50.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:50.577+0000 7fca9ea00640 1 -- 172.21.15.42:0/870551061 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7fca98006cf0 con 0x7fcaa80ac9d0 2024-07-04T17:34:50.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:50.577+0000 7fca9ea00640 1 --2- 172.21.15.42:0/870551061 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fca8403d4f0 0x7fca8403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:50.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:50.577+0000 7fca9ea00640 1 -- 172.21.15.42:0/870551061 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 3439+0+0 (secure 0 0 0) 0x7fca980515b0 con 0x7fcaa80ac9d0 2024-07-04T17:34:50.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:50.577+0000 7fca9fe00640 1 --2- 172.21.15.42:0/870551061 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fca8403d4f0 0x7fca8403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:50.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:50.578+0000 7fca9fe00640 1 --2- 172.21.15.42:0/870551061 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fca8403d4f0 0x7fca8403fa00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fca90034000 tx=0x7fca90004730 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:50.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:50.583+0000 7fca9ea00640 1 -- 172.21.15.42:0/870551061 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fcaa8144ca0 con 0x7fcaa80ac9d0 2024-07-04T17:34:50.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:50.845+0000 7fcaa7e00640 1 -- 172.21.15.42:0/870551061 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fcaa8141c50 con 0x7fcaa80ac9d0 2024-07-04T17:34:51.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.175+0000 7fca9ea00640 1 -- 172.21.15.42:0/870551061 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v109) ==== 136+0+0 (secure 0 0 0) 0x7fcaa80b1280 con 0x7fcaa80ac9d0 2024-07-04T17:34:51.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.180+0000 7fcaa7e00640 1 -- 172.21.15.42:0/870551061 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fca8403d4f0 msgr2=0x7fca8403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:51.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.180+0000 7fcaa7e00640 1 --2- 172.21.15.42:0/870551061 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fca8403d4f0 0x7fca8403fa00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fca90034000 tx=0x7fca90004730 comp rx=0 tx=0).stop 2024-07-04T17:34:51.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.180+0000 7fcaa7e00640 1 -- 172.21.15.42:0/870551061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcaa80ac9d0 msgr2=0x7fcaa8143100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:51.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.180+0000 7fcaa7e00640 1 --2- 172.21.15.42:0/870551061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcaa80ac9d0 0x7fcaa8143100 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fca9800c6b0 tx=0x7fca9800cb70 comp rx=0 tx=0).stop 2024-07-04T17:34:51.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.180+0000 7fcaa7e00640 1 -- 172.21.15.42:0/870551061 shutdown_connections 2024-07-04T17:34:51.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.180+0000 7fcaa7e00640 1 --2- 172.21.15.42:0/870551061 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fca8403d4f0 0x7fca8403fa00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:51.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.180+0000 7fcaa7e00640 1 --2- 172.21.15.42:0/870551061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcaa80ac9d0 0x7fcaa8143100 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:51.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.180+0000 7fcaa7e00640 1 -- 172.21.15.42:0/870551061 >> 172.21.15.42:0/870551061 conn(0x7fcaa80afaa0 msgr2=0x7fcaa813f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:51.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.180+0000 7fcaa7e00640 1 -- 172.21.15.42:0/870551061 shutdown_connections 2024-07-04T17:34:51.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.181+0000 7fcaa7e00640 1 -- 172.21.15.42:0/870551061 wait complete. 2024-07-04T17:34:51.218 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-07-04T17:34:51.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.508+0000 7efe01a00640 1 -- 172.21.15.42:0/3188719414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efdfc0bbef0 msgr2=0x7efdfc1414a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:51.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.508+0000 7efe01a00640 1 --2- 172.21.15.42:0/3188719414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efdfc0bbef0 0x7efdfc1414a0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7efdec0097b0 tx=0x7efdec02a530 comp rx=0 tx=0).stop 2024-07-04T17:34:51.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.508+0000 7efe01a00640 1 -- 172.21.15.42:0/3188719414 shutdown_connections 2024-07-04T17:34:51.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.508+0000 7efe01a00640 1 --2- 172.21.15.42:0/3188719414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efdfc0bbef0 0x7efdfc1414a0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:51.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.508+0000 7efe01a00640 1 -- 172.21.15.42:0/3188719414 >> 172.21.15.42:0/3188719414 conn(0x7efdfc0688a0 msgr2=0x7efdfc06db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:51.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.508+0000 7efe01a00640 1 -- 172.21.15.42:0/3188719414 shutdown_connections 2024-07-04T17:34:51.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.508+0000 7efe01a00640 1 -- 172.21.15.42:0/3188719414 wait complete. 2024-07-04T17:34:51.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.510+0000 7efe01a00640 1 Processor -- start 2024-07-04T17:34:51.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.510+0000 7efe01a00640 1 -- start start 2024-07-04T17:34:51.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.512+0000 7efe01a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efdfc0bbef0 0x7efdfc0bee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:51.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.512+0000 7efe01a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc0b79a0 con 0x7efdfc0bbef0 2024-07-04T17:34:51.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.512+0000 7efdfbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efdfc0bbef0 0x7efdfc0bee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:51.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.512+0000 7efdfbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efdfc0bbef0 0x7efdfc0bee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33978/0 (socket says 172.21.15.42:33978) 2024-07-04T17:34:51.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.512+0000 7efdfbe00640 1 -- 172.21.15.42:0/2443328903 learned_addr learned my addr 172.21.15.42:0/2443328903 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:51.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.512+0000 7efdfbe00640 1 -- 172.21.15.42:0/2443328903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdfc1dd2a0 con 0x7efdfc0bbef0 2024-07-04T17:34:51.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.513+0000 7efdfbe00640 1 --2- 172.21.15.42:0/2443328903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efdfc0bbef0 0x7efdfc0bee00 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7efdec02ab20 tx=0x7efdec033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:51.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.513+0000 7efdf3e00640 1 -- 172.21.15.42:0/2443328903 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdec0319b0 con 0x7efdfc0bbef0 2024-07-04T17:34:51.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.513+0000 7efdf3e00640 1 -- 172.21.15.42:0/2443328903 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efdec031c40 con 0x7efdfc0bbef0 2024-07-04T17:34:51.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.513+0000 7efdf3e00640 1 -- 172.21.15.42:0/2443328903 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdec0330d0 con 0x7efdfc0bbef0 2024-07-04T17:34:51.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.513+0000 7efe01a00640 1 -- 172.21.15.42:0/2443328903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdfc13f350 con 0x7efdfc0bbef0 2024-07-04T17:34:51.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.513+0000 7efe01a00640 1 -- 172.21.15.42:0/2443328903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdfc145280 con 0x7efdfc0bbef0 2024-07-04T17:34:51.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.515+0000 7efe01a00640 1 -- 172.21.15.42:0/2443328903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdfc0b8020 con 0x7efdfc0bbef0 2024-07-04T17:34:51.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.515+0000 7efdf3e00640 1 -- 172.21.15.42:0/2443328903 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7efdec033270 con 0x7efdfc0bbef0 2024-07-04T17:34:51.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.515+0000 7efdf3e00640 1 --2- 172.21.15.42:0/2443328903 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7efdd803d4f0 0x7efdd803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:51.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.515+0000 7efdf3e00640 1 -- 172.21.15.42:0/2443328903 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 3439+0+0 (secure 0 0 0) 0x7efdec072d90 con 0x7efdfc0bbef0 2024-07-04T17:34:51.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.516+0000 7efdfb400640 1 --2- 172.21.15.42:0/2443328903 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7efdd803d4f0 0x7efdd803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:51.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.517+0000 7efdfb400640 1 --2- 172.21.15.42:0/2443328903 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7efdd803d4f0 0x7efdd803fa00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efdfc06db00 tx=0x7efde4001290 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:51.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.522+0000 7efdf3e00640 1 -- 172.21.15.42:0/2443328903 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7efdfc1dd2a0 con 0x7efdfc0bbef0 2024-07-04T17:34:51.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.741+0000 7efe01a00640 1 -- 172.21.15.42:0/2443328903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7efdfc0c01a0 con 0x7efdfc0bbef0 2024-07-04T17:34:51.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.742+0000 7efdf3e00640 1 -- 172.21.15.42:0/2443328903 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v136) ==== 84+0+209 (secure 0 0 0) 0x7efdec03f030 con 0x7efdfc0bbef0 2024-07-04T17:34:51.743 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:34:51.743 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-07-04T17:34:51.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:34:51.743 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 1, 2024-07-04T17:34:51.743 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": ".mgr" 2024-07-04T17:34:51.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:51.743 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:34:51.744 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 24, 2024-07-04T17:34:51.744 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": "cephfs_metadata" 2024-07-04T17:34:51.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:34:51.744 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:34:51.744 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 25, 2024-07-04T17:34:51.744 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": "cephfs_data" 2024-07-04T17:34:51.744 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:34:51.744 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-07-04T17:34:51.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.744+0000 7efe01a00640 1 -- 172.21.15.42:0/2443328903 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7efdd803d4f0 msgr2=0x7efdd803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:51.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.744+0000 7efe01a00640 1 --2- 172.21.15.42:0/2443328903 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7efdd803d4f0 0x7efdd803fa00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efdfc06db00 tx=0x7efde4001290 comp rx=0 tx=0).stop 2024-07-04T17:34:51.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.744+0000 7efe01a00640 1 -- 172.21.15.42:0/2443328903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efdfc0bbef0 msgr2=0x7efdfc0bee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:51.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.744+0000 7efe01a00640 1 --2- 172.21.15.42:0/2443328903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efdfc0bbef0 0x7efdfc0bee00 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7efdec02ab20 tx=0x7efdec033ad0 comp rx=0 tx=0).stop 2024-07-04T17:34:51.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.744+0000 7efe01a00640 1 -- 172.21.15.42:0/2443328903 shutdown_connections 2024-07-04T17:34:51.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.744+0000 7efe01a00640 1 --2- 172.21.15.42:0/2443328903 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7efdd803d4f0 0x7efdd803fa00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:51.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.744+0000 7efe01a00640 1 --2- 172.21.15.42:0/2443328903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efdfc0bbef0 0x7efdfc0bee00 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:51.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.745+0000 7efe01a00640 1 -- 172.21.15.42:0/2443328903 >> 172.21.15.42:0/2443328903 conn(0x7efdfc0688a0 msgr2=0x7efdfc0b6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:51.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.745+0000 7efe01a00640 1 -- 172.21.15.42:0/2443328903 shutdown_connections 2024-07-04T17:34:51.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:51.745+0000 7efe01a00640 1 -- 172.21.15.42:0/2443328903 wait complete. 2024-07-04T17:34:51.772 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'backup_fs' 2024-07-04T17:34:51.773 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_metadata --pg_num_min 64 2024-07-04T17:34:52.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:52.065+0000 7f35eb200640 1 -- 172.21.15.42:0/1207778858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35ec06da60 msgr2=0x7f35ec0b0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:52.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:52.065+0000 7f35eb200640 1 --2- 172.21.15.42:0/1207778858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35ec06da60 0x7f35ec0b0db0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f35dc0097b0 tx=0x7f35dc02a4a0 comp rx=0 tx=0).stop 2024-07-04T17:34:52.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:52.066+0000 7f35eb200640 1 -- 172.21.15.42:0/1207778858 shutdown_connections 2024-07-04T17:34:52.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:52.066+0000 7f35eb200640 1 --2- 172.21.15.42:0/1207778858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35ec06da60 0x7f35ec0b0db0 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:52.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:52.066+0000 7f35eb200640 1 -- 172.21.15.42:0/1207778858 >> 172.21.15.42:0/1207778858 conn(0x7f35ec0afaa0 msgr2=0x7f35ec0afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:52.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:52.066+0000 7f35eb200640 1 -- 172.21.15.42:0/1207778858 shutdown_connections 2024-07-04T17:34:52.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:52.066+0000 7f35eb200640 1 -- 172.21.15.42:0/1207778858 wait complete. 2024-07-04T17:34:52.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:52.068+0000 7f35eb200640 1 Processor -- start 2024-07-04T17:34:52.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:52.068+0000 7f35eb200640 1 -- start start 2024-07-04T17:34:52.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:52.069+0000 7f35eb200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35ec06da60 0x7f35ec1d7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:52.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:52.069+0000 7f35eb200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec0ae590 con 0x7f35ec06da60 2024-07-04T17:34:52.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:52.070+0000 7f35e9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35ec06da60 0x7f35ec1d7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:52.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:52.070+0000 7f35e9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35ec06da60 0x7f35ec1d7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33994/0 (socket says 172.21.15.42:33994) 2024-07-04T17:34:52.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:52.070+0000 7f35e9e00640 1 -- 172.21.15.42:0/1970635011 learned_addr learned my addr 172.21.15.42:0/1970635011 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:52.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:52.070+0000 7f35e9e00640 1 -- 172.21.15.42:0/1970635011 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ec1d4c00 con 0x7f35ec06da60 2024-07-04T17:34:52.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:52.071+0000 7f35e9e00640 1 --2- 172.21.15.42:0/1970635011 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35ec06da60 0x7f35ec1d7700 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f35dc0098d0 tx=0x7f35dc033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:52.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:52.071+0000 7f35e2000640 1 -- 172.21.15.42:0/1970635011 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35dc031880 con 0x7f35ec06da60 2024-07-04T17:34:52.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:52.071+0000 7f35e2000640 1 -- 172.21.15.42:0/1970635011 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35dc031b10 con 0x7f35ec06da60 2024-07-04T17:34:52.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:52.071+0000 7f35e2000640 1 -- 172.21.15.42:0/1970635011 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35dc031df0 con 0x7f35ec06da60 2024-07-04T17:34:52.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:52.071+0000 7f35eb200640 1 -- 172.21.15.42:0/1970635011 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35ec13a3a0 con 0x7f35ec06da60 2024-07-04T17:34:52.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:52.071+0000 7f35eb200640 1 -- 172.21.15.42:0/1970635011 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35ec1d5720 con 0x7f35ec06da60 2024-07-04T17:34:52.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:52.072+0000 7f35eb200640 1 -- 172.21.15.42:0/1970635011 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35ec0b0db0 con 0x7f35ec06da60 2024-07-04T17:34:52.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:52.073+0000 7f35e2000640 1 -- 172.21.15.42:0/1970635011 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f35dc033070 con 0x7f35ec06da60 2024-07-04T17:34:52.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:52.073+0000 7f35e2000640 1 --2- 172.21.15.42:0/1970635011 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f35c803d4f0 0x7f35c803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:52.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:52.073+0000 7f35e2000640 1 -- 172.21.15.42:0/1970635011 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 3439+0+0 (secure 0 0 0) 0x7f35dc072d60 con 0x7f35ec06da60 2024-07-04T17:34:52.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:52.074+0000 7f35e3e00640 1 --2- 172.21.15.42:0/1970635011 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f35c803d4f0 0x7f35c803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:52.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:52.074+0000 7f35e3e00640 1 --2- 172.21.15.42:0/1970635011 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f35c803d4f0 0x7f35c803fa00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f35d4007fd0 tx=0x7f35d4007920 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:52.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:52.078+0000 7f35e2000640 1 -- 172.21.15.42:0/1970635011 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f35dc03f030 con 0x7f35ec06da60 2024-07-04T17:34:52.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:52.287+0000 7f35eb200640 1 -- 172.21.15.42:0/1970635011 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64} v 0) -- 0x7f35ec068e30 con 0x7f35ec06da60 2024-07-04T17:34:53.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.190+0000 7f35e2000640 1 -- 172.21.15.42:0/1970635011 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64}]=0 pool 'backup_fs_metadata' created v137) ==== 144+0+0 (secure 0 0 0) 0x7f35dc03a3a0 con 0x7f35ec06da60 2024-07-04T17:34:53.191 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'backup_fs_metadata' created 2024-07-04T17:34:53.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.194+0000 7f35eb200640 1 -- 172.21.15.42:0/1970635011 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f35c803d4f0 msgr2=0x7f35c803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:53.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.194+0000 7f35eb200640 1 --2- 172.21.15.42:0/1970635011 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f35c803d4f0 0x7f35c803fa00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f35d4007fd0 tx=0x7f35d4007920 comp rx=0 tx=0).stop 2024-07-04T17:34:53.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.194+0000 7f35eb200640 1 -- 172.21.15.42:0/1970635011 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35ec06da60 msgr2=0x7f35ec1d7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:53.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.194+0000 7f35eb200640 1 --2- 172.21.15.42:0/1970635011 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35ec06da60 0x7f35ec1d7700 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f35dc0098d0 tx=0x7f35dc033ad0 comp rx=0 tx=0).stop 2024-07-04T17:34:53.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.194+0000 7f35eb200640 1 -- 172.21.15.42:0/1970635011 shutdown_connections 2024-07-04T17:34:53.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.194+0000 7f35eb200640 1 --2- 172.21.15.42:0/1970635011 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f35c803d4f0 0x7f35c803fa00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:53.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.194+0000 7f35eb200640 1 --2- 172.21.15.42:0/1970635011 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35ec06da60 0x7f35ec1d7700 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:53.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.194+0000 7f35eb200640 1 -- 172.21.15.42:0/1970635011 >> 172.21.15.42:0/1970635011 conn(0x7f35ec0afaa0 msgr2=0x7f35ec0aebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:53.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.194+0000 7f35eb200640 1 -- 172.21.15.42:0/1970635011 shutdown_connections 2024-07-04T17:34:53.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.195+0000 7f35eb200640 1 -- 172.21.15.42:0/1970635011 wait complete. 2024-07-04T17:34:53.236 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-04T17:34:53.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.506+0000 7f2a0d200640 1 -- 172.21.15.42:0/2549785717 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a0806d160 msgr2=0x7f2a08071c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:53.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.506+0000 7f2a0d200640 1 --2- 172.21.15.42:0/2549785717 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a0806d160 0x7f2a08071c70 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f29f00097b0 tx=0x7f29f002a5d0 comp rx=0 tx=0).stop 2024-07-04T17:34:53.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.506+0000 7f2a0d200640 1 -- 172.21.15.42:0/2549785717 shutdown_connections 2024-07-04T17:34:53.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.506+0000 7f2a0d200640 1 --2- 172.21.15.42:0/2549785717 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a0806d160 0x7f2a08071c70 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:53.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.506+0000 7f2a0d200640 1 -- 172.21.15.42:0/2549785717 >> 172.21.15.42:0/2549785717 conn(0x7f2a0806c020 msgr2=0x7f2a0806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:53.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.506+0000 7f2a0d200640 1 -- 172.21.15.42:0/2549785717 shutdown_connections 2024-07-04T17:34:53.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.506+0000 7f2a0d200640 1 -- 172.21.15.42:0/2549785717 wait complete. 2024-07-04T17:34:53.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.508+0000 7f2a0d200640 1 Processor -- start 2024-07-04T17:34:53.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.508+0000 7f2a0d200640 1 -- start start 2024-07-04T17:34:53.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.510+0000 7f2a0d200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a0806d160 0x7f2a081d8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:53.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.510+0000 7f2a0d200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0813e3a0 con 0x7f2a0806d160 2024-07-04T17:34:53.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.510+0000 7f2a05e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a0806d160 0x7f2a081d8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:53.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.510+0000 7f2a05e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a0806d160 0x7f2a081d8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34012/0 (socket says 172.21.15.42:34012) 2024-07-04T17:34:53.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.510+0000 7f2a05e00640 1 -- 172.21.15.42:0/2431796873 learned_addr learned my addr 172.21.15.42:0/2431796873 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:53.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.510+0000 7f2a05e00640 1 -- 172.21.15.42:0/2431796873 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a080b6880 con 0x7f2a0806d160 2024-07-04T17:34:53.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.511+0000 7f2a05e00640 1 --2- 172.21.15.42:0/2431796873 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a0806d160 0x7f2a081d8860 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f29f800c6b0 tx=0x7f29f800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:53.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.511+0000 7f29fe000640 1 -- 172.21.15.42:0/2431796873 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29f8016020 con 0x7f2a0806d160 2024-07-04T17:34:53.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.511+0000 7f29fe000640 1 -- 172.21.15.42:0/2431796873 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29f800a5c0 con 0x7f2a0806d160 2024-07-04T17:34:53.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.511+0000 7f29fe000640 1 -- 172.21.15.42:0/2431796873 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29f800a960 con 0x7f2a0806d160 2024-07-04T17:34:53.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.511+0000 7f2a0d200640 1 -- 172.21.15.42:0/2431796873 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a081e75e0 con 0x7f2a0806d160 2024-07-04T17:34:53.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.511+0000 7f2a0d200640 1 -- 172.21.15.42:0/2431796873 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a080b6570 con 0x7f2a0806d160 2024-07-04T17:34:53.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.512+0000 7f2a0d200640 1 -- 172.21.15.42:0/2431796873 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a0813e9e0 con 0x7f2a0806d160 2024-07-04T17:34:53.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.513+0000 7f29fe000640 1 -- 172.21.15.42:0/2431796873 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f29f8019020 con 0x7f2a0806d160 2024-07-04T17:34:53.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.513+0000 7f29fe000640 1 --2- 172.21.15.42:0/2431796873 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f29e403d4f0 0x7f29e403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:53.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.514+0000 7f29fe000640 1 -- 172.21.15.42:0/2431796873 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 3812+0+0 (secure 0 0 0) 0x7f29f8051a80 con 0x7f2a0806d160 2024-07-04T17:34:53.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.514+0000 7f2a05400640 1 --2- 172.21.15.42:0/2431796873 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f29e403d4f0 0x7f29e403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:53.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.515+0000 7f2a05400640 1 --2- 172.21.15.42:0/2431796873 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f29e403d4f0 0x7f29e403fa00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f29f0034000 tx=0x7f29f0004730 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:53.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.522+0000 7f29fe000640 1 -- 172.21.15.42:0/2431796873 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f29f8020020 con 0x7f2a0806d160 2024-07-04T17:34:53.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:53.736+0000 7f2a0d200640 1 -- 172.21.15.42:0/2431796873 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2a080b6a50 con 0x7f2a0806d160 2024-07-04T17:34:54.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.206+0000 7f29fe000640 1 -- 172.21.15.42:0/2431796873 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'backup_fs_data' created v138) ==== 176+0+0 (secure 0 0 0) 0x7f2a0806d570 con 0x7f2a0806d160 2024-07-04T17:34:54.208 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'backup_fs_data' created 2024-07-04T17:34:54.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.210+0000 7f2a0d200640 1 -- 172.21.15.42:0/2431796873 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f29e403d4f0 msgr2=0x7f29e403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:54.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.210+0000 7f2a0d200640 1 --2- 172.21.15.42:0/2431796873 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f29e403d4f0 0x7f29e403fa00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f29f0034000 tx=0x7f29f0004730 comp rx=0 tx=0).stop 2024-07-04T17:34:54.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.210+0000 7f2a0d200640 1 -- 172.21.15.42:0/2431796873 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a0806d160 msgr2=0x7f2a081d8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:54.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.210+0000 7f2a0d200640 1 --2- 172.21.15.42:0/2431796873 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a0806d160 0x7f2a081d8860 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f29f800c6b0 tx=0x7f29f800cb70 comp rx=0 tx=0).stop 2024-07-04T17:34:54.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.211+0000 7f2a0d200640 1 -- 172.21.15.42:0/2431796873 shutdown_connections 2024-07-04T17:34:54.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.211+0000 7f2a0d200640 1 --2- 172.21.15.42:0/2431796873 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f29e403d4f0 0x7f29e403fa00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:54.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.211+0000 7f2a0d200640 1 --2- 172.21.15.42:0/2431796873 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a0806d160 0x7f2a081d8860 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:54.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.211+0000 7f2a0d200640 1 -- 172.21.15.42:0/2431796873 >> 172.21.15.42:0/2431796873 conn(0x7f2a0806c020 msgr2=0x7f2a080704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:54.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.211+0000 7f2a0d200640 1 -- 172.21.15.42:0/2431796873 shutdown_connections 2024-07-04T17:34:54.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.213+0000 7f2a0d200640 1 -- 172.21.15.42:0/2431796873 wait complete. 2024-07-04T17:34:54.268 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new backup_fs backup_fs_metadata backup_fs_data 2024-07-04T17:34:54.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.530+0000 7f747dc00640 1 -- 172.21.15.42:0/3715173113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74780b1a00 msgr2=0x7f74780b1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:54.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.530+0000 7f747dc00640 1 --2- 172.21.15.42:0/3715173113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74780b1a00 0x7f74780b1e10 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f74600097b0 tx=0x7f746002a4a0 comp rx=0 tx=0).stop 2024-07-04T17:34:54.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.530+0000 7f747dc00640 1 -- 172.21.15.42:0/3715173113 shutdown_connections 2024-07-04T17:34:54.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.530+0000 7f747dc00640 1 --2- 172.21.15.42:0/3715173113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74780b1a00 0x7f74780b1e10 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:54.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.530+0000 7f747dc00640 1 -- 172.21.15.42:0/3715173113 >> 172.21.15.42:0/3715173113 conn(0x7f74780bf4d0 msgr2=0x7f74780af870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:54.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.530+0000 7f747dc00640 1 -- 172.21.15.42:0/3715173113 shutdown_connections 2024-07-04T17:34:54.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.530+0000 7f747dc00640 1 -- 172.21.15.42:0/3715173113 wait complete. 2024-07-04T17:34:54.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.532+0000 7f747dc00640 1 Processor -- start 2024-07-04T17:34:54.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.532+0000 7f747dc00640 1 -- start start 2024-07-04T17:34:54.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.534+0000 7f747dc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74780b0160 0x7f7478150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:54.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.534+0000 7f747dc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f74780ae080 con 0x7f74780b0160 2024-07-04T17:34:54.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.534+0000 7f7476a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74780b0160 0x7f7478150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:54.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.534+0000 7f7476a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74780b0160 0x7f7478150ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55950/0 (socket says 172.21.15.42:55950) 2024-07-04T17:34:54.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.534+0000 7f7476a00640 1 -- 172.21.15.42:0/1604236952 learned_addr learned my addr 172.21.15.42:0/1604236952 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:54.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.534+0000 7f7476a00640 1 -- 172.21.15.42:0/1604236952 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74781d90f0 con 0x7f74780b0160 2024-07-04T17:34:54.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.535+0000 7f7476a00640 1 --2- 172.21.15.42:0/1604236952 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74780b0160 0x7f7478150ad0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f746800c3b0 tx=0x7f746800c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:54.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.535+0000 7f746ea00640 1 -- 172.21.15.42:0/1604236952 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7468011780 con 0x7f74780b0160 2024-07-04T17:34:54.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.535+0000 7f746ea00640 1 -- 172.21.15.42:0/1604236952 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7468017070 con 0x7f74780b0160 2024-07-04T17:34:54.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.535+0000 7f747dc00640 1 -- 172.21.15.42:0/1604236952 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74780b5790 con 0x7f74780b0160 2024-07-04T17:34:54.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.535+0000 7f746ea00640 1 -- 172.21.15.42:0/1604236952 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7468011c00 con 0x7f74780b0160 2024-07-04T17:34:54.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.535+0000 7f747dc00640 1 -- 172.21.15.42:0/1604236952 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7478073d00 con 0x7f74780b0160 2024-07-04T17:34:54.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.537+0000 7f747dc00640 1 -- 172.21.15.42:0/1604236952 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7478151010 con 0x7f74780b0160 2024-07-04T17:34:54.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.537+0000 7f746ea00640 1 -- 172.21.15.42:0/1604236952 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f746801c460 con 0x7f74780b0160 2024-07-04T17:34:54.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.538+0000 7f746ea00640 1 --2- 172.21.15.42:0/1604236952 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f745403d4f0 0x7f745403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:54.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.538+0000 7f746ea00640 1 -- 172.21.15.42:0/1604236952 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 4197+0+0 (secure 0 0 0) 0x7f7468051970 con 0x7f74780b0160 2024-07-04T17:34:54.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.538+0000 7f746fe00640 1 --2- 172.21.15.42:0/1604236952 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f745403d4f0 0x7f745403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:54.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.539+0000 7f746fe00640 1 --2- 172.21.15.42:0/1604236952 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f745403d4f0 0x7f745403fa00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7460034000 tx=0x7f7460004730 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:54.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.543+0000 7f746ea00640 1 -- 172.21.15.42:0/1604236952 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f74781d90f0 con 0x7f74780b0160 2024-07-04T17:34:54.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:54.792+0000 7f747dc00640 1 -- 172.21.15.42:0/1604236952 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"} v 0) -- 0x7f74780ae750 con 0x7f74780b0160 2024-07-04T17:34:55.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.208+0000 7f746ea00640 1 -- 172.21.15.42:0/1604236952 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"}]=0 new fs with metadata pool 26 and data pool 27 v110) ==== 183+0+0 (secure 0 0 0) 0x7f7468010c60 con 0x7f74780b0160 2024-07-04T17:34:55.210 INFO:teuthology.orchestra.run.smithi042.stderr:new fs with metadata pool 26 and data pool 27 2024-07-04T17:34:55.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.212+0000 7f747dc00640 1 -- 172.21.15.42:0/1604236952 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f745403d4f0 msgr2=0x7f745403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:55.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.212+0000 7f747dc00640 1 --2- 172.21.15.42:0/1604236952 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f745403d4f0 0x7f745403fa00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7460034000 tx=0x7f7460004730 comp rx=0 tx=0).stop 2024-07-04T17:34:55.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.212+0000 7f747dc00640 1 -- 172.21.15.42:0/1604236952 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74780b0160 msgr2=0x7f7478150ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:55.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.212+0000 7f747dc00640 1 --2- 172.21.15.42:0/1604236952 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74780b0160 0x7f7478150ad0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f746800c3b0 tx=0x7f746800c870 comp rx=0 tx=0).stop 2024-07-04T17:34:55.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.212+0000 7f747dc00640 1 -- 172.21.15.42:0/1604236952 shutdown_connections 2024-07-04T17:34:55.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.212+0000 7f747dc00640 1 --2- 172.21.15.42:0/1604236952 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f745403d4f0 0x7f745403fa00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:55.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.212+0000 7f747dc00640 1 --2- 172.21.15.42:0/1604236952 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74780b0160 0x7f7478150ad0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:55.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.212+0000 7f747dc00640 1 -- 172.21.15.42:0/1604236952 >> 172.21.15.42:0/1604236952 conn(0x7f74780bf4d0 msgr2=0x7f74780bc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:55.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.213+0000 7f747dc00640 1 -- 172.21.15.42:0/1604236952 shutdown_connections 2024-07-04T17:34:55.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.213+0000 7f747dc00640 1 -- 172.21.15.42:0/1604236952 wait complete. 2024-07-04T17:34:55.266 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:34:55.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.531+0000 7f84ac800640 1 -- 172.21.15.42:0/3479756865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a40b3650 msgr2=0x7f84a41462f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:55.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.531+0000 7f84ac800640 1 --2- 172.21.15.42:0/3479756865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a40b3650 0x7f84a41462f0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f84940097b0 tx=0x7f849402a4b0 comp rx=0 tx=0).stop 2024-07-04T17:34:55.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.531+0000 7f84ac800640 1 -- 172.21.15.42:0/3479756865 shutdown_connections 2024-07-04T17:34:55.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.531+0000 7f84ac800640 1 --2- 172.21.15.42:0/3479756865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a40b3650 0x7f84a41462f0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:55.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.531+0000 7f84ac800640 1 -- 172.21.15.42:0/3479756865 >> 172.21.15.42:0/3479756865 conn(0x7f84a40bf700 msgr2=0x7f84a40af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:55.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.531+0000 7f84ac800640 1 -- 172.21.15.42:0/3479756865 shutdown_connections 2024-07-04T17:34:55.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.531+0000 7f84ac800640 1 -- 172.21.15.42:0/3479756865 wait complete. 2024-07-04T17:34:55.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.533+0000 7f84ac800640 1 Processor -- start 2024-07-04T17:34:55.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.534+0000 7f84ac800640 1 -- start start 2024-07-04T17:34:55.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.535+0000 7f84ac800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a40b02b0 0x7f84a41e6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:55.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.535+0000 7f84ac800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f84a406c590 con 0x7f84a40b02b0 2024-07-04T17:34:55.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.535+0000 7f84a9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a40b02b0 0x7f84a41e6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:55.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.536+0000 7f84a9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a40b02b0 0x7f84a41e6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55958/0 (socket says 172.21.15.42:55958) 2024-07-04T17:34:55.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.536+0000 7f84a9e00640 1 -- 172.21.15.42:0/2252750274 learned_addr learned my addr 172.21.15.42:0/2252750274 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:55.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.536+0000 7f84a9e00640 1 -- 172.21.15.42:0/2252750274 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a40b8bd0 con 0x7f84a40b02b0 2024-07-04T17:34:55.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.536+0000 7f84a9e00640 1 --2- 172.21.15.42:0/2252750274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a40b02b0 0x7f84a41e6b70 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f8498001eb0 tx=0x7f849800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:55.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.537+0000 7f849e000640 1 -- 172.21.15.42:0/2252750274 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84980116c0 con 0x7f84a40b02b0 2024-07-04T17:34:55.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.537+0000 7f849e000640 1 -- 172.21.15.42:0/2252750274 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8498017070 con 0x7f84a40b02b0 2024-07-04T17:34:55.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.537+0000 7f849e000640 1 -- 172.21.15.42:0/2252750274 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8498011ae0 con 0x7f84a40b02b0 2024-07-04T17:34:55.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.537+0000 7f84ac800640 1 -- 172.21.15.42:0/2252750274 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a41e7210 con 0x7f84a40b02b0 2024-07-04T17:34:55.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.537+0000 7f84ac800640 1 -- 172.21.15.42:0/2252750274 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a40b88a0 con 0x7f84a40b02b0 2024-07-04T17:34:55.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.538+0000 7f84ac800640 1 -- 172.21.15.42:0/2252750274 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84a41462f0 con 0x7f84a40b02b0 2024-07-04T17:34:55.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.538+0000 7f849e000640 1 -- 172.21.15.42:0/2252750274 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f8498011c80 con 0x7f84a40b02b0 2024-07-04T17:34:55.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.539+0000 7f849e000640 1 --2- 172.21.15.42:0/2252750274 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f848803d4f0 0x7f848803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:55.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.539+0000 7f849e000640 1 -- 172.21.15.42:0/2252750274 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 4303+0+0 (secure 0 0 0) 0x7f84980519d0 con 0x7f84a40b02b0 2024-07-04T17:34:55.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.539+0000 7f84a9400640 1 --2- 172.21.15.42:0/2252750274 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f848803d4f0 0x7f848803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:55.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.540+0000 7f84a9400640 1 --2- 172.21.15.42:0/2252750274 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f848803d4f0 0x7f848803fa00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8494034000 tx=0x7f8494004730 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:55.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.544+0000 7f849e000640 1 -- 172.21.15.42:0/2252750274 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f84a40b06c0 con 0x7f84a40b02b0 2024-07-04T17:34:55.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.746+0000 7f84ac800640 1 -- 172.21.15.42:0/2252750274 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f84a40beb10 con 0x7f84a40b02b0 2024-07-04T17:34:55.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.748+0000 7f849e000640 1 -- 172.21.15.42:0/2252750274 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) ==== 74+0+14471 (secure 0 0 0) 0x7f849800a140 con 0x7f84a40b02b0 2024-07-04T17:34:55.750 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:34:55.750 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":139,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:34:54.794106+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:34:33.191097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-07-04T17:34:34.033408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:34:52.310342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:34:53.756688+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:34:55.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.751+0000 7f84ac800640 1 -- 172.21.15.42:0/2252750274 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f848803d4f0 msgr2=0x7f848803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:55.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.751+0000 7f84ac800640 1 --2- 172.21.15.42:0/2252750274 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f848803d4f0 0x7f848803fa00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8494034000 tx=0x7f8494004730 comp rx=0 tx=0).stop 2024-07-04T17:34:55.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.751+0000 7f84ac800640 1 -- 172.21.15.42:0/2252750274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a40b02b0 msgr2=0x7f84a41e6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:55.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.751+0000 7f84ac800640 1 --2- 172.21.15.42:0/2252750274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a40b02b0 0x7f84a41e6b70 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f8498001eb0 tx=0x7f849800c810 comp rx=0 tx=0).stop 2024-07-04T17:34:55.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.751+0000 7f84ac800640 1 -- 172.21.15.42:0/2252750274 shutdown_connections 2024-07-04T17:34:55.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.751+0000 7f84ac800640 1 --2- 172.21.15.42:0/2252750274 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f848803d4f0 0x7f848803fa00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:55.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.751+0000 7f84ac800640 1 --2- 172.21.15.42:0/2252750274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84a40b02b0 0x7f84a41e6b70 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:55.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.751+0000 7f84ac800640 1 -- 172.21.15.42:0/2252750274 >> 172.21.15.42:0/2252750274 conn(0x7f84a40bf700 msgr2=0x7f84a4075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:55.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.751+0000 7f84ac800640 1 -- 172.21.15.42:0/2252750274 shutdown_connections 2024-07-04T17:34:55.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:55.751+0000 7f84ac800640 1 -- 172.21.15.42:0/2252750274 wait complete. 2024-07-04T17:34:55.775 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:34:56.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.045+0000 7f7099a00640 1 -- 172.21.15.42:0/100376548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70940b3700 msgr2=0x7f70940b63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:56.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.045+0000 7f7099a00640 1 --2- 172.21.15.42:0/100376548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70940b3700 0x7f70940b63c0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f70800097b0 tx=0x7f708002a540 comp rx=0 tx=0).stop 2024-07-04T17:34:56.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.046+0000 7f7099a00640 1 -- 172.21.15.42:0/100376548 shutdown_connections 2024-07-04T17:34:56.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.046+0000 7f7099a00640 1 --2- 172.21.15.42:0/100376548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70940b3700 0x7f70940b63c0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:56.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.046+0000 7f7099a00640 1 -- 172.21.15.42:0/100376548 >> 172.21.15.42:0/100376548 conn(0x7f70940bebc0 msgr2=0x7f70940befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:56.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.046+0000 7f7099a00640 1 -- 172.21.15.42:0/100376548 shutdown_connections 2024-07-04T17:34:56.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.046+0000 7f7099a00640 1 -- 172.21.15.42:0/100376548 wait complete. 2024-07-04T17:34:56.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.048+0000 7f7099a00640 1 Processor -- start 2024-07-04T17:34:56.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.048+0000 7f7099a00640 1 -- start start 2024-07-04T17:34:56.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.049+0000 7f7099a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7094073f70 0x7f70940bcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:56.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.049+0000 7f7099a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f70940b1ce0 con 0x7f7094073f70 2024-07-04T17:34:56.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.049+0000 7f7093e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7094073f70 0x7f70940bcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:56.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.049+0000 7f7093e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7094073f70 0x7f70940bcfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55982/0 (socket says 172.21.15.42:55982) 2024-07-04T17:34:56.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.049+0000 7f7093e00640 1 -- 172.21.15.42:0/818523219 learned_addr learned my addr 172.21.15.42:0/818523219 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:56.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.050+0000 7f7093e00640 1 -- 172.21.15.42:0/818523219 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f709413e7c0 con 0x7f7094073f70 2024-07-04T17:34:56.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.050+0000 7f7093e00640 1 --2- 172.21.15.42:0/818523219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7094073f70 0x7f70940bcfd0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f708400cb40 tx=0x7f708400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:56.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.050+0000 7f708be00640 1 -- 172.21.15.42:0/818523219 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7084016020 con 0x7f7094073f70 2024-07-04T17:34:56.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.050+0000 7f708be00640 1 -- 172.21.15.42:0/818523219 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f708400abf0 con 0x7f7094073f70 2024-07-04T17:34:56.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.050+0000 7f7099a00640 1 -- 172.21.15.42:0/818523219 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7094073680 con 0x7f7094073f70 2024-07-04T17:34:56.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.050+0000 7f708be00640 1 -- 172.21.15.42:0/818523219 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7084012420 con 0x7f7094073f70 2024-07-04T17:34:56.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.050+0000 7f7099a00640 1 -- 172.21.15.42:0/818523219 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7094071c60 con 0x7f7094073f70 2024-07-04T17:34:56.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.051+0000 7f7099a00640 1 -- 172.21.15.42:0/818523219 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70940b2320 con 0x7f7094073f70 2024-07-04T17:34:56.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.052+0000 7f708be00640 1 -- 172.21.15.42:0/818523219 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f7084019020 con 0x7f7094073f70 2024-07-04T17:34:56.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.052+0000 7f708be00640 1 --2- 172.21.15.42:0/818523219 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f707403d4a0 0x7f707403f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:56.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.052+0000 7f708be00640 1 -- 172.21.15.42:0/818523219 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 4303+0+0 (secure 0 0 0) 0x7f70840522b0 con 0x7f7094073f70 2024-07-04T17:34:56.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.053+0000 7f7093400640 1 --2- 172.21.15.42:0/818523219 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f707403d4a0 0x7f707403f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:56.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.053+0000 7f7093400640 1 --2- 172.21.15.42:0/818523219 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f707403d4a0 0x7f707403f9b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7080034000 tx=0x7f7080004730 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:56.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.057+0000 7f708be00640 1 -- 172.21.15.42:0/818523219 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f7084020020 con 0x7f7094073f70 2024-07-04T17:34:56.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.283+0000 7f7099a00640 1 -- 172.21.15.42:0/818523219 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70941d6e90 con 0x7f7094073f70 2024-07-04T17:34:56.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.286+0000 7f708be00640 1 -- 172.21.15.42:0/818523219 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) ==== 74+0+14458 (secure 0 0 0) 0x7f7084011ca0 con 0x7f7094073f70 2024-07-04T17:34:56.288 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:34:56.288 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":140,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:34:56.219969+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:34:33.191097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-07-04T17:34:34.033408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:34:52.310342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:34:53.756688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:34:56.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.289+0000 7f7099a00640 1 -- 172.21.15.42:0/818523219 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f707403d4a0 msgr2=0x7f707403f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:56.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.289+0000 7f7099a00640 1 --2- 172.21.15.42:0/818523219 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f707403d4a0 0x7f707403f9b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7080034000 tx=0x7f7080004730 comp rx=0 tx=0).stop 2024-07-04T17:34:56.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.289+0000 7f7099a00640 1 -- 172.21.15.42:0/818523219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7094073f70 msgr2=0x7f70940bcfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:56.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.289+0000 7f7099a00640 1 --2- 172.21.15.42:0/818523219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7094073f70 0x7f70940bcfd0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f708400cb40 tx=0x7f708400cf40 comp rx=0 tx=0).stop 2024-07-04T17:34:56.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.289+0000 7f7099a00640 1 -- 172.21.15.42:0/818523219 shutdown_connections 2024-07-04T17:34:56.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.289+0000 7f7099a00640 1 --2- 172.21.15.42:0/818523219 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f707403d4a0 0x7f707403f9b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:56.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.290+0000 7f7099a00640 1 --2- 172.21.15.42:0/818523219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7094073f70 0x7f70940bcfd0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:56.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.290+0000 7f7099a00640 1 -- 172.21.15.42:0/818523219 >> 172.21.15.42:0/818523219 conn(0x7f70940bebc0 msgr2=0x7f70940bbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:56.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.290+0000 7f7099a00640 1 -- 172.21.15.42:0/818523219 shutdown_connections 2024-07-04T17:34:56.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.290+0000 7f7099a00640 1 -- 172.21.15.42:0/818523219 wait complete. 2024-07-04T17:34:56.316 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs standby_count_wanted 0 2024-07-04T17:34:56.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.600+0000 7f77d0400640 1 -- 172.21.15.42:0/3903774948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77c80712d0 msgr2=0x7f77c806d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:56.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.600+0000 7f77d0400640 1 --2- 172.21.15.42:0/3903774948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77c80712d0 0x7f77c806d130 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f77bc0097b0 tx=0x7f77bc02a540 comp rx=0 tx=0).stop 2024-07-04T17:34:56.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.600+0000 7f77d0400640 1 -- 172.21.15.42:0/3903774948 shutdown_connections 2024-07-04T17:34:56.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.600+0000 7f77d0400640 1 --2- 172.21.15.42:0/3903774948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77c80712d0 0x7f77c806d130 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:56.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.600+0000 7f77d0400640 1 -- 172.21.15.42:0/3903774948 >> 172.21.15.42:0/3903774948 conn(0x7f77c806c020 msgr2=0x7f77c806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:56.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.600+0000 7f77d0400640 1 -- 172.21.15.42:0/3903774948 shutdown_connections 2024-07-04T17:34:56.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.600+0000 7f77d0400640 1 -- 172.21.15.42:0/3903774948 wait complete. 2024-07-04T17:34:56.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.602+0000 7f77d0400640 1 Processor -- start 2024-07-04T17:34:56.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.602+0000 7f77d0400640 1 -- start start 2024-07-04T17:34:56.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.604+0000 7f77d0400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77c80712d0 0x7f77c8074000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:56.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.604+0000 7f77d0400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f77c8141900 con 0x7f77c80712d0 2024-07-04T17:34:56.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.604+0000 7f77cda00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77c80712d0 0x7f77c8074000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:56.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.604+0000 7f77cda00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77c80712d0 0x7f77c8074000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56012/0 (socket says 172.21.15.42:56012) 2024-07-04T17:34:56.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.604+0000 7f77cda00640 1 -- 172.21.15.42:0/2825221987 learned_addr learned my addr 172.21.15.42:0/2825221987 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:56.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.604+0000 7f77cda00640 1 -- 172.21.15.42:0/2825221987 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c81e3830 con 0x7f77c80712d0 2024-07-04T17:34:56.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.605+0000 7f77cda00640 1 --2- 172.21.15.42:0/2825221987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77c80712d0 0x7f77c8074000 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f77bc0098d0 tx=0x7f77bc033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:56.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.605+0000 7f77c5600640 1 -- 172.21.15.42:0/2825221987 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77bc031bc0 con 0x7f77c80712d0 2024-07-04T17:34:56.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.605+0000 7f77c5600640 1 -- 172.21.15.42:0/2825221987 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77bc031e50 con 0x7f77c80712d0 2024-07-04T17:34:56.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.605+0000 7f77d0400640 1 -- 172.21.15.42:0/2825221987 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c813d6e0 con 0x7f77c80712d0 2024-07-04T17:34:56.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.605+0000 7f77c5600640 1 -- 172.21.15.42:0/2825221987 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77bc033200 con 0x7f77c80712d0 2024-07-04T17:34:56.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.605+0000 7f77d0400640 1 -- 172.21.15.42:0/2825221987 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c813dbb0 con 0x7f77c80712d0 2024-07-04T17:34:56.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.607+0000 7f77d0400640 1 -- 172.21.15.42:0/2825221987 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77c806d130 con 0x7f77c80712d0 2024-07-04T17:34:56.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.607+0000 7f77c5600640 1 -- 172.21.15.42:0/2825221987 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f77bc0333a0 con 0x7f77c80712d0 2024-07-04T17:34:56.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.608+0000 7f77c5600640 1 --2- 172.21.15.42:0/2825221987 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f77a803d4f0 0x7f77a803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:56.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.608+0000 7f77c5600640 1 -- 172.21.15.42:0/2825221987 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 4303+0+0 (secure 0 0 0) 0x7f77bc073150 con 0x7f77c80712d0 2024-07-04T17:34:56.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.608+0000 7f77cd000640 1 --2- 172.21.15.42:0/2825221987 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f77a803d4f0 0x7f77a803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:56.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.609+0000 7f77cd000640 1 --2- 172.21.15.42:0/2825221987 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f77a803d4f0 0x7f77a803fa00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f77b40015c0 tx=0x7f77b4007a90 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:56.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.613+0000 7f77c5600640 1 -- 172.21.15.42:0/2825221987 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f77c81e3830 con 0x7f77c80712d0 2024-07-04T17:34:56.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:56.870+0000 7f77d0400640 1 -- 172.21.15.42:0/2825221987 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f77c8149370 con 0x7f77c80712d0 2024-07-04T17:34:57.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.235+0000 7f77c5600640 1 -- 172.21.15.42:0/2825221987 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"}]=0 v112) ==== 121+0+0 (secure 0 0 0) 0x7f77bc03f030 con 0x7f77c80712d0 2024-07-04T17:34:57.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.238+0000 7f77d0400640 1 -- 172.21.15.42:0/2825221987 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f77a803d4f0 msgr2=0x7f77a803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:57.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.238+0000 7f77d0400640 1 --2- 172.21.15.42:0/2825221987 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f77a803d4f0 0x7f77a803fa00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f77b40015c0 tx=0x7f77b4007a90 comp rx=0 tx=0).stop 2024-07-04T17:34:57.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.238+0000 7f77d0400640 1 -- 172.21.15.42:0/2825221987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77c80712d0 msgr2=0x7f77c8074000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:57.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.238+0000 7f77d0400640 1 --2- 172.21.15.42:0/2825221987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77c80712d0 0x7f77c8074000 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f77bc0098d0 tx=0x7f77bc033e40 comp rx=0 tx=0).stop 2024-07-04T17:34:57.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.239+0000 7f77d0400640 1 -- 172.21.15.42:0/2825221987 shutdown_connections 2024-07-04T17:34:57.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.239+0000 7f77d0400640 1 --2- 172.21.15.42:0/2825221987 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f77a803d4f0 0x7f77a803fa00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:57.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.239+0000 7f77d0400640 1 --2- 172.21.15.42:0/2825221987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77c80712d0 0x7f77c8074000 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:57.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.239+0000 7f77d0400640 1 -- 172.21.15.42:0/2825221987 >> 172.21.15.42:0/2825221987 conn(0x7f77c806c020 msgr2=0x7f77c80b09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:57.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.239+0000 7f77d0400640 1 -- 172.21.15.42:0/2825221987 shutdown_connections 2024-07-04T17:34:57.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.239+0000 7f77d0400640 1 -- 172.21.15.42:0/2825221987 wait complete. 2024-07-04T17:34:57.276 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-07-04T17:34:57.276 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs allow_standby_replay false 2024-07-04T17:34:57.459 INFO:tasks.ceph.mds.a.smithi042.stderr:2024-07-04T17:34:57.458+0000 7fe361e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-04T17:34:57.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.574+0000 7f6cf9600640 1 -- 172.21.15.42:0/2160090858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cf40b0db0 msgr2=0x7f6cf40abc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:57.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.574+0000 7f6cf9600640 1 --2- 172.21.15.42:0/2160090858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cf40b0db0 0x7f6cf40abc80 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00b2b0 tx=0x7f6cdc02ac80 comp rx=0 tx=0).stop 2024-07-04T17:34:57.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.575+0000 7f6cf9600640 1 -- 172.21.15.42:0/2160090858 shutdown_connections 2024-07-04T17:34:57.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.575+0000 7f6cf9600640 1 --2- 172.21.15.42:0/2160090858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cf40b0db0 0x7f6cf40abc80 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:57.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.575+0000 7f6cf9600640 1 -- 172.21.15.42:0/2160090858 >> 172.21.15.42:0/2160090858 conn(0x7f6cf40afaa0 msgr2=0x7f6cf40afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:57.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.575+0000 7f6cf9600640 1 -- 172.21.15.42:0/2160090858 shutdown_connections 2024-07-04T17:34:57.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.575+0000 7f6cf9600640 1 -- 172.21.15.42:0/2160090858 wait complete. 2024-07-04T17:34:57.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.577+0000 7f6cf9600640 1 Processor -- start 2024-07-04T17:34:57.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.577+0000 7f6cf9600640 1 -- start start 2024-07-04T17:34:57.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.579+0000 7f6cf9600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cf40ac5b0 0x7f6cf41d4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:57.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.579+0000 7f6cf9600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf40ae590 con 0x7f6cf40ac5b0 2024-07-04T17:34:57.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.579+0000 7f6cf3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cf40ac5b0 0x7f6cf41d4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:57.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.579+0000 7f6cf3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cf40ac5b0 0x7f6cf41d4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56044/0 (socket says 172.21.15.42:56044) 2024-07-04T17:34:57.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.579+0000 7f6cf3e00640 1 -- 172.21.15.42:0/529508598 learned_addr learned my addr 172.21.15.42:0/529508598 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:57.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.579+0000 7f6cf3e00640 1 -- 172.21.15.42:0/529508598 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf41d6c10 con 0x7f6cf40ac5b0 2024-07-04T17:34:57.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.580+0000 7f6cf3e00640 1 --2- 172.21.15.42:0/529508598 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cf40ac5b0 0x7f6cf41d4c10 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400c380 tx=0x7f6ce400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:57.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.580+0000 7f6cf0c00640 1 -- 172.21.15.42:0/529508598 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ce40116c0 con 0x7f6cf40ac5b0 2024-07-04T17:34:57.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.580+0000 7f6cf0c00640 1 -- 172.21.15.42:0/529508598 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ce4017070 con 0x7f6cf40ac5b0 2024-07-04T17:34:57.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.580+0000 7f6cf0c00640 1 -- 172.21.15.42:0/529508598 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ce4011b20 con 0x7f6cf40ac5b0 2024-07-04T17:34:57.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.580+0000 7f6cf9600640 1 -- 172.21.15.42:0/529508598 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cf413ae80 con 0x7f6cf40ac5b0 2024-07-04T17:34:57.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.580+0000 7f6cf9600640 1 -- 172.21.15.42:0/529508598 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cf41e4480 con 0x7f6cf40ac5b0 2024-07-04T17:34:57.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.582+0000 7f6cf9600640 1 -- 172.21.15.42:0/529508598 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cf40abc80 con 0x7f6cf40ac5b0 2024-07-04T17:34:57.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.582+0000 7f6cf0c00640 1 -- 172.21.15.42:0/529508598 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f6ce4011cc0 con 0x7f6cf40ac5b0 2024-07-04T17:34:57.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.582+0000 7f6cf0c00640 1 --2- 172.21.15.42:0/529508598 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f6cd003d520 0x7f6cd003fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:57.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.582+0000 7f6cf0c00640 1 -- 172.21.15.42:0/529508598 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 4303+0+0 (secure 0 0 0) 0x7f6ce40518e0 con 0x7f6cf40ac5b0 2024-07-04T17:34:57.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.583+0000 7f6ceb400640 1 --2- 172.21.15.42:0/529508598 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f6cd003d520 0x7f6cd003fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:57.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.583+0000 7f6ceb400640 1 --2- 172.21.15.42:0/529508598 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f6cd003d520 0x7f6cd003fa30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc035000 tx=0x7f6cdc004730 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:57.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.587+0000 7f6cf0c00640 1 -- 172.21.15.42:0/529508598 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f6ce4010cf0 con 0x7f6cf40ac5b0 2024-07-04T17:34:57.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:57.839+0000 7f6cf9600640 1 -- 172.21.15.42:0/529508598 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f6cf41d6930 con 0x7f6cf40ac5b0 2024-07-04T17:34:58.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.237+0000 7f6cf0c00640 1 -- 172.21.15.42:0/529508598 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"}]=0 v113) ==== 125+0+0 (secure 0 0 0) 0x7f6ce4019310 con 0x7f6cf40ac5b0 2024-07-04T17:34:58.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.240+0000 7f6cf9600640 1 -- 172.21.15.42:0/529508598 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f6cd003d520 msgr2=0x7f6cd003fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:58.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.240+0000 7f6cf9600640 1 --2- 172.21.15.42:0/529508598 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f6cd003d520 0x7f6cd003fa30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc035000 tx=0x7f6cdc004730 comp rx=0 tx=0).stop 2024-07-04T17:34:58.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.241+0000 7f6cf9600640 1 -- 172.21.15.42:0/529508598 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cf40ac5b0 msgr2=0x7f6cf41d4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:58.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.241+0000 7f6cf9600640 1 --2- 172.21.15.42:0/529508598 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cf40ac5b0 0x7f6cf41d4c10 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400c380 tx=0x7f6ce400c840 comp rx=0 tx=0).stop 2024-07-04T17:34:58.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.241+0000 7f6cf9600640 1 -- 172.21.15.42:0/529508598 shutdown_connections 2024-07-04T17:34:58.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.241+0000 7f6cf9600640 1 --2- 172.21.15.42:0/529508598 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f6cd003d520 0x7f6cd003fa30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:58.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.241+0000 7f6cf9600640 1 --2- 172.21.15.42:0/529508598 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cf40ac5b0 0x7f6cf41d4c10 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:58.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.241+0000 7f6cf9600640 1 -- 172.21.15.42:0/529508598 >> 172.21.15.42:0/529508598 conn(0x7f6cf40afaa0 msgr2=0x7f6cf4141180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:58.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.241+0000 7f6cf9600640 1 -- 172.21.15.42:0/529508598 shutdown_connections 2024-07-04T17:34:58.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.241+0000 7f6cf9600640 1 -- 172.21.15.42:0/529508598 wait complete. 2024-07-04T17:34:58.276 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'backup_fs' 2024-07-04T17:34:58.276 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:34:58.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.565+0000 7f18bce00640 1 -- 172.21.15.42:0/200939497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18b80b17d0 msgr2=0x7f18b80b1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:58.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.565+0000 7f18bce00640 1 --2- 172.21.15.42:0/200939497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18b80b17d0 0x7f18b80b1be0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f18a40097b0 tx=0x7f18a402a4b0 comp rx=0 tx=0).stop 2024-07-04T17:34:58.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.565+0000 7f18bce00640 1 -- 172.21.15.42:0/200939497 shutdown_connections 2024-07-04T17:34:58.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.565+0000 7f18bce00640 1 --2- 172.21.15.42:0/200939497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18b80b17d0 0x7f18b80b1be0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:58.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.565+0000 7f18bce00640 1 -- 172.21.15.42:0/200939497 >> 172.21.15.42:0/200939497 conn(0x7f18b80bebc0 msgr2=0x7f18b80befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:58.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.565+0000 7f18bce00640 1 -- 172.21.15.42:0/200939497 shutdown_connections 2024-07-04T17:34:58.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.565+0000 7f18bce00640 1 -- 172.21.15.42:0/200939497 wait complete. 2024-07-04T17:34:58.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.567+0000 7f18bce00640 1 Processor -- start 2024-07-04T17:34:58.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.567+0000 7f18bce00640 1 -- start start 2024-07-04T17:34:58.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.569+0000 7f18bce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18b806c200 0x7f18b8147fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:58.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.569+0000 7f18bce00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f18b814ae80 con 0x7f18b806c200 2024-07-04T17:34:58.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.569+0000 7f18b7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18b806c200 0x7f18b8147fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:58.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.569+0000 7f18b7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18b806c200 0x7f18b8147fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56064/0 (socket says 172.21.15.42:56064) 2024-07-04T17:34:58.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.569+0000 7f18b7400640 1 -- 172.21.15.42:0/3472024921 learned_addr learned my addr 172.21.15.42:0/3472024921 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:58.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.569+0000 7f18b7400640 1 -- 172.21.15.42:0/3472024921 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b806fc90 con 0x7f18b806c200 2024-07-04T17:34:58.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.570+0000 7f18b7400640 1 --2- 172.21.15.42:0/3472024921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18b806c200 0x7f18b8147fd0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f18a800c380 tx=0x7f18a800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:58.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.570+0000 7f18af400640 1 -- 172.21.15.42:0/3472024921 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18a80116c0 con 0x7f18b806c200 2024-07-04T17:34:58.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.570+0000 7f18af400640 1 -- 172.21.15.42:0/3472024921 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18a8017070 con 0x7f18b806c200 2024-07-04T17:34:58.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.570+0000 7f18af400640 1 -- 172.21.15.42:0/3472024921 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18a8011b20 con 0x7f18b806c200 2024-07-04T17:34:58.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.570+0000 7f18bce00640 1 -- 172.21.15.42:0/3472024921 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b81e12a0 con 0x7f18b806c200 2024-07-04T17:34:58.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.570+0000 7f18bce00640 1 -- 172.21.15.42:0/3472024921 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b8073430 con 0x7f18b806c200 2024-07-04T17:34:58.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.571+0000 7f18bce00640 1 -- 172.21.15.42:0/3472024921 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18b80b44b0 con 0x7f18b806c200 2024-07-04T17:34:58.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.571+0000 7f18af400640 1 -- 172.21.15.42:0/3472024921 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f18a8011cc0 con 0x7f18b806c200 2024-07-04T17:34:58.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.572+0000 7f18af400640 1 --2- 172.21.15.42:0/3472024921 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f189803d4f0 0x7f189803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:58.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.572+0000 7f18af400640 1 -- 172.21.15.42:0/3472024921 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 4303+0+0 (secure 0 0 0) 0x7f18a80519f0 con 0x7f18b806c200 2024-07-04T17:34:58.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.572+0000 7f18b6a00640 1 --2- 172.21.15.42:0/3472024921 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f189803d4f0 0x7f189803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:58.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.573+0000 7f18b6a00640 1 --2- 172.21.15.42:0/3472024921 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f189803d4f0 0x7f189803fa00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f18a4034000 tx=0x7f18a4004730 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:58.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.579+0000 7f18af400640 1 -- 172.21.15.42:0/3472024921 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f18b80b44b0 con 0x7f18b806c200 2024-07-04T17:34:58.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.830+0000 7f18bce00640 1 -- 172.21.15.42:0/3472024921 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18b814b4c0 con 0x7f18b806c200 2024-07-04T17:34:58.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.831+0000 7f18af400640 1 -- 172.21.15.42:0/3472024921 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+7380 (secure 0 0 0) 0x7f18a8010cf0 con 0x7f18b806c200 2024-07-04T17:34:58.832 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:34:58.833 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":113,"btime":"2024-07-04T17:34:58:236243+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5803,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/1184311185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":1184311185},{"type":"v1","addr":"172.21.15.42:6825","nonce":1184311185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103},{"gid":5808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/1815554749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1815554749},{"type":"v1","addr":"172.21.15.42:6827","nonce":1815554749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":5809,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/2027504764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2027504764},{"type":"v1","addr":"172.21.15.42:6829","nonce":2027504764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:34:35.049685+0000","modified":"2024-07-04T17:34:38.110000+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5818},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5818":{"gid":5818,"name":"d","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6833/1863956336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1863956336},{"type":"v1","addr":"172.21.15.42:6833","nonce":1863956336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5818,"qdb_cluster":[5818]},"id":12},{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:34:54.794041+0000","modified":"2024-07-04T17:34:57.841598+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5817},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5817":{"gid":5817,"name":"a","rank":0,"incarnation":110,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6831/4064998265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":4064998265},{"type":"v1","addr":"172.21.15.42:6831","nonce":4064998265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5817,"qdb_cluster":[5817]},"id":13}]} 2024-07-04T17:34:58.833 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 113 2024-07-04T17:34:58.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.834+0000 7f18bce00640 1 -- 172.21.15.42:0/3472024921 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f189803d4f0 msgr2=0x7f189803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:58.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.834+0000 7f18bce00640 1 --2- 172.21.15.42:0/3472024921 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f189803d4f0 0x7f189803fa00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f18a4034000 tx=0x7f18a4004730 comp rx=0 tx=0).stop 2024-07-04T17:34:58.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.834+0000 7f18bce00640 1 -- 172.21.15.42:0/3472024921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18b806c200 msgr2=0x7f18b8147fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:58.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.834+0000 7f18bce00640 1 --2- 172.21.15.42:0/3472024921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18b806c200 0x7f18b8147fd0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f18a800c380 tx=0x7f18a800c840 comp rx=0 tx=0).stop 2024-07-04T17:34:58.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.835+0000 7f18bce00640 1 -- 172.21.15.42:0/3472024921 shutdown_connections 2024-07-04T17:34:58.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.835+0000 7f18bce00640 1 --2- 172.21.15.42:0/3472024921 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f189803d4f0 0x7f189803fa00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:58.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.835+0000 7f18bce00640 1 --2- 172.21.15.42:0/3472024921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18b806c200 0x7f18b8147fd0 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:58.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.835+0000 7f18bce00640 1 -- 172.21.15.42:0/3472024921 >> 172.21.15.42:0/3472024921 conn(0x7f18b80bebc0 msgr2=0x7f18b81d72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:58.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.835+0000 7f18bce00640 1 -- 172.21.15.42:0/3472024921 shutdown_connections 2024-07-04T17:34:58.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:58.835+0000 7f18bce00640 1 -- 172.21.15.42:0/3472024921 wait complete. 2024-07-04T17:34:58.861 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:34:59.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.151+0000 7f6d58e00640 1 -- 172.21.15.42:0/3801746829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d540bc040 msgr2=0x7f6d540bd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:59.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.151+0000 7f6d58e00640 1 --2- 172.21.15.42:0/3801746829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d540bc040 0x7f6d540bd2d0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f6d440097b0 tx=0x7f6d4402a530 comp rx=0 tx=0).stop 2024-07-04T17:34:59.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.151+0000 7f6d58e00640 1 -- 172.21.15.42:0/3801746829 shutdown_connections 2024-07-04T17:34:59.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.151+0000 7f6d58e00640 1 --2- 172.21.15.42:0/3801746829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d540bc040 0x7f6d540bd2d0 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:59.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.151+0000 7f6d58e00640 1 -- 172.21.15.42:0/3801746829 >> 172.21.15.42:0/3801746829 conn(0x7f6d5406c940 msgr2=0x7f6d5406ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:59.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.151+0000 7f6d58e00640 1 -- 172.21.15.42:0/3801746829 shutdown_connections 2024-07-04T17:34:59.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.151+0000 7f6d58e00640 1 -- 172.21.15.42:0/3801746829 wait complete. 2024-07-04T17:34:59.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.153+0000 7f6d58e00640 1 Processor -- start 2024-07-04T17:34:59.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.153+0000 7f6d58e00640 1 -- start start 2024-07-04T17:34:59.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.154+0000 7f6d58e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d540bc040 0x7f6d540bfae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:59.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.154+0000 7f6d58e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6d540b7530 con 0x7f6d540bc040 2024-07-04T17:34:59.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.155+0000 7f6d51e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d540bc040 0x7f6d540bfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:59.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.155+0000 7f6d51e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d540bc040 0x7f6d540bfae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56094/0 (socket says 172.21.15.42:56094) 2024-07-04T17:34:59.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.155+0000 7f6d51e00640 1 -- 172.21.15.42:0/2784021554 learned_addr learned my addr 172.21.15.42:0/2784021554 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:59.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.155+0000 7f6d51e00640 1 -- 172.21.15.42:0/2784021554 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d540c25e0 con 0x7f6d540bc040 2024-07-04T17:34:59.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.156+0000 7f6d51e00640 1 --2- 172.21.15.42:0/2784021554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d540bc040 0x7f6d540bfae0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f6d440098d0 tx=0x7f6d44033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:59.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.156+0000 7f6d4a000640 1 -- 172.21.15.42:0/2784021554 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d44031880 con 0x7f6d540bc040 2024-07-04T17:34:59.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.156+0000 7f6d4a000640 1 -- 172.21.15.42:0/2784021554 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d44031b10 con 0x7f6d540bc040 2024-07-04T17:34:59.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.156+0000 7f6d4a000640 1 -- 172.21.15.42:0/2784021554 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d44031df0 con 0x7f6d540bc040 2024-07-04T17:34:59.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.156+0000 7f6d58e00640 1 -- 172.21.15.42:0/2784021554 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d5413fba0 con 0x7f6d540bc040 2024-07-04T17:34:59.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.156+0000 7f6d58e00640 1 -- 172.21.15.42:0/2784021554 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d540c1470 con 0x7f6d540bc040 2024-07-04T17:34:59.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.157+0000 7f6d58e00640 1 -- 172.21.15.42:0/2784021554 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d540b7c00 con 0x7f6d540bc040 2024-07-04T17:34:59.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.158+0000 7f6d4a000640 1 -- 172.21.15.42:0/2784021554 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f6d44033070 con 0x7f6d540bc040 2024-07-04T17:34:59.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.158+0000 7f6d4a000640 1 --2- 172.21.15.42:0/2784021554 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f6d3003d4f0 0x7f6d3003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:59.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.158+0000 7f6d4a000640 1 -- 172.21.15.42:0/2784021554 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 4303+0+0 (secure 0 0 0) 0x7f6d44073080 con 0x7f6d540bc040 2024-07-04T17:34:59.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.159+0000 7f6d51400640 1 --2- 172.21.15.42:0/2784021554 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f6d3003d4f0 0x7f6d3003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:59.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.159+0000 7f6d51400640 1 --2- 172.21.15.42:0/2784021554 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f6d3003d4f0 0x7f6d3003fa00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c002020 tx=0x7f6d3c001670 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:59.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.163+0000 7f6d4a000640 1 -- 172.21.15.42:0/2784021554 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f6d4403f030 con 0x7f6d540bc040 2024-07-04T17:34:59.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.380+0000 7f6d58e00640 1 -- 172.21.15.42:0/2784021554 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d540c23b0 con 0x7f6d540bc040 2024-07-04T17:34:59.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.382+0000 7f6d4a000640 1 -- 172.21.15.42:0/2784021554 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) ==== 74+0+14458 (secure 0 0 0) 0x7f6d4403a3a0 con 0x7f6d540bc040 2024-07-04T17:34:59.383 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:34:59.384 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":140,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:34:56.219969+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:34:33.191097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-07-04T17:34:34.033408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:34:52.310342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:34:53.756688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:34:59.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.385+0000 7f6d58e00640 1 -- 172.21.15.42:0/2784021554 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f6d3003d4f0 msgr2=0x7f6d3003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:59.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.385+0000 7f6d58e00640 1 --2- 172.21.15.42:0/2784021554 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f6d3003d4f0 0x7f6d3003fa00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c002020 tx=0x7f6d3c001670 comp rx=0 tx=0).stop 2024-07-04T17:34:59.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.385+0000 7f6d58e00640 1 -- 172.21.15.42:0/2784021554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d540bc040 msgr2=0x7f6d540bfae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:59.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.385+0000 7f6d58e00640 1 --2- 172.21.15.42:0/2784021554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d540bc040 0x7f6d540bfae0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f6d440098d0 tx=0x7f6d44033ad0 comp rx=0 tx=0).stop 2024-07-04T17:34:59.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.385+0000 7f6d58e00640 1 -- 172.21.15.42:0/2784021554 shutdown_connections 2024-07-04T17:34:59.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.385+0000 7f6d58e00640 1 --2- 172.21.15.42:0/2784021554 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f6d3003d4f0 0x7f6d3003fa00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:59.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.385+0000 7f6d58e00640 1 --2- 172.21.15.42:0/2784021554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d540bc040 0x7f6d540bfae0 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:59.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.385+0000 7f6d58e00640 1 -- 172.21.15.42:0/2784021554 >> 172.21.15.42:0/2784021554 conn(0x7f6d5406c940 msgr2=0x7f6d54140eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:59.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.386+0000 7f6d58e00640 1 -- 172.21.15.42:0/2784021554 shutdown_connections 2024-07-04T17:34:59.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.386+0000 7f6d58e00640 1 -- 172.21.15.42:0/2784021554 wait complete. 2024-07-04T17:34:59.411 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-04T17:34:59.412 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:34:59.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.710+0000 7f6f2d800640 1 -- 172.21.15.42:0/3877254237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f2806c2c0 msgr2=0x7f6f280b5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:59.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.710+0000 7f6f2d800640 1 --2- 172.21.15.42:0/3877254237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f2806c2c0 0x7f6f280b5650 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f6f1000b2b0 tx=0x7f6f1002ac80 comp rx=0 tx=0).stop 2024-07-04T17:34:59.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.710+0000 7f6f2d800640 1 -- 172.21.15.42:0/3877254237 shutdown_connections 2024-07-04T17:34:59.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.710+0000 7f6f2d800640 1 --2- 172.21.15.42:0/3877254237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f2806c2c0 0x7f6f280b5650 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:59.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.710+0000 7f6f2d800640 1 -- 172.21.15.42:0/3877254237 >> 172.21.15.42:0/3877254237 conn(0x7f6f280bebc0 msgr2=0x7f6f280befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:59.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.710+0000 7f6f2d800640 1 -- 172.21.15.42:0/3877254237 shutdown_connections 2024-07-04T17:34:59.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.710+0000 7f6f2d800640 1 -- 172.21.15.42:0/3877254237 wait complete. 2024-07-04T17:34:59.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.712+0000 7f6f2d800640 1 Processor -- start 2024-07-04T17:34:59.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.712+0000 7f6f2d800640 1 -- start start 2024-07-04T17:34:59.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.713+0000 7f6f2d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f2806c2c0 0x7f6f280701f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:59.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.713+0000 7f6f2d800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6f280b2810 con 0x7f6f2806c2c0 2024-07-04T17:34:59.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.714+0000 7f6f27e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f2806c2c0 0x7f6f280701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:59.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.714+0000 7f6f27e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f2806c2c0 0x7f6f280701f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56114/0 (socket says 172.21.15.42:56114) 2024-07-04T17:34:59.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.714+0000 7f6f27e00640 1 -- 172.21.15.42:0/3679985443 learned_addr learned my addr 172.21.15.42:0/3679985443 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:34:59.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.714+0000 7f6f27e00640 1 -- 172.21.15.42:0/3679985443 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f281e3730 con 0x7f6f2806c2c0 2024-07-04T17:34:59.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.714+0000 7f6f27e00640 1 --2- 172.21.15.42:0/3679985443 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f2806c2c0 0x7f6f280701f0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800c380 tx=0x7f6f1800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:59.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.715+0000 7f6f1fe00640 1 -- 172.21.15.42:0/3679985443 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f180116c0 con 0x7f6f2806c2c0 2024-07-04T17:34:59.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.715+0000 7f6f1fe00640 1 -- 172.21.15.42:0/3679985443 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f18017070 con 0x7f6f2806c2c0 2024-07-04T17:34:59.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.715+0000 7f6f1fe00640 1 -- 172.21.15.42:0/3679985443 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f18011b20 con 0x7f6f2806c2c0 2024-07-04T17:34:59.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.715+0000 7f6f2d800640 1 -- 172.21.15.42:0/3679985443 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f281e3400 con 0x7f6f2806c2c0 2024-07-04T17:34:59.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.715+0000 7f6f2d800640 1 -- 172.21.15.42:0/3679985443 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f280b07a0 con 0x7f6f2806c2c0 2024-07-04T17:34:59.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.716+0000 7f6f2d800640 1 -- 172.21.15.42:0/3679985443 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f280b5650 con 0x7f6f2806c2c0 2024-07-04T17:34:59.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.716+0000 7f6f1fe00640 1 -- 172.21.15.42:0/3679985443 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f6f18011cc0 con 0x7f6f2806c2c0 2024-07-04T17:34:59.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.717+0000 7f6f1fe00640 1 --2- 172.21.15.42:0/3679985443 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f6f0403d4f0 0x7f6f0403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:34:59.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.717+0000 7f6f1fe00640 1 -- 172.21.15.42:0/3679985443 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 4303+0+0 (secure 0 0 0) 0x7f6f180519f0 con 0x7f6f2806c2c0 2024-07-04T17:34:59.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.717+0000 7f6f27400640 1 --2- 172.21.15.42:0/3679985443 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f6f0403d4f0 0x7f6f0403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:34:59.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.718+0000 7f6f27400640 1 --2- 172.21.15.42:0/3679985443 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f6f0403d4f0 0x7f6f0403fa00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6f10035000 tx=0x7f6f10004730 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:34:59.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.722+0000 7f6f1fe00640 1 -- 172.21.15.42:0/3679985443 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f6f2806c6d0 con 0x7f6f2806c2c0 2024-07-04T17:34:59.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.927+0000 7f6f2d800640 1 -- 172.21.15.42:0/3679985443 --> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6f280715c0 con 0x7f6f0403d4f0 2024-07-04T17:34:59.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.942+0000 7f6f1fe00640 1 -- 172.21.15.42:0/3679985443 <== mgr.5692 v2:172.21.15.42:6834/4064094064 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+483258 (secure 0 0 0) 0x7f6f280715c0 con 0x7f6f0403d4f0 2024-07-04T17:34:59.951 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:34:59.954 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:34:59.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.952+0000 7f6f2d800640 1 -- 172.21.15.42:0/3679985443 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f6f0403d4f0 msgr2=0x7f6f0403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:59.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.953+0000 7f6f2d800640 1 --2- 172.21.15.42:0/3679985443 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f6f0403d4f0 0x7f6f0403fa00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6f10035000 tx=0x7f6f10004730 comp rx=0 tx=0).stop 2024-07-04T17:34:59.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.953+0000 7f6f2d800640 1 -- 172.21.15.42:0/3679985443 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f2806c2c0 msgr2=0x7f6f280701f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:34:59.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.953+0000 7f6f2d800640 1 --2- 172.21.15.42:0/3679985443 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f2806c2c0 0x7f6f280701f0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800c380 tx=0x7f6f1800c840 comp rx=0 tx=0).stop 2024-07-04T17:34:59.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.953+0000 7f6f2d800640 1 -- 172.21.15.42:0/3679985443 shutdown_connections 2024-07-04T17:34:59.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.953+0000 7f6f2d800640 1 --2- 172.21.15.42:0/3679985443 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f6f0403d4f0 0x7f6f0403fa00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:59.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.953+0000 7f6f2d800640 1 --2- 172.21.15.42:0/3679985443 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f2806c2c0 0x7f6f280701f0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:34:59.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.953+0000 7f6f2d800640 1 -- 172.21.15.42:0/3679985443 >> 172.21.15.42:0/3679985443 conn(0x7f6f280bebc0 msgr2=0x7f6f2813fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:34:59.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.953+0000 7f6f2d800640 1 -- 172.21.15.42:0/3679985443 shutdown_connections 2024-07-04T17:34:59.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:34:59.953+0000 7f6f2d800640 1 -- 172.21.15.42:0/3679985443 wait complete. 2024-07-04T17:34:59.982 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":45,"stamp":"2024-07-04T17:34:59.116184+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596885,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":596,"num_read_kb":1630,"num_write":279,"num_write_kb":1137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":195,"ondisk_log_size":195,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":362,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":111552,"kb_used_data":5524,"kb_used_omap":4,"kb_used_meta":105979,"kb_avail":283003968,"statfs":{"total":289910292480,"available":289796063232,"internally_reserved":0,"allocated":5656576,"data_stored":2931766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":108522831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.923660"},"pg_stats":[{"pgid":"27.3d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.236651+0000","last_change":"2024-07-04T17:34:55.282061+0000","last_active":"2024-07-04T17:34:56.236651+0000","last_peered":"2024-07-04T17:34:56.236651+0000","last_clean":"2024-07-04T17:34:56.236651+0000","last_became_active":"2024-07-04T17:34:55.281584+0000","last_became_peered":"2024-07-04T17:34:55.281584+0000","last_unstale":"2024-07-04T17:34:56.236651+0000","last_undegraded":"2024-07-04T17:34:56.236651+0000","last_fullsized":"2024-07-04T17:34:56.236651+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:05:12.120118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3f","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.248651+0000","last_change":"2024-07-04T17:34:35.545191+0000","last_active":"2024-07-04T17:34:56.248651+0000","last_peered":"2024-07-04T17:34:56.248651+0000","last_clean":"2024-07-04T17:34:56.248651+0000","last_became_active":"2024-07-04T17:34:35.542859+0000","last_became_peered":"2024-07-04T17:34:35.542859+0000","last_unstale":"2024-07-04T17:34:56.248651+0000","last_undegraded":"2024-07-04T17:34:56.248651+0000","last_fullsized":"2024-07-04T17:34:56.248651+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:36:40.642306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.243433+0000","last_change":"2024-07-04T17:34:55.246499+0000","last_active":"2024-07-04T17:34:56.243433+0000","last_peered":"2024-07-04T17:34:56.243433+0000","last_clean":"2024-07-04T17:34:56.243433+0000","last_became_active":"2024-07-04T17:34:55.240565+0000","last_became_peered":"2024-07-04T17:34:55.240565+0000","last_unstale":"2024-07-04T17:34:56.243433+0000","last_undegraded":"2024-07-04T17:34:56.243433+0000","last_fullsized":"2024-07-04T17:34:56.243433+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:04:06.751483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.247337+0000","last_change":"2024-07-04T17:34:35.541970+0000","last_active":"2024-07-04T17:34:56.247337+0000","last_peered":"2024-07-04T17:34:56.247337+0000","last_clean":"2024-07-04T17:34:56.247337+0000","last_became_active":"2024-07-04T17:34:35.541341+0000","last_became_peered":"2024-07-04T17:34:35.541341+0000","last_unstale":"2024-07-04T17:34:56.247337+0000","last_undegraded":"2024-07-04T17:34:56.247337+0000","last_fullsized":"2024-07-04T17:34:56.247337+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:06:30.239920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582457+0000","last_change":"2024-07-04T17:34:55.277627+0000","last_active":"2024-07-04T17:34:58.582457+0000","last_peered":"2024-07-04T17:34:58.582457+0000","last_clean":"2024-07-04T17:34:58.582457+0000","last_became_active":"2024-07-04T17:34:55.276422+0000","last_became_peered":"2024-07-04T17:34:55.276422+0000","last_unstale":"2024-07-04T17:34:58.582457+0000","last_undegraded":"2024-07-04T17:34:58.582457+0000","last_fullsized":"2024-07-04T17:34:58.582457+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:41:17.056030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.245821+0000","last_change":"2024-07-04T17:34:35.531108+0000","last_active":"2024-07-04T17:34:56.245821+0000","last_peered":"2024-07-04T17:34:56.245821+0000","last_clean":"2024-07-04T17:34:56.245821+0000","last_became_active":"2024-07-04T17:34:35.530755+0000","last_became_peered":"2024-07-04T17:34:35.530755+0000","last_unstale":"2024-07-04T17:34:56.245821+0000","last_undegraded":"2024-07-04T17:34:56.245821+0000","last_fullsized":"2024-07-04T17:34:56.245821+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:48:46.716399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.239846+0000","last_change":"2024-07-04T17:34:55.281142+0000","last_active":"2024-07-04T17:34:56.239846+0000","last_peered":"2024-07-04T17:34:56.239846+0000","last_clean":"2024-07-04T17:34:56.239846+0000","last_became_active":"2024-07-04T17:34:55.280578+0000","last_became_peered":"2024-07-04T17:34:55.280578+0000","last_unstale":"2024-07-04T17:34:56.239846+0000","last_undegraded":"2024-07-04T17:34:56.239846+0000","last_fullsized":"2024-07-04T17:34:56.239846+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:54:48.573105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.244690+0000","last_change":"2024-07-04T17:34:35.534705+0000","last_active":"2024-07-04T17:34:56.244690+0000","last_peered":"2024-07-04T17:34:56.244690+0000","last_clean":"2024-07-04T17:34:56.244690+0000","last_became_active":"2024-07-04T17:34:35.532134+0000","last_became_peered":"2024-07-04T17:34:35.532134+0000","last_unstale":"2024-07-04T17:34:56.244690+0000","last_undegraded":"2024-07-04T17:34:56.244690+0000","last_fullsized":"2024-07-04T17:34:56.244690+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:04:01.056622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.578758+0000","last_change":"2024-07-04T17:34:55.288989+0000","last_active":"2024-07-04T17:34:58.578758+0000","last_peered":"2024-07-04T17:34:58.578758+0000","last_clean":"2024-07-04T17:34:58.578758+0000","last_became_active":"2024-07-04T17:34:55.287981+0000","last_became_peered":"2024-07-04T17:34:55.287981+0000","last_unstale":"2024-07-04T17:34:58.578758+0000","last_undegraded":"2024-07-04T17:34:58.578758+0000","last_fullsized":"2024-07-04T17:34:58.578758+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:12:26.839696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.237095+0000","last_change":"2024-07-04T17:34:35.529627+0000","last_active":"2024-07-04T17:34:56.237095+0000","last_peered":"2024-07-04T17:34:56.237095+0000","last_clean":"2024-07-04T17:34:56.237095+0000","last_became_active":"2024-07-04T17:34:35.528998+0000","last_became_peered":"2024-07-04T17:34:35.528998+0000","last_unstale":"2024-07-04T17:34:56.237095+0000","last_undegraded":"2024-07-04T17:34:56.237095+0000","last_fullsized":"2024-07-04T17:34:56.237095+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:53:22.203053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.241804+0000","last_change":"2024-07-04T17:34:55.258684+0000","last_active":"2024-07-04T17:34:56.241804+0000","last_peered":"2024-07-04T17:34:56.241804+0000","last_clean":"2024-07-04T17:34:56.241804+0000","last_became_active":"2024-07-04T17:34:55.254309+0000","last_became_peered":"2024-07-04T17:34:55.254309+0000","last_unstale":"2024-07-04T17:34:56.241804+0000","last_undegraded":"2024-07-04T17:34:56.241804+0000","last_fullsized":"2024-07-04T17:34:56.241804+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:23:01.177074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.244028+0000","last_change":"2024-07-04T17:34:35.517987+0000","last_active":"2024-07-04T17:34:56.244028+0000","last_peered":"2024-07-04T17:34:56.244028+0000","last_clean":"2024-07-04T17:34:56.244028+0000","last_became_active":"2024-07-04T17:34:35.517658+0000","last_became_peered":"2024-07-04T17:34:35.517658+0000","last_unstale":"2024-07-04T17:34:56.244028+0000","last_undegraded":"2024-07-04T17:34:56.244028+0000","last_fullsized":"2024-07-04T17:34:56.244028+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:49:37.922530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.242707+0000","last_change":"2024-07-04T17:34:55.259361+0000","last_active":"2024-07-04T17:34:56.242707+0000","last_peered":"2024-07-04T17:34:56.242707+0000","last_clean":"2024-07-04T17:34:56.242707+0000","last_became_active":"2024-07-04T17:34:55.256213+0000","last_became_peered":"2024-07-04T17:34:55.256213+0000","last_unstale":"2024-07-04T17:34:56.242707+0000","last_undegraded":"2024-07-04T17:34:56.242707+0000","last_fullsized":"2024-07-04T17:34:56.242707+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:27:12.706155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.235049+0000","last_change":"2024-07-04T17:34:35.555625+0000","last_active":"2024-07-04T17:34:56.235049+0000","last_peered":"2024-07-04T17:34:56.235049+0000","last_clean":"2024-07-04T17:34:56.235049+0000","last_became_active":"2024-07-04T17:34:35.554652+0000","last_became_peered":"2024-07-04T17:34:35.554652+0000","last_unstale":"2024-07-04T17:34:56.235049+0000","last_undegraded":"2024-07-04T17:34:56.235049+0000","last_fullsized":"2024-07-04T17:34:56.235049+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:47:42.975532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.242948+0000","last_change":"2024-07-04T17:34:55.239188+0000","last_active":"2024-07-04T17:34:56.242948+0000","last_peered":"2024-07-04T17:34:56.242948+0000","last_clean":"2024-07-04T17:34:56.242948+0000","last_became_active":"2024-07-04T17:34:55.237333+0000","last_became_peered":"2024-07-04T17:34:55.237333+0000","last_unstale":"2024-07-04T17:34:56.242948+0000","last_undegraded":"2024-07-04T17:34:56.242948+0000","last_fullsized":"2024-07-04T17:34:56.242948+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:14:53.439738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.243972+0000","last_change":"2024-07-04T17:34:35.543396+0000","last_active":"2024-07-04T17:34:56.243972+0000","last_peered":"2024-07-04T17:34:56.243972+0000","last_clean":"2024-07-04T17:34:56.243972+0000","last_became_active":"2024-07-04T17:34:35.542958+0000","last_became_peered":"2024-07-04T17:34:35.542958+0000","last_unstale":"2024-07-04T17:34:56.243972+0000","last_undegraded":"2024-07-04T17:34:56.243972+0000","last_fullsized":"2024-07-04T17:34:56.243972+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:37:46.927489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.237183+0000","last_change":"2024-07-04T17:34:55.284874+0000","last_active":"2024-07-04T17:34:56.237183+0000","last_peered":"2024-07-04T17:34:56.237183+0000","last_clean":"2024-07-04T17:34:56.237183+0000","last_became_active":"2024-07-04T17:34:55.284544+0000","last_became_peered":"2024-07-04T17:34:55.284544+0000","last_unstale":"2024-07-04T17:34:56.237183+0000","last_undegraded":"2024-07-04T17:34:56.237183+0000","last_fullsized":"2024-07-04T17:34:56.237183+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:52:31.382449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.248631+0000","last_change":"2024-07-04T17:34:35.543199+0000","last_active":"2024-07-04T17:34:56.248631+0000","last_peered":"2024-07-04T17:34:56.248631+0000","last_clean":"2024-07-04T17:34:56.248631+0000","last_became_active":"2024-07-04T17:34:35.537240+0000","last_became_peered":"2024-07-04T17:34:35.537240+0000","last_unstale":"2024-07-04T17:34:56.248631+0000","last_undegraded":"2024-07-04T17:34:56.248631+0000","last_fullsized":"2024-07-04T17:34:56.248631+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:17:07.663136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.579818+0000","last_change":"2024-07-04T17:34:55.288619+0000","last_active":"2024-07-04T17:34:58.579818+0000","last_peered":"2024-07-04T17:34:58.579818+0000","last_clean":"2024-07-04T17:34:58.579818+0000","last_became_active":"2024-07-04T17:34:55.288057+0000","last_became_peered":"2024-07-04T17:34:55.288057+0000","last_unstale":"2024-07-04T17:34:58.579818+0000","last_undegraded":"2024-07-04T17:34:58.579818+0000","last_fullsized":"2024-07-04T17:34:58.579818+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:46:28.164230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582367+0000","last_change":"2024-07-04T17:34:35.552696+0000","last_active":"2024-07-04T17:34:58.582367+0000","last_peered":"2024-07-04T17:34:58.582367+0000","last_clean":"2024-07-04T17:34:58.582367+0000","last_became_active":"2024-07-04T17:34:35.552377+0000","last_became_peered":"2024-07-04T17:34:35.552377+0000","last_unstale":"2024-07-04T17:34:58.582367+0000","last_undegraded":"2024-07-04T17:34:58.582367+0000","last_fullsized":"2024-07-04T17:34:58.582367+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:54:28.842946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.240464+0000","last_change":"2024-07-04T17:34:55.281556+0000","last_active":"2024-07-04T17:34:56.240464+0000","last_peered":"2024-07-04T17:34:56.240464+0000","last_clean":"2024-07-04T17:34:56.240464+0000","last_became_active":"2024-07-04T17:34:55.281090+0000","last_became_peered":"2024-07-04T17:34:55.281090+0000","last_unstale":"2024-07-04T17:34:56.240464+0000","last_undegraded":"2024-07-04T17:34:56.240464+0000","last_fullsized":"2024-07-04T17:34:56.240464+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:41:49.878066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.239112+0000","last_change":"2024-07-04T17:34:35.535880+0000","last_active":"2024-07-04T17:34:56.239112+0000","last_peered":"2024-07-04T17:34:56.239112+0000","last_clean":"2024-07-04T17:34:56.239112+0000","last_became_active":"2024-07-04T17:34:35.535547+0000","last_became_peered":"2024-07-04T17:34:35.535547+0000","last_unstale":"2024-07-04T17:34:56.239112+0000","last_undegraded":"2024-07-04T17:34:56.239112+0000","last_fullsized":"2024-07-04T17:34:56.239112+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:22:16.634063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580166+0000","last_change":"2024-07-04T17:34:55.277218+0000","last_active":"2024-07-04T17:34:58.580166+0000","last_peered":"2024-07-04T17:34:58.580166+0000","last_clean":"2024-07-04T17:34:58.580166+0000","last_became_active":"2024-07-04T17:34:55.276877+0000","last_became_peered":"2024-07-04T17:34:55.276877+0000","last_unstale":"2024-07-04T17:34:58.580166+0000","last_undegraded":"2024-07-04T17:34:58.580166+0000","last_fullsized":"2024-07-04T17:34:58.580166+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:47:24.561498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580845+0000","last_change":"2024-07-04T17:34:35.561769+0000","last_active":"2024-07-04T17:34:58.580845+0000","last_peered":"2024-07-04T17:34:58.580845+0000","last_clean":"2024-07-04T17:34:58.580845+0000","last_became_active":"2024-07-04T17:34:35.560437+0000","last_became_peered":"2024-07-04T17:34:35.560437+0000","last_unstale":"2024-07-04T17:34:58.580845+0000","last_undegraded":"2024-07-04T17:34:58.580845+0000","last_fullsized":"2024-07-04T17:34:58.580845+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:52:11.652290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.578833+0000","last_change":"2024-07-04T17:34:55.254016+0000","last_active":"2024-07-04T17:34:58.578833+0000","last_peered":"2024-07-04T17:34:58.578833+0000","last_clean":"2024-07-04T17:34:58.578833+0000","last_became_active":"2024-07-04T17:34:55.253616+0000","last_became_peered":"2024-07-04T17:34:55.253616+0000","last_unstale":"2024-07-04T17:34:58.578833+0000","last_undegraded":"2024-07-04T17:34:58.578833+0000","last_fullsized":"2024-07-04T17:34:58.578833+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:08:01.918120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.244854+0000","last_change":"2024-07-04T17:34:35.514882+0000","last_active":"2024-07-04T17:34:56.244854+0000","last_peered":"2024-07-04T17:34:56.244854+0000","last_clean":"2024-07-04T17:34:56.244854+0000","last_became_active":"2024-07-04T17:34:35.514560+0000","last_became_peered":"2024-07-04T17:34:35.514560+0000","last_unstale":"2024-07-04T17:34:56.244854+0000","last_undegraded":"2024-07-04T17:34:56.244854+0000","last_fullsized":"2024-07-04T17:34:56.244854+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:46:59.706787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.241737+0000","last_change":"2024-07-04T17:34:55.238218+0000","last_active":"2024-07-04T17:34:56.241737+0000","last_peered":"2024-07-04T17:34:56.241737+0000","last_clean":"2024-07-04T17:34:56.241737+0000","last_became_active":"2024-07-04T17:34:55.237053+0000","last_became_peered":"2024-07-04T17:34:55.237053+0000","last_unstale":"2024-07-04T17:34:56.241737+0000","last_undegraded":"2024-07-04T17:34:56.241737+0000","last_fullsized":"2024-07-04T17:34:56.241737+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:47:25.955446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.244101+0000","last_change":"2024-07-04T17:34:35.538102+0000","last_active":"2024-07-04T17:34:56.244101+0000","last_peered":"2024-07-04T17:34:56.244101+0000","last_clean":"2024-07-04T17:34:56.244101+0000","last_became_active":"2024-07-04T17:34:35.536203+0000","last_became_peered":"2024-07-04T17:34:35.536203+0000","last_unstale":"2024-07-04T17:34:56.244101+0000","last_undegraded":"2024-07-04T17:34:56.244101+0000","last_fullsized":"2024-07-04T17:34:56.244101+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:15:42.373781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.244440+0000","last_change":"2024-07-04T17:34:55.260190+0000","last_active":"2024-07-04T17:34:56.244440+0000","last_peered":"2024-07-04T17:34:56.244440+0000","last_clean":"2024-07-04T17:34:56.244440+0000","last_became_active":"2024-07-04T17:34:55.259302+0000","last_became_peered":"2024-07-04T17:34:55.259302+0000","last_unstale":"2024-07-04T17:34:56.244440+0000","last_undegraded":"2024-07-04T17:34:56.244440+0000","last_fullsized":"2024-07-04T17:34:56.244440+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:04:05.855295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.579486+0000","last_change":"2024-07-04T17:34:35.542229+0000","last_active":"2024-07-04T17:34:58.579486+0000","last_peered":"2024-07-04T17:34:58.579486+0000","last_clean":"2024-07-04T17:34:58.579486+0000","last_became_active":"2024-07-04T17:34:35.541754+0000","last_became_peered":"2024-07-04T17:34:35.541754+0000","last_unstale":"2024-07-04T17:34:58.579486+0000","last_undegraded":"2024-07-04T17:34:58.579486+0000","last_fullsized":"2024-07-04T17:34:58.579486+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:35:52.678989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.579473+0000","last_change":"2024-07-04T17:34:55.264141+0000","last_active":"2024-07-04T17:34:58.579473+0000","last_peered":"2024-07-04T17:34:58.579473+0000","last_clean":"2024-07-04T17:34:58.579473+0000","last_became_active":"2024-07-04T17:34:55.263789+0000","last_became_peered":"2024-07-04T17:34:55.263789+0000","last_unstale":"2024-07-04T17:34:58.579473+0000","last_undegraded":"2024-07-04T17:34:58.579473+0000","last_fullsized":"2024-07-04T17:34:58.579473+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:04:32.541821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.235418+0000","last_change":"2024-07-04T17:34:35.525814+0000","last_active":"2024-07-04T17:34:56.235418+0000","last_peered":"2024-07-04T17:34:56.235418+0000","last_clean":"2024-07-04T17:34:56.235418+0000","last_became_active":"2024-07-04T17:34:35.525479+0000","last_became_peered":"2024-07-04T17:34:35.525479+0000","last_unstale":"2024-07-04T17:34:56.235418+0000","last_undegraded":"2024-07-04T17:34:56.235418+0000","last_fullsized":"2024-07-04T17:34:56.235418+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:44:57.595263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.245460+0000","last_change":"2024-07-04T17:34:55.259558+0000","last_active":"2024-07-04T17:34:56.245460+0000","last_peered":"2024-07-04T17:34:56.245460+0000","last_clean":"2024-07-04T17:34:56.245460+0000","last_became_active":"2024-07-04T17:34:55.258996+0000","last_became_peered":"2024-07-04T17:34:55.258996+0000","last_unstale":"2024-07-04T17:34:56.245460+0000","last_undegraded":"2024-07-04T17:34:56.245460+0000","last_fullsized":"2024-07-04T17:34:56.245460+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:40:04.267278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.240612+0000","last_change":"2024-07-04T17:34:35.554702+0000","last_active":"2024-07-04T17:34:56.240612+0000","last_peered":"2024-07-04T17:34:56.240612+0000","last_clean":"2024-07-04T17:34:56.240612+0000","last_became_active":"2024-07-04T17:34:35.554205+0000","last_became_peered":"2024-07-04T17:34:35.554205+0000","last_unstale":"2024-07-04T17:34:56.240612+0000","last_undegraded":"2024-07-04T17:34:56.240612+0000","last_fullsized":"2024-07-04T17:34:56.240612+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:18:48.550958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.244609+0000","last_change":"2024-07-04T17:34:55.252463+0000","last_active":"2024-07-04T17:34:56.244609+0000","last_peered":"2024-07-04T17:34:56.244609+0000","last_clean":"2024-07-04T17:34:56.244609+0000","last_became_active":"2024-07-04T17:34:55.251930+0000","last_became_peered":"2024-07-04T17:34:55.251930+0000","last_unstale":"2024-07-04T17:34:56.244609+0000","last_undegraded":"2024-07-04T17:34:56.244609+0000","last_fullsized":"2024-07-04T17:34:56.244609+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:52:31.382449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.241188+0000","last_change":"2024-07-04T17:34:35.536019+0000","last_active":"2024-07-04T17:34:56.241188+0000","last_peered":"2024-07-04T17:34:56.241188+0000","last_clean":"2024-07-04T17:34:56.241188+0000","last_became_active":"2024-07-04T17:34:35.535705+0000","last_became_peered":"2024-07-04T17:34:35.535705+0000","last_unstale":"2024-07-04T17:34:56.241188+0000","last_undegraded":"2024-07-04T17:34:56.241188+0000","last_fullsized":"2024-07-04T17:34:56.241188+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:06:39.223609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582360+0000","last_change":"2024-07-04T17:34:55.288462+0000","last_active":"2024-07-04T17:34:58.582360+0000","last_peered":"2024-07-04T17:34:58.582360+0000","last_clean":"2024-07-04T17:34:58.582360+0000","last_became_active":"2024-07-04T17:34:55.287324+0000","last_became_peered":"2024-07-04T17:34:55.287324+0000","last_unstale":"2024-07-04T17:34:58.582360+0000","last_undegraded":"2024-07-04T17:34:58.582360+0000","last_fullsized":"2024-07-04T17:34:58.582360+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:31:23.342645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582066+0000","last_change":"2024-07-04T17:34:35.562071+0000","last_active":"2024-07-04T17:34:58.582066+0000","last_peered":"2024-07-04T17:34:58.582066+0000","last_clean":"2024-07-04T17:34:58.582066+0000","last_became_active":"2024-07-04T17:34:35.560220+0000","last_became_peered":"2024-07-04T17:34:35.560220+0000","last_unstale":"2024-07-04T17:34:58.582066+0000","last_undegraded":"2024-07-04T17:34:58.582066+0000","last_fullsized":"2024-07-04T17:34:58.582066+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:27:08.229969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.240607+0000","last_change":"2024-07-04T17:34:55.283160+0000","last_active":"2024-07-04T17:34:56.240607+0000","last_peered":"2024-07-04T17:34:56.240607+0000","last_clean":"2024-07-04T17:34:56.240607+0000","last_became_active":"2024-07-04T17:34:55.282839+0000","last_became_peered":"2024-07-04T17:34:55.282839+0000","last_unstale":"2024-07-04T17:34:56.240607+0000","last_undegraded":"2024-07-04T17:34:56.240607+0000","last_fullsized":"2024-07-04T17:34:56.240607+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:38:09.271247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.238302+0000","last_change":"2024-07-04T17:34:35.553714+0000","last_active":"2024-07-04T17:34:56.238302+0000","last_peered":"2024-07-04T17:34:56.238302+0000","last_clean":"2024-07-04T17:34:56.238302+0000","last_became_active":"2024-07-04T17:34:35.553384+0000","last_became_peered":"2024-07-04T17:34:35.553384+0000","last_unstale":"2024-07-04T17:34:56.238302+0000","last_undegraded":"2024-07-04T17:34:56.238302+0000","last_fullsized":"2024-07-04T17:34:56.238302+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:47:20.883963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.241749+0000","last_change":"2024-07-04T17:34:55.239877+0000","last_active":"2024-07-04T17:34:56.241749+0000","last_peered":"2024-07-04T17:34:56.241749+0000","last_clean":"2024-07-04T17:34:56.241749+0000","last_became_active":"2024-07-04T17:34:55.239581+0000","last_became_peered":"2024-07-04T17:34:55.239581+0000","last_unstale":"2024-07-04T17:34:56.241749+0000","last_undegraded":"2024-07-04T17:34:56.241749+0000","last_fullsized":"2024-07-04T17:34:56.241749+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:20:07.104502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.237301+0000","last_change":"2024-07-04T17:34:35.554950+0000","last_active":"2024-07-04T17:34:56.237301+0000","last_peered":"2024-07-04T17:34:56.237301+0000","last_clean":"2024-07-04T17:34:56.237301+0000","last_became_active":"2024-07-04T17:34:35.554259+0000","last_became_peered":"2024-07-04T17:34:35.554259+0000","last_unstale":"2024-07-04T17:34:56.237301+0000","last_undegraded":"2024-07-04T17:34:56.237301+0000","last_fullsized":"2024-07-04T17:34:56.237301+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:59:46.128637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.233908+0000","last_change":"2024-07-04T17:34:55.272867+0000","last_active":"2024-07-04T17:34:56.233908+0000","last_peered":"2024-07-04T17:34:56.233908+0000","last_clean":"2024-07-04T17:34:56.233908+0000","last_became_active":"2024-07-04T17:34:55.270738+0000","last_became_peered":"2024-07-04T17:34:55.270738+0000","last_unstale":"2024-07-04T17:34:56.233908+0000","last_undegraded":"2024-07-04T17:34:56.233908+0000","last_fullsized":"2024-07-04T17:34:56.233908+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:16:21.117253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580453+0000","last_change":"2024-07-04T17:34:35.523276+0000","last_active":"2024-07-04T17:34:58.580453+0000","last_peered":"2024-07-04T17:34:58.580453+0000","last_clean":"2024-07-04T17:34:58.580453+0000","last_became_active":"2024-07-04T17:34:35.522611+0000","last_became_peered":"2024-07-04T17:34:35.522611+0000","last_unstale":"2024-07-04T17:34:58.580453+0000","last_undegraded":"2024-07-04T17:34:58.580453+0000","last_fullsized":"2024-07-04T17:34:58.580453+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:48:47.209945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580202+0000","last_change":"2024-07-04T17:34:55.281064+0000","last_active":"2024-07-04T17:34:58.580202+0000","last_peered":"2024-07-04T17:34:58.580202+0000","last_clean":"2024-07-04T17:34:58.580202+0000","last_became_active":"2024-07-04T17:34:55.280040+0000","last_became_peered":"2024-07-04T17:34:55.280040+0000","last_unstale":"2024-07-04T17:34:58.580202+0000","last_undegraded":"2024-07-04T17:34:58.580202+0000","last_fullsized":"2024-07-04T17:34:58.580202+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:45:05.455074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.244255+0000","last_change":"2024-07-04T17:34:35.546941+0000","last_active":"2024-07-04T17:34:56.244255+0000","last_peered":"2024-07-04T17:34:56.244255+0000","last_clean":"2024-07-04T17:34:56.244255+0000","last_became_active":"2024-07-04T17:34:35.544163+0000","last_became_peered":"2024-07-04T17:34:35.544163+0000","last_unstale":"2024-07-04T17:34:56.244255+0000","last_undegraded":"2024-07-04T17:34:56.244255+0000","last_fullsized":"2024-07-04T17:34:56.244255+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:15:08.178810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580021+0000","last_change":"2024-07-04T17:34:55.284190+0000","last_active":"2024-07-04T17:34:58.580021+0000","last_peered":"2024-07-04T17:34:58.580021+0000","last_clean":"2024-07-04T17:34:58.580021+0000","last_became_active":"2024-07-04T17:34:55.283868+0000","last_became_peered":"2024-07-04T17:34:55.283868+0000","last_unstale":"2024-07-04T17:34:58.580021+0000","last_undegraded":"2024-07-04T17:34:58.580021+0000","last_fullsized":"2024-07-04T17:34:58.580021+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:53:36.933359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.578815+0000","last_change":"2024-07-04T17:34:35.545420+0000","last_active":"2024-07-04T17:34:58.578815+0000","last_peered":"2024-07-04T17:34:58.578815+0000","last_clean":"2024-07-04T17:34:58.578815+0000","last_became_active":"2024-07-04T17:34:35.543935+0000","last_became_peered":"2024-07-04T17:34:35.543935+0000","last_unstale":"2024-07-04T17:34:58.578815+0000","last_undegraded":"2024-07-04T17:34:58.578815+0000","last_fullsized":"2024-07-04T17:34:58.578815+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:16:01.387094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582403+0000","last_change":"2024-07-04T17:34:55.285445+0000","last_active":"2024-07-04T17:34:58.582403+0000","last_peered":"2024-07-04T17:34:58.582403+0000","last_clean":"2024-07-04T17:34:58.582403+0000","last_became_active":"2024-07-04T17:34:55.285049+0000","last_became_peered":"2024-07-04T17:34:55.285049+0000","last_unstale":"2024-07-04T17:34:58.582403+0000","last_undegraded":"2024-07-04T17:34:58.582403+0000","last_fullsized":"2024-07-04T17:34:58.582403+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:20:51.546736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.248985+0000","last_change":"2024-07-04T17:34:35.536388+0000","last_active":"2024-07-04T17:34:56.248985+0000","last_peered":"2024-07-04T17:34:56.248985+0000","last_clean":"2024-07-04T17:34:56.248985+0000","last_became_active":"2024-07-04T17:34:35.536052+0000","last_became_peered":"2024-07-04T17:34:35.536052+0000","last_unstale":"2024-07-04T17:34:56.248985+0000","last_undegraded":"2024-07-04T17:34:56.248985+0000","last_fullsized":"2024-07-04T17:34:56.248985+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:08:18.792180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.238582+0000","last_change":"2024-07-04T17:34:55.274895+0000","last_active":"2024-07-04T17:34:56.238582+0000","last_peered":"2024-07-04T17:34:56.238582+0000","last_clean":"2024-07-04T17:34:56.238582+0000","last_became_active":"2024-07-04T17:34:55.274568+0000","last_became_peered":"2024-07-04T17:34:55.274568+0000","last_unstale":"2024-07-04T17:34:56.238582+0000","last_undegraded":"2024-07-04T17:34:56.238582+0000","last_fullsized":"2024-07-04T17:34:56.238582+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:16:51.380937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582501+0000","last_change":"2024-07-04T17:34:35.533907+0000","last_active":"2024-07-04T17:34:58.582501+0000","last_peered":"2024-07-04T17:34:58.582501+0000","last_clean":"2024-07-04T17:34:58.582501+0000","last_became_active":"2024-07-04T17:34:35.533596+0000","last_became_peered":"2024-07-04T17:34:35.533596+0000","last_unstale":"2024-07-04T17:34:58.582501+0000","last_undegraded":"2024-07-04T17:34:58.582501+0000","last_fullsized":"2024-07-04T17:34:58.582501+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:41:30.147907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.248085+0000","last_change":"2024-07-04T17:34:55.239623+0000","last_active":"2024-07-04T17:34:56.248085+0000","last_peered":"2024-07-04T17:34:56.248085+0000","last_clean":"2024-07-04T17:34:56.248085+0000","last_became_active":"2024-07-04T17:34:55.239022+0000","last_became_peered":"2024-07-04T17:34:55.239022+0000","last_unstale":"2024-07-04T17:34:56.248085+0000","last_undegraded":"2024-07-04T17:34:56.248085+0000","last_fullsized":"2024-07-04T17:34:56.248085+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:15:15.683297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.246912+0000","last_change":"2024-07-04T17:34:35.537742+0000","last_active":"2024-07-04T17:34:56.246912+0000","last_peered":"2024-07-04T17:34:56.246912+0000","last_clean":"2024-07-04T17:34:56.246912+0000","last_became_active":"2024-07-04T17:34:35.533422+0000","last_became_peered":"2024-07-04T17:34:35.533422+0000","last_unstale":"2024-07-04T17:34:56.246912+0000","last_undegraded":"2024-07-04T17:34:56.246912+0000","last_fullsized":"2024-07-04T17:34:56.246912+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:02:05.629708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580418+0000","last_change":"2024-07-04T17:34:55.277168+0000","last_active":"2024-07-04T17:34:58.580418+0000","last_peered":"2024-07-04T17:34:58.580418+0000","last_clean":"2024-07-04T17:34:58.580418+0000","last_became_active":"2024-07-04T17:34:55.276394+0000","last_became_peered":"2024-07-04T17:34:55.276394+0000","last_unstale":"2024-07-04T17:34:58.580418+0000","last_undegraded":"2024-07-04T17:34:58.580418+0000","last_fullsized":"2024-07-04T17:34:58.580418+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:41:59.736262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.238463+0000","last_change":"2024-07-04T17:34:35.553505+0000","last_active":"2024-07-04T17:34:56.238463+0000","last_peered":"2024-07-04T17:34:56.238463+0000","last_clean":"2024-07-04T17:34:56.238463+0000","last_became_active":"2024-07-04T17:34:35.553230+0000","last_became_peered":"2024-07-04T17:34:35.553230+0000","last_unstale":"2024-07-04T17:34:56.238463+0000","last_undegraded":"2024-07-04T17:34:56.238463+0000","last_fullsized":"2024-07-04T17:34:56.238463+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:49:38.920935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580315+0000","last_change":"2024-07-04T17:34:55.265408+0000","last_active":"2024-07-04T17:34:58.580315+0000","last_peered":"2024-07-04T17:34:58.580315+0000","last_clean":"2024-07-04T17:34:58.580315+0000","last_became_active":"2024-07-04T17:34:55.264653+0000","last_became_peered":"2024-07-04T17:34:55.264653+0000","last_unstale":"2024-07-04T17:34:58.580315+0000","last_undegraded":"2024-07-04T17:34:58.580315+0000","last_fullsized":"2024-07-04T17:34:58.580315+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:22:51.459466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580656+0000","last_change":"2024-07-04T17:34:35.524780+0000","last_active":"2024-07-04T17:34:58.580656+0000","last_peered":"2024-07-04T17:34:58.580656+0000","last_clean":"2024-07-04T17:34:58.580656+0000","last_became_active":"2024-07-04T17:34:35.524418+0000","last_became_peered":"2024-07-04T17:34:35.524418+0000","last_unstale":"2024-07-04T17:34:58.580656+0000","last_undegraded":"2024-07-04T17:34:58.580656+0000","last_fullsized":"2024-07-04T17:34:58.580656+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:16:31.650779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.579194+0000","last_change":"2024-07-04T17:34:55.286892+0000","last_active":"2024-07-04T17:34:58.579194+0000","last_peered":"2024-07-04T17:34:58.579194+0000","last_clean":"2024-07-04T17:34:58.579194+0000","last_became_active":"2024-07-04T17:34:55.285216+0000","last_became_peered":"2024-07-04T17:34:55.285216+0000","last_unstale":"2024-07-04T17:34:58.579194+0000","last_undegraded":"2024-07-04T17:34:58.579194+0000","last_fullsized":"2024-07-04T17:34:58.579194+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:51:03.831100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580803+0000","last_change":"2024-07-04T17:34:35.522942+0000","last_active":"2024-07-04T17:34:58.580803+0000","last_peered":"2024-07-04T17:34:58.580803+0000","last_clean":"2024-07-04T17:34:58.580803+0000","last_became_active":"2024-07-04T17:34:35.522210+0000","last_became_peered":"2024-07-04T17:34:35.522210+0000","last_unstale":"2024-07-04T17:34:58.580803+0000","last_undegraded":"2024-07-04T17:34:58.580803+0000","last_fullsized":"2024-07-04T17:34:58.580803+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:03:46.125136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.235849+0000","last_change":"2024-07-04T17:34:55.272255+0000","last_active":"2024-07-04T17:34:56.235849+0000","last_peered":"2024-07-04T17:34:56.235849+0000","last_clean":"2024-07-04T17:34:56.235849+0000","last_became_active":"2024-07-04T17:34:55.270558+0000","last_became_peered":"2024-07-04T17:34:55.270558+0000","last_unstale":"2024-07-04T17:34:56.235849+0000","last_undegraded":"2024-07-04T17:34:56.235849+0000","last_fullsized":"2024-07-04T17:34:56.235849+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:04:06.751483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.244843+0000","last_change":"2024-07-04T17:34:35.544485+0000","last_active":"2024-07-04T17:34:56.244843+0000","last_peered":"2024-07-04T17:34:56.244843+0000","last_clean":"2024-07-04T17:34:56.244843+0000","last_became_active":"2024-07-04T17:34:35.539252+0000","last_became_peered":"2024-07-04T17:34:35.539252+0000","last_unstale":"2024-07-04T17:34:56.244843+0000","last_undegraded":"2024-07-04T17:34:56.244843+0000","last_fullsized":"2024-07-04T17:34:56.244843+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:58:20.122138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.243351+0000","last_change":"2024-07-04T17:34:55.254639+0000","last_active":"2024-07-04T17:34:56.243351+0000","last_peered":"2024-07-04T17:34:56.243351+0000","last_clean":"2024-07-04T17:34:56.243351+0000","last_became_active":"2024-07-04T17:34:55.252455+0000","last_became_peered":"2024-07-04T17:34:55.252455+0000","last_unstale":"2024-07-04T17:34:56.243351+0000","last_undegraded":"2024-07-04T17:34:56.243351+0000","last_fullsized":"2024-07-04T17:34:56.243351+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:36:12.409147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.235635+0000","last_change":"2024-07-04T17:34:35.532311+0000","last_active":"2024-07-04T17:34:56.235635+0000","last_peered":"2024-07-04T17:34:56.235635+0000","last_clean":"2024-07-04T17:34:56.235635+0000","last_became_active":"2024-07-04T17:34:35.531704+0000","last_became_peered":"2024-07-04T17:34:35.531704+0000","last_unstale":"2024-07-04T17:34:56.235635+0000","last_undegraded":"2024-07-04T17:34:56.235635+0000","last_fullsized":"2024-07-04T17:34:56.235635+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:38:06.359176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580208+0000","last_change":"2024-07-04T17:34:55.273539+0000","last_active":"2024-07-04T17:34:58.580208+0000","last_peered":"2024-07-04T17:34:58.580208+0000","last_clean":"2024-07-04T17:34:58.580208+0000","last_became_active":"2024-07-04T17:34:55.272611+0000","last_became_peered":"2024-07-04T17:34:55.272611+0000","last_unstale":"2024-07-04T17:34:58.580208+0000","last_undegraded":"2024-07-04T17:34:58.580208+0000","last_fullsized":"2024-07-04T17:34:58.580208+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:49:52.552101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.238206+0000","last_change":"2024-07-04T17:34:54.267723+0000","last_active":"2024-07-04T17:34:56.238206+0000","last_peered":"2024-07-04T17:34:56.238206+0000","last_clean":"2024-07-04T17:34:56.238206+0000","last_became_active":"2024-07-04T17:34:54.267352+0000","last_became_peered":"2024-07-04T17:34:54.267352+0000","last_unstale":"2024-07-04T17:34:56.238206+0000","last_undegraded":"2024-07-04T17:34:56.238206+0000","last_fullsized":"2024-07-04T17:34:56.238206+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:12:50.370995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.247361+0000","last_change":"2024-07-04T17:34:35.536755+0000","last_active":"2024-07-04T17:34:56.247361+0000","last_peered":"2024-07-04T17:34:56.247361+0000","last_clean":"2024-07-04T17:34:56.247361+0000","last_became_active":"2024-07-04T17:34:35.535641+0000","last_became_peered":"2024-07-04T17:34:35.535641+0000","last_unstale":"2024-07-04T17:34:56.247361+0000","last_undegraded":"2024-07-04T17:34:56.247361+0000","last_fullsized":"2024-07-04T17:34:56.247361+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:00:03.434775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1f","version":"135'4","reported_seq":26,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.241514+0000","last_change":"2024-07-04T17:34:34.526194+0000","last_active":"2024-07-04T17:34:56.241514+0000","last_peered":"2024-07-04T17:34:56.241514+0000","last_clean":"2024-07-04T17:34:56.241514+0000","last_became_active":"2024-07-04T17:34:34.525749+0000","last_became_peered":"2024-07-04T17:34:34.525749+0000","last_unstale":"2024-07-04T17:34:56.241514+0000","last_undegraded":"2024-07-04T17:34:56.241514+0000","last_fullsized":"2024-07-04T17:34:56.241514+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:15:41.361322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.245620+0000","last_change":"2024-07-04T17:34:55.263366+0000","last_active":"2024-07-04T17:34:56.245620+0000","last_peered":"2024-07-04T17:34:56.245620+0000","last_clean":"2024-07-04T17:34:56.245620+0000","last_became_active":"2024-07-04T17:34:55.263036+0000","last_became_peered":"2024-07-04T17:34:55.263036+0000","last_unstale":"2024-07-04T17:34:56.245620+0000","last_undegraded":"2024-07-04T17:34:56.245620+0000","last_fullsized":"2024-07-04T17:34:56.245620+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:16:51.380937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1c","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.244471+0000","last_change":"2024-07-04T17:34:54.257931+0000","last_active":"2024-07-04T17:34:56.244471+0000","last_peered":"2024-07-04T17:34:56.244471+0000","last_clean":"2024-07-04T17:34:56.244471+0000","last_became_active":"2024-07-04T17:34:54.255905+0000","last_became_peered":"2024-07-04T17:34:54.255905+0000","last_unstale":"2024-07-04T17:34:56.244471+0000","last_undegraded":"2024-07-04T17:34:56.244471+0000","last_fullsized":"2024-07-04T17:34:56.244471+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:06:21.435358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582588+0000","last_change":"2024-07-04T17:34:35.548372+0000","last_active":"2024-07-04T17:34:58.582588+0000","last_peered":"2024-07-04T17:34:58.582588+0000","last_clean":"2024-07-04T17:34:58.582588+0000","last_became_active":"2024-07-04T17:34:35.547829+0000","last_became_peered":"2024-07-04T17:34:35.547829+0000","last_unstale":"2024-07-04T17:34:58.582588+0000","last_undegraded":"2024-07-04T17:34:58.582588+0000","last_fullsized":"2024-07-04T17:34:58.582588+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:37:49.541089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"135'1","reported_seq":23,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.577793+0000","last_change":"2024-07-04T17:34:34.505687+0000","last_active":"2024-07-04T17:34:58.577793+0000","last_peered":"2024-07-04T17:34:58.577793+0000","last_clean":"2024-07-04T17:34:58.577793+0000","last_became_active":"2024-07-04T17:34:34.505337+0000","last_became_peered":"2024-07-04T17:34:34.505337+0000","last_unstale":"2024-07-04T17:34:58.577793+0000","last_undegraded":"2024-07-04T17:34:58.577793+0000","last_fullsized":"2024-07-04T17:34:58.577793+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:57:27.037624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582563+0000","last_change":"2024-07-04T17:34:55.276891+0000","last_active":"2024-07-04T17:34:58.582563+0000","last_peered":"2024-07-04T17:34:58.582563+0000","last_clean":"2024-07-04T17:34:58.582563+0000","last_became_active":"2024-07-04T17:34:55.276381+0000","last_became_peered":"2024-07-04T17:34:55.276381+0000","last_unstale":"2024-07-04T17:34:58.582563+0000","last_undegraded":"2024-07-04T17:34:58.582563+0000","last_fullsized":"2024-07-04T17:34:58.582563+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:33:25.267671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1f","version":"139'4","reported_seq":18,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580901+0000","last_change":"2024-07-04T17:34:54.261384+0000","last_active":"2024-07-04T17:34:58.580901+0000","last_peered":"2024-07-04T17:34:58.580901+0000","last_clean":"2024-07-04T17:34:58.580901+0000","last_became_active":"2024-07-04T17:34:54.260019+0000","last_became_peered":"2024-07-04T17:34:54.260019+0000","last_unstale":"2024-07-04T17:34:58.580901+0000","last_undegraded":"2024-07-04T17:34:58.580901+0000","last_fullsized":"2024-07-04T17:34:58.580901+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:30:17.511938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.244945+0000","last_change":"2024-07-04T17:34:35.531247+0000","last_active":"2024-07-04T17:34:56.244945+0000","last_peered":"2024-07-04T17:34:56.244945+0000","last_clean":"2024-07-04T17:34:56.244945+0000","last_became_active":"2024-07-04T17:34:35.530846+0000","last_became_peered":"2024-07-04T17:34:35.530846+0000","last_unstale":"2024-07-04T17:34:56.244945+0000","last_undegraded":"2024-07-04T17:34:56.244945+0000","last_fullsized":"2024-07-04T17:34:56.244945+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:30:51.180015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1d","version":"135'2","reported_seq":24,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.247392+0000","last_change":"2024-07-04T17:34:34.512001+0000","last_active":"2024-07-04T17:34:56.247392+0000","last_peered":"2024-07-04T17:34:56.247392+0000","last_clean":"2024-07-04T17:34:56.247392+0000","last_became_active":"2024-07-04T17:34:34.511480+0000","last_became_peered":"2024-07-04T17:34:34.511480+0000","last_unstale":"2024-07-04T17:34:56.247392+0000","last_undegraded":"2024-07-04T17:34:56.247392+0000","last_fullsized":"2024-07-04T17:34:56.247392+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:46:03.790681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.249372+0000","last_change":"2024-07-04T17:34:55.239365+0000","last_active":"2024-07-04T17:34:56.249372+0000","last_peered":"2024-07-04T17:34:56.249372+0000","last_clean":"2024-07-04T17:34:56.249372+0000","last_became_active":"2024-07-04T17:34:55.237209+0000","last_became_peered":"2024-07-04T17:34:55.237209+0000","last_unstale":"2024-07-04T17:34:56.249372+0000","last_undegraded":"2024-07-04T17:34:56.249372+0000","last_fullsized":"2024-07-04T17:34:56.249372+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:41:49.878066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1e","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582123+0000","last_change":"2024-07-04T17:34:54.261119+0000","last_active":"2024-07-04T17:34:58.582123+0000","last_peered":"2024-07-04T17:34:58.582123+0000","last_clean":"2024-07-04T17:34:58.582123+0000","last_became_active":"2024-07-04T17:34:54.259804+0000","last_became_peered":"2024-07-04T17:34:54.259804+0000","last_unstale":"2024-07-04T17:34:58.582123+0000","last_undegraded":"2024-07-04T17:34:58.582123+0000","last_fullsized":"2024-07-04T17:34:58.582123+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:01:32.268460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.247098+0000","last_change":"2024-07-04T17:34:35.546422+0000","last_active":"2024-07-04T17:34:56.247098+0000","last_peered":"2024-07-04T17:34:56.247098+0000","last_clean":"2024-07-04T17:34:56.247098+0000","last_became_active":"2024-07-04T17:34:35.546061+0000","last_became_peered":"2024-07-04T17:34:35.546061+0000","last_unstale":"2024-07-04T17:34:56.247098+0000","last_undegraded":"2024-07-04T17:34:56.247098+0000","last_fullsized":"2024-07-04T17:34:56.247098+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:50:00.540847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1c","version":"135'2","reported_seq":24,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.577632+0000","last_change":"2024-07-04T17:34:34.506897+0000","last_active":"2024-07-04T17:34:58.577632+0000","last_peered":"2024-07-04T17:34:58.577632+0000","last_clean":"2024-07-04T17:34:58.577632+0000","last_became_active":"2024-07-04T17:34:34.506023+0000","last_became_peered":"2024-07-04T17:34:34.506023+0000","last_unstale":"2024-07-04T17:34:58.577632+0000","last_undegraded":"2024-07-04T17:34:58.577632+0000","last_fullsized":"2024-07-04T17:34:58.577632+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:36:57.170806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.242029+0000","last_change":"2024-07-04T17:34:55.255731+0000","last_active":"2024-07-04T17:34:56.242029+0000","last_peered":"2024-07-04T17:34:56.242029+0000","last_clean":"2024-07-04T17:34:56.242029+0000","last_became_active":"2024-07-04T17:34:55.255430+0000","last_became_peered":"2024-07-04T17:34:55.255430+0000","last_unstale":"2024-07-04T17:34:56.242029+0000","last_undegraded":"2024-07-04T17:34:56.242029+0000","last_fullsized":"2024-07-04T17:34:56.242029+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:55:44.449626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.242983+0000","last_change":"2024-07-04T17:34:54.264471+0000","last_active":"2024-07-04T17:34:56.242983+0000","last_peered":"2024-07-04T17:34:56.242983+0000","last_clean":"2024-07-04T17:34:56.242983+0000","last_became_active":"2024-07-04T17:34:54.263599+0000","last_became_peered":"2024-07-04T17:34:54.263599+0000","last_unstale":"2024-07-04T17:34:56.242983+0000","last_undegraded":"2024-07-04T17:34:56.242983+0000","last_fullsized":"2024-07-04T17:34:56.242983+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:08:30.179545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580809+0000","last_change":"2024-07-04T17:34:35.515987+0000","last_active":"2024-07-04T17:34:58.580809+0000","last_peered":"2024-07-04T17:34:58.580809+0000","last_clean":"2024-07-04T17:34:58.580809+0000","last_became_active":"2024-07-04T17:34:35.515158+0000","last_became_peered":"2024-07-04T17:34:35.515158+0000","last_unstale":"2024-07-04T17:34:58.580809+0000","last_undegraded":"2024-07-04T17:34:58.580809+0000","last_fullsized":"2024-07-04T17:34:58.580809+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:39:44.537119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"135'1","reported_seq":23,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.238878+0000","last_change":"2024-07-04T17:34:34.526306+0000","last_active":"2024-07-04T17:34:56.238878+0000","last_peered":"2024-07-04T17:34:56.238878+0000","last_clean":"2024-07-04T17:34:56.238878+0000","last_became_active":"2024-07-04T17:34:34.525936+0000","last_became_peered":"2024-07-04T17:34:34.525936+0000","last_unstale":"2024-07-04T17:34:56.238878+0000","last_undegraded":"2024-07-04T17:34:56.238878+0000","last_fullsized":"2024-07-04T17:34:56.238878+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:58:19.109679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.234302+0000","last_change":"2024-07-04T17:34:55.280775+0000","last_active":"2024-07-04T17:34:56.234302+0000","last_peered":"2024-07-04T17:34:56.234302+0000","last_clean":"2024-07-04T17:34:56.234302+0000","last_became_active":"2024-07-04T17:34:55.280449+0000","last_became_peered":"2024-07-04T17:34:55.280449+0000","last_unstale":"2024-07-04T17:34:56.234302+0000","last_undegraded":"2024-07-04T17:34:56.234302+0000","last_fullsized":"2024-07-04T17:34:56.234302+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:36:12.409147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.242124+0000","last_change":"2024-07-04T17:34:54.264992+0000","last_active":"2024-07-04T17:34:56.242124+0000","last_peered":"2024-07-04T17:34:56.242124+0000","last_clean":"2024-07-04T17:34:56.242124+0000","last_became_active":"2024-07-04T17:34:54.264254+0000","last_became_peered":"2024-07-04T17:34:54.264254+0000","last_unstale":"2024-07-04T17:34:56.242124+0000","last_undegraded":"2024-07-04T17:34:56.242124+0000","last_fullsized":"2024-07-04T17:34:56.242124+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:21:20.115129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.237494+0000","last_change":"2024-07-04T17:34:35.529792+0000","last_active":"2024-07-04T17:34:56.237494+0000","last_peered":"2024-07-04T17:34:56.237494+0000","last_clean":"2024-07-04T17:34:56.237494+0000","last_became_active":"2024-07-04T17:34:35.529159+0000","last_became_peered":"2024-07-04T17:34:35.529159+0000","last_unstale":"2024-07-04T17:34:56.237494+0000","last_undegraded":"2024-07-04T17:34:56.237494+0000","last_fullsized":"2024-07-04T17:34:56.237494+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:19:05.515444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.1a","version":"135'1","reported_seq":23,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.236666+0000","last_change":"2024-07-04T17:34:34.520915+0000","last_active":"2024-07-04T17:34:56.236666+0000","last_peered":"2024-07-04T17:34:56.236666+0000","last_clean":"2024-07-04T17:34:56.236666+0000","last_became_active":"2024-07-04T17:34:34.520164+0000","last_became_peered":"2024-07-04T17:34:34.520164+0000","last_unstale":"2024-07-04T17:34:56.236666+0000","last_undegraded":"2024-07-04T17:34:56.236666+0000","last_fullsized":"2024-07-04T17:34:56.236666+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:15:07.166351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.243630+0000","last_change":"2024-07-04T17:34:55.262931+0000","last_active":"2024-07-04T17:34:56.243630+0000","last_peered":"2024-07-04T17:34:56.243630+0000","last_clean":"2024-07-04T17:34:56.243630+0000","last_became_active":"2024-07-04T17:34:55.262623+0000","last_became_peered":"2024-07-04T17:34:55.262623+0000","last_unstale":"2024-07-04T17:34:56.243630+0000","last_undegraded":"2024-07-04T17:34:56.243630+0000","last_fullsized":"2024-07-04T17:34:56.243630+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:05:12.120118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1a","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.243597+0000","last_change":"2024-07-04T17:34:54.254747+0000","last_active":"2024-07-04T17:34:56.243597+0000","last_peered":"2024-07-04T17:34:56.243597+0000","last_clean":"2024-07-04T17:34:56.243597+0000","last_became_active":"2024-07-04T17:34:54.254412+0000","last_became_peered":"2024-07-04T17:34:54.254412+0000","last_unstale":"2024-07-04T17:34:56.243597+0000","last_undegraded":"2024-07-04T17:34:56.243597+0000","last_fullsized":"2024-07-04T17:34:56.243597+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:22:35.350759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.235418+0000","last_change":"2024-07-04T17:34:35.528571+0000","last_active":"2024-07-04T17:34:56.235418+0000","last_peered":"2024-07-04T17:34:56.235418+0000","last_clean":"2024-07-04T17:34:56.235418+0000","last_became_active":"2024-07-04T17:34:35.528217+0000","last_became_peered":"2024-07-04T17:34:35.528217+0000","last_unstale":"2024-07-04T17:34:56.235418+0000","last_undegraded":"2024-07-04T17:34:56.235418+0000","last_fullsized":"2024-07-04T17:34:56.235418+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:25:31.311940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.235826+0000","last_change":"2024-07-04T17:34:34.520226+0000","last_active":"2024-07-04T17:34:56.235826+0000","last_peered":"2024-07-04T17:34:56.235826+0000","last_clean":"2024-07-04T17:34:56.235826+0000","last_became_active":"2024-07-04T17:34:34.519880+0000","last_became_peered":"2024-07-04T17:34:34.519880+0000","last_unstale":"2024-07-04T17:34:56.235826+0000","last_undegraded":"2024-07-04T17:34:56.235826+0000","last_fullsized":"2024-07-04T17:34:56.235826+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:30:01.808525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.237006+0000","last_change":"2024-07-04T17:34:55.284186+0000","last_active":"2024-07-04T17:34:56.237006+0000","last_peered":"2024-07-04T17:34:56.237006+0000","last_clean":"2024-07-04T17:34:56.237006+0000","last_became_active":"2024-07-04T17:34:55.283840+0000","last_became_peered":"2024-07-04T17:34:55.283840+0000","last_unstale":"2024-07-04T17:34:56.237006+0000","last_undegraded":"2024-07-04T17:34:56.237006+0000","last_fullsized":"2024-07-04T17:34:56.237006+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:04:05.855295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.15","version":"139'3","reported_seq":17,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580863+0000","last_change":"2024-07-04T17:34:54.251183+0000","last_active":"2024-07-04T17:34:58.580863+0000","last_peered":"2024-07-04T17:34:58.580863+0000","last_clean":"2024-07-04T17:34:58.580863+0000","last_became_active":"2024-07-04T17:34:54.250831+0000","last_became_peered":"2024-07-04T17:34:54.250831+0000","last_unstale":"2024-07-04T17:34:58.580863+0000","last_undegraded":"2024-07-04T17:34:58.580863+0000","last_fullsized":"2024-07-04T17:34:58.580863+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:31:23.802489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.247488+0000","last_change":"2024-07-04T17:34:35.541668+0000","last_active":"2024-07-04T17:34:56.247488+0000","last_peered":"2024-07-04T17:34:56.247488+0000","last_clean":"2024-07-04T17:34:56.247488+0000","last_became_active":"2024-07-04T17:34:35.537142+0000","last_became_peered":"2024-07-04T17:34:35.537142+0000","last_unstale":"2024-07-04T17:34:56.247488+0000","last_undegraded":"2024-07-04T17:34:56.247488+0000","last_fullsized":"2024-07-04T17:34:56.247488+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:56:41.649364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.249541+0000","last_change":"2024-07-04T17:34:34.516054+0000","last_active":"2024-07-04T17:34:56.249541+0000","last_peered":"2024-07-04T17:34:56.249541+0000","last_clean":"2024-07-04T17:34:56.249541+0000","last_became_active":"2024-07-04T17:34:34.514177+0000","last_became_peered":"2024-07-04T17:34:34.514177+0000","last_unstale":"2024-07-04T17:34:56.249541+0000","last_undegraded":"2024-07-04T17:34:56.249541+0000","last_fullsized":"2024-07-04T17:34:56.249541+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:18:46.789057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580742+0000","last_change":"2024-07-04T17:34:55.277295+0000","last_active":"2024-07-04T17:34:58.580742+0000","last_peered":"2024-07-04T17:34:58.580742+0000","last_clean":"2024-07-04T17:34:58.580742+0000","last_became_active":"2024-07-04T17:34:55.275102+0000","last_became_peered":"2024-07-04T17:34:55.275102+0000","last_unstale":"2024-07-04T17:34:58.580742+0000","last_undegraded":"2024-07-04T17:34:58.580742+0000","last_fullsized":"2024-07-04T17:34:58.580742+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:09:53.591198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"139'6","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.237116+0000","last_change":"2024-07-04T17:34:54.270201+0000","last_active":"2024-07-04T17:34:56.237116+0000","last_peered":"2024-07-04T17:34:56.237116+0000","last_clean":"2024-07-04T17:34:56.237116+0000","last_became_active":"2024-07-04T17:34:54.269515+0000","last_became_peered":"2024-07-04T17:34:54.269515+0000","last_unstale":"2024-07-04T17:34:56.237116+0000","last_undegraded":"2024-07-04T17:34:56.237116+0000","last_fullsized":"2024-07-04T17:34:56.237116+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:22:12.441138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.249356+0000","last_change":"2024-07-04T17:34:35.546010+0000","last_active":"2024-07-04T17:34:56.249356+0000","last_peered":"2024-07-04T17:34:56.249356+0000","last_clean":"2024-07-04T17:34:56.249356+0000","last_became_active":"2024-07-04T17:34:35.536818+0000","last_became_peered":"2024-07-04T17:34:35.536818+0000","last_unstale":"2024-07-04T17:34:56.249356+0000","last_undegraded":"2024-07-04T17:34:56.249356+0000","last_fullsized":"2024-07-04T17:34:56.249356+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:24:20.145687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.241512+0000","last_change":"2024-07-04T17:34:34.526851+0000","last_active":"2024-07-04T17:34:56.241512+0000","last_peered":"2024-07-04T17:34:56.241512+0000","last_clean":"2024-07-04T17:34:56.241512+0000","last_became_active":"2024-07-04T17:34:34.526514+0000","last_became_peered":"2024-07-04T17:34:34.526514+0000","last_unstale":"2024-07-04T17:34:56.241512+0000","last_undegraded":"2024-07-04T17:34:56.241512+0000","last_fullsized":"2024-07-04T17:34:56.241512+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:37:45.915030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.247127+0000","last_change":"2024-07-04T17:34:55.238745+0000","last_active":"2024-07-04T17:34:56.247127+0000","last_peered":"2024-07-04T17:34:56.247127+0000","last_clean":"2024-07-04T17:34:56.247127+0000","last_became_active":"2024-07-04T17:34:55.238434+0000","last_became_peered":"2024-07-04T17:34:55.238434+0000","last_unstale":"2024-07-04T17:34:56.247127+0000","last_undegraded":"2024-07-04T17:34:56.247127+0000","last_fullsized":"2024-07-04T17:34:56.247127+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:54:55.103581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.248259+0000","last_change":"2024-07-04T17:34:54.257535+0000","last_active":"2024-07-04T17:34:56.248259+0000","last_peered":"2024-07-04T17:34:56.248259+0000","last_clean":"2024-07-04T17:34:56.248259+0000","last_became_active":"2024-07-04T17:34:54.256273+0000","last_became_peered":"2024-07-04T17:34:54.256273+0000","last_unstale":"2024-07-04T17:34:56.248259+0000","last_undegraded":"2024-07-04T17:34:56.248259+0000","last_fullsized":"2024-07-04T17:34:56.248259+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:36:46.193844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582076+0000","last_change":"2024-07-04T17:34:35.524150+0000","last_active":"2024-07-04T17:34:58.582076+0000","last_peered":"2024-07-04T17:34:58.582076+0000","last_clean":"2024-07-04T17:34:58.582076+0000","last_became_active":"2024-07-04T17:34:35.523835+0000","last_became_peered":"2024-07-04T17:34:35.523835+0000","last_unstale":"2024-07-04T17:34:58.582076+0000","last_undegraded":"2024-07-04T17:34:58.582076+0000","last_fullsized":"2024-07-04T17:34:58.582076+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:14:55.953139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.15","version":"135'3","reported_seq":25,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.239494+0000","last_change":"2024-07-04T17:34:34.525906+0000","last_active":"2024-07-04T17:34:56.239494+0000","last_peered":"2024-07-04T17:34:56.239494+0000","last_clean":"2024-07-04T17:34:56.239494+0000","last_became_active":"2024-07-04T17:34:34.525545+0000","last_became_peered":"2024-07-04T17:34:34.525545+0000","last_unstale":"2024-07-04T17:34:56.239494+0000","last_undegraded":"2024-07-04T17:34:56.239494+0000","last_fullsized":"2024-07-04T17:34:56.239494+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:15:15.110008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.239796+0000","last_change":"2024-07-04T17:34:55.274885+0000","last_active":"2024-07-04T17:34:56.239796+0000","last_peered":"2024-07-04T17:34:56.239796+0000","last_clean":"2024-07-04T17:34:56.239796+0000","last_became_active":"2024-07-04T17:34:55.273954+0000","last_became_peered":"2024-07-04T17:34:55.273954+0000","last_unstale":"2024-07-04T17:34:56.239796+0000","last_undegraded":"2024-07-04T17:34:56.239796+0000","last_fullsized":"2024-07-04T17:34:56.239796+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:15:15.683297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.245888+0000","last_change":"2024-07-04T17:34:54.257704+0000","last_active":"2024-07-04T17:34:56.245888+0000","last_peered":"2024-07-04T17:34:56.245888+0000","last_clean":"2024-07-04T17:34:56.245888+0000","last_became_active":"2024-07-04T17:34:54.257376+0000","last_became_peered":"2024-07-04T17:34:54.257376+0000","last_unstale":"2024-07-04T17:34:56.245888+0000","last_undegraded":"2024-07-04T17:34:56.245888+0000","last_fullsized":"2024-07-04T17:34:56.245888+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:59:05.459068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582464+0000","last_change":"2024-07-04T17:34:35.523356+0000","last_active":"2024-07-04T17:34:58.582464+0000","last_peered":"2024-07-04T17:34:58.582464+0000","last_clean":"2024-07-04T17:34:58.582464+0000","last_became_active":"2024-07-04T17:34:35.522584+0000","last_became_peered":"2024-07-04T17:34:35.522584+0000","last_unstale":"2024-07-04T17:34:58.582464+0000","last_undegraded":"2024-07-04T17:34:58.582464+0000","last_fullsized":"2024-07-04T17:34:58.582464+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:34:41.701054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"136'8","reported_seq":30,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.245341+0000","last_change":"2024-07-04T17:34:34.513760+0000","last_active":"2024-07-04T17:34:56.245341+0000","last_peered":"2024-07-04T17:34:56.245341+0000","last_clean":"2024-07-04T17:34:56.245341+0000","last_became_active":"2024-07-04T17:34:34.512609+0000","last_became_peered":"2024-07-04T17:34:34.512609+0000","last_unstale":"2024-07-04T17:34:56.245341+0000","last_undegraded":"2024-07-04T17:34:56.245341+0000","last_fullsized":"2024-07-04T17:34:56.245341+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:08:12.565423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3309,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.234441+0000","last_change":"2024-07-04T17:34:55.280804+0000","last_active":"2024-07-04T17:34:56.234441+0000","last_peered":"2024-07-04T17:34:56.234441+0000","last_clean":"2024-07-04T17:34:56.234441+0000","last_became_active":"2024-07-04T17:34:55.280549+0000","last_became_peered":"2024-07-04T17:34:55.280549+0000","last_unstale":"2024-07-04T17:34:56.234441+0000","last_undegraded":"2024-07-04T17:34:56.234441+0000","last_fullsized":"2024-07-04T17:34:56.234441+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:36:40.598708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.234453+0000","last_change":"2024-07-04T17:34:54.263803+0000","last_active":"2024-07-04T17:34:56.234453+0000","last_peered":"2024-07-04T17:34:56.234453+0000","last_clean":"2024-07-04T17:34:56.234453+0000","last_became_active":"2024-07-04T17:34:54.263497+0000","last_became_peered":"2024-07-04T17:34:54.263497+0000","last_unstale":"2024-07-04T17:34:56.234453+0000","last_undegraded":"2024-07-04T17:34:56.234453+0000","last_fullsized":"2024-07-04T17:34:56.234453+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:02:47.521123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.244176+0000","last_change":"2024-07-04T17:34:35.537826+0000","last_active":"2024-07-04T17:34:56.244176+0000","last_peered":"2024-07-04T17:34:56.244176+0000","last_clean":"2024-07-04T17:34:56.244176+0000","last_became_active":"2024-07-04T17:34:35.535781+0000","last_became_peered":"2024-07-04T17:34:35.535781+0000","last_unstale":"2024-07-04T17:34:56.244176+0000","last_undegraded":"2024-07-04T17:34:56.244176+0000","last_fullsized":"2024-07-04T17:34:56.244176+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:26:50.917096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"135'2","reported_seq":24,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.577427+0000","last_change":"2024-07-04T17:34:34.507682+0000","last_active":"2024-07-04T17:34:58.577427+0000","last_peered":"2024-07-04T17:34:58.577427+0000","last_clean":"2024-07-04T17:34:58.577427+0000","last_became_active":"2024-07-04T17:34:34.507353+0000","last_became_peered":"2024-07-04T17:34:34.507353+0000","last_unstale":"2024-07-04T17:34:58.577427+0000","last_undegraded":"2024-07-04T17:34:58.577427+0000","last_fullsized":"2024-07-04T17:34:58.577427+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:47:18.975316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.578931+0000","last_change":"2024-07-04T17:34:55.240381+0000","last_active":"2024-07-04T17:34:58.578931+0000","last_peered":"2024-07-04T17:34:58.578931+0000","last_clean":"2024-07-04T17:34:58.578931+0000","last_became_active":"2024-07-04T17:34:55.239891+0000","last_became_peered":"2024-07-04T17:34:55.239891+0000","last_unstale":"2024-07-04T17:34:58.578931+0000","last_undegraded":"2024-07-04T17:34:58.578931+0000","last_fullsized":"2024-07-04T17:34:58.578931+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:08:19.765528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"139'8","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.234731+0000","last_change":"2024-07-04T17:34:54.264205+0000","last_active":"2024-07-04T17:34:56.234731+0000","last_peered":"2024-07-04T17:34:56.234731+0000","last_clean":"2024-07-04T17:34:56.234731+0000","last_became_active":"2024-07-04T17:34:54.263909+0000","last_became_peered":"2024-07-04T17:34:54.263909+0000","last_unstale":"2024-07-04T17:34:56.234731+0000","last_undegraded":"2024-07-04T17:34:56.234731+0000","last_fullsized":"2024-07-04T17:34:56.234731+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:03:24.700218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.238004+0000","last_change":"2024-07-04T17:34:35.529515+0000","last_active":"2024-07-04T17:34:56.238004+0000","last_peered":"2024-07-04T17:34:56.238004+0000","last_clean":"2024-07-04T17:34:56.238004+0000","last_became_active":"2024-07-04T17:34:35.528532+0000","last_became_peered":"2024-07-04T17:34:35.528532+0000","last_unstale":"2024-07-04T17:34:56.238004+0000","last_undegraded":"2024-07-04T17:34:56.238004+0000","last_fullsized":"2024-07-04T17:34:56.238004+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:33:17.769730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.577366+0000","last_change":"2024-07-04T17:34:34.512082+0000","last_active":"2024-07-04T17:34:58.577366+0000","last_peered":"2024-07-04T17:34:58.577366+0000","last_clean":"2024-07-04T17:34:58.577366+0000","last_became_active":"2024-07-04T17:34:34.511518+0000","last_became_peered":"2024-07-04T17:34:34.511518+0000","last_unstale":"2024-07-04T17:34:58.577366+0000","last_undegraded":"2024-07-04T17:34:58.577366+0000","last_fullsized":"2024-07-04T17:34:58.577366+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:34:32.553466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.243378+0000","last_change":"2024-07-04T17:34:55.260121+0000","last_active":"2024-07-04T17:34:56.243378+0000","last_peered":"2024-07-04T17:34:56.243378+0000","last_clean":"2024-07-04T17:34:56.243378+0000","last_became_active":"2024-07-04T17:34:55.257916+0000","last_became_peered":"2024-07-04T17:34:55.257916+0000","last_unstale":"2024-07-04T17:34:56.243378+0000","last_undegraded":"2024-07-04T17:34:56.243378+0000","last_fullsized":"2024-07-04T17:34:56.243378+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:36:40.598708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.236136+0000","last_change":"2024-07-04T17:34:54.272879+0000","last_active":"2024-07-04T17:34:56.236136+0000","last_peered":"2024-07-04T17:34:56.236136+0000","last_clean":"2024-07-04T17:34:56.236136+0000","last_became_active":"2024-07-04T17:34:54.272295+0000","last_became_peered":"2024-07-04T17:34:54.272295+0000","last_unstale":"2024-07-04T17:34:56.236136+0000","last_undegraded":"2024-07-04T17:34:56.236136+0000","last_fullsized":"2024-07-04T17:34:56.236136+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:37:16.899961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.578924+0000","last_change":"2024-07-04T17:34:35.539061+0000","last_active":"2024-07-04T17:34:58.578924+0000","last_peered":"2024-07-04T17:34:58.578924+0000","last_clean":"2024-07-04T17:34:58.578924+0000","last_became_active":"2024-07-04T17:34:35.538660+0000","last_became_peered":"2024-07-04T17:34:35.538660+0000","last_unstale":"2024-07-04T17:34:58.578924+0000","last_undegraded":"2024-07-04T17:34:58.578924+0000","last_fullsized":"2024-07-04T17:34:58.578924+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:36:20.868550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.577356+0000","last_change":"2024-07-04T17:34:34.513736+0000","last_active":"2024-07-04T17:34:58.577356+0000","last_peered":"2024-07-04T17:34:58.577356+0000","last_clean":"2024-07-04T17:34:58.577356+0000","last_became_active":"2024-07-04T17:34:34.512396+0000","last_became_peered":"2024-07-04T17:34:34.512396+0000","last_unstale":"2024-07-04T17:34:58.577356+0000","last_undegraded":"2024-07-04T17:34:58.577356+0000","last_fullsized":"2024-07-04T17:34:58.577356+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:25:30.776827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.243001+0000","last_change":"2024-07-04T17:34:55.259648+0000","last_active":"2024-07-04T17:34:56.243001+0000","last_peered":"2024-07-04T17:34:56.243001+0000","last_clean":"2024-07-04T17:34:56.243001+0000","last_became_active":"2024-07-04T17:34:55.256614+0000","last_became_peered":"2024-07-04T17:34:55.256614+0000","last_unstale":"2024-07-04T17:34:56.243001+0000","last_undegraded":"2024-07-04T17:34:56.243001+0000","last_fullsized":"2024-07-04T17:34:56.243001+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:16:21.117253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580678+0000","last_change":"2024-07-04T17:34:54.251365+0000","last_active":"2024-07-04T17:34:58.580678+0000","last_peered":"2024-07-04T17:34:58.580678+0000","last_clean":"2024-07-04T17:34:58.580678+0000","last_became_active":"2024-07-04T17:34:54.250997+0000","last_became_peered":"2024-07-04T17:34:54.250997+0000","last_unstale":"2024-07-04T17:34:58.580678+0000","last_undegraded":"2024-07-04T17:34:58.580678+0000","last_fullsized":"2024-07-04T17:34:58.580678+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:53:35.789378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.579622+0000","last_change":"2024-07-04T17:34:35.540239+0000","last_active":"2024-07-04T17:34:58.579622+0000","last_peered":"2024-07-04T17:34:58.579622+0000","last_clean":"2024-07-04T17:34:58.579622+0000","last_became_active":"2024-07-04T17:34:35.539911+0000","last_became_peered":"2024-07-04T17:34:35.539911+0000","last_unstale":"2024-07-04T17:34:58.579622+0000","last_undegraded":"2024-07-04T17:34:58.579622+0000","last_fullsized":"2024-07-04T17:34:58.579622+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:03:47.021324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"135'8","reported_seq":30,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.236664+0000","last_change":"2024-07-04T17:34:34.526204+0000","last_active":"2024-07-04T17:34:56.236664+0000","last_peered":"2024-07-04T17:34:56.236664+0000","last_clean":"2024-07-04T17:34:56.236664+0000","last_became_active":"2024-07-04T17:34:34.525811+0000","last_became_peered":"2024-07-04T17:34:34.525811+0000","last_unstale":"2024-07-04T17:34:56.236664+0000","last_undegraded":"2024-07-04T17:34:56.236664+0000","last_fullsized":"2024-07-04T17:34:56.236664+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:49:36.910071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.236782+0000","last_change":"2024-07-04T17:34:55.271371+0000","last_active":"2024-07-04T17:34:56.236782+0000","last_peered":"2024-07-04T17:34:56.236782+0000","last_clean":"2024-07-04T17:34:56.236782+0000","last_became_active":"2024-07-04T17:34:55.271036+0000","last_became_peered":"2024-07-04T17:34:55.271036+0000","last_unstale":"2024-07-04T17:34:56.236782+0000","last_undegraded":"2024-07-04T17:34:56.236782+0000","last_fullsized":"2024-07-04T17:34:56.236782+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:49:06.940104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.237408+0000","last_change":"2024-07-04T17:34:54.272111+0000","last_active":"2024-07-04T17:34:56.237408+0000","last_peered":"2024-07-04T17:34:56.237408+0000","last_clean":"2024-07-04T17:34:56.237408+0000","last_became_active":"2024-07-04T17:34:54.271789+0000","last_became_peered":"2024-07-04T17:34:54.271789+0000","last_unstale":"2024-07-04T17:34:56.237408+0000","last_undegraded":"2024-07-04T17:34:56.237408+0000","last_fullsized":"2024-07-04T17:34:56.237408+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:10:04.594848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582645+0000","last_change":"2024-07-04T17:34:35.530930+0000","last_active":"2024-07-04T17:34:58.582645+0000","last_peered":"2024-07-04T17:34:58.582645+0000","last_clean":"2024-07-04T17:34:58.582645+0000","last_became_active":"2024-07-04T17:34:35.527474+0000","last_became_peered":"2024-07-04T17:34:35.527474+0000","last_unstale":"2024-07-04T17:34:58.582645+0000","last_undegraded":"2024-07-04T17:34:58.582645+0000","last_fullsized":"2024-07-04T17:34:58.582645+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:51:21.803840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"135'1","reported_seq":23,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.241143+0000","last_change":"2024-07-04T17:34:34.526961+0000","last_active":"2024-07-04T17:34:56.241143+0000","last_peered":"2024-07-04T17:34:56.241143+0000","last_clean":"2024-07-04T17:34:56.241143+0000","last_became_active":"2024-07-04T17:34:34.526338+0000","last_became_peered":"2024-07-04T17:34:34.526338+0000","last_unstale":"2024-07-04T17:34:56.241143+0000","last_undegraded":"2024-07-04T17:34:56.241143+0000","last_fullsized":"2024-07-04T17:34:56.241143+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:46:36.828153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580500+0000","last_change":"2024-07-04T17:34:55.263811+0000","last_active":"2024-07-04T17:34:58.580500+0000","last_peered":"2024-07-04T17:34:58.580500+0000","last_clean":"2024-07-04T17:34:58.580500+0000","last_became_active":"2024-07-04T17:34:55.262987+0000","last_became_peered":"2024-07-04T17:34:55.262987+0000","last_unstale":"2024-07-04T17:34:58.580500+0000","last_undegraded":"2024-07-04T17:34:58.580500+0000","last_fullsized":"2024-07-04T17:34:58.580500+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:51:29.870162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.237662+0000","last_change":"2024-07-04T17:34:54.270824+0000","last_active":"2024-07-04T17:34:56.237662+0000","last_peered":"2024-07-04T17:34:56.237662+0000","last_clean":"2024-07-04T17:34:56.237662+0000","last_became_active":"2024-07-04T17:34:54.270379+0000","last_became_peered":"2024-07-04T17:34:54.270379+0000","last_unstale":"2024-07-04T17:34:56.237662+0000","last_undegraded":"2024-07-04T17:34:56.237662+0000","last_fullsized":"2024-07-04T17:34:56.237662+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:25:50.505982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582706+0000","last_change":"2024-07-04T17:34:35.546152+0000","last_active":"2024-07-04T17:34:58.582706+0000","last_peered":"2024-07-04T17:34:58.582706+0000","last_clean":"2024-07-04T17:34:58.582706+0000","last_became_active":"2024-07-04T17:34:35.545573+0000","last_became_peered":"2024-07-04T17:34:35.545573+0000","last_unstale":"2024-07-04T17:34:58.582706+0000","last_undegraded":"2024-07-04T17:34:58.582706+0000","last_fullsized":"2024-07-04T17:34:58.582706+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:20:21.078798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.e","version":"135'2","reported_seq":24,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.247700+0000","last_change":"2024-07-04T17:34:34.516533+0000","last_active":"2024-07-04T17:34:56.247700+0000","last_peered":"2024-07-04T17:34:56.247700+0000","last_clean":"2024-07-04T17:34:56.247700+0000","last_became_active":"2024-07-04T17:34:34.515492+0000","last_became_peered":"2024-07-04T17:34:34.515492+0000","last_unstale":"2024-07-04T17:34:56.247700+0000","last_undegraded":"2024-07-04T17:34:56.247700+0000","last_fullsized":"2024-07-04T17:34:56.247700+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:06:02.654676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582181+0000","last_change":"2024-07-04T17:34:55.286421+0000","last_active":"2024-07-04T17:34:58.582181+0000","last_peered":"2024-07-04T17:34:58.582181+0000","last_clean":"2024-07-04T17:34:58.582181+0000","last_became_active":"2024-07-04T17:34:55.285847+0000","last_became_peered":"2024-07-04T17:34:55.285847+0000","last_unstale":"2024-07-04T17:34:58.582181+0000","last_undegraded":"2024-07-04T17:34:58.582181+0000","last_fullsized":"2024-07-04T17:34:58.582181+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:06:46.200018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.f","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.248318+0000","last_change":"2024-07-04T17:34:54.266573+0000","last_active":"2024-07-04T17:34:56.248318+0000","last_peered":"2024-07-04T17:34:56.248318+0000","last_clean":"2024-07-04T17:34:56.248318+0000","last_became_active":"2024-07-04T17:34:54.265975+0000","last_became_peered":"2024-07-04T17:34:54.265975+0000","last_unstale":"2024-07-04T17:34:56.248318+0000","last_undegraded":"2024-07-04T17:34:56.248318+0000","last_fullsized":"2024-07-04T17:34:56.248318+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:37:34.579661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.245005+0000","last_change":"2024-07-04T17:34:35.544212+0000","last_active":"2024-07-04T17:34:56.245005+0000","last_peered":"2024-07-04T17:34:56.245005+0000","last_clean":"2024-07-04T17:34:56.245005+0000","last_became_active":"2024-07-04T17:34:35.543570+0000","last_became_peered":"2024-07-04T17:34:35.543570+0000","last_unstale":"2024-07-04T17:34:56.245005+0000","last_undegraded":"2024-07-04T17:34:56.245005+0000","last_fullsized":"2024-07-04T17:34:56.245005+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:38:02.545198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.d","version":"135'1","reported_seq":23,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.577667+0000","last_change":"2024-07-04T17:34:34.514159+0000","last_active":"2024-07-04T17:34:58.577667+0000","last_peered":"2024-07-04T17:34:58.577667+0000","last_clean":"2024-07-04T17:34:58.577667+0000","last_became_active":"2024-07-04T17:34:34.511974+0000","last_became_peered":"2024-07-04T17:34:34.511974+0000","last_unstale":"2024-07-04T17:34:58.577667+0000","last_undegraded":"2024-07-04T17:34:58.577667+0000","last_fullsized":"2024-07-04T17:34:58.577667+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:20:11.531736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.248100+0000","last_change":"2024-07-04T17:34:55.261562+0000","last_active":"2024-07-04T17:34:56.248100+0000","last_peered":"2024-07-04T17:34:56.248100+0000","last_clean":"2024-07-04T17:34:56.248100+0000","last_became_active":"2024-07-04T17:34:55.254887+0000","last_became_peered":"2024-07-04T17:34:55.254887+0000","last_unstale":"2024-07-04T17:34:56.248100+0000","last_undegraded":"2024-07-04T17:34:56.248100+0000","last_fullsized":"2024-07-04T17:34:56.248100+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:54:48.573105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.e","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.240226+0000","last_change":"2024-07-04T17:34:54.271754+0000","last_active":"2024-07-04T17:34:56.240226+0000","last_peered":"2024-07-04T17:34:56.240226+0000","last_clean":"2024-07-04T17:34:56.240226+0000","last_became_active":"2024-07-04T17:34:54.271472+0000","last_became_peered":"2024-07-04T17:34:54.271472+0000","last_unstale":"2024-07-04T17:34:56.240226+0000","last_undegraded":"2024-07-04T17:34:56.240226+0000","last_fullsized":"2024-07-04T17:34:56.240226+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:05:22.077162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582520+0000","last_change":"2024-07-04T17:34:35.523503+0000","last_active":"2024-07-04T17:34:58.582520+0000","last_peered":"2024-07-04T17:34:58.582520+0000","last_clean":"2024-07-04T17:34:58.582520+0000","last_became_active":"2024-07-04T17:34:35.522995+0000","last_became_peered":"2024-07-04T17:34:35.522995+0000","last_unstale":"2024-07-04T17:34:58.582520+0000","last_undegraded":"2024-07-04T17:34:58.582520+0000","last_fullsized":"2024-07-04T17:34:58.582520+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:24:36.746418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.577607+0000","last_change":"2024-07-04T17:34:34.508754+0000","last_active":"2024-07-04T17:34:58.577607+0000","last_peered":"2024-07-04T17:34:58.577607+0000","last_clean":"2024-07-04T17:34:58.577607+0000","last_became_active":"2024-07-04T17:34:34.507614+0000","last_became_peered":"2024-07-04T17:34:34.507614+0000","last_unstale":"2024-07-04T17:34:58.577607+0000","last_undegraded":"2024-07-04T17:34:58.577607+0000","last_fullsized":"2024-07-04T17:34:58.577607+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:09:44.865693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.242349+0000","last_change":"2024-07-04T17:34:55.240066+0000","last_active":"2024-07-04T17:34:56.242349+0000","last_peered":"2024-07-04T17:34:56.242349+0000","last_clean":"2024-07-04T17:34:56.242349+0000","last_became_active":"2024-07-04T17:34:55.237881+0000","last_became_peered":"2024-07-04T17:34:55.237881+0000","last_unstale":"2024-07-04T17:34:56.242349+0000","last_undegraded":"2024-07-04T17:34:56.242349+0000","last_fullsized":"2024-07-04T17:34:56.242349+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:56:38.177831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.579023+0000","last_change":"2024-07-04T17:34:54.248960+0000","last_active":"2024-07-04T17:34:58.579023+0000","last_peered":"2024-07-04T17:34:58.579023+0000","last_clean":"2024-07-04T17:34:58.579023+0000","last_became_active":"2024-07-04T17:34:54.248440+0000","last_became_peered":"2024-07-04T17:34:54.248440+0000","last_unstale":"2024-07-04T17:34:58.579023+0000","last_undegraded":"2024-07-04T17:34:58.579023+0000","last_fullsized":"2024-07-04T17:34:58.579023+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:58:17.189484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580924+0000","last_change":"2024-07-04T17:34:35.515740+0000","last_active":"2024-07-04T17:34:58.580924+0000","last_peered":"2024-07-04T17:34:58.580924+0000","last_clean":"2024-07-04T17:34:58.580924+0000","last_became_active":"2024-07-04T17:34:35.514774+0000","last_became_peered":"2024-07-04T17:34:35.514774+0000","last_unstale":"2024-07-04T17:34:58.580924+0000","last_undegraded":"2024-07-04T17:34:58.580924+0000","last_fullsized":"2024-07-04T17:34:58.580924+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:54:35.373423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"135'2","reported_seq":24,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.244843+0000","last_change":"2024-07-04T17:34:34.515650+0000","last_active":"2024-07-04T17:34:56.244843+0000","last_peered":"2024-07-04T17:34:56.244843+0000","last_clean":"2024-07-04T17:34:56.244843+0000","last_became_active":"2024-07-04T17:34:34.514347+0000","last_became_peered":"2024-07-04T17:34:34.514347+0000","last_unstale":"2024-07-04T17:34:56.244843+0000","last_undegraded":"2024-07-04T17:34:56.244843+0000","last_fullsized":"2024-07-04T17:34:56.244843+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:34:16.111397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.242266+0000","last_change":"2024-07-04T17:34:55.259732+0000","last_active":"2024-07-04T17:34:56.242266+0000","last_peered":"2024-07-04T17:34:56.242266+0000","last_clean":"2024-07-04T17:34:56.242266+0000","last_became_active":"2024-07-04T17:34:55.258543+0000","last_became_peered":"2024-07-04T17:34:55.258543+0000","last_unstale":"2024-07-04T17:34:56.242266+0000","last_undegraded":"2024-07-04T17:34:56.242266+0000","last_fullsized":"2024-07-04T17:34:56.242266+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:32:40.557322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.242314+0000","last_change":"2024-07-04T17:34:54.264077+0000","last_active":"2024-07-04T17:34:56.242314+0000","last_peered":"2024-07-04T17:34:56.242314+0000","last_clean":"2024-07-04T17:34:56.242314+0000","last_became_active":"2024-07-04T17:34:54.263723+0000","last_became_peered":"2024-07-04T17:34:54.263723+0000","last_unstale":"2024-07-04T17:34:56.242314+0000","last_undegraded":"2024-07-04T17:34:56.242314+0000","last_fullsized":"2024-07-04T17:34:56.242314+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:18:44.495991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.238334+0000","last_change":"2024-07-04T17:34:35.529352+0000","last_active":"2024-07-04T17:34:56.238334+0000","last_peered":"2024-07-04T17:34:56.238334+0000","last_clean":"2024-07-04T17:34:56.238334+0000","last_became_active":"2024-07-04T17:34:35.528095+0000","last_became_peered":"2024-07-04T17:34:35.528095+0000","last_unstale":"2024-07-04T17:34:56.238334+0000","last_undegraded":"2024-07-04T17:34:56.238334+0000","last_fullsized":"2024-07-04T17:34:56.238334+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:53:30.230859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.237063+0000","last_change":"2024-07-04T17:34:34.520687+0000","last_active":"2024-07-04T17:34:56.237063+0000","last_peered":"2024-07-04T17:34:56.237063+0000","last_clean":"2024-07-04T17:34:56.237063+0000","last_became_active":"2024-07-04T17:34:34.520328+0000","last_became_peered":"2024-07-04T17:34:34.520328+0000","last_unstale":"2024-07-04T17:34:56.237063+0000","last_undegraded":"2024-07-04T17:34:56.237063+0000","last_fullsized":"2024-07-04T17:34:56.237063+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:26:49.904637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.243687+0000","last_change":"2024-07-04T17:34:55.249029+0000","last_active":"2024-07-04T17:34:56.243687+0000","last_peered":"2024-07-04T17:34:56.243687+0000","last_clean":"2024-07-04T17:34:56.243687+0000","last_became_active":"2024-07-04T17:34:55.248635+0000","last_became_peered":"2024-07-04T17:34:55.248635+0000","last_unstale":"2024-07-04T17:34:56.243687+0000","last_undegraded":"2024-07-04T17:34:56.243687+0000","last_fullsized":"2024-07-04T17:34:56.243687+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:49:06.940104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.b","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.579740+0000","last_change":"2024-07-04T17:34:54.250904+0000","last_active":"2024-07-04T17:34:58.579740+0000","last_peered":"2024-07-04T17:34:58.579740+0000","last_clean":"2024-07-04T17:34:58.579740+0000","last_became_active":"2024-07-04T17:34:54.247680+0000","last_became_peered":"2024-07-04T17:34:54.247680+0000","last_unstale":"2024-07-04T17:34:58.579740+0000","last_undegraded":"2024-07-04T17:34:58.579740+0000","last_fullsized":"2024-07-04T17:34:58.579740+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:23:57.570366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.579048+0000","last_change":"2024-07-04T17:34:35.525251+0000","last_active":"2024-07-04T17:34:58.579048+0000","last_peered":"2024-07-04T17:34:58.579048+0000","last_clean":"2024-07-04T17:34:58.579048+0000","last_became_active":"2024-07-04T17:34:35.524606+0000","last_became_peered":"2024-07-04T17:34:35.524606+0000","last_unstale":"2024-07-04T17:34:58.579048+0000","last_undegraded":"2024-07-04T17:34:58.579048+0000","last_fullsized":"2024-07-04T17:34:58.579048+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:18:18.311591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.245175+0000","last_change":"2024-07-04T17:34:34.515246+0000","last_active":"2024-07-04T17:34:56.245175+0000","last_peered":"2024-07-04T17:34:56.245175+0000","last_clean":"2024-07-04T17:34:56.245175+0000","last_became_active":"2024-07-04T17:34:34.514888+0000","last_became_peered":"2024-07-04T17:34:34.514888+0000","last_unstale":"2024-07-04T17:34:56.245175+0000","last_undegraded":"2024-07-04T17:34:56.245175+0000","last_fullsized":"2024-07-04T17:34:56.245175+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:35:07.644625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.238752+0000","last_change":"2024-07-04T17:34:55.275519+0000","last_active":"2024-07-04T17:34:56.238752+0000","last_peered":"2024-07-04T17:34:56.238752+0000","last_clean":"2024-07-04T17:34:56.238752+0000","last_became_active":"2024-07-04T17:34:55.274909+0000","last_became_peered":"2024-07-04T17:34:55.274909+0000","last_unstale":"2024-07-04T17:34:56.238752+0000","last_undegraded":"2024-07-04T17:34:56.238752+0000","last_fullsized":"2024-07-04T17:34:56.238752+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:54:55.103581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580097+0000","last_change":"2024-07-04T17:34:54.250256+0000","last_active":"2024-07-04T17:34:58.580097+0000","last_peered":"2024-07-04T17:34:58.580097+0000","last_clean":"2024-07-04T17:34:58.580097+0000","last_became_active":"2024-07-04T17:34:54.249493+0000","last_became_peered":"2024-07-04T17:34:54.249493+0000","last_unstale":"2024-07-04T17:34:58.580097+0000","last_undegraded":"2024-07-04T17:34:58.580097+0000","last_fullsized":"2024-07-04T17:34:58.580097+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:17:55.698458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.237412+0000","last_change":"2024-07-04T17:34:35.541160+0000","last_active":"2024-07-04T17:34:56.237412+0000","last_peered":"2024-07-04T17:34:56.237412+0000","last_clean":"2024-07-04T17:34:56.237412+0000","last_became_active":"2024-07-04T17:34:35.540869+0000","last_became_peered":"2024-07-04T17:34:35.540869+0000","last_unstale":"2024-07-04T17:34:56.237412+0000","last_undegraded":"2024-07-04T17:34:56.237412+0000","last_fullsized":"2024-07-04T17:34:56.237412+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:41:18.824553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.236897+0000","last_change":"2024-07-04T17:34:34.521170+0000","last_active":"2024-07-04T17:34:56.236897+0000","last_peered":"2024-07-04T17:34:56.236897+0000","last_clean":"2024-07-04T17:34:56.236897+0000","last_became_active":"2024-07-04T17:34:34.520731+0000","last_became_peered":"2024-07-04T17:34:34.520731+0000","last_unstale":"2024-07-04T17:34:56.236897+0000","last_undegraded":"2024-07-04T17:34:56.236897+0000","last_fullsized":"2024-07-04T17:34:56.236897+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:30:50.167556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.237504+0000","last_change":"2024-07-04T17:34:55.283231+0000","last_active":"2024-07-04T17:34:56.237504+0000","last_peered":"2024-07-04T17:34:56.237504+0000","last_clean":"2024-07-04T17:34:56.237504+0000","last_became_active":"2024-07-04T17:34:55.282939+0000","last_became_peered":"2024-07-04T17:34:55.282939+0000","last_unstale":"2024-07-04T17:34:56.237504+0000","last_undegraded":"2024-07-04T17:34:56.237504+0000","last_fullsized":"2024-07-04T17:34:56.237504+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:40:04.267278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.245883+0000","last_change":"2024-07-04T17:34:54.265406+0000","last_active":"2024-07-04T17:34:56.245883+0000","last_peered":"2024-07-04T17:34:56.245883+0000","last_clean":"2024-07-04T17:34:56.245883+0000","last_became_active":"2024-07-04T17:34:54.264774+0000","last_became_peered":"2024-07-04T17:34:54.264774+0000","last_unstale":"2024-07-04T17:34:56.245883+0000","last_undegraded":"2024-07-04T17:34:56.245883+0000","last_fullsized":"2024-07-04T17:34:56.245883+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:20:07.780705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.241264+0000","last_change":"2024-07-04T17:34:35.534458+0000","last_active":"2024-07-04T17:34:56.241264+0000","last_peered":"2024-07-04T17:34:56.241264+0000","last_clean":"2024-07-04T17:34:56.241264+0000","last_became_active":"2024-07-04T17:34:35.534123+0000","last_became_peered":"2024-07-04T17:34:35.534123+0000","last_unstale":"2024-07-04T17:34:56.241264+0000","last_undegraded":"2024-07-04T17:34:56.241264+0000","last_fullsized":"2024-07-04T17:34:56.241264+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:55:02.890933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"135'2","reported_seq":24,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.577672+0000","last_change":"2024-07-04T17:34:34.516580+0000","last_active":"2024-07-04T17:34:58.577672+0000","last_peered":"2024-07-04T17:34:58.577672+0000","last_clean":"2024-07-04T17:34:58.577672+0000","last_became_active":"2024-07-04T17:34:34.513629+0000","last_became_peered":"2024-07-04T17:34:34.513629+0000","last_unstale":"2024-07-04T17:34:58.577672+0000","last_undegraded":"2024-07-04T17:34:58.577672+0000","last_fullsized":"2024-07-04T17:34:58.577672+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:31:26.410567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.581976+0000","last_change":"2024-07-04T17:34:55.271389+0000","last_active":"2024-07-04T17:34:58.581976+0000","last_peered":"2024-07-04T17:34:58.581976+0000","last_clean":"2024-07-04T17:34:58.581976+0000","last_became_active":"2024-07-04T17:34:55.266272+0000","last_became_peered":"2024-07-04T17:34:55.266272+0000","last_unstale":"2024-07-04T17:34:58.581976+0000","last_undegraded":"2024-07-04T17:34:58.581976+0000","last_fullsized":"2024-07-04T17:34:58.581976+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:11:00.326191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.237859+0000","last_change":"2024-07-04T17:34:54.270486+0000","last_active":"2024-07-04T17:34:56.237859+0000","last_peered":"2024-07-04T17:34:56.237859+0000","last_clean":"2024-07-04T17:34:56.237859+0000","last_became_active":"2024-07-04T17:34:54.269722+0000","last_became_peered":"2024-07-04T17:34:54.269722+0000","last_unstale":"2024-07-04T17:34:56.237859+0000","last_undegraded":"2024-07-04T17:34:56.237859+0000","last_fullsized":"2024-07-04T17:34:56.237859+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:47:38.704471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582751+0000","last_change":"2024-07-04T17:34:35.545883+0000","last_active":"2024-07-04T17:34:58.582751+0000","last_peered":"2024-07-04T17:34:58.582751+0000","last_clean":"2024-07-04T17:34:58.582751+0000","last_became_active":"2024-07-04T17:34:35.545154+0000","last_became_peered":"2024-07-04T17:34:35.545154+0000","last_unstale":"2024-07-04T17:34:58.582751+0000","last_undegraded":"2024-07-04T17:34:58.582751+0000","last_fullsized":"2024-07-04T17:34:58.582751+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:36:58.592291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"135'1","reported_seq":23,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.577643+0000","last_change":"2024-07-04T17:34:34.505851+0000","last_active":"2024-07-04T17:34:58.577643+0000","last_peered":"2024-07-04T17:34:58.577643+0000","last_clean":"2024-07-04T17:34:58.577643+0000","last_became_active":"2024-07-04T17:34:34.505548+0000","last_became_peered":"2024-07-04T17:34:34.505548+0000","last_unstale":"2024-07-04T17:34:58.577643+0000","last_undegraded":"2024-07-04T17:34:58.577643+0000","last_fullsized":"2024-07-04T17:34:58.577643+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:17:04.987342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582029+0000","last_change":"2024-07-04T17:34:55.286697+0000","last_active":"2024-07-04T17:34:58.582029+0000","last_peered":"2024-07-04T17:34:58.582029+0000","last_clean":"2024-07-04T17:34:58.582029+0000","last_became_active":"2024-07-04T17:34:55.286239+0000","last_became_peered":"2024-07-04T17:34:55.286239+0000","last_unstale":"2024-07-04T17:34:58.582029+0000","last_undegraded":"2024-07-04T17:34:58.582029+0000","last_fullsized":"2024-07-04T17:34:58.582029+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:55:24.645280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.7","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.239893+0000","last_change":"2024-07-04T17:34:54.271614+0000","last_active":"2024-07-04T17:34:56.239893+0000","last_peered":"2024-07-04T17:34:56.239893+0000","last_clean":"2024-07-04T17:34:56.239893+0000","last_became_active":"2024-07-04T17:34:54.271161+0000","last_became_peered":"2024-07-04T17:34:54.271161+0000","last_unstale":"2024-07-04T17:34:56.239893+0000","last_undegraded":"2024-07-04T17:34:56.239893+0000","last_fullsized":"2024-07-04T17:34:56.239893+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:34:52.282620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.245100+0000","last_change":"2024-07-04T17:34:35.543872+0000","last_active":"2024-07-04T17:34:56.245100+0000","last_peered":"2024-07-04T17:34:56.245100+0000","last_clean":"2024-07-04T17:34:56.245100+0000","last_became_active":"2024-07-04T17:34:35.542863+0000","last_became_peered":"2024-07-04T17:34:35.542863+0000","last_unstale":"2024-07-04T17:34:56.245100+0000","last_undegraded":"2024-07-04T17:34:56.245100+0000","last_fullsized":"2024-07-04T17:34:56.245100+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:28:18.564334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.577527+0000","last_change":"2024-07-04T17:34:34.505554+0000","last_active":"2024-07-04T17:34:58.577527+0000","last_peered":"2024-07-04T17:34:58.577527+0000","last_clean":"2024-07-04T17:34:58.577527+0000","last_became_active":"2024-07-04T17:34:34.505151+0000","last_became_peered":"2024-07-04T17:34:34.505151+0000","last_unstale":"2024-07-04T17:34:58.577527+0000","last_undegraded":"2024-07-04T17:34:58.577527+0000","last_fullsized":"2024-07-04T17:34:58.577527+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:17:53.346868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580833+0000","last_change":"2024-07-04T17:34:55.287357+0000","last_active":"2024-07-04T17:34:58.580833+0000","last_peered":"2024-07-04T17:34:58.580833+0000","last_clean":"2024-07-04T17:34:58.580833+0000","last_became_active":"2024-07-04T17:34:55.286890+0000","last_became_peered":"2024-07-04T17:34:55.286890+0000","last_unstale":"2024-07-04T17:34:58.580833+0000","last_undegraded":"2024-07-04T17:34:58.580833+0000","last_fullsized":"2024-07-04T17:34:58.580833+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:10:36.543674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"140'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.246992+0000","last_change":"2024-07-04T17:34:54.267192+0000","last_active":"2024-07-04T17:34:56.246992+0000","last_peered":"2024-07-04T17:34:56.246992+0000","last_clean":"2024-07-04T17:34:56.246992+0000","last_became_active":"2024-07-04T17:34:54.266751+0000","last_became_peered":"2024-07-04T17:34:54.266751+0000","last_unstale":"2024-07-04T17:34:56.246992+0000","last_undegraded":"2024-07-04T17:34:56.246992+0000","last_fullsized":"2024-07-04T17:34:56.246992+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:56:37.308739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.247157+0000","last_change":"2024-07-04T17:34:35.543911+0000","last_active":"2024-07-04T17:34:56.247157+0000","last_peered":"2024-07-04T17:34:56.247157+0000","last_clean":"2024-07-04T17:34:56.247157+0000","last_became_active":"2024-07-04T17:34:35.533364+0000","last_became_peered":"2024-07-04T17:34:35.533364+0000","last_unstale":"2024-07-04T17:34:56.247157+0000","last_undegraded":"2024-07-04T17:34:56.247157+0000","last_fullsized":"2024-07-04T17:34:56.247157+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:13:03.582706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.577485+0000","last_change":"2024-07-04T17:34:34.507394+0000","last_active":"2024-07-04T17:34:58.577485+0000","last_peered":"2024-07-04T17:34:58.577485+0000","last_clean":"2024-07-04T17:34:58.577485+0000","last_became_active":"2024-07-04T17:34:34.506560+0000","last_became_peered":"2024-07-04T17:34:34.506560+0000","last_unstale":"2024-07-04T17:34:58.577485+0000","last_undegraded":"2024-07-04T17:34:58.577485+0000","last_fullsized":"2024-07-04T17:34:58.577485+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:12:30.641841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.234817+0000","last_change":"2024-07-04T17:34:55.289868+0000","last_active":"2024-07-04T17:34:56.234817+0000","last_peered":"2024-07-04T17:34:56.234817+0000","last_clean":"2024-07-04T17:34:56.234817+0000","last_became_active":"2024-07-04T17:34:55.286082+0000","last_became_peered":"2024-07-04T17:34:55.286082+0000","last_unstale":"2024-07-04T17:34:56.234817+0000","last_undegraded":"2024-07-04T17:34:56.234817+0000","last_fullsized":"2024-07-04T17:34:56.234817+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:18:38.041750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.578888+0000","last_change":"2024-07-04T17:34:54.246417+0000","last_active":"2024-07-04T17:34:58.578888+0000","last_peered":"2024-07-04T17:34:58.578888+0000","last_clean":"2024-07-04T17:34:58.578888+0000","last_became_active":"2024-07-04T17:34:54.246033+0000","last_became_peered":"2024-07-04T17:34:54.246033+0000","last_unstale":"2024-07-04T17:34:58.578888+0000","last_undegraded":"2024-07-04T17:34:58.578888+0000","last_fullsized":"2024-07-04T17:34:58.578888+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:24:28.116695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.244471+0000","last_change":"2024-07-04T17:34:35.543760+0000","last_active":"2024-07-04T17:34:56.244471+0000","last_peered":"2024-07-04T17:34:56.244471+0000","last_clean":"2024-07-04T17:34:56.244471+0000","last_became_active":"2024-07-04T17:34:35.533337+0000","last_became_peered":"2024-07-04T17:34:35.533337+0000","last_unstale":"2024-07-04T17:34:56.244471+0000","last_undegraded":"2024-07-04T17:34:56.244471+0000","last_fullsized":"2024-07-04T17:34:56.244471+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:15:16.122467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.245193+0000","last_change":"2024-07-04T17:34:34.513417+0000","last_active":"2024-07-04T17:34:56.245193+0000","last_peered":"2024-07-04T17:34:56.245193+0000","last_clean":"2024-07-04T17:34:56.245193+0000","last_became_active":"2024-07-04T17:34:34.513092+0000","last_became_peered":"2024-07-04T17:34:34.513092+0000","last_unstale":"2024-07-04T17:34:56.245193+0000","last_undegraded":"2024-07-04T17:34:56.245193+0000","last_fullsized":"2024-07-04T17:34:56.245193+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:04:50.268887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.242818+0000","last_change":"2024-07-04T17:34:55.260018+0000","last_active":"2024-07-04T17:34:56.242818+0000","last_peered":"2024-07-04T17:34:56.242818+0000","last_clean":"2024-07-04T17:34:56.242818+0000","last_became_active":"2024-07-04T17:34:55.259357+0000","last_became_peered":"2024-07-04T17:34:55.259357+0000","last_unstale":"2024-07-04T17:34:56.242818+0000","last_undegraded":"2024-07-04T17:34:56.242818+0000","last_fullsized":"2024-07-04T17:34:56.242818+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:59:14.916680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.242547+0000","last_change":"2024-07-04T17:34:54.251972+0000","last_active":"2024-07-04T17:34:56.242547+0000","last_peered":"2024-07-04T17:34:56.242547+0000","last_clean":"2024-07-04T17:34:56.242547+0000","last_became_active":"2024-07-04T17:34:54.251663+0000","last_became_peered":"2024-07-04T17:34:54.251663+0000","last_unstale":"2024-07-04T17:34:56.242547+0000","last_undegraded":"2024-07-04T17:34:56.242547+0000","last_fullsized":"2024-07-04T17:34:56.242547+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:10:30.523911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.238631+0000","last_change":"2024-07-04T17:34:35.521764+0000","last_active":"2024-07-04T17:34:56.238631+0000","last_peered":"2024-07-04T17:34:56.238631+0000","last_clean":"2024-07-04T17:34:56.238631+0000","last_became_active":"2024-07-04T17:34:35.521455+0000","last_became_peered":"2024-07-04T17:34:35.521455+0000","last_unstale":"2024-07-04T17:34:56.238631+0000","last_undegraded":"2024-07-04T17:34:56.238631+0000","last_fullsized":"2024-07-04T17:34:56.238631+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:54:12.411851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.244467+0000","last_change":"2024-07-04T17:34:34.518708+0000","last_active":"2024-07-04T17:34:56.244467+0000","last_peered":"2024-07-04T17:34:56.244467+0000","last_clean":"2024-07-04T17:34:56.244467+0000","last_became_active":"2024-07-04T17:34:34.518368+0000","last_became_peered":"2024-07-04T17:34:34.518368+0000","last_unstale":"2024-07-04T17:34:56.244467+0000","last_undegraded":"2024-07-04T17:34:56.244467+0000","last_fullsized":"2024-07-04T17:34:56.244467+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:58:34.507931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.579670+0000","last_change":"2024-07-04T17:34:55.284100+0000","last_active":"2024-07-04T17:34:58.579670+0000","last_peered":"2024-07-04T17:34:58.579670+0000","last_clean":"2024-07-04T17:34:58.579670+0000","last_became_active":"2024-07-04T17:34:55.283787+0000","last_became_peered":"2024-07-04T17:34:55.283787+0000","last_unstale":"2024-07-04T17:34:58.579670+0000","last_undegraded":"2024-07-04T17:34:58.579670+0000","last_fullsized":"2024-07-04T17:34:58.579670+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:34:58.855161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.243671+0000","last_change":"2024-07-04T17:34:54.255923+0000","last_active":"2024-07-04T17:34:56.243671+0000","last_peered":"2024-07-04T17:34:56.243671+0000","last_clean":"2024-07-04T17:34:56.243671+0000","last_became_active":"2024-07-04T17:34:54.255556+0000","last_became_peered":"2024-07-04T17:34:54.255556+0000","last_unstale":"2024-07-04T17:34:56.243671+0000","last_undegraded":"2024-07-04T17:34:56.243671+0000","last_fullsized":"2024-07-04T17:34:56.243671+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:54:31.128546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.243878+0000","last_change":"2024-07-04T17:34:35.520630+0000","last_active":"2024-07-04T17:34:56.243878+0000","last_peered":"2024-07-04T17:34:56.243878+0000","last_clean":"2024-07-04T17:34:56.243878+0000","last_became_active":"2024-07-04T17:34:35.520357+0000","last_became_peered":"2024-07-04T17:34:35.520357+0000","last_unstale":"2024-07-04T17:34:56.243878+0000","last_undegraded":"2024-07-04T17:34:56.243878+0000","last_fullsized":"2024-07-04T17:34:56.243878+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:46:37.840612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.245539+0000","last_change":"2024-07-04T17:34:34.518104+0000","last_active":"2024-07-04T17:34:56.245539+0000","last_peered":"2024-07-04T17:34:56.245539+0000","last_clean":"2024-07-04T17:34:56.245539+0000","last_became_active":"2024-07-04T17:34:34.513906+0000","last_became_peered":"2024-07-04T17:34:34.513906+0000","last_unstale":"2024-07-04T17:34:56.245539+0000","last_undegraded":"2024-07-04T17:34:56.245539+0000","last_fullsized":"2024-07-04T17:34:56.245539+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:43:24.362777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.243367+0000","last_change":"2024-07-04T17:34:55.259916+0000","last_active":"2024-07-04T17:34:56.243367+0000","last_peered":"2024-07-04T17:34:56.243367+0000","last_clean":"2024-07-04T17:34:56.243367+0000","last_became_active":"2024-07-04T17:34:55.257014+0000","last_became_peered":"2024-07-04T17:34:55.257014+0000","last_unstale":"2024-07-04T17:34:56.243367+0000","last_undegraded":"2024-07-04T17:34:56.243367+0000","last_fullsized":"2024-07-04T17:34:56.243367+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:18:38.041750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.243756+0000","last_change":"2024-07-04T17:34:54.267250+0000","last_active":"2024-07-04T17:34:56.243756+0000","last_peered":"2024-07-04T17:34:56.243756+0000","last_clean":"2024-07-04T17:34:56.243756+0000","last_became_active":"2024-07-04T17:34:54.266937+0000","last_became_peered":"2024-07-04T17:34:54.266937+0000","last_unstale":"2024-07-04T17:34:56.243756+0000","last_undegraded":"2024-07-04T17:34:56.243756+0000","last_fullsized":"2024-07-04T17:34:56.243756+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:33:36.486426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.237638+0000","last_change":"2024-07-04T17:34:35.538841+0000","last_active":"2024-07-04T17:34:56.237638+0000","last_peered":"2024-07-04T17:34:56.237638+0000","last_clean":"2024-07-04T17:34:56.237638+0000","last_became_active":"2024-07-04T17:34:35.538551+0000","last_became_peered":"2024-07-04T17:34:35.538551+0000","last_unstale":"2024-07-04T17:34:56.237638+0000","last_undegraded":"2024-07-04T17:34:56.237638+0000","last_fullsized":"2024-07-04T17:34:56.237638+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:02:46.435969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.244048+0000","last_change":"2024-07-04T17:34:34.514509+0000","last_active":"2024-07-04T17:34:56.244048+0000","last_peered":"2024-07-04T17:34:56.244048+0000","last_clean":"2024-07-04T17:34:56.244048+0000","last_became_active":"2024-07-04T17:34:34.511266+0000","last_became_peered":"2024-07-04T17:34:34.511266+0000","last_unstale":"2024-07-04T17:34:56.244048+0000","last_undegraded":"2024-07-04T17:34:56.244048+0000","last_fullsized":"2024-07-04T17:34:56.244048+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:18:52.279136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580727+0000","last_change":"2024-07-04T17:34:55.263706+0000","last_active":"2024-07-04T17:34:58.580727+0000","last_peered":"2024-07-04T17:34:58.580727+0000","last_clean":"2024-07-04T17:34:58.580727+0000","last_became_active":"2024-07-04T17:34:55.263339+0000","last_became_peered":"2024-07-04T17:34:55.263339+0000","last_unstale":"2024-07-04T17:34:58.580727+0000","last_undegraded":"2024-07-04T17:34:58.580727+0000","last_fullsized":"2024-07-04T17:34:58.580727+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:30:29.811812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.243471+0000","last_change":"2024-07-04T17:34:54.267924+0000","last_active":"2024-07-04T17:34:56.243471+0000","last_peered":"2024-07-04T17:34:56.243471+0000","last_clean":"2024-07-04T17:34:56.243471+0000","last_became_active":"2024-07-04T17:34:54.267590+0000","last_became_peered":"2024-07-04T17:34:54.267590+0000","last_unstale":"2024-07-04T17:34:56.243471+0000","last_undegraded":"2024-07-04T17:34:56.243471+0000","last_fullsized":"2024-07-04T17:34:56.243471+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:53:48.947554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.579164+0000","last_change":"2024-07-04T17:34:35.515886+0000","last_active":"2024-07-04T17:34:58.579164+0000","last_peered":"2024-07-04T17:34:58.579164+0000","last_clean":"2024-07-04T17:34:58.579164+0000","last_became_active":"2024-07-04T17:34:35.515383+0000","last_became_peered":"2024-07-04T17:34:35.515383+0000","last_unstale":"2024-07-04T17:34:58.579164+0000","last_undegraded":"2024-07-04T17:34:58.579164+0000","last_fullsized":"2024-07-04T17:34:58.579164+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:04:52.389960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.577369+0000","last_change":"2024-07-04T17:34:34.515873+0000","last_active":"2024-07-04T17:34:58.577369+0000","last_peered":"2024-07-04T17:34:58.577369+0000","last_clean":"2024-07-04T17:34:58.577369+0000","last_became_active":"2024-07-04T17:34:34.515559+0000","last_became_peered":"2024-07-04T17:34:34.515559+0000","last_unstale":"2024-07-04T17:34:58.577369+0000","last_undegraded":"2024-07-04T17:34:58.577369+0000","last_fullsized":"2024-07-04T17:34:58.577369+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:21:52.711984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"122'115","reported_seq":549,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.244233+0000","last_change":"2024-07-04T17:26:04.953559+0000","last_active":"2024-07-04T17:34:56.244233+0000","last_peered":"2024-07-04T17:34:56.244233+0000","last_clean":"2024-07-04T17:34:56.244233+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:34:56.244233+0000","last_undegraded":"2024-07-04T17:34:56.244233+0000","last_fullsized":"2024-07-04T17:34:56.244233+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_clean_scrub_stamp":"2024-07-04T17:26:03.933432+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.064181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1630,"num_write":187,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":4107,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":3},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1630,"num_write":187,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738473,"num_pgs":126,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36804,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335036,"statfs":{"total":96636764160,"available":96599076864,"internally_reserved":0,"allocated":1490944,"data_stored":582416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738474,"num_pgs":124,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37376,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334464,"statfs":{"total":96636764160,"available":96598491136,"internally_reserved":0,"allocated":2084864,"data_stored":1174871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738472,"num_pgs":112,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37372,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334468,"statfs":{"total":96636764160,"available":96598495232,"internally_reserved":0,"allocated":2080768,"data_stored":1174479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:35:00.043 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:35:00.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.337+0000 7f0b7e400640 1 -- 172.21.15.42:0/2152070205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b7806cea0 msgr2=0x7f0b780b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:35:00.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.337+0000 7f0b7e400640 1 --2- 172.21.15.42:0/2152070205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b7806cea0 0x7f0b780b3030 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f0b68009230 tx=0x7f0b6802a480 comp rx=0 tx=0).stop 2024-07-04T17:35:00.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.337+0000 7f0b7e400640 1 -- 172.21.15.42:0/2152070205 shutdown_connections 2024-07-04T17:35:00.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.337+0000 7f0b7e400640 1 --2- 172.21.15.42:0/2152070205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b7806cea0 0x7f0b780b3030 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:35:00.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.337+0000 7f0b7e400640 1 -- 172.21.15.42:0/2152070205 >> 172.21.15.42:0/2152070205 conn(0x7f0b780bf700 msgr2=0x7f0b780af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:35:00.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.337+0000 7f0b7e400640 1 -- 172.21.15.42:0/2152070205 shutdown_connections 2024-07-04T17:35:00.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.337+0000 7f0b7e400640 1 -- 172.21.15.42:0/2152070205 wait complete. 2024-07-04T17:35:00.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.339+0000 7f0b7e400640 1 Processor -- start 2024-07-04T17:35:00.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.340+0000 7f0b7e400640 1 -- start start 2024-07-04T17:35:00.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.341+0000 7f0b7e400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b7806cea0 0x7f0b780ad160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:35:00.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.341+0000 7f0b7e400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7806ef50 con 0x7f0b7806cea0 2024-07-04T17:35:00.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.341+0000 7f0b77400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b7806cea0 0x7f0b780ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:35:00.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.341+0000 7f0b77400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b7806cea0 0x7f0b780ad160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56142/0 (socket says 172.21.15.42:56142) 2024-07-04T17:35:00.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.341+0000 7f0b77400640 1 -- 172.21.15.42:0/3958632971 learned_addr learned my addr 172.21.15.42:0/3958632971 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:35:00.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.342+0000 7f0b77400640 1 -- 172.21.15.42:0/3958632971 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b780ae410 con 0x7f0b7806cea0 2024-07-04T17:35:00.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.342+0000 7f0b77400640 1 --2- 172.21.15.42:0/3958632971 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b7806cea0 0x7f0b780ad160 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f0b68009290 tx=0x7f0b68031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:35:00.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.342+0000 7f0b6f400640 1 -- 172.21.15.42:0/3958632971 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b68041070 con 0x7f0b7806cea0 2024-07-04T17:35:00.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.342+0000 7f0b6f400640 1 -- 172.21.15.42:0/3958632971 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b68003be0 con 0x7f0b7806cea0 2024-07-04T17:35:00.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.342+0000 7f0b7e400640 1 -- 172.21.15.42:0/3958632971 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b781467f0 con 0x7f0b7806cea0 2024-07-04T17:35:00.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.343+0000 7f0b6f400640 1 -- 172.21.15.42:0/3958632971 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b6803b440 con 0x7f0b7806cea0 2024-07-04T17:35:00.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.343+0000 7f0b7e400640 1 -- 172.21.15.42:0/3958632971 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b781462f0 con 0x7f0b7806cea0 2024-07-04T17:35:00.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.344+0000 7f0b7e400640 1 -- 172.21.15.42:0/3958632971 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b781469c0 con 0x7f0b7806cea0 2024-07-04T17:35:00.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.344+0000 7f0b6f400640 1 -- 172.21.15.42:0/3958632971 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f0b68044070 con 0x7f0b7806cea0 2024-07-04T17:35:00.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.345+0000 7f0b6f400640 1 --2- 172.21.15.42:0/3958632971 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f0b5803d4f0 0x7f0b5803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:35:00.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.345+0000 7f0b6f400640 1 -- 172.21.15.42:0/3958632971 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 4303+0+0 (secure 0 0 0) 0x7f0b68073370 con 0x7f0b7806cea0 2024-07-04T17:35:00.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.345+0000 7f0b76a00640 1 --2- 172.21.15.42:0/3958632971 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f0b5803d4f0 0x7f0b5803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:35:00.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.346+0000 7f0b76a00640 1 --2- 172.21.15.42:0/3958632971 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f0b5803d4f0 0x7f0b5803fa00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0b64002020 tx=0x7f0b64001670 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:35:00.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.350+0000 7f0b6f400640 1 -- 172.21.15.42:0/3958632971 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f0b6803b5e0 con 0x7f0b7806cea0 2024-07-04T17:35:00.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.557+0000 7f0b7e400640 1 -- 172.21.15.42:0/3958632971 --> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0b780ae110 con 0x7f0b5803d4f0 2024-07-04T17:35:00.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.578+0000 7f0b6f400640 1 -- 172.21.15.42:0/3958632971 <== mgr.5692 v2:172.21.15.42:6834/4064094064 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+483258 (secure 0 0 0) 0x7f0b780ae110 con 0x7f0b5803d4f0 2024-07-04T17:35:00.581 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:35:00.590 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:35:00.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.585+0000 7f0b7e400640 1 -- 172.21.15.42:0/3958632971 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f0b5803d4f0 msgr2=0x7f0b5803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:35:00.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.585+0000 7f0b7e400640 1 --2- 172.21.15.42:0/3958632971 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f0b5803d4f0 0x7f0b5803fa00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0b64002020 tx=0x7f0b64001670 comp rx=0 tx=0).stop 2024-07-04T17:35:00.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.585+0000 7f0b7e400640 1 -- 172.21.15.42:0/3958632971 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b7806cea0 msgr2=0x7f0b780ad160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:35:00.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.585+0000 7f0b7e400640 1 --2- 172.21.15.42:0/3958632971 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b7806cea0 0x7f0b780ad160 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f0b68009290 tx=0x7f0b68031a90 comp rx=0 tx=0).stop 2024-07-04T17:35:00.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.585+0000 7f0b7e400640 1 -- 172.21.15.42:0/3958632971 shutdown_connections 2024-07-04T17:35:00.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.585+0000 7f0b7e400640 1 --2- 172.21.15.42:0/3958632971 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f0b5803d4f0 0x7f0b5803fa00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:35:00.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.585+0000 7f0b7e400640 1 --2- 172.21.15.42:0/3958632971 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0b7806cea0 0x7f0b780ad160 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:35:00.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.585+0000 7f0b7e400640 1 -- 172.21.15.42:0/3958632971 >> 172.21.15.42:0/3958632971 conn(0x7f0b780bf700 msgr2=0x7f0b781d7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:35:00.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.585+0000 7f0b7e400640 1 -- 172.21.15.42:0/3958632971 shutdown_connections 2024-07-04T17:35:00.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.585+0000 7f0b7e400640 1 -- 172.21.15.42:0/3958632971 wait complete. 2024-07-04T17:35:00.612 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":45,"stamp":"2024-07-04T17:34:59.116184+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596885,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":596,"num_read_kb":1630,"num_write":279,"num_write_kb":1137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":195,"ondisk_log_size":195,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":362,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":111552,"kb_used_data":5524,"kb_used_omap":4,"kb_used_meta":105979,"kb_avail":283003968,"statfs":{"total":289910292480,"available":289796063232,"internally_reserved":0,"allocated":5656576,"data_stored":2931766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":108522831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.923660"},"pg_stats":[{"pgid":"27.3d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.236651+0000","last_change":"2024-07-04T17:34:55.282061+0000","last_active":"2024-07-04T17:34:56.236651+0000","last_peered":"2024-07-04T17:34:56.236651+0000","last_clean":"2024-07-04T17:34:56.236651+0000","last_became_active":"2024-07-04T17:34:55.281584+0000","last_became_peered":"2024-07-04T17:34:55.281584+0000","last_unstale":"2024-07-04T17:34:56.236651+0000","last_undegraded":"2024-07-04T17:34:56.236651+0000","last_fullsized":"2024-07-04T17:34:56.236651+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:05:12.120118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3f","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.248651+0000","last_change":"2024-07-04T17:34:35.545191+0000","last_active":"2024-07-04T17:34:56.248651+0000","last_peered":"2024-07-04T17:34:56.248651+0000","last_clean":"2024-07-04T17:34:56.248651+0000","last_became_active":"2024-07-04T17:34:35.542859+0000","last_became_peered":"2024-07-04T17:34:35.542859+0000","last_unstale":"2024-07-04T17:34:56.248651+0000","last_undegraded":"2024-07-04T17:34:56.248651+0000","last_fullsized":"2024-07-04T17:34:56.248651+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:36:40.642306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.243433+0000","last_change":"2024-07-04T17:34:55.246499+0000","last_active":"2024-07-04T17:34:56.243433+0000","last_peered":"2024-07-04T17:34:56.243433+0000","last_clean":"2024-07-04T17:34:56.243433+0000","last_became_active":"2024-07-04T17:34:55.240565+0000","last_became_peered":"2024-07-04T17:34:55.240565+0000","last_unstale":"2024-07-04T17:34:56.243433+0000","last_undegraded":"2024-07-04T17:34:56.243433+0000","last_fullsized":"2024-07-04T17:34:56.243433+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:04:06.751483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.247337+0000","last_change":"2024-07-04T17:34:35.541970+0000","last_active":"2024-07-04T17:34:56.247337+0000","last_peered":"2024-07-04T17:34:56.247337+0000","last_clean":"2024-07-04T17:34:56.247337+0000","last_became_active":"2024-07-04T17:34:35.541341+0000","last_became_peered":"2024-07-04T17:34:35.541341+0000","last_unstale":"2024-07-04T17:34:56.247337+0000","last_undegraded":"2024-07-04T17:34:56.247337+0000","last_fullsized":"2024-07-04T17:34:56.247337+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:06:30.239920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582457+0000","last_change":"2024-07-04T17:34:55.277627+0000","last_active":"2024-07-04T17:34:58.582457+0000","last_peered":"2024-07-04T17:34:58.582457+0000","last_clean":"2024-07-04T17:34:58.582457+0000","last_became_active":"2024-07-04T17:34:55.276422+0000","last_became_peered":"2024-07-04T17:34:55.276422+0000","last_unstale":"2024-07-04T17:34:58.582457+0000","last_undegraded":"2024-07-04T17:34:58.582457+0000","last_fullsized":"2024-07-04T17:34:58.582457+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:41:17.056030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.245821+0000","last_change":"2024-07-04T17:34:35.531108+0000","last_active":"2024-07-04T17:34:56.245821+0000","last_peered":"2024-07-04T17:34:56.245821+0000","last_clean":"2024-07-04T17:34:56.245821+0000","last_became_active":"2024-07-04T17:34:35.530755+0000","last_became_peered":"2024-07-04T17:34:35.530755+0000","last_unstale":"2024-07-04T17:34:56.245821+0000","last_undegraded":"2024-07-04T17:34:56.245821+0000","last_fullsized":"2024-07-04T17:34:56.245821+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:48:46.716399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.239846+0000","last_change":"2024-07-04T17:34:55.281142+0000","last_active":"2024-07-04T17:34:56.239846+0000","last_peered":"2024-07-04T17:34:56.239846+0000","last_clean":"2024-07-04T17:34:56.239846+0000","last_became_active":"2024-07-04T17:34:55.280578+0000","last_became_peered":"2024-07-04T17:34:55.280578+0000","last_unstale":"2024-07-04T17:34:56.239846+0000","last_undegraded":"2024-07-04T17:34:56.239846+0000","last_fullsized":"2024-07-04T17:34:56.239846+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:54:48.573105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.244690+0000","last_change":"2024-07-04T17:34:35.534705+0000","last_active":"2024-07-04T17:34:56.244690+0000","last_peered":"2024-07-04T17:34:56.244690+0000","last_clean":"2024-07-04T17:34:56.244690+0000","last_became_active":"2024-07-04T17:34:35.532134+0000","last_became_peered":"2024-07-04T17:34:35.532134+0000","last_unstale":"2024-07-04T17:34:56.244690+0000","last_undegraded":"2024-07-04T17:34:56.244690+0000","last_fullsized":"2024-07-04T17:34:56.244690+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:04:01.056622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.578758+0000","last_change":"2024-07-04T17:34:55.288989+0000","last_active":"2024-07-04T17:34:58.578758+0000","last_peered":"2024-07-04T17:34:58.578758+0000","last_clean":"2024-07-04T17:34:58.578758+0000","last_became_active":"2024-07-04T17:34:55.287981+0000","last_became_peered":"2024-07-04T17:34:55.287981+0000","last_unstale":"2024-07-04T17:34:58.578758+0000","last_undegraded":"2024-07-04T17:34:58.578758+0000","last_fullsized":"2024-07-04T17:34:58.578758+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:12:26.839696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.237095+0000","last_change":"2024-07-04T17:34:35.529627+0000","last_active":"2024-07-04T17:34:56.237095+0000","last_peered":"2024-07-04T17:34:56.237095+0000","last_clean":"2024-07-04T17:34:56.237095+0000","last_became_active":"2024-07-04T17:34:35.528998+0000","last_became_peered":"2024-07-04T17:34:35.528998+0000","last_unstale":"2024-07-04T17:34:56.237095+0000","last_undegraded":"2024-07-04T17:34:56.237095+0000","last_fullsized":"2024-07-04T17:34:56.237095+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:53:22.203053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.241804+0000","last_change":"2024-07-04T17:34:55.258684+0000","last_active":"2024-07-04T17:34:56.241804+0000","last_peered":"2024-07-04T17:34:56.241804+0000","last_clean":"2024-07-04T17:34:56.241804+0000","last_became_active":"2024-07-04T17:34:55.254309+0000","last_became_peered":"2024-07-04T17:34:55.254309+0000","last_unstale":"2024-07-04T17:34:56.241804+0000","last_undegraded":"2024-07-04T17:34:56.241804+0000","last_fullsized":"2024-07-04T17:34:56.241804+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:23:01.177074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.244028+0000","last_change":"2024-07-04T17:34:35.517987+0000","last_active":"2024-07-04T17:34:56.244028+0000","last_peered":"2024-07-04T17:34:56.244028+0000","last_clean":"2024-07-04T17:34:56.244028+0000","last_became_active":"2024-07-04T17:34:35.517658+0000","last_became_peered":"2024-07-04T17:34:35.517658+0000","last_unstale":"2024-07-04T17:34:56.244028+0000","last_undegraded":"2024-07-04T17:34:56.244028+0000","last_fullsized":"2024-07-04T17:34:56.244028+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:49:37.922530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.242707+0000","last_change":"2024-07-04T17:34:55.259361+0000","last_active":"2024-07-04T17:34:56.242707+0000","last_peered":"2024-07-04T17:34:56.242707+0000","last_clean":"2024-07-04T17:34:56.242707+0000","last_became_active":"2024-07-04T17:34:55.256213+0000","last_became_peered":"2024-07-04T17:34:55.256213+0000","last_unstale":"2024-07-04T17:34:56.242707+0000","last_undegraded":"2024-07-04T17:34:56.242707+0000","last_fullsized":"2024-07-04T17:34:56.242707+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:27:12.706155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.235049+0000","last_change":"2024-07-04T17:34:35.555625+0000","last_active":"2024-07-04T17:34:56.235049+0000","last_peered":"2024-07-04T17:34:56.235049+0000","last_clean":"2024-07-04T17:34:56.235049+0000","last_became_active":"2024-07-04T17:34:35.554652+0000","last_became_peered":"2024-07-04T17:34:35.554652+0000","last_unstale":"2024-07-04T17:34:56.235049+0000","last_undegraded":"2024-07-04T17:34:56.235049+0000","last_fullsized":"2024-07-04T17:34:56.235049+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:47:42.975532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.242948+0000","last_change":"2024-07-04T17:34:55.239188+0000","last_active":"2024-07-04T17:34:56.242948+0000","last_peered":"2024-07-04T17:34:56.242948+0000","last_clean":"2024-07-04T17:34:56.242948+0000","last_became_active":"2024-07-04T17:34:55.237333+0000","last_became_peered":"2024-07-04T17:34:55.237333+0000","last_unstale":"2024-07-04T17:34:56.242948+0000","last_undegraded":"2024-07-04T17:34:56.242948+0000","last_fullsized":"2024-07-04T17:34:56.242948+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:14:53.439738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.243972+0000","last_change":"2024-07-04T17:34:35.543396+0000","last_active":"2024-07-04T17:34:56.243972+0000","last_peered":"2024-07-04T17:34:56.243972+0000","last_clean":"2024-07-04T17:34:56.243972+0000","last_became_active":"2024-07-04T17:34:35.542958+0000","last_became_peered":"2024-07-04T17:34:35.542958+0000","last_unstale":"2024-07-04T17:34:56.243972+0000","last_undegraded":"2024-07-04T17:34:56.243972+0000","last_fullsized":"2024-07-04T17:34:56.243972+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:37:46.927489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.237183+0000","last_change":"2024-07-04T17:34:55.284874+0000","last_active":"2024-07-04T17:34:56.237183+0000","last_peered":"2024-07-04T17:34:56.237183+0000","last_clean":"2024-07-04T17:34:56.237183+0000","last_became_active":"2024-07-04T17:34:55.284544+0000","last_became_peered":"2024-07-04T17:34:55.284544+0000","last_unstale":"2024-07-04T17:34:56.237183+0000","last_undegraded":"2024-07-04T17:34:56.237183+0000","last_fullsized":"2024-07-04T17:34:56.237183+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:52:31.382449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.248631+0000","last_change":"2024-07-04T17:34:35.543199+0000","last_active":"2024-07-04T17:34:56.248631+0000","last_peered":"2024-07-04T17:34:56.248631+0000","last_clean":"2024-07-04T17:34:56.248631+0000","last_became_active":"2024-07-04T17:34:35.537240+0000","last_became_peered":"2024-07-04T17:34:35.537240+0000","last_unstale":"2024-07-04T17:34:56.248631+0000","last_undegraded":"2024-07-04T17:34:56.248631+0000","last_fullsized":"2024-07-04T17:34:56.248631+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:17:07.663136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.579818+0000","last_change":"2024-07-04T17:34:55.288619+0000","last_active":"2024-07-04T17:34:58.579818+0000","last_peered":"2024-07-04T17:34:58.579818+0000","last_clean":"2024-07-04T17:34:58.579818+0000","last_became_active":"2024-07-04T17:34:55.288057+0000","last_became_peered":"2024-07-04T17:34:55.288057+0000","last_unstale":"2024-07-04T17:34:58.579818+0000","last_undegraded":"2024-07-04T17:34:58.579818+0000","last_fullsized":"2024-07-04T17:34:58.579818+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:46:28.164230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582367+0000","last_change":"2024-07-04T17:34:35.552696+0000","last_active":"2024-07-04T17:34:58.582367+0000","last_peered":"2024-07-04T17:34:58.582367+0000","last_clean":"2024-07-04T17:34:58.582367+0000","last_became_active":"2024-07-04T17:34:35.552377+0000","last_became_peered":"2024-07-04T17:34:35.552377+0000","last_unstale":"2024-07-04T17:34:58.582367+0000","last_undegraded":"2024-07-04T17:34:58.582367+0000","last_fullsized":"2024-07-04T17:34:58.582367+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:54:28.842946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.240464+0000","last_change":"2024-07-04T17:34:55.281556+0000","last_active":"2024-07-04T17:34:56.240464+0000","last_peered":"2024-07-04T17:34:56.240464+0000","last_clean":"2024-07-04T17:34:56.240464+0000","last_became_active":"2024-07-04T17:34:55.281090+0000","last_became_peered":"2024-07-04T17:34:55.281090+0000","last_unstale":"2024-07-04T17:34:56.240464+0000","last_undegraded":"2024-07-04T17:34:56.240464+0000","last_fullsized":"2024-07-04T17:34:56.240464+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:41:49.878066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.239112+0000","last_change":"2024-07-04T17:34:35.535880+0000","last_active":"2024-07-04T17:34:56.239112+0000","last_peered":"2024-07-04T17:34:56.239112+0000","last_clean":"2024-07-04T17:34:56.239112+0000","last_became_active":"2024-07-04T17:34:35.535547+0000","last_became_peered":"2024-07-04T17:34:35.535547+0000","last_unstale":"2024-07-04T17:34:56.239112+0000","last_undegraded":"2024-07-04T17:34:56.239112+0000","last_fullsized":"2024-07-04T17:34:56.239112+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:22:16.634063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580166+0000","last_change":"2024-07-04T17:34:55.277218+0000","last_active":"2024-07-04T17:34:58.580166+0000","last_peered":"2024-07-04T17:34:58.580166+0000","last_clean":"2024-07-04T17:34:58.580166+0000","last_became_active":"2024-07-04T17:34:55.276877+0000","last_became_peered":"2024-07-04T17:34:55.276877+0000","last_unstale":"2024-07-04T17:34:58.580166+0000","last_undegraded":"2024-07-04T17:34:58.580166+0000","last_fullsized":"2024-07-04T17:34:58.580166+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:47:24.561498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580845+0000","last_change":"2024-07-04T17:34:35.561769+0000","last_active":"2024-07-04T17:34:58.580845+0000","last_peered":"2024-07-04T17:34:58.580845+0000","last_clean":"2024-07-04T17:34:58.580845+0000","last_became_active":"2024-07-04T17:34:35.560437+0000","last_became_peered":"2024-07-04T17:34:35.560437+0000","last_unstale":"2024-07-04T17:34:58.580845+0000","last_undegraded":"2024-07-04T17:34:58.580845+0000","last_fullsized":"2024-07-04T17:34:58.580845+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:52:11.652290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.578833+0000","last_change":"2024-07-04T17:34:55.254016+0000","last_active":"2024-07-04T17:34:58.578833+0000","last_peered":"2024-07-04T17:34:58.578833+0000","last_clean":"2024-07-04T17:34:58.578833+0000","last_became_active":"2024-07-04T17:34:55.253616+0000","last_became_peered":"2024-07-04T17:34:55.253616+0000","last_unstale":"2024-07-04T17:34:58.578833+0000","last_undegraded":"2024-07-04T17:34:58.578833+0000","last_fullsized":"2024-07-04T17:34:58.578833+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:08:01.918120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.244854+0000","last_change":"2024-07-04T17:34:35.514882+0000","last_active":"2024-07-04T17:34:56.244854+0000","last_peered":"2024-07-04T17:34:56.244854+0000","last_clean":"2024-07-04T17:34:56.244854+0000","last_became_active":"2024-07-04T17:34:35.514560+0000","last_became_peered":"2024-07-04T17:34:35.514560+0000","last_unstale":"2024-07-04T17:34:56.244854+0000","last_undegraded":"2024-07-04T17:34:56.244854+0000","last_fullsized":"2024-07-04T17:34:56.244854+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:46:59.706787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.241737+0000","last_change":"2024-07-04T17:34:55.238218+0000","last_active":"2024-07-04T17:34:56.241737+0000","last_peered":"2024-07-04T17:34:56.241737+0000","last_clean":"2024-07-04T17:34:56.241737+0000","last_became_active":"2024-07-04T17:34:55.237053+0000","last_became_peered":"2024-07-04T17:34:55.237053+0000","last_unstale":"2024-07-04T17:34:56.241737+0000","last_undegraded":"2024-07-04T17:34:56.241737+0000","last_fullsized":"2024-07-04T17:34:56.241737+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:47:25.955446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.244101+0000","last_change":"2024-07-04T17:34:35.538102+0000","last_active":"2024-07-04T17:34:56.244101+0000","last_peered":"2024-07-04T17:34:56.244101+0000","last_clean":"2024-07-04T17:34:56.244101+0000","last_became_active":"2024-07-04T17:34:35.536203+0000","last_became_peered":"2024-07-04T17:34:35.536203+0000","last_unstale":"2024-07-04T17:34:56.244101+0000","last_undegraded":"2024-07-04T17:34:56.244101+0000","last_fullsized":"2024-07-04T17:34:56.244101+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:15:42.373781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.244440+0000","last_change":"2024-07-04T17:34:55.260190+0000","last_active":"2024-07-04T17:34:56.244440+0000","last_peered":"2024-07-04T17:34:56.244440+0000","last_clean":"2024-07-04T17:34:56.244440+0000","last_became_active":"2024-07-04T17:34:55.259302+0000","last_became_peered":"2024-07-04T17:34:55.259302+0000","last_unstale":"2024-07-04T17:34:56.244440+0000","last_undegraded":"2024-07-04T17:34:56.244440+0000","last_fullsized":"2024-07-04T17:34:56.244440+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:04:05.855295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.579486+0000","last_change":"2024-07-04T17:34:35.542229+0000","last_active":"2024-07-04T17:34:58.579486+0000","last_peered":"2024-07-04T17:34:58.579486+0000","last_clean":"2024-07-04T17:34:58.579486+0000","last_became_active":"2024-07-04T17:34:35.541754+0000","last_became_peered":"2024-07-04T17:34:35.541754+0000","last_unstale":"2024-07-04T17:34:58.579486+0000","last_undegraded":"2024-07-04T17:34:58.579486+0000","last_fullsized":"2024-07-04T17:34:58.579486+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:35:52.678989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.579473+0000","last_change":"2024-07-04T17:34:55.264141+0000","last_active":"2024-07-04T17:34:58.579473+0000","last_peered":"2024-07-04T17:34:58.579473+0000","last_clean":"2024-07-04T17:34:58.579473+0000","last_became_active":"2024-07-04T17:34:55.263789+0000","last_became_peered":"2024-07-04T17:34:55.263789+0000","last_unstale":"2024-07-04T17:34:58.579473+0000","last_undegraded":"2024-07-04T17:34:58.579473+0000","last_fullsized":"2024-07-04T17:34:58.579473+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:04:32.541821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.235418+0000","last_change":"2024-07-04T17:34:35.525814+0000","last_active":"2024-07-04T17:34:56.235418+0000","last_peered":"2024-07-04T17:34:56.235418+0000","last_clean":"2024-07-04T17:34:56.235418+0000","last_became_active":"2024-07-04T17:34:35.525479+0000","last_became_peered":"2024-07-04T17:34:35.525479+0000","last_unstale":"2024-07-04T17:34:56.235418+0000","last_undegraded":"2024-07-04T17:34:56.235418+0000","last_fullsized":"2024-07-04T17:34:56.235418+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:44:57.595263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.245460+0000","last_change":"2024-07-04T17:34:55.259558+0000","last_active":"2024-07-04T17:34:56.245460+0000","last_peered":"2024-07-04T17:34:56.245460+0000","last_clean":"2024-07-04T17:34:56.245460+0000","last_became_active":"2024-07-04T17:34:55.258996+0000","last_became_peered":"2024-07-04T17:34:55.258996+0000","last_unstale":"2024-07-04T17:34:56.245460+0000","last_undegraded":"2024-07-04T17:34:56.245460+0000","last_fullsized":"2024-07-04T17:34:56.245460+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:40:04.267278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.240612+0000","last_change":"2024-07-04T17:34:35.554702+0000","last_active":"2024-07-04T17:34:56.240612+0000","last_peered":"2024-07-04T17:34:56.240612+0000","last_clean":"2024-07-04T17:34:56.240612+0000","last_became_active":"2024-07-04T17:34:35.554205+0000","last_became_peered":"2024-07-04T17:34:35.554205+0000","last_unstale":"2024-07-04T17:34:56.240612+0000","last_undegraded":"2024-07-04T17:34:56.240612+0000","last_fullsized":"2024-07-04T17:34:56.240612+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:18:48.550958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.244609+0000","last_change":"2024-07-04T17:34:55.252463+0000","last_active":"2024-07-04T17:34:56.244609+0000","last_peered":"2024-07-04T17:34:56.244609+0000","last_clean":"2024-07-04T17:34:56.244609+0000","last_became_active":"2024-07-04T17:34:55.251930+0000","last_became_peered":"2024-07-04T17:34:55.251930+0000","last_unstale":"2024-07-04T17:34:56.244609+0000","last_undegraded":"2024-07-04T17:34:56.244609+0000","last_fullsized":"2024-07-04T17:34:56.244609+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:52:31.382449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.241188+0000","last_change":"2024-07-04T17:34:35.536019+0000","last_active":"2024-07-04T17:34:56.241188+0000","last_peered":"2024-07-04T17:34:56.241188+0000","last_clean":"2024-07-04T17:34:56.241188+0000","last_became_active":"2024-07-04T17:34:35.535705+0000","last_became_peered":"2024-07-04T17:34:35.535705+0000","last_unstale":"2024-07-04T17:34:56.241188+0000","last_undegraded":"2024-07-04T17:34:56.241188+0000","last_fullsized":"2024-07-04T17:34:56.241188+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:06:39.223609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582360+0000","last_change":"2024-07-04T17:34:55.288462+0000","last_active":"2024-07-04T17:34:58.582360+0000","last_peered":"2024-07-04T17:34:58.582360+0000","last_clean":"2024-07-04T17:34:58.582360+0000","last_became_active":"2024-07-04T17:34:55.287324+0000","last_became_peered":"2024-07-04T17:34:55.287324+0000","last_unstale":"2024-07-04T17:34:58.582360+0000","last_undegraded":"2024-07-04T17:34:58.582360+0000","last_fullsized":"2024-07-04T17:34:58.582360+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:31:23.342645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582066+0000","last_change":"2024-07-04T17:34:35.562071+0000","last_active":"2024-07-04T17:34:58.582066+0000","last_peered":"2024-07-04T17:34:58.582066+0000","last_clean":"2024-07-04T17:34:58.582066+0000","last_became_active":"2024-07-04T17:34:35.560220+0000","last_became_peered":"2024-07-04T17:34:35.560220+0000","last_unstale":"2024-07-04T17:34:58.582066+0000","last_undegraded":"2024-07-04T17:34:58.582066+0000","last_fullsized":"2024-07-04T17:34:58.582066+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:27:08.229969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.240607+0000","last_change":"2024-07-04T17:34:55.283160+0000","last_active":"2024-07-04T17:34:56.240607+0000","last_peered":"2024-07-04T17:34:56.240607+0000","last_clean":"2024-07-04T17:34:56.240607+0000","last_became_active":"2024-07-04T17:34:55.282839+0000","last_became_peered":"2024-07-04T17:34:55.282839+0000","last_unstale":"2024-07-04T17:34:56.240607+0000","last_undegraded":"2024-07-04T17:34:56.240607+0000","last_fullsized":"2024-07-04T17:34:56.240607+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:38:09.271247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.238302+0000","last_change":"2024-07-04T17:34:35.553714+0000","last_active":"2024-07-04T17:34:56.238302+0000","last_peered":"2024-07-04T17:34:56.238302+0000","last_clean":"2024-07-04T17:34:56.238302+0000","last_became_active":"2024-07-04T17:34:35.553384+0000","last_became_peered":"2024-07-04T17:34:35.553384+0000","last_unstale":"2024-07-04T17:34:56.238302+0000","last_undegraded":"2024-07-04T17:34:56.238302+0000","last_fullsized":"2024-07-04T17:34:56.238302+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:47:20.883963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.241749+0000","last_change":"2024-07-04T17:34:55.239877+0000","last_active":"2024-07-04T17:34:56.241749+0000","last_peered":"2024-07-04T17:34:56.241749+0000","last_clean":"2024-07-04T17:34:56.241749+0000","last_became_active":"2024-07-04T17:34:55.239581+0000","last_became_peered":"2024-07-04T17:34:55.239581+0000","last_unstale":"2024-07-04T17:34:56.241749+0000","last_undegraded":"2024-07-04T17:34:56.241749+0000","last_fullsized":"2024-07-04T17:34:56.241749+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:20:07.104502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.237301+0000","last_change":"2024-07-04T17:34:35.554950+0000","last_active":"2024-07-04T17:34:56.237301+0000","last_peered":"2024-07-04T17:34:56.237301+0000","last_clean":"2024-07-04T17:34:56.237301+0000","last_became_active":"2024-07-04T17:34:35.554259+0000","last_became_peered":"2024-07-04T17:34:35.554259+0000","last_unstale":"2024-07-04T17:34:56.237301+0000","last_undegraded":"2024-07-04T17:34:56.237301+0000","last_fullsized":"2024-07-04T17:34:56.237301+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:59:46.128637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.233908+0000","last_change":"2024-07-04T17:34:55.272867+0000","last_active":"2024-07-04T17:34:56.233908+0000","last_peered":"2024-07-04T17:34:56.233908+0000","last_clean":"2024-07-04T17:34:56.233908+0000","last_became_active":"2024-07-04T17:34:55.270738+0000","last_became_peered":"2024-07-04T17:34:55.270738+0000","last_unstale":"2024-07-04T17:34:56.233908+0000","last_undegraded":"2024-07-04T17:34:56.233908+0000","last_fullsized":"2024-07-04T17:34:56.233908+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:16:21.117253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580453+0000","last_change":"2024-07-04T17:34:35.523276+0000","last_active":"2024-07-04T17:34:58.580453+0000","last_peered":"2024-07-04T17:34:58.580453+0000","last_clean":"2024-07-04T17:34:58.580453+0000","last_became_active":"2024-07-04T17:34:35.522611+0000","last_became_peered":"2024-07-04T17:34:35.522611+0000","last_unstale":"2024-07-04T17:34:58.580453+0000","last_undegraded":"2024-07-04T17:34:58.580453+0000","last_fullsized":"2024-07-04T17:34:58.580453+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:48:47.209945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580202+0000","last_change":"2024-07-04T17:34:55.281064+0000","last_active":"2024-07-04T17:34:58.580202+0000","last_peered":"2024-07-04T17:34:58.580202+0000","last_clean":"2024-07-04T17:34:58.580202+0000","last_became_active":"2024-07-04T17:34:55.280040+0000","last_became_peered":"2024-07-04T17:34:55.280040+0000","last_unstale":"2024-07-04T17:34:58.580202+0000","last_undegraded":"2024-07-04T17:34:58.580202+0000","last_fullsized":"2024-07-04T17:34:58.580202+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:45:05.455074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.244255+0000","last_change":"2024-07-04T17:34:35.546941+0000","last_active":"2024-07-04T17:34:56.244255+0000","last_peered":"2024-07-04T17:34:56.244255+0000","last_clean":"2024-07-04T17:34:56.244255+0000","last_became_active":"2024-07-04T17:34:35.544163+0000","last_became_peered":"2024-07-04T17:34:35.544163+0000","last_unstale":"2024-07-04T17:34:56.244255+0000","last_undegraded":"2024-07-04T17:34:56.244255+0000","last_fullsized":"2024-07-04T17:34:56.244255+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:15:08.178810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580021+0000","last_change":"2024-07-04T17:34:55.284190+0000","last_active":"2024-07-04T17:34:58.580021+0000","last_peered":"2024-07-04T17:34:58.580021+0000","last_clean":"2024-07-04T17:34:58.580021+0000","last_became_active":"2024-07-04T17:34:55.283868+0000","last_became_peered":"2024-07-04T17:34:55.283868+0000","last_unstale":"2024-07-04T17:34:58.580021+0000","last_undegraded":"2024-07-04T17:34:58.580021+0000","last_fullsized":"2024-07-04T17:34:58.580021+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:53:36.933359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.578815+0000","last_change":"2024-07-04T17:34:35.545420+0000","last_active":"2024-07-04T17:34:58.578815+0000","last_peered":"2024-07-04T17:34:58.578815+0000","last_clean":"2024-07-04T17:34:58.578815+0000","last_became_active":"2024-07-04T17:34:35.543935+0000","last_became_peered":"2024-07-04T17:34:35.543935+0000","last_unstale":"2024-07-04T17:34:58.578815+0000","last_undegraded":"2024-07-04T17:34:58.578815+0000","last_fullsized":"2024-07-04T17:34:58.578815+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:16:01.387094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582403+0000","last_change":"2024-07-04T17:34:55.285445+0000","last_active":"2024-07-04T17:34:58.582403+0000","last_peered":"2024-07-04T17:34:58.582403+0000","last_clean":"2024-07-04T17:34:58.582403+0000","last_became_active":"2024-07-04T17:34:55.285049+0000","last_became_peered":"2024-07-04T17:34:55.285049+0000","last_unstale":"2024-07-04T17:34:58.582403+0000","last_undegraded":"2024-07-04T17:34:58.582403+0000","last_fullsized":"2024-07-04T17:34:58.582403+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:20:51.546736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.248985+0000","last_change":"2024-07-04T17:34:35.536388+0000","last_active":"2024-07-04T17:34:56.248985+0000","last_peered":"2024-07-04T17:34:56.248985+0000","last_clean":"2024-07-04T17:34:56.248985+0000","last_became_active":"2024-07-04T17:34:35.536052+0000","last_became_peered":"2024-07-04T17:34:35.536052+0000","last_unstale":"2024-07-04T17:34:56.248985+0000","last_undegraded":"2024-07-04T17:34:56.248985+0000","last_fullsized":"2024-07-04T17:34:56.248985+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:08:18.792180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.238582+0000","last_change":"2024-07-04T17:34:55.274895+0000","last_active":"2024-07-04T17:34:56.238582+0000","last_peered":"2024-07-04T17:34:56.238582+0000","last_clean":"2024-07-04T17:34:56.238582+0000","last_became_active":"2024-07-04T17:34:55.274568+0000","last_became_peered":"2024-07-04T17:34:55.274568+0000","last_unstale":"2024-07-04T17:34:56.238582+0000","last_undegraded":"2024-07-04T17:34:56.238582+0000","last_fullsized":"2024-07-04T17:34:56.238582+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:16:51.380937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582501+0000","last_change":"2024-07-04T17:34:35.533907+0000","last_active":"2024-07-04T17:34:58.582501+0000","last_peered":"2024-07-04T17:34:58.582501+0000","last_clean":"2024-07-04T17:34:58.582501+0000","last_became_active":"2024-07-04T17:34:35.533596+0000","last_became_peered":"2024-07-04T17:34:35.533596+0000","last_unstale":"2024-07-04T17:34:58.582501+0000","last_undegraded":"2024-07-04T17:34:58.582501+0000","last_fullsized":"2024-07-04T17:34:58.582501+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:41:30.147907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.248085+0000","last_change":"2024-07-04T17:34:55.239623+0000","last_active":"2024-07-04T17:34:56.248085+0000","last_peered":"2024-07-04T17:34:56.248085+0000","last_clean":"2024-07-04T17:34:56.248085+0000","last_became_active":"2024-07-04T17:34:55.239022+0000","last_became_peered":"2024-07-04T17:34:55.239022+0000","last_unstale":"2024-07-04T17:34:56.248085+0000","last_undegraded":"2024-07-04T17:34:56.248085+0000","last_fullsized":"2024-07-04T17:34:56.248085+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:15:15.683297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.246912+0000","last_change":"2024-07-04T17:34:35.537742+0000","last_active":"2024-07-04T17:34:56.246912+0000","last_peered":"2024-07-04T17:34:56.246912+0000","last_clean":"2024-07-04T17:34:56.246912+0000","last_became_active":"2024-07-04T17:34:35.533422+0000","last_became_peered":"2024-07-04T17:34:35.533422+0000","last_unstale":"2024-07-04T17:34:56.246912+0000","last_undegraded":"2024-07-04T17:34:56.246912+0000","last_fullsized":"2024-07-04T17:34:56.246912+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:02:05.629708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580418+0000","last_change":"2024-07-04T17:34:55.277168+0000","last_active":"2024-07-04T17:34:58.580418+0000","last_peered":"2024-07-04T17:34:58.580418+0000","last_clean":"2024-07-04T17:34:58.580418+0000","last_became_active":"2024-07-04T17:34:55.276394+0000","last_became_peered":"2024-07-04T17:34:55.276394+0000","last_unstale":"2024-07-04T17:34:58.580418+0000","last_undegraded":"2024-07-04T17:34:58.580418+0000","last_fullsized":"2024-07-04T17:34:58.580418+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:41:59.736262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.238463+0000","last_change":"2024-07-04T17:34:35.553505+0000","last_active":"2024-07-04T17:34:56.238463+0000","last_peered":"2024-07-04T17:34:56.238463+0000","last_clean":"2024-07-04T17:34:56.238463+0000","last_became_active":"2024-07-04T17:34:35.553230+0000","last_became_peered":"2024-07-04T17:34:35.553230+0000","last_unstale":"2024-07-04T17:34:56.238463+0000","last_undegraded":"2024-07-04T17:34:56.238463+0000","last_fullsized":"2024-07-04T17:34:56.238463+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:49:38.920935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580315+0000","last_change":"2024-07-04T17:34:55.265408+0000","last_active":"2024-07-04T17:34:58.580315+0000","last_peered":"2024-07-04T17:34:58.580315+0000","last_clean":"2024-07-04T17:34:58.580315+0000","last_became_active":"2024-07-04T17:34:55.264653+0000","last_became_peered":"2024-07-04T17:34:55.264653+0000","last_unstale":"2024-07-04T17:34:58.580315+0000","last_undegraded":"2024-07-04T17:34:58.580315+0000","last_fullsized":"2024-07-04T17:34:58.580315+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:22:51.459466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580656+0000","last_change":"2024-07-04T17:34:35.524780+0000","last_active":"2024-07-04T17:34:58.580656+0000","last_peered":"2024-07-04T17:34:58.580656+0000","last_clean":"2024-07-04T17:34:58.580656+0000","last_became_active":"2024-07-04T17:34:35.524418+0000","last_became_peered":"2024-07-04T17:34:35.524418+0000","last_unstale":"2024-07-04T17:34:58.580656+0000","last_undegraded":"2024-07-04T17:34:58.580656+0000","last_fullsized":"2024-07-04T17:34:58.580656+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:16:31.650779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.579194+0000","last_change":"2024-07-04T17:34:55.286892+0000","last_active":"2024-07-04T17:34:58.579194+0000","last_peered":"2024-07-04T17:34:58.579194+0000","last_clean":"2024-07-04T17:34:58.579194+0000","last_became_active":"2024-07-04T17:34:55.285216+0000","last_became_peered":"2024-07-04T17:34:55.285216+0000","last_unstale":"2024-07-04T17:34:58.579194+0000","last_undegraded":"2024-07-04T17:34:58.579194+0000","last_fullsized":"2024-07-04T17:34:58.579194+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:51:03.831100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580803+0000","last_change":"2024-07-04T17:34:35.522942+0000","last_active":"2024-07-04T17:34:58.580803+0000","last_peered":"2024-07-04T17:34:58.580803+0000","last_clean":"2024-07-04T17:34:58.580803+0000","last_became_active":"2024-07-04T17:34:35.522210+0000","last_became_peered":"2024-07-04T17:34:35.522210+0000","last_unstale":"2024-07-04T17:34:58.580803+0000","last_undegraded":"2024-07-04T17:34:58.580803+0000","last_fullsized":"2024-07-04T17:34:58.580803+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:03:46.125136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.235849+0000","last_change":"2024-07-04T17:34:55.272255+0000","last_active":"2024-07-04T17:34:56.235849+0000","last_peered":"2024-07-04T17:34:56.235849+0000","last_clean":"2024-07-04T17:34:56.235849+0000","last_became_active":"2024-07-04T17:34:55.270558+0000","last_became_peered":"2024-07-04T17:34:55.270558+0000","last_unstale":"2024-07-04T17:34:56.235849+0000","last_undegraded":"2024-07-04T17:34:56.235849+0000","last_fullsized":"2024-07-04T17:34:56.235849+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:04:06.751483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.244843+0000","last_change":"2024-07-04T17:34:35.544485+0000","last_active":"2024-07-04T17:34:56.244843+0000","last_peered":"2024-07-04T17:34:56.244843+0000","last_clean":"2024-07-04T17:34:56.244843+0000","last_became_active":"2024-07-04T17:34:35.539252+0000","last_became_peered":"2024-07-04T17:34:35.539252+0000","last_unstale":"2024-07-04T17:34:56.244843+0000","last_undegraded":"2024-07-04T17:34:56.244843+0000","last_fullsized":"2024-07-04T17:34:56.244843+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:58:20.122138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.243351+0000","last_change":"2024-07-04T17:34:55.254639+0000","last_active":"2024-07-04T17:34:56.243351+0000","last_peered":"2024-07-04T17:34:56.243351+0000","last_clean":"2024-07-04T17:34:56.243351+0000","last_became_active":"2024-07-04T17:34:55.252455+0000","last_became_peered":"2024-07-04T17:34:55.252455+0000","last_unstale":"2024-07-04T17:34:56.243351+0000","last_undegraded":"2024-07-04T17:34:56.243351+0000","last_fullsized":"2024-07-04T17:34:56.243351+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:36:12.409147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.235635+0000","last_change":"2024-07-04T17:34:35.532311+0000","last_active":"2024-07-04T17:34:56.235635+0000","last_peered":"2024-07-04T17:34:56.235635+0000","last_clean":"2024-07-04T17:34:56.235635+0000","last_became_active":"2024-07-04T17:34:35.531704+0000","last_became_peered":"2024-07-04T17:34:35.531704+0000","last_unstale":"2024-07-04T17:34:56.235635+0000","last_undegraded":"2024-07-04T17:34:56.235635+0000","last_fullsized":"2024-07-04T17:34:56.235635+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:38:06.359176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580208+0000","last_change":"2024-07-04T17:34:55.273539+0000","last_active":"2024-07-04T17:34:58.580208+0000","last_peered":"2024-07-04T17:34:58.580208+0000","last_clean":"2024-07-04T17:34:58.580208+0000","last_became_active":"2024-07-04T17:34:55.272611+0000","last_became_peered":"2024-07-04T17:34:55.272611+0000","last_unstale":"2024-07-04T17:34:58.580208+0000","last_undegraded":"2024-07-04T17:34:58.580208+0000","last_fullsized":"2024-07-04T17:34:58.580208+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:49:52.552101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.238206+0000","last_change":"2024-07-04T17:34:54.267723+0000","last_active":"2024-07-04T17:34:56.238206+0000","last_peered":"2024-07-04T17:34:56.238206+0000","last_clean":"2024-07-04T17:34:56.238206+0000","last_became_active":"2024-07-04T17:34:54.267352+0000","last_became_peered":"2024-07-04T17:34:54.267352+0000","last_unstale":"2024-07-04T17:34:56.238206+0000","last_undegraded":"2024-07-04T17:34:56.238206+0000","last_fullsized":"2024-07-04T17:34:56.238206+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:12:50.370995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.247361+0000","last_change":"2024-07-04T17:34:35.536755+0000","last_active":"2024-07-04T17:34:56.247361+0000","last_peered":"2024-07-04T17:34:56.247361+0000","last_clean":"2024-07-04T17:34:56.247361+0000","last_became_active":"2024-07-04T17:34:35.535641+0000","last_became_peered":"2024-07-04T17:34:35.535641+0000","last_unstale":"2024-07-04T17:34:56.247361+0000","last_undegraded":"2024-07-04T17:34:56.247361+0000","last_fullsized":"2024-07-04T17:34:56.247361+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:00:03.434775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1f","version":"135'4","reported_seq":26,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.241514+0000","last_change":"2024-07-04T17:34:34.526194+0000","last_active":"2024-07-04T17:34:56.241514+0000","last_peered":"2024-07-04T17:34:56.241514+0000","last_clean":"2024-07-04T17:34:56.241514+0000","last_became_active":"2024-07-04T17:34:34.525749+0000","last_became_peered":"2024-07-04T17:34:34.525749+0000","last_unstale":"2024-07-04T17:34:56.241514+0000","last_undegraded":"2024-07-04T17:34:56.241514+0000","last_fullsized":"2024-07-04T17:34:56.241514+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:15:41.361322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.245620+0000","last_change":"2024-07-04T17:34:55.263366+0000","last_active":"2024-07-04T17:34:56.245620+0000","last_peered":"2024-07-04T17:34:56.245620+0000","last_clean":"2024-07-04T17:34:56.245620+0000","last_became_active":"2024-07-04T17:34:55.263036+0000","last_became_peered":"2024-07-04T17:34:55.263036+0000","last_unstale":"2024-07-04T17:34:56.245620+0000","last_undegraded":"2024-07-04T17:34:56.245620+0000","last_fullsized":"2024-07-04T17:34:56.245620+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:16:51.380937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1c","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.244471+0000","last_change":"2024-07-04T17:34:54.257931+0000","last_active":"2024-07-04T17:34:56.244471+0000","last_peered":"2024-07-04T17:34:56.244471+0000","last_clean":"2024-07-04T17:34:56.244471+0000","last_became_active":"2024-07-04T17:34:54.255905+0000","last_became_peered":"2024-07-04T17:34:54.255905+0000","last_unstale":"2024-07-04T17:34:56.244471+0000","last_undegraded":"2024-07-04T17:34:56.244471+0000","last_fullsized":"2024-07-04T17:34:56.244471+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:06:21.435358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582588+0000","last_change":"2024-07-04T17:34:35.548372+0000","last_active":"2024-07-04T17:34:58.582588+0000","last_peered":"2024-07-04T17:34:58.582588+0000","last_clean":"2024-07-04T17:34:58.582588+0000","last_became_active":"2024-07-04T17:34:35.547829+0000","last_became_peered":"2024-07-04T17:34:35.547829+0000","last_unstale":"2024-07-04T17:34:58.582588+0000","last_undegraded":"2024-07-04T17:34:58.582588+0000","last_fullsized":"2024-07-04T17:34:58.582588+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:37:49.541089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"135'1","reported_seq":23,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.577793+0000","last_change":"2024-07-04T17:34:34.505687+0000","last_active":"2024-07-04T17:34:58.577793+0000","last_peered":"2024-07-04T17:34:58.577793+0000","last_clean":"2024-07-04T17:34:58.577793+0000","last_became_active":"2024-07-04T17:34:34.505337+0000","last_became_peered":"2024-07-04T17:34:34.505337+0000","last_unstale":"2024-07-04T17:34:58.577793+0000","last_undegraded":"2024-07-04T17:34:58.577793+0000","last_fullsized":"2024-07-04T17:34:58.577793+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:57:27.037624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582563+0000","last_change":"2024-07-04T17:34:55.276891+0000","last_active":"2024-07-04T17:34:58.582563+0000","last_peered":"2024-07-04T17:34:58.582563+0000","last_clean":"2024-07-04T17:34:58.582563+0000","last_became_active":"2024-07-04T17:34:55.276381+0000","last_became_peered":"2024-07-04T17:34:55.276381+0000","last_unstale":"2024-07-04T17:34:58.582563+0000","last_undegraded":"2024-07-04T17:34:58.582563+0000","last_fullsized":"2024-07-04T17:34:58.582563+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:33:25.267671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1f","version":"139'4","reported_seq":18,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580901+0000","last_change":"2024-07-04T17:34:54.261384+0000","last_active":"2024-07-04T17:34:58.580901+0000","last_peered":"2024-07-04T17:34:58.580901+0000","last_clean":"2024-07-04T17:34:58.580901+0000","last_became_active":"2024-07-04T17:34:54.260019+0000","last_became_peered":"2024-07-04T17:34:54.260019+0000","last_unstale":"2024-07-04T17:34:58.580901+0000","last_undegraded":"2024-07-04T17:34:58.580901+0000","last_fullsized":"2024-07-04T17:34:58.580901+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:30:17.511938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.244945+0000","last_change":"2024-07-04T17:34:35.531247+0000","last_active":"2024-07-04T17:34:56.244945+0000","last_peered":"2024-07-04T17:34:56.244945+0000","last_clean":"2024-07-04T17:34:56.244945+0000","last_became_active":"2024-07-04T17:34:35.530846+0000","last_became_peered":"2024-07-04T17:34:35.530846+0000","last_unstale":"2024-07-04T17:34:56.244945+0000","last_undegraded":"2024-07-04T17:34:56.244945+0000","last_fullsized":"2024-07-04T17:34:56.244945+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:30:51.180015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1d","version":"135'2","reported_seq":24,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.247392+0000","last_change":"2024-07-04T17:34:34.512001+0000","last_active":"2024-07-04T17:34:56.247392+0000","last_peered":"2024-07-04T17:34:56.247392+0000","last_clean":"2024-07-04T17:34:56.247392+0000","last_became_active":"2024-07-04T17:34:34.511480+0000","last_became_peered":"2024-07-04T17:34:34.511480+0000","last_unstale":"2024-07-04T17:34:56.247392+0000","last_undegraded":"2024-07-04T17:34:56.247392+0000","last_fullsized":"2024-07-04T17:34:56.247392+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:46:03.790681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.249372+0000","last_change":"2024-07-04T17:34:55.239365+0000","last_active":"2024-07-04T17:34:56.249372+0000","last_peered":"2024-07-04T17:34:56.249372+0000","last_clean":"2024-07-04T17:34:56.249372+0000","last_became_active":"2024-07-04T17:34:55.237209+0000","last_became_peered":"2024-07-04T17:34:55.237209+0000","last_unstale":"2024-07-04T17:34:56.249372+0000","last_undegraded":"2024-07-04T17:34:56.249372+0000","last_fullsized":"2024-07-04T17:34:56.249372+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:41:49.878066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1e","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582123+0000","last_change":"2024-07-04T17:34:54.261119+0000","last_active":"2024-07-04T17:34:58.582123+0000","last_peered":"2024-07-04T17:34:58.582123+0000","last_clean":"2024-07-04T17:34:58.582123+0000","last_became_active":"2024-07-04T17:34:54.259804+0000","last_became_peered":"2024-07-04T17:34:54.259804+0000","last_unstale":"2024-07-04T17:34:58.582123+0000","last_undegraded":"2024-07-04T17:34:58.582123+0000","last_fullsized":"2024-07-04T17:34:58.582123+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:01:32.268460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.247098+0000","last_change":"2024-07-04T17:34:35.546422+0000","last_active":"2024-07-04T17:34:56.247098+0000","last_peered":"2024-07-04T17:34:56.247098+0000","last_clean":"2024-07-04T17:34:56.247098+0000","last_became_active":"2024-07-04T17:34:35.546061+0000","last_became_peered":"2024-07-04T17:34:35.546061+0000","last_unstale":"2024-07-04T17:34:56.247098+0000","last_undegraded":"2024-07-04T17:34:56.247098+0000","last_fullsized":"2024-07-04T17:34:56.247098+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:50:00.540847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1c","version":"135'2","reported_seq":24,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.577632+0000","last_change":"2024-07-04T17:34:34.506897+0000","last_active":"2024-07-04T17:34:58.577632+0000","last_peered":"2024-07-04T17:34:58.577632+0000","last_clean":"2024-07-04T17:34:58.577632+0000","last_became_active":"2024-07-04T17:34:34.506023+0000","last_became_peered":"2024-07-04T17:34:34.506023+0000","last_unstale":"2024-07-04T17:34:58.577632+0000","last_undegraded":"2024-07-04T17:34:58.577632+0000","last_fullsized":"2024-07-04T17:34:58.577632+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:36:57.170806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.242029+0000","last_change":"2024-07-04T17:34:55.255731+0000","last_active":"2024-07-04T17:34:56.242029+0000","last_peered":"2024-07-04T17:34:56.242029+0000","last_clean":"2024-07-04T17:34:56.242029+0000","last_became_active":"2024-07-04T17:34:55.255430+0000","last_became_peered":"2024-07-04T17:34:55.255430+0000","last_unstale":"2024-07-04T17:34:56.242029+0000","last_undegraded":"2024-07-04T17:34:56.242029+0000","last_fullsized":"2024-07-04T17:34:56.242029+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:55:44.449626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.242983+0000","last_change":"2024-07-04T17:34:54.264471+0000","last_active":"2024-07-04T17:34:56.242983+0000","last_peered":"2024-07-04T17:34:56.242983+0000","last_clean":"2024-07-04T17:34:56.242983+0000","last_became_active":"2024-07-04T17:34:54.263599+0000","last_became_peered":"2024-07-04T17:34:54.263599+0000","last_unstale":"2024-07-04T17:34:56.242983+0000","last_undegraded":"2024-07-04T17:34:56.242983+0000","last_fullsized":"2024-07-04T17:34:56.242983+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:08:30.179545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580809+0000","last_change":"2024-07-04T17:34:35.515987+0000","last_active":"2024-07-04T17:34:58.580809+0000","last_peered":"2024-07-04T17:34:58.580809+0000","last_clean":"2024-07-04T17:34:58.580809+0000","last_became_active":"2024-07-04T17:34:35.515158+0000","last_became_peered":"2024-07-04T17:34:35.515158+0000","last_unstale":"2024-07-04T17:34:58.580809+0000","last_undegraded":"2024-07-04T17:34:58.580809+0000","last_fullsized":"2024-07-04T17:34:58.580809+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:39:44.537119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"135'1","reported_seq":23,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.238878+0000","last_change":"2024-07-04T17:34:34.526306+0000","last_active":"2024-07-04T17:34:56.238878+0000","last_peered":"2024-07-04T17:34:56.238878+0000","last_clean":"2024-07-04T17:34:56.238878+0000","last_became_active":"2024-07-04T17:34:34.525936+0000","last_became_peered":"2024-07-04T17:34:34.525936+0000","last_unstale":"2024-07-04T17:34:56.238878+0000","last_undegraded":"2024-07-04T17:34:56.238878+0000","last_fullsized":"2024-07-04T17:34:56.238878+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:58:19.109679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.234302+0000","last_change":"2024-07-04T17:34:55.280775+0000","last_active":"2024-07-04T17:34:56.234302+0000","last_peered":"2024-07-04T17:34:56.234302+0000","last_clean":"2024-07-04T17:34:56.234302+0000","last_became_active":"2024-07-04T17:34:55.280449+0000","last_became_peered":"2024-07-04T17:34:55.280449+0000","last_unstale":"2024-07-04T17:34:56.234302+0000","last_undegraded":"2024-07-04T17:34:56.234302+0000","last_fullsized":"2024-07-04T17:34:56.234302+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:36:12.409147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.242124+0000","last_change":"2024-07-04T17:34:54.264992+0000","last_active":"2024-07-04T17:34:56.242124+0000","last_peered":"2024-07-04T17:34:56.242124+0000","last_clean":"2024-07-04T17:34:56.242124+0000","last_became_active":"2024-07-04T17:34:54.264254+0000","last_became_peered":"2024-07-04T17:34:54.264254+0000","last_unstale":"2024-07-04T17:34:56.242124+0000","last_undegraded":"2024-07-04T17:34:56.242124+0000","last_fullsized":"2024-07-04T17:34:56.242124+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:21:20.115129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.237494+0000","last_change":"2024-07-04T17:34:35.529792+0000","last_active":"2024-07-04T17:34:56.237494+0000","last_peered":"2024-07-04T17:34:56.237494+0000","last_clean":"2024-07-04T17:34:56.237494+0000","last_became_active":"2024-07-04T17:34:35.529159+0000","last_became_peered":"2024-07-04T17:34:35.529159+0000","last_unstale":"2024-07-04T17:34:56.237494+0000","last_undegraded":"2024-07-04T17:34:56.237494+0000","last_fullsized":"2024-07-04T17:34:56.237494+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:19:05.515444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.1a","version":"135'1","reported_seq":23,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.236666+0000","last_change":"2024-07-04T17:34:34.520915+0000","last_active":"2024-07-04T17:34:56.236666+0000","last_peered":"2024-07-04T17:34:56.236666+0000","last_clean":"2024-07-04T17:34:56.236666+0000","last_became_active":"2024-07-04T17:34:34.520164+0000","last_became_peered":"2024-07-04T17:34:34.520164+0000","last_unstale":"2024-07-04T17:34:56.236666+0000","last_undegraded":"2024-07-04T17:34:56.236666+0000","last_fullsized":"2024-07-04T17:34:56.236666+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:15:07.166351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.243630+0000","last_change":"2024-07-04T17:34:55.262931+0000","last_active":"2024-07-04T17:34:56.243630+0000","last_peered":"2024-07-04T17:34:56.243630+0000","last_clean":"2024-07-04T17:34:56.243630+0000","last_became_active":"2024-07-04T17:34:55.262623+0000","last_became_peered":"2024-07-04T17:34:55.262623+0000","last_unstale":"2024-07-04T17:34:56.243630+0000","last_undegraded":"2024-07-04T17:34:56.243630+0000","last_fullsized":"2024-07-04T17:34:56.243630+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:05:12.120118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1a","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.243597+0000","last_change":"2024-07-04T17:34:54.254747+0000","last_active":"2024-07-04T17:34:56.243597+0000","last_peered":"2024-07-04T17:34:56.243597+0000","last_clean":"2024-07-04T17:34:56.243597+0000","last_became_active":"2024-07-04T17:34:54.254412+0000","last_became_peered":"2024-07-04T17:34:54.254412+0000","last_unstale":"2024-07-04T17:34:56.243597+0000","last_undegraded":"2024-07-04T17:34:56.243597+0000","last_fullsized":"2024-07-04T17:34:56.243597+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:22:35.350759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.235418+0000","last_change":"2024-07-04T17:34:35.528571+0000","last_active":"2024-07-04T17:34:56.235418+0000","last_peered":"2024-07-04T17:34:56.235418+0000","last_clean":"2024-07-04T17:34:56.235418+0000","last_became_active":"2024-07-04T17:34:35.528217+0000","last_became_peered":"2024-07-04T17:34:35.528217+0000","last_unstale":"2024-07-04T17:34:56.235418+0000","last_undegraded":"2024-07-04T17:34:56.235418+0000","last_fullsized":"2024-07-04T17:34:56.235418+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:25:31.311940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.235826+0000","last_change":"2024-07-04T17:34:34.520226+0000","last_active":"2024-07-04T17:34:56.235826+0000","last_peered":"2024-07-04T17:34:56.235826+0000","last_clean":"2024-07-04T17:34:56.235826+0000","last_became_active":"2024-07-04T17:34:34.519880+0000","last_became_peered":"2024-07-04T17:34:34.519880+0000","last_unstale":"2024-07-04T17:34:56.235826+0000","last_undegraded":"2024-07-04T17:34:56.235826+0000","last_fullsized":"2024-07-04T17:34:56.235826+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:30:01.808525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.237006+0000","last_change":"2024-07-04T17:34:55.284186+0000","last_active":"2024-07-04T17:34:56.237006+0000","last_peered":"2024-07-04T17:34:56.237006+0000","last_clean":"2024-07-04T17:34:56.237006+0000","last_became_active":"2024-07-04T17:34:55.283840+0000","last_became_peered":"2024-07-04T17:34:55.283840+0000","last_unstale":"2024-07-04T17:34:56.237006+0000","last_undegraded":"2024-07-04T17:34:56.237006+0000","last_fullsized":"2024-07-04T17:34:56.237006+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:04:05.855295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.15","version":"139'3","reported_seq":17,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580863+0000","last_change":"2024-07-04T17:34:54.251183+0000","last_active":"2024-07-04T17:34:58.580863+0000","last_peered":"2024-07-04T17:34:58.580863+0000","last_clean":"2024-07-04T17:34:58.580863+0000","last_became_active":"2024-07-04T17:34:54.250831+0000","last_became_peered":"2024-07-04T17:34:54.250831+0000","last_unstale":"2024-07-04T17:34:58.580863+0000","last_undegraded":"2024-07-04T17:34:58.580863+0000","last_fullsized":"2024-07-04T17:34:58.580863+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:31:23.802489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.247488+0000","last_change":"2024-07-04T17:34:35.541668+0000","last_active":"2024-07-04T17:34:56.247488+0000","last_peered":"2024-07-04T17:34:56.247488+0000","last_clean":"2024-07-04T17:34:56.247488+0000","last_became_active":"2024-07-04T17:34:35.537142+0000","last_became_peered":"2024-07-04T17:34:35.537142+0000","last_unstale":"2024-07-04T17:34:56.247488+0000","last_undegraded":"2024-07-04T17:34:56.247488+0000","last_fullsized":"2024-07-04T17:34:56.247488+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:56:41.649364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.249541+0000","last_change":"2024-07-04T17:34:34.516054+0000","last_active":"2024-07-04T17:34:56.249541+0000","last_peered":"2024-07-04T17:34:56.249541+0000","last_clean":"2024-07-04T17:34:56.249541+0000","last_became_active":"2024-07-04T17:34:34.514177+0000","last_became_peered":"2024-07-04T17:34:34.514177+0000","last_unstale":"2024-07-04T17:34:56.249541+0000","last_undegraded":"2024-07-04T17:34:56.249541+0000","last_fullsized":"2024-07-04T17:34:56.249541+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:18:46.789057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580742+0000","last_change":"2024-07-04T17:34:55.277295+0000","last_active":"2024-07-04T17:34:58.580742+0000","last_peered":"2024-07-04T17:34:58.580742+0000","last_clean":"2024-07-04T17:34:58.580742+0000","last_became_active":"2024-07-04T17:34:55.275102+0000","last_became_peered":"2024-07-04T17:34:55.275102+0000","last_unstale":"2024-07-04T17:34:58.580742+0000","last_undegraded":"2024-07-04T17:34:58.580742+0000","last_fullsized":"2024-07-04T17:34:58.580742+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:09:53.591198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"139'6","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.237116+0000","last_change":"2024-07-04T17:34:54.270201+0000","last_active":"2024-07-04T17:34:56.237116+0000","last_peered":"2024-07-04T17:34:56.237116+0000","last_clean":"2024-07-04T17:34:56.237116+0000","last_became_active":"2024-07-04T17:34:54.269515+0000","last_became_peered":"2024-07-04T17:34:54.269515+0000","last_unstale":"2024-07-04T17:34:56.237116+0000","last_undegraded":"2024-07-04T17:34:56.237116+0000","last_fullsized":"2024-07-04T17:34:56.237116+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:22:12.441138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.249356+0000","last_change":"2024-07-04T17:34:35.546010+0000","last_active":"2024-07-04T17:34:56.249356+0000","last_peered":"2024-07-04T17:34:56.249356+0000","last_clean":"2024-07-04T17:34:56.249356+0000","last_became_active":"2024-07-04T17:34:35.536818+0000","last_became_peered":"2024-07-04T17:34:35.536818+0000","last_unstale":"2024-07-04T17:34:56.249356+0000","last_undegraded":"2024-07-04T17:34:56.249356+0000","last_fullsized":"2024-07-04T17:34:56.249356+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:24:20.145687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.241512+0000","last_change":"2024-07-04T17:34:34.526851+0000","last_active":"2024-07-04T17:34:56.241512+0000","last_peered":"2024-07-04T17:34:56.241512+0000","last_clean":"2024-07-04T17:34:56.241512+0000","last_became_active":"2024-07-04T17:34:34.526514+0000","last_became_peered":"2024-07-04T17:34:34.526514+0000","last_unstale":"2024-07-04T17:34:56.241512+0000","last_undegraded":"2024-07-04T17:34:56.241512+0000","last_fullsized":"2024-07-04T17:34:56.241512+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:37:45.915030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.247127+0000","last_change":"2024-07-04T17:34:55.238745+0000","last_active":"2024-07-04T17:34:56.247127+0000","last_peered":"2024-07-04T17:34:56.247127+0000","last_clean":"2024-07-04T17:34:56.247127+0000","last_became_active":"2024-07-04T17:34:55.238434+0000","last_became_peered":"2024-07-04T17:34:55.238434+0000","last_unstale":"2024-07-04T17:34:56.247127+0000","last_undegraded":"2024-07-04T17:34:56.247127+0000","last_fullsized":"2024-07-04T17:34:56.247127+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:54:55.103581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.248259+0000","last_change":"2024-07-04T17:34:54.257535+0000","last_active":"2024-07-04T17:34:56.248259+0000","last_peered":"2024-07-04T17:34:56.248259+0000","last_clean":"2024-07-04T17:34:56.248259+0000","last_became_active":"2024-07-04T17:34:54.256273+0000","last_became_peered":"2024-07-04T17:34:54.256273+0000","last_unstale":"2024-07-04T17:34:56.248259+0000","last_undegraded":"2024-07-04T17:34:56.248259+0000","last_fullsized":"2024-07-04T17:34:56.248259+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:36:46.193844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582076+0000","last_change":"2024-07-04T17:34:35.524150+0000","last_active":"2024-07-04T17:34:58.582076+0000","last_peered":"2024-07-04T17:34:58.582076+0000","last_clean":"2024-07-04T17:34:58.582076+0000","last_became_active":"2024-07-04T17:34:35.523835+0000","last_became_peered":"2024-07-04T17:34:35.523835+0000","last_unstale":"2024-07-04T17:34:58.582076+0000","last_undegraded":"2024-07-04T17:34:58.582076+0000","last_fullsized":"2024-07-04T17:34:58.582076+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:14:55.953139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.15","version":"135'3","reported_seq":25,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.239494+0000","last_change":"2024-07-04T17:34:34.525906+0000","last_active":"2024-07-04T17:34:56.239494+0000","last_peered":"2024-07-04T17:34:56.239494+0000","last_clean":"2024-07-04T17:34:56.239494+0000","last_became_active":"2024-07-04T17:34:34.525545+0000","last_became_peered":"2024-07-04T17:34:34.525545+0000","last_unstale":"2024-07-04T17:34:56.239494+0000","last_undegraded":"2024-07-04T17:34:56.239494+0000","last_fullsized":"2024-07-04T17:34:56.239494+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:15:15.110008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.239796+0000","last_change":"2024-07-04T17:34:55.274885+0000","last_active":"2024-07-04T17:34:56.239796+0000","last_peered":"2024-07-04T17:34:56.239796+0000","last_clean":"2024-07-04T17:34:56.239796+0000","last_became_active":"2024-07-04T17:34:55.273954+0000","last_became_peered":"2024-07-04T17:34:55.273954+0000","last_unstale":"2024-07-04T17:34:56.239796+0000","last_undegraded":"2024-07-04T17:34:56.239796+0000","last_fullsized":"2024-07-04T17:34:56.239796+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:15:15.683297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.245888+0000","last_change":"2024-07-04T17:34:54.257704+0000","last_active":"2024-07-04T17:34:56.245888+0000","last_peered":"2024-07-04T17:34:56.245888+0000","last_clean":"2024-07-04T17:34:56.245888+0000","last_became_active":"2024-07-04T17:34:54.257376+0000","last_became_peered":"2024-07-04T17:34:54.257376+0000","last_unstale":"2024-07-04T17:34:56.245888+0000","last_undegraded":"2024-07-04T17:34:56.245888+0000","last_fullsized":"2024-07-04T17:34:56.245888+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:59:05.459068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582464+0000","last_change":"2024-07-04T17:34:35.523356+0000","last_active":"2024-07-04T17:34:58.582464+0000","last_peered":"2024-07-04T17:34:58.582464+0000","last_clean":"2024-07-04T17:34:58.582464+0000","last_became_active":"2024-07-04T17:34:35.522584+0000","last_became_peered":"2024-07-04T17:34:35.522584+0000","last_unstale":"2024-07-04T17:34:58.582464+0000","last_undegraded":"2024-07-04T17:34:58.582464+0000","last_fullsized":"2024-07-04T17:34:58.582464+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:34:41.701054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"136'8","reported_seq":30,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.245341+0000","last_change":"2024-07-04T17:34:34.513760+0000","last_active":"2024-07-04T17:34:56.245341+0000","last_peered":"2024-07-04T17:34:56.245341+0000","last_clean":"2024-07-04T17:34:56.245341+0000","last_became_active":"2024-07-04T17:34:34.512609+0000","last_became_peered":"2024-07-04T17:34:34.512609+0000","last_unstale":"2024-07-04T17:34:56.245341+0000","last_undegraded":"2024-07-04T17:34:56.245341+0000","last_fullsized":"2024-07-04T17:34:56.245341+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:08:12.565423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3309,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.234441+0000","last_change":"2024-07-04T17:34:55.280804+0000","last_active":"2024-07-04T17:34:56.234441+0000","last_peered":"2024-07-04T17:34:56.234441+0000","last_clean":"2024-07-04T17:34:56.234441+0000","last_became_active":"2024-07-04T17:34:55.280549+0000","last_became_peered":"2024-07-04T17:34:55.280549+0000","last_unstale":"2024-07-04T17:34:56.234441+0000","last_undegraded":"2024-07-04T17:34:56.234441+0000","last_fullsized":"2024-07-04T17:34:56.234441+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:36:40.598708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.234453+0000","last_change":"2024-07-04T17:34:54.263803+0000","last_active":"2024-07-04T17:34:56.234453+0000","last_peered":"2024-07-04T17:34:56.234453+0000","last_clean":"2024-07-04T17:34:56.234453+0000","last_became_active":"2024-07-04T17:34:54.263497+0000","last_became_peered":"2024-07-04T17:34:54.263497+0000","last_unstale":"2024-07-04T17:34:56.234453+0000","last_undegraded":"2024-07-04T17:34:56.234453+0000","last_fullsized":"2024-07-04T17:34:56.234453+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:02:47.521123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.244176+0000","last_change":"2024-07-04T17:34:35.537826+0000","last_active":"2024-07-04T17:34:56.244176+0000","last_peered":"2024-07-04T17:34:56.244176+0000","last_clean":"2024-07-04T17:34:56.244176+0000","last_became_active":"2024-07-04T17:34:35.535781+0000","last_became_peered":"2024-07-04T17:34:35.535781+0000","last_unstale":"2024-07-04T17:34:56.244176+0000","last_undegraded":"2024-07-04T17:34:56.244176+0000","last_fullsized":"2024-07-04T17:34:56.244176+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:26:50.917096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"135'2","reported_seq":24,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.577427+0000","last_change":"2024-07-04T17:34:34.507682+0000","last_active":"2024-07-04T17:34:58.577427+0000","last_peered":"2024-07-04T17:34:58.577427+0000","last_clean":"2024-07-04T17:34:58.577427+0000","last_became_active":"2024-07-04T17:34:34.507353+0000","last_became_peered":"2024-07-04T17:34:34.507353+0000","last_unstale":"2024-07-04T17:34:58.577427+0000","last_undegraded":"2024-07-04T17:34:58.577427+0000","last_fullsized":"2024-07-04T17:34:58.577427+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:47:18.975316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.578931+0000","last_change":"2024-07-04T17:34:55.240381+0000","last_active":"2024-07-04T17:34:58.578931+0000","last_peered":"2024-07-04T17:34:58.578931+0000","last_clean":"2024-07-04T17:34:58.578931+0000","last_became_active":"2024-07-04T17:34:55.239891+0000","last_became_peered":"2024-07-04T17:34:55.239891+0000","last_unstale":"2024-07-04T17:34:58.578931+0000","last_undegraded":"2024-07-04T17:34:58.578931+0000","last_fullsized":"2024-07-04T17:34:58.578931+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:08:19.765528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"139'8","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.234731+0000","last_change":"2024-07-04T17:34:54.264205+0000","last_active":"2024-07-04T17:34:56.234731+0000","last_peered":"2024-07-04T17:34:56.234731+0000","last_clean":"2024-07-04T17:34:56.234731+0000","last_became_active":"2024-07-04T17:34:54.263909+0000","last_became_peered":"2024-07-04T17:34:54.263909+0000","last_unstale":"2024-07-04T17:34:56.234731+0000","last_undegraded":"2024-07-04T17:34:56.234731+0000","last_fullsized":"2024-07-04T17:34:56.234731+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:03:24.700218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.238004+0000","last_change":"2024-07-04T17:34:35.529515+0000","last_active":"2024-07-04T17:34:56.238004+0000","last_peered":"2024-07-04T17:34:56.238004+0000","last_clean":"2024-07-04T17:34:56.238004+0000","last_became_active":"2024-07-04T17:34:35.528532+0000","last_became_peered":"2024-07-04T17:34:35.528532+0000","last_unstale":"2024-07-04T17:34:56.238004+0000","last_undegraded":"2024-07-04T17:34:56.238004+0000","last_fullsized":"2024-07-04T17:34:56.238004+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:33:17.769730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.577366+0000","last_change":"2024-07-04T17:34:34.512082+0000","last_active":"2024-07-04T17:34:58.577366+0000","last_peered":"2024-07-04T17:34:58.577366+0000","last_clean":"2024-07-04T17:34:58.577366+0000","last_became_active":"2024-07-04T17:34:34.511518+0000","last_became_peered":"2024-07-04T17:34:34.511518+0000","last_unstale":"2024-07-04T17:34:58.577366+0000","last_undegraded":"2024-07-04T17:34:58.577366+0000","last_fullsized":"2024-07-04T17:34:58.577366+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:34:32.553466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.243378+0000","last_change":"2024-07-04T17:34:55.260121+0000","last_active":"2024-07-04T17:34:56.243378+0000","last_peered":"2024-07-04T17:34:56.243378+0000","last_clean":"2024-07-04T17:34:56.243378+0000","last_became_active":"2024-07-04T17:34:55.257916+0000","last_became_peered":"2024-07-04T17:34:55.257916+0000","last_unstale":"2024-07-04T17:34:56.243378+0000","last_undegraded":"2024-07-04T17:34:56.243378+0000","last_fullsized":"2024-07-04T17:34:56.243378+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:36:40.598708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.236136+0000","last_change":"2024-07-04T17:34:54.272879+0000","last_active":"2024-07-04T17:34:56.236136+0000","last_peered":"2024-07-04T17:34:56.236136+0000","last_clean":"2024-07-04T17:34:56.236136+0000","last_became_active":"2024-07-04T17:34:54.272295+0000","last_became_peered":"2024-07-04T17:34:54.272295+0000","last_unstale":"2024-07-04T17:34:56.236136+0000","last_undegraded":"2024-07-04T17:34:56.236136+0000","last_fullsized":"2024-07-04T17:34:56.236136+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:37:16.899961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.578924+0000","last_change":"2024-07-04T17:34:35.539061+0000","last_active":"2024-07-04T17:34:58.578924+0000","last_peered":"2024-07-04T17:34:58.578924+0000","last_clean":"2024-07-04T17:34:58.578924+0000","last_became_active":"2024-07-04T17:34:35.538660+0000","last_became_peered":"2024-07-04T17:34:35.538660+0000","last_unstale":"2024-07-04T17:34:58.578924+0000","last_undegraded":"2024-07-04T17:34:58.578924+0000","last_fullsized":"2024-07-04T17:34:58.578924+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:36:20.868550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.577356+0000","last_change":"2024-07-04T17:34:34.513736+0000","last_active":"2024-07-04T17:34:58.577356+0000","last_peered":"2024-07-04T17:34:58.577356+0000","last_clean":"2024-07-04T17:34:58.577356+0000","last_became_active":"2024-07-04T17:34:34.512396+0000","last_became_peered":"2024-07-04T17:34:34.512396+0000","last_unstale":"2024-07-04T17:34:58.577356+0000","last_undegraded":"2024-07-04T17:34:58.577356+0000","last_fullsized":"2024-07-04T17:34:58.577356+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:25:30.776827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.243001+0000","last_change":"2024-07-04T17:34:55.259648+0000","last_active":"2024-07-04T17:34:56.243001+0000","last_peered":"2024-07-04T17:34:56.243001+0000","last_clean":"2024-07-04T17:34:56.243001+0000","last_became_active":"2024-07-04T17:34:55.256614+0000","last_became_peered":"2024-07-04T17:34:55.256614+0000","last_unstale":"2024-07-04T17:34:56.243001+0000","last_undegraded":"2024-07-04T17:34:56.243001+0000","last_fullsized":"2024-07-04T17:34:56.243001+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:16:21.117253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580678+0000","last_change":"2024-07-04T17:34:54.251365+0000","last_active":"2024-07-04T17:34:58.580678+0000","last_peered":"2024-07-04T17:34:58.580678+0000","last_clean":"2024-07-04T17:34:58.580678+0000","last_became_active":"2024-07-04T17:34:54.250997+0000","last_became_peered":"2024-07-04T17:34:54.250997+0000","last_unstale":"2024-07-04T17:34:58.580678+0000","last_undegraded":"2024-07-04T17:34:58.580678+0000","last_fullsized":"2024-07-04T17:34:58.580678+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:53:35.789378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.579622+0000","last_change":"2024-07-04T17:34:35.540239+0000","last_active":"2024-07-04T17:34:58.579622+0000","last_peered":"2024-07-04T17:34:58.579622+0000","last_clean":"2024-07-04T17:34:58.579622+0000","last_became_active":"2024-07-04T17:34:35.539911+0000","last_became_peered":"2024-07-04T17:34:35.539911+0000","last_unstale":"2024-07-04T17:34:58.579622+0000","last_undegraded":"2024-07-04T17:34:58.579622+0000","last_fullsized":"2024-07-04T17:34:58.579622+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:03:47.021324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"135'8","reported_seq":30,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.236664+0000","last_change":"2024-07-04T17:34:34.526204+0000","last_active":"2024-07-04T17:34:56.236664+0000","last_peered":"2024-07-04T17:34:56.236664+0000","last_clean":"2024-07-04T17:34:56.236664+0000","last_became_active":"2024-07-04T17:34:34.525811+0000","last_became_peered":"2024-07-04T17:34:34.525811+0000","last_unstale":"2024-07-04T17:34:56.236664+0000","last_undegraded":"2024-07-04T17:34:56.236664+0000","last_fullsized":"2024-07-04T17:34:56.236664+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:49:36.910071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.236782+0000","last_change":"2024-07-04T17:34:55.271371+0000","last_active":"2024-07-04T17:34:56.236782+0000","last_peered":"2024-07-04T17:34:56.236782+0000","last_clean":"2024-07-04T17:34:56.236782+0000","last_became_active":"2024-07-04T17:34:55.271036+0000","last_became_peered":"2024-07-04T17:34:55.271036+0000","last_unstale":"2024-07-04T17:34:56.236782+0000","last_undegraded":"2024-07-04T17:34:56.236782+0000","last_fullsized":"2024-07-04T17:34:56.236782+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:49:06.940104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.237408+0000","last_change":"2024-07-04T17:34:54.272111+0000","last_active":"2024-07-04T17:34:56.237408+0000","last_peered":"2024-07-04T17:34:56.237408+0000","last_clean":"2024-07-04T17:34:56.237408+0000","last_became_active":"2024-07-04T17:34:54.271789+0000","last_became_peered":"2024-07-04T17:34:54.271789+0000","last_unstale":"2024-07-04T17:34:56.237408+0000","last_undegraded":"2024-07-04T17:34:56.237408+0000","last_fullsized":"2024-07-04T17:34:56.237408+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:10:04.594848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582645+0000","last_change":"2024-07-04T17:34:35.530930+0000","last_active":"2024-07-04T17:34:58.582645+0000","last_peered":"2024-07-04T17:34:58.582645+0000","last_clean":"2024-07-04T17:34:58.582645+0000","last_became_active":"2024-07-04T17:34:35.527474+0000","last_became_peered":"2024-07-04T17:34:35.527474+0000","last_unstale":"2024-07-04T17:34:58.582645+0000","last_undegraded":"2024-07-04T17:34:58.582645+0000","last_fullsized":"2024-07-04T17:34:58.582645+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:51:21.803840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"135'1","reported_seq":23,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.241143+0000","last_change":"2024-07-04T17:34:34.526961+0000","last_active":"2024-07-04T17:34:56.241143+0000","last_peered":"2024-07-04T17:34:56.241143+0000","last_clean":"2024-07-04T17:34:56.241143+0000","last_became_active":"2024-07-04T17:34:34.526338+0000","last_became_peered":"2024-07-04T17:34:34.526338+0000","last_unstale":"2024-07-04T17:34:56.241143+0000","last_undegraded":"2024-07-04T17:34:56.241143+0000","last_fullsized":"2024-07-04T17:34:56.241143+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:46:36.828153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580500+0000","last_change":"2024-07-04T17:34:55.263811+0000","last_active":"2024-07-04T17:34:58.580500+0000","last_peered":"2024-07-04T17:34:58.580500+0000","last_clean":"2024-07-04T17:34:58.580500+0000","last_became_active":"2024-07-04T17:34:55.262987+0000","last_became_peered":"2024-07-04T17:34:55.262987+0000","last_unstale":"2024-07-04T17:34:58.580500+0000","last_undegraded":"2024-07-04T17:34:58.580500+0000","last_fullsized":"2024-07-04T17:34:58.580500+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:51:29.870162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.237662+0000","last_change":"2024-07-04T17:34:54.270824+0000","last_active":"2024-07-04T17:34:56.237662+0000","last_peered":"2024-07-04T17:34:56.237662+0000","last_clean":"2024-07-04T17:34:56.237662+0000","last_became_active":"2024-07-04T17:34:54.270379+0000","last_became_peered":"2024-07-04T17:34:54.270379+0000","last_unstale":"2024-07-04T17:34:56.237662+0000","last_undegraded":"2024-07-04T17:34:56.237662+0000","last_fullsized":"2024-07-04T17:34:56.237662+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:25:50.505982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582706+0000","last_change":"2024-07-04T17:34:35.546152+0000","last_active":"2024-07-04T17:34:58.582706+0000","last_peered":"2024-07-04T17:34:58.582706+0000","last_clean":"2024-07-04T17:34:58.582706+0000","last_became_active":"2024-07-04T17:34:35.545573+0000","last_became_peered":"2024-07-04T17:34:35.545573+0000","last_unstale":"2024-07-04T17:34:58.582706+0000","last_undegraded":"2024-07-04T17:34:58.582706+0000","last_fullsized":"2024-07-04T17:34:58.582706+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:20:21.078798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.e","version":"135'2","reported_seq":24,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.247700+0000","last_change":"2024-07-04T17:34:34.516533+0000","last_active":"2024-07-04T17:34:56.247700+0000","last_peered":"2024-07-04T17:34:56.247700+0000","last_clean":"2024-07-04T17:34:56.247700+0000","last_became_active":"2024-07-04T17:34:34.515492+0000","last_became_peered":"2024-07-04T17:34:34.515492+0000","last_unstale":"2024-07-04T17:34:56.247700+0000","last_undegraded":"2024-07-04T17:34:56.247700+0000","last_fullsized":"2024-07-04T17:34:56.247700+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:06:02.654676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582181+0000","last_change":"2024-07-04T17:34:55.286421+0000","last_active":"2024-07-04T17:34:58.582181+0000","last_peered":"2024-07-04T17:34:58.582181+0000","last_clean":"2024-07-04T17:34:58.582181+0000","last_became_active":"2024-07-04T17:34:55.285847+0000","last_became_peered":"2024-07-04T17:34:55.285847+0000","last_unstale":"2024-07-04T17:34:58.582181+0000","last_undegraded":"2024-07-04T17:34:58.582181+0000","last_fullsized":"2024-07-04T17:34:58.582181+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:06:46.200018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.f","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.248318+0000","last_change":"2024-07-04T17:34:54.266573+0000","last_active":"2024-07-04T17:34:56.248318+0000","last_peered":"2024-07-04T17:34:56.248318+0000","last_clean":"2024-07-04T17:34:56.248318+0000","last_became_active":"2024-07-04T17:34:54.265975+0000","last_became_peered":"2024-07-04T17:34:54.265975+0000","last_unstale":"2024-07-04T17:34:56.248318+0000","last_undegraded":"2024-07-04T17:34:56.248318+0000","last_fullsized":"2024-07-04T17:34:56.248318+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:37:34.579661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.245005+0000","last_change":"2024-07-04T17:34:35.544212+0000","last_active":"2024-07-04T17:34:56.245005+0000","last_peered":"2024-07-04T17:34:56.245005+0000","last_clean":"2024-07-04T17:34:56.245005+0000","last_became_active":"2024-07-04T17:34:35.543570+0000","last_became_peered":"2024-07-04T17:34:35.543570+0000","last_unstale":"2024-07-04T17:34:56.245005+0000","last_undegraded":"2024-07-04T17:34:56.245005+0000","last_fullsized":"2024-07-04T17:34:56.245005+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:38:02.545198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.d","version":"135'1","reported_seq":23,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.577667+0000","last_change":"2024-07-04T17:34:34.514159+0000","last_active":"2024-07-04T17:34:58.577667+0000","last_peered":"2024-07-04T17:34:58.577667+0000","last_clean":"2024-07-04T17:34:58.577667+0000","last_became_active":"2024-07-04T17:34:34.511974+0000","last_became_peered":"2024-07-04T17:34:34.511974+0000","last_unstale":"2024-07-04T17:34:58.577667+0000","last_undegraded":"2024-07-04T17:34:58.577667+0000","last_fullsized":"2024-07-04T17:34:58.577667+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:20:11.531736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.248100+0000","last_change":"2024-07-04T17:34:55.261562+0000","last_active":"2024-07-04T17:34:56.248100+0000","last_peered":"2024-07-04T17:34:56.248100+0000","last_clean":"2024-07-04T17:34:56.248100+0000","last_became_active":"2024-07-04T17:34:55.254887+0000","last_became_peered":"2024-07-04T17:34:55.254887+0000","last_unstale":"2024-07-04T17:34:56.248100+0000","last_undegraded":"2024-07-04T17:34:56.248100+0000","last_fullsized":"2024-07-04T17:34:56.248100+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:54:48.573105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.e","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.240226+0000","last_change":"2024-07-04T17:34:54.271754+0000","last_active":"2024-07-04T17:34:56.240226+0000","last_peered":"2024-07-04T17:34:56.240226+0000","last_clean":"2024-07-04T17:34:56.240226+0000","last_became_active":"2024-07-04T17:34:54.271472+0000","last_became_peered":"2024-07-04T17:34:54.271472+0000","last_unstale":"2024-07-04T17:34:56.240226+0000","last_undegraded":"2024-07-04T17:34:56.240226+0000","last_fullsized":"2024-07-04T17:34:56.240226+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:05:22.077162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582520+0000","last_change":"2024-07-04T17:34:35.523503+0000","last_active":"2024-07-04T17:34:58.582520+0000","last_peered":"2024-07-04T17:34:58.582520+0000","last_clean":"2024-07-04T17:34:58.582520+0000","last_became_active":"2024-07-04T17:34:35.522995+0000","last_became_peered":"2024-07-04T17:34:35.522995+0000","last_unstale":"2024-07-04T17:34:58.582520+0000","last_undegraded":"2024-07-04T17:34:58.582520+0000","last_fullsized":"2024-07-04T17:34:58.582520+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:24:36.746418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.577607+0000","last_change":"2024-07-04T17:34:34.508754+0000","last_active":"2024-07-04T17:34:58.577607+0000","last_peered":"2024-07-04T17:34:58.577607+0000","last_clean":"2024-07-04T17:34:58.577607+0000","last_became_active":"2024-07-04T17:34:34.507614+0000","last_became_peered":"2024-07-04T17:34:34.507614+0000","last_unstale":"2024-07-04T17:34:58.577607+0000","last_undegraded":"2024-07-04T17:34:58.577607+0000","last_fullsized":"2024-07-04T17:34:58.577607+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:09:44.865693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.242349+0000","last_change":"2024-07-04T17:34:55.240066+0000","last_active":"2024-07-04T17:34:56.242349+0000","last_peered":"2024-07-04T17:34:56.242349+0000","last_clean":"2024-07-04T17:34:56.242349+0000","last_became_active":"2024-07-04T17:34:55.237881+0000","last_became_peered":"2024-07-04T17:34:55.237881+0000","last_unstale":"2024-07-04T17:34:56.242349+0000","last_undegraded":"2024-07-04T17:34:56.242349+0000","last_fullsized":"2024-07-04T17:34:56.242349+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:56:38.177831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.579023+0000","last_change":"2024-07-04T17:34:54.248960+0000","last_active":"2024-07-04T17:34:58.579023+0000","last_peered":"2024-07-04T17:34:58.579023+0000","last_clean":"2024-07-04T17:34:58.579023+0000","last_became_active":"2024-07-04T17:34:54.248440+0000","last_became_peered":"2024-07-04T17:34:54.248440+0000","last_unstale":"2024-07-04T17:34:58.579023+0000","last_undegraded":"2024-07-04T17:34:58.579023+0000","last_fullsized":"2024-07-04T17:34:58.579023+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:58:17.189484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580924+0000","last_change":"2024-07-04T17:34:35.515740+0000","last_active":"2024-07-04T17:34:58.580924+0000","last_peered":"2024-07-04T17:34:58.580924+0000","last_clean":"2024-07-04T17:34:58.580924+0000","last_became_active":"2024-07-04T17:34:35.514774+0000","last_became_peered":"2024-07-04T17:34:35.514774+0000","last_unstale":"2024-07-04T17:34:58.580924+0000","last_undegraded":"2024-07-04T17:34:58.580924+0000","last_fullsized":"2024-07-04T17:34:58.580924+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:54:35.373423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"135'2","reported_seq":24,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.244843+0000","last_change":"2024-07-04T17:34:34.515650+0000","last_active":"2024-07-04T17:34:56.244843+0000","last_peered":"2024-07-04T17:34:56.244843+0000","last_clean":"2024-07-04T17:34:56.244843+0000","last_became_active":"2024-07-04T17:34:34.514347+0000","last_became_peered":"2024-07-04T17:34:34.514347+0000","last_unstale":"2024-07-04T17:34:56.244843+0000","last_undegraded":"2024-07-04T17:34:56.244843+0000","last_fullsized":"2024-07-04T17:34:56.244843+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:34:16.111397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.242266+0000","last_change":"2024-07-04T17:34:55.259732+0000","last_active":"2024-07-04T17:34:56.242266+0000","last_peered":"2024-07-04T17:34:56.242266+0000","last_clean":"2024-07-04T17:34:56.242266+0000","last_became_active":"2024-07-04T17:34:55.258543+0000","last_became_peered":"2024-07-04T17:34:55.258543+0000","last_unstale":"2024-07-04T17:34:56.242266+0000","last_undegraded":"2024-07-04T17:34:56.242266+0000","last_fullsized":"2024-07-04T17:34:56.242266+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:32:40.557322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.242314+0000","last_change":"2024-07-04T17:34:54.264077+0000","last_active":"2024-07-04T17:34:56.242314+0000","last_peered":"2024-07-04T17:34:56.242314+0000","last_clean":"2024-07-04T17:34:56.242314+0000","last_became_active":"2024-07-04T17:34:54.263723+0000","last_became_peered":"2024-07-04T17:34:54.263723+0000","last_unstale":"2024-07-04T17:34:56.242314+0000","last_undegraded":"2024-07-04T17:34:56.242314+0000","last_fullsized":"2024-07-04T17:34:56.242314+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:18:44.495991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.238334+0000","last_change":"2024-07-04T17:34:35.529352+0000","last_active":"2024-07-04T17:34:56.238334+0000","last_peered":"2024-07-04T17:34:56.238334+0000","last_clean":"2024-07-04T17:34:56.238334+0000","last_became_active":"2024-07-04T17:34:35.528095+0000","last_became_peered":"2024-07-04T17:34:35.528095+0000","last_unstale":"2024-07-04T17:34:56.238334+0000","last_undegraded":"2024-07-04T17:34:56.238334+0000","last_fullsized":"2024-07-04T17:34:56.238334+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:53:30.230859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.237063+0000","last_change":"2024-07-04T17:34:34.520687+0000","last_active":"2024-07-04T17:34:56.237063+0000","last_peered":"2024-07-04T17:34:56.237063+0000","last_clean":"2024-07-04T17:34:56.237063+0000","last_became_active":"2024-07-04T17:34:34.520328+0000","last_became_peered":"2024-07-04T17:34:34.520328+0000","last_unstale":"2024-07-04T17:34:56.237063+0000","last_undegraded":"2024-07-04T17:34:56.237063+0000","last_fullsized":"2024-07-04T17:34:56.237063+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:26:49.904637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.243687+0000","last_change":"2024-07-04T17:34:55.249029+0000","last_active":"2024-07-04T17:34:56.243687+0000","last_peered":"2024-07-04T17:34:56.243687+0000","last_clean":"2024-07-04T17:34:56.243687+0000","last_became_active":"2024-07-04T17:34:55.248635+0000","last_became_peered":"2024-07-04T17:34:55.248635+0000","last_unstale":"2024-07-04T17:34:56.243687+0000","last_undegraded":"2024-07-04T17:34:56.243687+0000","last_fullsized":"2024-07-04T17:34:56.243687+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:49:06.940104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.b","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.579740+0000","last_change":"2024-07-04T17:34:54.250904+0000","last_active":"2024-07-04T17:34:58.579740+0000","last_peered":"2024-07-04T17:34:58.579740+0000","last_clean":"2024-07-04T17:34:58.579740+0000","last_became_active":"2024-07-04T17:34:54.247680+0000","last_became_peered":"2024-07-04T17:34:54.247680+0000","last_unstale":"2024-07-04T17:34:58.579740+0000","last_undegraded":"2024-07-04T17:34:58.579740+0000","last_fullsized":"2024-07-04T17:34:58.579740+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:23:57.570366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.579048+0000","last_change":"2024-07-04T17:34:35.525251+0000","last_active":"2024-07-04T17:34:58.579048+0000","last_peered":"2024-07-04T17:34:58.579048+0000","last_clean":"2024-07-04T17:34:58.579048+0000","last_became_active":"2024-07-04T17:34:35.524606+0000","last_became_peered":"2024-07-04T17:34:35.524606+0000","last_unstale":"2024-07-04T17:34:58.579048+0000","last_undegraded":"2024-07-04T17:34:58.579048+0000","last_fullsized":"2024-07-04T17:34:58.579048+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:18:18.311591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.245175+0000","last_change":"2024-07-04T17:34:34.515246+0000","last_active":"2024-07-04T17:34:56.245175+0000","last_peered":"2024-07-04T17:34:56.245175+0000","last_clean":"2024-07-04T17:34:56.245175+0000","last_became_active":"2024-07-04T17:34:34.514888+0000","last_became_peered":"2024-07-04T17:34:34.514888+0000","last_unstale":"2024-07-04T17:34:56.245175+0000","last_undegraded":"2024-07-04T17:34:56.245175+0000","last_fullsized":"2024-07-04T17:34:56.245175+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:35:07.644625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.238752+0000","last_change":"2024-07-04T17:34:55.275519+0000","last_active":"2024-07-04T17:34:56.238752+0000","last_peered":"2024-07-04T17:34:56.238752+0000","last_clean":"2024-07-04T17:34:56.238752+0000","last_became_active":"2024-07-04T17:34:55.274909+0000","last_became_peered":"2024-07-04T17:34:55.274909+0000","last_unstale":"2024-07-04T17:34:56.238752+0000","last_undegraded":"2024-07-04T17:34:56.238752+0000","last_fullsized":"2024-07-04T17:34:56.238752+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:54:55.103581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580097+0000","last_change":"2024-07-04T17:34:54.250256+0000","last_active":"2024-07-04T17:34:58.580097+0000","last_peered":"2024-07-04T17:34:58.580097+0000","last_clean":"2024-07-04T17:34:58.580097+0000","last_became_active":"2024-07-04T17:34:54.249493+0000","last_became_peered":"2024-07-04T17:34:54.249493+0000","last_unstale":"2024-07-04T17:34:58.580097+0000","last_undegraded":"2024-07-04T17:34:58.580097+0000","last_fullsized":"2024-07-04T17:34:58.580097+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:17:55.698458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.237412+0000","last_change":"2024-07-04T17:34:35.541160+0000","last_active":"2024-07-04T17:34:56.237412+0000","last_peered":"2024-07-04T17:34:56.237412+0000","last_clean":"2024-07-04T17:34:56.237412+0000","last_became_active":"2024-07-04T17:34:35.540869+0000","last_became_peered":"2024-07-04T17:34:35.540869+0000","last_unstale":"2024-07-04T17:34:56.237412+0000","last_undegraded":"2024-07-04T17:34:56.237412+0000","last_fullsized":"2024-07-04T17:34:56.237412+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:41:18.824553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.236897+0000","last_change":"2024-07-04T17:34:34.521170+0000","last_active":"2024-07-04T17:34:56.236897+0000","last_peered":"2024-07-04T17:34:56.236897+0000","last_clean":"2024-07-04T17:34:56.236897+0000","last_became_active":"2024-07-04T17:34:34.520731+0000","last_became_peered":"2024-07-04T17:34:34.520731+0000","last_unstale":"2024-07-04T17:34:56.236897+0000","last_undegraded":"2024-07-04T17:34:56.236897+0000","last_fullsized":"2024-07-04T17:34:56.236897+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:30:50.167556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.237504+0000","last_change":"2024-07-04T17:34:55.283231+0000","last_active":"2024-07-04T17:34:56.237504+0000","last_peered":"2024-07-04T17:34:56.237504+0000","last_clean":"2024-07-04T17:34:56.237504+0000","last_became_active":"2024-07-04T17:34:55.282939+0000","last_became_peered":"2024-07-04T17:34:55.282939+0000","last_unstale":"2024-07-04T17:34:56.237504+0000","last_undegraded":"2024-07-04T17:34:56.237504+0000","last_fullsized":"2024-07-04T17:34:56.237504+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:40:04.267278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.245883+0000","last_change":"2024-07-04T17:34:54.265406+0000","last_active":"2024-07-04T17:34:56.245883+0000","last_peered":"2024-07-04T17:34:56.245883+0000","last_clean":"2024-07-04T17:34:56.245883+0000","last_became_active":"2024-07-04T17:34:54.264774+0000","last_became_peered":"2024-07-04T17:34:54.264774+0000","last_unstale":"2024-07-04T17:34:56.245883+0000","last_undegraded":"2024-07-04T17:34:56.245883+0000","last_fullsized":"2024-07-04T17:34:56.245883+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:20:07.780705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.241264+0000","last_change":"2024-07-04T17:34:35.534458+0000","last_active":"2024-07-04T17:34:56.241264+0000","last_peered":"2024-07-04T17:34:56.241264+0000","last_clean":"2024-07-04T17:34:56.241264+0000","last_became_active":"2024-07-04T17:34:35.534123+0000","last_became_peered":"2024-07-04T17:34:35.534123+0000","last_unstale":"2024-07-04T17:34:56.241264+0000","last_undegraded":"2024-07-04T17:34:56.241264+0000","last_fullsized":"2024-07-04T17:34:56.241264+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:55:02.890933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"135'2","reported_seq":24,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.577672+0000","last_change":"2024-07-04T17:34:34.516580+0000","last_active":"2024-07-04T17:34:58.577672+0000","last_peered":"2024-07-04T17:34:58.577672+0000","last_clean":"2024-07-04T17:34:58.577672+0000","last_became_active":"2024-07-04T17:34:34.513629+0000","last_became_peered":"2024-07-04T17:34:34.513629+0000","last_unstale":"2024-07-04T17:34:58.577672+0000","last_undegraded":"2024-07-04T17:34:58.577672+0000","last_fullsized":"2024-07-04T17:34:58.577672+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:31:26.410567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.581976+0000","last_change":"2024-07-04T17:34:55.271389+0000","last_active":"2024-07-04T17:34:58.581976+0000","last_peered":"2024-07-04T17:34:58.581976+0000","last_clean":"2024-07-04T17:34:58.581976+0000","last_became_active":"2024-07-04T17:34:55.266272+0000","last_became_peered":"2024-07-04T17:34:55.266272+0000","last_unstale":"2024-07-04T17:34:58.581976+0000","last_undegraded":"2024-07-04T17:34:58.581976+0000","last_fullsized":"2024-07-04T17:34:58.581976+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:11:00.326191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.237859+0000","last_change":"2024-07-04T17:34:54.270486+0000","last_active":"2024-07-04T17:34:56.237859+0000","last_peered":"2024-07-04T17:34:56.237859+0000","last_clean":"2024-07-04T17:34:56.237859+0000","last_became_active":"2024-07-04T17:34:54.269722+0000","last_became_peered":"2024-07-04T17:34:54.269722+0000","last_unstale":"2024-07-04T17:34:56.237859+0000","last_undegraded":"2024-07-04T17:34:56.237859+0000","last_fullsized":"2024-07-04T17:34:56.237859+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:47:38.704471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582751+0000","last_change":"2024-07-04T17:34:35.545883+0000","last_active":"2024-07-04T17:34:58.582751+0000","last_peered":"2024-07-04T17:34:58.582751+0000","last_clean":"2024-07-04T17:34:58.582751+0000","last_became_active":"2024-07-04T17:34:35.545154+0000","last_became_peered":"2024-07-04T17:34:35.545154+0000","last_unstale":"2024-07-04T17:34:58.582751+0000","last_undegraded":"2024-07-04T17:34:58.582751+0000","last_fullsized":"2024-07-04T17:34:58.582751+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:36:58.592291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"135'1","reported_seq":23,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.577643+0000","last_change":"2024-07-04T17:34:34.505851+0000","last_active":"2024-07-04T17:34:58.577643+0000","last_peered":"2024-07-04T17:34:58.577643+0000","last_clean":"2024-07-04T17:34:58.577643+0000","last_became_active":"2024-07-04T17:34:34.505548+0000","last_became_peered":"2024-07-04T17:34:34.505548+0000","last_unstale":"2024-07-04T17:34:58.577643+0000","last_undegraded":"2024-07-04T17:34:58.577643+0000","last_fullsized":"2024-07-04T17:34:58.577643+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:17:04.987342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.582029+0000","last_change":"2024-07-04T17:34:55.286697+0000","last_active":"2024-07-04T17:34:58.582029+0000","last_peered":"2024-07-04T17:34:58.582029+0000","last_clean":"2024-07-04T17:34:58.582029+0000","last_became_active":"2024-07-04T17:34:55.286239+0000","last_became_peered":"2024-07-04T17:34:55.286239+0000","last_unstale":"2024-07-04T17:34:58.582029+0000","last_undegraded":"2024-07-04T17:34:58.582029+0000","last_fullsized":"2024-07-04T17:34:58.582029+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:55:24.645280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.7","version":"139'2","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.239893+0000","last_change":"2024-07-04T17:34:54.271614+0000","last_active":"2024-07-04T17:34:56.239893+0000","last_peered":"2024-07-04T17:34:56.239893+0000","last_clean":"2024-07-04T17:34:56.239893+0000","last_became_active":"2024-07-04T17:34:54.271161+0000","last_became_peered":"2024-07-04T17:34:54.271161+0000","last_unstale":"2024-07-04T17:34:56.239893+0000","last_undegraded":"2024-07-04T17:34:56.239893+0000","last_fullsized":"2024-07-04T17:34:56.239893+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:34:52.282620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.245100+0000","last_change":"2024-07-04T17:34:35.543872+0000","last_active":"2024-07-04T17:34:56.245100+0000","last_peered":"2024-07-04T17:34:56.245100+0000","last_clean":"2024-07-04T17:34:56.245100+0000","last_became_active":"2024-07-04T17:34:35.542863+0000","last_became_peered":"2024-07-04T17:34:35.542863+0000","last_unstale":"2024-07-04T17:34:56.245100+0000","last_undegraded":"2024-07-04T17:34:56.245100+0000","last_fullsized":"2024-07-04T17:34:56.245100+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:28:18.564334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.577527+0000","last_change":"2024-07-04T17:34:34.505554+0000","last_active":"2024-07-04T17:34:58.577527+0000","last_peered":"2024-07-04T17:34:58.577527+0000","last_clean":"2024-07-04T17:34:58.577527+0000","last_became_active":"2024-07-04T17:34:34.505151+0000","last_became_peered":"2024-07-04T17:34:34.505151+0000","last_unstale":"2024-07-04T17:34:58.577527+0000","last_undegraded":"2024-07-04T17:34:58.577527+0000","last_fullsized":"2024-07-04T17:34:58.577527+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:17:53.346868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580833+0000","last_change":"2024-07-04T17:34:55.287357+0000","last_active":"2024-07-04T17:34:58.580833+0000","last_peered":"2024-07-04T17:34:58.580833+0000","last_clean":"2024-07-04T17:34:58.580833+0000","last_became_active":"2024-07-04T17:34:55.286890+0000","last_became_peered":"2024-07-04T17:34:55.286890+0000","last_unstale":"2024-07-04T17:34:58.580833+0000","last_undegraded":"2024-07-04T17:34:58.580833+0000","last_fullsized":"2024-07-04T17:34:58.580833+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:10:36.543674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"140'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.246992+0000","last_change":"2024-07-04T17:34:54.267192+0000","last_active":"2024-07-04T17:34:56.246992+0000","last_peered":"2024-07-04T17:34:56.246992+0000","last_clean":"2024-07-04T17:34:56.246992+0000","last_became_active":"2024-07-04T17:34:54.266751+0000","last_became_peered":"2024-07-04T17:34:54.266751+0000","last_unstale":"2024-07-04T17:34:56.246992+0000","last_undegraded":"2024-07-04T17:34:56.246992+0000","last_fullsized":"2024-07-04T17:34:56.246992+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:56:37.308739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.247157+0000","last_change":"2024-07-04T17:34:35.543911+0000","last_active":"2024-07-04T17:34:56.247157+0000","last_peered":"2024-07-04T17:34:56.247157+0000","last_clean":"2024-07-04T17:34:56.247157+0000","last_became_active":"2024-07-04T17:34:35.533364+0000","last_became_peered":"2024-07-04T17:34:35.533364+0000","last_unstale":"2024-07-04T17:34:56.247157+0000","last_undegraded":"2024-07-04T17:34:56.247157+0000","last_fullsized":"2024-07-04T17:34:56.247157+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T21:13:03.582706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.577485+0000","last_change":"2024-07-04T17:34:34.507394+0000","last_active":"2024-07-04T17:34:58.577485+0000","last_peered":"2024-07-04T17:34:58.577485+0000","last_clean":"2024-07-04T17:34:58.577485+0000","last_became_active":"2024-07-04T17:34:34.506560+0000","last_became_peered":"2024-07-04T17:34:34.506560+0000","last_unstale":"2024-07-04T17:34:58.577485+0000","last_undegraded":"2024-07-04T17:34:58.577485+0000","last_fullsized":"2024-07-04T17:34:58.577485+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:12:30.641841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.234817+0000","last_change":"2024-07-04T17:34:55.289868+0000","last_active":"2024-07-04T17:34:56.234817+0000","last_peered":"2024-07-04T17:34:56.234817+0000","last_clean":"2024-07-04T17:34:56.234817+0000","last_became_active":"2024-07-04T17:34:55.286082+0000","last_became_peered":"2024-07-04T17:34:55.286082+0000","last_unstale":"2024-07-04T17:34:56.234817+0000","last_undegraded":"2024-07-04T17:34:56.234817+0000","last_fullsized":"2024-07-04T17:34:56.234817+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:18:38.041750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.578888+0000","last_change":"2024-07-04T17:34:54.246417+0000","last_active":"2024-07-04T17:34:58.578888+0000","last_peered":"2024-07-04T17:34:58.578888+0000","last_clean":"2024-07-04T17:34:58.578888+0000","last_became_active":"2024-07-04T17:34:54.246033+0000","last_became_peered":"2024-07-04T17:34:54.246033+0000","last_unstale":"2024-07-04T17:34:58.578888+0000","last_undegraded":"2024-07-04T17:34:58.578888+0000","last_fullsized":"2024-07-04T17:34:58.578888+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:24:28.116695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.244471+0000","last_change":"2024-07-04T17:34:35.543760+0000","last_active":"2024-07-04T17:34:56.244471+0000","last_peered":"2024-07-04T17:34:56.244471+0000","last_clean":"2024-07-04T17:34:56.244471+0000","last_became_active":"2024-07-04T17:34:35.533337+0000","last_became_peered":"2024-07-04T17:34:35.533337+0000","last_unstale":"2024-07-04T17:34:56.244471+0000","last_undegraded":"2024-07-04T17:34:56.244471+0000","last_fullsized":"2024-07-04T17:34:56.244471+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T00:15:16.122467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.245193+0000","last_change":"2024-07-04T17:34:34.513417+0000","last_active":"2024-07-04T17:34:56.245193+0000","last_peered":"2024-07-04T17:34:56.245193+0000","last_clean":"2024-07-04T17:34:56.245193+0000","last_became_active":"2024-07-04T17:34:34.513092+0000","last_became_peered":"2024-07-04T17:34:34.513092+0000","last_unstale":"2024-07-04T17:34:56.245193+0000","last_undegraded":"2024-07-04T17:34:56.245193+0000","last_fullsized":"2024-07-04T17:34:56.245193+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:04:50.268887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.242818+0000","last_change":"2024-07-04T17:34:55.260018+0000","last_active":"2024-07-04T17:34:56.242818+0000","last_peered":"2024-07-04T17:34:56.242818+0000","last_clean":"2024-07-04T17:34:56.242818+0000","last_became_active":"2024-07-04T17:34:55.259357+0000","last_became_peered":"2024-07-04T17:34:55.259357+0000","last_unstale":"2024-07-04T17:34:56.242818+0000","last_undegraded":"2024-07-04T17:34:56.242818+0000","last_fullsized":"2024-07-04T17:34:56.242818+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T03:59:14.916680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.242547+0000","last_change":"2024-07-04T17:34:54.251972+0000","last_active":"2024-07-04T17:34:56.242547+0000","last_peered":"2024-07-04T17:34:56.242547+0000","last_clean":"2024-07-04T17:34:56.242547+0000","last_became_active":"2024-07-04T17:34:54.251663+0000","last_became_peered":"2024-07-04T17:34:54.251663+0000","last_unstale":"2024-07-04T17:34:56.242547+0000","last_undegraded":"2024-07-04T17:34:56.242547+0000","last_fullsized":"2024-07-04T17:34:56.242547+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:10:30.523911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.238631+0000","last_change":"2024-07-04T17:34:35.521764+0000","last_active":"2024-07-04T17:34:56.238631+0000","last_peered":"2024-07-04T17:34:56.238631+0000","last_clean":"2024-07-04T17:34:56.238631+0000","last_became_active":"2024-07-04T17:34:35.521455+0000","last_became_peered":"2024-07-04T17:34:35.521455+0000","last_unstale":"2024-07-04T17:34:56.238631+0000","last_undegraded":"2024-07-04T17:34:56.238631+0000","last_fullsized":"2024-07-04T17:34:56.238631+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:54:12.411851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.244467+0000","last_change":"2024-07-04T17:34:34.518708+0000","last_active":"2024-07-04T17:34:56.244467+0000","last_peered":"2024-07-04T17:34:56.244467+0000","last_clean":"2024-07-04T17:34:56.244467+0000","last_became_active":"2024-07-04T17:34:34.518368+0000","last_became_peered":"2024-07-04T17:34:34.518368+0000","last_unstale":"2024-07-04T17:34:56.244467+0000","last_undegraded":"2024-07-04T17:34:56.244467+0000","last_fullsized":"2024-07-04T17:34:56.244467+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:58:34.507931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.579670+0000","last_change":"2024-07-04T17:34:55.284100+0000","last_active":"2024-07-04T17:34:58.579670+0000","last_peered":"2024-07-04T17:34:58.579670+0000","last_clean":"2024-07-04T17:34:58.579670+0000","last_became_active":"2024-07-04T17:34:55.283787+0000","last_became_peered":"2024-07-04T17:34:55.283787+0000","last_unstale":"2024-07-04T17:34:58.579670+0000","last_undegraded":"2024-07-04T17:34:58.579670+0000","last_fullsized":"2024-07-04T17:34:58.579670+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:34:58.855161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.243671+0000","last_change":"2024-07-04T17:34:54.255923+0000","last_active":"2024-07-04T17:34:56.243671+0000","last_peered":"2024-07-04T17:34:56.243671+0000","last_clean":"2024-07-04T17:34:56.243671+0000","last_became_active":"2024-07-04T17:34:54.255556+0000","last_became_peered":"2024-07-04T17:34:54.255556+0000","last_unstale":"2024-07-04T17:34:56.243671+0000","last_undegraded":"2024-07-04T17:34:56.243671+0000","last_fullsized":"2024-07-04T17:34:56.243671+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:54:31.128546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.243878+0000","last_change":"2024-07-04T17:34:35.520630+0000","last_active":"2024-07-04T17:34:56.243878+0000","last_peered":"2024-07-04T17:34:56.243878+0000","last_clean":"2024-07-04T17:34:56.243878+0000","last_became_active":"2024-07-04T17:34:35.520357+0000","last_became_peered":"2024-07-04T17:34:35.520357+0000","last_unstale":"2024-07-04T17:34:56.243878+0000","last_undegraded":"2024-07-04T17:34:56.243878+0000","last_fullsized":"2024-07-04T17:34:56.243878+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:46:37.840612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.245539+0000","last_change":"2024-07-04T17:34:34.518104+0000","last_active":"2024-07-04T17:34:56.245539+0000","last_peered":"2024-07-04T17:34:56.245539+0000","last_clean":"2024-07-04T17:34:56.245539+0000","last_became_active":"2024-07-04T17:34:34.513906+0000","last_became_peered":"2024-07-04T17:34:34.513906+0000","last_unstale":"2024-07-04T17:34:56.245539+0000","last_undegraded":"2024-07-04T17:34:56.245539+0000","last_fullsized":"2024-07-04T17:34:56.245539+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T23:43:24.362777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.243367+0000","last_change":"2024-07-04T17:34:55.259916+0000","last_active":"2024-07-04T17:34:56.243367+0000","last_peered":"2024-07-04T17:34:56.243367+0000","last_clean":"2024-07-04T17:34:56.243367+0000","last_became_active":"2024-07-04T17:34:55.257014+0000","last_became_peered":"2024-07-04T17:34:55.257014+0000","last_unstale":"2024-07-04T17:34:56.243367+0000","last_undegraded":"2024-07-04T17:34:56.243367+0000","last_fullsized":"2024-07-04T17:34:56.243367+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:18:38.041750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":14,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.243756+0000","last_change":"2024-07-04T17:34:54.267250+0000","last_active":"2024-07-04T17:34:56.243756+0000","last_peered":"2024-07-04T17:34:56.243756+0000","last_clean":"2024-07-04T17:34:56.243756+0000","last_became_active":"2024-07-04T17:34:54.266937+0000","last_became_peered":"2024-07-04T17:34:54.266937+0000","last_unstale":"2024-07-04T17:34:56.243756+0000","last_undegraded":"2024-07-04T17:34:56.243756+0000","last_fullsized":"2024-07-04T17:34:56.243756+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:33:36.486426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.237638+0000","last_change":"2024-07-04T17:34:35.538841+0000","last_active":"2024-07-04T17:34:56.237638+0000","last_peered":"2024-07-04T17:34:56.237638+0000","last_clean":"2024-07-04T17:34:56.237638+0000","last_became_active":"2024-07-04T17:34:35.538551+0000","last_became_peered":"2024-07-04T17:34:35.538551+0000","last_unstale":"2024-07-04T17:34:56.237638+0000","last_undegraded":"2024-07-04T17:34:56.237638+0000","last_fullsized":"2024-07-04T17:34:56.237638+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T05:02:46.435969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.244048+0000","last_change":"2024-07-04T17:34:34.514509+0000","last_active":"2024-07-04T17:34:56.244048+0000","last_peered":"2024-07-04T17:34:56.244048+0000","last_clean":"2024-07-04T17:34:56.244048+0000","last_became_active":"2024-07-04T17:34:34.511266+0000","last_became_peered":"2024-07-04T17:34:34.511266+0000","last_unstale":"2024-07-04T17:34:56.244048+0000","last_undegraded":"2024-07-04T17:34:56.244048+0000","last_fullsized":"2024-07-04T17:34:56.244048+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:18:52.279136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":12,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.580727+0000","last_change":"2024-07-04T17:34:55.263706+0000","last_active":"2024-07-04T17:34:58.580727+0000","last_peered":"2024-07-04T17:34:58.580727+0000","last_clean":"2024-07-04T17:34:58.580727+0000","last_became_active":"2024-07-04T17:34:55.263339+0000","last_became_peered":"2024-07-04T17:34:55.263339+0000","last_unstale":"2024-07-04T17:34:58.580727+0000","last_undegraded":"2024-07-04T17:34:58.580727+0000","last_fullsized":"2024-07-04T17:34:58.580727+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:54.192271+0000","last_clean_scrub_stamp":"2024-07-04T17:34:54.192271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T22:30:29.811812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"139'1","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.243471+0000","last_change":"2024-07-04T17:34:54.267924+0000","last_active":"2024-07-04T17:34:56.243471+0000","last_peered":"2024-07-04T17:34:56.243471+0000","last_clean":"2024-07-04T17:34:56.243471+0000","last_became_active":"2024-07-04T17:34:54.267590+0000","last_became_peered":"2024-07-04T17:34:54.267590+0000","last_unstale":"2024-07-04T17:34:56.243471+0000","last_undegraded":"2024-07-04T17:34:56.243471+0000","last_fullsized":"2024-07-04T17:34:56.243471+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:53.178808+0000","last_clean_scrub_stamp":"2024-07-04T17:34:53.178808+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T04:53:48.947554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":20,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.579164+0000","last_change":"2024-07-04T17:34:35.515886+0000","last_active":"2024-07-04T17:34:58.579164+0000","last_peered":"2024-07-04T17:34:58.579164+0000","last_clean":"2024-07-04T17:34:58.579164+0000","last_became_active":"2024-07-04T17:34:35.515383+0000","last_became_peered":"2024-07-04T17:34:35.515383+0000","last_unstale":"2024-07-04T17:34:58.579164+0000","last_undegraded":"2024-07-04T17:34:58.579164+0000","last_fullsized":"2024-07-04T17:34:58.579164+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:34.462113+0000","last_clean_scrub_stamp":"2024-07-04T17:34:34.462113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:04:52.389960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":22,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:58.577369+0000","last_change":"2024-07-04T17:34:34.515873+0000","last_active":"2024-07-04T17:34:58.577369+0000","last_peered":"2024-07-04T17:34:58.577369+0000","last_clean":"2024-07-04T17:34:58.577369+0000","last_became_active":"2024-07-04T17:34:34.515559+0000","last_became_peered":"2024-07-04T17:34:34.515559+0000","last_unstale":"2024-07-04T17:34:58.577369+0000","last_undegraded":"2024-07-04T17:34:58.577369+0000","last_fullsized":"2024-07-04T17:34:58.577369+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:34:33.449654+0000","last_clean_scrub_stamp":"2024-07-04T17:34:33.449654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T01:21:52.711984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"122'115","reported_seq":549,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-07-04T17:34:56.244233+0000","last_change":"2024-07-04T17:26:04.953559+0000","last_active":"2024-07-04T17:34:56.244233+0000","last_peered":"2024-07-04T17:34:56.244233+0000","last_clean":"2024-07-04T17:34:56.244233+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:34:56.244233+0000","last_undegraded":"2024-07-04T17:34:56.244233+0000","last_fullsized":"2024-07-04T17:34:56.244233+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_clean_scrub_stamp":"2024-07-04T17:26:03.933432+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.064181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1630,"num_write":187,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":4107,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":3},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":596,"num_read_kb":1630,"num_write":187,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738473,"num_pgs":126,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36804,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335036,"statfs":{"total":96636764160,"available":96599076864,"internally_reserved":0,"allocated":1490944,"data_stored":582416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738474,"num_pgs":124,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37376,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334464,"statfs":{"total":96636764160,"available":96598491136,"internally_reserved":0,"allocated":2084864,"data_stored":1174871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738472,"num_pgs":112,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37372,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334468,"statfs":{"total":96636764160,"available":96598495232,"internally_reserved":0,"allocated":2080768,"data_stored":1174479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:35:00.619 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-04T17:35:00.619 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:35:00.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.949+0000 7fcef2000640 1 -- 172.21.15.42:0/3385647492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fceec06cea0 msgr2=0x7fceec0b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:35:01.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.949+0000 7fcef2000640 1 --2- 172.21.15.42:0/3385647492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fceec06cea0 0x7fceec0b3030 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fcedc0097b0 tx=0x7fcedc02a530 comp rx=0 tx=0).stop 2024-07-04T17:35:01.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.949+0000 7fcef2000640 1 -- 172.21.15.42:0/3385647492 shutdown_connections 2024-07-04T17:35:01.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.949+0000 7fcef2000640 1 --2- 172.21.15.42:0/3385647492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fceec06cea0 0x7fceec0b3030 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:35:01.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.949+0000 7fcef2000640 1 -- 172.21.15.42:0/3385647492 >> 172.21.15.42:0/3385647492 conn(0x7fceec0bf700 msgr2=0x7fceec0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:35:01.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.949+0000 7fcef2000640 1 -- 172.21.15.42:0/3385647492 shutdown_connections 2024-07-04T17:35:01.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.949+0000 7fcef2000640 1 -- 172.21.15.42:0/3385647492 wait complete. 2024-07-04T17:35:01.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.951+0000 7fcef2000640 1 Processor -- start 2024-07-04T17:35:01.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.951+0000 7fcef2000640 1 -- start start 2024-07-04T17:35:01.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.953+0000 7fcef2000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fceec06cea0 0x7fceec0ae090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:35:01.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.953+0000 7fcef2000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fceec06ef50 con 0x7fceec06cea0 2024-07-04T17:35:01.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.953+0000 7fceeb400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fceec06cea0 0x7fceec0ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:35:01.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.953+0000 7fceeb400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fceec06cea0 0x7fceec0ae090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56146/0 (socket says 172.21.15.42:56146) 2024-07-04T17:35:01.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.953+0000 7fceeb400640 1 -- 172.21.15.42:0/1497625787 learned_addr learned my addr 172.21.15.42:0/1497625787 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:35:01.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.953+0000 7fceeb400640 1 -- 172.21.15.42:0/1497625787 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceec14a200 con 0x7fceec06cea0 2024-07-04T17:35:01.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.954+0000 7fceeb400640 1 --2- 172.21.15.42:0/1497625787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fceec06cea0 0x7fceec0ae090 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fcedc02ab20 tx=0x7fcedc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:35:01.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.954+0000 7fcee3e00640 1 -- 172.21.15.42:0/1497625787 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcedc0035f0 con 0x7fceec06cea0 2024-07-04T17:35:01.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.954+0000 7fcee3e00640 1 -- 172.21.15.42:0/1497625787 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcedc003880 con 0x7fceec06cea0 2024-07-04T17:35:01.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.954+0000 7fcee3e00640 1 -- 172.21.15.42:0/1497625787 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcedc003b60 con 0x7fceec06cea0 2024-07-04T17:35:01.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.954+0000 7fcef2000640 1 -- 172.21.15.42:0/1497625787 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceec1d4490 con 0x7fceec06cea0 2024-07-04T17:35:01.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.954+0000 7fcef2000640 1 -- 172.21.15.42:0/1497625787 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceec144350 con 0x7fceec06cea0 2024-07-04T17:35:01.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.955+0000 7fcef2000640 1 -- 172.21.15.42:0/1497625787 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceec0b3030 con 0x7fceec06cea0 2024-07-04T17:35:01.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.956+0000 7fcee3e00640 1 -- 172.21.15.42:0/1497625787 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7fcedc003d00 con 0x7fceec06cea0 2024-07-04T17:35:01.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.956+0000 7fcee3e00640 1 --2- 172.21.15.42:0/1497625787 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fcec803d4f0 0x7fcec803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:35:01.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.956+0000 7fcee3e00640 1 -- 172.21.15.42:0/1497625787 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 4303+0+0 (secure 0 0 0) 0x7fcedc073330 con 0x7fceec06cea0 2024-07-04T17:35:01.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.957+0000 7fcee2a00640 1 --2- 172.21.15.42:0/1497625787 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fcec803d4f0 0x7fcec803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:35:01.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.957+0000 7fcee2a00640 1 --2- 172.21.15.42:0/1497625787 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fcec803d4f0 0x7fcec803fa00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fced4007fd0 tx=0x7fced4007920 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:35:01.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:00.962+0000 7fcee3e00640 1 -- 172.21.15.42:0/1497625787 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fcedc03f030 con 0x7fceec06cea0 2024-07-04T17:35:01.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:01.212+0000 7fcef2000640 1 -- 172.21.15.42:0/1497625787 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fceec0af670 con 0x7fceec06cea0 2024-07-04T17:35:01.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:01.214+0000 7fcee3e00640 1 -- 172.21.15.42:0/1497625787 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+7380 (secure 0 0 0) 0x7fcedc073080 con 0x7fceec06cea0 2024-07-04T17:35:01.215 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:35:01.215 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":113,"btime":"2024-07-04T17:34:58:236243+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5803,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/1184311185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":1184311185},{"type":"v1","addr":"172.21.15.42:6825","nonce":1184311185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103},{"gid":5808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/1815554749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1815554749},{"type":"v1","addr":"172.21.15.42:6827","nonce":1815554749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":5809,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/2027504764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2027504764},{"type":"v1","addr":"172.21.15.42:6829","nonce":2027504764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:34:35.049685+0000","modified":"2024-07-04T17:34:38.110000+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5818},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5818":{"gid":5818,"name":"d","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6833/1863956336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1863956336},{"type":"v1","addr":"172.21.15.42:6833","nonce":1863956336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5818,"qdb_cluster":[5818]},"id":12},{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:34:54.794041+0000","modified":"2024-07-04T17:34:57.841598+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5817},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5817":{"gid":5817,"name":"a","rank":0,"incarnation":110,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6831/4064998265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":4064998265},{"type":"v1","addr":"172.21.15.42:6831","nonce":4064998265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5817,"qdb_cluster":[5817]},"id":13}]} 2024-07-04T17:35:01.215 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 113 2024-07-04T17:35:01.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:01.216+0000 7fcef2000640 1 -- 172.21.15.42:0/1497625787 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fcec803d4f0 msgr2=0x7fcec803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:35:01.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:01.216+0000 7fcef2000640 1 --2- 172.21.15.42:0/1497625787 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fcec803d4f0 0x7fcec803fa00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fced4007fd0 tx=0x7fced4007920 comp rx=0 tx=0).stop 2024-07-04T17:35:01.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:01.216+0000 7fcef2000640 1 -- 172.21.15.42:0/1497625787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fceec06cea0 msgr2=0x7fceec0ae090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:35:01.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:01.216+0000 7fcef2000640 1 --2- 172.21.15.42:0/1497625787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fceec06cea0 0x7fceec0ae090 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fcedc02ab20 tx=0x7fcedc004040 comp rx=0 tx=0).stop 2024-07-04T17:35:01.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:01.217+0000 7fcef2000640 1 -- 172.21.15.42:0/1497625787 shutdown_connections 2024-07-04T17:35:01.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:01.217+0000 7fcef2000640 1 --2- 172.21.15.42:0/1497625787 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fcec803d4f0 0x7fcec803fa00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:35:01.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:01.217+0000 7fcef2000640 1 --2- 172.21.15.42:0/1497625787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fceec06cea0 0x7fceec0ae090 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:35:01.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:01.217+0000 7fcef2000640 1 -- 172.21.15.42:0/1497625787 >> 172.21.15.42:0/1497625787 conn(0x7fceec0bf700 msgr2=0x7fceec0bc260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:35:01.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:01.217+0000 7fcef2000640 1 -- 172.21.15.42:0/1497625787 shutdown_connections 2024-07-04T17:35:01.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:01.217+0000 7fcef2000640 1 -- 172.21.15.42:0/1497625787 wait complete. 2024-07-04T17:35:01.565 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-04T17:35:01.746 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 113, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-04T17:34:54.794041+0000', 'modified': '2024-07-04T17:34:57.841598+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5817}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5817': {'gid': 5817, 'name': 'a', 'rank': 0, 'incarnation': 110, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.42:6831/4064998265', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6830', 'nonce': 4064998265}, {'type': 'v1', 'addr': '172.21.15.42:6831', 'nonce': 4064998265}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'backup_fs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5817, 'qdb_cluster': [5817]} 2024-07-04T17:35:01.746 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-04T17:35:01.747 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-07-04T17:35:02.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.046+0000 7f48fb800640 1 -- 172.21.15.42:0/1180417734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48f40b02b0 msgr2=0x7f48f40b3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:35:02.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.046+0000 7f48fb800640 1 --2- 172.21.15.42:0/1180417734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48f40b02b0 0x7f48f40b3650 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f48e40097b0 tx=0x7f48e402a530 comp rx=0 tx=0).stop 2024-07-04T17:35:02.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.046+0000 7f48fb800640 1 -- 172.21.15.42:0/1180417734 shutdown_connections 2024-07-04T17:35:02.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.046+0000 7f48fb800640 1 --2- 172.21.15.42:0/1180417734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48f40b02b0 0x7f48f40b3650 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:35:02.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.046+0000 7f48fb800640 1 -- 172.21.15.42:0/1180417734 >> 172.21.15.42:0/1180417734 conn(0x7f48f40bf700 msgr2=0x7f48f40af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:35:02.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.046+0000 7f48fb800640 1 -- 172.21.15.42:0/1180417734 shutdown_connections 2024-07-04T17:35:02.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.046+0000 7f48fb800640 1 -- 172.21.15.42:0/1180417734 wait complete. 2024-07-04T17:35:02.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.048+0000 7f48fb800640 1 Processor -- start 2024-07-04T17:35:02.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.048+0000 7f48fb800640 1 -- start start 2024-07-04T17:35:02.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.050+0000 7f48fb800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48f40b02b0 0x7f48f41d6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:35:02.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.050+0000 7f48fb800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f48f41463b0 con 0x7f48f40b02b0 2024-07-04T17:35:02.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.050+0000 7f48f8e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48f40b02b0 0x7f48f41d6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:35:02.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.050+0000 7f48f8e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48f40b02b0 0x7f48f41d6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56166/0 (socket says 172.21.15.42:56166) 2024-07-04T17:35:02.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.050+0000 7f48f8e00640 1 -- 172.21.15.42:0/4058145367 learned_addr learned my addr 172.21.15.42:0/4058145367 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:35:02.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.051+0000 7f48f8e00640 1 -- 172.21.15.42:0/4058145367 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f41d7a50 con 0x7f48f40b02b0 2024-07-04T17:35:02.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.051+0000 7f48f8e00640 1 --2- 172.21.15.42:0/4058145367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48f40b02b0 0x7f48f41d6f60 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f48e40098d0 tx=0x7f48e4033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:35:02.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.051+0000 7f48f0c00640 1 -- 172.21.15.42:0/4058145367 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48e4031880 con 0x7f48f40b02b0 2024-07-04T17:35:02.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.051+0000 7f48f0c00640 1 -- 172.21.15.42:0/4058145367 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48e4031b10 con 0x7f48f40b02b0 2024-07-04T17:35:02.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.051+0000 7f48f0c00640 1 -- 172.21.15.42:0/4058145367 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48e4031df0 con 0x7f48f40b02b0 2024-07-04T17:35:02.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.051+0000 7f48fb800640 1 -- 172.21.15.42:0/4058145367 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48f41d7550 con 0x7f48f40b02b0 2024-07-04T17:35:02.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.052+0000 7f48fb800640 1 -- 172.21.15.42:0/4058145367 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48f41d8fd0 con 0x7f48f40b02b0 2024-07-04T17:35:02.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.053+0000 7f48f0c00640 1 -- 172.21.15.42:0/4058145367 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7f48e4033070 con 0x7f48f40b02b0 2024-07-04T17:35:02.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.053+0000 7f48f0c00640 1 --2- 172.21.15.42:0/4058145367 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f48d403d540 0x7f48d403fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:35:02.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.054+0000 7f48f0c00640 1 -- 172.21.15.42:0/4058145367 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 4303+0+0 (secure 0 0 0) 0x7f48e4073000 con 0x7f48f40b02b0 2024-07-04T17:35:02.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.054+0000 7f48f3e00640 1 --2- 172.21.15.42:0/4058145367 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f48d403d540 0x7f48d403fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:35:02.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.055+0000 7f48f3e00640 1 --2- 172.21.15.42:0/4058145367 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f48d403d540 0x7f48d403fa50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f48e0002020 tx=0x7f48e0001610 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:35:02.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.056+0000 7f4902835740 1 Processor -- start 2024-07-04T17:35:02.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.056+0000 7f4902835740 1 -- start start 2024-07-04T17:35:02.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.057+0000 7f4902835740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eaec6b9270 0x55eaeca5fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:35:02.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.057+0000 7f4902835740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55eaec9d05b0 con 0x55eaec6b9270 2024-07-04T17:35:02.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.057+0000 7f48f8e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eaec6b9270 0x55eaeca5fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:35:02.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.057+0000 7f48f8e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eaec6b9270 0x55eaeca5fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56168/0 (socket says 172.21.15.42:56168) 2024-07-04T17:35:02.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.057+0000 7f48f8e00640 1 -- 172.21.15.42:0/2182870566 learned_addr learned my addr 172.21.15.42:0/2182870566 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:35:02.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.058+0000 7f48f8e00640 1 -- 172.21.15.42:0/2182870566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eaec724850 con 0x55eaec6b9270 2024-07-04T17:35:02.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.058+0000 7f48f8e00640 1 --2- 172.21.15.42:0/2182870566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eaec6b9270 0x55eaeca5fb70 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f48e4033bb0 tx=0x7f48e403bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34194b1480bbea2a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:35:02.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.058+0000 7f48e9600640 1 -- 172.21.15.42:0/2182870566 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48e4070300 con 0x55eaec6b9270 2024-07-04T17:35:02.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.058+0000 7f48e9600640 1 -- 172.21.15.42:0/2182870566 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48e4003970 con 0x55eaec6b9270 2024-07-04T17:35:02.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.058+0000 7f48e9600640 1 -- 172.21.15.42:0/2182870566 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48e40393a0 con 0x55eaec6b9270 2024-07-04T17:35:02.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.059+0000 7f4902835740 1 -- 172.21.15.42:0/2182870566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eaec6b9270 msgr2=0x55eaeca5fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:35:02.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.059+0000 7f4902835740 1 --2- 172.21.15.42:0/2182870566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eaec6b9270 0x55eaeca5fb70 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f48e4033bb0 tx=0x7f48e403bf90 comp rx=0 tx=0).stop 2024-07-04T17:35:02.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.059+0000 7f4902835740 1 -- 172.21.15.42:0/2182870566 shutdown_connections 2024-07-04T17:35:02.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.059+0000 7f4902835740 1 --2- 172.21.15.42:0/2182870566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eaec6b9270 0x55eaeca5fb70 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:35:02.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.059+0000 7f4902835740 1 -- 172.21.15.42:0/2182870566 >> 172.21.15.42:0/2182870566 conn(0x55eaec6ad3c0 msgr2=0x55eaeca5e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:35:02.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.059+0000 7f4902835740 1 -- 172.21.15.42:0/2182870566 shutdown_connections 2024-07-04T17:35:02.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.059+0000 7f4902835740 1 -- 172.21.15.42:0/2182870566 wait complete. 2024-07-04T17:35:02.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.060+0000 7f4902835740 1 -- start start 2024-07-04T17:35:02.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.060+0000 7f4902835740 1 Processor -- start 2024-07-04T17:35:02.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.062+0000 7f4902835740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eaec6b9270 0x55eaeca82a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:35:02.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.062+0000 7f4902835740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55eaec9c3b90 con 0x55eaec6b9270 2024-07-04T17:35:02.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.062+0000 7f4902835740 10 client.0 _ll_register_callbacks cb 0x55eaec9688a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T17:35:02.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.062+0000 7f48f8e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eaec6b9270 0x55eaeca82a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:35:02.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.062+0000 7f48f8e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eaec6b9270 0x55eaeca82a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56178/0 (socket says 172.21.15.42:56178) 2024-07-04T17:35:02.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.062+0000 7f48f8e00640 1 -- 172.21.15.42:0/2288505856 learned_addr learned my addr 172.21.15.42:0/2288505856 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:35:02.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.062+0000 7f48f8e00640 1 -- 172.21.15.42:0/2288505856 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eaec79b6b0 con 0x55eaec6b9270 2024-07-04T17:35:02.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.063+0000 7f48f8e00640 1 --2- 172.21.15.42:0/2288505856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eaec6b9270 0x55eaeca82a20 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f48e4031fb0 tx=0x7f48e4033b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c031674ad9ae44 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:35:02.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.063+0000 7f48de000640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-07-04T17:35:02.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.063+0000 7f48de000640 1 -- 172.21.15.42:0/2288505856 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48e40393a0 con 0x55eaec6b9270 2024-07-04T17:35:02.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.063+0000 7f48de000640 1 -- 172.21.15.42:0/2288505856 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 4303+0+0 (secure 0 0 0) 0x7f48e4069030 con 0x55eaec6b9270 2024-07-04T17:35:02.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.063+0000 7f4902835740 1 -- 172.21.15.42:0/2288505856 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55eaec9606c0 con 0x55eaec6b9270 2024-07-04T17:35:02.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.063+0000 7f48de000640 1 -- 172.21.15.42:0/2288505856 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f48e405b050 con 0x55eaec6b9270 2024-07-04T17:35:02.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.063+0000 7f4902835740 10 client.5928 fetch_fsmap learned FSMap version 113 2024-07-04T17:35:02.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.063+0000 7f4902835740 1 -- 172.21.15.42:0/2288505856 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x55eaeca80a90 con 0x55eaec6b9270 2024-07-04T17:35:02.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.064+0000 7f48de000640 1 -- 172.21.15.42:0/2288505856 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55eaec9606c0 con 0x55eaec6b9270 2024-07-04T17:35:02.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.064+0000 7f48de000640 1 -- 172.21.15.42:0/2288505856 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 113) ==== 4754+0+0 (secure 0 0 0) 0x7f48e406a9a0 con 0x55eaec6b9270 2024-07-04T17:35:02.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.064+0000 7f4902835740 10 client.5928 fetch_fsmap finished waiting for FSMap version 113 2024-07-04T17:35:02.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.064+0000 7f4902835740 10 client.5928 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-04T17:35:02.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.064+0000 7f4902835740 20 client.5928 populate_metadata read hostname 'smithi042' 2024-07-04T17:35:02.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.064+0000 7f4902835740 1 --2- 172.21.15.42:0/2288505856 >> [v2:172.21.15.42:6830/4064998265,v1:172.21.15.42:6831/4064998265] conn(0x55eaec6903b0 0x55eaeca87000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:35:02.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.064+0000 7f4902835740 4 client.5928 mds_command: new command op to 5817 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T17:35:02.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.064+0000 7f4902835740 1 -- 172.21.15.42:0/2288505856 --> [v2:172.21.15.42:6830/4064998265,v1:172.21.15.42:6831/4064998265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eaec9d2500 con 0x55eaec6903b0 2024-07-04T17:35:02.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.064+0000 7f48f3e00640 1 --2- 172.21.15.42:0/2288505856 >> [v2:172.21.15.42:6830/4064998265,v1:172.21.15.42:6831/4064998265] conn(0x55eaec6903b0 0x55eaeca87000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:35:02.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.065+0000 7f48f3e00640 1 --2- 172.21.15.42:0/2288505856 >> [v2:172.21.15.42:6830/4064998265,v1:172.21.15.42:6831/4064998265] conn(0x55eaec6903b0 0x55eaeca87000 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfdf9c73e3b56789 server_cookie=ba062fbd703a9148 in_seq=0 out_seq=0 2024-07-04T17:35:02.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.065+0000 7f48de000640 10 client.5928 ms_handle_connect on v2:172.21.15.42:6830/4064998265 2024-07-04T17:35:02.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.066+0000 7f48de000640 1 -- 172.21.15.42:0/2288505856 <== mds.0 v2:172.21.15.42:6830/4064998265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55eaec9d2500 con 0x55eaec6903b0 2024-07-04T17:35:02.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.066+0000 7f48de000640 10 client.5928 handle_command_reply: tid=0 2024-07-04T17:35:02.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.066+0000 7f4902835740 1 client.5928 shutdown 2024-07-04T17:35:02.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.066+0000 7f4902835740 1 -- 172.21.15.42:0/2288505856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eaec6b9270 msgr2=0x55eaeca82a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:35:02.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.066+0000 7f4902835740 1 --2- 172.21.15.42:0/2288505856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eaec6b9270 0x55eaeca82a20 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f48e4031fb0 tx=0x7f48e4033b90 comp rx=0 tx=0).stop 2024-07-04T17:35:02.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.066+0000 7f4902835740 1 -- 172.21.15.42:0/2288505856 shutdown_connections 2024-07-04T17:35:02.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.066+0000 7f4902835740 1 --2- 172.21.15.42:0/2288505856 >> [v2:172.21.15.42:6830/4064998265,v1:172.21.15.42:6831/4064998265] conn(0x55eaec6903b0 0x55eaeca87000 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:35:02.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.066+0000 7f4902835740 1 --2- 172.21.15.42:0/2288505856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eaec6b9270 0x55eaeca82a20 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:35:02.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.066+0000 7f4902835740 1 -- 172.21.15.42:0/2288505856 >> 172.21.15.42:0/2288505856 conn(0x55eaec6ad3c0 msgr2=0x55eaeca59080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:35:02.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.066+0000 7f48de000640 0 client.5928 ms_handle_reset on v2:172.21.15.42:6830/4064998265 2024-07-04T17:35:02.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.066+0000 7f4902835740 1 -- 172.21.15.42:0/2288505856 shutdown_connections 2024-07-04T17:35:02.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.066+0000 7f4902835740 1 -- 172.21.15.42:0/2288505856 wait complete. 2024-07-04T17:35:02.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.067+0000 7f4902835740 20 client.5928 trim_cache size 0 max 16384 2024-07-04T17:35:02.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.088+0000 7f4902835740 1 Processor -- start 2024-07-04T17:35:02.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.088+0000 7f4902835740 1 -- start start 2024-07-04T17:35:02.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.089+0000 7f4902835740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eaec6b9270 0x55eaeca5b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:35:02.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.089+0000 7f4902835740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f48f413cbb0 con 0x55eaec6b9270 2024-07-04T17:35:02.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.089+0000 7f48f8e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eaec6b9270 0x55eaeca5b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:35:02.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.090+0000 7f48f8e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eaec6b9270 0x55eaeca5b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56192/0 (socket says 172.21.15.42:56192) 2024-07-04T17:35:02.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.090+0000 7f48f8e00640 1 -- 172.21.15.42:0/2864787175 learned_addr learned my addr 172.21.15.42:0/2864787175 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:35:02.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.090+0000 7f48f8e00640 1 -- 172.21.15.42:0/2864787175 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eaeca5fee0 con 0x55eaec6b9270 2024-07-04T17:35:02.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.090+0000 7f48f8e00640 1 --2- 172.21.15.42:0/2864787175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eaec6b9270 0x55eaeca5b9e0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f48e4039c70 tx=0x7f48e403bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4f1e30c153e5f5 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:35:02.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.090+0000 7f48e8c00640 1 -- 172.21.15.42:0/2864787175 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48e40035f0 con 0x55eaec6b9270 2024-07-04T17:35:02.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.091+0000 7f48e8c00640 1 -- 172.21.15.42:0/2864787175 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48e406aab0 con 0x55eaec6b9270 2024-07-04T17:35:02.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.091+0000 7f48e8c00640 1 -- 172.21.15.42:0/2864787175 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48e4039890 con 0x55eaec6b9270 2024-07-04T17:35:02.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.091+0000 7f4902835740 1 -- 172.21.15.42:0/2864787175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eaec6b9270 msgr2=0x55eaeca5b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:35:02.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.091+0000 7f4902835740 1 --2- 172.21.15.42:0/2864787175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eaec6b9270 0x55eaeca5b9e0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f48e4039c70 tx=0x7f48e403bf90 comp rx=0 tx=0).stop 2024-07-04T17:35:02.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.091+0000 7f4902835740 1 -- 172.21.15.42:0/2864787175 shutdown_connections 2024-07-04T17:35:02.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.091+0000 7f4902835740 1 --2- 172.21.15.42:0/2864787175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eaec6b9270 0x55eaeca5b9e0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:35:02.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.091+0000 7f4902835740 1 -- 172.21.15.42:0/2864787175 >> 172.21.15.42:0/2864787175 conn(0x55eaec6ad3c0 msgr2=0x55eaeca8c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:35:02.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.091+0000 7f4902835740 1 -- 172.21.15.42:0/2864787175 shutdown_connections 2024-07-04T17:35:02.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.091+0000 7f4902835740 1 -- 172.21.15.42:0/2864787175 wait complete. 2024-07-04T17:35:02.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.092+0000 7f4902835740 1 -- start start 2024-07-04T17:35:02.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.093+0000 7f4902835740 1 Processor -- start 2024-07-04T17:35:02.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.094+0000 7f4902835740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eaec6b9270 0x55eaecaaf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:35:02.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.094+0000 7f4902835740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55eaec9bfb50 con 0x55eaec6b9270 2024-07-04T17:35:02.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.094+0000 7f4902835740 10 client.0 _ll_register_callbacks cb 0x7f48e4009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T17:35:02.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.094+0000 7f48f8e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eaec6b9270 0x55eaecaaf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:35:02.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.094+0000 7f48f8e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eaec6b9270 0x55eaecaaf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56206/0 (socket says 172.21.15.42:56206) 2024-07-04T17:35:02.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.094+0000 7f48f8e00640 1 -- 172.21.15.42:0/1830605494 learned_addr learned my addr 172.21.15.42:0/1830605494 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:35:02.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.095+0000 7f48f8e00640 1 -- 172.21.15.42:0/1830605494 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eaeca82d90 con 0x55eaec6b9270 2024-07-04T17:35:02.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.095+0000 7f48f8e00640 1 --2- 172.21.15.42:0/1830605494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eaec6b9270 0x55eaecaaf690 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f48e4031fb0 tx=0x7f48e4071bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f940a505a125978b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:35:02.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.095+0000 7f48de000640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-07-04T17:35:02.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.095+0000 7f48de000640 1 -- 172.21.15.42:0/1830605494 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48e407a020 con 0x55eaec6b9270 2024-07-04T17:35:02.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.095+0000 7f48de000640 1 -- 172.21.15.42:0/1830605494 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 4303+0+0 (secure 0 0 0) 0x7f48e4069040 con 0x55eaec6b9270 2024-07-04T17:35:02.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.095+0000 7f4902835740 1 -- 172.21.15.42:0/1830605494 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48e406aab0 con 0x55eaec6b9270 2024-07-04T17:35:02.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.096+0000 7f48de000640 1 -- 172.21.15.42:0/1830605494 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f48e4033360 con 0x55eaec6b9270 2024-07-04T17:35:02.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.096+0000 7f4902835740 10 client.5930 fetch_fsmap learned FSMap version 113 2024-07-04T17:35:02.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.096+0000 7f4902835740 1 -- 172.21.15.42:0/1830605494 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x55eaeca82d90 con 0x55eaec6b9270 2024-07-04T17:35:02.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.096+0000 7f48de000640 1 -- 172.21.15.42:0/1830605494 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48e406aab0 con 0x55eaec6b9270 2024-07-04T17:35:02.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.096+0000 7f48de000640 1 -- 172.21.15.42:0/1830605494 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 113) ==== 4754+0+0 (secure 0 0 0) 0x7f48e406a2d0 con 0x55eaec6b9270 2024-07-04T17:35:02.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.096+0000 7f4902835740 10 client.5930 fetch_fsmap finished waiting for FSMap version 113 2024-07-04T17:35:02.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.096+0000 7f4902835740 10 client.5930 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-04T17:35:02.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.096+0000 7f4902835740 20 client.5930 populate_metadata read hostname 'smithi042' 2024-07-04T17:35:02.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.096+0000 7f4902835740 1 --2- 172.21.15.42:0/1830605494 >> [v2:172.21.15.42:6830/4064998265,v1:172.21.15.42:6831/4064998265] conn(0x55eaec677e40 0x55eaecb08c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:35:02.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.096+0000 7f4902835740 4 client.5930 mds_command: new command op to 5817 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-04T17:35:02.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.096+0000 7f4902835740 1 -- 172.21.15.42:0/1830605494 --> [v2:172.21.15.42:6830/4064998265,v1:172.21.15.42:6831/4064998265] -- command(tid 0: {"prefix": "status"}) -- 0x55eaeca97c80 con 0x55eaec677e40 2024-07-04T17:35:02.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.096+0000 7f48f3e00640 1 --2- 172.21.15.42:0/1830605494 >> [v2:172.21.15.42:6830/4064998265,v1:172.21.15.42:6831/4064998265] conn(0x55eaec677e40 0x55eaecb08c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:35:02.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.097+0000 7f48f3e00640 1 --2- 172.21.15.42:0/1830605494 >> [v2:172.21.15.42:6830/4064998265,v1:172.21.15.42:6831/4064998265] conn(0x55eaec677e40 0x55eaecb08c60 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc6f4bd80bfed6a0 server_cookie=973690df21ea641c in_seq=0 out_seq=0 2024-07-04T17:35:02.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.097+0000 7f48de000640 10 client.5930 ms_handle_connect on v2:172.21.15.42:6830/4064998265 2024-07-04T17:35:02.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.097+0000 7f48de000640 1 -- 172.21.15.42:0/1830605494 <== mds.0 v2:172.21.15.42:6830/4064998265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+337 (crc 0 0 0) 0x55eaeca97c80 con 0x55eaec677e40 2024-07-04T17:35:02.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.097+0000 7f48de000640 10 client.5930 handle_command_reply: tid=0 2024-07-04T17:35:02.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.097+0000 7f4902835740 1 client.5930 shutdown 2024-07-04T17:35:02.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.098+0000 7f4902835740 1 -- 172.21.15.42:0/1830605494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eaec6b9270 msgr2=0x55eaecaaf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:35:02.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.098+0000 7f4902835740 1 --2- 172.21.15.42:0/1830605494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eaec6b9270 0x55eaecaaf690 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f48e4031fb0 tx=0x7f48e4071bb0 comp rx=0 tx=0).stop 2024-07-04T17:35:02.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.098+0000 7f4902835740 1 -- 172.21.15.42:0/1830605494 shutdown_connections 2024-07-04T17:35:02.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.098+0000 7f4902835740 1 --2- 172.21.15.42:0/1830605494 >> [v2:172.21.15.42:6830/4064998265,v1:172.21.15.42:6831/4064998265] conn(0x55eaec677e40 0x55eaecb08c60 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:35:02.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.098+0000 7f4902835740 1 --2- 172.21.15.42:0/1830605494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55eaec6b9270 0x55eaecaaf690 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:35:02.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.098+0000 7f4902835740 1 -- 172.21.15.42:0/1830605494 >> 172.21.15.42:0/1830605494 conn(0x55eaec6ad3c0 msgr2=0x55eaeca5b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:35:02.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.098+0000 7f48de000640 0 client.5930 ms_handle_reset on v2:172.21.15.42:6830/4064998265 2024-07-04T17:35:02.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.098+0000 7f4902835740 1 -- 172.21.15.42:0/1830605494 shutdown_connections 2024-07-04T17:35:02.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.098+0000 7f4902835740 1 -- 172.21.15.42:0/1830605494 wait complete. 2024-07-04T17:35:02.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.098+0000 7f4902835740 20 client.5930 trim_cache size 0 max 16384 2024-07-04T17:35:02.101 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:35:02.101 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "be4f300a-93ef-4489-b540-37ece39b0aac", 2024-07-04T17:35:02.101 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-07-04T17:35:02.101 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 5817, 2024-07-04T17:35:02.101 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-07-04T17:35:02.101 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-07-04T17:35:02.101 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "backup_fs", 2024-07-04T17:35:02.101 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 6.8864060760000001, 2024-07-04T17:35:02.101 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 113, 2024-07-04T17:35:02.101 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 140, 2024-07-04T17:35:02.101 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 140, 2024-07-04T17:35:02.101 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 31.434827944999999 2024-07-04T17:35:02.101 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:35:02.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.099+0000 7f48fb800640 1 -- 172.21.15.42:0/4058145367 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f48d403d540 msgr2=0x7f48d403fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:35:02.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.099+0000 7f48fb800640 1 --2- 172.21.15.42:0/4058145367 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f48d403d540 0x7f48d403fa50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f48e0002020 tx=0x7f48e0001610 comp rx=0 tx=0).stop 2024-07-04T17:35:02.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.099+0000 7f48fb800640 1 -- 172.21.15.42:0/4058145367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48f40b02b0 msgr2=0x7f48f41d6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:35:02.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.099+0000 7f48fb800640 1 --2- 172.21.15.42:0/4058145367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48f40b02b0 0x7f48f41d6f60 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f48e40098d0 tx=0x7f48e4033ad0 comp rx=0 tx=0).stop 2024-07-04T17:35:02.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.099+0000 7f48fb800640 1 -- 172.21.15.42:0/4058145367 shutdown_connections 2024-07-04T17:35:02.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.099+0000 7f48fb800640 1 --2- 172.21.15.42:0/4058145367 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7f48d403d540 0x7f48d403fa50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:35:02.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.099+0000 7f48fb800640 1 --2- 172.21.15.42:0/4058145367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48f40b02b0 0x7f48f41d6f60 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:35:02.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.099+0000 7f48fb800640 1 -- 172.21.15.42:0/4058145367 >> 172.21.15.42:0/4058145367 conn(0x7f48f40bf700 msgr2=0x7f48f4075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:35:02.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.099+0000 7f48fb800640 1 -- 172.21.15.42:0/4058145367 shutdown_connections 2024-07-04T17:35:02.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.099+0000 7f48fb800640 1 -- 172.21.15.42:0/4058145367 wait complete. 2024-07-04T17:35:02.125 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable mirroring 2024-07-04T17:35:02.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.413+0000 7fcb5fe00640 1 -- 172.21.15.42:0/1155892039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb600abda0 msgr2=0x7fcb600a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:35:02.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.413+0000 7fcb5fe00640 1 --2- 172.21.15.42:0/1155892039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb600abda0 0x7fcb600a94d0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fcb480097b0 tx=0x7fcb4802a4a0 comp rx=0 tx=0).stop 2024-07-04T17:35:02.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.413+0000 7fcb5fe00640 1 -- 172.21.15.42:0/1155892039 shutdown_connections 2024-07-04T17:35:02.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.413+0000 7fcb5fe00640 1 --2- 172.21.15.42:0/1155892039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb600abda0 0x7fcb600a94d0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:35:02.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.413+0000 7fcb5fe00640 1 -- 172.21.15.42:0/1155892039 >> 172.21.15.42:0/1155892039 conn(0x7fcb600afaa0 msgr2=0x7fcb600afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:35:02.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.413+0000 7fcb5fe00640 1 -- 172.21.15.42:0/1155892039 shutdown_connections 2024-07-04T17:35:02.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.413+0000 7fcb5fe00640 1 -- 172.21.15.42:0/1155892039 wait complete. 2024-07-04T17:35:02.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.415+0000 7fcb5fe00640 1 Processor -- start 2024-07-04T17:35:02.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.415+0000 7fcb5fe00640 1 -- start start 2024-07-04T17:35:02.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.416+0000 7fcb5fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb60070240 0x7fcb600b3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:35:02.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.417+0000 7fcb5fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6013f510 con 0x7fcb60070240 2024-07-04T17:35:02.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.417+0000 7fcb5ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb60070240 0x7fcb600b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:35:02.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.417+0000 7fcb5ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb60070240 0x7fcb600b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56220/0 (socket says 172.21.15.42:56220) 2024-07-04T17:35:02.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.417+0000 7fcb5ea00640 1 -- 172.21.15.42:0/466410006 learned_addr learned my addr 172.21.15.42:0/466410006 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:35:02.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.417+0000 7fcb5ea00640 1 -- 172.21.15.42:0/466410006 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb601d51b0 con 0x7fcb60070240 2024-07-04T17:35:02.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.418+0000 7fcb5ea00640 1 --2- 172.21.15.42:0/466410006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb60070240 0x7fcb600b3560 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000c310 tx=0x7fcb5000c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:35:02.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.418+0000 7fcb56a00640 1 -- 172.21.15.42:0/466410006 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb500116c0 con 0x7fcb60070240 2024-07-04T17:35:02.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.418+0000 7fcb56a00640 1 -- 172.21.15.42:0/466410006 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb50017070 con 0x7fcb60070240 2024-07-04T17:35:02.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.418+0000 7fcb56a00640 1 -- 172.21.15.42:0/466410006 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb50011b60 con 0x7fcb60070240 2024-07-04T17:35:02.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.418+0000 7fcb5fe00640 1 -- 172.21.15.42:0/466410006 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb601e6f10 con 0x7fcb60070240 2024-07-04T17:35:02.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.418+0000 7fcb5fe00640 1 -- 172.21.15.42:0/466410006 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb601e6c40 con 0x7fcb60070240 2024-07-04T17:35:02.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.419+0000 7fcb5fe00640 1 -- 172.21.15.42:0/466410006 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb6013fbc0 con 0x7fcb60070240 2024-07-04T17:35:02.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.420+0000 7fcb56a00640 1 -- 172.21.15.42:0/466410006 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 52) ==== 49615+0+0 (secure 0 0 0) 0x7fcb50011d00 con 0x7fcb60070240 2024-07-04T17:35:02.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.420+0000 7fcb56a00640 1 --2- 172.21.15.42:0/466410006 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fcb3c03d4f0 0x7fcb3c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:35:02.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.420+0000 7fcb56a00640 1 -- 172.21.15.42:0/466410006 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 4303+0+0 (secure 0 0 0) 0x7fcb50051920 con 0x7fcb60070240 2024-07-04T17:35:02.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.420+0000 7fcb57e00640 1 --2- 172.21.15.42:0/466410006 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fcb3c03d4f0 0x7fcb3c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:35:02.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.421+0000 7fcb57e00640 1 --2- 172.21.15.42:0/466410006 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fcb3c03d4f0 0x7fcb3c03fa00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcb48034000 tx=0x7fcb48004730 comp rx=0 tx=0).ready entity=mgr.5692 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:35:02.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.426+0000 7fcb56a00640 1 -- 172.21.15.42:0/466410006 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fcb5000a140 con 0x7fcb60070240 2024-07-04T17:35:02.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:02.692+0000 7fcb5fe00640 1 -- 172.21.15.42:0/466410006 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "mirroring"} v 0) -- 0x7fcb6014f8d0 con 0x7fcb60070240 2024-07-04T17:35:03.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.273+0000 7fcb56a00640 1 -- 172.21.15.42:0/466410006 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "mirroring"}]=0 v53) ==== 88+0+0 (secure 0 0 0) 0x7fcb5001ac60 con 0x7fcb60070240 2024-07-04T17:35:03.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.278+0000 7fcb5fe00640 1 -- 172.21.15.42:0/466410006 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fcb3c03d4f0 msgr2=0x7fcb3c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:35:03.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.278+0000 7fcb5fe00640 1 --2- 172.21.15.42:0/466410006 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fcb3c03d4f0 0x7fcb3c03fa00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcb48034000 tx=0x7fcb48004730 comp rx=0 tx=0).stop 2024-07-04T17:35:03.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.278+0000 7fcb5fe00640 1 -- 172.21.15.42:0/466410006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb60070240 msgr2=0x7fcb600b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:35:03.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.278+0000 7fcb5fe00640 1 --2- 172.21.15.42:0/466410006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb60070240 0x7fcb600b3560 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000c310 tx=0x7fcb5000c7d0 comp rx=0 tx=0).stop 2024-07-04T17:35:03.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.278+0000 7fcb5fe00640 1 -- 172.21.15.42:0/466410006 shutdown_connections 2024-07-04T17:35:03.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.278+0000 7fcb5fe00640 1 --2- 172.21.15.42:0/466410006 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fcb3c03d4f0 0x7fcb3c03fa00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:35:03.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.278+0000 7fcb5fe00640 1 --2- 172.21.15.42:0/466410006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb60070240 0x7fcb600b3560 secure :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000c310 tx=0x7fcb5000c7d0 comp rx=0 tx=0).stop 2024-07-04T17:35:03.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.278+0000 7fcb5fe00640 1 -- 172.21.15.42:0/466410006 >> 172.21.15.42:0/466410006 conn(0x7fcb600afaa0 msgr2=0x7fcb600aa900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:35:03.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.282+0000 7fcb5fe00640 1 -- 172.21.15.42:0/466410006 shutdown_connections 2024-07-04T17:35:03.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.283+0000 7fcb5fe00640 1 -- 172.21.15.42:0/466410006 wait complete. 2024-07-04T17:35:03.329 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:35:03.329 DEBUG:teuthology.orchestra.run.smithi042:counter dump for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok counter dump 2024-07-04T17:35:03.552 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:35:03.552 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker": [ 2024-07-04T17:35:03.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:03.553 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:35:03.553 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "0" 2024-07-04T17:35:03.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:03.553 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:03.553 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:35:03.553 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:35:03.553 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:03.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:03.553 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:03.553 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:35:03.553 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "1" 2024-07-04T17:35:03.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:03.554 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:03.554 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:35:03.554 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:35:03.554 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:03.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:03.554 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:03.554 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:35:03.554 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "2" 2024-07-04T17:35:03.554 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:03.554 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:03.554 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:35:03.554 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:35:03.555 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:03.555 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:03.555 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:03.555 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": [ 2024-07-04T17:35:03.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:03.555 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:03.555 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:03.555 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 112, 2024-07-04T17:35:03.555 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 6, 2024-07-04T17:35:03.555 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 459376, 2024-07-04T17:35:03.555 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 1440, 2024-07-04T17:35:03.555 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:35:03.556 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:35:03.556 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.041576179, 2024-07-04T17:35:03.556 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.004906519, 2024-07-04T17:35:03.556 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.022135410, 2024-07-04T17:35:03.556 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.012265884, 2024-07-04T17:35:03.556 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:35:03.556 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 6, 2024-07-04T17:35:03.556 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.004539022, 2024-07-04T17:35:03.556 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000756503 2024-07-04T17:35:03.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:03.556 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:35:03.557 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:03.557 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:03.557 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:03.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:03.557 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 459376, 2024-07-04T17:35:03.557 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 1440 2024-07-04T17:35:03.557 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:03.557 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:03.557 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:03.557 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": [ 2024-07-04T17:35:03.557 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:03.558 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:03.558 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:03.558 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 10, 2024-07-04T17:35:03.558 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 117, 2024-07-04T17:35:03.558 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 1440, 2024-07-04T17:35:03.558 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 829264, 2024-07-04T17:35:03.558 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 22, 2024-07-04T17:35:03.558 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:35:03.558 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.052794638, 2024-07-04T17:35:03.558 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.017004208, 2024-07-04T17:35:03.558 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.017946178, 2024-07-04T17:35:03.559 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:35:03.559 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:35:03.559 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 107, 2024-07-04T17:35:03.559 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.094742013, 2024-07-04T17:35:03.559 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000885439 2024-07-04T17:35:03.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:03.559 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:35:03.559 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:03.559 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:03.559 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:03.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:03.559 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 1440, 2024-07-04T17:35:03.560 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 829264 2024-07-04T17:35:03.560 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:03.560 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:03.560 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:03.560 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": [ 2024-07-04T17:35:03.560 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:03.560 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:03.560 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:03.560 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 59, 2024-07-04T17:35:03.560 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 9, 2024-07-04T17:35:03.560 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 2436656, 2024-07-04T17:35:03.561 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 1904, 2024-07-04T17:35:03.561 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:35:03.561 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:35:03.561 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.069173247, 2024-07-04T17:35:03.561 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.004855749, 2024-07-04T17:35:03.561 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.118435574, 2024-07-04T17:35:03.561 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:35:03.561 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:35:03.561 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 9, 2024-07-04T17:35:03.561 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.003402160, 2024-07-04T17:35:03.561 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000378017 2024-07-04T17:35:03.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:03.562 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:35:03.562 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:03.562 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:03.562 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:03.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:03.562 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 2436656, 2024-07-04T17:35:03.562 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 1904 2024-07-04T17:35:03.562 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:03.562 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:03.562 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:03.563 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": [ 2024-07-04T17:35:03.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:03.563 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:03.563 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:03.563 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-07-04T17:35:03.563 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-07-04T17:35:03.563 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:03.563 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:03.563 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:03.563 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror": [ 2024-07-04T17:35:03.563 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:03.564 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:03.564 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:03.564 INFO:teuthology.orchestra.run.smithi042.stdout: "mirrored_filesystems": 0, 2024-07-04T17:35:03.564 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_enable_failures": 0 2024-07-04T17:35:03.564 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:03.564 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:03.564 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:03.564 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": [ 2024-07-04T17:35:03.564 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:03.564 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:03.564 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:03.564 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-07-04T17:35:03.564 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-07-04T17:35:03.565 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T17:35:03.565 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-07-04T17:35:03.565 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T17:35:03.565 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-07-04T17:35:03.565 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T17:35:03.565 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T17:35:03.565 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T17:35:03.565 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T17:35:03.565 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T17:35:03.565 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-07-04T17:35:03.565 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T17:35:03.565 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T17:35:03.566 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-07-04T17:35:03.566 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-07-04T17:35:03.566 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-07-04T17:35:03.566 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-07-04T17:35:03.566 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T17:35:03.566 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T17:35:03.566 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T17:35:03.566 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T17:35:03.566 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T17:35:03.566 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-07-04T17:35:03.566 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-07-04T17:35:03.567 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-07-04T17:35:03.567 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T17:35:03.567 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T17:35:03.567 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-07-04T17:35:03.567 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-07-04T17:35:03.567 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-07-04T17:35:03.567 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-07-04T17:35:03.567 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T17:35:03.567 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-07-04T17:35:03.567 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T17:35:03.567 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-07-04T17:35:03.567 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 148170, 2024-07-04T17:35:03.568 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 29, 2024-07-04T17:35:03.568 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-07-04T17:35:03.568 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-07-04T17:35:03.568 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-07-04T17:35:03.568 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-07-04T17:35:03.568 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-07-04T17:35:03.568 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-07-04T17:35:03.568 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-07-04T17:35:03.568 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-07-04T17:35:03.568 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 4592, 2024-07-04T17:35:03.568 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 37, 2024-07-04T17:35:03.569 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T17:35:03.569 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-07-04T17:35:03.569 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-07-04T17:35:03.569 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-07-04T17:35:03.569 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 0, 2024-07-04T17:35:03.569 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 0, 2024-07-04T17:35:03.569 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-07-04T17:35:03.569 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-07-04T17:35:03.569 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-07-04T17:35:03.569 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-07-04T17:35:03.569 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:03.569 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:03.570 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:03.570 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter": [ 2024-07-04T17:35:03.570 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:03.570 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:03.570 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:03.570 INFO:teuthology.orchestra.run.smithi042.stdout: "op_active": 0, 2024-07-04T17:35:03.570 INFO:teuthology.orchestra.run.smithi042.stdout: "op_laggy": 0, 2024-07-04T17:35:03.570 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send": 0, 2024-07-04T17:35:03.570 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send_bytes": 0, 2024-07-04T17:35:03.570 INFO:teuthology.orchestra.run.smithi042.stdout: "op_resend": 0, 2024-07-04T17:35:03.570 INFO:teuthology.orchestra.run.smithi042.stdout: "op_reply": 0, 2024-07-04T17:35:03.570 INFO:teuthology.orchestra.run.smithi042.stdout: "op_latency": { 2024-07-04T17:35:03.571 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:03.571 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:03.571 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:03.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:03.571 INFO:teuthology.orchestra.run.smithi042.stdout: "op_inflight": 0, 2024-07-04T17:35:03.571 INFO:teuthology.orchestra.run.smithi042.stdout: "oplen_avg": { 2024-07-04T17:35:03.571 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:03.571 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0 2024-07-04T17:35:03.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:03.571 INFO:teuthology.orchestra.run.smithi042.stdout: "op": 0, 2024-07-04T17:35:03.571 INFO:teuthology.orchestra.run.smithi042.stdout: "op_r": 0, 2024-07-04T17:35:03.571 INFO:teuthology.orchestra.run.smithi042.stdout: "op_w": 0, 2024-07-04T17:35:03.571 INFO:teuthology.orchestra.run.smithi042.stdout: "op_rmw": 0, 2024-07-04T17:35:03.572 INFO:teuthology.orchestra.run.smithi042.stdout: "op_pg": 0, 2024-07-04T17:35:03.572 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_stat": 0, 2024-07-04T17:35:03.572 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_create": 0, 2024-07-04T17:35:03.572 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_read": 0, 2024-07-04T17:35:03.572 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_write": 0, 2024-07-04T17:35:03.572 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writefull": 0, 2024-07-04T17:35:03.572 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writesame": 0, 2024-07-04T17:35:03.572 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_append": 0, 2024-07-04T17:35:03.572 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_zero": 0, 2024-07-04T17:35:03.572 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_truncate": 0, 2024-07-04T17:35:03.572 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_delete": 0, 2024-07-04T17:35:03.572 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_mapext": 0, 2024-07-04T17:35:03.572 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_sparse_read": 0, 2024-07-04T17:35:03.573 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_clonerange": 0, 2024-07-04T17:35:03.573 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_getxattr": 0, 2024-07-04T17:35:03.573 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_setxattr": 0, 2024-07-04T17:35:03.573 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_cmpxattr": 0, 2024-07-04T17:35:03.573 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_rmxattr": 0, 2024-07-04T17:35:03.573 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_resetxattrs": 0, 2024-07-04T17:35:03.573 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_call": 0, 2024-07-04T17:35:03.573 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_watch": 0, 2024-07-04T17:35:03.573 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_notify": 0, 2024-07-04T17:35:03.573 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T17:35:03.573 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls": 0, 2024-07-04T17:35:03.573 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls_filter": 0, 2024-07-04T17:35:03.574 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_other": 0, 2024-07-04T17:35:03.574 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_active": 0, 2024-07-04T17:35:03.574 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_send": 0, 2024-07-04T17:35:03.574 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_resend": 0, 2024-07-04T17:35:03.574 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_ping": 0, 2024-07-04T17:35:03.574 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_active": 0, 2024-07-04T17:35:03.574 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_send": 0, 2024-07-04T17:35:03.574 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_resend": 0, 2024-07-04T17:35:03.574 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_active": 0, 2024-07-04T17:35:03.574 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_send": 0, 2024-07-04T17:35:03.575 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_resend": 0, 2024-07-04T17:35:03.575 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_active": 0, 2024-07-04T17:35:03.575 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_send": 0, 2024-07-04T17:35:03.575 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_resend": 0, 2024-07-04T17:35:03.575 INFO:teuthology.orchestra.run.smithi042.stdout: "command_active": 0, 2024-07-04T17:35:03.575 INFO:teuthology.orchestra.run.smithi042.stdout: "command_send": 0, 2024-07-04T17:35:03.575 INFO:teuthology.orchestra.run.smithi042.stdout: "command_resend": 0, 2024-07-04T17:35:03.575 INFO:teuthology.orchestra.run.smithi042.stdout: "map_epoch": 0, 2024-07-04T17:35:03.575 INFO:teuthology.orchestra.run.smithi042.stdout: "map_full": 0, 2024-07-04T17:35:03.575 INFO:teuthology.orchestra.run.smithi042.stdout: "map_inc": 0, 2024-07-04T17:35:03.575 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_sessions": 0, 2024-07-04T17:35:03.575 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_open": 0, 2024-07-04T17:35:03.575 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_close": 0, 2024-07-04T17:35:03.576 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_laggy": 0, 2024-07-04T17:35:03.576 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_wr": 0, 2024-07-04T17:35:03.576 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_rd": 0, 2024-07-04T17:35:03.576 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_del": 0 2024-07-04T17:35:03.576 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:03.576 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:03.576 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:03.576 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-client": [ 2024-07-04T17:35:03.576 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:03.576 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:03.577 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:03.577 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:35:03.577 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:35:03.577 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:35:03.577 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 112, 2024-07-04T17:35:03.577 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 444064, 2024-07-04T17:35:03.577 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:35:03.577 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 112, 2024-07-04T17:35:03.577 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:35:03.577 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:35:03.577 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 112, 2024-07-04T17:35:03.577 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 444064, 2024-07-04T17:35:03.578 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:35:03.578 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:03.578 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:03.578 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:03.578 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:03.578 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:03.578 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:03.578 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:03.578 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-radosclient": [ 2024-07-04T17:35:03.578 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:03.578 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:03.579 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:03.579 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:35:03.579 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:35:03.579 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:35:03.579 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 66, 2024-07-04T17:35:03.579 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 2428224, 2024-07-04T17:35:03.579 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:35:03.579 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 66, 2024-07-04T17:35:03.579 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:35:03.579 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:35:03.579 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 66, 2024-07-04T17:35:03.580 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 2428224, 2024-07-04T17:35:03.580 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:35:03.580 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:03.580 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:03.580 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:03.580 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:03.580 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:03.580 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:03.580 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:03.580 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_bytes": [ 2024-07-04T17:35:03.580 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:03.580 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:03.581 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:03.581 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:35:03.581 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:35:03.581 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:35:03.581 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:35:03.581 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:35:03.581 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:35:03.581 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:35:03.581 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:35:03.581 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:35:03.581 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:35:03.581 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:35:03.582 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:35:03.582 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:03.582 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:03.582 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:03.582 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:03.582 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:03.582 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:03.582 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:03.582 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_ops": [ 2024-07-04T17:35:03.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:03.582 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:03.582 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:03.583 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:35:03.583 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1024, 2024-07-04T17:35:03.583 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:35:03.583 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:35:03.583 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:35:03.583 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:35:03.583 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:35:03.583 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:35:03.583 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:35:03.583 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:35:03.583 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:35:03.583 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:35:03.583 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:03.584 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:03.584 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:03.584 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:03.584 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:03.584 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:03.584 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-07-04T17:35:03.584 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:35:03.585 DEBUG:tasks.cephfs.test_mirroring:command returned={ "AsyncMessenger::Worker": [ { "labels": { "id": "0" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "1" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "2" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } } ], "AsyncMessenger::Worker-0": [ { "labels": {}, "counters": { "msgr_recv_messages": 112, "msgr_send_messages": 6, "msgr_recv_bytes": 459376, "msgr_send_bytes": 1440, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.041576179, "msgr_running_send_time": 0.004906519, "msgr_running_recv_time": 0.022135410, "msgr_running_fast_dispatch_time": 0.012265884, "msgr_send_messages_queue_lat": { "avgcount": 6, "sum": 0.004539022, "avgtime": 0.000756503 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 459376, "msgr_send_encrypted_bytes": 1440 } } ], "AsyncMessenger::Worker-1": [ { "labels": {}, "counters": { "msgr_recv_messages": 10, "msgr_send_messages": 117, "msgr_recv_bytes": 1440, "msgr_send_bytes": 829264, "msgr_created_connections": 22, "msgr_active_connections": 1, "msgr_running_total_time": 0.052794638, "msgr_running_send_time": 0.017004208, "msgr_running_recv_time": 0.017946178, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 107, "sum": 0.094742013, "avgtime": 0.000885439 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 1440, "msgr_send_encrypted_bytes": 829264 } } ], "AsyncMessenger::Worker-2": [ { "labels": {}, "counters": { "msgr_recv_messages": 59, "msgr_send_messages": 9, "msgr_recv_bytes": 2436656, "msgr_send_bytes": 1904, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.069173247, "msgr_running_send_time": 0.004855749, "msgr_running_recv_time": 0.118435574, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 9, "sum": 0.003402160, "avgtime": 0.000378017 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 2436656, "msgr_send_encrypted_bytes": 1904 } } ], "cct": [ { "labels": {}, "counters": { "total_workers": 0, "unhealthy_workers": 0 } } ], "cephfs_mirror": [ { "labels": {}, "counters": { "mirrored_filesystems": 0, "mirror_enable_failures": 0 } } ], "mempool": [ { "labels": {}, "counters": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_items": 0, "bluefs_bytes": 0, "bluefs_items": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "buffer_anon_bytes": 148170, "buffer_anon_items": 29, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 4592, "osdmap_items": 37, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "mds_co_bytes": 0, "mds_co_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 } } ], "objecter": [ { "labels": {}, "counters": { "op_active": 0, "op_laggy": 0, "op_send": 0, "op_send_bytes": 0, "op_resend": 0, "op_reply": 0, "op_latency": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "op_inflight": 0, "oplen_avg": { "avgcount": 0, "sum": 0 }, "op": 0, "op_r": 0, "op_w": 0, "op_rmw": 0, "op_pg": 0, "osdop_stat": 0, "osdop_create": 0, "osdop_read": 0, "osdop_write": 0, "osdop_writefull": 0, "osdop_writesame": 0, "osdop_append": 0, "osdop_zero": 0, "osdop_truncate": 0, "osdop_delete": 0, "osdop_mapext": 0, "osdop_sparse_read": 0, "osdop_clonerange": 0, "osdop_getxattr": 0, "osdop_setxattr": 0, "osdop_cmpxattr": 0, "osdop_rmxattr": 0, "osdop_resetxattrs": 0, "osdop_call": 0, "osdop_watch": 0, "osdop_notify": 0, "osdop_src_cmpxattr": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_other": 0, "linger_active": 0, "linger_send": 0, "linger_resend": 0, "linger_ping": 0, "poolop_active": 0, "poolop_send": 0, "poolop_resend": 0, "poolstat_active": 0, "poolstat_send": 0, "poolstat_resend": 0, "statfs_active": 0, "statfs_send": 0, "statfs_resend": 0, "command_active": 0, "command_send": 0, "command_resend": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "osd_sessions": 0, "osd_session_open": 0, "osd_session_close": 0, "osd_laggy": 0, "omap_wr": 0, "omap_rd": 0, "omap_del": 0 } } ], "throttle-msgr_dispatch_throttler-client": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 112, "get_sum": 444064, "get_or_fail_fail": 0, "get_or_fail_success": 112, "take": 0, "take_sum": 0, "put": 112, "put_sum": 444064, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-msgr_dispatch_throttler-radosclient": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 66, "get_sum": 2428224, "get_or_fail_fail": 0, "get_or_fail_success": 66, "take": 0, "take_sum": 0, "put": 66, "put_sum": 2428224, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_bytes": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_ops": [ { "labels": {}, "counters": { "val": 0, "max": 1024, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ] } 2024-07-04T17:35:03.585 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snapshot mirror enable cephfs 2024-07-04T17:35:03.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.861+0000 7fe4a0c00640 1 -- 172.21.15.42:0/3684107376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe49c0bb1a0 msgr2=0x7fe49c0b3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:35:03.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.861+0000 7fe4a0c00640 1 --2- 172.21.15.42:0/3684107376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe49c0bb1a0 0x7fe49c0b3570 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fe4840097b0 tx=0x7fe48402a540 comp rx=0 tx=0).stop 2024-07-04T17:35:03.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.861+0000 7fe4a0c00640 1 -- 172.21.15.42:0/3684107376 shutdown_connections 2024-07-04T17:35:03.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.861+0000 7fe4a0c00640 1 --2- 172.21.15.42:0/3684107376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe49c0bb1a0 0x7fe49c0b3570 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:35:03.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.861+0000 7fe4a0c00640 1 -- 172.21.15.42:0/3684107376 >> 172.21.15.42:0/3684107376 conn(0x7fe49c06e6d0 msgr2=0x7fe49c073960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:35:03.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.861+0000 7fe4a0c00640 1 -- 172.21.15.42:0/3684107376 shutdown_connections 2024-07-04T17:35:03.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.861+0000 7fe4a0c00640 1 -- 172.21.15.42:0/3684107376 wait complete. 2024-07-04T17:35:03.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.863+0000 7fe4a0c00640 1 Processor -- start 2024-07-04T17:35:03.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.863+0000 7fe4a0c00640 1 -- start start 2024-07-04T17:35:03.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.864+0000 7fe4a0c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe49c0b7e40 0x7fe49c0a99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:35:03.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.864+0000 7fe4a0c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c0bc5b0 con 0x7fe49c0b7e40 2024-07-04T17:35:03.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.865+0000 7fe499e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe49c0b7e40 0x7fe49c0a99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:35:03.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.865+0000 7fe499e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe49c0b7e40 0x7fe49c0a99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37392/0 (socket says 172.21.15.42:37392) 2024-07-04T17:35:03.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.865+0000 7fe499e00640 1 -- 172.21.15.42:0/1532537545 learned_addr learned my addr 172.21.15.42:0/1532537545 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:35:03.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.865+0000 7fe499e00640 1 -- 172.21.15.42:0/1532537545 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49c0aa010 con 0x7fe49c0b7e40 2024-07-04T17:35:03.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.865+0000 7fe499e00640 1 --2- 172.21.15.42:0/1532537545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe49c0b7e40 0x7fe49c0a99a0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00c380 tx=0x7fe48c00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:35:03.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.866+0000 7fe492000640 1 -- 172.21.15.42:0/1532537545 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe48c0116c0 con 0x7fe49c0b7e40 2024-07-04T17:35:03.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.866+0000 7fe492000640 1 -- 172.21.15.42:0/1532537545 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe48c017070 con 0x7fe49c0b7e40 2024-07-04T17:35:03.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.866+0000 7fe492000640 1 -- 172.21.15.42:0/1532537545 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe48c011b20 con 0x7fe49c0b7e40 2024-07-04T17:35:03.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.866+0000 7fe4a0c00640 1 -- 172.21.15.42:0/1532537545 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe49c1e4730 con 0x7fe49c0b7e40 2024-07-04T17:35:03.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.866+0000 7fe4a0c00640 1 -- 172.21.15.42:0/1532537545 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe49c0c26c0 con 0x7fe49c0b7e40 2024-07-04T17:35:03.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.867+0000 7fe4a0c00640 1 -- 172.21.15.42:0/1532537545 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe49c0bcc60 con 0x7fe49c0b7e40 2024-07-04T17:35:03.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.867+0000 7fe492000640 1 -- 172.21.15.42:0/1532537545 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 53) ==== 49628+0+0 (secure 0 0 0) 0x7fe48c011cc0 con 0x7fe49c0b7e40 2024-07-04T17:35:03.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.868+0000 7fe492000640 1 --2- 172.21.15.42:0/1532537545 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fe47803d540 0x7fe47803fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:35:03.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.868+0000 7fe492000640 1 -- 172.21.15.42:0/1532537545 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 4303+0+0 (secure 0 0 0) 0x7fe48c051aa0 con 0x7fe49c0b7e40 2024-07-04T17:35:03.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.868+0000 7fe499400640 1 -- 172.21.15.42:0/1532537545 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fe47803d540 msgr2=0x7fe47803fa50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/4064094064 2024-07-04T17:35:03.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.868+0000 7fe499400640 1 --2- 172.21.15.42:0/1532537545 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fe47803d540 0x7fe47803fa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:35:03.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:03.873+0000 7fe492000640 1 -- 172.21.15.42:0/1532537545 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fe48c00a140 con 0x7fe49c0b7e40 2024-07-04T17:35:04.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:04.070+0000 7fe499400640 1 -- 172.21.15.42:0/1532537545 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fe47803d540 msgr2=0x7fe47803fa50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/4064094064 2024-07-04T17:35:04.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:04.070+0000 7fe499400640 1 --2- 172.21.15.42:0/1532537545 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fe47803d540 0x7fe47803fa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:35:04.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:04.100+0000 7fe4a0c00640 1 -- 172.21.15.42:0/1532537545 --> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] -- mgr_command(tid 0: {"prefix": "fs snapshot mirror enable", "fs_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7fe49c0c38e0 con 0x7fe47803d540 2024-07-04T17:35:04.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:04.472+0000 7fe499400640 1 -- 172.21.15.42:0/1532537545 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fe47803d540 msgr2=0x7fe47803fa50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/4064094064 2024-07-04T17:35:04.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:04.472+0000 7fe499400640 1 --2- 172.21.15.42:0/1532537545 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fe47803d540 0x7fe47803fa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-07-04T17:35:05.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:05.273+0000 7fe499400640 1 -- 172.21.15.42:0/1532537545 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fe47803d540 msgr2=0x7fe47803fa50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/4064094064 2024-07-04T17:35:05.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:05.273+0000 7fe499400640 1 --2- 172.21.15.42:0/1532537545 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fe47803d540 0x7fe47803fa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-07-04T17:35:06.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:06.875+0000 7fe499400640 1 -- 172.21.15.42:0/1532537545 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fe47803d540 msgr2=0x7fe47803fa50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/4064094064 2024-07-04T17:35:06.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:06.875+0000 7fe499400640 1 --2- 172.21.15.42:0/1532537545 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fe47803d540 0x7fe47803fa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-07-04T17:35:07.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:07.930+0000 7fe492000640 1 -- 172.21.15.42:0/1532537545 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 54) ==== 49135+0+0 (secure 0 0 0) 0x7fe48c010cd0 con 0x7fe49c0b7e40 2024-07-04T17:35:07.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:07.930+0000 7fe492000640 1 -- 172.21.15.42:0/1532537545 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fe47803d540 msgr2=0x7fe47803fa50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:35:07.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:07.930+0000 7fe492000640 1 --2- 172.21.15.42:0/1532537545 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fe47803d540 0x7fe47803fa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:35:08.017 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:35:08.016+0000 7f8b70c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:35:08.017 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:35:08.016+0000 7f8b70c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:35:08.017 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:35:08.016+0000 7f8b70c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:35:08.017 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:35:08.016+0000 7f8b70c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:35:08.018 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:35:08.016+0000 7f8b70c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:35:08.018 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:35:08.016+0000 7f8b76600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:35:08.018 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:35:08.016+0000 7f8b76600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:35:08.018 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:35:08.016+0000 7f8b76600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:35:08.018 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:35:08.016+0000 7f8b76600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:35:08.018 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:35:08.016+0000 7f8b76600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:35:08.037 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:35:08.035+0000 7f8b76600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:35:08.037 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:35:08.035+0000 7f8b70c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:35:08.037 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:35:08.035+0000 7f8b76600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:35:08.037 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:35:08.035+0000 7f8b76600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:35:08.037 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:35:08.035+0000 7f8b70c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:35:08.037 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:35:08.035+0000 7f8b70c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:35:08.037 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:35:08.035+0000 7f8b76600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:35:08.037 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:35:08.035+0000 7f8b76600640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:35:08.037 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:35:08.036+0000 7f8b70c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:35:08.037 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:35:08.036+0000 7f8b70c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:35:08.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:08.936+0000 7fe492000640 1 -- 172.21.15.42:0/1532537545 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mgrmap(e 55) ==== 49262+0+0 (secure 0 0 0) 0x7fe48c038f80 con 0x7fe49c0b7e40 2024-07-04T17:35:08.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:08.937+0000 7fe492000640 1 --2- 172.21.15.42:0/1532537545 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fe478046b80 0x7fe478048fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:35:08.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:08.937+0000 7fe492000640 1 -- 172.21.15.42:0/1532537545 --> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] -- mgr_command(tid 0: {"prefix": "fs snapshot mirror enable", "fs_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7fe49c0c38e0 con 0x7fe478046b80 2024-07-04T17:35:08.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:08.940+0000 7fe499400640 1 --2- 172.21.15.42:0/1532537545 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fe478046b80 0x7fe478048fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:35:08.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:08.950+0000 7fe499400640 1 --2- 172.21.15.42:0/1532537545 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fe478046b80 0x7fe478048fa0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe484034000 tx=0x7fe484004730 comp rx=0 tx=0).ready entity=mgr.5936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:35:09.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:09.937+0000 7fe492000640 1 -- 172.21.15.42:0/1532537545 <== mgr.5936 v2:172.21.15.42:6834/3991875201 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fe49c0c38e0 con 0x7fe478046b80 2024-07-04T17:35:09.938 INFO:teuthology.orchestra.run.smithi042.stdout:{} 2024-07-04T17:35:09.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:09.941+0000 7fe4a0c00640 1 -- 172.21.15.42:0/1532537545 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fe478046b80 msgr2=0x7fe478048fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:35:09.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:09.941+0000 7fe4a0c00640 1 --2- 172.21.15.42:0/1532537545 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fe478046b80 0x7fe478048fa0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe484034000 tx=0x7fe484004730 comp rx=0 tx=0).stop 2024-07-04T17:35:09.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:09.941+0000 7fe4a0c00640 1 -- 172.21.15.42:0/1532537545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe49c0b7e40 msgr2=0x7fe49c0a99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:35:09.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:09.941+0000 7fe4a0c00640 1 --2- 172.21.15.42:0/1532537545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe49c0b7e40 0x7fe49c0a99a0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00c380 tx=0x7fe48c00c840 comp rx=0 tx=0).stop 2024-07-04T17:35:09.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:09.941+0000 7fe4a0c00640 1 -- 172.21.15.42:0/1532537545 shutdown_connections 2024-07-04T17:35:09.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:09.941+0000 7fe4a0c00640 1 --2- 172.21.15.42:0/1532537545 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fe478046b80 0x7fe478048fa0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:35:09.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:09.941+0000 7fe4a0c00640 1 --2- 172.21.15.42:0/1532537545 >> [v2:172.21.15.42:6834/4064094064,v1:172.21.15.42:6835/4064094064] conn(0x7fe47803d540 0x7fe47803fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:35:09.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:09.941+0000 7fe4a0c00640 1 --2- 172.21.15.42:0/1532537545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe49c0b7e40 0x7fe49c0a99a0 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:35:09.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:09.941+0000 7fe4a0c00640 1 -- 172.21.15.42:0/1532537545 >> 172.21.15.42:0/1532537545 conn(0x7fe49c06e6d0 msgr2=0x7fe49c143970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:35:09.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:09.942+0000 7fe4a0c00640 1 -- 172.21.15.42:0/1532537545 shutdown_connections 2024-07-04T17:35:09.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:09.942+0000 7fe4a0c00640 1 -- 172.21.15.42:0/1532537545 wait complete. 2024-07-04T17:35:19.978 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:35:19.979 DEBUG:teuthology.orchestra.run.smithi042:mirror status for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@12 2024-07-04T17:35:20.226 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:35:20.226 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_inst": "172.21.15.42:0/2937116305", 2024-07-04T17:35:20.226 INFO:teuthology.orchestra.run.smithi042.stdout: "peers": {}, 2024-07-04T17:35:20.226 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_dirs": { 2024-07-04T17:35:20.226 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_count": 0 2024-07-04T17:35:20.226 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.226 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:35:20.244 DEBUG:tasks.cephfs.test_mirroring:command returned={ "rados_inst": "172.21.15.42:0/2937116305", "peers": {}, "snap_dirs": { "dir_count": 0 } } 2024-07-04T17:35:20.244 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:35:20.244 DEBUG:teuthology.orchestra.run.smithi042:counter dump for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok counter dump 2024-07-04T17:35:20.491 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:35:20.491 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker": [ 2024-07-04T17:35:20.491 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:20.491 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:35:20.492 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "0" 2024-07-04T17:35:20.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:20.492 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:20.492 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:35:20.492 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:35:20.492 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:20.492 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:20.492 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:35:20.492 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "1" 2024-07-04T17:35:20.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:20.492 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:20.492 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:35:20.493 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:35:20.493 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:20.493 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:20.493 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:35:20.493 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "2" 2024-07-04T17:35:20.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:20.493 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:20.493 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:35:20.493 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:35:20.493 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.493 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.493 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:20.494 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": [ 2024-07-04T17:35:20.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:20.494 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:20.494 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:20.494 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 113, 2024-07-04T17:35:20.494 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 6, 2024-07-04T17:35:20.494 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 464272, 2024-07-04T17:35:20.494 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 1440, 2024-07-04T17:35:20.494 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:35:20.494 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:35:20.494 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.042113731, 2024-07-04T17:35:20.494 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.005087183, 2024-07-04T17:35:20.494 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.022322308, 2024-07-04T17:35:20.495 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.012338182, 2024-07-04T17:35:20.495 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:35:20.495 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 6, 2024-07-04T17:35:20.495 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.004539022, 2024-07-04T17:35:20.495 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000756503 2024-07-04T17:35:20.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:20.495 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:35:20.495 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:20.495 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:20.495 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:20.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:20.495 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 464272, 2024-07-04T17:35:20.495 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 1440 2024-07-04T17:35:20.495 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.496 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.496 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:20.496 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": [ 2024-07-04T17:35:20.496 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:20.496 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:20.496 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:20.496 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 11, 2024-07-04T17:35:20.496 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 121, 2024-07-04T17:35:20.496 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 1584, 2024-07-04T17:35:20.496 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 910416, 2024-07-04T17:35:20.496 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 23, 2024-07-04T17:35:20.497 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 2, 2024-07-04T17:35:20.497 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.057362474, 2024-07-04T17:35:20.497 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.017960288, 2024-07-04T17:35:20.497 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.020130039, 2024-07-04T17:35:20.497 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:35:20.497 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:35:20.497 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 111, 2024-07-04T17:35:20.497 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.106943404, 2024-07-04T17:35:20.497 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000963454 2024-07-04T17:35:20.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:20.497 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:35:20.497 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:20.497 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:20.497 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:20.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:20.498 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 1584, 2024-07-04T17:35:20.498 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 910416 2024-07-04T17:35:20.498 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.498 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.498 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:20.498 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": [ 2024-07-04T17:35:20.498 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:20.498 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:20.498 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:20.498 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 63, 2024-07-04T17:35:20.498 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 9, 2024-07-04T17:35:20.499 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 2634896, 2024-07-04T17:35:20.499 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 1904, 2024-07-04T17:35:20.499 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:35:20.499 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:35:20.499 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.073837690, 2024-07-04T17:35:20.499 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.005031675, 2024-07-04T17:35:20.499 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.126993902, 2024-07-04T17:35:20.499 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:35:20.499 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:35:20.499 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 9, 2024-07-04T17:35:20.499 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.003402160, 2024-07-04T17:35:20.499 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000378017 2024-07-04T17:35:20.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:20.499 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:35:20.500 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:20.500 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:20.500 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:20.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:20.500 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 2634896, 2024-07-04T17:35:20.500 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 1904 2024-07-04T17:35:20.500 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.500 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.500 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:20.500 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": [ 2024-07-04T17:35:20.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:20.500 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:20.500 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:20.500 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-07-04T17:35:20.501 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-07-04T17:35:20.501 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.501 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.501 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:20.501 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror": [ 2024-07-04T17:35:20.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:20.501 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:20.501 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:20.501 INFO:teuthology.orchestra.run.smithi042.stdout: "mirrored_filesystems": 1, 2024-07-04T17:35:20.501 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_enable_failures": 0 2024-07-04T17:35:20.501 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.501 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.501 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:20.502 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_mirrored_filesystems": [ 2024-07-04T17:35:20.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:20.502 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:35:20.502 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem": "cephfs" 2024-07-04T17:35:20.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:20.502 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:20.502 INFO:teuthology.orchestra.run.smithi042.stdout: "mirroring_peers": 0, 2024-07-04T17:35:20.502 INFO:teuthology.orchestra.run.smithi042.stdout: "directory_count": 0 2024-07-04T17:35:20.502 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.502 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.502 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:20.502 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": [ 2024-07-04T17:35:20.502 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:20.503 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:20.503 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:20.503 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-07-04T17:35:20.503 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-07-04T17:35:20.503 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T17:35:20.503 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-07-04T17:35:20.503 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T17:35:20.503 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-07-04T17:35:20.503 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T17:35:20.503 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T17:35:20.503 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T17:35:20.503 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T17:35:20.503 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T17:35:20.503 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-07-04T17:35:20.503 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T17:35:20.503 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T17:35:20.504 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-07-04T17:35:20.504 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-07-04T17:35:20.504 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-07-04T17:35:20.504 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-07-04T17:35:20.504 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T17:35:20.504 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T17:35:20.504 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T17:35:20.504 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T17:35:20.504 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T17:35:20.504 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-07-04T17:35:20.504 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-07-04T17:35:20.504 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-07-04T17:35:20.504 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T17:35:20.505 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T17:35:20.505 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-07-04T17:35:20.505 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-07-04T17:35:20.505 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-07-04T17:35:20.505 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-07-04T17:35:20.505 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T17:35:20.505 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-07-04T17:35:20.505 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T17:35:20.505 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-07-04T17:35:20.505 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 174433, 2024-07-04T17:35:20.505 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 55, 2024-07-04T17:35:20.505 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-07-04T17:35:20.505 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-07-04T17:35:20.505 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-07-04T17:35:20.506 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-07-04T17:35:20.506 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-07-04T17:35:20.506 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-07-04T17:35:20.506 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-07-04T17:35:20.506 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-07-04T17:35:20.506 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 18720, 2024-07-04T17:35:20.506 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 169, 2024-07-04T17:35:20.506 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T17:35:20.506 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-07-04T17:35:20.506 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-07-04T17:35:20.506 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-07-04T17:35:20.506 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 0, 2024-07-04T17:35:20.506 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 0, 2024-07-04T17:35:20.507 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-07-04T17:35:20.507 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-07-04T17:35:20.507 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-07-04T17:35:20.507 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-07-04T17:35:20.507 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.507 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.507 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:20.507 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter": [ 2024-07-04T17:35:20.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:20.507 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:20.507 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:20.507 INFO:teuthology.orchestra.run.smithi042.stdout: "op_active": 0, 2024-07-04T17:35:20.507 INFO:teuthology.orchestra.run.smithi042.stdout: "op_laggy": 0, 2024-07-04T17:35:20.507 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send": 0, 2024-07-04T17:35:20.508 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send_bytes": 0, 2024-07-04T17:35:20.508 INFO:teuthology.orchestra.run.smithi042.stdout: "op_resend": 0, 2024-07-04T17:35:20.508 INFO:teuthology.orchestra.run.smithi042.stdout: "op_reply": 0, 2024-07-04T17:35:20.508 INFO:teuthology.orchestra.run.smithi042.stdout: "op_latency": { 2024-07-04T17:35:20.508 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:20.508 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:20.508 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:20.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:20.508 INFO:teuthology.orchestra.run.smithi042.stdout: "op_inflight": 0, 2024-07-04T17:35:20.508 INFO:teuthology.orchestra.run.smithi042.stdout: "oplen_avg": { 2024-07-04T17:35:20.508 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:20.508 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0 2024-07-04T17:35:20.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:20.509 INFO:teuthology.orchestra.run.smithi042.stdout: "op": 0, 2024-07-04T17:35:20.509 INFO:teuthology.orchestra.run.smithi042.stdout: "op_r": 0, 2024-07-04T17:35:20.509 INFO:teuthology.orchestra.run.smithi042.stdout: "op_w": 0, 2024-07-04T17:35:20.509 INFO:teuthology.orchestra.run.smithi042.stdout: "op_rmw": 0, 2024-07-04T17:35:20.509 INFO:teuthology.orchestra.run.smithi042.stdout: "op_pg": 0, 2024-07-04T17:35:20.509 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_stat": 0, 2024-07-04T17:35:20.509 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_create": 0, 2024-07-04T17:35:20.509 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_read": 0, 2024-07-04T17:35:20.509 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_write": 0, 2024-07-04T17:35:20.509 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writefull": 0, 2024-07-04T17:35:20.509 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writesame": 0, 2024-07-04T17:35:20.509 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_append": 0, 2024-07-04T17:35:20.509 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_zero": 0, 2024-07-04T17:35:20.509 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_truncate": 0, 2024-07-04T17:35:20.510 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_delete": 0, 2024-07-04T17:35:20.510 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_mapext": 0, 2024-07-04T17:35:20.510 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_sparse_read": 0, 2024-07-04T17:35:20.510 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_clonerange": 0, 2024-07-04T17:35:20.510 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_getxattr": 0, 2024-07-04T17:35:20.510 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_setxattr": 0, 2024-07-04T17:35:20.510 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_cmpxattr": 0, 2024-07-04T17:35:20.510 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_rmxattr": 0, 2024-07-04T17:35:20.510 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_resetxattrs": 0, 2024-07-04T17:35:20.510 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_call": 0, 2024-07-04T17:35:20.510 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_watch": 0, 2024-07-04T17:35:20.510 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_notify": 0, 2024-07-04T17:35:20.510 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T17:35:20.511 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls": 0, 2024-07-04T17:35:20.511 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls_filter": 0, 2024-07-04T17:35:20.511 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_other": 0, 2024-07-04T17:35:20.511 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_active": 0, 2024-07-04T17:35:20.511 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_send": 0, 2024-07-04T17:35:20.511 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_resend": 0, 2024-07-04T17:35:20.511 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_ping": 0, 2024-07-04T17:35:20.511 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_active": 0, 2024-07-04T17:35:20.511 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_send": 0, 2024-07-04T17:35:20.511 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_resend": 0, 2024-07-04T17:35:20.511 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_active": 0, 2024-07-04T17:35:20.511 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_send": 0, 2024-07-04T17:35:20.511 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_resend": 0, 2024-07-04T17:35:20.511 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_active": 0, 2024-07-04T17:35:20.511 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_send": 0, 2024-07-04T17:35:20.512 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_resend": 0, 2024-07-04T17:35:20.512 INFO:teuthology.orchestra.run.smithi042.stdout: "command_active": 0, 2024-07-04T17:35:20.512 INFO:teuthology.orchestra.run.smithi042.stdout: "command_send": 0, 2024-07-04T17:35:20.512 INFO:teuthology.orchestra.run.smithi042.stdout: "command_resend": 0, 2024-07-04T17:35:20.512 INFO:teuthology.orchestra.run.smithi042.stdout: "map_epoch": 0, 2024-07-04T17:35:20.512 INFO:teuthology.orchestra.run.smithi042.stdout: "map_full": 0, 2024-07-04T17:35:20.512 INFO:teuthology.orchestra.run.smithi042.stdout: "map_inc": 0, 2024-07-04T17:35:20.512 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_sessions": 0, 2024-07-04T17:35:20.512 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_open": 0, 2024-07-04T17:35:20.512 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_close": 0, 2024-07-04T17:35:20.512 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_laggy": 0, 2024-07-04T17:35:20.512 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_wr": 0, 2024-07-04T17:35:20.512 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_rd": 0, 2024-07-04T17:35:20.512 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_del": 0 2024-07-04T17:35:20.513 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.513 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.513 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:20.513 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-client": [ 2024-07-04T17:35:20.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:20.513 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:20.513 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:20.513 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:35:20.513 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:35:20.513 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:35:20.513 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 113, 2024-07-04T17:35:20.513 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 448818, 2024-07-04T17:35:20.513 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:35:20.513 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 113, 2024-07-04T17:35:20.513 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:35:20.514 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:35:20.514 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 113, 2024-07-04T17:35:20.514 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 448818, 2024-07-04T17:35:20.514 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:35:20.514 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:20.514 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:20.514 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:20.514 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.514 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.514 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.514 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:20.514 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-radosclient": [ 2024-07-04T17:35:20.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:20.514 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:20.515 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:20.515 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:35:20.515 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:35:20.515 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:35:20.515 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 71, 2024-07-04T17:35:20.515 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 2625940, 2024-07-04T17:35:20.515 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:35:20.515 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 71, 2024-07-04T17:35:20.515 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:35:20.515 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:35:20.515 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 71, 2024-07-04T17:35:20.515 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 2625940, 2024-07-04T17:35:20.515 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:35:20.515 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:20.516 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:20.516 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:20.516 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.516 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.516 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.516 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:20.516 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_bytes": [ 2024-07-04T17:35:20.516 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:20.516 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:20.516 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:20.516 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:35:20.516 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:35:20.516 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:35:20.517 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:35:20.517 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:35:20.517 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:35:20.517 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:35:20.517 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:35:20.517 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:35:20.517 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:35:20.517 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:35:20.517 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:35:20.517 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:20.517 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:20.517 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:20.517 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_ops": [ 2024-07-04T17:35:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:35:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1024, 2024-07-04T17:35:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:35:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:35:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:35:20.518 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:35:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:35:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:35:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:35:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:35:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:35:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:35:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.519 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-07-04T17:35:20.520 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:35:20.520 DEBUG:tasks.cephfs.test_mirroring:command returned={ "AsyncMessenger::Worker": [ { "labels": { "id": "0" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "1" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "2" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } } ], "AsyncMessenger::Worker-0": [ { "labels": {}, "counters": { "msgr_recv_messages": 113, "msgr_send_messages": 6, "msgr_recv_bytes": 464272, "msgr_send_bytes": 1440, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.042113731, "msgr_running_send_time": 0.005087183, "msgr_running_recv_time": 0.022322308, "msgr_running_fast_dispatch_time": 0.012338182, "msgr_send_messages_queue_lat": { "avgcount": 6, "sum": 0.004539022, "avgtime": 0.000756503 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 464272, "msgr_send_encrypted_bytes": 1440 } } ], "AsyncMessenger::Worker-1": [ { "labels": {}, "counters": { "msgr_recv_messages": 11, "msgr_send_messages": 121, "msgr_recv_bytes": 1584, "msgr_send_bytes": 910416, "msgr_created_connections": 23, "msgr_active_connections": 2, "msgr_running_total_time": 0.057362474, "msgr_running_send_time": 0.017960288, "msgr_running_recv_time": 0.020130039, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 111, "sum": 0.106943404, "avgtime": 0.000963454 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 1584, "msgr_send_encrypted_bytes": 910416 } } ], "AsyncMessenger::Worker-2": [ { "labels": {}, "counters": { "msgr_recv_messages": 63, "msgr_send_messages": 9, "msgr_recv_bytes": 2634896, "msgr_send_bytes": 1904, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.073837690, "msgr_running_send_time": 0.005031675, "msgr_running_recv_time": 0.126993902, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 9, "sum": 0.003402160, "avgtime": 0.000378017 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 2634896, "msgr_send_encrypted_bytes": 1904 } } ], "cct": [ { "labels": {}, "counters": { "total_workers": 0, "unhealthy_workers": 0 } } ], "cephfs_mirror": [ { "labels": {}, "counters": { "mirrored_filesystems": 1, "mirror_enable_failures": 0 } } ], "cephfs_mirror_mirrored_filesystems": [ { "labels": { "filesystem": "cephfs" }, "counters": { "mirroring_peers": 0, "directory_count": 0 } } ], "mempool": [ { "labels": {}, "counters": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_items": 0, "bluefs_bytes": 0, "bluefs_items": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "buffer_anon_bytes": 174433, "buffer_anon_items": 55, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 18720, "osdmap_items": 169, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "mds_co_bytes": 0, "mds_co_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 } } ], "objecter": [ { "labels": {}, "counters": { "op_active": 0, "op_laggy": 0, "op_send": 0, "op_send_bytes": 0, "op_resend": 0, "op_reply": 0, "op_latency": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "op_inflight": 0, "oplen_avg": { "avgcount": 0, "sum": 0 }, "op": 0, "op_r": 0, "op_w": 0, "op_rmw": 0, "op_pg": 0, "osdop_stat": 0, "osdop_create": 0, "osdop_read": 0, "osdop_write": 0, "osdop_writefull": 0, "osdop_writesame": 0, "osdop_append": 0, "osdop_zero": 0, "osdop_truncate": 0, "osdop_delete": 0, "osdop_mapext": 0, "osdop_sparse_read": 0, "osdop_clonerange": 0, "osdop_getxattr": 0, "osdop_setxattr": 0, "osdop_cmpxattr": 0, "osdop_rmxattr": 0, "osdop_resetxattrs": 0, "osdop_call": 0, "osdop_watch": 0, "osdop_notify": 0, "osdop_src_cmpxattr": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_other": 0, "linger_active": 0, "linger_send": 0, "linger_resend": 0, "linger_ping": 0, "poolop_active": 0, "poolop_send": 0, "poolop_resend": 0, "poolstat_active": 0, "poolstat_send": 0, "poolstat_resend": 0, "statfs_active": 0, "statfs_send": 0, "statfs_resend": 0, "command_active": 0, "command_send": 0, "command_resend": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "osd_sessions": 0, "osd_session_open": 0, "osd_session_close": 0, "osd_laggy": 0, "omap_wr": 0, "omap_rd": 0, "omap_del": 0 } } ], "throttle-msgr_dispatch_throttler-client": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 113, "get_sum": 448818, "get_or_fail_fail": 0, "get_or_fail_success": 113, "take": 0, "take_sum": 0, "put": 113, "put_sum": 448818, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-msgr_dispatch_throttler-radosclient": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 71, "get_sum": 2625940, "get_or_fail_fail": 0, "get_or_fail_success": 71, "take": 0, "take_sum": 0, "put": 71, "put_sum": 2625940, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_bytes": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_ops": [ { "labels": {}, "counters": { "val": 0, "max": 1024, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ] } 2024-07-04T17:35:20.521 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:35:20.521 DEBUG:teuthology.orchestra.run.smithi042:counter dump for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok counter dump 2024-07-04T17:35:20.755 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:35:20.755 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker": [ 2024-07-04T17:35:20.755 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:20.755 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:35:20.755 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "0" 2024-07-04T17:35:20.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:20.756 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:20.756 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:35:20.756 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:35:20.756 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:20.756 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:20.756 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:35:20.756 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "1" 2024-07-04T17:35:20.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:20.756 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:20.756 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:35:20.756 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:35:20.756 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:20.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:20.757 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:35:20.757 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "2" 2024-07-04T17:35:20.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:20.757 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:20.757 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:35:20.757 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:35:20.757 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.757 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.757 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:20.757 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": [ 2024-07-04T17:35:20.757 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:20.757 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:20.758 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:20.758 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 113, 2024-07-04T17:35:20.758 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 6, 2024-07-04T17:35:20.758 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 464272, 2024-07-04T17:35:20.758 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 1440, 2024-07-04T17:35:20.758 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:35:20.758 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:35:20.758 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.042113731, 2024-07-04T17:35:20.758 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.005087183, 2024-07-04T17:35:20.758 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.022322308, 2024-07-04T17:35:20.758 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.012338182, 2024-07-04T17:35:20.758 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:35:20.758 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 6, 2024-07-04T17:35:20.759 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.004539022, 2024-07-04T17:35:20.759 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000756503 2024-07-04T17:35:20.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:20.759 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:35:20.759 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:20.759 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:20.759 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:20.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:20.759 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 464272, 2024-07-04T17:35:20.759 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 1440 2024-07-04T17:35:20.759 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.759 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.759 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:20.759 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": [ 2024-07-04T17:35:20.759 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:20.760 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:20.760 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:20.760 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 11, 2024-07-04T17:35:20.760 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 121, 2024-07-04T17:35:20.760 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 1584, 2024-07-04T17:35:20.760 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 910416, 2024-07-04T17:35:20.760 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 23, 2024-07-04T17:35:20.760 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 2, 2024-07-04T17:35:20.760 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.057362474, 2024-07-04T17:35:20.760 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.017960288, 2024-07-04T17:35:20.760 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.020130039, 2024-07-04T17:35:20.760 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:35:20.760 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:35:20.760 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 111, 2024-07-04T17:35:20.761 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.106943404, 2024-07-04T17:35:20.761 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000963454 2024-07-04T17:35:20.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:20.761 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:35:20.761 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:20.761 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:20.761 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:20.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:20.761 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 1584, 2024-07-04T17:35:20.761 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 910416 2024-07-04T17:35:20.761 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.761 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.761 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:20.761 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": [ 2024-07-04T17:35:20.762 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:20.762 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:20.762 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:20.762 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 63, 2024-07-04T17:35:20.762 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 9, 2024-07-04T17:35:20.762 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 2634896, 2024-07-04T17:35:20.762 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 1904, 2024-07-04T17:35:20.762 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:35:20.762 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:35:20.762 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.073837690, 2024-07-04T17:35:20.762 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.005031675, 2024-07-04T17:35:20.762 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.126993902, 2024-07-04T17:35:20.762 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:35:20.763 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:35:20.763 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 9, 2024-07-04T17:35:20.763 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.003402160, 2024-07-04T17:35:20.763 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000378017 2024-07-04T17:35:20.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:20.763 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:35:20.763 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:20.763 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:20.763 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:20.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:20.763 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 2634896, 2024-07-04T17:35:20.763 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 1904 2024-07-04T17:35:20.763 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.763 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.764 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:20.764 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": [ 2024-07-04T17:35:20.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:20.764 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:20.764 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:20.764 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-07-04T17:35:20.764 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-07-04T17:35:20.764 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.764 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.764 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:20.764 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror": [ 2024-07-04T17:35:20.764 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:20.764 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:20.764 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:20.765 INFO:teuthology.orchestra.run.smithi042.stdout: "mirrored_filesystems": 1, 2024-07-04T17:35:20.765 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_enable_failures": 0 2024-07-04T17:35:20.765 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.765 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.765 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:20.765 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_mirrored_filesystems": [ 2024-07-04T17:35:20.765 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:20.765 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:35:20.765 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem": "cephfs" 2024-07-04T17:35:20.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:20.765 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:20.765 INFO:teuthology.orchestra.run.smithi042.stdout: "mirroring_peers": 0, 2024-07-04T17:35:20.765 INFO:teuthology.orchestra.run.smithi042.stdout: "directory_count": 0 2024-07-04T17:35:20.765 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.766 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.766 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:20.766 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": [ 2024-07-04T17:35:20.766 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:20.766 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:20.766 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:20.766 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-07-04T17:35:20.766 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-07-04T17:35:20.766 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T17:35:20.766 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-07-04T17:35:20.766 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T17:35:20.766 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-07-04T17:35:20.766 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T17:35:20.767 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T17:35:20.767 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T17:35:20.767 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T17:35:20.767 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T17:35:20.767 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-07-04T17:35:20.767 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T17:35:20.767 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T17:35:20.767 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-07-04T17:35:20.767 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-07-04T17:35:20.767 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-07-04T17:35:20.767 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-07-04T17:35:20.767 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T17:35:20.767 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T17:35:20.767 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T17:35:20.768 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T17:35:20.768 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T17:35:20.768 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-07-04T17:35:20.768 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-07-04T17:35:20.768 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-07-04T17:35:20.768 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T17:35:20.768 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T17:35:20.768 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-07-04T17:35:20.768 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-07-04T17:35:20.768 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-07-04T17:35:20.768 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-07-04T17:35:20.768 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T17:35:20.768 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-07-04T17:35:20.768 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T17:35:20.769 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-07-04T17:35:20.769 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 174433, 2024-07-04T17:35:20.769 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 55, 2024-07-04T17:35:20.769 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-07-04T17:35:20.769 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-07-04T17:35:20.769 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-07-04T17:35:20.769 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-07-04T17:35:20.769 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-07-04T17:35:20.769 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-07-04T17:35:20.769 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-07-04T17:35:20.769 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-07-04T17:35:20.769 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 18720, 2024-07-04T17:35:20.769 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 169, 2024-07-04T17:35:20.769 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T17:35:20.770 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-07-04T17:35:20.770 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-07-04T17:35:20.770 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-07-04T17:35:20.770 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 0, 2024-07-04T17:35:20.770 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 0, 2024-07-04T17:35:20.770 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-07-04T17:35:20.770 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-07-04T17:35:20.770 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-07-04T17:35:20.770 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-07-04T17:35:20.770 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.770 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.770 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:20.770 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter": [ 2024-07-04T17:35:20.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:20.771 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:20.771 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:20.771 INFO:teuthology.orchestra.run.smithi042.stdout: "op_active": 0, 2024-07-04T17:35:20.771 INFO:teuthology.orchestra.run.smithi042.stdout: "op_laggy": 0, 2024-07-04T17:35:20.771 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send": 0, 2024-07-04T17:35:20.771 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send_bytes": 0, 2024-07-04T17:35:20.771 INFO:teuthology.orchestra.run.smithi042.stdout: "op_resend": 0, 2024-07-04T17:35:20.771 INFO:teuthology.orchestra.run.smithi042.stdout: "op_reply": 0, 2024-07-04T17:35:20.771 INFO:teuthology.orchestra.run.smithi042.stdout: "op_latency": { 2024-07-04T17:35:20.771 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:20.771 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:20.771 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:20.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:20.771 INFO:teuthology.orchestra.run.smithi042.stdout: "op_inflight": 0, 2024-07-04T17:35:20.772 INFO:teuthology.orchestra.run.smithi042.stdout: "oplen_avg": { 2024-07-04T17:35:20.772 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:20.772 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0 2024-07-04T17:35:20.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:20.772 INFO:teuthology.orchestra.run.smithi042.stdout: "op": 0, 2024-07-04T17:35:20.772 INFO:teuthology.orchestra.run.smithi042.stdout: "op_r": 0, 2024-07-04T17:35:20.772 INFO:teuthology.orchestra.run.smithi042.stdout: "op_w": 0, 2024-07-04T17:35:20.772 INFO:teuthology.orchestra.run.smithi042.stdout: "op_rmw": 0, 2024-07-04T17:35:20.772 INFO:teuthology.orchestra.run.smithi042.stdout: "op_pg": 0, 2024-07-04T17:35:20.772 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_stat": 0, 2024-07-04T17:35:20.772 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_create": 0, 2024-07-04T17:35:20.772 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_read": 0, 2024-07-04T17:35:20.772 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_write": 0, 2024-07-04T17:35:20.772 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writefull": 0, 2024-07-04T17:35:20.773 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writesame": 0, 2024-07-04T17:35:20.773 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_append": 0, 2024-07-04T17:35:20.773 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_zero": 0, 2024-07-04T17:35:20.773 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_truncate": 0, 2024-07-04T17:35:20.773 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_delete": 0, 2024-07-04T17:35:20.773 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_mapext": 0, 2024-07-04T17:35:20.773 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_sparse_read": 0, 2024-07-04T17:35:20.773 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_clonerange": 0, 2024-07-04T17:35:20.773 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_getxattr": 0, 2024-07-04T17:35:20.773 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_setxattr": 0, 2024-07-04T17:35:20.773 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_cmpxattr": 0, 2024-07-04T17:35:20.773 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_rmxattr": 0, 2024-07-04T17:35:20.773 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_resetxattrs": 0, 2024-07-04T17:35:20.773 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_call": 0, 2024-07-04T17:35:20.774 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_watch": 0, 2024-07-04T17:35:20.774 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_notify": 0, 2024-07-04T17:35:20.774 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T17:35:20.774 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls": 0, 2024-07-04T17:35:20.774 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls_filter": 0, 2024-07-04T17:35:20.774 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_other": 0, 2024-07-04T17:35:20.774 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_active": 0, 2024-07-04T17:35:20.774 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_send": 0, 2024-07-04T17:35:20.774 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_resend": 0, 2024-07-04T17:35:20.774 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_ping": 0, 2024-07-04T17:35:20.774 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_active": 0, 2024-07-04T17:35:20.774 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_send": 0, 2024-07-04T17:35:20.774 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_resend": 0, 2024-07-04T17:35:20.774 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_active": 0, 2024-07-04T17:35:20.775 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_send": 0, 2024-07-04T17:35:20.775 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_resend": 0, 2024-07-04T17:35:20.775 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_active": 0, 2024-07-04T17:35:20.775 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_send": 0, 2024-07-04T17:35:20.775 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_resend": 0, 2024-07-04T17:35:20.775 INFO:teuthology.orchestra.run.smithi042.stdout: "command_active": 0, 2024-07-04T17:35:20.775 INFO:teuthology.orchestra.run.smithi042.stdout: "command_send": 0, 2024-07-04T17:35:20.775 INFO:teuthology.orchestra.run.smithi042.stdout: "command_resend": 0, 2024-07-04T17:35:20.775 INFO:teuthology.orchestra.run.smithi042.stdout: "map_epoch": 0, 2024-07-04T17:35:20.775 INFO:teuthology.orchestra.run.smithi042.stdout: "map_full": 0, 2024-07-04T17:35:20.775 INFO:teuthology.orchestra.run.smithi042.stdout: "map_inc": 0, 2024-07-04T17:35:20.775 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_sessions": 0, 2024-07-04T17:35:20.775 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_open": 0, 2024-07-04T17:35:20.775 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_close": 0, 2024-07-04T17:35:20.776 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_laggy": 0, 2024-07-04T17:35:20.776 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_wr": 0, 2024-07-04T17:35:20.776 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_rd": 0, 2024-07-04T17:35:20.776 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_del": 0 2024-07-04T17:35:20.776 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.776 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.776 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:20.776 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-client": [ 2024-07-04T17:35:20.776 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:20.776 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:20.776 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:20.776 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:35:20.776 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:35:20.776 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:35:20.777 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 113, 2024-07-04T17:35:20.777 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 448818, 2024-07-04T17:35:20.777 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:35:20.777 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 113, 2024-07-04T17:35:20.777 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:35:20.777 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:35:20.777 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 113, 2024-07-04T17:35:20.777 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 448818, 2024-07-04T17:35:20.777 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:35:20.777 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:20.777 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:20.777 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:20.777 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.777 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.778 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.778 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:20.778 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-radosclient": [ 2024-07-04T17:35:20.778 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:20.778 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:20.778 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:20.778 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:35:20.778 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:35:20.778 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:35:20.778 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 71, 2024-07-04T17:35:20.778 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 2625940, 2024-07-04T17:35:20.778 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:35:20.778 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 71, 2024-07-04T17:35:20.779 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:35:20.779 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:35:20.779 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 71, 2024-07-04T17:35:20.779 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 2625940, 2024-07-04T17:35:20.779 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:35:20.779 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:20.779 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:20.779 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:20.779 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.779 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.779 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.779 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:20.779 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_bytes": [ 2024-07-04T17:35:20.779 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:20.779 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:20.780 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:20.780 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:35:20.780 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:35:20.780 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:35:20.780 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:35:20.780 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:35:20.780 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:35:20.780 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:35:20.780 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:35:20.780 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:35:20.780 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:35:20.780 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:35:20.780 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:35:20.780 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:20.781 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:20.781 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:20.781 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.781 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.781 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.781 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:20.781 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_ops": [ 2024-07-04T17:35:20.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:20.781 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:20.781 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:20.781 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:35:20.781 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1024, 2024-07-04T17:35:20.781 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:35:20.781 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:35:20.781 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:35:20.781 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:35:20.782 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:35:20.782 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:35:20.782 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:35:20.782 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:35:20.782 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:35:20.782 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:35:20.782 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:20.782 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:20.782 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:20.782 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.782 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.782 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:20.782 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-07-04T17:35:20.782 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:35:20.783 DEBUG:tasks.cephfs.test_mirroring:command returned={ "AsyncMessenger::Worker": [ { "labels": { "id": "0" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "1" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "2" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } } ], "AsyncMessenger::Worker-0": [ { "labels": {}, "counters": { "msgr_recv_messages": 113, "msgr_send_messages": 6, "msgr_recv_bytes": 464272, "msgr_send_bytes": 1440, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.042113731, "msgr_running_send_time": 0.005087183, "msgr_running_recv_time": 0.022322308, "msgr_running_fast_dispatch_time": 0.012338182, "msgr_send_messages_queue_lat": { "avgcount": 6, "sum": 0.004539022, "avgtime": 0.000756503 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 464272, "msgr_send_encrypted_bytes": 1440 } } ], "AsyncMessenger::Worker-1": [ { "labels": {}, "counters": { "msgr_recv_messages": 11, "msgr_send_messages": 121, "msgr_recv_bytes": 1584, "msgr_send_bytes": 910416, "msgr_created_connections": 23, "msgr_active_connections": 2, "msgr_running_total_time": 0.057362474, "msgr_running_send_time": 0.017960288, "msgr_running_recv_time": 0.020130039, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 111, "sum": 0.106943404, "avgtime": 0.000963454 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 1584, "msgr_send_encrypted_bytes": 910416 } } ], "AsyncMessenger::Worker-2": [ { "labels": {}, "counters": { "msgr_recv_messages": 63, "msgr_send_messages": 9, "msgr_recv_bytes": 2634896, "msgr_send_bytes": 1904, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.073837690, "msgr_running_send_time": 0.005031675, "msgr_running_recv_time": 0.126993902, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 9, "sum": 0.003402160, "avgtime": 0.000378017 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 2634896, "msgr_send_encrypted_bytes": 1904 } } ], "cct": [ { "labels": {}, "counters": { "total_workers": 0, "unhealthy_workers": 0 } } ], "cephfs_mirror": [ { "labels": {}, "counters": { "mirrored_filesystems": 1, "mirror_enable_failures": 0 } } ], "cephfs_mirror_mirrored_filesystems": [ { "labels": { "filesystem": "cephfs" }, "counters": { "mirroring_peers": 0, "directory_count": 0 } } ], "mempool": [ { "labels": {}, "counters": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_items": 0, "bluefs_bytes": 0, "bluefs_items": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "buffer_anon_bytes": 174433, "buffer_anon_items": 55, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 18720, "osdmap_items": 169, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "mds_co_bytes": 0, "mds_co_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 } } ], "objecter": [ { "labels": {}, "counters": { "op_active": 0, "op_laggy": 0, "op_send": 0, "op_send_bytes": 0, "op_resend": 0, "op_reply": 0, "op_latency": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "op_inflight": 0, "oplen_avg": { "avgcount": 0, "sum": 0 }, "op": 0, "op_r": 0, "op_w": 0, "op_rmw": 0, "op_pg": 0, "osdop_stat": 0, "osdop_create": 0, "osdop_read": 0, "osdop_write": 0, "osdop_writefull": 0, "osdop_writesame": 0, "osdop_append": 0, "osdop_zero": 0, "osdop_truncate": 0, "osdop_delete": 0, "osdop_mapext": 0, "osdop_sparse_read": 0, "osdop_clonerange": 0, "osdop_getxattr": 0, "osdop_setxattr": 0, "osdop_cmpxattr": 0, "osdop_rmxattr": 0, "osdop_resetxattrs": 0, "osdop_call": 0, "osdop_watch": 0, "osdop_notify": 0, "osdop_src_cmpxattr": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_other": 0, "linger_active": 0, "linger_send": 0, "linger_resend": 0, "linger_ping": 0, "poolop_active": 0, "poolop_send": 0, "poolop_resend": 0, "poolstat_active": 0, "poolstat_send": 0, "poolstat_resend": 0, "statfs_active": 0, "statfs_send": 0, "statfs_resend": 0, "command_active": 0, "command_send": 0, "command_resend": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "osd_sessions": 0, "osd_session_open": 0, "osd_session_close": 0, "osd_laggy": 0, "omap_wr": 0, "omap_rd": 0, "omap_del": 0 } } ], "throttle-msgr_dispatch_throttler-client": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 113, "get_sum": 448818, "get_or_fail_fail": 0, "get_or_fail_success": 113, "take": 0, "take_sum": 0, "put": 113, "put_sum": 448818, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-msgr_dispatch_throttler-radosclient": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 71, "get_sum": 2625940, "get_or_fail_fail": 0, "get_or_fail_success": 71, "take": 0, "take_sum": 0, "put": 71, "put_sum": 2625940, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_bytes": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_ops": [ { "labels": {}, "counters": { "val": 0, "max": 1024, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ] } 2024-07-04T17:35:20.784 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snapshot mirror peer_add cephfs client.mirror_remote@ceph backup_fs 2024-07-04T17:35:21.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:21.088+0000 7f9478a00640 1 -- 172.21.15.42:0/1382370743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94740b1d90 msgr2=0x7f94740aaa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:35:21.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:21.088+0000 7f9478a00640 1 --2- 172.21.15.42:0/1382370743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94740b1d90 0x7f94740aaa50 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f94640097b0 tx=0x7f946402a540 comp rx=0 tx=0).stop 2024-07-04T17:35:21.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:21.088+0000 7f9478a00640 1 -- 172.21.15.42:0/1382370743 shutdown_connections 2024-07-04T17:35:21.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:21.088+0000 7f9478a00640 1 --2- 172.21.15.42:0/1382370743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94740b1d90 0x7f94740aaa50 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:35:21.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:21.088+0000 7f9478a00640 1 -- 172.21.15.42:0/1382370743 >> 172.21.15.42:0/1382370743 conn(0x7f94740bf700 msgr2=0x7f94740af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:35:21.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:21.088+0000 7f9478a00640 1 -- 172.21.15.42:0/1382370743 shutdown_connections 2024-07-04T17:35:21.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:21.088+0000 7f9478a00640 1 -- 172.21.15.42:0/1382370743 wait complete. 2024-07-04T17:35:21.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:21.090+0000 7f9478a00640 1 Processor -- start 2024-07-04T17:35:21.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:21.091+0000 7f9478a00640 1 -- start start 2024-07-04T17:35:21.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:21.092+0000 7f9478a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94740a9f30 0x7f94740acbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:35:21.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:21.092+0000 7f9478a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f947406c020 con 0x7f94740a9f30 2024-07-04T17:35:21.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:21.092+0000 7f9471e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94740a9f30 0x7f94740acbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:35:21.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:21.092+0000 7f9471e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94740a9f30 0x7f94740acbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50258/0 (socket says 172.21.15.42:50258) 2024-07-04T17:35:21.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:21.092+0000 7f9471e00640 1 -- 172.21.15.42:0/2413493608 learned_addr learned my addr 172.21.15.42:0/2413493608 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:35:21.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:21.092+0000 7f9471e00640 1 -- 172.21.15.42:0/2413493608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94741e4220 con 0x7f94740a9f30 2024-07-04T17:35:21.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:21.093+0000 7f9471e00640 1 --2- 172.21.15.42:0/2413493608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94740a9f30 0x7f94740acbe0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f94640098d0 tx=0x7f9464033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:35:21.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:21.093+0000 7f946aa00640 1 -- 172.21.15.42:0/2413493608 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9464031bc0 con 0x7f94740a9f30 2024-07-04T17:35:21.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:21.093+0000 7f946aa00640 1 -- 172.21.15.42:0/2413493608 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9464031e50 con 0x7f94740a9f30 2024-07-04T17:35:21.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:21.093+0000 7f946aa00640 1 -- 172.21.15.42:0/2413493608 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9464033200 con 0x7f94740a9f30 2024-07-04T17:35:21.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:21.093+0000 7f9478a00640 1 -- 172.21.15.42:0/2413493608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f947413b6b0 con 0x7f94740a9f30 2024-07-04T17:35:21.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:21.093+0000 7f9478a00640 1 -- 172.21.15.42:0/2413493608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f947413c980 con 0x7f94740a9f30 2024-07-04T17:35:21.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:21.095+0000 7f946aa00640 1 -- 172.21.15.42:0/2413493608 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 57) ==== 49681+0+0 (secure 0 0 0) 0x7f94640333a0 con 0x7f94740a9f30 2024-07-04T17:35:21.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:21.095+0000 7f9478a00640 1 -- 172.21.15.42:0/2413493608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f947406c710 con 0x7f94740a9f30 2024-07-04T17:35:21.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:21.095+0000 7f946aa00640 1 --2- 172.21.15.42:0/2413493608 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f945003d5d0 0x7f945003fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:35:21.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:21.095+0000 7f946aa00640 1 -- 172.21.15.42:0/2413493608 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 4733+0+0 (secure 0 0 0) 0x7f94640735e0 con 0x7f94740a9f30 2024-07-04T17:35:21.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:21.096+0000 7f9469400640 1 --2- 172.21.15.42:0/2413493608 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f945003d5d0 0x7f945003fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:35:21.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:21.096+0000 7f9469400640 1 --2- 172.21.15.42:0/2413493608 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f945003d5d0 0x7f945003fae0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f945c0015c0 tx=0x7f945c007a90 comp rx=0 tx=0).ready entity=mgr.5936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:35:21.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:21.101+0000 7f946aa00640 1 -- 172.21.15.42:0/2413493608 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f947413c980 con 0x7f94740a9f30 2024-07-04T17:35:21.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:21.328+0000 7f9478a00640 1 -- 172.21.15.42:0/2413493608 --> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] -- mgr_command(tid 0: {"prefix": "fs snapshot mirror peer_add", "fs_name": "cephfs", "remote_cluster_spec": "client.mirror_remote@ceph", "remote_fs_name": "backup_fs", "target": ["mon-mgr", ""]}) -- 0x7f947413ba50 con 0x7f945003d5d0 2024-07-04T17:35:21.359 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:35:21.359+0000 7f8b8e200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:35:21.359 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:35:21.359+0000 7f8b8e200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:35:21.360 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:35:21.359+0000 7f8b8e200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:35:21.360 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:35:21.359+0000 7f8b8e200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:35:21.360 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:35:21.359+0000 7f8b8e200640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:35:22.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:22.232+0000 7f946aa00640 1 -- 172.21.15.42:0/2413493608 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 58) ==== 49734+0+0 (secure 0 0 0) 0x7f94640035f0 con 0x7f94740a9f30 2024-07-04T17:35:22.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:22.237+0000 7f946aa00640 1 -- 172.21.15.42:0/2413493608 <== mgr.5936 v2:172.21.15.42:6834/3991875201 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f947413ba50 con 0x7f945003d5d0 2024-07-04T17:35:22.238 INFO:teuthology.orchestra.run.smithi042.stdout:{} 2024-07-04T17:35:22.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:22.241+0000 7f9478a00640 1 -- 172.21.15.42:0/2413493608 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f945003d5d0 msgr2=0x7f945003fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:35:22.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:22.241+0000 7f9478a00640 1 --2- 172.21.15.42:0/2413493608 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f945003d5d0 0x7f945003fae0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f945c0015c0 tx=0x7f945c007a90 comp rx=0 tx=0).stop 2024-07-04T17:35:22.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:22.241+0000 7f9478a00640 1 -- 172.21.15.42:0/2413493608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94740a9f30 msgr2=0x7f94740acbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:35:22.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:22.241+0000 7f9478a00640 1 --2- 172.21.15.42:0/2413493608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94740a9f30 0x7f94740acbe0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f94640098d0 tx=0x7f9464033e40 comp rx=0 tx=0).stop 2024-07-04T17:35:22.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:22.241+0000 7f9478a00640 1 -- 172.21.15.42:0/2413493608 shutdown_connections 2024-07-04T17:35:22.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:22.241+0000 7f9478a00640 1 --2- 172.21.15.42:0/2413493608 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f945003d5d0 0x7f945003fae0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:35:22.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:22.241+0000 7f9478a00640 1 --2- 172.21.15.42:0/2413493608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94740a9f30 0x7f94740acbe0 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:35:22.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:22.241+0000 7f9478a00640 1 -- 172.21.15.42:0/2413493608 >> 172.21.15.42:0/2413493608 conn(0x7f94740bf700 msgr2=0x7f94740bdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:35:22.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:22.242+0000 7f9478a00640 1 -- 172.21.15.42:0/2413493608 shutdown_connections 2024-07-04T17:35:22.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:22.242+0000 7f9478a00640 1 -- 172.21.15.42:0/2413493608 wait complete. 2024-07-04T17:35:31.695 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-04T17:35:32.277 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:35:32.277 DEBUG:teuthology.orchestra.run.smithi042:mirror status for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@12 2024-07-04T17:35:32.525 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:35:32.526 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_inst": "172.21.15.42:0/2937116305", 2024-07-04T17:35:32.526 INFO:teuthology.orchestra.run.smithi042.stdout: "peers": { 2024-07-04T17:35:32.526 INFO:teuthology.orchestra.run.smithi042.stdout: "dedc342a-5e11-4049-987c-94fa3cabd62f": { 2024-07-04T17:35:32.526 INFO:teuthology.orchestra.run.smithi042.stdout: "remote": { 2024-07-04T17:35:32.526 INFO:teuthology.orchestra.run.smithi042.stdout: "client_name": "client.mirror_remote", 2024-07-04T17:35:32.526 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_name": "ceph", 2024-07-04T17:35:32.526 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "backup_fs" 2024-07-04T17:35:32.526 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:32.526 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:32.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:32.526 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_dirs": { 2024-07-04T17:35:32.527 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_count": 0 2024-07-04T17:35:32.527 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:32.527 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:35:32.543 DEBUG:tasks.cephfs.test_mirroring:command returned={ "rados_inst": "172.21.15.42:0/2937116305", "peers": { "dedc342a-5e11-4049-987c-94fa3cabd62f": { "remote": { "client_name": "client.mirror_remote", "cluster_name": "ceph", "fs_name": "backup_fs" } } }, "snap_dirs": { "dir_count": 0 } } 2024-07-04T17:35:32.544 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:35:32.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:32.839+0000 7fbc05000640 1 -- 172.21.15.42:0/1613583227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc000b2d20 msgr2=0x7fbc000bb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:35:32.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:32.839+0000 7fbc05000640 1 --2- 172.21.15.42:0/1613583227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc000b2d20 0x7fbc000bb1f0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80097b0 tx=0x7fbbe802a4a0 comp rx=0 tx=0).stop 2024-07-04T17:35:32.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:32.839+0000 7fbc05000640 1 -- 172.21.15.42:0/1613583227 shutdown_connections 2024-07-04T17:35:32.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:32.839+0000 7fbc05000640 1 --2- 172.21.15.42:0/1613583227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc000b2d20 0x7fbc000bb1f0 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:35:32.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:32.839+0000 7fbc05000640 1 -- 172.21.15.42:0/1613583227 >> 172.21.15.42:0/1613583227 conn(0x7fbc000bf700 msgr2=0x7fbc000af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:35:32.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:32.839+0000 7fbc05000640 1 -- 172.21.15.42:0/1613583227 shutdown_connections 2024-07-04T17:35:32.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:32.839+0000 7fbc05000640 1 -- 172.21.15.42:0/1613583227 wait complete. 2024-07-04T17:35:32.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:32.841+0000 7fbc05000640 1 Processor -- start 2024-07-04T17:35:32.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:32.841+0000 7fbc05000640 1 -- start start 2024-07-04T17:35:32.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:32.843+0000 7fbc05000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc000b2d20 0x7fbc00143b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:35:32.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:32.843+0000 7fbc05000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0006c020 con 0x7fbc000b2d20 2024-07-04T17:35:32.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:32.843+0000 7fbbfde00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc000b2d20 0x7fbc00143b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:35:32.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:32.843+0000 7fbbfde00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc000b2d20 0x7fbc00143b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50214/0 (socket says 172.21.15.42:50214) 2024-07-04T17:35:32.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:32.843+0000 7fbbfde00640 1 -- 172.21.15.42:0/3403979402 learned_addr learned my addr 172.21.15.42:0/3403979402 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:35:32.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:32.843+0000 7fbbfde00640 1 -- 172.21.15.42:0/3403979402 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc00143890 con 0x7fbc000b2d20 2024-07-04T17:35:32.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:32.844+0000 7fbbfde00640 1 --2- 172.21.15.42:0/3403979402 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc000b2d20 0x7fbc00143b10 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fbbf000c3b0 tx=0x7fbbf000c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:35:32.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:32.844+0000 7fbbf6a00640 1 -- 172.21.15.42:0/3403979402 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbbf0011780 con 0x7fbc000b2d20 2024-07-04T17:35:32.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:32.844+0000 7fbbf6a00640 1 -- 172.21.15.42:0/3403979402 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbf0017070 con 0x7fbc000b2d20 2024-07-04T17:35:32.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:32.844+0000 7fbc05000640 1 -- 172.21.15.42:0/3403979402 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc001d8980 con 0x7fbc000b2d20 2024-07-04T17:35:32.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:32.844+0000 7fbbf6a00640 1 -- 172.21.15.42:0/3403979402 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbbf0011c00 con 0x7fbc000b2d20 2024-07-04T17:35:32.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:32.844+0000 7fbc05000640 1 -- 172.21.15.42:0/3403979402 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc0013e540 con 0x7fbc000b2d20 2024-07-04T17:35:32.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:32.845+0000 7fbc05000640 1 -- 172.21.15.42:0/3403979402 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc0006c710 con 0x7fbc000b2d20 2024-07-04T17:35:32.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:32.845+0000 7fbbf6a00640 1 -- 172.21.15.42:0/3403979402 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 58) ==== 49734+0+0 (secure 0 0 0) 0x7fbbf001c460 con 0x7fbc000b2d20 2024-07-04T17:35:32.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:32.846+0000 7fbbf6a00640 1 --2- 172.21.15.42:0/3403979402 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fbbdc03d660 0x7fbbdc03fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:35:32.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:32.846+0000 7fbbf6a00640 1 -- 172.21.15.42:0/3403979402 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 4733+0+0 (secure 0 0 0) 0x7fbbf0051d30 con 0x7fbc000b2d20 2024-07-04T17:35:32.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:32.846+0000 7fbbf5400640 1 --2- 172.21.15.42:0/3403979402 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fbbdc03d660 0x7fbbdc03fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:35:32.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:32.847+0000 7fbbf5400640 1 --2- 172.21.15.42:0/3403979402 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fbbdc03d660 0x7fbbdc03fb70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8034000 tx=0x7fbbe8004730 comp rx=0 tx=0).ready entity=mgr.5936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:35:32.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:32.851+0000 7fbbf6a00640 1 -- 172.21.15.42:0/3403979402 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fbbf000a140 con 0x7fbc000b2d20 2024-07-04T17:35:33.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:33.104+0000 7fbc05000640 1 -- 172.21.15.42:0/3403979402 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc001445a0 con 0x7fbc000b2d20 2024-07-04T17:35:33.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:33.105+0000 7fbbf6a00640 1 -- 172.21.15.42:0/3403979402 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) ==== 95+0+7539 (secure 0 0 0) 0x7fbbf001c850 con 0x7fbc000b2d20 2024-07-04T17:35:33.107 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:35:33.107 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":115,"btime":"2024-07-04T17:35:22:210884+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5803,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/1184311185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":1184311185},{"type":"v1","addr":"172.21.15.42:6825","nonce":1184311185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103},{"gid":5808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/1815554749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1815554749},{"type":"v1","addr":"172.21.15.42:6827","nonce":1815554749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":5809,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/2027504764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2027504764},{"type":"v1","addr":"172.21.15.42:6829","nonce":2027504764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:34:35.049685+0000","modified":"2024-07-04T17:35:21.376518+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5818},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5818":{"gid":5818,"name":"d","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6833/1863956336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1863956336},{"type":"v1","addr":"172.21.15.42:6833","nonce":1863956336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5818,"qdb_cluster":[5818]},"id":12,"mirror_info":{"peers":{"dedc342a-5e11-4049-987c-94fa3cabd62f":{"remote":{"client_name":"client.mirror_remote","cluster_name":"ceph","fs_name":"backup_fs"}}}}},{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:34:54.794041+0000","modified":"2024-07-04T17:34:57.841598+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5817},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5817":{"gid":5817,"name":"a","rank":0,"incarnation":110,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6831/4064998265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":4064998265},{"type":"v1","addr":"172.21.15.42:6831","nonce":4064998265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5817,"qdb_cluster":[5817]},"id":13}]} 2024-07-04T17:35:33.107 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 115 2024-07-04T17:35:33.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:33.108+0000 7fbc05000640 1 -- 172.21.15.42:0/3403979402 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fbbdc03d660 msgr2=0x7fbbdc03fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:35:33.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:33.108+0000 7fbc05000640 1 --2- 172.21.15.42:0/3403979402 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fbbdc03d660 0x7fbbdc03fb70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8034000 tx=0x7fbbe8004730 comp rx=0 tx=0).stop 2024-07-04T17:35:33.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:33.108+0000 7fbc05000640 1 -- 172.21.15.42:0/3403979402 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc000b2d20 msgr2=0x7fbc00143b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:35:33.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:33.108+0000 7fbc05000640 1 --2- 172.21.15.42:0/3403979402 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc000b2d20 0x7fbc00143b10 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fbbf000c3b0 tx=0x7fbbf000c870 comp rx=0 tx=0).stop 2024-07-04T17:35:33.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:33.109+0000 7fbc05000640 1 -- 172.21.15.42:0/3403979402 shutdown_connections 2024-07-04T17:35:33.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:33.109+0000 7fbc05000640 1 --2- 172.21.15.42:0/3403979402 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fbbdc03d660 0x7fbbdc03fb70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:35:33.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:33.109+0000 7fbc05000640 1 --2- 172.21.15.42:0/3403979402 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc000b2d20 0x7fbc00143b10 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:35:33.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:33.109+0000 7fbc05000640 1 -- 172.21.15.42:0/3403979402 >> 172.21.15.42:0/3403979402 conn(0x7fbc000bf700 msgr2=0x7fbc00075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:35:33.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:33.109+0000 7fbc05000640 1 -- 172.21.15.42:0/3403979402 shutdown_connections 2024-07-04T17:35:33.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:35:33.109+0000 7fbc05000640 1 -- 172.21.15.42:0/3403979402 wait complete. 2024-07-04T17:35:33.138 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:35:33.138 DEBUG:teuthology.orchestra.run.smithi042:counter dump for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok counter dump 2024-07-04T17:35:33.377 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:35:33.377 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker": [ 2024-07-04T17:35:33.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:33.378 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:35:33.378 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "0" 2024-07-04T17:35:33.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:33.378 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:33.378 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:35:33.378 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:35:33.378 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:33.378 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:33.378 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:35:33.378 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "1" 2024-07-04T17:35:33.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:33.378 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:33.378 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:35:33.379 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:35:33.379 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:33.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:33.379 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:35:33.379 INFO:teuthology.orchestra.run.smithi042.stdout: "id": "2" 2024-07-04T17:35:33.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:33.379 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:33.379 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_ready_timeouts": 0, 2024-07-04T17:35:33.379 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_connection_idle_timeouts": 0 2024-07-04T17:35:33.379 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.379 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.379 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:33.379 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": [ 2024-07-04T17:35:33.379 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:33.380 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:33.380 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:33.380 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 115, 2024-07-04T17:35:33.380 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 7, 2024-07-04T17:35:33.380 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 469600, 2024-07-04T17:35:33.380 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 1776, 2024-07-04T17:35:33.380 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:35:33.380 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:35:33.380 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.042949823, 2024-07-04T17:35:33.380 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.005296419, 2024-07-04T17:35:33.380 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.022749177, 2024-07-04T17:35:33.380 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.012515151, 2024-07-04T17:35:33.380 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:35:33.380 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 7, 2024-07-04T17:35:33.381 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.004621411, 2024-07-04T17:35:33.381 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000660201 2024-07-04T17:35:33.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:33.381 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:35:33.381 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:33.381 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:33.381 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:33.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:33.381 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 469600, 2024-07-04T17:35:33.381 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 1776 2024-07-04T17:35:33.381 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.381 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.381 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:33.381 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": [ 2024-07-04T17:35:33.382 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:33.382 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:33.382 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:33.382 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 11, 2024-07-04T17:35:33.382 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 123, 2024-07-04T17:35:33.382 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 1584, 2024-07-04T17:35:33.382 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 911152, 2024-07-04T17:35:33.382 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 23, 2024-07-04T17:35:33.382 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 2, 2024-07-04T17:35:33.382 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.057703712, 2024-07-04T17:35:33.382 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.018253936, 2024-07-04T17:35:33.382 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.020130039, 2024-07-04T17:35:33.382 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:35:33.382 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:35:33.383 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 113, 2024-07-04T17:35:33.383 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.107177070, 2024-07-04T17:35:33.383 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000948469 2024-07-04T17:35:33.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:33.383 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:35:33.383 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:33.383 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:33.383 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:33.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:33.383 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 1584, 2024-07-04T17:35:33.383 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 911152 2024-07-04T17:35:33.383 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.383 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.384 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:33.384 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": [ 2024-07-04T17:35:33.384 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:33.384 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:33.384 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:33.384 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 65, 2024-07-04T17:35:33.384 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 10, 2024-07-04T17:35:33.384 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 2685536, 2024-07-04T17:35:33.384 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 2240, 2024-07-04T17:35:33.384 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-07-04T17:35:33.384 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-07-04T17:35:33.385 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.075993449, 2024-07-04T17:35:33.385 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.005252027, 2024-07-04T17:35:33.385 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.130562373, 2024-07-04T17:35:33.385 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T17:35:33.385 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T17:35:33.385 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 10, 2024-07-04T17:35:33.385 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.003480880, 2024-07-04T17:35:33.385 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000348088 2024-07-04T17:35:33.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:33.385 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-07-04T17:35:33.385 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:33.385 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:33.385 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:33.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:33.386 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 2685536, 2024-07-04T17:35:33.386 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 2240 2024-07-04T17:35:33.386 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.386 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.386 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:33.386 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": [ 2024-07-04T17:35:33.386 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:33.386 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:33.386 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:33.386 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-07-04T17:35:33.386 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-07-04T17:35:33.386 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.386 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.387 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:33.387 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror": [ 2024-07-04T17:35:33.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:33.387 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:33.387 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:33.387 INFO:teuthology.orchestra.run.smithi042.stdout: "mirrored_filesystems": 1, 2024-07-04T17:35:33.387 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_enable_failures": 0 2024-07-04T17:35:33.387 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.387 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.387 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:33.387 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_mirrored_filesystems": [ 2024-07-04T17:35:33.387 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:33.388 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:35:33.388 INFO:teuthology.orchestra.run.smithi042.stdout: "filesystem": "cephfs" 2024-07-04T17:35:33.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:33.388 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:33.388 INFO:teuthology.orchestra.run.smithi042.stdout: "mirroring_peers": 1, 2024-07-04T17:35:33.388 INFO:teuthology.orchestra.run.smithi042.stdout: "directory_count": 0 2024-07-04T17:35:33.388 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.388 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.388 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:33.388 INFO:teuthology.orchestra.run.smithi042.stdout: "cephfs_mirror_peers": [ 2024-07-04T17:35:33.388 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:33.388 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": { 2024-07-04T17:35:33.388 INFO:teuthology.orchestra.run.smithi042.stdout: "peer_cluster_filesystem": "backup_fs", 2024-07-04T17:35:33.388 INFO:teuthology.orchestra.run.smithi042.stdout: "peer_cluster_name": "ceph", 2024-07-04T17:35:33.389 INFO:teuthology.orchestra.run.smithi042.stdout: "source_filesystem": "cephfs", 2024-07-04T17:35:33.389 INFO:teuthology.orchestra.run.smithi042.stdout: "source_fscid": "12" 2024-07-04T17:35:33.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:33.389 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:33.389 INFO:teuthology.orchestra.run.smithi042.stdout: "snaps_synced": 0, 2024-07-04T17:35:33.389 INFO:teuthology.orchestra.run.smithi042.stdout: "snaps_deleted": 0, 2024-07-04T17:35:33.389 INFO:teuthology.orchestra.run.smithi042.stdout: "snaps_renamed": 0, 2024-07-04T17:35:33.389 INFO:teuthology.orchestra.run.smithi042.stdout: "sync_failures": 0, 2024-07-04T17:35:33.389 INFO:teuthology.orchestra.run.smithi042.stdout: "avg_sync_time": { 2024-07-04T17:35:33.389 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:33.389 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:33.389 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:33.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:33.390 INFO:teuthology.orchestra.run.smithi042.stdout: "sync_bytes": 0 2024-07-04T17:35:33.390 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.390 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.390 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:33.390 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": [ 2024-07-04T17:35:33.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:33.390 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:33.390 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:33.390 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-07-04T17:35:33.390 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-07-04T17:35:33.390 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T17:35:33.390 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-07-04T17:35:33.391 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T17:35:33.391 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-07-04T17:35:33.391 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T17:35:33.391 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T17:35:33.391 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T17:35:33.391 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T17:35:33.391 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T17:35:33.391 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-07-04T17:35:33.391 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T17:35:33.391 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T17:35:33.391 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-07-04T17:35:33.391 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-07-04T17:35:33.391 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-07-04T17:35:33.391 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-07-04T17:35:33.391 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T17:35:33.392 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T17:35:33.392 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T17:35:33.392 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T17:35:33.392 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T17:35:33.392 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-07-04T17:35:33.392 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-07-04T17:35:33.392 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-07-04T17:35:33.392 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T17:35:33.392 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T17:35:33.392 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-07-04T17:35:33.392 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-07-04T17:35:33.392 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-07-04T17:35:33.392 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-07-04T17:35:33.392 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T17:35:33.393 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-07-04T17:35:33.393 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T17:35:33.393 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-07-04T17:35:33.393 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 193188, 2024-07-04T17:35:33.393 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 77, 2024-07-04T17:35:33.393 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-07-04T17:35:33.393 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-07-04T17:35:33.393 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-07-04T17:35:33.393 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-07-04T17:35:33.393 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-07-04T17:35:33.393 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-07-04T17:35:33.393 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-07-04T17:35:33.393 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-07-04T17:35:33.393 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 32848, 2024-07-04T17:35:33.393 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 301, 2024-07-04T17:35:33.394 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T17:35:33.394 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-07-04T17:35:33.394 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-07-04T17:35:33.394 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-07-04T17:35:33.394 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 0, 2024-07-04T17:35:33.394 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 0, 2024-07-04T17:35:33.394 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-07-04T17:35:33.394 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-07-04T17:35:33.394 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-07-04T17:35:33.394 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-07-04T17:35:33.394 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.394 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.394 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:33.395 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter": [ 2024-07-04T17:35:33.395 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:33.395 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:33.395 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:33.395 INFO:teuthology.orchestra.run.smithi042.stdout: "op_active": 0, 2024-07-04T17:35:33.395 INFO:teuthology.orchestra.run.smithi042.stdout: "op_laggy": 0, 2024-07-04T17:35:33.395 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send": 0, 2024-07-04T17:35:33.395 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send_bytes": 0, 2024-07-04T17:35:33.395 INFO:teuthology.orchestra.run.smithi042.stdout: "op_resend": 0, 2024-07-04T17:35:33.395 INFO:teuthology.orchestra.run.smithi042.stdout: "op_reply": 0, 2024-07-04T17:35:33.395 INFO:teuthology.orchestra.run.smithi042.stdout: "op_latency": { 2024-07-04T17:35:33.395 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:33.395 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:33.396 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:33.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:33.396 INFO:teuthology.orchestra.run.smithi042.stdout: "op_inflight": 0, 2024-07-04T17:35:33.396 INFO:teuthology.orchestra.run.smithi042.stdout: "oplen_avg": { 2024-07-04T17:35:33.396 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:33.396 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0 2024-07-04T17:35:33.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:33.396 INFO:teuthology.orchestra.run.smithi042.stdout: "op": 0, 2024-07-04T17:35:33.396 INFO:teuthology.orchestra.run.smithi042.stdout: "op_r": 0, 2024-07-04T17:35:33.396 INFO:teuthology.orchestra.run.smithi042.stdout: "op_w": 0, 2024-07-04T17:35:33.396 INFO:teuthology.orchestra.run.smithi042.stdout: "op_rmw": 0, 2024-07-04T17:35:33.396 INFO:teuthology.orchestra.run.smithi042.stdout: "op_pg": 0, 2024-07-04T17:35:33.396 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_stat": 0, 2024-07-04T17:35:33.396 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_create": 0, 2024-07-04T17:35:33.397 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_read": 0, 2024-07-04T17:35:33.397 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_write": 0, 2024-07-04T17:35:33.397 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writefull": 0, 2024-07-04T17:35:33.397 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writesame": 0, 2024-07-04T17:35:33.397 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_append": 0, 2024-07-04T17:35:33.397 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_zero": 0, 2024-07-04T17:35:33.397 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_truncate": 0, 2024-07-04T17:35:33.397 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_delete": 0, 2024-07-04T17:35:33.397 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_mapext": 0, 2024-07-04T17:35:33.397 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_sparse_read": 0, 2024-07-04T17:35:33.397 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_clonerange": 0, 2024-07-04T17:35:33.397 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_getxattr": 0, 2024-07-04T17:35:33.397 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_setxattr": 0, 2024-07-04T17:35:33.398 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_cmpxattr": 0, 2024-07-04T17:35:33.398 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_rmxattr": 0, 2024-07-04T17:35:33.398 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_resetxattrs": 0, 2024-07-04T17:35:33.398 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_call": 0, 2024-07-04T17:35:33.398 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_watch": 0, 2024-07-04T17:35:33.398 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_notify": 0, 2024-07-04T17:35:33.398 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T17:35:33.398 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls": 0, 2024-07-04T17:35:33.398 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls_filter": 0, 2024-07-04T17:35:33.398 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_other": 0, 2024-07-04T17:35:33.398 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_active": 0, 2024-07-04T17:35:33.399 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_send": 0, 2024-07-04T17:35:33.399 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_resend": 0, 2024-07-04T17:35:33.399 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_ping": 0, 2024-07-04T17:35:33.399 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_active": 0, 2024-07-04T17:35:33.399 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_send": 0, 2024-07-04T17:35:33.399 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_resend": 0, 2024-07-04T17:35:33.399 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_active": 0, 2024-07-04T17:35:33.399 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_send": 0, 2024-07-04T17:35:33.399 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_resend": 0, 2024-07-04T17:35:33.399 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_active": 0, 2024-07-04T17:35:33.399 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_send": 0, 2024-07-04T17:35:33.400 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_resend": 0, 2024-07-04T17:35:33.400 INFO:teuthology.orchestra.run.smithi042.stdout: "command_active": 0, 2024-07-04T17:35:33.400 INFO:teuthology.orchestra.run.smithi042.stdout: "command_send": 0, 2024-07-04T17:35:33.400 INFO:teuthology.orchestra.run.smithi042.stdout: "command_resend": 0, 2024-07-04T17:35:33.400 INFO:teuthology.orchestra.run.smithi042.stdout: "map_epoch": 0, 2024-07-04T17:35:33.400 INFO:teuthology.orchestra.run.smithi042.stdout: "map_full": 0, 2024-07-04T17:35:33.400 INFO:teuthology.orchestra.run.smithi042.stdout: "map_inc": 0, 2024-07-04T17:35:33.400 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_sessions": 0, 2024-07-04T17:35:33.400 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_open": 0, 2024-07-04T17:35:33.400 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_close": 0, 2024-07-04T17:35:33.400 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_laggy": 0, 2024-07-04T17:35:33.400 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_wr": 0, 2024-07-04T17:35:33.401 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_rd": 0, 2024-07-04T17:35:33.401 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_del": 0 2024-07-04T17:35:33.401 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.401 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.401 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:33.401 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-client": [ 2024-07-04T17:35:33.401 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:33.401 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:33.401 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:33.401 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:35:33.401 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:35:33.401 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:35:33.402 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 115, 2024-07-04T17:35:33.402 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 453875, 2024-07-04T17:35:33.402 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:35:33.402 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 115, 2024-07-04T17:35:33.402 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:35:33.402 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:35:33.402 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 115, 2024-07-04T17:35:33.402 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 453875, 2024-07-04T17:35:33.402 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:35:33.402 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:33.402 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:33.402 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:33.402 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.403 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.403 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.403 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:33.403 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-radosclient": [ 2024-07-04T17:35:33.403 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:33.403 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:33.403 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:33.403 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:35:33.403 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:35:33.403 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:35:33.403 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 73, 2024-07-04T17:35:33.403 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 2676302, 2024-07-04T17:35:33.403 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:35:33.403 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 73, 2024-07-04T17:35:33.403 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:35:33.403 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:35:33.404 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 73, 2024-07-04T17:35:33.404 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 2676302, 2024-07-04T17:35:33.404 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:35:33.404 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:33.404 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:33.404 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:33.404 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.404 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.404 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.404 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:33.404 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_bytes": [ 2024-07-04T17:35:33.404 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:33.404 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:33.404 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:33.404 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:35:33.405 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-07-04T17:35:33.405 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:35:33.405 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:35:33.405 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:35:33.405 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:35:33.405 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:35:33.405 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:35:33.405 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:35:33.405 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:35:33.405 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:35:33.405 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:35:33.405 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:33.405 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:33.405 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:33.405 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.405 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.406 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.406 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-07-04T17:35:33.406 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_ops": [ 2024-07-04T17:35:33.406 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-07-04T17:35:33.406 INFO:teuthology.orchestra.run.smithi042.stdout: "labels": {}, 2024-07-04T17:35:33.406 INFO:teuthology.orchestra.run.smithi042.stdout: "counters": { 2024-07-04T17:35:33.406 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-07-04T17:35:33.406 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1024, 2024-07-04T17:35:33.406 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-07-04T17:35:33.406 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-07-04T17:35:33.406 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-07-04T17:35:33.406 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-07-04T17:35:33.406 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-07-04T17:35:33.407 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-07-04T17:35:33.407 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-07-04T17:35:33.407 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2024-07-04T17:35:33.407 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2024-07-04T17:35:33.407 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-07-04T17:35:33.407 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-07-04T17:35:33.407 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-07-04T17:35:33.407 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-07-04T17:35:33.407 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.407 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.407 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.407 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-07-04T17:35:33.407 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:35:33.408 DEBUG:tasks.cephfs.test_mirroring:command returned={ "AsyncMessenger::Worker": [ { "labels": { "id": "0" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "1" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } }, { "labels": { "id": "2" }, "counters": { "msgr_connection_ready_timeouts": 0, "msgr_connection_idle_timeouts": 0 } } ], "AsyncMessenger::Worker-0": [ { "labels": {}, "counters": { "msgr_recv_messages": 115, "msgr_send_messages": 7, "msgr_recv_bytes": 469600, "msgr_send_bytes": 1776, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.042949823, "msgr_running_send_time": 0.005296419, "msgr_running_recv_time": 0.022749177, "msgr_running_fast_dispatch_time": 0.012515151, "msgr_send_messages_queue_lat": { "avgcount": 7, "sum": 0.004621411, "avgtime": 0.000660201 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 469600, "msgr_send_encrypted_bytes": 1776 } } ], "AsyncMessenger::Worker-1": [ { "labels": {}, "counters": { "msgr_recv_messages": 11, "msgr_send_messages": 123, "msgr_recv_bytes": 1584, "msgr_send_bytes": 911152, "msgr_created_connections": 23, "msgr_active_connections": 2, "msgr_running_total_time": 0.057703712, "msgr_running_send_time": 0.018253936, "msgr_running_recv_time": 0.020130039, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 113, "sum": 0.107177070, "avgtime": 0.000948469 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 1584, "msgr_send_encrypted_bytes": 911152 } } ], "AsyncMessenger::Worker-2": [ { "labels": {}, "counters": { "msgr_recv_messages": 65, "msgr_send_messages": 10, "msgr_recv_bytes": 2685536, "msgr_send_bytes": 2240, "msgr_created_connections": 2, "msgr_active_connections": 1, "msgr_running_total_time": 0.075993449, "msgr_running_send_time": 0.005252027, "msgr_running_recv_time": 0.130562373, "msgr_running_fast_dispatch_time": 0.000000000, "msgr_send_messages_queue_lat": { "avgcount": 10, "sum": 0.003480880, "avgtime": 0.000348088 }, "msgr_handle_ack_lat": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "msgr_recv_encrypted_bytes": 2685536, "msgr_send_encrypted_bytes": 2240 } } ], "cct": [ { "labels": {}, "counters": { "total_workers": 0, "unhealthy_workers": 0 } } ], "cephfs_mirror": [ { "labels": {}, "counters": { "mirrored_filesystems": 1, "mirror_enable_failures": 0 } } ], "cephfs_mirror_mirrored_filesystems": [ { "labels": { "filesystem": "cephfs" }, "counters": { "mirroring_peers": 1, "directory_count": 0 } } ], "cephfs_mirror_peers": [ { "labels": { "peer_cluster_filesystem": "backup_fs", "peer_cluster_name": "ceph", "source_filesystem": "cephfs", "source_fscid": "12" }, "counters": { "snaps_synced": 0, "snaps_deleted": 0, "snaps_renamed": 0, "sync_failures": 0, "avg_sync_time": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "sync_bytes": 0 } } ], "mempool": [ { "labels": {}, "counters": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_items": 0, "bluefs_bytes": 0, "bluefs_items": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "buffer_anon_bytes": 193188, "buffer_anon_items": 77, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 32848, "osdmap_items": 301, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "mds_co_bytes": 0, "mds_co_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 } } ], "objecter": [ { "labels": {}, "counters": { "op_active": 0, "op_laggy": 0, "op_send": 0, "op_send_bytes": 0, "op_resend": 0, "op_reply": 0, "op_latency": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 }, "op_inflight": 0, "oplen_avg": { "avgcount": 0, "sum": 0 }, "op": 0, "op_r": 0, "op_w": 0, "op_rmw": 0, "op_pg": 0, "osdop_stat": 0, "osdop_create": 0, "osdop_read": 0, "osdop_write": 0, "osdop_writefull": 0, "osdop_writesame": 0, "osdop_append": 0, "osdop_zero": 0, "osdop_truncate": 0, "osdop_delete": 0, "osdop_mapext": 0, "osdop_sparse_read": 0, "osdop_clonerange": 0, "osdop_getxattr": 0, "osdop_setxattr": 0, "osdop_cmpxattr": 0, "osdop_rmxattr": 0, "osdop_resetxattrs": 0, "osdop_call": 0, "osdop_watch": 0, "osdop_notify": 0, "osdop_src_cmpxattr": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_other": 0, "linger_active": 0, "linger_send": 0, "linger_resend": 0, "linger_ping": 0, "poolop_active": 0, "poolop_send": 0, "poolop_resend": 0, "poolstat_active": 0, "poolstat_send": 0, "poolstat_resend": 0, "statfs_active": 0, "statfs_send": 0, "statfs_resend": 0, "command_active": 0, "command_send": 0, "command_resend": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "osd_sessions": 0, "osd_session_open": 0, "osd_session_close": 0, "osd_laggy": 0, "omap_wr": 0, "omap_rd": 0, "omap_del": 0 } } ], "throttle-msgr_dispatch_throttler-client": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 115, "get_sum": 453875, "get_or_fail_fail": 0, "get_or_fail_success": 115, "take": 0, "take_sum": 0, "put": 115, "put_sum": 453875, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-msgr_dispatch_throttler-radosclient": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 73, "get_sum": 2676302, "get_or_fail_fail": 0, "get_or_fail_success": 73, "take": 0, "take_sum": 0, "put": 73, "put_sum": 2676302, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_bytes": [ { "labels": {}, "counters": { "val": 0, "max": 104857600, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ], "throttle-objecter_ops": [ { "labels": {}, "counters": { "val": 0, "max": 1024, "get_started": 0, "get": 0, "get_sum": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "take": 0, "take_sum": 0, "put": 0, "put_sum": 0, "wait": { "avgcount": 0, "sum": 0.000000000, "avgtime": 0.000000000 } } } ] } 2024-07-04T17:35:33.408 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:35:33.409 DEBUG:teuthology.orchestra.run.smithi042:mirror status for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@12 2024-07-04T17:35:33.656 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:35:33.656 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_inst": "172.21.15.42:0/2937116305", 2024-07-04T17:35:33.656 INFO:teuthology.orchestra.run.smithi042.stdout: "peers": { 2024-07-04T17:35:33.656 INFO:teuthology.orchestra.run.smithi042.stdout: "dedc342a-5e11-4049-987c-94fa3cabd62f": { 2024-07-04T17:35:33.657 INFO:teuthology.orchestra.run.smithi042.stdout: "remote": { 2024-07-04T17:35:33.657 INFO:teuthology.orchestra.run.smithi042.stdout: "client_name": "client.mirror_remote", 2024-07-04T17:35:33.657 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_name": "ceph", 2024-07-04T17:35:33.657 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "backup_fs" 2024-07-04T17:35:33.657 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.657 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:33.657 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_dirs": { 2024-07-04T17:35:33.657 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_count": 0 2024-07-04T17:35:33.657 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.657 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:35:33.675 DEBUG:tasks.cephfs.test_mirroring:command returned={ "rados_inst": "172.21.15.42:0/2937116305", "peers": { "dedc342a-5e11-4049-987c-94fa3cabd62f": { "remote": { "client_name": "client.mirror_remote", "cluster_name": "ceph", "fs_name": "backup_fs" } } }, "snap_dirs": { "dir_count": 0 } } 2024-07-04T17:35:33.675 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:35:33.675 DEBUG:teuthology.orchestra.run.smithi042:mirror status for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@12 2024-07-04T17:35:33.917 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:35:33.917 INFO:teuthology.orchestra.run.smithi042.stdout: "rados_inst": "172.21.15.42:0/2937116305", 2024-07-04T17:35:33.917 INFO:teuthology.orchestra.run.smithi042.stdout: "peers": { 2024-07-04T17:35:33.917 INFO:teuthology.orchestra.run.smithi042.stdout: "dedc342a-5e11-4049-987c-94fa3cabd62f": { 2024-07-04T17:35:33.917 INFO:teuthology.orchestra.run.smithi042.stdout: "remote": { 2024-07-04T17:35:33.917 INFO:teuthology.orchestra.run.smithi042.stdout: "client_name": "client.mirror_remote", 2024-07-04T17:35:33.917 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_name": "ceph", 2024-07-04T17:35:33.917 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "backup_fs" 2024-07-04T17:35:33.917 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.917 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-07-04T17:35:33.918 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_dirs": { 2024-07-04T17:35:33.918 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_count": 0 2024-07-04T17:35:33.918 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-07-04T17:35:33.918 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:35:33.934 DEBUG:tasks.cephfs.test_mirroring:command returned={ "rados_inst": "172.21.15.42:0/2937116305", "peers": { "dedc342a-5e11-4049-987c-94fa3cabd62f": { "remote": { "client_name": "client.mirror_remote", "cluster_name": "ceph", "fs_name": "backup_fs" } } }, "snap_dirs": { "dir_count": 0 } } 2024-07-04T17:35:33.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:35:33.934 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec cat /var/run/ceph/cephfs-mirror.pid) 2024-07-04T17:35:33.994 INFO:teuthology.orchestra.run.smithi042.stdout:42081 2024-07-04T17:35:33.995 DEBUG:tasks.cephfs.test_mirroring:SIGSTOP to cephfs-mirror pid 42081 2024-07-04T17:35:33.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:35:33.996 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec kill -SIGSTOP 42081) 2024-07-04T17:36:01.819 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-04T17:36:14.065 DEBUG:tasks.cephfs.test_mirroring:SIGCONT to cephfs-mirror 2024-07-04T17:36:14.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:36:14.065 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec kill -SIGCONT 42081) 2024-07-04T17:36:14.128 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:36:14.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.413+0000 7fed92600640 1 -- 172.21.15.42:0/2627845146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed8c14d550 msgr2=0x7fed8c0bd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:36:14.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.413+0000 7fed92600640 1 --2- 172.21.15.42:0/2627845146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed8c14d550 0x7fed8c0bd100 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0097b0 tx=0x7fed7c02a540 comp rx=0 tx=0).stop 2024-07-04T17:36:14.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.413+0000 7fed92600640 1 -- 172.21.15.42:0/2627845146 shutdown_connections 2024-07-04T17:36:14.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.413+0000 7fed92600640 1 --2- 172.21.15.42:0/2627845146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed8c14d550 0x7fed8c0bd100 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:36:14.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.413+0000 7fed92600640 1 -- 172.21.15.42:0/2627845146 >> 172.21.15.42:0/2627845146 conn(0x7fed8c0c03e0 msgr2=0x7fed8c0c07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:36:14.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.413+0000 7fed92600640 1 -- 172.21.15.42:0/2627845146 shutdown_connections 2024-07-04T17:36:14.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.413+0000 7fed92600640 1 -- 172.21.15.42:0/2627845146 wait complete. 2024-07-04T17:36:14.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.415+0000 7fed92600640 1 Processor -- start 2024-07-04T17:36:14.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.416+0000 7fed92600640 1 -- start start 2024-07-04T17:36:14.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.417+0000 7fed92600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed8c0ad8e0 0x7fed8c0b0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:36:14.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.417+0000 7fed92600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c14df30 con 0x7fed8c0ad8e0 2024-07-04T17:36:14.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.417+0000 7fed8b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed8c0ad8e0 0x7fed8c0b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:36:14.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.417+0000 7fed8b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed8c0ad8e0 0x7fed8c0b0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40024/0 (socket says 172.21.15.42:40024) 2024-07-04T17:36:14.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.417+0000 7fed8b400640 1 -- 172.21.15.42:0/3684789923 learned_addr learned my addr 172.21.15.42:0/3684789923 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:36:14.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.418+0000 7fed8b400640 1 -- 172.21.15.42:0/3684789923 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8c14f670 con 0x7fed8c0ad8e0 2024-07-04T17:36:14.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.418+0000 7fed8b400640 1 --2- 172.21.15.42:0/3684789923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed8c0ad8e0 0x7fed8c0b0680 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0098d0 tx=0x7fed7c033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:36:14.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.418+0000 7fed83400640 1 -- 172.21.15.42:0/3684789923 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed7c031bc0 con 0x7fed8c0ad8e0 2024-07-04T17:36:14.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.419+0000 7fed83400640 1 -- 172.21.15.42:0/3684789923 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed7c031e50 con 0x7fed8c0ad8e0 2024-07-04T17:36:14.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.419+0000 7fed92600640 1 -- 172.21.15.42:0/3684789923 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed8c1e4140 con 0x7fed8c0ad8e0 2024-07-04T17:36:14.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.419+0000 7fed83400640 1 -- 172.21.15.42:0/3684789923 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed7c033200 con 0x7fed8c0ad8e0 2024-07-04T17:36:14.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.419+0000 7fed92600640 1 -- 172.21.15.42:0/3684789923 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed8c1427c0 con 0x7fed8c0ad8e0 2024-07-04T17:36:14.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.420+0000 7fed83400640 1 -- 172.21.15.42:0/3684789923 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 58) ==== 49734+0+0 (secure 0 0 0) 0x7fed7c0333a0 con 0x7fed8c0ad8e0 2024-07-04T17:36:14.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.420+0000 7fed92600640 1 -- 172.21.15.42:0/3684789923 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed8c0bd100 con 0x7fed8c0ad8e0 2024-07-04T17:36:14.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.420+0000 7fed83400640 1 --2- 172.21.15.42:0/3684789923 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fed6c03d610 0x7fed6c03fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:36:14.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.421+0000 7fed83400640 1 -- 172.21.15.42:0/3684789923 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 4863+0+0 (secure 0 0 0) 0x7fed7c073750 con 0x7fed8c0ad8e0 2024-07-04T17:36:14.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.421+0000 7fed8aa00640 1 --2- 172.21.15.42:0/3684789923 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fed6c03d610 0x7fed6c03fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:36:14.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.422+0000 7fed8aa00640 1 --2- 172.21.15.42:0/3684789923 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fed6c03d610 0x7fed6c03fb20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fed780015c0 tx=0x7fed78007a90 comp rx=0 tx=0).ready entity=mgr.5936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:36:14.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.426+0000 7fed83400640 1 -- 172.21.15.42:0/3684789923 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fed7c03f030 con 0x7fed8c0ad8e0 2024-07-04T17:36:14.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.642+0000 7fed92600640 1 -- 172.21.15.42:0/3684789923 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed8c1429f0 con 0x7fed8c0ad8e0 2024-07-04T17:36:14.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.644+0000 7fed83400640 1 -- 172.21.15.42:0/3684789923 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+15289 (secure 0 0 0) 0x7fed7c03a3a0 con 0x7fed8c0ad8e0 2024-07-04T17:36:14.645 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:36:14.645 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":144,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:36:09.312059+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:34:33.191097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-07-04T17:34:34.033408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1699999570846558,"score_stable":1.1699999570846558,"optimal_score":1,"raw_score_acting":1.1699999570846558,"raw_score_stable":1.1699999570846558,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:34:52.310342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:34:53.756688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.d","mappings":[{"from":0,"to":1}]},{"pgid":"25.25","mappings":[{"from":0,"to":2}]},{"pgid":"27.2d","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/2937116305":"2024-07-04T18:36:04.169055+0000","172.21.15.42:0/3482046390":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/1404855452":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/1956174992":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/3978634668":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/3294509689":"2024-07-05T17:35:07.918359+0000","172.21.15.42:6835/4064094064":"2024-07-05T17:35:07.918359+0000","172.21.15.42:6834/4064094064":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/2180653394":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/2328963399":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/644737563":"2024-07-05T17:35:07.918359+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:36:14.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.646+0000 7fed92600640 1 -- 172.21.15.42:0/3684789923 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fed6c03d610 msgr2=0x7fed6c03fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:36:14.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.646+0000 7fed92600640 1 --2- 172.21.15.42:0/3684789923 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fed6c03d610 0x7fed6c03fb20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fed780015c0 tx=0x7fed78007a90 comp rx=0 tx=0).stop 2024-07-04T17:36:14.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.646+0000 7fed92600640 1 -- 172.21.15.42:0/3684789923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed8c0ad8e0 msgr2=0x7fed8c0b0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:36:14.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.646+0000 7fed92600640 1 --2- 172.21.15.42:0/3684789923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed8c0ad8e0 0x7fed8c0b0680 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0098d0 tx=0x7fed7c033e40 comp rx=0 tx=0).stop 2024-07-04T17:36:14.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.647+0000 7fed92600640 1 -- 172.21.15.42:0/3684789923 shutdown_connections 2024-07-04T17:36:14.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.647+0000 7fed92600640 1 --2- 172.21.15.42:0/3684789923 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fed6c03d610 0x7fed6c03fb20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:36:14.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.647+0000 7fed92600640 1 --2- 172.21.15.42:0/3684789923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed8c0ad8e0 0x7fed8c0b0680 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:36:14.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.647+0000 7fed92600640 1 -- 172.21.15.42:0/3684789923 >> 172.21.15.42:0/3684789923 conn(0x7fed8c0c03e0 msgr2=0x7fed8c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:36:14.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.647+0000 7fed92600640 1 -- 172.21.15.42:0/3684789923 shutdown_connections 2024-07-04T17:36:14.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:14.647+0000 7fed92600640 1 -- 172.21.15.42:0/3684789923 wait complete. 2024-07-04T17:36:31.953 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-04T17:36:54.679 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-07-04T17:36:54.679 DEBUG:teuthology.orchestra.run.smithi042:mirror status for fs: cephfs> ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror status cephfs@12 2024-07-04T17:36:54.930 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:36:54.930 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "blocklisted" 2024-07-04T17:36:54.930 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:36:54.949 DEBUG:tasks.cephfs.test_mirroring:command returned={ "state": "blocklisted" } 2024-07-04T17:36:54.950 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable mirroring 2024-07-04T17:36:55.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:55.236+0000 7f1416a00640 1 -- 172.21.15.42:0/1165234591 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14100b0db0 msgr2=0x7f14100abb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:36:55.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:55.236+0000 7f1416a00640 1 --2- 172.21.15.42:0/1165234591 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14100b0db0 0x7f14100abb80 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f13fc0097b0 tx=0x7f13fc02a540 comp rx=0 tx=0).stop 2024-07-04T17:36:55.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:55.236+0000 7f1416a00640 1 -- 172.21.15.42:0/1165234591 shutdown_connections 2024-07-04T17:36:55.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:55.236+0000 7f1416a00640 1 --2- 172.21.15.42:0/1165234591 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14100b0db0 0x7f14100abb80 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:36:55.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:55.236+0000 7f1416a00640 1 -- 172.21.15.42:0/1165234591 >> 172.21.15.42:0/1165234591 conn(0x7f14100afaa0 msgr2=0x7f14100afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:36:55.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:55.236+0000 7f1416a00640 1 -- 172.21.15.42:0/1165234591 shutdown_connections 2024-07-04T17:36:55.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:55.236+0000 7f1416a00640 1 -- 172.21.15.42:0/1165234591 wait complete. 2024-07-04T17:36:55.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:55.238+0000 7f1416a00640 1 Processor -- start 2024-07-04T17:36:55.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:55.238+0000 7f1416a00640 1 -- start start 2024-07-04T17:36:55.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:55.239+0000 7f1416a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14100ac5b0 0x7f14101e1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:36:55.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:55.239+0000 7f1416a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f14100ae640 con 0x7f14100ac5b0 2024-07-04T17:36:55.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:55.240+0000 7f1415600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14100ac5b0 0x7f14101e1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:36:55.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:55.240+0000 7f1415600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14100ac5b0 0x7f14101e1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43650/0 (socket says 172.21.15.42:43650) 2024-07-04T17:36:55.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:55.240+0000 7f1415600640 1 -- 172.21.15.42:0/68119968 learned_addr learned my addr 172.21.15.42:0/68119968 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:36:55.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:55.240+0000 7f1415600640 1 -- 172.21.15.42:0/68119968 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1410152e10 con 0x7f14100ac5b0 2024-07-04T17:36:55.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:55.241+0000 7f1415600640 1 --2- 172.21.15.42:0/68119968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14100ac5b0 0x7f14101e1220 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f140400c340 tx=0x7f140400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:36:55.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:55.241+0000 7f140d600640 1 -- 172.21.15.42:0/68119968 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1404017070 con 0x7f14100ac5b0 2024-07-04T17:36:55.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:55.241+0000 7f140d600640 1 -- 172.21.15.42:0/68119968 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14040035f0 con 0x7f14100ac5b0 2024-07-04T17:36:55.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:55.241+0000 7f1416a00640 1 -- 172.21.15.42:0/68119968 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14101e1cb0 con 0x7f14100ac5b0 2024-07-04T17:36:55.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:55.241+0000 7f140d600640 1 -- 172.21.15.42:0/68119968 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1404003970 con 0x7f14100ac5b0 2024-07-04T17:36:55.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:55.241+0000 7f1416a00640 1 -- 172.21.15.42:0/68119968 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f141013d2b0 con 0x7f14100ac5b0 2024-07-04T17:36:55.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:55.242+0000 7f1416a00640 1 -- 172.21.15.42:0/68119968 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14100abb80 con 0x7f14100ac5b0 2024-07-04T17:36:55.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:55.242+0000 7f140d600640 1 -- 172.21.15.42:0/68119968 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 59) ==== 49477+0+0 (secure 0 0 0) 0x7f140401a070 con 0x7f14100ac5b0 2024-07-04T17:36:55.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:55.243+0000 7f140d600640 1 --2- 172.21.15.42:0/68119968 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f13f003d390 0x7f13f003f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:36:55.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:55.243+0000 7f140d600640 1 -- 172.21.15.42:0/68119968 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 4863+0+0 (secure 0 0 0) 0x7f1404050e70 con 0x7f14100ac5b0 2024-07-04T17:36:55.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:55.243+0000 7f1414c00640 1 --2- 172.21.15.42:0/68119968 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f13f003d390 0x7f13f003f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:36:55.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:55.245+0000 7f1414c00640 1 --2- 172.21.15.42:0/68119968 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f13f003d390 0x7f13f003f8a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f13fc034000 tx=0x7f13fc004730 comp rx=0 tx=0).ready entity=mgr.5936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:36:55.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:55.249+0000 7f140d600640 1 -- 172.21.15.42:0/68119968 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f1404003da0 con 0x7f14100ac5b0 2024-07-04T17:36:55.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:55.525+0000 7f1416a00640 1 -- 172.21.15.42:0/68119968 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "mirroring"} v 0) -- 0x7f1410068e30 con 0x7f14100ac5b0 2024-07-04T17:36:56.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.408+0000 7f140d600640 1 -- 172.21.15.42:0/68119968 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "mirroring"}]=0 v60) ==== 89+0+0 (secure 0 0 0) 0x7f140404f750 con 0x7f14100ac5b0 2024-07-04T17:36:56.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.415+0000 7f1416a00640 1 -- 172.21.15.42:0/68119968 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f13f003d390 msgr2=0x7f13f003f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:36:56.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.415+0000 7f1416a00640 1 --2- 172.21.15.42:0/68119968 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f13f003d390 0x7f13f003f8a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f13fc034000 tx=0x7f13fc004730 comp rx=0 tx=0).stop 2024-07-04T17:36:56.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.415+0000 7f1416a00640 1 -- 172.21.15.42:0/68119968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14100ac5b0 msgr2=0x7f14101e1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:36:56.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.415+0000 7f1416a00640 1 --2- 172.21.15.42:0/68119968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14100ac5b0 0x7f14101e1220 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f140400c340 tx=0x7f140400c800 comp rx=0 tx=0).stop 2024-07-04T17:36:56.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.416+0000 7f1416a00640 1 -- 172.21.15.42:0/68119968 shutdown_connections 2024-07-04T17:36:56.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.416+0000 7f1416a00640 1 --2- 172.21.15.42:0/68119968 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f13f003d390 0x7f13f003f8a0 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f13fc034000 tx=0x7f13fc004730 comp rx=0 tx=0).stop 2024-07-04T17:36:56.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.416+0000 7f1416a00640 1 --2- 172.21.15.42:0/68119968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14100ac5b0 0x7f14101e1220 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:36:56.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.416+0000 7f1416a00640 1 -- 172.21.15.42:0/68119968 >> 172.21.15.42:0/68119968 conn(0x7f14100afaa0 msgr2=0x7f1410073be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:36:56.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.418+0000 7f1416a00640 1 -- 172.21.15.42:0/68119968 shutdown_connections 2024-07-04T17:36:56.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.418+0000 7f1416a00640 1 -- 172.21.15.42:0/68119968 wait complete. 2024-07-04T17:36:56.446 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-07-04T17:36:56.447 DEBUG:teuthology.orchestra.run.smithi042:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-07-04T17:36:56.542 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-07-04T17:36:56.542 INFO:tasks.cephfs.mount:Terminating background process 2024-07-04T17:36:56.542 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:36:56.542 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:36:56.585 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:36:56.585 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:36:56.663 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-04T17:36:56.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.977+0000 7fc452400640 1 -- 172.21.15.42:0/3401060663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc44c0b2d20 msgr2=0x7fc44c0bb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:36:56.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.977+0000 7fc452400640 1 --2- 172.21.15.42:0/3401060663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc44c0b2d20 0x7fc44c0bb1f0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fc4340097b0 tx=0x7fc43402a4a0 comp rx=0 tx=0).stop 2024-07-04T17:36:56.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.977+0000 7fc452400640 1 -- 172.21.15.42:0/3401060663 shutdown_connections 2024-07-04T17:36:56.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.977+0000 7fc452400640 1 --2- 172.21.15.42:0/3401060663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc44c0b2d20 0x7fc44c0bb1f0 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:36:56.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.977+0000 7fc452400640 1 -- 172.21.15.42:0/3401060663 >> 172.21.15.42:0/3401060663 conn(0x7fc44c0bf700 msgr2=0x7fc44c0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:36:56.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.977+0000 7fc452400640 1 -- 172.21.15.42:0/3401060663 shutdown_connections 2024-07-04T17:36:56.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.977+0000 7fc452400640 1 -- 172.21.15.42:0/3401060663 wait complete. 2024-07-04T17:36:56.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.979+0000 7fc452400640 1 Processor -- start 2024-07-04T17:36:56.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.979+0000 7fc452400640 1 -- start start 2024-07-04T17:36:56.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.980+0000 7fc452400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc44c0b2d20 0x7fc44c0aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:36:56.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.981+0000 7fc452400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc44c06c020 con 0x7fc44c0b2d20 2024-07-04T17:36:56.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.981+0000 7fc44b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc44c0b2d20 0x7fc44c0aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:36:56.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.981+0000 7fc44b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc44c0b2d20 0x7fc44c0aee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43686/0 (socket says 172.21.15.42:43686) 2024-07-04T17:36:56.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.981+0000 7fc44b400640 1 -- 172.21.15.42:0/3778088986 learned_addr learned my addr 172.21.15.42:0/3778088986 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:36:56.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.981+0000 7fc44b400640 1 -- 172.21.15.42:0/3778088986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc44c1e4e10 con 0x7fc44c0b2d20 2024-07-04T17:36:56.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.982+0000 7fc44b400640 1 --2- 172.21.15.42:0/3778088986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc44c0b2d20 0x7fc44c0aee10 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fc43c00c320 tx=0x7fc43c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:36:56.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.982+0000 7fc443400640 1 -- 172.21.15.42:0/3778088986 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc43c0116c0 con 0x7fc44c0b2d20 2024-07-04T17:36:56.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.982+0000 7fc443400640 1 -- 172.21.15.42:0/3778088986 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc43c017070 con 0x7fc44c0b2d20 2024-07-04T17:36:56.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.982+0000 7fc452400640 1 -- 172.21.15.42:0/3778088986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc44c073530 con 0x7fc44c0b2d20 2024-07-04T17:36:56.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.982+0000 7fc443400640 1 -- 172.21.15.42:0/3778088986 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc43c011b40 con 0x7fc44c0b2d20 2024-07-04T17:36:56.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.982+0000 7fc452400640 1 -- 172.21.15.42:0/3778088986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc44c073ea0 con 0x7fc44c0b2d20 2024-07-04T17:36:56.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.983+0000 7fc452400640 1 -- 172.21.15.42:0/3778088986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc44c06c710 con 0x7fc44c0b2d20 2024-07-04T17:36:56.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.984+0000 7fc443400640 1 -- 172.21.15.42:0/3778088986 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 60) ==== 49464+0+0 (secure 0 0 0) 0x7fc43c01c460 con 0x7fc44c0b2d20 2024-07-04T17:36:56.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.985+0000 7fc443400640 1 --2- 172.21.15.42:0/3778088986 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fc42803d2f0 0x7fc42803f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:36:56.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.985+0000 7fc443400640 1 -- 172.21.15.42:0/3778088986 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 4863+0+0 (secure 0 0 0) 0x7fc43c051940 con 0x7fc44c0b2d20 2024-07-04T17:36:56.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.985+0000 7fc443e00640 1 -- 172.21.15.42:0/3778088986 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fc42803d2f0 msgr2=0x7fc42803f800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/3991875201 2024-07-04T17:36:56.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.985+0000 7fc443e00640 1 --2- 172.21.15.42:0/3778088986 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fc42803d2f0 0x7fc42803f800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:36:56.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:56.990+0000 7fc443400640 1 -- 172.21.15.42:0/3778088986 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fc43c01c800 con 0x7fc44c0b2d20 2024-07-04T17:36:57.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.187+0000 7fc443e00640 1 -- 172.21.15.42:0/3778088986 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fc42803d2f0 msgr2=0x7fc42803f800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/3991875201 2024-07-04T17:36:57.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.187+0000 7fc443e00640 1 --2- 172.21.15.42:0/3778088986 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fc42803d2f0 0x7fc42803f800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:36:57.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.215+0000 7fc452400640 1 -- 172.21.15.42:0/3778088986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc44c0b5fd0 con 0x7fc44c0b2d20 2024-07-04T17:36:57.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.216+0000 7fc443400640 1 -- 172.21.15.42:0/3778088986 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 11 entries v144) ==== 81+0+643 (secure 0 0 0) 0x7fc43c01cad0 con 0x7fc44c0b2d20 2024-07-04T17:36:57.216 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2937116305 2024-07-04T18:36:04.169055+0000 2024-07-04T17:36:57.216 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3482046390 2024-07-05T17:35:07.918359+0000 2024-07-04T17:36:57.216 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1404855452 2024-07-05T17:35:07.918359+0000 2024-07-04T17:36:57.216 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1956174992 2024-07-05T17:35:07.918359+0000 2024-07-04T17:36:57.216 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3978634668 2024-07-05T17:35:07.918359+0000 2024-07-04T17:36:57.216 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3294509689 2024-07-05T17:35:07.918359+0000 2024-07-04T17:36:57.216 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6835/4064094064 2024-07-05T17:35:07.918359+0000 2024-07-04T17:36:57.217 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6834/4064094064 2024-07-05T17:35:07.918359+0000 2024-07-04T17:36:57.217 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2180653394 2024-07-05T17:35:07.918359+0000 2024-07-04T17:36:57.217 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2328963399 2024-07-05T17:35:07.918359+0000 2024-07-04T17:36:57.217 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/644737563 2024-07-05T17:35:07.918359+0000 2024-07-04T17:36:57.217 INFO:teuthology.orchestra.run.smithi042.stderr:listed 11 entries 2024-07-04T17:36:57.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.218+0000 7fc452400640 1 -- 172.21.15.42:0/3778088986 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fc42803d2f0 msgr2=0x7fc42803f800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:36:57.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.218+0000 7fc452400640 1 --2- 172.21.15.42:0/3778088986 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fc42803d2f0 0x7fc42803f800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:36:57.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.219+0000 7fc452400640 1 -- 172.21.15.42:0/3778088986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc44c0b2d20 msgr2=0x7fc44c0aee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:36:57.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.219+0000 7fc452400640 1 --2- 172.21.15.42:0/3778088986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc44c0b2d20 0x7fc44c0aee10 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fc43c00c320 tx=0x7fc43c00c7e0 comp rx=0 tx=0).stop 2024-07-04T17:36:57.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.219+0000 7fc452400640 1 -- 172.21.15.42:0/3778088986 shutdown_connections 2024-07-04T17:36:57.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.219+0000 7fc452400640 1 --2- 172.21.15.42:0/3778088986 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fc42803d2f0 0x7fc42803f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:36:57.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.219+0000 7fc452400640 1 --2- 172.21.15.42:0/3778088986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc44c0b2d20 0x7fc44c0aee10 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:36:57.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.219+0000 7fc452400640 1 -- 172.21.15.42:0/3778088986 >> 172.21.15.42:0/3778088986 conn(0x7fc44c0bf700 msgr2=0x7fc44c141080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:36:57.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.219+0000 7fc452400640 1 -- 172.21.15.42:0/3778088986 shutdown_connections 2024-07-04T17:36:57.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.219+0000 7fc452400640 1 -- 172.21.15.42:0/3778088986 wait complete. 2024-07-04T17:36:57.244 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi042.front.sepia.ceph.com... 2024-07-04T17:36:57.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:36:57.244 DEBUG:teuthology.orchestra.run.smithi042:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-07-04T17:36:57.322 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[70393]: fuse finished with error 0 and tester_r 0 2024-07-04T17:36:57.328 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-07-04T17:36:57.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:36:57.329 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-04T17:36:57.496 INFO:tasks.cephfs.mount:Terminating background process 2024-07-04T17:36:57.497 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:36:57.497 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:36:57.590 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:36:57.590 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:36:57.639 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-07-04T17:36:57.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.901+0000 7ffbb9400640 1 -- 172.21.15.42:0/855810817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbb40bad70 msgr2=0x7ffbb406c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:36:57.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.901+0000 7ffbb9400640 1 --2- 172.21.15.42:0/855810817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbb40bad70 0x7ffbb406c2c0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7ffba4005fd0 tx=0x7ffba402ae00 comp rx=0 tx=0).stop 2024-07-04T17:36:57.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.901+0000 7ffbb9400640 1 -- 172.21.15.42:0/855810817 shutdown_connections 2024-07-04T17:36:57.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.902+0000 7ffbb9400640 1 --2- 172.21.15.42:0/855810817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbb40bad70 0x7ffbb406c2c0 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:36:57.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.902+0000 7ffbb9400640 1 -- 172.21.15.42:0/855810817 >> 172.21.15.42:0/855810817 conn(0x7ffbb40bebc0 msgr2=0x7ffbb40befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:36:57.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.902+0000 7ffbb9400640 1 -- 172.21.15.42:0/855810817 shutdown_connections 2024-07-04T17:36:57.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.902+0000 7ffbb9400640 1 -- 172.21.15.42:0/855810817 wait complete. 2024-07-04T17:36:57.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.904+0000 7ffbb9400640 1 Processor -- start 2024-07-04T17:36:57.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.904+0000 7ffbb9400640 1 -- start start 2024-07-04T17:36:57.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.905+0000 7ffbb9400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbb40b5650 0x7ffbb40bd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:36:57.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.905+0000 7ffbb9400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb40b27c0 con 0x7ffbb40b5650 2024-07-04T17:36:57.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.905+0000 7ffbb3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbb40b5650 0x7ffbb40bd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:36:57.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.906+0000 7ffbb3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbb40b5650 0x7ffbb40bd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43698/0 (socket says 172.21.15.42:43698) 2024-07-04T17:36:57.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.906+0000 7ffbb3e00640 1 -- 172.21.15.42:0/3877541743 learned_addr learned my addr 172.21.15.42:0/3877541743 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:36:57.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.906+0000 7ffbb3e00640 1 -- 172.21.15.42:0/3877541743 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb40b0690 con 0x7ffbb40b5650 2024-07-04T17:36:57.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.906+0000 7ffbb3e00640 1 --2- 172.21.15.42:0/3877541743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbb40b5650 0x7ffbb40bd6c0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7ffba400b3c0 tx=0x7ffba402ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:36:57.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.907+0000 7ffbabe00640 1 -- 172.21.15.42:0/3877541743 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffba4041020 con 0x7ffbb40b5650 2024-07-04T17:36:57.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.907+0000 7ffbabe00640 1 -- 172.21.15.42:0/3877541743 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffba40094b0 con 0x7ffbb40b5650 2024-07-04T17:36:57.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.907+0000 7ffbabe00640 1 -- 172.21.15.42:0/3877541743 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffba4009830 con 0x7ffbb40b5650 2024-07-04T17:36:57.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.907+0000 7ffbb9400640 1 -- 172.21.15.42:0/3877541743 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb41d8b00 con 0x7ffbb40b5650 2024-07-04T17:36:57.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.907+0000 7ffbb9400640 1 -- 172.21.15.42:0/3877541743 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb40afb50 con 0x7ffbb40b5650 2024-07-04T17:36:57.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.908+0000 7ffbabe00640 1 -- 172.21.15.42:0/3877541743 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 60) ==== 49464+0+0 (secure 0 0 0) 0x7ffba4044020 con 0x7ffbb40b5650 2024-07-04T17:36:57.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.908+0000 7ffbb9400640 1 -- 172.21.15.42:0/3877541743 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbb406c2c0 con 0x7ffbb40b5650 2024-07-04T17:36:57.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.909+0000 7ffbabe00640 1 --2- 172.21.15.42:0/3877541743 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7ffb9403d340 0x7ffb9403f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:36:57.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.909+0000 7ffbabe00640 1 -- 172.21.15.42:0/3877541743 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 4863+0+0 (secure 0 0 0) 0x7ffba4039040 con 0x7ffbb40b5650 2024-07-04T17:36:57.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.909+0000 7ffbb3400640 1 -- 172.21.15.42:0/3877541743 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7ffb9403d340 msgr2=0x7ffb9403f850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/3991875201 2024-07-04T17:36:57.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.909+0000 7ffbb3400640 1 --2- 172.21.15.42:0/3877541743 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7ffb9403d340 0x7ffb9403f850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:36:57.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:57.914+0000 7ffbabe00640 1 -- 172.21.15.42:0/3877541743 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7ffba404b030 con 0x7ffbb40b5650 2024-07-04T17:36:58.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.110+0000 7ffbb3400640 1 -- 172.21.15.42:0/3877541743 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7ffb9403d340 msgr2=0x7ffb9403f850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/3991875201 2024-07-04T17:36:58.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.110+0000 7ffbb3400640 1 --2- 172.21.15.42:0/3877541743 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7ffb9403d340 0x7ffb9403f850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:36:58.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.140+0000 7ffbb9400640 1 -- 172.21.15.42:0/3877541743 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ffbb40bdc00 con 0x7ffbb40b5650 2024-07-04T17:36:58.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.140+0000 7ffbabe00640 1 -- 172.21.15.42:0/3877541743 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 11 entries v144) ==== 81+0+643 (secure 0 0 0) 0x7ffba4030e80 con 0x7ffbb40b5650 2024-07-04T17:36:58.140 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2937116305 2024-07-04T18:36:04.169055+0000 2024-07-04T17:36:58.140 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3482046390 2024-07-05T17:35:07.918359+0000 2024-07-04T17:36:58.141 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1404855452 2024-07-05T17:35:07.918359+0000 2024-07-04T17:36:58.141 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1956174992 2024-07-05T17:35:07.918359+0000 2024-07-04T17:36:58.141 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3978634668 2024-07-05T17:35:07.918359+0000 2024-07-04T17:36:58.141 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3294509689 2024-07-05T17:35:07.918359+0000 2024-07-04T17:36:58.141 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6835/4064094064 2024-07-05T17:35:07.918359+0000 2024-07-04T17:36:58.141 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6834/4064094064 2024-07-05T17:35:07.918359+0000 2024-07-04T17:36:58.141 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2180653394 2024-07-05T17:35:07.918359+0000 2024-07-04T17:36:58.141 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2328963399 2024-07-05T17:35:07.918359+0000 2024-07-04T17:36:58.141 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/644737563 2024-07-05T17:35:07.918359+0000 2024-07-04T17:36:58.141 INFO:teuthology.orchestra.run.smithi042.stderr:listed 11 entries 2024-07-04T17:36:58.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.143+0000 7ffbb9400640 1 -- 172.21.15.42:0/3877541743 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7ffb9403d340 msgr2=0x7ffb9403f850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:36:58.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.143+0000 7ffbb9400640 1 --2- 172.21.15.42:0/3877541743 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7ffb9403d340 0x7ffb9403f850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:36:58.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.143+0000 7ffbb9400640 1 -- 172.21.15.42:0/3877541743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbb40b5650 msgr2=0x7ffbb40bd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:36:58.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.143+0000 7ffbb9400640 1 --2- 172.21.15.42:0/3877541743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbb40b5650 0x7ffbb40bd6c0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7ffba400b3c0 tx=0x7ffba402ccf0 comp rx=0 tx=0).stop 2024-07-04T17:36:58.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.143+0000 7ffbb9400640 1 -- 172.21.15.42:0/3877541743 shutdown_connections 2024-07-04T17:36:58.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.143+0000 7ffbb9400640 1 --2- 172.21.15.42:0/3877541743 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7ffb9403d340 0x7ffb9403f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:36:58.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.143+0000 7ffbb9400640 1 --2- 172.21.15.42:0/3877541743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffbb40b5650 0x7ffbb40bd6c0 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:36:58.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.144+0000 7ffbb9400640 1 -- 172.21.15.42:0/3877541743 >> 172.21.15.42:0/3877541743 conn(0x7ffbb40bebc0 msgr2=0x7ffbb41403e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:36:58.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.144+0000 7ffbb9400640 1 -- 172.21.15.42:0/3877541743 shutdown_connections 2024-07-04T17:36:58.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.144+0000 7ffbb9400640 1 -- 172.21.15.42:0/3877541743 wait complete. 2024-07-04T17:36:58.171 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi042.front.sepia.ceph.com... 2024-07-04T17:36:58.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:36:58.171 DEBUG:teuthology.orchestra.run.smithi042:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-07-04T17:36:58.242 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi042.stderr:ceph-fuse[70862]: fuse finished with error 0 and tester_r 0 2024-07-04T17:36:58.247 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-07-04T17:36:58.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:36:58.247 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-07-04T17:36:58.417 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:36:58.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.737+0000 7f5810000640 1 -- 172.21.15.42:0/666360510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58080b3030 msgr2=0x7f58080b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:36:58.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.737+0000 7f5810000640 1 --2- 172.21.15.42:0/666360510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58080b3030 0x7f58080b01f0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f57f4005fd0 tx=0x7f57f402ae80 comp rx=0 tx=0).stop 2024-07-04T17:36:58.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.738+0000 7f5810000640 1 -- 172.21.15.42:0/666360510 shutdown_connections 2024-07-04T17:36:58.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.738+0000 7f5810000640 1 --2- 172.21.15.42:0/666360510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58080b3030 0x7f58080b01f0 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:36:58.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.738+0000 7f5810000640 1 -- 172.21.15.42:0/666360510 >> 172.21.15.42:0/666360510 conn(0x7f58080bf700 msgr2=0x7f58080af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:36:58.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.738+0000 7f5810000640 1 -- 172.21.15.42:0/666360510 shutdown_connections 2024-07-04T17:36:58.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.738+0000 7f5810000640 1 -- 172.21.15.42:0/666360510 wait complete. 2024-07-04T17:36:58.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.740+0000 7f5810000640 1 Processor -- start 2024-07-04T17:36:58.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.740+0000 7f5810000640 1 -- start start 2024-07-04T17:36:58.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.741+0000 7f5810000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f580806cea0 0x7f58080ad180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:36:58.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.741+0000 7f5810000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f580806ef50 con 0x7f580806cea0 2024-07-04T17:36:58.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.741+0000 7f580d600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f580806cea0 0x7f58080ad180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:36:58.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.741+0000 7f580d600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f580806cea0 0x7f58080ad180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43718/0 (socket says 172.21.15.42:43718) 2024-07-04T17:36:58.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.742+0000 7f580d600640 1 -- 172.21.15.42:0/511872122 learned_addr learned my addr 172.21.15.42:0/511872122 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:36:58.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.742+0000 7f580d600640 1 -- 172.21.15.42:0/511872122 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5808073970 con 0x7f580806cea0 2024-07-04T17:36:58.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.742+0000 7f580d600640 1 --2- 172.21.15.42:0/511872122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f580806cea0 0x7f58080ad180 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00c380 tx=0x7f57fc00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:36:58.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.743+0000 7f5805600640 1 -- 172.21.15.42:0/511872122 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57fc017070 con 0x7f580806cea0 2024-07-04T17:36:58.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.743+0000 7f5805600640 1 -- 172.21.15.42:0/511872122 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f57fc0052f0 con 0x7f580806cea0 2024-07-04T17:36:58.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.743+0000 7f5805600640 1 -- 172.21.15.42:0/511872122 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57fc0035f0 con 0x7f580806cea0 2024-07-04T17:36:58.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.743+0000 7f5810000640 1 -- 172.21.15.42:0/511872122 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58080adbe0 con 0x7f580806cea0 2024-07-04T17:36:58.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.743+0000 7f5810000640 1 -- 172.21.15.42:0/511872122 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5808073e10 con 0x7f580806cea0 2024-07-04T17:36:58.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.744+0000 7f5810000640 1 -- 172.21.15.42:0/511872122 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58080b01f0 con 0x7f580806cea0 2024-07-04T17:36:58.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.744+0000 7f5805600640 1 -- 172.21.15.42:0/511872122 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 60) ==== 49464+0+0 (secure 0 0 0) 0x7f57fc01a070 con 0x7f580806cea0 2024-07-04T17:36:58.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.745+0000 7f5805600640 1 --2- 172.21.15.42:0/511872122 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f57e803d340 0x7f57e803f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:36:58.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.745+0000 7f5805600640 1 -- 172.21.15.42:0/511872122 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 4863+0+0 (secure 0 0 0) 0x7f57fc050f00 con 0x7f580806cea0 2024-07-04T17:36:58.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.745+0000 7f580cc00640 1 -- 172.21.15.42:0/511872122 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f57e803d340 msgr2=0x7f57e803f850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/3991875201 2024-07-04T17:36:58.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.745+0000 7f580cc00640 1 --2- 172.21.15.42:0/511872122 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f57e803d340 0x7f57e803f850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:36:58.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.750+0000 7f5805600640 1 -- 172.21.15.42:0/511872122 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f57fc016170 con 0x7f580806cea0 2024-07-04T17:36:58.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.947+0000 7f580cc00640 1 -- 172.21.15.42:0/511872122 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f57e803d340 msgr2=0x7f57e803f850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/3991875201 2024-07-04T17:36:58.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:58.947+0000 7f580cc00640 1 --2- 172.21.15.42:0/511872122 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f57e803d340 0x7f57e803f850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:36:59.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.000+0000 7f5810000640 1 -- 172.21.15.42:0/511872122 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5808073b40 con 0x7f580806cea0 2024-07-04T17:36:59.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.001+0000 7f5805600640 1 -- 172.21.15.42:0/511872122 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) ==== 95+0+7539 (secure 0 0 0) 0x7f57fc04f7e0 con 0x7f580806cea0 2024-07-04T17:36:59.001 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:36:59.001 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":115,"btime":"2024-07-04T17:35:22:210884+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5803,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/1184311185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":1184311185},{"type":"v1","addr":"172.21.15.42:6825","nonce":1184311185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103},{"gid":5808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/1815554749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1815554749},{"type":"v1","addr":"172.21.15.42:6827","nonce":1815554749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":5809,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/2027504764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2027504764},{"type":"v1","addr":"172.21.15.42:6829","nonce":2027504764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:34:35.049685+0000","modified":"2024-07-04T17:35:21.376518+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5818},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5818":{"gid":5818,"name":"d","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6833/1863956336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1863956336},{"type":"v1","addr":"172.21.15.42:6833","nonce":1863956336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5818,"qdb_cluster":[5818]},"id":12,"mirror_info":{"peers":{"dedc342a-5e11-4049-987c-94fa3cabd62f":{"remote":{"client_name":"client.mirror_remote","cluster_name":"ceph","fs_name":"backup_fs"}}}}},{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:34:54.794041+0000","modified":"2024-07-04T17:34:57.841598+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5817},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5817":{"gid":5817,"name":"a","rank":0,"incarnation":110,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6831/4064998265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":4064998265},{"type":"v1","addr":"172.21.15.42:6831","nonce":4064998265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5817,"qdb_cluster":[5817]},"id":13}]} 2024-07-04T17:36:59.002 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 115 2024-07-04T17:36:59.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.004+0000 7f5810000640 1 -- 172.21.15.42:0/511872122 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f57e803d340 msgr2=0x7f57e803f850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:36:59.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.004+0000 7f5810000640 1 --2- 172.21.15.42:0/511872122 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f57e803d340 0x7f57e803f850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:36:59.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.004+0000 7f5810000640 1 -- 172.21.15.42:0/511872122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f580806cea0 msgr2=0x7f58080ad180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:36:59.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.004+0000 7f5810000640 1 --2- 172.21.15.42:0/511872122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f580806cea0 0x7f58080ad180 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00c380 tx=0x7f57fc00c840 comp rx=0 tx=0).stop 2024-07-04T17:36:59.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.004+0000 7f5810000640 1 -- 172.21.15.42:0/511872122 shutdown_connections 2024-07-04T17:36:59.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.004+0000 7f5810000640 1 --2- 172.21.15.42:0/511872122 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f57e803d340 0x7f57e803f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:36:59.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.004+0000 7f5810000640 1 --2- 172.21.15.42:0/511872122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f580806cea0 0x7f58080ad180 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:36:59.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.004+0000 7f5810000640 1 -- 172.21.15.42:0/511872122 >> 172.21.15.42:0/511872122 conn(0x7f58080bf700 msgr2=0x7f58080aa3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:36:59.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.005+0000 7f5810000640 1 -- 172.21.15.42:0/511872122 shutdown_connections 2024-07-04T17:36:59.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.005+0000 7f5810000640 1 -- 172.21.15.42:0/511872122 wait complete. 2024-07-04T17:36:59.031 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:36:59.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.288+0000 7facb7a00640 1 -- 172.21.15.42:0/121851488 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facb00b38b0 msgr2=0x7facb00b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:36:59.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.288+0000 7facb7a00640 1 --2- 172.21.15.42:0/121851488 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facb00b38b0 0x7facb00b2d20 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7faca4005fd0 tx=0x7faca402a5f0 comp rx=0 tx=0).stop 2024-07-04T17:36:59.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.288+0000 7facb7a00640 1 -- 172.21.15.42:0/121851488 shutdown_connections 2024-07-04T17:36:59.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.288+0000 7facb7a00640 1 --2- 172.21.15.42:0/121851488 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facb00b38b0 0x7facb00b2d20 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:36:59.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.288+0000 7facb7a00640 1 -- 172.21.15.42:0/121851488 >> 172.21.15.42:0/121851488 conn(0x7facb00bf700 msgr2=0x7facb00af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:36:59.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.288+0000 7facb7a00640 1 -- 172.21.15.42:0/121851488 shutdown_connections 2024-07-04T17:36:59.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.288+0000 7facb7a00640 1 -- 172.21.15.42:0/121851488 wait complete. 2024-07-04T17:36:59.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.290+0000 7facb7a00640 1 Processor -- start 2024-07-04T17:36:59.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.291+0000 7facb7a00640 1 -- start start 2024-07-04T17:36:59.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.292+0000 7facb7a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facb00b1e00 0x7facb01d4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:36:59.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.292+0000 7facb7a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7facb00b06e0 con 0x7facb00b1e00 2024-07-04T17:36:59.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.292+0000 7facb5000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facb00b1e00 0x7facb01d4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:36:59.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.292+0000 7facb5000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facb00b1e00 0x7facb01d4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43742/0 (socket says 172.21.15.42:43742) 2024-07-04T17:36:59.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.292+0000 7facb5000640 1 -- 172.21.15.42:0/679790872 learned_addr learned my addr 172.21.15.42:0/679790872 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:36:59.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.293+0000 7facb5000640 1 -- 172.21.15.42:0/679790872 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb01d6740 con 0x7facb00b1e00 2024-07-04T17:36:59.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.293+0000 7facb5000640 1 --2- 172.21.15.42:0/679790872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facb00b1e00 0x7facb01d4df0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7faca4009850 tx=0x7faca40049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:36:59.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.293+0000 7facacc00640 1 -- 172.21.15.42:0/679790872 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faca403e030 con 0x7facb00b1e00 2024-07-04T17:36:59.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.293+0000 7facacc00640 1 -- 172.21.15.42:0/679790872 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faca40035f0 con 0x7facb00b1e00 2024-07-04T17:36:59.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.293+0000 7facacc00640 1 -- 172.21.15.42:0/679790872 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faca4035040 con 0x7facb00b1e00 2024-07-04T17:36:59.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.293+0000 7facb7a00640 1 -- 172.21.15.42:0/679790872 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facb00aa540 con 0x7facb00b1e00 2024-07-04T17:36:59.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.293+0000 7facb7a00640 1 -- 172.21.15.42:0/679790872 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facb01d5330 con 0x7facb00b1e00 2024-07-04T17:36:59.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.295+0000 7facb7a00640 1 -- 172.21.15.42:0/679790872 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facb006c590 con 0x7facb00b1e00 2024-07-04T17:36:59.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.295+0000 7facacc00640 1 -- 172.21.15.42:0/679790872 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 60) ==== 49464+0+0 (secure 0 0 0) 0x7faca4041030 con 0x7facb00b1e00 2024-07-04T17:36:59.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.295+0000 7facacc00640 1 --2- 172.21.15.42:0/679790872 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fac9003d340 0x7fac9003f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:36:59.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.295+0000 7facacc00640 1 -- 172.21.15.42:0/679790872 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 4863+0+0 (secure 0 0 0) 0x7faca403f070 con 0x7facb00b1e00 2024-07-04T17:36:59.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.295+0000 7facafe00640 1 -- 172.21.15.42:0/679790872 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fac9003d340 msgr2=0x7fac9003f850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/3991875201 2024-07-04T17:36:59.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.295+0000 7facafe00640 1 --2- 172.21.15.42:0/679790872 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fac9003d340 0x7fac9003f850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:36:59.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.300+0000 7facacc00640 1 -- 172.21.15.42:0/679790872 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7faca404bd60 con 0x7facb00b1e00 2024-07-04T17:36:59.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.497+0000 7facafe00640 1 -- 172.21.15.42:0/679790872 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fac9003d340 msgr2=0x7fac9003f850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/3991875201 2024-07-04T17:36:59.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.497+0000 7facafe00640 1 --2- 172.21.15.42:0/679790872 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fac9003d340 0x7fac9003f850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:36:59.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.557+0000 7facb7a00640 1 -- 172.21.15.42:0/679790872 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7facb0069b70 con 0x7facb00b1e00 2024-07-04T17:36:59.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.558+0000 7facacc00640 1 -- 172.21.15.42:0/679790872 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) ==== 95+0+7539 (secure 0 0 0) 0x7faca4030cc0 con 0x7facb00b1e00 2024-07-04T17:36:59.558 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:36:59.559 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":115,"btime":"2024-07-04T17:35:22:210884+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5803,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/1184311185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":1184311185},{"type":"v1","addr":"172.21.15.42:6825","nonce":1184311185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103},{"gid":5808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/1815554749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1815554749},{"type":"v1","addr":"172.21.15.42:6827","nonce":1815554749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":5809,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/2027504764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2027504764},{"type":"v1","addr":"172.21.15.42:6829","nonce":2027504764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:34:35.049685+0000","modified":"2024-07-04T17:35:21.376518+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5818},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5818":{"gid":5818,"name":"d","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6833/1863956336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1863956336},{"type":"v1","addr":"172.21.15.42:6833","nonce":1863956336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5818,"qdb_cluster":[5818]},"id":12,"mirror_info":{"peers":{"dedc342a-5e11-4049-987c-94fa3cabd62f":{"remote":{"client_name":"client.mirror_remote","cluster_name":"ceph","fs_name":"backup_fs"}}}}},{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:34:54.794041+0000","modified":"2024-07-04T17:34:57.841598+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5817},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5817":{"gid":5817,"name":"a","rank":0,"incarnation":110,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6831/4064998265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":4064998265},{"type":"v1","addr":"172.21.15.42:6831","nonce":4064998265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5817,"qdb_cluster":[5817]},"id":13}]} 2024-07-04T17:36:59.559 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 115 2024-07-04T17:36:59.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.561+0000 7facb7a00640 1 -- 172.21.15.42:0/679790872 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fac9003d340 msgr2=0x7fac9003f850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:36:59.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.561+0000 7facb7a00640 1 --2- 172.21.15.42:0/679790872 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fac9003d340 0x7fac9003f850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:36:59.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.561+0000 7facb7a00640 1 -- 172.21.15.42:0/679790872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facb00b1e00 msgr2=0x7facb01d4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:36:59.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.561+0000 7facb7a00640 1 --2- 172.21.15.42:0/679790872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facb00b1e00 0x7facb01d4df0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7faca4009850 tx=0x7faca40049d0 comp rx=0 tx=0).stop 2024-07-04T17:36:59.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.561+0000 7facb7a00640 1 -- 172.21.15.42:0/679790872 shutdown_connections 2024-07-04T17:36:59.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.561+0000 7facb7a00640 1 --2- 172.21.15.42:0/679790872 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7fac9003d340 0x7fac9003f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:36:59.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.561+0000 7facb7a00640 1 --2- 172.21.15.42:0/679790872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facb00b1e00 0x7facb01d4df0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:36:59.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.561+0000 7facb7a00640 1 -- 172.21.15.42:0/679790872 >> 172.21.15.42:0/679790872 conn(0x7facb00bf700 msgr2=0x7facb01412f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:36:59.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.562+0000 7facb7a00640 1 -- 172.21.15.42:0/679790872 shutdown_connections 2024-07-04T17:36:59.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.562+0000 7facb7a00640 1 -- 172.21.15.42:0/679790872 wait complete. 2024-07-04T17:36:59.588 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:36:59.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.859+0000 7f795b200640 1 -- 172.21.15.42:0/3649822249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f795c0b5ae0 msgr2=0x7f795c0c38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:36:59.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.859+0000 7f795b200640 1 --2- 172.21.15.42:0/3649822249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f795c0b5ae0 0x7f795c0c38b0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f794c005fd0 tx=0x7f794c02a5f0 comp rx=0 tx=0).stop 2024-07-04T17:36:59.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.859+0000 7f795b200640 1 -- 172.21.15.42:0/3649822249 shutdown_connections 2024-07-04T17:36:59.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.859+0000 7f795b200640 1 --2- 172.21.15.42:0/3649822249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f795c0b5ae0 0x7f795c0c38b0 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:36:59.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.859+0000 7f795b200640 1 -- 172.21.15.42:0/3649822249 >> 172.21.15.42:0/3649822249 conn(0x7f795c06dd10 msgr2=0x7f795c06e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:36:59.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.859+0000 7f795b200640 1 -- 172.21.15.42:0/3649822249 shutdown_connections 2024-07-04T17:36:59.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.859+0000 7f795b200640 1 -- 172.21.15.42:0/3649822249 wait complete. 2024-07-04T17:36:59.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.861+0000 7f795b200640 1 Processor -- start 2024-07-04T17:36:59.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.861+0000 7f795b200640 1 -- start start 2024-07-04T17:36:59.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.862+0000 7f795b200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f795c0b5ae0 0x7f795c0ba990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:36:59.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.862+0000 7f795b200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f795c0b76d0 con 0x7f795c0b5ae0 2024-07-04T17:36:59.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.863+0000 7f7959e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f795c0b5ae0 0x7f795c0ba990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:36:59.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.863+0000 7f7959e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f795c0b5ae0 0x7f795c0ba990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43750/0 (socket says 172.21.15.42:43750) 2024-07-04T17:36:59.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.863+0000 7f7959e00640 1 -- 172.21.15.42:0/1834364584 learned_addr learned my addr 172.21.15.42:0/1834364584 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:36:59.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.863+0000 7f7959e00640 1 -- 172.21.15.42:0/1834364584 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795c0bbb50 con 0x7f795c0b5ae0 2024-07-04T17:36:59.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.863+0000 7f7959e00640 1 --2- 172.21.15.42:0/1834364584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f795c0b5ae0 0x7f795c0ba990 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f794c034040 tx=0x7f794c0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:36:59.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.864+0000 7f7952000640 1 -- 172.21.15.42:0/1834364584 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f794c03e030 con 0x7f795c0b5ae0 2024-07-04T17:36:59.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.864+0000 7f7952000640 1 -- 172.21.15.42:0/1834364584 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f794c0035f0 con 0x7f795c0b5ae0 2024-07-04T17:36:59.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.864+0000 7f7952000640 1 -- 172.21.15.42:0/1834364584 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f794c035040 con 0x7f795c0b5ae0 2024-07-04T17:36:59.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.864+0000 7f795b200640 1 -- 172.21.15.42:0/1834364584 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f795c13b750 con 0x7f795c0b5ae0 2024-07-04T17:36:59.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.864+0000 7f795b200640 1 -- 172.21.15.42:0/1834364584 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f795c13db20 con 0x7f795c0b5ae0 2024-07-04T17:36:59.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.865+0000 7f795b200640 1 -- 172.21.15.42:0/1834364584 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f795c0b7d30 con 0x7f795c0b5ae0 2024-07-04T17:36:59.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.865+0000 7f7952000640 1 -- 172.21.15.42:0/1834364584 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 60) ==== 49464+0+0 (secure 0 0 0) 0x7f794c041030 con 0x7f795c0b5ae0 2024-07-04T17:36:59.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.866+0000 7f7952000640 1 --2- 172.21.15.42:0/1834364584 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f793c03d340 0x7f793c03f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:36:59.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.866+0000 7f7952000640 1 -- 172.21.15.42:0/1834364584 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 4863+0+0 (secure 0 0 0) 0x7f794c03f070 con 0x7f795c0b5ae0 2024-07-04T17:36:59.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.866+0000 7f7959400640 1 -- 172.21.15.42:0/1834364584 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f793c03d340 msgr2=0x7f793c03f850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/3991875201 2024-07-04T17:36:59.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.866+0000 7f7959400640 1 --2- 172.21.15.42:0/1834364584 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f793c03d340 0x7f793c03f850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:36:59.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:36:59.875+0000 7f7952000640 1 -- 172.21.15.42:0/1834364584 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f794c04bd60 con 0x7f795c0b5ae0 2024-07-04T17:37:00.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.068+0000 7f7959400640 1 -- 172.21.15.42:0/1834364584 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f793c03d340 msgr2=0x7f793c03f850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/3991875201 2024-07-04T17:37:00.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.068+0000 7f7959400640 1 --2- 172.21.15.42:0/1834364584 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f793c03d340 0x7f793c03f850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:37:00.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.085+0000 7f795b200640 1 -- 172.21.15.42:0/1834364584 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f795c068dc0 con 0x7f795c0b5ae0 2024-07-04T17:37:00.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.087+0000 7f7952000640 1 -- 172.21.15.42:0/1834364584 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+15289 (secure 0 0 0) 0x7f794c030cc0 con 0x7f795c0b5ae0 2024-07-04T17:37:00.088 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:37:00.088 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":144,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:36:09.312059+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:34:33.191097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-07-04T17:34:34.033408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1699999570846558,"score_stable":1.1699999570846558,"optimal_score":1,"raw_score_acting":1.1699999570846558,"raw_score_stable":1.1699999570846558,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:34:52.310342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:34:53.756688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.d","mappings":[{"from":0,"to":1}]},{"pgid":"25.25","mappings":[{"from":0,"to":2}]},{"pgid":"27.2d","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/2937116305":"2024-07-04T18:36:04.169055+0000","172.21.15.42:0/3482046390":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/1404855452":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/1956174992":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/3978634668":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/3294509689":"2024-07-05T17:35:07.918359+0000","172.21.15.42:6835/4064094064":"2024-07-05T17:35:07.918359+0000","172.21.15.42:6834/4064094064":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/2180653394":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/2328963399":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/644737563":"2024-07-05T17:35:07.918359+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:37:00.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.090+0000 7f795b200640 1 -- 172.21.15.42:0/1834364584 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f793c03d340 msgr2=0x7f793c03f850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:37:00.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.090+0000 7f795b200640 1 --2- 172.21.15.42:0/1834364584 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f793c03d340 0x7f793c03f850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:00.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.091+0000 7f795b200640 1 -- 172.21.15.42:0/1834364584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f795c0b5ae0 msgr2=0x7f795c0ba990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:00.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.091+0000 7f795b200640 1 --2- 172.21.15.42:0/1834364584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f795c0b5ae0 0x7f795c0ba990 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f794c034040 tx=0x7f794c0049d0 comp rx=0 tx=0).stop 2024-07-04T17:37:00.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.091+0000 7f795b200640 1 -- 172.21.15.42:0/1834364584 shutdown_connections 2024-07-04T17:37:00.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.091+0000 7f795b200640 1 --2- 172.21.15.42:0/1834364584 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f793c03d340 0x7f793c03f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:00.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.091+0000 7f795b200640 1 --2- 172.21.15.42:0/1834364584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f795c0b5ae0 0x7f795c0ba990 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:00.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.091+0000 7f795b200640 1 -- 172.21.15.42:0/1834364584 >> 172.21.15.42:0/1834364584 conn(0x7f795c06dd10 msgr2=0x7f795c0c1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:00.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.091+0000 7f795b200640 1 -- 172.21.15.42:0/1834364584 shutdown_connections 2024-07-04T17:37:00.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.091+0000 7f795b200640 1 -- 172.21.15.42:0/1834364584 wait complete. 2024-07-04T17:37:00.119 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-04T17:37:00.120 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:37:00.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.387+0000 7f88f7200640 1 -- 172.21.15.42:0/1571531360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88f0141590 msgr2=0x7f88f01419a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:00.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.387+0000 7f88f7200640 1 --2- 172.21.15.42:0/1571531360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88f0141590 0x7f88f01419a0 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f88d8005fd0 tx=0x7f88d802a5f0 comp rx=0 tx=0).stop 2024-07-04T17:37:00.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.387+0000 7f88f7200640 1 -- 172.21.15.42:0/1571531360 shutdown_connections 2024-07-04T17:37:00.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.387+0000 7f88f7200640 1 --2- 172.21.15.42:0/1571531360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88f0141590 0x7f88f01419a0 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:00.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.387+0000 7f88f7200640 1 -- 172.21.15.42:0/1571531360 >> 172.21.15.42:0/1571531360 conn(0x7f88f006c940 msgr2=0x7f88f006ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:00.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.387+0000 7f88f7200640 1 -- 172.21.15.42:0/1571531360 shutdown_connections 2024-07-04T17:37:00.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.387+0000 7f88f7200640 1 -- 172.21.15.42:0/1571531360 wait complete. 2024-07-04T17:37:00.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.389+0000 7f88f7200640 1 Processor -- start 2024-07-04T17:37:00.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.390+0000 7f88f7200640 1 -- start start 2024-07-04T17:37:00.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.391+0000 7f88f7200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88f00bc040 0x7f88f00b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:00.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.391+0000 7f88f7200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f88f00b7530 con 0x7f88f00bc040 2024-07-04T17:37:00.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.391+0000 7f88efe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88f00bc040 0x7f88f00b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:00.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.391+0000 7f88efe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88f00bc040 0x7f88f00b2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43776/0 (socket says 172.21.15.42:43776) 2024-07-04T17:37:00.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.391+0000 7f88efe00640 1 -- 172.21.15.42:0/2268484759 learned_addr learned my addr 172.21.15.42:0/2268484759 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:37:00.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.392+0000 7f88efe00640 1 -- 172.21.15.42:0/2268484759 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f00c4210 con 0x7f88f00bc040 2024-07-04T17:37:00.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.392+0000 7f88efe00640 1 --2- 172.21.15.42:0/2268484759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88f00bc040 0x7f88f00b2ce0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f88e000c380 tx=0x7f88e000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:00.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.392+0000 7f88ecc00640 1 -- 172.21.15.42:0/2268484759 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88e0017070 con 0x7f88f00bc040 2024-07-04T17:37:00.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.392+0000 7f88ecc00640 1 -- 172.21.15.42:0/2268484759 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88e00052f0 con 0x7f88f00bc040 2024-07-04T17:37:00.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.392+0000 7f88ecc00640 1 -- 172.21.15.42:0/2268484759 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88e00035f0 con 0x7f88f00bc040 2024-07-04T17:37:00.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.393+0000 7f88f7200640 1 -- 172.21.15.42:0/2268484759 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88f01e45a0 con 0x7f88f00bc040 2024-07-04T17:37:00.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.393+0000 7f88f7200640 1 -- 172.21.15.42:0/2268484759 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88f00c3c40 con 0x7f88f00bc040 2024-07-04T17:37:00.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.394+0000 7f88f7200640 1 -- 172.21.15.42:0/2268484759 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88f00b7bb0 con 0x7f88f00bc040 2024-07-04T17:37:00.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.395+0000 7f88ecc00640 1 -- 172.21.15.42:0/2268484759 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 60) ==== 49464+0+0 (secure 0 0 0) 0x7f88e001a070 con 0x7f88f00bc040 2024-07-04T17:37:00.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.395+0000 7f88ecc00640 1 --2- 172.21.15.42:0/2268484759 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f88cc03d340 0x7f88cc03f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:00.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.395+0000 7f88ecc00640 1 -- 172.21.15.42:0/2268484759 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 4863+0+0 (secure 0 0 0) 0x7f88e0050e40 con 0x7f88f00bc040 2024-07-04T17:37:00.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.395+0000 7f88e7e00640 1 -- 172.21.15.42:0/2268484759 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f88cc03d340 msgr2=0x7f88cc03f850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/3991875201 2024-07-04T17:37:00.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.395+0000 7f88e7e00640 1 --2- 172.21.15.42:0/2268484759 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f88cc03d340 0x7f88cc03f850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:37:00.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.401+0000 7f88ecc00640 1 -- 172.21.15.42:0/2268484759 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f88e0016170 con 0x7f88f00bc040 2024-07-04T17:37:00.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.596+0000 7f88e7e00640 1 -- 172.21.15.42:0/2268484759 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f88cc03d340 msgr2=0x7f88cc03f850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/3991875201 2024-07-04T17:37:00.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.596+0000 7f88e7e00640 1 --2- 172.21.15.42:0/2268484759 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f88cc03d340 0x7f88cc03f850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:37:00.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.643+0000 7f88f7200640 1 -- 172.21.15.42:0/2268484759 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88f00c43e0 con 0x7f88f00bc040 2024-07-04T17:37:00.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.644+0000 7f88ecc00640 1 -- 172.21.15.42:0/2268484759 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) ==== 95+0+7539 (secure 0 0 0) 0x7f88e00192c0 con 0x7f88f00bc040 2024-07-04T17:37:00.644 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:37:00.644 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":115,"btime":"2024-07-04T17:35:22:210884+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5803,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/1184311185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":1184311185},{"type":"v1","addr":"172.21.15.42:6825","nonce":1184311185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103},{"gid":5808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/1815554749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1815554749},{"type":"v1","addr":"172.21.15.42:6827","nonce":1815554749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":5809,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/2027504764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2027504764},{"type":"v1","addr":"172.21.15.42:6829","nonce":2027504764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:34:35.049685+0000","modified":"2024-07-04T17:35:21.376518+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5818},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5818":{"gid":5818,"name":"d","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6833/1863956336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1863956336},{"type":"v1","addr":"172.21.15.42:6833","nonce":1863956336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5818,"qdb_cluster":[5818]},"id":12,"mirror_info":{"peers":{"dedc342a-5e11-4049-987c-94fa3cabd62f":{"remote":{"client_name":"client.mirror_remote","cluster_name":"ceph","fs_name":"backup_fs"}}}}},{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:34:54.794041+0000","modified":"2024-07-04T17:34:57.841598+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5817},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5817":{"gid":5817,"name":"a","rank":0,"incarnation":110,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6831/4064998265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":4064998265},{"type":"v1","addr":"172.21.15.42:6831","nonce":4064998265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5817,"qdb_cluster":[5817]},"id":13}]} 2024-07-04T17:37:00.644 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 115 2024-07-04T17:37:00.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.647+0000 7f88f7200640 1 -- 172.21.15.42:0/2268484759 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f88cc03d340 msgr2=0x7f88cc03f850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:37:00.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.647+0000 7f88f7200640 1 --2- 172.21.15.42:0/2268484759 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f88cc03d340 0x7f88cc03f850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:00.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.647+0000 7f88f7200640 1 -- 172.21.15.42:0/2268484759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88f00bc040 msgr2=0x7f88f00b2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:00.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.647+0000 7f88f7200640 1 --2- 172.21.15.42:0/2268484759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88f00bc040 0x7f88f00b2ce0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f88e000c380 tx=0x7f88e000c840 comp rx=0 tx=0).stop 2024-07-04T17:37:00.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.647+0000 7f88f7200640 1 -- 172.21.15.42:0/2268484759 shutdown_connections 2024-07-04T17:37:00.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.647+0000 7f88f7200640 1 --2- 172.21.15.42:0/2268484759 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f88cc03d340 0x7f88cc03f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:00.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.647+0000 7f88f7200640 1 --2- 172.21.15.42:0/2268484759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88f00bc040 0x7f88f00b2ce0 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:00.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.647+0000 7f88f7200640 1 -- 172.21.15.42:0/2268484759 >> 172.21.15.42:0/2268484759 conn(0x7f88f006c940 msgr2=0x7f88f0145cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:00.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.647+0000 7f88f7200640 1 -- 172.21.15.42:0/2268484759 shutdown_connections 2024-07-04T17:37:00.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.647+0000 7f88f7200640 1 -- 172.21.15.42:0/2268484759 wait complete. 2024-07-04T17:37:00.673 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:37:00.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.938+0000 7f596ac00640 1 -- 172.21.15.42:0/118879840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f596406e0c0 msgr2=0x7f59640b2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:00.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.938+0000 7f596ac00640 1 --2- 172.21.15.42:0/118879840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f596406e0c0 0x7f59640b2cb0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f5958005fd0 tx=0x7f595802a560 comp rx=0 tx=0).stop 2024-07-04T17:37:00.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.938+0000 7f596ac00640 1 -- 172.21.15.42:0/118879840 shutdown_connections 2024-07-04T17:37:00.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.938+0000 7f596ac00640 1 --2- 172.21.15.42:0/118879840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f596406e0c0 0x7f59640b2cb0 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:00.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.938+0000 7f596ac00640 1 -- 172.21.15.42:0/118879840 >> 172.21.15.42:0/118879840 conn(0x7f59640bfd20 msgr2=0x7f59640c0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:00.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.939+0000 7f596ac00640 1 -- 172.21.15.42:0/118879840 shutdown_connections 2024-07-04T17:37:00.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.939+0000 7f596ac00640 1 -- 172.21.15.42:0/118879840 wait complete. 2024-07-04T17:37:00.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.941+0000 7f596ac00640 1 Processor -- start 2024-07-04T17:37:00.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.941+0000 7f596ac00640 1 -- start start 2024-07-04T17:37:00.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.943+0000 7f596ac00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f596406c570 0x7f5964144060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:00.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.943+0000 7f596ac00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f596414baa0 con 0x7f596406c570 2024-07-04T17:37:00.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.943+0000 7f5969800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f596406c570 0x7f5964144060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:00.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.943+0000 7f5969800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f596406c570 0x7f5964144060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43796/0 (socket says 172.21.15.42:43796) 2024-07-04T17:37:00.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.943+0000 7f5969800640 1 -- 172.21.15.42:0/11265913 learned_addr learned my addr 172.21.15.42:0/11265913 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:37:00.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.944+0000 7f5969800640 1 -- 172.21.15.42:0/11265913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59640af010 con 0x7f596406c570 2024-07-04T17:37:00.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.944+0000 7f5969800640 1 --2- 172.21.15.42:0/11265913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f596406c570 0x7f5964144060 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f5958009850 tx=0x7f59580049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:00.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.944+0000 7f5961600640 1 -- 172.21.15.42:0/11265913 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f595803e030 con 0x7f596406c570 2024-07-04T17:37:00.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.945+0000 7f5961600640 1 -- 172.21.15.42:0/11265913 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59580035f0 con 0x7f596406c570 2024-07-04T17:37:00.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.945+0000 7f5961600640 1 -- 172.21.15.42:0/11265913 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5958035040 con 0x7f596406c570 2024-07-04T17:37:00.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.945+0000 7f596ac00640 1 -- 172.21.15.42:0/11265913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59640af2e0 con 0x7f596406c570 2024-07-04T17:37:00.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.945+0000 7f596ac00640 1 -- 172.21.15.42:0/11265913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5964144a40 con 0x7f596406c570 2024-07-04T17:37:00.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.946+0000 7f596ac00640 1 -- 172.21.15.42:0/11265913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f596414c190 con 0x7f596406c570 2024-07-04T17:37:00.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.946+0000 7f5961600640 1 -- 172.21.15.42:0/11265913 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 60) ==== 49464+0+0 (secure 0 0 0) 0x7f5958041030 con 0x7f596406c570 2024-07-04T17:37:00.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.947+0000 7f5961600640 1 --2- 172.21.15.42:0/11265913 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f594403d340 0x7f594403f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:00.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.947+0000 7f5961600640 1 -- 172.21.15.42:0/11265913 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 4863+0+0 (secure 0 0 0) 0x7f595803f070 con 0x7f596406c570 2024-07-04T17:37:00.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.947+0000 7f5968e00640 1 -- 172.21.15.42:0/11265913 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f594403d340 msgr2=0x7f594403f850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/3991875201 2024-07-04T17:37:00.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.947+0000 7f5968e00640 1 --2- 172.21.15.42:0/11265913 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f594403d340 0x7f594403f850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-04T17:37:00.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:00.952+0000 7f5961600640 1 -- 172.21.15.42:0/11265913 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f595804bd60 con 0x7f596406c570 2024-07-04T17:37:01.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.149+0000 7f5968e00640 1 -- 172.21.15.42:0/11265913 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f594403d340 msgr2=0x7f594403f850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6834/3991875201 2024-07-04T17:37:01.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.149+0000 7f5968e00640 1 --2- 172.21.15.42:0/11265913 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f594403d340 0x7f594403f850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-04T17:37:01.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.177+0000 7f5961600640 1 -- 172.21.15.42:0/11265913 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 61) ==== 49122+0+0 (secure 0 0 0) 0x7f5958031400 con 0x7f596406c570 2024-07-04T17:37:01.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.177+0000 7f5961600640 1 -- 172.21.15.42:0/11265913 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f594403d340 msgr2=0x7f594403f850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T17:37:01.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.177+0000 7f5961600640 1 --2- 172.21.15.42:0/11265913 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f594403d340 0x7f594403f850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:01.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.205+0000 7f596ac00640 1 -- 172.21.15.42:0/11265913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f596414b250 con 0x7f596406c570 2024-07-04T17:37:01.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.206+0000 7f5961600640 1 -- 172.21.15.42:0/11265913 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) ==== 95+0+7539 (secure 0 0 0) 0x7f5958069d30 con 0x7f596406c570 2024-07-04T17:37:01.206 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:37:01.206 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":115,"btime":"2024-07-04T17:35:22:210884+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5803,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/1184311185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":1184311185},{"type":"v1","addr":"172.21.15.42:6825","nonce":1184311185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103},{"gid":5808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/1815554749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1815554749},{"type":"v1","addr":"172.21.15.42:6827","nonce":1815554749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":5809,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/2027504764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2027504764},{"type":"v1","addr":"172.21.15.42:6829","nonce":2027504764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:34:35.049685+0000","modified":"2024-07-04T17:35:21.376518+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5818},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5818":{"gid":5818,"name":"d","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.42:6833/1863956336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1863956336},{"type":"v1","addr":"172.21.15.42:6833","nonce":1863956336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5818,"qdb_cluster":[5818]},"id":12,"mirror_info":{"peers":{"dedc342a-5e11-4049-987c-94fa3cabd62f":{"remote":{"client_name":"client.mirror_remote","cluster_name":"ceph","fs_name":"backup_fs"}}}}},{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:34:54.794041+0000","modified":"2024-07-04T17:34:57.841598+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5817},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5817":{"gid":5817,"name":"a","rank":0,"incarnation":110,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6831/4064998265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":4064998265},{"type":"v1","addr":"172.21.15.42:6831","nonce":4064998265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5817,"qdb_cluster":[5817]},"id":13}]} 2024-07-04T17:37:01.206 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 115 2024-07-04T17:37:01.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.208+0000 7f596ac00640 1 -- 172.21.15.42:0/11265913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f596406c570 msgr2=0x7f5964144060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:01.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.208+0000 7f596ac00640 1 --2- 172.21.15.42:0/11265913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f596406c570 0x7f5964144060 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f5958009850 tx=0x7f59580049d0 comp rx=0 tx=0).stop 2024-07-04T17:37:01.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.208+0000 7f596ac00640 1 -- 172.21.15.42:0/11265913 shutdown_connections 2024-07-04T17:37:01.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.208+0000 7f596ac00640 1 --2- 172.21.15.42:0/11265913 >> [v2:172.21.15.42:6834/3991875201,v1:172.21.15.42:6835/3991875201] conn(0x7f594403d340 0x7f594403f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:01.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.208+0000 7f596ac00640 1 --2- 172.21.15.42:0/11265913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f596406c570 0x7f5964144060 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:01.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.208+0000 7f596ac00640 1 -- 172.21.15.42:0/11265913 >> 172.21.15.42:0/11265913 conn(0x7f59640bfd20 msgr2=0x7f5964075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:01.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.208+0000 7f596ac00640 1 -- 172.21.15.42:0/11265913 shutdown_connections 2024-07-04T17:37:01.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.208+0000 7f596ac00640 1 -- 172.21.15.42:0/11265913 wait complete. 2024-07-04T17:37:01.236 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:37:01.256 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:37:01.257+0000 7fc3bda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:37:01.257 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:37:01.257+0000 7fc3bda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:37:01.257 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:37:01.257+0000 7fc3bda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:37:01.257 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:37:01.257+0000 7fc3bda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:37:01.257 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:37:01.257+0000 7fc3bda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:37:01.257 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:37:01.257+0000 7fc3c0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:37:01.257 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:37:01.257+0000 7fc3c0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:37:01.258 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:37:01.257+0000 7fc3c0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:37:01.258 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:37:01.257+0000 7fc3c0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:37:01.258 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:37:01.257+0000 7fc3c0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:37:01.281 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:37:01.281+0000 7fc3c0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:37:01.281 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:37:01.281+0000 7fc3c0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:37:01.282 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:37:01.281+0000 7fc3c0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:37:01.282 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:37:01.281+0000 7fc3c0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:37:01.282 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:37:01.281+0000 7fc3c0c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:37:01.282 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:37:01.282+0000 7fc3bda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:37:01.282 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:37:01.282+0000 7fc3bda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:37:01.282 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:37:01.282+0000 7fc3bda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:37:01.283 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:37:01.282+0000 7fc3bda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:37:01.283 INFO:tasks.ceph.mgr.x.smithi042.stderr:2024-07-04T17:37:01.282+0000 7fc3bda00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T17:37:01.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.499+0000 7f6d99000640 1 -- 172.21.15.42:0/2578740887 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d9406c020 msgr2=0x7f6d9414a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:01.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.499+0000 7f6d99000640 1 --2- 172.21.15.42:0/2578740887 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d9406c020 0x7f6d9414a9e0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f6d800097b0 tx=0x7f6d8002a540 comp rx=0 tx=0).stop 2024-07-04T17:37:01.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.500+0000 7f6d99000640 1 -- 172.21.15.42:0/2578740887 shutdown_connections 2024-07-04T17:37:01.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.500+0000 7f6d99000640 1 --2- 172.21.15.42:0/2578740887 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d9406c020 0x7f6d9414a9e0 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:01.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.500+0000 7f6d99000640 1 -- 172.21.15.42:0/2578740887 >> 172.21.15.42:0/2578740887 conn(0x7f6d940bebc0 msgr2=0x7f6d940befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:01.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.500+0000 7f6d99000640 1 -- 172.21.15.42:0/2578740887 shutdown_connections 2024-07-04T17:37:01.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.500+0000 7f6d99000640 1 -- 172.21.15.42:0/2578740887 wait complete. 2024-07-04T17:37:01.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.502+0000 7f6d99000640 1 Processor -- start 2024-07-04T17:37:01.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.502+0000 7f6d99000640 1 -- start start 2024-07-04T17:37:01.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.503+0000 7f6d99000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d9406c020 0x7f6d940aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:01.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.503+0000 7f6d99000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9414b550 con 0x7f6d9406c020 2024-07-04T17:37:01.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.503+0000 7f6d93400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d9406c020 0x7f6d940aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:01.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.503+0000 7f6d93400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d9406c020 0x7f6d940aeb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43944/0 (socket says 172.21.15.42:43944) 2024-07-04T17:37:01.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.503+0000 7f6d93400640 1 -- 172.21.15.42:0/430927527 learned_addr learned my addr 172.21.15.42:0/430927527 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:37:01.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.504+0000 7f6d93400640 1 -- 172.21.15.42:0/430927527 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d940b6d50 con 0x7f6d9406c020 2024-07-04T17:37:01.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.504+0000 7f6d93400640 1 --2- 172.21.15.42:0/430927527 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d9406c020 0x7f6d940aeb50 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f6d84001eb0 tx=0x7f6d8400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:01.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.504+0000 7f6d8b400640 1 -- 172.21.15.42:0/430927527 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d840116c0 con 0x7f6d9406c020 2024-07-04T17:37:01.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.505+0000 7f6d8b400640 1 -- 172.21.15.42:0/430927527 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d84017070 con 0x7f6d9406c020 2024-07-04T17:37:01.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.505+0000 7f6d8b400640 1 -- 172.21.15.42:0/430927527 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d84011ae0 con 0x7f6d9406c020 2024-07-04T17:37:01.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.505+0000 7f6d99000640 1 -- 172.21.15.42:0/430927527 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d941e7450 con 0x7f6d9406c020 2024-07-04T17:37:01.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.505+0000 7f6d99000640 1 -- 172.21.15.42:0/430927527 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d940b7020 con 0x7f6d9406c020 2024-07-04T17:37:01.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.506+0000 7f6d8b400640 1 -- 172.21.15.42:0/430927527 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 61) ==== 49122+0+0 (secure 0 0 0) 0x7f6d84011c80 con 0x7f6d9406c020 2024-07-04T17:37:01.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.506+0000 7f6d99000640 1 -- 172.21.15.42:0/430927527 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d9414bc80 con 0x7f6d9406c020 2024-07-04T17:37:01.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.507+0000 7f6d8b400640 1 -- 172.21.15.42:0/430927527 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5164+0+0 (secure 0 0 0) 0x7f6d84051510 con 0x7f6d9406c020 2024-07-04T17:37:01.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.512+0000 7f6d8b400640 1 -- 172.21.15.42:0/430927527 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f6d8401c6f0 con 0x7f6d9406c020 2024-07-04T17:37:01.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.725+0000 7f6d99000640 1 -- 172.21.15.42:0/430927527 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d94068e30 con 0x7f6d9406c020 2024-07-04T17:37:01.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.728+0000 7f6d8b400640 1 -- 172.21.15.42:0/430927527 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+15728 (secure 0 0 0) 0x7f6d84010cd0 con 0x7f6d9406c020 2024-07-04T17:37:01.729 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:37:01.729 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":145,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:37:01.159772+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-07-04T17:34:33.191097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-07-04T17:34:34.033408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1699999570846558,"score_stable":1.1699999570846558,"optimal_score":1,"raw_score_acting":1.1699999570846558,"raw_score_stable":1.1699999570846558,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:34:52.310342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:34:53.756688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"25.d","mappings":[{"from":0,"to":1}]},{"pgid":"25.25","mappings":[{"from":0,"to":2}]},{"pgid":"27.2d","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1132989435":"2024-07-05T17:37:01.159615+0000","172.21.15.42:0/3471044288":"2024-07-05T17:37:01.159615+0000","172.21.15.42:6834/3991875201":"2024-07-05T17:37:01.159615+0000","172.21.15.42:0/4263654752":"2024-07-05T17:37:01.159615+0000","172.21.15.42:0/2937116305":"2024-07-04T18:36:04.169055+0000","172.21.15.42:0/3482046390":"2024-07-05T17:35:07.918359+0000","172.21.15.42:6835/3991875201":"2024-07-05T17:37:01.159615+0000","172.21.15.42:0/1404855452":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/2049026251":"2024-07-05T17:37:01.159615+0000","172.21.15.42:0/1956174992":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/3978634668":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/3294509689":"2024-07-05T17:35:07.918359+0000","172.21.15.42:6835/4064094064":"2024-07-05T17:35:07.918359+0000","172.21.15.42:6834/4064094064":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/528150101":"2024-07-05T17:37:01.159615+0000","172.21.15.42:0/2180653394":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/2328963399":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/644737563":"2024-07-05T17:35:07.918359+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:37:01.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.731+0000 7f6d99000640 1 -- 172.21.15.42:0/430927527 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d9406c020 msgr2=0x7f6d940aeb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:01.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.731+0000 7f6d99000640 1 --2- 172.21.15.42:0/430927527 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d9406c020 0x7f6d940aeb50 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f6d84001eb0 tx=0x7f6d8400c810 comp rx=0 tx=0).stop 2024-07-04T17:37:01.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.731+0000 7f6d99000640 1 -- 172.21.15.42:0/430927527 shutdown_connections 2024-07-04T17:37:01.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.731+0000 7f6d99000640 1 --2- 172.21.15.42:0/430927527 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6d9406c020 0x7f6d940aeb50 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:01.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.731+0000 7f6d99000640 1 -- 172.21.15.42:0/430927527 >> 172.21.15.42:0/430927527 conn(0x7f6d940bebc0 msgr2=0x7f6d941d6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:01.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.731+0000 7f6d99000640 1 -- 172.21.15.42:0/430927527 shutdown_connections 2024-07-04T17:37:01.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:01.732+0000 7f6d99000640 1 -- 172.21.15.42:0/430927527 wait complete. 2024-07-04T17:37:01.756 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-07-04T17:37:02.093 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-04T17:37:02.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.095+0000 7f8403200640 1 -- 172.21.15.42:0/3141637077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f840406cf90 msgr2=0x7f8404141dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:02.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.095+0000 7f8403200640 1 --2- 172.21.15.42:0/3141637077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f840406cf90 0x7f8404141dd0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f83f4005fd0 tx=0x7f83f402a5f0 comp rx=0 tx=0).stop 2024-07-04T17:37:02.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.095+0000 7f8403200640 1 -- 172.21.15.42:0/3141637077 shutdown_connections 2024-07-04T17:37:02.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.095+0000 7f8403200640 1 --2- 172.21.15.42:0/3141637077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f840406cf90 0x7f8404141dd0 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:02.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.095+0000 7f8403200640 1 -- 172.21.15.42:0/3141637077 >> 172.21.15.42:0/3141637077 conn(0x7f84040bebc0 msgr2=0x7f84040befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:02.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.095+0000 7f8403200640 1 -- 172.21.15.42:0/3141637077 shutdown_connections 2024-07-04T17:37:02.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.095+0000 7f8403200640 1 -- 172.21.15.42:0/3141637077 wait complete. 2024-07-04T17:37:02.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.097+0000 7f8403200640 1 Processor -- start 2024-07-04T17:37:02.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.097+0000 7f8403200640 1 -- start start 2024-07-04T17:37:02.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.099+0000 7f8403200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f840406cf90 0x7f8404139f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:02.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.099+0000 7f8403200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f84040babe0 con 0x7f840406cf90 2024-07-04T17:37:02.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.099+0000 7f8401e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f840406cf90 0x7f8404139f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:02.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.099+0000 7f8401e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f840406cf90 0x7f8404139f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43958/0 (socket says 172.21.15.42:43958) 2024-07-04T17:37:02.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.099+0000 7f8401e00640 1 -- 172.21.15.42:0/118485747 learned_addr learned my addr 172.21.15.42:0/118485747 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:37:02.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.099+0000 7f8401e00640 1 -- 172.21.15.42:0/118485747 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f840413bc90 con 0x7f840406cf90 2024-07-04T17:37:02.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.100+0000 7f8401e00640 1 --2- 172.21.15.42:0/118485747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f840406cf90 0x7f8404139f60 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f83f4009850 tx=0x7f83f40049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:02.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.100+0000 7f83fa000640 1 -- 172.21.15.42:0/118485747 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83f403e030 con 0x7f840406cf90 2024-07-04T17:37:02.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.100+0000 7f83fa000640 1 -- 172.21.15.42:0/118485747 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83f40035f0 con 0x7f840406cf90 2024-07-04T17:37:02.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.100+0000 7f83fa000640 1 -- 172.21.15.42:0/118485747 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83f4035040 con 0x7f840406cf90 2024-07-04T17:37:02.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.100+0000 7f8403200640 1 -- 172.21.15.42:0/118485747 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8404138d10 con 0x7f840406cf90 2024-07-04T17:37:02.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.101+0000 7f8403200640 1 -- 172.21.15.42:0/118485747 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8404139230 con 0x7f840406cf90 2024-07-04T17:37:02.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.102+0000 7f8403200640 1 -- 172.21.15.42:0/118485747 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f840413bab0 con 0x7f840406cf90 2024-07-04T17:37:02.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.102+0000 7f83fa000640 1 -- 172.21.15.42:0/118485747 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 61) ==== 49122+0+0 (secure 0 0 0) 0x7f83f4041030 con 0x7f840406cf90 2024-07-04T17:37:02.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.103+0000 7f83fa000640 1 -- 172.21.15.42:0/118485747 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5164+0+0 (secure 0 0 0) 0x7f83f403f070 con 0x7f840406cf90 2024-07-04T17:37:02.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.108+0000 7f83fa000640 1 -- 172.21.15.42:0/118485747 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f840413bab0 con 0x7f840406cf90 2024-07-04T17:37:02.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.173+0000 7f83fa000640 1 -- 172.21.15.42:0/118485747 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 62) ==== 49249+0+0 (secure 0 0 0) 0x7f83f4003990 con 0x7f840406cf90 2024-07-04T17:37:02.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.173+0000 7f83fa000640 1 --2- 172.21.15.42:0/118485747 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f83e0044170 0x7f83e0046590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:02.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.175+0000 7f83fbe00640 1 --2- 172.21.15.42:0/118485747 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f83e0044170 0x7f83e0046590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:02.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.178+0000 7f83fbe00640 1 --2- 172.21.15.42:0/118485747 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f83e0044170 0x7f83e0046590 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f83ec001560 tx=0x7f83ec0079f0 comp rx=0 tx=0).ready entity=mgr.5988 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:02.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.376+0000 7f8403200640 1 -- 172.21.15.42:0/118485747 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f84040bb1b0 con 0x7f840406cf90 2024-07-04T17:37:02.376 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-07-04T17:37:02.377+0000 7f8616e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-04T17:37:02.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.378+0000 7f83fa000640 1 -- 172.21.15.42:0/118485747 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v116) ==== 161+0+0 (secure 0 0 0) 0x7f83f402e260 con 0x7f840406cf90 2024-07-04T17:37:02.378 INFO:teuthology.orchestra.run.smithi042.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-04T17:37:02.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.382+0000 7f8403200640 1 -- 172.21.15.42:0/118485747 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f83e0044170 msgr2=0x7f83e0046590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:02.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.382+0000 7f8403200640 1 --2- 172.21.15.42:0/118485747 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f83e0044170 0x7f83e0046590 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f83ec001560 tx=0x7f83ec0079f0 comp rx=0 tx=0).stop 2024-07-04T17:37:02.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.382+0000 7f8403200640 1 -- 172.21.15.42:0/118485747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f840406cf90 msgr2=0x7f8404139f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:02.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.382+0000 7f8403200640 1 --2- 172.21.15.42:0/118485747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f840406cf90 0x7f8404139f60 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f83f4009850 tx=0x7f83f40049d0 comp rx=0 tx=0).stop 2024-07-04T17:37:02.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.383+0000 7f8403200640 1 -- 172.21.15.42:0/118485747 shutdown_connections 2024-07-04T17:37:02.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.383+0000 7f8403200640 1 --2- 172.21.15.42:0/118485747 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f83e0044170 0x7f83e0046590 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:02.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.383+0000 7f8403200640 1 --2- 172.21.15.42:0/118485747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f840406cf90 0x7f8404139f60 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:02.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.383+0000 7f8403200640 1 -- 172.21.15.42:0/118485747 >> 172.21.15.42:0/118485747 conn(0x7f84040bebc0 msgr2=0x7f84040b9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:02.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.383+0000 7f8403200640 1 -- 172.21.15.42:0/118485747 shutdown_connections 2024-07-04T17:37:02.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.383+0000 7f8403200640 1 -- 172.21.15.42:0/118485747 wait complete. 2024-07-04T17:37:02.415 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-07-04T17:37:02.553 INFO:tasks.ceph.mds.d.smithi042.stdout:starting mds.d at 2024-07-04T17:37:02.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.684+0000 7fe8c5a00640 1 -- 172.21.15.42:0/1294302838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8c006dc80 msgr2=0x7fe8c00b24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:02.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.684+0000 7fe8c5a00640 1 --2- 172.21.15.42:0/1294302838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8c006dc80 0x7fe8c00b24a0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fe8b000a8c0 tx=0x7fe8b002b7d0 comp rx=0 tx=0).stop 2024-07-04T17:37:02.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.684+0000 7fe8c5a00640 1 -- 172.21.15.42:0/1294302838 shutdown_connections 2024-07-04T17:37:02.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.684+0000 7fe8c5a00640 1 --2- 172.21.15.42:0/1294302838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8c006dc80 0x7fe8c00b24a0 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:02.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.684+0000 7fe8c5a00640 1 -- 172.21.15.42:0/1294302838 >> 172.21.15.42:0/1294302838 conn(0x7fe8c006fe40 msgr2=0x7fe8c006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:02.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.684+0000 7fe8c5a00640 1 -- 172.21.15.42:0/1294302838 shutdown_connections 2024-07-04T17:37:02.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.684+0000 7fe8c5a00640 1 -- 172.21.15.42:0/1294302838 wait complete. 2024-07-04T17:37:02.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.686+0000 7fe8c5a00640 1 Processor -- start 2024-07-04T17:37:02.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.686+0000 7fe8c5a00640 1 -- start start 2024-07-04T17:37:02.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.688+0000 7fe8c5a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8c006dc80 0x7fe8c00bf050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:02.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.688+0000 7fe8c5a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c0071d70 con 0x7fe8c006dc80 2024-07-04T17:37:02.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.688+0000 7fe8bea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8c006dc80 0x7fe8c00bf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:02.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.688+0000 7fe8bea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8c006dc80 0x7fe8c00bf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43982/0 (socket says 172.21.15.42:43982) 2024-07-04T17:37:02.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.688+0000 7fe8bea00640 1 -- 172.21.15.42:0/3817557390 learned_addr learned my addr 172.21.15.42:0/3817557390 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:37:02.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.689+0000 7fe8bea00640 1 -- 172.21.15.42:0/3817557390 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c01d3b20 con 0x7fe8c006dc80 2024-07-04T17:37:02.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.689+0000 7fe8bea00640 1 --2- 172.21.15.42:0/3817557390 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8c006dc80 0x7fe8c00bf050 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fe8b000a9e0 tx=0x7fe8b0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:02.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.689+0000 7fe8b7400640 1 -- 172.21.15.42:0/3817557390 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8b0039040 con 0x7fe8c006dc80 2024-07-04T17:37:02.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.689+0000 7fe8b7400640 1 -- 172.21.15.42:0/3817557390 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8b0044070 con 0x7fe8c006dc80 2024-07-04T17:37:02.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.689+0000 7fe8b7400640 1 -- 172.21.15.42:0/3817557390 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8b00085d0 con 0x7fe8c006dc80 2024-07-04T17:37:02.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.689+0000 7fe8c5a00640 1 -- 172.21.15.42:0/3817557390 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8c01e4690 con 0x7fe8c006dc80 2024-07-04T17:37:02.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.690+0000 7fe8c5a00640 1 -- 172.21.15.42:0/3817557390 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8c0141a00 con 0x7fe8c006dc80 2024-07-04T17:37:02.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.691+0000 7fe8c5a00640 1 -- 172.21.15.42:0/3817557390 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8c00bd620 con 0x7fe8c006dc80 2024-07-04T17:37:02.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.691+0000 7fe8b7400640 1 -- 172.21.15.42:0/3817557390 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 62) ==== 49249+0+0 (secure 0 0 0) 0x7fe8b0032070 con 0x7fe8c006dc80 2024-07-04T17:37:02.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.691+0000 7fe8b7400640 1 --2- 172.21.15.42:0/3817557390 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fe89c03d100 0x7fe89c03f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:02.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.691+0000 7fe8b7400640 1 -- 172.21.15.42:0/3817557390 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5250+0+0 (secure 0 0 0) 0x7fe8b0072860 con 0x7fe8c006dc80 2024-07-04T17:37:02.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.692+0000 7fe8b6000640 1 --2- 172.21.15.42:0/3817557390 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fe89c03d100 0x7fe89c03f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:02.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.693+0000 7fe8b6000640 1 --2- 172.21.15.42:0/3817557390 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fe89c03d100 0x7fe89c03f610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe8c00746d0 tx=0x7fe8a8007a90 comp rx=0 tx=0).ready entity=mgr.5988 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:02.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.697+0000 7fe8b7400640 1 -- 172.21.15.42:0/3817557390 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fe8b003fd40 con 0x7fe8c006dc80 2024-07-04T17:37:02.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:02.956+0000 7fe8c5a00640 1 -- 172.21.15.42:0/3817557390 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe8c00b24a0 con 0x7fe8c006dc80 2024-07-04T17:37:03.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.415+0000 7fe8b7400640 1 -- 172.21.15.42:0/3817557390 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v117) ==== 104+0+0 (secure 0 0 0) 0x7fe8b0074cf0 con 0x7fe8c006dc80 2024-07-04T17:37:03.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.419+0000 7fe8c5a00640 1 -- 172.21.15.42:0/3817557390 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fe89c03d100 msgr2=0x7fe89c03f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:03.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.419+0000 7fe8c5a00640 1 --2- 172.21.15.42:0/3817557390 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fe89c03d100 0x7fe89c03f610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe8c00746d0 tx=0x7fe8a8007a90 comp rx=0 tx=0).stop 2024-07-04T17:37:03.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.419+0000 7fe8c5a00640 1 -- 172.21.15.42:0/3817557390 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8c006dc80 msgr2=0x7fe8c00bf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:03.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.419+0000 7fe8c5a00640 1 --2- 172.21.15.42:0/3817557390 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8c006dc80 0x7fe8c00bf050 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fe8b000a9e0 tx=0x7fe8b0003b90 comp rx=0 tx=0).stop 2024-07-04T17:37:03.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.419+0000 7fe8c5a00640 1 -- 172.21.15.42:0/3817557390 shutdown_connections 2024-07-04T17:37:03.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.419+0000 7fe8c5a00640 1 --2- 172.21.15.42:0/3817557390 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fe89c03d100 0x7fe89c03f610 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:03.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.419+0000 7fe8c5a00640 1 --2- 172.21.15.42:0/3817557390 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8c006dc80 0x7fe8c00bf050 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:03.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.419+0000 7fe8c5a00640 1 -- 172.21.15.42:0/3817557390 >> 172.21.15.42:0/3817557390 conn(0x7fe8c006fe40 msgr2=0x7fe8c00735a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:03.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.419+0000 7fe8c5a00640 1 -- 172.21.15.42:0/3817557390 shutdown_connections 2024-07-04T17:37:03.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.419+0000 7fe8c5a00640 1 -- 172.21.15.42:0/3817557390 wait complete. 2024-07-04T17:37:03.458 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-04T17:37:03.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.752+0000 7f372fe00640 1 -- 172.21.15.42:0/67310334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f373006da60 msgr2=0x7f37300b0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:03.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.752+0000 7f372fe00640 1 --2- 172.21.15.42:0/67310334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f373006da60 0x7f37300b0db0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f372000a8c0 tx=0x7f372002b7d0 comp rx=0 tx=0).stop 2024-07-04T17:37:03.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.752+0000 7f372fe00640 1 -- 172.21.15.42:0/67310334 shutdown_connections 2024-07-04T17:37:03.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.752+0000 7f372fe00640 1 --2- 172.21.15.42:0/67310334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f373006da60 0x7f37300b0db0 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:03.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.752+0000 7f372fe00640 1 -- 172.21.15.42:0/67310334 >> 172.21.15.42:0/67310334 conn(0x7f37300afaa0 msgr2=0x7f37300afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:03.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.752+0000 7f372fe00640 1 -- 172.21.15.42:0/67310334 shutdown_connections 2024-07-04T17:37:03.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.752+0000 7f372fe00640 1 -- 172.21.15.42:0/67310334 wait complete. 2024-07-04T17:37:03.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.754+0000 7f372fe00640 1 Processor -- start 2024-07-04T17:37:03.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.755+0000 7f372fe00640 1 -- start start 2024-07-04T17:37:03.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.756+0000 7f372fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f373006da60 0x7f3730143aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:03.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.756+0000 7f372fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f37300ae590 con 0x7f373006da60 2024-07-04T17:37:03.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.756+0000 7f372ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f373006da60 0x7f3730143aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:03.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.756+0000 7f372ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f373006da60 0x7f3730143aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44020/0 (socket says 172.21.15.42:44020) 2024-07-04T17:37:03.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.756+0000 7f372ea00640 1 -- 172.21.15.42:0/2920261469 learned_addr learned my addr 172.21.15.42:0/2920261469 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:37:03.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.757+0000 7f372ea00640 1 -- 172.21.15.42:0/2920261469 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37301e6e50 con 0x7f373006da60 2024-07-04T17:37:03.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.757+0000 7f372ea00640 1 --2- 172.21.15.42:0/2920261469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f373006da60 0x7f3730143aa0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f3720000c00 tx=0x7f3720003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:03.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.757+0000 7f3727400640 1 -- 172.21.15.42:0/2920261469 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3720039040 con 0x7f373006da60 2024-07-04T17:37:03.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.757+0000 7f3727400640 1 -- 172.21.15.42:0/2920261469 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3720044070 con 0x7f373006da60 2024-07-04T17:37:03.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.757+0000 7f3727400640 1 -- 172.21.15.42:0/2920261469 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37200085d0 con 0x7f373006da60 2024-07-04T17:37:03.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.757+0000 7f372fe00640 1 -- 172.21.15.42:0/2920261469 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f373013aed0 con 0x7f373006da60 2024-07-04T17:37:03.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.757+0000 7f372fe00640 1 -- 172.21.15.42:0/2920261469 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37301d7790 con 0x7f373006da60 2024-07-04T17:37:03.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.759+0000 7f372fe00640 1 -- 172.21.15.42:0/2920261469 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37301d7960 con 0x7f373006da60 2024-07-04T17:37:03.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.759+0000 7f3727400640 1 -- 172.21.15.42:0/2920261469 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 63) ==== 49559+0+0 (secure 0 0 0) 0x7f3720032070 con 0x7f373006da60 2024-07-04T17:37:03.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.759+0000 7f3727400640 1 --2- 172.21.15.42:0/2920261469 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f371003d460 0x7f371003f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:03.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.759+0000 7f3727400640 1 -- 172.21.15.42:0/2920261469 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5250+0+0 (secure 0 0 0) 0x7f3720072d90 con 0x7f373006da60 2024-07-04T17:37:03.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.760+0000 7f372e000640 1 --2- 172.21.15.42:0/2920261469 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f371003d460 0x7f371003f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:03.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.760+0000 7f372e000640 1 --2- 172.21.15.42:0/2920261469 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f371003d460 0x7f371003f970 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f371c0015c0 tx=0x7f371c007a90 comp rx=0 tx=0).ready entity=mgr.5988 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:03.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.764+0000 7f3727400640 1 -- 172.21.15.42:0/2920261469 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f3720008950 con 0x7f373006da60 2024-07-04T17:37:03.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:03.975+0000 7f372fe00640 1 -- 172.21.15.42:0/2920261469 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f37300b0db0 con 0x7f373006da60 2024-07-04T17:37:04.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.426+0000 7f3727400640 1 -- 172.21.15.42:0/2920261469 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v147) ==== 181+0+0 (secure 0 0 0) 0x7f3720008cd0 con 0x7f373006da60 2024-07-04T17:37:04.427 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' removed 2024-07-04T17:37:04.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.431+0000 7f372fe00640 1 -- 172.21.15.42:0/2920261469 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f371003d460 msgr2=0x7f371003f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:04.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.431+0000 7f372fe00640 1 --2- 172.21.15.42:0/2920261469 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f371003d460 0x7f371003f970 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f371c0015c0 tx=0x7f371c007a90 comp rx=0 tx=0).stop 2024-07-04T17:37:04.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.431+0000 7f372fe00640 1 -- 172.21.15.42:0/2920261469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f373006da60 msgr2=0x7f3730143aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:04.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.431+0000 7f372fe00640 1 --2- 172.21.15.42:0/2920261469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f373006da60 0x7f3730143aa0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f3720000c00 tx=0x7f3720003b90 comp rx=0 tx=0).stop 2024-07-04T17:37:04.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.432+0000 7f372fe00640 1 -- 172.21.15.42:0/2920261469 shutdown_connections 2024-07-04T17:37:04.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.432+0000 7f372fe00640 1 --2- 172.21.15.42:0/2920261469 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f371003d460 0x7f371003f970 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:04.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.432+0000 7f372fe00640 1 --2- 172.21.15.42:0/2920261469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f373006da60 0x7f3730143aa0 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:04.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.432+0000 7f372fe00640 1 -- 172.21.15.42:0/2920261469 >> 172.21.15.42:0/2920261469 conn(0x7f37300afaa0 msgr2=0x7f37300aebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:04.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.434+0000 7f372fe00640 1 -- 172.21.15.42:0/2920261469 shutdown_connections 2024-07-04T17:37:04.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.434+0000 7f372fe00640 1 -- 172.21.15.42:0/2920261469 wait complete. 2024-07-04T17:37:04.477 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-04T17:37:04.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.757+0000 7f12fda00640 1 -- 172.21.15.42:0/2578646575 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12f80aea60 msgr2=0x7f12f80abba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:04.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.757+0000 7f12fda00640 1 --2- 172.21.15.42:0/2578646575 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12f80aea60 0x7f12f80abba0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f12e8005fd0 tx=0x7f12e802aac0 comp rx=0 tx=0).stop 2024-07-04T17:37:04.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.757+0000 7f12fda00640 1 -- 172.21.15.42:0/2578646575 shutdown_connections 2024-07-04T17:37:04.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.757+0000 7f12fda00640 1 --2- 172.21.15.42:0/2578646575 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12f80aea60 0x7f12f80abba0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:04.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.757+0000 7f12fda00640 1 -- 172.21.15.42:0/2578646575 >> 172.21.15.42:0/2578646575 conn(0x7f12f80afaa0 msgr2=0x7f12f80afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:04.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.757+0000 7f12fda00640 1 -- 172.21.15.42:0/2578646575 shutdown_connections 2024-07-04T17:37:04.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.757+0000 7f12fda00640 1 -- 172.21.15.42:0/2578646575 wait complete. 2024-07-04T17:37:04.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.760+0000 7f12fda00640 1 Processor -- start 2024-07-04T17:37:04.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.760+0000 7f12fda00640 1 -- start start 2024-07-04T17:37:04.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.761+0000 7f12fda00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12f80ac0e0 0x7f12f80708d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:04.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.761+0000 7f12fda00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f12f813f580 con 0x7f12f80ac0e0 2024-07-04T17:37:04.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.762+0000 7f12f7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12f80ac0e0 0x7f12f80708d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:04.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.762+0000 7f12f7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12f80ac0e0 0x7f12f80708d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56062/0 (socket says 172.21.15.42:56062) 2024-07-04T17:37:04.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.762+0000 7f12f7e00640 1 -- 172.21.15.42:0/3331862553 learned_addr learned my addr 172.21.15.42:0/3331862553 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:37:04.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.762+0000 7f12f7e00640 1 -- 172.21.15.42:0/3331862553 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12f8146690 con 0x7f12f80ac0e0 2024-07-04T17:37:04.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.762+0000 7f12f7e00640 1 --2- 172.21.15.42:0/3331862553 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12f80ac0e0 0x7f12f80708d0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f12e8006010 tx=0x7f12e8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:04.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.763+0000 7f12efe00640 1 -- 172.21.15.42:0/3331862553 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12e8044070 con 0x7f12f80ac0e0 2024-07-04T17:37:04.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.763+0000 7f12efe00640 1 -- 172.21.15.42:0/3331862553 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12e8004250 con 0x7f12f80ac0e0 2024-07-04T17:37:04.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.763+0000 7f12efe00640 1 -- 172.21.15.42:0/3331862553 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12e8004590 con 0x7f12f80ac0e0 2024-07-04T17:37:04.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.763+0000 7f12fda00640 1 -- 172.21.15.42:0/3331862553 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12f8073110 con 0x7f12f80ac0e0 2024-07-04T17:37:04.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.763+0000 7f12fda00640 1 -- 172.21.15.42:0/3331862553 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12f813e6e0 con 0x7f12f80ac0e0 2024-07-04T17:37:04.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.764+0000 7f12fda00640 1 -- 172.21.15.42:0/3331862553 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12f813fb00 con 0x7f12f80ac0e0 2024-07-04T17:37:04.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.764+0000 7f12efe00640 1 -- 172.21.15.42:0/3331862553 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 63) ==== 49559+0+0 (secure 0 0 0) 0x7f12e8047070 con 0x7f12f80ac0e0 2024-07-04T17:37:04.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.765+0000 7f12efe00640 1 --2- 172.21.15.42:0/3331862553 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f12d803d460 0x7f12d803f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:04.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.765+0000 7f12efe00640 1 -- 172.21.15.42:0/3331862553 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 4812+0+0 (secure 0 0 0) 0x7f12e803c070 con 0x7f12f80ac0e0 2024-07-04T17:37:04.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.765+0000 7f12f7400640 1 --2- 172.21.15.42:0/3331862553 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f12d803d460 0x7f12d803f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:04.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.766+0000 7f12f7400640 1 --2- 172.21.15.42:0/3331862553 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f12d803d460 0x7f12d803f970 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f12e4001560 tx=0x7f12e40079f0 comp rx=0 tx=0).ready entity=mgr.5988 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:04.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.770+0000 7f12efe00640 1 -- 172.21.15.42:0/3331862553 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f12e8032030 con 0x7f12f80ac0e0 2024-07-04T17:37:04.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:04.982+0000 7f12fda00640 1 -- 172.21.15.42:0/3331862553 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f12f8068e30 con 0x7f12f80ac0e0 2024-07-04T17:37:05.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.444+0000 7f12efe00640 1 -- 172.21.15.42:0/3331862553 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v148) ==== 169+0+0 (secure 0 0 0) 0x7f12e8047360 con 0x7f12f80ac0e0 2024-07-04T17:37:05.445 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' removed 2024-07-04T17:37:05.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.450+0000 7f12fda00640 1 -- 172.21.15.42:0/3331862553 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f12d803d460 msgr2=0x7f12d803f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:05.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.450+0000 7f12fda00640 1 --2- 172.21.15.42:0/3331862553 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f12d803d460 0x7f12d803f970 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f12e4001560 tx=0x7f12e40079f0 comp rx=0 tx=0).stop 2024-07-04T17:37:05.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.450+0000 7f12fda00640 1 -- 172.21.15.42:0/3331862553 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12f80ac0e0 msgr2=0x7f12f80708d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:05.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.450+0000 7f12fda00640 1 --2- 172.21.15.42:0/3331862553 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12f80ac0e0 0x7f12f80708d0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f12e8006010 tx=0x7f12e8003f10 comp rx=0 tx=0).stop 2024-07-04T17:37:05.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.451+0000 7f12fda00640 1 -- 172.21.15.42:0/3331862553 shutdown_connections 2024-07-04T17:37:05.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.451+0000 7f12fda00640 1 --2- 172.21.15.42:0/3331862553 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f12d803d460 0x7f12d803f970 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:05.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.451+0000 7f12fda00640 1 --2- 172.21.15.42:0/3331862553 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12f80ac0e0 0x7f12f80708d0 secure :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f12e8006010 tx=0x7f12e8003f10 comp rx=0 tx=0).stop 2024-07-04T17:37:05.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.451+0000 7f12fda00640 1 -- 172.21.15.42:0/3331862553 >> 172.21.15.42:0/3331862553 conn(0x7f12f80afaa0 msgr2=0x7f12f80b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:05.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.455+0000 7f12fda00640 1 -- 172.21.15.42:0/3331862553 shutdown_connections 2024-07-04T17:37:05.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.456+0000 7f12fda00640 1 -- 172.21.15.42:0/3331862553 wait complete. 2024-07-04T17:37:05.497 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:37:05.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.759+0000 7fa319c00640 1 -- 172.21.15.42:0/294054418 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa31414d550 msgr2=0x7fa3140bd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:05.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.759+0000 7fa319c00640 1 --2- 172.21.15.42:0/294054418 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa31414d550 0x7fa3140bd100 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7fa3040097b0 tx=0x7fa30402a530 comp rx=0 tx=0).stop 2024-07-04T17:37:05.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.759+0000 7fa319c00640 1 -- 172.21.15.42:0/294054418 shutdown_connections 2024-07-04T17:37:05.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.759+0000 7fa319c00640 1 --2- 172.21.15.42:0/294054418 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa31414d550 0x7fa3140bd100 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:05.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.759+0000 7fa319c00640 1 -- 172.21.15.42:0/294054418 >> 172.21.15.42:0/294054418 conn(0x7fa3140c03e0 msgr2=0x7fa3140c07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:05.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.759+0000 7fa319c00640 1 -- 172.21.15.42:0/294054418 shutdown_connections 2024-07-04T17:37:05.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.759+0000 7fa319c00640 1 -- 172.21.15.42:0/294054418 wait complete. 2024-07-04T17:37:05.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.762+0000 7fa319c00640 1 Processor -- start 2024-07-04T17:37:05.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.762+0000 7fa319c00640 1 -- start start 2024-07-04T17:37:05.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.764+0000 7fa319c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3140ad8e0 0x7fa3140b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:05.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.764+0000 7fa319c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa31414df30 con 0x7fa3140ad8e0 2024-07-04T17:37:05.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.764+0000 7fa312a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3140ad8e0 0x7fa3140b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:05.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.764+0000 7fa312a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3140ad8e0 0x7fa3140b09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56082/0 (socket says 172.21.15.42:56082) 2024-07-04T17:37:05.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.764+0000 7fa312a00640 1 -- 172.21.15.42:0/977781692 learned_addr learned my addr 172.21.15.42:0/977781692 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:37:05.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.764+0000 7fa312a00640 1 -- 172.21.15.42:0/977781692 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3141d8d60 con 0x7fa3140ad8e0 2024-07-04T17:37:05.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.765+0000 7fa312a00640 1 --2- 172.21.15.42:0/977781692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3140ad8e0 0x7fa3140b09c0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7fa3040098d0 tx=0x7fa304033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:05.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.765+0000 7fa30aa00640 1 -- 172.21.15.42:0/977781692 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa304031a90 con 0x7fa3140ad8e0 2024-07-04T17:37:05.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.765+0000 7fa30aa00640 1 -- 172.21.15.42:0/977781692 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa304031d20 con 0x7fa3140ad8e0 2024-07-04T17:37:05.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.765+0000 7fa30aa00640 1 -- 172.21.15.42:0/977781692 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa304033200 con 0x7fa3140ad8e0 2024-07-04T17:37:05.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.765+0000 7fa319c00640 1 -- 172.21.15.42:0/977781692 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3141e78f0 con 0x7fa3140ad8e0 2024-07-04T17:37:05.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.765+0000 7fa319c00640 1 -- 172.21.15.42:0/977781692 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa314151400 con 0x7fa3140ad8e0 2024-07-04T17:37:05.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.766+0000 7fa319c00640 1 -- 172.21.15.42:0/977781692 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3140738d0 con 0x7fa3140ad8e0 2024-07-04T17:37:05.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.767+0000 7fa30aa00640 1 -- 172.21.15.42:0/977781692 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 64) ==== 49615+0+0 (secure 0 0 0) 0x7fa3040333a0 con 0x7fa3140ad8e0 2024-07-04T17:37:05.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.767+0000 7fa30aa00640 1 --2- 172.21.15.42:0/977781692 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fa2f403d4f0 0x7fa2f403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:05.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.767+0000 7fa30aa00640 1 -- 172.21.15.42:0/977781692 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4340+0+0 (secure 0 0 0) 0x7fa304072ff0 con 0x7fa3140ad8e0 2024-07-04T17:37:05.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.767+0000 7fa312000640 1 --2- 172.21.15.42:0/977781692 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fa2f403d4f0 0x7fa2f403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:05.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.768+0000 7fa312000640 1 --2- 172.21.15.42:0/977781692 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fa2f403d4f0 0x7fa2f403fa00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa300002020 tx=0x7fa300001670 comp rx=0 tx=0).ready entity=mgr.5988 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:05.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:05.773+0000 7fa30aa00640 1 -- 172.21.15.42:0/977781692 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fa3140738d0 con 0x7fa3140ad8e0 2024-07-04T17:37:06.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.022+0000 7fa319c00640 1 -- 172.21.15.42:0/977781692 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3140c10b0 con 0x7fa3140ad8e0 2024-07-04T17:37:06.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.024+0000 7fa30aa00640 1 -- 172.21.15.42:0/977781692 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) ==== 95+0+6062 (secure 0 0 0) 0x7fa3140c10b0 con 0x7fa3140ad8e0 2024-07-04T17:37:06.025 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:37:06.025 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":117,"btime":"2024-07-04T17:37:03:413873+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5803,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/1184311185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":1184311185},{"type":"v1","addr":"172.21.15.42:6825","nonce":1184311185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103},{"gid":5808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/1815554749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1815554749},{"type":"v1","addr":"172.21.15.42:6827","nonce":1815554749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":5809,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/2027504764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2027504764},{"type":"v1","addr":"172.21.15.42:6829","nonce":2027504764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":6027,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6833/4188603550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":4188603550},{"type":"v1","addr":"172.21.15.42:6833","nonce":4188603550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":117}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:34:54.794041+0000","modified":"2024-07-04T17:34:57.841598+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5817},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5817":{"gid":5817,"name":"a","rank":0,"incarnation":110,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6831/4064998265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":4064998265},{"type":"v1","addr":"172.21.15.42:6831","nonce":4064998265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5817,"qdb_cluster":[5817]},"id":13}]} 2024-07-04T17:37:06.026 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 117 2024-07-04T17:37:06.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.027+0000 7fa319c00640 1 -- 172.21.15.42:0/977781692 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fa2f403d4f0 msgr2=0x7fa2f403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:06.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.027+0000 7fa319c00640 1 --2- 172.21.15.42:0/977781692 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fa2f403d4f0 0x7fa2f403fa00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa300002020 tx=0x7fa300001670 comp rx=0 tx=0).stop 2024-07-04T17:37:06.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.027+0000 7fa319c00640 1 -- 172.21.15.42:0/977781692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3140ad8e0 msgr2=0x7fa3140b09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:06.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.027+0000 7fa319c00640 1 --2- 172.21.15.42:0/977781692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3140ad8e0 0x7fa3140b09c0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7fa3040098d0 tx=0x7fa304033c10 comp rx=0 tx=0).stop 2024-07-04T17:37:06.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.027+0000 7fa319c00640 1 -- 172.21.15.42:0/977781692 shutdown_connections 2024-07-04T17:37:06.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.027+0000 7fa319c00640 1 --2- 172.21.15.42:0/977781692 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fa2f403d4f0 0x7fa2f403fa00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:06.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.027+0000 7fa319c00640 1 --2- 172.21.15.42:0/977781692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa3140ad8e0 0x7fa3140b09c0 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:06.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.027+0000 7fa319c00640 1 -- 172.21.15.42:0/977781692 >> 172.21.15.42:0/977781692 conn(0x7fa3140c03e0 msgr2=0x7fa3140b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:06.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.027+0000 7fa319c00640 1 -- 172.21.15.42:0/977781692 shutdown_connections 2024-07-04T17:37:06.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.027+0000 7fa319c00640 1 -- 172.21.15.42:0/977781692 wait complete. 2024-07-04T17:37:06.056 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:37:06.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.367+0000 7f0f71e00640 1 -- 172.21.15.42:0/4152617879 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f6c06cea0 msgr2=0x7f0f6c0b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:06.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.367+0000 7f0f71e00640 1 --2- 172.21.15.42:0/4152617879 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f6c06cea0 0x7f0f6c0b3030 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f0f5800a8c0 tx=0x7f0f5802b860 comp rx=0 tx=0).stop 2024-07-04T17:37:06.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.367+0000 7f0f71e00640 1 -- 172.21.15.42:0/4152617879 shutdown_connections 2024-07-04T17:37:06.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.367+0000 7f0f71e00640 1 --2- 172.21.15.42:0/4152617879 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f6c06cea0 0x7f0f6c0b3030 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:06.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.367+0000 7f0f71e00640 1 -- 172.21.15.42:0/4152617879 >> 172.21.15.42:0/4152617879 conn(0x7f0f6c0bf700 msgr2=0x7f0f6c0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:06.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.367+0000 7f0f71e00640 1 -- 172.21.15.42:0/4152617879 shutdown_connections 2024-07-04T17:37:06.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.368+0000 7f0f71e00640 1 -- 172.21.15.42:0/4152617879 wait complete. 2024-07-04T17:37:06.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.370+0000 7f0f71e00640 1 Processor -- start 2024-07-04T17:37:06.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.370+0000 7f0f71e00640 1 -- start start 2024-07-04T17:37:06.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.371+0000 7f0f71e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f6c06cea0 0x7f0f6c1d3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:06.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.371+0000 7f0f71e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c0b0370 con 0x7f0f6c06cea0 2024-07-04T17:37:06.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.372+0000 7f0f6aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f6c06cea0 0x7f0f6c1d3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:06.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.372+0000 7f0f6aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f6c06cea0 0x7f0f6c1d3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56098/0 (socket says 172.21.15.42:56098) 2024-07-04T17:37:06.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.372+0000 7f0f6aa00640 1 -- 172.21.15.42:0/842305836 learned_addr learned my addr 172.21.15.42:0/842305836 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:37:06.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.372+0000 7f0f6aa00640 1 -- 172.21.15.42:0/842305836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f6c1d3840 con 0x7f0f6c06cea0 2024-07-04T17:37:06.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.372+0000 7f0f6aa00640 1 --2- 172.21.15.42:0/842305836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f6c06cea0 0x7f0f6c1d3ce0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c00cb40 tx=0x7f0f5c0021e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:06.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.373+0000 7f0f63400640 1 -- 172.21.15.42:0/842305836 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f5c016020 con 0x7f0f6c06cea0 2024-07-04T17:37:06.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.373+0000 7f0f63400640 1 -- 172.21.15.42:0/842305836 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f5c00abd0 con 0x7f0f6c06cea0 2024-07-04T17:37:06.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.373+0000 7f0f63400640 1 -- 172.21.15.42:0/842305836 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f5c012420 con 0x7f0f6c06cea0 2024-07-04T17:37:06.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.373+0000 7f0f71e00640 1 -- 172.21.15.42:0/842305836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f6c1d52a0 con 0x7f0f6c06cea0 2024-07-04T17:37:06.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.373+0000 7f0f71e00640 1 -- 172.21.15.42:0/842305836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f6c1d2ec0 con 0x7f0f6c06cea0 2024-07-04T17:37:06.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.374+0000 7f0f71e00640 1 -- 172.21.15.42:0/842305836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f6c1d3660 con 0x7f0f6c06cea0 2024-07-04T17:37:06.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.374+0000 7f0f63400640 1 -- 172.21.15.42:0/842305836 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 64) ==== 49615+0+0 (secure 0 0 0) 0x7f0f5c019020 con 0x7f0f6c06cea0 2024-07-04T17:37:06.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.375+0000 7f0f63400640 1 --2- 172.21.15.42:0/842305836 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f0f4c03d4f0 0x7f0f4c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:06.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.375+0000 7f0f63400640 1 -- 172.21.15.42:0/842305836 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4340+0+0 (secure 0 0 0) 0x7f0f5c052310 con 0x7f0f6c06cea0 2024-07-04T17:37:06.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.375+0000 7f0f6a000640 1 --2- 172.21.15.42:0/842305836 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f0f4c03d4f0 0x7f0f4c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:06.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.376+0000 7f0f6a000640 1 --2- 172.21.15.42:0/842305836 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f0f4c03d4f0 0x7f0f4c03fa00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0f58036000 tx=0x7f0f58004730 comp rx=0 tx=0).ready entity=mgr.5988 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:06.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.381+0000 7f0f63400640 1 -- 172.21.15.42:0/842305836 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f0f6c1d3660 con 0x7f0f6c06cea0 2024-07-04T17:37:06.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.595+0000 7f0f71e00640 1 -- 172.21.15.42:0/842305836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f6c0b3030 con 0x7f0f6c06cea0 2024-07-04T17:37:06.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.596+0000 7f0f63400640 1 -- 172.21.15.42:0/842305836 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) ==== 74+0+11925 (secure 0 0 0) 0x7f0f5c020020 con 0x7f0f6c06cea0 2024-07-04T17:37:06.597 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:37:06.597 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":148,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:37:05.438206+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:34:52.310342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:34:53.756688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"27.2d","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6832/1863956336":"2024-07-05T17:37:02.377167+0000","172.21.15.42:6833/1863956336":"2024-07-05T17:37:02.377167+0000","172.21.15.42:0/1132989435":"2024-07-05T17:37:01.159615+0000","172.21.15.42:0/3471044288":"2024-07-05T17:37:01.159615+0000","172.21.15.42:6834/3991875201":"2024-07-05T17:37:01.159615+0000","172.21.15.42:0/4263654752":"2024-07-05T17:37:01.159615+0000","172.21.15.42:0/2937116305":"2024-07-04T18:36:04.169055+0000","172.21.15.42:0/3482046390":"2024-07-05T17:35:07.918359+0000","172.21.15.42:6835/3991875201":"2024-07-05T17:37:01.159615+0000","172.21.15.42:0/1404855452":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/2049026251":"2024-07-05T17:37:01.159615+0000","172.21.15.42:0/1956174992":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/3978634668":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/3294509689":"2024-07-05T17:35:07.918359+0000","172.21.15.42:6835/4064094064":"2024-07-05T17:35:07.918359+0000","172.21.15.42:6834/4064094064":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/528150101":"2024-07-05T17:37:01.159615+0000","172.21.15.42:0/2180653394":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/2328963399":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/644737563":"2024-07-05T17:35:07.918359+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:37:06.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.599+0000 7f0f71e00640 1 -- 172.21.15.42:0/842305836 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f0f4c03d4f0 msgr2=0x7f0f4c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:06.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.599+0000 7f0f71e00640 1 --2- 172.21.15.42:0/842305836 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f0f4c03d4f0 0x7f0f4c03fa00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0f58036000 tx=0x7f0f58004730 comp rx=0 tx=0).stop 2024-07-04T17:37:06.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.599+0000 7f0f71e00640 1 -- 172.21.15.42:0/842305836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f6c06cea0 msgr2=0x7f0f6c1d3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:06.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.599+0000 7f0f71e00640 1 --2- 172.21.15.42:0/842305836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f6c06cea0 0x7f0f6c1d3ce0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c00cb40 tx=0x7f0f5c0021e0 comp rx=0 tx=0).stop 2024-07-04T17:37:06.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.599+0000 7f0f71e00640 1 -- 172.21.15.42:0/842305836 shutdown_connections 2024-07-04T17:37:06.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.599+0000 7f0f71e00640 1 --2- 172.21.15.42:0/842305836 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f0f4c03d4f0 0x7f0f4c03fa00 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:06.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.599+0000 7f0f71e00640 1 --2- 172.21.15.42:0/842305836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f6c06cea0 0x7f0f6c1d3ce0 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:06.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.599+0000 7f0f71e00640 1 -- 172.21.15.42:0/842305836 >> 172.21.15.42:0/842305836 conn(0x7f0f6c0bf700 msgr2=0x7f0f6c14bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:06.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.600+0000 7f0f71e00640 1 -- 172.21.15.42:0/842305836 shutdown_connections 2024-07-04T17:37:06.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.600+0000 7f0f71e00640 1 -- 172.21.15.42:0/842305836 wait complete. 2024-07-04T17:37:06.627 INFO:tasks.cephfs.filesystem:Destroying file system backup_fs and related pools 2024-07-04T17:37:06.627 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:37:06.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.907+0000 7fd20cc00640 1 -- 172.21.15.42:0/3050066021 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd20806edd0 msgr2=0x7fd2080b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:06.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.907+0000 7fd20cc00640 1 --2- 172.21.15.42:0/3050066021 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd20806edd0 0x7fd2080b3030 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fd1f80097b0 tx=0x7fd1f802a540 comp rx=0 tx=0).stop 2024-07-04T17:37:06.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.907+0000 7fd20cc00640 1 -- 172.21.15.42:0/3050066021 shutdown_connections 2024-07-04T17:37:06.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.907+0000 7fd20cc00640 1 --2- 172.21.15.42:0/3050066021 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd20806edd0 0x7fd2080b3030 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:06.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.907+0000 7fd20cc00640 1 -- 172.21.15.42:0/3050066021 >> 172.21.15.42:0/3050066021 conn(0x7fd2080bf700 msgr2=0x7fd2080af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:06.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.907+0000 7fd20cc00640 1 -- 172.21.15.42:0/3050066021 shutdown_connections 2024-07-04T17:37:06.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.908+0000 7fd20cc00640 1 -- 172.21.15.42:0/3050066021 wait complete. 2024-07-04T17:37:06.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.910+0000 7fd20cc00640 1 Processor -- start 2024-07-04T17:37:06.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.910+0000 7fd20cc00640 1 -- start start 2024-07-04T17:37:06.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.911+0000 7fd20cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd20806cea0 0x7fd20814b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:06.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.911+0000 7fd20cc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd2080b0580 con 0x7fd20806cea0 2024-07-04T17:37:06.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.911+0000 7fd205e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd20806cea0 0x7fd20814b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:06.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.911+0000 7fd205e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd20806cea0 0x7fd20814b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56124/0 (socket says 172.21.15.42:56124) 2024-07-04T17:37:06.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.911+0000 7fd205e00640 1 -- 172.21.15.42:0/386006215 learned_addr learned my addr 172.21.15.42:0/386006215 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:37:06.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.912+0000 7fd205e00640 1 -- 172.21.15.42:0/386006215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2080ad030 con 0x7fd20806cea0 2024-07-04T17:37:06.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.912+0000 7fd205e00640 1 --2- 172.21.15.42:0/386006215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd20806cea0 0x7fd20814b7d0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fd1f802aab0 tx=0x7fd1f8032e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:06.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.912+0000 7fd1fe000640 1 -- 172.21.15.42:0/386006215 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1f8040070 con 0x7fd20806cea0 2024-07-04T17:37:06.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.913+0000 7fd1fe000640 1 -- 172.21.15.42:0/386006215 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1f80038f0 con 0x7fd20806cea0 2024-07-04T17:37:06.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.913+0000 7fd1fe000640 1 -- 172.21.15.42:0/386006215 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1f8003c10 con 0x7fd20806cea0 2024-07-04T17:37:06.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.913+0000 7fd20cc00640 1 -- 172.21.15.42:0/386006215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd208145350 con 0x7fd20806cea0 2024-07-04T17:37:06.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.913+0000 7fd20cc00640 1 -- 172.21.15.42:0/386006215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2080ac5f0 con 0x7fd20806cea0 2024-07-04T17:37:06.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.914+0000 7fd20cc00640 1 -- 172.21.15.42:0/386006215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2080b3030 con 0x7fd20806cea0 2024-07-04T17:37:06.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.914+0000 7fd1fe000640 1 -- 172.21.15.42:0/386006215 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 64) ==== 49615+0+0 (secure 0 0 0) 0x7fd1f8039040 con 0x7fd20806cea0 2024-07-04T17:37:06.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.914+0000 7fd1fe000640 1 --2- 172.21.15.42:0/386006215 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fd1e403d4f0 0x7fd1e403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:06.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.915+0000 7fd1fe000640 1 -- 172.21.15.42:0/386006215 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4340+0+0 (secure 0 0 0) 0x7fd1f8073450 con 0x7fd20806cea0 2024-07-04T17:37:06.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.915+0000 7fd1ffe00640 1 --2- 172.21.15.42:0/386006215 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fd1e403d4f0 0x7fd1e403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:06.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.915+0000 7fd1ffe00640 1 --2- 172.21.15.42:0/386006215 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fd1e403d4f0 0x7fd1e403fa00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd1f00015c0 tx=0x7fd1f0007a90 comp rx=0 tx=0).ready entity=mgr.5988 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:06.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:06.919+0000 7fd1fe000640 1 -- 172.21.15.42:0/386006215 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fd1f8045c60 con 0x7fd20806cea0 2024-07-04T17:37:07.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.170+0000 7fd20cc00640 1 -- 172.21.15.42:0/386006215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2080ad200 con 0x7fd20806cea0 2024-07-04T17:37:07.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.171+0000 7fd1fe000640 1 -- 172.21.15.42:0/386006215 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) ==== 95+0+6062 (secure 0 0 0) 0x7fd1f80730c0 con 0x7fd20806cea0 2024-07-04T17:37:07.172 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:37:07.172 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":117,"btime":"2024-07-04T17:37:03:413873+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5803,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/1184311185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":1184311185},{"type":"v1","addr":"172.21.15.42:6825","nonce":1184311185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103},{"gid":5808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/1815554749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1815554749},{"type":"v1","addr":"172.21.15.42:6827","nonce":1815554749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":5809,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/2027504764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2027504764},{"type":"v1","addr":"172.21.15.42:6829","nonce":2027504764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":6027,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6833/4188603550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":4188603550},{"type":"v1","addr":"172.21.15.42:6833","nonce":4188603550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":117}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:34:54.794041+0000","modified":"2024-07-04T17:34:57.841598+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5817},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5817":{"gid":5817,"name":"a","rank":0,"incarnation":110,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6831/4064998265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":4064998265},{"type":"v1","addr":"172.21.15.42:6831","nonce":4064998265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5817,"qdb_cluster":[5817]},"id":13}]} 2024-07-04T17:37:07.172 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 117 2024-07-04T17:37:07.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.174+0000 7fd20cc00640 1 -- 172.21.15.42:0/386006215 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fd1e403d4f0 msgr2=0x7fd1e403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:07.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.174+0000 7fd20cc00640 1 --2- 172.21.15.42:0/386006215 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fd1e403d4f0 0x7fd1e403fa00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd1f00015c0 tx=0x7fd1f0007a90 comp rx=0 tx=0).stop 2024-07-04T17:37:07.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.174+0000 7fd20cc00640 1 -- 172.21.15.42:0/386006215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd20806cea0 msgr2=0x7fd20814b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:07.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.174+0000 7fd20cc00640 1 --2- 172.21.15.42:0/386006215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd20806cea0 0x7fd20814b7d0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fd1f802aab0 tx=0x7fd1f8032e30 comp rx=0 tx=0).stop 2024-07-04T17:37:07.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.174+0000 7fd20cc00640 1 -- 172.21.15.42:0/386006215 shutdown_connections 2024-07-04T17:37:07.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.174+0000 7fd20cc00640 1 --2- 172.21.15.42:0/386006215 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fd1e403d4f0 0x7fd1e403fa00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:07.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.174+0000 7fd20cc00640 1 --2- 172.21.15.42:0/386006215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd20806cea0 0x7fd20814b7d0 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:07.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.174+0000 7fd20cc00640 1 -- 172.21.15.42:0/386006215 >> 172.21.15.42:0/386006215 conn(0x7fd2080bf700 msgr2=0x7fd208075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:07.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.174+0000 7fd20cc00640 1 -- 172.21.15.42:0/386006215 shutdown_connections 2024-07-04T17:37:07.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.174+0000 7fd20cc00640 1 -- 172.21.15.42:0/386006215 wait complete. 2024-07-04T17:37:07.202 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:37:07.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.492+0000 7f9847200640 1 -- 172.21.15.42:0/2176772955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98480bad70 msgr2=0x7f984806c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:07.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.492+0000 7f9847200640 1 --2- 172.21.15.42:0/2176772955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98480bad70 0x7f984806c2c0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f98380097b0 tx=0x7f983802a4b0 comp rx=0 tx=0).stop 2024-07-04T17:37:07.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.492+0000 7f9847200640 1 -- 172.21.15.42:0/2176772955 shutdown_connections 2024-07-04T17:37:07.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.492+0000 7f9847200640 1 --2- 172.21.15.42:0/2176772955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98480bad70 0x7f984806c2c0 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:07.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.492+0000 7f9847200640 1 -- 172.21.15.42:0/2176772955 >> 172.21.15.42:0/2176772955 conn(0x7f98480bebc0 msgr2=0x7f98480befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:07.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.493+0000 7f9847200640 1 -- 172.21.15.42:0/2176772955 shutdown_connections 2024-07-04T17:37:07.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.493+0000 7f9847200640 1 -- 172.21.15.42:0/2176772955 wait complete. 2024-07-04T17:37:07.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.495+0000 7f9847200640 1 Processor -- start 2024-07-04T17:37:07.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.495+0000 7f9847200640 1 -- start start 2024-07-04T17:37:07.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.497+0000 7f9847200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98480b5650 0x7f98480ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:07.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.497+0000 7f9847200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f98480b27c0 con 0x7f98480b5650 2024-07-04T17:37:07.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.497+0000 7f9845e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98480b5650 0x7f98480ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:07.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.497+0000 7f9845e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98480b5650 0x7f98480ae2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56144/0 (socket says 172.21.15.42:56144) 2024-07-04T17:37:07.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.497+0000 7f9845e00640 1 -- 172.21.15.42:0/2384936146 learned_addr learned my addr 172.21.15.42:0/2384936146 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:37:07.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.498+0000 7f9845e00640 1 -- 172.21.15.42:0/2384936146 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98480adda0 con 0x7f98480b5650 2024-07-04T17:37:07.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.498+0000 7f9845e00640 1 --2- 172.21.15.42:0/2384936146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98480b5650 0x7f98480ae2c0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f98380098d0 tx=0x7f9838033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:07.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.499+0000 7f983e000640 1 -- 172.21.15.42:0/2384936146 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9838031bc0 con 0x7f98480b5650 2024-07-04T17:37:07.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.499+0000 7f983e000640 1 -- 172.21.15.42:0/2384936146 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9838031e50 con 0x7f98480b5650 2024-07-04T17:37:07.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.499+0000 7f983e000640 1 -- 172.21.15.42:0/2384936146 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9838033200 con 0x7f98480b5650 2024-07-04T17:37:07.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.499+0000 7f9847200640 1 -- 172.21.15.42:0/2384936146 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98481e6f20 con 0x7f98480b5650 2024-07-04T17:37:07.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.499+0000 7f9847200640 1 -- 172.21.15.42:0/2384936146 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98480adac0 con 0x7f98480b5650 2024-07-04T17:37:07.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.500+0000 7f9847200640 1 -- 172.21.15.42:0/2384936146 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f984806c2c0 con 0x7f98480b5650 2024-07-04T17:37:07.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.500+0000 7f983e000640 1 -- 172.21.15.42:0/2384936146 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 64) ==== 49615+0+0 (secure 0 0 0) 0x7f98380333a0 con 0x7f98480b5650 2024-07-04T17:37:07.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.501+0000 7f983e000640 1 --2- 172.21.15.42:0/2384936146 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f982803d4f0 0x7f982803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:07.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.501+0000 7f983e000640 1 -- 172.21.15.42:0/2384936146 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4340+0+0 (secure 0 0 0) 0x7f9838073190 con 0x7f98480b5650 2024-07-04T17:37:07.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.501+0000 7f9845400640 1 --2- 172.21.15.42:0/2384936146 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f982803d4f0 0x7f982803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:07.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.502+0000 7f9845400640 1 --2- 172.21.15.42:0/2384936146 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f982803d4f0 0x7f982803fa00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f98340015c0 tx=0x7f9834007a90 comp rx=0 tx=0).ready entity=mgr.5988 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:07.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.506+0000 7f983e000640 1 -- 172.21.15.42:0/2384936146 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f983803f030 con 0x7f98480b5650 2024-07-04T17:37:07.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.761+0000 7f9847200640 1 -- 172.21.15.42:0/2384936146 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9848068e30 con 0x7f98480b5650 2024-07-04T17:37:07.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.762+0000 7f983e000640 1 -- 172.21.15.42:0/2384936146 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) ==== 95+0+6062 (secure 0 0 0) 0x7f9838033690 con 0x7f98480b5650 2024-07-04T17:37:07.763 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:37:07.764 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":117,"btime":"2024-07-04T17:37:03:413873+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5803,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/1184311185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":1184311185},{"type":"v1","addr":"172.21.15.42:6825","nonce":1184311185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103},{"gid":5808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/1815554749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1815554749},{"type":"v1","addr":"172.21.15.42:6827","nonce":1815554749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":5809,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/2027504764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2027504764},{"type":"v1","addr":"172.21.15.42:6829","nonce":2027504764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":6027,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6833/4188603550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":4188603550},{"type":"v1","addr":"172.21.15.42:6833","nonce":4188603550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":117}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T17:34:54.794041+0000","modified":"2024-07-04T17:34:57.841598+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5817},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5817":{"gid":5817,"name":"a","rank":0,"incarnation":110,"state":"up:active","state_seq":8,"addr":"172.21.15.42:6831/4064998265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":4064998265},{"type":"v1","addr":"172.21.15.42:6831","nonce":4064998265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5817,"qdb_cluster":[5817]},"id":13}]} 2024-07-04T17:37:07.764 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 117 2024-07-04T17:37:07.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.765+0000 7f9847200640 1 -- 172.21.15.42:0/2384936146 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f982803d4f0 msgr2=0x7f982803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:07.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.765+0000 7f9847200640 1 --2- 172.21.15.42:0/2384936146 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f982803d4f0 0x7f982803fa00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f98340015c0 tx=0x7f9834007a90 comp rx=0 tx=0).stop 2024-07-04T17:37:07.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.765+0000 7f9847200640 1 -- 172.21.15.42:0/2384936146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98480b5650 msgr2=0x7f98480ae2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:07.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.765+0000 7f9847200640 1 --2- 172.21.15.42:0/2384936146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98480b5650 0x7f98480ae2c0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f98380098d0 tx=0x7f9838033e40 comp rx=0 tx=0).stop 2024-07-04T17:37:07.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.765+0000 7f9847200640 1 -- 172.21.15.42:0/2384936146 shutdown_connections 2024-07-04T17:37:07.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.765+0000 7f9847200640 1 --2- 172.21.15.42:0/2384936146 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f982803d4f0 0x7f982803fa00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:07.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.765+0000 7f9847200640 1 --2- 172.21.15.42:0/2384936146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98480b5650 0x7f98480ae2c0 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:07.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.765+0000 7f9847200640 1 -- 172.21.15.42:0/2384936146 >> 172.21.15.42:0/2384936146 conn(0x7f98480bebc0 msgr2=0x7f98480bcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:07.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.765+0000 7f9847200640 1 -- 172.21.15.42:0/2384936146 shutdown_connections 2024-07-04T17:37:07.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:07.765+0000 7f9847200640 1 -- 172.21.15.42:0/2384936146 wait complete. 2024-07-04T17:37:07.791 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:37:08.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.082+0000 7fed2d800640 1 -- 172.21.15.42:0/2093412932 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed2806cea0 msgr2=0x7fed280b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:08.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.082+0000 7fed2d800640 1 --2- 172.21.15.42:0/2093412932 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed2806cea0 0x7fed280b3030 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7fed1800b2b0 tx=0x7fed1802ad10 comp rx=0 tx=0).stop 2024-07-04T17:37:08.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.082+0000 7fed2d800640 1 -- 172.21.15.42:0/2093412932 shutdown_connections 2024-07-04T17:37:08.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.082+0000 7fed2d800640 1 --2- 172.21.15.42:0/2093412932 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed2806cea0 0x7fed280b3030 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:08.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.082+0000 7fed2d800640 1 -- 172.21.15.42:0/2093412932 >> 172.21.15.42:0/2093412932 conn(0x7fed280bf700 msgr2=0x7fed280af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:08.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.082+0000 7fed2d800640 1 -- 172.21.15.42:0/2093412932 shutdown_connections 2024-07-04T17:37:08.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.082+0000 7fed2d800640 1 -- 172.21.15.42:0/2093412932 wait complete. 2024-07-04T17:37:08.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.084+0000 7fed2d800640 1 Processor -- start 2024-07-04T17:37:08.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.084+0000 7fed2d800640 1 -- start start 2024-07-04T17:37:08.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.085+0000 7fed2d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed2806cea0 0x7fed28146130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:08.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.085+0000 7fed2d800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fed2806ef50 con 0x7fed2806cea0 2024-07-04T17:37:08.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.086+0000 7fed26a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed2806cea0 0x7fed28146130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:08.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.086+0000 7fed26a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed2806cea0 0x7fed28146130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56152/0 (socket says 172.21.15.42:56152) 2024-07-04T17:37:08.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.086+0000 7fed26a00640 1 -- 172.21.15.42:0/987624598 learned_addr learned my addr 172.21.15.42:0/987624598 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:37:08.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.086+0000 7fed26a00640 1 -- 172.21.15.42:0/987624598 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed28146720 con 0x7fed2806cea0 2024-07-04T17:37:08.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.086+0000 7fed26a00640 1 --2- 172.21.15.42:0/987624598 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed2806cea0 0x7fed28146130 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7fed18000c00 tx=0x7fed180097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:08.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.087+0000 7fed1ea00640 1 -- 172.21.15.42:0/987624598 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed18038040 con 0x7fed2806cea0 2024-07-04T17:37:08.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.087+0000 7fed1ea00640 1 -- 172.21.15.42:0/987624598 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed18041070 con 0x7fed2806cea0 2024-07-04T17:37:08.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.087+0000 7fed2d800640 1 -- 172.21.15.42:0/987624598 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed281dc980 con 0x7fed2806cea0 2024-07-04T17:37:08.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.087+0000 7fed1ea00640 1 -- 172.21.15.42:0/987624598 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed18004420 con 0x7fed2806cea0 2024-07-04T17:37:08.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.087+0000 7fed2d800640 1 -- 172.21.15.42:0/987624598 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed28149a10 con 0x7fed2806cea0 2024-07-04T17:37:08.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.088+0000 7fed1ea00640 1 -- 172.21.15.42:0/987624598 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 64) ==== 49615+0+0 (secure 0 0 0) 0x7fed1802b870 con 0x7fed2806cea0 2024-07-04T17:37:08.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.088+0000 7fed2d800640 1 -- 172.21.15.42:0/987624598 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed280b3030 con 0x7fed2806cea0 2024-07-04T17:37:08.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.089+0000 7fed1ea00640 1 --2- 172.21.15.42:0/987624598 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fed0803d4f0 0x7fed0803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:08.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.089+0000 7fed1ea00640 1 -- 172.21.15.42:0/987624598 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4340+0+0 (secure 0 0 0) 0x7fed180723c0 con 0x7fed2806cea0 2024-07-04T17:37:08.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.089+0000 7fed26000640 1 --2- 172.21.15.42:0/987624598 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fed0803d4f0 0x7fed0803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:08.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.090+0000 7fed26000640 1 --2- 172.21.15.42:0/987624598 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fed0803d4f0 0x7fed0803fa00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fed140015c0 tx=0x7fed14007a90 comp rx=0 tx=0).ready entity=mgr.5988 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:08.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.094+0000 7fed1ea00640 1 -- 172.21.15.42:0/987624598 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fed1804b020 con 0x7fed2806cea0 2024-07-04T17:37:08.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.303+0000 7fed2d800640 1 -- 172.21.15.42:0/987624598 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed28146be0 con 0x7fed2806cea0 2024-07-04T17:37:08.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.305+0000 7fed1ea00640 1 -- 172.21.15.42:0/987624598 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) ==== 74+0+11925 (secure 0 0 0) 0x7fed18041220 con 0x7fed2806cea0 2024-07-04T17:37:08.306 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:37:08.307 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":148,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:37:05.438206+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2024-07-04T17:34:52.310342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2024-07-04T17:34:53.756688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"27.2d","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6832/1863956336":"2024-07-05T17:37:02.377167+0000","172.21.15.42:6833/1863956336":"2024-07-05T17:37:02.377167+0000","172.21.15.42:0/1132989435":"2024-07-05T17:37:01.159615+0000","172.21.15.42:0/3471044288":"2024-07-05T17:37:01.159615+0000","172.21.15.42:6834/3991875201":"2024-07-05T17:37:01.159615+0000","172.21.15.42:0/4263654752":"2024-07-05T17:37:01.159615+0000","172.21.15.42:0/2937116305":"2024-07-04T18:36:04.169055+0000","172.21.15.42:0/3482046390":"2024-07-05T17:35:07.918359+0000","172.21.15.42:6835/3991875201":"2024-07-05T17:37:01.159615+0000","172.21.15.42:0/1404855452":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/2049026251":"2024-07-05T17:37:01.159615+0000","172.21.15.42:0/1956174992":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/3978634668":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/3294509689":"2024-07-05T17:35:07.918359+0000","172.21.15.42:6835/4064094064":"2024-07-05T17:35:07.918359+0000","172.21.15.42:6834/4064094064":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/528150101":"2024-07-05T17:37:01.159615+0000","172.21.15.42:0/2180653394":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/2328963399":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/644737563":"2024-07-05T17:35:07.918359+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:37:08.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.308+0000 7fed2d800640 1 -- 172.21.15.42:0/987624598 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fed0803d4f0 msgr2=0x7fed0803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:08.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.308+0000 7fed2d800640 1 --2- 172.21.15.42:0/987624598 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fed0803d4f0 0x7fed0803fa00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fed140015c0 tx=0x7fed14007a90 comp rx=0 tx=0).stop 2024-07-04T17:37:08.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.309+0000 7fed2d800640 1 -- 172.21.15.42:0/987624598 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed2806cea0 msgr2=0x7fed28146130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:08.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.309+0000 7fed2d800640 1 --2- 172.21.15.42:0/987624598 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed2806cea0 0x7fed28146130 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7fed18000c00 tx=0x7fed180097d0 comp rx=0 tx=0).stop 2024-07-04T17:37:08.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.309+0000 7fed2d800640 1 -- 172.21.15.42:0/987624598 shutdown_connections 2024-07-04T17:37:08.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.309+0000 7fed2d800640 1 --2- 172.21.15.42:0/987624598 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fed0803d4f0 0x7fed0803fa00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:08.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.309+0000 7fed2d800640 1 --2- 172.21.15.42:0/987624598 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed2806cea0 0x7fed28146130 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:08.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.309+0000 7fed2d800640 1 -- 172.21.15.42:0/987624598 >> 172.21.15.42:0/987624598 conn(0x7fed280bf700 msgr2=0x7fed2814a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:08.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.309+0000 7fed2d800640 1 -- 172.21.15.42:0/987624598 shutdown_connections 2024-07-04T17:37:08.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.309+0000 7fed2d800640 1 -- 172.21.15.42:0/987624598 wait complete. 2024-07-04T17:37:08.337 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail backup_fs 2024-07-04T17:37:08.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.615+0000 7fcfb7200640 1 -- 172.21.15.42:0/1690478562 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfb80b8f80 msgr2=0x7fcfb806c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:08.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.615+0000 7fcfb7200640 1 --2- 172.21.15.42:0/1690478562 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfb80b8f80 0x7fcfb806c020 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fcfa80097b0 tx=0x7fcfa802a530 comp rx=0 tx=0).stop 2024-07-04T17:37:08.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.615+0000 7fcfb7200640 1 -- 172.21.15.42:0/1690478562 shutdown_connections 2024-07-04T17:37:08.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.615+0000 7fcfb7200640 1 --2- 172.21.15.42:0/1690478562 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfb80b8f80 0x7fcfb806c020 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:08.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.615+0000 7fcfb7200640 1 -- 172.21.15.42:0/1690478562 >> 172.21.15.42:0/1690478562 conn(0x7fcfb80bebc0 msgr2=0x7fcfb80befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:08.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.615+0000 7fcfb7200640 1 -- 172.21.15.42:0/1690478562 shutdown_connections 2024-07-04T17:37:08.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.615+0000 7fcfb7200640 1 -- 172.21.15.42:0/1690478562 wait complete. 2024-07-04T17:37:08.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.617+0000 7fcfb7200640 1 Processor -- start 2024-07-04T17:37:08.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.617+0000 7fcfb7200640 1 -- start start 2024-07-04T17:37:08.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.618+0000 7fcfb7200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfb80b2110 0x7fcfb80a9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:08.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.618+0000 7fcfb7200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb80b28b0 con 0x7fcfb80b2110 2024-07-04T17:37:08.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.619+0000 7fcfb5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfb80b2110 0x7fcfb80a9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:08.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.619+0000 7fcfb5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfb80b2110 0x7fcfb80a9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56170/0 (socket says 172.21.15.42:56170) 2024-07-04T17:37:08.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.619+0000 7fcfb5e00640 1 -- 172.21.15.42:0/4182768229 learned_addr learned my addr 172.21.15.42:0/4182768229 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:37:08.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.619+0000 7fcfb5e00640 1 -- 172.21.15.42:0/4182768229 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb80b1280 con 0x7fcfb80b2110 2024-07-04T17:37:08.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.619+0000 7fcfb5e00640 1 --2- 172.21.15.42:0/4182768229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfb80b2110 0x7fcfb80a9780 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fcfa80098d0 tx=0x7fcfa8033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:08.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.620+0000 7fcfae000640 1 -- 172.21.15.42:0/4182768229 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcfa8031880 con 0x7fcfb80b2110 2024-07-04T17:37:08.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.620+0000 7fcfae000640 1 -- 172.21.15.42:0/4182768229 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcfa8031b10 con 0x7fcfb80b2110 2024-07-04T17:37:08.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.620+0000 7fcfae000640 1 -- 172.21.15.42:0/4182768229 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcfa8031df0 con 0x7fcfb80b2110 2024-07-04T17:37:08.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.620+0000 7fcfb7200640 1 -- 172.21.15.42:0/4182768229 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfb80a9e80 con 0x7fcfb80b2110 2024-07-04T17:37:08.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.620+0000 7fcfb7200640 1 -- 172.21.15.42:0/4182768229 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfb80b8670 con 0x7fcfb80b2110 2024-07-04T17:37:08.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.621+0000 7fcfb7200640 1 -- 172.21.15.42:0/4182768229 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfb80b82a0 con 0x7fcfb80b2110 2024-07-04T17:37:08.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.621+0000 7fcfae000640 1 -- 172.21.15.42:0/4182768229 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 64) ==== 49615+0+0 (secure 0 0 0) 0x7fcfa8033070 con 0x7fcfb80b2110 2024-07-04T17:37:08.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.622+0000 7fcfae000640 1 --2- 172.21.15.42:0/4182768229 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fcf9403d4f0 0x7fcf9403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:08.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.622+0000 7fcfae000640 1 -- 172.21.15.42:0/4182768229 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4340+0+0 (secure 0 0 0) 0x7fcfa80730f0 con 0x7fcfb80b2110 2024-07-04T17:37:08.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.622+0000 7fcfafe00640 1 --2- 172.21.15.42:0/4182768229 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fcf9403d4f0 0x7fcf9403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:08.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.623+0000 7fcfafe00640 1 --2- 172.21.15.42:0/4182768229 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fcf9403d4f0 0x7fcf9403fa00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0007fd0 tx=0x7fcfa0007920 comp rx=0 tx=0).ready entity=mgr.5988 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:08.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.627+0000 7fcfae000640 1 -- 172.21.15.42:0/4182768229 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fcfa803f030 con 0x7fcfb80b2110 2024-07-04T17:37:08.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.874+0000 7fcfb7200640 1 -- 172.21.15.42:0/4182768229 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "backup_fs"} v 0) -- 0x7fcfb80b0ba0 con 0x7fcfb80b2110 2024-07-04T17:37:08.876 INFO:tasks.ceph.mon.a.smithi042.stderr:2024-07-04T17:37:08.875+0000 7f8616e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-04T17:37:08.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.878+0000 7fcfae000640 1 -- 172.21.15.42:0/4182768229 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "backup_fs"}]=0 backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v118) ==== 167+0+0 (secure 0 0 0) 0x7fcfa803a3a0 con 0x7fcfb80b2110 2024-07-04T17:37:08.879 INFO:teuthology.orchestra.run.smithi042.stderr:backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-04T17:37:08.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.881+0000 7fcfb7200640 1 -- 172.21.15.42:0/4182768229 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fcf9403d4f0 msgr2=0x7fcf9403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:08.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.881+0000 7fcfb7200640 1 --2- 172.21.15.42:0/4182768229 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fcf9403d4f0 0x7fcf9403fa00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0007fd0 tx=0x7fcfa0007920 comp rx=0 tx=0).stop 2024-07-04T17:37:08.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.881+0000 7fcfb7200640 1 -- 172.21.15.42:0/4182768229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfb80b2110 msgr2=0x7fcfb80a9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:08.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.881+0000 7fcfb7200640 1 --2- 172.21.15.42:0/4182768229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfb80b2110 0x7fcfb80a9780 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fcfa80098d0 tx=0x7fcfa8033ad0 comp rx=0 tx=0).stop 2024-07-04T17:37:08.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.882+0000 7fcfb7200640 1 -- 172.21.15.42:0/4182768229 shutdown_connections 2024-07-04T17:37:08.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.882+0000 7fcfb7200640 1 --2- 172.21.15.42:0/4182768229 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fcf9403d4f0 0x7fcf9403fa00 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:08.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.882+0000 7fcfb7200640 1 --2- 172.21.15.42:0/4182768229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfb80b2110 0x7fcfb80a9780 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:08.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.882+0000 7fcfb7200640 1 -- 172.21.15.42:0/4182768229 >> 172.21.15.42:0/4182768229 conn(0x7fcfb80bebc0 msgr2=0x7fcfb80bcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:08.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.882+0000 7fcfb7200640 1 -- 172.21.15.42:0/4182768229 shutdown_connections 2024-07-04T17:37:08.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:08.883+0000 7fcfb7200640 1 -- 172.21.15.42:0/4182768229 wait complete. 2024-07-04T17:37:08.904 INFO:tasks.ceph.mds.a.smithi042.stderr: -8707> 2024-07-04T17:34:57.458+0000 7fe361e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-04T17:37:08.913 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm backup_fs --yes-i-really-mean-it 2024-07-04T17:37:09.070 INFO:tasks.ceph.mds.a.smithi042.stdout:starting mds.a at 2024-07-04T17:37:09.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.205+0000 7f9443600640 1 -- 172.21.15.42:0/556982506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f943c0b2550 msgr2=0x7f943c0af6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:09.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.205+0000 7f9443600640 1 --2- 172.21.15.42:0/556982506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f943c0b2550 0x7f943c0af6f0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f942c0097b0 tx=0x7f942c02a530 comp rx=0 tx=0).stop 2024-07-04T17:37:09.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.205+0000 7f9443600640 1 -- 172.21.15.42:0/556982506 shutdown_connections 2024-07-04T17:37:09.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.205+0000 7f9443600640 1 --2- 172.21.15.42:0/556982506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f943c0b2550 0x7f943c0af6f0 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:09.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.205+0000 7f9443600640 1 -- 172.21.15.42:0/556982506 >> 172.21.15.42:0/556982506 conn(0x7f943c06fe40 msgr2=0x7f943c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:09.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.205+0000 7f9443600640 1 -- 172.21.15.42:0/556982506 shutdown_connections 2024-07-04T17:37:09.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.206+0000 7f9443600640 1 -- 172.21.15.42:0/556982506 wait complete. 2024-07-04T17:37:09.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.207+0000 7f9443600640 1 Processor -- start 2024-07-04T17:37:09.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.208+0000 7f9443600640 1 -- start start 2024-07-04T17:37:09.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.209+0000 7f9443600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f943c0addb0 0x7f943c0b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:09.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.209+0000 7f9443600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f943c1556c0 con 0x7f943c0addb0 2024-07-04T17:37:09.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.209+0000 7f9440c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f943c0addb0 0x7f943c0b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:09.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.209+0000 7f9440c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f943c0addb0 0x7f943c0b4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56202/0 (socket says 172.21.15.42:56202) 2024-07-04T17:37:09.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.209+0000 7f9440c00640 1 -- 172.21.15.42:0/2989117682 learned_addr learned my addr 172.21.15.42:0/2989117682 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:37:09.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.210+0000 7f9440c00640 1 -- 172.21.15.42:0/2989117682 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f943c071250 con 0x7f943c0addb0 2024-07-04T17:37:09.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.210+0000 7f9440c00640 1 --2- 172.21.15.42:0/2989117682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f943c0addb0 0x7f943c0b4c20 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f942c0098d0 tx=0x7f942c033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:09.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.210+0000 7f9438c00640 1 -- 172.21.15.42:0/2989117682 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f942c031880 con 0x7f943c0addb0 2024-07-04T17:37:09.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.210+0000 7f9438c00640 1 -- 172.21.15.42:0/2989117682 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f942c031b10 con 0x7f943c0addb0 2024-07-04T17:37:09.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.210+0000 7f9443600640 1 -- 172.21.15.42:0/2989117682 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f943c0b5160 con 0x7f943c0addb0 2024-07-04T17:37:09.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.210+0000 7f9438c00640 1 -- 172.21.15.42:0/2989117682 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f942c031df0 con 0x7f943c0addb0 2024-07-04T17:37:09.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.210+0000 7f9443600640 1 -- 172.21.15.42:0/2989117682 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f943c154a40 con 0x7f943c0addb0 2024-07-04T17:37:09.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.212+0000 7f9438c00640 1 -- 172.21.15.42:0/2989117682 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 64) ==== 49615+0+0 (secure 0 0 0) 0x7f942c0330d0 con 0x7f943c0addb0 2024-07-04T17:37:09.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.212+0000 7f9443600640 1 -- 172.21.15.42:0/2989117682 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f943c0af6f0 con 0x7f943c0addb0 2024-07-04T17:37:09.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.212+0000 7f9438c00640 1 --2- 172.21.15.42:0/2989117682 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f941803d4a0 0x7f941803f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:09.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.212+0000 7f9438c00640 1 -- 172.21.15.42:0/2989117682 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4426+0+0 (secure 0 0 0) 0x7f942c0731f0 con 0x7f943c0addb0 2024-07-04T17:37:09.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.212+0000 7f943be00640 1 --2- 172.21.15.42:0/2989117682 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f941803d4a0 0x7f941803f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:09.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.213+0000 7f943be00640 1 --2- 172.21.15.42:0/2989117682 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f941803d4a0 0x7f941803f9b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f94240015e0 tx=0x7f9424001290 comp rx=0 tx=0).ready entity=mgr.5988 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:09.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.218+0000 7f9438c00640 1 -- 172.21.15.42:0/2989117682 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f942c03f030 con 0x7f943c0addb0 2024-07-04T17:37:09.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.472+0000 7f9443600640 1 -- 172.21.15.42:0/2989117682 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true} v 0) -- 0x7f943c069b70 con 0x7f943c0addb0 2024-07-04T17:37:09.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.885+0000 7f9438c00640 1 -- 172.21.15.42:0/2989117682 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true}]=0 v119) ==== 107+0+0 (secure 0 0 0) 0x7f942c03a560 con 0x7f943c0addb0 2024-07-04T17:37:09.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.889+0000 7f9443600640 1 -- 172.21.15.42:0/2989117682 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f941803d4a0 msgr2=0x7f941803f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:09.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.889+0000 7f9443600640 1 --2- 172.21.15.42:0/2989117682 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f941803d4a0 0x7f941803f9b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f94240015e0 tx=0x7f9424001290 comp rx=0 tx=0).stop 2024-07-04T17:37:09.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.889+0000 7f9443600640 1 -- 172.21.15.42:0/2989117682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f943c0addb0 msgr2=0x7f943c0b4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:09.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.889+0000 7f9443600640 1 --2- 172.21.15.42:0/2989117682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f943c0addb0 0x7f943c0b4c20 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f942c0098d0 tx=0x7f942c033ad0 comp rx=0 tx=0).stop 2024-07-04T17:37:09.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.889+0000 7f9443600640 1 -- 172.21.15.42:0/2989117682 shutdown_connections 2024-07-04T17:37:09.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.889+0000 7f9443600640 1 --2- 172.21.15.42:0/2989117682 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f941803d4a0 0x7f941803f9b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:09.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.889+0000 7f9443600640 1 --2- 172.21.15.42:0/2989117682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f943c0addb0 0x7f943c0b4c20 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:09.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.889+0000 7f9443600640 1 -- 172.21.15.42:0/2989117682 >> 172.21.15.42:0/2989117682 conn(0x7f943c06fe40 msgr2=0x7f943c145f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:09.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.889+0000 7f9443600640 1 -- 172.21.15.42:0/2989117682 shutdown_connections 2024-07-04T17:37:09.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:09.889+0000 7f9443600640 1 -- 172.21.15.42:0/2989117682 wait complete. 2024-07-04T17:37:09.921 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_metadata backup_fs_metadata --yes-i-really-really-mean-it 2024-07-04T17:37:10.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.214+0000 7fdd84c00640 1 -- 172.21.15.42:0/2112881502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd800ab2f0 msgr2=0x7fdd800a9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:10.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.214+0000 7fdd84c00640 1 --2- 172.21.15.42:0/2112881502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd800ab2f0 0x7fdd800a9990 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fdd700097b0 tx=0x7fdd7002a540 comp rx=0 tx=0).stop 2024-07-04T17:37:10.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.214+0000 7fdd84c00640 1 -- 172.21.15.42:0/2112881502 shutdown_connections 2024-07-04T17:37:10.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.214+0000 7fdd84c00640 1 --2- 172.21.15.42:0/2112881502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd800ab2f0 0x7fdd800a9990 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:10.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.214+0000 7fdd84c00640 1 -- 172.21.15.42:0/2112881502 >> 172.21.15.42:0/2112881502 conn(0x7fdd8006c020 msgr2=0x7fdd8006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:10.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.214+0000 7fdd84c00640 1 -- 172.21.15.42:0/2112881502 shutdown_connections 2024-07-04T17:37:10.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.214+0000 7fdd84c00640 1 -- 172.21.15.42:0/2112881502 wait complete. 2024-07-04T17:37:10.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.216+0000 7fdd84c00640 1 Processor -- start 2024-07-04T17:37:10.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.217+0000 7fdd84c00640 1 -- start start 2024-07-04T17:37:10.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.218+0000 7fdd84c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd8006d0f0 0x7fdd800ae480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:10.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.218+0000 7fdd84c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8013f430 con 0x7fdd8006d0f0 2024-07-04T17:37:10.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.218+0000 7fdd7de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd8006d0f0 0x7fdd800ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:10.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.218+0000 7fdd7de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd8006d0f0 0x7fdd800ae480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56228/0 (socket says 172.21.15.42:56228) 2024-07-04T17:37:10.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.218+0000 7fdd7de00640 1 -- 172.21.15.42:0/1735824217 learned_addr learned my addr 172.21.15.42:0/1735824217 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:37:10.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.219+0000 7fdd7de00640 1 -- 172.21.15.42:0/1735824217 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd801e33d0 con 0x7fdd8006d0f0 2024-07-04T17:37:10.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.219+0000 7fdd7de00640 1 --2- 172.21.15.42:0/1735824217 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd8006d0f0 0x7fdd800ae480 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fdd7002ab30 tx=0x7fdd70032ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:10.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.219+0000 7fdd76000640 1 -- 172.21.15.42:0/1735824217 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd70040070 con 0x7fdd8006d0f0 2024-07-04T17:37:10.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.219+0000 7fdd76000640 1 -- 172.21.15.42:0/1735824217 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd700036e0 con 0x7fdd8006d0f0 2024-07-04T17:37:10.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.219+0000 7fdd84c00640 1 -- 172.21.15.42:0/1735824217 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd800ae9c0 con 0x7fdd8006d0f0 2024-07-04T17:37:10.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.219+0000 7fdd76000640 1 -- 172.21.15.42:0/1735824217 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd70003a40 con 0x7fdd8006d0f0 2024-07-04T17:37:10.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.219+0000 7fdd84c00640 1 -- 172.21.15.42:0/1735824217 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd800bf7c0 con 0x7fdd8006d0f0 2024-07-04T17:37:10.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.221+0000 7fdd84c00640 1 -- 172.21.15.42:0/1735824217 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd8013fb00 con 0x7fdd8006d0f0 2024-07-04T17:37:10.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.221+0000 7fdd76000640 1 -- 172.21.15.42:0/1735824217 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 64) ==== 49615+0+0 (secure 0 0 0) 0x7fdd70039040 con 0x7fdd8006d0f0 2024-07-04T17:37:10.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.221+0000 7fdd76000640 1 --2- 172.21.15.42:0/1735824217 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fdd5c03d4f0 0x7fdd5c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:10.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.221+0000 7fdd76000640 1 -- 172.21.15.42:0/1735824217 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4426+0+0 (secure 0 0 0) 0x7fdd700732f0 con 0x7fdd8006d0f0 2024-07-04T17:37:10.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.222+0000 7fdd7d400640 1 --2- 172.21.15.42:0/1735824217 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fdd5c03d4f0 0x7fdd5c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:10.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.222+0000 7fdd7d400640 1 --2- 172.21.15.42:0/1735824217 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fdd5c03d4f0 0x7fdd5c03fa00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdd68002020 tx=0x7fdd68001610 comp rx=0 tx=0).ready entity=mgr.5988 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:10.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.227+0000 7fdd76000640 1 -- 172.21.15.42:0/1735824217 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fdd7003b3b0 con 0x7fdd8006d0f0 2024-07-04T17:37:10.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.444+0000 7fdd84c00640 1 -- 172.21.15.42:0/1735824217 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdd801e35a0 con 0x7fdd8006d0f0 2024-07-04T17:37:10.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.887+0000 7fdd76000640 1 -- 172.21.15.42:0/1735824217 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_metadata' removed v150) ==== 190+0+0 (secure 0 0 0) 0x7fdd7004ee10 con 0x7fdd8006d0f0 2024-07-04T17:37:10.888 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'backup_fs_metadata' removed 2024-07-04T17:37:10.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.894+0000 7fdd84c00640 1 -- 172.21.15.42:0/1735824217 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fdd5c03d4f0 msgr2=0x7fdd5c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:10.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.894+0000 7fdd84c00640 1 --2- 172.21.15.42:0/1735824217 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fdd5c03d4f0 0x7fdd5c03fa00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdd68002020 tx=0x7fdd68001610 comp rx=0 tx=0).stop 2024-07-04T17:37:10.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.894+0000 7fdd84c00640 1 -- 172.21.15.42:0/1735824217 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd8006d0f0 msgr2=0x7fdd800ae480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:10.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.894+0000 7fdd84c00640 1 --2- 172.21.15.42:0/1735824217 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd8006d0f0 0x7fdd800ae480 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fdd7002ab30 tx=0x7fdd70032ac0 comp rx=0 tx=0).stop 2024-07-04T17:37:10.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.895+0000 7fdd84c00640 1 -- 172.21.15.42:0/1735824217 shutdown_connections 2024-07-04T17:37:10.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.895+0000 7fdd84c00640 1 --2- 172.21.15.42:0/1735824217 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fdd5c03d4f0 0x7fdd5c03fa00 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:10.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.895+0000 7fdd84c00640 1 --2- 172.21.15.42:0/1735824217 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd8006d0f0 0x7fdd800ae480 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:10.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.895+0000 7fdd84c00640 1 -- 172.21.15.42:0/1735824217 >> 172.21.15.42:0/1735824217 conn(0x7fdd8006c020 msgr2=0x7fdd801411a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:10.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.896+0000 7fdd84c00640 1 -- 172.21.15.42:0/1735824217 shutdown_connections 2024-07-04T17:37:10.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:10.896+0000 7fdd84c00640 1 -- 172.21.15.42:0/1735824217 wait complete. 2024-07-04T17:37:10.940 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_data backup_fs_data --yes-i-really-really-mean-it 2024-07-04T17:37:11.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.214+0000 7efcdb200640 1 -- 172.21.15.42:0/1758711143 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efcdc0b1640 msgr2=0x7efcdc13db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:11.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.214+0000 7efcdb200640 1 --2- 172.21.15.42:0/1758711143 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efcdc0b1640 0x7efcdc13db60 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7efcc400a8c0 tx=0x7efcc402b860 comp rx=0 tx=0).stop 2024-07-04T17:37:11.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.214+0000 7efcdb200640 1 -- 172.21.15.42:0/1758711143 shutdown_connections 2024-07-04T17:37:11.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.214+0000 7efcdb200640 1 --2- 172.21.15.42:0/1758711143 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efcdc0b1640 0x7efcdc13db60 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:11.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.214+0000 7efcdb200640 1 -- 172.21.15.42:0/1758711143 >> 172.21.15.42:0/1758711143 conn(0x7efcdc0afaa0 msgr2=0x7efcdc0afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:11.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.214+0000 7efcdb200640 1 -- 172.21.15.42:0/1758711143 shutdown_connections 2024-07-04T17:37:11.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.214+0000 7efcdb200640 1 -- 172.21.15.42:0/1758711143 wait complete. 2024-07-04T17:37:11.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.217+0000 7efcdb200640 1 Processor -- start 2024-07-04T17:37:11.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.217+0000 7efcdb200640 1 -- start start 2024-07-04T17:37:11.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.218+0000 7efcdb200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efcdc0b1640 0x7efcdc139a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:11.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.218+0000 7efcdb200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efcdc0b0f60 con 0x7efcdc0b1640 2024-07-04T17:37:11.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.218+0000 7efcd9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efcdc0b1640 0x7efcdc139a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:11.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.219+0000 7efcd9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efcdc0b1640 0x7efcdc139a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56250/0 (socket says 172.21.15.42:56250) 2024-07-04T17:37:11.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.219+0000 7efcd9e00640 1 -- 172.21.15.42:0/3361397964 learned_addr learned my addr 172.21.15.42:0/3361397964 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:37:11.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.219+0000 7efcd9e00640 1 -- 172.21.15.42:0/3361397964 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcdc137910 con 0x7efcdc0b1640 2024-07-04T17:37:11.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.219+0000 7efcd9e00640 1 --2- 172.21.15.42:0/3361397964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efcdc0b1640 0x7efcdc139a30 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7efccc00c380 tx=0x7efccc00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:11.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.220+0000 7efcd2000640 1 -- 172.21.15.42:0/3361397964 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efccc0116c0 con 0x7efcdc0b1640 2024-07-04T17:37:11.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.220+0000 7efcd2000640 1 -- 172.21.15.42:0/3361397964 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efccc017070 con 0x7efcdc0b1640 2024-07-04T17:37:11.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.220+0000 7efcdb200640 1 -- 172.21.15.42:0/3361397964 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcdc0b3ab0 con 0x7efcdc0b1640 2024-07-04T17:37:11.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.220+0000 7efcd2000640 1 -- 172.21.15.42:0/3361397964 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efccc011b20 con 0x7efcdc0b1640 2024-07-04T17:37:11.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.220+0000 7efcdb200640 1 -- 172.21.15.42:0/3361397964 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcdc141320 con 0x7efcdc0b1640 2024-07-04T17:37:11.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.221+0000 7efcdb200640 1 -- 172.21.15.42:0/3361397964 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcdc139f70 con 0x7efcdc0b1640 2024-07-04T17:37:11.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.221+0000 7efcd2000640 1 -- 172.21.15.42:0/3361397964 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 64) ==== 49615+0+0 (secure 0 0 0) 0x7efccc011cc0 con 0x7efcdc0b1640 2024-07-04T17:37:11.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.222+0000 7efcd2000640 1 --2- 172.21.15.42:0/3361397964 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7efcb803d4f0 0x7efcb803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:11.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.222+0000 7efcd2000640 1 -- 172.21.15.42:0/3361397964 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 3982+0+0 (secure 0 0 0) 0x7efccc0518d0 con 0x7efcdc0b1640 2024-07-04T17:37:11.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.222+0000 7efcd3e00640 1 --2- 172.21.15.42:0/3361397964 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7efcb803d4f0 0x7efcb803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:11.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.223+0000 7efcd3e00640 1 --2- 172.21.15.42:0/3361397964 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7efcb803d4f0 0x7efcb803fa00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7efcc4036000 tx=0x7efcc4004730 comp rx=0 tx=0).ready entity=mgr.5988 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:11.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.228+0000 7efcd2000640 1 -- 172.21.15.42:0/3361397964 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7efccc00a140 con 0x7efcdc0b1640 2024-07-04T17:37:11.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.456+0000 7efcdb200640 1 -- 172.21.15.42:0/3361397964 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7efcdc13db60 con 0x7efcdc0b1640 2024-07-04T17:37:11.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.889+0000 7efcd2000640 1 -- 172.21.15.42:0/3361397964 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_data' removed v151) ==== 178+0+0 (secure 0 0 0) 0x7efccc010cf0 con 0x7efcdc0b1640 2024-07-04T17:37:11.891 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'backup_fs_data' removed 2024-07-04T17:37:11.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.894+0000 7efcdb200640 1 -- 172.21.15.42:0/3361397964 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7efcb803d4f0 msgr2=0x7efcb803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:11.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.894+0000 7efcdb200640 1 --2- 172.21.15.42:0/3361397964 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7efcb803d4f0 0x7efcb803fa00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7efcc4036000 tx=0x7efcc4004730 comp rx=0 tx=0).stop 2024-07-04T17:37:11.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.894+0000 7efcdb200640 1 -- 172.21.15.42:0/3361397964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efcdc0b1640 msgr2=0x7efcdc139a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:11.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.894+0000 7efcdb200640 1 --2- 172.21.15.42:0/3361397964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efcdc0b1640 0x7efcdc139a30 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7efccc00c380 tx=0x7efccc00c840 comp rx=0 tx=0).stop 2024-07-04T17:37:11.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.895+0000 7efcdb200640 1 -- 172.21.15.42:0/3361397964 shutdown_connections 2024-07-04T17:37:11.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.895+0000 7efcdb200640 1 --2- 172.21.15.42:0/3361397964 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7efcb803d4f0 0x7efcb803fa00 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:11.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.895+0000 7efcdb200640 1 --2- 172.21.15.42:0/3361397964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efcdc0b1640 0x7efcdc139a30 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:11.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.895+0000 7efcdb200640 1 -- 172.21.15.42:0/3361397964 >> 172.21.15.42:0/3361397964 conn(0x7efcdc0afaa0 msgr2=0x7efcdc075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:11.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.899+0000 7efcdb200640 1 -- 172.21.15.42:0/3361397964 shutdown_connections 2024-07-04T17:37:11.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:11.900+0000 7efcdb200640 1 -- 172.21.15.42:0/3361397964 wait complete. 2024-07-04T17:37:11.936 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 22 2024-07-04T17:37:12.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.205+0000 7f0f51e00640 1 -- 172.21.15.42:0/3962861871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f4c06c2c0 msgr2=0x7f0f4c0b5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:12.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.205+0000 7f0f51e00640 1 --2- 172.21.15.42:0/3962861871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f4c06c2c0 0x7f0f4c0b5650 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f0f3400b2b0 tx=0x7f0f3402ac80 comp rx=0 tx=0).stop 2024-07-04T17:37:12.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.206+0000 7f0f51e00640 1 -- 172.21.15.42:0/3962861871 shutdown_connections 2024-07-04T17:37:12.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.206+0000 7f0f51e00640 1 --2- 172.21.15.42:0/3962861871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f4c06c2c0 0x7f0f4c0b5650 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:12.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.206+0000 7f0f51e00640 1 -- 172.21.15.42:0/3962861871 >> 172.21.15.42:0/3962861871 conn(0x7f0f4c0bebc0 msgr2=0x7f0f4c0befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:12.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.206+0000 7f0f51e00640 1 -- 172.21.15.42:0/3962861871 shutdown_connections 2024-07-04T17:37:12.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.206+0000 7f0f51e00640 1 -- 172.21.15.42:0/3962861871 wait complete. 2024-07-04T17:37:12.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.208+0000 7f0f51e00640 1 Processor -- start 2024-07-04T17:37:12.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.209+0000 7f0f51e00640 1 -- start start 2024-07-04T17:37:12.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.210+0000 7f0f51e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f4c06c2c0 0x7f0f4c0b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:12.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.210+0000 7f0f51e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4c0b2810 con 0x7f0f4c06c2c0 2024-07-04T17:37:12.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.210+0000 7f0f50a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f4c06c2c0 0x7f0f4c0b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:12.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.210+0000 7f0f50a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f4c06c2c0 0x7f0f4c0b6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56266/0 (socket says 172.21.15.42:56266) 2024-07-04T17:37:12.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.210+0000 7f0f50a00640 1 -- 172.21.15.42:0/1635699741 learned_addr learned my addr 172.21.15.42:0/1635699741 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:37:12.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.211+0000 7f0f50a00640 1 -- 172.21.15.42:0/1635699741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f4c1d5390 con 0x7f0f4c06c2c0 2024-07-04T17:37:12.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.211+0000 7f0f50a00640 1 --2- 172.21.15.42:0/1635699741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f4c06c2c0 0x7f0f4c0b6310 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c00c380 tx=0x7f0f3c00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:12.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.211+0000 7f0f49600640 1 -- 172.21.15.42:0/1635699741 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f3c0116c0 con 0x7f0f4c06c2c0 2024-07-04T17:37:12.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.211+0000 7f0f49600640 1 -- 172.21.15.42:0/1635699741 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f3c017070 con 0x7f0f4c06c2c0 2024-07-04T17:37:12.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.211+0000 7f0f49600640 1 -- 172.21.15.42:0/1635699741 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f3c011b20 con 0x7f0f4c06c2c0 2024-07-04T17:37:12.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.211+0000 7f0f51e00640 1 -- 172.21.15.42:0/1635699741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f4c0b6900 con 0x7f0f4c06c2c0 2024-07-04T17:37:12.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.211+0000 7f0f51e00640 1 -- 172.21.15.42:0/1635699741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f4c1dcab0 con 0x7f0f4c06c2c0 2024-07-04T17:37:12.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.213+0000 7f0f51e00640 1 -- 172.21.15.42:0/1635699741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f4c0b5650 con 0x7f0f4c06c2c0 2024-07-04T17:37:12.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.213+0000 7f0f49600640 1 -- 172.21.15.42:0/1635699741 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 64) ==== 49615+0+0 (secure 0 0 0) 0x7f0f3c011cc0 con 0x7f0f4c06c2c0 2024-07-04T17:37:12.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.214+0000 7f0f49600640 1 --2- 172.21.15.42:0/1635699741 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f0f2803d4f0 0x7f0f2803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:12.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.214+0000 7f0f49600640 1 -- 172.21.15.42:0/1635699741 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 3533+0+0 (secure 0 0 0) 0x7f0f3c051640 con 0x7f0f4c06c2c0 2024-07-04T17:37:12.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.214+0000 7f0f43e00640 1 --2- 172.21.15.42:0/1635699741 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f0f2803d4f0 0x7f0f2803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:12.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.215+0000 7f0f43e00640 1 --2- 172.21.15.42:0/1635699741 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f0f2803d4f0 0x7f0f2803fa00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0f34035000 tx=0x7f0f34004730 comp rx=0 tx=0).ready entity=mgr.5988 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:12.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.219+0000 7f0f49600640 1 -- 172.21.15.42:0/1635699741 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f0f4c06c6d0 con 0x7f0f4c06c2c0 2024-07-04T17:37:12.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.426+0000 7f0f51e00640 1 -- 172.21.15.42:0/1635699741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config reset", "num": 22} v 0) -- 0x7f0f4c068e30 con 0x7f0f4c06c2c0 2024-07-04T17:37:12.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.433+0000 7f0f49600640 1 -- 172.21.15.42:0/1635699741 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 22}]=0 v34) ==== 71+0+0 (secure 0 0 0) 0x7f0f3c0513b0 con 0x7f0f4c06c2c0 2024-07-04T17:37:12.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.436+0000 7f0f51e00640 1 -- 172.21.15.42:0/1635699741 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f0f2803d4f0 msgr2=0x7f0f2803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:12.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.436+0000 7f0f51e00640 1 --2- 172.21.15.42:0/1635699741 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f0f2803d4f0 0x7f0f2803fa00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0f34035000 tx=0x7f0f34004730 comp rx=0 tx=0).stop 2024-07-04T17:37:12.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.436+0000 7f0f51e00640 1 -- 172.21.15.42:0/1635699741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f4c06c2c0 msgr2=0x7f0f4c0b6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:12.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.436+0000 7f0f51e00640 1 --2- 172.21.15.42:0/1635699741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f4c06c2c0 0x7f0f4c0b6310 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c00c380 tx=0x7f0f3c00c840 comp rx=0 tx=0).stop 2024-07-04T17:37:12.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.436+0000 7f0f51e00640 1 -- 172.21.15.42:0/1635699741 shutdown_connections 2024-07-04T17:37:12.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.436+0000 7f0f51e00640 1 --2- 172.21.15.42:0/1635699741 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f0f2803d4f0 0x7f0f2803fa00 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:12.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.436+0000 7f0f51e00640 1 --2- 172.21.15.42:0/1635699741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f4c06c2c0 0x7f0f4c0b6310 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:12.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.436+0000 7f0f51e00640 1 -- 172.21.15.42:0/1635699741 >> 172.21.15.42:0/1635699741 conn(0x7f0f4c0bebc0 msgr2=0x7f0f4c140aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:12.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.436+0000 7f0f51e00640 1 -- 172.21.15.42:0/1635699741 shutdown_connections 2024-07-04T17:37:12.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.436+0000 7f0f51e00640 1 -- 172.21.15.42:0/1635699741 wait complete. 2024-07-04T17:37:12.466 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_mirroring.TestMirroring.test_cephfs_mirror_blocklist' 2024-07-04T17:37:12.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.757+0000 7fc2d5e00640 1 -- 172.21.15.42:0/4226730611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2d00b2d20 msgr2=0x7fc2d00bb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:12.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.757+0000 7fc2d5e00640 1 --2- 172.21.15.42:0/4226730611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2d00b2d20 0x7fc2d00bb1f0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7fc2b80097b0 tx=0x7fc2b802a4a0 comp rx=0 tx=0).stop 2024-07-04T17:37:12.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.757+0000 7fc2d5e00640 1 -- 172.21.15.42:0/4226730611 shutdown_connections 2024-07-04T17:37:12.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.757+0000 7fc2d5e00640 1 --2- 172.21.15.42:0/4226730611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2d00b2d20 0x7fc2d00bb1f0 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:12.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.757+0000 7fc2d5e00640 1 -- 172.21.15.42:0/4226730611 >> 172.21.15.42:0/4226730611 conn(0x7fc2d00bf700 msgr2=0x7fc2d00af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:12.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.757+0000 7fc2d5e00640 1 -- 172.21.15.42:0/4226730611 shutdown_connections 2024-07-04T17:37:12.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.757+0000 7fc2d5e00640 1 -- 172.21.15.42:0/4226730611 wait complete. 2024-07-04T17:37:12.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.759+0000 7fc2d5e00640 1 Processor -- start 2024-07-04T17:37:12.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.759+0000 7fc2d5e00640 1 -- start start 2024-07-04T17:37:12.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.760+0000 7fc2d5e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2d00b2d20 0x7fc2d01d6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:12.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.760+0000 7fc2d5e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d006c020 con 0x7fc2d00b2d20 2024-07-04T17:37:12.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.761+0000 7fc2cea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2d00b2d20 0x7fc2d01d6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:12.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.761+0000 7fc2cea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2d00b2d20 0x7fc2d01d6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56282/0 (socket says 172.21.15.42:56282) 2024-07-04T17:37:12.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.761+0000 7fc2cea00640 1 -- 172.21.15.42:0/383431932 learned_addr learned my addr 172.21.15.42:0/383431932 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:37:12.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.761+0000 7fc2cea00640 1 -- 172.21.15.42:0/383431932 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d01e1370 con 0x7fc2d00b2d20 2024-07-04T17:37:12.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.761+0000 7fc2cea00640 1 --2- 172.21.15.42:0/383431932 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2d00b2d20 0x7fc2d01d6cc0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000c3b0 tx=0x7fc2c000c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:12.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.762+0000 7fc2c7400640 1 -- 172.21.15.42:0/383431932 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2c0011780 con 0x7fc2d00b2d20 2024-07-04T17:37:12.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.762+0000 7fc2c7400640 1 -- 172.21.15.42:0/383431932 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2c0017070 con 0x7fc2d00b2d20 2024-07-04T17:37:12.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.762+0000 7fc2c7400640 1 -- 172.21.15.42:0/383431932 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2c0011c00 con 0x7fc2d00b2d20 2024-07-04T17:37:12.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.762+0000 7fc2d5e00640 1 -- 172.21.15.42:0/383431932 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2d01e1600 con 0x7fc2d00b2d20 2024-07-04T17:37:12.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.762+0000 7fc2d5e00640 1 -- 172.21.15.42:0/383431932 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2d00acb90 con 0x7fc2d00b2d20 2024-07-04T17:37:12.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.763+0000 7fc2d5e00640 1 -- 172.21.15.42:0/383431932 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2d0137e00 con 0x7fc2d00b2d20 2024-07-04T17:37:12.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.763+0000 7fc2c7400640 1 -- 172.21.15.42:0/383431932 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 64) ==== 49615+0+0 (secure 0 0 0) 0x7fc2c001c460 con 0x7fc2d00b2d20 2024-07-04T17:37:12.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.764+0000 7fc2c7400640 1 --2- 172.21.15.42:0/383431932 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fc2ac03d4f0 0x7fc2ac03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:12.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.764+0000 7fc2c7400640 1 -- 172.21.15.42:0/383431932 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 3533+0+0 (secure 0 0 0) 0x7fc2c0051540 con 0x7fc2d00b2d20 2024-07-04T17:37:12.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.764+0000 7fc2c7e00640 1 --2- 172.21.15.42:0/383431932 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fc2ac03d4f0 0x7fc2ac03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:12.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.765+0000 7fc2c7e00640 1 --2- 172.21.15.42:0/383431932 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fc2ac03d4f0 0x7fc2ac03fa00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8034000 tx=0x7fc2b8004730 comp rx=0 tx=0).ready entity=mgr.5988 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:12.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:12.769+0000 7fc2c7400640 1 -- 172.21.15.42:0/383431932 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fc2c0011da0 con 0x7fc2d00b2d20 2024-07-04T17:37:13.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:13.058+0000 7fc2d5e00640 1 -- 172.21.15.42:0/383431932 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_mirroring.TestMirroring.test_cephfs_mirror_blocklist"]} v 0) -- 0x7fc2d006c710 con 0x7fc2d00b2d20 2024-07-04T17:37:13.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:13.909+0000 7fc2c7400640 1 -- 172.21.15.42:0/383431932 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_mirroring.TestMirroring.test_cephfs_mirror_blocklist"]}]=0 v540) ==== 149+0+0 (secure 0 0 0) 0x7fc2c00512b0 con 0x7fc2d00b2d20 2024-07-04T17:37:13.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:13.912+0000 7fc2d5e00640 1 -- 172.21.15.42:0/383431932 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fc2ac03d4f0 msgr2=0x7fc2ac03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:13.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:13.912+0000 7fc2d5e00640 1 --2- 172.21.15.42:0/383431932 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fc2ac03d4f0 0x7fc2ac03fa00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8034000 tx=0x7fc2b8004730 comp rx=0 tx=0).stop 2024-07-04T17:37:13.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:13.912+0000 7fc2d5e00640 1 -- 172.21.15.42:0/383431932 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2d00b2d20 msgr2=0x7fc2d01d6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:13.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:13.912+0000 7fc2d5e00640 1 --2- 172.21.15.42:0/383431932 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2d00b2d20 0x7fc2d01d6cc0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000c3b0 tx=0x7fc2c000c870 comp rx=0 tx=0).stop 2024-07-04T17:37:13.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:13.913+0000 7fc2d5e00640 1 -- 172.21.15.42:0/383431932 shutdown_connections 2024-07-04T17:37:13.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:13.913+0000 7fc2d5e00640 1 --2- 172.21.15.42:0/383431932 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fc2ac03d4f0 0x7fc2ac03fa00 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:13.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:13.913+0000 7fc2d5e00640 1 --2- 172.21.15.42:0/383431932 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2d00b2d20 0x7fc2d01d6cc0 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:13.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:13.913+0000 7fc2d5e00640 1 -- 172.21.15.42:0/383431932 >> 172.21.15.42:0/383431932 conn(0x7fc2d00bf700 msgr2=0x7fc2d0140070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:13.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:13.913+0000 7fc2d5e00640 1 -- 172.21.15.42:0/383431932 shutdown_connections 2024-07-04T17:37:13.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:13.913+0000 7fc2d5e00640 1 -- 172.21.15.42:0/383431932 wait complete. 2024-07-04T17:37:13.944 INFO:tasks.cephfs_test_runner:test_cephfs_mirror_blocklist (tasks.cephfs.test_mirroring.TestMirroring) ... ERROR 2024-07-04T17:37:13.944 INFO:tasks.cephfs_test_runner: 2024-07-04T17:37:13.944 INFO:tasks.cephfs_test_runner:====================================================================== 2024-07-04T17:37:13.944 INFO:tasks.cephfs_test_runner:ERROR: test_cephfs_mirror_blocklist (tasks.cephfs.test_mirroring.TestMirroring) 2024-07-04T17:37:13.944 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-07-04T17:37:13.944 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-07-04T17:37:13.944 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_9dba1e836b4756bf3a32be2a4936f371dff55008/qa/tasks/cephfs/test_mirroring.py", line 517, in test_cephfs_mirror_blocklist 2024-07-04T17:37:13.944 INFO:tasks.cephfs_test_runner: rados_inst_new = self.get_mirror_rados_addr(self.primary_fs_name, self.primary_fs_id) 2024-07-04T17:37:13.945 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_9dba1e836b4756bf3a32be2a4936f371dff55008/qa/tasks/cephfs/test_mirroring.py", line 281, in get_mirror_rados_addr 2024-07-04T17:37:13.945 INFO:tasks.cephfs_test_runner: return res['rados_inst'] 2024-07-04T17:37:13.945 INFO:tasks.cephfs_test_runner:KeyError: 'rados_inst' 2024-07-04T17:37:13.945 INFO:tasks.cephfs_test_runner: 2024-07-04T17:37:13.945 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-07-04T17:37:13.945 INFO:tasks.cephfs_test_runner:Ran 6 tests in 629.746s 2024-07-04T17:37:13.945 INFO:tasks.cephfs_test_runner: 2024-07-04T17:37:13.945 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-07-04T17:37:13.945 INFO:tasks.cephfs_test_runner: 2024-07-04T17:37:13.945 INFO:tasks.cephfs_test_runner:====================================================================== 2024-07-04T17:37:13.945 INFO:tasks.cephfs_test_runner:ERROR: test_cephfs_mirror_blocklist (tasks.cephfs.test_mirroring.TestMirroring) 2024-07-04T17:37:13.945 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-07-04T17:37:13.945 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-07-04T17:37:13.946 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_9dba1e836b4756bf3a32be2a4936f371dff55008/qa/tasks/cephfs/test_mirroring.py", line 517, in test_cephfs_mirror_blocklist 2024-07-04T17:37:13.946 INFO:tasks.cephfs_test_runner: rados_inst_new = self.get_mirror_rados_addr(self.primary_fs_name, self.primary_fs_id) 2024-07-04T17:37:13.946 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_9dba1e836b4756bf3a32be2a4936f371dff55008/qa/tasks/cephfs/test_mirroring.py", line 281, in get_mirror_rados_addr 2024-07-04T17:37:13.946 INFO:tasks.cephfs_test_runner: return res['rados_inst'] 2024-07-04T17:37:13.946 INFO:tasks.cephfs_test_runner:KeyError: 'rados_inst' 2024-07-04T17:37:13.946 INFO:tasks.cephfs_test_runner: 2024-07-04T17:37:13.946 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_a4a89a3d72be587ba6bd8509090508e8a19e74c5/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_9dba1e836b4756bf3a32be2a4936f371dff55008/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_cephfs_mirror_blocklist (tasks.cephfs.test_mirroring.TestMirroring) 2024-07-04T17:37:14.124 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=3acb7460b79345ec872a8beaa4e05116 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_a4a89a3d72be587ba6bd8509090508e8a19e74c5/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_9dba1e836b4756bf3a32be2a4936f371dff55008/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_cephfs_mirror_blocklist (tasks.cephfs.test_mirroring.TestMirroring) 2024-07-04T17:37:14.126 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-07-04T17:37:14.136 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-07-04T17:37:14.159 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-07-04T17:37:14.160 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:37:14.160 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:37:14.222 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-07-04T17:37:14.222 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-07-04T17:37:14.222 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T17:37:14.276 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-07-04T17:37:14.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:37:14.276 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-07-04T17:37:14.333 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-04T17:37:14.334 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2024-07-04T17:37:14.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:37:14.335 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T17:37:14.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:37:14.426 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-07-04T17:37:14.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T17:37:14.504 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link delete ceph-brx 2024-07-04T17:37:14.621 DEBUG:teuthology.run_tasks:Unwinding manager cephfs-mirror 2024-07-04T17:37:14.632 DEBUG:tasks.cephfs_mirror.client.mirror:waiting for process to exit 2024-07-04T17:37:14.632 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:37:14.632 INFO:tasks.cephfs_mirror.client.mirror:Stopped 2024-07-04T17:37:14.632 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-07-04T17:37:14.660 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-07-04T17:37:14.660 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:37:14.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:14.934+0000 7fadd3400640 1 -- 172.21.15.42:0/3866813040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadcc06cea0 msgr2=0x7fadcc0b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:14.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:14.934+0000 7fadd3400640 1 --2- 172.21.15.42:0/3866813040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadcc06cea0 0x7fadcc0b3030 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fadc000b2b0 tx=0x7fadc002ad10 comp rx=0 tx=0).stop 2024-07-04T17:37:14.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:14.934+0000 7fadd3400640 1 -- 172.21.15.42:0/3866813040 shutdown_connections 2024-07-04T17:37:14.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:14.934+0000 7fadd3400640 1 --2- 172.21.15.42:0/3866813040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadcc06cea0 0x7fadcc0b3030 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:14.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:14.934+0000 7fadd3400640 1 -- 172.21.15.42:0/3866813040 >> 172.21.15.42:0/3866813040 conn(0x7fadcc0bf700 msgr2=0x7fadcc0af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:14.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:14.934+0000 7fadd3400640 1 -- 172.21.15.42:0/3866813040 shutdown_connections 2024-07-04T17:37:14.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:14.934+0000 7fadd3400640 1 -- 172.21.15.42:0/3866813040 wait complete. 2024-07-04T17:37:14.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:14.936+0000 7fadd3400640 1 Processor -- start 2024-07-04T17:37:14.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:14.936+0000 7fadd3400640 1 -- start start 2024-07-04T17:37:14.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:14.937+0000 7fadd3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadcc06cea0 0x7fadcc148d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:14.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:14.938+0000 7fadd3400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fadcc06ef50 con 0x7fadcc06cea0 2024-07-04T17:37:14.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:14.938+0000 7fadd0a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadcc06cea0 0x7fadcc148d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:14.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:14.938+0000 7fadd0a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadcc06cea0 0x7fadcc148d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44042/0 (socket says 172.21.15.42:44042) 2024-07-04T17:37:14.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:14.938+0000 7fadd0a00640 1 -- 172.21.15.42:0/4058512254 learned_addr learned my addr 172.21.15.42:0/4058512254 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:37:14.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:14.938+0000 7fadd0a00640 1 -- 172.21.15.42:0/4058512254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadcc149570 con 0x7fadcc06cea0 2024-07-04T17:37:14.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:14.939+0000 7fadd0a00640 1 --2- 172.21.15.42:0/4058512254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadcc06cea0 0x7fadcc148d80 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fadc0000c00 tx=0x7fadc00097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:14.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:14.939+0000 7fadc8c00640 1 -- 172.21.15.42:0/4058512254 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadc0038040 con 0x7fadcc06cea0 2024-07-04T17:37:14.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:14.939+0000 7fadc8c00640 1 -- 172.21.15.42:0/4058512254 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fadc0041070 con 0x7fadcc06cea0 2024-07-04T17:37:14.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:14.939+0000 7fadd3400640 1 -- 172.21.15.42:0/4058512254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadcc0708a0 con 0x7fadcc06cea0 2024-07-04T17:37:14.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:14.939+0000 7fadc8c00640 1 -- 172.21.15.42:0/4058512254 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadc0004420 con 0x7fadcc06cea0 2024-07-04T17:37:14.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:14.939+0000 7fadd3400640 1 -- 172.21.15.42:0/4058512254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadcc1497b0 con 0x7fadcc06cea0 2024-07-04T17:37:14.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:14.941+0000 7fadd3400640 1 -- 172.21.15.42:0/4058512254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadcc0b3030 con 0x7fadcc06cea0 2024-07-04T17:37:14.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:14.941+0000 7fadc8c00640 1 -- 172.21.15.42:0/4058512254 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 64) ==== 49615+0+0 (secure 0 0 0) 0x7fadc002b870 con 0x7fadcc06cea0 2024-07-04T17:37:14.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:14.941+0000 7fadc8c00640 1 --2- 172.21.15.42:0/4058512254 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fadac03d4f0 0x7fadac03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:14.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:14.941+0000 7fadc8c00640 1 -- 172.21.15.42:0/4058512254 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 3533+0+0 (secure 0 0 0) 0x7fadc0071d20 con 0x7fadcc06cea0 2024-07-04T17:37:14.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:14.942+0000 7fadcbe00640 1 --2- 172.21.15.42:0/4058512254 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fadac03d4f0 0x7fadac03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:14.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:14.943+0000 7fadcbe00640 1 --2- 172.21.15.42:0/4058512254 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fadac03d4f0 0x7fadac03fa00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fadb80015c0 tx=0x7fadb8007a90 comp rx=0 tx=0).ready entity=mgr.5988 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:14.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:14.947+0000 7fadc8c00640 1 -- 172.21.15.42:0/4058512254 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fadc0045db0 con 0x7fadcc06cea0 2024-07-04T17:37:15.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.158+0000 7fadd3400640 1 -- 172.21.15.42:0/4058512254 --> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fadcc1daa40 con 0x7fadac03d4f0 2024-07-04T17:37:15.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.159+0000 7fadc8c00640 1 -- 172.21.15.42:0/4058512254 <== mgr.5988 v2:172.21.15.42:6834/2161106357 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+9727 (secure 0 0 0) 0x7fadcc1daa40 con 0x7fadac03d4f0 2024-07-04T17:37:15.161 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:37:15.161 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:37:15.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.162+0000 7fadd3400640 1 -- 172.21.15.42:0/4058512254 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fadac03d4f0 msgr2=0x7fadac03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:15.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.162+0000 7fadd3400640 1 --2- 172.21.15.42:0/4058512254 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fadac03d4f0 0x7fadac03fa00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fadb80015c0 tx=0x7fadb8007a90 comp rx=0 tx=0).stop 2024-07-04T17:37:15.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.162+0000 7fadd3400640 1 -- 172.21.15.42:0/4058512254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadcc06cea0 msgr2=0x7fadcc148d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:15.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.162+0000 7fadd3400640 1 --2- 172.21.15.42:0/4058512254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadcc06cea0 0x7fadcc148d80 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fadc0000c00 tx=0x7fadc00097d0 comp rx=0 tx=0).stop 2024-07-04T17:37:15.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.162+0000 7fadd3400640 1 -- 172.21.15.42:0/4058512254 shutdown_connections 2024-07-04T17:37:15.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.162+0000 7fadd3400640 1 --2- 172.21.15.42:0/4058512254 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fadac03d4f0 0x7fadac03fa00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:15.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.162+0000 7fadd3400640 1 --2- 172.21.15.42:0/4058512254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadcc06cea0 0x7fadcc148d80 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:15.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.162+0000 7fadd3400640 1 -- 172.21.15.42:0/4058512254 >> 172.21.15.42:0/4058512254 conn(0x7fadcc0bf700 msgr2=0x7fadcc14a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:15.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.162+0000 7fadd3400640 1 -- 172.21.15.42:0/4058512254 shutdown_connections 2024-07-04T17:37:15.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.162+0000 7fadd3400640 1 -- 172.21.15.42:0/4058512254 wait complete. 2024-07-04T17:37:15.192 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2024-07-04T17:37:13.198237+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":716,"num_read_kb":1982,"num_write":211,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129,"ondisk_log_size":129,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":153,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":2,"kb":283115520,"kb_used":111800,"kb_used_data":5776,"kb_used_omap":4,"kb_used_meta":105979,"kb_avail":283003720,"statfs":{"total":289910292480,"available":289795809280,"internally_reserved":0,"allocated":5914624,"data_stored":3069582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":108522831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6803,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-63,"num_write_kb":-30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002023"},"pg_stats":[{"pgid":"1.0","version":"146'129","reported_seq":619,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-07-04T17:37:10.911980+0000","last_change":"2024-07-04T17:26:04.953559+0000","last_active":"2024-07-04T17:37:10.911980+0000","last_peered":"2024-07-04T17:37:10.911980+0000","last_clean":"2024-07-04T17:37:10.911980+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:37:10.911980+0000","last_undegraded":"2024-07-04T17:37:10.911980+0000","last_fullsized":"2024-07-04T17:37:10.911980+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_clean_scrub_stamp":"2024-07-04T17:26:03.933432+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.064181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":716,"num_read_kb":1982,"num_write":211,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":716,"num_read_kb":1982,"num_write":211,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129,"ondisk_log_size":129,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738498,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36880,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334960,"statfs":{"total":96636764160,"available":96598999040,"internally_reserved":0,"allocated":1576960,"data_stored":630936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738499,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37464,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334376,"statfs":{"total":96636764160,"available":96598401024,"internally_reserved":0,"allocated":2170880,"data_stored":1221304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738497,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37456,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334384,"statfs":{"total":96636764160,"available":96598409216,"internally_reserved":0,"allocated":2166784,"data_stored":1217342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:37:15.193 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:37:15.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.493+0000 7eff5e400640 1 -- 172.21.15.42:0/4215716432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff580b1760 msgr2=0x7eff580b8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:15.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.493+0000 7eff5e400640 1 --2- 172.21.15.42:0/4215716432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff580b1760 0x7eff580b8da0 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7eff480097b0 tx=0x7eff4802a4b0 comp rx=0 tx=0).stop 2024-07-04T17:37:15.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.493+0000 7eff5e400640 1 -- 172.21.15.42:0/4215716432 shutdown_connections 2024-07-04T17:37:15.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.493+0000 7eff5e400640 1 --2- 172.21.15.42:0/4215716432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff580b1760 0x7eff580b8da0 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:15.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.493+0000 7eff5e400640 1 -- 172.21.15.42:0/4215716432 >> 172.21.15.42:0/4215716432 conn(0x7eff580bebc0 msgr2=0x7eff580befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:15.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.493+0000 7eff5e400640 1 -- 172.21.15.42:0/4215716432 shutdown_connections 2024-07-04T17:37:15.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.493+0000 7eff5e400640 1 -- 172.21.15.42:0/4215716432 wait complete. 2024-07-04T17:37:15.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.495+0000 7eff5e400640 1 Processor -- start 2024-07-04T17:37:15.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.495+0000 7eff5e400640 1 -- start start 2024-07-04T17:37:15.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.497+0000 7eff5e400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff5806c020 0x7eff58072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:15.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.497+0000 7eff5e400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7eff5814c030 con 0x7eff5806c020 2024-07-04T17:37:15.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.497+0000 7eff5d000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff5806c020 0x7eff58072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:15.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.497+0000 7eff5d000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff5806c020 0x7eff58072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44054/0 (socket says 172.21.15.42:44054) 2024-07-04T17:37:15.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.497+0000 7eff5d000640 1 -- 172.21.15.42:0/1710513872 learned_addr learned my addr 172.21.15.42:0/1710513872 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:37:15.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.497+0000 7eff5d000640 1 -- 172.21.15.42:0/1710513872 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff58148e80 con 0x7eff5806c020 2024-07-04T17:37:15.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.498+0000 7eff5d000640 1 --2- 172.21.15.42:0/1710513872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff5806c020 0x7eff58072af0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7eff480098d0 tx=0x7eff48033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:15.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.498+0000 7eff55600640 1 -- 172.21.15.42:0/1710513872 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff48031bc0 con 0x7eff5806c020 2024-07-04T17:37:15.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.498+0000 7eff55600640 1 -- 172.21.15.42:0/1710513872 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff48031e50 con 0x7eff5806c020 2024-07-04T17:37:15.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.498+0000 7eff55600640 1 -- 172.21.15.42:0/1710513872 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff48033260 con 0x7eff5806c020 2024-07-04T17:37:15.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.498+0000 7eff5e400640 1 -- 172.21.15.42:0/1710513872 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff58149320 con 0x7eff5806c020 2024-07-04T17:37:15.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.498+0000 7eff5e400640 1 -- 172.21.15.42:0/1710513872 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff58142690 con 0x7eff5806c020 2024-07-04T17:37:15.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.499+0000 7eff55600640 1 -- 172.21.15.42:0/1710513872 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 64) ==== 49615+0+0 (secure 0 0 0) 0x7eff48033400 con 0x7eff5806c020 2024-07-04T17:37:15.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.499+0000 7eff5e400640 1 -- 172.21.15.42:0/1710513872 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff5814f320 con 0x7eff5806c020 2024-07-04T17:37:15.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.500+0000 7eff55600640 1 --2- 172.21.15.42:0/1710513872 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7eff3403d4f0 0x7eff3403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:15.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.500+0000 7eff55600640 1 -- 172.21.15.42:0/1710513872 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 3533+0+0 (secure 0 0 0) 0x7eff48072f10 con 0x7eff5806c020 2024-07-04T17:37:15.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.500+0000 7eff4fe00640 1 --2- 172.21.15.42:0/1710513872 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7eff3403d4f0 0x7eff3403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:15.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.501+0000 7eff4fe00640 1 --2- 172.21.15.42:0/1710513872 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7eff3403d4f0 0x7eff3403fa00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7eff400015c0 tx=0x7eff40007a90 comp rx=0 tx=0).ready entity=mgr.5988 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:15.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.505+0000 7eff55600640 1 -- 172.21.15.42:0/1710513872 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7eff4803f030 con 0x7eff5806c020 2024-07-04T17:37:15.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.714+0000 7eff5e400640 1 -- 172.21.15.42:0/1710513872 --> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7eff580b79d0 con 0x7eff3403d4f0 2024-07-04T17:37:15.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.715+0000 7eff55600640 1 -- 172.21.15.42:0/1710513872 <== mgr.5988 v2:172.21.15.42:6834/2161106357 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+9727 (secure 0 0 0) 0x7eff580b79d0 con 0x7eff3403d4f0 2024-07-04T17:37:15.716 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:37:15.716 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:37:15.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.718+0000 7eff5e400640 1 -- 172.21.15.42:0/1710513872 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7eff3403d4f0 msgr2=0x7eff3403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:15.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.718+0000 7eff5e400640 1 --2- 172.21.15.42:0/1710513872 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7eff3403d4f0 0x7eff3403fa00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7eff400015c0 tx=0x7eff40007a90 comp rx=0 tx=0).stop 2024-07-04T17:37:15.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.718+0000 7eff5e400640 1 -- 172.21.15.42:0/1710513872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff5806c020 msgr2=0x7eff58072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:15.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.718+0000 7eff5e400640 1 --2- 172.21.15.42:0/1710513872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff5806c020 0x7eff58072af0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7eff480098d0 tx=0x7eff48033e40 comp rx=0 tx=0).stop 2024-07-04T17:37:15.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.718+0000 7eff5e400640 1 -- 172.21.15.42:0/1710513872 shutdown_connections 2024-07-04T17:37:15.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.718+0000 7eff5e400640 1 --2- 172.21.15.42:0/1710513872 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7eff3403d4f0 0x7eff3403fa00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:15.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.718+0000 7eff5e400640 1 --2- 172.21.15.42:0/1710513872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff5806c020 0x7eff58072af0 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:15.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.718+0000 7eff5e400640 1 -- 172.21.15.42:0/1710513872 >> 172.21.15.42:0/1710513872 conn(0x7eff580bebc0 msgr2=0x7eff580b2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:15.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.718+0000 7eff5e400640 1 -- 172.21.15.42:0/1710513872 shutdown_connections 2024-07-04T17:37:15.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:15.718+0000 7eff5e400640 1 -- 172.21.15.42:0/1710513872 wait complete. 2024-07-04T17:37:15.747 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2024-07-04T17:37:15.198961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":716,"num_read_kb":1982,"num_write":211,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129,"ondisk_log_size":129,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":153,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":2,"kb":283115520,"kb_used":111800,"kb_used_data":5776,"kb_used_omap":4,"kb_used_meta":105979,"kb_avail":283003720,"statfs":{"total":289910292480,"available":289795809280,"internally_reserved":0,"allocated":5914624,"data_stored":3069582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":108522831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6803,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-63,"num_write_kb":-30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.317051"},"pg_stats":[{"pgid":"1.0","version":"146'129","reported_seq":619,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-07-04T17:37:10.911980+0000","last_change":"2024-07-04T17:26:04.953559+0000","last_active":"2024-07-04T17:37:10.911980+0000","last_peered":"2024-07-04T17:37:10.911980+0000","last_clean":"2024-07-04T17:37:10.911980+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:37:10.911980+0000","last_undegraded":"2024-07-04T17:37:10.911980+0000","last_fullsized":"2024-07-04T17:37:10.911980+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_clean_scrub_stamp":"2024-07-04T17:26:03.933432+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.064181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":716,"num_read_kb":1982,"num_write":211,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":716,"num_read_kb":1982,"num_write":211,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129,"ondisk_log_size":129,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738498,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36880,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334960,"statfs":{"total":96636764160,"available":96598999040,"internally_reserved":0,"allocated":1576960,"data_stored":630936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738499,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37464,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334376,"statfs":{"total":96636764160,"available":96598401024,"internally_reserved":0,"allocated":2170880,"data_stored":1221304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738497,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37456,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334384,"statfs":{"total":96636764160,"available":96598409216,"internally_reserved":0,"allocated":2166784,"data_stored":1217342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:37:15.747 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-07-04T17:37:15.748 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:37:16.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.046+0000 7fd5f9200640 1 -- 172.21.15.42:0/502686142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5f406d580 msgr2=0x7fd5f406c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:16.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.046+0000 7fd5f9200640 1 --2- 172.21.15.42:0/502686142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5f406d580 0x7fd5f406c2c0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7fd5e400a8c0 tx=0x7fd5e402b7d0 comp rx=0 tx=0).stop 2024-07-04T17:37:16.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.046+0000 7fd5f9200640 1 -- 172.21.15.42:0/502686142 shutdown_connections 2024-07-04T17:37:16.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.046+0000 7fd5f9200640 1 --2- 172.21.15.42:0/502686142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5f406d580 0x7fd5f406c2c0 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:16.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.046+0000 7fd5f9200640 1 -- 172.21.15.42:0/502686142 >> 172.21.15.42:0/502686142 conn(0x7fd5f40bebc0 msgr2=0x7fd5f40befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:16.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.046+0000 7fd5f9200640 1 -- 172.21.15.42:0/502686142 shutdown_connections 2024-07-04T17:37:16.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.046+0000 7fd5f9200640 1 -- 172.21.15.42:0/502686142 wait complete. 2024-07-04T17:37:16.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.049+0000 7fd5f9200640 1 Processor -- start 2024-07-04T17:37:16.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.049+0000 7fd5f9200640 1 -- start start 2024-07-04T17:37:16.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.050+0000 7fd5f9200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5f40bad70 0x7fd5f40af550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:16.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.050+0000 7fd5f9200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f40b2810 con 0x7fd5f40bad70 2024-07-04T17:37:16.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.050+0000 7fd5f3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5f40bad70 0x7fd5f40af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:16.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.051+0000 7fd5f3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5f40bad70 0x7fd5f40af550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44058/0 (socket says 172.21.15.42:44058) 2024-07-04T17:37:16.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.051+0000 7fd5f3400640 1 -- 172.21.15.42:0/2647304723 learned_addr learned my addr 172.21.15.42:0/2647304723 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:37:16.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.051+0000 7fd5f3400640 1 -- 172.21.15.42:0/2647304723 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f41e6770 con 0x7fd5f40bad70 2024-07-04T17:37:16.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.051+0000 7fd5f3400640 1 --2- 172.21.15.42:0/2647304723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5f40bad70 0x7fd5f40af550 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fd5e400a9e0 tx=0x7fd5e4003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:16.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.052+0000 7fd5eb400640 1 -- 172.21.15.42:0/2647304723 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5e4039040 con 0x7fd5f40bad70 2024-07-04T17:37:16.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.052+0000 7fd5eb400640 1 -- 172.21.15.42:0/2647304723 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5e4044070 con 0x7fd5f40bad70 2024-07-04T17:37:16.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.052+0000 7fd5eb400640 1 -- 172.21.15.42:0/2647304723 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5e4003e40 con 0x7fd5f40bad70 2024-07-04T17:37:16.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.052+0000 7fd5f9200640 1 -- 172.21.15.42:0/2647304723 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5f40b8b90 con 0x7fd5f40bad70 2024-07-04T17:37:16.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.052+0000 7fd5f9200640 1 -- 172.21.15.42:0/2647304723 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5f413cf30 con 0x7fd5f40bad70 2024-07-04T17:37:16.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.053+0000 7fd5f9200640 1 -- 172.21.15.42:0/2647304723 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5f406c2c0 con 0x7fd5f40bad70 2024-07-04T17:37:16.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.053+0000 7fd5eb400640 1 -- 172.21.15.42:0/2647304723 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 64) ==== 49615+0+0 (secure 0 0 0) 0x7fd5e4032070 con 0x7fd5f40bad70 2024-07-04T17:37:16.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.054+0000 7fd5eb400640 1 --2- 172.21.15.42:0/2647304723 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fd5d403d4f0 0x7fd5d403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:16.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.054+0000 7fd5eb400640 1 -- 172.21.15.42:0/2647304723 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 3533+0+0 (secure 0 0 0) 0x7fd5e40727a0 con 0x7fd5f40bad70 2024-07-04T17:37:16.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.054+0000 7fd5f2a00640 1 --2- 172.21.15.42:0/2647304723 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fd5d403d4f0 0x7fd5d403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:16.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.055+0000 7fd5f2a00640 1 --2- 172.21.15.42:0/2647304723 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fd5d403d4f0 0x7fd5d403fa00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0007fd0 tx=0x7fd5e0007920 comp rx=0 tx=0).ready entity=mgr.5988 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:16.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.059+0000 7fd5eb400640 1 -- 172.21.15.42:0/2647304723 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fd5e4031c30 con 0x7fd5f40bad70 2024-07-04T17:37:16.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.264+0000 7fd5f9200640 1 -- 172.21.15.42:0/2647304723 --> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd5f40b8ef0 con 0x7fd5d403d4f0 2024-07-04T17:37:16.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.265+0000 7fd5eb400640 1 -- 172.21.15.42:0/2647304723 <== mgr.5988 v2:172.21.15.42:6834/2161106357 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+9727 (secure 0 0 0) 0x7fd5f40b8ef0 con 0x7fd5d403d4f0 2024-07-04T17:37:16.266 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:37:16.267 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:37:16.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.268+0000 7fd5f9200640 1 -- 172.21.15.42:0/2647304723 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fd5d403d4f0 msgr2=0x7fd5d403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:16.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.268+0000 7fd5f9200640 1 --2- 172.21.15.42:0/2647304723 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fd5d403d4f0 0x7fd5d403fa00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0007fd0 tx=0x7fd5e0007920 comp rx=0 tx=0).stop 2024-07-04T17:37:16.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.268+0000 7fd5f9200640 1 -- 172.21.15.42:0/2647304723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5f40bad70 msgr2=0x7fd5f40af550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:16.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.268+0000 7fd5f9200640 1 --2- 172.21.15.42:0/2647304723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5f40bad70 0x7fd5f40af550 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fd5e400a9e0 tx=0x7fd5e4003b90 comp rx=0 tx=0).stop 2024-07-04T17:37:16.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.268+0000 7fd5f9200640 1 -- 172.21.15.42:0/2647304723 shutdown_connections 2024-07-04T17:37:16.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.268+0000 7fd5f9200640 1 --2- 172.21.15.42:0/2647304723 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fd5d403d4f0 0x7fd5d403fa00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:16.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.268+0000 7fd5f9200640 1 --2- 172.21.15.42:0/2647304723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5f40bad70 0x7fd5f40af550 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:16.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.268+0000 7fd5f9200640 1 -- 172.21.15.42:0/2647304723 >> 172.21.15.42:0/2647304723 conn(0x7fd5f40bebc0 msgr2=0x7fd5f4141770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:16.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.268+0000 7fd5f9200640 1 -- 172.21.15.42:0/2647304723 shutdown_connections 2024-07-04T17:37:16.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.268+0000 7fd5f9200640 1 -- 172.21.15.42:0/2647304723 wait complete. 2024-07-04T17:37:16.295 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2024-07-04T17:37:15.198961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":716,"num_read_kb":1982,"num_write":211,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129,"ondisk_log_size":129,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":153,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":2,"kb":283115520,"kb_used":111800,"kb_used_data":5776,"kb_used_omap":4,"kb_used_meta":105979,"kb_avail":283003720,"statfs":{"total":289910292480,"available":289795809280,"internally_reserved":0,"allocated":5914624,"data_stored":3069582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":108522831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6803,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-63,"num_write_kb":-30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.317051"},"pg_stats":[{"pgid":"1.0","version":"146'129","reported_seq":619,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-07-04T17:37:10.911980+0000","last_change":"2024-07-04T17:26:04.953559+0000","last_active":"2024-07-04T17:37:10.911980+0000","last_peered":"2024-07-04T17:37:10.911980+0000","last_clean":"2024-07-04T17:37:10.911980+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:37:10.911980+0000","last_undegraded":"2024-07-04T17:37:10.911980+0000","last_fullsized":"2024-07-04T17:37:10.911980+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_clean_scrub_stamp":"2024-07-04T17:26:03.933432+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.064181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":716,"num_read_kb":1982,"num_write":211,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":716,"num_read_kb":1982,"num_write":211,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129,"ondisk_log_size":129,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738498,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36880,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334960,"statfs":{"total":96636764160,"available":96598999040,"internally_reserved":0,"allocated":1576960,"data_stored":630936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738499,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37464,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334376,"statfs":{"total":96636764160,"available":96598401024,"internally_reserved":0,"allocated":2170880,"data_stored":1221304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738497,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37456,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334384,"statfs":{"total":96636764160,"available":96598409216,"internally_reserved":0,"allocated":2166784,"data_stored":1217342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:37:16.296 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T17:37:16.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.590+0000 7f172b800640 1 -- 172.21.15.42:0/3454479630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17240b2d20 msgr2=0x7f17240bb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:16.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.590+0000 7f172b800640 1 --2- 172.21.15.42:0/3454479630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17240b2d20 0x7f17240bb1f0 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f170c0097b0 tx=0x7f170c02a4b0 comp rx=0 tx=0).stop 2024-07-04T17:37:16.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.590+0000 7f172b800640 1 -- 172.21.15.42:0/3454479630 shutdown_connections 2024-07-04T17:37:16.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.590+0000 7f172b800640 1 --2- 172.21.15.42:0/3454479630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17240b2d20 0x7f17240bb1f0 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:16.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.590+0000 7f172b800640 1 -- 172.21.15.42:0/3454479630 >> 172.21.15.42:0/3454479630 conn(0x7f17240bf700 msgr2=0x7f17240af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:16.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.590+0000 7f172b800640 1 -- 172.21.15.42:0/3454479630 shutdown_connections 2024-07-04T17:37:16.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.590+0000 7f172b800640 1 -- 172.21.15.42:0/3454479630 wait complete. 2024-07-04T17:37:16.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.592+0000 7f172b800640 1 Processor -- start 2024-07-04T17:37:16.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.592+0000 7f172b800640 1 -- start start 2024-07-04T17:37:16.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.594+0000 7f172b800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17240b2d20 0x7f172413c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:16.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.594+0000 7f172b800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f172406c020 con 0x7f17240b2d20 2024-07-04T17:37:16.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.594+0000 7f1728e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17240b2d20 0x7f172413c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:16.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.594+0000 7f1728e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17240b2d20 0x7f172413c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44078/0 (socket says 172.21.15.42:44078) 2024-07-04T17:37:16.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.594+0000 7f1728e00640 1 -- 172.21.15.42:0/1235905209 learned_addr learned my addr 172.21.15.42:0/1235905209 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:37:16.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.594+0000 7f1728e00640 1 -- 172.21.15.42:0/1235905209 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17241e4050 con 0x7f17240b2d20 2024-07-04T17:37:16.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.595+0000 7f1728e00640 1 --2- 172.21.15.42:0/1235905209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17240b2d20 0x7f172413c530 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f171400c380 tx=0x7f171400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:16.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.595+0000 7f1720c00640 1 -- 172.21.15.42:0/1235905209 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17140116c0 con 0x7f17240b2d20 2024-07-04T17:37:16.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.595+0000 7f1720c00640 1 -- 172.21.15.42:0/1235905209 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1714017070 con 0x7f17240b2d20 2024-07-04T17:37:16.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.595+0000 7f172b800640 1 -- 172.21.15.42:0/1235905209 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f172413ca70 con 0x7f17240b2d20 2024-07-04T17:37:16.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.595+0000 7f1720c00640 1 -- 172.21.15.42:0/1235905209 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1714011b20 con 0x7f17240b2d20 2024-07-04T17:37:16.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.595+0000 7f172b800640 1 -- 172.21.15.42:0/1235905209 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f172413e600 con 0x7f17240b2d20 2024-07-04T17:37:16.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.597+0000 7f1720c00640 1 -- 172.21.15.42:0/1235905209 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 64) ==== 49615+0+0 (secure 0 0 0) 0x7f1714011cc0 con 0x7f17240b2d20 2024-07-04T17:37:16.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.597+0000 7f172b800640 1 -- 172.21.15.42:0/1235905209 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f172413e7d0 con 0x7f17240b2d20 2024-07-04T17:37:16.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.597+0000 7f1720c00640 1 --2- 172.21.15.42:0/1235905209 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f170003d4f0 0x7f170003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:16.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.597+0000 7f1720c00640 1 -- 172.21.15.42:0/1235905209 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 3533+0+0 (secure 0 0 0) 0x7f1714051710 con 0x7f17240b2d20 2024-07-04T17:37:16.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.597+0000 7f1723e00640 1 --2- 172.21.15.42:0/1235905209 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f170003d4f0 0x7f170003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:16.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.598+0000 7f1723e00640 1 --2- 172.21.15.42:0/1235905209 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f170003d4f0 0x7f170003fa00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f170c034000 tx=0x7f170c004730 comp rx=0 tx=0).ready entity=mgr.5988 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:16.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.603+0000 7f1720c00640 1 -- 172.21.15.42:0/1235905209 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f172413e7d0 con 0x7f17240b2d20 2024-07-04T17:37:16.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.818+0000 7f172b800640 1 -- 172.21.15.42:0/1235905209 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f172406c710 con 0x7f17240b2d20 2024-07-04T17:37:16.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.820+0000 7f1720c00640 1 -- 172.21.15.42:0/1235905209 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) ==== 74+0+8159 (secure 0 0 0) 0x7f171400a140 con 0x7f17240b2d20 2024-07-04T17:37:16.821 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:37:16.821 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":151,"fsid":"be4f300a-93ef-4489-b540-37ece39b0aac","created":"2024-07-04T17:25:51.056303+0000","modified":"2024-07-04T17:37:11.888370+0000","last_up_change":"2024-07-04T17:26:02.931381+0000","last_in_change":"2024-07-04T17:25:54.262511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T17:26:03.297630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7868bbeb-8129-4123-bc11-d96d23f65a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6801","nonce":584646941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6803","nonce":584646941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6807","nonce":584646941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":584646941},{"type":"v1","addr":"172.21.15.42:6805","nonce":584646941}]},"public_addr":"172.21.15.42:6801/584646941","cluster_addr":"172.21.15.42:6803/584646941","heartbeat_back_addr":"172.21.15.42:6807/584646941","heartbeat_front_addr":"172.21.15.42:6805/584646941","state":["exists","up"]},{"osd":1,"uuid":"8d5148e1-e5d7-4bca-abef-00e5b1f9e35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6817","nonce":3110347191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6819","nonce":3110347191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6823","nonce":3110347191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3110347191},{"type":"v1","addr":"172.21.15.42:6821","nonce":3110347191}]},"public_addr":"172.21.15.42:6817/3110347191","cluster_addr":"172.21.15.42:6819/3110347191","heartbeat_back_addr":"172.21.15.42:6823/3110347191","heartbeat_front_addr":"172.21.15.42:6821/3110347191","state":["exists","up"]},{"osd":2,"uuid":"c24efcdf-9c7d-46d6-a323-76fb5250d5fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6809","nonce":2100781882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6811","nonce":2100781882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6815","nonce":2100781882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2100781882},{"type":"v1","addr":"172.21.15.42:6813","nonce":2100781882}]},"public_addr":"172.21.15.42:6809/2100781882","cluster_addr":"172.21.15.42:6811/2100781882","heartbeat_back_addr":"172.21.15.42:6815/2100781882","heartbeat_front_addr":"172.21.15.42:6813/2100781882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.816520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.947008+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T17:25:59.875795+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6831/4064998265":"2024-07-05T17:37:08.876030+0000","172.21.15.42:6832/1863956336":"2024-07-05T17:37:02.377167+0000","172.21.15.42:6833/1863956336":"2024-07-05T17:37:02.377167+0000","172.21.15.42:0/1132989435":"2024-07-05T17:37:01.159615+0000","172.21.15.42:0/3471044288":"2024-07-05T17:37:01.159615+0000","172.21.15.42:6834/3991875201":"2024-07-05T17:37:01.159615+0000","172.21.15.42:0/4263654752":"2024-07-05T17:37:01.159615+0000","172.21.15.42:0/2937116305":"2024-07-04T18:36:04.169055+0000","172.21.15.42:0/3482046390":"2024-07-05T17:35:07.918359+0000","172.21.15.42:6835/3991875201":"2024-07-05T17:37:01.159615+0000","172.21.15.42:0/1404855452":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/2049026251":"2024-07-05T17:37:01.159615+0000","172.21.15.42:0/1956174992":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/3978634668":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/3294509689":"2024-07-05T17:35:07.918359+0000","172.21.15.42:6835/4064094064":"2024-07-05T17:35:07.918359+0000","172.21.15.42:6830/4064998265":"2024-07-05T17:37:08.876030+0000","172.21.15.42:6834/4064094064":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/528150101":"2024-07-05T17:37:01.159615+0000","172.21.15.42:0/2180653394":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/2328963399":"2024-07-05T17:35:07.918359+0000","172.21.15.42:0/644737563":"2024-07-05T17:35:07.918359+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T17:37:16.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.823+0000 7f172b800640 1 -- 172.21.15.42:0/1235905209 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f170003d4f0 msgr2=0x7f170003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:16.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.823+0000 7f172b800640 1 --2- 172.21.15.42:0/1235905209 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f170003d4f0 0x7f170003fa00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f170c034000 tx=0x7f170c004730 comp rx=0 tx=0).stop 2024-07-04T17:37:16.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.823+0000 7f172b800640 1 -- 172.21.15.42:0/1235905209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17240b2d20 msgr2=0x7f172413c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:16.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.823+0000 7f172b800640 1 --2- 172.21.15.42:0/1235905209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17240b2d20 0x7f172413c530 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f171400c380 tx=0x7f171400c840 comp rx=0 tx=0).stop 2024-07-04T17:37:16.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.823+0000 7f172b800640 1 -- 172.21.15.42:0/1235905209 shutdown_connections 2024-07-04T17:37:16.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.823+0000 7f172b800640 1 --2- 172.21.15.42:0/1235905209 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f170003d4f0 0x7f170003fa00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:16.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.823+0000 7f172b800640 1 --2- 172.21.15.42:0/1235905209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17240b2d20 0x7f172413c530 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:16.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.823+0000 7f172b800640 1 -- 172.21.15.42:0/1235905209 >> 172.21.15.42:0/1235905209 conn(0x7f17240bf700 msgr2=0x7f1724075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:16.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.824+0000 7f172b800640 1 -- 172.21.15.42:0/1235905209 shutdown_connections 2024-07-04T17:37:16.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:16.824+0000 7f172b800640 1 -- 172.21.15.42:0/1235905209 wait complete. 2024-07-04T17:37:17.851 INFO:tasks.ceph:Scrubbing osd.0 2024-07-04T17:37:17.852 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-07-04T17:37:18.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.161+0000 7f0afd800640 1 -- 172.21.15.42:0/2678222817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0af80b1100 msgr2=0x7f0af813f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:18.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.161+0000 7f0afd800640 1 --2- 172.21.15.42:0/2678222817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0af80b1100 0x7f0af813f800 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8009fd0 tx=0x7f0ae802bae0 comp rx=0 tx=0).stop 2024-07-04T17:37:18.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.161+0000 7f0afd800640 1 -- 172.21.15.42:0/2678222817 shutdown_connections 2024-07-04T17:37:18.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.161+0000 7f0afd800640 1 --2- 172.21.15.42:0/2678222817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0af80b1100 0x7f0af813f800 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:18.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.161+0000 7f0afd800640 1 -- 172.21.15.42:0/2678222817 >> 172.21.15.42:0/2678222817 conn(0x7f0af806c020 msgr2=0x7f0af806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:18.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.161+0000 7f0afd800640 1 -- 172.21.15.42:0/2678222817 shutdown_connections 2024-07-04T17:37:18.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.161+0000 7f0afd800640 1 -- 172.21.15.42:0/2678222817 wait complete. 2024-07-04T17:37:18.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.163+0000 7f0afd800640 1 Processor -- start 2024-07-04T17:37:18.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.164+0000 7f0afd800640 1 -- start start 2024-07-04T17:37:18.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.165+0000 7f0afd800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0af80a94d0 0x7f0af8134190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:18.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.165+0000 7f0afd800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0af81401e0 con 0x7f0af80a94d0 2024-07-04T17:37:18.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.165+0000 7f0af6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0af80a94d0 0x7f0af8134190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:18.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.165+0000 7f0af6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0af80a94d0 0x7f0af8134190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44106/0 (socket says 172.21.15.42:44106) 2024-07-04T17:37:18.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.165+0000 7f0af6a00640 1 -- 172.21.15.42:0/81956290 learned_addr learned my addr 172.21.15.42:0/81956290 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:37:18.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.166+0000 7f0af6a00640 1 -- 172.21.15.42:0/81956290 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af8135770 con 0x7f0af80a94d0 2024-07-04T17:37:18.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.166+0000 7f0af6a00640 1 --2- 172.21.15.42:0/81956290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0af80a94d0 0x7f0af8134190 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0001eb0 tx=0x7f0ae000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:18.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.166+0000 7f0aeea00640 1 -- 172.21.15.42:0/81956290 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ae00116c0 con 0x7f0af80a94d0 2024-07-04T17:37:18.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.166+0000 7f0aeea00640 1 -- 172.21.15.42:0/81956290 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ae0017070 con 0x7f0af80a94d0 2024-07-04T17:37:18.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.166+0000 7f0aeea00640 1 -- 172.21.15.42:0/81956290 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ae0011ae0 con 0x7f0af80a94d0 2024-07-04T17:37:18.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.166+0000 7f0afd800640 1 -- 172.21.15.42:0/81956290 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0af81e23e0 con 0x7f0af80a94d0 2024-07-04T17:37:18.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.166+0000 7f0afd800640 1 -- 172.21.15.42:0/81956290 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0af8135e50 con 0x7f0af80a94d0 2024-07-04T17:37:18.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.167+0000 7f0afd800640 1 -- 172.21.15.42:0/81956290 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0af813f800 con 0x7f0af80a94d0 2024-07-04T17:37:18.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.168+0000 7f0aeea00640 1 -- 172.21.15.42:0/81956290 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 64) ==== 49615+0+0 (secure 0 0 0) 0x7f0ae0011c80 con 0x7f0af80a94d0 2024-07-04T17:37:18.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.168+0000 7f0aeea00640 1 --2- 172.21.15.42:0/81956290 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f0ad403d4f0 0x7f0ad403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:18.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.168+0000 7f0aeea00640 1 -- 172.21.15.42:0/81956290 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 3533+0+0 (secure 0 0 0) 0x7f0ae0051610 con 0x7f0af80a94d0 2024-07-04T17:37:18.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.169+0000 7f0aeea00640 1 --2- 172.21.15.42:0/81956290 >> [v2:172.21.15.42:6800/584646941,v1:172.21.15.42:6801/584646941] conn(0x7f0ad4042830 0x7f0ad4044cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:18.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.169+0000 7f0aeea00640 1 -- 172.21.15.42:0/81956290 --> [v2:172.21.15.42:6800/584646941,v1:172.21.15.42:6801/584646941] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0ae0018ec0 con 0x7f0ad4042830 2024-07-04T17:37:18.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.169+0000 7f0aeea00640 1 -- 172.21.15.42:0/81956290 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) ==== 24+0+0 (secure 0 0 0) 0x7f0ae0058050 con 0x7f0af80a94d0 2024-07-04T17:37:18.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.169+0000 7f0af6000640 1 --2- 172.21.15.42:0/81956290 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f0ad403d4f0 0x7f0ad403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:18.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.169+0000 7f0af7400640 1 --2- 172.21.15.42:0/81956290 >> [v2:172.21.15.42:6800/584646941,v1:172.21.15.42:6801/584646941] conn(0x7f0ad4042830 0x7f0ad4044cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:18.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.170+0000 7f0af6000640 1 --2- 172.21.15.42:0/81956290 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f0ad403d4f0 0x7f0ad403fa00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8035000 tx=0x7f0ae8004ab0 comp rx=0 tx=0).ready entity=mgr.5988 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:18.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.170+0000 7f0af7400640 1 --2- 172.21.15.42:0/81956290 >> [v2:172.21.15.42:6800/584646941,v1:172.21.15.42:6801/584646941] conn(0x7f0ad4042830 0x7f0ad4044cc0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:18.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.171+0000 7f0aeea00640 1 -- 172.21.15.42:0/81956290 <== osd.0 v2:172.21.15.42:6800/584646941 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0ae0018ec0 con 0x7f0ad4042830 2024-07-04T17:37:18.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.186+0000 7f0afd800640 1 -- 172.21.15.42:0/81956290 --> [v2:172.21.15.42:6800/584646941,v1:172.21.15.42:6801/584646941] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0af80b1100 con 0x7f0ad4042830 2024-07-04T17:37:18.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.186+0000 7f0aeea00640 1 -- 172.21.15.42:0/81956290 <== osd.0 v2:172.21.15.42:6800/584646941 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0af80b1100 con 0x7f0ad4042830 2024-07-04T17:37:18.187 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:37:18.187 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-07-04T17:37:18.187 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:37:18.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.186+0000 7f0afd800640 1 -- 172.21.15.42:0/81956290 >> [v2:172.21.15.42:6800/584646941,v1:172.21.15.42:6801/584646941] conn(0x7f0ad4042830 msgr2=0x7f0ad4044cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:18.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.186+0000 7f0afd800640 1 --2- 172.21.15.42:0/81956290 >> [v2:172.21.15.42:6800/584646941,v1:172.21.15.42:6801/584646941] conn(0x7f0ad4042830 0x7f0ad4044cc0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:18.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.187+0000 7f0afd800640 1 -- 172.21.15.42:0/81956290 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f0ad403d4f0 msgr2=0x7f0ad403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:18.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.187+0000 7f0afd800640 1 --2- 172.21.15.42:0/81956290 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f0ad403d4f0 0x7f0ad403fa00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8035000 tx=0x7f0ae8004ab0 comp rx=0 tx=0).stop 2024-07-04T17:37:18.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.187+0000 7f0afd800640 1 -- 172.21.15.42:0/81956290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0af80a94d0 msgr2=0x7f0af8134190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:18.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.187+0000 7f0afd800640 1 --2- 172.21.15.42:0/81956290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0af80a94d0 0x7f0af8134190 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0001eb0 tx=0x7f0ae000c810 comp rx=0 tx=0).stop 2024-07-04T17:37:18.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.187+0000 7f0afd800640 1 -- 172.21.15.42:0/81956290 shutdown_connections 2024-07-04T17:37:18.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.187+0000 7f0afd800640 1 --2- 172.21.15.42:0/81956290 >> [v2:172.21.15.42:6800/584646941,v1:172.21.15.42:6801/584646941] conn(0x7f0ad4042830 0x7f0ad4044cc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:18.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.187+0000 7f0afd800640 1 --2- 172.21.15.42:0/81956290 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f0ad403d4f0 0x7f0ad403fa00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:18.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.187+0000 7f0afd800640 1 --2- 172.21.15.42:0/81956290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0af80a94d0 0x7f0af8134190 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:18.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.187+0000 7f0afd800640 1 -- 172.21.15.42:0/81956290 >> 172.21.15.42:0/81956290 conn(0x7f0af806c020 msgr2=0x7f0af81d7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:18.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.187+0000 7f0afd800640 1 -- 172.21.15.42:0/81956290 shutdown_connections 2024-07-04T17:37:18.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.187+0000 7f0afd800640 1 -- 172.21.15.42:0/81956290 wait complete. 2024-07-04T17:37:18.213 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-07-04T17:37:18.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.505+0000 7f2a2a000640 1 -- 172.21.15.42:0/1393895643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a2406cea0 msgr2=0x7f2a240b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:18.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.505+0000 7f2a2a000640 1 --2- 172.21.15.42:0/1393895643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a2406cea0 0x7f2a240b3030 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f2a100097b0 tx=0x7f2a1002a530 comp rx=0 tx=0).stop 2024-07-04T17:37:18.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.505+0000 7f2a2a000640 1 -- 172.21.15.42:0/1393895643 shutdown_connections 2024-07-04T17:37:18.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.505+0000 7f2a2a000640 1 --2- 172.21.15.42:0/1393895643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a2406cea0 0x7f2a240b3030 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:18.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.505+0000 7f2a2a000640 1 -- 172.21.15.42:0/1393895643 >> 172.21.15.42:0/1393895643 conn(0x7f2a240bf700 msgr2=0x7f2a240af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:18.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.506+0000 7f2a2a000640 1 -- 172.21.15.42:0/1393895643 shutdown_connections 2024-07-04T17:37:18.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.506+0000 7f2a2a000640 1 -- 172.21.15.42:0/1393895643 wait complete. 2024-07-04T17:37:18.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.508+0000 7f2a2a000640 1 Processor -- start 2024-07-04T17:37:18.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.508+0000 7f2a2a000640 1 -- start start 2024-07-04T17:37:18.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.509+0000 7f2a2a000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a2406cea0 0x7f2a241d5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:18.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.509+0000 7f2a2a000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2a240b0580 con 0x7f2a2406cea0 2024-07-04T17:37:18.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.510+0000 7f2a23400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a2406cea0 0x7f2a241d5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:18.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.510+0000 7f2a23400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a2406cea0 0x7f2a241d5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44124/0 (socket says 172.21.15.42:44124) 2024-07-04T17:37:18.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.510+0000 7f2a23400640 1 -- 172.21.15.42:0/1936248740 learned_addr learned my addr 172.21.15.42:0/1936248740 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:37:18.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.510+0000 7f2a23400640 1 -- 172.21.15.42:0/1936248740 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a241e34a0 con 0x7f2a2406cea0 2024-07-04T17:37:18.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.511+0000 7f2a23400640 1 --2- 172.21.15.42:0/1936248740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a2406cea0 0x7f2a241d5c50 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400c3b0 tx=0x7f2a1400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:18.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.511+0000 7f2a1b400640 1 -- 172.21.15.42:0/1936248740 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a14011780 con 0x7f2a2406cea0 2024-07-04T17:37:18.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.511+0000 7f2a1b400640 1 -- 172.21.15.42:0/1936248740 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a14017070 con 0x7f2a2406cea0 2024-07-04T17:37:18.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.511+0000 7f2a1b400640 1 -- 172.21.15.42:0/1936248740 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a14011c00 con 0x7f2a2406cea0 2024-07-04T17:37:18.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.511+0000 7f2a2a000640 1 -- 172.21.15.42:0/1936248740 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a241e3690 con 0x7f2a2406cea0 2024-07-04T17:37:18.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.511+0000 7f2a2a000640 1 -- 172.21.15.42:0/1936248740 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a241498e0 con 0x7f2a2406cea0 2024-07-04T17:37:18.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.512+0000 7f2a2a000640 1 -- 172.21.15.42:0/1936248740 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a241d6430 con 0x7f2a2406cea0 2024-07-04T17:37:18.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.512+0000 7f2a1b400640 1 -- 172.21.15.42:0/1936248740 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 64) ==== 49615+0+0 (secure 0 0 0) 0x7f2a1401c460 con 0x7f2a2406cea0 2024-07-04T17:37:18.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.513+0000 7f2a1b400640 1 --2- 172.21.15.42:0/1936248740 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f2a0403d4f0 0x7f2a0403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:18.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.513+0000 7f2a1b400640 1 -- 172.21.15.42:0/1936248740 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 3533+0+0 (secure 0 0 0) 0x7f2a14051640 con 0x7f2a2406cea0 2024-07-04T17:37:18.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.513+0000 7f2a22a00640 1 --2- 172.21.15.42:0/1936248740 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f2a0403d4f0 0x7f2a0403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:18.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.514+0000 7f2a22a00640 1 --2- 172.21.15.42:0/1936248740 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f2a0403d4f0 0x7f2a0403fa00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2a10034000 tx=0x7f2a10004730 comp rx=0 tx=0).ready entity=mgr.5988 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:18.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.518+0000 7f2a1b400640 1 -- 172.21.15.42:0/1936248740 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f2a241d6430 con 0x7f2a2406cea0 2024-07-04T17:37:18.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.734+0000 7f2a2a000640 1 -- 172.21.15.42:0/1936248740 --> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f2a24149bd0 con 0x7f2a0403d4f0 2024-07-04T17:37:18.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.735+0000 7f2a1b400640 1 -- 172.21.15.42:0/1936248740 <== mgr.5988 v2:172.21.15.42:6834/2161106357 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2a24149bd0 con 0x7f2a0403d4f0 2024-07-04T17:37:18.735 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 0 to deep-scrub 2024-07-04T17:37:18.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.737+0000 7f2a2a000640 1 -- 172.21.15.42:0/1936248740 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f2a0403d4f0 msgr2=0x7f2a0403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:18.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.738+0000 7f2a2a000640 1 --2- 172.21.15.42:0/1936248740 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f2a0403d4f0 0x7f2a0403fa00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2a10034000 tx=0x7f2a10004730 comp rx=0 tx=0).stop 2024-07-04T17:37:18.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.738+0000 7f2a2a000640 1 -- 172.21.15.42:0/1936248740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a2406cea0 msgr2=0x7f2a241d5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:18.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.738+0000 7f2a2a000640 1 --2- 172.21.15.42:0/1936248740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a2406cea0 0x7f2a241d5c50 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400c3b0 tx=0x7f2a1400c870 comp rx=0 tx=0).stop 2024-07-04T17:37:18.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.738+0000 7f2a2a000640 1 -- 172.21.15.42:0/1936248740 shutdown_connections 2024-07-04T17:37:18.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.738+0000 7f2a2a000640 1 --2- 172.21.15.42:0/1936248740 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f2a0403d4f0 0x7f2a0403fa00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:18.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.738+0000 7f2a2a000640 1 --2- 172.21.15.42:0/1936248740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a2406cea0 0x7f2a241d5c50 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:18.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.738+0000 7f2a2a000640 1 -- 172.21.15.42:0/1936248740 >> 172.21.15.42:0/1936248740 conn(0x7f2a240bf700 msgr2=0x7f2a240bb740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:18.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.738+0000 7f2a2a000640 1 -- 172.21.15.42:0/1936248740 shutdown_connections 2024-07-04T17:37:18.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:18.738+0000 7f2a2a000640 1 -- 172.21.15.42:0/1936248740 wait complete. 2024-07-04T17:37:18.766 INFO:tasks.ceph:Scrubbing osd.1 2024-07-04T17:37:18.766 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-07-04T17:37:19.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.065+0000 7f3623e00640 1 -- 172.21.15.42:0/2190198662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36240b0db0 msgr2=0x7f36240abc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:19.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.065+0000 7f3623e00640 1 --2- 172.21.15.42:0/2190198662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36240b0db0 0x7f36240abc80 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f360c009230 tx=0x7f360c02a3f0 comp rx=0 tx=0).stop 2024-07-04T17:37:19.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.065+0000 7f3623e00640 1 -- 172.21.15.42:0/2190198662 shutdown_connections 2024-07-04T17:37:19.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.065+0000 7f3623e00640 1 --2- 172.21.15.42:0/2190198662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36240b0db0 0x7f36240abc80 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:19.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.065+0000 7f3623e00640 1 -- 172.21.15.42:0/2190198662 >> 172.21.15.42:0/2190198662 conn(0x7f36240afaa0 msgr2=0x7f36240afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:19.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.065+0000 7f3623e00640 1 -- 172.21.15.42:0/2190198662 shutdown_connections 2024-07-04T17:37:19.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.066+0000 7f3623e00640 1 -- 172.21.15.42:0/2190198662 wait complete. 2024-07-04T17:37:19.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.068+0000 7f3623e00640 1 Processor -- start 2024-07-04T17:37:19.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.068+0000 7f3623e00640 1 -- start start 2024-07-04T17:37:19.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.069+0000 7f3623e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36240ac5b0 0x7f36241d2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:19.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.069+0000 7f3623e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f36240ae590 con 0x7f36240ac5b0 2024-07-04T17:37:19.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.069+0000 7f3622a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36240ac5b0 0x7f36241d2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:19.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.070+0000 7f3622a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36240ac5b0 0x7f36241d2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44140/0 (socket says 172.21.15.42:44140) 2024-07-04T17:37:19.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.070+0000 7f3622a00640 1 -- 172.21.15.42:0/869354448 learned_addr learned my addr 172.21.15.42:0/869354448 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:37:19.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.070+0000 7f3622a00640 1 -- 172.21.15.42:0/869354448 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36241d4e70 con 0x7f36240ac5b0 2024-07-04T17:37:19.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.070+0000 7f3622a00640 1 --2- 172.21.15.42:0/869354448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36240ac5b0 0x7f36241d2f80 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f361400c3b0 tx=0x7f361400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:19.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.071+0000 7f361b400640 1 -- 172.21.15.42:0/869354448 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3614011780 con 0x7f36240ac5b0 2024-07-04T17:37:19.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.071+0000 7f361b400640 1 -- 172.21.15.42:0/869354448 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3614017070 con 0x7f36240ac5b0 2024-07-04T17:37:19.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.071+0000 7f361b400640 1 -- 172.21.15.42:0/869354448 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3614011c00 con 0x7f36240ac5b0 2024-07-04T17:37:19.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.071+0000 7f3623e00640 1 -- 172.21.15.42:0/869354448 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3624139c00 con 0x7f36240ac5b0 2024-07-04T17:37:19.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.071+0000 7f3623e00640 1 -- 172.21.15.42:0/869354448 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36241e4230 con 0x7f36240ac5b0 2024-07-04T17:37:19.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.072+0000 7f3623e00640 1 -- 172.21.15.42:0/869354448 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f36240abc80 con 0x7f36240ac5b0 2024-07-04T17:37:19.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.072+0000 7f361b400640 1 -- 172.21.15.42:0/869354448 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 64) ==== 49615+0+0 (secure 0 0 0) 0x7f361401c460 con 0x7f36240ac5b0 2024-07-04T17:37:19.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.073+0000 7f361b400640 1 --2- 172.21.15.42:0/869354448 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f360003d4f0 0x7f360003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:19.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.073+0000 7f361b400640 1 -- 172.21.15.42:0/869354448 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 3533+0+0 (secure 0 0 0) 0x7f36140515c0 con 0x7f36240ac5b0 2024-07-04T17:37:19.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.073+0000 7f361b400640 1 --2- 172.21.15.42:0/869354448 >> [v2:172.21.15.42:6816/3110347191,v1:172.21.15.42:6817/3110347191] conn(0x7f3600042830 0x7f3600044cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:19.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.073+0000 7f361b400640 1 -- 172.21.15.42:0/869354448 --> [v2:172.21.15.42:6816/3110347191,v1:172.21.15.42:6817/3110347191] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f361401fe80 con 0x7f3600042830 2024-07-04T17:37:19.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.073+0000 7f361b400640 1 -- 172.21.15.42:0/869354448 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) ==== 24+0+0 (secure 0 0 0) 0x7f3614058050 con 0x7f36240ac5b0 2024-07-04T17:37:19.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.073+0000 7f361be00640 1 --2- 172.21.15.42:0/869354448 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f360003d4f0 0x7f360003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:19.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.073+0000 7f3623400640 1 --2- 172.21.15.42:0/869354448 >> [v2:172.21.15.42:6816/3110347191,v1:172.21.15.42:6817/3110347191] conn(0x7f3600042830 0x7f3600044cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:19.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.074+0000 7f361be00640 1 --2- 172.21.15.42:0/869354448 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f360003d4f0 0x7f360003fa00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f360c034000 tx=0x7f360c004730 comp rx=0 tx=0).ready entity=mgr.5988 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:19.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.074+0000 7f3623400640 1 --2- 172.21.15.42:0/869354448 >> [v2:172.21.15.42:6816/3110347191,v1:172.21.15.42:6817/3110347191] conn(0x7f3600042830 0x7f3600044cc0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:19.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.075+0000 7f361b400640 1 -- 172.21.15.42:0/869354448 <== osd.1 v2:172.21.15.42:6816/3110347191 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f361401fe80 con 0x7f3600042830 2024-07-04T17:37:19.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.091+0000 7f3623e00640 1 -- 172.21.15.42:0/869354448 --> [v2:172.21.15.42:6816/3110347191,v1:172.21.15.42:6817/3110347191] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f362413cd20 con 0x7f3600042830 2024-07-04T17:37:19.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.091+0000 7f361b400640 1 -- 172.21.15.42:0/869354448 <== osd.1 v2:172.21.15.42:6816/3110347191 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f362413cd20 con 0x7f3600042830 2024-07-04T17:37:19.092 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:37:19.092 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-07-04T17:37:19.093 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:37:19.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.092+0000 7f3623e00640 1 -- 172.21.15.42:0/869354448 >> [v2:172.21.15.42:6816/3110347191,v1:172.21.15.42:6817/3110347191] conn(0x7f3600042830 msgr2=0x7f3600044cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:19.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.092+0000 7f3623e00640 1 --2- 172.21.15.42:0/869354448 >> [v2:172.21.15.42:6816/3110347191,v1:172.21.15.42:6817/3110347191] conn(0x7f3600042830 0x7f3600044cc0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:19.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.092+0000 7f3623e00640 1 -- 172.21.15.42:0/869354448 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f360003d4f0 msgr2=0x7f360003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:19.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.092+0000 7f3623e00640 1 --2- 172.21.15.42:0/869354448 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f360003d4f0 0x7f360003fa00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f360c034000 tx=0x7f360c004730 comp rx=0 tx=0).stop 2024-07-04T17:37:19.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.092+0000 7f3623e00640 1 -- 172.21.15.42:0/869354448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36240ac5b0 msgr2=0x7f36241d2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:19.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.092+0000 7f3623e00640 1 --2- 172.21.15.42:0/869354448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36240ac5b0 0x7f36241d2f80 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f361400c3b0 tx=0x7f361400c870 comp rx=0 tx=0).stop 2024-07-04T17:37:19.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.092+0000 7f3623e00640 1 -- 172.21.15.42:0/869354448 shutdown_connections 2024-07-04T17:37:19.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.092+0000 7f3623e00640 1 --2- 172.21.15.42:0/869354448 >> [v2:172.21.15.42:6816/3110347191,v1:172.21.15.42:6817/3110347191] conn(0x7f3600042830 0x7f3600044cc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:19.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.092+0000 7f3623e00640 1 --2- 172.21.15.42:0/869354448 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f360003d4f0 0x7f360003fa00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:19.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.092+0000 7f3623e00640 1 --2- 172.21.15.42:0/869354448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36240ac5b0 0x7f36241d2f80 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:19.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.092+0000 7f3623e00640 1 -- 172.21.15.42:0/869354448 >> 172.21.15.42:0/869354448 conn(0x7f36240afaa0 msgr2=0x7f3624075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:19.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.093+0000 7f3623e00640 1 -- 172.21.15.42:0/869354448 shutdown_connections 2024-07-04T17:37:19.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.093+0000 7f3623e00640 1 -- 172.21.15.42:0/869354448 wait complete. 2024-07-04T17:37:19.120 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-07-04T17:37:19.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.409+0000 7f98fd600640 1 -- 172.21.15.42:0/1040525543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98f80bad70 msgr2=0x7f98f806c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:19.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.409+0000 7f98fd600640 1 --2- 172.21.15.42:0/1040525543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98f80bad70 0x7f98f806c2c0 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f98e00097b0 tx=0x7f98e002a4a0 comp rx=0 tx=0).stop 2024-07-04T17:37:19.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.409+0000 7f98fd600640 1 -- 172.21.15.42:0/1040525543 shutdown_connections 2024-07-04T17:37:19.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.409+0000 7f98fd600640 1 --2- 172.21.15.42:0/1040525543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98f80bad70 0x7f98f806c2c0 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:19.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.409+0000 7f98fd600640 1 -- 172.21.15.42:0/1040525543 >> 172.21.15.42:0/1040525543 conn(0x7f98f80bebc0 msgr2=0x7f98f80befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:19.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.409+0000 7f98fd600640 1 -- 172.21.15.42:0/1040525543 shutdown_connections 2024-07-04T17:37:19.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.409+0000 7f98fd600640 1 -- 172.21.15.42:0/1040525543 wait complete. 2024-07-04T17:37:19.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.411+0000 7f98fd600640 1 Processor -- start 2024-07-04T17:37:19.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.411+0000 7f98fd600640 1 -- start start 2024-07-04T17:37:19.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.413+0000 7f98fd600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98f806d580 0x7f98f8071af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:19.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.413+0000 7f98fd600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f98f80b2810 con 0x7f98f806d580 2024-07-04T17:37:19.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.413+0000 7f98f7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98f806d580 0x7f98f8071af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:19.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.413+0000 7f98f7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98f806d580 0x7f98f8071af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44168/0 (socket says 172.21.15.42:44168) 2024-07-04T17:37:19.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.413+0000 7f98f7e00640 1 -- 172.21.15.42:0/4088510237 learned_addr learned my addr 172.21.15.42:0/4088510237 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:37:19.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.414+0000 7f98f7e00640 1 -- 172.21.15.42:0/4088510237 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f80713f0 con 0x7f98f806d580 2024-07-04T17:37:19.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.414+0000 7f98f7e00640 1 --2- 172.21.15.42:0/4088510237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98f806d580 0x7f98f8071af0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f98e800c3b0 tx=0x7f98e800c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:19.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.414+0000 7f98f4c00640 1 -- 172.21.15.42:0/4088510237 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98e8011780 con 0x7f98f806d580 2024-07-04T17:37:19.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.414+0000 7f98f4c00640 1 -- 172.21.15.42:0/4088510237 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98e8017070 con 0x7f98f806d580 2024-07-04T17:37:19.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.415+0000 7f98f4c00640 1 -- 172.21.15.42:0/4088510237 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98e8011c00 con 0x7f98f806d580 2024-07-04T17:37:19.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.415+0000 7f98fd600640 1 -- 172.21.15.42:0/4088510237 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98f8072200 con 0x7f98f806d580 2024-07-04T17:37:19.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.415+0000 7f98fd600640 1 -- 172.21.15.42:0/4088510237 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98f8072e80 con 0x7f98f806d580 2024-07-04T17:37:19.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.416+0000 7f98fd600640 1 -- 172.21.15.42:0/4088510237 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98f8072030 con 0x7f98f806d580 2024-07-04T17:37:19.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.416+0000 7f98f4c00640 1 -- 172.21.15.42:0/4088510237 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 64) ==== 49615+0+0 (secure 0 0 0) 0x7f98e801c460 con 0x7f98f806d580 2024-07-04T17:37:19.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.416+0000 7f98f4c00640 1 --2- 172.21.15.42:0/4088510237 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f98d403d4f0 0x7f98d403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:19.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.417+0000 7f98f4c00640 1 -- 172.21.15.42:0/4088510237 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 3533+0+0 (secure 0 0 0) 0x7f98e8051550 con 0x7f98f806d580 2024-07-04T17:37:19.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.417+0000 7f98efe00640 1 --2- 172.21.15.42:0/4088510237 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f98d403d4f0 0x7f98d403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:19.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.418+0000 7f98efe00640 1 --2- 172.21.15.42:0/4088510237 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f98d403d4f0 0x7f98d403fa00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f98e0034000 tx=0x7f98e0004730 comp rx=0 tx=0).ready entity=mgr.5988 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:19.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.422+0000 7f98f4c00640 1 -- 172.21.15.42:0/4088510237 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f98e801c800 con 0x7f98f806d580 2024-07-04T17:37:19.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.632+0000 7f98fd600640 1 -- 172.21.15.42:0/4088510237 --> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f98f80734e0 con 0x7f98d403d4f0 2024-07-04T17:37:19.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.633+0000 7f98f4c00640 1 -- 172.21.15.42:0/4088510237 <== mgr.5988 v2:172.21.15.42:6834/2161106357 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f98f80734e0 con 0x7f98d403d4f0 2024-07-04T17:37:19.634 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 1 to deep-scrub 2024-07-04T17:37:19.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.636+0000 7f98fd600640 1 -- 172.21.15.42:0/4088510237 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f98d403d4f0 msgr2=0x7f98d403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:19.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.636+0000 7f98fd600640 1 --2- 172.21.15.42:0/4088510237 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f98d403d4f0 0x7f98d403fa00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f98e0034000 tx=0x7f98e0004730 comp rx=0 tx=0).stop 2024-07-04T17:37:19.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.636+0000 7f98fd600640 1 -- 172.21.15.42:0/4088510237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98f806d580 msgr2=0x7f98f8071af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:19.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.636+0000 7f98fd600640 1 --2- 172.21.15.42:0/4088510237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98f806d580 0x7f98f8071af0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f98e800c3b0 tx=0x7f98e800c870 comp rx=0 tx=0).stop 2024-07-04T17:37:19.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.636+0000 7f98fd600640 1 -- 172.21.15.42:0/4088510237 shutdown_connections 2024-07-04T17:37:19.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.636+0000 7f98fd600640 1 --2- 172.21.15.42:0/4088510237 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f98d403d4f0 0x7f98d403fa00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:19.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.636+0000 7f98fd600640 1 --2- 172.21.15.42:0/4088510237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98f806d580 0x7f98f8071af0 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:19.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.636+0000 7f98fd600640 1 -- 172.21.15.42:0/4088510237 >> 172.21.15.42:0/4088510237 conn(0x7f98f80bebc0 msgr2=0x7f98f8142c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:19.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.636+0000 7f98fd600640 1 -- 172.21.15.42:0/4088510237 shutdown_connections 2024-07-04T17:37:19.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.636+0000 7f98fd600640 1 -- 172.21.15.42:0/4088510237 wait complete. 2024-07-04T17:37:19.665 INFO:tasks.ceph:Scrubbing osd.2 2024-07-04T17:37:19.665 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-07-04T17:37:19.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.964+0000 7fc47bc00640 1 -- 172.21.15.42:0/2764136748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc474073470 msgr2=0x7fc4740a9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:19.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.964+0000 7fc47bc00640 1 --2- 172.21.15.42:0/2764136748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc474073470 0x7fc4740a9990 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7fc45c0097b0 tx=0x7fc45c02a540 comp rx=0 tx=0).stop 2024-07-04T17:37:19.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.965+0000 7fc47bc00640 1 -- 172.21.15.42:0/2764136748 shutdown_connections 2024-07-04T17:37:19.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.965+0000 7fc47bc00640 1 --2- 172.21.15.42:0/2764136748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc474073470 0x7fc4740a9990 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:19.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.965+0000 7fc47bc00640 1 -- 172.21.15.42:0/2764136748 >> 172.21.15.42:0/2764136748 conn(0x7fc47406c020 msgr2=0x7fc47406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:19.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.965+0000 7fc47bc00640 1 -- 172.21.15.42:0/2764136748 shutdown_connections 2024-07-04T17:37:19.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.965+0000 7fc47bc00640 1 -- 172.21.15.42:0/2764136748 wait complete. 2024-07-04T17:37:19.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.968+0000 7fc47bc00640 1 Processor -- start 2024-07-04T17:37:19.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.968+0000 7fc47bc00640 1 -- start start 2024-07-04T17:37:19.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.969+0000 7fc47bc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc474073470 0x7fc4740ac2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:19.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.969+0000 7fc47bc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc47413f430 con 0x7fc474073470 2024-07-04T17:37:19.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.970+0000 7fc479200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc474073470 0x7fc4740ac2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:19.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.970+0000 7fc479200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc474073470 0x7fc4740ac2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44190/0 (socket says 172.21.15.42:44190) 2024-07-04T17:37:19.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.970+0000 7fc479200640 1 -- 172.21.15.42:0/2187918174 learned_addr learned my addr 172.21.15.42:0/2187918174 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:37:19.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.970+0000 7fc479200640 1 -- 172.21.15.42:0/2187918174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4740ac910 con 0x7fc474073470 2024-07-04T17:37:19.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.971+0000 7fc479200640 1 --2- 172.21.15.42:0/2187918174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc474073470 0x7fc4740ac2f0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7fc46400c380 tx=0x7fc46400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:19.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.971+0000 7fc470c00640 1 -- 172.21.15.42:0/2187918174 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4640116c0 con 0x7fc474073470 2024-07-04T17:37:19.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.971+0000 7fc470c00640 1 -- 172.21.15.42:0/2187918174 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc464017070 con 0x7fc474073470 2024-07-04T17:37:19.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.971+0000 7fc47bc00640 1 -- 172.21.15.42:0/2187918174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4741e1f30 con 0x7fc474073470 2024-07-04T17:37:19.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.971+0000 7fc470c00640 1 -- 172.21.15.42:0/2187918174 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc464011b20 con 0x7fc474073470 2024-07-04T17:37:19.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.971+0000 7fc47bc00640 1 -- 172.21.15.42:0/2187918174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4740aef80 con 0x7fc474073470 2024-07-04T17:37:19.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.972+0000 7fc47bc00640 1 -- 172.21.15.42:0/2187918174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc47413fb00 con 0x7fc474073470 2024-07-04T17:37:19.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.973+0000 7fc470c00640 1 -- 172.21.15.42:0/2187918174 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 64) ==== 49615+0+0 (secure 0 0 0) 0x7fc464011cc0 con 0x7fc474073470 2024-07-04T17:37:19.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.973+0000 7fc470c00640 1 --2- 172.21.15.42:0/2187918174 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fc45003d4f0 0x7fc45003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:19.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.973+0000 7fc470c00640 1 -- 172.21.15.42:0/2187918174 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 3533+0+0 (secure 0 0 0) 0x7fc464051620 con 0x7fc474073470 2024-07-04T17:37:19.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.973+0000 7fc470c00640 1 --2- 172.21.15.42:0/2187918174 >> [v2:172.21.15.42:6808/2100781882,v1:172.21.15.42:6809/2100781882] conn(0x7fc450042830 0x7fc450044cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:19.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.973+0000 7fc470c00640 1 -- 172.21.15.42:0/2187918174 --> [v2:172.21.15.42:6808/2100781882,v1:172.21.15.42:6809/2100781882] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc464018ec0 con 0x7fc450042830 2024-07-04T17:37:19.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.973+0000 7fc470c00640 1 -- 172.21.15.42:0/2187918174 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=151) ==== 24+0+0 (secure 0 0 0) 0x7fc464058050 con 0x7fc474073470 2024-07-04T17:37:19.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.973+0000 7fc473e00640 1 --2- 172.21.15.42:0/2187918174 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fc45003d4f0 0x7fc45003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:19.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.974+0000 7fc479c00640 1 --2- 172.21.15.42:0/2187918174 >> [v2:172.21.15.42:6808/2100781882,v1:172.21.15.42:6809/2100781882] conn(0x7fc450042830 0x7fc450044cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:19.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.974+0000 7fc473e00640 1 --2- 172.21.15.42:0/2187918174 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fc45003d4f0 0x7fc45003fa00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc45c034000 tx=0x7fc45c004730 comp rx=0 tx=0).ready entity=mgr.5988 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:19.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.975+0000 7fc479c00640 1 --2- 172.21.15.42:0/2187918174 >> [v2:172.21.15.42:6808/2100781882,v1:172.21.15.42:6809/2100781882] conn(0x7fc450042830 0x7fc450044cc0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:19.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.976+0000 7fc470c00640 1 -- 172.21.15.42:0/2187918174 <== osd.2 v2:172.21.15.42:6808/2100781882 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc464018ec0 con 0x7fc450042830 2024-07-04T17:37:19.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.993+0000 7fc47bc00640 1 -- 172.21.15.42:0/2187918174 --> [v2:172.21.15.42:6808/2100781882,v1:172.21.15.42:6809/2100781882] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc4740b4900 con 0x7fc450042830 2024-07-04T17:37:19.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.994+0000 7fc470c00640 1 -- 172.21.15.42:0/2187918174 <== osd.2 v2:172.21.15.42:6808/2100781882 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc4740b4900 con 0x7fc450042830 2024-07-04T17:37:19.995 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-07-04T17:37:19.995 INFO:teuthology.orchestra.run.smithi042.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-07-04T17:37:19.995 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-07-04T17:37:19.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.994+0000 7fc47bc00640 1 -- 172.21.15.42:0/2187918174 >> [v2:172.21.15.42:6808/2100781882,v1:172.21.15.42:6809/2100781882] conn(0x7fc450042830 msgr2=0x7fc450044cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:19.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.994+0000 7fc47bc00640 1 --2- 172.21.15.42:0/2187918174 >> [v2:172.21.15.42:6808/2100781882,v1:172.21.15.42:6809/2100781882] conn(0x7fc450042830 0x7fc450044cc0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:19.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.994+0000 7fc47bc00640 1 -- 172.21.15.42:0/2187918174 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fc45003d4f0 msgr2=0x7fc45003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:19.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.994+0000 7fc47bc00640 1 --2- 172.21.15.42:0/2187918174 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fc45003d4f0 0x7fc45003fa00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc45c034000 tx=0x7fc45c004730 comp rx=0 tx=0).stop 2024-07-04T17:37:19.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.994+0000 7fc47bc00640 1 -- 172.21.15.42:0/2187918174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc474073470 msgr2=0x7fc4740ac2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:19.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.994+0000 7fc47bc00640 1 --2- 172.21.15.42:0/2187918174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc474073470 0x7fc4740ac2f0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7fc46400c380 tx=0x7fc46400c840 comp rx=0 tx=0).stop 2024-07-04T17:37:19.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.995+0000 7fc47bc00640 1 -- 172.21.15.42:0/2187918174 shutdown_connections 2024-07-04T17:37:19.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.995+0000 7fc47bc00640 1 --2- 172.21.15.42:0/2187918174 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fc45003d4f0 0x7fc45003fa00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:19.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.995+0000 7fc47bc00640 1 --2- 172.21.15.42:0/2187918174 >> [v2:172.21.15.42:6808/2100781882,v1:172.21.15.42:6809/2100781882] conn(0x7fc450042830 0x7fc450044cc0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:19.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.995+0000 7fc47bc00640 1 --2- 172.21.15.42:0/2187918174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc474073470 0x7fc4740ac2f0 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:19.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.995+0000 7fc47bc00640 1 -- 172.21.15.42:0/2187918174 >> 172.21.15.42:0/2187918174 conn(0x7fc47406c020 msgr2=0x7fc4740b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:19.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.995+0000 7fc47bc00640 1 -- 172.21.15.42:0/2187918174 shutdown_connections 2024-07-04T17:37:19.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:19.995+0000 7fc47bc00640 1 -- 172.21.15.42:0/2187918174 wait complete. 2024-07-04T17:37:20.025 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-07-04T17:37:20.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.315+0000 7fc4c7a00640 1 -- 172.21.15.42:0/4008225320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4c006cea0 msgr2=0x7fc4c00b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:20.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.315+0000 7fc4c7a00640 1 --2- 172.21.15.42:0/4008225320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4c006cea0 0x7fc4c00b3030 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4009230 tx=0x7fc4b402a480 comp rx=0 tx=0).stop 2024-07-04T17:37:20.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.315+0000 7fc4c7a00640 1 -- 172.21.15.42:0/4008225320 shutdown_connections 2024-07-04T17:37:20.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.315+0000 7fc4c7a00640 1 --2- 172.21.15.42:0/4008225320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4c006cea0 0x7fc4c00b3030 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:20.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.315+0000 7fc4c7a00640 1 -- 172.21.15.42:0/4008225320 >> 172.21.15.42:0/4008225320 conn(0x7fc4c00bf700 msgr2=0x7fc4c00af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:20.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.316+0000 7fc4c7a00640 1 -- 172.21.15.42:0/4008225320 shutdown_connections 2024-07-04T17:37:20.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.316+0000 7fc4c7a00640 1 -- 172.21.15.42:0/4008225320 wait complete. 2024-07-04T17:37:20.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.318+0000 7fc4c7a00640 1 Processor -- start 2024-07-04T17:37:20.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.318+0000 7fc4c7a00640 1 -- start start 2024-07-04T17:37:20.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.319+0000 7fc4c7a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4c006cea0 0x7fc4c00be3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:20.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.319+0000 7fc4c7a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c006ef50 con 0x7fc4c006cea0 2024-07-04T17:37:20.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.320+0000 7fc4c5000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4c006cea0 0x7fc4c00be3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:20.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.320+0000 7fc4c5000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4c006cea0 0x7fc4c00be3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44208/0 (socket says 172.21.15.42:44208) 2024-07-04T17:37:20.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.320+0000 7fc4c5000640 1 -- 172.21.15.42:0/79954294 learned_addr learned my addr 172.21.15.42:0/79954294 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:37:20.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.320+0000 7fc4c5000640 1 -- 172.21.15.42:0/79954294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c01e7040 con 0x7fc4c006cea0 2024-07-04T17:37:20.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.320+0000 7fc4c5000640 1 --2- 172.21.15.42:0/79954294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4c006cea0 0x7fc4c00be3f0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4009290 tx=0x7fc4b4031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:20.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.321+0000 7fc4bcc00640 1 -- 172.21.15.42:0/79954294 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4b4041070 con 0x7fc4c006cea0 2024-07-04T17:37:20.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.321+0000 7fc4bcc00640 1 -- 172.21.15.42:0/79954294 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4b4003be0 con 0x7fc4c006cea0 2024-07-04T17:37:20.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.321+0000 7fc4bcc00640 1 -- 172.21.15.42:0/79954294 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4b403b440 con 0x7fc4c006cea0 2024-07-04T17:37:20.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.321+0000 7fc4c7a00640 1 -- 172.21.15.42:0/79954294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c0071fc0 con 0x7fc4c006cea0 2024-07-04T17:37:20.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.321+0000 7fc4c7a00640 1 -- 172.21.15.42:0/79954294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c013f850 con 0x7fc4c006cea0 2024-07-04T17:37:20.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.322+0000 7fc4c7a00640 1 -- 172.21.15.42:0/79954294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4c00b3030 con 0x7fc4c006cea0 2024-07-04T17:37:20.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.323+0000 7fc4bcc00640 1 -- 172.21.15.42:0/79954294 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 64) ==== 49615+0+0 (secure 0 0 0) 0x7fc4b4044070 con 0x7fc4c006cea0 2024-07-04T17:37:20.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.323+0000 7fc4bcc00640 1 --2- 172.21.15.42:0/79954294 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fc4a003d4f0 0x7fc4a003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:20.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.323+0000 7fc4bcc00640 1 -- 172.21.15.42:0/79954294 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 3533+0+0 (secure 0 0 0) 0x7fc4b4073070 con 0x7fc4c006cea0 2024-07-04T17:37:20.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.324+0000 7fc4bfe00640 1 --2- 172.21.15.42:0/79954294 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fc4a003d4f0 0x7fc4a003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:20.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.324+0000 7fc4bfe00640 1 --2- 172.21.15.42:0/79954294 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fc4a003d4f0 0x7fc4a003fa00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac007fd0 tx=0x7fc4ac007920 comp rx=0 tx=0).ready entity=mgr.5988 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:20.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.328+0000 7fc4bcc00640 1 -- 172.21.15.42:0/79954294 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fc4b403b5e0 con 0x7fc4c006cea0 2024-07-04T17:37:20.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.539+0000 7fc4c7a00640 1 -- 172.21.15.42:0/79954294 --> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fc4c0072940 con 0x7fc4a003d4f0 2024-07-04T17:37:20.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.540+0000 7fc4bcc00640 1 -- 172.21.15.42:0/79954294 <== mgr.5988 v2:172.21.15.42:6834/2161106357 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc4c0072940 con 0x7fc4a003d4f0 2024-07-04T17:37:20.541 INFO:teuthology.orchestra.run.smithi042.stderr:instructed osd(s) 2 to deep-scrub 2024-07-04T17:37:20.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.543+0000 7fc4c7a00640 1 -- 172.21.15.42:0/79954294 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fc4a003d4f0 msgr2=0x7fc4a003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:20.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.543+0000 7fc4c7a00640 1 --2- 172.21.15.42:0/79954294 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fc4a003d4f0 0x7fc4a003fa00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac007fd0 tx=0x7fc4ac007920 comp rx=0 tx=0).stop 2024-07-04T17:37:20.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.543+0000 7fc4c7a00640 1 -- 172.21.15.42:0/79954294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4c006cea0 msgr2=0x7fc4c00be3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:20.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.543+0000 7fc4c7a00640 1 --2- 172.21.15.42:0/79954294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4c006cea0 0x7fc4c00be3f0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4009290 tx=0x7fc4b4031a90 comp rx=0 tx=0).stop 2024-07-04T17:37:20.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.543+0000 7fc4c7a00640 1 -- 172.21.15.42:0/79954294 shutdown_connections 2024-07-04T17:37:20.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.543+0000 7fc4c7a00640 1 --2- 172.21.15.42:0/79954294 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fc4a003d4f0 0x7fc4a003fa00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:20.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.543+0000 7fc4c7a00640 1 --2- 172.21.15.42:0/79954294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4c006cea0 0x7fc4c00be3f0 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:20.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.543+0000 7fc4c7a00640 1 -- 172.21.15.42:0/79954294 >> 172.21.15.42:0/79954294 conn(0x7fc4c00bf700 msgr2=0x7fc4c00bc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:20.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.543+0000 7fc4c7a00640 1 -- 172.21.15.42:0/79954294 shutdown_connections 2024-07-04T17:37:20.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.543+0000 7fc4c7a00640 1 -- 172.21.15.42:0/79954294 wait complete. 2024-07-04T17:37:20.571 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:37:20.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.860+0000 7f64dee00640 1 -- 172.21.15.42:0/1612985086 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64d80b2d20 msgr2=0x7f64d80bb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:20.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.860+0000 7f64dee00640 1 --2- 172.21.15.42:0/1612985086 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64d80b2d20 0x7f64d80bb1f0 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f64c40097b0 tx=0x7f64c402a4b0 comp rx=0 tx=0).stop 2024-07-04T17:37:20.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.860+0000 7f64dee00640 1 -- 172.21.15.42:0/1612985086 shutdown_connections 2024-07-04T17:37:20.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.860+0000 7f64dee00640 1 --2- 172.21.15.42:0/1612985086 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64d80b2d20 0x7f64d80bb1f0 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:20.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.860+0000 7f64dee00640 1 -- 172.21.15.42:0/1612985086 >> 172.21.15.42:0/1612985086 conn(0x7f64d80bf700 msgr2=0x7f64d80af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:20.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.860+0000 7f64dee00640 1 -- 172.21.15.42:0/1612985086 shutdown_connections 2024-07-04T17:37:20.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.860+0000 7f64dee00640 1 -- 172.21.15.42:0/1612985086 wait complete. 2024-07-04T17:37:20.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.862+0000 7f64dee00640 1 Processor -- start 2024-07-04T17:37:20.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.863+0000 7f64dee00640 1 -- start start 2024-07-04T17:37:20.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.864+0000 7f64dee00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64d80b2d20 0x7f64d813c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:20.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.864+0000 7f64dee00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f64d806c020 con 0x7f64d80b2d20 2024-07-04T17:37:20.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.864+0000 7f64d7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64d80b2d20 0x7f64d813c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:20.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.864+0000 7f64d7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64d80b2d20 0x7f64d813c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44236/0 (socket says 172.21.15.42:44236) 2024-07-04T17:37:20.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.864+0000 7f64d7e00640 1 -- 172.21.15.42:0/1771216107 learned_addr learned my addr 172.21.15.42:0/1771216107 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:37:20.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.865+0000 7f64d7e00640 1 -- 172.21.15.42:0/1771216107 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64d81e4050 con 0x7f64d80b2d20 2024-07-04T17:37:20.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.865+0000 7f64d7e00640 1 --2- 172.21.15.42:0/1771216107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64d80b2d20 0x7f64d813c530 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f64c800c380 tx=0x7f64c800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:20.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.865+0000 7f64cfe00640 1 -- 172.21.15.42:0/1771216107 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64c80116c0 con 0x7f64d80b2d20 2024-07-04T17:37:20.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.865+0000 7f64cfe00640 1 -- 172.21.15.42:0/1771216107 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64c8017070 con 0x7f64d80b2d20 2024-07-04T17:37:20.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.865+0000 7f64cfe00640 1 -- 172.21.15.42:0/1771216107 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64c8011b20 con 0x7f64d80b2d20 2024-07-04T17:37:20.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.865+0000 7f64dee00640 1 -- 172.21.15.42:0/1771216107 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64d813ca70 con 0x7f64d80b2d20 2024-07-04T17:37:20.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.865+0000 7f64dee00640 1 -- 172.21.15.42:0/1771216107 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64d813e600 con 0x7f64d80b2d20 2024-07-04T17:37:20.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.867+0000 7f64dee00640 1 -- 172.21.15.42:0/1771216107 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64d813e7d0 con 0x7f64d80b2d20 2024-07-04T17:37:20.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.867+0000 7f64cfe00640 1 -- 172.21.15.42:0/1771216107 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 64) ==== 49615+0+0 (secure 0 0 0) 0x7f64c8011cc0 con 0x7f64d80b2d20 2024-07-04T17:37:20.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.867+0000 7f64cfe00640 1 --2- 172.21.15.42:0/1771216107 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f64b803d4f0 0x7f64b803fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:20.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.868+0000 7f64cfe00640 1 -- 172.21.15.42:0/1771216107 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 3533+0+0 (secure 0 0 0) 0x7f64c8051660 con 0x7f64d80b2d20 2024-07-04T17:37:20.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.868+0000 7f64d7400640 1 --2- 172.21.15.42:0/1771216107 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f64b803d4f0 0x7f64b803fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:20.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.869+0000 7f64d7400640 1 --2- 172.21.15.42:0/1771216107 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f64b803d4f0 0x7f64b803fa00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f64c4034000 tx=0x7f64c4004730 comp rx=0 tx=0).ready entity=mgr.5988 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:20.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:20.873+0000 7f64cfe00640 1 -- 172.21.15.42:0/1771216107 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f64d813e7d0 con 0x7f64d80b2d20 2024-07-04T17:37:21.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:21.078+0000 7f64dee00640 1 -- 172.21.15.42:0/1771216107 --> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f64d813d010 con 0x7f64b803d4f0 2024-07-04T17:37:21.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:21.079+0000 7f64cfe00640 1 -- 172.21.15.42:0/1771216107 <== mgr.5988 v2:172.21.15.42:6834/2161106357 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+9698 (secure 0 0 0) 0x7f64d813d010 con 0x7f64b803d4f0 2024-07-04T17:37:21.080 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:37:21.080 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:37:21.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:21.081+0000 7f64dee00640 1 -- 172.21.15.42:0/1771216107 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f64b803d4f0 msgr2=0x7f64b803fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:21.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:21.081+0000 7f64dee00640 1 --2- 172.21.15.42:0/1771216107 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f64b803d4f0 0x7f64b803fa00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f64c4034000 tx=0x7f64c4004730 comp rx=0 tx=0).stop 2024-07-04T17:37:21.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:21.081+0000 7f64dee00640 1 -- 172.21.15.42:0/1771216107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64d80b2d20 msgr2=0x7f64d813c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:21.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:21.081+0000 7f64dee00640 1 --2- 172.21.15.42:0/1771216107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64d80b2d20 0x7f64d813c530 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f64c800c380 tx=0x7f64c800c840 comp rx=0 tx=0).stop 2024-07-04T17:37:21.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:21.081+0000 7f64dee00640 1 -- 172.21.15.42:0/1771216107 shutdown_connections 2024-07-04T17:37:21.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:21.081+0000 7f64dee00640 1 --2- 172.21.15.42:0/1771216107 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f64b803d4f0 0x7f64b803fa00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:21.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:21.081+0000 7f64dee00640 1 --2- 172.21.15.42:0/1771216107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64d80b2d20 0x7f64d813c530 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:21.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:21.081+0000 7f64dee00640 1 -- 172.21.15.42:0/1771216107 >> 172.21.15.42:0/1771216107 conn(0x7f64d80bf700 msgr2=0x7f64d8075150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:21.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:21.082+0000 7f64dee00640 1 -- 172.21.15.42:0/1771216107 shutdown_connections 2024-07-04T17:37:21.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:21.082+0000 7f64dee00640 1 -- 172.21.15.42:0/1771216107 wait complete. 2024-07-04T17:37:21.114 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-07-04T17:37:19.199997+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":716,"num_read_kb":1982,"num_write":211,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129,"ondisk_log_size":129,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":2,"kb":283115520,"kb_used":111844,"kb_used_data":5816,"kb_used_omap":4,"kb_used_meta":105979,"kb_avail":283003676,"statfs":{"total":289910292480,"available":289795764224,"internally_reserved":0,"allocated":5955584,"data_stored":3086078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":108522831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.310210"},"pg_stats":[{"pgid":"1.0","version":"146'129","reported_seq":621,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-07-04T17:37:12.614040+0000","last_change":"2024-07-04T17:26:04.953559+0000","last_active":"2024-07-04T17:37:12.614040+0000","last_peered":"2024-07-04T17:37:12.614040+0000","last_clean":"2024-07-04T17:37:12.614040+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:37:12.614040+0000","last_undegraded":"2024-07-04T17:37:12.614040+0000","last_fullsized":"2024-07-04T17:37:12.614040+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T17:26:03.933432+0000","last_clean_scrub_stamp":"2024-07-04T17:26:03.933432+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:25:39.064181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":716,"num_read_kb":1982,"num_write":211,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":716,"num_read_kb":1982,"num_write":211,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129,"ondisk_log_size":129,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738499,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36896,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334944,"statfs":{"total":96636764160,"available":96598982656,"internally_reserved":0,"allocated":1589248,"data_stored":635114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738500,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37472,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334368,"statfs":{"total":96636764160,"available":96598392832,"internally_reserved":0,"allocated":2183168,"data_stored":1225482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738498,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37476,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334364,"statfs":{"total":96636764160,"available":96598388736,"internally_reserved":0,"allocated":2183168,"data_stored":1225482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:37:21.116 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-07-04T17:26:03.933432+0000 time.struct_time(tm_year=2024, tm_mon=7, tm_mday=4, tm_hour=17, tm_min=26, tm_sec=3, tm_wday=3, tm_yday=186, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=7, tm_mday=4, tm_hour=17, tm_min=37, tm_sec=16, tm_wday=3, tm_yday=186, tm_isdst=0) 2024-07-04T17:37:21.117 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-07-04T17:37:32.191 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-04T17:37:41.118 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T17:37:41.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.424+0000 7f5449400640 1 -- 172.21.15.42:0/1010027330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54440b02b0 msgr2=0x7f54440b3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:41.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.424+0000 7f5449400640 1 --2- 172.21.15.42:0/1010027330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54440b02b0 0x7f54440b3650 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f54340097b0 tx=0x7f543402a4b0 comp rx=0 tx=0).stop 2024-07-04T17:37:41.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.424+0000 7f5449400640 1 -- 172.21.15.42:0/1010027330 shutdown_connections 2024-07-04T17:37:41.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.424+0000 7f5449400640 1 --2- 172.21.15.42:0/1010027330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54440b02b0 0x7f54440b3650 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:41.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.424+0000 7f5449400640 1 -- 172.21.15.42:0/1010027330 >> 172.21.15.42:0/1010027330 conn(0x7f54440bf700 msgr2=0x7f54440af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:41.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.424+0000 7f5449400640 1 -- 172.21.15.42:0/1010027330 shutdown_connections 2024-07-04T17:37:41.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.424+0000 7f5449400640 1 -- 172.21.15.42:0/1010027330 wait complete. 2024-07-04T17:37:41.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.426+0000 7f5449400640 1 Processor -- start 2024-07-04T17:37:41.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.426+0000 7f5449400640 1 -- start start 2024-07-04T17:37:41.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.428+0000 7f5449400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54440b02b0 0x7f54440b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:41.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.428+0000 7f5449400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f544406c710 con 0x7f54440b02b0 2024-07-04T17:37:41.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.428+0000 7f5442800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54440b02b0 0x7f54440b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:41.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.428+0000 7f5442800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54440b02b0 0x7f54440b7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37592/0 (socket says 172.21.15.42:37592) 2024-07-04T17:37:41.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.428+0000 7f5442800640 1 -- 172.21.15.42:0/713850498 learned_addr learned my addr 172.21.15.42:0/713850498 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:37:41.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.428+0000 7f5442800640 1 -- 172.21.15.42:0/713850498 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54440b9420 con 0x7f54440b02b0 2024-07-04T17:37:41.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.429+0000 7f5442800640 1 --2- 172.21.15.42:0/713850498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54440b02b0 0x7f54440b7140 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f54340098d0 tx=0x7f5434033e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:41.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.429+0000 7f543b400640 1 -- 172.21.15.42:0/713850498 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5434031bc0 con 0x7f54440b02b0 2024-07-04T17:37:41.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.429+0000 7f543b400640 1 -- 172.21.15.42:0/713850498 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5434031e50 con 0x7f54440b02b0 2024-07-04T17:37:41.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.429+0000 7f543b400640 1 -- 172.21.15.42:0/713850498 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5434033260 con 0x7f54440b02b0 2024-07-04T17:37:41.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.429+0000 7f5449400640 1 -- 172.21.15.42:0/713850498 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54441e4570 con 0x7f54440b02b0 2024-07-04T17:37:41.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.429+0000 7f5449400640 1 -- 172.21.15.42:0/713850498 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54440ba900 con 0x7f54440b02b0 2024-07-04T17:37:41.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.430+0000 7f5449400640 1 -- 172.21.15.42:0/713850498 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f544406cd50 con 0x7f54440b02b0 2024-07-04T17:37:41.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.431+0000 7f543b400640 1 -- 172.21.15.42:0/713850498 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 64) ==== 49615+0+0 (secure 0 0 0) 0x7f5434033400 con 0x7f54440b02b0 2024-07-04T17:37:41.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.431+0000 7f543b400640 1 --2- 172.21.15.42:0/713850498 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f542003d4f0 0x7f542003fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:41.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.431+0000 7f543b400640 1 -- 172.21.15.42:0/713850498 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 3533+0+0 (secure 0 0 0) 0x7f5434072e50 con 0x7f54440b02b0 2024-07-04T17:37:41.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.431+0000 7f5439e00640 1 --2- 172.21.15.42:0/713850498 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f542003d4f0 0x7f542003fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:41.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.432+0000 7f5439e00640 1 --2- 172.21.15.42:0/713850498 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f542003d4f0 0x7f542003fa00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f542c0015c0 tx=0x7f542c007a90 comp rx=0 tx=0).ready entity=mgr.5988 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:41.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.436+0000 7f543b400640 1 -- 172.21.15.42:0/713850498 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f543403f030 con 0x7f54440b02b0 2024-07-04T17:37:41.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.646+0000 7f5449400640 1 -- 172.21.15.42:0/713850498 --> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f54440b8510 con 0x7f542003d4f0 2024-07-04T17:37:41.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.647+0000 7f543b400640 1 -- 172.21.15.42:0/713850498 <== mgr.5988 v2:172.21.15.42:6834/2161106357 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+9667 (secure 0 0 0) 0x7f54440b8510 con 0x7f542003d4f0 2024-07-04T17:37:41.649 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:37:41.649 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-07-04T17:37:41.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.650+0000 7f5449400640 1 -- 172.21.15.42:0/713850498 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f542003d4f0 msgr2=0x7f542003fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:41.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.650+0000 7f5449400640 1 --2- 172.21.15.42:0/713850498 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f542003d4f0 0x7f542003fa00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f542c0015c0 tx=0x7f542c007a90 comp rx=0 tx=0).stop 2024-07-04T17:37:41.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.650+0000 7f5449400640 1 -- 172.21.15.42:0/713850498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54440b02b0 msgr2=0x7f54440b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:41.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.650+0000 7f5449400640 1 --2- 172.21.15.42:0/713850498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54440b02b0 0x7f54440b7140 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f54340098d0 tx=0x7f5434033e40 comp rx=0 tx=0).stop 2024-07-04T17:37:41.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.650+0000 7f5449400640 1 -- 172.21.15.42:0/713850498 shutdown_connections 2024-07-04T17:37:41.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.650+0000 7f5449400640 1 --2- 172.21.15.42:0/713850498 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f542003d4f0 0x7f542003fa00 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:41.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.650+0000 7f5449400640 1 --2- 172.21.15.42:0/713850498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54440b02b0 0x7f54440b7140 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:41.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.650+0000 7f5449400640 1 -- 172.21.15.42:0/713850498 >> 172.21.15.42:0/713850498 conn(0x7f54440bf700 msgr2=0x7f5444145320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:41.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.650+0000 7f5449400640 1 -- 172.21.15.42:0/713850498 shutdown_connections 2024-07-04T17:37:41.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.650+0000 7f5449400640 1 -- 172.21.15.42:0/713850498 wait complete. 2024-07-04T17:37:41.682 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-07-04T17:37:41.207387+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":716,"num_read_kb":1982,"num_write":211,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":2,"kb":283115520,"kb_used":111832,"kb_used_data":5816,"kb_used_omap":4,"kb_used_meta":105979,"kb_avail":283003688,"statfs":{"total":289910292480,"available":289795776512,"internally_reserved":0,"allocated":5955584,"data_stored":3086078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":108522831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004571"},"pg_stats":[{"pgid":"1.0","version":"151'130","reported_seq":631,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-07-04T17:37:19.988516+0000","last_change":"2024-07-04T17:37:19.988553+0000","last_active":"2024-07-04T17:37:19.988516+0000","last_peered":"2024-07-04T17:37:19.988516+0000","last_clean":"2024-07-04T17:37:19.988516+0000","last_became_active":"2024-07-04T17:26:04.952628+0000","last_became_peered":"2024-07-04T17:26:04.952628+0000","last_unstale":"2024-07-04T17:37:19.988516+0000","last_undegraded":"2024-07-04T17:37:19.988516+0000","last_fullsized":"2024-07-04T17:37:19.988516+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'130","last_scrub_stamp":"2024-07-04T17:37:19.988419+0000","last_deep_scrub":"151'130","last_deep_scrub_stamp":"2024-07-04T17:37:19.988419+0000","last_clean_scrub_stamp":"2024-07-04T17:37:19.988419+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-07-06T02:10:45.561676+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":716,"num_read_kb":1982,"num_write":211,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":716,"num_read_kb":1982,"num_write":211,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738503,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36896,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334944,"statfs":{"total":96636764160,"available":96598982656,"internally_reserved":0,"allocated":1589248,"data_stored":635114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738504,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37468,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334372,"statfs":{"total":96636764160,"available":96598396928,"internally_reserved":0,"allocated":2183168,"data_stored":1225482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738502,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37468,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334372,"statfs":{"total":96636764160,"available":96598396928,"internally_reserved":0,"allocated":2183168,"data_stored":1225482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T17:37:41.683 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-07-04T17:37:41.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.969+0000 7f7e13e00640 1 -- 172.21.15.42:0/3005148200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e140b0e70 msgr2=0x7f7e140ac9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:41.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.969+0000 7f7e13e00640 1 --2- 172.21.15.42:0/3005148200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e140b0e70 0x7f7e140ac9d0 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f7e040097b0 tx=0x7f7e0402a4a0 comp rx=0 tx=0).stop 2024-07-04T17:37:41.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.970+0000 7f7e13e00640 1 -- 172.21.15.42:0/3005148200 shutdown_connections 2024-07-04T17:37:41.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.970+0000 7f7e13e00640 1 --2- 172.21.15.42:0/3005148200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e140b0e70 0x7f7e140ac9d0 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:41.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.970+0000 7f7e13e00640 1 -- 172.21.15.42:0/3005148200 >> 172.21.15.42:0/3005148200 conn(0x7f7e140afaa0 msgr2=0x7f7e140afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:41.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.970+0000 7f7e13e00640 1 -- 172.21.15.42:0/3005148200 shutdown_connections 2024-07-04T17:37:41.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.970+0000 7f7e13e00640 1 -- 172.21.15.42:0/3005148200 wait complete. 2024-07-04T17:37:41.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.972+0000 7f7e13e00640 1 Processor -- start 2024-07-04T17:37:41.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.972+0000 7f7e13e00640 1 -- start start 2024-07-04T17:37:41.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.973+0000 7f7e13e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e140ab700 0x7f7e140b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:41.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.973+0000 7f7e13e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7e141415a0 con 0x7f7e140ab700 2024-07-04T17:37:41.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.974+0000 7f7e12a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e140ab700 0x7f7e140b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:41.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.974+0000 7f7e12a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e140ab700 0x7f7e140b6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37600/0 (socket says 172.21.15.42:37600) 2024-07-04T17:37:41.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.974+0000 7f7e12a00640 1 -- 172.21.15.42:0/1949425022 learned_addr learned my addr 172.21.15.42:0/1949425022 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:37:41.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.974+0000 7f7e12a00640 1 -- 172.21.15.42:0/1949425022 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e140b5c80 con 0x7f7e140ab700 2024-07-04T17:37:41.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.974+0000 7f7e12a00640 1 --2- 172.21.15.42:0/1949425022 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e140ab700 0x7f7e140b6180 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f7e040098d0 tx=0x7f7e04033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:41.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.975+0000 7f7e0aa00640 1 -- 172.21.15.42:0/1949425022 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e04031a90 con 0x7f7e140ab700 2024-07-04T17:37:41.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.975+0000 7f7e0aa00640 1 -- 172.21.15.42:0/1949425022 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e04031d20 con 0x7f7e140ab700 2024-07-04T17:37:41.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.975+0000 7f7e0aa00640 1 -- 172.21.15.42:0/1949425022 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e04033200 con 0x7f7e140ab700 2024-07-04T17:37:41.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.975+0000 7f7e13e00640 1 -- 172.21.15.42:0/1949425022 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e141d84d0 con 0x7f7e140ab700 2024-07-04T17:37:41.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.975+0000 7f7e13e00640 1 -- 172.21.15.42:0/1949425022 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e140ab120 con 0x7f7e140ab700 2024-07-04T17:37:41.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.976+0000 7f7e13e00640 1 -- 172.21.15.42:0/1949425022 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e14141c50 con 0x7f7e140ab700 2024-07-04T17:37:41.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.976+0000 7f7e0aa00640 1 -- 172.21.15.42:0/1949425022 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 64) ==== 49615+0+0 (secure 0 0 0) 0x7f7e040333a0 con 0x7f7e140ab700 2024-07-04T17:37:41.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.977+0000 7f7e0aa00640 1 --2- 172.21.15.42:0/1949425022 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f7df403d4f0 0x7f7df403fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:41.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.977+0000 7f7e0aa00640 1 -- 172.21.15.42:0/1949425022 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 3533+0+0 (secure 0 0 0) 0x7f7e04072d80 con 0x7f7e140ab700 2024-07-04T17:37:41.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.977+0000 7f7e12000640 1 --2- 172.21.15.42:0/1949425022 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f7df403d4f0 0x7f7df403fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:41.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.978+0000 7f7e12000640 1 --2- 172.21.15.42:0/1949425022 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f7df403d4f0 0x7f7df403fa00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7e000015e0 tx=0x7f7e00001290 comp rx=0 tx=0).ready entity=mgr.5988 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:41.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:41.982+0000 7f7e0aa00640 1 -- 172.21.15.42:0/1949425022 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7f7e0403f030 con 0x7f7e140ab700 2024-07-04T17:37:42.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.198+0000 7f7e13e00640 1 -- 172.21.15.42:0/1949425022 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f7e14068e30 con 0x7f7e140ab700 2024-07-04T17:37:42.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.204+0000 7f7e0aa00640 1 -- 172.21.15.42:0/1949425022 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v35)=0 v35) ==== 123+0+0 (secure 0 0 0) 0x7f7e0404b3a0 con 0x7f7e140ab700 2024-07-04T17:37:42.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.208+0000 7f7e0aa00640 1 -- 172.21.15.42:0/1949425022 <== mon.0 v2:172.21.15.42:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f7e04030e10 con 0x7f7e140ab700 2024-07-04T17:37:42.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.210+0000 7f7e13e00640 1 -- 172.21.15.42:0/1949425022 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f7df403d4f0 msgr2=0x7f7df403fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:42.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.210+0000 7f7e13e00640 1 --2- 172.21.15.42:0/1949425022 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f7df403d4f0 0x7f7df403fa00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7e000015e0 tx=0x7f7e00001290 comp rx=0 tx=0).stop 2024-07-04T17:37:42.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.210+0000 7f7e13e00640 1 -- 172.21.15.42:0/1949425022 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e140ab700 msgr2=0x7f7e140b6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:42.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.210+0000 7f7e13e00640 1 --2- 172.21.15.42:0/1949425022 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e140ab700 0x7f7e140b6180 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f7e040098d0 tx=0x7f7e04033c10 comp rx=0 tx=0).stop 2024-07-04T17:37:42.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.210+0000 7f7e13e00640 1 -- 172.21.15.42:0/1949425022 shutdown_connections 2024-07-04T17:37:42.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.210+0000 7f7e13e00640 1 --2- 172.21.15.42:0/1949425022 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7f7df403d4f0 0x7f7df403fa00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:42.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.210+0000 7f7e13e00640 1 --2- 172.21.15.42:0/1949425022 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7e140ab700 0x7f7e140b6180 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:42.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.210+0000 7f7e13e00640 1 -- 172.21.15.42:0/1949425022 >> 172.21.15.42:0/1949425022 conn(0x7f7e140afaa0 msgr2=0x7f7e1413e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:42.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.210+0000 7f7e13e00640 1 -- 172.21.15.42:0/1949425022 shutdown_connections 2024-07-04T17:37:42.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.210+0000 7f7e13e00640 1 -- 172.21.15.42:0/1949425022 wait complete. 2024-07-04T17:37:42.241 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-04T17:37:42.242 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-07-04T17:37:42.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.527+0000 7fa776a00640 1 -- 172.21.15.42:0/3012375068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7700b38f0 msgr2=0x7fa7700b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:42.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.527+0000 7fa776a00640 1 --2- 172.21.15.42:0/3012375068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7700b38f0 0x7fa7700b2d20 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7fa760005fd0 tx=0x7fa76002a5f0 comp rx=0 tx=0).stop 2024-07-04T17:37:42.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.527+0000 7fa776a00640 1 -- 172.21.15.42:0/3012375068 shutdown_connections 2024-07-04T17:37:42.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.527+0000 7fa776a00640 1 --2- 172.21.15.42:0/3012375068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7700b38f0 0x7fa7700b2d20 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:42.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.527+0000 7fa776a00640 1 -- 172.21.15.42:0/3012375068 >> 172.21.15.42:0/3012375068 conn(0x7fa7700bf700 msgr2=0x7fa7700af900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:42.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.527+0000 7fa776a00640 1 -- 172.21.15.42:0/3012375068 shutdown_connections 2024-07-04T17:37:42.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.527+0000 7fa776a00640 1 -- 172.21.15.42:0/3012375068 wait complete. 2024-07-04T17:37:42.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.529+0000 7fa776a00640 1 Processor -- start 2024-07-04T17:37:42.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.529+0000 7fa776a00640 1 -- start start 2024-07-04T17:37:42.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.531+0000 7fa776a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7700b01f0 0x7fa7700bad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:42.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.531+0000 7fa776a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa77006cd40 con 0x7fa7700b01f0 2024-07-04T17:37:42.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.531+0000 7fa76fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7700b01f0 0x7fa7700bad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:42.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.531+0000 7fa76fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7700b01f0 0x7fa7700bad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37614/0 (socket says 172.21.15.42:37614) 2024-07-04T17:37:42.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.531+0000 7fa76fe00640 1 -- 172.21.15.42:0/1061598440 learned_addr learned my addr 172.21.15.42:0/1061598440 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-07-04T17:37:42.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.531+0000 7fa76fe00640 1 -- 172.21.15.42:0/1061598440 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7700aea60 con 0x7fa7700b01f0 2024-07-04T17:37:42.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.532+0000 7fa76fe00640 1 --2- 172.21.15.42:0/1061598440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7700b01f0 0x7fa7700bad60 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7fa760009850 tx=0x7fa760003890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:42.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.532+0000 7fa767e00640 1 -- 172.21.15.42:0/1061598440 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa76003e030 con 0x7fa7700b01f0 2024-07-04T17:37:42.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.532+0000 7fa767e00640 1 -- 172.21.15.42:0/1061598440 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7fa760004730 con 0x7fa7700b01f0 2024-07-04T17:37:42.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.532+0000 7fa767e00640 1 -- 172.21.15.42:0/1061598440 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa760035040 con 0x7fa7700b01f0 2024-07-04T17:37:42.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.532+0000 7fa776a00640 1 -- 172.21.15.42:0/1061598440 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7701d9990 con 0x7fa7700b01f0 2024-07-04T17:37:42.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.532+0000 7fa776a00640 1 -- 172.21.15.42:0/1061598440 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa770073200 con 0x7fa7700b01f0 2024-07-04T17:37:42.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.534+0000 7fa776a00640 1 -- 172.21.15.42:0/1061598440 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa77006d4e0 con 0x7fa7700b01f0 2024-07-04T17:37:42.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.534+0000 7fa767e00640 1 -- 172.21.15.42:0/1061598440 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 64) ==== 49615+0+0 (secure 0 0 0) 0x7fa760041030 con 0x7fa7700b01f0 2024-07-04T17:37:42.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.534+0000 7fa767e00640 1 --2- 172.21.15.42:0/1061598440 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fa74c03d4f0 0x7fa74c03fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T17:37:42.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.534+0000 7fa767e00640 1 -- 172.21.15.42:0/1061598440 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 3533+0+0 (secure 0 0 0) 0x7fa76003f070 con 0x7fa7700b01f0 2024-07-04T17:37:42.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.535+0000 7fa76f400640 1 --2- 172.21.15.42:0/1061598440 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fa74c03d4f0 0x7fa74c03fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T17:37:42.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.535+0000 7fa76f400640 1 --2- 172.21.15.42:0/1061598440 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fa74c03d4f0 0x7fa74c03fa00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa770144ef0 tx=0x7fa758007d70 comp rx=0 tx=0).ready entity=mgr.5988 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T17:37:42.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.540+0000 7fa767e00640 1 -- 172.21.15.42:0/1061598440 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191195 (secure 0 0 0) 0x7fa76003acd0 con 0x7fa7700b01f0 2024-07-04T17:37:42.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.792+0000 7fa776a00640 1 -- 172.21.15.42:0/1061598440 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa770074090 con 0x7fa7700b01f0 2024-07-04T17:37:42.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.794+0000 7fa767e00640 1 -- 172.21.15.42:0/1061598440 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) ==== 95+0+4742 (secure 0 0 0) 0x7fa760072330 con 0x7fa7700b01f0 2024-07-04T17:37:42.795 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-07-04T17:37:42.795 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":119,"btime":"2024-07-04T17:37:09:880230+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5803,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6825/1184311185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":1184311185},{"type":"v1","addr":"172.21.15.42:6825","nonce":1184311185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103},{"gid":5808,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/1815554749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1815554749},{"type":"v1","addr":"172.21.15.42:6827","nonce":1815554749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":5809,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/2027504764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2027504764},{"type":"v1","addr":"172.21.15.42:6829","nonce":2027504764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":6027,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6833/4188603550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":4188603550},{"type":"v1","addr":"172.21.15.42:6833","nonce":4188603550}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":117},{"gid":6059,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6831/2222684149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":2222684149},{"type":"v1","addr":"172.21.15.42:6831","nonce":2222684149}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":119}],"filesystems":[]} 2024-07-04T17:37:42.796 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 119 2024-07-04T17:37:42.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.797+0000 7fa776a00640 1 -- 172.21.15.42:0/1061598440 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fa74c03d4f0 msgr2=0x7fa74c03fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:42.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.797+0000 7fa776a00640 1 --2- 172.21.15.42:0/1061598440 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fa74c03d4f0 0x7fa74c03fa00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa770144ef0 tx=0x7fa758007d70 comp rx=0 tx=0).stop 2024-07-04T17:37:42.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.797+0000 7fa776a00640 1 -- 172.21.15.42:0/1061598440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7700b01f0 msgr2=0x7fa7700bad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T17:37:42.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.797+0000 7fa776a00640 1 --2- 172.21.15.42:0/1061598440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7700b01f0 0x7fa7700bad60 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7fa760009850 tx=0x7fa760003890 comp rx=0 tx=0).stop 2024-07-04T17:37:42.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.797+0000 7fa776a00640 1 -- 172.21.15.42:0/1061598440 shutdown_connections 2024-07-04T17:37:42.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.797+0000 7fa776a00640 1 --2- 172.21.15.42:0/1061598440 >> [v2:172.21.15.42:6834/2161106357,v1:172.21.15.42:6835/2161106357] conn(0x7fa74c03d4f0 0x7fa74c03fa00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:42.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.797+0000 7fa776a00640 1 --2- 172.21.15.42:0/1061598440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7700b01f0 0x7fa7700bad60 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T17:37:42.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.797+0000 7fa776a00640 1 -- 172.21.15.42:0/1061598440 >> 172.21.15.42:0/1061598440 conn(0x7fa7700bf700 msgr2=0x7fa770140f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T17:37:42.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.797+0000 7fa776a00640 1 -- 172.21.15.42:0/1061598440 shutdown_connections 2024-07-04T17:37:42.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-07-04T17:37:42.797+0000 7fa776a00640 1 -- 172.21.15.42:0/1061598440 wait complete. 2024-07-04T17:37:42.824 DEBUG:tasks.cephfs.filesystem:already dead... 2024-07-04T17:37:42.824 INFO:teuthology.misc:Shutting down mds daemons... 2024-07-04T17:37:42.824 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-07-04T17:37:42.824 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:37:42.853 INFO:tasks.ceph.mds.a:Stopped 2024-07-04T17:37:42.853 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-07-04T17:37:42.854 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:37:42.942 INFO:tasks.ceph.mds.b:Stopped 2024-07-04T17:37:42.943 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-07-04T17:37:42.943 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:37:42.989 INFO:tasks.ceph.mds.c:Stopped 2024-07-04T17:37:42.990 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-07-04T17:37:42.990 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:37:43.046 INFO:tasks.ceph.mds.d:Stopped 2024-07-04T17:37:43.046 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-07-04T17:37:43.046 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:37:43.104 INFO:tasks.ceph.mds.e:Stopped 2024-07-04T17:37:43.105 INFO:teuthology.misc:Shutting down osd daemons... 2024-07-04T17:37:43.105 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-07-04T17:37:43.105 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:37:46.353 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-07-04T17:37:49.207 INFO:tasks.ceph.osd.0:Stopped 2024-07-04T17:37:49.207 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-07-04T17:37:49.207 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:37:51.655 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-07-04T17:37:51.656 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-07-04T17:37:55.308 INFO:tasks.ceph.osd.1:Stopped 2024-07-04T17:37:55.309 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-07-04T17:37:55.309 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:37:55.381 INFO:tasks.ceph.osd.2:Stopped 2024-07-04T17:37:55.381 INFO:teuthology.misc:Shutting down mgr daemons... 2024-07-04T17:37:55.382 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-07-04T17:37:55.382 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:37:56.757 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2024-07-04T17:37:56.757 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-07-04T17:38:01.484 INFO:tasks.ceph.mgr.x:Stopped 2024-07-04T17:38:01.484 INFO:teuthology.misc:Shutting down mon daemons... 2024-07-04T17:38:01.485 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-07-04T17:38:01.485 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T17:38:01.531 INFO:tasks.ceph.mon.a:Stopped 2024-07-04T17:38:01.531 INFO:tasks.ceph:Checking cluster log for badness... 2024-07-04T17:38:01.531 DEBUG:teuthology.orchestra.run.smithi042:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-07-04T17:38:01.633 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi042.front.sepia.ceph.com 2024-07-04T17:38:01.633 DEBUG:teuthology.orchestra.run.smithi042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-07-04T17:38:01.758 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2024-07-04T17:38:01.802 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi042.front.sepia.ceph.com 2024-07-04T17:38:01.803 DEBUG:teuthology.orchestra.run.smithi042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-07-04T17:38:01.948 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi042.front.sepia.ceph.com 2024-07-04T17:38:01.948 DEBUG:teuthology.orchestra.run.smithi042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-07-04T17:38:02.080 INFO:tasks.ceph:Archiving mon data... 2024-07-04T17:38:02.081 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-07-04_16:36:07-fs-wip-rishabh-testing-20240701.141150-debug-testing-default-smithi/7787245/data/mon.a.tgz 2024-07-04T17:38:02.081 DEBUG:teuthology.orchestra.run.smithi042:> mktemp 2024-07-04T17:38:02.133 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/tmp.m2RZPp3uNm 2024-07-04T17:38:02.134 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.m2RZPp3uNm 2024-07-04T17:38:02.320 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-04T17:38:02.488 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0666 /tmp/tmp.m2RZPp3uNm 2024-07-04T17:38:02.615 DEBUG:teuthology.orchestra.remote:smithi042:/tmp/tmp.m2RZPp3uNm is 1MB 2024-07-04T17:38:02.728 DEBUG:teuthology.orchestra.run.smithi042:> rm -fr /tmp/tmp.m2RZPp3uNm 2024-07-04T17:38:02.774 INFO:tasks.ceph:Cleaning ceph cluster... 2024-07-04T17:38:02.775 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-07-04T17:38:02.944 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-07-04T17:38:02.944 INFO:tasks.ceph:Archiving crash dumps... 2024-07-04T17:38:02.946 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-07-04_16:36:07-fs-wip-rishabh-testing-20240701.141150-debug-testing-default-smithi/7787245/remote/smithi042/crash 2024-07-04T17:38:02.947 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-07-04T17:38:03.021 INFO:tasks.ceph:Shutting down logrotate 2024-07-04T17:38:03.021 DEBUG:teuthology.orchestra.run.smithi042:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-07-04T17:38:03.128 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-07-04T17:38:03.204 INFO:tasks.ceph:Compressing logs... 2024-07-04T17:38:03.204 DEBUG:teuthology.orchestra.run.smithi042:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-07-04T17:38:03.275 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71318.log 2024-07-04T17:38:03.276 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58439.log 2024-07-04T17:38:03.277 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.71318.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.71318.log.gz 2024-07-04T17:38:03.278 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65171.log 2024-07-04T17:38:03.278 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58018.log 2024-07-04T17:38:03.279 INFO:teuthology.orchestra.run.smithi042.stderr: 83.5% -- replaced with /var/log/ceph/ceph-client.admin.58439.log.gz 2024-07-04T17:38:03.279 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.65171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53535.log 2024-07-04T17:38:03.280 INFO:teuthology.orchestra.run.smithi042.stderr: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.65171.log.gz 2024-07-04T17:38:03.280 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58018.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.58018.log.gz 2024-07-04T17:38:03.281 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44026.log 2024-07-04T17:38:03.281 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71137.log 85.5% 2024-07-04T17:38:03.282 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53535.log.gz 2024-07-04T17:38:03.282 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49677.log 2024-07-04T17:38:03.283 INFO:teuthology.orchestra.run.smithi042.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.44026.log.gz 2024-07-04T17:38:03.283 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.71137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54576.log 2024-07-04T17:38:03.283 INFO:teuthology.orchestra.run.smithi042.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.71137.log.gz 2024-07-04T17:38:03.284 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49196.log 2024-07-04T17:38:03.284 INFO:teuthology.orchestra.run.smithi042.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.49677.log.gz 2024-07-04T17:38:03.285 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60402.log 2024-07-04T17:38:03.285 INFO:teuthology.orchestra.run.smithi042.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.54576.log.gz 2024-07-04T17:38:03.286 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57330.log 85.1% 2024-07-04T17:38:03.286 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49196.log.gz 2024-07-04T17:38:03.287 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40706.log 2024-07-04T17:38:03.287 INFO:teuthology.orchestra.run.smithi042.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.60402.log.gz 2024-07-04T17:38:03.288 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50451.log 2024-07-04T17:38:03.288 INFO:teuthology.orchestra.run.smithi042.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.57330.log.gz 2024-07-04T17:38:03.289 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68926.log 2024-07-04T17:38:03.289 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.40706.log.gz 2024-07-04T17:38:03.290 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66840.log 2024-07-04T17:38:03.290 INFO:teuthology.orchestra.run.smithi042.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.50451.log.gz 2024-07-04T17:38:03.291 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.68926.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.68926.log.gz 2024-07-04T17:38:03.291 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43229.log 2024-07-04T17:38:03.292 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.66840.log: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.66840.log.gz 2024-07-04T17:38:03.292 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66612.log 2024-07-04T17:38:03.293 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.1.43229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74466.log 2024-07-04T17:38:03.293 INFO:teuthology.orchestra.run.smithi042.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.1.43229.log.gz 2024-07-04T17:38:03.294 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.66612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61987.log 2024-07-04T17:38:03.294 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.66612.log.gz 2024-07-04T17:38:03.295 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.74466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65087.log 2024-07-04T17:38:03.295 INFO:teuthology.orchestra.run.smithi042.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.74466.log.gz 2024-07-04T17:38:03.296 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.61987.log: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.61987.log.gz 2024-07-04T17:38:03.296 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49635.log 2024-07-04T17:38:03.298 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.65087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60830.log 2024-07-04T17:38:03.298 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.65087.log.gz 2024-07-04T17:38:03.298 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66301.log 2024-07-04T17:38:03.298 INFO:teuthology.orchestra.run.smithi042.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.49635.log.gz 2024-07-04T17:38:03.299 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.0.60830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71824.log 2024-07-04T17:38:03.300 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.66301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36336.log 2024-07-04T17:38:03.300 INFO:teuthology.orchestra.run.smithi042.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.66301.log.gz 2024-07-04T17:38:03.300 INFO:teuthology.orchestra.run.smithi042.stderr: 93.4% -- replaced with /var/log/ceph/ceph-client.0.60830.log.gz 2024-07-04T17:38:03.301 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.71824.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.71824.log.gz 2024-07-04T17:38:03.301 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51270.log 2024-07-04T17:38:03.302 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69440.log 2024-07-04T17:38:03.302 INFO:teuthology.orchestra.run.smithi042.stderr: 73.1% -- replaced with /var/log/ceph/ceph-client.admin.36336.log.gz 2024-07-04T17:38:03.303 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.0.51270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36731.log 2024-07-04T17:38:03.304 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.69440.log: gzip 85.1% -- replaced with /var/log/ceph/ceph-client.admin.69440.log.gz -5 2024-07-04T17:38:03.304 INFO:teuthology.orchestra.run.smithi042.stderr: --verbose -- /var/log/ceph/ceph-client.admin.57381.log 2024-07-04T17:38:03.305 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55469.log 2024-07-04T17:38:03.305 INFO:teuthology.orchestra.run.smithi042.stderr: 83.3% -- replaced with /var/log/ceph/ceph-client.admin.36731.log.gz 2024-07-04T17:38:03.306 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57381.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.57381.log.gz 2024-07-04T17:38:03.306 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47113.log 2024-07-04T17:38:03.307 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43495.log 2024-07-04T17:38:03.308 INFO:teuthology.orchestra.run.smithi042.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.55469.log.gz 2024-07-04T17:38:03.308 INFO:teuthology.orchestra.run.smithi042.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.0.51270.log.gz 2024-07-04T17:38:03.309 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47113.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.47113.log.gz 2024-07-04T17:38:03.309 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53978.log 2024-07-04T17:38:03.309 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54805.log 2024-07-04T17:38:03.310 INFO:teuthology.orchestra.run.smithi042.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.43495.log.gz 2024-07-04T17:38:03.310 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74424.log 2024-07-04T17:38:03.310 INFO:teuthology.orchestra.run.smithi042.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.53978.log.gz 2024-07-04T17:38:03.311 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54805.log: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.54805.log.gz 2024-07-04T17:38:03.311 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52325.log 2024-07-04T17:38:03.312 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.74424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73335.log 2024-07-04T17:38:03.313 INFO:teuthology.orchestra.run.smithi042.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.74424.log.gz 2024-07-04T17:38:03.313 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63817.log 2024-07-04T17:38:03.314 INFO:teuthology.orchestra.run.smithi042.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.52325.log.gz 2024-07-04T17:38:03.315 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.73335.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.73335.log.gz 2024-07-04T17:38:03.315 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45444.log 2024-07-04T17:38:03.316 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.63817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50758.log 2024-07-04T17:38:03.316 INFO:teuthology.orchestra.run.smithi042.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.63817.log.gz 2024-07-04T17:38:03.317 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56092.log 2024-07-04T17:38:03.317 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.45444.log.gz 2024-07-04T17:38:03.318 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50758.log: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.50758.log.gz 2024-07-04T17:38:03.318 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44187.log 2024-07-04T17:38:03.319 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46742.log 2024-07-04T17:38:03.319 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.56092.log.gz 2024-07-04T17:38:03.320 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41676.log 2024-07-04T17:38:03.320 INFO:teuthology.orchestra.run.smithi042.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.44187.log.gz 2024-07-04T17:38:03.321 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46742.log: gzip -5 85.4% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.46742.log.gz /var/log/ceph/ceph-client.admin.48570.log 2024-07-04T17:38:03.321 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-07-04T17:38:03.322 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63027.log 2024-07-04T17:38:03.322 INFO:teuthology.orchestra.run.smithi042.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.41676.log.gz 2024-07-04T17:38:03.323 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63523.log 2024-07-04T17:38:03.323 INFO:teuthology.orchestra.run.smithi042.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.48570.log.gz 2024-07-04T17:38:03.324 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.63027.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.63027.log.gz 2024-07-04T17:38:03.324 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67583.log 2024-07-04T17:38:03.325 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.63523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45402.log 2024-07-04T17:38:03.325 INFO:teuthology.orchestra.run.smithi042.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.63523.log.gz 2024-07-04T17:38:03.326 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.67583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58637.log 2024-07-04T17:38:03.326 INFO:teuthology.orchestra.run.smithi042.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.67583.log.gz 2024-07-04T17:38:03.327 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45402.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.45402.log.gz 2024-07-04T17:38:03.327 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58827.log 2024-07-04T17:38:03.328 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44881.log 2024-07-04T17:38:03.329 INFO:teuthology.orchestra.run.smithi042.stderr: 83.1% -- replaced with /var/log/ceph/ceph-client.admin.58637.log.gz 2024-07-04T17:38:03.330 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65363.log 2024-07-04T17:38:03.330 INFO:teuthology.orchestra.run.smithi042.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.58827.log.gz 2024-07-04T17:38:03.331 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44881.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.44881.log.gz 2024-07-04T17:38:03.331 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64960.log 2024-07-04T17:38:03.332 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.65363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65129.log 2024-07-04T17:38:03.332 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.65363.log.gz 2024-07-04T17:38:03.333 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.64960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73847.log 2024-07-04T17:38:03.333 INFO:teuthology.orchestra.run.smithi042.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.64960.log.gz 2024-07-04T17:38:03.334 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.65129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45570.log 2024-07-04T17:38:03.334 INFO:teuthology.orchestra.run.smithi042.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.65129.log.gz 2024-07-04T17:38:03.335 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.73847.log: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.73847.log.gzgzip 2024-07-04T17:38:03.335 INFO:teuthology.orchestra.run.smithi042.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48826.log 2024-07-04T17:38:03.336 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41718.log 2024-07-04T17:38:03.336 INFO:teuthology.orchestra.run.smithi042.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.45570.log.gz 2024-07-04T17:38:03.337 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66714.log 2024-07-04T17:38:03.337 INFO:teuthology.orchestra.run.smithi042.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.48826.log.gz 2024-07-04T17:38:03.338 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41718.log: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.41718.log.gz 2024-07-04T17:38:03.338 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57721.log 2024-07-04T17:38:03.339 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.66714.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-07-04T17:38:03.339 INFO:teuthology.orchestra.run.smithi042.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.66714.log.gz 2024-07-04T17:38:03.340 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61618.log 2024-07-04T17:38:03.340 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.57721.log.gz 2024-07-04T17:38:03.341 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52367.log 2024-07-04T17:38:03.344 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.61618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45221.log 2024-07-04T17:38:03.344 INFO:teuthology.orchestra.run.smithi042.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.61618.log.gz 2024-07-04T17:38:03.344 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52367.log: 87.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-07-04T17:38:03.344 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48658.log 2024-07-04T17:38:03.345 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.52367.log.gz 2024-07-04T17:38:03.348 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62179.log 2024-07-04T17:38:03.348 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.45221.log.gz 2024-07-04T17:38:03.348 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54847.log 2024-07-04T17:38:03.348 INFO:teuthology.orchestra.run.smithi042.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.48658.log.gz 2024-07-04T17:38:03.349 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.62179.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.62179.log.gz 2024-07-04T17:38:03.349 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67693.log 2024-07-04T17:38:03.349 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44355.log 2024-07-04T17:38:03.349 INFO:teuthology.orchestra.run.smithi042.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.54847.log.gz 2024-07-04T17:38:03.353 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.67693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61574.log 2024-07-04T17:38:03.353 INFO:teuthology.orchestra.run.smithi042.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.67693.log.gz 2024-07-04T17:38:03.353 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44355.log: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.44355.log.gzgzip 2024-07-04T17:38:03.354 INFO:teuthology.orchestra.run.smithi042.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.71095.log 2024-07-04T17:38:03.354 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.61574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42819.log 2024-07-04T17:38:03.354 INFO:teuthology.orchestra.run.smithi042.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.61574.log.gz 2024-07-04T17:38:03.354 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.71095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48341.log 2024-07-04T17:38:03.355 INFO:teuthology.orchestra.run.smithi042.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.71095.log.gz 2024-07-04T17:38:03.355 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42819.log: 85.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.42819.log.gz -5 2024-07-04T17:38:03.355 INFO:teuthology.orchestra.run.smithi042.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53852.log 2024-07-04T17:38:03.357 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49982.log 2024-07-04T17:38:03.358 INFO:teuthology.orchestra.run.smithi042.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.48341.log.gz 2024-07-04T17:38:03.358 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68319.log 2024-07-04T17:38:03.358 INFO:teuthology.orchestra.run.smithi042.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.53852.log.gz 2024-07-04T17:38:03.359 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49982.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.49982.log.gz 2024-07-04T17:38:03.359 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74130.log 2024-07-04T17:38:03.359 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.68319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40062.log 2024-07-04T17:38:03.359 INFO:teuthology.orchestra.run.smithi042.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.68319.log.gz 2024-07-04T17:38:03.361 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.74130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45696.log 2024-07-04T17:38:03.361 INFO:teuthology.orchestra.run.smithi042.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.74130.log.gz 2024-07-04T17:38:03.361 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40062.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.40062.log.gz 2024-07-04T17:38:03.361 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68163.log 2024-07-04T17:38:03.361 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70007.log 2024-07-04T17:38:03.362 INFO:teuthology.orchestra.run.smithi042.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.45696.log.gz 2024-07-04T17:38:03.362 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.68163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63565.log 2024-07-04T17:38:03.363 INFO:teuthology.orchestra.run.smithi042.stderr: 82.9% -- replaced with /var/log/ceph/ceph-client.admin.68163.log.gz 2024-07-04T17:38:03.363 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.70007.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.70007.log.gz 2024-07-04T17:38:03.363 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63775.log 2024-07-04T17:38:03.364 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.63565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45360.log 2024-07-04T17:38:03.364 INFO:teuthology.orchestra.run.smithi042.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.63565.log.gz 2024-07-04T17:38:03.365 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.63775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49082.log 2024-07-04T17:38:03.365 INFO:teuthology.orchestra.run.smithi042.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.63775.log.gz 2024-07-04T17:38:03.365 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45360.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.45360.log.gz 2024-07-04T17:38:03.366 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59695.log 2024-07-04T17:38:03.366 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50363.log 2024-07-04T17:38:03.367 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.49082.log.gz 2024-07-04T17:38:03.367 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44923.log 2024-07-04T17:38:03.368 INFO:teuthology.orchestra.run.smithi042.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.59695.log.gz 2024-07-04T17:38:03.369 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50363.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.50363.log.gz 2024-07-04T17:38:03.369 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69530.log 2024-07-04T17:38:03.370 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58247.log 2024-07-04T17:38:03.370 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.44923.log.gz 2024-07-04T17:38:03.372 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.69530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59463.log 2024-07-04T17:38:03.372 INFO:teuthology.orchestra.run.smithi042.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.69530.log.gz 2024-07-04T17:38:03.372 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58247.log: 85.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.58247.log.gz -5 2024-07-04T17:38:03.372 INFO:teuthology.orchestra.run.smithi042.stderr: --verbose -- /var/log/ceph/ceph-osd.2.log 2024-07-04T17:38:03.373 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71742.log 2024-07-04T17:38:03.373 INFO:teuthology.orchestra.run.smithi042.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.59463.log.gz 2024-07-04T17:38:03.374 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71574.log 2024-07-04T17:38:03.375 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.71742.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.71742.log.gz 2024-07-04T17:38:03.375 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66672.log 2024-07-04T17:38:03.376 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.71574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43800.log 2024-07-04T17:38:03.376 INFO:teuthology.orchestra.run.smithi042.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.71574.log.gz 2024-07-04T17:38:03.377 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.66672.log: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.66672.log.gz 2024-07-04T17:38:03.378 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41584.log 2024-07-04T17:38:03.378 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58289.log 2024-07-04T17:38:03.379 INFO:teuthology.orchestra.run.smithi042.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.43800.log.gz 2024-07-04T17:38:03.379 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41584.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.41584.log.gz 2024-07-04T17:38:03.380 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64046.log 2024-07-04T17:38:03.380 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68758.log 2024-07-04T17:38:03.381 INFO:teuthology.orchestra.run.smithi042.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.58289.log.gz 2024-07-04T17:38:03.382 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.64046.log: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.64046.log.gz 2024-07-04T17:38:03.382 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64088.log 2024-07-04T17:38:03.383 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.68758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55511.log 2024-07-04T17:38:03.383 INFO:teuthology.orchestra.run.smithi042.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.68758.log.gz 2024-07-04T17:38:03.384 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.64088.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.64088.log.gz 2024-07-04T17:38:03.384 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49154.log 2024-07-04T17:38:03.385 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71532.log 2024-07-04T17:38:03.385 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.55511.log.gz 2024-07-04T17:38:03.386 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49154.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.49154.log.gz 2024-07-04T17:38:03.386 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66990.log 2024-07-04T17:38:03.387 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.71532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53384.log 2024-07-04T17:38:03.388 INFO:teuthology.orchestra.run.smithi042.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.71532.log.gz 2024-07-04T17:38:03.388 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.66990.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.66990.log.gz 2024-07-04T17:38:03.388 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37276.log 2024-07-04T17:38:03.389 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5/var/log/ceph/ceph-client.admin.53384.log: --verbose -- /var/log/ceph/ceph-client.admin.41298.log 2024-07-04T17:38:03.390 INFO:teuthology.orchestra.run.smithi042.stderr: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.53384.log.gz 2024-07-04T17:38:03.390 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37276.log: 83.5% -- replaced with /var/log/ceph/ceph-client.admin.37276.log.gz 2024-07-04T17:38:03.391 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59253.log 2024-07-04T17:38:03.391 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63649.log 2024-07-04T17:38:03.392 INFO:teuthology.orchestra.run.smithi042.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.41298.log.gz 2024-07-04T17:38:03.393 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59253.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.59253.log.gz 2024-07-04T17:38:03.393 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70393.log 2024-07-04T17:38:03.394 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.63649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49490.log 2024-07-04T17:38:03.394 INFO:teuthology.orchestra.run.smithi042.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.63649.log.gz 2024-07-04T17:38:03.395 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.0.70393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46040.log 2024-07-04T17:38:03.397 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50590.log 2024-07-04T17:38:03.397 INFO:teuthology.orchestra.run.smithi042.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.49490.log.gz 2024-07-04T17:38:03.398 INFO:teuthology.orchestra.run.smithi042.stderr: 94.9% -- replaced with /var/log/ceph/ceph-client.0.70393.log.gz 2024-07-04T17:38:03.398 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.0.46040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52413.log 2024-07-04T17:38:03.399 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50590.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.50590.log.gz 2024-07-04T17:38:03.400 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44965.log 2024-07-04T17:38:03.401 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65255.log 2024-07-04T17:38:03.402 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52413.log: /var/log/ceph/ceph-client.admin.44965.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.44965.log.gz 2024-07-04T17:38:03.402 INFO:teuthology.orchestra.run.smithi042.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.52413.log.gz 2024-07-04T17:38:03.402 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72812.log 2024-07-04T17:38:03.403 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49364.log 2024-07-04T17:38:03.404 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.65255.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.65255.log.gz 2024-07-04T17:38:03.404 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.72812.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.72812.log.gz 2024-07-04T17:38:03.404 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51461.log 2024-07-04T17:38:03.406 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50800.log 2024-07-04T17:38:03.407 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49364.log: /var/log/ceph/ceph-client.admin.51461.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.49364.log.gz 2024-07-04T17:38:03.407 INFO:teuthology.orchestra.run.smithi042.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.51461.log.gz 2024-07-04T17:38:03.407 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63962.log 2024-07-04T17:38:03.408 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50800.log: 93.6% -- replaced with /var/log/ceph/ceph-client.0.46040.log.gz 2024-07-04T17:38:03.408 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41337.log 2024-07-04T17:38:03.409 INFO:teuthology.orchestra.run.smithi042.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.50800.log.gz 2024-07-04T17:38:03.409 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.63962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72935.log 2024-07-04T17:38:03.409 INFO:teuthology.orchestra.run.smithi042.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.63962.log.gz 2024-07-04T17:38:03.410 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60274.log 2024-07-04T17:38:03.410 INFO:teuthology.orchestra.run.smithi042.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.41337.log.gz 2024-07-04T17:38:03.411 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.72935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68800.log 2024-07-04T17:38:03.411 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.72935.log.gz 2024-07-04T17:38:03.412 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60274.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-07-04T17:38:03.413 INFO:teuthology.orchestra.run.smithi042.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.60274.log.gz 2024-07-04T17:38:03.414 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.68800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36554.log 2024-07-04T17:38:03.414 INFO:teuthology.orchestra.run.smithi042.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.68800.log.gz 2024-07-04T17:38:03.415 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58397.log 2024-07-04T17:38:03.415 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36554.log: 83.0% -- replaced with /var/log/ceph/ceph-client.admin.36554.log.gz 2024-07-04T17:38:03.416 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67823.log 2024-07-04T17:38:03.417 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47323.log 2024-07-04T17:38:03.418 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58397.log: /var/log/ceph/ceph-client.admin.67823.log: 83.3% -- replaced with /var/log/ceph/ceph-client.admin.58397.log.gz 2024-07-04T17:38:03.418 INFO:teuthology.orchestra.run.smithi042.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.67823.log.gz 2024-07-04T17:38:03.419 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69242.log 2024-07-04T17:38:03.419 INFO:teuthology.orchestra.run.smithi042.stderr: 89.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-07-04T17:38:03.419 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62071.log 2024-07-04T17:38:03.420 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47323.log: /var/log/ceph/ceph-client.admin.69242.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47323.log.gz 2024-07-04T17:38:03.420 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.69242.log.gz 2024-07-04T17:38:03.421 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49845.log 2024-07-04T17:38:03.421 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.62071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68445.log 2024-07-04T17:38:03.421 INFO:teuthology.orchestra.run.smithi042.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.62071.log.gz 2024-07-04T17:38:03.422 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41818.log 2024-07-04T17:38:03.423 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.49845.log.gz 2024-07-04T17:38:03.423 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.68445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44271.log 2024-07-04T17:38:03.424 INFO:teuthology.orchestra.run.smithi042.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.68445.log.gz 2024-07-04T17:38:03.425 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44500.log 2024-07-04T17:38:03.425 INFO:teuthology.orchestra.run.smithi042.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.41818.log.gz 2024-07-04T17:38:03.425 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72458.log 2024-07-04T17:38:03.426 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.44271.log.gz 2024-07-04T17:38:03.426 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.mirror_remote.42081.log 2024-07-04T17:38:03.427 INFO:teuthology.orchestra.run.smithi042.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.44500.log.gz 2024-07-04T17:38:03.428 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.72458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71490.log 2024-07-04T17:38:03.428 INFO:teuthology.orchestra.run.smithi042.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.72458.log.gz 2024-07-04T17:38:03.429 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.mirror_remote.42081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41904.log 2024-07-04T17:38:03.431 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.71490.log: 93.4% -- replaced with /var/log/ceph/ceph-client.mirror_remote.42081.log.gz 2024-07-04T17:38:03.431 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59295.log 2024-07-04T17:38:03.431 INFO:teuthology.orchestra.run.smithi042.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.71490.log.gz 2024-07-04T17:38:03.432 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74568.log 2024-07-04T17:38:03.432 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.41904.log.gz 2024-07-04T17:38:03.433 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59295.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.59295.log.gz 2024-07-04T17:38:03.433 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65045.log 2024-07-04T17:38:03.433 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.74568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.66110.log 2024-07-04T17:38:03.434 INFO:teuthology.orchestra.run.smithi042.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.74568.log.gz 2024-07-04T17:38:03.435 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.65045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41440.log 2024-07-04T17:38:03.435 INFO:teuthology.orchestra.run.smithi042.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.65045.log.gz 2024-07-04T17:38:03.435 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.1.66110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64914.log 2024-07-04T17:38:03.437 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41440.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.41440.log.gz 2024-07-04T17:38:03.437 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65003.log 2024-07-04T17:38:03.437 INFO:teuthology.orchestra.run.smithi042.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.1.66110.log.gz 2024-07-04T17:38:03.438 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47071.log 2024-07-04T17:38:03.439 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.64914.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.64914.log.gz 2024-07-04T17:38:03.439 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.65003.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.65003.log.gz 2024-07-04T17:38:03.439 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68487.log 2024-07-04T17:38:03.440 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46784.log 2024-07-04T17:38:03.441 INFO:teuthology.orchestra.run.smithi042.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.47071.log.gz 2024-07-04T17:38:03.441 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.68487.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.68487.log.gz 2024-07-04T17:38:03.442 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49887.log 2024-07-04T17:38:03.442 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69923.log 2024-07-04T17:38:03.442 INFO:teuthology.orchestra.run.smithi042.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.46784.log.gz 2024-07-04T17:38:03.443 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37028.log 2024-07-04T17:38:03.444 INFO:teuthology.orchestra.run.smithi042.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.49887.log.gz 2024-07-04T17:38:03.444 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.69923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48460.log 2024-07-04T17:38:03.445 INFO:teuthology.orchestra.run.smithi042.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.69923.log.gz 2024-07-04T17:38:03.446 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58492.log 2024-07-04T17:38:03.446 INFO:teuthology.orchestra.run.smithi042.stderr: 83.4% -- replaced with /var/log/ceph/ceph-client.admin.37028.log.gz 2024-07-04T17:38:03.447 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48460.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-07-04T17:38:03.447 INFO:teuthology.orchestra.run.smithi042.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.48460.log.gz 2024-07-04T17:38:03.448 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40878.log 2024-07-04T17:38:03.448 INFO:teuthology.orchestra.run.smithi042.stderr: 83.0% -- replaced with /var/log/ceph/ceph-client.admin.58492.log.gz 2024-07-04T17:38:03.449 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62558.log 2024-07-04T17:38:03.450 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40878.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.40878.log.gz 2024-07-04T17:38:03.450 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44458.log 2024-07-04T17:38:03.451 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.61299.log 2024-07-04T17:38:03.452 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.62558.log: /var/log/ceph/ceph-client.admin.44458.log: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.62558.log.gz 2024-07-04T17:38:03.453 INFO:teuthology.orchestra.run.smithi042.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.44458.log.gz 2024-07-04T17:38:03.453 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50321.log 2024-07-04T17:38:03.453 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49448.log 2024-07-04T17:38:03.455 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.1.61299.log: /var/log/ceph/ceph-client.admin.50321.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.50321.log.gz 2024-07-04T17:38:03.455 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40146.log 2024-07-04T17:38:03.456 INFO:teuthology.orchestra.run.smithi042.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.1.61299.log.gz 2024-07-04T17:38:03.456 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69625.log 2024-07-04T17:38:03.457 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49448.log: /var/log/ceph/ceph-client.admin.40146.log: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.49448.log.gz 2024-07-04T17:38:03.457 INFO:teuthology.orchestra.run.smithi042.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.40146.log.gz 2024-07-04T17:38:03.457 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40441.log 2024-07-04T17:38:03.458 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41946.log 2024-07-04T17:38:03.459 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.69625.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.69625.log.gz 2024-07-04T17:38:03.459 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40441.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.40441.log.gz 2024-07-04T17:38:03.459 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54390.log 2024-07-04T17:38:03.460 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55099.log 2024-07-04T17:38:03.461 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41946.log: /var/log/ceph/ceph-client.admin.54390.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.41946.log.gz 2024-07-04T17:38:03.462 INFO:teuthology.orchestra.run.smithi042.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.54390.log.gz 2024-07-04T17:38:03.462 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45486.log 2024-07-04T17:38:03.463 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55826.log 2024-07-04T17:38:03.463 INFO:teuthology.orchestra.run.smithi042.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.55099.log.gz 2024-07-04T17:38:03.464 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45486.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.45486.log.gz 2024-07-04T17:38:03.464 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40962.log 2024-07-04T17:38:03.465 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.mirror_remote.36173.log 2024-07-04T17:38:03.466 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55826.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.55826.log.gz 2024-07-04T17:38:03.466 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40962.log: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.40962.log.gz 2024-07-04T17:38:03.466 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60444.log 2024-07-04T17:38:03.467 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69667.log 2024-07-04T17:38:03.468 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.mirror_remote.36173.log: /var/log/ceph/ceph-client.admin.60444.log: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.60444.log.gz 2024-07-04T17:38:03.468 INFO:teuthology.orchestra.run.smithi042.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.mirror_remote.36173.log.gz 2024-07-04T17:38:03.468 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46872.log 2024-07-04T17:38:03.469 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47938.log 2024-07-04T17:38:03.470 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.69667.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.69667.log.gz 2024-07-04T17:38:03.470 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46872.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.46872.log.gz 2024-07-04T17:38:03.471 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62029.log 2024-07-04T17:38:03.474 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63195.log 2024-07-04T17:38:03.474 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47938.log: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.47938.log.gz 2024-07-04T17:38:03.475 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.62029.log: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.62029.log.gz 2024-07-04T17:38:03.475 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.46509.log 2024-07-04T17:38:03.475 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.63195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55331.log 2024-07-04T17:38:03.475 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.63195.log.gz 2024-07-04T17:38:03.476 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.1.46509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71364.log 2024-07-04T17:38:03.476 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55331.log: 94.0% -- replaced with /var/log/ceph/ceph-client.1.46509.log.gz 2024-07-04T17:38:03.477 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.51739.log 2024-07-04T17:38:03.477 INFO:teuthology.orchestra.run.smithi042.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.55331.log.gz 2024-07-04T17:38:03.477 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.71364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61861.log 2024-07-04T17:38:03.478 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.71364.log.gz 2024-07-04T17:38:03.479 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.1.51739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62829.log 2024-07-04T17:38:03.480 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.61861.log: 93.6% -- replaced with /var/log/ceph/ceph-client.1.51739.log.gz 2024-07-04T17:38:03.480 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53442.log 2024-07-04T17:38:03.480 INFO:teuthology.orchestra.run.smithi042.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.61861.log.gz 2024-07-04T17:38:03.481 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.62829.log: -- /var/log/ceph/ceph-client.admin.71225.log 2024-07-04T17:38:03.481 INFO:teuthology.orchestra.run.smithi042.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.62829.log.gz 2024-07-04T17:38:03.482 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53442.log: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.53442.log.gz 2024-07-04T17:38:03.482 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74382.log 2024-07-04T17:38:03.483 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55955.log 2024-07-04T17:38:03.484 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.71225.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.71225.log.gz 2024-07-04T17:38:03.484 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.74382.log: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.74382.log.gz 2024-07-04T17:38:03.484 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44103.log 2024-07-04T17:38:03.485 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41046.log 2024-07-04T17:38:03.486 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55955.log: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.55955.log.gz 2024-07-04T17:38:03.486 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44103.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.44103.log.gz 2024-07-04T17:38:03.487 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41130.log 2024-07-04T17:38:03.487 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57511.log 2024-07-04T17:38:03.488 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41046.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.41046.log.gz 2024-07-04T17:38:03.488 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41130.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.41130.log.gz 2024-07-04T17:38:03.489 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59337.log 2024-07-04T17:38:03.490 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40299.log 2024-07-04T17:38:03.490 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57511.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.57511.log.gz 2024-07-04T17:38:03.491 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59337.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.59337.log.gz 2024-07-04T17:38:03.491 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70115.log 2024-07-04T17:38:03.492 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64369.log 2024-07-04T17:38:03.492 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.40299.log.gz 2024-07-04T17:38:03.493 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.70115.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.70115.log.gz 2024-07-04T17:38:03.493 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59048.log 2024-07-04T17:38:03.494 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71448.log 2024-07-04T17:38:03.495 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.64369.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.64369.log.gz 2024-07-04T17:38:03.495 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59048.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.59048.log.gz 2024-07-04T17:38:03.495 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57242.log 2024-07-04T17:38:03.496 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64274.log 2024-07-04T17:38:03.497 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.71448.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.71448.log.gz 2024-07-04T17:38:03.497 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57242.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.57242.log.gz 2024-07-04T17:38:03.497 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42148.log 2024-07-04T17:38:03.498 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50926.log/var/log/ceph/ceph-client.admin.64274.log: 2024-07-04T17:38:03.499 INFO:teuthology.orchestra.run.smithi042.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.64274.log.gz 2024-07-04T17:38:03.500 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42148.log: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.42148.log.gz 2024-07-04T17:38:03.500 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71406.log 2024-07-04T17:38:03.500 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50674.log 2024-07-04T17:38:03.501 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50926.log: /var/log/ceph/ceph-client.admin.71406.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.50926.log.gz 2024-07-04T17:38:03.502 INFO:teuthology.orchestra.run.smithi042.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.71406.log.gz 2024-07-04T17:38:03.502 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61490.log 2024-07-04T17:38:03.503 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50674.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.50674.log.gz 2024-07-04T17:38:03.503 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68205.log 2024-07-04T17:38:03.504 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.61490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64872.log 2024-07-04T17:38:03.505 INFO:teuthology.orchestra.run.smithi042.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.61490.log.gz 2024-07-04T17:38:03.505 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.68205.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.68205.log.gz 2024-07-04T17:38:03.505 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-07-04T17:38:03.507 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.64872.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.64872.log.gz 2024-07-04T17:38:03.507 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49280.log 2024-07-04T17:38:03.508 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68571.log 2024-07-04T17:38:03.509 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66756.log 2024-07-04T17:38:03.509 INFO:teuthology.orchestra.run.smithi042.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.49280.log.gz 2024-07-04T17:38:03.511 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.68571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74298.log 2024-07-04T17:38:03.511 INFO:teuthology.orchestra.run.smithi042.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.68571.log.gz 2024-07-04T17:38:03.512 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.66756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54432.log 2024-07-04T17:38:03.512 INFO:teuthology.orchestra.run.smithi042.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.66756.log.gz 2024-07-04T17:38:03.513 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.74298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73701.log 2024-07-04T17:38:03.514 INFO:teuthology.orchestra.run.smithi042.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.74298.log.gz 2024-07-04T17:38:03.514 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37214.log 2024-07-04T17:38:03.515 INFO:teuthology.orchestra.run.smithi042.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.54432.log.gz 2024-07-04T17:38:03.516 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.73701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36216.log 2024-07-04T17:38:03.516 INFO:teuthology.orchestra.run.smithi042.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.73701.log.gz 2024-07-04T17:38:03.517 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60009.log 2024-07-04T17:38:03.518 INFO:teuthology.orchestra.run.smithi042.stderr: 83.6% -- replaced with /var/log/ceph/ceph-client.admin.37214.log.gz 2024-07-04T17:38:03.518 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36216.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36216.log.gz 2024-07-04T17:38:03.518 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74088.log 2024-07-04T17:38:03.520 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57156.log 2024-07-04T17:38:03.520 INFO:teuthology.orchestra.run.smithi042.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.60009.log.gz 2024-07-04T17:38:03.521 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.74088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54889.log 2024-07-04T17:38:03.521 INFO:teuthology.orchestra.run.smithi042.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.74088.log.gz 2024-07-04T17:38:03.522 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53894.log 2024-07-04T17:38:03.522 INFO:teuthology.orchestra.run.smithi042.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.57156.log.gz 2024-07-04T17:38:03.524 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53269.log 2024-07-04T17:38:03.524 INFO:teuthology.orchestra.run.smithi042.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.54889.log.gz 2024-07-04T17:38:03.525 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53894.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.53894.log.gz 2024-07-04T17:38:03.525 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51930.log 2024-07-04T17:38:03.526 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53269.log: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.53269.log.gz 2024-07-04T17:38:03.526 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55911.log 2024-07-04T17:38:03.528 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40836.log 2024-07-04T17:38:03.528 INFO:teuthology.orchestra.run.smithi042.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.51930.log.gz 2024-07-04T17:38:03.529 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55911.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.55911.log.gz 2024-07-04T17:38:03.529 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-07-04T17:38:03.531 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74340.log 2024-07-04T17:38:03.531 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.40836.log.gz 2024-07-04T17:38:03.532 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41458.log 2024-07-04T17:38:03.534 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.74340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52014.log 2024-07-04T17:38:03.534 INFO:teuthology.orchestra.run.smithi042.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.74340.log.gz 2024-07-04T17:38:03.535 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44397.log 2024-07-04T17:38:03.535 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.41458.log.gz 2024-07-04T17:38:03.536 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73275.log 2024-07-04T17:38:03.536 INFO:teuthology.orchestra.run.smithi042.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.52014.log.gz 2024-07-04T17:38:03.538 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63733.log 2024-07-04T17:38:03.538 INFO:teuthology.orchestra.run.smithi042.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.44397.log.gz 2024-07-04T17:38:03.539 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.73275.log: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.73275.log.gz 2024-07-04T17:38:03.539 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58331.log 2024-07-04T17:38:03.540 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.63733.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.63733.log.gz 2024-07-04T17:38:03.540 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63607.log 2024-07-04T17:38:03.542 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36171.log 2024-07-04T17:38:03.542 INFO:teuthology.orchestra.run.smithi042.stderr: 83.3% -- replaced with /var/log/ceph/ceph-client.admin.58331.log.gz 2024-07-04T17:38:03.543 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.63607.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.63607.log.gz 2024-07-04T17:38:03.543 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36195.log 2024-07-04T17:38:03.544 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36171.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36171.log.gz 2024-07-04T17:38:03.544 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55645.log 2024-07-04T17:38:03.546 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36195.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36195.log.gzgzip 2024-07-04T17:38:03.546 INFO:teuthology.orchestra.run.smithi042.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.73019.log 2024-07-04T17:38:03.547 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68613.log 2024-07-04T17:38:03.547 INFO:teuthology.orchestra.run.smithi042.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.55645.log.gz 2024-07-04T17:38:03.548 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.73019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60552.log 2024-07-04T17:38:03.548 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.73019.log.gz 2024-07-04T17:38:03.550 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.68613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48742.log 2024-07-04T17:38:03.550 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.68613.log.gz 2024-07-04T17:38:03.551 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60552.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.60552.log.gz 2024-07-04T17:38:03.552 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60102.log 2024-07-04T17:38:03.552 INFO:teuthology.orchestra.run.smithi042.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.48742.log.gz 2024-07-04T17:38:03.554 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-07-04T17:38:03.555 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41988.log 2024-07-04T17:38:03.555 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.60102.log.gz 2024-07-04T17:38:03.556 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53033.log 2024-07-04T17:38:03.558 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41988.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.41988.log.gz 2024-07-04T17:38:03.559 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53033.log: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.53033.log.gz 2024-07-04T17:38:03.559 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72695.log 2024-07-04T17:38:03.561 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72977.log 2024-07-04T17:38:03.562 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.72695.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.72695.log.gz 2024-07-04T17:38:03.562 INFO:teuthology.orchestra.run.smithi042.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-07-04T17:38:03.562 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41590.log 2024-07-04T17:38:03.564 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.72977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67865.log 2024-07-04T17:38:03.564 INFO:teuthology.orchestra.run.smithi042.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.72977.log.gz 2024-07-04T17:38:03.564 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41590.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.41590.log.gz 2024-07-04T17:38:03.565 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62616.log 2024-07-04T17:38:03.566 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55557.log 2024-07-04T17:38:03.566 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.67865.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.67865.log.gz 2024-07-04T17:38:03.567 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.62616.log: gzip -5 --verbose -- 84.7% -- replaced with /var/log/ceph/ceph-client.admin.62616.log.gz 2024-07-04T17:38:03.568 INFO:teuthology.orchestra.run.smithi042.stderr: /var/log/ceph/ceph-client.admin.65297.log 2024-07-04T17:38:03.568 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55557.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.55557.log.gz 2024-07-04T17:38:03.569 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68361.log 2024-07-04T17:38:03.570 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.65297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57286.log 2024-07-04T17:38:03.571 INFO:teuthology.orchestra.run.smithi042.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.65297.log.gz 2024-07-04T17:38:03.571 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.68361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50217.log 2024-07-04T17:38:03.571 INFO:teuthology.orchestra.run.smithi042.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.68361.log.gz 2024-07-04T17:38:03.573 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64232.log 2024-07-04T17:38:03.573 INFO:teuthology.orchestra.run.smithi042.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.57286.log.gz 2024-07-04T17:38:03.574 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50217.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.50217.log.gz 2024-07-04T17:38:03.575 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.64232.log: -- /var/log/ceph/ceph-client.admin.50279.log 2024-07-04T17:38:03.575 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.64232.log.gz 2024-07-04T17:38:03.576 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45314.log 2024-07-04T17:38:03.577 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50279.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.50279.log.gz 2024-07-04T17:38:03.577 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53326.log 2024-07-04T17:38:03.578 INFO:teuthology.orchestra.run.smithi042.stderr:gzip/var/log/ceph/ceph-client.admin.45314.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44145.log 2024-07-04T17:38:03.579 INFO:teuthology.orchestra.run.smithi042.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.45314.log.gz 2024-07-04T17:38:03.579 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53326.log: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.53326.log.gz 2024-07-04T17:38:03.580 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60486.log 2024-07-04T17:38:03.581 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61021.log 2024-07-04T17:38:03.582 INFO:teuthology.orchestra.run.smithi042.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.44145.log.gz 2024-07-04T17:38:03.583 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56134.log 2024-07-04T17:38:03.583 INFO:teuthology.orchestra.run.smithi042.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.60486.log.gz 2024-07-04T17:38:03.584 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.61021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63153.log 2024-07-04T17:38:03.584 INFO:teuthology.orchestra.run.smithi042.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.61021.log.gz 2024-07-04T17:38:03.585 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42226.log 2024-07-04T17:38:03.586 INFO:teuthology.orchestra.run.smithi042.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.56134.log.gz 2024-07-04T17:38:03.587 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.63153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61819.log 2024-07-04T17:38:03.587 INFO:teuthology.orchestra.run.smithi042.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.63153.log.gz 2024-07-04T17:38:03.587 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42226.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.42226.log.gz 2024-07-04T17:38:03.588 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64644.log 2024-07-04T17:38:03.588 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.61819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47011.log 2024-07-04T17:38:03.589 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.61819.log.gz 2024-07-04T17:38:03.590 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.64644.log: 91.3% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-07-04T17:38:03.590 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59875.log 2024-07-04T17:38:03.590 INFO:teuthology.orchestra.run.smithi042.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.64644.log.gz 2024-07-04T17:38:03.591 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47011.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.47011.log.gz 2024-07-04T17:38:03.592 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59875.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.59875.log.gz 2024-07-04T17:38:03.593 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40794.log 2024-07-04T17:38:03.594 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61945.log 2024-07-04T17:38:03.595 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40794.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.31245.log 2024-07-04T17:38:03.595 INFO:teuthology.orchestra.run.smithi042.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.40794.log.gz 2024-07-04T17:38:03.596 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.61945.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-07-04T17:38:03.597 INFO:teuthology.orchestra.run.smithi042.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.61945.log.gz 2024-07-04T17:38:03.598 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph.tmp-client.admin.31245.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.31245.log.gz 2024-07-04T17:38:03.598 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56005.log 2024-07-04T17:38:03.599 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52719.log 2024-07-04T17:38:03.600 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51972.log 2024-07-04T17:38:03.600 INFO:teuthology.orchestra.run.smithi042.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.56005.log.gz 2024-07-04T17:38:03.601 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54062.log 2024-07-04T17:38:03.602 INFO:teuthology.orchestra.run.smithi042.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52719.log.gz 2024-07-04T17:38:03.603 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.51972.log: 85.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54234.log 2024-07-04T17:38:03.603 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51972.log.gz 2024-07-04T17:38:03.604 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69755.log 2024-07-04T17:38:03.604 INFO:teuthology.orchestra.run.smithi042.stderr: 83.4% -- replaced with /var/log/ceph/ceph-client.admin.54062.log.gz 2024-07-04T17:38:03.605 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55194.log 2024-07-04T17:38:03.606 INFO:teuthology.orchestra.run.smithi042.stderr: 83.1% -- replaced with /var/log/ceph/ceph-client.admin.54234.log.gz 2024-07-04T17:38:03.607 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.69755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41212.log 2024-07-04T17:38:03.607 INFO:teuthology.orchestra.run.smithi042.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.69755.log.gz 2024-07-04T17:38:03.608 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63409.log 2024-07-04T17:38:03.608 INFO:teuthology.orchestra.run.smithi042.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.55194.log.gz 2024-07-04T17:38:03.609 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41212.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.41212.log.gz 2024-07-04T17:38:03.611 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.63409.log: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.63409.log.gz 2024-07-04T17:38:03.611 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62113.log 2024-07-04T17:38:03.612 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59558.log 2024-07-04T17:38:03.614 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.62113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40664.log 2024-07-04T17:38:03.614 INFO:teuthology.orchestra.run.smithi042.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.62113.log.gz 2024-07-04T17:38:03.615 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66473.log 2024-07-04T17:38:03.615 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.59558.log.gz 2024-07-04T17:38:03.616 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68674.log 2024-07-04T17:38:03.616 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.40664.log.gz 2024-07-04T17:38:03.617 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.66473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49593.log 2024-07-04T17:38:03.617 INFO:teuthology.orchestra.run.smithi042.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.66473.log.gz 2024-07-04T17:38:03.618 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.68674.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.68674.log.gz 2024-07-04T17:38:03.619 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49593.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.49593.log.gz 2024-07-04T17:38:03.620 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68968.log 2024-07-04T17:38:03.621 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58911.log 2024-07-04T17:38:03.622 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.68968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52553.log 2024-07-04T17:38:03.623 INFO:teuthology.orchestra.run.smithi042.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.68968.log.gz 2024-07-04T17:38:03.624 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69574.log 2024-07-04T17:38:03.624 INFO:teuthology.orchestra.run.smithi042.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.58911.log.gz 2024-07-04T17:38:03.625 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69713.log 2024-07-04T17:38:03.625 INFO:teuthology.orchestra.run.smithi042.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.52553.log.gz 2024-07-04T17:38:03.626 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.69574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41088.log 2024-07-04T17:38:03.626 INFO:teuthology.orchestra.run.smithi042.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.69574.log.gz 2024-07-04T17:38:03.628 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.69713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66924.log 85.3% -- replaced with /var/log/ceph/ceph-client.admin.69713.log.gz 2024-07-04T17:38:03.628 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-07-04T17:38:03.629 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41088.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.41088.log.gz 2024-07-04T17:38:03.629 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74172.log 2024-07-04T17:38:03.631 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.66924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41210.log 2024-07-04T17:38:03.631 INFO:teuthology.orchestra.run.smithi042.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.66924.log.gz 2024-07-04T17:38:03.632 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.74172.log: 84.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47197.log 2024-07-04T17:38:03.632 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.74172.log.gz 2024-07-04T17:38:03.633 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60190.log 2024-07-04T17:38:03.633 INFO:teuthology.orchestra.run.smithi042.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.41210.log.gz 2024-07-04T17:38:03.635 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47197.log: gzip -5 --verbose -- 84.9% -- replaced with /var/log/ceph/ceph-client.admin.47197.log.gz 2024-07-04T17:38:03.635 INFO:teuthology.orchestra.run.smithi042.stderr: /var/log/ceph/ceph-client.admin.49803.log 2024-07-04T17:38:03.635 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60190.log: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.60190.log.gz 2024-07-04T17:38:03.636 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54490.log 2024-07-04T17:38:03.637 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49803.log: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.49803.log.gz 2024-07-04T17:38:03.637 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46700.log 2024-07-04T17:38:03.638 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54490.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.54490.log.gz 2024-07-04T17:38:03.639 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50407.log 2024-07-04T17:38:03.640 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44835.log 2024-07-04T17:38:03.640 INFO:teuthology.orchestra.run.smithi042.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.46700.log.gz 2024-07-04T17:38:03.642 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71616.log 2024-07-04T17:38:03.642 INFO:teuthology.orchestra.run.smithi042.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.50407.log.gz 2024-07-04T17:38:03.643 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49040.log 2024-07-04T17:38:03.643 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.44835.log.gz 2024-07-04T17:38:03.644 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.71616.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-07-04T17:38:03.645 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.71616.log.gz 2024-07-04T17:38:03.645 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49040.log: 83.0% -- replaced with /var/log/ceph/ceph-client.admin.49040.log.gz 2024-07-04T17:38:03.648 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47819.log 2024-07-04T17:38:03.649 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45612.log 2024-07-04T17:38:03.651 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47819.log: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.47819.log.gz 2024-07-04T17:38:03.652 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73233.log 2024-07-04T17:38:03.652 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45612.log: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.45612.log.gz 2024-07-04T17:38:03.653 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54276.log 2024-07-04T17:38:03.654 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.73233.log: 83.3% -- replaced with /var/log/ceph/ceph-client.admin.73233.log.gz 2024-07-04T17:38:03.656 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.54276.log: -- /var/log/ceph/ceph-client.admin.58595.log 2024-07-04T17:38:03.656 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.54276.log.gz 2024-07-04T17:38:03.657 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53211.log 2024-07-04T17:38:03.658 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58595.log: 83.1% -- replaced with /var/log/ceph/ceph-client.admin.58595.log.gz 2024-07-04T17:38:03.659 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43453.log 2024-07-04T17:38:03.660 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53211.log: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.53211.log.gz 2024-07-04T17:38:03.661 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40752.log 2024-07-04T17:38:03.662 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43453.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.43453.log.gz 2024-07-04T17:38:03.663 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55015.log 2024-07-04T17:38:03.664 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40752.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.40752.log.gz 2024-07-04T17:38:03.665 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43537.log 2024-07-04T17:38:03.666 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55015.log: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.55015.log.gz 2024-07-04T17:38:03.667 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36308.log 2024-07-04T17:38:03.668 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43537.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.43537.log.gz 2024-07-04T17:38:03.668 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57553.log 2024-07-04T17:38:03.669 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36308.log: 73.4% -- replaced with /var/log/ceph/ceph-client.admin.36308.log.gz 2024-07-04T17:38:03.670 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47389.log/var/log/ceph/ceph-client.admin.57553.log: 2024-07-04T17:38:03.671 INFO:teuthology.orchestra.run.smithi042.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.57553.log.gz 2024-07-04T17:38:03.672 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40569.log 2024-07-04T17:38:03.673 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47389.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.47389.log.gz 2024-07-04T17:38:03.674 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46231.log 2024-07-04T17:38:03.675 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40569.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.40569.log.gz 2024-07-04T17:38:03.676 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49406.log 2024-07-04T17:38:03.677 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46231.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.46231.log.gz 2024-07-04T17:38:03.678 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61680.log 2024-07-04T17:38:03.679 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49406.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.49406.log.gz 2024-07-04T17:38:03.680 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62943.log 2024-07-04T17:38:03.681 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.61680.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.61680.log.gz 2024-07-04T17:38:03.682 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59921.log 2024-07-04T17:38:03.683 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.62943.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.62943.log.gz 2024-07-04T17:38:03.684 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47505.log 2024-07-04T17:38:03.685 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59921.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.59921.log.gz 2024-07-04T17:38:03.685 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43621.log 2024-07-04T17:38:03.688 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47505.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47505.log.gz 2024-07-04T17:38:03.688 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68884.log 2024-07-04T17:38:03.689 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43621.log: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.43621.log.gz 2024-07-04T17:38:03.689 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41862.log 2024-07-04T17:38:03.691 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.68884.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.68884.log.gz 2024-07-04T17:38:03.692 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48784.log 2024-07-04T17:38:03.692 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41862.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.41862.log.gz 2024-07-04T17:38:03.693 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41464.log 2024-07-04T17:38:03.695 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48784.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.48784.log.gz 2024-07-04T17:38:03.696 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40920.log 2024-07-04T17:38:03.696 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41464.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.41464.log.gz 2024-07-04T17:38:03.697 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45654.log 2024-07-04T17:38:03.699 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40920.log: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.40920.log.gz 2024-07-04T17:38:03.699 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49719.log 2024-07-04T17:38:03.700 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45654.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.45654.log.gz 2024-07-04T17:38:03.701 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54660.log 2024-07-04T17:38:03.702 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49719.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.49719.log.gz 2024-07-04T17:38:03.703 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36420.log/var/log/ceph/ceph-client.admin.54660.log: 2024-07-04T17:38:03.704 INFO:teuthology.orchestra.run.smithi042.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.54660.log.gz 2024-07-04T17:38:03.705 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61532.log 2024-07-04T17:38:03.706 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36420.log: 83.2% -- replaced with /var/log/ceph/ceph-client.admin.36420.log.gz 2024-07-04T17:38:03.707 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46828.log 2024-07-04T17:38:03.708 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.61532.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.61532.log.gz 2024-07-04T17:38:03.709 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64776.log 2024-07-04T17:38:03.710 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46828.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.46828.log.gz 2024-07-04T17:38:03.711 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72280.log 2024-07-04T17:38:03.712 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.64776.log: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.64776.log.gz 2024-07-04T17:38:03.713 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68716.log 2024-07-04T17:38:03.714 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.72280.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.72280.log.gz 2024-07-04T17:38:03.715 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43858.log 2024-07-04T17:38:03.716 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.68716.log: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.68716.log.gz 2024-07-04T17:38:03.717 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57829.log 2024-07-04T17:38:03.717 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43858.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.43858.log.gz 2024-07-04T17:38:03.718 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64130.log 2024-07-04T17:38:03.720 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57829.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.57829.log.gz 2024-07-04T17:38:03.721 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63451.log 2024-07-04T17:38:03.721 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.64130.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.64130.log.gz 2024-07-04T17:38:03.722 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59151.log 2024-07-04T17:38:03.723 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.63451.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.63451.log.gz 2024-07-04T17:38:03.724 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36460.log 2024-07-04T17:38:03.725 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59151.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.59151.log.gz 2024-07-04T17:38:03.726 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48868.log 2024-07-04T17:38:03.727 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36460.log: 82.9% -- replaced with /var/log/ceph/ceph-client.admin.36460.log.gz 2024-07-04T17:38:03.728 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60148.log 2024-07-04T17:38:03.729 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48868.log: 83.3% -- replaced with /var/log/ceph/ceph-client.admin.48868.log.gz 2024-07-04T17:38:03.730 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73449.log 2024-07-04T17:38:03.731 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60148.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.60148.log.gz 2024-07-04T17:38:03.732 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50716.log 2024-07-04T17:38:03.733 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.73449.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.73449.log.gz 2024-07-04T17:38:03.734 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41211.log 2024-07-04T17:38:03.735 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50716.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.50716.log.gz 2024-07-04T17:38:03.736 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69486.log 2024-07-04T17:38:03.738 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41211.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.41211.log.gz 2024-07-04T17:38:03.738 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57763.log 2024-07-04T17:38:03.739 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.69486.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.69486.log.gz 2024-07-04T17:38:03.739 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50502.log 2024-07-04T17:38:03.741 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57763.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.57763.log.gz 2024-07-04T17:38:03.743 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50502.log: gzip -5 --verbose 85.5% -- replaced with /var/log/ceph/ceph-client.admin.50502.log.gz 2024-07-04T17:38:03.743 INFO:teuthology.orchestra.run.smithi042.stderr: -- /var/log/ceph/ceph-client.admin.44313.log 2024-07-04T17:38:03.744 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63288.log 2024-07-04T17:38:03.745 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44313.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.44313.log.gz 2024-07-04T17:38:03.745 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63920.log 2024-07-04T17:38:03.746 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.63288.log: 83.3% -- replaced with /var/log/ceph/ceph-client.admin.63288.log.gz 2024-07-04T17:38:03.747 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36661.log/var/log/ceph/ceph-client.admin.63920.log: 2024-07-04T17:38:03.748 INFO:teuthology.orchestra.run.smithi042.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.63920.log.gz 2024-07-04T17:38:03.749 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45272.log 2024-07-04T17:38:03.750 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36661.log: 83.4% -- replaced with /var/log/ceph/ceph-client.admin.36661.log.gz 2024-07-04T17:38:03.751 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73743.log 2024-07-04T17:38:03.752 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45272.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.45272.log.gz 2024-07-04T17:38:03.753 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73617.log 2024-07-04T17:38:03.754 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.73743.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.73743.log.gz 2024-07-04T17:38:03.755 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55601.log 2024-07-04T17:38:03.756 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.73617.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.73617.log.gz 2024-07-04T17:38:03.757 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50548.log 2024-07-04T17:38:03.758 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55601.log: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.55601.log.gz 2024-07-04T17:38:03.759 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63069.log 2024-07-04T17:38:03.760 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50548.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.50548.log.gz 2024-07-04T17:38:03.761 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73931.log 2024-07-04T17:38:03.763 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.63069.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.63069.log.gz 2024-07-04T17:38:03.763 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62261.log 2024-07-04T17:38:03.764 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.73931.log: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.73931.log.gz 2024-07-04T17:38:03.764 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48700.log 2024-07-04T17:38:03.766 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.62261.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.62261.log.gz 2024-07-04T17:38:03.768 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.48700.log: /var/log/ceph/ceph-client.admin.47996.log 2024-07-04T17:38:03.768 INFO:teuthology.orchestra.run.smithi042.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.48700.log.gz 2024-07-04T17:38:03.768 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60232.log 2024-07-04T17:38:03.770 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47996.log: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.47996.log.gz 2024-07-04T17:38:03.771 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67370.log 2024-07-04T17:38:03.771 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60232.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.60232.log.gz 2024-07-04T17:38:03.772 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53936.log 2024-07-04T17:38:03.773 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.67370.log: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.67370.log.gz 2024-07-04T17:38:03.774 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69797.log 2024-07-04T17:38:03.775 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53936.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.53936.log.gz 2024-07-04T17:38:03.776 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56200.log 2024-07-04T17:38:03.777 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.69797.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.69797.log.gz 2024-07-04T17:38:03.778 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71676.log 2024-07-04T17:38:03.779 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56200.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.56200.log.gz 2024-07-04T17:38:03.781 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.71676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67949.log 2024-07-04T17:38:03.782 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48229.log 2024-07-04T17:38:03.784 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.67949.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.71676.log.gz 2024-07-04T17:38:03.784 INFO:teuthology.orchestra.run.smithi042.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.67949.log.gz 2024-07-04T17:38:03.786 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48229.log: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.48229.log.gz 2024-07-04T17:38:03.786 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57912.log 2024-07-04T17:38:03.788 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59421.log 2024-07-04T17:38:03.789 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57912.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.57912.log.gz 2024-07-04T17:38:03.791 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59421.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.59421.log.gz 2024-07-04T17:38:03.791 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69965.log 2024-07-04T17:38:03.793 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64506.log 2024-07-04T17:38:03.794 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.69965.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.69965.log.gz 2024-07-04T17:38:03.795 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.64506.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.64506.log.gz 2024-07-04T17:38:03.796 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44793.log 2024-07-04T17:38:03.797 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67072.log 2024-07-04T17:38:03.798 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44793.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.44793.log.gz 2024-07-04T17:38:03.800 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.67072.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67072.log.gz 2024-07-04T17:38:03.801 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66343.log 2024-07-04T17:38:03.802 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43968.log 2024-07-04T17:38:03.803 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.66343.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.66343.log.gz 2024-07-04T17:38:03.805 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43968.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.43968.log.gz 2024-07-04T17:38:03.805 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43663.log 2024-07-04T17:38:03.807 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45133.log 2024-07-04T17:38:03.808 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43663.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.43663.log.gz 2024-07-04T17:38:03.810 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45133.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.45133.log.gz 2024-07-04T17:38:03.810 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59109.log 2024-07-04T17:38:03.812 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36622.log 2024-07-04T17:38:03.813 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59109.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.59109.log.gz 2024-07-04T17:38:03.815 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.mirror.42081.log 2024-07-04T17:38:03.816 INFO:teuthology.orchestra.run.smithi042.stderr: 83.2% -- replaced with /var/log/ceph/ceph-client.admin.36622.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52241.log 2024-07-04T17:38:03.816 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-07-04T17:38:03.818 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.mirror.42081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37075.log 2024-07-04T17:38:03.819 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52241.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.52241.log.gz 2024-07-04T17:38:03.820 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45762.log 2024-07-04T17:38:03.821 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37075.log: 83.4% -- replaced with /var/log/ceph/ceph-client.admin.37075.log.gz 2024-07-04T17:38:03.823 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45762.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.e.log 2024-07-04T17:38:03.824 INFO:teuthology.orchestra.run.smithi042.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.45762.log.gz 2024-07-04T17:38:03.825 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36675.log 2024-07-04T17:38:03.827 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74214.log 2024-07-04T17:38:03.828 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36675.log: 83.3% -- replaced with /var/log/ceph/ceph-client.admin.36675.log.gz 2024-07-04T17:38:03.831 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67991.log 2024-07-04T17:38:03.832 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.74214.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.74214.log.gz 2024-07-04T17:38:03.834 INFO:teuthology.orchestra.run.smithi042.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.mirror.42081.log.gz 2024-07-04T17:38:03.836 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.67991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64686.log 2024-07-04T17:38:03.837 INFO:teuthology.orchestra.run.smithi042.stderr: 83.5% -- replaced with /var/log/ceph/ceph-client.admin.67991.log.gz 2024-07-04T17:38:03.839 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69839.log 2024-07-04T17:38:03.843 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.64686.log: 85.2%gzip -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.64686.log.gz -- /var/log/ceph/ceph-client.admin.59965.log 2024-07-04T17:38:03.843 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-07-04T17:38:03.844 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.69839.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.69839.log.gz 2024-07-04T17:38:03.845 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.mirror.42071.log 2024-07-04T17:38:03.847 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59965.log: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.59965.log.gz 2024-07-04T17:38:03.849 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60060.log 2024-07-04T17:38:03.850 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.mirror.42071.log: 69.0% -- replaced with /var/log/ceph/ceph-client.mirror.42071.log.gz 2024-07-04T17:38:03.853 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50884.log 2024-07-04T17:38:03.853 INFO:teuthology.orchestra.run.smithi042.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mds.e.log.gz 2024-07-04T17:38:03.854 INFO:teuthology.orchestra.run.smithi042.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.60060.log.gz 2024-07-04T17:38:03.855 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72575.log 2024-07-04T17:38:03.856 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50884.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.50884.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74608.log 2024-07-04T17:38:03.856 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-07-04T17:38:03.857 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.72575.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.72575.log.gz 2024-07-04T17:38:03.858 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63859.log 2024-07-04T17:38:03.859 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.74608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72893.log 2024-07-04T17:38:03.859 INFO:teuthology.orchestra.run.smithi042.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.74608.log.gz 2024-07-04T17:38:03.860 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.63859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65213.log 2024-07-04T17:38:03.860 INFO:teuthology.orchestra.run.smithi042.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.63859.log.gz 2024-07-04T17:38:03.861 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.72893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56049.log 2024-07-04T17:38:03.862 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.72893.log.gz 2024-07-04T17:38:03.863 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.65213.log: gzip -5 --verbose 85.0% -- /var/log/ceph/ceph-client.admin.53764.log 2024-07-04T17:38:03.863 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65213.log.gz 2024-07-04T17:38:03.864 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56049.log: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.56049.log.gz 2024-07-04T17:38:03.864 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66798.log 2024-07-04T17:38:03.866 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43579.log 2024-07-04T17:38:03.866 INFO:teuthology.orchestra.run.smithi042.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.53764.log.gz 2024-07-04T17:38:03.867 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.66798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57595.log 2024-07-04T17:38:03.867 INFO:teuthology.orchestra.run.smithi042.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.66798.log.gz 2024-07-04T17:38:03.868 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43579.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.43579.log.gz 2024-07-04T17:38:03.869 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66524.log 2024-07-04T17:38:03.869 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57595.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.57595.log.gz 2024-07-04T17:38:03.870 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60316.log 2024-07-04T17:38:03.871 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.66524.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.66524.log.gz 2024-07-04T17:38:03.872 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69105.log 2024-07-04T17:38:03.873 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36361.log 2024-07-04T17:38:03.873 INFO:teuthology.orchestra.run.smithi042.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.60316.log.gz 2024-07-04T17:38:03.874 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.69105.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.69105.log.gz 2024-07-04T17:38:03.875 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54973.log 2024-07-04T17:38:03.875 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36361.log: 74.1% -- replaced with /var/log/ceph/ceph-client.admin.36361.log.gz 2024-07-04T17:38:03.876 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69881.log 2024-07-04T17:38:03.877 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57199.log 2024-07-04T17:38:03.878 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.54973.log.gz 2024-07-04T17:38:03.879 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.69881.log: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.69881.log.gz 2024-07-04T17:38:03.883 INFO:teuthology.orchestra.run.smithi042.stderr:gzip/var/log/ceph/ceph-client.admin.57199.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.66566.log 2024-07-04T17:38:03.884 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.57199.log.gz 2024-07-04T17:38:03.884 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68842.log 2024-07-04T17:38:03.884 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.66566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73889.log 2024-07-04T17:38:03.884 INFO:teuthology.orchestra.run.smithi042.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.66566.log.gz 2024-07-04T17:38:03.884 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.68842.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.68842.log.gz 2024-07-04T17:38:03.885 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49551.log 2024-07-04T17:38:03.885 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.73889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63691.log 2024-07-04T17:38:03.885 INFO:teuthology.orchestra.run.smithi042.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.73889.log.gz 2024-07-04T17:38:03.886 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40613.log 2024-07-04T17:38:03.886 INFO:teuthology.orchestra.run.smithi042.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.49551.log.gz 2024-07-04T17:38:03.887 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.63691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65641.log 2024-07-04T17:38:03.888 INFO:teuthology.orchestra.run.smithi042.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.63691.log.gz 2024-07-04T17:38:03.889 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69398.log 2024-07-04T17:38:03.889 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.40613.log.gz 2024-07-04T17:38:03.890 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.0.65641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71276.log 2024-07-04T17:38:03.891 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.69398.log: 93.2% -- replaced with /var/log/ceph/ceph-client.0.65641.log.gz 85.3% -- replaced with /var/log/ceph/ceph-client.admin.69398.log.gz 2024-07-04T17:38:03.892 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-07-04T17:38:03.893 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.71276.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.71276.log.gz 2024-07-04T17:38:03.893 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68529.log 2024-07-04T17:38:03.894 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53654.log 2024-07-04T17:38:03.896 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.68529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36266.log 2024-07-04T17:38:03.896 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.68529.log.gz 2024-07-04T17:38:03.897 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53654.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.53654.log.gz 2024-07-04T17:38:03.898 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36266.log: 73.4% -- replaced with /var/log/ceph/ceph-client.admin.36266.log.gz 2024-07-04T17:38:03.899 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71181.log 2024-07-04T17:38:03.899 INFO:teuthology.orchestra.run.smithi042.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-07-04T17:38:03.900 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41760.log 2024-07-04T17:38:03.901 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.71181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52153.log 2024-07-04T17:38:03.901 INFO:teuthology.orchestra.run.smithi042.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.71181.log.gz 2024-07-04T17:38:03.902 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45091.log 2024-07-04T17:38:03.902 INFO:teuthology.orchestra.run.smithi042.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.41760.log.gz 2024-07-04T17:38:03.903 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52153.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.52153.log.gz 2024-07-04T17:38:03.904 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46923.log 2024-07-04T17:38:03.904 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45091.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.45091.log.gz 2024-07-04T17:38:03.905 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67907.log 2024-07-04T17:38:03.906 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42072.log 2024-07-04T17:38:03.906 INFO:teuthology.orchestra.run.smithi042.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.46923.log.gz 2024-07-04T17:38:03.908 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.67907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54020.log 2024-07-04T17:38:03.908 INFO:teuthology.orchestra.run.smithi042.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.67907.log.gz 2024-07-04T17:38:03.909 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.42072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47239.log 2024-07-04T17:38:03.909 INFO:teuthology.orchestra.run.smithi042.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.42072.log.gz 2024-07-04T17:38:03.910 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56478.log 2024-07-04T17:38:03.910 INFO:teuthology.orchestra.run.smithi042.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.54020.log.gz 2024-07-04T17:38:03.911 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70584.log 2024-07-04T17:38:03.911 INFO:teuthology.orchestra.run.smithi042.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.47239.log.gz 2024-07-04T17:38:03.912 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.0.56478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70049.log 2024-07-04T17:38:03.914 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.70584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63111.log 2024-07-04T17:38:03.914 INFO:teuthology.orchestra.run.smithi042.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.56478.log.gz 2024-07-04T17:38:03.914 INFO:teuthology.orchestra.run.smithi042.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.70584.log.gz 2024-07-04T17:38:03.915 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.70049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71053.log 2024-07-04T17:38:03.915 INFO:teuthology.orchestra.run.smithi042.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.70049.log.gz 2024-07-04T17:38:03.916 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.63111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57679.log 2024-07-04T17:38:03.916 INFO:teuthology.orchestra.run.smithi042.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.63111.log.gz 2024-07-04T17:38:03.917 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.71053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66882.log 2024-07-04T17:38:03.917 INFO:teuthology.orchestra.run.smithi042.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.71053.log.gz 2024-07-04T17:38:03.919 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50632.log 2024-07-04T17:38:03.919 INFO:teuthology.orchestra.run.smithi042.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.57679.log.gz 2024-07-04T17:38:03.919 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.66882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45177.log 2024-07-04T17:38:03.920 INFO:teuthology.orchestra.run.smithi042.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.66882.log.gz 2024-07-04T17:38:03.921 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58964.log 2024-07-04T17:38:03.921 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.50632.log.gz 2024-07-04T17:38:03.922 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48128.log 2024-07-04T17:38:03.922 INFO:teuthology.orchestra.run.smithi042.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.45177.log.gz 2024-07-04T17:38:03.923 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58964.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.58964.log.gz 2024-07-04T17:38:03.924 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49761.log 2024-07-04T17:38:03.924 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48128.log: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.48128.log.gz 2024-07-04T17:38:03.925 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40525.log 2024-07-04T17:38:03.926 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52058.log 2024-07-04T17:38:03.926 INFO:teuthology.orchestra.run.smithi042.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.49761.log.gz 2024-07-04T17:38:03.927 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58869.log 2024-07-04T17:38:03.927 INFO:teuthology.orchestra.run.smithi042.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.40525.log.gz 2024-07-04T17:38:03.929 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36152.log 2024-07-04T17:38:03.929 INFO:teuthology.orchestra.run.smithi042.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.52058.log.gz 2024-07-04T17:38:03.930 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57637.log 2024-07-04T17:38:03.930 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.58869.log.gz 2024-07-04T17:38:03.931 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36152.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36152.log.gz 2024-07-04T17:38:03.931 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73491.log 2024-07-04T17:38:03.932 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55738.log 2024-07-04T17:38:03.933 INFO:teuthology.orchestra.run.smithi042.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.57637.log.gz 2024-07-04T17:38:03.934 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.73491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67464.log 2024-07-04T17:38:03.934 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.73491.log.gz 2024-07-04T17:38:03.935 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49322.log 2024-07-04T17:38:03.935 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.55738.log.gz 2024-07-04T17:38:03.936 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.67464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61773.log 2024-07-04T17:38:03.936 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.67464.log.gz 2024-07-04T17:38:03.937 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67781.log 2024-07-04T17:38:03.938 INFO:teuthology.orchestra.run.smithi042.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.49322.log.gz 2024-07-04T17:38:03.939 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.61773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52102.log 2024-07-04T17:38:03.939 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.61773.log.gz 2024-07-04T17:38:03.940 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.67781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47155.log 2024-07-04T17:38:03.940 INFO:teuthology.orchestra.run.smithi042.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.67781.log.gz 2024-07-04T17:38:03.941 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41338.log 2024-07-04T17:38:03.941 INFO:teuthology.orchestra.run.smithi042.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.52102.log.gz 2024-07-04T17:38:03.942 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47155.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.47155.log.gz 2024-07-04T17:38:03.942 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73575.log 2024-07-04T17:38:03.943 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41338.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.41338.log.gz 2024-07-04T17:38:03.944 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74650.log 2024-07-04T17:38:03.945 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.73575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55696.log 2024-07-04T17:38:03.945 INFO:teuthology.orchestra.run.smithi042.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.73575.log.gz 2024-07-04T17:38:03.946 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.74650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49238.log 2024-07-04T17:38:03.946 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.74650.log.gz 2024-07-04T17:38:03.948 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44229.log 2024-07-04T17:38:03.948 INFO:teuthology.orchestra.run.smithi042.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.55696.log.gz 2024-07-04T17:38:03.949 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.49238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56669.log 2024-07-04T17:38:03.949 INFO:teuthology.orchestra.run.smithi042.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.49238.log.gz 2024-07-04T17:38:03.949 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68277.log 2024-07-04T17:38:03.950 INFO:teuthology.orchestra.run.smithi042.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.44229.log.gz 2024-07-04T17:38:03.951 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.56669.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-07-04T17:38:03.951 INFO:teuthology.orchestra.run.smithi042.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.56669.log.gz 2024-07-04T17:38:03.952 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.68277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36526.log 2024-07-04T17:38:03.953 INFO:teuthology.orchestra.run.smithi042.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.68277.log.gz 2024-07-04T17:38:03.954 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73804.log 2024-07-04T17:38:03.955 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55784.log 2024-07-04T17:38:03.955 INFO:teuthology.orchestra.run.smithi042.stderr: 83.1% -- replaced with /var/log/ceph/ceph-client.admin.36526.log.gz 2024-07-04T17:38:03.956 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.73804.log: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.73804.log.gz 2024-07-04T17:38:03.956 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54618.log 2024-07-04T17:38:03.957 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55784.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.55784.log.gz 2024-07-04T17:38:03.958 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.56947.log 2024-07-04T17:38:03.959 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64186.log 2024-07-04T17:38:03.959 INFO:teuthology.orchestra.run.smithi042.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.54618.log.gz 2024-07-04T17:38:03.961 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.1.56947.log: /var/log/ceph/ceph-client.admin.64186.log: 90.3% -- replaced with /var/log/ceph/ceph-client.1.56947.log.gz 2024-07-04T17:38:03.961 INFO:teuthology.orchestra.run.smithi042.stderr: 85.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48054.log 2024-07-04T17:38:03.962 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64186.log.gz 2024-07-04T17:38:03.962 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55057.log 2024-07-04T17:38:03.964 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.48054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37115.log 2024-07-04T17:38:03.964 INFO:teuthology.orchestra.run.smithi042.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.48054.log.gz 2024-07-04T17:38:03.965 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55057.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.55057.log.gz 2024-07-04T17:38:03.965 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58128.log 2024-07-04T17:38:03.966 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.37115.log: 83.2% -- replaced with /var/log/ceph/ceph-client.admin.37115.log.gz 2024-07-04T17:38:03.967 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41566.log 2024-07-04T17:38:03.968 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64004.log 2024-07-04T17:38:03.968 INFO:teuthology.orchestra.run.smithi042.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.58128.log.gz 2024-07-04T17:38:03.970 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41566.log: 85.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.41566.log.gz 2024-07-04T17:38:03.970 INFO:teuthology.orchestra.run.smithi042.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43705.log 2024-07-04T17:38:03.970 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.64004.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.64004.log.gz 2024-07-04T17:38:03.971 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74256.log 2024-07-04T17:38:03.972 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.43705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61731.log 2024-07-04T17:38:03.972 INFO:teuthology.orchestra.run.smithi042.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.43705.log.gz 2024-07-04T17:38:03.973 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.74256.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.74256.log.gz 2024-07-04T17:38:03.974 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.61731.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.61731.log.gz 2024-07-04T17:38:03.975 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73659.log 2024-07-04T17:38:03.976 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53153.log 2024-07-04T17:38:03.978 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.73659.log: gzip -5 --verbose 85.3% -- replaced with /var/log/ceph/ceph-client.admin.73659.log.gz 2024-07-04T17:38:03.978 INFO:teuthology.orchestra.run.smithi042.stderr: -- /var/log/ceph/ceph-client.admin.65832.log 2024-07-04T17:38:03.978 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.53153.log: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.53153.log.gz 2024-07-04T17:38:03.979 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54763.log 2024-07-04T17:38:03.980 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.65832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44637.log 2024-07-04T17:38:03.980 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.65832.log.gz 2024-07-04T17:38:03.982 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45528.log 2024-07-04T17:38:03.982 INFO:teuthology.orchestra.run.smithi042.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.54763.log.gz 2024-07-04T17:38:03.983 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.44637.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.44637.log.gz 2024-07-04T17:38:03.983 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59211.log 2024-07-04T17:38:03.984 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73061.log 2024-07-04T17:38:03.985 INFO:teuthology.orchestra.run.smithi042.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.45528.log.gz 2024-07-04T17:38:03.986 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5/var/log/ceph/ceph-client.admin.59211.log: --verbose -- /var/log/ceph/ceph-client.admin.58534.log 2024-07-04T17:38:03.987 INFO:teuthology.orchestra.run.smithi042.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.59211.log.gz 2024-07-04T17:38:03.987 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.73061.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.73061.log.gz 2024-07-04T17:38:03.987 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54931.log 2024-07-04T17:38:03.989 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52283.log 2024-07-04T17:38:03.989 INFO:teuthology.orchestra.run.smithi042.stderr: 83.5% -- replaced with /var/log/ceph/ceph-client.admin.58534.log.gz 2024-07-04T17:38:03.990 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54931.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.54931.log.gz 2024-07-04T17:38:03.990 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59379.log 2024-07-04T17:38:03.991 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72121.log 2024-07-04T17:38:03.992 INFO:teuthology.orchestra.run.smithi042.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.52283.log.gz 2024-07-04T17:38:03.993 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58205.log 2024-07-04T17:38:03.993 INFO:teuthology.orchestra.run.smithi042.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.59379.log.gz 2024-07-04T17:38:03.994 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.72121.log: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.72121.log.gz 2024-07-04T17:38:03.995 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58205.log: 85.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52619.log 2024-07-04T17:38:03.996 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58205.log.gz 2024-07-04T17:38:03.996 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52511.log 2024-07-04T17:38:03.998 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54348.log 2024-07-04T17:38:03.998 INFO:teuthology.orchestra.run.smithi042.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.52619.log.gz 2024-07-04T17:38:03.999 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52511.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.52511.log.gz 2024-07-04T17:38:03.999 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54702.log 2024-07-04T17:38:04.000 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54348.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.54348.log.gz 2024-07-04T17:38:04.001 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64732.log 2024-07-04T17:38:04.002 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54534.log 2024-07-04T17:38:04.002 INFO:teuthology.orchestra.run.smithi042.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.54702.log.gz 2024-07-04T17:38:04.003 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.64732.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.64732.log.gz 2024-07-04T17:38:04.003 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41004.log 2024-07-04T17:38:04.004 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.54534.log: 85.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74508.log 2024-07-04T17:38:04.005 INFO:teuthology.orchestra.run.smithi042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54534.log.gz 2024-07-04T17:38:04.006 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.41004.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-07-04T17:38:04.006 INFO:teuthology.orchestra.run.smithi042.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.41004.log.gz 2024-07-04T17:38:04.007 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.74508.log: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.74508.log.gz 2024-07-04T17:38:04.007 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59833.log 2024-07-04T17:38:04.009 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46965.log 2024-07-04T17:38:04.010 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59833.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.59833.log.gz 2024-07-04T17:38:04.011 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45049.log 2024-07-04T17:38:04.011 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.46965.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.46965.log.gz 2024-07-04T17:38:04.012 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69010.log 2024-07-04T17:38:04.013 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45049.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.45049.log.gz 2024-07-04T17:38:04.014 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50992.log 2024-07-04T17:38:04.015 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.69010.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.69010.log.gz 2024-07-04T17:38:04.016 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68403.log 2024-07-04T17:38:04.017 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50992.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.50992.log.gz 2024-07-04T17:38:04.017 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52199.log 2024-07-04T17:38:04.019 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.68403.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.68403.log.gz 2024-07-04T17:38:04.020 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.52199.log: /var/log/ceph/ceph-client.admin.50842.log 2024-07-04T17:38:04.020 INFO:teuthology.orchestra.run.smithi042.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.52199.log.gz 2024-07-04T17:38:04.021 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62710.log 2024-07-04T17:38:04.023 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.50842.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.50842.log.gz 2024-07-04T17:38:04.024 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58785.log 2024-07-04T17:38:04.025 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.62710.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.62710.log.gz 2024-07-04T17:38:04.025 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52469.log 2024-07-04T17:38:04.026 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.58785.log: 83.2% -- replaced with /var/log/ceph/ceph-client.admin.58785.log.gz 2024-07-04T17:38:04.026 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36930.log 2024-07-04T17:38:04.028 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.52469.log: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.52469.log.gz 2024-07-04T17:38:04.028 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55868.log 2024-07-04T17:38:04.030 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36930.log: 83.1% -- replaced with /var/log/ceph/ceph-client.admin.36930.log.gz 2024-07-04T17:38:04.030 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45007.log 2024-07-04T17:38:04.031 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.55868.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.55868.log.gz 2024-07-04T17:38:04.032 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64820.log 2024-07-04T17:38:04.033 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.45007.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.45007.log.gz 2024-07-04T17:38:04.034 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61903.log 2024-07-04T17:38:04.034 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.64820.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.64820.log.gz 2024-07-04T17:38:04.035 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47281.log 2024-07-04T17:38:04.036 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.61903.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.61903.log.gz 2024-07-04T17:38:04.038 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36707.log 2024-07-04T17:38:04.038 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.47281.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.47281.log.gz 2024-07-04T17:38:04.039 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42663.log 2024-07-04T17:38:04.040 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36707.log: 83.4% -- replaced with /var/log/ceph/ceph-client.admin.36707.log.gz 2024-07-04T17:38:04.041 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.0.42663.log: /var/log/ceph/ceph-client.admin.57469.log 2024-07-04T17:38:04.042 INFO:teuthology.orchestra.run.smithi042.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.0.42663.log.gz 2024-07-04T17:38:04.043 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66429.log 2024-07-04T17:38:04.044 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57469.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.57469.log.gz 2024-07-04T17:38:04.045 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60360.log 2024-07-04T17:38:04.045 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.66429.log: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.66429.log.gz 2024-07-04T17:38:04.047 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57423.log 2024-07-04T17:38:04.047 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.60360.log: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.60360.log.gz 2024-07-04T17:38:04.048 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59006.log 2024-07-04T17:38:04.049 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.57423.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.57423.log.gz 2024-07-04T17:38:04.050 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66385.log 2024-07-04T17:38:04.051 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.59006.log: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.59006.log.gz 2024-07-04T17:38:04.053 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40483.log/var/log/ceph/ceph-client.admin.66385.log: 2024-07-04T17:38:04.053 INFO:teuthology.orchestra.run.smithi042.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.66385.log.gz 2024-07-04T17:38:04.054 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73533.log 2024-07-04T17:38:04.056 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.40483.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.40483.log.gz 2024-07-04T17:38:04.056 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36604.log 2024-07-04T17:38:04.057 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.73533.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.73533.log.gz 2024-07-04T17:38:04.057 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73407.log 2024-07-04T17:38:04.058 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.admin.36604.log: 83.0% -- replaced with /var/log/ceph/ceph-client.admin.36604.log.gz 2024-07-04T17:38:04.060 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.73407.log: -- /var/log/ceph/ceph-client.1.70862.log 2024-07-04T17:38:04.060 INFO:teuthology.orchestra.run.smithi042.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.73407.log.gz 2024-07-04T17:38:04.064 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.1.70862.log: 95.3% -- replaced with /var/log/ceph/ceph-client.1.70862.log.gz 2024-07-04T17:38:04.078 INFO:teuthology.orchestra.run.smithi042.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-07-04T17:38:04.183 INFO:teuthology.orchestra.run.smithi042.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-07-04T17:38:04.336 INFO:teuthology.orchestra.run.smithi042.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-07-04T17:38:04.993 INFO:teuthology.orchestra.run.smithi042.stderr: 94.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-07-04T17:38:05.234 INFO:teuthology.orchestra.run.smithi042.stderr: 94.3% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-07-04T17:38:05.257 INFO:teuthology.orchestra.run.smithi042.stderr: 94.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-07-04T17:38:05.260 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-07-04T17:38:05.261 INFO:teuthology.orchestra.run.smithi042.stderr:real 0m2.013s 2024-07-04T17:38:05.261 INFO:teuthology.orchestra.run.smithi042.stderr:user 0m6.256s 2024-07-04T17:38:05.261 INFO:teuthology.orchestra.run.smithi042.stderr:sys 0m1.971s 2024-07-04T17:38:05.262 INFO:tasks.ceph:Archiving logs... 2024-07-04T17:38:05.262 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-07-04_16:36:07-fs-wip-rishabh-testing-20240701.141150-debug-testing-default-smithi/7787245/remote/smithi042/log 2024-07-04T17:38:05.263 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /var/log/ceph -- . 2024-07-04T17:38:06.484 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-07-04T17:38:06.500 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-07-04T17:38:06.501 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-07-04T17:38:06.620 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-07-04T17:38:06.629 INFO:teuthology.task.clock:Checking final clock skew... 2024-07-04T17:38:06.630 DEBUG:teuthology.orchestra.run.smithi042:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-04T17:38:06.688 INFO:teuthology.orchestra.run.smithi042.stderr:bash: line 1: ntpq: command not found 2024-07-04T17:38:06.698 INFO:teuthology.orchestra.run.smithi042.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-04T17:38:06.698 INFO:teuthology.orchestra.run.smithi042.stdout:=============================================================================== 2024-07-04T17:38:06.698 INFO:teuthology.orchestra.run.smithi042.stdout:^* hv01.front.sepia.ceph.com 3 6 377 49 +1750us[+1753us] +/- 69ms 2024-07-04T17:38:06.699 INFO:teuthology.orchestra.run.smithi042.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 51 -575us[ -572us] +/- 58ms 2024-07-04T17:38:06.699 INFO:teuthology.orchestra.run.smithi042.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 50 -1482us[-1479us] +/- 93ms 2024-07-04T17:38:06.699 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-07-04T17:38:06.700 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-07-04T17:38:06.709 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-07-04T17:38:06.709 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-07-04T17:38:06.739 DEBUG:teuthology.orchestra.run.smithi042:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-07-04T17:38:06.980 DEBUG:teuthology.orchestra.run.smithi042:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-04T17:38:07.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-04T17:38:07.076 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-07-04T17:38:07.085 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-07-04T17:38:07.134 INFO:teuthology.task.internal:Duration was 1282.749847 seconds 2024-07-04T17:38:07.135 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-07-04T17:38:07.169 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-07-04T17:38:07.169 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-07-04T17:38:07.329 INFO:teuthology.orchestra.run.smithi042.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-04T17:38:07.641 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-07-04T17:38:07.641 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi042.front.sepia.ceph.com 2024-07-04T17:38:07.641 DEBUG:teuthology.orchestra.run.smithi042:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-07-04T17:38:07.704 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-07-04T17:38:07.704 DEBUG:teuthology.orchestra.run.smithi042:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-07-04T17:38:07.781 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-07-04T17:38:07.781 DEBUG:teuthology.orchestra.run.smithi042:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-07-04T17:38:08.011 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-07-04T17:38:08.023 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-07-04T17:38:08.023 DEBUG:teuthology.orchestra.run.smithi042:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-07-04T17:38:08.111 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-07-04T17:38:08.121 DEBUG:teuthology.orchestra.run.smithi042:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-07-04T17:38:08.221 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern = core 2024-07-04T17:38:08.262 DEBUG:teuthology.orchestra.run.smithi042:> test -e /home/ubuntu/cephtest/archive/coredump 2024-07-04T17:38:08.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-04T17:38:08.306 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-07-04T17:38:08.316 INFO:teuthology.task.internal:Transferring archived files... 2024-07-04T17:38:08.317 DEBUG:teuthology.misc:Transferring archived files from smithi042:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-07-04_16:36:07-fs-wip-rishabh-testing-20240701.141150-debug-testing-default-smithi/7787245/remote/smithi042 2024-07-04T17:38:08.317 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-07-04T17:38:08.446 INFO:teuthology.task.internal:Removing archive directory... 2024-07-04T17:38:08.446 DEBUG:teuthology.orchestra.run.smithi042:> rm -rf -- /home/ubuntu/cephtest/archive 2024-07-04T17:38:08.545 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-07-04T17:38:08.555 INFO:teuthology.task.internal:Not uploading archives. 2024-07-04T17:38:08.555 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-07-04T17:38:08.577 INFO:teuthology.task.internal:Tidying up after the test... 2024-07-04T17:38:08.578 DEBUG:teuthology.orchestra.run.smithi042:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-07-04T17:38:08.638 INFO:teuthology.orchestra.run.smithi042.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 4 17:38 /home/ubuntu/cephtest 2024-07-04T17:38:08.641 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-07-04T17:38:08.653 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-07-04T17:38:08.713 INFO:teuthology.run:Summary data: description: fs/mirror/{begin/{0-install 1-ceph 2-logrotate 3-modules} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} conf/{client mds mgr mon osd} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health pg_health} supported-random-distros$/{centos_latest} tasks/mirror} duration: 1282.7498469352722 failure_reason: 'Test failure: test_cephfs_mirror_blocklist (tasks.cephfs.test_mirroring.TestMirroring)' flavor: default mon.a-kernel-sha1: 0d32a7bba447dd2470f4de43560317375ca9e9c8 owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=3acb7460b79345ec872a8beaa4e05116 status: fail success: false 2024-07-04T17:38:08.714 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-07-04T17:38:08.800 INFO:teuthology.run:FAIL